Browse Source

Merge branch 'debugger-json-proxy-prototype'

pull/195/head
Sami Vaarala 10 years ago
parent
commit
d7f2bde35a
  1. 2
      RELEASES.rst
  2. 4
      debugger/Makefile
  3. 71
      debugger/README.rst
  4. 20
      debugger/duk_classnames.yaml
  5. 561
      debugger/duk_debug.js
  6. 36
      debugger/duk_debugcommands.yaml
  7. 3
      debugger/package.json
  8. 198
      doc/debugger.rst
  9. 21
      website/guide/debugger.html

2
RELEASES.rst

@ -817,7 +817,7 @@ Planned
* Add first iteration of experimental debugger support which is based on a * Add first iteration of experimental debugger support which is based on a
narrow debug API and a standard debug protocol; Duktape also provides an narrow debug API and a standard debug protocol; Duktape also provides an
example debug client with a web UI example debug client with a web UI, and a JSON debug proxy
* Add support "fastints", i.e. for using integer arithmetic when possible * Add support "fastints", i.e. for using integer arithmetic when possible
in a transparent manner (improves performance for some workloads on soft in a transparent manner (improves performance for some workloads on soft

4
debugger/Makefile

@ -16,6 +16,10 @@ all: run
run: node_modules static/socket.io-1.2.0.js static/jquery-1.11.1.min.js static/reset.css static/jquery-ui.min.js static/jquery-ui.min.css static/images run: node_modules static/socket.io-1.2.0.js static/jquery-1.11.1.min.js static/reset.css static/jquery-ui.min.js static/jquery-ui.min.css static/images
$(NODE) duk_debug.js --source-dirs=$(SOURCEDIRS) $(NODE) duk_debug.js --source-dirs=$(SOURCEDIRS)
.PHONY: runproxy
runproxy: node_modules static/socket.io-1.2.0.js static/jquery-1.11.1.min.js static/reset.css static/jquery-ui.min.js static/jquery-ui.min.css static/images
$(NODE) duk_debug.js --json-proxy
.PHONY: clean .PHONY: clean
clean: clean:
@rm -f static/socket.io-1.2.0.js @rm -f static/socket.io-1.2.0.js

71
debugger/README.rst

@ -1,6 +1,6 @@
==================== =========================================
Duktape debug client Duktape debug client and JSON debug proxy
==================== =========================================
Overview Overview
======== ========
@ -10,12 +10,15 @@ Overview
Debugger web UI which connects to the Duktape command line tool or any other Debugger web UI which connects to the Duktape command line tool or any other
target supporting the example TCP transport (``examples/debug-trans-socket``). target supporting the example TCP transport (``examples/debug-trans-socket``).
Also provides a JSON debug proxy with a JSON mapping for the Duktape debug
protocol.
For detailed documentation of the debugger internals, see `debugger.rst`__. For detailed documentation of the debugger internals, see `debugger.rst`__.
__ https://github.com/svaarala/duktape/blob/master/doc/debugger.rst __ https://github.com/svaarala/duktape/blob/master/doc/debugger.rst
Usage Using the debugger web UI
===== =========================
Some prerequisites: Some prerequisites:
@ -71,6 +74,52 @@ The debug client automatically attaches to the debug target on startup.
If you start the debug target later, you'll need to click "Attach" in the If you start the debug target later, you'll need to click "Attach" in the
web UI. web UI.
Using the JSON debug proxy
==========================
A JSON debug proxy is also provided by ``duk_debug.js``::
# Same prerequisites as above
$ make runproxy
Start Duktape command line (or whatever your target is)::
$ cd <duktape checkout>/ecmascript-testcases/
$ ../duk --debugger test-dev-mandel2-func.js
You can then connect to localhost:9093 and interact with the proxy.
Here's an example session using telnet and manually typed in commands
The ``-->`` (send) and ``<--`` (receiver) markers have been added for
readability and are not part of the stream::
$ telnet localhost 9093
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
<-- {"notify":"_Connected","args":["1 10199 v1.1.0-275-gbd4d610-dirty duk command built from Duktape repo"]}
<-- {"notify":"Status","command":1,"args":[1,"test-dev-mandel2-func.js","global",58,0]}
--> {"request":"BasicInfo"}
<-- {"reply":true,"args":[10199,"v1.1.0-275-gbd4d610-dirty","duk command built from Duktape repo",1]}
--> {"request":"Eval", "args":[ "print(Math.PI)" ]}
<-- {"notify":"Print","command":2,"args":["3.141592653589793\n"]}
<-- {"reply":true,"args":[0,{"type":"undefined"}]}
--> {"request":"Resume"}
<-- {"reply":true,"args":[]}
<-- {"notify":"Status","command":1,"args":[0,"test-dev-mandel2-func.js","global",58,0]}
<-- {"notify":"Status","command":1,"args":[0,"test-dev-mandel2-func.js","global",58,0]}
<-- {"notify":"Print","command":2,"args":["................................................................................\n"]}
<-- {"notify":"Print","command":2,"args":["................................................................................\n"]}
<-- {"notify":"Print","command":2,"args":["................................................................................\n"]}
[...]
<-- {"notify":"_Disconnecting"}
A telnet connection allows you to experiment with debug commands by simply
copy-pasting debug commands to the telnet session. This is useful even if
you decide to implement the binary protocol directly.
The debug target used by the proxy can be configured with ``duk_debug.js``
command line options.
Source search path Source search path
================== ==================
@ -171,8 +220,8 @@ Implement the debug transport callbacks needed by ``duk_debugger_attach()``.
See ``doc/debugger.rst`` for details and ``examples/debug-trans-socket`` See ``doc/debugger.rst`` for details and ``examples/debug-trans-socket``
for example running code for a TCP transport. for example running code for a TCP transport.
Debug client alternative 1: TCP proxy Debug client alternative 1: duk_debug.js + custom TCP proxy
------------------------------------- -----------------------------------------------------------
If you don't want to change ``duk_debug.js`` you can implement a TCP proxy If you don't want to change ``duk_debug.js`` you can implement a TCP proxy
which accepts a TCP connection from ``duk_debug.js`` and then uses your which accepts a TCP connection from ``duk_debug.js`` and then uses your
@ -188,8 +237,8 @@ clients too (perhaps custom scripts talking to the target etc).
You could also use netcat and implement your proxy so that it talks to You could also use netcat and implement your proxy so that it talks to
``duk_debug.js`` using stdin/stdout. ``duk_debug.js`` using stdin/stdout.
Debug client alternative 2: NodeJS stream Debug client alternative 2: duk_debug.js + custom NodeJS stream
----------------------------------------- ---------------------------------------------------------------
To make ``duk_debug.js`` use a custom transport you need to: To make ``duk_debug.js`` use a custom transport you need to:
@ -215,3 +264,7 @@ your custom transport.
You'll also need to implement the client part of the Duktape debugger You'll also need to implement the client part of the Duktape debugger
protocol. See ``doc/debugger.rst`` for the specification and ``duk_debug.js`` protocol. See ``doc/debugger.rst`` for the specification and ``duk_debug.js``
for example running code which should illustrate the protocol in more detail. for example running code which should illustrate the protocol in more detail.
The JSON debug proxy allows you to implement a debug client without needing
to implement the Duktape binary debug protocol. The JSON protocol provides
a roughly 1:1 mapping to the binary protocol but with an easier syntax.

20
debugger/duk_classnames.yaml

@ -0,0 +1,20 @@
# Must match C header
- unused
- Arguments
- Array
- Boolean
- Date
- Error
- Function
- JSON
- Math
- Number
- Object
- RegExp
- String
- global
- ObjEnv
- DecEnv
- Buffer
- Pointer
- Thread

561
debugger/duk_debug.js

@ -23,6 +23,7 @@ var stream = require('stream');
var path = require('path'); var path = require('path');
var fs = require('fs'); var fs = require('fs');
var net = require('net'); var net = require('net');
var byline = require('byline');
var util = require('util'); var util = require('util');
var readline = require('readline'); var readline = require('readline');
var sprintf = require('sprintf').sprintf; var sprintf = require('sprintf').sprintf;
@ -34,6 +35,8 @@ var yaml = require('yamljs');
var optTargetHost = '127.0.0.1'; var optTargetHost = '127.0.0.1';
var optTargetPort = 9091; var optTargetPort = 9091;
var optHttpPort = 9092; var optHttpPort = 9092;
var optJsonProxyPort = 9093;
var optJsonProxy = false;
var optSourceSearchDirs = [ '../ecmascript-testcases' ]; var optSourceSearchDirs = [ '../ecmascript-testcases' ];
var optDumpDebugRead = null; var optDumpDebugRead = null;
var optDumpDebugWrite = null; var optDumpDebugWrite = null;
@ -79,38 +82,22 @@ var ERR_TOOMANY = 0x02;
var ERR_NOTFOUND = 0x03; var ERR_NOTFOUND = 0x03;
// Marker objects for special protocol values // Marker objects for special protocol values
var DVAL_EOM = { EOM: true }; var DVAL_EOM = { type: 'eom' };
var DVAL_REQ = { REQ: true }; var DVAL_REQ = { type: 'req' };
var DVAL_REP = { REP: true }; var DVAL_REP = { type: 'rep' };
var DVAL_ERR = { ERR: true }; var DVAL_ERR = { type: 'err' };
var DVAL_NFY = { NFY: true }; var DVAL_NFY = { type: 'nfy' };
// String map for commands (debug dumping). A single map works (instead of // String map for commands (debug dumping). A single map works (instead of
// separate maps for each direction) because command numbers don't currently // separate maps for each direction) because command numbers don't currently
// overlap. // overlap.
var debugCommandNames = {}; var debugCommandNames = yaml.load('duk_debugcommands.yaml');
debugCommandNames[0x01] = 'Status';
debugCommandNames[0x02] = 'Print'; // Map debug command names to numbers.
debugCommandNames[0x03] = 'Alert'; var debugCommandNumbers = {};
debugCommandNames[0x04] = 'Log'; debugCommandNames.forEach(function (k, i) {
debugCommandNames[0x05] = 'Gc'; debugCommandNumbers[k] = i;
debugCommandNames[0x10] = 'BasicInfo'; });
debugCommandNames[0x11] = 'TriggerStatus';
debugCommandNames[0x12] = 'Pause';
debugCommandNames[0x13] = 'Resume';
debugCommandNames[0x14] = 'StepInto';
debugCommandNames[0x15] = 'StepOver';
debugCommandNames[0x16] = 'StepOut';
debugCommandNames[0x17] = 'ListBreak';
debugCommandNames[0x18] = 'AddBreak';
debugCommandNames[0x19] = 'DelBreak';
debugCommandNames[0x1a] = 'GetVar';
debugCommandNames[0x1b] = 'PutVar';
debugCommandNames[0x1c] = 'GetCallStack';
debugCommandNames[0x1d] = 'GetLocals';
debugCommandNames[0x1e] = 'Eval';
debugCommandNames[0x1f] = 'Detach';
debugCommandNames[0x20] = 'DumpHeap';
// Duktape heaphdr type constants, must match C headers // Duktape heaphdr type constants, must match C headers
var DUK_HTYPE_STRING = 1; var DUK_HTYPE_STRING = 1;
@ -118,27 +105,7 @@ var DUK_HTYPE_OBJECT = 2;
var DUK_HTYPE_BUFFER = 3; var DUK_HTYPE_BUFFER = 3;
// Duktape internal class numbers, must match C headers // Duktape internal class numbers, must match C headers
var dukClassNames = [ var dukClassNames = yaml.load('duk_classnames.yaml');
'unused',
'Arguments',
'Array',
'Boolean',
'Date',
'Error',
'Function',
'JSON',
'Math',
'Number',
'Object',
'RegExp',
'String',
'global',
'ObjEnv',
'DecEnv',
'Buffer',
'Pointer',
'Thread'
];
// Bytecode opcode/extraop metadata // Bytecode opcode/extraop metadata
var dukOpcodes = yaml.load('duk_opcodes.yaml') var dukOpcodes = yaml.load('duk_opcodes.yaml')
@ -206,11 +173,19 @@ function stringToDebugString(str) {
/* Pretty print a dvalue. Useful for dumping etc. */ /* Pretty print a dvalue. Useful for dumping etc. */
function prettyDebugValue(x) { function prettyDebugValue(x) {
if (x === DVAL_EOM) { return 'EOM'; } if (typeof x === 'object' && x !== null) {
if (x === DVAL_REQ) { return 'REQ'; } if (x.type === 'eom') {
if (x === DVAL_REP) { return 'REP'; } return 'EOM';
if (x === DVAL_ERR) { return 'ERR'; } } else if (x.type === 'req') {
if (x === DVAL_NFY) { return 'NFY'; } return 'REQ';
} else if (x.type === 'rep') {
return 'REP';
} else if (x.type === 'err') {
return 'ERR';
} else if (x.type === 'nfy') {
return 'NFY';
}
}
return JSON.stringify(x); return JSON.stringify(x);
} }
@ -300,50 +275,49 @@ function prettyUiStringUnquoted(x, cliplen) {
* so that styling etc. could take typing into account. * so that styling etc. could take typing into account.
*/ */
function prettyUiDebugValue(x, cliplen) { function prettyUiDebugValue(x, cliplen) {
var t;
if (x === DVAL_EOM) { return 'EOM'; }
if (x === DVAL_REQ) { return 'REQ'; }
if (x === DVAL_REP) { return 'REP'; }
if (x === DVAL_ERR) { return 'ERR'; }
if (x === DVAL_NFY) { return 'NFY'; }
if (x === undefined) { return 'undefined'; }
if (x === null) { return 'null'; }
if (typeof x === 'boolean') {
return x ? 'true' : 'false';
}
if (typeof x === 'string') {
return prettyUiString(x, cliplen);
}
if (typeof x === 'number') {
// Debug protocol integer
return prettyUiNumber(x);
}
if (typeof x === 'object' && x !== null) { if (typeof x === 'object' && x !== null) {
// Note: typeof null === 'object', so null special case explicitly // Note: typeof null === 'object', so null special case explicitly
if (typeof x.UNUSED !== 'undefined') { if (x.type === 'eom') {
return 'EOM';
} else if (x.type === 'req') {
return 'REQ';
} else if (x.type === 'rep') {
return 'REP';
} else if (x.type === 'err') {
return 'ERR';
} else if (x.type === 'nfy') {
return 'NFY';
} else if (x.type === 'unused') {
return 'unused'; return 'unused';
} } else if (x.type === 'undefined') {
if (typeof x.BUF !== 'undefined') { return 'undefined';
return '|' + x.BUF + '|'; } else if (x.type === 'buffer') {
} return '|' + x.data + '|';
if (typeof x.OBJ !== 'undefined') { } else if (x.type === 'object') {
return '[object ' + (dukClassNames[x.class] || ('class ' + x.class)) + ']'; return '[object ' + (dukClassNames[x.class] || ('class ' + x.class)) + ' ' + x.pointer + ']';
} } else if (x.type === 'pointer') {
if (typeof x.PTR !== 'undefined') { return '<pointer ' + x.pointer + '>';
return '<pointer ' + x.PTR + '>'; } else if (x.type === 'lightfunc') {
} return '<lightfunc 0x' + x.flags.toString(16) + ' ' + x.pointer + '>';
if (typeof x.LFUNC !== 'undefined') { } else if (x.type === 'number') {
return '<lightfunc 0x' + x.flags.toString(16) + ' ' + x.LFUNC + '>';
}
if (typeof x.NUM !== 'undefined') {
// duk_tval number, any IEEE double // duk_tval number, any IEEE double
return prettyUiNumber(x.value); var tmp = new Buffer(x.data, 'hex'); // decode into hex
var val = tmp.readDoubleBE(0); // big endian ieee double
return prettyUiNumber(val);
} }
} else if (x === null) {
return 'null';
} else if (typeof x === 'boolean') {
return x ? 'true' : 'false';
} else if (typeof x === 'string') {
return prettyUiString(x, cliplen);
} else if (typeof x === 'number') {
// Debug protocol integer
return prettyUiNumber(x);
} }
// We shouldn't come here, but if we do, JSON is a reasonable default. // We shouldn't come here, but if we do, JSON is a reasonable default.
return JSON.stringify(t); return JSON.stringify(x);
} }
/* Pretty print a debugger message given as an array of parsed dvalues. /* Pretty print a debugger message given as an array of parsed dvalues.
@ -551,6 +525,7 @@ function DebugProtocolParser(inputStream,
this.prevBytes = 0; this.prevBytes = 0;
this.bytesPerSec = 0; this.bytesPerSec = 0;
this.statsTimer = null; this.statsTimer = null;
this.readableNumberValue = true;
events.EventEmitter.call(this); events.EventEmitter.call(this);
@ -693,7 +668,7 @@ function DebugProtocolParser(inputStream,
if (buf.length >= 5 + len) { if (buf.length >= 5 + len) {
v = new Buffer(len); v = new Buffer(len);
buf.copy(v, 0, 5, 5 + len); buf.copy(v, 0, 5, 5 + len);
v = { BUF: v.toString('hex') }; v = { type: 'buffer', data: v.toString('hex') };
consume(5 + len); consume(5 + len);
// Value could be a Node.js buffer directly, but // Value could be a Node.js buffer directly, but
// we prefer all dvalues to be JSON compatible // we prefer all dvalues to be JSON compatible
@ -706,7 +681,7 @@ function DebugProtocolParser(inputStream,
if (buf.length >= 3 + len) { if (buf.length >= 3 + len) {
v = new Buffer(len); v = new Buffer(len);
buf.copy(v, 0, 3, 3 + len); buf.copy(v, 0, 3, 3 + len);
v = { BUF: v.toString('hex') }; v = { type: 'buffer', data: v.toString('hex') };
consume(3 + len); consume(3 + len);
// Value could be a Node.js buffer directly, but // Value could be a Node.js buffer directly, but
// we prefer all dvalues to be JSON compatible // we prefer all dvalues to be JSON compatible
@ -714,11 +689,11 @@ function DebugProtocolParser(inputStream,
} }
break; break;
case 0x15: // unused/none case 0x15: // unused/none
v = { UNUSED: true }; v = { type: 'unused' };
consume(1); consume(1);
break; break;
case 0x16: // undefined case 0x16: // undefined
v = undefined; v = { type: 'undefined' };
gotValue = true; // indicate 'v' is actually set gotValue = true; // indicate 'v' is actually set
consume(1); consume(1);
break; break;
@ -739,7 +714,13 @@ function DebugProtocolParser(inputStream,
if (buf.length >= 9) { if (buf.length >= 9) {
v = new Buffer(8); v = new Buffer(8);
buf.copy(v, 0, 1, 9); buf.copy(v, 0, 1, 9);
v = { NUM: v.toString('hex'), value: v.readDoubleBE(0) }; v = { type: 'number', data: v.toString('hex') }
if (_this.readableNumberValue) {
// The _value key should not be used programmatically,
// it is just there to make the dumps more readable.
v._value = buf.readDoubleBE(1);
}
consume(9); consume(9);
} }
break; break;
@ -749,7 +730,7 @@ function DebugProtocolParser(inputStream,
if (buf.length >= 3 + len) { if (buf.length >= 3 + len) {
v = new Buffer(len); v = new Buffer(len);
buf.copy(v, 0, 3, 3 + len); buf.copy(v, 0, 3, 3 + len);
v = { OBJ: v.toString('hex'), class: buf[1] }; v = { type: 'object', 'class': buf[1], pointer: v.toString('hex') };
consume(3 + len); consume(3 + len);
} }
} }
@ -760,7 +741,7 @@ function DebugProtocolParser(inputStream,
if (buf.length >= 2 + len) { if (buf.length >= 2 + len) {
v = new Buffer(len); v = new Buffer(len);
buf.copy(v, 0, 2, 2 + len); buf.copy(v, 0, 2, 2 + len);
v = { PTR: v.toString('hex') }; v = { type: 'pointer', pointer: v.toString('hex') };
consume(2 + len); consume(2 + len);
} }
} }
@ -771,7 +752,7 @@ function DebugProtocolParser(inputStream,
if (buf.length >= 4 + len) { if (buf.length >= 4 + len) {
v = new Buffer(len); v = new Buffer(len);
buf.copy(v, 0, 4, 4 + len); buf.copy(v, 0, 4, 4 + len);
v = { LFUNC: v.toString('hex'), flags: buf.readUInt16BE(1) }; v = { type: 'lightfunc', flags: buf.readUInt16BE(1), pointer: v.toString('hex') };
consume(4 + len); consume(4 + len);
} }
} }
@ -782,7 +763,7 @@ function DebugProtocolParser(inputStream,
if (buf.length >= 2 + len) { if (buf.length >= 2 + len) {
v = new Buffer(len); v = new Buffer(len);
buf.copy(v, 0, 2, 2 + len); buf.copy(v, 0, 2, 2 + len);
v = { HEAPPTR: v.toString('hex') }; v = { type: 'heapptr', pointer: v.toString('hex') };
consume(2 + len); consume(2 + len);
} }
} }
@ -858,20 +839,94 @@ DebugProtocolParser.prototype.close = function () {
*/ */
function formatDebugValue(v) { function formatDebugValue(v) {
var buf; var buf, dec, len;
// See doc/debugger.rst for format description. // See doc/debugger.rst for format description.
if (v === DVAL_EOM) { if (typeof v === 'object' && v !== null) {
return new Buffer([ 0x00 ]); // Note: typeof null === 'object', so null special case explicitly
} else if (v === DVAL_REQ) { if (v.type === 'eom') {
return new Buffer([ 0x01 ]); return new Buffer([ 0x00 ]);
} else if (v === DVAL_REP) { } else if (v.type === 'req') {
return new Buffer([ 0x02 ]); return new Buffer([ 0x01 ]);
} else if (v === DVAL_ERR) { } else if (v.type === 'rep') {
return new Buffer([ 0x03 ]); return new Buffer([ 0x02 ]);
} else if (v === DVAL_NFY) { } else if (v.type === 'err') {
return new Buffer([ 0x04 ]); return new Buffer([ 0x03 ]);
} else if (v.type === 'nfy') {
return new Buffer([ 0x04 ]);
} else if (v.type === 'unused') {
return new Buffer([ 0x15 ]);
} else if (v.type === 'undefined') {
return new Buffer([ 0x16 ]);
} else if (v.type === 'number') {
dec = new Buffer(v.data, 'hex');
len = dec.length;
if (len !== 8) {
throw new TypeError('value cannot be converted to dvalue: ' + JSON.stringify(v));
}
buf = new Buffer(1 + len);
buf[0] = 0x1a;
dec.copy(buf, 1);
return buf;
} else if (v.type === 'buffer') {
dec = new Buffer(v.data, 'hex');
len = dec.length;
if (len <= 0xffff) {
buf = new Buffer(3 + len);
buf[0] = 0x14;
buf[1] = (len >> 8) & 0xff;
buf[2] = (len >> 0) & 0xff;
dec.copy(buf, 3);
return buf;
} else {
buf = new Buffer(5 + len);
buf[0] = 0x13;
buf[1] = (len >> 24) & 0xff;
buf[2] = (len >> 16) & 0xff;
buf[3] = (len >> 8) & 0xff;
buf[4] = (len >> 0) & 0xff;
dec.copy(buf, 5);
return buf;
}
} else if (v.type === 'object') {
dec = new Buffer(v.pointer, 'hex');
len = dec.length;
buf = new Buffer(3 + len);
buf[0] = 0x1b;
buf[1] = v.class;
buf[2] = len;
dec.copy(buf, 3);
return buf;
} else if (v.type === 'pointer') {
dec = new Buffer(v.pointer, 'hex');
len = dec.length;
buf = new Buffer(2 + len);
buf[0] = 0x1c;
buf[1] = len;
dec.copy(buf, 2);
return buf;
} else if (v.type === 'lightfunc') {
dec = new Buffer(v.pointer, 'hex');
len = dec.length;
buf = new Buffer(4 + len);
buf[0] = 0x1d;
buf[1] = (v.flags >> 8) & 0xff;
buf[2] = v.flags & 0xff;
buf[3] = len;
dec.copy(buf, 4);
return buf;
} else if (v.type === 'heapptr') {
dec = new Buffer(v.pointer, 'hex');
len = dec.length;
buf = new Buffer(2 + len);
buf[0] = 0x1e;
buf[1] = len;
dec.copy(buf, 2);
return buf;
}
} else if (v === null) {
return new Buffer([ 0x17 ]);
} else if (typeof v === 'boolean') { } else if (typeof v === 'boolean') {
return new Buffer([ v ? 0x18 : 0x19 ]); return new Buffer([ v ? 0x18 : 0x19 ]);
} else if (typeof v === 'number') { } else if (typeof v === 'number') {
@ -930,10 +985,8 @@ function formatDebugValue(v) {
} }
} }
// XXX: missing support for various types (pointer, object, lightfunc, buffer) // Shouldn't come here.
// as they're not needed right now. throw new TypeError('value cannot be converted to dvalue: ' + JSON.stringify(v));
throw new TypeError('value cannot be converted to dvalue: ' + v);
} }
/* /*
@ -1386,10 +1439,10 @@ Debugger.prototype.sendGetBytecodeRequest = function () {
preformatted = preformatted.join('\n') + '\n'; preformatted = preformatted.join('\n') + '\n';
ret = { ret = {
"constants": consts, constants: consts,
"functions": funcs, functions: funcs,
"bytecode": bcode, bytecode: bcode,
"preformatted": preformatted preformatted: preformatted
}; };
return ret; return ret;
@ -2073,6 +2126,251 @@ DebugWebServer.prototype.emitLocals = function () {
this.socket.emit('locals', newMsg); this.socket.emit('locals', newMsg);
}; };
/*
* JSON debug proxy
*/
function DebugProxy(serverPort) {
this.serverPort = serverPort;
this.server = null;
this.socket = null;
this.targetStream = null;
this.inputParser = null;
// preformatted dvalues
this.dval_eom = formatDebugValue(DVAL_EOM);
this.dval_req = formatDebugValue(DVAL_REQ);
this.dval_rep = formatDebugValue(DVAL_REP);
this.dval_nfy = formatDebugValue(DVAL_NFY);
this.dval_err = formatDebugValue(DVAL_ERR);
}
DebugProxy.prototype.determineCommandNumber = function (cmdString, cmdNumber) {
var ret;
if (typeof cmdString === 'string') {
ret = debugCommandNumbers[cmdString];
}
ret = ret || cmdNumber;
if (typeof ret !== 'number') {
throw Error('cannot figure out command number for "' + cmdString + '" (' + cmdNumber + ')');
}
return ret;
};
DebugProxy.prototype.commandNumberToString = function (id) {
return debugCommandNames[id] || String(id);
};
DebugProxy.prototype.formatDvalues = function (args) {
if (!args) {
return [];
}
return args.map(function (v) {
return formatDebugValue(v);
});
};
DebugProxy.prototype.writeJson = function (val) {
this.socket.write(JSON.stringify(val) + '\n');
};
DebugProxy.prototype.writeJsonSafe = function (val) {
try {
this.writeJson(val);
} catch (e) {
console.log('Failed to write JSON in writeJsonSafe, ignoring: ' + e);
}
};
DebugProxy.prototype.disconnectJsonClient = function () {
if (this.socket) {
this.socket.destroy();
this.socket = null;
}
};
DebugProxy.prototype.disconnectTarget = function () {
if (this.inputParser) {
this.inputParser.close();
this.inputParser = null;
}
if (this.targetStream) {
this.targetStream.destroy();
this.targetStream = null;
}
};
DebugProxy.prototype.run = function () {
var _this = this;
console.log('Waiting for client connections on port ' + this.serverPort);
this.server = net.createServer(function (socket) {
console.log('JSON proxy client connected');
_this.disconnectJsonClient();
_this.disconnectTarget();
// A byline-parser is simple and good enough for now (assume
// compact JSON with no newlines).
var socketByline = byline(socket);
_this.socket = socket;
socketByline.on('data', function (line) {
try {
// console.log('Received json proxy input line: ' + line.toString('utf8'));
var msg = JSON.parse(line.toString('utf8'));
var first_dval;
var args_dvalues = _this.formatDvalues(msg.args);
var last_dval = _this.dval_eom;
var cmd;
if (msg.request) {
// "request" can be a string or "true"
first_dval = _this.dval_req;
cmd = _this.determineCommandNumber(msg.request, msg.command);
} else if (msg.reply) {
first_dval = _this.dval_rep;
} else if (msg.notify) {
// "notify" can be a string or "true"
first_dval = _this.dval_nfy;
cmd = _this.determineCommandNumber(msg.notify, msg.command);
} else if (msg.error) {
first_dval = _this.dval_err;
} else {
throw new Error('Invalid input JSON message: ' + JSON.stringify(msg));
}
_this.targetStream.write(first_dval);
if (cmd) {
_this.targetStream.write(formatDebugValue(cmd));
}
args_dvalues.forEach(function (v) {
_this.targetStream.write(v);
});
_this.targetStream.write(last_dval);
} catch (e) {
console.log(e);
_this.writeJsonSafe({
notify: '_Error',
args: [ 'Failed to handle input json message: ' + e ]
});
_this.disconnectJsonClient();
_this.disconnectTarget();
}
});
_this.connectToTarget();
}).listen(this.serverPort);
};
DebugProxy.prototype.connectToTarget = function () {
var _this = this;
console.log('Connecting to ' + optTargetHost + ':' + optTargetPort + '...');
this.targetStream = new net.Socket();
this.targetStream.connect(optTargetPort, optTargetHost, function () {
console.log('Debug transport connected');
});
this.inputParser = new DebugProtocolParser(
this.targetStream,
null,
optDumpDebugRead,
optDumpDebugPretty,
optDumpDebugPretty ? 'Recv: ' : null,
null,
null // console logging is done at a higher level to match request/response
);
// Don't add a '_value' key to numbers.
this.inputParser.readableNumberValue = false;
this.inputParser.on('transport-close', function () {
console.log('Debug transport closed');
_this.writeJsonSafe({
notify: '_Disconnecting'
});
_this.disconnectJsonClient();
_this.disconnectTarget();
});
this.inputParser.on('transport-error', function (err) {
console.log('Debug transport error', err);
_this.writeJsonSafe({
notify: '_Error',
args: [ String(err) ]
});
});
this.inputParser.on('protocol-version', function (msg) {
var ver = msg.protocolVersion;
console.log('Debug version identification:', msg.versionIdentification);
_this.writeJson({
notify: '_Connected',
args: [ msg.versionIdentification ] // raw identification string
});
if (ver !== 1) {
console.log('Protocol version ' + ver + ' unsupported, dropping connection');
}
});
this.inputParser.on('debug-message', function (msg) {
var t;
//console.log(msg);
if (typeof msg[0] !== 'object' || msg[0] === null) {
throw new Error('unexpected initial dvalue: ' + msg[0]);
} else if (msg.type === 'eom') {
throw new Error('unexpected initial dvalue: ' + msg[0]);
} else if (msg.type === 'req') {
if (typeof msg[1] !== 'number') {
throw new Error('unexpected request command number: ' + msg[1]);
}
t = {
request: _this.commandNumberToString(msg[1]),
command: msg[1],
args: msg.slice(2, msg.length - 1)
}
_this.writeJson(t);
} else if (msg[0].type === 'rep') {
t = {
reply: true,
args: msg.slice(1, msg.length - 1)
}
_this.writeJson(t);
} else if (msg[0].type === 'err') {
t = {
error: true,
args: msg.slice(1, msg.length - 1)
}
_this.writeJson(t);
} else if (msg[0].type === 'nfy') {
if (typeof msg[1] !== 'number') {
throw new Error('unexpected notify command number: ' + msg[1]);
}
t = {
notify: _this.commandNumberToString(msg[1]),
command: msg[1],
args: msg.slice(2, msg.length - 1)
}
_this.writeJson(t);
} else {
throw new Error('unexpected initial dvalue: ' + msg[0]);
}
});
this.inputParser.on('stats-update', function () {
});
};
/* /*
* Command line parsing and initialization * Command line parsing and initialization
*/ */
@ -2093,6 +2391,12 @@ function main() {
if (argv['http-port']) { if (argv['http-port']) {
optHttpPort = argv['http-port']; optHttpPort = argv['http-port'];
} }
if (argv['json-proxy-port']) {
optJsonProxyPort = argv['json-proxy-port'];
}
if (argv['json-proxy']) {
optJsonProxy = argv['json-proxy'];
}
if (argv['source-dirs']) { if (argv['source-dirs']) {
optSourceSearchDirs = argv['source-dirs'].split(path.delimiter); optSourceSearchDirs = argv['source-dirs'].split(path.delimiter);
} }
@ -2116,6 +2420,8 @@ function main() {
console.log(' --target-host: ' + optTargetHost); console.log(' --target-host: ' + optTargetHost);
console.log(' --target-port: ' + optTargetPort); console.log(' --target-port: ' + optTargetPort);
console.log(' --http-port: ' + optHttpPort); console.log(' --http-port: ' + optHttpPort);
console.log(' --json-proxy-port: ' + optJsonProxyPort);
console.log(' --json-proxy: ' + optJsonProxy);
console.log(' --source-dirs: ' + optSourceSearchDirs.join(' ')); console.log(' --source-dirs: ' + optSourceSearchDirs.join(' '));
console.log(' --dump-debug-read: ' + optDumpDebugRead); console.log(' --dump-debug-read: ' + optDumpDebugRead);
console.log(' --dump-debug-write: ' + optDumpDebugWrite); console.log(' --dump-debug-write: ' + optDumpDebugWrite);
@ -2126,12 +2432,19 @@ function main() {
// Create debugger and web UI singletons, tie them together and // Create debugger and web UI singletons, tie them together and
// start them. // start them.
var dbg = new Debugger(); if (optJsonProxy) {
var web = new DebugWebServer(); console.log('Starting in JSON proxy mode, JSON port: ' + optJsonProxyPort);
dbg.web = web;
web.dbg = dbg; var prx = new DebugProxy(optJsonProxyPort);
dbg.run(); prx.run();
web.run(); } else {
var dbg = new Debugger();
var web = new DebugWebServer();
dbg.web = web;
web.dbg = dbg;
dbg.run();
web.run();
}
} }
main(); main();

36
debugger/duk_debugcommands.yaml

@ -0,0 +1,36 @@
# Debug command names. A single map works for now because command names
# provided by client/target don't overlap.
- Reserved_0
- Status
- Print
- Alert
- Log
- Gc
- Reserved_6
- Reserved_7
- Reserved_8
- Reserved_9
- Reserved_10
- Reserved_11
- Reserved_12
- Reserved_13
- Reserved_14
- Reserved_15
- BasicInfo
- TriggerStatus
- Pause
- Resume
- StepInto
- StepOver
- StepOut
- ListBreak
- AddBreak
- DelBreak
- GetVar
- PutVar
- GetCallStack
- GetLocals
- Eval
- Detach
- DumpHeap
- GetBytecode

3
debugger/package.json

@ -20,7 +20,8 @@
"readline": "0.0.5", "readline": "0.0.5",
"util": "~0.10.3", "util": "~0.10.3",
"http": "0.0.0", "http": "0.0.0",
"yamljs": "~0.2.1" "yamljs": "~0.2.1",
"byline": "~4.2.1"
}, },
"main": "duk_debug.js" "main": "duk_debug.js"
} }

198
doc/debugger.rst

@ -36,7 +36,11 @@ Duktape debugging architecture is based on the following major pieces:
* A **debug API** to attach/detach a debugger to a Duktape heap. * A **debug API** to attach/detach a debugger to a Duktape heap.
* A **debug client**, running off target, which provides a user interface. * A **debug client**, running off target, which implements the other
debug protocol endpoint and provides a user interface.
* An optional **JSON debug protocol proxy** which provides an easier
JSON-based interface for talking to the debug target.
This document describes these pieces in detail. This document describes these pieces in detail.
@ -84,6 +88,12 @@ debug protocol version, so your debug client may need changes from time to
time as the Duktape debug protocol evolves. The debug protocol is versioned time as the Duktape debug protocol evolves. The debug protocol is versioned
with the same semantic versioning principles as the Duktape API. with the same semantic versioning principles as the Duktape API.
You can implement the binary debug protocol directly in your debug client,
but an easier option is to use the JSON mapping of the debug protocol which
is much more user friendly. Duktape includes a proxy server which converts
between the JSON mapping and the binary debug protocol (which actually runs
on the target).
Example debug client and server Example debug client and server
------------------------------- -------------------------------
@ -657,7 +667,7 @@ some cases:
| | | length in network order and buffer | | | | length in network order and buffer |
| | | data follows initial byte | | | | data follows initial byte |
+-----------------------+-----------+---------------------------------------+ +-----------------------+-----------+---------------------------------------+
| 0x14 <uint16> <data> | buffer | 2-byte string, unsigned 16-bit buffer | | 0x14 <uint16> <data> | buffer | 2-byte buffer, unsigned 16-bit buffer |
| | | length in network order and buffer | | | | length in network order and buffer |
| | | data follows initial byte | | | | data follows initial byte |
+-----------------------+-----------+---------------------------------------+ +-----------------------+-----------+---------------------------------------+
@ -919,7 +929,8 @@ gracefully in a few common cases (but certainly not all).
Text representation of dvalues and debug messages Text representation of dvalues and debug messages
------------------------------------------------- -------------------------------------------------
**This is an informative convention only.** **This is an informative convention only used in this document and in
duk_debug.js dumps.**
The Duktape debug client uses the following convention for representing The Duktape debug client uses the following convention for representing
dvalues as text: dvalues as text:
@ -934,8 +945,7 @@ dvalues as text:
of the codepoints U+0080...U+00FF which unfortunately looks funny (ASCII of the codepoints U+0080...U+00FF which unfortunately looks funny (ASCII
only serialization would be preferable). only serialization would be preferable).
* Other types are JSON encoded from their internal representation, see * Other types are JSON encoded like in the JSON mapping, see below.
``duk_debug.js`` for details.
Debug messages are then simply represented as one-liners containing all the Debug messages are then simply represented as one-liners containing all the
related dvalues (including message type marker and EOM) separate by spaces. related dvalues (including message type marker and EOM) separate by spaces.
@ -960,34 +970,107 @@ string inside Duktape. Note that some Duktape strings are intentionally
invalid UTF-8 so mapping to Unicode is not always an option. This string invalid UTF-8 so mapping to Unicode is not always an option. This string
mapping is also used to represent buffer data. mapping is also used to represent buffer data.
JSON representation of dvalues and debug messages JSON mapping for debug protocol
------------------------------------------------- ===============================
The mapping described in this section is used to map debug dvalues and
messages into JSON values. The mapping is used to implement a JSON
debug proxy which allows a debug client to interact with a debug target
using clean JSON messages alone without implementing the binary protocol
at all.
JSON representation of dvalues
------------------------------
* Unused::
{ "type": "unused" }
**Not currently used, might be useful for a debugger JSON proxy for easier * Undefined::
debug client writing. This is an informative convention only.**
Debug values and debug messages can also be mapped 1:1 to JSON objects as { "type": "undefined" }
described below. This might be useful e.g. to provide a JSON debug proxy
which would make it easier to write a custom debugger UI.
Dvalues: * Null, true, and false map directly to JSON::
* All integers map directly to JSON number type. null
true
false
* Integers map directly to JSON number type::
1234
* Any numbers that can't be represented without loss as JSON numbers
(e.g. infinity, NaN, negative zero) are expressed as::
// data contains IEEE double in big endian hex encoded bytes
// (here Math.PI)
{ "type": "number", "data": "400921fb54442d18" }
* Strings are mapped like in the text representation, i.e. bytes 0x00...0xff * Strings are mapped like in the text representation, i.e. bytes 0x00...0xff
map to Unicode codepoints U+0000...U+00FF, to maintain byte exactness and map to Unicode codepoints U+0000...U+00FF::
to represent non-UTF-8 strings correctly. Buffers are expressed as strings.
// the 4-byte string 0xde 0xad 0xbe 0xef
"\u00de\00ad\00be\00ef"
This representation is used because it is byte exact, represents non-UTF-8
strings correctly, but is still human readable for most practical (ASCII)
strings.
* Buffer data is represented in hex encoded form wrapped in an object::
{ "type": "buffer", "data": "deadbeef" }
* **XXX: pointers, buffers, etc** * The message framing dvalues (EOM, REQ, REP, NFY, ERR) are not visible in
the JSON protocol. They are used by ``duk_debug.js`` internally with the
format::
{ "type": "eom" }
{ "type": "req" }
{ "type": "rep" }
{ "type": "err" }
{ "type": "nfy" }
* Object::
// class is a number, pointer is hex-encoded
{ "type": "object", "class": 10, "pointer": "deadbeef" }
* Pointer::
// pointer is hex-encoded
{ "type": "pointer", "pointer": "deadbeef" }
* Lightfunc::
// flags is a 16-bit integer represented as a JSON number,
// pointer is hex-encoded
{ "type": "lightfunc", "flags": 1234, "pointer": "deadbeef" }
* Heap pointer::
// pointer is hex-encoded
{ "type": "heapptr", "pointer": "deadbeef" }
JSON representation of debug messages
-------------------------------------
Messages are represented as JSON objects, with the message type marker and the Messages are represented as JSON objects, with the message type marker and the
EOM marker removed, as follows. EOM marker removed, as follows.
Request messages have a 'request' key which contains the command number, and Request messages have a 'request' key which contains the command name (if
'args' which contains remaining dvalues (EOM omitted):: known) or "true" (if not known), a 'command' key which contains the command
number, and 'args' which contains remaining dvalues (EOM omitted)::
{
"request": "AddBreak",
"command": 24,
"args": [ "foo.js", 123 ]
}
{ {
"request": 24, "request": true,
"command": 24,
"args": [ "foo.js", 123 ] "args": [ "foo.js", 123 ]
} }
@ -1008,14 +1091,76 @@ contain the error arguments (EOM omitted)::
"args": [ 2, "no space for breakpoint" ] "args": [ 2, "no space for breakpoint" ]
} }
Notify messages have a 'notify' key with the notify command number, and an Notify messages have a 'notify' key with the notify command name (if known)
'args' for arguments (EOM omitted):: or "true" (if not known), a 'command' key which contains the command number,
and an 'args' for arguments (EOM omitted)::
{ {
"notify": 1, "notify": "Status",
"command": 1,
"args": [ 0, "foo.js", "frob", 123, 808 ] "args": [ 0, "foo.js", "frob", 123, 808 ]
} }
{
"notify": true,
"command": 1,
"args": [ 0, "foo.js", "frob", 123, 808 ]
}
If an argument list is empty, 'args' can be omitted from any message.
The request and notify message contain both a request/notify command name and
a number. The intent is to allow debug clients to use command names (rather
than numbers). The command name/number is resolved as follows:
* If command name is present, look up the command name from command metadata.
If the command is known, use the command number in the command metadata and
ignore a possible 'command' key.
* If command number is present, use it verbatim if the name lookup failed.
* If no command number is present, fail.
Other JSON messages
-------------------
In addition to the core message formats above, there are a few custom messages
for debug protocol version info and transport events. These are expressed as
"notify" messages with a special command name beginning with an underscore, and
no command number.
When connecting to a debug target, a version identification line is received.
This line doesn't follow the dvalue format, so it is transmitted specially::
{
"notify": "_Connected",
"args": [ "1 10199 v1.1.0-173-gecd806e-dirty duk command built from Duktape repo" ]
}
When a transport error occurs (not necessarily a terminal error)::
{
"notify": "_Error",
"args": [ "some kind of error" ]
}
When the JSON connection is just about to be disconnected::
{
"notify": "_Disconnecting"
}
JSON protocol line formatting
-----------------------------
JSON messages are sent by encoding them in compact one-liner form and
terminating a message with a newline (single LF character, 0x0a).
(Note that the examples above are formatted in multiline format which
is **not** allowed; this is simply for clarity.)
This convention makes is easy to read and write messages. Messages can
be easily cut-pasted, and message logs can be grepped effectively.
Extending the protocol and version compatibility Extending the protocol and version compatibility
================================================ ================================================
@ -2617,10 +2762,3 @@ Currently the list of breakpoints is not cleared by attach or detach, so if
you detach and then re-attach, old breakpoints are still set. The debug you detach and then re-attach, old breakpoints are still set. The debug
client can just delete all breakpoints on attach, but it'd be cleaner to client can just delete all breakpoints on attach, but it'd be cleaner to
remove the breakpoints on either attach or detach. remove the breakpoints on either attach or detach.
Complete the JSON format and add a JSON proxy
---------------------------------------------
A JSON proxy would make it much easier to implement a debug client, as debug
messages can be read and written as simple JSON messages with existing JSON
libraries.

21
website/guide/debugger.html

@ -13,7 +13,10 @@ very minimal memory footprint.</p>
<ul> <ul>
<li>Duktape provides a built-in <b>debug protocol</b> which is the same for all <li>Duktape provides a built-in <b>debug protocol</b> which is the same for all
applications. The application doesn't need to parse or understand the applications. The application doesn't need to parse or understand the
debug protocol.</li> debug protocol. The debug protocol is a compact binary protocol so that
it works well on low memory targets with low speed connectivity. There
is a <b>JSON mapping</b> for the debug protocol and a JSON debug proxy to
make it easier to integrate a debug client.</li>
<li>The debug protocol runs over a reliable, stream-based <b>debug transport</b>. <li>The debug protocol runs over a reliable, stream-based <b>debug transport</b>.
To maximize portability, the concrete transport is provided by application To maximize portability, the concrete transport is provided by application
code as a set of callbacks implementing a stream interface. A streamed code as a set of callbacks implementing a stream interface. A streamed
@ -21,7 +24,8 @@ very minimal memory footprint.</p>
usage very low.</li> usage very low.</li>
<li>A <b>debug client</b> terminates the transport connection and uses the Duktape <li>A <b>debug client</b> terminates the transport connection and uses the Duktape
debug protocol to interact with Duktape internals: pause/resume, stepping, debug protocol to interact with Duktape internals: pause/resume, stepping,
breakpoints, eval, etc.</li> breakpoints, eval, etc. You can also use the JSON debug proxy for easier
integration.</li>
<li>A very narrow <b>debug API</b> is used by the application code to attach and <li>A very narrow <b>debug API</b> is used by the application code to attach and
detach a debugger, and to provide the callbacks needed to implement the detach a debugger, and to provide the callbacks needed to implement the
debug transport. All other debug activity happens through the debug debug transport. All other debug activity happens through the debug
@ -47,11 +51,14 @@ pieces you need to get started with debugging using a TCP transport:</p>
<a href="https://github.com/svaarala/duktape/blob/master/debugger/">duk_debug.js</a></li> <a href="https://github.com/svaarala/duktape/blob/master/debugger/">duk_debug.js</a></li>
</ul> </ul>
<p>The Node.js based debugger web UI can connect to the Duktape command line, <p>The Node.js based debugger web UI (<code>duk_debug.js</code>) can connect
but can also talk directly with any other target implementing a TCP transport. to the Duktape command line, but can also talk directly with any other target
You can also customize it to use a different transport or use a proxy which implementing a TCP transport. You can also customize it to use a different
converts between TCP and your custom transport. It's also possible to write transport or use a proxy which converts between TCP and your custom transport.
your own debug client from scratch and e.g. integrate it to a custom IDE.</p> It's also possible to write your own debug client from scratch and e.g.
integrate it to a custom IDE. You can integrate directly with a debug target
using the binary debug protocol, or use the JSON proxy provided by
<code>duk_debug.js</code>.</p>
<p>For more details on the implementation and how to get started, see:</p> <p>For more details on the implementation and how to get started, see:</p>
<ul> <ul>

Loading…
Cancel
Save