Remove thr->callstack as a monolithic array and replace it with a linked list
of duk_activations. thr->callstack_curr is the current call (or NULL if no
call is in progress), and act->parent chains to a previous call or NULL.
thr->callstack_top is kept because it's needed by some internals at present;
it may be removed in the future.
When the flag is set, there is either no subclass C struct for the
duk_hobject, or there is a subclass C struct but there are no references
needing DECREF/marking in the struct.
This allows DECREF and mark-and-sweep to handle duk_hobjects with less
overhead for the common cases of plain objects and arrays (and some other
less commonly collected structs like duk_hnatfunc).
Also change Duktape.Thread.prototype internal class from Thread to Object:
with the other changes internal code now assumes that if an object's class
is Thread, it has the duk_hthread memory layout which wouldn't be the case
for Duktape.Thread.prototype.
One bottleneck in refzero and mark-and-sweep handling is checking whether an
object has an own or inherited _Finalizer property. This check walked the
prototype chain and did a property lookup for every object. Because a
finalizer is usually not present, the prototype chain would almost always be
walked to completion.
Improve this behavior by:
* Adding a DUK_HOBJECT_FLAG_HAVE_FINALIZER flag. The flag is set when the
object has an own _Finalizer property with a callable value, and cleared
otherwise. The flag is *only* set by duk_set_finalizer(), so any other
means of changing the internal _Finalizer property will leave the flag out
of sync (which causes a finalizer run to be skipped).
* Adding duk_hobject_has_finalizer_fast() which checks for finalizer existence
by walking the prototype chain, but only checking the flag, not the property
table.
* Use the fast finalizer check in refzero and mark-and-sweep.
Out-of sync cases:
* If the flag is set but there is no actual finalizer, the object will go
through finalizer processing when garbage collecting. This is harmless:
the finalizer call will fail and the object will be garbage collected, but
with some potential delay (especially for mark-and-sweep).
* If the flag is cleared but there is an actual finalizer, the finalizer will
be ignored.
Related changes:
* When duk_dump_function() is called, zero DUK_HOBJECT_FLAG_HAVE_FINALIZER on
serialization, so it won't be set when the function is loaded back. If this
is not done, the loaded function will (harmlessly) go through finalizer
processing when garbage collected.
* Update debugger artificial properties to include "have_finalizer" flag.
Other changes:
* A few DUK_UNLIKELY() attributes for prototype sanity limits which are
almost never hit.
Throw notify might be thrown from an empty callstack e.g. if duk_throw()
is called by C code outside of any call. If the throw argument is not an
error instance, the code would attempt to access the topmost activation
even if one didn't exist.
* Replace the two alternative algorithms with a single one which works for
both desktop and low memory cases.
* Basic algorithm is a hash table with size 2^N, hash mask is simply
(size - 1), e.g. if size is 0x100, mask is 0xFF. duk_hstring has a 'next'
pointer (single linked list) for chaining strings mapping to the same
slot.
* Change plain buffers to inherit from Uint8Array. This affects a lot of
small things like Object.prototype.toString() output, enumeration of plain
buffers, etc. It also changes JSON serialization for plain buffers because
the index properties are enumerable as with Uint8Array instances.
* Disable JSON stringify fastpath for plain buffers for now so that the
virtual index properties get serialized correctly.
* Remove ArrayBuffer non-standard virtual properties.
* Remove DataView non-standard virtual properties.
* Move .byteLength, .byteOffset, .BYTES_PER_ELEMENT, and .buffer into
inherited getters as required in ES6. However, the .length property
remains a virtual own property for now (it too is an inherited getter
in ES6).
* Move ArrayBuffer.allocPlain() and ArrayBuffer.plainOf() to
Uint8Array.allocPlain() and Uint8Array.plainOf() to match the
semantics change for plain buffers.
* Fix Node.js buffer .slice() behavior, the returned Node.js buffer
would have ArrayBuffer.isView == 0 which doesn't match the revised
Node.js behavior (Buffers being Uint8Array instances)
* Reject ArrayBuffers with a view offset/length in Node.js Buffer .slice()
rather than accept such ArrayBuffers without actually respecting the
view offset/length.
* Allow a plain buffer or a lightfunc as a constructor "replacement object"
return value.
* Comment on duk_buffer_to_string() safety w.r.t. potentially pushing
Symbol values.
* Went through all duk_push_(l)string() call sites too.
* Minor footprint optimization for pushing empty strings (use more
compact internal helper).
* Strings with 0xFF byte prefix are considered special symbols: they have
typeof "symbol" but still mostly behave as strings (e.g. allow ToString)
so that existing code dealing with internal keys, especially inside
Duktape, can work with fewer changes.
* Strings with 0x80 byte prefix are global symbols, e.g. Symbol.for('foo')
creates the byte representatio: 0x80 "foo"
* Strings with 0x81 byte prefix are unique symbols; the 0x81 byte is followed
by the Symbol description, and an internal string component ensuring
uniqueness is separated by a 0xFF byte (which can never appear anywhere in
an extended UTF-8 string). The unique suffix is up to Duktape internals,
currently two 32-bit counters are used. For example:
0x81 "mySymbol" 0xFF "0-17".
* Well-known symbols use the 0x81 prefix but lack a unique suffix, so their
format is 0x81 <description> 0xFF.
* ES6 distinguishes between an undefined symbol description and an empty
string symbol description. This distinction is not currently visible via
Ecmascript bindings but may be visible in the future. Append an extra
0xFF to the unique suffix when the description is undefined, i.e.
0x81 0xFF <unique suffix> 0xFF.
* Callstack index is required for most debug commands which accept one
* Index comes before other parameters, e.g. Eval has the signature:
REQ 0x1e <index> <code>
* Eval accepts null for the callstack index, indicating an indirect
eval.
These can be used whenever we're 100% certain that the value stack index
exists and the type matches expected type. When these are true, a
duk_hstring, duk_hbuffer, or duk_hobject pointer fetch can be inlined to
small code.
* Plain buffers still inherit from ArrayBuffer.prototype.
* Plain buffers won't object coerce, so Object(plainBuffer) fails.
* All buffer object related methods throw an error; their function bodies
are essentially empty. Note that this includes bindings such as
String.fromBuffer(), ArrayBuffer.allocPlain(), ArrayBuffer.plainOf(),
and so on. In essence, you can index plain buffers in Ecmascript but
the buffer values must be created via the C API.
* Duktape custom bindings like Duktape.dec('hex', 'deadbeef') still work
and produce plain buffers.
Change handling of plain buffers so that they behave like ArrayBuffer
instances to Ecmascript code, with limitations such as not being
extensible and all properties being virtualized. This simplifies
Ecmascript code as plain buffers are just lightweight ArrayBuffers
(similarly to how lightfuncs appear as function objects). There are
a lot of small changes in how the built-in objects and methods, and
the C API deals with plain buffer values.
Also make a few small changes to plain pointer and lightfunc handling
to improve consistency with how plain buffers are now handled.
Improve readability by doing the following renames:
* duk_hcompiledfunction -> duk_hcompfunc
* duk_hnativefunction -> duk_hnatfunc
* duk_hbufferobject -> duk_hbufobj
Corresponding renames for all caps defines.
Scanbuild fixes:
* duk_cmdline.c realloc() failure could leak memory, fixed.
* Other fixed scanbuild errors were harmless.
* Remaining errors are invalid memory access in panic (intentional
segfault) and a Date handling case where garbage data is intentionally
assigned as doing so is harmless and intended.
Avoid unnecessary index re-check in num coercion.
* GetBytecode defaults to callstack top as before, with the small change
that a missing callstack top (empty callstack) results in an error instead
of a dummy fake response.
* GetBytecode also accepts a callstack index (-1...-N) and an object/heap
target pointer.
* Fix minor bugs in duk_debug_read_tval() call sites: they assumed that a
duk_tval was pushed even when the read failed and nothing was pushed.
The fix is to stop processing an inbound message if duk_debug_read_tval()
fails and avoid making assumptions about value stack shape.
When Duktape receives an AppRequest, it pushes all the dvalues in the
message to the value stack and calls the request callback. That callback
can process the message and optionally push its own values to the stack,
which will be sent back to the client as a reply.
It is also possible for the target to send application-specific notifys
to the client by calling duk_debugger_notify(). These will be received
by the client as an AppNotify message.
NULL all debugger transport I/O callbacks (i.e. callbacks other than
detaching_cb) whenever the transport indicates an error has occurred.
By the API contract we're not allowed to call any of them after transport
read/write error so it's safest to NULL them all on a transport error.
This is not strictly necessary because only the write_cb is used by
detach1() currently before NULLing the callbacks anyway. However, NULLing
them early on transport error ensures there's no potential to call them
incorrectly even when the code is reorganized later.
Fix a bug in `duk_debug_write_byte()` where a transport write callback
error (indicating a transport detached/broken condition) did not NULL
the transport `dbg_write_cb` as intended.
As a result, the detach1() handler (which runs immediately afterwards)
would try to write a "Detaching" notify and invoke the transport write
callback after it already returned an error. This is a violation of the
debug transport contract.
Fixed by making duk_debug_write_byte() simply call duk_debug_write_bytes()
which has correct handling for the write_cb NULLing. This reduces call sites
at the cost of being a little less optimal. However, the same approach is
already used for read helpers so this change makes the handling symmetric.
Fix two separate bugs:
* Potential to re-enter the debugger recursively when a detach happened
inside the debugger message loop.
* Potential for detach initiated from outside the debugger message loop to
be never finalized.
The re-entry bug was caused by `dbg_processing` being set to 0 in detach1()
while in the debugger message loop. This caused the potential for the
debugger code to be re-entered recursively before the detach was finalized
using detach2(), which could have various harmful side effects. The fix is
to keep `dbg_processing` set to 1 at all times when still in the debugger
message loop.
The detach finalization bug was triggered by a debug transport callback
called outside the debugger message loop indicating the transport connection
had broken (this would concretely happen e.g. for a periodic Status notify).
This caused detach1() to run which, among other things, sets dbg_read_cb to
NULL and dbg_detaching to 1.
The intent is then for detach2() to finalize the detach (this is separated
from detach1() to allow reattach-during-detached-callback to be handled
correctly). However, when dbg_read_cb was set to NULL outside the debugger
message loop, the message loop would never be entered again -- and only the
message loop has handling for finalizing a detach using detach2().
The fix for this is to allow the executor to enter the debugger message loop
when a detach is pending (dbg_read_cb == NULL and dbg_detaching == 1) so that
the detach can be correctly finalized.
The message loop detach2() handling now also supports the corner case where
user code reattaches in the detached callback but a detach is immediately
triggered while still inside the detached callback. This may happen because
`duk_debugger_attach()` writes the debugger protocol handshake line inline
inside the API call using the transport write callback, and that callback
may fail and indicate a transport error.
The fix is to then issue another detached callback etc, until we are either
cleanly detached or cleanly attached and can proceed.
Finally, the control flow was changed so that the `dbg_processing` flag is
set and cleared inside the debugger message loop now rather than in several
separate call sites outside of it. This should be less error prone.
Both bugs fixed in this commit were introduced in Duktape 1.4.0 as part of
adding the Detaching notify and splitting detach handling into detach1()
and detach2(). As such, the fixes don't need to be backported to Duktape
1.3.x etc.
Separate call handling into three functions: unprotected call wrapper,
protected call wrapper, and a shared inner call handler. Having setjmp
only in a smaller wrapper reduces problems with compiler, including:
* Spurious volatile warnings
* Performance impact on compilers where optimizations must be disabled for
any function involving a setjmp
Also make every DUK_SETJMP() call site compatible with a later refactoring
to supporting C++ exceptions by handling the success case in the then-block
and the error in the else-block. This allows a try-catch structure to
trivially replace the setjmp construct.
Split call error handling to separate function which makes it easier to
share for C++ exception handling which may need a catch (xxx) for Duktape
errors and a separate catch (...) for other errors accidentally thrown by
user code.
Other minor cleanups in call handling.
Sami Vaarala
Bruce Pascoe