Remove thr->callstack as a monolithic array and replace it with a linked list
of duk_activations. thr->callstack_curr is the current call (or NULL if no
call is in progress), and act->parent chains to a previous call or NULL.
thr->callstack_top is kept because it's needed by some internals at present;
it may be removed in the future.
* Add a h_assert_refcount field to duk_heaphdr when assertions are enabled.
* When doing mark-and-sweep, clear h_assert_refcount, perform mark-and-sweep
processing normally, and assert for correct refcounts for objects that
remain in heap_allocated after sweeping. (Refcounts for objects prior to
sweeping won't match those computed via reachability roots.)
* Improve FASTREFS asserts for refcounting and mark-and-sweep.
When the flag is set, there is either no subclass C struct for the
duk_hobject, or there is a subclass C struct but there are no references
needing DECREF/marking in the struct.
This allows DECREF and mark-and-sweep to handle duk_hobjects with less
overhead for the common cases of plain objects and arrays (and some other
less commonly collected structs like duk_hnatfunc).
Also change Duktape.Thread.prototype internal class from Thread to Object:
with the other changes internal code now assumes that if an object's class
is Thread, it has the duk_hthread memory layout which wouldn't be the case
for Duktape.Thread.prototype.
The refcount field must be chosen so that there's never any wrapping,
otherwise memory unsafe behavior will happen.
For the default configuration a size_t is used for refcount; it's
guaranteed not to wrap because there can't be enough references to
wrap it. For low memory targets it's possible to configure 16-bit
refcounts. This is fine as long as memory pools are small enough
to never contain more than 2^16-1 references.
One bottleneck in refzero and mark-and-sweep handling is checking whether an
object has an own or inherited _Finalizer property. This check walked the
prototype chain and did a property lookup for every object. Because a
finalizer is usually not present, the prototype chain would almost always be
walked to completion.
Improve this behavior by:
* Adding a DUK_HOBJECT_FLAG_HAVE_FINALIZER flag. The flag is set when the
object has an own _Finalizer property with a callable value, and cleared
otherwise. The flag is *only* set by duk_set_finalizer(), so any other
means of changing the internal _Finalizer property will leave the flag out
of sync (which causes a finalizer run to be skipped).
* Adding duk_hobject_has_finalizer_fast() which checks for finalizer existence
by walking the prototype chain, but only checking the flag, not the property
table.
* Use the fast finalizer check in refzero and mark-and-sweep.
Out-of sync cases:
* If the flag is set but there is no actual finalizer, the object will go
through finalizer processing when garbage collecting. This is harmless:
the finalizer call will fail and the object will be garbage collected, but
with some potential delay (especially for mark-and-sweep).
* If the flag is cleared but there is an actual finalizer, the finalizer will
be ignored.
Related changes:
* When duk_dump_function() is called, zero DUK_HOBJECT_FLAG_HAVE_FINALIZER on
serialization, so it won't be set when the function is loaded back. If this
is not done, the loaded function will (harmlessly) go through finalizer
processing when garbage collected.
* Update debugger artificial properties to include "have_finalizer" flag.
Other changes:
* A few DUK_UNLIKELY() attributes for prototype sanity limits which are
almost never hit.
* Replace the two alternative algorithms with a single one which works for
both desktop and low memory cases.
* Basic algorithm is a hash table with size 2^N, hash mask is simply
(size - 1), e.g. if size is 0x100, mask is 0xFF. duk_hstring has a 'next'
pointer (single linked list) for chaining strings mapping to the same
slot.
* Change plain buffers to inherit from Uint8Array. This affects a lot of
small things like Object.prototype.toString() output, enumeration of plain
buffers, etc. It also changes JSON serialization for plain buffers because
the index properties are enumerable as with Uint8Array instances.
* Disable JSON stringify fastpath for plain buffers for now so that the
virtual index properties get serialized correctly.
* Remove ArrayBuffer non-standard virtual properties.
* Remove DataView non-standard virtual properties.
* Move .byteLength, .byteOffset, .BYTES_PER_ELEMENT, and .buffer into
inherited getters as required in ES6. However, the .length property
remains a virtual own property for now (it too is an inherited getter
in ES6).
* Move ArrayBuffer.allocPlain() and ArrayBuffer.plainOf() to
Uint8Array.allocPlain() and Uint8Array.plainOf() to match the
semantics change for plain buffers.
* Fix Node.js buffer .slice() behavior, the returned Node.js buffer
would have ArrayBuffer.isView == 0 which doesn't match the revised
Node.js behavior (Buffers being Uint8Array instances)
* Reject ArrayBuffers with a view offset/length in Node.js Buffer .slice()
rather than accept such ArrayBuffers without actually respecting the
view offset/length.
* Allow a plain buffer or a lightfunc as a constructor "replacement object"
return value.
* Plain buffers still inherit from ArrayBuffer.prototype.
* Plain buffers won't object coerce, so Object(plainBuffer) fails.
* All buffer object related methods throw an error; their function bodies
are essentially empty. Note that this includes bindings such as
String.fromBuffer(), ArrayBuffer.allocPlain(), ArrayBuffer.plainOf(),
and so on. In essence, you can index plain buffers in Ecmascript but
the buffer values must be created via the C API.
* Duktape custom bindings like Duktape.dec('hex', 'deadbeef') still work
and produce plain buffers.
Improve readability by doing the following renames:
* duk_hcompiledfunction -> duk_hcompfunc
* duk_hnativefunction -> duk_hnatfunc
* duk_hbufferobject -> duk_hbufobj
Corresponding renames for all caps defines.
- Set FINALIZED only when actually running finalizer to prevent
a finalizer running twice unless explicitly rescued (and the
flag cleared by the rescuing mark-and-sweep or refzero code).
- Add a guard to avoid re-finalizing until FINALIZED is explicitly
cleared on rescue by either mark-and-sweep or refcounting.
- Prevent mark-and-sweep and refzero from running when heap is
destroyed and finalizers are forcibly executed.
- Add a mark-and-sweep flag to skip finalizers: move any finalizable
objects back to heap_allocated. This is needed for correct
finalizer handling in heap destruction.
- Add second finalizer argument; if true, it indicates that the
heap is being destroyed and rescuing an object is not possible.
Finalizer should therefore free all native resources without
relying on the finalizer to be called again.
- Add multiple finalizer rounds for heap destruction to deal with
finalizers which create further finalizable objects. Also add
a sanity limit for this process to catch runaway finalizers.
- Explicit Proxy check just before running finalizer: don't run
finalizer for Proxy objects even when call site is buggy.
Src fiexs
Change the current value stack policy from "unused above top" to either
"undefined above top" or "garbage above top", depending on a config
option. Change mark-and-sweep and debug print code to only process
entries between [0,top[ in either case.
Both policies have potential upsides and downsides based on performance
measurement. This commit provides both policies; "undefined above top"
will probably be the only policy left however, because "garbage above
top" is only better in a few cases and mostly without refcounts.
Other minor improvements:
- Rework index validation to use duk_uidx_t and more shared code.
- Add cached thr->valstack_size value
When refzero runs, call a fake finalizer for every object (other than the
fake finalizer itself) so ensure call-related side effects occur for every
refzero.
Also remove the NULL check for refcounts: call sites where a NULL target value
(duk_tval * or duk_heaphdr *) may occur must use a NULL-tolerant variant
explicitly. This is for performance and to reduce call site size when the
refcount operations are inlined.
Using inline macros is basically the same as a forced inline when using a
good compiler and the single source file model. Macros make this more
portable, and will allow refcount updates to happen even directly from
Duktape API macros later.
Also remove mostly unused old debug code.
Debug code doesn't have access to 'heap' so it cannot decode pointers.
Cause an #error for now if both debug prints and pointer compression
are enabled at the same time.
Remove duk_debug_hobject.c from make and dist. It was out of date and
not used in practice anymore.
Memory optimization work for very low memory devices (96 to 256kB system RAM).
Overall changes are:
- 16-bit fields for various internal structures to reduce their size
- Heap pointer compression to reduce pointer size to 16 bits
When DUK_OPT_LIGHTFUNC_BUILTINS and the new low memory options are enabled,
Duktape initial heap memory usage is about 23kB (compared to baseline of
about 45kB) on x86.
Unless low memory feature options are enabled, there should be no visible
changes to Duktape behavior.
More detailed changes:
- 16-bit changes for duk_heaphdr: pointer compression, refcount
- 16-bit changes for duk_hstring: hash, blen, and clen can all be 16 bits,
use 0xFFFF as string byte length limit (call sites ensure this limit is
never exceeded)
- 16-bit changes for duk_hbuffer, use 0xFFFF as buffer length limit
- 16-bit fields for hobject size (entry part, array part), drop hash part
since it's not usually needed for extremely low memory environments
- 16-bit changes for duk_hcompiledfunction
- Heap pointer packing for stringtable
- Heap pointer packing for 'strs' built-in strings list (saves around 600
to 700 bytes but may not be a good tradeoff because call site size will
increase)
Other changes:
- Heaphdr NULL init fix. The original macros were broken: the double/single
linked macro variants were the wrong way around. Now sets through macro
to work properly with compressed pointers.
- Rename duk_hbuffer CURR_DATA_PTR -> DATA_PTR to reduce macro length
(previous name was tediously long)
- Rename buffer "usable_size" to "alloc_size" throughout as they have been
the same for a while now (they used to differ when buffer had an extra NUL).
- Add memory optimization markers to Duktape.env (pointer compression and
individual 16-bit field options)
- Rename a few internal fields for clarity: duk_hobject 'p' to 'props',
heap->st to heap->strtable
- Add a safety check for buffer alloc size (should not be triggered but
prevents wrapping if call sites don't properly check for sizes)
- Other minor cleanups
A lot of changes to add preliminary lightfunc support:
* Add LIGHTFUNC tagged type to duk_tval.h and API.
* Internal changes for preliminary to support lightfuncs in call handling
and other operations (FIXMEs left in obvious places where support is
still missing after this commit)
* Preliminary Ecmascript and API testcases for lightfuncs
Detailed notes:
* Because magic is signed, reading it back involves sign extension which is
quite verbose to do in C. Use macros for reading the magic value and other
bit fields encoded in the flags.
* Function.prototype.bind(): the 'length' property of a bound function now
comes out wrong. We could simply look up the virtual 'length' property
even if h_target is NULL: no extra code and binding is relatively rare in
hot paths. Rewrite more cleanly in any case.
* The use flag DUK_USE_LIGHTFUNC_BUILTINS controls the forced lightfunc
conversion of built-ins. This results in non-compliant built-ins but
significant memory savings in very memory poor environments.
* Reject eval(), Thread.yield/resume as lightfuncs. These functions have
current assertions that they must be called as fully fledged functions.
* Lightfuncs are serialized like ordinary functions for JSON, JX, and JC
by this diff.
* Add 'magic' to activation for lightfuncs. It will be needed for lightweight
functions: we don't have the duk_tval related to the lightfunc, so we must
copy the magic value to the activation when a call is made.
* When lightfuncs are used as property lookup base values, continue property
lookup from the Function.prototype object. This is necessary to allow e.g.
``func.call()`` and ``func.apply()`` to be used.
* Call handling had to be reworked for lightfuncs, especially how bound
function chains are handled. This is a relatively large change but is
necessary to support lightweight functions properly in bound function
resolution.
The current solution is not ideal. The bytecode executor will first try an
ecma-to-ecma call setup which resolves the bound function chain first. If
the final, unbound function is not viable (a native function) the call setup
returns with an error code. The caller will then perform a normal call.
Although bound function resolution has already been done, the normal call
handling code will re-do it (and detect there is nothing to do).
This situation could be avoided by decoupling bound function handling and
effective this binding computation from the actual call setup. The caller
could then to do this prestep first, and only then decide whether to use an
ecma-to-ecma call or an ordinary heavyweight call.
Remove duk__find_nonbound_function as unused.
* Use indirect magic to allow LIGHTFUNCs for Date. Most of the built-in
functions not directly eligible as lightfuncs are the Date built-in methods,
whose magic values contain too much information to fit into the 8-bit magic
of a LIGHTFUNC value.
To work around this, add an array (duk__date_magics[]) containing the
actual control flags needed by the built-ins, and make the Date built-in
magic value an index into this table. With this change Date built-ins are
successfully converted to lightfuncs.
Testcase fixes:
- Whitespace fixes
- Print error for indirect eval error to make diagnosis easier
- Fix error string to match errmsg updated in this branch
Sami Vaarala