* Add internal wrappers also to DUK_MEMMOVE(), DUK_MEMSET(), and
DUK_MEMZERO().
* Use the wrappers everywhere for consistency: the zero-size cases will then
always be safe, and if the target is fine with invalid pointers in the zero
size case, the whole check can be omitted easily.
* Remove a few zero size checks as they're no longer necessary.
When noreturn attribute is not available, avoid warnings by e.g. returning
dummy values (never actually used because of e.g. a longjmp) with the
DUK_WO_NORETURN() macro.
Clang warns about tmp being used uninitialized, but it is always assigned to
because the for-loop preceding the assignment runs from 1 to 3 and assigns to
tmp unconditionally on the first round. Work around the warning by assigning
tmp to zero unnecessarily.
* Change duk_bool_to to duk_small_uint_t from duk_small_int_t. This may
cause some sign warnings in calling code.
* Reject attempt to unpack an array-like value whose length is 2G or over;
previously was not checked explicitly, and the length was cast to duk_idx_t
with a sign change and the unpack would then later fail. Now it fails with
a clean RangeError.
* Add wrap check for Node.js Buffer.concat().
* API DUK_TYPE_xxx, DUK_TYPE_MASK_xxx, flag constants etc are now unsigned.
Both duk_hthread and duk_context typedefs resolve to struct duk_hthread
internally. In external API duk_context resolves to struct duk_hthread
which is intentionally left undefined as the struct itself is not
dereferenced. Change internal code to use duk_hthread exclusively which
removes unnecessary and awkward thr <-> ctx casts from internals.
The basic guidelines are:
* Public API uses duk_context in prototype declarations. The intent is to
hide the internal type, and there's already a wide dependency on the
type name.
* All internal code, both declarations and definitions, use duk_hthread
exclusively. This is done even for API functions, i.e. an API function
declared as "void duk_foo(duk_context *ctx);" is then defined as
"void duk_foo(duk_hthread *thr);".
* Strings with 0xFF byte prefix are considered special symbols: they have
typeof "symbol" but still mostly behave as strings (e.g. allow ToString)
so that existing code dealing with internal keys, especially inside
Duktape, can work with fewer changes.
* Strings with 0x80 byte prefix are global symbols, e.g. Symbol.for('foo')
creates the byte representatio: 0x80 "foo"
* Strings with 0x81 byte prefix are unique symbols; the 0x81 byte is followed
by the Symbol description, and an internal string component ensuring
uniqueness is separated by a 0xFF byte (which can never appear anywhere in
an extended UTF-8 string). The unique suffix is up to Duktape internals,
currently two 32-bit counters are used. For example:
0x81 "mySymbol" 0xFF "0-17".
* Well-known symbols use the 0x81 prefix but lack a unique suffix, so their
format is 0x81 <description> 0xFF.
* ES6 distinguishes between an undefined symbol description and an empty
string symbol description. This distinction is not currently visible via
Ecmascript bindings but may be visible in the future. Append an extra
0xFF to the unique suffix when the description is undefined, i.e.
0x81 0xFF <unique suffix> 0xFF.
These can be used whenever we're 100% certain that the value stack index
exists and the type matches expected type. When these are true, a
duk_hstring, duk_hbuffer, or duk_hobject pointer fetch can be inlined to
small code.
This avoids the need for a function call and up to two property lookups
for a `Math.pow()` invocation, as well as allowing expressions like
`2 ** 16` to be inlined at compile time.
Exponentiation uses the same internal handler as `Math.pow()`, per the
ES7 specification.
Usage:
x = base ** exp;
x **= 2;
* Use shared error macros and shared error handler to reduce the size of call
sites of common errors.
* Make zero argument DUK_ERROR() calls non-vararg calls to reduce call site
footprint. Non-vararg calls have smaller call sites and because there are
a lot of call sites, this turns out to be significant.
* Remove variadic macros from internal DUK_ERROR() macro set and add separate
macros for argument counts 0 to 4; this is more portable and requires less
conditional code, and works well when a non-vararg call is used for most
error call sites.
* Rework macro / function argument order for the error path, try to keep 'thr'
in the same argument slot to avoid unnecessary register moves.
* Pack linenumber and error code into a single 32-bit argument when possible,
removes one more constant load from the call site.
* Convert some internal errors to RangeErrors when the underlying cause is an
implementation limit (such as a compiler temp limit) rather than an actual
unexpected internal situation.
* Simplify and share a few error messages to reduce string count.
* Use a single step value encoding approach instead of a two-step one
* Rework slow path value encoding to rewind: by simply rewinding the
ufwriter the awkward two-step approach can be avoided.
* Also rework automatic "naked" key escape code to be a bit simpler.
* Fix automatic unboxing of strings, numbers, etc in JSON.stringify()
fast path; it needs to invoke .toString() / .valueOf() explicitly
if they have been replaced in the object itself or in the prototype
chain.
* Fix regexp maximum quantifier digit check off-by-one (rejected 9
digits)
* Allow two more cases of invalid quantifier to fall back to parsing
the left curly as a literal.
* Some ifdef/if defined, whitespace, etc trivia.
Cast through "void *" when casting from e.g. a duk_uint8_t pointer
to a pointer with more alignment requirements (such as a struct pointer).
This is probably the best fix for cases where one also needs to do
pointer arithmetic (for which a char pointer is preferable). In cases
where pointer arithmetic is not necessarily, the original pointer could
in some cases be changed from a char pointer to a void pointer.
Memory optimization work for very low memory devices (96 to 256kB system RAM).
Overall changes are:
- 16-bit fields for various internal structures to reduce their size
- Heap pointer compression to reduce pointer size to 16 bits
When DUK_OPT_LIGHTFUNC_BUILTINS and the new low memory options are enabled,
Duktape initial heap memory usage is about 23kB (compared to baseline of
about 45kB) on x86.
Unless low memory feature options are enabled, there should be no visible
changes to Duktape behavior.
More detailed changes:
- 16-bit changes for duk_heaphdr: pointer compression, refcount
- 16-bit changes for duk_hstring: hash, blen, and clen can all be 16 bits,
use 0xFFFF as string byte length limit (call sites ensure this limit is
never exceeded)
- 16-bit changes for duk_hbuffer, use 0xFFFF as buffer length limit
- 16-bit fields for hobject size (entry part, array part), drop hash part
since it's not usually needed for extremely low memory environments
- 16-bit changes for duk_hcompiledfunction
- Heap pointer packing for stringtable
- Heap pointer packing for 'strs' built-in strings list (saves around 600
to 700 bytes but may not be a good tradeoff because call site size will
increase)
Other changes:
- Heaphdr NULL init fix. The original macros were broken: the double/single
linked macro variants were the wrong way around. Now sets through macro
to work properly with compressed pointers.
- Rename duk_hbuffer CURR_DATA_PTR -> DATA_PTR to reduce macro length
(previous name was tediously long)
- Rename buffer "usable_size" to "alloc_size" throughout as they have been
the same for a while now (they used to differ when buffer had an extra NUL).
- Add memory optimization markers to Duktape.env (pointer compression and
individual 16-bit field options)
- Rename a few internal fields for clarity: duk_hobject 'p' to 'props',
heap->st to heap->strtable
- Add a safety check for buffer alloc size (should not be triggered but
prevents wrapping if call sites don't properly check for sizes)
- Other minor cleanups
Unused variables in some option combinations. Some of these were caused by
having -DDUK_OPT_DEBUG but no debug print options.
The GCC -O4 array subscript warning seems spurious and is probably caused by
GCC not being 100% sure 'count' is always valid. The call sites for
duk__advance_chars() have a variety of argument values, including those
computed using shifts etc. Add an explicit check for the valid range which
silences the GCC error. The check also protects against regressions that
would cause an invalid count to be used. With the check in place,
potentially memory unsafe behavior will now cause an internal error instead.
If the form DUK_ASSERT(x >= 0 && x < y) is used, gcc will cause a spurious
warning:
duk_api_stack.c:149:33: error: assuming signed overflow does not occur when assuming that (X - c) > X is always false [-Werror=strict-overflow]
duk_error.h:131:8: note: in definition of macro ‘DUK_ASSERT’
By breaking down the assert into two, i.e. DUK_ASSERT(x >= 0) and
DUK_ASSERT(x < y) the warning disappears.
Sami Vaarala
Michael Drake
Bruce Pascoe
Julien Hamaide