Merge pull request #719 from svaarala/v1.5.0-release-prep

Release preparations for 1.5.0 release

RELEASES.rst

@@ -1416,10 +1416,7 @@ Released
 * Fix memory unsafe behavior when an external plain buffer was allocated
   with heap pointer compression enabled (DUK_USE_HEAPPTR16) (GH-618)
 
-Planned
-=======
-
-1.5.0 (XXXX-XX-XX)
+1.5.0 (2016-05-03)
 ------------------
 
 * Add support for placing Duktape and user strings and objects into ROM
@@ -1590,6 +1587,9 @@ Planned
 
 * Internal footprint improvement: reduce error call site size (GH-661)
 
+Planned
+=======
+
 2.0.0 (XXXX-XX-XX)
 ------------------
 

doc/release-checklist.rst

@@ -64,6 +64,9 @@ Checklist for ordinary releases
   - Check that ``duk_tval`` is packed by default on x86 and unpacked on
     x64
 
+  - util/checklist_compile_test.sh: linux compiler/arch combinations,
+    run in dist, check output manually
+
   - Platform / compiler combinations (incomplete, should be automated):
 
     + Linux x86-64 gcc

doc/release-notes-v1-5.rst

@@ -5,13 +5,34 @@ Duktape 1.5 release notes
 Release overview
 ================
 
-FIXME.
+Main changes in this release (see RELEASES.rst for full details):
+
+* Low memory improvements: support for ROM strings and objects, reduce string
+  memory footprint by 4 bytes.
+
+* Debugger improvements: heap object inspection, application specific
+  commands and notifications, duk_debugger_pause() API, DukLuv-based
+  debug proxy for easier packaging of a proxy into an application.
+
+* Improve error instance .stack format, minor changes to error message
+  strings.
+
+* CommonJS module improvements: module.filename and module.name support,
+  improve stack trace formatting of require() calls and module wrapper
+  functions.
+
+* Emscripten compatibility improvements (Emscripten code can now be executed
+  without fixups): RegExp parser accepts unescaped curly braces, Function
+  ``.toString()`` output format has been modified slightly to match Emscripten
+  expectations.
+
+* Minor fixes, performance, and portability improvements.
 
 Upgrading from Duktape 1.4.x
 ============================
 
 No action (other than recompiling) should be needed for most users to upgrade
-from Duktape v1.3.x.  Note the following:
+from Duktape v1.4.x.  Note the following:
 
 * When a debugger is attached and Duktape is in a paused state garbage
   collection is now disabled by default.  As a result, garbage created during
@@ -29,23 +50,35 @@ from Duktape v1.3.x.  Note the following:
   a debug client which assumes, for instance, that Eval result numbers are
   always in IEEE double format.
 
+* Because the format of error ``.stack`` property has been changed in this
+  release, any code parsing the stack trace format may need changes.
+
+* Because the Function ``.toString()`` output format has been changed in this
+  release (to be more Emscripten compatible), any code expecting a specific
+  ``.toString()`` output format may need changes.
+
 There are bug fixes and other minor behavioral changes which may affect some
 applications, see ``RELEASES.rst`` for details.
 
 Known issues
 ============
 
-This release has the following known issues worth noting.
+This release has the following known issues worth noting:
 
-Ecmascript features
--------------------
+* Non-compliant behavior for array indices near 2G or 4G elements.
 
-* FIXME
+* RegExp parser is strict and won't accept some real world RegExps which
+  are technically not compliant with Ecmascript E5/E5.1 specification.
 
-Portability and platforms
--------------------------
+* Final mantissa bit rounding issues in the internal number-to-string
+  conversion.
 
-* FIXME
+* On FreeBSD 10.x (at least 10.1 and 10.2): Clang with ``-m32`` generates
+  incorrect code for union assignments needed by Duktape's 8-byte packed
+  value encoding (see
+  https://github.com/svaarala/duktape/blob/master/misc/clang_aliasing.c).
+  The issue can be detected by defining ``DUK_OPT_SELF_TESTS``.  A workaround
+  is to avoid packed types in this case by defining ``DUK_OPT_NO_PACKED_TVAL``.
 
 Raw issues from test runs
 =========================
@@ -53,15 +86,61 @@ Raw issues from test runs
 API tests
 ---------
 
-* FIXME
+::
 
+    test-to-number.c: fail; 15 diff lines; known issue: number parsing bug for strings containing NUL characters (e.g. '\u0000')
 
 Ecmascript tests
 ----------------
 
-* FIXME
+::
+
+    test-bi-array-proto-push: fail; 30 diff lines; known issue: array length above 2^32-1 not supported
+    test-bi-array-push-maxlen: fail; 17 diff lines; known issue: array length above 2^32-1 not supported
+    test-bi-date-tzoffset-brute-fi: fail; 12 diff lines; known issue: year 1970 deviates from expected, Duktape uses equiv. year for 1970 on purpose at the moment; requires special feature options: test case has been written for Finnish locale
+    test-bi-function-nonstd-caller-prop: fail; 178 diff lines; requires special feature options: DUK_OPT_NONSTD_FUNC_CALLER_PROPERTY
+    test-bi-global-parseint: fail; 108 diff lines; known issue: rounding differences for parsing integers larger than 2^53
+    test-bi-json-dec-types: fail; 21 diff lines; known issue: '\x' should be allowed by eval() but not by JSON.parse(), Duktape rejects '\x' in both
+    test-bi-json-enc-proplist-dups: fail; 8 diff lines; known issue: JSON.stringify() can be given a property list to serialize; duplicates should be eliminated but Duktape (and other engines) will happily serialize a property multiple times
+    test-bi-json-enc-proxy: fail; 18 diff lines; known issue: JSON enumeration behavior for Proxy targets is incomplete and uses 'enumerate' trap instead of 'ownKeys' trap
+    test-bi-number-proto-toexponential: fail; 75 diff lines; known issue: corner case rounding errors in toExponential()
+    test-bi-number-proto-tostring: fail; 46 diff lines; known issue: expect strings to be checked, but probably Duktape rounding issues
+    test-bi-proxy-object-tostring: fail; 6 diff lines; known issue: Object class handling for Proxy objects is incomplete
+    test-bi-regexp-gh39: fail; 5 diff lines; known issue: requires leniency for non-standard regexps
+    test-bug-dataview-buffer-prop: fail; 20 diff lines; known issue: DataView .buffer property misleading when DataView argument is not an ArrayBuffer (custom behavior)
+    test-bug-enum-shadow-nonenumerable: fail; 12 diff lines; known issue: corner case enumeration semantics, not sure what correct behavior is (test262 ch12/12.6/12.6.4/12.6.4-2)
+    test-bug-invalid-oct-as-dec: fail; 16 diff lines; known issue: V8/Rhino parse invalid octal constants as decimal values, Duktape doesn't at the moment
+    test-bug-json-parse-__proto__: fail; 18 diff lines; known issue: when ES6 __proto__ enabled, JSON.parse() parses '__proto__' property incorrectly when a specially crafted reviver is used
+    test-bug-numconv-1e23: fail; 10 diff lines; known issue: corner case in floating point parse rounding
+    test-bug-numconv-denorm-toprec: fail; 7 diff lines; known issue: in a denormal corner case toPrecision() can output a zero leading digit
+    test-bug-tonumber-u0000: fail; 7 diff lines; known issue: '\u0000' should ToNumber() coerce to NaN, but now coerces to zero like an empty string
+    test-dev-bound-thread-start-func: fail; 13 diff lines; known issue: initial function of a new coroutine cannot be bound
+    test-dev-func-cons-args: fail; 18 diff lines; known issue: corner cases for 'new Function()' when arguments and code are given as strings
+    test-dev-lightfunc-accessor: fail; 50 diff lines; requires special feature options: DUK_OPT_LIGHTFUNC_BUILTINS
+    test-dev-lightfunc-finalizer: fail; 8 diff lines; requires special feature options: DUK_OPT_LIGHTFUNC_BUILTINS
+    test-dev-lightfunc: fail; 459 diff lines; requires special feature options: DUK_OPT_LIGHTFUNC_BUILTINS
+    test-dev-yield-after-callapply: fail; 8 diff lines; known issue: yield() not allowed when function called via Function.prototype.(call|apply)()
+    test-lex-unterminated-hex-uni-escape: fail; 29 diff lines; known issue: unterminated hex escapes should be parsed leniently, e.g. '\uX' -> 'uX' but Duktape now refuses to parse them
+    test-numconv-parse-misc: fail; 12 diff lines; known issue: rounding corner case for 1e+23 (parses/prints as 1.0000000000000001e+23)
+    test-numconv-tostring-gen: fail; 257 diff lines; known issue: rounding corner cases in number-to-string coercion
+    test-numconv-tostring-misc: fail; 6 diff lines; known issue: rounding corner case, 1e+23 string coerces to 1.0000000000000001e+23
+    test-regexp-empty-quantified: fail; 15 diff lines; known issue: a suitable empty quantified (e.g. '(x*)*') causes regexp parsing to terminate due to step limit
+    test-regexp-invalid-charclass: fail; 7 diff lines; known issue: some invalid character classes are accepted (e.g. '[\d-z]' and '[z-x]')
+    test-stmt-for-in-lhs: fail; 29 diff lines; known issue: for-in allows some invalid left-hand-side expressions which cause a runtime ReferenceError instead of a compile-time SyntaxError (e.g. 'for (a+b in [0,1]) {...}')
 
 test262
 -------
 
-* FIXME
+::
+
+    ch12/12.6/12.6.4/12.6.4-2 in non-strict mode   // diagnosed: enumeration corner case issue, see test-bug-enum-shadow-nonenumerable.js
+    ch15/15.10/15.10.2/15.10.2.5/S15.10.2.5_A1_T5 in non-strict mode   // diagnosed: Duktape bug, matching /(a*)b\1+/ against 'baaaac' causes first capture to match the empty string; the '\1+' part will then use the '+' quantifier over the empty string.  As there is no handling to empty quantified now, Duktape bails out with a RangeError.
+    ch15/15.10/15.10.2/15.10.2.9/S15.10.2.9_A1_T5 in non-strict mode   // diagnosed: Duktape bug, matching /(a*)b\1+/ against 'baaac' causes first capture to be empty, the '\1+' part will then quantify over an empty string leading to Duktape RangeError (there is no proper handling for an empty quantified now)
+    ch15/15.4/15.4.4/15.4.4.10/S15.4.4.10_A3_T3 in non-strict mode   // diagnosed: probably Duktape bug related to long array corner cases or 'length' sign handling (C typing?)
+    ch15/15.4/15.4.4/15.4.4.12/S15.4.4.12_A3_T3 in non-strict mode   // diagnosed: probably Duktape bug related to long array corner cases or 'length' sign handling (C typing?)
+    ch15/15.4/15.4.4/15.4.4.14/15.4.4.14-5-12 in non-strict mode   // diagnosed: Array length over 2G, not supported right now
+    ch15/15.4/15.4.4/15.4.4.14/15.4.4.14-5-16 in non-strict mode   // diagnosed: Array length over 2G, not supported right now
+    ch15/15.4/15.4.4/15.4.4.14/15.4.4.14-9-9 in non-strict mode   // diagnosed: a.indexOf(<n>,4294967290) returns -1 for all indices n=2,3,4,5 but is supposed to return 4294967294 for n=2.  The cause is long array corner case handling, possibly signed length handling (C typing?)
+    ch15/15.4/15.4.4/15.4.4.15/15.4.4.15-5-12 in non-strict mode   // diagnosed: probably Duktape bug: long array corner cases (C typing?)
+    ch15/15.4/15.4.4/15.4.4.15/15.4.4.15-5-16 in non-strict mode   // diagnosed: probably Duktape bug: long array corner cases (C typing?)
+    ch15/15.4/15.4.4/15.4.4.15/15.4.4.15-8-9 in non-strict mode   // diagnosed: probably Duktape bug: long array corner cases (C typing?)

runtests/runtests.js

@@ -245,7 +245,8 @@ function executeTest(options, callback) {
         execopts = {
             maxBuffer: 128 * 1024 * 1024,
             timeout: timeout,
-            stdio: 'pipe'
+            stdio: 'pipe',
+            killSignal: 'SIGKILL'
         };
 
         //console.log(cmdline);
@@ -282,7 +283,8 @@ function executeTest(options, callback) {
         execopts = {
             maxBuffer: 128 * 1024 * 1024,
             timeout: timeout,
-            stdio: 'pipe'
+            stdio: 'pipe',
+            killSignal: 'SIGKILL'
         };
 
         console.log(options.testPath, cmdline);

src/duk_api_public.h.in

@@ -87,7 +87,7 @@ struct duk_number_list_entry {
  * have 99 for patch level (e.g. 0.10.99 would be a development version
  * after 0.10.0 but before the next official release).
  */
-#define DUK_VERSION                       10499L
+#define DUK_VERSION                       10500L
 
 /* Git commit, describe, and branch for Duktape build.  Useful for
  * non-official snapshot builds so that application code can easily log

tests/api/test-all-public-symbols.c

@@ -54,6 +54,7 @@ static duk_ret_t test_func(duk_context *ctx) {
 	(void) duk_debugger_cooperate(ctx);
 	(void) duk_debugger_detach(ctx);
 	(void) duk_debugger_notify(ctx, 0);
+	(void) duk_debugger_pause(ctx);
 	(void) duk_decode_string(ctx, 0, NULL, NULL);
 	(void) duk_def_prop(ctx, 0, 0);
 	(void) duk_del_prop_index(ctx, 0, 0);

tests/ecmascript/test-dev-refcount-leak-basic.js

@@ -9,7 +9,7 @@
  *      # Set LOOP_COUNT to 1 manually and re-run the test
  *      $ valgrind ./duk --no-heap-destroy test-dev-refcount-leak-basic.js
  *
- *  Inspect the final "still allocated" to ensure that in botj cases we arrive at
+ *  Inspect the final "still allocated" to ensure that in both cases we arrive at
  *  the same baseline allocation at the end.  There's a large loop count in this
  *  test to magnify any leaks.  Using LOOP_COUNT 0 is OK but there may be small
  *  differences due to the value stack and/or call stack having a different size
@@ -26,15 +26,21 @@
  *  should provide a reasonable regression harness.  The basic coverage here
  *  is based on:
  *
- *      $ cd src/; grep DECREF UPDREF *.c > /tmp/worklist
+ *      $ cd src/; grep DECREF *.c > /tmp/worklist
+ *      $ cd src/; grep UPDREF *.c >> /tmp/worklist
  *
  *  Run the test with and without fastint support because that affects the
  *  code paths executed.  It's best to test with mark-and-sweep disabled so
  *  that garbage with broken refcounts won't get collected.
  *
  *  NOTE! To avoid circular references, all functions are made cycle free by
- *  forcing .prototype to null.  If this is not done, and mark-and-sweep is
- *  disabled, all functions will be leaks.
+ *  forcing .prototype to null.  Named function expressions ('function foo() {}')
+ *  are also in a circular reference with the intermediate scope object (which
+ *  provides the name binding) because the function references the scope via
+ *  _LexEnv and the scope object references the function.  Avoid named function
+ *  expressions in the test for this reason.  If these are not done, and
+ *  mark-and-sweep is disabled, all functions will be "leaks" until the heap
+ *  is destroyed.
  *
  *  NOTE! Leaving function instances (even cycle free ones) in the function
  *  registers when the function exits leads to a circular reference: the
@@ -213,9 +219,8 @@ this.T.objectMiscTest = function objectMiscTest() {
 
     // duk_hobject_props.c: duk_hobject_delprop_raw() delete accessor
     x = {};
-    setter = function setter() {}; setter.prototype = null;
-    getter = function getter() {}; getter.prototype = null;
-
+    setter = function () {}; setter.prototype = null;
+    getter = function () {}; getter.prototype = null;
     Object.defineProperty(x, 'prop', {
         set: setter, get: getter, configurable: true
     });

util/checklist_compile_test.sh

@@ -0,0 +1,37 @@
+#!/bin/bash
+#
+#  Helper script for release checklist compile test.
+#
+
+set -x
+
+OPTS1="-DDUK_OPT_SELF_TESTS"
+OPTS2="-DDUK_OPT_SELF_TESTS -DDUK_OPT_ASSERTIONS -DDUK_OPT_DEBUG -DDUK_OPT_DPRINT"
+
+TESTCODE="for (var arr=[]; arr.length < 1e6; arr.push(1)); Duktape.compact(arr); var estimate = Duktape.info(arr)[4] / arr.length; print('duk_tval size approximation:', estimate);"
+
+for compiler in gcc clang; do
+	for archopt in -m64 -m32; do
+		echo "*** $compiler $archopt"
+
+		echo "- single source, normal options"
+		rm -f duk
+		$compiler -o duk $archopt -Os -pedantic -std=c99 -Wall -fstrict-aliasing -fomit-frame-pointer -I./src ${OPTS1} src/duktape.c examples/cmdline/duk_cmdline.c -lm
+		./duk -e 'print(Duktape.env)' -e "$TESTCODE" mandel.js
+
+		echo "- single source, debug options"
+		rm -f duk
+		$compiler -o duk $archopt -Os -pedantic -std=c99 -Wall -fstrict-aliasing -fomit-frame-pointer -I./src ${OPTS2} src/duktape.c examples/cmdline/duk_cmdline.c -lm
+		./duk -e 'print(Duktape.env)' -e "$TESTCODE" mandel.js
+
+		echo "- separate sources, normal options"
+		rm -f duk
+		$compiler -o duk $archopt -Os -pedantic -std=c99 -Wall -fstrict-aliasing -fomit-frame-pointer -I./src-separate $OPTS1 src-separate/*.c examples/cmdline/duk_cmdline.c -lm
+		./duk -e 'print(Duktape.env)' -e "$TESTCODE" mandel.js
+
+		echo "- separate sources, debug options"
+		rm -f duk
+		$compiler -o duk $archopt -Os -pedantic -std=c99 -Wall -fstrict-aliasing -fomit-frame-pointer -I./src-separate $OPTS2 src-separate/*.c examples/cmdline/duk_cmdline.c -lm
+		./duk -e 'print(Duktape.env)' -e "$TESTCODE" mandel.js
+	done
+done

website/guide/gettingstarted.html

@@ -40,7 +40,7 @@ by editing the Makefile:
 <p>You can now run Ecmascript code interactively:</p>
 <pre>
 $ ./duk
-((o) Duktape 1.4.0 (v1.4.0)
+((o) Duktape 1.5.0 (v1.5.0)
 duk&gt; print('Hello world!')
 Hello world!
 = undefined

website/guide/intro.html

@@ -10,9 +10,9 @@ Once you're familiar with the basics, there is a concise
 <a href="http://wiki.duktape.org">Duktape Wiki</a> provides more detailed
 examples and best practices.</p>
 
-<p>This document doesn't cover Duktape internals (see the
+<p>This document doesn't cover Duktape internals, see the
 <a href="https://github.com/svaarala/duktape/tree/master/doc">Duktape repo</a>
-if you wish to tinker with them).</p>
+if you wish to tinker with them.</p>
 
 <h2>What is Duktape?</h2>
 

website/index/index.html

@@ -37,7 +37,7 @@ Duktape API to call Ecmascript functions from C code and vice versa.</p>
 <h1>Main features</h1>
 <ul>
 <li>Embeddable, portable, compact:
-    can run on platforms with 256kB flash and 96kB system RAM</li>
+    can run on platforms with 256kB flash and 64kB system RAM</li>
 <li><a href="http://www.ecma-international.org/ecma-262/5.1/">Ecmascript E5/E5.1</a>
     compliant, some features borrowed from
     <a href="http://www.ecma-international.org/ecma-262/6.0/index.html">Ecmascript E6</a></li>
@@ -59,11 +59,12 @@ Duktape API to call Ecmascript functions from C code and vice versa.</p>
 <h1>Code and RAM footprint</h1>
 <table>
 <tr><td>&nbsp;</td><td>x86 default</td><td>x86 lowmem</td><td>x86 full lowmem</td></tr>
-<tr><td>Code</td><td>224kB</td><td>198kB</td><td>234kB</td></tr>  <!-- FIXME: to be updated -->
-<tr><td>Startup RAM</td><td>70kB</td><td>43kB</td><td>3kB</td></tr>
+<tr><td>Code</td><td>170kB</td><td>150kB</td><td>200kB</td></tr>
+<tr><td>Startup RAM</td><td>68kB</td><td>35kB</td><td>4kB</td></tr>
 </table>
 
-<p>Full <a href="https://github.com/svaarala/duktape/blob/master/doc/low-memory.rst">lowmem</a>
+<p>See <a href="https://github.com/svaarala/duktape/blob/master/doc/low-memory.rst#optimizing-code-footprint">GCC options</a>
+for minimizing code footprint.  Full <a href="https://github.com/svaarala/duktape/blob/master/doc/low-memory.rst">lowmem</a>
 uses "pointer compression" and ROM-based strings/objects.  ROM-based strings/objects can
 also be used without other low memory options.</p>