From 00a8627e15fcbb01912f111a9df91476ac545ce6 Mon Sep 17 00:00:00 2001 From: Sami Vaarala Date: Fri, 26 Aug 2016 00:29:34 +0300 Subject: [PATCH] Documentation changes for tools reorg --- config/README.rst | 8 +++--- dist-files/README.rst | 8 +++++- doc/duk-config.rst | 26 +++++++++--------- doc/low-memory.rst | 16 +++++------ tools/README.rst | 63 +++++++++++++++++++++++++++++++++++++++++++ util/README.rst | 6 +++++ 6 files changed, 101 insertions(+), 26 deletions(-) create mode 100644 tools/README.rst create mode 100644 util/README.rst diff --git a/config/README.rst b/config/README.rst index 1d172261..58dae2ed 100644 --- a/config/README.rst +++ b/config/README.rst @@ -25,15 +25,15 @@ Usage To create an autodetect duk_config.h header (compatible with Duktape 1.2.x):: - $ python config/genconfig.py --metadata config --output /tmp/duk_config.h \ - autodetect-header + $ python tools/genconfig.py --metadata config --output /tmp/duk_config.h \ + duk-config-header To create a barebones duk_config.h header for a specific platform (easier to edit manually):: - $ python config/genconfig.py --metadata config --output /tmp/duk_config.h \ + $ python tools/genconfig.py --metadata config --output /tmp/duk_config.h \ --platform linux --compiler gcc --architecture x64 \ - barebones-header + duk-config-header There are further commands to e.g. autogenerate config option documentation; see ``genconfig.py`` for details. diff --git a/dist-files/README.rst b/dist-files/README.rst index b741f643..6521e146 100644 --- a/dist-files/README.rst +++ b/dist-files/README.rst @@ -67,9 +67,15 @@ This distributable contains: * ``src-separate/``: main Duktape library in multiple files format. -* ``config/``: genconfig utility for creating duk_config.h configuration +* ``src-input/``: raw input source files used for a config-and-prepare which + recreates the combined/separate prepared sources with specific options. + +* ``tools/``: various Python tools, such as prepare_sources.py for doing a + config-and-prepare and genconfig.py for creating duk_config.h configuration files, see: http://wiki.duktape.org/Configuring.html. +* ``config/``: configuration metadata for genconfig.py. + * ``examples/``: further examples for using Duktape. Although Duktape itself is widely portable, some of the examples are Linux only. For instance the ``eventloop`` example illustrates how ``setTimeout()`` diff --git a/doc/duk-config.rst b/doc/duk-config.rst index 8a2029f1..03398cfa 100644 --- a/doc/duk-config.rst +++ b/doc/duk-config.rst @@ -94,7 +94,7 @@ Using genconfig Overview of genconfig --------------------- -Genconfig (``config/genconfig.py``) is a helper script which provides +Genconfig (``tools/genconfig.py``) is a helper script which provides several commands related to config handling: * Generate a ``duk_config.h`` for a specified platform, compiler, and @@ -121,16 +121,16 @@ To generate an autodetect header suitable for directly supported platforms # The --metadata option can point to a metadata directory or a tar.gz # file with packed metadata (included in end user distributable). - $ cd duktape-1.4.0 - $ python config/genconfig.py \ + $ cd duktape-2.0.0 + $ python tools/genconfig.py \ --metadata config/genconfig_metadata.tar.gz \ --output /tmp/duk_config.h \ duk-config-header # The same command using unpacked metadata present in Duktape source repo. - $ cd duktape - $ python config/genconfig.py \ + $ cd duktape-2.0.0 + $ python tools/genconfig.py \ --metadata config/ \ --output /tmp/duk_config.h \ duk-config-header @@ -145,7 +145,7 @@ through a file or inline. If you're building Duktape as a DLL, you should use the ``--dll`` option:: - $ python config/genconfig.py \ + $ python tools/genconfig.py \ --metadata config/ \ --dll \ --output /tmp/duk_config.h \ @@ -167,7 +167,7 @@ Generating a barebones duk_config.h To generate a barebones header you need to specify a platform, compiler, and architecture for genconfig:: - $ python config/genconfig.py \ + $ python tools/genconfig.py \ --metadata config/ \ --platform linux \ --compiler gcc \ @@ -229,7 +229,7 @@ file, and a few options are then tweaked using the C compiler format. An autodetect header is then generated:: $ cd duktape - $ python config/genconfig.py \ + $ python tools/genconfig.py \ --metadata config/ \ --option-file low_memory.yaml \ -DDUK_USE_TRACEBACK_DEPTH=100 \ @@ -254,7 +254,7 @@ This file, another override file, and a few inline YAML forced options could be used as follows to generate a barebones header:: $ cd duktape - $ python config/genconfig.py \ + $ python tools/genconfig.py \ --metadata config/ \ --platform linux \ --compiler gcc \ @@ -264,7 +264,7 @@ could be used as follows to generate a barebones header:: --option-yaml 'DUK_USE_JX: false' \ --option-yaml 'DUK_USE_JC: false' \ --output /tmp/duk_config.h \ - barebones-header + duk-config-header For inline YAML, multiple forced options can be given either by using a YAML value with multiple keys, or by using multiple options:: @@ -335,7 +335,7 @@ sanity checks (if enabled). For example, to generate a barebones header with two fixup headers:: - $ python config/genconfig.py \ + $ python tools/genconfig.py \ --metadata config/ \ --platform linux \ --compiler gcc \ @@ -343,7 +343,7 @@ For example, to generate a barebones header with two fixup headers:: --fixup-file my_env_strings.h \ --fixup-file my_no_json_fastpath.h \ --output /tmp/duk_config.h \ - barebones-header + duk-config-header The ``my_env_strings.h`` fixup header could be:: @@ -476,7 +476,7 @@ header:: # my_custom.h is applied after generated header; functionally similar # to Duktape 1.2.x duk_custom.h - $ python config/ genconfig.py [...] --fixup-file my_custom.h [...] + $ python tools/genconfig.py [...] --fixup-file my_custom.h [...] A genconfig-generated barebones header also has the following line near the end for detecting where to add override defines; this is easy to detect reliably:: diff --git a/doc/low-memory.rst b/doc/low-memory.rst index 573e674b..0c66fd65 100644 --- a/doc/low-memory.rst +++ b/doc/low-memory.rst @@ -344,12 +344,12 @@ The following may be appropriate when even less memory is available is more memory efficient: it creates a writable (empty) global object which inherits from the ROM global object. - - Rerun ``make_dist.py`` with ``--rom-support`` to create a distributable - with support for ROM builtins. ROM builtin support is not enabled by - default because it increases the size of ``duktape.c`` considerably. - Add the option ``--rom-auto-lightfunc`` to convert built-in function - properties into lightfuncs to reduce ROM footprint. (See - ``util/example_rombuild.sh`` for some very simple examples.) + - Rerun ``prepare_sources.py`` with ``--rom-support`` to create prepared + sources with support for ROM builtins. ROM builtin support is not + enabled by default because it increases the size of ``duktape.c`` + considerably. Add the option ``--rom-auto-lightfunc`` to convert + built-in function properties into lightfuncs to reduce ROM footprint. + (See ``util/example_rombuild.sh`` for some very simple examples.) - Moving built-ins into ROM makes them read-only which has some side effects. Some side effects are technical compliance issues while @@ -377,8 +377,8 @@ The following may be appropriate when even less memory is available + ``src/builtins.yaml``: documents some more format details - + ``util/example_rombuild.sh``: illustrates how to run ``make_dist.py`` - with user builtins + + ``util/example_rombuild.sh``: illustrates how to run + ``prepare_sources.py`` with user builtins * Consider using lightfuncs for representing function properties of ROM built-ins. diff --git a/tools/README.rst b/tools/README.rst new file mode 100644 index 00000000..3fb8c40c --- /dev/null +++ b/tools/README.rst @@ -0,0 +1,63 @@ +============= +Duktape tools +============= + +This directory contains various Duktape Python tools which are included in +the end user distributable. + +The main use case is config-and-prepare which simultaneously creates +a ``duk_config.h`` configuration file and prepares source files for +compilation. These two operations are combined because: + +1. Configuration options may affect source file preparation. + +2. Some features involve metaprogramming. For example, when using ROM + built-ins YAML metadata is used for automatic code generation for the + ROM object initializers. + +There are also other miscellaneous tools, e.g. utilities to scan potential +fixed strings from source files, dump bytecode, and resolving file/line +from combined source. + +Because this tooling is part of the end user distributable, the tooling must +be as portable as possible. For example: + +* Avoid Unix path separators. + +* Avoid depending on Python executable name, use ``sys.executable`` instead + to launch Python commands. + +The tooling has been written for Python 2.x and there's no support for +Python 3.x at present. + +TODO +==== + +- Go through all .py files once more and figure out what to move into tools + +- Go through all moves, and git grep for "call sites" + +- Add dist/config/genconfig.py copy for convenience? + +STEPS +===== + +- Move all scripts that should be in the end user distributable into 'tools/'. + +- Change all references for the scripts for their new location so that existing + dist and build works. No other changes at this point. + +- Split make_dist.py into two parts: + + 1. Revised dist part which first copies files into the end user distributable + and then runs the prepare step *in the distributable* for the default + configuration. + + 2. Separate config-and-prepare utility which does most of the current dist + processing. This will take the --rom-auto-lightfunc and other stuff now. + It also interfaces with genconfig. + +- Update documentation: new locations (e.g. in genconfig docs, READMEs etc), + new config process, ROM built-in stuff, etc. + +- At this point the initial rework is complete. diff --git a/util/README.rst b/util/README.rst new file mode 100644 index 00000000..b5275738 --- /dev/null +++ b/util/README.rst @@ -0,0 +1,6 @@ +======================= +Miscellaneous utilities +======================= + +This directory contains miscellaneous Python, Shell, and other utilities +which are not part of the end user distributable.