duktape/doc/modules.txt

=======
Modules
=======

Introduction
============

This document discusses the barebone module framework built into Duktape:

* There is a module search function which must be provided by user code.
  The search function maps a module ID into a source code string or throws
  an error.  To remain portable and unaware of file systems and such,
  Duktape does not provide a default module search function.

* Ecmascript modules are defined using the CommonJS module format:

  - http://wiki.commonjs.org/wiki/Modules/1.1.1

* C modules loaded from DLL files have a custom convention to allow
  module initialization code to run when a C module is loaded.

Using modules from Ecmascript code (require)
============================================

Duktape provides a global ``require()`` function which allows a module to be
loaded based on a module identifier::

  var lib = require('package/lib')

The module identifier may contain forward slashes to provide some hierarchy.
There are both absolute and relative identifiers::

  // absolute identifier, resolves to 'a/b' everywhere
  var mod = require('a/b');

  // relative identifier, resolves to 'package/c' inside package/lib
  var mod = require('./c');

  // relative identifier, resolves to 'foo/xyz' inside foo/bar/quux
  var mod = require('../xyz');

Relative paths are resolved starting from the absolute root of the current
module, i.e. the module calling ``require()``, see separate section on the
resolution algorithm.

A module with a certain resolved identifier is loaded only once per a global
environment: ``Duktape.loaded`` keeps track of modules which are fully or
partially loaded, mapping a resolved identifier to the module's ``exports``
table.  If a required module has already been (fully or partially) loaded,
the same ``exports`` value is returned without further action.

If a module has not yet been loaded Duktape calls ``Duktape.find()``, a module
search function which must be provided by the user (there is no default).
The search function is given the resolved identifier of the module, and is
expected to return the module source code as a string or throw an error.
If the module is found, a fresh ``exports`` object is created and registered
to ``Duktape.loaded``.  This registration is done before evaluating the module
source code, so that circular dependencies work as required.

The user ``Duktape.find()`` function encapsulates functionality such as
module search paths and file I/O.  This is a nice division of labor as it
encapsulates practically all of the platform dependent parts of module
loading, while keeping the user callback unaware of almost every other
aspect of module loading.  Using stub module search functions is also easy,
which is good for testing.

Ecmascript modules follow the CommonJS format, e.g. ``package/lib`` could
be represented by the source file::

  exports.add = function (a, b) {
    return a + b;
  };

CommonJS instructs that modules should be evaluated with certain bindings
in force.  Duktape currently implements the CommonJS requirements by simply
wrapping the module code inside some footer/header code::

  (function (require, exports, module) {
    /* module code here */
  })

So the example module would become::

  (function (require, exports, module) {
    exports.add = function (a, b) {
      return a + b;
    };
    // return value is ignored
  })

When evaluated, the expression results in a function object (denoted ``F``)
which is then called (more or less) like::

  var exports = {};
  F.call(exports,                 /* exports also used as 'this' binding */
         require,                 /* require method */
         exports,                 /* exports */
         { id: 'package/lib' });  /* module */

The return value of this call is ignored.

The first argument is a new function object whose underlying native function
is the same as the global ``require()`` function.  This fresh function is
needed to facilitate resolution of relative module identifiers: relative
identifers are resolved relative to the current module.  This is easiest
to manage by providing a new function and storing the resolution path in a
property of that function.  Native code can then pick up the resolution path
from the function object.

The second argument is an initially empty object.  The module code adds
all exported symbols into the object which is then registered into
``Duktape.mods[id]`` and returned from ``require()``.

The third argument provides the module with its own, resolved identifier.
The value in ``module.id`` is guaranteed to be in absolute form, and resolve
to the module itself if required from any other module.  Duktape doesn't
currently support ``module.exports`` like NodeJS, as it is not required by
CommonJS.

CommonJS module identifier resolution
=====================================

CommonJS specifies that identifier terms must be "camelCase":

* http://wiki.commonjs.org/wiki/Modules/1.1#Module_Identifiers

Some interpret this to mean that e.g. a dash character is not allowed.
Such an interpretation seems counterproductive because e.g. filenames
often contain dashes, underscores, etc.  Duktape allows terms to contain
any characters (including non-ASCII and white space) except that:

* A term must not begin with a period (``.``) to simplify resolution.
  Such terms are rejected.

* A term cannot contain a forward slash, which (of course) gets
  interpreted as a separator.

* A term cannot contain a U+0000 character.  Such terms are currently
  not rejected.  Instead, they terminate the resolution as if the
  requested identifier had ended.

If user code wishes to impose further limits, the module search function
can check a resolved identifier and throw an error if it is not of a
desirable form.

module.exports
==============

NodeJS allows the default ``exports`` value to be changed by the module being
loaded; it can even be replaced e.g. by a function (it's normally an object
value).  To change the value, the module must assign to ``module.exports``
which initially has the same value as ``exports``:

* http://timnew.github.io/blog/2012/04/20/exports_vs_module_exports_in_node_js/

Duktape doesn't currently support assignment to ``module.exports``.

Using modules from C (duk_require_module)
=========================================

**FIXME**

C modules (DLLs)
================

**FIXME**

Background
==========

Module frameworks
-----------------

Ecmascript has not traditionally had a module mechanism.  In browser
environments a web page can load multiple script files in a specific
order, each of them introducing more global symbols.  This is not very
elegant because the order of loading must be correct in case any code
runs during loading.  Several module mechanisms have since been created
for the browser environment to make writing modular Ecmascript easier.
Similar needs also exist in non-browser environments and several mechanisms
have been defined.

References summarizing several module frameworks:

* http://addyosmani.com/writing-modular-js/
* http://wiki.commonjs.org/wiki/Modules

Module loading APIs or "formats":

* Asynchronous Module Definition (AMD) API:

  - https://github.com/amdjs/amdjs-api/wiki/AMD

* CommonJS:

  - http://wiki.commonjs.org/wiki/Modules/1.1.1
  - https://github.com/joyent/node/blob/master/lib/module.js
  - https://github.com/commonjs/commonjs/tree/master/tests/modules
  - http://requirejs.org/docs/commonjs.html
  - http://dailyjs.com/2010/10/18/modules/

* NodeJS, more or less CommonJS:

  - http://nodejs.org/docs/v0.11.13/api/modules.html

* ES6:

  - https://people.mozilla.org/~jorendorff/es6-draft.html#sec-modules

AMD is optimized for the web client side, and requires callback based
asynchronous module loading.  This model is not very convenient for
server side programming, or fully fledged application programming which
is more natural with Duktape.

CommonJS module format is a server side module mechanism which seems most
appropriate to be the default Duktape mechanism.

Some NodeJS tests
=================

This section illustrates some NodeJS module loader features, as it's nice
to align with NodeJS behavior when possible.

Assignments
-----------

Test module::

  // test.js
  var foo = 123;     // not visible outside
  bar = 234;         // assigned to global object
  this.quux = 345;   // exported from module
  exports.baz = 456; // exported from module

Test code::

  > var t = require('./test');
  undefined
  > console.log(JSON.stringify(t));
  {"quux":345,"baz":456}
  undefined
  > console.log(bar);
  234

Future work
===========

module.exports
--------------

Could add support to ``module.exports``.

Lua-like module loader
----------------------

The lowest level module mechanism could also be similar to what Lua does.
A module would be cached as in CommonJS so that it would only be loaded
once per global context.  Modules could be loaded with a user callback
which takes a module ID and returns the loaded module object (same as the
``exports``) value to be registered into the module cache.

The upside of this approach is flexibility: most of the CommonJS module
mechanism can be implemented on top of this.

One downside is that the module loading mechanism would not be a common one
and most users would need to implement or borrow a standard module loader.
Another downside is that a Lua-like mechanism doesn't deal with circular
module loading while the CommonJS one does (to some extent).