So it's possible to know when an external C function is being called at the
top-level, eg by JavaScript without any intermediate C->JS->C calls.
Signed-off-by: Damien George <damien@micropython.org>
Instead of raising KeyError. These semantics match JavaScript behaviour
and make it much more seamless to pass Python dicts through to JavaScript
as though they were JavaScript {} objects.
Signed-off-by: Damien George <damien@micropython.org>
This adds a new undefined singleton to Python, that corresponds directly to
JavaScript `undefined`. It's accessible via `js.undefined`.
Signed-off-by: Damien George <damien@micropython.org>
This reverts part of commit fa23e4b093, to
make it so that Python `None` converts to JavaScript `null` (and JavaScript
`null` already converts to Python `None`). That's consistent with how the
`json` module converts these values back and forth.
Signed-off-by: Damien George <damien@micropython.org>
And change Py None conversion so it converts to JS undefined.
The semantics for conversion of these objects are then:
- Python None -> JavaScript undefined
- JavaScript undefined -> Python None
- JavaScript null -> Python None
This follows Pyodide:
https://pyodide.org/en/stable/usage/type-conversions.html
Signed-off-by: Damien George <damien@micropython.org>
This commit defines a new `JsException` exception type which is used on the
Python side to wrap JavaScript errors. That's then used when a JavaScript
Promise is rejected, and the reason is then converted to a `JsException`
for the Python side to handle.
This new exception is exposed as `jsffi.JsException`.
Signed-off-by: Damien George <damien@micropython.org>
JavaScript semantics are such that the caller of an async function does not
need to await that function for it to run to completion. This commit makes
that behaviour also apply to top-level async Python code run via
`runPythonAsync()`.
Signed-off-by: Damien George <damien@micropython.org>
The `reason` in a rejected promise should be an instance of `Error`. That
leads to better error messages on the JavaScript side.
Signed-off-by: Damien George <damien@micropython.org>
This allows a simple way to run the existing asyncio tests under the
webassembly port, which doesn't support `asyncio.run()`.
Signed-off-by: Damien George <damien@micropython.org>
This commit adds a significant portion of the existing MicroPython asyncio
module to the webassembly port, using parts of the existing asyncio code
and some custom JavaScript parts.
The key difference to the standard asyncio is that this version uses the
JavaScript runtime to do the actual scheduling and waiting on events, eg
Promise fulfillment, timeouts, fetching URLs.
This implementation does not include asyncio.run(). Instead one just uses
asyncio.create_task(..) to start tasks and then returns to the JavaScript.
Then JavaScript will run the tasks.
The implementation here tries to reuse as much existing asyncio code as
possible, and gets all the semantics correct for things like cancellation
and asyncio.wait_for. An alternative approach would reimplement Task,
Event, etc using JavaScript Promise's. That approach is very difficult to
get right when trying to implement cancellation (because it's not possible
to cancel a JavaScript Promise).
Signed-off-by: Damien George <damien@micropython.org>
When a Promise is rejected on the JavaScript side, the reject reason should
be thrown into the encapsulating generator on the Python side.
Signed-off-by: Damien George <damien@micropython.org>
An exception on the Python side should be passed to the Promise reject
callback on the JavaScript side.
Signed-off-by: Damien George <damien@micropython.org>
Otherwise Emscripten allocates it on the Emscripten C stack, which will
overflow for large amounts of code.
Fixes issue #14307.
Signed-off-by: Damien George <damien@micropython.org>
In modularize mode, the `_createMicroPythonModule()` constructor must be
await'ed on, before `Module` is ready to use.
Signed-off-by: Damien George <damien@micropython.org>
This optimises the case where a Python function is, for example, stored to
a JavaScript attribute and then later retrieved from Python. The Python
function no longer needs to be a proxy with double proxying needed for the
call from Python -> JavaScript -> Python.
Signed-off-by: Damien George <damien@micropython.org>
This commit adds a pyscript variant for use in https://pyscript.net/.
The configuration is:
- No ASYNCIFY, in order to keep the WASM size down and have good
performance.
- MICROPY_CONFIG_ROM_LEVEL_FULL_FEATURES to enable most features.
- Custom manifest that includes many of the python-stdlib libraries.
- MICROPY_GC_SPLIT_HEAP_AUTO to increase GC heap size instead of doing a
collection when memory is exhausted. This is needed because ASYNCIFY is
disabled. Instead the GC collection is run at the top-level before
executing any Python code.
- No MICROPY_VARIANT_ENABLE_JS_HOOK because there is no asynchronous
keyboard input to interrupt a running script.
Signed-off-by: Damien George <damien@micropython.org>
With this commit, `interpreter.runPythonAsync(code)` can now be used to run
Python code that uses `await` at the top level. That will yield up to
JavaScript and produce a thenable, which the JavaScript runtime can then
resume. Also implemented is the ability for Python code to await on
JavaScript promises/thenables. For example, outer JavaScript code can
await on `runPythonAsync(code)` which then runs Python code that does
`await js.fetch(url)`. The entire chain of calls will be suspended until
the fetch completes.
Signed-off-by: Damien George <damien@micropython.org>
This commit improves the webassembly port by adding:
- Proxying of Python objects to JavaScript with a PyProxy type that lives
on the JavaScript side. PyProxy implements JavaScript Proxy traps such
as has, get, set and ownKeys, to make Python objects have functionality
on the JavaScript side.
- Proxying of JavaScript objects to Python with a JsProxy type that lives
on the Python side. JsProxy passes through calls, attributes,
subscription and iteration from Python to JavaScript.
- A top-level API on the JavaScript side to construct a MicroPython
interpreter instance via `loadMicroPython()`. That function returns an
object that can be used to execute Python code, access the Python globals
dict, access the Emscripten filesystem, and other things. This API is
based on the API provided by Pyodide (https://pyodide.org/). As part of
this, the top-level file is changed from `micropython.js` to
`micropython.mjs`.
- A Python `js` module which can be used to access all JavaScript-side
symbols, for example the DOM when run within a browser.
- A Python `jsffi` module with various helper functions like
`create_proxy()` and `to_js()`.
- A dedenting lexer which automatically dedents Python source code if every
non-empty line in that source starts with a common whitespace prefix.
This is very helpful when Python source code is indented within a string
within HTML or JavaScript for formatting reasons.
Signed-off-by: Damien George <damien@micropython.org>
This commit cleans up and generalises the Makefile, adds support for
variants (following the unix port) and adds the "standard" variant as the
default variant.
Signed-off-by: Damien George <damien@micropython.org>
When enabled the GC will not reclaim any memory on a call to
`gc_collect()`. Instead it will grow the heap.
Signed-off-by: Damien George <damien@micropython.org>
The STATIC macro was introduced a very long time ago in commit
d5df6cd44a. The original reason for this was
to have the option to define it to nothing so that all static functions
become global functions and therefore visible to certain debug tools, so
one could do function size comparison and other things.
This STATIC feature is rarely (if ever) used. And with the use of LTO and
heavy inline optimisation, analysing the size of individual functions when
they are not static is not a good representation of the size of code when
fully optimised.
So the macro does not have much use and it's simpler to just remove it.
Then you know exactly what it's doing. For example, newcomers don't have
to learn what the STATIC macro is and why it exists. Reading the code is
also less "loud" with a lowercase static.
One other minor point in favour of removing it, is that it stops bugs with
`STATIC inline`, which should always be `static inline`.
Methodology for this commit was:
1) git ls-files | egrep '\.[ch]$' | \
xargs sed -Ei "s/(^| )STATIC($| )/\1static\2/"
2) Do some manual cleanup in the diff by searching for the word STATIC in
comments and changing those back.
3) "git-grep STATIC docs/", manually fixed those cases.
4) "rg -t python STATIC", manually fixed codegen lines that used STATIC.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
It's no longer supported by Emscripten (at least at 3.1.55). And it's not
needed when the output is WASM, which it is by default.
Signed-off-by: Damien George <damien@micropython.org>
MicroPython code may rely on the return value of sys.stdout.buffer.write()
to reflect the number of bytes actually written. While in most scenarios a
write() operation is successful, there are cases where it fails, leading to
data loss. This problem arises because, currently, write() merely returns
the number of bytes it was supposed to write, without indication of
failure.
One scenario where write() might fail, is where USB is used and the
receiving end doesn't read quickly enough to empty the receive buffer. In
that case, write() on the MicroPython side can timeout, resulting in the
loss of data without any indication, a behavior observed notably in
communication between a Pi Pico as a client and a Linux host using the ACM
driver.
A complex issue arises with mp_hal_stdout_tx_strn() when it involves
multiple outputs, such as USB, dupterm and hardware UART. The challenge is
in handling cases where writing to one output is successful, but another
fails, either fully or partially. This patch implements the following
solution:
mp_hal_stdout_tx_strn() attempts to write len bytes to all of the possible
destinations for that data, and returns the minimum successful write
length.
The implementation of this is complicated by several factors:
- multiple outputs may be enabled or disabled at compiled time
- multiple outputs may be enabled or disabled at runtime
- mp_os_dupterm_tx_strn() is one such output, optionally containing
multiple additional outputs
- each of these outputs may or may not be able to report success
- each of these outputs may or may not be able to report partial writes
As a result, there's no single strategy that fits all ports, necessitating
unique logic for each instance of mp_hal_stdout_tx_strn().
Note that addressing sys.stdout.write() is more complex due to its data
modification process ("cooked" output), and it remains unchanged in this
patch. Developers who are concerned about accurate return values from
write operations should use sys.stdout.buffer.write().
This patch might disrupt some existing code, but it's also expected to
resolve issues, considering that the peculiar return value behavior of
sys.stdout.buffer.write() is not well-documented and likely not widely
known. Therefore, it's improbable that much existing code relies on the
previous behavior.
Signed-off-by: Maarten van der Schrieck <maarten@thingsconnected.nl>
If a non-string buffer was passed to execfile, then it would be passed
as a non-null-terminated char* to mp_lexer_new_from_file.
This changes mp_lexer_new_from_file to take a qstr instead (as in almost
all cases a qstr will be created from this input anyway to set the
`__file__` attribute on the module).
This now makes execfile require a string (not generic buffer) argument,
which is probably a good fix to make anyway.
Fixes issue #12522.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
When the "typeof window" check is run within a web worker the window is
undefined, causing an error because "require" is only defined in a Node
environment. Change the logic to reflect the true intentions of when this
code should run, ie in Node only.
Signed-off-by: Damien George <damien@micropython.org>
This fixes a bug where `gc.collect()` would crash due to
emscripten_scan_stack being called synchronously within mp_js_do_str. The
fix is to make mp_js_do_str asynchronous.
Fixes#10692.
Signed-off-by: Eli Bierman <eli@elib.dev>
Damien George
Angus Gratton
Maarten van der Schrieck
Jim Mussared
Nicholas H.Tollervey
elibdev