This function was implemented as an experiment, and was enabled only in
unix port. To remind, it allows to access arbitrary files frozen as
source modules (vs bytecode).
However, further experimentation showed that the same functionality can
be implemented with frozen bytecode. The process requires more steps, but
with suitable toolset it doesn't matter patch. This process is:
1. Convert binary files into "Python resource module" with
tools/mpy_bin2res.py.
2. Freeze as the bytecode.
3. Use micropython-lib's pkg_resources.resource_stream() to access it.
In other words, the extra step is using tools/mpy_bin2res.py (because
there would be wrapper for uio.resource_stream() anyway).
Going frozen bytecode route allows more flexibility, and same/additional
efficiency:
1. Frozen source support can be disabled altogether for additional code
savings.
2. Resources could be also accessed as a buffer, not just as a stream.
There're few caveats too:
1. It wasn't actually profiled the overhead of storing a resource in
"Python resource module" vs storing it directly, but it's assumed that
overhead is small.
2. The "efficiency" claim above applies to the case when resource
file is frozen as the bytecode. If it's not, it actually will take a
lot of RAM on loading. But in this case, the resource file should not
be used (i.e. generated) in the first place, and micropython-lib's
pkg_resources.resource_stream() implementation has the appropriate
fallback to read the raw files instead. This still poses some distribution
issues, e.g. to deployable to baremetal ports (which almost certainly
would require freezeing as the bytecode), a distribution package should
include the resource module. But for non-freezing deployment, presense
of resource module will lead to memory inefficiency.
All the discussion above reminds why uio.resource_stream() was implemented
in the first place - to address some of the issues above. However, since
then, frozen bytecode approach seems to prevail, so, while there're still
some issues to address with it, this change is being made.
This change saves 488 bytes for the unix x86_64 port.
The SHA1 hashing functionality is provided via the "axtls" library's
implementation, and hence is unavailable when the "axtls" library is not being
used. This change provides the same SHA1 hashing functionality when using the
"mbedtls" library by using its implementation instead.
With inplace methods now disabled by default, it makes sense to enable
reverse methods, as they allow for more useful features, e.g. allow
for datetime module to implement both 2 * HOUR and HOUR * 2 (where
HOUR is e.g. timedelta object).
This adds a new configuration option to print runtime warnings and errors to
stderr. On Unix, CPython prints warnings and unhandled exceptions to stderr,
so the unix port here is configured to use this option.
The unix port already printed unhandled exceptions on the main thread to
stderr. This patch fixes unhandled exceptions on other threads and warnings
(issue #2838) not printing on stderr.
Additionally, a couple tests needed to be fixed to handle this new behavior.
This is done by also capturing stderr when running tests.
This is to keep the top-level directory clean, to make it clear what is
core and what is a port, and to allow the repository to grow with new ports
in a sustainable way.
The with semantics of this function is close to
pkg_resources.resource_stream() function from setuptools, which
is the canonical way to access non-source files belonging to a package
(resources), regardless of what medium the package uses (e.g. individual
source files vs zip archive). In the case of MicroPython, this function
allows to access resources which are frozen into the executable, besides
accessing resources in the file system.
This is initial stage of the implementation, which actually doesn't
implement "package" part of the semantics, just accesses frozen resources
from "root", or filesystem resource - from current dir.
Prior to making this a config option it was previously available on these
(and all other) ports, and it makes sense to keep it enabled for mpy-cross
as well as ports that have a decent amount of space for the code.
Updated modbuiltin.c to add conditional support for 3-arg calls to
pow() using MICROPY_PY_BUILTINS_POW3 config parameter. Added support in
objint_mpz.c for for optimised implementation.
Implementations of persistent-code reader are provided for POSIX systems
and systems using FatFS. Macros to use these are MICROPY_READER_POSIX and
MICROPY_READER_FATFS respectively. If an alternative implementation is
needed then a port can define the function mp_reader_new_file.
This type was used only for the typedef of mp_obj_t, which is now defined
by the object representation. So we can now remove this unused typedef,
to simplify the mpconfigport.h file.
To filter out even prototypes of mp_stream_posix_*() functions, which
require POSIX types like ssize_t & off_t, which may be not available in
some ports.
When built for Linux, libffi includes very bloated and workaround exec-alloc
implementation required to work around SELinux and other "sekuritee" features
which real people don't use. MicroPython has own alloc-exec implementation,
used to alloc memory for @micropython.native code. With this option enabled,
uPy's implementation will override libffi's. This saves 11K on x86_64 (and
that accounts for more than half of the libffi code size).
TODO: Possibly, we want to refactor this option to allow either use uPy's
implementation even for libffi, or allow to use libffi's implementation even
for uPy.
Disabled by default, enabled in unix port. Need for this method easily
pops up when working with text UI/reporting, and coding workalike
manually again and again counter-productive.
To use frozen bytecode make a subdirectory under the unix/ directory
(eg frozen/), put .py files there, then run:
make FROZEN_MPY_DIR=frozen
Be sure to build from scratch. The .py files will then be available for
importing.
The config variable MICROPY_MODULE_FROZEN is now made of two separate
parts: MICROPY_MODULE_FROZEN_STR and MICROPY_MODULE_FROZEN_MPY. This
allows to have none, either or both of frozen strings and frozen mpy
files (aka frozen bytecode).
Calling it from mp_init() is too late for some ports (like Unix), and leads
to incomplete stack frame being captured, with following GC issues. So, now
each port should call mp_stack_ctrl_init() on its own, ASAP after startup,
and taking special precautions so it really was called before stack variables
get allocated (because if such variable with a pointer is missed, it may lead
to over-collecting (typical symptom is segfaulting)).
Seedable and reproducible pseudo-random number generator. Implemented
functions are getrandbits(n) (n <= 32) and seed().
The algorithm used is Yasmarang by Ilya Levin:
http://www.literatecode.com/yasmarang
Paul Sokolovsky
Christopher Cooper
David Lechner
Damien George
Alexander Steffen
Pavol Rusnak
Nicko van Someren
ilovezfs