As a prerequisite to upgrading to Zephyr v2.7.0, upgrade CI to use
Zephyr docker image v0.21.0. In particular, this is needed to pick up a
newer CMake version because Zephyr v2.7.0 increased the minimum CMake
version required to 3.20.0.
Signed-off-by: Maureen Helm <maureen.helm@intel.com>
This allows the remote MicroPython instance to create and delete
directories from the mounted host filesystem in addition to the already
existing functionality of reading, creating, and modifying files.
Signed-off-by: Michael Bentley <mikebentley15@gmail.com>
This changes makemanifest.py & mpy-tool.py to merge string and mpy names
into the same list (now mp_frozen_names).
The various paths for loading a frozen module (mp_find_frozen_module) and
checking existence of a frozen module (mp_frozen_stat) use a common
function that searches this list.
In addition, the frozen lookup will now only take place if the path starts
with ".frozen", which needs to be added to sys.path.
This fixes issues #1804, #2322, #3509, #6419.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Takes the functionality from tools/make-frozen.py, adds support for
multiple frozen directories, and moves it to tools/makemanifest.py.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Any board with a board.json file will be built. ESP32-based boards will be
built using the IDF at $IDF_PATH_V42, all other MCU variants (S2, S3, C3)
will be built using the IDF at $IDF_PATH_V44.
Signed-off-by: Damien George <damien@micropython.org>
Any board with a board.json file will be built. Additional variants for
certain pyboards will also be built by the explicit build-stm32-extra.sh
script. Both .dfu and .hex files will be made available.
Also build boards in a sorted order, and don't stop building if a single
board fails.
Signed-off-by: Damien George <damien@micropython.org>
This is to make the builds for all nucleo/discovery boards uniform, so they
can be treated the same by the auto build scripts.
The CI script is updated to explicitly enable mboot and packing, to test
these features.
Signed-off-by: Damien George <damien@micropython.org>
There is no release of IDF v4.4 yet but master is now on v5.0-dev so a
specific commit must be chosen to stick to v4.4.
Signed-off-by: Damien George <damien@micropython.org>
IDF v4.4 does not have an official release so for now use the latest
master. Also remove building GENERIC with no options (all the other boards
are no-option builds), to keep CI time reasonable.
Signed-off-by: Damien George <damien@micropython.org>
This commit removes all parts of code associated with the existing
MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE optimisation option, including the
-mcache-lookup-bc option to mpy-cross.
This feature originally provided a significant performance boost for Unix,
but wasn't able to be enabled for MCU targets (due to frozen bytecode), and
added significant extra complexity to generating and distributing .mpy
files.
The equivalent performance gain is now provided by the combination of
MICROPY_OPT_LOAD_ATTR_FAST_PATH and MICROPY_OPT_MAP_LOOKUP_CACHE (which has
been enabled on the unix port in the previous commit).
It's hard to provide precise performance numbers, but tests have been run
on a wide variety of architectures (x86-64, ARM Cortex, Aarch64, RISC-V,
xtensa) and they all generally agree on the qualitative improvements seen
by the combination of MICROPY_OPT_LOAD_ATTR_FAST_PATH and
MICROPY_OPT_MAP_LOOKUP_CACHE.
For example, on a "quiet" Linux x64 environment (i3-5010U @ 2.10GHz) the
change from CACHE_MAP_LOOKUP_IN_BYTECODE, to LOAD_ATTR_FAST_PATH combined
with MAP_LOOKUP_CACHE is:
diff of scores (higher is better)
N=2000 M=2000 bccache -> attrmapcache diff diff% (error%)
bm_chaos.py 13742.56 -> 13905.67 : +163.11 = +1.187% (+/-3.75%)
bm_fannkuch.py 60.13 -> 61.34 : +1.21 = +2.012% (+/-2.11%)
bm_fft.py 113083.20 -> 114793.68 : +1710.48 = +1.513% (+/-1.57%)
bm_float.py 256552.80 -> 243908.29 : -12644.51 = -4.929% (+/-1.90%)
bm_hexiom.py 521.93 -> 625.41 : +103.48 = +19.826% (+/-0.40%)
bm_nqueens.py 197544.25 -> 217713.12 : +20168.87 = +10.210% (+/-3.01%)
bm_pidigits.py 8072.98 -> 8198.75 : +125.77 = +1.558% (+/-3.22%)
misc_aes.py 17283.45 -> 16480.52 : -802.93 = -4.646% (+/-0.82%)
misc_mandel.py 99083.99 -> 128939.84 : +29855.85 = +30.132% (+/-5.88%)
misc_pystone.py 83860.10 -> 82592.56 : -1267.54 = -1.511% (+/-2.27%)
misc_raytrace.py 21490.40 -> 22227.23 : +736.83 = +3.429% (+/-1.88%)
This shows that the new optimisations are at least as good as the existing
inline-bytecode-caching, and are sometimes much better (because the new
ones apply caching to a wider variety of map lookups).
The new optimisations can also benefit code generated by the native
emitter, because they apply to the runtime rather than the generated code.
The improvement for the native emitter when LOAD_ATTR_FAST_PATH and
MAP_LOOKUP_CACHE are enabled is (same Linux environment as above):
diff of scores (higher is better)
N=2000 M=2000 native -> nat-attrmapcache diff diff% (error%)
bm_chaos.py 14130.62 -> 15464.68 : +1334.06 = +9.441% (+/-7.11%)
bm_fannkuch.py 74.96 -> 76.16 : +1.20 = +1.601% (+/-1.80%)
bm_fft.py 166682.99 -> 168221.86 : +1538.87 = +0.923% (+/-4.20%)
bm_float.py 233415.23 -> 265524.90 : +32109.67 = +13.756% (+/-2.57%)
bm_hexiom.py 628.59 -> 734.17 : +105.58 = +16.796% (+/-1.39%)
bm_nqueens.py 225418.44 -> 232926.45 : +7508.01 = +3.331% (+/-3.10%)
bm_pidigits.py 6322.00 -> 6379.52 : +57.52 = +0.910% (+/-5.62%)
misc_aes.py 20670.10 -> 27223.18 : +6553.08 = +31.703% (+/-1.56%)
misc_mandel.py 138221.11 -> 152014.01 : +13792.90 = +9.979% (+/-2.46%)
misc_pystone.py 85032.14 -> 105681.44 : +20649.30 = +24.284% (+/-2.25%)
misc_raytrace.py 19800.01 -> 23350.73 : +3550.72 = +17.933% (+/-2.79%)
In summary, compared to MICROPY_OPT_CACHE_MAP_LOOKUP_IN_BYTECODE, the new
MICROPY_OPT_LOAD_ATTR_FAST_PATH and MICROPY_OPT_MAP_LOOKUP_CACHE options:
- are simpler;
- take less code size;
- are faster (generally);
- work with code generated by the native emitter;
- can be used on embedded targets with a small and constant RAM overhead;
- allow the same .mpy bytecode to run on all targets.
See #7680 for further discussion. And see also #7653 for a discussion
about simplifying mpy-cross options.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
- Moves definition of BOARD_FLASH_SIZE and other header files related to
flash configuration into the Makefile.
- Adds board specific clock_config.h.
- Adds board.h, pin_mux.h, and peripherals.h as they are
required by NXP MCU SDK in order to use our own clock_config.h.
- Renames board specific FlexSPI configuration files.
- Updates flash frequency of MIMXRT1020_EVK
- Creates separated flash_config files for QSPI NOR and
QSPI Hyper flash.
- Unifies VFS start address to be @ 1M for 1010 and 1020 boards.
- Unifies 1050EVK boards
- Adds support to both NOR and HyperFlash on boards with
both capabilities.
- Adds automatic FlexRAM initialization to start-up code based on
linker script and NXP HAL.
- Applies code formatting to all files in mimxrt port.
With this change the flash configuration is restructured and
organized. This simplifies the configuration process and
provides a better overview of each board's settings. With the integration
of clock_config.h, board.h, pin_mux.h, and peripherals.h we gain better
control of the settings and clock configurations. Furthermore the
implementation of an explicit FlexRAM setup improves the system
performance and allows for performance tuning.
Signed-off-by: Philipp Ebensberger
To keep things neat and tidy, we ensure that each file has 1 and only 1
newline at the end of each file.
Signed-off-by: David Lechner <david@pybricks.com>
This feature {x=} was introduced in Python 3.8 so needs a separate .exp
file to run on earlier Python versions.
See https://bugs.python.org/issue36817
Signed-off-by: Damien George <damien@micropython.org>
This makes it work like --no-follow and --no-exclusive using a mutex group
and dest. Although the current implementation with BooleanOptionAction is
neater it requires Python 3.9, so don't use this feature.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The --no-exclusive flag was accidentally added to the mutex group in
178198a01d.
The --soft-reset flag was accidentally added to the mutex group in
41adf17830.
These flags can be specified independently to --[no-]follow so should not
be in that mutex group.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
And using "-B" means mpy-cross is forcefully rebuilt, sometimes with
invalid CFLAGS_EXTRA options which makes the auto-build fail.
Signed-off-by: Damien George <damien@micropython.org>
The sys.stdin.buffer and sys.stdout.buffer streams work just as well (and
are just as fast) as pyb.USB_VCP on stm32 devices, so there's no need to
have the USB_VCP specialisation code, which just adds complexity.
Also, on stm32 devices with both USB and UART (or other serial interface),
if something other than the USB_VCP port is used for the serial connection
then mpremote mount will not work because it will default to reading and
writing on USB_VCP instead of the other connected serial stream.
As part of this simplification, support for a second port as input is
removed (this feature was never exposed to the user).
Signed-off-by: Damien George <damien@micropython.org>
This implements (most of) the PEP-498 spec for f-strings and is based on
https://github.com/micropython/micropython/pull/4998 by @klardotsh.
It is implemented in the lexer as a syntax translation to `str.format`:
f"{a}" --> "{}".format(a)
It also supports:
f"{a=}" --> "a={}".format(a)
This is done by extracting the arguments into a temporary vstr buffer,
then after the string has been tokenized, the lexer input queue is saved
and the contents of the temporary vstr buffer are injected into the lexer
instead.
There are four main limitations:
- raw f-strings (`fr` or `rf` prefixes) are not supported and will raise
`SyntaxError: raw f-strings are not supported`.
- literal concatenation of f-strings with adjacent strings will fail
"{}" f"{a}" --> "{}{}".format(a) (str.format will incorrectly use
the braces from the non-f-string)
f"{a}" f"{a}" --> "{}".format(a) "{}".format(a) (cannot concatenate)
- PEP-498 requires the full parser to understand the interpolated
argument, however because this entirely runs in the lexer it cannot
resolve nested braces in expressions like
f"{'}'}"
- The !r, !s, and !a conversions are not supported.
Includes tests and cpydiffs.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
For consistency with other board-level config macros that begin with
MICROPY_HW_USB.
Also allow boards in the mimxrt, nrf and samd ports to configure these
values.
Signed-off-by: Damien George <damien@micropython.org>
Maureen Helm
Damien George
Sergei Silnov
Michael Bentley
Jim Mussared
Christian Decker
iabdalkader
Dave Hylands
robert-hh
Philipp Ebensberger
Patrick Van Oosterwijck
Seon Rozenblum
David Lechner
Glenn Ruben Bakke
Tobias Thyrrestrup
Michel Bouwmans