If `array.append()` fails with an exception due to heap exhaustion, the
next attempt to grow the buffer will cause a buffer overflow because the
free slot count is increased before performing the allocation, and will
stay as if the allocation succeeded.
Signed-off-by: Yoctopuce <dev@yoctopuce.com>
Fixes and improvements to `int.to_bytes()` are:
- No longer overflows if byte size is 0 (closes#13041).
- Raises OverflowError in any case where number won't fit into byte length
(now matches CPython, previously MicroPython would return a truncated
bytes object).
- Document that `micropython int.to_bytes()` doesn't implement the optional
signed kwarg, but will behave as if `signed=True` when the integer is
negative (this is the current behaviour). Add tests for this also.
Requires changes for small ints, MPZ large ints, and "long long" large
ints.
Adds a new set of unit tests for ints between 32 and 64 bits to increase
coverage of "long long" large ints, which are otherwise untested.
Tested on unix port (64 bit small ints, MPZ long ints) and Zephyr STM32WB
board (32 bit small ints, long long large ints).
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
The code generating the entry to the finally handler of an async-with
statement was simply wrong for the case of the native emitter. Among other
things the layout of the stack was incorrect.
This is fixed by this commit. The setup of the async-with finally handler
is now put in a dedicated emit function, for both the bytecode and native
emitters to implement in their own way (the bytecode emitter is unchanged,
just factored to a function).
With this fix all of the async-with tests now work when using the native
emitter.
Signed-off-by: Damien George <damien@micropython.org>
A value thrown/injected into a native generator needs to be stored in a
dedicated variable outside `nlr_buf_t`, following the `inject_exc` variable
in `py/vm.c`.
Signed-off-by: Damien George <damien@micropython.org>
This emitter prints out pseudo-machine instructions, instead of the usual
output of the native emitter. It can be enabled on any port via
`MICROPY_EMIT_NATIVE_DEBUG` (make sure other native emitters are disabled)
but the easiest way to use it is with mpy-cross:
$ mpy-cross -march=debug file.py
Signed-off-by: Damien George <damien@micropython.org>
Selected load/store code sequences have been optimised for RV32IMC when the
chance to use fewer and smaller opcodes was possible.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
MPY files can now hold generated RV32IMC native code. This can be
accomplished by passing the `-march=rv32imc` flag to mpy-cross.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This adds a native code generation backend for RISC-V RV32I CPUs, currently
limited to the I, M, and C instruction sets.
Signed-off-by: Alessandro Gatti <a.gatti@frob.it>
This fixes a minor issue in the changes made by
7dff38fdc190d7b731fad8319d2ae8aa13fde18a: the type flags for deque were
meant to be conditionalized based on MICROPY_PY_COLLECTIONS_DEQUE_ITER, but
the computed conditionalized value wasn't used.
Signed-off-by: Dan Halbert <halbert@halwitz.org>
Allows passing in a callback to `TaskQueue()` that is called when something
is pushed on to the queue.
Signed-off-by: Damien George <damien@micropython.org>
Support for raw str/bytes already exists, and extending that to raw
f-strings is easy. It also reduces code size because it eliminates an
error message.
Signed-off-by: Damien George <damien@micropython.org>
This is quite a simple and small change to support concatenation of
adjacent f-strings, and improve compatibility with CPython.
Signed-off-by: Damien George <damien@micropython.org>
Also define `mp_type_bytearray`. These all help to write native modules.
Signed-off-by: Brian Pugh <bnp117@gmail.com>
Signed-off-by: Damien George <damien@micropython.org>
Although the original motivation given for the workaround[1] is correct,
nlr.o and nlrthumb.o are linked with a small enough distance that the
problem does not occur, and the workaround isn't necessary. The distance
between the b instruction and its target (nlr_push_tail) is just 64
bytes[2], well within the ±2046 byte range addressable by an
unconditional branch instruction in Thumb mode.
The workaround induces a relocation in the text section (textrel), which
isn't supported everywhere, notably not on musl-libc[3], where it causes
a crash on start-up. With the workaround removed, micropython works on an
ARMv5T Linux system built with musl-libc.
This commit changes nlrthumb.c to use a direct jump by default, but
leaves the long jump workaround as an option for those cases where it's
actually needed.
[1]: commit dd376a239d
Author: Damien George <damien.p.george@gmail.com>
Date: Fri Sep 1 15:25:29 2017 +1000
py/nlrthumb: Get working again on standard Thumb arch (ie not Thumb2).
"b" on Thumb might not be long enough for the jump to nlr_push_tail so
it must be done indirectly.
[2]: Excerpt from objdump -d micropython:
000095c4 <nlr_push_tail>:
95c4: b510 push {r4, lr}
95c6: 0004 movs r4, r0
95c8: f02d fd42 bl 37050 <mp_thread_get_state>
95cc: 6943 ldr r3, [r0, #20]
95ce: 6023 str r3, [r4, #0]
95d0: 6144 str r4, [r0, #20]
95d2: 2000 movs r0, #0
95d4: bd10 pop {r4, pc}
000095d6 <nlr_pop>:
95d6: b510 push {r4, lr}
95d8: f02d fd3a bl 37050 <mp_thread_get_state>
95dc: 6943 ldr r3, [r0, #20]
95de: 681b ldr r3, [r3, #0]
95e0: 6143 str r3, [r0, #20]
95e2: bd10 pop {r4, pc}
000095e4 <nlr_push>:
95e4: 60c4 str r4, [r0, #12]
95e6: 6105 str r5, [r0, #16]
95e8: 6146 str r6, [r0, #20]
95ea: 6187 str r7, [r0, #24]
95ec: 4641 mov r1, r8
95ee: 61c1 str r1, [r0, #28]
95f0: 4649 mov r1, r9
95f2: 6201 str r1, [r0, #32]
95f4: 4651 mov r1, sl
95f6: 6241 str r1, [r0, #36] @ 0x24
95f8: 4659 mov r1, fp
95fa: 6281 str r1, [r0, #40] @ 0x28
95fc: 4669 mov r1, sp
95fe: 62c1 str r1, [r0, #44] @ 0x2c
9600: 4671 mov r1, lr
9602: 6081 str r1, [r0, #8]
9604: e7de b.n 95c4 <nlr_push_tail>
[3]: https://www.openwall.com/lists/musl/2020/09/25/4
Signed-off-by: J. Neuschäfer <j.ne@posteo.net>
Two cases, one assigning to a slice.
Closes https://github.com/micropython/micropython/issues/13283
Second is extending a slice from itself, similar logic.
In both cases the problem occurs when m_renew causes realloc to move the
buffer, leaving a dangling pointer behind.
There are more complex and hard to fix cases when either argument is a
memoryview into the buffer, currently resizing to a new address breaks
memoryviews into that object.
Reproducing this bug and confirming the fix was done by running the unix
port under valgrind with GC-aware extensions.
Note in default configurations with GIL this bug exists but has no impact
(the free buffer won't be reused while the function is still executing, and
is no longer referenced after it returns).
Signed-off-by: Angus Gratton <angus@redyak.com.au>
This commit swaps the order of the `flags` and `name` struct initialisers
for `mp_obj_type_t`, to fix an incompatibility with C++. The original
order of the initialiser didn't match the definition of the type, and
although that's still legal C, it's not legal C++.
Signed-off-by: Vonasmic <kasarkal123@gmail.com>
This is required because the .mpy native ABI was changed by the
introduction of `mp_proto_fun_t`, see commits:
- 416465d81e
- 5e3006f117
- e2ff00e811
And three `mp_binary` functions were added to `mp_fun_table` in
commit d2276f0d41.
Signed-off-by: Damien George <damien@micropython.org>
These are needed to read/write array.array objects, which is useful in
native code to provide fast extensions that work with big arrays of data.
Signed-off-by: Damien George <damien@micropython.org>
Newer versions of gcc (14 and up) have more sophisticated dead-code
detection, and the asm clobbers list needs to contain "memory" to inform
the compiler that the asm code actually does something.
Tested that adding this "memory" line does not change the generated code on
ARM Thumb2, x86-64 and Xtensa targets (using gcc 13.2).
Fixes issue #14115.
Signed-off-by: Damien George <damien@micropython.org>
Enabled by MICROPY_COMPILE_ALLOW_TOP_LEVEL_AWAIT. When enabled, this means
that scope such as module-level functions and REPL statements can yield.
The outer C code must then handle this yielded generator.
Signed-off-by: Damien George <damien@micropython.org>
This fixes a bug where a random Python object may become
un-garbage-collectable until an enclosing Python file (compiled on device)
finishes executing.
Details:
The mp_parse_tree_t structure is stored on the stack in top-level functions
such as parse_compile_execute() in pyexec.c (and others).
Although it quickly falls out of scope in these functions, it is usually
still in the current stack frame when the compiled code executes. (Compiler
dependent, but usually it's one stack push per function.)
This means if any Python object happens to allocate at the same address as
the (freed) root parse tree chunk, it's un-garbage-collectable as there's a
(dangling) pointer up the stack referencing this same address.
As reported by @GitHubsSilverBullet here:
https://github.com/orgs/micropython/discussions/14116#discussioncomment-8837214
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
This commit implements the 'e' half-float format: 10-bit mantissa, 5-bit
exponent. It uses native _Float16 if supported by the compiler, otherwise
uses custom bitshifting encoding/decoding routines.
Signed-off-by: Matthias Urlichs <matthias@urlichs.de>
Signed-off-by: Damien George <damien@micropython.org>
ASM_NOT_REG is optional, it can be synthesised by xor(reg, -1).
ASM_NEG_REG can also be synthesised with a subtraction, but most
architectures have a dedicated instruction for it.
Signed-off-by: Damien George <damien@micropython.org>
Add `pop()`, `appendleft()`, and `extend()` methods, support iteration
and indexing, and initializing from an existing sequence.
Iteration and indexing (subscription) have independent configuration flags
to enable them. They are enabled by default at the same level that
collections.deque is enabled (the extra features level).
Also add tests for checking new behavior.
Signed-off-by: Damien George <damien@micropython.org>
This new machine-module driver provides a "USBDevice" singleton object and
a shim TinyUSB "runtime" driver that delegates the descriptors and all of
the TinyUSB callbacks to Python functions. This allows writing arbitrary
USB devices in pure Python. It's also possible to have a base built-in
USB device implemented in C (eg CDC, or CDC+MSC) and a Python USB device
added on top of that.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
Disabled by default, but enabled on all boards that previously had
`MICROPY_PY_MACHINE_BARE_METAL_FUNCS` enabled.
Signed-off-by: Damien George <damien@micropython.org>
The STATIC macro was introduced a very long time ago in commit
d5df6cd44a. The original reason for this was
to have the option to define it to nothing so that all static functions
become global functions and therefore visible to certain debug tools, so
one could do function size comparison and other things.
This STATIC feature is rarely (if ever) used. And with the use of LTO and
heavy inline optimisation, analysing the size of individual functions when
they are not static is not a good representation of the size of code when
fully optimised.
So the macro does not have much use and it's simpler to just remove it.
Then you know exactly what it's doing. For example, newcomers don't have
to learn what the STATIC macro is and why it exists. Reading the code is
also less "loud" with a lowercase static.
One other minor point in favour of removing it, is that it stops bugs with
`STATIC inline`, which should always be `static inline`.
Methodology for this commit was:
1) git ls-files | egrep '\.[ch]$' | \
xargs sed -Ei "s/(^| )STATIC($| )/\1static\2/"
2) Do some manual cleanup in the diff by searching for the word STATIC in
comments and changing those back.
3) "git-grep STATIC docs/", manually fixed those cases.
4) "rg -t python STATIC", manually fixed codegen lines that used STATIC.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
This call used to be needed when there was an `emit_bc_pre()` function that
needed to be called at the start of each emitted bytecode. But in
8e7745eb31 that function was removed and now
the call to `mp_emit_bc_adjust_stack_size()` does nothing when adjusting by
0 entries, so it can be removed.
Signed-off-by: Damien George <damien@micropython.org>
For boards with MICROPY_CONFIG_ROM_LEVEL_AT_LEAST_CORE_FEATURES and up.
This gets samd21 boards working (which need the vfs module in _boot.py),
B_L072Z_LRWAN1, and nrf boards with smaller MCUs that use CORE or BASIC
feature levels.
Signed-off-by: robert-hh <robert@hammelrath.com>
Yoctopuce
Angus Gratton
Damien George
Alessandro Gatti
Dan Halbert
Brian Pugh
J. Neuschäfer
Vonasmic
stijn
Jim Mussared
Matthias Urlichs
Dash Peters
iabdalkader
Daniël van de Giessen
robert-hh