The STATIC macro was introduced a very long time ago in commit
d5df6cd44a. The original reason for this was
to have the option to define it to nothing so that all static functions
become global functions and therefore visible to certain debug tools, so
one could do function size comparison and other things.
This STATIC feature is rarely (if ever) used. And with the use of LTO and
heavy inline optimisation, analysing the size of individual functions when
they are not static is not a good representation of the size of code when
fully optimised.
So the macro does not have much use and it's simpler to just remove it.
Then you know exactly what it's doing. For example, newcomers don't have
to learn what the STATIC macro is and why it exists. Reading the code is
also less "loud" with a lowercase static.
One other minor point in favour of removing it, is that it stops bugs with
`STATIC inline`, which should always be `static inline`.
Methodology for this commit was:
1) git ls-files | egrep '\.[ch]$' | \
xargs sed -Ei "s/(^| )STATIC($| )/\1static\2/"
2) Do some manual cleanup in the diff by searching for the word STATIC in
comments and changing those back.
3) "git-grep STATIC docs/", manually fixed those cases.
4) "rg -t python STATIC", manually fixed codegen lines that used STATIC.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
The current `ssl` module has quite a few differences to the CPython
implementation. This change moves the MicroPython variant to a new `tls`
module and provides a wrapper module for `ssl` (in micropython-lib).
Users who only rely on implemented comparible behavior can continue to use
`ssl`, while users that rely on non-compatible behavior should switch to
`tls`. Then we can make the facade in `ssl` more strictly adhere to
CPython.
Signed-off-by: Felix Dörre <felix@dogcraft.de>
Changes include:
- Some mbedtls source files renamed or deprecated.
- Our `mbedtls_config.h` files are renamed to `mbedtls_config_port.h`, so
they don't clash with mbedtls's new default configuration file named
`mbedtls_config.h`.
- MBEDTLS_TLS_DEFAULT_ALLOW_SHA1_IN_KEY_EXCHANGE is deprecated.
- MBEDTLS_HAVE_TIME now requires an `mbedtls_ms_time` function to be
defined but it's only used for TLSv1.3 (currently not enabled in
MicroPython so there is a lazy implementation, i.e. seconds * 1000).
- `tests/multi_net/ssl_data.py` is removed (due to deprecation of
MBEDTLS_TLS_DEFAULT_ALLOW_SHA1_IN_KEY_EXCHANGE), there are the existing
`ssl_cert_rsa.py` and `sslcontext_server_client.py` tests which do very
similar, simple SSL data transfer.
- Tests now use an EC key by default (they are smaller and faster), and the
RSA key has been regenerated due to the old PKCS encoding used by openssl
rsa command, see
https://stackoverflow.com/questions/40822328/openssl-rsa-key-pem-and-der-conversion-does-not-match
(and `tests/README.md` has been updated accordingly).
Signed-off-by: Carlos Gil <carlosgilglez@gmail.com>
This will apply to bare-arm and minimal, as well as the minimal unix
variant.
Change the default to MICROPY_QSTR_BYTES_IN_HASH=1 for the CORE,BASIC
levels, 2 for >=EXTRA.
Removes explicit setting of MICROPY_QSTR_BYTES_IN_HASH==1 in ports that
don't set the feature level (because 1 is implied by the default level,
CORE). Applies to cc3200, pic16bt, powerpc.
Removes explicit setting for nRF (which sets feature level). Also for samd,
which sets CORE for d21 and FULL for d51. This means that d21 is unchanged
with MICROPY_QSTR_BYTES_IN_HASH==1, but d51 now moves from 1 to 2 (roughly
adds 1kiB).
The only remaining port which explicitly set bytes-in-hash is rp2 because
it's high-flash (hence CORE level) but lowish-SRAM, so it's worthwhile
saving the RAM for runtime qstrs.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
MicroPython code may rely on the return value of sys.stdout.buffer.write()
to reflect the number of bytes actually written. While in most scenarios a
write() operation is successful, there are cases where it fails, leading to
data loss. This problem arises because, currently, write() merely returns
the number of bytes it was supposed to write, without indication of
failure.
One scenario where write() might fail, is where USB is used and the
receiving end doesn't read quickly enough to empty the receive buffer. In
that case, write() on the MicroPython side can timeout, resulting in the
loss of data without any indication, a behavior observed notably in
communication between a Pi Pico as a client and a Linux host using the ACM
driver.
A complex issue arises with mp_hal_stdout_tx_strn() when it involves
multiple outputs, such as USB, dupterm and hardware UART. The challenge is
in handling cases where writing to one output is successful, but another
fails, either fully or partially. This patch implements the following
solution:
mp_hal_stdout_tx_strn() attempts to write len bytes to all of the possible
destinations for that data, and returns the minimum successful write
length.
The implementation of this is complicated by several factors:
- multiple outputs may be enabled or disabled at compiled time
- multiple outputs may be enabled or disabled at runtime
- mp_os_dupterm_tx_strn() is one such output, optionally containing
multiple additional outputs
- each of these outputs may or may not be able to report success
- each of these outputs may or may not be able to report partial writes
As a result, there's no single strategy that fits all ports, necessitating
unique logic for each instance of mp_hal_stdout_tx_strn().
Note that addressing sys.stdout.write() is more complex due to its data
modification process ("cooked" output), and it remains unchanged in this
patch. Developers who are concerned about accurate return values from
write operations should use sys.stdout.buffer.write().
This patch might disrupt some existing code, but it's also expected to
resolve issues, considering that the peculiar return value behavior of
sys.stdout.buffer.write() is not well-documented and likely not widely
known. Therefore, it's improbable that much existing code relies on the
previous behavior.
Signed-off-by: Maarten van der Schrieck <maarten@thingsconnected.nl>
Necessary to get coverage of the new event functions.
Deletes the case that called usleep(delay) for mp_hal_delay_ms(), it seems
like this wouldn't have ever happened anyhow (MICROPY_EVENT_POOL_HOOK is
always defined for the unix port).
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
All ports using this common configuration already enable time/date
validation, so this commit is a no-op change.
Signed-off-by: Damien George <damien@micropython.org>
Also move MICROPY_PY_PENDSV_ENTER/REENTER/EXIT to mphalport.h, for ports
where these are not already there.
This helps separate the hardware implementation of these macros from the
MicroPython configuration (eg for renesas-ra and stm32, the IRQ static
inline helper functions can now be moved to irq.h).
Signed-off-by: Damien George <damien@micropython.org>
And use it in all ports. The ports are unchanged, except esp8266 which now
just returns None from this function instead of the time elapsed (to match
other ports), and qemu-arm which gains this function.
Signed-off-by: Damien George <damien@micropython.org>
This is a code factoring to have the dict for the machine module in one
location, and all the ports use that same dict. The machine.soft_reset()
function implementation is also factored because it's the same for all
ports that did already implement it. Eventually more functions/bindings
can be factored.
All ports remain functionally the same, except:
- cc3200 port: gains soft_reset, mem8, mem16, mem32, Signal; loses POWER_ON
(which was a legacy constant, replaced long ago by PWRON_RESET)
- nrf port: gains Signal
- qemu-arm port: gains soft_reset
- unix port: gains soft_reset
- zephyr port: gains soft_reset, mem8, mem16, mem32
Signed-off-by: Damien George <damien@micropython.org>
This fixes the case where e.g.
struct foo_t {
mp_obj_t x;
uint16_t y;
char buf[];
};
will have `sizeof(struct foo_t)==8`, but `offsetof(struct foo_t, buf)==6`.
When computing the size to allocate for `m_new_obj_var` we need to use
offsetof to avoid over-allocating. This is important especially when it
might cause it to spill over into another GC block.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
The contents of machine_mem.h, machine_i2c.h and machine_spi.h have been
moved into extmod/modmachine.h.
Signed-off-by: Damien George <damien@micropython.org>
The contents of machine_bitstream.h, machine_pinbase.h, machine_pulse.h and
machine_signal.h have been moved into extmod/modmachine.h.
Signed-off-by: Damien George <damien@micropython.org>
If a non-string buffer was passed to execfile, then it would be passed
as a non-null-terminated char* to mp_lexer_new_from_file.
This changes mp_lexer_new_from_file to take a qstr instead (as in almost
all cases a qstr will be created from this input anyway to set the
`__file__` attribute on the module).
This now makes execfile require a string (not generic buffer) argument,
which is probably a good fix to make anyway.
Fixes issue #12522.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This provides a way to build a non-DEBUG host binary that still has symbols
and debug information.
Document this for the unix port, and update a comment in the unix port
Makefile.
This work was funded through GitHub Sponsors.
Signed-off-by: Angus Gratton <angus@redyak.com.au>
- Superfluous comments in MP_DEFINE_CONST_OBJ_TYPE stop correct macro
expanding.
- MP_ERROR_TEXT now gives mp_rom_error_text_t, but we want plain const
char *.
Signed-off-by: David Yang <mmyangfl@gmail.com>
In CPython, `_thread.start_new_thread()` returns an ID that is the same ID
that is returned by `_thread.get_ident()`. The current MicroPython
implementation of `_thread.start_new_thread()` always returns `None`.
This modifies the required functions to return a value. The native thread
id is returned since this can be used for interop with other functions, for
example, `pthread_kill()` on *nix. `_thread.get_ident()` is also modified
to return the native thread id so that the values match and avoids the need
for a separate `native_id` attribute.
Fixes issue #12153.
Signed-off-by: David Lechner <david@pybricks.com>
The only reason that const had to be disabled was to make the test output
match CPython when const was involved. Instead, this commit fixes the test
to handle the lines where const is used.
Also:
- remove the special handling for MICROPY_PERSISTENT_CODE_SAVE in
unix/mpconfigport.h, and make this automatic.
- move the check for MICROPY_PERSISTENT_CODE_SAVE to where it's used (like
we do for other similar checks) and add a comment explaining it.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
A previous commit removed the unix-specific select module implementation
and made unix use the common one.
This commit adds an optimisation so that the system poll function is used
when polling objects that have a file descriptor. With this optimisation
enabled, if code registers both file-descriptor-based objects, and non-
file-descriptor-based objects with select.poll() then the following occurs:
- the system poll is called for all file-descriptor-based objects with a
timeout of 1ms
- then the bare-metal polling implementation is used for remaining objects,
which calls into their ioctl method (which can be in C or Python)
In the case where all objects have file descriptors, the system poll is
called with the full timeout requested by the caller. That makes it as
efficient as possible in the case everything has a file descriptor.
Benefits of this approach:
- all ports use the same select module implementation
- the unix port now supports polling of all objects and matches bare metal
implementations
- it's still efficient for existing cases where only files and sockets are
polled (on unix)
- the bare metal implementation does not change
- polling of SSL objects will now work on unix by calling in to the ioctl
method on SSL objects (this is required for asyncio ssl support)
Note that extmod/vfs_posix_file.c has poll disable when the optimisation is
enabled, because the code is not reachable when the optimisation is used.
Signed-off-by: Damien George <damien@micropython.org>
The unix port has a custom select module which only works with objects that
have a file descriptor, eg files and sockets. On the other hand, bare
metal ports use the common extmod/modselect.c implementation of the select
module that supports polling of arbitrary objects, as long as those objects
provide a MP_STREAM_POLL in their ioctl implementation (which can be done
in C or Python).
This commit removes the unix-specific code and makes unix use the common
one provided by extmod/modselect.c instead. All objects with file
descriptors implement MP_STREAM_POLL so they continue to work.
Signed-off-by: Damien George <damien@micropython.org>
The asyncio module now has much better CPython compatibility and
deserves to be just called "asyncio".
This will avoid people having to write `from uasyncio import asyncio`.
Renames all files, and updates port manifests to use the new path. Also
renames the built-in _uasyncio to _asyncio.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Following how mkrules.cmake works. This makes it easy for a port to enable
frozen code, by defining FROZEN_MANIFEST in its Makefile.
Signed-off-by: Damien George <damien@micropython.org>
This is a MicroPython-specific module that existed to support the old
version of uasyncio. It's undocumented and not enabled on all ports and
takes up code size unnecessarily.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Previously sys.path could be modified by append/pop or slice assignment.
This allows `sys.path = [...]`, which can be simpler in many cases, but
also improves CPython compatibility.
It also allows sys.path to be set to a tuple which means that you can
clear sys.path (e.g. temporarily) with no allocations.
This also makes sys.path (and sys.argv for consistency) able to be disabled
via mpconfig. The unix port (and upytesthelper) require them, so they
explicitly verify that they're enabled.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
Updates any includes, and references from Makefiles/CMake.
This essentially reverts what was done long ago in commit
136b5cbd76
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This renames the builtin-modules, such that help('modules') and printing
the module object will show "module" rather than "umodule".
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This makes it so that sub-packages are resolved relative to their parent's
`__path__`, rather than re-resolving each parent's filesystem path.
The previous behavior was that `import foo.bar` would first re-search
`sys.path` for `foo`, then use the resulting path to find `bar`.
For already-loaded and u-prefixed modules, because we no longer need to
build the path from level to level, we no longer unnecessarily search
the filesystem. This should improve startup time.
Explicitly makes the resolving process clear:
- Loaded modules are returned immediately without touching the filesystem.
- Exact-match of builtins are also returned immediately.
- Then the filesystem search happens.
- If that fails, then the weak-link handling is applied.
This maintains the existing behavior: if a user writes `import time` they
will get time.py if it exits, otherwise the built-in utime. Whereas `import
utime` will always return the built-in.
This also fixes a regression from a7fa18c203
where we search the filesystem for built-ins. It is now only possible to
override u-prefixed builtins. This will remove a lot of filesystem stats
at startup, as micropython-specific modules (e.g. `pyb`) will no longer
attempt to look at the filesystem.
Added several improvements to the comments and some minor renaming and
refactoring to make it clearer how the import mechanism works. Overall
code size diff is +56 bytes on STM32.
This work was funded through GitHub Sponsors.
Signed-off-by: Jim Mussared <jim.mussared@gmail.com>
This modifies the unix port Makefile to use += for QSTR_DEFS and
QSTR_GLOBAL_DEPENDENCIES so that variants can add additional files if
needed (similar to stm32 port).
Signed-off-by: David Lechner <david@pybricks.com>
Damien George
Angus Gratton
YAMAMOTO Takashi
Felix Dörre
Carlosgg
Jim Mussared
Maarten van der Schrieck
vsfos
David Yang
David Lechner
Brett Cannon