https://www.python.org/dev/peps/pep-0475/
This implements something similar to PEP 475 on the unix port, and for the
VfsPosix class.
There are a few differences from the CPython implementation:
- Since we call mp_handle_pending() between any ENITR's, additional
functions could be called if MICROPY_ENABLE_SCHEDULER is enabled, not
just signal handlers.
- CPython only handles signal on the main thread, so other threads will
raise InterruptedError instead of retrying. On MicroPython,
mp_handle_pending() will currently raise exceptions on any thread.
A new macro MP_HAL_RETRY_SYSCALL is introduced to reduce duplicated code
and ensure that all instances behave the same. This will also allow other
ports that use POSIX-like system calls (and use, eg, VfsPosix) to provide
their own implementation if needed.
Addition of GIL EXIT/ENTER pairs are:
- modos: release the GIL during system calls. CPython does this as well.
- moduselect: release the GIL during the poll() syscall. This call can be
blocking, so it is important to allow other threads to run at this time.
- modusocket: release the GIL during system calls. Many of these calls can
be blocking, so it is important to allow other threads to run.
- unix_mphal: release the GIL during the read and write syscalls in
mp_hal_stdin_rx_chr and mp_hal_stdout_tx_strn. If we don't do this
threads are blocked when the REPL or the builtin input function are used.
- file, main, mpconfigport.h: release GIL during syscalls in built-in
functions that could block.
This mechanism will scale to to an arbitrary number of pollable objects, so
long as they implement the MP_STREAM_GET_FILENO ioctl. Since ussl objects
pass through ioctl requests transparently to the underlying socket object,
it will allow ussl sockets to be polled. And a user object with uio.IOBase
as a base could support polling.
This is to keep the top-level directory clean, to make it clear what is
core and what is a port, and to allow the repository to grow with new ports
in a sustainable way.
Allows to iterate over the following without allocating on the heap:
- tuple
- list
- string, bytes
- bytearray, array
- dict (not dict.keys, dict.values, dict.items)
- set, frozenset
Allows to call the following without heap memory:
- all, any, min, max, sum
TODO: still need to allocate stack memory in bytecode for iter_buf.
ipoll() allows to poll streams without allocating any memory: this method
returns an iterator (a poll object itself), and the iterator yields
preallocated "callee-owned tuple" with polling results for each active
stream. The only operation a caller is allowed to do with this tuple is
extracting values from it (storing the tuple as a whole somewhere is
not allowed).
This makes unix "uselect" compatible with baremetal "uselect". Previosuly,
unix version accepted file/socket objects, but internally converted that
to file descriptors, and that's what .poll() returned. To acheive new
behavior, file-like objects are stored internally in an array, in addition
to existing array of struct pollfd. This array is created only on first
case of file-like object being passed to .register(). If only raw fd's are
passed, there will be no additional memory used comparing to the original
implementation.
This includes file and socket objects, backed by Unix file descriptor.
This improves compatibility with stmhal's uselect (and convenience of
use), though not completely: return value from poll.poll() is still
raw file descriptor.
After an I/O event is triggered for fd, event flags are automatically reset,
so no further events are reported until new event flags are set. This is
an optimization for uasyncio, required to account for coroutine semantics:
each coroutine issues explicit read/write async call, and once that trigger,
no events should be reported to coroutine, unless it again explicitly
requests it. One-shot mode saves one linear scan over the poll array.
Per CPython docs, "Registering a file descriptor that’s already registered
is not an error, and has the same effect as registering the descriptor
exactly once."
https://docs.python.org/3/library/select.html#select.poll.register
That's somewhat ambiguous, what's implemented here is that if fd si not
yet registered, it is registered. Otherwise, the effect is equivalent to
modify() method.
This allows the mp_obj_t type to be configured to something other than a
pointer-sized primitive type.
This patch also includes additional changes to allow the code to compile
when sizeof(mp_uint_t) != sizeof(void*), such as using size_t instead of
mp_uint_t, and various casts.
David Lechner
Damien George
Paul Sokolovsky
Alexander Steffen
Renato Aguiar