In building some `libc-test` tests, I found these functions were not
compiled in. This change adds `pthread_barrier_init`,
`pthread_barrier_wait`, and `pthread_barrier_destroy` to the
`THREAD_MODEL=posix` build. As has been done with previous pthreads PRs,
this PR skips any inter-process locking by removing any calls to
`__vm_lock` and friends. If in the future WASI gains the "process"
concept, then these locations (and the pre-existing ones) will need to
be modified.
The pthreads API exposes functions for querying the attributes of a
thread. This change allows these functions to be compiled in the
`THREAD_MODEL=posix` build. Some functions are skipped (and documented);
they can be added if/when needed. This change is motivated by a
`libc-test` test that uses these functions.
as __wasi_errno_t is uint16_t, with the current coding,
__pthread_create will never see negative return values from
wasi:thread_spawn.
eg. (int)(uint16_t)-1 == 65535.
1e4e2433bc
enabled sign-ext and mutable-globals by default, which adds
corresponding __wasm_-prefixed #defines.
9e956995db
changed the definition of __GNUC_VA_LIST to match that of GCC headers,
leaving it without a value.
I've now tested the zero-inode path on a Wasm engine specially-modified
to have `fd_readdir` set inode numbers to zero. Fix two bugs this turned up:
- Increment `buffer_processed`, as noticed by @yamt
- Don't do an `fstatat` on "..", because that references a path outside
of the directory, which gets a permission-denied error.
* test: run a subset of tests from `libc-test`
This change introduces a `test` directory that retrieves the `libc-test`
suite, compiles a subset of the tests using `wasi-libc`, and runs them
with Wasmtime.
* ci: run tests during CI
This change includes some fixups to the filesystem to place Clang's
runtime library for `wasm32-wasi` in the right location. Note that this
CI action is limited to a single OS--Linux.
* If `fd_readdir` returns a zero inode, call `fstatat` to get the inode value.
On some systems, `fd_readdir` may not implement the `d_ino` field and
may set it to zero. When this happens, have wasi-libc call `fstatat` to
get the inode number.
See the discussion in
https://github.com/WebAssembly/wasi-filesystem/issues/65 for details.
* Update the `d_type` field too, in case it changes.
We download LLVM releases built for Ubuntu 18 because LLVM doesn't
always have builds for different versions, but the builds we use
depend on libtinfo5 which isn't installed on Ubuntu 20 by default.
So install it.
This function returns the address of `errno`, which makes it easier to
access from non-C languages since `errno` is a thread-local variable
which requires a special ABI.
* threads: implement init of TLS and stack pointer
* fix: rename wasi_snapshot_preview2_thread_spawn to wasi_thread_spawn
Signed-off-by: Harald Hoyer <harald@profian.com>
* fix: change signature of wasi_thread_start
Signed-off-by: Harald Hoyer <harald@profian.com>
* fix: pthread_exit for WASI
Can't use `exit()` because it is too high level.
Have to unlock the thread list.
Signed-off-by: Harald Hoyer <harald@profian.com>
* fix: initialize struct pthread for the main thread
Signed-off-by: Harald Hoyer <harald@profian.com>
* fix: store the aligned stack minus `struct start_args`
Signed-off-by: Harald Hoyer <harald@profian.com>
Signed-off-by: Harald Hoyer <harald@profian.com>
Co-authored-by: Harald Hoyer <harald@profian.com>
- Avoid using Emscripten-specific functions
- Avoid using a constructor.
- Add support for allocating memory at `__heap_base`.
- Adjust the max-align value.
- Disable functions that wasi-libc doesn't currently publish.
- Add `__libc_` aliases.
* Add a check to command modules to ensure that they're only started once.
Wasm command modules should only be called once per instance, because
the programming model doesn't leave linear memory in a reusable state
when the program exits. As use cases arise for loading wasm modules in
environments that want to treat them like reactors, add a safety check
to ensure that command modules are used according to their
expectations.
* threads: implement `pthread_create`
As described in the [`wasi-threads`] proposal, this change implements
`pthread_create` using the new `wasi_thread_spawn(void *arg)` API. As
described there, `wasi-libc` exports the thread entry point with the
expected name, `wasi_thread_start`, and then unwraps the passed argument
`struct` to invoke the user function with the user argument `struct`.
[`wasi-threads`]: https://github.com/WebAssembly/wasi-threads/pull/5
Previously, the TID was only passed to the child thread entry point; the
parent thread was forced to wait until the child thread set the TID in
the pthread structure. With this change, the TID will be passed not only
to the child thread but also returned to the parent thread, so that
either side can make progress. The `i32.store` becomes an
`i32.atomic.store` to avoid concurrent writes.
Previously, "new-style commmands" considered every user-defined
export to be a potential command entrypoint, so wasi-libc and wasm-ld
cooperated to run the user's static constructors on each entrypoint.
This form of new-style command turned out not to be useful, and it
interferes with some use cases, so disable it.
This is done by making an explicit call to `__wasm_call_ctors`, which
tells wasm-ld that it shouldn't synthesize any calls to
`__wasm_call_ctors` on its own.
I'm not sure if the function really has to be exported. If so, we should
probably move it to a separate compilation unit, otherwise it will be defined
multiple times (e.g. in `strerror.o` and `__lctrans.o`) causing linker errors.
However, I don't see a reason (at least for now) to export this function,
therefore making it static in this PR.
The implementation is not as efficient as for native Linux platform due
to lack of FUTEX_REQUEUE-like system call in WASI.
For now we wake all the waiters which is inefficient; if that becomes
a bottleneck, I suggest we'll revisit the implementation.
[POSIX semaphores] come in two forms: named and unnamed. Roughly, named
semaphores use files to implement locking across processes; unnamed
semaphores use a shared memory region to implement locking across
threads in the same process. Since WASI currently has no process concept
(and it is relatively unclear how to map the WASI files as shared
memory), only the unnamed semaphores are supported by this changed. This
means that `sem_open`, `sem_close`, and `sem_unlink` will not available
to programs compiled with a threads-enabled `wasi-libc`.
[POSIX semaphores]: https://man7.org/linux/man-pages/man7/sem_overview.7.html
This change adds pthread's mutex support to the `THREAD_MODEL=posix`
build of wasi-libc. Some less-common features are unsupported and
documented here:
- mutex robust lists are disabled and their use will return a runtime
error; currently WASI does not support the concept of multiple
processes so maintaining robust mutexes across processes does not yet
make sense in wasi-libc
- timed locks with priority inheritance (PI) are disabled and will act
as any other mutex; this feature is related to task priorities which
is not yet relevant in the WASI ecosystem (see [priority-inheritance
futexes](https://man7.org/linux/man-pages/man2/futex.2.html))
- thread cancellation is ignored; this feature is difficult to support
and @sunfishcode would likely want to discuss this before adding it at
some later time.
* Fixes for the THREAD_MODEL=posix build
* Fix expected symbols from previous commit
* Enable `lock` in `random.c` when threads are enabled
This uses the `_REENTRANT` definition to indicate when the `lock` should
be available.
* Disable `aio.h` when compiling for threads
In talking to @sunfishcode about `aio.h`, this functionality is not yet
a primary concern (it was already disabled in the default,
single-threaded mode). Additionally, this change adds expectation lines
for the new symbols/includes added and removed by `pthread.h`.
This change was reached by running:
```console
$ git diff --no-index expected/wasm32-wasi sysroot/share/wasm32-wasi > patch.diff
# replace "sysroot/share" with "expected" in `patch.diff`
$ git apply patch.diff --reject
# manually fix any rejections
```
* Specify the TLS model until LLVM 15 is released
The `-ftls-model` configuration can be removed once https://reviews.llvm.org/D130053 makes its way into an upstream release.
* Rename `__wasi_libc_pthread_self` to `__wasilibc_pthread_self`
The symbol is still undefined, though.
* Add different sets of expected output based on THREAD_MODEL
* Re-add trailing whitespace to `predefined-macros.txt`
@sbc100 wanted to retain the whitespace trailing after certain
predefined macro lines. This change restores that whitespace from
upstream and re-generates the POSIX version using the following command:
```console
$ git diff --no-index expected/wasm32-wasi/posix/predefined-macros.txt sysroot/share/wasm32-wasi/predefined-macros.txt | sed 's/sysroot\/share\/wasm32-wasi/expected\/wasm32-wasi\/posix/' | git apply
```
* Protect `preopens.c` against concurrent access
* Only build thread-capable wasi-libc on latest version of Clang
* Use `thrd_sleep` from MUSL instead of aliasing `nanosleep`
* Define `pthread_setcancelstate` in `THREAD_MODEL=posix` builds
There are other options here (e.g., always define the `pthread_*`
symbols with stubs) but until we discuss that this is an intermediate
working step.
* Define a Wasm global to store `pthread_self`
* Remove `g_needs_dynamic_alloc` global
* Document the state of pthread support
* review: de-duplicate symbols based on #314
* review: only define `__wasilibc_cwd_{un}lock` when needed
* review: add #ifdefs to `__pthread_setcancelstate`
* review: add additional #ifdefs to `pthread_self.c`
* review: put lock definition behind #ifdef _REENTRANT
* review: remove pthread_setcancelstate.c
* review: re-fix indentation
* review: alias __clock_nanosleep in bottom half
* review: remove extra line
Co-authored-by: Sam Clegg <sbc@chromium.org>
For reproducible .a files in spite of non-deterministic
filesystem readdir order
Without this patch, find returned files in filesystem order
and llvm-ar used that order to create .a files.
This change extracts the `weak*`-related parts of #303 as a separate PR.
Note that this is slightly strange in that it uses some top-half MUSL
headers in the bottom-half code, but discussion around this led me to
believe that the advantages of, e.g., `LOCK` made this worthwhile.
Beyond just changing uses of `weak` to `__weak__`, we also MUSL's `weak`
and `weak_alias` macros in a few more places.
* Delete several blocks of unused code.
Delete several pieces of code from libc-bottom-half/cloudlibc that aren't in
use on wasi-libc.
* Delete more of `_CLOCK_PROCESS_CPUTIME_ID` or `_CLOCK_THREAD_CPUTIME_ID`.
* Add a `getpagesize` function.
This adds a `getpagesize` function. This interface is deprecated in POSIX,
but it's sufficiently widely used and not problematic in practice.
* Use musl's `getpagesize`.
* Enable the `getpagesize` declaration in unistd.h.
* Remove support for `__original_main`.
This was used in old LLVM versions. Recent LLVM versions all emit either
`__main_void` or `__main_argv_argc`, so we can use those directly.
* Update the CI to use LLVM 14.0.0.
Previously, utimensat would leave the mtim and/or atim timestamps
uninitialized when the `MTIM_NOW` or `ATIM_NOW` were in use, because
that means the respective timestamps are not used.
However, clang now automatically adds `noundef` to the arguments in
functions like `__wasi_path_filestat_set_times`, and there are cases
where simplifycfg can see paths where the uninitialized values are
passed to those `noundef` arguments.
To fix this, change the utimens code to zero out the timestamps when
they aren't in use, to avoid passing uninitialized arguments.
YAMAMOTO Takashi
Andrew Brown
Dan Gohman
Mike Hommey
Yuta Saito
Marcin Kolny
韩朴宇
Marcin Kolny
Bernhard M. Wiedemann
msirringhaus
Cheng Shao