* Implement the critical part of wasi_thread_start in asm
It's fragile to set up the critical part of C environment in C.
* Specify --target for asm files as well
* wasi_thread_start: Move __tls_base initialization to asm as well
To make it easier to create a sysroot with both triples.
Eg.
```
make -j4 CC=/opt/wasi-sdk-16.0/bin/clang
make -j4 CC=/opt/wasi-sdk-16.0/bin/clang THREAD_MODEL=posix
```
In building some `libc-test` tests, I found these functions were not
compiled in. This change adds `pthread_barrier_init`,
`pthread_barrier_wait`, and `pthread_barrier_destroy` to the
`THREAD_MODEL=posix` build. As has been done with previous pthreads PRs,
this PR skips any inter-process locking by removing any calls to
`__vm_lock` and friends. If in the future WASI gains the "process"
concept, then these locations (and the pre-existing ones) will need to
be modified.
The pthreads API exposes functions for querying the attributes of a
thread. This change allows these functions to be compiled in the
`THREAD_MODEL=posix` build. Some functions are skipped (and documented);
they can be added if/when needed. This change is motivated by a
`libc-test` test that uses these functions.
1e4e2433bc
enabled sign-ext and mutable-globals by default, which adds
corresponding __wasm_-prefixed #defines.
9e956995db
changed the definition of __GNUC_VA_LIST to match that of GCC headers,
leaving it without a value.
* test: run a subset of tests from `libc-test`
This change introduces a `test` directory that retrieves the `libc-test`
suite, compiles a subset of the tests using `wasi-libc`, and runs them
with Wasmtime.
* ci: run tests during CI
This change includes some fixups to the filesystem to place Clang's
runtime library for `wasm32-wasi` in the right location. Note that this
CI action is limited to a single OS--Linux.
* threads: implement init of TLS and stack pointer
* fix: rename wasi_snapshot_preview2_thread_spawn to wasi_thread_spawn
Signed-off-by: Harald Hoyer <harald@profian.com>
* fix: change signature of wasi_thread_start
Signed-off-by: Harald Hoyer <harald@profian.com>
* fix: pthread_exit for WASI
Can't use `exit()` because it is too high level.
Have to unlock the thread list.
Signed-off-by: Harald Hoyer <harald@profian.com>
* fix: initialize struct pthread for the main thread
Signed-off-by: Harald Hoyer <harald@profian.com>
* fix: store the aligned stack minus `struct start_args`
Signed-off-by: Harald Hoyer <harald@profian.com>
Signed-off-by: Harald Hoyer <harald@profian.com>
Co-authored-by: Harald Hoyer <harald@profian.com>
- Avoid using Emscripten-specific functions
- Avoid using a constructor.
- Add support for allocating memory at `__heap_base`.
- Adjust the max-align value.
- Disable functions that wasi-libc doesn't currently publish.
- Add `__libc_` aliases.
* threads: implement `pthread_create`
As described in the [`wasi-threads`] proposal, this change implements
`pthread_create` using the new `wasi_thread_spawn(void *arg)` API. As
described there, `wasi-libc` exports the thread entry point with the
expected name, `wasi_thread_start`, and then unwraps the passed argument
`struct` to invoke the user function with the user argument `struct`.
[`wasi-threads`]: https://github.com/WebAssembly/wasi-threads/pull/5
Previously, the TID was only passed to the child thread entry point; the
parent thread was forced to wait until the child thread set the TID in
the pthread structure. With this change, the TID will be passed not only
to the child thread but also returned to the parent thread, so that
either side can make progress. The `i32.store` becomes an
`i32.atomic.store` to avoid concurrent writes.
The implementation is not as efficient as for native Linux platform due
to lack of FUTEX_REQUEUE-like system call in WASI.
For now we wake all the waiters which is inefficient; if that becomes
a bottleneck, I suggest we'll revisit the implementation.
[POSIX semaphores] come in two forms: named and unnamed. Roughly, named
semaphores use files to implement locking across processes; unnamed
semaphores use a shared memory region to implement locking across
threads in the same process. Since WASI currently has no process concept
(and it is relatively unclear how to map the WASI files as shared
memory), only the unnamed semaphores are supported by this changed. This
means that `sem_open`, `sem_close`, and `sem_unlink` will not available
to programs compiled with a threads-enabled `wasi-libc`.
[POSIX semaphores]: https://man7.org/linux/man-pages/man7/sem_overview.7.html
This change adds pthread's mutex support to the `THREAD_MODEL=posix`
build of wasi-libc. Some less-common features are unsupported and
documented here:
- mutex robust lists are disabled and their use will return a runtime
error; currently WASI does not support the concept of multiple
processes so maintaining robust mutexes across processes does not yet
make sense in wasi-libc
- timed locks with priority inheritance (PI) are disabled and will act
as any other mutex; this feature is related to task priorities which
is not yet relevant in the WASI ecosystem (see [priority-inheritance
futexes](https://man7.org/linux/man-pages/man2/futex.2.html))
- thread cancellation is ignored; this feature is difficult to support
and @sunfishcode would likely want to discuss this before adding it at
some later time.
* Fixes for the THREAD_MODEL=posix build
* Fix expected symbols from previous commit
* Enable `lock` in `random.c` when threads are enabled
This uses the `_REENTRANT` definition to indicate when the `lock` should
be available.
* Disable `aio.h` when compiling for threads
In talking to @sunfishcode about `aio.h`, this functionality is not yet
a primary concern (it was already disabled in the default,
single-threaded mode). Additionally, this change adds expectation lines
for the new symbols/includes added and removed by `pthread.h`.
This change was reached by running:
```console
$ git diff --no-index expected/wasm32-wasi sysroot/share/wasm32-wasi > patch.diff
# replace "sysroot/share" with "expected" in `patch.diff`
$ git apply patch.diff --reject
# manually fix any rejections
```
* Specify the TLS model until LLVM 15 is released
The `-ftls-model` configuration can be removed once https://reviews.llvm.org/D130053 makes its way into an upstream release.
* Rename `__wasi_libc_pthread_self` to `__wasilibc_pthread_self`
The symbol is still undefined, though.
* Add different sets of expected output based on THREAD_MODEL
* Re-add trailing whitespace to `predefined-macros.txt`
@sbc100 wanted to retain the whitespace trailing after certain
predefined macro lines. This change restores that whitespace from
upstream and re-generates the POSIX version using the following command:
```console
$ git diff --no-index expected/wasm32-wasi/posix/predefined-macros.txt sysroot/share/wasm32-wasi/predefined-macros.txt | sed 's/sysroot\/share\/wasm32-wasi/expected\/wasm32-wasi\/posix/' | git apply
```
* Protect `preopens.c` against concurrent access
* Only build thread-capable wasi-libc on latest version of Clang
* Use `thrd_sleep` from MUSL instead of aliasing `nanosleep`
* Define `pthread_setcancelstate` in `THREAD_MODEL=posix` builds
There are other options here (e.g., always define the `pthread_*`
symbols with stubs) but until we discuss that this is an intermediate
working step.
* Define a Wasm global to store `pthread_self`
* Remove `g_needs_dynamic_alloc` global
* Document the state of pthread support
* review: de-duplicate symbols based on #314
* review: only define `__wasilibc_cwd_{un}lock` when needed
* review: add #ifdefs to `__pthread_setcancelstate`
* review: add additional #ifdefs to `pthread_self.c`
* review: put lock definition behind #ifdef _REENTRANT
* review: remove pthread_setcancelstate.c
* review: re-fix indentation
* review: alias __clock_nanosleep in bottom half
* review: remove extra line
Co-authored-by: Sam Clegg <sbc@chromium.org>
For reproducible .a files in spite of non-deterministic
filesystem readdir order
Without this patch, find returned files in filesystem order
and llvm-ar used that order to create .a files.
This change extracts the `weak*`-related parts of #303 as a separate PR.
Note that this is slightly strange in that it uses some top-half MUSL
headers in the bottom-half code, but discussion around this led me to
believe that the advantages of, e.g., `LOCK` made this worthwhile.
Beyond just changing uses of `weak` to `__weak__`, we also MUSL's `weak`
and `weak_alias` macros in a few more places.
* Add a `getpagesize` function.
This adds a `getpagesize` function. This interface is deprecated in POSIX,
but it's sufficiently widely used and not problematic in practice.
* Use musl's `getpagesize`.
* Enable the `getpagesize` declaration in unistd.h.
The order that things happen in appears to have changed, and the
include-all.c script is now generated at a time when there can be
C++ headers in the sysroot, so adjust the script to exclude C++
headers.
If using a custom toolchain not on the path, explicitly invoking
ccache/sccache is required (also easier to debug that the right compiler
is being invoked). Fixes#253.
Add a build flag MALLOC_IMPL that can be set to dlmalloc or none
(defaulting to dlmalloc) which controls the malloc implementation to
use. The dlmalloc option is the same as before, but selecting none
removes dlmalloc from the libc build.
This flag replaces the BUILD_DLMALLOC flag, which never worked and thus
can be removed without breaking any builds. By switching to MALLOC_IMPL,
there is a clear path towards a different heap implementation, such as
mimalloc.
YAMAMOTO Takashi
Shengyun Zhou
Petr Penzin
Andrew Brown
Mike Hommey
Yuta Saito
Marcin Kolny
韩朴宇
Dan Gohman
Marcin Kolny
Bernhard M. Wiedemann
Cheng Shao
John Ericson
Vitali Lovich
Ayke van Laethem