Instead of having crt1-command.o always call `__main_void` which is then
overridden by the linker, have crt1-command.o use weak symbols and test
both symbols for their presence. This should help libc.so, as it mean
libc.so won't need to depend on any of the main-related symbols.
Given there are already AF_* definitions, and they are (now) essentially
synonyms, we add those definitions to enable compilation of code that
already use PF_* macros.
In a multi-threaded execution we need to make sure that only exactly one
thread initializes malloc. The function try_init_allocator() can't
easily be made thread-safe, so just move the call to
try_init_allocator() inside the block that holds the lock.
This commit effectively drops the support of older wasm-ld. (LLVM <15.0.7).
We have two relevant use cases:
* `memory.grow` use outside of malloc
(eg. used by polyfill preview1 binaries)
* `--init-memory` to somehow preallocate heap
(eg. avoid dynamic allocations, especially on small environments)
While https://github.com/WebAssembly/wasi-libc/pull/377
fixed the former, it broke the latter if you are using
an older LLVM, which doesn't provide the `__heap_end` symbol,
to link your module.
As we couldn't come up with a solution which satisfies all parties,
this commit simply makes it require new enough LLVM which provides
`__heap_end`. After all, a link-time failure is more friendly to users
than failing later in a subtle way.
This changes the front-page documentation to:
- use `wasi-libc` instead of "WASI Libc"
- explain how to build the pthreads-enabled `wasm32-wasi-threads` target
* Convert preopen initialization to be lazy.
Insteead of eagerly initializing the preopens in a static constructor,
perform preopen initialization the first time it's needed, or before a
close or a renumber which might disrupt the file descriptor space.
And, use a weak symbol with a stub function for use by `close` or
`fd_renumber`, we that they can trigger preopen initialization only
if it's actually needed.
This way, if a program doesn't contain any calls to any function that
needs preopens, it can avoid linking in the preopen initialization code.
And if it contains calls but doesn't execute them at runtime, it can
avoid executing the preopen intiailization code.
A downside here is that this may cause problems for users that call
`__wasi_fd_close` or `__wasi_fd_renumber` directly and close over
overwrite some preopens before libc has a chance to scan them. To
partially address this, this PR does add a declaration for
`__wasilibc_populate_preopens` to <wasi/libc.h> so that users can call
it manually if they need to.
* Fix calling `internal_register_preopened_fd` with the lock held.
Factor out the lock acquisition from the implementation of
`internal_register_preopened_fd` so that we can call it from
`__wasilibc_populate_preopens` with the lock held.
This commit fixes the ability to build `wasi-libc` with `-g` options and
possibly without `-O2` options as well. I've found this useful when
debugging issues as historically that the build fails when `-g` is
passed or optimizations are removed due to the checks against these
expectation files. This commit adds more filters to the list of macros
to ensure that optimization/debug related ones are all removed from the
expectation lists.
This makes the output of the build a lot more concise and easy to read.
The only real change here is to build each of the crt1 startup files
individually instead to trying to build them all in a single clang
invocation (that latter doesn't allow for -o to be specified which is
a pretty severe limitation, so its best avoided anyway).
It also reduces the size of the `ar` command line for libc itself from
78017 to 43609 (on my machine), which sadly is still tool long for win32
I believe.
Calling _initialize multiple times is undefined behavior, since the
ctors are not guaranteed to be idempotent. We should have this safety
check which is similar to #329.
* Change `wasm32-wasi-pthread` to `wasm32-wasi-threads`
After some thought, I think that we should rename the `THREAD_MODEL=posix` build to avoid confusion. Why? Though in this project the use of this target does involve pthreads, it will not be so in other standard libraries or languages (see, e.g., https://github.com/rust-lang/compiler-team/issues/574). I think it would be preferable to emphasize the "threads" Wasm-level proposal and the "wasi-threads" proposal rather than the specific details of which threading API is being exposed.
* fix: rename the `expected` output directory as well
* Don't use sbrk(0) to determine the initial heap size
This commit changes the `try_init_allocator` function as part of
dlmalloc to not use `sbrk(0)` to determine the initial heap size. The
purpose of this function is to use the extra memory at the end of linear
memory for the initial allocation heap before `memory.grow` is used to
allocate more memory. To learn the extent of this region the code
previously would use `sbrk(0)` to find the current size of linear
memory. This does not work, however, when other systems have called
`memory.grow` before this function is called. For example if another
allocator is used or if another component of a wasm binary grows memory
for its own purposes then that memory will be incorrectly claimed to be
owned by dlmalloc.
Instead this commit rounds up the `__heap_base` address to the nearest
page size, since that must be allocatable. Otherwise anything above this
rounded address is assumed to be used by something else, even if it's
addressable.
* Use `__heap_end` if defined
* Move mstate initialization earlier
* Implement the critical part of wasi_thread_start in asm
It's fragile to set up the critical part of C environment in C.
* Specify --target for asm files as well
* wasi_thread_start: Move __tls_base initialization to asm as well
To make it easier to create a sysroot with both triples.
Eg.
```
make -j4 CC=/opt/wasi-sdk-16.0/bin/clang
make -j4 CC=/opt/wasi-sdk-16.0/bin/clang THREAD_MODEL=posix
```
When a user calls `open` with a path that does not have a corresponding
preopen, set errno to `ENOENT` rather than `ENOTCAPABLE`. This
conceptually represents an attempt to open a path which has not been
provided within the sandbox, so it's more accurately represented as
"not present" rather than "insufficient capabilities".
The current wasi-threads has no thread-exit functionality.
Thus it isn't straightforward to implement pthread_exit
without leaking thread context. This commit simply disables
pthread_exit for now.
Also, instead of abusing `wasi_proc_exit` for thread exit,
make `wasi_thread_start` return.
Note: `wasi_proc_exit` is supposed to terminate all threads
in the "process", not only the calling thread.
Note: Depending on the conclusion of the discussion about
`wasi_thread_exit`, we might revisit this change later.
References:
https://github.com/WebAssembly/wasi-threads/issues/7https://github.com/WebAssembly/wasi-threads/pull/17
When compiling with `-z stack-size` flag, only the main thread's stack
size is set to the specified value and other threads use musl's default value.
That's inconsistent with LLD's `-Wl,-stack_size`.
I think we can make it similar to MUSL's behavior, where thread's stack
size can be set via `PT_GNU_STACK` program header (via `-Wl,-z,stack-size`
flag).
Configuring stack size through `pthread_attr_t` still work as expected and
overrides the defaults ([pthread_create.c](be1ffd6a9e/libc-top-half/musl/src/thread/pthread_create.c (L362)))
default settings.
In building some `libc-test` tests, I found these functions were not
compiled in. This change adds `pthread_barrier_init`,
`pthread_barrier_wait`, and `pthread_barrier_destroy` to the
`THREAD_MODEL=posix` build. As has been done with previous pthreads PRs,
this PR skips any inter-process locking by removing any calls to
`__vm_lock` and friends. If in the future WASI gains the "process"
concept, then these locations (and the pre-existing ones) will need to
be modified.
The pthreads API exposes functions for querying the attributes of a
thread. This change allows these functions to be compiled in the
`THREAD_MODEL=posix` build. Some functions are skipped (and documented);
they can be added if/when needed. This change is motivated by a
`libc-test` test that uses these functions.
as __wasi_errno_t is uint16_t, with the current coding,
__pthread_create will never see negative return values from
wasi:thread_spawn.
eg. (int)(uint16_t)-1 == 65535.
1e4e2433bc
enabled sign-ext and mutable-globals by default, which adds
corresponding __wasm_-prefixed #defines.
9e956995db
changed the definition of __GNUC_VA_LIST to match that of GCC headers,
leaving it without a value.
I've now tested the zero-inode path on a Wasm engine specially-modified
to have `fd_readdir` set inode numbers to zero. Fix two bugs this turned up:
- Increment `buffer_processed`, as noticed by @yamt
- Don't do an `fstatat` on "..", because that references a path outside
of the directory, which gets a permission-denied error.
* test: run a subset of tests from `libc-test`
This change introduces a `test` directory that retrieves the `libc-test`
suite, compiles a subset of the tests using `wasi-libc`, and runs them
with Wasmtime.
* ci: run tests during CI
This change includes some fixups to the filesystem to place Clang's
runtime library for `wasm32-wasi` in the right location. Note that this
CI action is limited to a single OS--Linux.
Dan Gohman
Marcin Kolny
Moritz Sichert
YAMAMOTO Takashi
Josh Stone
Andrew Brown
Mike Hommey
Jiri Pospisil
Alex Crichton
Wenyong Huang
Cheng Shao
Pier Angelo Vendrame
Sam Clegg
Catherine
Shengyun Zhou
Petr Penzin
Yuta Saito