This commit aims to address a discrepancy in Wasmtime where the world
supported by `wasmtime serve` is too large today. This includes
WIT interfaces which are not specified in `wasi:http/proxy` such as
`wasi:filesystem/types`, aka access to a filesystem.
This commit slims down `wasmtime serve` to, by default, only supporting
the `wasi:http/proxy` world. Like with `wasmtime run` various CLI flags
can be passed to enable more interfaces, however:
* `-Scommon` - this enables "common" interfaces such as
`wasi:filesystem`, `wasi:sockets`, and `wasi:cli/*`.
* `-Snn` - this enables wasi-nn
It's expected that more will get extended here over time too.
This change is enabled by a third build of the adapter, a "proxy" mode.
In this mode most functions are cfg'd to return `ERRNO_NOTSUP` to
indicate that the runtime does not support it. Notably this includes the
filesystem, arguments, and environment variables.
This change is tested by updating all `api_proxy*` tests to use this new
adapter which is now required that many previous interfaces are no
longer present by default in the proxy world.
* Replace the preview2 table's HashMap storage with a Vec
* Forgot to reserve index `2`
* Stop reserving 0, 1, and 2 in the table
* Exercise the free queue
* Remove unnecessary `_` suffix
* Remove push_child_
* Review feedback - switch free node tracking from a queue to a list
* Add `with_capacity` back in
* Fix comments
* Simplify pop_free_list
* Simplify iter_entries
* Winch: cleanup stack in br_if in non-fallthrough case
* Remove unnecessary refetch of sp_offsets
* Refactoring based on PR feedback
* Have SPOffset implement Ord
* mpk: allow forcing MPK during tests
For testing on machines on which we know MPK is enabled, we want to be
able to force-enable MPK, ensuring we get coverage of MPK-related code.
This change adds a `WASMTIME_TEST_FORCE_MPK` environment variable which,
when set, sets the pooling allocator configuration to force-enable MPK.
This variable, like the `WASMTIME_TEST_NO_HOG_MEMORY` variable it is
styled from, could be used in CI workflows on which we know MPK should
be available.
* review: only check `WASMTIME_TEST_FORCE_MPK` in tests
Checking the environment variable at runtime is too invasive and could
lead to unexpected behavior. This limits the use of
`WASMTIME_TEST_FORCE_MPK` to the `wast` tests and any tests that use the
`small_pool_config`.
This commit updates to the latest wasm-tools and `wit-bindgen` to bring
the family of crates forward. This update notably includes Nick's work
on packed indices in the `wasmparser` crate for validation for the
upcoming implementation of GC types. This meant that translation from
`wasmparser` types to Wasmtime types now may work with a "type id"
instead of just a type index which required plumbing not only Wasmtime's
own type information but additionally `wasmparser`'s type information
throughout translation.
This required a fair bit of refactoring to get this working but no
change in functionality is intended, only a different way of doing
everything prior.
This commit updates the semantics of `fd_{seek,tell}` on preview1 to
match native Unix when used with appending files. On Unix `write` claims
to always update the file position pointer to the end of the file, so
this commit implements that instead of the previous logic of ignoring
the position update for appending files. This currently requires an
extra roundtrip via `stat` to figure out the size of the file, but for
now that seems to be the best that can be done.
Closes#7583
* wasi: test for mtime accuracy to 1ms only
* add accurate time configuration
* fix conditional
* extend mtime accuracy cases
* remove unnecessary diff
* more cases
* reworking
* u64 tweak
* use direct precision checks in assertions
* Winch: fix bug by spilling when calling a func
* Forgot to commit new filetest
* Only support WasmHeapType::Func
* Elaborate on call_indirect jump details
* Update docs for call
* Verify stack is only consts and memory entries
This bug was discovered when testing with QEMU: the documentation states
that "bit 3" should be checked but this is a 0-based bit. The check
previously performed a 1-based "bit 3" check, which tests the UMIP
feature. This change switches to use the correct bit.
* ci: log CPU details when testing
When testing, there are certain CPU-dependent features that influence
Cranelift's codegen (e.g., availability of AVX512 instructions). This
additional CI step logs the current CPU information to aid in
troubleshooting, such as the MPK-related troubleshooting over in #7445.
Also, if we let this run in CI for a while, we may be able to run
queries on the logs to determine how often jobs run on servers with
certain features enabled.
prtest:full
* Add Windows variant of 'lscpu'
* Add MacOS variant of 'lscpu'
* move wasmtime-wasi's unit test for stdin to a separate integration test
fork is always a terrible idea, but when we wrote this test, we couldn't
think of an alternative method. alex showed us how
`/tests/host_segfault.rs` works, which solves a similar problem for
measuring process behavior without forking.
the forking version of this test would occasionally hang in the child's
creation of a tokio runtime because std Once is not fork-safe (nor
should it be. nothing should be fork-safe. forks are an abomination).
so instead, this is now a separate integration test with `harness =
false` that will exec itself in order to run the child.
* wasmtime-wasi: add tests to package include
* Configure Rust lints at the workspace level
This commit adds necessary configuration knobs to have lints configured
at the workspace level in Wasmtime rather than the crate level. This
uses a feature of Cargo first released with 1.74.0 (last week) of the
`[workspace.lints]` table. This should help create a more consistent set
of lints applied across all crates in our workspace in addition to
possibly running select clippy lints on CI as well.
* Move `unused_extern_crates` to the workspace level
This commit configures a `deny` lint level for the
`unused_extern_crates` lint to the workspace level rather than the
previous configuration at the individual crate level.
* Move `trivial_numeric_casts` to workspace level
* Change workspace lint levels to `warn`
CI will ensure that these don't get checked into the codebase and
otherwise provide fewer speed bumps for in-process development.
* Move `unstable_features` lint to workspace level
* Move `unused_import_braces` lint to workspace level
* Start running Clippy on CI
This commit configures our CI to run `cargo clippy --workspace` for all
merged PRs. Historically this hasn't been all the feasible due to the
amount of configuration required to control the number of warnings on
CI, but with Cargo's new `[lint]` table it's possible to have a
one-liner to silence all lints from Clippy by default. This commit by
default sets the `all` lint in Clippy to `allow` to by-default disable
warnings from Clippy. The goal of this PR is to enable selective access
to Clippy lints for Wasmtime on CI.
* Selectively enable `clippy::cast_sign_loss`
This would have fixed#7558 so try to head off future issues with that
by warning against this situation in a few crates. This lint is still
quite noisy though for Cranelift for example so it's not worthwhile at
this time to enable it for the whole workspace.
* Fix CI error
prtest:full
This commit fixes a bug in initializing memory segments of 32-bit
memories where if the offset was negative when viewed as a signed
integer the offset was incorrectly sign-extended to a 64-bit value
instead of zero-extended. This commit replaces an `i32`-to-`u64` cast
with an `i32`-to-`u32` cast followed by a `u32`-to-`u64` cast which
performs the zero extend.
Closes#7558
* rename preview1_remove_directory_trailing_slashes to preview1_remove_directory
and get rid its testing of the inconsistient behavior of removing a
directory using a trailing slash.
Then, unmark it as should_panic everywhere, so we have test coverage of
the behaviors that are consistient
prtest:full
* delete preview1_path_rename_file_trailing_slashes
this only tests behavior which is not consistient across platforms. the
rest of the behavior of path_rename is covered well by the path_rename
test and path_rename_dir_trailing_slashes
* debugging: see what CI tells me the preview1_interesting_paths chokes on in windows
* more debugging information
* interesting paths: windows allows a trailing nul apparently
* preview1_remove_directory: every os says notdir, apparently!
* no more should_panic tests!
* Invert logic to use `__builtin_{setjmp,longjmp}`
Instead of having the catch-all be the compiler intrinsics instead have
the catch-all be `sig{set,long}jmp`. It looks like GCC implements the
`__builtin_*` intrinsics but Clang only implements them on x86_64.
* Differentiate gcc/clang better
This commit solidifies the approach for unreachable code handling in
control flow.
Prior to this change, at unconditional jump sites, the compiler would
reset the machine stack as well as the value stack. Even though this
appoach might seem natural at first, it actually broke several of the
invariants that must be met at the end of each contol block, this was
specially noticeable with programs that conditionally entered in an
unreachable state, like for example
```wat
(module
(func (;0;) (param i32) (result i32)
local.get 0
local.get 0
if (result i32)
i32.const 1
return
else
i32.const 2
end
i32.sub
)
(export "main" (func 0))
)
```
The approach followed in this commit ensures that all the invariants are
met and introduces more guardrails around those invariants. In short,
instead of resetting the value stack at unconditional jump sites, the
value stack handling is deferred until the reachability analysis
restores the reachability of the code generation process, ensuring that
the value stack contains the exact amount of values expected by the
frame where reachability is restored. Given that unconditional jumps
reset the machine stack, when the reachability of the code generation
process is restored, the SP offset is also restored which should match
the size of the value stack.
This commit implements the `wasi_unstable` module, sometimes referred to
as "preview0", in the `wasmtime-wasi` crate. Previously this was only
implemented by the `wasi-common` crate but now this is implemented for
both meaning that the switch to preview2 won't lose this functionality.
The preview0 WITX files are vendored like the preview1 files and the
implementation of preview0 is exclusively implemented by delegating to
the preview1 implementation.
This commit slims down the default WASI interfaces needed by the adapter
by deferring the need to infer whether a stdio stream is a tty until
it's requested.
This commit removes the usage of wasi:sockets from the preview1 adapter.
It's never been implemented but has the side effect of requiring TCP
support to be imported into components even though it's never used. This
commit removes the stubs in the adapter to get filled in at a later date
if necessary.
It looks like clang does not expose `__builtin_{set,long}jmp` for some platforms.
This was a known issue previously for AArch64 and S390X, but it looks like it also affects RISC-V.
The fix here is to use libc's `sig{set,long}jmp` instead.
* Update `errno` crate to drop unused dependency
Small change to our dependency graph.
* Update rayon-core to drop some dependency edges
Just a bit of pruning for our dependency graph.
* Update num_cpus dependency to drop a dependency
Minor pruning of our dependency graph.
* winch: Introduce `ABIParams` and `ABIResults`
This commit prepares Winch to support WebAssembly Multi-Value.
The most notorious piece of this change is the introduction of the
`ABIParams` and `ABIResults` structs which are type wrappers around the
concept of an `ABIOperand`, which is the underlying main representation
of a param or result.
This change also consolidates how the size for WebAssembly types is
derived by introducing `ABI::sizeof`, as well as introducing
`ABI::stack_slot_size` to concretely indicate the stack slot size in
bytes for stack params, which is ABI dependent.
* winch: Add the necessary ABI building blocks for multi-value
This change adds the necessary changes at the ABI level in order to
handle multi-value.
The most notable modifications in this change are:
* Modifying Winch's default ABI to reverse the order of results,
ensuring that results that go in the stack should always come first;
this makes it easier to respect the following two stack invariants:
* Spilled memory values always precede register values
* Spilled values are stored from oldest to newest, matching their
respective locations on the machine stack.
* Modify all calling conventions supported by Winch so that only one result, the first one is stored in
registers. This differs from their vanilla counterparts in that these
ABIs can handle multiple results in registers. Given that Winch is not
a generic code generator, keeping the ABI close to what Wasmtime
expects makes it easier to pass multiple results at trampolines.
* Add more multi-value tests
This commit adds more tests for multi-value and improves documentation.
prtest:full
* Address review feedback
* Implement the missing http-error-code function
* Add concrete error conversion functions
* Test the error returned when writing too much
* Remove an unused import
* Only log errors when they will get the default representation
* Use `source` instead of `into_cause`
* wit deps: use 0.2.0-rc-2023-11-10 from WebAssembly/wasi-* repos
* local wits: depend on rc-2023-11-10
* wit-bindgen invocation: use rc-2023-11-10
* wasi-http wit subdir: sync with wasi
* wasi-http wit-bindgen: update for 11-10 rc
* wasi-preview1-component-adapter: imports are from 11-10 rc
* cli test component-basic: use wasi rc 11-10
* Modify outgoing-body.finish to return a result
* Validate content length when finishing an outgoing-body
* Track written vs expected, instead of decrementing
* Switch to the new errors
* Move content-length tracking to BodyWriteStream
* Review feedback
* Remove a TODO
Allows users to avoid runaway rules that match too much stuff.
No statistically significant speed up to compilation on sightglass, but good to
have as a safeguard anyways.
Fixes#7500
* Migrate to a more specific error-code variant in wasi-http
Co-authored-by: Pat Hickey <phickey@fastly.com>
* Optional fields, and align with upstream pr
* Update for upstream changes to the error-code variant
* Sync with the upstream implementation
* Missed updating an error for riscv64 and s390x
* More debuggable error
prtest:full
* Try to stabilize the test on windows
---------
Co-authored-by: Pat Hickey <phickey@fastly.com>
Alex Crichton
Trevor Elliott
Dave Bakker
Tyler Rockwood
scottmcm
Jeffrey Charles
Caio Ramos Casimiro
Andrew Brown
Mrmaxmeier
Collin Styles
Guy Bedford
pyroMechanical
Brett Cannon
vwkd
Pat Hickey
Guy Bedford
wasmtime-publish
Saúl Cabrera
Afonso Bordado
Nick Fitzgerald