* Don't differentiate between decommitting table vs stack pages
All implementations are the same either way, and by treating them differently,
it makes it difficult to prototype integration with batching syscalls that need
to treat them homogenously.
Co-Authored-By: Jamey Sharp <jsharp@fastly.com>
* Remove the `madvise_dontneed` function from the system virtual memory interface
This was already implemented by the existing `decommit_pages` function, we just
need to determine what the behavior is for the left over mapping. Specifically,
whether it is zeroed or restored to the original mapping (e.g. a CoW image).
This allows us to handle `decommit_pages` and what previously were calls to
`madvise_dontneed` homogenously, which enables us to prototype new system calls
that batch decommits together and therefore must treat them all the same.
Co-Authored-By: Jamey Sharp <jsharp@fastly.com>
---------
Co-authored-by: Jamey Sharp <jsharp@fastly.com>
* wasmtime(gc): Fix wasm-to-native trampoline lookup for subtyping
Previously, we would look up a wasm-to-native trampoline in the Wasm module
based on the host function's type. With Wasm GC and subtyping, this becomes
problematic because a Wasm module can import a function of type `T` but the host
can define a function of type `U` where `U <: T`. And if the Wasm has never
defined type `U` then it wouldn't have a trampoline for it. But our trampolines
don't actually care, they treat all reference values within the same type
hierarchy identically. So the trampoline for `T` would have worked in
practice. But once we find a trampoline for a function, we cache it and reuse it
every time that function is used in the same store again. Even if the function
is imported with its precise type somewhere else. So then we would have a
trampoline of the wrong type. But this happened to be okay in practice because
the trampolines happen not to inspect their arguments or do anything with them
other than forward them between calling convention locations. But relying on
that accidental invariant seems fragile and like a gun aimed at the future's
feet.
This commit makes that invariant non-accidental, centering it and hopefully
making it less fragile by doing so, by making every function type have an
associated "trampoline type". A trampoline type is the original function type
but where all the reference types in its params and results are replaced with
the nullable top versions, e.g. `(ref $my_struct)` is replaced with `(ref null
any)`. Often a function type is its own associated trampoline type, as is the
case for all functions that don't have take or return any references, for
example. Then, all trampoline lookup begins by first getting the trampoline type
of the actual function type, or actual import type, and then only afterwards
finding for the pre-compiled trampoline in the Wasm module.
Fixes https://github.com/bytecodealliance/wasmtime/issues/8432
Co-Authored-By: Jamey Sharp <jsharp@fastly.com>
* Fix no-std build
---------
Co-authored-by: Jamey Sharp <jsharp@fastly.com>
* Use WASI builder directly in C API
This commit updates the C API to use the `WasiCtxBuilder` directly
within `wasi_config_t` instead of buffering up the options separately.
This keeps the behavior of the Rust-based API more similar to the C API
and should also help resolve#8552 due to errors being returned more
eagerly in the builder-based API.
This additionally makes some minor modifications to the C APIs here as
appropriate.
Close#8552
* Review comments
Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=68735.
That fuzzbug bisected to the call-indirect caching changes, but this
turned out to be a red herring (the options added in that PR mean that
the fuzzbug config deserializes differently prior to the commit). In
any case, it's an easy fix -- it appears that V8 added a new error
message, so we need to add it to the allowlist of messages that we
expect for a table out-of-bounds condition.
This is extremely useful for cases where the default optimizations just
are not enough.
Background: [neovim](https://github.com/neovim/neovim) is interested to
add wasmtime support in https://github.com/neovim/neovim/pull/28415 but
we noticed that including wasmtime, even when not using wasmtime
directly, heavily affects runtime performance. This is not only
reflected in the increased startuptime but affects the runtime
performance overall.
Here are the benchmarks for startuptimes for different configurations.
Important to note is that all of runtime is affected, but the
startuptime is a decent proxy to measure runtime performance:
```
No wasm
Time (mean ± σ): 50.5 ms ± 1.5 ms [User: 32.8 ms, System: 12.3 ms]
Range (min … max): 48.3 ms … 54.4 ms 56 runs
Wasm, lto=thin
Time (mean ± σ): 104.9 ms ± 3.5 ms [User: 86.5 ms, System: 12.7 ms]
Range (min … max): 99.5 ms … 111.1 ms 26 runs
Wasm, lto=true
Time (mean ± σ): 83.8 ms ± 2.5 ms [User: 65.8 ms, System: 12.1 ms]
Range (min … max): 80.5 ms … 93.3 ms 31 runs
Wasm, lto=true, strip="none", incremental=false, codegen-units=1, panic="abort"
Time (mean ± σ): 53.1 ms ± 1.0 ms [User: 35.5 ms, System: 12.5 ms]
Range (min … max): 50.6 ms … 55.5 ms 54 runs
```
* Bump Wasmtime's MSRV to 1.76.0
* Update Rust in CI to 1.78.0, the current stable
* Update nightly tests to the latest nightly
prtest:full
* Fix check-cfg with nightly
* More check-cfg fixes
* Remove an async cfg
This is no longer specified for the root crate.
* Move definition of Wasmtime's nightly into one place
Don't change a bunch of places when this is updated, try to update just
one single location instead.
* Always fall back to custom platform for Wasmtime
This commit updates Wasmtime's platform support to no longer require an
opt-in `RUSTFLAGS` `--cfg` flag to be specified. With `no_std` becoming
officially supported this should provide a better onboarding experience
where the fallback custom platform is used. This will cause linker
errors if the symbols aren't implemented and searching/googling should
lead back to our docs/repo (eventually, hopefully).
* Change Wasmtime's TLS state to a single pointer
This commit updates the management of TLS to rely on just a single
pointer rather than a pair of a pointer and a `bool`. Additionally
management of the TLS state is pushed into platform-specific modules to
enable different means of managing it, namely the "custom" platform now
has a C function required to implement TLS state for Wasmtime.
* Delay conversion to `Instant` in atomic intrinsics
The `Duration` type is available in `no_std` but the `Instant` type is
not. The intention is to only support the `threads` proposal if `std` is
active but to assist with this split push the `Duration` further into
Wasmtime to avoid using a type that can't be mentioned in `no_std`.
* Gate more parts of Wasmtime on the `profiling` feature
Move `serde_json` to an optional dependency and gate the guest profiler
entirely on the `profiling` feature.
* Refactor conversion to `anyhow::Error` in `wasmtime-environ`
Have a dedicated trait for consuming `self` in addition to a
`Result`-friendly trait.
* Gate `gimli` in Wasmtime on `addr2line`
Cut down the dependency list if `addr2line` isn't enabled since then
the dependency is not used. While here additionally lift the version
requirement for `addr2line` up to the workspace level.
* Update `bindgen!` to have `no_std`-compatible output
Pull most types from Wasmtime's `__internal` module as the source of
truth.
* Use an `Option` for `gc_store` instead of `OnceCell`
No need for synchronization here when mutability is already available in
the necessary contexts.
* Enable embedder-defined host feature detection
* Add `#![no_std]` support to the `wasmtime` crate
This commit enables compiling the `runtime`, `gc`, and `component-model`
features of the `wasmtime` crate on targets that do not have `std`. This
tags the crate as `#![no_std]` and then updates everything internally to
import from `core` or `alloc` and adapt for the various idioms. This
ended up requiring some relatively extensive changes, but nothing too
too bad in the grand scheme of things.
* Require `std` for the perfmap profiling agent
prtest:full
* Fix build on wasm
* Fix windows build
* Remove unused import
* Fix Windows/Unix build without `std` feature
* Fix some doc links
* Remove unused import
* Fix build of wasi-common in isolation
* Fix no_std build on macos
* Re-fix build
* Fix standalone build of wasmtime-cli-flags
* Resolve a merge conflict
* Review comments
* Remove unused import
* Call-indirect caching: protect against out-of-bounds table index during prescan.
Call-indirect caching requires a "prescan" of a module's code section
during compilation in order to know which tables are possibly written,
and to count call-indirect callsites so that each separate function
compilation can enable caching if possible and use the right slot
range.
This prescan is not integrated with the usual validation logic (nor
should it be, probably, for performance), so it's possible that an
out-of-bounds table index or other illegal instruction could be
present. We previously indexed into an internal data
structure (`table_plans`) with this index, allowing for a compilation
panic on certain invalid modules before validation would have caught
it. This PR fixes that with a one-off check at the single point that
we interpret a parameter (the table index) from an instruction during
the prescan.
* Add test case.
* Respect pooling allocation options in `wasmtime serve`
This commit updates the processing of pooling allocator options in
`wasmtime serve`. Previously the pooling allocator was enabled by
default but the options to configure it weren't processed due to how
this default-enable was implemented. The option to enable it by default
for `wasmtime serve`, but only `wasmtime serve`, is now processed
differently in a way that handles various other
pooling-allocator-related options.
Closes#8504
* Fix compile of bench api
* Fix test build
* Ignore newly added test as it's flaky
This commit fixes a fuzz bug that popped up where the
`cache_call_indirects` feature wasn't reading memory64-based offsets
correctly. This is due to (not great) API design in `wasmparser`
(introduced by me) where wasmparser by default won't read 64-bit offsets
unless explicitly allowed to. This is to handle how spec tests assert
that overlong 32-bit encodings are invalid as the memory64 proposal
isn't merged into the spec yet.
The fix here is to call `allow_memarg64` with whether memory64 is
enabled or not and then that'll enable reading these overlong and/or
larger offsets correctly.
As per [this comment], the call-indirect caching options should really
be in the `-O` option namespace (for optimizations), not `-W` (for
semantically-visible Wasm standards options); the original PR simply put
them in the wrong place, and this PR moves them.
[this comment]: https://github.com/bytecodealliance/wasmtime/pull/8509#discussion_r1589594152
* Enable the tail calling convention by default
Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
* Move conditional default initialization to its own method
* Fix comment about when tail calls get enabled automatically
---------
Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
This commit removes all our `#[cfg_attr(..., doc(cfg(...)))]`
annotations throughout Wasmtime and `wasmtime-wasi`. These are all
replaced with `feature(doc_auto_cfg)` which automatically infers the
attribute to show rather than requiring us to duplicate it.
Spot-checking the docs this looks just-as-readable while being much
easier to maintain over time.
Similar to https://github.com/bytecodealliance/wasmtime/pull/8481 but for struct
types instead of array types.
Note that this is support for only defining these types in Wasm or the host; we
don't support allocating instances of these types yet. That will come in follow
up PRs.
* Migrate the `wasmtime-environ` crate to `no_std`
This commit migrates the `wasmtime-environ` crate to by default being
tagged with `#![no_std]`. Only the `component-model` and `gc` features
are able to be built without `std`, all other features will implicitly
activate the `std` feature as they currently require it one way or
another. CI is updated to build `wasmtime-environ` with these two
features active on a no_std platform.
This additionally, for the workspace, disables the `std` feature for the
`target-lexicon`, `indexmap`, `object`, and `gimli` dependencies. For
object/gimli all other crates in the workspace now enable the `std`
feature, but for `wasmtime-environ` this activation is omitted.
The `thiserror` dependency was dropped from `wasmtime-environ` and
additionally `hashbrown` was added for explicit usage of maps.
* Always enable `std` for environ for now
prtest:full
* Add some more std features
* wasi-nn: remove Git submodule
To more closely align with the conventions in the `wasmtime-wasi` and
`wasmtime-wasi-http` crates, this change removes the Git submodule that
previously provided the WIT and WITX files for `wasmtime-wasi-nn`. Like
those other crates, the syncing of wasi-nn WIT and WITX files will
happen manually for the time being. This is the first PR towards
upgrading the wasi-nn implementation to match recent spec changes and
better preview2-ABI compatibility.
prtest:full
* ci: auto-vendor the wasi-nn WIT files
* Double the default allowed table elements
This commit doubles the default allowed table elements per table in the
pooling allocator from 10k to 20k. This helps to, by default, run the
module produced in #8504.
* Update docs on deafults
`wasi-nn`'s test program suite is light at the moment but, in order to
expand it, this change factors out some of the common bits that are
being used in the `test-programs` crate. Since all of the tests perform
some kind of image classification, the new `nn` module gains `classify`
and `sort_results` functions to help with this exact case.
prtest:full
* Migrate the wasmtime-types crate to no_std
This commit is where no_std for Wasmtime starts to get a bit
interesting. Specifically the `wasmtime-types` crate is the first crate
that depends on some nontrivial crates that also need to be migrated to
`no_std`. This PR disables the default feature of `wasmparser` by
default and additionally does the same for `serde`. This enables them to
compile in `no_std` contexts by default and default features will be
enabled elsewhere in this repository as necessary.
This also opts to drop the `thiserror` dependency entirely in favor of a
manual `Display` implementation with a cfg'd implementation of `Error`.
As before CI checks are added for `wasmtime-types` with a `no_std`
target itself to ensure the crate and all dependencies all avoid `std`.
* Fix adapter build
* Wasmtime: add one-entry call-indirect caching.
In WebAssembly, an indirect call is somewhat slow, because of the
indirection required by CFI (control-flow integrity) sandboxing. In
particular, a "function pointer" in most source languages compiled to
Wasm is represented by an index into a table of funcrefs. The
`call_indirect` instruction then has to do the following steps to invoke
a function pointer:
- Load the funcref table's base and length values from the vmctx.
- Bounds-check the invoked index against the actual table size; trap if
out-of-bounds.
- Spectre mitigation (cmove) on that bounds-check.
- Load the `vmfuncref` from the table given base and index.
- For lazy table init, check if this is a non-initialized funcref
pointer, and initialize the entry.
- Load the signature from the funcref struct and compare it against the
`call_indirect`'s expected signature; trap if wrong.
- Load the actual code pointer for the callee's Wasm-ABI entry point.
- Load the callee vmctx (which may be different for a cross-module
call).
- Put that vmctx in arg 0, our vmctx in arg 1, and invoke the loaded
code pointer with an indirect call instruction.
Compare and contrast to the process involved in invoking a native
function pointer:
- Invoke the code pointer with an indirect call instruction.
This overhead buys us something -- it is part of the SFI sandbox
boundary -- but it is very repetitive and unnecessary work in *most*
cases when indirect function calls are performed repeatedly (such as
within an inner loop).
This PR introduces the idea of *caching*: if we know that the result of
all the above checks won't change, then if we use the same index as "the
last time" (for some definition), we can skip straight to the "invoke
the code pointer" step, with a cached code pointer from that last time.
Concretely, it introduces a two-word struct inlined into the vmctx for
each `call_indirect` instruction in the module (up to a limit):
- The last invoked index;
- The code pointer that index corresponded to.
When compiling the module, we check whether the table could possibly be
mutable at a given index once read: any instructions like `table.set`,
or the whole table exported thus writable from the outside. We also
check whether index 0 is a non-null funcref. If neither of these things
are true, then we know we can cache an index-to-code-pointer mapping,
and we know we can use index 0 as a sentinel for "no cached value".
We then make use of the struct for each indirect call site and generate
code to check if the index matches; if so, call cached pointer; if not,
load the vmfuncref, check the signature, check that the callee vmctx is
the same as caller (intra-module case), and stash the code pointer and
index away (fill the cache), then make the call.
On an in-development branch of SpiderMonkey-in-Wasm with ICs (using
indirect calls), this is about a 20% speedup; I haven't yet measured on
other benchmarks. It is expected that this might be an
instantiation-time slowdown due to a larger vmctx (but we could use
madvise to zero if needed).
This feature is off by default right now.
* Addressed review feedback.
* Added some more comments.
* Allow unused VMCallIndirectCache struct (defined for parity with other bits but not needed in actual runtime).
* Add a limit to the number of call-indirect cache slots.
* Fix merge conflict: handle ConstOp element offset.
* Review feedback.
* wasmtime: Use ConstExpr for element segment offsets
This shouldn't change any behavior currently, but prepares us for
supporting extended constant expressions.
* Fix clippy::cast_sign_loss lint
* Expose `wasmtime-runtime` as `crate::runtime::vm` internally for the `wasmtime` crate
* Rewrite uses of `wasmtime_runtime` to `crate::runtime::vm`
* Remove dep on `wasmtime-runtime` from `wasmtime-cli`
* Move the `wasmtime-runtime` crate into the `wasmtime::runtime::vm` module
* Update labeler for merged crates
* Fix `publish verify`
prtest:full
Instead of
> Performing build step for
'wasmtime-crate''WASMTIME_CARGO_BINARY-NOTFOUND' is not recognized as an
internal or external command, operable program or batch file.
this will now instead output
> "cargo" was not found. Ensure "cargo" is in PATH. Aborting...
This commit adds support for defining array types from Wasm or the host, and
managing them inside the engine's types registry. It does not introduce support
for allocating or manipulating array values. That functionality will come in
future pull requests.
* Add an example for wasi-nn WinML backend.
This example is similar as classification-component-onnx, but it's not a wasm
component, and it's for WinML backend. It also includes a step by step
instruction for running this example.
* Touch up the documentation for this example
This change removes some duplicated information and tweaks some of the
wording.
---------
Co-authored-by: Andrew Brown <andrew.brown@intel.com>
* Gate type-builder types from `wasmtime-environ` on `compile`
This commit gates the `*Builder` types related to building sets of types
in the `wasmtime-environ` crate on the `compile` feature. This helps
bring in less code when the feature is disabled and helps exclude some
dependencies for the upcoming `no_std` migration as well.
This commit doesn't change anything, it's just moving code around.
* Remove no-longer-needed import
prtest:full
Currently the CMakeLists.txt is designed to only be consumed by a parent
project, which limits its flexibility. Adding a `project()` function
allows it to also be built as its own project.
Also set `USES_TERMINAL_BUILD` to `TRUE` in `ExternalProject_Add()`,
which allows getting the installation progress output from `cargo
build`.
* Start migrating some Wasmtime crates to no_std
This commit is the first in what will be multiple PRs to migrate
Wasmtime to being compatible with `#![no_std]`. This work is outlined
in #8341 and the rough plan I have in mind is to go on a crate-by-crate
basis and use CI as a "ratchet" to ensure that `no_std` compat is
preserved. In that sense this PR is a bit of a template for future PRs.
This PR migrates a few small crates to `no_std`, basically those that
need no changes beyond simply adding the attribute. The nontrivial parts
introduced in this PR are:
* CI is introduced to verify that a subset of crates can indeed be
built on a `no_std` target. The target selected is
`x86_64-unknown-none` which is known to not have `std` and will result
in a build error if it's attempted to be used.
* The `anyhow` crate, which `wasmtime-jit-icache-coherence` now depends
on, has its `std` feature disabled by default in Wasmtime's workspace.
This means that some crates which require `std` now need to explicitly
enable the feature, but it means that by-default its usage is
appropriate for `no_std`.
The first point should provide CI checks that compatibility with
`no_std` indeed works, at least from an "it compiles" perspective. Note
that it's not sufficient to test with a target like
`x86_64-unknown-linux-gnu` because `extern crate std` will work on that
target, even when `#![no_std]` is active.
The second point however is likely to increase maintenance burden
in Wasmtime unfortunately. Namely we'll inevitably, either here or in
the future, forget to turn on some feature for some crate that's not
covered in CI checks. While I've tried to do my best here in covering it
there's no guarantee that everything will work and the combinatorial
explosion of what could be checked in CI can't all be added to CI.
Instead we'll have to rely on bug fixes, users, and perhaps point
releases to add more use cases to CI over time as we see fit.
* Add another std feature
* Another std feature
* Enable anyhow/std for another crate
* Activate `std` in more crates
* Fix miri build
* Fix compile on riscv64
prtest:full
* Fix min-platform example build
* Fix icache-coherence again
Wasmtime and Cranelift have a few miscellaenous use cases for "just take
this Rust type and make it bytes", for example Wasmtime's serialization
of internal metadata into a compiled module. Previously Wasmtime used
the `bincode` crate for performing these tasks as the format was
generally optimized to be small and fast, not general purpose (e.g.
JSON). The `bincode` crate on crates.io doesn't work on `no_std`,
however, and with the work in #8341 that's an issue now for Wasmtime.
This crate switches instead to the `postcard` crate. This crate is
listed in Serde's documentation as:
> Postcard, a no_std and embedded-systems friendly compact binary
> format.
While I've not personally used it before it checks all the boxes we
relied on `bincode` for and additionally works with `no_std`. After
auditing the crate this commit then switches out Wasmtime's usage of
`bincode` for `postcard` throughout the repository.
* Allow sleep form of poll_oneoff for wasmtime-wasi
Allow sleep form of poll_oneoff for wasmtime-wasi crate, which just
sleeps for relative time using sleep from the std::thread.
This patch derives from the commit "Support sleep forms of poll_oneoff"
6b40724d18. Applying this patch would
help improve the real-time performance of wasmtime.
Closes#8428
Signed-off-by: Daichi Fukui <daichi1.fukui@toshiba.co.jp>
* Run cargo fmt
---------
Signed-off-by: Daichi Fukui <daichi1.fukui@toshiba.co.jp>
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
Alex Crichton
Nick Fitzgerald
FrankReh
Lann
Chris Fallin
dundargoc
Trevor Elliott
wasmtime-publish
Jamey Sharp
Andrew Brown
iximeow
Saúl Cabrera
jianjunz
Brian
daichifukui