* Make the Wasmtime CLI use async
This means that interrupting a running Wasm program will now work
correctly, even when the program is blocked on I/O or waiting on a timeout or
some such.
This also involved making `wasi-threads` async-compatible.
Co-Authored-By: Alex Crichton <alex@alexcrichton.com>
* rustfmt
* Make `run` command enable the `tokio` feature
* Add a test for CLI, timeouts, and sleeping forever
* Fix warning
---------
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
This allows, for example, getting `trace!` logs from Cranelift out of
`wasmtime-cli` tests, rather than needing to modify `cranelift-codegen`'s
default features or whatever.
* Bump wasm-tools dependencies
This commit updates the wasm-tools dependencies to their 216 track of
versions. This will be followed-up with some updates to how some
features are managed.
* Update vets and depend on crates.io
* Cranelift: Add a new backend for emitting Pulley bytecode
This commit adds two new backends for Cranelift that emits 32- and 64-bit Pulley
bytecode. The backends are both actually the same, with a common implementation
living in `cranelift/codegen/src/isa/pulley_shared`. Each backend configures an
ISA flag that determines the pointer size, and lowering inspects this flag's
value when lowering memory accesses.
To avoid multiple ISLE compilation units, and to avoid compiling duplicate
copies of Pulley's generated `MInst`, I couldn't use `MInst` as the `MachInst`
implementation directly. Instead, there is an `InstAndKind` type that is a
newtype over the generated `MInst` but which also carries a phantom type
parameter that implements the `PulleyTargetKind` trait. There are two
implementations of this trait, a 32- and 64-bit version. This is necessary
because there are various static trait methods for the mach backend which we
must implement, and which return the pointer width, but don't have access to any
`self`. Therefore, we are forced to monomorphize some amount of code. This type
parameter is fairly infectious, and all the "big" backend
types (`PulleyBackend<P>`, `PulleyABICallSite<P>`, etc...) are parameterized
over it. Nonetheless, not everything is parameterized over a `PulleyTargetKind`,
and we manage to avoid duplicate `MInst` definitions and lowering code.
Note that many methods are still stubbed out with `todo!`s. It is expected that
we will fill in those implementations as the work on Pulley progresses.
* Trust the `pulley-interpreter` crate, as it is part of our workspace
* fix some clippy warnings
* Fix a dead-code warning from inside generated code
* Use a helper for emitting br_if+comparison instructions
* Add a helper for converting `Reg` to `pulley_interpreter::XReg`
* Add version to pulley workspace dependency
* search the pulley directory for crates in the publish script
* Use cmake to build wasmtime-c-api
* Properly expose features when building via cmake
* Install all headers to same directory
* Add vets
* attempt to fix ci
* Run all tests on CI
prtest:full
* Set CARGO_BUILD_TARGET; add CMakeLists to package
* Update comment on github action
* Attempt to fix android build
* Fix source dir modifications of c-api build
* Re-add BINARY_DIR option
* Fix build
* Move header installation to a cmake script
Try to avoid dealing with cmake configuration/platforms/etc.
* Tweak build of headers
* Install headers in build dir for examples
* Add cmake files to dist, fix header install dir
---------
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
Also set their default levels to `warn` instead of `deny` to assist with
using clippy locally to avoid compilation failures and collecting as
many as possible in one run. Note that CI will still deny-by-default
through `-Dwarnings`.
* Derive `Copy` for `Val`
* Fix `clippy::clone_on_copy` for the whole repo
* Enforce `clippy::clone_on_copy` for the workspace
* fix some more clippy::clone_on_copy that got missed
* Update gimli to 0.29.0
Gimli 0.29.0 includes a fix for corrupt .eh_frame unwind tables being
generated in certain cases. Wasmtime isn't affected by this as it never
sets an LSDA, but cg_clif will need this for supporting unwinding from
exceptions.
I also had to update a couple of other crates to prevent crates being
included in multiple versions.
* Add vets for new dependencies
---------
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
* Introduce the `pulley-interpreter` crate
This commit is the first step towards implementing
https://github.com/bytecodealliance/rfcs/pull/35
This commit introduces the `pulley-interpreter` crate which contains the Pulley
bytecode definition, encoder, decoder, disassembler, and interpreter.
This is still very much a work in progress! It is expected that we will tweak
encodings and bytecode definitions, that we will overhaul the interpreter (to,
for example, optionally support the unstable Rust `explicit_tail_calls`
feature), and otherwise make large changes. This is just a starting point to get
the ball rolling.
Subsequent commits and pull requests will do things like add the Cranelift
backend to produce Pulley bytecode from Wasm as well as the runtime integration
to run the Pulley interpreter inside Wasmtime.
* remove stray fn main
* Add small tests for special x registers
* Remove now-unused import
* always generate 0 pc rel offsets in arbitrary
* Add doc_auto_cfg feature for docs.rs
* enable all optional features for docs.rs
* Consolidate `BytecodeStream::{advance,get1,get2,...}` into `BytecodeStream::read`
* fix fuzz targets build
* inherit workspace lints in pulley's fuzz crate
* Merge fuzz targets into one target; fix a couple small fuzz bugs
* Add Pulley to our cargo vet config
* Add pulley as a crate to publish
* Move Pulley fuzz target into top level fuzz directory
* Bump MSRV to 1.78.0
This commit updates to use Rust 1.80 in CI as well as updating the
minimum-supported-rust-version to 1.78. Additionally the nightly used in
testing in CI is updated as well.
prtest:full
* Fix compat with latest nightly in tests
* Fix some more nightly warnings
* Ignore new tests on MIRI
* clif in wasmtime explore
* clif in wasmtime explore: do not continue if clif directory exists
* clif in explorer: run prettier
* explorer: use flex instead of width
* explorer: fix scrolling
* explorer: use temp directory for clif
* explorer: tempfile is an optional dependency
* clif in explorer: do not display clif if compiling with winch
* Update wasm-tools and wit-bindgen crates
This keeps things updated and pulls in some new features from
wasm-tools:
* Multiple returns in the component model are now gated by default. This
is reflected in a new `Config` option and CLI flag. The hope is to
remove this feature is no one ends up needing it.
* Support for more `shared` things were added but the feature is always
disabled so internal handlers just panic.
Tests were updated to not use multiple returns where appropriate, except
for one test which is specifically testing for multiple returns and how
it's reflected into the Rust type system.
* Fix fuzz build
* Update the wasm-tools family of crates
This notably brings in a limitation where component model flags types
must have 32 or fewer flags in accordance with the transition plan of
https://github.com/WebAssembly/component-model/issues/370. A feature
flag is added to go back to the previous behavior to avoid breaking
anyone too much.
This additionally brings in a fix for a panic when validating invalid
modules with tail calls.
* Add vet entries
* upgrade to wasm-tools 0.211.1
* code review
* cargo vet: auto imports
* fuzzing: fix wasm-smith changes
* fuzzing: changes for HeapType
* Configure features on `Parser` when parsing
---------
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
* wasmtime: Add profile markers around host-calls
The output of the guest profiler can be misleading around hostcalls.
Whatever happened to be the last sample before the hostcall appears to
run for the entire time of the hostcall. This change ensures that we can
see the actual call stack at the time of the hostcall, and get a visual
indication of which periods are not spent executing guest code.
* wasmtime-cli needs wasmtime/call-hook, but wasmtime itself doesn't
In general, embedders that wish to use the new functionality likely will
need to enable the wasmtime/call-hook feature in order to get Wasmtime
to notify them of when to call into the profiler. However embedders
could consider other alternatives, such as calling the profiler from
selected hostcall implementations.
* Implement semver compatibility for exports
This commit is an implementation of component model semver compatibility
for export lookups. Previously in #7994 component imports were made
semver-aware to ensure that bumping version numbers would not be a
breaking change. This commit implements the same feature for component
exports. This required some refactoring to move the definition of semver
compat around and the previous refactoring in #8786 enables frontloading
this work to happen before instantiation.
Closes#8395
* Review comments
* Fix tests
* Introduce the `cranelift-bitset` crate
The eventual goal is to deduplicate bitset types between Cranelift and Wasmtime,
especially their use in stack maps.
* Use the `cranelift-bitset` crate inside both Cranelift and Wasmtime
Mostly for stack maps, also for a variety of other random things where
`cranelift_codegen::bitset::BitSet` was previously used.
* Fix stack maps unit test in cranelift-codegen
* Uncomment `no_std` declaration
* Fix `CompountBitSet::reserve` method
* Fix `CompoundBitSet::insert` method
* Keep track of the max in a `CompoundBitSet`
Makes a bunch of other stuff easier, and will be needed for replacing
`cranelift_entity::EntitySet`'s bitset with this thing anyways.
* Add missing parens
* Fix a bug around insert and reserve
* Implement `with_capacity` in terms of `new` and `reserve`
* Rename `reserve` to `ensure_capacity`
* wasi-adapter: Implement provider crate that embeds the adapter binaries
* Upgrade wasi adapters to the latest version
* Update adapter docs
* Recompile asi adapters with 1.78
* Recompile wasi adapters with 1.79
* Add some debugging to adapter build script
* Fix script debugging
* Compute wasi adapter version based on latest adapter commit hash
* Try to bless wasi adapters again
* Try to work around CI auto-merges
* Revert to just using workspace version
* Add the wasi adapter provider to the crate publication list
* Use wasi adapter provider in artifacts test + explicit MSRV in CI
* Explicit adapter crate version
* Small fix
* Remove version info from adapter metadata
* Check but don't install rust toolchain in build script
* Bless after rebase
---------
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
* wasi-nn: remove some unncecessary panics from test programs
* Make `libtest-mimic` a workspace dependency
* wasi-nn: use \`libtest-mimic\` for testing
wasi-nn's testing story is complicated by different levels of support on
different platforms (some backends work on certain architectures, others
only work on certain OSes, etc.). This change migrates the `testing`
module, which was included in `src`, to exist solely under `tests`. It
also dynamically checks whether each test is runnable and then chooses
whether to ignore it with a `libtest-mimic` flag. This ensures we can
see all the tests all the time and whether they are running or not,
which is helpful during development.
* Refactor for more subtle `ignore` behavior
On any development machine, with no prior setup, we should be able to
compile and move past the ignored tests without issue:
```console
$ cargo test -- --quiet
running 4 tests
iiii
```
With the proper setup and enabling the right features, tests that are
able to run should do so (eliding a bunch of test output):
```console
$ cargo test --all-features -- --quiet
running 4 tests
iii.
```
On CI, tests that _should_ pass will fail if they can't run:
```console
$ CI=1 cargo test --all-features -- --quiet
iFF.
```
prtest:full
* Add missing `use`
* fix: share download lock between checks
* fix: typo, winml usedx preloaded model
* fix: revert to previous winml behavior
This test was reusing the ONNX test for some reason.
* fix: fully qualify bail!
* Disable memory protection keys by default at compile time
This commit gates memory protection keys behind a new Cargo feature
which is disabled by default. Memory protection keys are already
disabled by default on all platforms and are only configured to possibly
work with Linux x64. When enabled, however, it unconditionally adds a
small amount of overhead to WebAssembly entries/exits even if the
feature is disabled at runtime for the same reason that the `call-hook`
feature adds overhead. With `call-hook` being disabled by default
in #8808 it seemed reasonable to additionally gate memory protection
keys to avoid needing to disable features in Wasmtime to get the best
performance wasm<->host calls.
* Enable Wasmtime feature for fuzzing
* Disable `call-hook` crate feature by default
This commit disables the `call-hook` feature for the Wasmtime crate
added in #8795 by default. The rationale is that this has a slight cost
to all embeddings even if the feature isn't used and it's not expected
to be that widely used of a feature, so off-by-default seems like a more
appropriate default.
* Enable all features in doc build
* More doc fixes
* Migrate usage of the `memoffset` crate to `core::mem::offset_of!`
* C-string literals such as `c"foo"` are now possible
* Some helpers for fixed-length slices such as `slice::first_chunk_mut`
are starting to stabilize.
This accompanies today's release of Rust 1.79.0. This means that CI will
now use the latest stable of 1.79.0 for testing primarily. Additionally
this updates the pinned nightly commit used for testing to today's
nightly and fixes a few warnings that uncovered. I plan on having a
follow-up that leverages some new APIs and features in 1.77.0.
This commit updates the wit-bindgen family of crates and additionally
adds a `features` key to the `bindgen!` macro. This brings it in line
with `wit-bindgen`'s support which enables usage of the new `@since` and
`@unstable` features of WIT.
* Enable rustc's `unused-lifetimes` lint
This is allow-by-default doesn't seem to have any false positives in
Wasmtime's codebase so enable it by default to help clean up vestiges of
old refactorings.
* Remove another unused lifetime
* Remove another unused lifetime
* wasmtime: Introduce the `test-macros` crate
This commit introduces the `test-macros` crate. The whole idea behind this crate is to export a single or multiple macros to make it easier to configure Wasmtime for integration tests. The main use-case at this time, is running a subset of the integration tests with Cranelift and Winch. This crate could be extended to serve other use-cases, like testing pooling allocator and/or GC configurations.
This commit introduces a single example of how this macro could be used. If there's agreement on merging this change in some shape or form, I'll follow up with migrating the current tests to use `#[wasmtime_test]` where applicable.
Part of what's implemented in this PR was discussed in Cranelift's meeting on [April 24th, 2024](https://github.com/bytecodealliance/meetings/blob/main/cranelift/2024/cranelift-04-24.md), however there are several discussion points that are still "in the air", like for example, what's the best way to avoid the combinatorial explosion problem for the potential test matrix.
* Add crate license
* Clippy fixes
* Remove test-macros from members
* Clean up test-macros Cargo.toml
- Fix version
- Add `[lints]`
- Add publish key
* Add `TestConfig` and simpify parsing
This commit adds a `TestConfig` struct that holds the supported Wasmtime
test configuration. Additonally this commit introduces a partial
function parser, in which only the attributes, visibility, and signature
are fully parsed, leaving the function body as an opaque `TokenStream`
This commit removes the support in the `wasmtime` CLI for old CLI
options which were present in Wasmtime 13 and prior. This compatibility
was added in #7385 and backported to the Wasmtime 14 release #7395.
Wasmtime 14.0.0, which did not have this compatibility shim, was
released on 2023-10-20. Wasmtime 14.0.3, which restored compatibility
with this shim, was released on 2023-10-30. This means that Wasmtime
since 2023-10-30 has been warning users about differences in the old and
new CLI.
This commit will be released with Wasmtime 22 which will means that
users will have had an 8-month transition window for warnings to
migrate. The hope is that this is sufficient but it's also not too too
burdensome to carry for longer if necessary.
This was pioneered in bytecodealliance/wasm-tools#1562 and looks to be
working well so I've copied over similar metadata for Wasmtime. I've
seen the `cargo binstall` method of installing binaries to be somewhat
common and we've already got all the necessary binaries so it seemed
nice to add support. Wasmtime doesn't name the artifacts after Rust
target names so some manual configuration is required here, but
otherwise this hopefully isn't too costly to maintain.
* Update wit-bindgen
This commit updates wit-bindgen to 0.25 and applies some "extra
trickery" to work around the now-default providing of the realloc
symbol.
* Add audits
Nick Fitzgerald
Alex Crichton
bjorn3
Ryan Patterson
wasmtime-publish
Hamir Mahal
Xinzhao Xu
Vulcain
Pat Hickey
Jamey Sharp
Juniper Tyree
Andrew Brown
Philip Craig
Saúl Cabrera