* Gate support for the wasm `threads` proposal behind a Cargo feature
This commit moves support for the `threads` proposal behind a new
on-by-default Cargo feature: `threads`. This is intended to support
building Wasmtime with fewer runtime dependencies such as those required
for the atomic operations on memories.
This additionally adds the `gc` feature in a few missing places too.
* Fix compile of C API without threads
* Rename `-S common` to `-S cli`.
The "common" in `-S common` came from "wasi-common" which came from the
idea of having code in common between Wasmtime, Lucet, and others. It
doesn't have a clear meaning for end users, and has a risk of being
interpreted as "common" functionality that's generally available
everywhere.
This PR renames `-S common` to `-S cli`, and documents it as including
the WASI CLI APIs, to clarify its purpose `-S common` is still accepted,
with a warning.
* Fix formatting in RELEASES.md.
* Disable the warning about `-S common` for now.
* Support hex numbers in CLI arguments
This enables `-O static-memory-maximum-size=0x10`, for example.
Previously this was a parse error.
* Discover `disas` tests in subdirectories
* Deny unknown fields when parsing test configs
Make sure we don't accidentally have other configuration. Useful when
porting tests over.
* Update `make-load-store-tests.sh` to new syntax
Pass various flags and such.
* Move all `*.wat` tests from `cranelift/filetests` into `tests/disas`
* Migrate all load/store tests
* Migrate all other `*.wat` tests to `tests/disas`
* Use opt-level=0 with table instructions
* Add CLI flags for instance limits
While experimenting with `wasmtime serve`, I found it helpful to be able
to configure some of the maximum limits of the various objects stored in
the pooling allocator. This change surfaces some new `-O total-*` flags
for controlling these limits from the command line.
* review: prepend flags with `pooling_*`
* Wasmtime: Add a `gc` cargo feature
This controls whether support for `ExternRef` and its associated deferred,
reference-counting garbage collector is enabled at compile time or not. It will
also be used for similarly for Wasmtime's full Wasm GC support as that gets
added.
* Add CI for `gc` Cargo feature
* Cut down on the number of `#[cfg(feature = "gc")]`s outside the implementation of `[VM]ExternRef`
* Fix wasmparser reference types configuration with GC disabled/enabled
* More config fix
* doc cfg
* Make the dummy `VMExternRefActivationsTable` inhabited
* Fix winch tests
* final review bits
* Enable wasmtime's gc cargo feature for the C API
* Enable wasmtime's gc cargo feature from wasmtime-cli-flags
* enable gc cargo feature in a couple other crates
* mpk: allow configuring MPK from the command line
This allows the following usage:
```console
$ wasmtime --pooling-allocator=y --memory-protection-keys=y ...
```
* mpk: add CLI test for failure
* review: move `bail!` outside the pooling allocator block
* fix: use absolute dependency path
* Add options to WasiCtx for toggling TCP and UDP on and off
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Address PR feedback
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Add test for turning off tcp
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
---------
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Configure Rust lints at the workspace level
This commit adds necessary configuration knobs to have lints configured
at the workspace level in Wasmtime rather than the crate level. This
uses a feature of Cargo first released with 1.74.0 (last week) of the
`[workspace.lints]` table. This should help create a more consistent set
of lints applied across all crates in our workspace in addition to
possibly running select clippy lints on CI as well.
* Move `unused_extern_crates` to the workspace level
This commit configures a `deny` lint level for the
`unused_extern_crates` lint to the workspace level rather than the
previous configuration at the individual crate level.
* Move `trivial_numeric_casts` to workspace level
* Change workspace lint levels to `warn`
CI will ensure that these don't get checked into the codebase and
otherwise provide fewer speed bumps for in-process development.
* Move `unstable_features` lint to workspace level
* Move `unused_import_braces` lint to workspace level
* Start running Clippy on CI
This commit configures our CI to run `cargo clippy --workspace` for all
merged PRs. Historically this hasn't been all the feasible due to the
amount of configuration required to control the number of warnings on
CI, but with Cargo's new `[lint]` table it's possible to have a
one-liner to silence all lints from Clippy by default. This commit by
default sets the `all` lint in Clippy to `allow` to by-default disable
warnings from Clippy. The goal of this PR is to enable selective access
to Clippy lints for Wasmtime on CI.
* Selectively enable `clippy::cast_sign_loss`
This would have fixed#7558 so try to head off future issues with that
by warning against this situation in a few crates. This lint is still
quite noisy though for Cranelift for example so it's not worthwhile at
this time to enable it for the whole workspace.
* Fix CI error
prtest:full
This commit implements the `wasi_unstable` module, sometimes referred to
as "preview0", in the `wasmtime-wasi` crate. Previously this was only
implemented by the `wasi-common` crate but now this is implemented for
both meaning that the switch to preview2 won't lose this functionality.
The preview0 WITX files are vendored like the preview1 files and the
implementation of preview0 is exclusively implemented by delegating to
the preview1 implementation.
In #7239 we added a `tracing-log` subscriber that prints logs to stderr
if enabled with an environment variable. It included logic to add ANSI
color sequences when stderr is a terminal, for legibility. Unfortunately
it seems that while this logic *enabled* colors on a terminal, it did
not *disable* colors on a non-terminal; so redirects of stderr to a file
would result in ANSI color sequences being captured in that file.
Specifically, the builder seems not to default to no-color; so rather
than enable-or-nothing, we should explicitly enable or disable always.
Fixes#7435.
This commit fixes logging under `-D log-to-files` mode to not panic when
logging happens in WASI-spawned thread which won't have initialization
for the logger invoked already.
Closes#7418
* Add compatibility shims for Wasmtime 13 CLI
This commit introduces a compatibility shim for the Wasmtime 13 CLI and
prior. The goal of this commit is to address concerns raised in #7336
and other locations as well. While the new CLI cannot be un-shipped at
this point this PR attempts to ameliorate the situation somewhat through
a few avenues:
* A complete copy of the old CLI parser is now included in `wasmtime` by
default.
* The `WASMTIME_NEW_CLI=0` environment variable can force usage of the
old CLI parser for the `run` and `compile` commands.
* The `WASMTIME_NEW_CLI=1` environment variable can force usage of the
new CLI parser.
* Otherwise both the old and the new CLI parser are executed. Depending
on the result one is selected to be executed, possibly with a warning
printed.
* If both CLI parsers succeed but produce the same result, then no
warning is emitted and execution continues as usual.
* If both CLI parsers succeed but produce different results then a
warning is emitted indicating this. The warning points to #7384 which
has further examples of how to squash the warning. The warning also
mentions the above env var to turn the warning off. In this situation
the old semantics are used at this time instead of the new semantics.
It's intended that eventually this will change in the future.
* If the new CLI succeeds and the old CLI fails then the new semantics
are executed without warning.
* If the old CLI succeeds and the new CLI fails then a warning is issued
and the old semantics are used.
* If both the old and the new CLI fail to parse then the error message
for the new CLI is emitted.
Note that this doesn't add up to a perfect transition. The hope is that
most of the major cases of change at the very least have something
printed. My plan is to land this on `main` and then backport it to the
14.0.0 branch as a 14.0.3 release.
* Wordsmith messages
* Update messages
* More wording updates
* Fix grammar
* More updates
* Move `wasmtime explore` behind a Cargo feature
Enable this Cargo feature by default, but enable building the CLI
without the `explore` subcommand.
* Move the `wast` subcommand behind a Cargo feature
* Move support for `wat` behind a CLI feature
This was already conditional in crates such as `wasmtime` and this makes
it an optional dependency of the CLI as well.
* Move CLI cache support behind a Cargo feature
Additionally refactor `wasmtime-cli-flags` to not unconditionally pull
in caching support by removing its `default` feature and appropriately
enabling it from the CLI.
* Move `rayon` behind an optional feature
* Move `http-body-util` dependency behind `serve` feature
* Add a Cargo feature for compiling out log statements
This sets the static features of `log` and `tracing` to statically
remove all log statements from the binary to cut down on binary size.
* Move logging support behind a Cargo feature
Enables statically removing logging support in addition to the previous
compiling out log statements themselves.
* Move demangling support behind a Cargo feature
* Enable building the CLI without cranelift
Compile out the `compile` subcommand for example.
* Gate all profiling support behind one feature flag
This commit moves vtune/jitdump support behind a single `profiling`
feature flag that additionally includes the guest profiler dependencies
now too.
* Move support for core dumps behind a feature flag
* Move addr2line behind a feature
* Fix rebase
* Document cargo features and a minimal build
* Tidy up the source a bit
* Rename compile-out-logging
* Document disabling logging
* Note the host architecture as well
* Fix tests
* Fix tests
* Mention debuginfo stripping
* Fix CI configuration for checking features
* Fix book tests
* Update lock file after rebase
* Enable coredump feature by default
* PCC: add facts to global values, parse and print them. No verification yet.
Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
* PCC: propagate facts on GV loads and check them.
Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
* PCC: support propagating facts on iteratively-elaborated GVs as well.
Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
* PCC: fix up Wasmtime uses of GVs after refactors to memflags handling.
Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
* PCC: working end-to-end for static memories!
Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
* PCC: add toplevel Wasmtime option `-C enable-pcc=y`.
* Fix filetests build.
* Review feedback, and blessed test updates due to GV legalization changes.
---------
Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
Some things I noticed from #7239 which are very much not critical but I
figure might be nice-to-haves:
* Move the logging configuration to the `wasmtime-cli-flags` crate with
the other logging configuration to keep it in one place.
* Remove `pretty_env_logger` since `tracing-subscriber` probably
supplants it.
* Don't explicitly inherit env vars in tests since that happens
automatically with `Command`.
* Implement the `wasi:sockets/ip-name-lookup` interface
This commit is an initial implementation of the new `ip-name-lookup`
interface from the `wasi-sockets` proposal. The intention is to get a
sketch of what an implementation would look like for the preview2
release later this year. Notable features of this implementation are:
* Name lookups are disabled by default and must be explicitly enabled in
`WasiCtx`. This seems like a reasonable default for now while the full
set of configuration around this is settled.
* I've added new "typed" methods to `preview2::Table` to avoid the need
for extra helpers when using resources.
* A new `-Sallow-ip-name-lookup` option is added to control this on the
CLI.
* Implementation-wise this uses the blocking resolution in the Rust
standard library, built on `getaddrinfo`. This doesn't invoke
`getaddrinfo` "raw", however, so information such as error details can
be lost in translation. This will probably need to change in the
future.
Closes#7070
* Validate the input domain name.
* Use a crate-level helper for `spawn_blocking`
---------
Co-authored-by: Dave Bakker <github@davebakker.io>
This commit makes it so that the library type for core dumps is serializable
into the standard binary format for core dumps.
Additionally, this commit makes it so that we use the library type for
generating core dumps in the CLI. We previously were using a one-off
implementation of core dump generation that only had backtrace information and
no instances, modules, globals, or memories included. The library type has all
that information, so the core dumps produced by our CLI will both be more
featureful and be generated by shared code paths going forward.
Along the way, implementing all this required some new helper methods sprinkled
throughout `wasmtime` and `wasmtime-runtime`:
* `wasmtime::Instance::module`: get the module that a `wasmtime::Instance` is an
instance of. This is public, since it seems generally useful. This involved
adding a new return value from `ModuleRegistry::register_module` that is an
identifier that can be used to recover a reference to the registered module.
* `wasmtime::Instance::all_{globals,memories}`: get the full global/memory index
space. I made these `pub(crate)` out of caution. I don't think we want to commit
to exposing non-exported things in the public API, even if we internally need
them for debugging-related features like core dumps. These also needed
corresponding methods inside `wasmtime-runtime`.
* `wasmtime::{Global,Memory}::hash_key`: this was needed to work around the fact
that each time you call `{Global,Memory}::from_wasmtime`, it creates a new
entry in the `StoreData` and so you can get duplicates. But we need to key some
hash maps on globals and memories when constructing core dumps, so we can't
treat the underlying `Stored<T>` as a hash key because it isn't stable across
duplicate `StoreData` entries. So we have these new methods. They are only
`pub(crate)`, are definitely implementation details, and aren't exposed in the
public API.
* `wasmtime::FrameInfo::module`: Each frame in a backtrace now keeps a handle to
its associated module instead of just the name. This is publicly exposed
because it seems generally useful. This means I also deprecated
`wasmtime::FrameInfo::module_name` since you can now instead do
`frame.module().name()` to get that exact same info. I updated callers inside
the repo.
* Redesign Wasmtime's CLI
This commit follows through on discussion from #6741 to redesign the
flags that the `wasmtime` binary accepts on the CLI. Almost all flags
have been renamed/moved and will require callers to update. The main
motivation here is to cut down on the forest of options in `wasmtime -h`
which are difficult to mentally group together and understand.
The main change implemented here is to move options behind "option
groups" which are intended to be abbreviated with a single letter:
* `-O foo` - an optimization or performance-tuning related option
* `-C foo` - a codegen option affecting the compilation process.
* `-D foo` - a debug-related option
* `-W foo` - a wasm-related option, for example changing wasm semantics
* `-S foo` - a WASI-related option, configuring various proposals for example
Each option group can be explored by passing `help`, for example `-O
help`. This will print all options within the group along with their
help message. Additionally `-O help-long` can be passed to print the
full comment for each option if desired.
Option groups can be specified multiple times on the command line, for
example `-Wrelaxed-simd -Wthreads`. They can also be combined together
with commas as `-Wrelaxed-simd,threads`. Configuration works as a "last
option wins" so `-Ccache,cache=n` would end up with a compilation
cache disabled.
Boolean options can be specified as `-C foo` to enable `foo`, or they
can be specified with `-Cfoo=$val` with any of `y`, `n`, `yes`, `no`,
`true`, or `false`. All other options require a `=foo` value to be
passed and the parsing depends on the type.
This commit additionally applies a few small refactorings to the CLI as
well. For example the help text no longer prints information about wasm
features after printing the option help. This is still available via
`-Whelp` as all wasm features have moved from `--wasm-features` to `-W`.
Additionally flags are no longer conditionally compiled in, but instead
all flags are always supported. A runtime error is returned if support
for a flag is not compiled in. Additionally the "experimental" name of
WASI proposals has been dropped in favor of just the name of the
proposal, for example `--wasi nn` instead of `--wasi-modules
experimental-wasi-nn`. This is intended to mirror how wasm proposals
don't have "experimental" in the name and an opt-in is required
regardless.
A full listing of flags and how they have changed is:
| old cli flag | new cli flag |
|-----------------------------------------------|-------------------------------------------------|
| `-O, --optimize` | removed |
| `--opt-level <LEVEL>` | `-O opt-level=N` |
| `--dynamic-memory-guard-size <SIZE>` | `-O dynamic-memory-guard-size=...` |
| `--static-memory-forced` | `-O static-memory-forced` |
| `--static-memory-guard-size <SIZE>` | `-O static-memory-guard-size=N` |
| `--static-memory-maximum-size <MAXIMUM>` | `-O static-memory-maximum-size=N` |
| `--dynamic-memory-reserved-for-growth <SIZE>` | `-O dynamic-memory-reserved-for-growth=...` |
| `--pooling-allocator` | `-O pooling-allocator` |
| `--disable-memory-init-cow` | `-O memory-init-cow=no` |
| `--compiler <COMPILER>` | `-C compiler=..` |
| `--enable-cranelift-debug-verifier` | `-C cranelift-debug-verifier` |
| `--cranelift-enable <SETTING>` | `-C cranelift-NAME` |
| `--cranelift-set <NAME=VALUE>` | `-C cranelift-NAME=VALUE` |
| `--config <CONFIG_PATH>` | `-C cache-config=..` |
| `--disable-cache` | `-C cache=no` |
| `--disable-parallel-compilation` | `-C parallel-compilation=no` |
| `-g` | `-D debug-info` |
| `--disable-address-map` | `-D address-map=no` |
| `--disable-logging` | `-D logging=no` |
| `--log-to-files` | `-D log-to-files` |
| `--coredump-on-trap <PATH>` | `-D coredump=..` |
| `--wasm-features all` | `-W all-proposals` |
| `--wasm-features -all` | `-W all-proposals=n` |
| `--wasm-features bulk-memory` | `-W bulk-memory` |
| `--wasm-features multi-memory` | `-W multi-memory` |
| `--wasm-features multi-value` | `-W multi-value` |
| `--wasm-features reference-types` | `-W reference-types` |
| `--wasm-features simd` | `-W simd` |
| `--wasm-features tail-call` | `-W tail-call` |
| `--wasm-features threads` | `-W threads` |
| `--wasm-features memory64` | `-W memory64` |
| `--wasm-features copmonent-model` | `-W component-model` |
| `--wasm-features function-references` | `-W function-references` |
| `--relaxed-simd-deterministic` | `-W relaxed-simd-deterministic` |
| `--enable-cranelift-nan-canonicalization` | `-W nan-canonicalization` |
| `--fuel <N>` | `-W fuel=N` |
| `--epoch-interruption` | `-W epoch-interruption` |
| `--allow-unknown-exports` | `-W unknown-exports-allow` |
| `--trap-unknown-imports` | `-W unknown-imports-trap` |
| `--default-values-unknown-imports` | `-W unknown-imports-default` |
| `--max-instances <MAX_INSTANCES>` | `-W max-instances=N` |
| `--max-memories <MAX_MEMORIES>` | `-W max-memories=N` |
| `--max-memory-size <BYTES>` | `-W max-memory-size=N` |
| `--max-table-elements <MAX_TABLE_ELEMENTS>` | `-W max-table-elements=N` |
| `--max-tables <MAX_TABLES>` | `-W max-tables=N` |
| `--max-wasm-stack <MAX_WASM_STACK>` | `-W max-wasm-stack=N` |
| `--trap-on-grow-failure` | `-W trap-on-grow-failure` |
| `--wasm-timeout <TIME>` | `-W timeout=N` |
| `--wmemcheck` | `-W wmemcheck` |
| `--wasi-modules default` | removed |
| `--wasi-modules -default` | removed |
| `--wasi-modules wasi-common` | `-S common` |
| `--wasi-modules -wasi-common` | `-S common=n` |
| `--wasi-modules experimental-wasi-nn` | `-S nn` |
| `--wasi-modules experimental-wasi-threads` | `-S threads` |
| `--wasi-modules experimental-wasi-http` | `-S http` |
| `--listenfd` | `-S listenfd` |
| `--tcplisten <SOCKET ADDRESS>` | `-S tcplisten=...` |
| `--wasi-nn-graph <FORMAT::HOST>` | `-S nn-graph=FORMAT::HOST` |
| `--preview2` | `-S preview2` |
| `--dir <DIRECTORY>` | `--dir ...` |
| `--mapdir <GUEST_DIR::HOST_DIR>` | `--dir a::b` |
* Be more descriptive with help text
* Document `=val` is optional for `-Ccranelift-xxx`
* Fix compile after rebase
* Fix rebase of `--inherit-network`
* Fix wasi-http test
* Fix compile without pooling allocator support
* Update some flags in docs
* Fix bench-api build
* Update flags for gdb/lldb tests
* Fixup optimization flags
prtest:full
Wasmtime's CI does not run clippy so there's no enforcement of this
configuration. Additionally the configuration per-crate is not uniformly
applied across all of the Wasmtime workspace and is only on some
historical crates. Because we don't run clippy in CI this commit removes
all of the clippy annotations for allow/warn/deny from the source.
* Remove the implementation of wasi-crypto
This commit is a follow-up to the discussion on #6732. This removes
Wasmtime's implementation of the wasi-crypto proposal from in-tree along
with its various support in CI, configuration, etc. See the discussion
on #6732 for the full information but at a high level the main reasons
for removing the implementation at this time are:
* There is not currently an active maintainer of the Wasmtime
integration here for wasi-crypto.
* There are known issues with the code quality of the implementation
such as transmutes of guest-owned memory to `&'static mut [u8]` and
known unsafety in dependencies.
* The size and breadth of the dependency tree brings maintenance burden
and overhead to managing Wasmtime's dependency tree.
As mentioned on the issue this commit does not mean that Wasmtime
doesn't want to implement the wasi-crypto proposal. Instead the "tier 3"
status of wasi-crypto needs to be re-attained to be included back
in-tree, which would mean resolving the above issues.
Note that this commit is intentionally just after the 13.0.0 branch
point which means that this is slated for Wasmtime 14 to be released on
September 20.
* Remove some cfgs
* Remove wasi-crypto CI
* Wasmtime: Add support for Wasm tail calls
This adds the `Config::wasm_tail_call` method and `--wasm-features tail-call`
CLI flag to enable the Wasm tail calls proposal in Wasmtime.
This PR is mostly just plumbing and enabling tests, since all the prerequisite
work (Wasmtime trampoline overhauls and Cranelift tail calls) was completed in
earlier pull requests.
When Wasm tail calls are enabled, Wasm code uses the `tail` calling
convention. The `tail` calling convention is known to cause a 1-7% slow down for
regular code that isn't using tail calls, which is why it isn't used
unconditionally. This involved shepherding `Tunables` through to Wasm signature
construction methods. The eventual plan is for the `tail` calling convention to
be used unconditionally, but not until the performance regression is
addressed. This work is tracked in
https://github.com/bytecodealliance/wasmtime/issues/6759
Additionally while our x86-64, aarch64, and riscv64 backends support tail calls,
the s390x backend does not support them yet. Attempts to use tail calls on s390x
will return errors. Support for s390x is tracked in
https://github.com/bytecodealliance/wasmtime/issues/6530
* Store `Tunables` inside the `Compiler`
Instead of passing as an argument to every `Compiler` method.
* Cranelift: Support "direct" return calls on riscv64
They still use `jalr` instead of `jal` but this allows us to use the `RiscvCall`
reloc, which Wasmtime handles. Before we were using `LoadExternalName` which
produces an `Abs8` reloc, which Wasmtime intentionally does not handle since
that involves patching code at runtime, which makes loading code slower.
* Fix tests that assume tail call support on s390x
* Remove deny.toml exception for wasm-coredump-builder
This isn't used any more so no need to continue to list this.
* Update Wasmtime's pretty_env_logger dependency
This removes a `deny.toml` exception for that crate, but `openvino-sys`
still depends on `pretty_env_logger 0.4.0` so a new exception is added
for that.
* Update criterion and clap dependencies
This commit started out by updating the `criterion` dependency to remove
an entry in `deny.toml`, but that ended up transitively requiring a
`clap` dependency upgrade from 3.x to 4.x because `criterion` uses
pieces of clap 4.x. Most of this commit is then dedicated to updating
clap 3.x to 4.x which was relatively simple, mostly renaming attributes
here and there.
* Update gimli-related dependencies
I originally wanted to remove the `indexmap` clause in `deny.toml` but
enough dependencies haven't updated from 1.9 to 2.0 that it wasn't
possible. In the meantime though this updates some various dependencies
to bring them to the latest and a few of them now use `indexmap` 2.0.
* Update deps to remove `windows-sys 0.45.0`
This involved updating tokio/mio and then providing new audits for new
crates. The tokio exemption was updated from its old version to the new
version and tokio remains un-audited.
* Update `syn` to 2.x.x
This required a bit of rewriting for the component-macro related bits
but otherwise was pretty straightforward. The `syn` 1.x.x track is still
present in the wasi-crypto tree at this time.
I've additionally added some trusted audits for my own publications of
`wasm-bindgen`
* Update bitflags to 2.x.x
This updates Wasmtime's dependency on the `bitflags` crate to the 2.x.x
track to keep it up-to-date.
* Update the cap-std family of crates
This bumps them all to the next major version to keep up with updates.
I've additionally added trusted entries for publishes of cap-std crates
from Dan.
There's still lingering references to rustix 0.37.x which will need to
get weeded out over time.
* Update memoffset dependency to latest
Avoids having two versions in our crate graph.
* Fix tests
* Update try_from for wiggle flags
* Fix build on AArch64 Linux
* Enable `event` for rustix on Windows too
* Upgrade file-per-thread-logger to v0.2.0
Signed-off-by: Benjamin Bouvier <public@benj.me>
* Update audits too
Signed-off-by: Benjamin Bouvier <public@benj.me>
---------
Signed-off-by: Benjamin Bouvier <public@benj.me>
* Make wasmtime-types type check
* Make wasmtime-environ type check.
* Make wasmtime-runtime type check
* Make cranelift-wasm type check
* Make wasmtime-cranelift type check
* Make wasmtime type check
* Make wasmtime-wast type check
* Make testsuite compile
* Address Luna's comments
* Restore compatibility with effect-handlers/wasm-tools#func-ref-2
* Add function refs feature flag; support testing
* Provide function references support in helpers
- Always support Index in blocktypes
- Support Index as table type by pretending to be Func
- Etc
* Implement ref.as_non_null
* Add br_on_null
* Update Cargo.lock to use wasm-tools with peek
This will ultimately be reverted when we refer to
wasm-tools#function-references, which doesn't have peek, but does have type
annotations on CallRef
* Add call_ref
* Support typed function references in ref.null
* Implement br_on_non_null
* Remove extraneous flag; default func refs false
* Use IndirectCallToNull trap code for call_ref
* Factor common call_indirect / call_ref into a fn
* Remove copypasta clippy attribute / format
* Add a some more tests for typed table instructions
There certainly need to be many more, but this at least catches the bugs fixed
in the next commit
* Fix missing typed cases for table_grow, table_fill
* Document trap code; remove answered question
* Mark wasm-tools to wasmtime reftype infallible
* Fix reversed conditional
* Scope externref/funcref shorthands within WasmRefType
* Merge with upstream
* Make wasmtime compile again
* Fix warnings
* Remove Bot from the type algebra
* Fix table tests.
`wast::Cranelift::spec::function_references::table`
`wast::Cranelift::spec::function_references::table_pooling`
* Fix table{get,set} tests.
```
wast::Cranelift::misc::function_references::table_get
wast::Cranelift::misc::function_references::table_get_pooling
wast::Cranelift::misc::function_references::table_set
wast::Cranelift::misc::function_references::table_set_pooling
```
* Insert subtype check to fix local_get tests.
```
wast::Cranelift::spec::function_references::local_get
wast::Cranelift::spec::function_references::local_get_pooling
```
* Fix compilation of `br_on_non_null`.
The branch destinations were the other way round... :-)
Fixes the following test failures:
```
wast::Cranelift::spec::function_references::br_on_non_null
wast::Cranelift::spec::function_references::br_on_non_null_pooling
```
* Fix ref_as_non_null tests.
The test was failing due to the wrong error message being printed. As
per upstream folks' suggest we were using the trap code
`IndirectCallToNull`, but it produces an unexpected error message.
This commit reinstates the `NullReference` trap code. It produces the
expected error message. We will have to chat with the maintainers
upstream about how to handle these "test failures".
Fixes the following test failures:
```
wast::Cranelift::spec::function_references::ref_as_non_null
wast::Cranelift::spec::function_references::ref_as_non_null_pooling
```
* Fix a call_ref regression.
* Fix global tests.
Extend `is_matching_assert_invalid_error_message` to circumvent the textual error message failure.
Fixes the following test failures:
```
wast::Cranelift::spec::function_references::global
wast::Cranelift::spec::function_references::global_pooling
```
* Cargo update
* Update
* Spell out some cases in match_val
* Disgusting hack to subvert limitations of type reconstruction.
In the function `wasmtime::values::Val::ty()` attempts to reconstruct
the type of its underlying value purely based on the shape of the
value. With function references proposal this sort of reconstruction
is no longer complete as a source reference type may have been
nullable. Nullability is not inferrable by looking at the shape of the
runtime object alone.
Consequently, the runtime cannot reconstruct the type for
`Val::FuncRef` and `Val::ExternRef` by looking at their respective
shapes.
* Address workflows comments.
* null reference => null_reference for CLIF parsing compliance.
* Delete duplicate-loads-dynamic-memory-egraph (again)
* Idiomatic code change.
* Nullability subtyping + fix non-null storage check.
This commit removes the `hacky_eq` check in `func.rs`. Instead it is
replaced by a subtype check. This subtype check occurs in
`externals.rs` too.
This commit also fixes a bug. Previously, it was possible to store a
null reference into a non-null table cell. I have added to new test
cases for this bug: one for funcrefs and another for externrefs.
* Trigger unimplemented for typed function references. Format values.rs
* run cargo fmt
* Explicitly match on HeapType::Extern.
* Address cranelift-related feedback
* Remove PartialEq,Eq from ValType, RefType, HeapType.
* Pin wasmparser to a fairly recent commit.
* Run cargo fmt
* Ignore tail call tests.
* Remove garbage
* Revert changes to wasmtime public API.
* Run cargo fmt
* Get more CI passing (#19)
* Undo Cargo.lock changes
* Fix build of cranelift tests
* Implement link-time matches relation. Disable tests failing due to lack of public API support.
* Run cargo fmt
* Run cargo fmt
* Initial implementation of eager table initialization
* Tidy up eager table initialisation
* Cargo fmt
* Ignore type-equivalence test
* Replace TODOs with descriptive comments.
* Various changes found during review (#21)
* Clarify a comment
This isn't only used for null references
* Resolve a TODO in local init
Don't initialize non-nullable locals to null, instead skip
initialization entirely and wasm validation will ensure it's always
initialized in the scope where it's used.
* Clarify a comment and skipping the null check.
* Remove a stray comment
* Change representation of `WasmHeapType`
Use a `SignatureIndex` instead of a `u32` which while not 100% correct
should be more correct. This additionally renames the `Index` variant to
`TypedFunc` to leave space for future types which aren't functions to
not all go into an `Index` variant.
This required updates to Winch because `wasmtime_environ` types can no
longer be converted back to their `wasmparser` equivalents. Additionally
this means that all type translation needs to go through some form of
context to resolve indices which is now encapsulated in a `TypeConvert`
trait implemented in various locations.
* Refactor table initialization
Reduce some duplication and simplify some data structures to have a more
direct form of table initialization and a bit more graceful handling of
element-initialized tables. Additionally element-initialize tables are
now treated the same as if there's a large element segment initializing
them.
* Clean up some unrelated chagnes
* Simplify Table bindings slightly
* Remove a no-longer-needed TODO
* Add a FIXME for `SignatureIndex` in `WasmHeapType`
* Add a FIXME for panicking on exposing function-references types
* Fix a warning on nightly
* Fix tests for winch and cranelift
* Cargo fmt
* Fix arity mismatch in aarch64/abi
---------
Co-authored-by: Daniel Hillerström <daniel.hillerstrom@ed.ac.uk>
Co-authored-by: Daniel Hillerström <daniel.hillerstrom@huawei.com>
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
* Fold guest profiling flags into `--profile` CLI option
This commit removes the `--profile-guest` and `--profile-guest-interval`
CLI flags and folds them into the preexisting `--profile` flag.
Specifying guest profiling can now be done with `--profile guest` which
has a default path/interval to write to. The path/interval can be
configured with `--profile guest,path.json` and `--profile
guest,path.json,10ms`.
* Update src/commands/run.rs
Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
---------
Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
* Integrate experimental HTTP into wasmtime.
* Reset Cargo.lock
* Switch to bail!, plumb options partially.
* Implement timeouts.
* Remove generated files & wasm, add Makefile
* Remove generated code textfile
* Update crates/wasi-http/Cargo.toml
Co-authored-by: Eduardo de Moura Rodrigues <16357187+eduardomourar@users.noreply.github.com>
* Update crates/wasi-http/Cargo.toml
Co-authored-by: Eduardo de Moura Rodrigues <16357187+eduardomourar@users.noreply.github.com>
* Extract streams from request/response.
* Fix read for len < buffer length.
* Formatting.
* types impl: swap todos for traps
* streams_impl: idioms, and swap todos for traps
* component impl: idioms, swap all unwraps for traps, swap all todos for traps
* http impl: idiom
* Remove an unnecessary mut.
* Remove an unsupported function.
* Switch to the tokio runtime for the HTTP request.
* Add a rust example.
* Update to latest wit definition
* Remove example code.
* wip: start writing a http test...
* finish writing the outbound request example
havent executed it yet
* better debug output
* wasi-http: some stubs required for rust rewrite of the example
* add wasi_http tests to test-programs
* CI: run the http tests
* Fix some warnings.
* bump new deps to latest releases (#3)
* Add tests for wasi-http to test-programs (#2)
* wip: start writing a http test...
* finish writing the outbound request example
havent executed it yet
* better debug output
* wasi-http: some stubs required for rust rewrite of the example
* add wasi_http tests to test-programs
* CI: run the http tests
* bump new deps to latest releases
h2 0.3.16
http 0.2.9
mio 0.8.6
openssl 0.10.48
openssl-sys 0.9.83
tokio 1.26.0
---------
Co-authored-by: Brendan Burns <bburns@microsoft.com>
* Update crates/test-programs/tests/http_tests/runtime/wasi_http_tests.rs
* Update crates/test-programs/tests/http_tests/runtime/wasi_http_tests.rs
* Update crates/test-programs/tests/http_tests/runtime/wasi_http_tests.rs
* wasi-http: fix cargo.toml file and publish script to work together (#4)
unfortunately, the publish script doesn't use a proper toml parser (in
order to not have any dependencies), so the whitespace has to be the
trivial expected case.
then, add wasi-http to the list of crates to publish.
* Update crates/test-programs/build.rs
* Switch to rustls
* Cleanups.
* Merge switch to rustls.
* Formatting
* Remove libssl install
* Fix tests.
* Rename wasi-http -> wasmtime-wasi-http
* prtest:full
Conditionalize TLS on riscv64gc.
* prtest:full
Fix formatting, also disable tls on s390x
* prtest:full
Add a path parameter to wit-bindgen, remove symlink.
* prtest:full
Fix tests for places where SSL isn't supported.
* Update crates/wasi-http/Cargo.toml
---------
Co-authored-by: Eduardo de Moura Rodrigues <16357187+eduardomourar@users.noreply.github.com>
Co-authored-by: Pat Hickey <phickey@fastly.com>
Co-authored-by: Pat Hickey <pat@moreproductive.org>
* Adding in trampoline compiling method for ISA
* Adding support for indirect call to memory address
* Refactoring frame to externalize defined locals, so it removes WASM depedencies in trampoline case
* Adding initial version of trampoline for testing
* Refactoring trampoline to be re-used by other architectures
* Initial wiring for winch with wasmtime
* Add a Wasmtime CLI option to select `winch`
This is effectively an option to select the `Strategy` enumeration.
* Implement `Compiler::compile_function` for Winch
Hook this into the `TargetIsa::compile_function` hook as well. Currently
this doesn't take into account `Tunables`, but that's left as a TODO for
later.
* Filling out Winch append_code method
* Adding back in changes from previous branch
Most of these are a WIP. It's missing trampolines for x64, but a basic
one exists for aarch64. It's missing the handling of arguments that
exist on the stack.
It currently imports `cranelift_wasm::WasmFuncType` since it's what's
passed to the `Compiler` trait. It's a bit awkward to use in the
`winch_codegen` crate since it mostly operates on `wasmparser` types.
I've had to hack in a conversion to get things working. Long term, I'm
not sure it's wise to rely on this type but it seems like it's easier on
the Cranelift side when creating the stub IR.
* Small API changes to make integration easier
* Adding in new FuncEnv, only a stub for now
* Removing unneeded parts of the old PoC, and refactoring trampoline code
* Moving FuncEnv into a separate file
* More comments for trampolines
* Adding in winch integration tests for first pass
* Using new addressing method to fix stack pointer error
* Adding test for stack arguments
* Only run tests on x86 for now, it's more complete for winch
* Add in missing documentation after rebase
* Updating based on feedback in draft PR
* Fixing formatting on doc comment for argv register
* Running formatting
* Lock updates, and turning on winch feature flags during tests
* Updating configuration with comments to no longer gate Strategy enum
* Using the winch-environ FuncEnv, but it required changing the sig
* Proper comment formatting
* Removing wasmtime-winch from dev-dependencies, adding the winch feature makes this not necessary
* Update doc attr to include winch check
* Adding winch feature to doc generation, which seems to fix the feature error in CI
* Add the `component-model` feature to the cargo doc invocation in CI
To match the metadata used by the docs.rs invocation when building docs.
* Add a comment clarifying the usage of `component-model` for docs.rs
* Correctly order wasmtime-winch and winch-environ in the publish script
* Ensure x86 test dependencies are included in cfg(target_arch)
* Further constrain Winch tests to x86_64 _and_ unix
---------
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
Co-authored-by: Saúl Cabrera <saulecabrera@gmail.com>
* Add support for generating perf maps for simple perf profiling
* add missing enum entry in C code
* bugfix: use hexa when printing the code region's length too (thanks bjorn3!)
* sanitize file name + use bufwriter
* introduce --profile CLI flag for wasmtime
* Update doc and doc comments for new --profile option
* remove redundant FromStr import
* Apply review feedback: make_line receives a Write impl, report errors
* fix tests?
* better docs
Maps to the corresponding `wasmtime::Config` option. The motivation here
is largely completeness and was something I was looking into with the
failures in #5970
* Initial support for the Relaxed SIMD proposal
This commit adds initial scaffolding and support for the Relaxed SIMD
proposal for WebAssembly. Codegen support is supported on the x64 and
AArch64 backends on this time.
The purpose of this commit is to get all the boilerplate out of the way
in terms of plumbing through a new feature, adding tests, etc. The tests
are copied from the upstream repository at this time while the
WebAssembly/testsuite repository hasn't been updated.
A summary of changes made in this commit are:
* Lowerings for all relaxed simd opcodes have been added, currently all
exhibiting deterministic behavior. This means that few lowerings are
optimal on the x86 backend, but on the AArch64 backend, for example,
all lowerings should be optimal.
* Support is added to codegen to, eventually, conditionally generate
different code based on input codegen flags. This is intended to
enable codegen to more efficient instructions on x86 by default, for
example, while still allowing embedders to force
architecture-independent semantics and behavior. One good example of
this is the `f32x4.relaxed_fmadd` instruction which when deterministic
forces the `fma` instruction, but otherwise if the backend doesn't
have support for `fma` then intermediate operations are performed
instead.
* Lowerings of `iadd_pairwise` for `i16x8` and `i32x4` were added to the
x86 backend as they're now exercised by the deterministic lowerings of
relaxed simd instructions.
* Sample codegen tests for added for x86 and aarch64 for some relaxed
simd instructions.
* Wasmtime embedder support for the relaxed-simd proposal and forcing
determinism have been added to `Config` and the CLI.
* Support has been added to the `*.wast` runtime execution for the
`(either ...)` matcher used in the relaxed-simd proposal.
* Tests for relaxed-simd are run both with a default `Engine` as well as
a "force deterministic" `Engine` to test both configurations.
* All tests from the upstream repository were copied into Wasmtime.
These tests should be deleted when WebAssembly/testsuite is updated.
* x64: Add x86-specific lowerings for relaxed simd
This commit builds on the prior commit and adds an array of `x86_*`
instructions to Cranelift which have semantics that match their
corresponding x86 equivalents. Translation for relaxed simd is then
additionally updated to conditionally generate different CLIF for
relaxed simd instructions depending on whether the target is x86 or not.
This means that for AArch64 no changes are made but for x86 most relaxed
instructions now lower to some x86-equivalent with slightly different
semantics than the "deterministic" lowering.
* Add libcall support for fma to Wasmtime
This will be required to implement the `f32x4.relaxed_madd` instruction
(and others) when an x86 host doesn't specify the `has_fma` feature.
* Ignore relaxed-simd tests on s390x and riscv64
* Enable relaxed-simd tests on s390x
* Update cranelift/codegen/meta/src/shared/instructions.rs
Co-authored-by: Andrew Brown <andrew.brown@intel.com>
* Add a FIXME from review
* Add notes about deterministic semantics
* Don't default `has_native_fma` to `true`
* Review comments and rebase fixes
---------
Co-authored-by: Andrew Brown <andrew.brown@intel.com>
This commit includes a set of changes that add initial support for `wasi-threads` to Wasmtime:
* feat: remove mutability from the WasiCtx Table
This patch adds interior mutability to the WasiCtx Table and the Table elements.
Major pain points:
* `File` only needs `RwLock<cap_std::fs::File>` to implement
`File::set_fdflags()` on Windows, because of [1]
* Because `File` needs a `RwLock` and `RwLock*Guard` cannot
be hold across an `.await`, The `async` from
`async fn num_ready_bytes(&self)` had to be removed
* Because `File` needs a `RwLock` and `RwLock*Guard` cannot
be dereferenced in `pollable`, the signature of
`fn pollable(&self) -> Option<rustix::fd::BorrowedFd>`
changed to `fn pollable(&self) -> Option<Arc<dyn AsFd + '_>>`
[1] da238e324e/src/fs/fd_flags.rs (L210-L217)
* wasi-threads: add an initial implementation
This change is a first step toward implementing `wasi-threads` in
Wasmtime. We may find that it has some missing pieces, but the core
functionality is there: when `wasi::thread_spawn` is called by a running
WebAssembly module, a function named `wasi_thread_start` is found in the
module's exports and called in a new instance. The shared memory of the
original instance is reused in the new instance.
This new WASI proposal is in its early stages and details are still
being hashed out in the [spec] and [wasi-libc] repositories. Due to its
experimental state, the `wasi-threads` functionality is hidden behind
both a compile-time and runtime flag: one must build with `--features
wasi-threads` but also run the Wasmtime CLI with `--wasm-features
threads` and `--wasi-modules experimental-wasi-threads`. One can
experiment with `wasi-threads` by running:
```console
$ cargo run --features wasi-threads -- \
--wasm-features threads --wasi-modules experimental-wasi-threads \
<a threads-enabled module>
```
Threads-enabled Wasm modules are not yet easy to build. Hopefully this
is resolved soon, but in the meantime see the use of
`THREAD_MODEL=posix` in the [wasi-libc] repository for some clues on
what is necessary. Wiggle complicates things by requiring the Wasm
memory to be exported with a certain name and `wasi-threads` also
expects that memory to be imported; this build-time obstacle can be
overcome with the `--import-memory --export-memory` flags only available
in the latest Clang tree. Due to all of this, the included tests are
written directly in WAT--run these with:
```console
$ cargo test --features wasi-threads -p wasmtime-cli -- cli_tests
```
[spec]: https://github.com/WebAssembly/wasi-threads
[wasi-libc]: https://github.com/WebAssembly/wasi-libc
This change does not protect the WASI implementations themselves from
concurrent access. This is already complete in previous commits or left
for future commits in certain cases (e.g., wasi-nn).
* wasi-threads: factor out process exit logic
As is being discussed [elsewhere], either calling `proc_exit` or
trapping in any thread should halt execution of all threads. The
Wasmtime CLI already has logic for adapting a WebAssembly error code to
a code expected in each OS. This change factors out this logic to a new
function, `maybe_exit_on_error`, for use within the `wasi-threads`
implementation.
This will work reasonably well for CLI users of Wasmtime +
`wasi-threads`, but embedders will want something better in the future:
when a `wasi-threads` threads fails, they may not want their application
to exit. Handling this is tricky, because it will require cancelling the
threads spawned by the `wasi-threads` implementation, something that is
not trivial to do in Rust. With this change, we defer that work until
later in order to provide a working implementation of `wasi-threads` for
experimentation.
[elsewhere]: https://github.com/WebAssembly/wasi-threads/pull/17
* review: work around `fd_fdstat_set_flags`
In order to make progress with wasi-threads, this change temporarily
works around limitations induced by `wasi-common`'s
`fd_fdstat_set_flags` to allow `&mut self` use in the implementation.
Eventual resolution is tracked in
https://github.com/bytecodealliance/wasmtime/issues/5643. This change
makes several related helper functions (e.g., `set_fdflags`) take `&mut
self` as well.
* test: use `wait`/`notify` to improve `threads.wat` test
Previously, the test simply executed in a loop for some hardcoded number
of iterations. This changes uses `wait` and `notify` and atomic
operations to keep track of when the spawned threads are done and join
on the main thread appropriately.
* various fixes and tweaks due to the PR review
---------
Signed-off-by: Harald Hoyer <harald@profian.com>
Co-authored-by: Harald Hoyer <harald@profian.com>
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
* Wasmtime: Add `Config::disable_cache`
* bench-api: Always disable the cache
* bench-api: Always get a `Config` from CLI flags
This commit fixes an issue that I ran into just now where benchmarking
one `*.so` with `--engine-flags` was giving wildly unexpected results
comparing to something without `--engine-flags`. The root cause here
appears to that when specifying `--engine-flags` the CLI parsing code is
used to create a `Config` and when omitted a `Config::new` instance is
created. The main difference between these is that for the CLI caching
is enabled by default and for `Config::new` it is not. Coupled with the
fact that caching doesn't really work for the `main` branch this ended
up giving wild results.
The fix here is to first always use the CLI parsing code to create a
`Config` to ensure that a config is consistently created. Next the
`--disable-cache` flag is unconditionally passed to the CLI parsing to
ensure that compilation actually happens.
Once applied this enables comparing an engine without flags and an
engine with flags which provides consistent results.
* Fix compile error
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
* Unconditionally use `MemoryImageSlot`
This commit removes the internal branching within the pooling instance
allocator to sometimes use a `MemoryImageSlot` and sometimes now.
Instead this is now unconditionally used in all situations on all
platforms. This fixes an issue where the state of a slot could get
corrupted if modules being instantiated switched from having images to
not having an image or vice versa.
The bulk of this commit is the removal of the `memory-init-cow`
compile-time feature in addition to adding Windows support to the
`cow.rs` file.
* Fix compile on Unix
* Add a stricter assertion for static memory bounds
Double-check that when a memory is allocated the configuration required
is satisfied by the pooling allocator.
This commit changes the APIs in the `wasmtime` crate for configuring the
pooling allocator. I plan on adding a few more configuration options in
the near future and the current structure was feeling unwieldy for
adding these new abstractions.
The previous `struct`-based API has been replaced with a builder-style
API in a similar shape as to `Config`. This is done to help make it
easier to add more configuration options in the future through adding
more methods as opposed to adding more field which could break prior
initializations.
* Leverage Cargo's workspace inheritance feature
This commit is an attempt to reduce the complexity of the Cargo
manifests in this repository with Cargo's workspace-inheritance feature
becoming stable in Rust 1.64.0. This feature allows specifying fields in
the root workspace `Cargo.toml` which are then reused throughout the
workspace. For example this PR shares definitions such as:
* All of the Wasmtime-family of crates now use `version.workspace =
true` to have a single location which defines the version number.
* All crates use `edition.workspace = true` to have one default edition
for the entire workspace.
* Common dependencies are listed in `[workspace.dependencies]` to avoid
typing the same version number in a lot of different places (e.g. the
`wasmparser = "0.89.0"` is now in just one spot.
Currently the workspace-inheritance feature doesn't allow having two
different versions to inherit, so all of the Cranelift-family of crates
still manually specify their version. The inter-crate dependencies,
however, are shared amongst the root workspace.
This feature can be seen as a method of "preprocessing" of sorts for
Cargo manifests. This will help us develop Wasmtime but shouldn't have
any actual impact on the published artifacts -- everything's dependency
lists are still the same.
* Fix wasi-crypto tests
This commit replaces #4869 and represents the actual version bump that
should have happened had I remembered to bump the in-tree version of
Wasmtime to 1.0.0 prior to the branch-cut date. Alas!
* Update tracing-core to a version which doesn't depend on lazy-static.
* Update crossbeam-utils to a version that doesn't depend on lazy-static.
* Update crossbeam-epoch to a version that doesn't depend on lazy-static.
* Update clap to a version that doesn't depend on lazy-static.
* Convert Wasmtime's own use of lazy_static to once_cell.
* Make `GDB_REGISTRATION`'s comment a doc comment.
* Fix compilation on Windows.
This commit refactored `Config` to use a seperate `CompilerConfig` field instead
of operating on `CompilerBuilder` directly to make all its methods idempotent.
Fixes#4189
Alex Crichton
Dan Gohman
Andrew Brown
Nick Fitzgerald
Xinzhao Xu
Ryan Levick
Chris Fallin
Benjamin Bouvier
Luna P-C
Brendan Burns
Kevin Rizzo
Tshepang Mbambo
Cheng Shao
wasmtime-publish
Pure White