This commit updates Wasmtime to support `global.get` in constant
expressions when located in table initializers and element segments.
Pre-reference-types this never came up because there was no valid
`global.get` that would typecheck. After the reference-types proposal
landed however this became possible but Wasmtime did not support it.
This was surfaced in #6705 when the spec test suite was updated and has
a new test that exercises this functionality.
This commit both updates the spec test suite and additionally adds
support for this new form of element segment and table initialization
expression.
The fact that Wasmtime hasn't supported this until now also means that
we have a gap in our fuzz-testing infrastructure. The `wasm-smith`
generator is being updated in bytecodealliance/wasm-tools#1426 to
generate modules with this particular feature and I've tested that with
that PR fuzzing here eventually generates an error before this PR.
Closes#6705
* Update some CI dependencies
* Update to the latest nightly toolchain
* Update mdbook
* Update QEMU for cross-compiled testing
* Update `cargo nextest` for usage with MIRI
prtest:full
* Remove lots of unnecessary imports
* Downgrade qemu as 8.2.1 seems to segfault
* Remove more imports
* Remove unused winch trait method
* Fix warnings about unused trait methods
* More unused imports
* More unused imports
This commit updates the allocation scheme for resources in the component
model to start at 1 instead of 0 when communicating with components.
This is an implementation of WebAssembly/component-model#284.
While this broke a number of tests we have this shouldn't actually break
any components in practice. The broken tests were all overly-precise in
their assertions and error messages and this shouldn't idiomatically
come up in any guest language, so this should not be a practically
breaking change.
This change additionally places an upper limit on the maximum
allocatable index at `1 << 30` which is also specified in the above PR.
* Wasmtime: Finish support for the typed function references proposal
While we supported the function references proposal inside Wasm, we didn't
support it on the "outside" in the Wasmtime embedder APIs. So much of the work
here is exposing typed function references, and their type system updates, in
the embedder API. These changes include:
* `ValType::FuncRef` and `ValType::ExternRef` are gone, replaced with the
introduction of the `RefType` and `HeapType` types and a
`ValType::Ref(RefType)` variant.
* `ValType` and `FuncType` no longer implement `Eq` and `PartialEq`. Instead
there are `ValType::matches` and `FuncType::matches` methods which check
directional subtyping. I also added `ValType::eq` and `FuncType::eq` static
methods for the rare case where someone needs to check precise equality, but
that is almost never actually the case, 99.99% of the time you want to check
subtyping.
* There are also public `Val::matches_ty` predicates for checking if a value is
an instance of a type, as well as internal helpers like
`Val::ensure_matches_ty` that return a formatted error if the value does not
match the given type. These helpers are used throughout Wasmtime internals
now.
* There is now a dedicated `wasmtime::Ref` type that represents reference
values. Table operations have been updated to take and return `Ref`s rather
than `Val`s.
Furthermore, this commit also includes type registry changes to correctly manage
lifetimes of types that reference other types. This wasn't previously an issue
because the only thing that could reference types that reference other types was
a Wasm module that added all the types that could reference each other at the
same time and removed them all at the same time. But now that the previously
discussed work to expose these things in the embedder API is done, type lifetime
management in the registry becomes a little trickier because the embedder might
grab a reference to a type that references another type, and then unload the
Wasm module that originally defined that type, but then the user should still be
able use that type and the other types it transtively references. Before, we
were refcounting individual registry entries. Now, we still are refcounting
individual entries, but now we are also accounting for type-to-type references
and adding a new type to the registry will increment the refcounts of each of
the types that it references, and removing a type from the registry will
decrement the refcounts of each of the types it references, and then recursively
(logically, not literally) remove any types whose refcount has now reached zero.
Additionally, this PR adds support for subtyping to `Func::typed`- and
`Func::wrap`-style APIs. For result types, you can always use a supertype of the
WebAssembly function's actual declared return type in `Func::typed`. And for
param types, you can always use a subtype of the Wasm function's actual declared
param type. Doing these things essentially erases information but is always
correct. But additionally, for functions which take a reference to a concrete
type as a parameter, you can also use the concrete type's supertype. Consider a
WebAssembly function that takes a reference to a function with a concrete type:
`(ref null <func type index>)`. In this scenario, there is no static
`wasmtime::Foo` Rust type that corresponds to that particular Wasm-defined
concrete reference type because Wasm modules are loaded dynamically at
runtime. You *could* do `f.typed::<Option<NoFunc>, ()>()`, and while that is
correctly typed and valid, it is often overly restrictive. The only value you
could call the resulting typed function with is the null function reference, but
we'd like to call it with non-null function references that happen to be of the
correct type. Therefore, `f.typed<Option<Func>, ()>()` is also allowed in this
case, even though `Option<Func>` represents `(ref null func)` which is the
supertype, not subtype, of `(ref null <func type index>)`. This does imply some
minimal dynamic type checks in this case, but it is supported for better
ergonomics, to enable passing non-null references into the function.
We can investigate whether it is possible to use generic type parameters and
combinators to define Rust types that precisely match concrete reference types
in future, follow-up pull requests. But for now, we've made things usable, at
least.
Finally, this also takes the first baby step towards adding support for the Wasm
GC proposal. Right now the only thing that is supported is `nofunc` references,
and this was mainly to make testing function reference subtyping easier. But
that does mean that supporting `nofunc` references entailed also adding a
`wasmtime::NoFunc` type as well as the `Config::wasm_gc(enabled)` knob. So we
officially have an in-progress implementation of Wasm GC in Wasmtime after this
PR lands!
Fixes https://github.com/bytecodealliance/wasmtime/issues/6455
* Fix WAT in test to be valid
* Check that dependent features are enabled for function-references and GC
* Remove unnecessary engine parameters from a few methods
Ever since `FuncType`'s internal `RegisteredType` holds onto its own `Engine`,
we don't need these anymore.
Still useful to keep the `Engine` parameter around for the `ensure_matches`
methods because that can be used to check correct store/engine usage for
embedders.
* Add missing dependent feature enabling for some tests
* Remove copy-paste bit from test
* match self to show it is uninhabited
* Add a missing `is_v128` method
* Short circuit a few func type comparisons
* Turn comment into part of doc comment
* Add test for `Global::new` and subtyping
* Add tests for embedder API, tables, and subtyping
* Add an embedder API test for setting globals and subtyping
* Construct realloc's type from its index, rather than from scratch
* Help LLVM better optimize our dynamic type checks in `TypedFunc::call_raw`
* Fix call benchmark compilation
* Change `WasmParams::into_abi` to take the whole func type instead of iter of params
* Fix doc links
prtest:full
* Fix size assertion on s390x
* Allow `traps.rs` tests to pass even when `RUST_BACKTRACE=1` on nightly Rust
Instead of asserting exact error messages, which `anyhow` auto adds Rust
backtrace to on nightly, assert that the important bits are there.
* Use `contains` on the whole error message when possible
* mpk: restore PKRU state when a fiber resumes execution
Previously, when a fiber was suspended, other computation could change
the PKRU state on the current CPU. This means that the fiber could be
resumed with a different PKRU state. This could be bad, resulting in
situations in which the fiber can access more memory slots than it
should or cannot even access its own memory slots.
This change saves the PKRU state prior to a fiber being suspended. When
the fiber resumes execution, that PKRU state is restored.
* mpk: check correct PKRU switching on async suspension
This adds a test that alternately polls two Wasm instances in a loop.
Since the instances are async, we can set up epochs to suspend each
fiber as we iterate over a loop. Because we alternate between the two
instances, it checks that `AsyncCx::block_on` has correctly restored the
PKRU bits; otherwise we should see test failures. In the process of
writing this test I discovered #7942, which can be solved separately
(it has to do with the interaction between memory images, _not_ used
here, and MPK).
prtest:full
* fix: condition the PKRU context switches
Not all stores have protection keys and MPK is not always enabled. This
change checks for these conditions before context-switching the PKRU
bits.
* WIP: try to make wasi-common self contained.
* rebase: cargo.lock
* remove all dependencies between wasi-common and wasmtime-wasi
* use wasi-common directly throughout tests, benches, examples, cli run
* wasi-threads: use wasi-common's maybe_exit_on_error in spawned thread
not a very modular design, but at this point wasi-common and
wasi-threads are forever wed
* fix wasmtime's docs
* re-introduce wasmtime-wasi's exports of wasi-common definitions behind deprecated
* factor out determining i32 process exit code
and remove libc dep because rustix provides the same constant
* commands/run: inline the logic about aborting on trap
since this is the sole place in the codebase its used
* Add high-level summary to wasi-common's top-level doc comment.
* c-api: fix use of wasi_cap_std_sync => wasi_common::sync, wasmtime_wasi => wasi_common
* fix tokio example
* think better of combining downcast and masking into one method
* fix references to wasmtime_wasi in docs
prtest:full
* benches: use wasi-common
* cfg-if around use of rustix::process because that doesnt exist on windows
* wasi-common: include tests, caught by verify-publish
* fix another bench
* exit requires wasmtime dep. caught by verify-publish.
* Perform stronger typechecks of host-owned resources
Right now the abstraction for host-owned resources in Wasmtime is quite
simple, it's "just an index". This can be error prone because the host
can suffer both from use-after-free and ABA-style problems. While
there's not much that can be done about use-after-free the previous
implementation would accidentally enable "AB" style problems where if a
host-owned resource index was deallocated and then reallocated with
another type the original handle would still work. While not a major bug
this can be confusing and additionally violate's our guarantees as a
component runtime to guests to ensure that resources always have valid
`rep`s going into components.
This commit adds a new layer of storage which keeps track of the
`ResourceType` for all host-owned resources. This means that resources
going into a host-owned table now record their type and coming out of a
host-owned table a typecheck is always performed. Note that guests can
continue to skip this logic as they already have per-type tables and so
won't suffer the same issue.
This change is inspired by my taking a look at #7883. The crux of the
issue was a typo where a resource was reused by accident which led to
confusing behavior due to the reuse. This change instead makes it return
an error more quickly and doesn't allow the component to see any invalid
state.
Closes#7883
* Fix test
* Use generation counters, not types
* Review comments
* Refactor `wasmtime::FuncType` to hold a handle to its registered type
Rather than holding a copy of the type directly, it now holds a `RegisteredType`
which internally is
* A `VMSharedTypeIndex` pointing into the engine's types registry.
* An `Arc` handle to the engine's type registry.
* An `Arc` handle to the actual type.
The last exists only to keep it so that accessing a `wasmtime::FuncType`'s
parameters and results fast, avoiding any new locking on call hot paths.
This is helping set the stage for further types and `TypeRegistry` refactors
needed for Wasm GC.
* Update the C API for the function types refactor
prtest:full
* rustfmt
* Fix benches build
* feat: support component instance import type introspection
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
* refactor: implement `import_types` on `Linker`
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
* refactor: flatten `types::ComponentItem` members
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
* refactor: remove `types` re-exports
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
* feat: implement component type introspection
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
* chore: derive `Debug` for `Field`
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
* refactor: return `ExactSizeIterator` for component function parameters/results
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
* tests: add `introspection` test
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
* chore: derive `Clone` on `ComponentItem` types
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
* feat: support module and instance type introspection
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
---------
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
* Enable the component model by default
This commit enables the component model by default in the embedding API
and the CLI. This means that an opt-in of `-W component-model` is no
longer required and additionally `.wasm_component_model(true)` is no
longer required. Note that this won't impact existing embeddings since
the component model feature doesn't do much less `wasmtime::component`
is used, and if that's being used this is probably good news for you.
* Fix non-component model build
* WASI: copy in the version 0.2.0 wits
* wasmtime's wits: use versions 0.2.0 of wasi packages
* bindgens and other fixed version strings: change 0.2.0-rc-etc to 0.2.0
* mpk: allow configuring MPK from the command line
This allows the following usage:
```console
$ wasmtime --pooling-allocator=y --memory-protection-keys=y ...
```
* mpk: add CLI test for failure
* review: move `bail!` outside the pooling allocator block
* fix: use absolute dependency path
* fix: allow dynamic owned resources to be used as borrowed parameters
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
* tests: add `can_use_own_for_borrow` test
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
---------
Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>
* wasi: pull in contents of wasi-sockets, wasi-http, and wasi-cli 0.2.0-rc-2024-01-16
* command-extended and test worlds: use rc-2024-01-16
* sockets implementation: v6only is now mandatory
* adapter: cli imports and exports are from rc-2024-01-16 now
* eliminate ipv6-only methods and tests
* a v6_client.blocking_connect(net, v4_listener) will always fail
with INVAL right away.
* eliminate the paths where a v6 client is allowed to connect to v4.
* eliminate the udp_dual_stack_conversation from udp_sample_application
* component-basic: update wasi:cli version
* wasi-http: sync wit directory
* wasi-http: fix import version
* code review from dave
* test both ipv4 address on v6 socket, and ipv6-mapped-ipv4 on v6 socket, both fail
* Add a small test checking that pollables can be used many times
* Tests where two commands have their stdin/stdout piped together
* Pipe together wasmtime subprocesses instead of managing the buffer
* Update the wasm-tools family of crates
Brings in support for validating gc instructions, but they're all left
disabled for now.
* Update fuzz test case generation
* More test fixes, remove stray files
* More test fixes
* Rebase
The test simply tries looking up example.com and ensuring it fails with
`PermanentResolverFailure`. Additionally, `PermanentResolverFailure` is
removed as one of the allowed errors in the normal ip name lookup test
as those allowed errors should only be transient lookup errors and
`PermanentResolverFailure` is very much not a transient error.
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Ensure remote_address is allowed when creating UDP stream
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Check addr validity after UDP state check and wrap pool in Arc
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Check provided addr in outgoing stream
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Add test for turning off udp
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Model pool on UDP types as optional to catch bad state bugs
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
---------
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Add options to WasiCtx for toggling TCP and UDP on and off
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Address PR feedback
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Add test for turning off tcp
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
---------
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* wasi-cli: update to version 0.2.0-rc-2023-12-05
* wasi-http: update to 0.2.0-rc-2023-12-05
* fix versions in extra (non-wasi) wits
* component adapter: fixes to use cli/imports world, correct versions
* wasmtime-wasi: cli/reactor is now cli/imports
* sync wasi-http/wit with wasi/wit
* fix cli-test component-basic.wat version
* mpk: optimize layout of protected stripes, again
This is another attempt at #7603, attempting reduce the slab layout
sizes of MPK-protected stripes. While experimenting with the limits of
MPK-protected memory pools, @alexcrichton and I discovered that the
current slab layout calculations were too conservative. This meant that
the memory pool could not pack in as many memories as it should have
been able: we were expecting, but not seeing, ~15x more memory slots
over non-MPK memory pools.
This change brings together several fixes:
- it more aggressively divides up the stripes (as in b212152)
- it eliminates an extra stripe (as in 8813a30)
- it replaces some uses of `checked_*` with `saturating_*` (as in
fb22a20)
- it improves some documentation
- and, crucially, it reports back a larger value for
`memory_and_guard_size`
The failures observed with #7603 when run with MPK
(`WASMTIME_TEST_FORCE_MPK=1 cargo test`) were due to `Store::wasm_fault`
not being able to identify which memory an OOB address belonged to.
This is because the `MemoryPool` was underreporting the size of the
region in which OOB accesses would fault. The correct value is provided
by the new `SlabLayout::bytes_to_next_stripe_slot`: any OOB access
within that larger region must fault because (1) the other stripes have
different protection keys and (2) a `Store` can only use one protection
key. We also use (2) to guarantee that `Store::wasm_fault` will be able
to calculate the Wasm address from the raw address.
This change also provides a new `traps` test that will reproduce the
failures from #7603; if we observe `SIGABRT` from that test, it will be
a regression.
* fix: make test x86-specific
* mpk: allow forcing MPK during tests
For testing on machines on which we know MPK is enabled, we want to be
able to force-enable MPK, ensuring we get coverage of MPK-related code.
This change adds a `WASMTIME_TEST_FORCE_MPK` environment variable which,
when set, sets the pooling allocator configuration to force-enable MPK.
This variable, like the `WASMTIME_TEST_NO_HOG_MEMORY` variable it is
styled from, could be used in CI workflows on which we know MPK should
be available.
* review: only check `WASMTIME_TEST_FORCE_MPK` in tests
Checking the environment variable at runtime is too invasive and could
lead to unexpected behavior. This limits the use of
`WASMTIME_TEST_FORCE_MPK` to the `wast` tests and any tests that use the
`small_pool_config`.
* Configure Rust lints at the workspace level
This commit adds necessary configuration knobs to have lints configured
at the workspace level in Wasmtime rather than the crate level. This
uses a feature of Cargo first released with 1.74.0 (last week) of the
`[workspace.lints]` table. This should help create a more consistent set
of lints applied across all crates in our workspace in addition to
possibly running select clippy lints on CI as well.
* Move `unused_extern_crates` to the workspace level
This commit configures a `deny` lint level for the
`unused_extern_crates` lint to the workspace level rather than the
previous configuration at the individual crate level.
* Move `trivial_numeric_casts` to workspace level
* Change workspace lint levels to `warn`
CI will ensure that these don't get checked into the codebase and
otherwise provide fewer speed bumps for in-process development.
* Move `unstable_features` lint to workspace level
* Move `unused_import_braces` lint to workspace level
* Start running Clippy on CI
This commit configures our CI to run `cargo clippy --workspace` for all
merged PRs. Historically this hasn't been all the feasible due to the
amount of configuration required to control the number of warnings on
CI, but with Cargo's new `[lint]` table it's possible to have a
one-liner to silence all lints from Clippy by default. This commit by
default sets the `all` lint in Clippy to `allow` to by-default disable
warnings from Clippy. The goal of this PR is to enable selective access
to Clippy lints for Wasmtime on CI.
* Selectively enable `clippy::cast_sign_loss`
This would have fixed#7558 so try to head off future issues with that
by warning against this situation in a few crates. This lint is still
quite noisy though for Cranelift for example so it's not worthwhile at
this time to enable it for the whole workspace.
* Fix CI error
prtest:full
This commit fixes a bug in initializing memory segments of 32-bit
memories where if the offset was negative when viewed as a signed
integer the offset was incorrectly sign-extended to a 64-bit value
instead of zero-extended. This commit replaces an `i32`-to-`u64` cast
with an `i32`-to-`u32` cast followed by a `u32`-to-`u64` cast which
performs the zero extend.
Closes#7558
This commit implements the `wasi_unstable` module, sometimes referred to
as "preview0", in the `wasmtime-wasi` crate. Previously this was only
implemented by the `wasi-common` crate but now this is implemented for
both meaning that the switch to preview2 won't lose this functionality.
The preview0 WITX files are vendored like the preview1 files and the
implementation of preview0 is exclusively implemented by delegating to
the preview1 implementation.
* wit deps: use 0.2.0-rc-2023-11-10 from WebAssembly/wasi-* repos
* local wits: depend on rc-2023-11-10
* wit-bindgen invocation: use rc-2023-11-10
* wasi-http wit subdir: sync with wasi
* wasi-http wit-bindgen: update for 11-10 rc
* wasi-preview1-component-adapter: imports are from 11-10 rc
* cli test component-basic: use wasi rc 11-10
This commit fixes a mistake where these tests were not being run on
macOS after signal handling was moved behind a `Config` flag instead of
being a crate feature. Additionally these tests are moved to their own
test file to ensure that each process either uses Mach Ports or signals
(this one uses signals which is non-default).
* Fix writes being flushed to stdio
This commit fixes an accidental issue with writing to stdout with the
`wasi-common` implementation where writes were line-buffered by the Rust
standard library rather than being flushed out immediately as WASI
semantics required.
Closes#7437
* Fix warning in test
In Wasmtime 13 and prior the `--dir` argument was unconditionally used
to open a host dir as the same name and in the guest. In Wasmtime 14+
though this argument is being repurposed with an optional trailing
`::GUEST` to configure the guest directory. This means that
`--dir`-with-remapping behavior is actually unusable without the
environment variable configuration from #7385 due to it parsing
differently in the old and the new.
This commit updates the situation by adding `::`-parsing to the old CLI
meaning that both the old and the new parse this the same way. This will
break any scripts that open host directories with two colons in their
path, but that seems niche enough we can handle that later.
* Add compatibility shims for Wasmtime 13 CLI
This commit introduces a compatibility shim for the Wasmtime 13 CLI and
prior. The goal of this commit is to address concerns raised in #7336
and other locations as well. While the new CLI cannot be un-shipped at
this point this PR attempts to ameliorate the situation somewhat through
a few avenues:
* A complete copy of the old CLI parser is now included in `wasmtime` by
default.
* The `WASMTIME_NEW_CLI=0` environment variable can force usage of the
old CLI parser for the `run` and `compile` commands.
* The `WASMTIME_NEW_CLI=1` environment variable can force usage of the
new CLI parser.
* Otherwise both the old and the new CLI parser are executed. Depending
on the result one is selected to be executed, possibly with a warning
printed.
* If both CLI parsers succeed but produce the same result, then no
warning is emitted and execution continues as usual.
* If both CLI parsers succeed but produce different results then a
warning is emitted indicating this. The warning points to #7384 which
has further examples of how to squash the warning. The warning also
mentions the above env var to turn the warning off. In this situation
the old semantics are used at this time instead of the new semantics.
It's intended that eventually this will change in the future.
* If the new CLI succeeds and the old CLI fails then the new semantics
are executed without warning.
* If the old CLI succeeds and the new CLI fails then a warning is issued
and the old semantics are used.
* If both the old and the new CLI fail to parse then the error message
for the new CLI is emitted.
Note that this doesn't add up to a perfect transition. The hope is that
most of the major cases of change at the very least have something
printed. My plan is to land this on `main` and then backport it to the
14.0.0 branch as a 14.0.3 release.
* Wordsmith messages
* Update messages
* More wording updates
* Fix grammar
* More updates
Currently Wasmtime has two implementations of the `wasi_snapshot_preview1` set
of APIs. The now-historical implementation lives in the `wasi-common`
crate and the more recent implementation lives in the `wasmtime-wasi`
crate. The main difference is that the `wasmtime-wasi` implementation is
based on the implementation of preview2, meaning that the preview1
implementation is a shim in that direction. Additionally currently the
preview2 implementation of preview1 is accessible via the `-Spreview2`
flag on the CLI.
This commit updates the interpretation of the `-Spreview2` flag and the
defaults around which implementation to choose. By default the
preview1-built-on-preview2 implementation (the new `wasmtime-wasi`
implementation) is selected now. This means that the `wasi-common`
implementation is disabled by default. There are still two use cases to
keep running the `wasi-common` implementation, however:
* When running modules that import from `wasi_unstable`, the "snapshot"
before `wasi_snapshot_preview1`, currently `wasi-common` is required.
The shims to implement `wasi_unstable` have not yet been implemented
in the `wasmtime-wasi` crate.
* When running with WASI threads (`-Sthreads`) the preview2
implementation does not work. This is because the preview2
implementation expects mutable access to the table which is not
granted when threads are enabled.
Tests using `wasi_unstable` now pass `-Spreview2=n` to explicitly
request the old `wasi-common` implementation. Additionally the
`wasi-common` implementation is still selected by default when
`-Sthreads` is passed (enabling the WASI threads proposal).
* mpk: add Wasm test
This adds a Wasmtime-level test checking that, when MPK is available on
the system, an MPK-configured memory pool obeys the same invariants as
other memory pool configurations (e.g., `guards_present` for normal
allocation, `guards_present_pooling` for non-MPK pooling allocation).
It also fixes a bug. A previous commit had updated the validation logic
of `MemoryPool` to check that the memory plan's bound would be reached
before the next slot of the same stripe. But `MemoryPool::allocate` has
some "double-check" logic that was triggered by this test. It now
matches the logic in `MemoryPool::validate`.
* mpk: explicitly disable MPK on certain tests
Some Wasmtime tests rely on specific limits to the memory pool. When MPK
is enabled, these tests fail because MPK splits access to the pool
slices among different stores. This change does not yet enable MPK,
though locally I run with MPK enabled. With this commit and MPK enabled
locally, all Wasmtime tests now pass.
* mpk: add `max_memory_protection_keys`
If Wasmtime is ever embedded in an application that also uses memory
protection keys, it could be useful to limit how many Wasmtime
allocates and uses. This came up while examining `*.wast` tests: if
there was no way limiting the number of keys used, then those tests
configured a pool that reserved too much memory. This change takes that
further to attempt to limit the initial number of keys allocated. The
unfortunate side effect of using a `OnceLock` is that the `max` setting
is only applicable on the first invocation, the one that sets the
`OnceLock`.
* mpk: use two protection keys for WAST tests
This change stems from how slicing memory slots into MPK-protected
regions limits the number of memories each store can access: e.g., with
fifteen keys in use, a store only has access to a fifteenth of the
available slots. If we simply multiple the number of memory slots needed
to run the `*.wast` spec tests by fifteen, we run out of available
memory. This limits the number of protection keys used to two, which
still allows us to test the functionality without reserving too much
memory.
* mpk: ensure `keys` only ever returns `max` items
This addresses a review comment to slice the list of keys down to the
`max` hint regardless of how many are allocated in the first invocation.
* fix: remove warning about unused parameter
In an effort to simplify the many fuel related APIs, simplify the
interface here to a single counter with get and set methods.
Additionally the async yield is reduced to an interval of the total fuel
instead of injecting fuel, so it's easy to still reason about how much
fuel is left even with yielding turned on.
Internally this works by keeping two counters - one the VM uses to
increment towards 0 for fuel, the other to track how much is in
"reserve". Then when we're out of gas, we pull from the reserve to
refuel and continue. We use the reserve in two cases: one for overflow
of the fuel (which is an i64 and the API expresses fuel as u64) and the
other for async yieling, which then the yield interval acts as a cap to
how much we can refuel with.
This also means that `get_fuel` can return the full range of `u64`
before this change it could only return up to `i64::MAX`. This is
important because this PR is removing the functionality to track fuel
consumption, and this makes the API less error prone for embedders to
track consumption themselves.
Careful to note that the VM counter that is stored as `i64` can be
positive if an instruction "costs" multiple units of fuel when the fuel
ran out.
prtest:full
Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
* Update WASI versions to `0.2.0-rc-2023-11-05`
This commit updates the version numbers on `main` to no longer clash
with the 14.0.0 release after #7299. The version number is chosen as the
branch point for the 15.0.0 release of Wasmtime, at which point we'll
update the versions again.
* Update another version
This commit changes the `--dir` argument on the `wasmtime` CLI to be
`HOST::GUEST` rather than `GUEST::HOST`. This matches Docker for example
and is a little more consistent with only `--dir path` where the first
argument is always treated as a host directory.
In terms of breaking-ness the movement from `--mapdir` to `--dir` hasn't
been released with Wasmtime 14 yet so my hope is that this can land on
both `main` and Wasmtime 14.0.0 before it's released to avoid any
breakage other than existing scripts migrating from `--mapdir` to
`--dir`.
* Put versions in all WASI WIT files
This commit starts exercising the versioning feature of WIT by ensuring
that all WASI descriptions have a version associated with them. The
version chosen is 0.2.0 which reflects the upcoming "preview 2" release
where in theory 0.1.0 was claimed by preview1. This is intended to stay
as 0.2.0 for now and we'll determine how best to update these numbers in
the future once preview2 is released.
Closes#7171
* Allow omitting versions in `with` keys
As a convenience for now this enables omitting the version of an
interface from a `with` key. This has a risk of not working well if two
packages are present and one has a version and one doesn't, but that's
left as a PR to fix in the future as the benefit of avoiding repetition
seems good for now.
* Allow omitting versions in trappable_error_types
* Use 0.2.0-rc-2023-10-18 as a version number
* More test fixes
* Fix another test
* Enable threads, multi-memory, and relaxed-simd by default
This commit enables by default three proposals that were advanced to
stage 4 in the proposal process at last week's in-person CG meeting.
These proposals are:
* `threads` - atomic instructions and shared memory
* `multi-memory` - the ability to have more than one linear memory
* `relaxed-simd` - new simd instructions that may differ across
platforms
These proposals are now all enabled by default in Wasmtime. Note that
they can still be selectively disabled via `Config` options.
* Fix an x64-specific test
* Support set_fuel in store APIs
Fixes: https://github.com/bytecodealliance/wasmtime/issues/5109
Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
* rename set_fuel to reset_fuel
To make it more clear that consumed fuel is being reset.
Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
* update out of date documentation for fuel in C API
Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
---------
Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
Alex Crichton
Nick Fitzgerald
Andrew Brown
Pat Hickey
Matheus Cardoso
Roman Volosatovs
Trevor Elliott
Ryan Levick
Tyler Rockwood