* Fix test expectations
[automatically-tag-and-release-this-commit]
* CI: upgrade to qemu 8.0.4. (#6849)
* CI: upgrade to qemu 8.0.4.
This should resolve a bug in qemu triggered by changes in #6820
(see
[here](https://github.com/bytecodealliance/wasmtime/pull/6820#issuecomment-1678908361)).
* ci: Update QEMU patch for 8.0.2
---------
Co-authored-by: Afonso Bordado <afonsobordado@az8.co>
---------
Co-authored-by: Chris Fallin <chris@cfallin.org>
Co-authored-by: Afonso Bordado <afonsobordado@az8.co>
* x64: Fix an off-by-one in in `i64x2.shr_s`
This commit fixes a mistake from #6372 where one of the immediates to a
`pshufd` instruction was off-by-one in terms of bits. This fixes the
behavior of the wasm `i64x2.shr_s` instruction with constant shift
amounts larger than 32.
* Update release notes
* More release notes
* x64: Remove recursion in `to_amode` helper
This commit removes the recursion present in the x64 backend's
`to_amode` and `to_amode_add` helpers. The recursion is currently
unbounded and controlled by user input meaning it's not too hard to
craft user input which triggers stack overflow in the host. By removing
recursion there's no need to worry about this since the stack depth will
never be hit.
The main concern with removing recursion is that code quality may not be
quite as good any more. The purpose of the recursion here is to "hunt
for constants" and update the immediate `Offset32`, and now without
recursion only at most one constant is found and folded instead of an
arbitrary number of constants as before. This should continue to produce
the same code as before so long as optimizations are enabled, but
without optimizations this will produce worse code than before.
Note with a hypothetical mid-end optimization that does this constant
folding for us the rules here could be further simplified to purely
consider the shape of the input `Value` to amode computation without
considering constants at all.
* Perform a `--locked` install of `cargo vet`
* Try adding a `minimal-versions` test to CI
This commit is an attempt to add another CI check for Wasmtime to ensure
that the minimum version bounds on all of our dependencies are accurate.
This is not a stable feature of Cargo and thus requires usage of
nightly. Additionally one of the reasons it's not stable is that the DX
is not great as many minimal-versions errors come from transitive
dependencies that we can't do anything about. Nevertheless I figure it
might be good to try out having it on CI and see how it fares.
This is inspired by #6755 where we picked up a dependency on a newer
version of `system-interface` but forgot to update the minimum version
bound. Whether or not this prevents the issue or causes too many
headaches is I think yet to be seen.
* Update some minimum version requirements
Currently Cargo's interpretation of prerelease version doesn't quite
match semver's, meaning that hyper, which is taking semver's
interpretation, has broken Wasmtime builds without lock files. The rc.4
version of hyper released is no longer compatible with our
usage which means that `cargo install wasmtime-cli` currently fails. It
passes, however, with `--locked`.
To resolve this the dependency on hyper gets an `=` dependency for now
to force it to be at this version.
Cranelift: upgrade to regalloc2 0.9.2. (#6726)
* Cranelift: upgrade to regalloc2 0.9.2.
This pulls in bytecodealliance/regalloc2#152, which fixes a bug that is
reachable on RISC-V: when two different register classes have the same
stackslot size, the register allocation result might share a slot
between two different classes, which can result in moves between classes
that will cause a panic. The fix properly separates slots by class.
* cargo-vet update for regalloc2 0.9.2.
* Add WasiCtxBuilder setters for the two clock types
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Move two clocks to dedicated fields in WasiCtx
* Rename clock traits to Host prefix convention
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Fix tests
---------
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Shepherd along type information in lifting/lowering
This commit is a large refactor to the component runtime of Wasmtime to
shepherd along type information when lifting and lowering values.
Previously lifting and lowering would assume type information given
context such as "surely lowering a `u32` must lower into the type
`InterfaceType::U32`" or "lowering a `Val` works as it knows its own
type". This is still true, and this commit isn't changing these
features. The rationale for this commit instead stems from the upcoming
implementation of resources in Wasmtime.
Resources are trickier than all existing types in Wasmtime because what
exactly is the type of a resource depends on who you're asking. For
example the host might have one type called `http::Headers` but a
component could import it as two distinct types:
(component
(import "headers1" (type $h1 (sub resource)))
(import "headers2" (type $h2 (sub resource)))
;; ...
)
in the above component the `$h1` and `$h2` types will each get their own
table at runtime for managing their state. This means that if the host
instantiates the component with `http::Headers` supplied as both types
then the same type on the outside maps to two different types inside.
This means that the lowering of a host-defined type into the component
is now dependent on the "name" that the component has for the type,
basically if the function used `$h1` or `$h2`. This overall means that
the type that the component assigned for a function is significant as
part of lifting and lowering. Hence the rationale for this commit,
threading around this type information.
The major change in this commit is updates to the `Lift` and `Lower`
traits. Previously they took a mishmash of parameters and now they
needed to take more parameters, so I've updated them with:
* `Lift` operations take a `&LiftContext<'_>` and an `InterfaceType` as
contextual information. The context stores the store, the options, and
type information. The `InterfaceType` is the type that's being lifted,
which would indicate which resource table to load from for example.
* `Lower` operations now take a `&mut LowerContext<'_, T>` and an
`InterfaceType`. The `LowerContext` is similar to its lift cousin
where it stores the store, options, and type information.
The different context passed in to `lift` and `load`, for example, is no
longer distinguished and both simply take a `&LiftContext<'_>` which
simplifies things a bit.
This refactoring was pretty far reaching and touches quite a bit of the
component model implementation. This is because basically everything
deals with type information as types can be recursively nested in one
another. I've taken the liberty to make code continue to be
ergonomic/understandable where appropriate so some "shapes" of code are
now different to continue to accommodate readability and
maintainability.
Finally it's worth noting that this should not have any actual function
impact on components running today (or tomorrow). User-facing APIs
haven't changed at all here and it's just the guts that are refactored.
One unfortunate aspect, though, is that this is going to be a small perf
hit on lifting/lowering due to the fact that type information
essentially needs to be "iterated" over during the lifting/lowering
process. This iteration involves index lookups in `&ComponentTypes`
along with assertions that when you lower `Vec<T>` that the type is
`InterfaceType::List(i)`. These assertions should always succeed, and in
theory could become some sort of `unreachable_unchecked` in the future,
but for now it's all left as safe checks/panics for us to optimize at a
later date if necessary.
* Fill out a TODO comment
* Fill out more comments
Fixing typo of wasmtime_wasi_nn -> wasmtime_wasi_crypto that was causing
a build error when building the bench-api with the wasi-crypto feature
enabled.
Commons up some code paths and sets the stage for other architectures. This
should have fewer calls back and forth between architecture specific and
independent bits of code, which I have found hard to keep track of. Now,
lowering tail calls is done in architecture specific code that can call out to
architecture independent helpers as needed. Before it was architecture
independent code that would call architecture specific hooks that would call
architecture independent helpers. Too much stuff split across too many
layers. This new approach removes at least one layer of indirection and
unnecessarily confusing abstraction.
This commits removes an assert that checked that the stack pointer
position at the end of a call should be greater or equal than the
position registered at the callsite.
Even though this is true in most cases, there are cases in which this is
invariant is not met and as well as there are cases in which the stack pointer will
inevitably be greater than the position registered at callsite:
1. When the call setup doesn't spill any values and instead it
only consumes memory values from the value stack, the stack pointer
can end up being less than what it was at the callsite.
2. When the call setup spills values that are not going to be consumed
by the call (not used as params to the function) the stack pointer
position can end up being greater than what it was at the callsite.
The assert was originally introduced to ensure the right deallocation of
stack space consumed by the call, and it could be improved by applying
the heuristics mentioned above, but I prefer to remove it since we
already assert when emitting the epilogue that both the value stack and
machine stack are in the correct state when fishing compilation.
This change includes an extra test in which the original invariant
doesn't hold (case 2 described above occurs).
* Remove Wasmtime ABIs from Cranelift
This commit removes the `Wasmtime*` family of ABIs from Cranelift. These
were originally added to support multi-value in Wasmtime via the
`TypedFunc` API, but they should now no longer be necessary. In general
this is a higher-level Wasmtime concern than something all backends of
Cranelift should have to deal with.
Today with recent refactorings it's possible to remove the reliance on
ABI details for multi-value and instead codify it directly into the
Cranelift IR generated. For example wasm calls are able to have a
"purely internal" ABI which Wasmtime's Rust code doesn't see at all, and
the Rust code only interacts with the native ABI. The native ABI is
redefined to be what the previous Wasmtime ABIs were, which is to return
the first of a 2+ value return through a register (native return value)
and everything else through a return pointer.
* Remove some wasmtime_system_v usage in tests
* Add back WasmtimeSystemV for s390x
* Fix some docs and references in winch
* Fix another doc link
Pass in the calling convention so we can make an informed decision of which
register to use as a stack-limit temporary register. We need to choose different
registers on x64, for example, when using the `tail` calling convention vs
system v calling convention.
This is a leftover from #3302 where we used to inject a special `vmctx`
struct if the test requested it. We've removed that capability in #5386
but accidentally left this in, which caused some weird handling of these
test invocations.
* poll.wit: we can returl a list<bool> now
* adapter: fixes for poll-oneoff returning list<bool>
* wasi preview2: fixes for poll-oneoff returning list<bool>
* adapter: manually import poll-oneoff to avoid pulling in std to allocate return vec
* comment describing the skip functions
* preview2: refactor WasiCtxBuilder impl, and make WasiCtx fields private
The optional-fields WasiCtxBuilder was an intermediate stepping stone
which has outlived its usefulness - there is now only one sensible
default value for each field, so we no longer need to expose the Default
(empty) constructor.
There is no need for the WasiCtx::builder method being an alias for
default, and it creates user confusion.
Finally, the WasiCtx itself is a private implementation detail for use
in this crate only. The WasiCtxBuilder is the only way to customize
its contents. We don't want to let users modify the fields of WasiCtx
after it has been used by a wasm guest, because the guest may have made
assumptions that fields won't change - e.g. the stderr field is fetched
once by the adapter and assumed to always be the same, environment
variables are copied into libc once and assumed to always be the same.
* fix tests
On x86_64, for example, the temporary register was previously hardcoded to `r11`
which was available as a non-argument, caller-saved register on both system v
and fast call. However in the `tail` calling convention it is used as an
argument register.
This commit plumbs through the calling convention to where we are choosing a
temporary register for stack probes so that we can make an informed decision.
Fixes#6640
When the size of the tail callee's and tail caller's stack arguments is the
same, we don't need to copy the return address and then write it to the stack
again, since its original location will also be its final location. That is, we
were previously reading a value and then writing it back to the exact same
location.
* [wit-bindgen] provide more control over type ownership
This replaces the `duplicate_if_necessary` parameter to
`wasmtime::component::bindgen` with a new `ownership` parameter which provides
finer-grained control over whether and how generated types own their fields.
The default is `Ownership::Owning`, which means types own their fields
regardless of how they are used in functions. These types passed by reference
when used as parameters to guest-exported functions. Note that this also
affects how unnamed types (e.g. `list<list<string>>`) are passed: using a
reference only at the top level (e.g. `&[Vec<String>]` instead of `&[&[&str]]`,
which is more difficult to construct when using non-`'static` data).
The other option is `Ownership::Borrowing`, which includes a
`duplicate_if_necessary` field, providing the same code generation strategy as
was used prior to this change.
If we're happy with this approach, I'll open another PR in the `wit-bindgen`
repo to match.
This also fixes a bug that caused named types to be considered owned and/or
borrowed when they shouldn't have been due to having fields with unnamed types
which were owned and/or borrowed in unrelated interfaces.
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* fix test breakage
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
---------
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* Lean on wasmparser's type information when translating components
Before this commit Wasmtime would build up its own representation of
type information independently of wasmparser, effectively duplicating
things such as type scopes and managing indices. This to some degree is
required because Wasmtime's type information is serialized into compiled
images and wasmparser's information isn't easily serialized. With the
advent of resources in components, however, the task of doing this
correctly has become much more difficult.
When component translation was first written it was more difficult to
acquire type information from `wasmparser::Validator`. Nowadays there's
helpful type information exposed every step of the way which makes it
much easier to get at the types of items while we're translating rather
than only at the very end (or not at all). This is one of the
motivations for this commit where now it's possible to avoid duplicating
the work `wasmparser` is doing whereas before it wasn't as easy.
Additionally with resources in the component model they perform namely a
number of involved type substitutions during instantiation of arbitrary
components which isn't implemented in Wasmtime today. Some of this will
be required for Wasmtime to correctly implement resources so instead of
doing all that again I've decided to replace Wasmtime's management of
types with wasmparser's management of types.
The main difference in this commit is that Wasmtime no longer tracks
type information during translation and that conversion into Wasmtime's
type hierarchy now has a different entry point. Previously conversion
would happen based on raw types read from the wasm file (think
index-based things) whereas now translation happens on `wasmparser`'s
parsed and validated hierarchy of types (think ID-based things rather
than index-based things). This makes translation slightly more involved
but overall it's largely performing the same work.
One gotcha with this PR is that core wasm modules using GC types and
typed function references could theoretically have worked previously but
they no longer work. It turns out that `wasmparser` is not correctly
surfacing type information in components for core modules that use GC
types, namely because `wasmparser`'s validated type hierarchy uses the
same core wasm types as what's read raw from the type section. This in
turn means that the index-based format can't be resolved. This is a bug
in `wasmparser` which will need resolving but is a big chunk of work to
take on, so for now the component model will panic on these sorts of
modules (which are disabled by default anyway).
Overall the end-goal of this commit is to ease the implementation of
resources a bit by heavily relying on `wasmparser`'s understanding of
resources, chiefly the functionality of performing type substitutions on
subcomponent instantiations.
* Fix a warning
* Fix factc build
* Refactor compilation of component-related functions
Extract some common functionality out to helper functions to reduce the
boilerplate involved in compiling component-related functions. This is
in preparation for upcoming resource-related work which adds more
functions to compile.
* Fix a typo
* aarch64: Fix `AuthenticatedRet` when stack bytes are popped
This commit fixes an accidental issue with #6478 where when pointer
authentication was enabled and stack bytes are being popped during a
return this didn't work. In this situation an authenticated return
instruction was used, such as `retab`, and no extra stack bytes were
popped. The fix here is to use the non-`retab` path which handles stack
bytes being popped if there are stack bytes to pop.
Closes#6567
* Still use `retab` for `is_hint: false`
The memory-store format of `pextrw` requires SSE4.1 despite `pextrw`
itself only requiring SSE2. This commit updates this lowering to require
an extra feature.
* x64: Fix ISA requirement for `pmaddubsw`
This was erroneously listed as SSE2 but this is actually an SSSE3
instruction. This commit updates the ISA requirement as well as related
lowerings to have everything work with an SSE2 baseline as well which is
mostly plumbing to make sure the relaxed-simd dot product instructions
use it when it's available but otherwise fall-back to the deterministic
lowering.
* Add require ssse3 feature
* Fix preexisting wat test
This commit removes a setting for Cranelift which I've found a bit
confusing historically and I think is no longer necessary. The setting
is currently documented as enabling SIMD instructions, but that only
sort of works for the x64 backend and none of the other backends look at
it. Historically this was used to flag to Cranelift that a higher x64
baseline feature set is required for codegen but as of #6625 that's no
longer necessary.
Otherwise it seems more Cranelift-like nowadays to say that vector
instructions generate SIMD instructions where non-vector instructions
probably don't, but may still depending on activated CPU features. In
that sense I'm not sure if a dedicated `enable_simd` setting is still
motivated, so this PR removes it.
This renames some features in the x86 backend such as `use_avx_simd` to
`use_avx` since the `_simd` part is no longer part of the computation
now that `enable_simd` is gone.
* issue-6592: args accumulator match entry block params
* Formatting
* Formatting
* added push_non_formal to aarch64 and x86 implementations of add_ret_area_ptr case in compute_arg_locs
---------
Co-authored-by: Cameron <cameron@Camerons-MacBook-Pro.local>
All instructions in Cranelift now have lowerings for SSE2 as a baseline,
even if they're not exactly the speediest things in the world. This
enables lowering the baseline required for the SIMD proposal for
WebAssembly to SSE2, the base features set of x86_64. Lots of tests were
updated here to remove explicit `has_foo=false` annotations as they no
longer have any effect.
Additionally fuzzing has been updated to enable disabling `sse3` and
`ssse3` which will help stress-test all previously-added lowerings.
* Add a `set_random` function to `WasiCtxBuilder`.
Add a `set_random` function to `WasiCtxBuilder`, allowing users that
construct a default interface via `WasiCtxBuilder::default()` or
`WasiCtx::builder()` to initialize the `random` state.
Fixes#6576.
* Rename `set_random_for_testing` to `set_secure_random_to_custom_generator`
Remove the `cfg(test)`, and add comments advising users of the potential
hazards.
* rustfmt
* Fix documentation links.
Alex Crichton
wasmtime-publish
Trevor Elliott
Chris Fallin
Saúl Cabrera
Afonso Bordado
Ryan Levick
Stephen Doyle
Nick Fitzgerald
Pat Hickey
Joel Dice
Bobby Holley
Cameron Harris
Dan Gohman
Sofia