* handle interface functions correctly in `component::Linker::func_new`
Many months ago, I implemented `func_new`, but only supporting top-level
function imports. If you tried to link a host function under an imported
interface, it would mistakenly treat it as a top-level function and either error
out if it couldn't find a corresponding type definition in the passed
`&Component`; or, if it found a top-level function that happened to have the
same name, it would use that type (which would coincidentally work if the type
happens to match, but lead to a runtime error later on otherwise).
This fixes the issue by looking up the correct component instance when necessary
and getting the type from there.
Note I've made no effort to optimize for performance here. Happy to revisit
that if there's a need.
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* print names in `func_new`, not intern indexes
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* use `Vec` instead of linked list in `LinkerInstance`
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
---------
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
* Fix a typo in an `unsafe impl Send/Sync`
This commit fixes a mistake in the `PrePatchedFuncRef` type where it has
an `unsafe impl` to make it send/sync but the target of the impl was
mistakenly `InstancePre<T>`
Note that this doesn't actually have any impact on the send/sync-ness of
`InstancePre<T>` since it's not storing an instance of `T`, so it's
always `Send`/`Sync` anyway. Nevertheless this reduces the scope of
unsafety slightly as was originally intended.
* Remove the `PrePatchedFuncRef` entirely
No longer necessary for `Send`/`Sync`-ness any more
* winch(x64): Fix a couple of issues with control flow
This change fixes two issues with the control flow implementation found
when working on https://github.com/bytecodealliance/wasmtime/pull/6610
The two fixes included in this change are:
1. Correctly handle the stack pointer at jump sites: when emitting an
unconditional jump, the stack pointer might be left unbalanced due to
register spilling, this could cause invalid memory accesses.
2. Explicitly track the exit label of the if block: previously the
continuation label of the if block was implicitly treated as the exit
label, which would cause undefined behaviour in programs that use
unconditional branches in the `if` and `else` branches. This slight
disadvantage of the approach in this change is that it involves special casing the
emission of the end of an if block (without else) to account for the
continuation label, since the continuation label is only naturally
bound when there's an else branch.
* Expand on how `pop_sp_for_branch` works
* Update v8 and proc-macro2 dependencies
Gets them both compiling on the latest nightly so we can unpin the Rust
compiler version in OSS-Fuzz.
* Update nightly in CI
* Run `cargo vet` on automated version bumps
As shown in the original CI of #6686 the recent changes for `cargo vet`
0.8.0 mean that the previous release process no longer works since it
requires changes to the lock file for `cargo vet`. This updates the
version bumping process to run `cargo vet` as part of the committed
changes which hopefully will get everything to succeed. To ensure that
the same version of `cargo vet` is used in both locations the
installation procedure was extracted into its own separate little action.
* Configure shell to run in
* Add WasiCtxBuilder setters for the two clock types
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Move two clocks to dedicated fields in WasiCtx
* Rename clock traits to Host prefix convention
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Fix tests
---------
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Shepherd along type information in lifting/lowering
This commit is a large refactor to the component runtime of Wasmtime to
shepherd along type information when lifting and lowering values.
Previously lifting and lowering would assume type information given
context such as "surely lowering a `u32` must lower into the type
`InterfaceType::U32`" or "lowering a `Val` works as it knows its own
type". This is still true, and this commit isn't changing these
features. The rationale for this commit instead stems from the upcoming
implementation of resources in Wasmtime.
Resources are trickier than all existing types in Wasmtime because what
exactly is the type of a resource depends on who you're asking. For
example the host might have one type called `http::Headers` but a
component could import it as two distinct types:
(component
(import "headers1" (type $h1 (sub resource)))
(import "headers2" (type $h2 (sub resource)))
;; ...
)
in the above component the `$h1` and `$h2` types will each get their own
table at runtime for managing their state. This means that if the host
instantiates the component with `http::Headers` supplied as both types
then the same type on the outside maps to two different types inside.
This means that the lowering of a host-defined type into the component
is now dependent on the "name" that the component has for the type,
basically if the function used `$h1` or `$h2`. This overall means that
the type that the component assigned for a function is significant as
part of lifting and lowering. Hence the rationale for this commit,
threading around this type information.
The major change in this commit is updates to the `Lift` and `Lower`
traits. Previously they took a mishmash of parameters and now they
needed to take more parameters, so I've updated them with:
* `Lift` operations take a `&LiftContext<'_>` and an `InterfaceType` as
contextual information. The context stores the store, the options, and
type information. The `InterfaceType` is the type that's being lifted,
which would indicate which resource table to load from for example.
* `Lower` operations now take a `&mut LowerContext<'_, T>` and an
`InterfaceType`. The `LowerContext` is similar to its lift cousin
where it stores the store, options, and type information.
The different context passed in to `lift` and `load`, for example, is no
longer distinguished and both simply take a `&LiftContext<'_>` which
simplifies things a bit.
This refactoring was pretty far reaching and touches quite a bit of the
component model implementation. This is because basically everything
deals with type information as types can be recursively nested in one
another. I've taken the liberty to make code continue to be
ergonomic/understandable where appropriate so some "shapes" of code are
now different to continue to accommodate readability and
maintainability.
Finally it's worth noting that this should not have any actual function
impact on components running today (or tomorrow). User-facing APIs
haven't changed at all here and it's just the guts that are refactored.
One unfortunate aspect, though, is that this is going to be a small perf
hit on lifting/lowering due to the fact that type information
essentially needs to be "iterated" over during the lifting/lowering
process. This iteration involves index lookups in `&ComponentTypes`
along with assertions that when you lower `Vec<T>` that the type is
`InterfaceType::List(i)`. These assertions should always succeed, and in
theory could become some sort of `unreachable_unchecked` in the future,
but for now it's all left as safe checks/panics for us to optimize at a
later date if necessary.
* Fill out a TODO comment
* Fill out more comments
Fixing typo of wasmtime_wasi_nn -> wasmtime_wasi_crypto that was causing
a build error when building the bench-api with the wasi-crypto feature
enabled.
Commons up some code paths and sets the stage for other architectures. This
should have fewer calls back and forth between architecture specific and
independent bits of code, which I have found hard to keep track of. Now,
lowering tail calls is done in architecture specific code that can call out to
architecture independent helpers as needed. Before it was architecture
independent code that would call architecture specific hooks that would call
architecture independent helpers. Too much stuff split across too many
layers. This new approach removes at least one layer of indirection and
unnecessarily confusing abstraction.
This commits removes an assert that checked that the stack pointer
position at the end of a call should be greater or equal than the
position registered at the callsite.
Even though this is true in most cases, there are cases in which this is
invariant is not met and as well as there are cases in which the stack pointer will
inevitably be greater than the position registered at callsite:
1. When the call setup doesn't spill any values and instead it
only consumes memory values from the value stack, the stack pointer
can end up being less than what it was at the callsite.
2. When the call setup spills values that are not going to be consumed
by the call (not used as params to the function) the stack pointer
position can end up being greater than what it was at the callsite.
The assert was originally introduced to ensure the right deallocation of
stack space consumed by the call, and it could be improved by applying
the heuristics mentioned above, but I prefer to remove it since we
already assert when emitting the epilogue that both the value stack and
machine stack are in the correct state when fishing compilation.
This change includes an extra test in which the original invariant
doesn't hold (case 2 described above occurs).
* Remove Wasmtime ABIs from Cranelift
This commit removes the `Wasmtime*` family of ABIs from Cranelift. These
were originally added to support multi-value in Wasmtime via the
`TypedFunc` API, but they should now no longer be necessary. In general
this is a higher-level Wasmtime concern than something all backends of
Cranelift should have to deal with.
Today with recent refactorings it's possible to remove the reliance on
ABI details for multi-value and instead codify it directly into the
Cranelift IR generated. For example wasm calls are able to have a
"purely internal" ABI which Wasmtime's Rust code doesn't see at all, and
the Rust code only interacts with the native ABI. The native ABI is
redefined to be what the previous Wasmtime ABIs were, which is to return
the first of a 2+ value return through a register (native return value)
and everything else through a return pointer.
* Remove some wasmtime_system_v usage in tests
* Add back WasmtimeSystemV for s390x
* Fix some docs and references in winch
* Fix another doc link
Pass in the calling convention so we can make an informed decision of which
register to use as a stack-limit temporary register. We need to choose different
registers on x64, for example, when using the `tail` calling convention vs
system v calling convention.
This is a leftover from #3302 where we used to inject a special `vmctx`
struct if the test requested it. We've removed that capability in #5386
but accidentally left this in, which caused some weird handling of these
test invocations.
* poll.wit: we can returl a list<bool> now
* adapter: fixes for poll-oneoff returning list<bool>
* wasi preview2: fixes for poll-oneoff returning list<bool>
* adapter: manually import poll-oneoff to avoid pulling in std to allocate return vec
* comment describing the skip functions
* preview2: refactor WasiCtxBuilder impl, and make WasiCtx fields private
The optional-fields WasiCtxBuilder was an intermediate stepping stone
which has outlived its usefulness - there is now only one sensible
default value for each field, so we no longer need to expose the Default
(empty) constructor.
There is no need for the WasiCtx::builder method being an alias for
default, and it creates user confusion.
Finally, the WasiCtx itself is a private implementation detail for use
in this crate only. The WasiCtxBuilder is the only way to customize
its contents. We don't want to let users modify the fields of WasiCtx
after it has been used by a wasm guest, because the guest may have made
assumptions that fields won't change - e.g. the stderr field is fetched
once by the adapter and assumed to always be the same, environment
variables are copied into libc once and assumed to always be the same.
* fix tests
On x86_64, for example, the temporary register was previously hardcoded to `r11`
which was available as a non-argument, caller-saved register on both system v
and fast call. However in the `tail` calling convention it is used as an
argument register.
This commit plumbs through the calling convention to where we are choosing a
temporary register for stack probes so that we can make an informed decision.
Fixes#6640
When the size of the tail callee's and tail caller's stack arguments is the
same, we don't need to copy the return address and then write it to the stack
again, since its original location will also be its final location. That is, we
were previously reading a value and then writing it back to the exact same
location.
* [wit-bindgen] provide more control over type ownership
This replaces the `duplicate_if_necessary` parameter to
`wasmtime::component::bindgen` with a new `ownership` parameter which provides
finer-grained control over whether and how generated types own their fields.
The default is `Ownership::Owning`, which means types own their fields
regardless of how they are used in functions. These types passed by reference
when used as parameters to guest-exported functions. Note that this also
affects how unnamed types (e.g. `list<list<string>>`) are passed: using a
reference only at the top level (e.g. `&[Vec<String>]` instead of `&[&[&str]]`,
which is more difficult to construct when using non-`'static` data).
The other option is `Ownership::Borrowing`, which includes a
`duplicate_if_necessary` field, providing the same code generation strategy as
was used prior to this change.
If we're happy with this approach, I'll open another PR in the `wit-bindgen`
repo to match.
This also fixes a bug that caused named types to be considered owned and/or
borrowed when they shouldn't have been due to having fields with unnamed types
which were owned and/or borrowed in unrelated interfaces.
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* fix test breakage
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
---------
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* Lean on wasmparser's type information when translating components
Before this commit Wasmtime would build up its own representation of
type information independently of wasmparser, effectively duplicating
things such as type scopes and managing indices. This to some degree is
required because Wasmtime's type information is serialized into compiled
images and wasmparser's information isn't easily serialized. With the
advent of resources in components, however, the task of doing this
correctly has become much more difficult.
When component translation was first written it was more difficult to
acquire type information from `wasmparser::Validator`. Nowadays there's
helpful type information exposed every step of the way which makes it
much easier to get at the types of items while we're translating rather
than only at the very end (or not at all). This is one of the
motivations for this commit where now it's possible to avoid duplicating
the work `wasmparser` is doing whereas before it wasn't as easy.
Additionally with resources in the component model they perform namely a
number of involved type substitutions during instantiation of arbitrary
components which isn't implemented in Wasmtime today. Some of this will
be required for Wasmtime to correctly implement resources so instead of
doing all that again I've decided to replace Wasmtime's management of
types with wasmparser's management of types.
The main difference in this commit is that Wasmtime no longer tracks
type information during translation and that conversion into Wasmtime's
type hierarchy now has a different entry point. Previously conversion
would happen based on raw types read from the wasm file (think
index-based things) whereas now translation happens on `wasmparser`'s
parsed and validated hierarchy of types (think ID-based things rather
than index-based things). This makes translation slightly more involved
but overall it's largely performing the same work.
One gotcha with this PR is that core wasm modules using GC types and
typed function references could theoretically have worked previously but
they no longer work. It turns out that `wasmparser` is not correctly
surfacing type information in components for core modules that use GC
types, namely because `wasmparser`'s validated type hierarchy uses the
same core wasm types as what's read raw from the type section. This in
turn means that the index-based format can't be resolved. This is a bug
in `wasmparser` which will need resolving but is a big chunk of work to
take on, so for now the component model will panic on these sorts of
modules (which are disabled by default anyway).
Overall the end-goal of this commit is to ease the implementation of
resources a bit by heavily relying on `wasmparser`'s understanding of
resources, chiefly the functionality of performing type substitutions on
subcomponent instantiations.
* Fix a warning
* Fix factc build
* Refactor compilation of component-related functions
Extract some common functionality out to helper functions to reduce the
boilerplate involved in compiling component-related functions. This is
in preparation for upcoming resource-related work which adds more
functions to compile.
* Fix a typo
* aarch64: Fix `AuthenticatedRet` when stack bytes are popped
This commit fixes an accidental issue with #6478 where when pointer
authentication was enabled and stack bytes are being popped during a
return this didn't work. In this situation an authenticated return
instruction was used, such as `retab`, and no extra stack bytes were
popped. The fix here is to use the non-`retab` path which handles stack
bytes being popped if there are stack bytes to pop.
Closes#6567
* Still use `retab` for `is_hint: false`
The memory-store format of `pextrw` requires SSE4.1 despite `pextrw`
itself only requiring SSE2. This commit updates this lowering to require
an extra feature.
* x64: Fix ISA requirement for `pmaddubsw`
This was erroneously listed as SSE2 but this is actually an SSSE3
instruction. This commit updates the ISA requirement as well as related
lowerings to have everything work with an SSE2 baseline as well which is
mostly plumbing to make sure the relaxed-simd dot product instructions
use it when it's available but otherwise fall-back to the deterministic
lowering.
* Add require ssse3 feature
* Fix preexisting wat test
This commit removes a setting for Cranelift which I've found a bit
confusing historically and I think is no longer necessary. The setting
is currently documented as enabling SIMD instructions, but that only
sort of works for the x64 backend and none of the other backends look at
it. Historically this was used to flag to Cranelift that a higher x64
baseline feature set is required for codegen but as of #6625 that's no
longer necessary.
Otherwise it seems more Cranelift-like nowadays to say that vector
instructions generate SIMD instructions where non-vector instructions
probably don't, but may still depending on activated CPU features. In
that sense I'm not sure if a dedicated `enable_simd` setting is still
motivated, so this PR removes it.
This renames some features in the x86 backend such as `use_avx_simd` to
`use_avx` since the `_simd` part is no longer part of the computation
now that `enable_simd` is gone.
* issue-6592: args accumulator match entry block params
* Formatting
* Formatting
* added push_non_formal to aarch64 and x86 implementations of add_ret_area_ptr case in compute_arg_locs
---------
Co-authored-by: Cameron <cameron@Camerons-MacBook-Pro.local>
All instructions in Cranelift now have lowerings for SSE2 as a baseline,
even if they're not exactly the speediest things in the world. This
enables lowering the baseline required for the SIMD proposal for
WebAssembly to SSE2, the base features set of x86_64. Lots of tests were
updated here to remove explicit `has_foo=false` annotations as they no
longer have any effect.
Additionally fuzzing has been updated to enable disabling `sse3` and
`ssse3` which will help stress-test all previously-added lowerings.
* Add a `set_random` function to `WasiCtxBuilder`.
Add a `set_random` function to `WasiCtxBuilder`, allowing users that
construct a default interface via `WasiCtxBuilder::default()` or
`WasiCtx::builder()` to initialize the `random` state.
Fixes#6576.
* Rename `set_random_for_testing` to `set_secure_random_to_custom_generator`
Remove the `cfg(test)`, and add comments advising users of the potential
hazards.
* rustfmt
* Fix documentation links.
Joel Dice
Alex Crichton
Saúl Cabrera
Afonso Bordado
wasmtime-publish
Andrew Brown
Ryan Levick
Stephen Doyle
Nick Fitzgerald
Pat Hickey
Bobby Holley
Cameron Harris
Dan Gohman
Sofia