* Rework the riscv64 prologue to spill clobbers with positive offsets to SP
* Also modify gen_clobber_restore to load positive offsets from SP
This fixes a bug: we're not supposed to write below SP, as a signal
handler is allowed to clobber anything below it. Writing our clobbers to
negative offsets of SP leaves a window open where they might be
clobbered if we receive a signal while executing the prologue or
epilogue.
* Fix x64 clobber offsets with unwind info
The stack offsets provided to UnwindInst::SaveReg instructions are
relative to the beginning of the clobber area of the stack, and on main
the x64 backend computes those offsets by assuming a SP-relative offset
and then adjusting it. This is problematic, as any changes to the frame
layout might require updates in two locations: where we compute the
initial SP offset, and where we compute the adjustment to make the
offset relative to the clobber area again.
This PR fixes this issue by making the offset we track in the loop
always relative to the beginning of the clobber area, and adding the
offset from SP only when emitting the store. This relies on the
assumption that the clobber area is sized according to the constraints
of the registers stored within it (16-byte alignment is required for
floating point registers) but this assumption is enforced by existing
behavior in compute_clobber_size.
This also fixes a long-standing bug with floating point register spill
offsets, as the offset was saved before the alignment to a 16-byte
offset was made, leaving the resulting unwind info potentially pointing
at the wrong offset.
* Make the same refactoring to clobber_restore
This reverts the key parts of e3a08d4c40
(#8151), because it turns out that we didn't need that abstraction.
Several changes in the last month have enabled this:
- #8292 and then #8316 allow us to refer to either incoming or outgoing
argument areas in a (mostly) consistent way
- #8327, #8377, and #8383 demonstrate that we never need to delay
writing stack arguments directly to their final location
prtest:full
* Fix pcc error on some workloads
- `attach_constant_fact` should create facts for constants based on their type
- `MovK` generates incorrect fact ranges
* Update cranelift/codegen/src/isa/aarch64/pcc.rs
Co-authored-by: Jamey Sharp <jamey@minilop.net>
* adjust test format
---------
Co-authored-by: Jamey Sharp <jamey@minilop.net>
The `compute_addr` helper only works on real address modes, and one of
its callers checks that it only passes a real address mode to it. On x64
we have different Rust types for address modes which are real versus
synthetic, so we can simplify a little by reflecting this requirement in
the types.
Since the vector of clobbered registers is sorted first by class, we can
return slices of that vector instead of allocating a temporary vector
for each class.
This relies on the `RealReg` sort order and on the frame layout's
`clobbered_callee_saves` being sorted in natural order.
Semantically, a "real register" is supposed to be a physical register,
so let's use the type dedicated for that purpose.
This has the advantage that `PReg` is only one byte while `VReg` is four
bytes, so the arrays where we record collections of `RealReg` become
smaller.
There was an implementation of `From<VReg> for RealReg` which was not a
sensible conversion, because not all `VReg`s are valid `RealReg`s. I
could have replaced it with a `TryFrom` implementation but it wasn't
used anywhere important, so I'm just deleting it instead.
Winch was using that `VReg`->`RealReg` conversion, but only in the
implementation of another conversion that was itself unused, so I'm
deleting that conversion as well. It's easy to implement correctly (the
Winch `Reg` type is identical to `RealReg`, so all conversions for the
latter are readily available) but as far as I can tell Winch doesn't
need to use Cranelift's register wrappers or RA2's virtual register
type, so it's simpler to just delete those conversions.
The riscv64 backend was relying on quirks of the existing conversions
between `RealReg` and `VReg` when emitting clobber saves and restores.
Just using the generic conversions between `RealReg` and `Reg` is
simpler and works correctly with the rest of these changes.
* Ensure that indirect tail calls on x64 don't clobber the destination register
* Add the fuzzbug to the x64 isa tests as well
* Use `reg_early_def` in the ReturnCall case as well
This method isn't actually correct if `split_off` is used on the
returned values again as it's missing some extra indexing processing.
That being said nothing in Wasmtime currently uses it, so remove it
entirely instead of keeping it around.
* Avoid copying the frame for tail calls on aarch64
To mirror the implementation in the x64 backend, switch to eagerly
reserving enough space in the incoming argument area for any tail call
present in the function being compiled.
prtest:macos-arm64
Co-authored-by: Jamey Sharp <jsharp@fastly.com>
* Add a TODO about how to remove the direct SP manipulation
* Fix comments
---------
Co-authored-by: Jamey Sharp <jsharp@fastly.com>
`Callee::probestack_min_frame` was always set to the guard-page size.
That is the same as the `guard_size` local in `gen_prologue`, which was
also the only place which used `probestack_min_frame`. So don't even
bother storing it, just compute it from the flags as needed.
Also, remove the long-obsolete `probestack_func_adjusts_sp` setting,
which hasn't been used since at least 2021. We may actually want to do
something like what that setting described, but even if we do, it should
be a choice the backend makes rather than a global setting.
* c-api: Create `RootScope` where necessary
This commit changes the `wasmtime_val_t::{from_val, to_val}` methods to
take a `RootScope` instead of any `AsContextMut`. This then required a
number of refactorings in callers to ensure that a `RootScope` was
created for any function that needed one. This is required to ensure
that GC references in the C API aren't forced to live for the entire
lifetime of the store.
This additionally added `*_unrooted` variants which do the same thing
but don't require `RootScope`. This was needed for when the C API calls
out to the embedder through a function call because a new `RootScope`
wouldn't work for return values (they're bound to a scope within the
closure when we want them to outlive the closure). In these situations
though we know a `RootScope` is already present at the entrypoint.
Closes#8367
* Review comments
`return_call` instructions reuse the incoming argument area of the caller's
frame. As such if the caller's incoming argument area is not exactly the right
size for the callee, some resizing will need to take place to ensure that the
return address, frame pointer, clobbers, and stack slots don't get overwritten.
The current solution on the main branch for the x64 backend is to explicitly
resize the frame via `ShrinkArgumentArea` or `GrowArgumentArea` right before
the `return_call` arguments are written to the stack, ensuring that there is
sufficient space. While this does work, it does make a `return_call` more
expensive when the resizing is necessary.
To simplify this, we instead resize the incoming argument area in the function
prologue to accommodate the largest possible argument area of any `return_call`
instruction in the function. We then shrink back down when necessary before an
individual `return_call`. This simplifies the implementation of tail calls on
x86_64, as we no longer need to move the entire frame, just the return address
before we jump to the tail-callee.
Co-authored-by: Jamey Sharp <jsharp@fastly.com>
* Use `--locked` on all `cargo install` in CI
Prevents any updates to rustc or crates from accidentally causing issues
by ensuring that the same set of deps is used over time.
* Remove rust/WASI markdown parser example
The documentation referring to this example was removed in #6994 and
that forgot to remove this as well. This example is building without a
lock file which is causing issues in #8368.
* c-api: Fix alignment of `wasmtime_val_*`
This commit fixes an issue where `wasmtime_val_raw_t` had an incorrect
alignment. In Rust `ValRaw` contains a `u128` which has an alignment of
16 but in C the representation had a smaller alignment meaning that the
alignment of the two structures was different. This was seen to cause
alignment faults when structure were passed from C++ to Rust, for
example.
This commit changes the Rust representation of `ValRaw`'s `v128` field
to do the same as C which is to use `[u8; 16]`. This avoids the need to
raise the alignment in C which appears to be nontrivial. Cranelift is
appropriately adjusted to understand that loads/stores from `ValRaw` are
no longer aligned. Technically this only applies to the `v128` field but
it's not too bad to apply it to the other fields as well.
* Try alternate syntax for alignof
* Tidy up some headers related to shared memory
* Don't declare an anonymous `struct wasmtime_sharedmemory`, instead
`#include` the actual definition.
* Fix an issue where a header in `sharedmemory.h` referred to a type in
`extern.h` which wasn't `#include`'d. This function,
`wasmtime_sharedmemory_into_extern`, additionally isn't necessary as
it's no different than manually constructing it. Fix this by removing
this function.
* Run clang-format
cranelift-fuzzgen unconditionally runs the NaN Canonicalization pass on all functions that it generates. This is so that we can ensure that when running in the interpreter vs natively we get the same bitpattern for all NaN's.
Until now we just picked a random ISA (the host ISA), disabled the verifier and ran the pass with that. This was because the ISA didn't really matter for the passes that we wanted to run.
In #8313 the ISA now drives some codgen decisions for the NaN Canonicalization pass. Namely, if the ISA supports Vectors, it tries to use that.
In #8359 there was a fuzz bug reported where fuzzgen generated vector code for RISC-V without the `has_v` flag, something that should *never* happen, because we simply cannot compile that code.
It turns out that fuzzgen did not generate vector code itself. But since we were passing the host ISA to the nan canonicalization pass, it assumed that it could use vectors and did so. But the actual target isa did not support vectors.
To fix this, we now correctly pass the target isa that we are building a function for.
Restores support for `externref` in `wasmtime_val_t`, methods for manipulating
them and getting their wrapped host data, and examples/tests for these things.
Additionally adds support for `anyref` in `wasmtime_val_t`, clone/delete methods
similar to those for `externref`, and a few `i31ref`-specific methods. Also adds
C and Rust example / test for working with `anyref`.
* Fix running wasi-common tests on CI
Turns out we haven't been running wasi-common tests for some time in CI
and they've started failing. Force enable the test at all times and then
fix the test failures. The test failures here were introduced in #8277
and weren't caught due to the test not running and the fix was to relax
the implementation of `fd_pread` to avoid taking multiple mutable
borrows.
* Fix CI
Currently we've got a good number of WASI tests and they're all
relatively large. We also can run a single test in up to three
configurations:
* As-is with a module
* As a component in "sync" mode
* As a component in "async" mode
In debug mode compilation of all these modules can take a significant
chunk of time (20-30s in total for test suites) This commit updates
these test suites to use an in-memory per-process incremental cache
backed by a simple `Mutex<HashMap>`. This gives some good speedups in
debug mode, locally the wasi-common, wasmtime-wasi, and
wasmtime-wasi-http test suites were reduced from 32 to 17 seconds. I'd
expect larger speedups on less-parallel machines such as our CI.
* wasmtime-c-api: switch from using wasi-common to wasmtime-wasi
* Fix WasiP1Ctx references, and stop eagerly opening dirs for preopens
* Add OutputFile to skip async writes in stdout/stderr
---------
Co-authored-by: Trevor Elliott <telliott@fastly.com>
Previously, `wasmtime_types::EngineOrModuleTypeIndex` had a raw `u32` for its
engine-level-canonicalized variant. This change allows it to store a
`VMSharedTypeIndex` directly, giving us some additional static assurance that we
aren't messing up our different index types and lets us remove a ton of
conversions back and forth between raw `u32`s and `VMSharedTypeIndex`.
These are all passing now that we support typed function references in the
embedder API and our `wasmparser` and `wast` deps have been updated to versions
that fix the issues referenced in the old comments.
* c-api: Better differentiate between `wasm.h` and `wasmtime.h` APIs
This renames some types and adds some type aliases to help us better distinguish
between `wasm.h` APIs and `wasmtime.h` APIs, primarily for `Store`-related
types. In general, `WasmFoo` is related to `wasm.h` and `WasmtimeFoo` is related
to `wasmtime.h`.
* `StoreRef` -> `WasmStoreRef`
* Introduce the `WasmStore[Data]` and `WasmStoreContext[Mut]` aliases
* `StoreData` -> `WasmtimeStoreData`
* `CStoreContext[Mut]` -> `WasmtimeStoreContext[Mut]`
* Introduce the `Wasmtime{Store,Caller}` aliases
* c-api: Improve non-support of GC references in `wasm.h` APIs
A couple small tweaks: error message improvements, exhaustive matching, etc...
* Document HttpResult and HttpError
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Move error functions into error module
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Document the body module (and rearrange slightly)
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Make FinishMessage as it does not need to be public
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Rename as_body_error to as_body_size_error
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Make http_impl module non-public since it doesn't need to be
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Document the io module
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Document the proxy module
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Make types_impl module non-public since it doesn't need to be
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* WasiHttpCtx and WasiHttpView documented inline
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Add top level crate documentation
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Document proxy add_to_linker with example
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Deny missing docs
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Make HostOutgoingBody::body_output_stream non public
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
---------
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
This renames some types and adds some type aliases to help us better distinguish
between `wasm.h` APIs and `wasmtime.h` APIs, primarily for `Store`-related
types. In general, `WasmFoo` is related to `wasm.h` and `WasmtimeFoo` is related
to `wasmtime.h`.
* `StoreRef` -> `WasmStoreRef`
* Introduce the `WasmStore[Data]` and `WasmStoreContext[Mut]` aliases
* `StoreData` -> `WasmtimeStoreData`
* `CStoreContext[Mut]` -> `WasmtimeStoreContext[Mut]`
* Introduce the `Wasmtime{Store,Caller}` aliases
* Explicitly document why `blocking_mode` is being ignored.
* Don't write-in-a-loop to doing a partial write and then returning an
error.
* Spawn a background read/write when the embedding cannot block the
current thread.
* Don't panic on overflows for addition.
* Fix panic on reads with shared memory
Follow-ups to #8303
* Don't gate on `cargo vet` for PRs
Gate the `cargo vet` step for the merge queue by default but for PRs
don't fail the PR's CI and instead allow entering the queue with a
failing `cargo vet` entry. This should help make it easier to land a
`cargo vet` entry on `main` outside of a PR while not requiring the PR
is rebased.
* Fix syntax
* Trigger cargo vet failures
* Fix conditional
* Try to fix conditional
* More changes
* Revert "Trigger cargo vet failures"
This reverts commit 14ba8cb52c.
* Update Cargo.lock but also have vet pass
* Try different syntax
* Revert "Update Cargo.lock but also have vet pass"
This reverts commit 1679838848.
* Remove debugging
* Spruce up some comments
* Update docs
* Force users to construct WasiHttpCtx through a constructor
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Rename OutgoingRequest to OutgoingRequestConfig and breakout actual request
This rename makes the role of the type easier to understand and draws a
bigger difference between it and HostOutgoingRequest.
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Remove the authority field from the OutgoingRequestConfig
The authority is already encoded in the request itself.
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Add doc comments to the major request/response resource types
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Add more doc comments to resource types
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Rename `IncomingResponseInternal` to `IncomingResponse`
Calling a public type "internal" seems strange. This also adds docs to
the type.
* Small additional changes
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Refactor HttpView::send_request
Don't require implementors to do resource management.
* Make worker task optional
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Fix CI
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
---------
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
The worker is a handle to a tokio task where the connection is being
read from. It should be kept alive in order to continue to be able to
read the response body. However, the worker _is_ kept alive on
`HostIncomingBody`. Arguably this the exact place where the task should be kept
as once that type is dropped there's no longer a need to keep the worker
around.
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
This commit is a follow-up to https://github.com/bytecodealliance/wasmtime/pull/8248, in which the `winch-tools` tool was removed.
It's in my todo list to follow-up with a PR with more in-deptch documentation about Winch.
By default previously all return types were wrapped in
`wasmtime::Result<T>` to enable any import to return a trap to the wasm
guest. This is a fair bit of boilerplate, however, and it's easy to
accidentally turn a normal error into a trap. This is additionally
somewhat of a power-user method as many imports probably don't end up
wanting to trap.
This commit adds a new configuration option to `bindgen!`:
`trappable_imports`, and switches the default to `false`. The previous
behavior can be recovered with `trappable_imports: true`.
Trevor Elliott
Jamey Sharp
Feilong Jiang
Alona Enraght-Moony
Caleb Schoepp
Nick Fitzgerald
Alex Crichton
Peter Huene
Jacob Snell
Edoardo Vacchi
Ryan Levick
Afonso Bordado
Pat Hickey
liutao
Saúl Cabrera