* Update handling of wasm features and `*.wast` testing
This commit is an attempt at making it easier to manage the set of
variables we have in play with `*.wast` testing and wasm features.
The Cranelift and Winch backends support different sets of wasm features
at this time and historically Cranelift has also had
architecture-specific support for wasm features. This is hoped to help
simplify management of all of this in the future in addition to making
`*.wast` testing more robust. Notable changes here are:
* A `Config` no longer tracks a `WasmFeatures` but instead only tracks
explicitly disabled/enabled features. The final `WasmFeatures` is then
computed given the result of compiler configuration.
* Default-enabled features now start from nothing instead of
`wasmparser`'s defaults to avoid future breakage there.
* Each compiler configuration and/or backend now has a listed set of
"this feature may panic" wasm features. These are all disabled by
default and if explicitly enabled a `Config::validate` error is
returned.
* All `*.wast` tests are now run through both Cranelift and Winch. Tests
are now asserted to either fail or pass depending on configuration and
the whole test will fail if the result doesn't match the expectation.
* Many `gc` proposal tests which are now passing are flagged as passing
now. Failing `winch` tests are now explicitly listed instead of using
a litany of patterns.
* Change panicking features for winch
* Fix builds without a compiler
* winch: Solidify bounds check for dynamic heaps
This commit fixes and edge case for bounds checks for dynamic heaps.
https://github.com/bytecodealliance/wasmtime/pull/8157/files erroneously
tied the bounds check operation (more concretely the overflow check) to the size derived from from the heap
type. Even though offsets and access sizes are validated ahead-of-time
and bound to the heap type, in the case of overflow checking, we must
ensure that the operation size is tied to the target's pointer size to
avoid clamping the access size and offset addition, which would result
in missing an out-of-bounds memory access.
This commit also adds a disassembly test to avoid introducing
regressions in the future.
Additionally, this commit adds more comments around why `pointer_size`
is used for certain bounds checking operations.
* Update disassembly test
This commit simplifies the selection of an ABI for wasm functions now
that all Cranelift backends implement tail calls. All wasm functions use
the `Tail` calling convention except when the `winch_callable` tunable
is enabled meaning that Winch-generated functions are being called.
This then additionally simplifies the activation of the tail call
proposal. It's not unconditionally active and the same across all
compilers. The Winch compiler is updated to return an error for
unsupported instructions rather than panicking so the embedder API is
suitable for feeding unsupported modules to Winch. This means that tail
calls in Winch behaves similarly to GC in Cranelift or other unsupported
proposals like SIMD in Winch.
* Cranelift(x64): provide better panic messages for `Gpr::new(reg).unwrap()` et al
Co-Authored-By: Jamey Sharp <jsharp@fastly.com>
* Fix compile error due to bad mechanical replacement
---------
Co-authored-by: Jamey Sharp <jsharp@fastly.com>
* Add v128.const support to Winch
* Remove next_vr and vector_reg_for methods
* Adjust alignment and slot size for v128
* Forgot to update disas tests
* Update unit tests
* Use 8 byte stack slot sizes
* Fix broken unit tests, add tests for vecs, and use ty size for vecs
* winch: Improve scratch register handling
This commit doesn't introduce any new behavior. It's mostly a follow-up
to https://github.com/bytecodealliance/wasmtime/pull/7977.
This commit tries to reduce the repetitive pattern used to obtain the
scrtach register by introducing a macro similar to the existing `vmctx!`
macro.
This commit also improves the macro definition by using fully qualified
paths.
* Fix unused imports
This register was used as an additional scratch register in the context
of trampolines.
As of https://github.com/bytecodealliance/wasmtime/pull/8109,
trampolines are implemented with Cranelift, so this register alias is
unused.
* Remove Opcode from MachCallSite
It doesn't seem useful and in several locations we already use a
different Opcode from what the user actually wrote. For example for
tls_value, the implicit stackprobe or the implicit memcpy call for
struct args.
This also makes the recorded call sites more accurately represent the
actual set of call sites by avoiding special casing of pseudo
instructions that happen to include a call.
* Don't precompute the set of defs to remove from the clobbers
Instead remove the actual set of defs from the clobber set once they are
known. This reduces the risk of getting the clobber set wrong and makes
it easier to handle cases where the exact set is not known in advance.
Fixes: https://github.com/bytecodealliance/wasmtime/issues/8848
Similar to all the control instructions, any state must be explicitly
saved before emitting the code for `br_if`.
This commit ensures that live locals and registers are explicilty saved
before emitting the code for `br_if`. Prior to this commit, live
locals and registers were not saved every time causing incorrect
behavior in cases where the calculation of the conditional argument
didn't trigger a spill.
This change introduces the explicit spill after calculating the branch
condition argument to minimize memory traffic in case the conditional is
already in a register.
* upgrade to wasm-tools 0.211.1
* code review
* cargo vet: auto imports
* fuzzing: fix wasm-smith changes
* fuzzing: changes for HeapType
* Configure features on `Parser` when parsing
---------
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
* Const-propagate some offsets in `VMOffsets`
Before this commit all offsets to all fields in `VMOffsets` were stored
as fields within `VMOffset` itself. All of the fields at the start of
`VMOffsets`, however, are statically known given the pointer size.
Notably this means that the use of `HostPtr` in the runtime still was
forcing a dynamic lookup of these static offsets.
This commit refactors this to reflect all static offsets based solely on
the pointer size in the `PtrSize` trait, removing all the fields from
`VMOffsets`. All the dynamically sized fields, however, remain in
`VMOffsets`.
* Fix expected error message
* Wasmtime: Implement the custom-page-sizes proposal
This commit adds support for the custom-page-sizes proposal to Wasmtime:
https://github.com/WebAssembly/custom-page-sizes
I've migrated, fixed some bugs within, and extended the `*.wast` tests for this
proposal from the `wasm-tools` repository. I intend to upstream them into the
proposal shortly.
There is a new `wasmtime::Config::wasm_custom_page_sizes_proposal` method to
enable or disable the proposal. It is disabled by default.
Our fuzzing config has been updated to turn this feature on/off as dictated by
the arbitrary input given to us from the fuzzer.
Additionally, there were getting to be so many constructors for
`wasmtime::MemoryType` that I added a builder rather than add yet another
constructor.
In general, we store the `log2(page_size)` rather than the page size
directly. This helps cut down on invalid states and properties we need to
assert.
I've also intentionally written this code such that supporting any power of two
page size (rather than just the exact values `1` and `65536` that are currently
valid) will essentially just involve updating `wasmparser`'s validation and
removing some debug asserts in Wasmtime.
* Update error string expectation
* Remove debug logging
* Use a right shift instead of a division
* fix error message expectation again
* remove page size from VMMemoryDefinition
* fix size of VMMemoryDefinition again
* Only dynamically check for `-1` sentinel for 1-byte page sizes
* Import functions that are used a few times
* Better handle overflows when rounding up to the host page size
Propagate errors instead of returning a value that is not actually a rounded up
version of the input.
Delay rounding up various config sizes until runtime instead of eagerly doing it
at config time (which isn't even guaranteed to work, so we already had to have a
backup plan to round up at runtime, since we might be cross-compiling wasm or
not have the runtime feature enabled).
* Fix some anyhow and nostd errors
* Add missing rounding up to host page size at runtime
* Add validate feature to wasmparser dep
* Add some new rounding in a few places, due to no longer rounding in config methods
* Avoid actually trying to allocate the whole address space in the `massive_64_bit_still_limited` test
The point of the test is to ensure that we hit the limiter, so just cancel the
allocation from the limiter, and otherwise avoid MIRI attempting to allocate a
bunch of memory after we hit the limiter.
* prtest:full
* Revert "Avoid actually trying to allocate the whole address space in the `massive_64_bit_still_limited` test"
This reverts commit ccfa34a78dd3d53e49a6158ca03077d42ce8bcd7.
* miri: don't attempt to allocate more than 4GiB of memory
It seems that rather than returning a null pointer from `std::alloc::alloc`,
miri will sometimes choose to simply crash the whole program.
* remove duplicate prelude import after rebasing
Fixes: https://github.com/bytecodealliance/wasmtime/issues/8632
This commit fixes the handling of f64 comparison operations in Winch.
f64 comparison operations are defined as
[f64 f64] -> [i32]
The previous implemementation was widening the result to `[i64]` which
caused issues which stack shuffling in multi-value returns.
* wasmtime: Make table lazy-init configurable
Lazy initialization of tables has trade-offs that we haven't explored in
a while. Making it configurable makes it easier to test the effects of
these trade-offs on a variety of WebAssembly programs, and allows
embedders to decide whether the trade-offs are worth-while for their use
cases.
* Review comments
* Change `MemoryStyle::Static` to store bytes, not pages
This commit is inspired by me looking at some configuration in the
pooling allocator and noticing that configuration of wasm pages vs bytes
of linear memory is somewhat inconsistent in `Config`. In the end I'd
like to remove or update the `memory_pages` configuration in the pooling
allocator to being bytes of linear memory instead to be more consistent
with `Config` (and additionally anticipate the custom-page-sizes
wasm proposal where terms-of-pages will become ambiguous). The first
step in this change is to update one of the lowest layered usages of
pages, the `MemoryStyle::Static` configuration.
Note that this is not a trivial conversion because the purpose of
carrying around pages instead of bytes is that bytes may overflow where
overflow-with-pages typically happens during validation. This means that
extra care is taken to handle errors related to overflow to ensure that
everything is still reported at the same time.
* Update crates/wasmtime/src/runtime/vm/instance/allocator/pooling/memory_pool.rs
Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
* Fix tests
* Really fix tests
---------
Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
This is the final type system change for Wasm GC: the ability to explicitly
declare supertypes and finality. A final type may not be a supertype of another
type. A concrete heap type matches another concrete heap type if its concrete
type is a subtype (potentially transitively) of the other heap type's concrete
type.
Next, I'll begin support for allocating GC structs and arrays at runtime.
I've also implemented `O(1)` subtype checking in the types registry:
In a type system with single inheritance, the subtyping relationships between
all types form a set of trees. The root of each tree is a type that has no
supertype; each node's immediate children are the types that directly subtype
that node.
For example, consider these types:
class Base {}
class A subtypes Base {}
class B subtypes Base {}
class C subtypes A {}
class D subtypes A {}
class E subtypes C {}
These types produce the following tree:
Base
/ \
A B
/ \
C D
/
E
Note the following properties:
1. If `sub` is a subtype of `sup` (either directly or transitively) then
`sup` *must* be on the path from `sub` up to the root of `sub`'s tree.
2. Additionally, `sup` *must* be the `i`th node down from the root in that path,
where `i` is the length of the path from `sup` to its tree's root.
Therefore, if we maintain a vector containing the path to the root for each
type, then we can simply check if `sup` is at index `supertypes(sup).len()`
within `supertypes(sub)`.
* Rename `WasmHeapType::Concrete(_)` to `WasmHeapType::ConcreteFunc(_)`
* Rename `wasmtime::HeapType::Concrete` to `wasmtime::HeapType::ConcreteFunc`
* Introduce Wasm sub- and composite-types
Right now, these are only ever final function types that don't have a supertype,
but this refactoring paves the way for array and struct types, and lets us make
sure that `match`es are exhaustive for when we add new enum variants. (Although
I did add an `unwrap_func` helper for use when it is clear that the type should
be a function type, and if it isn't then we should panic.)
* x64: Rename fields of `CmpRmiR` and swap them
This commit is the start of work that is the equivalent of #8362 but for
comparisons of general purpose instructions. Here only the instruction
arguments are swapped and renamed. All callers preserve the
right-to-left ordering and a subsequent commit will swap them to
left-to-right.
* x64: Swap `test_rr` operands in Winch
* x64: Swap operands of `cmp_rr` in Winch
* x64: Swap operands of `cmp_ir` in Winch
* x64: Swap operands for the `cmp` masm method in Winch
This additionally needed to affect the `branch` method. Methods now
additionally document what's the left-hand-side and what's the
right-hand-side.
Of note here was that the `branch` instruction actually used the terms
"lhs" and "rhs" incorrectly where the left/right-ness was actually
swapped in the actual comparison. This causes no issues, however, since
`branch` was only ever used when both operands were the same or a
reflexive condition was used.
* x64: Swap operands for `Inst::cmp_rmi_r`
* x64: Swap operand order of `cmp_rmi_r` ISLE helper
Also update all callers to swap orders as well.
* x64: Swap operand order of `x64_test` helper
* Swap operand order of `x64_cmp_imm` helper
* x64: Swap operand order of `cmp`
* x64: Define `x64_cmp_imm` with `x64_cmp`
Minor refactoring which clarifies that this is just a normal `cmp`
except with a different signature.
* x64: Use `x64_cmp_imm` in a few more locations
A bit easier on the eyes to read.
* x64: Refactor float comparisons and their representations
Currently the `XmmCmpRmR` instruction variant has a `dst` and a `src`
field. The instruction doesn't actually write to `dst`, however, and the
constructor of `xmm_cmp_rm_r` takes the `src` first followed by the
`dst`. This is inconsistent with most other xmm-related instructions
where the "src1" comes first and the "src2", which is a memory operand,
comes second. This memory-operand-second pattern also matches the Intel
manuals more closely.
This commit refactors the `XmmCmpRmR` instruction variant with the
following changes:
* `dst` is renamed to `src1`
* `src` is renamed to `src2`
* The `xmm_cmp_rm_r` helpers, and callers, swapped their arguments to
take `Xmm` first and `XmmMem` second.
* The `x64_ptest` instruction followed suit as it was modelled after the
same.
* Callers of `x64_ucomis` swapped their arguments to preserve the
operand orders.
* The `Inst::xmm_cmp_rm_r` helper swapped operand order and additionally
callers were updated.
* The VCode rendering of `XmmCmpRmR` swapped order of its operands,
explaining changes in rendering of filetests (although machine code is
not changing here).
The changes were then additionally propagated to Winch as well. In Winch
the `src`/`dst` naming was inherited so it was renamed to `src1` and
`src2` which swapped operands as well. In the case of Winch there was
additionally an accident in `float_cmp_op` where values were popped in
reverse order. This swapping-of-swapping all worked out prior, but to
get all the names to align correctly I've swapped this to be more
consistent. Sorry there's a lot of swaps-of-swaps here but the basic
idea is that the low-level instruction constructor swapped arguments so
to preserve the same (correct) output today something else needed to be
swapped. In Winch's case it wasn't the immediate caller of the
instruction constructor since that method looked correct, but it was
instead a higher-level `float_cmp_op` which then called a helper which
then called the low-level constructor which had operands swapped.
* Review comments
Semantically, a "real register" is supposed to be a physical register,
so let's use the type dedicated for that purpose.
This has the advantage that `PReg` is only one byte while `VReg` is four
bytes, so the arrays where we record collections of `RealReg` become
smaller.
There was an implementation of `From<VReg> for RealReg` which was not a
sensible conversion, because not all `VReg`s are valid `RealReg`s. I
could have replaced it with a `TryFrom` implementation but it wasn't
used anywhere important, so I'm just deleting it instead.
Winch was using that `VReg`->`RealReg` conversion, but only in the
implementation of another conversion that was itself unused, so I'm
deleting that conversion as well. It's easy to implement correctly (the
Winch `Reg` type is identical to `RealReg`, so all conversions for the
latter are readily available) but as far as I can tell Winch doesn't
need to use Cranelift's register wrappers or RA2's virtual register
type, so it's simpler to just delete those conversions.
The riscv64 backend was relying on quirks of the existing conversions
between `RealReg` and `VReg` when emitting clobber saves and restores.
Just using the generic conversions between `RealReg` and `Reg` is
simpler and works correctly with the rest of these changes.
No new functionality is introduced as part of this change.
This commit introduces a small refactoring of Winch's `MacroAssembler` trait by removing multiple methods that can be treated in an ISA-indepedent way by relying on the base building blocks already provided by the `MacroAssembler`. This change simplifies the process of adding support for other backends to Winch.
* cranelift/aarch64: Remove types from address-modes
Previously, the precondition on every use of AMode::*Offset was that the
type specified in the address mode must be the same size as the access
type of the memory instruction where the address mode was used.
However, at the point where we replace these magic address modes with
real modes that the target architecture understands, we know what
instruction is using the address, so we can get the type from there.
This simplifies constructing addresses while also ensuring that the
types are always correct.
The changes here center around threading the access type through
`mem_finalize` and `AMode::pretty_print`, then deleting it from all
`AMode` variants.
In the process I introduced a helper method on `Inst` to centralize the
mapping from instruction to access type, and used that everywhere.
* Update Winch too
* winch: Add support for address maps
Closes https://github.com/bytecodealliance/wasmtime/issues/8095
This commit adds support for generating address maps for Winch.
Give that source code locations and machine code offsets are machine independent, one objective of this change is introduce minimal methods to the MacroAssesmbler and Asssembler implementations and tries to accomodate the bulk of the work in the ISA independent `CodeGen` module.
* Update method documentation to match implementation
* Fix rustdoc warnings on Nightly
I noticed during a failed doc build of another PR we've got a number of
warnings being emitted, so resolve all those here.
* Fix more warnings
* Fix rebase conflicts
* Switch Winch tests to ATT syntax
* Update all test expectations
* Move all winch tests to `disas` folder
* Add `test = "winch"` to `disas`
* Add `test = "winch"` to all winch test files
* Stub out bits to get AArch64 Winch tests working
* Update expectations for all aarch64 winch tests
* Update flags in Winch tests
Use CLI syntax as that's what `flags` was repurposes as in the new test
suite.
* Update all test expectations for x64 winch
* Omit more offsets by default
* Delete now-dead code
* Update an error message
* Update non-winch test expectations
* Exit through Cranelift-generated trampolines for builtins
This commit changes how builtin functions in Wasmtime (think
`memory.grow`) are implemented. These functions are required to exit
through some manner of trampoline to handle runtime requirements for
backtracing right now. Currently this is done via inline assembly for
each architecture (or external assembly for s390x). This is a bit
unfortunate as it's a lot of hand-coding and making sure everything is
right, and it's not easy to update as it's multiple platforms to update.
The change in this commit is to instead use Cranelift-generated
trampolines for this purpose instead. The path for invoking a builtin
function now looks like:
* Wasm code calls a statically known symbol for each builtin.
* The statically known symbol will perform exit trampoline duties (e.g.
pc/fp/etc) and then load a function pointer to the host
implementation.
* The host implementation is invoked and then proceeds as usual.
The main new piece for this PR is that all wasm modules and functions
are compiled in parallel but an output of this compilation phase is what
builtin functions are required. All builtin functions are then unioned
together into one set and then anything required is generated just
afterwards. That means that only one builtin-trampoline per-module is
generated per-builtin.
This work is inspired by #8135 and my own personal desire to have as
much about our ABI details flowing through Cranelift as we can. This in
theory makes it more flexible to deal with future improvements to our
ABI.
prtest:full
* Fix some build issues
* Update winch test expectations
* Update Winch to use new builtin shims.
This commit refactors the Winch compiler to use the new trampolines for
all Wasmtime builtins created in the previous commits. This required a
fair bit of refactoring to handle plumbing through a new kind of
relocation and function call.
Winch's `FuncEnv` now contains a `PrimaryMap` from `UserExternalNameRef`
to `UserExternalName`. This is because there's now more than one kind of
name than just wasm function relocations, so the raw index space of
`UserExternalNameRef` is no longer applicable. This required threading
`FuncEnv` to more locations along with some refactorings to ensure that
lifetimes work out ok.
The `CompiledFunction` no longer stores a trait object of how to map
name refs to names and now directly has a `Primarymap`. This also means
that Winch's return value from its `TargetIsa` is a `CompiledFunction`
as opposed to the previous just-a-`MachBuffer` so it can also package up
all the relocation information. This ends up having `winch-codegen`
depend on `wasmtime-cranelift-shared` as a new dependency.
* Review feedback
The wasmtime-cranelift-shared crate is not as useful as it once was, as
it's no longer possible to build wasmtime with only winch; winch uses
the trampolines generated by cranelift now.
* Remove callee saves from Winch's MacroAssembler trait
prtest:mingw-x64
* Remove the unused callee_saved_regs function
* Removed the unused callee_saved function from Winch's aarch64 backend
* Remove additional unused functions from the Winch ABI trait
Alex Crichton
Saúl Cabrera
Vulcain
Bruce Mitchener
Hamir Mahal
Nick Fitzgerald
Jeffrey Charles
Edoardo Vacchi
bjorn3
Pat Hickey
FrankReh
Jamey Sharp
RoboSchmied
Trevor Elliott