* winch: Improve result handling for Aarch64
This commit introduces missing tests for Aarch64's ABI implementation.
Compared to the x64 counterpart, the aarch64 implementation was missing
tests for multi-value.
Additionally, this commit fixes an issue with how multi-value returns
are handlded in Winch's default calling convention, which only allows
1 register result, independent of the register class.
This commit also refactors the register indexing environment so that it
can be easily shared across the existing backends.
* Keep comments on Aarch64 register assignment
* Update handling of wasm features and `*.wast` testing
This commit is an attempt at making it easier to manage the set of
variables we have in play with `*.wast` testing and wasm features.
The Cranelift and Winch backends support different sets of wasm features
at this time and historically Cranelift has also had
architecture-specific support for wasm features. This is hoped to help
simplify management of all of this in the future in addition to making
`*.wast` testing more robust. Notable changes here are:
* A `Config` no longer tracks a `WasmFeatures` but instead only tracks
explicitly disabled/enabled features. The final `WasmFeatures` is then
computed given the result of compiler configuration.
* Default-enabled features now start from nothing instead of
`wasmparser`'s defaults to avoid future breakage there.
* Each compiler configuration and/or backend now has a listed set of
"this feature may panic" wasm features. These are all disabled by
default and if explicitly enabled a `Config::validate` error is
returned.
* All `*.wast` tests are now run through both Cranelift and Winch. Tests
are now asserted to either fail or pass depending on configuration and
the whole test will fail if the result doesn't match the expectation.
* Many `gc` proposal tests which are now passing are flagged as passing
now. Failing `winch` tests are now explicitly listed instead of using
a litany of patterns.
* Change panicking features for winch
* Fix builds without a compiler
* winch: Solidify bounds check for dynamic heaps
This commit fixes and edge case for bounds checks for dynamic heaps.
https://github.com/bytecodealliance/wasmtime/pull/8157/files erroneously
tied the bounds check operation (more concretely the overflow check) to the size derived from from the heap
type. Even though offsets and access sizes are validated ahead-of-time
and bound to the heap type, in the case of overflow checking, we must
ensure that the operation size is tied to the target's pointer size to
avoid clamping the access size and offset addition, which would result
in missing an out-of-bounds memory access.
This commit also adds a disassembly test to avoid introducing
regressions in the future.
Additionally, this commit adds more comments around why `pointer_size`
is used for certain bounds checking operations.
* Update disassembly test
Right now this is only on some crates such as `wasmtime` itself and
`wasmtime-cli`, but by applying it to all crates it helps with version
selection of those using just Cranelift for example.
This commit simplifies the selection of an ABI for wasm functions now
that all Cranelift backends implement tail calls. All wasm functions use
the `Tail` calling convention except when the `winch_callable` tunable
is enabled meaning that Winch-generated functions are being called.
This then additionally simplifies the activation of the tail call
proposal. It's not unconditionally active and the same across all
compilers. The Winch compiler is updated to return an error for
unsupported instructions rather than panicking so the embedder API is
suitable for feeding unsupported modules to Winch. This means that tail
calls in Winch behaves similarly to GC in Cranelift or other unsupported
proposals like SIMD in Winch.
* Cranelift(x64): provide better panic messages for `Gpr::new(reg).unwrap()` et al
Co-Authored-By: Jamey Sharp <jsharp@fastly.com>
* Fix compile error due to bad mechanical replacement
---------
Co-authored-by: Jamey Sharp <jsharp@fastly.com>
* Add v128.const support to Winch
* Remove next_vr and vector_reg_for methods
* Adjust alignment and slot size for v128
* Forgot to update disas tests
* Update unit tests
* Use 8 byte stack slot sizes
* Fix broken unit tests, add tests for vecs, and use ty size for vecs
* winch: Improve scratch register handling
This commit doesn't introduce any new behavior. It's mostly a follow-up
to https://github.com/bytecodealliance/wasmtime/pull/7977.
This commit tries to reduce the repetitive pattern used to obtain the
scrtach register by introducing a macro similar to the existing `vmctx!`
macro.
This commit also improves the macro definition by using fully qualified
paths.
* Fix unused imports
This register was used as an additional scratch register in the context
of trampolines.
As of https://github.com/bytecodealliance/wasmtime/pull/8109,
trampolines are implemented with Cranelift, so this register alias is
unused.
* Remove Opcode from MachCallSite
It doesn't seem useful and in several locations we already use a
different Opcode from what the user actually wrote. For example for
tls_value, the implicit stackprobe or the implicit memcpy call for
struct args.
This also makes the recorded call sites more accurately represent the
actual set of call sites by avoiding special casing of pseudo
instructions that happen to include a call.
* Don't precompute the set of defs to remove from the clobbers
Instead remove the actual set of defs from the clobber set once they are
known. This reduces the risk of getting the clobber set wrong and makes
it easier to handle cases where the exact set is not known in advance.
Fixes: https://github.com/bytecodealliance/wasmtime/issues/8848
Similar to all the control instructions, any state must be explicitly
saved before emitting the code for `br_if`.
This commit ensures that live locals and registers are explicilty saved
before emitting the code for `br_if`. Prior to this commit, live
locals and registers were not saved every time causing incorrect
behavior in cases where the calculation of the conditional argument
didn't trigger a spill.
This change introduces the explicit spill after calculating the branch
condition argument to minimize memory traffic in case the conditional is
already in a register.
* upgrade to wasm-tools 0.211.1
* code review
* cargo vet: auto imports
* fuzzing: fix wasm-smith changes
* fuzzing: changes for HeapType
* Configure features on `Parser` when parsing
---------
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
* Const-propagate some offsets in `VMOffsets`
Before this commit all offsets to all fields in `VMOffsets` were stored
as fields within `VMOffset` itself. All of the fields at the start of
`VMOffsets`, however, are statically known given the pointer size.
Notably this means that the use of `HostPtr` in the runtime still was
forcing a dynamic lookup of these static offsets.
This commit refactors this to reflect all static offsets based solely on
the pointer size in the `PtrSize` trait, removing all the fields from
`VMOffsets`. All the dynamically sized fields, however, remain in
`VMOffsets`.
* Fix expected error message
* Wasmtime: Implement the custom-page-sizes proposal
This commit adds support for the custom-page-sizes proposal to Wasmtime:
https://github.com/WebAssembly/custom-page-sizes
I've migrated, fixed some bugs within, and extended the `*.wast` tests for this
proposal from the `wasm-tools` repository. I intend to upstream them into the
proposal shortly.
There is a new `wasmtime::Config::wasm_custom_page_sizes_proposal` method to
enable or disable the proposal. It is disabled by default.
Our fuzzing config has been updated to turn this feature on/off as dictated by
the arbitrary input given to us from the fuzzer.
Additionally, there were getting to be so many constructors for
`wasmtime::MemoryType` that I added a builder rather than add yet another
constructor.
In general, we store the `log2(page_size)` rather than the page size
directly. This helps cut down on invalid states and properties we need to
assert.
I've also intentionally written this code such that supporting any power of two
page size (rather than just the exact values `1` and `65536` that are currently
valid) will essentially just involve updating `wasmparser`'s validation and
removing some debug asserts in Wasmtime.
* Update error string expectation
* Remove debug logging
* Use a right shift instead of a division
* fix error message expectation again
* remove page size from VMMemoryDefinition
* fix size of VMMemoryDefinition again
* Only dynamically check for `-1` sentinel for 1-byte page sizes
* Import functions that are used a few times
* Better handle overflows when rounding up to the host page size
Propagate errors instead of returning a value that is not actually a rounded up
version of the input.
Delay rounding up various config sizes until runtime instead of eagerly doing it
at config time (which isn't even guaranteed to work, so we already had to have a
backup plan to round up at runtime, since we might be cross-compiling wasm or
not have the runtime feature enabled).
* Fix some anyhow and nostd errors
* Add missing rounding up to host page size at runtime
* Add validate feature to wasmparser dep
* Add some new rounding in a few places, due to no longer rounding in config methods
* Avoid actually trying to allocate the whole address space in the `massive_64_bit_still_limited` test
The point of the test is to ensure that we hit the limiter, so just cancel the
allocation from the limiter, and otherwise avoid MIRI attempting to allocate a
bunch of memory after we hit the limiter.
* prtest:full
* Revert "Avoid actually trying to allocate the whole address space in the `massive_64_bit_still_limited` test"
This reverts commit ccfa34a78dd3d53e49a6158ca03077d42ce8bcd7.
* miri: don't attempt to allocate more than 4GiB of memory
It seems that rather than returning a null pointer from `std::alloc::alloc`,
miri will sometimes choose to simply crash the whole program.
* remove duplicate prelude import after rebasing
Fixes: https://github.com/bytecodealliance/wasmtime/issues/8632
This commit fixes the handling of f64 comparison operations in Winch.
f64 comparison operations are defined as
[f64 f64] -> [i32]
The previous implemementation was widening the result to `[i64]` which
caused issues which stack shuffling in multi-value returns.
* wasmtime: Make table lazy-init configurable
Lazy initialization of tables has trade-offs that we haven't explored in
a while. Making it configurable makes it easier to test the effects of
these trade-offs on a variety of WebAssembly programs, and allows
embedders to decide whether the trade-offs are worth-while for their use
cases.
* Review comments
* Change `MemoryStyle::Static` to store bytes, not pages
This commit is inspired by me looking at some configuration in the
pooling allocator and noticing that configuration of wasm pages vs bytes
of linear memory is somewhat inconsistent in `Config`. In the end I'd
like to remove or update the `memory_pages` configuration in the pooling
allocator to being bytes of linear memory instead to be more consistent
with `Config` (and additionally anticipate the custom-page-sizes
wasm proposal where terms-of-pages will become ambiguous). The first
step in this change is to update one of the lowest layered usages of
pages, the `MemoryStyle::Static` configuration.
Note that this is not a trivial conversion because the purpose of
carrying around pages instead of bytes is that bytes may overflow where
overflow-with-pages typically happens during validation. This means that
extra care is taken to handle errors related to overflow to ensure that
everything is still reported at the same time.
* Update crates/wasmtime/src/runtime/vm/instance/allocator/pooling/memory_pool.rs
Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
* Fix tests
* Really fix tests
---------
Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
This is the final type system change for Wasm GC: the ability to explicitly
declare supertypes and finality. A final type may not be a supertype of another
type. A concrete heap type matches another concrete heap type if its concrete
type is a subtype (potentially transitively) of the other heap type's concrete
type.
Next, I'll begin support for allocating GC structs and arrays at runtime.
I've also implemented `O(1)` subtype checking in the types registry:
In a type system with single inheritance, the subtyping relationships between
all types form a set of trees. The root of each tree is a type that has no
supertype; each node's immediate children are the types that directly subtype
that node.
For example, consider these types:
class Base {}
class A subtypes Base {}
class B subtypes Base {}
class C subtypes A {}
class D subtypes A {}
class E subtypes C {}
These types produce the following tree:
Base
/ \
A B
/ \
C D
/
E
Note the following properties:
1. If `sub` is a subtype of `sup` (either directly or transitively) then
`sup` *must* be on the path from `sub` up to the root of `sub`'s tree.
2. Additionally, `sup` *must* be the `i`th node down from the root in that path,
where `i` is the length of the path from `sup` to its tree's root.
Therefore, if we maintain a vector containing the path to the root for each
type, then we can simply check if `sup` is at index `supertypes(sup).len()`
within `supertypes(sub)`.
* Rename `WasmHeapType::Concrete(_)` to `WasmHeapType::ConcreteFunc(_)`
* Rename `wasmtime::HeapType::Concrete` to `wasmtime::HeapType::ConcreteFunc`
* Introduce Wasm sub- and composite-types
Right now, these are only ever final function types that don't have a supertype,
but this refactoring paves the way for array and struct types, and lets us make
sure that `match`es are exhaustive for when we add new enum variants. (Although
I did add an `unwrap_func` helper for use when it is clear that the type should
be a function type, and if it isn't then we should panic.)
* x64: Rename fields of `CmpRmiR` and swap them
This commit is the start of work that is the equivalent of #8362 but for
comparisons of general purpose instructions. Here only the instruction
arguments are swapped and renamed. All callers preserve the
right-to-left ordering and a subsequent commit will swap them to
left-to-right.
* x64: Swap `test_rr` operands in Winch
* x64: Swap operands of `cmp_rr` in Winch
* x64: Swap operands of `cmp_ir` in Winch
* x64: Swap operands for the `cmp` masm method in Winch
This additionally needed to affect the `branch` method. Methods now
additionally document what's the left-hand-side and what's the
right-hand-side.
Of note here was that the `branch` instruction actually used the terms
"lhs" and "rhs" incorrectly where the left/right-ness was actually
swapped in the actual comparison. This causes no issues, however, since
`branch` was only ever used when both operands were the same or a
reflexive condition was used.
* x64: Swap operands for `Inst::cmp_rmi_r`
* x64: Swap operand order of `cmp_rmi_r` ISLE helper
Also update all callers to swap orders as well.
* x64: Swap operand order of `x64_test` helper
* Swap operand order of `x64_cmp_imm` helper
* x64: Swap operand order of `cmp`
* x64: Define `x64_cmp_imm` with `x64_cmp`
Minor refactoring which clarifies that this is just a normal `cmp`
except with a different signature.
* x64: Use `x64_cmp_imm` in a few more locations
A bit easier on the eyes to read.
* x64: Refactor float comparisons and their representations
Currently the `XmmCmpRmR` instruction variant has a `dst` and a `src`
field. The instruction doesn't actually write to `dst`, however, and the
constructor of `xmm_cmp_rm_r` takes the `src` first followed by the
`dst`. This is inconsistent with most other xmm-related instructions
where the "src1" comes first and the "src2", which is a memory operand,
comes second. This memory-operand-second pattern also matches the Intel
manuals more closely.
This commit refactors the `XmmCmpRmR` instruction variant with the
following changes:
* `dst` is renamed to `src1`
* `src` is renamed to `src2`
* The `xmm_cmp_rm_r` helpers, and callers, swapped their arguments to
take `Xmm` first and `XmmMem` second.
* The `x64_ptest` instruction followed suit as it was modelled after the
same.
* Callers of `x64_ucomis` swapped their arguments to preserve the
operand orders.
* The `Inst::xmm_cmp_rm_r` helper swapped operand order and additionally
callers were updated.
* The VCode rendering of `XmmCmpRmR` swapped order of its operands,
explaining changes in rendering of filetests (although machine code is
not changing here).
The changes were then additionally propagated to Winch as well. In Winch
the `src`/`dst` naming was inherited so it was renamed to `src1` and
`src2` which swapped operands as well. In the case of Winch there was
additionally an accident in `float_cmp_op` where values were popped in
reverse order. This swapping-of-swapping all worked out prior, but to
get all the names to align correctly I've swapped this to be more
consistent. Sorry there's a lot of swaps-of-swaps here but the basic
idea is that the low-level instruction constructor swapped arguments so
to preserve the same (correct) output today something else needed to be
swapped. In Winch's case it wasn't the immediate caller of the
instruction constructor since that method looked correct, but it was
instead a higher-level `float_cmp_op` which then called a helper which
then called the low-level constructor which had operands swapped.
* Review comments
Semantically, a "real register" is supposed to be a physical register,
so let's use the type dedicated for that purpose.
This has the advantage that `PReg` is only one byte while `VReg` is four
bytes, so the arrays where we record collections of `RealReg` become
smaller.
There was an implementation of `From<VReg> for RealReg` which was not a
sensible conversion, because not all `VReg`s are valid `RealReg`s. I
could have replaced it with a `TryFrom` implementation but it wasn't
used anywhere important, so I'm just deleting it instead.
Winch was using that `VReg`->`RealReg` conversion, but only in the
implementation of another conversion that was itself unused, so I'm
deleting that conversion as well. It's easy to implement correctly (the
Winch `Reg` type is identical to `RealReg`, so all conversions for the
latter are readily available) but as far as I can tell Winch doesn't
need to use Cranelift's register wrappers or RA2's virtual register
type, so it's simpler to just delete those conversions.
The riscv64 backend was relying on quirks of the existing conversions
between `RealReg` and `VReg` when emitting clobber saves and restores.
Just using the generic conversions between `RealReg` and `Reg` is
simpler and works correctly with the rest of these changes.
No new functionality is introduced as part of this change.
This commit introduces a small refactoring of Winch's `MacroAssembler` trait by removing multiple methods that can be treated in an ISA-indepedent way by relying on the base building blocks already provided by the `MacroAssembler`. This change simplifies the process of adding support for other backends to Winch.
* cranelift/aarch64: Remove types from address-modes
Previously, the precondition on every use of AMode::*Offset was that the
type specified in the address mode must be the same size as the access
type of the memory instruction where the address mode was used.
However, at the point where we replace these magic address modes with
real modes that the target architecture understands, we know what
instruction is using the address, so we can get the type from there.
This simplifies constructing addresses while also ensuring that the
types are always correct.
The changes here center around threading the access type through
`mem_finalize` and `AMode::pretty_print`, then deleting it from all
`AMode` variants.
In the process I introduced a helper method on `Inst` to centralize the
mapping from instruction to access type, and used that everywhere.
* Update Winch too
* winch: Add support for address maps
Closes https://github.com/bytecodealliance/wasmtime/issues/8095
This commit adds support for generating address maps for Winch.
Give that source code locations and machine code offsets are machine independent, one objective of this change is introduce minimal methods to the MacroAssesmbler and Asssembler implementations and tries to accomodate the bulk of the work in the ISA independent `CodeGen` module.
* Update method documentation to match implementation
Saúl Cabrera
Alex Crichton
Vulcain
Bruce Mitchener
wasmtime-publish
Hamir Mahal
Nick Fitzgerald
Jeffrey Charles
Edoardo Vacchi
bjorn3
Pat Hickey
FrankReh
Jamey Sharp
RoboSchmied