this is causing a link error because pulldown_cmark is available both in
the rust_wasi_markdown_parser deps directory and, now, in the root of
the project as well.
Many of the tests for lowering features use `extractor`s I think since
when they were added that was the only option. Nowadays a `pure
constructor` with `if-let` feels more natural for this, so this commit
updates all existing extractors used to test features to use a `pure
constructor` instead. This additionally updates the names to match the
cranelift codegen setting name to ensure that a consistent name is used.
This is an unnecessary restriction: applications shouldn't be relying on
any ordering of preview1 fds, besides that other files are not found in
the stdio range.
This behavior changed with the introduction of the component adapter,
but I am making a separate PR to edit the test rather than make it part
of #6391.
Currently all the jobs in GitHub Actions are configured to cancel the
entire run if any of them fail. This is similar to `fail-fast: true` but
that can't be used because the jobs aren't all part of the same matrix.
The original goal for this was to save CI time on the merge queue where
if something failed it should get out of the merge queue quickly instead
of waiting for all other jobs to succeed and then report the one quick
failure.
This seems to run into an issue, though, where if a Windows builder
fails then running cancellation will instead mark the Windows builder as
cancelled instead of failed. This can be confusing for test failures
because nothing looks obvious and each log needs to be manually pored
over.
To help address this the commit here disables auto-cancellation of the
"test" matrix of jobs, re-enabling `fail-fast: true` at the same time.
This way test if a test job fails it won't actually cancel everything
else so everything else will run to completion, but given that tests are
typically the longest part of the build that's hopefully ok.
This change intends to add a new lowering for `MachInst::LoadExtName`
(the pseudo-instruction behind CLIF's `func_addr` and `symbol_addr`)
that uses RIP-relative addressing when the name is local to the module
(`colocated`). The existing behavior is retained, which currently loads
from the global offset table when the `is_pic` flag is set (a
`X86GOTPCRel4` relocation) or loads a relocated immediate (an `Abs8`
relocation). After this change, a `lea` instruction will calculate a
RIP-relative address if the name in question is marked with CLIF's
`colocated` flag.
Why is this necessary? Though some users of Cranelift, like
`cranelift-jit`, understand how to relocate the existing `Abs8`
relocations (see [`CompiledBlob::perform_relocations`]), Wasmtime does
not. Though technically Wasmtime does apply relocations (see
[`CodeMemory::apply_relocations`]), these are only for libcalls.
Wasmtime will attempt to resolve module-local functions, the ones
targeted by `func_addr`, via some internal relocation patching in
[`ModuleTextBuilder::append_func`]. When it realizes it cannot patch one
of these functions, it panics because [`LabelUse::from_reloc`] only
understands the `X86CallPCRel4` relocation. This prevents the use of
`func_addr` when translating Wasm code. Since no current Wasmtime
translation uses `func_addr` directly, this change is not technically
required, but it does feel like the right option to add, normalizing
`func_addr`'s behavior for local functions to `MachInst::CallKnown`'s
behavior and making it more useful for other Cranelift users in the
future.
[`CompiledBlob::perform_relocations`]: 752c7ea4dd/cranelift/jit/src/compiled_blob.rs (L48)
[`CodeMemory::apply_relocations`]: 752c7ea4dd/crates/jit/src/code_memory.rs (L269)
[`ModuleTextBuilder::append_func`]: 752c7ea4dd/crates/cranelift-shared/src/obj.rs (L145)
[`LabelUse::from_reloc`]: 752c7ea4dd/cranelift/codegen/src/isa/x64/inst/mod.rs (L2759)
* wasi-tests and wasi-http-tests no longer have their own workspace
* wasi-tests: fix warnings
* rewrite the test-programs build.rs to generate {package}_modules.rs and _components.rs
The style is cribbed from preview2-prototying repo, but I ended up
refactoring it a bit.
* better escaping should help with windows?
* long form cap-std-sync and tokio test suites
* convert wasi-http test
* fixes, comments
* apply cargo fmt to whole workspace
* bump test-programs and wasi-http-tests to all use common dependency versions
wit-bindgen 0.6.0 and wit-component 0.7.4
* add new audits
* cargo vet prune
* package and supply chain updates to fix vulnerabilities
h2 upgraded from 0.3.16 -> 0.3.19 to fix vulnerability
tempfile upgraded from 0.3.3 -> 0.3.5 to eliminate dep on vulnerable
remove_dir_all
* deny: temporarily allow duplicate wasm-encoder, wasmparser, wit-parser
prtest:full
* convert more dependencies to { workspace = true }
Alex asked me to do thsi for wit-component and wit-bindgen, and I found
a few more (cfg-if, tempfile, filecheck, anyhow...
I also reorganized the workspace dependencies section to make the ones
our team maintains more clearly separated from our external
dependencies.
* test-programs build: ensure that the user writes a #[test] for each module, component
* fix build of wasi-tests on windows
* misspelled macos
* mark wasi-tests crate test=false so we dont try building it natively...
* mark wasi-http-tests test=false as well
* try getting the cargo keys right
* just exclude wasi-tests and wasi-http-tests in run-tests.sh
* interesting paths fails on windows
* misspelling so nice i did it twice
* new cargo deny exception: ignore all of wit-bindgen's dependencies
* auto-import wildcard vets
* x64: Update i64x2 `sshr` implementation
The previous implementation fell back to a scalar implementation by
extracting lanes and re-inserting them. These operations are
significantly more complicated without SSE 4.1, however. Comparing
against LLVM's lowerings, as well, it looks like a different strategy is
used which avoids extracting and inserting lanes. This commit updates to
follow the strategy matched by LLVM's lowerings which additionally has
the benefit of providing a non-SSE4.1 implementation.
* Update an outdated comment
* Update wasm-tools crates to latest versions.
This included stubbing out unimplemented GC-related things and
updating coredump generation to include the coredump spec changes.
* cargo vet
* address review comments
* riscv64: Use Vector Regclass
* riscv64: Add assert to `Inst::Mov`
It isn't ready yet
* riscv64: Add SIMD vconst large test
This was meant to exercise the changes in #6324 but was failing in RISC-V due to some missing regalloc bits.
* riscv64: Restrict spill slot size
* riscv64: Mark v0 as preferred
* riscv64: Const compute clobbers
* wasmtime: Refactor compilation orchestration
Before, the acts of discovering what we need to compile and actually compiling
those things was intertwined. This meant that we weren't fanning out as wide as
we could in parallel compilation because we would do things like compile all
Wasm functions in parallel, and all wasm-to-native trampolines in parallel, but
not compile those two groups in parallel with each other.
Now we split discovery and compilation into separate phases: first we discover
all the work we need to do, then we do it all in parallel at once.
* Address review comments
* winch: Implement new trampolines
This change is a follow-up to
https://github.com/bytecodealliance/wasmtime/pull/6262, in which the new
trampolines, described [here](https://github.com/bytecodealliance/rfcs/blob/main/accepted/tail-calls.md#new-trampolines-and-vmcallercheckedanyfunc-changes),
were introduced to Wasmtime.
This change, focuses on the `array-to-wasm`,
`native-to-wasm` and `wasm-to-native` trampolines to restore Winch's
working state prior to the introduction of the new trampolines. It's
worth noting that the new approach for trampolines make it easier to support
the `TypedFunc` API in Winch. Prior to the introduction of the new
trampolines, it was not obvious how to approach it.
This change also introduces a pinned register that will hold the
`VMContext` pointer, which is loaded in the `*-to-wasm` trampolines;
the `VMContext` register is a pre-requisite to this change to support
the `wasm-to-native` trampolines.
Lastly, with the introduction of the `VMContext` register and the
`wasm-to-native` trampolines, this change also introduces support for
calling function imports, which is a variation of the already existing
calls to locally defined functions.
The other notable piece of this change aside from the trampolines is
`winch-codegen`'s dependency on `wasmtime-environ`. Winch is so closely
tied to the concepts exposed by the wasmtime crates that it makes sense
to tie them together, even though the separation provides some
advantages like easier testing in some cases, in the long run, there's
probably going to be less need to test Winch in isolation and rather
we'd rely more on integration style tests which require all of Wasmtime
pieces anyway (fuzzing, spec tests, etc).
This change doesn't update the existing implmenetation of
`winch_codegen::FuncEnv`, but the intention is to update that part after
this change.
prtest:full
* tests: Ignore miri in Winch integration tests
* Remove hardcoded alignment and addend
* Add a cranelift setting for padding between basic blocks
Various relocations, jumps, and such require special handling in
`MachBuffer` with respect to islands to ensure that everything gets
emitted correctly. This commit adds a setting to synthetically insert
padding at the end of every basic block to help stress this logic with
more minimal test cases. The setting is disabled by default but is
something that we should be able to turn on during fuzzing, for example.
* aarch64: Fix out-of-range `Ldr19` relocations
This commit fixes a bug in the AArch64 backend, and possibly others,
where constants were unconditionally forced to be at the end of the
function when they sometimes couldn't be. For example the `Ldr19`
relocation has a 512k range meaning that if an instruction near the
beginning of a function accesses a constant at the end of a function and
the function is >1M, then the relocation cannot be resolved. This is all
handled internally with `MachBuffer`'s handling of islands but the
problem with constants is that the labels (and the constant values)
weren't defined until the end of the function.
The first attempt at fixing this was to move the calls to
`defer_constant` to the beginning of emission. This would enable the
constants to get deferred as necessary. This was problematic, however,
because it only solved the forwards case (aka your constant was forced
to the end of the function which is too far away). The backwards case,
aka your constant is way too far behind you, was a new problem that
arose.
To fix all of these issues constants are now handled differently inside
of the `MachBuffer`. Previously constants were all pre-assigned a
label-per-constant and all references to the constant would use that
single label. Instead a new heuristic has been added where constants
record their size/alignment at the start of emission and labels are
lazily deferred. When a label for a constant is requested then a label
is lazily allocated or a previously-allocated label for this constant is
returned. When an island is emitted then all emitted constants get
their labels cleared. This intends to balance the previous functionality
of multiple uses of a constant only emit the constant once with fixing
this issue with simplicity as well. This means that constants may get
emitted multiple times, since each reference to a constant after an
island is generated will be guaranteed to generate a new label, even if
it's in-range to access. This can perhaps be fixed in the future with a
more clever API where the `LabelUse` is passed into the function which
converts a constant to a label, but that's left as a refactoring for a
future date.
This commit also moves an `alignment: u32` field into the
`MachBufferFinalized` itself since that's now a function of whatever
constants actually got emitted. Additionally note that constant
emission in the middle of a function doesn't actually emit anything,
instead recording markers of where constants need to go. Then when a
buffer is finalized the constants are passed in to get access to the
data which fills in everything as it's referenced.
* Fuzz the `bb_padding_log2` setting
This commit hooks up the previously-added setting to Cranelift to
Wasmtime's fuzzing infrastructure. This will automatically configure the
setting based on the fuzz input to add a bit of "chaos" to the emitted
code. This should hopefully help expose the issue fixed previously via
fuzzing which otherwise won't generate massive functions.
* Realign back to an instruction boundary
Otherwise misaligned instructions were getting emitted and tripping
various asserts.
* Fix riscv64 testing
* Rename codegen setting to bb_padding_log2_minus_one
Allow for inserting one byte of padding.
* Doc updates
* Thread through shared flags differently
Don't use `EmitInfo`, instead pass in to vcode emission
* Fix s390x tests
* Combine island calculations during vcode emission
Fixes an off-by-just-a-few error if the two island checks are done
separately after a basic block.
* Update to latest wasm-tools crates
This commit pushes through the full update of the wasm-tools crates
through Wasmtime. There are two major features which changed, both
related to components, which required updates in Wasmtime:
* Resource types are now implemented in wasm-tools and they're not yet
implemented in Wasmtime so I've stubbed out the integration point with
panics as reminders to come back and implement them.
* There are new validation rules about how aggregate types must be
named. This doesn't affect runtime internals at all but was done on
behalf of code generators. This did however affect a number of tests
which have to ensure that types are exported.
* Fix more tests
* Add vet entries
* Switch wasmtime-wasi-http to using Wasmtime's version
This should use the same versioning scheme as all the other `wasmtime-*`
crates.
* Fixup more directives
Pat Hickey
Alex Crichton
Trevor Elliott
Afonso Bordado
Andrew Brown
Jamey Sharp
Rainy Sinclair
Brendan Burns
Saúl Cabrera
Nick Fitzgerald
Daniel Bevenius
Pat Hickey