* Pulley: Get tests passing with permissive provenance under MIRI
Co-Authored-By: Alex Crichton <alex@alexcrichton.com>
* Pulley: Get tests passing with strict provenance in MIRI
Co-Authored-By: Alex Crichton <alex@alexcrichton.com>
* Test Pulley under MIRI in CI
* Test Pulley with all cargo features in CI
* Quiet a warning for certain build configs
---------
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
* CI: Check that Pulley builds for `no_std` targets
* CI: Test Pulley on 32-bit targets
This adds CI testing for Pulley on the following 32-bit targets:
* `i686-unknown-linux-gnu`
* `armv7-unknown-linux-gnueabihf`
The previous commit added checks that Pulley builds for `no_std` targets.
Our existing test sharding means that it is already tested on a big-endian
platform (`s390x-unknown-linux-gnu`) as well as on different OSes (Linux,
Windows, and macOS).
So altogether we are now testing a pretty wide range of portability dimensions
for Pulley in CI:
* 32- vs. 64-bit architectures
* `std` vs. `no_std`
* Little- vs. big-endian
* Linux vs. Windows vs. macOS
I think our CI coverage for Pulley is in a pretty good place now!
Co-Authored-By: Alex Crichton <alex@alexcrichton.com>
---------
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
* Add wasi adapter provider template which is materialised in CI
* Add rustfmt component to adapter CI
* Draft an extra publish step for the adapter provider
* Check adapter provider in a separate step with adapter artifacts
* Use artifact downloads in the publish action as well
* Record results from adapter provider step as well
* Refactor to use composite actions
* Add missing shell property
* Fix spelling mistake
* Try using the env context
* wasi-adapter: Implement provider crate that embeds the adapter binaries
* Upgrade wasi adapters to the latest version
* Update adapter docs
* Recompile asi adapters with 1.78
* Recompile wasi adapters with 1.79
* Add some debugging to adapter build script
* Fix script debugging
* Compute wasi adapter version based on latest adapter commit hash
* Try to bless wasi adapters again
* Try to work around CI auto-merges
* Revert to just using workspace version
* Add the wasi adapter provider to the crate publication list
* Use wasi adapter provider in artifacts test + explicit MSRV in CI
* Explicit adapter crate version
* Small fix
* Remove version info from adapter metadata
* Check but don't install rust toolchain in build script
* Bless after rebase
---------
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
* Disable `call-hook` crate feature by default
This commit disables the `call-hook` feature for the Wasmtime crate
added in #8795 by default. The rationale is that this has a slight cost
to all embeddings even if the feature isn't used and it's not expected
to be that widely used of a feature, so off-by-default seems like a more
appropriate default.
* Enable all features in doc build
* More doc fixes
This commit is an attempt to migrate from
`JamesIves/github-pages-deploy-action`, a third party action, to what
appears to be a more official action under the `github.com/actions`
organization. This was first pioneered in
bytecodealliance/wasm-tools#1597 and it looked like it would work well
over here too.
This test was disabled because GitHub Actions Windows Server image
doesn't have desktop experience included. But it looks like we can have
a standalone WinML binary downloaded from ONNX Runtime project.
Wasi-nn WinML backend and ONNX Runtime backend now share the same test
code as they accept the same input, and they are expected to produce the
same result.
This change also make wasi-nn WinML backend as a default feature.
prtest:full
* Fix some aspects of running debug tests in CI
This commit fixes the filter used to run tests in CI to include all
debug with a broader filter to include the whole `debug` module of
tests. This fixes a mistake where recent tests added weren't running in
CI.
This then additionally adds a `run-dwarf` filter in CI to run these
tests on any changes to filenames containing "debug" in the name.
* Test out the marker to run dwarf tests in CI
prtest:debug
This is another take at improving the documentation for `bindgen!` in
Wasmtime. This commit takes a leaf out of the book of
bytecodealliance/wit-bindgen#871 to organize the documentation of the
macro a bit more rather than having one giant doc block that can be
difficult to explore. The macro's documentation itself is now mostly a
reference of all the options that can be specified. There is now a new
documentation-only module which serves a few purposes:
* Individual examples are organized per-submodule to be a bit more
digestable.
* Each example has an example of the generated code in addition to the
source code used for each example.
* All examples are tested on CI to compile (none are run).
My hope is that this makes it easier to expand the docs here further
over time with niche features as they arise or with various options that
the macro has. This is one of the lynchpins of Wasmtime's support for
the component model so it seems pretty important to have a good
onboarding experience here.
Along the way I've implemented a few more niche options for the
`bindgen!` macro that I found necessary, such as configuring the
`wasmtime` crate and where it's located.
* Refactor how release notes are managed
This commit updates how Wasmtime manages release notes across released
versions of Wasmtime. One of the most onerous parts about releases right
now is writing release notes in all the locations and making sure
they're all up-to-date and in-sync. This is inevitably forgotten in some
cases and various pieces will slip through the cracks. The basic idea of
this PR is to change our release notes to only document the release
branch that they're on. All historical release notes are relegated to
historical branches.
With this change there's no longer any need to backport or forward-port
release notes for any changes. Instead release notes are written once
on one branch and that's it.
The major downside of this change is that there's no easy way to get a
bird's eye view of all release notes for Wasmtime any more. If necessary
that could theoretically be automated in the future (like
https://releases.rs/), but for now this feels like an acceptable
compromise to make releases much easier.
The contents of this PR are to update `RELEASES.md` with back-links to
historical release notes as well as the various pieces of automation we
have about managing release notes.
* Add more notes about the release process
* Fix a typo
* Make it more obvious that patch releases are documented
* Refactor installation of C API and features supported
This commit overhauls and refactors the management of the building of
the C API. Instead of this being script-based it's now done entirely
through CMake and CMake is the primary focus for building the C API. For
example building the C API release artifacts is now done through CMake
instead of through a shell script's `cargo build` and manually moving
artifacts.
The benefits that this brings are:
* The C API now properly hides symbols in its header files that weren't
enabled at build time. This is done through a new build-time generated
`conf.h` templated on a `conf.h.in` file in the source tree.
* The C API's project now supports enabling/disabling Cargo features to
have finer-grained support over what's included (plus auto-management
of the header file).
* Building the C API and managing it is now exclusively done through
CMake. For example invoking `doxygen` now lives in CMake, installation
lives there, etc.
The `CMakeLists.txt` file for the C API is overhauled in the process of
doing this. The build now primarily matches on the Rust target being
built rather than the host target. Additionally installation will now
install both the static and shared libraries instead of just one.
Additionally during this refactoring various bits and pieces of
Android-specific code were all removed. Management of the C toolchain
feels best left in scope of the caller (e.g. configuring `CC_*` env vars
and such) rather than here.
prtest:full
* Don't use `option` for optional strings
* Invert release build check
Also adjust some indentation
* Fix more indentation
* Remove no-longer-used variable
* Reduce duplication in feature macro
* Add Android release binaries to CI
This commit is inspired by #6480 and historical asks for Android
binaries. This does the bare minimum necessary to configure C compilers
such that we can produce binaries but I'll admit that I'm no Android
developer myself so I have no idea if these are actually suitable for
use anywhere. Otherwise though this build subsumes the preexisting check
in CI that the build works for Android, so that part is removed too.
This additionally changes how the NDK is managed from before. Previously
a GitHub Action was used to download Java and the NDK and additionally
used the `cargo ndk` subcommand. That's all removed now in favor of
configuring C compilers directly with a pre-installed version of the NDK
which should help reduce the CI dependencies a bit.
* Review comments
* List Android as tier 3 target
* Bump Wasmtime's MSRV to 1.76.0
* Update Rust in CI to 1.78.0, the current stable
* Update nightly tests to the latest nightly
prtest:full
* Fix check-cfg with nightly
* More check-cfg fixes
* Remove an async cfg
This is no longer specified for the root crate.
* Move definition of Wasmtime's nightly into one place
Don't change a bunch of places when this is updated, try to update just
one single location instead.
* Update Wasmtime's tier stability documentation
Move some items between tiers and add a few misc items here and there.
* Update platform support documentation
Re-word lots of this since it was originally written, link to the tiers
of support page, and rewrite the section on `no_std`.
* Update the `min-platform` example with no_std
This commit updates the preexisting `min-platform` example to no longer
require Nightly Rust and instead use the `no_std` support now added to
Wasmtime. This involved:
* Change the build process to produce a staticlib which is then manually
converted via `cc` into a shared library for the native Linux platform.
* Compile the modules outside of the embedding and only `deserialize`
within the embedding.
* Update the `indexmap` dependency to pick up a bug fix required in
`no_std` mode (apparently, it fails on indexmap@2.0.0 and passes at
2.2.6, I didn't dig much further).
This commit additionally makes the `wasmtime-platform.h` header file
generated by the example a release artifact for Wasmtime itself. The
header itself is touched up a bit by configuring some more `cbindgen`
options as well.
* Fix clippy build
prtest:full
* Review comments
* Pass gc-sections to linking the library
* Always fall back to custom platform for Wasmtime
This commit updates Wasmtime's platform support to no longer require an
opt-in `RUSTFLAGS` `--cfg` flag to be specified. With `no_std` becoming
officially supported this should provide a better onboarding experience
where the fallback custom platform is used. This will cause linker
errors if the symbols aren't implemented and searching/googling should
lead back to our docs/repo (eventually, hopefully).
* Change Wasmtime's TLS state to a single pointer
This commit updates the management of TLS to rely on just a single
pointer rather than a pair of a pointer and a `bool`. Additionally
management of the TLS state is pushed into platform-specific modules to
enable different means of managing it, namely the "custom" platform now
has a C function required to implement TLS state for Wasmtime.
* Delay conversion to `Instant` in atomic intrinsics
The `Duration` type is available in `no_std` but the `Instant` type is
not. The intention is to only support the `threads` proposal if `std` is
active but to assist with this split push the `Duration` further into
Wasmtime to avoid using a type that can't be mentioned in `no_std`.
* Gate more parts of Wasmtime on the `profiling` feature
Move `serde_json` to an optional dependency and gate the guest profiler
entirely on the `profiling` feature.
* Refactor conversion to `anyhow::Error` in `wasmtime-environ`
Have a dedicated trait for consuming `self` in addition to a
`Result`-friendly trait.
* Gate `gimli` in Wasmtime on `addr2line`
Cut down the dependency list if `addr2line` isn't enabled since then
the dependency is not used. While here additionally lift the version
requirement for `addr2line` up to the workspace level.
* Update `bindgen!` to have `no_std`-compatible output
Pull most types from Wasmtime's `__internal` module as the source of
truth.
* Use an `Option` for `gc_store` instead of `OnceCell`
No need for synchronization here when mutability is already available in
the necessary contexts.
* Enable embedder-defined host feature detection
* Add `#![no_std]` support to the `wasmtime` crate
This commit enables compiling the `runtime`, `gc`, and `component-model`
features of the `wasmtime` crate on targets that do not have `std`. This
tags the crate as `#![no_std]` and then updates everything internally to
import from `core` or `alloc` and adapt for the various idioms. This
ended up requiring some relatively extensive changes, but nothing too
too bad in the grand scheme of things.
* Require `std` for the perfmap profiling agent
prtest:full
* Fix build on wasm
* Fix windows build
* Remove unused import
* Fix Windows/Unix build without `std` feature
* Fix some doc links
* Remove unused import
* Fix build of wasi-common in isolation
* Fix no_std build on macos
* Re-fix build
* Fix standalone build of wasmtime-cli-flags
* Resolve a merge conflict
* Review comments
* Remove unused import
* Migrate the `wasmtime-environ` crate to `no_std`
This commit migrates the `wasmtime-environ` crate to by default being
tagged with `#![no_std]`. Only the `component-model` and `gc` features
are able to be built without `std`, all other features will implicitly
activate the `std` feature as they currently require it one way or
another. CI is updated to build `wasmtime-environ` with these two
features active on a no_std platform.
This additionally, for the workspace, disables the `std` feature for the
`target-lexicon`, `indexmap`, `object`, and `gimli` dependencies. For
object/gimli all other crates in the workspace now enable the `std`
feature, but for `wasmtime-environ` this activation is omitted.
The `thiserror` dependency was dropped from `wasmtime-environ` and
additionally `hashbrown` was added for explicit usage of maps.
* Always enable `std` for environ for now
prtest:full
* Add some more std features
* Migrate the wasmtime-types crate to no_std
This commit is where no_std for Wasmtime starts to get a bit
interesting. Specifically the `wasmtime-types` crate is the first crate
that depends on some nontrivial crates that also need to be migrated to
`no_std`. This PR disables the default feature of `wasmparser` by
default and additionally does the same for `serde`. This enables them to
compile in `no_std` contexts by default and default features will be
enabled elsewhere in this repository as necessary.
This also opts to drop the `thiserror` dependency entirely in favor of a
manual `Display` implementation with a cfg'd implementation of `Error`.
As before CI checks are added for `wasmtime-types` with a `no_std`
target itself to ensure the crate and all dependencies all avoid `std`.
* Fix adapter build
* Expose `wasmtime-runtime` as `crate::runtime::vm` internally for the `wasmtime` crate
* Rewrite uses of `wasmtime_runtime` to `crate::runtime::vm`
* Remove dep on `wasmtime-runtime` from `wasmtime-cli`
* Move the `wasmtime-runtime` crate into the `wasmtime::runtime::vm` module
* Update labeler for merged crates
* Fix `publish verify`
prtest:full
Looks like GitHub is changing `macos-latest` to arm64 so change the test
builder that test x64 macos to `macos-13` which is the last builder that
wasn't arm64. Additionally drop `macos-14` from the C API tests since
testing `macos-latest` should be sufficient enough.
* Start migrating some Wasmtime crates to no_std
This commit is the first in what will be multiple PRs to migrate
Wasmtime to being compatible with `#![no_std]`. This work is outlined
in #8341 and the rough plan I have in mind is to go on a crate-by-crate
basis and use CI as a "ratchet" to ensure that `no_std` compat is
preserved. In that sense this PR is a bit of a template for future PRs.
This PR migrates a few small crates to `no_std`, basically those that
need no changes beyond simply adding the attribute. The nontrivial parts
introduced in this PR are:
* CI is introduced to verify that a subset of crates can indeed be
built on a `no_std` target. The target selected is
`x86_64-unknown-none` which is known to not have `std` and will result
in a build error if it's attempted to be used.
* The `anyhow` crate, which `wasmtime-jit-icache-coherence` now depends
on, has its `std` feature disabled by default in Wasmtime's workspace.
This means that some crates which require `std` now need to explicitly
enable the feature, but it means that by-default its usage is
appropriate for `no_std`.
The first point should provide CI checks that compatibility with
`no_std` indeed works, at least from an "it compiles" perspective. Note
that it's not sufficient to test with a target like
`x86_64-unknown-linux-gnu` because `extern crate std` will work on that
target, even when `#![no_std]` is active.
The second point however is likely to increase maintenance burden
in Wasmtime unfortunately. Namely we'll inevitably, either here or in
the future, forget to turn on some feature for some crate that's not
covered in CI checks. While I've tried to do my best here in covering it
there's no guarantee that everything will work and the combinatorial
explosion of what could be checked in CI can't all be added to CI.
Instead we'll have to rely on bug fixes, users, and perhaps point
releases to add more use cases to CI over time as we see fit.
* Add another std feature
* Another std feature
* Enable anyhow/std for another crate
* Activate `std` in more crates
* Fix miri build
* Fix compile on riscv64
prtest:full
* Fix min-platform example build
* Fix icache-coherence again
* add cloning for String attributes
* use into_owned instead of to_vec to avoid a clone if possible.
* runs, but does not substitute
* show vars from c program, start cleanup
* tiday
* resolve conflicts
* remove WASI folder
* Add module_builder
add dwarf_package to state for cache
* move dwarf loading to module_environ.rs
pass the dwarf as binary as low as module_environ.rs
* pass dwarf package rather than add to debug_info
* tidy option/result nested if
* revert some toml and whitespace.
* add features cranelift,winch to module_builder and compute_artifacts
remove some `use`s
* address some feedback
remove unused 'use's
* address some feedback
remove unused 'use's
* move wat feature condition to cover whole method.
* More feedback
Another try at wat feature move
* Another try at wat feature move
* change gimli exemption version
add typed-arena exemption
* add None for c-api
* move `use` to #cfg
* fix another config build
* revert unwanted code deletion
* move inner function closer to use
* revert extra param to Module::new
* workaround object crate bug.
* add missing parameter
* add missing parameter
* Merge remote-tracking branch 'origin/main' into dwarf-att-string
# Conflicts:
# crates/wasmtime/src/engine.rs
# crates/wasmtime/src/runtime/module.rs
# src/common.rs
* remove moduke
* use common gimli version of 28.1
* remove wasm feature, revert gimli version
* remove use of object for wasm dwarf
* remove NativeFile workaround, add feature for dwp loading
* sync winch signature
* revert bench api change
* add dwarf for no cache feature
* put back merge loss of module kind
* remove param from docs
* add dwarf fission lldb test
* simplify and include test source
* clang-format
* address feedback, remove packages
add docs
simplify return type
* remove Default use on ModuleTypesBuilder
* Remove an `unwrap()` and use `if let` instead
* Use `&[u8]` instead of `&Vec<u8>`
* Remove an `unwrap()` and return `None` instead
* Clean up some code in `transform_dwarf`
* Clean up some code in `replace_unit_from_split_dwarf`
* Clean up some code in `split_unit`
* Minor refactorings and documentation in `CodeBuilder`
* Restrict visibility of `dwarf_package_binary`
* Revert supply-chain folder changes
* Fix compile error on nightly
* prtest:full
* prtest:full
* prtest:full
* prtest:full
* prtest:full
* prtest:full
* prtest:full
* prtest:full
* use lldb 15
* prtest:full
* prtest:full
* load dwp when loading wasm bytes with path
* correct source file name
* remove debug
---------
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
* Use `--locked` on all `cargo install` in CI
Prevents any updates to rustc or crates from accidentally causing issues
by ensuring that the same set of deps is used over time.
* Remove rust/WASI markdown parser example
The documentation referring to this example was removed in #6994 and
that forgot to remove this as well. This example is building without a
lock file which is causing issues in #8368.
* Don't gate on `cargo vet` for PRs
Gate the `cargo vet` step for the merge queue by default but for PRs
don't fail the PR's CI and instead allow entering the queue with a
failing `cargo vet` entry. This should help make it easier to land a
`cargo vet` entry on `main` outside of a PR while not requiring the PR
is rebased.
* Fix syntax
* Trigger cargo vet failures
* Fix conditional
* Try to fix conditional
* More changes
* Revert "Trigger cargo vet failures"
This reverts commit 14ba8cb52c.
* Update Cargo.lock but also have vet pass
* Try different syntax
* Revert "Update Cargo.lock but also have vet pass"
This reverts commit 1679838848.
* Remove debugging
* Spruce up some comments
* Update docs
Along the same lines as #8280 I occasionally get emails about failures
here but that's not too too useful, so ignore the errors here to get
triaged elsewhere if necessary.
Currently this workflow fails about 50% of the time due to what appears
to be rate limiting issues. I'm personally receiving emails every time
this fails and would rather not continue to receive emails. I've updated
this to allow failure to hopefully stop notifying me about this.
This commit refactors the `wasmtime-runtime` crate to avoid the
`std::panic` module entirely if it's compiled with `panic=abort`. From
an optimization perspective this is not really required since it'll
optimize the same either way with `-Cpanic=abort`, but avoiding
`std::panic` can help make the code a bit more portable. This
refactoring bundles in the `catch_unwind` with the longjmp of the panic
to keep the `#[cfg]` in one location. Callers are then updated as
appropriate.
* Gate support for the wasm `threads` proposal behind a Cargo feature
This commit moves support for the `threads` proposal behind a new
on-by-default Cargo feature: `threads`. This is intended to support
building Wasmtime with fewer runtime dependencies such as those required
for the atomic operations on memories.
This additionally adds the `gc` feature in a few missing places too.
* Fix compile of C API without threads
* Disassemble `*.cwasm` for `compile` disas tests
This commit changes how the `compile` mode of the `disas` test suite
works. Previously this would use `--emit-clif` and run the Cranelift
pipeline for each individual function and use the custom VCode-based
disassembly for instruction output. This commit instead uses the raw
binary coming out of Wasmtime. The ELF file itself is parsed and is
disassembled in a manner similar to Winch tests.
The goal of this commit is somewhat twofold:
* Lay the groundwork to migrate all Winch-based filetests to
`tests/disas`.
* Test the raw output from Cranelift/Wasmtime which includes
optimizations like branch chomping in the `MachBuffer`.
This commit doesn't itself move the Winch tests yet, that's left for a
future commit.
* Update all test expectations for new output
* Fix PR-based CI when too many files are changed
Xinzhao Xu
Nick Fitzgerald
Alex Crichton
Marceline Cramer
Juniper Tyree
jianjunz
FrankReh
yowl
Trevor Elliott