* Update WASI versions to `0.2.0-rc-2023-11-05`
This commit updates the version numbers on `main` to no longer clash
with the 14.0.0 release after #7299. The version number is chosen as the
branch point for the 15.0.0 release of Wasmtime, at which point we'll
update the versions again.
* Update another version
This commit changes the `--dir` argument on the `wasmtime` CLI to be
`HOST::GUEST` rather than `GUEST::HOST`. This matches Docker for example
and is a little more consistent with only `--dir path` where the first
argument is always treated as a host directory.
In terms of breaking-ness the movement from `--mapdir` to `--dir` hasn't
been released with Wasmtime 14 yet so my hope is that this can land on
both `main` and Wasmtime 14.0.0 before it's released to avoid any
breakage other than existing scripts migrating from `--mapdir` to
`--dir`.
* PCC: draw the rest of the owl: fully-working PCC on hello-world Wasm on aarch64
This needed a bit more inference / magic than I was hoping for at first,
specifically around constants and adds. Some instructions can now
generate facts on their output registers, even if not stated. This
breaks away from the "breadcrumbs" idea, but seems to be the most
practical solution to a general problem that we have mini-lowering steps
in various places without careful preservation of PCC facts. Two
particular aspects:
- Constants: amodes on aarch64 can decompose into new
constant-generation instructions, and we need precise ranges on those
to properly check them. To avoid making the ISLE rules nightmarish,
it's best to reuse the existing semantics definitions of the Add* ALU
insts, and add a few rules for MovK/MovZ/MovN.
- Adds: similarly, amodes decompose into de-novo add instructions with
no facts. To handle this, there's now a notion of "propagating" facts
that cause an instruction with a propagating fact on the input to
generate a fact on the output.
Together, these heuristics mean that we'll eagerly generate a fact for
`mem(mt0, 0, 0) + 8 -> mem(mt0, 8, 8)`, but we won't, for example,
generate ranges on every single integer operation.
With these changes and a few other misc fixes, this PR can now check a
nontrivial "hello world" Wasm on aarch64 down to the machine-code level:
```
$ target/release/wasmtime compile -C enable-pcc=y ../wasm-tests/helloworld-rs.wasm
```
* Review feedback.
* Move `wasmtime explore` behind a Cargo feature
Enable this Cargo feature by default, but enable building the CLI
without the `explore` subcommand.
* Move the `wast` subcommand behind a Cargo feature
* Move support for `wat` behind a CLI feature
This was already conditional in crates such as `wasmtime` and this makes
it an optional dependency of the CLI as well.
* Move CLI cache support behind a Cargo feature
Additionally refactor `wasmtime-cli-flags` to not unconditionally pull
in caching support by removing its `default` feature and appropriately
enabling it from the CLI.
* Move `rayon` behind an optional feature
* Move `http-body-util` dependency behind `serve` feature
* Add a Cargo feature for compiling out log statements
This sets the static features of `log` and `tracing` to statically
remove all log statements from the binary to cut down on binary size.
* Move logging support behind a Cargo feature
Enables statically removing logging support in addition to the previous
compiling out log statements themselves.
* Move demangling support behind a Cargo feature
* Enable building the CLI without cranelift
Compile out the `compile` subcommand for example.
* Gate all profiling support behind one feature flag
This commit moves vtune/jitdump support behind a single `profiling`
feature flag that additionally includes the guest profiler dependencies
now too.
* Move support for core dumps behind a feature flag
* Move addr2line behind a feature
* Fix rebase
* Document cargo features and a minimal build
* Tidy up the source a bit
* Rename compile-out-logging
* Document disabling logging
* Note the host architecture as well
* Fix tests
* Fix tests
* Mention debuginfo stripping
* Fix CI configuration for checking features
* Fix book tests
* Update lock file after rebase
* Enable coredump feature by default
While valid in WIT I keep finding it jarring to have different
indentation in `wasi:sockets`. Additionally all other WASI WITs are
using four spaces or two, so make it a bit more consistent.
* Implement support for `thread` in `*.wast` tests
This commit implements support for `thread` and `wait` in `*.wast` files
and imports the upstream spec test suite from the `threads` proposal.
This additionally and hopefully makes it a bit easier to write threaded
tests in the future if necessary too.
* Fix compile of fuzzing
This adds a new `send_request` method to `WasiHttpView`, allowing embedders to
override the default implementation with their own if the desire. The default
implementation behaves exactly as before.
I've also added a few new `wasi-http` tests: one to test the above, and two
others to test streaming and concurrency. These tests are ports of the
`test_wasi_http_echo` and `test_wasi_http_hash_all` tests in the
[Spin](https://github.com/fermyon/spin) integration test suite. The component
they instantiate is likewise ported from the Spin
`wasi-http-rust-streaming-outgoing-body` component.
Fixes#7259
Signed-off-by: Joel Dice <joel.dice@fermyon.com>
* Put versions in all WASI WIT files
This commit starts exercising the versioning feature of WIT by ensuring
that all WASI descriptions have a version associated with them. The
version chosen is 0.2.0 which reflects the upcoming "preview 2" release
where in theory 0.1.0 was claimed by preview1. This is intended to stay
as 0.2.0 for now and we'll determine how best to update these numbers in
the future once preview2 is released.
Closes#7171
* Allow omitting versions in `with` keys
As a convenience for now this enables omitting the version of an
interface from a `with` key. This has a risk of not working well if two
packages are present and one has a version and one doesn't, but that's
left as a PR to fix in the future as the benefit of avoiding repetition
seems good for now.
* Allow omitting versions in trappable_error_types
* Use 0.2.0-rc-2023-10-18 as a version number
* More test fixes
* Fix another test
* Remove usage of `BTreeMap` for compiler flags
No need for a datastructure here really, a simple list with static
strings works alright.
* Fix winch compile and a warning
* Fix test compile
* c-api: Remove reexport of wasmtime crate
This is a follow-up to #6765 to remove this reexport since it was
originally added to use both the C API and the `wasmtime` crate in the
same downstream crate, but this should be possible through Cargo with:
[dependencies]
wasmtime = "13"
wasmtime-c-api = { version = "13", package = "wasmtime-c-api" }
and that way `wasmtime::*` is available as well as `wasmtime_c_api::*`
* Add `pub use wasmtime;`
* Enable threads, multi-memory, and relaxed-simd by default
This commit enables by default three proposals that were advanced to
stage 4 in the proposal process at last week's in-person CG meeting.
These proposals are:
* `threads` - atomic instructions and shared memory
* `multi-memory` - the ability to have more than one linear memory
* `relaxed-simd` - new simd instructions that may differ across
platforms
These proposals are now all enabled by default in Wasmtime. Note that
they can still be selectively disabled via `Config` options.
* Fix an x64-specific test
This turns out to be quite simple: the fundamental operation during
egraph-based optimization is to *merge* eclasses, which is an assertion
that their value is equal. Since the values of either side of the merge
are equal, a fact about one side is a fact about the other, and
vice-versa.
Since we only support *one* fact at most per value, we can't take the
union of all facts; instead, if one side is missing a fact, or if both
sides have exactly the same fact, we keep that one; otherwise we go to a
special "conflict" fact that can't support any check. This is edging
closer to a fact-lattice, but I opted not to go there with a full
meet-function merge yet, for simplicity. (It's a little complex because
of the "minimum fact" we can know about a value based on its type -- if
we're going to do something better, I think we should account for that
too.)
In any case, that complexity mostly isn't needed, because the two cases
that happen in reality are (i) opt rules rewrite to a new node, which
will have no facts at all, so facts just propagate; or (ii) GVN merges
two values in the input program into one, but if both are the same
value, in practice the Wasm PCC frontend (for example) should be
producing the same facts on both values, so the merge is trivial.
* PCC: add facts to global values, parse and print them. No verification yet.
Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
* PCC: propagate facts on GV loads and check them.
Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
* PCC: support propagating facts on iteratively-elaborated GVs as well.
Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
* PCC: fix up Wasmtime uses of GVs after refactors to memflags handling.
Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
* PCC: working end-to-end for static memories!
Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
* PCC: add toplevel Wasmtime option `-C enable-pcc=y`.
* Fix filetests build.
* Review feedback, and blessed test updates due to GV legalization changes.
---------
Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
This is needed for soundness when verifying accesses to memtype fields:
it's not enough to know that we're accessing an offset in `0` up to
`field_offset` inclusive, we need to know the access is actually to
`field_offset`.
The simplest change that validates this turned out to be the most
general one: making ranges two-sided rather than one-sided. The
transform is *mostly* mechanical, but a few new tests verify that ranges
are updated on both sides, and some fail-tests verify that "fuzzily
imprecise" pointers to struct fields fail to validate.
Some notable changes here are:
* The `wasm-tools` crates have enabled the `relaxed-simd`, `threads`,
and `multi-memory` proposals by default. For now I've left these
disabled-by-default in Wasmtime to get enabled in a future PR.
* The `wast` crate has support for parsing `thread` and `wait`
constructs from the `threads` proposal for WebAssembly. They're left
unimplemented for now and return errors. This will get filled in in a
future update.
This removes the need for the awkward "max-range fact is subsumed by
anything" rule noted by @fitzgen in [this
comment](https://github.com/bytecodealliance/wasmtime/pull/7231#discussion_r1358573147).
It also makes checking a little more efficient and logically clear, as
only the facts that the frontend/producer added are verified, rather
than all default facts as well.
* riscv64: Refactor `rotl` rules
Move from `inst.isle` to `lower.isle` since it's the only caller,
reorganize the rules to be a bit cleaner, add immediate shifting
specializations.
* riscv64: Refactor `rotr` lowerings
Same as the prior `rotl` lowerings, move the rules to `lower.isle` and
additionally add constant rules.
* Fix shift-by-128
* Remove empty comments
* cranelift: Add egraph rules for `bswap`
WebAssembly doesn't currently have a byte-swapping instruction so this
commit pattern matches what LLVM currently generates to simplify to a
single `bswap` instruction in CLIF which can be lowered within each
backend to respective native instructions.
* Use an `optimize` instead of `compile` test
This commit optimizes some `select_spectre_guard` patterns which are
frequently generated with the wasm backend when dynamic bounds checks
are enabled by pattern-matching when one of the values being selected is
zero. This enables shaving off a few instructions which can be constant
folded away.
* Support set_fuel in store APIs
Fixes: https://github.com/bytecodealliance/wasmtime/issues/5109
Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
* rename set_fuel to reset_fuel
To make it more clear that consumed fuel is being reset.
Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
* update out of date documentation for fuel in C API
Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
---------
Signed-off-by: Tyler Rockwood <rockwood@redpanda.com>
This commit removes a clause in `register_module` which was preventing
`perf` profiling the first function in a module because it was located
at address 0. This check is relatively dated at this point and I think I
originally added it in error (and/or it was copied from somewhere else),
so this commit removes it.
Some things I noticed from #7239 which are very much not critical but I
figure might be nice-to-haves:
* Move the logging configuration to the `wasmtime-cli-flags` crate with
the other logging configuration to keep it in one place.
* Remove `pretty_env_logger` since `tracing-subscriber` probably
supplants it.
* Don't explicitly inherit env vars in tests since that happens
automatically with `Command`.
* riscv64: Constants are always sign-extended
Skip generating extra sign-extension instructions in this case because
the materialization of a constant will implicitly sign-extend into the
entire register.
* riscv64: Rename `lower_int_compare` helper
Try to reserve `lower_*` as taking a thing and producing a `*Reg`.
Rename this helper to `is_nonzero_cmp` to represent how it's testing for
nonzero and producing a comparison.
* riscv64: Rename some float comparison helpers
* `FCmp` -> `FloatCompare`
* `emit_fcmp` -> `fcmp_to_float_compare`
* `lower_fcmp` -> `lower_float_compare`
Make some room for upcoming integer comparison functions.
* riscv64: Remove `ordered` helper
This is only used by one lowering so inline its definition directly.
* riscv64: Remove the `Icmp` pseudo-instruction
This commit is the first in a few steps to reimplement #6112 and #7113.
The `Icmp` pseudo-instruction is removed here and necessary
functionality is all pushed into ISLE lowerings. This enables deleting
the `lower_br_icmp` helper in `emit.rs` as it's no longer necessary,
meaning that all conditional branches should ideally be generated in
lowering rather than pseudo-instructions to benefit from various
optimizations.
Currently the lowering is the bare-bones minimum to get things working.
This involved creating a helper to lower an `IntegerCompare` into an
`XReg` via negation/swapping args/etc. In generated code this removes
branches and makes `icmp` a straight-line instruction for non-128-bit
arguments.
* riscv64: Remove an unused helper
* riscv64: Optimize comparisons with 0
Use the `x0` register which is always zero where possible and avoid
unnecessary `xor`s against this register.
* riscv64: Specialize `a < $imm`
* riscv64: Optimize Equal/NotEqual against constants
* riscv64: Optimize LessThan with constant argument
* Optimize `a <= $imm`
* riscv64: Optimize `a >= $imm`
* riscv64: Add comment for new helper
* Use i64 in icmp optimizations
Matches the sign-extension that happens at the hardware layer.
* Correct some sign extensions
* riscv64: Don't assume immediates are extended
* riscv64: Fix encoding for `c.addi4spn`
* riscv64: Remove `icmp` lowerings which modify constants
These aren't correct and will need updating
* Add regression test
* riscv64: Fix handling unsigned comparisons with constants
---------
Co-authored-by: Afonso Bordado <afonsobordado@az8.co>
In #7220, we began logging the arguments of WebAssembly `wait` and
`notify` as these instructions are executed. This change prints the
addresses using hexadecimal, which is simply more convenient when
dealing with addressing.
* winch: Add known a subset of known libcalls and improve call emission
This change is a follow up to:
- https://github.com/bytecodealliance/wasmtime/pull/7155
- https://github.com/bytecodealliance/wasmtime/pull/7035
One of the objectives of this change is to make it easy to emit
function calls at the MacroAssembler layer, for cases in which it's
challenging to know ahead-of-time if a particular functionality can be
achieved natively (e.g. rounding and SSE4.2). The original implementation
of function call emission, made this objective difficult to achieve and
it was also difficult to reason about.
I decided to simplify the overall approach to function calls as part of
this PR; in essence, the `call` module now exposes a single function
`FnCall::emit` which is reponsible of gathtering the dependencies and
orchestrating the emission of the call. This new approach deliberately
avoids holding any state regarding the function call for simplicity.
This change also standardizes the usage of `Callee` as the main
entrypoint for function call emission, as of this change 4 `Callee`
types exist (`Local`, `Builtin`, `Import`, `FuncRef`), each callee kind
is mappable to a `CalleeKind` which is the materialized version of
a callee which Cranelift understands.
This change also moves the creation of the `BuiltinFunctions` to the
`ISA` level given that they can be safely used accross multiple function
compilations.
Finally, this change also introduces support for some of the
"well-known" libcalls and hooks those libcalls at the
`MacroAssembler::float_round` callsite.
--
prtest:full
* Review comments
* Remove unnecessary `into_iter`
* Fix remaining lifetime parameter names
* Update to arbitrary 1.3.1
And use workspace dependencies for arbitrary.
* Prune cargo vet's supply-chain files
This is the mechanical changes made by running `cargo vet prune` which was
suggested to me when I ran `cargo vet`.
* PCC: add checking of struct field types on loads and stores.
Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
* PCC: check that blockparams uphold facts appropriately.
Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
* PCC: infer facts based on types' value ranges; add a mock vmctx example.
Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
* Fix formatting.
* Add two fixes required by new default facts:
- Allow any fact at all to subsume a trivially-true range fact that
states a bitslice is within the maximum range for its width.
- Clamp the maximum value of a bitslice at the maximum value for the
slice's width.
---------
Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
* wit: delete wasi logging package
missed this in a code review
* command extended: use include keyword, delete import of logging
* sync http wit definitions
* delete redundant cli test
cli_panic is almost exactly the same as cli_exit_panic, so delete the first one
* split stream pollable lifetimes test into two preview2 tests
Alex Crichton
Chris Fallin
Victor Bieszka
Jeffrey Charles
Joel Dice
Afonso Bordado
Pat Hickey
Tyler Rockwood
Andrew Brown
Saúl Cabrera
Nick Fitzgerald
Kelly Thomas Kline