This new fuzz target exercises sequences of `table.get`s, `table.set`s, and
GCs.
It already found a couple bugs:
* Some leaks due to ref count cycles between stores and host-defined functions
closing over those stores.
* If there are no live references for a PC, Cranelift can avoid emiting an
associated stack map. This was running afoul of a debug assertion.
* Add a spec test fuzzer for Config
This commit adds a new fuzzer which is intended to run on oss-fuzz. This
fuzzer creates and arbitrary `Config` which *should* pass spec tests and
then asserts that it does so. The goal here is to weed out any
accidental bugs in global configuration which could cause
non-spec-compliant behavior.
* Move implementation to `fuzzing` crate
... but turn it back on in CI by default. The `binaryen-sys` crate
builds binaryen from source, which is a drag on CI for a few reasons:
* This is quite large and takes a good deal of time to build
* The debug build directory for binaryen is 4GB large
In an effort to both save time and disk space on the builders this
commit adds a `binaryen` feature to the `wasmtime-fuzz` crate. This
feature is enabled specifically when running the fuzzers on CI, but it
is disabled during the typical `cargo test --all` command. This means
that the test builders should save an extra 4G of space and be a bit
speedier now that they don't build a giant wad of C++.
We'll need to update the OSS-fuzz integration to enable the `binaryen`
feature when executing `cargo fuzz build`, and I'll do that once this
gets closer to landing.
This commit removes the two fuzz targets that we imported from cranelift
when cranelift merged in. These have both uncovered a few issues in the
fuzz targets themselves, for example:
* `translate_module` - this doesn't verify the wasm is valid a head of
time and cranelift is known to panic on translating invalid wasm
modules. We also already do a lot of fuzzing of translation of wasm
modules, so this isn't necessarily buying us anything over what we're
already fuzzing.
* `reader_parse_test` - discovered in #1205 we already found some "bugs"
in this but it may not necessarily rise to the level of "needs to be
run on oss-fuzz for us to find more bugs" yet. It looks like this is
still somewhat internal so we can re-enable when we've got folks to
fix the fuzz bugs coming in.
Closes#1205
* Move compilation into Module from Instance.
* Fix fuzzing
* Use wasmtime::Module in fuzzing crates
Instead of wasmtime_jit.
* Compile eagerly.
* Review fixes.
* Always use the saved name.
* Preserve the former behavior for fuzzing oracle
* Remove usage of `CompilationStrategy` from `Config`
This commit removes the public API usage of the internal
`CompilationStrategy` enumeration from the `Config` type in the
`wasmtime` crate. To do this the `enum` was copied locally into the
crate and renamed `Strategy`. The high-level description of this change
is:
* The `Config::strategy` method now takes a locally-defined `Strategy`
enumeration instead of an internal type.
* The contents of `Strategy` are always the same, not relying on Cargo
features to indicate which variants are present. This avoids
unnecessary downstream `#[cfg]`.
* A `lightbeam` feature was added to the `wasmtime` crate itself to
lightbeam compilation support.
* The `Config::strategy` method is now fallible. It returns a runtime
error if support for the selected strategy wasn't compiled in.
* The `Strategy` enum is listed as `#[non_exhaustive]` so we can safely
add variants over time to it.
This reduces the public crate dependencies of the `wasmtime` crate
itself, removing the need to reach into internal crates even more!
cc #708
* Fix fuzz targets
* Update nightly used to build releases
* Run rustfmt
We only generate *valid* sequences of API calls. To do this, we keep track of
what objects we've already created in earlier API calls via the `Scope` struct.
To generate even-more-pathological sequences of API calls, we use [swarm
testing]:
> In swarm testing, the usual practice of potentially including all features
> in every test case is abandoned. Rather, a large “swarm” of randomly
> generated configurations, each of which omits some features, is used, with
> configurations receiving equal resources.
[swarm testing]: https://www.cs.utah.edu/~regehr/papers/swarm12.pdf
There are more public APIs and instance introspection APIs that we have than
this fuzzer exercises right now. We will need a better generator of valid Wasm
than `wasm-opt -ttf` to really get the most out of those currently-unexercised
APIs, since the Wasm modules generated by `wasm-opt -ttf` don't import and
export a huge variety of things.
When the test case that causes the failure can successfully be disassembled to
WAT, we get logs like this:
```
[2019-11-26T18:48:46Z INFO wasmtime_fuzzing] Wrote WAT disassembly to: /home/fitzgen/wasmtime/crates/fuzzing/target/scratch/8437-0.wat
[2019-11-26T18:48:46Z INFO wasmtime_fuzzing] If this fuzz test fails, copy `/home/fitzgen/wasmtime/crates/fuzzing/target/scratch/8437-0.wat` to `wasmtime/crates/fuzzing/tests/regressions/my-regression.wat` and add the following test to `wasmtime/crates/fuzzing/tests/regressions.rs`:
```
#[test]
fn my_fuzzing_regression_test() {
let data = wat::parse_str(
include_str!("./regressions/my-regression.wat")
).unwrap();
oracles::instantiate(data, CompilationStrategy::Auto)
}
```
```
If the test case cannot be disassembled to WAT, then we get logs like this:
```
[2019-11-26T18:48:46Z INFO wasmtime_fuzzing] Wrote Wasm test case to: /home/fitzgen/wasmtime/crates/fuzzing/target/scratch/8437-0.wasm
[2019-11-26T18:48:46Z INFO wasmtime_fuzzing] Failed to disassemble Wasm into WAT:
Bad magic number (at offset 0)
Stack backtrace:
Run with RUST_LIB_BACKTRACE=1 env variable to display a backtrace
[2019-11-26T18:48:46Z INFO wasmtime_fuzzing] If this fuzz test fails, copy `/home/fitzgen/wasmtime/crates/fuzzing/target/scratch/8437-0.wasm` to `wasmtime/crates/fuzzing/tests/regressions/my-regression.wasm` and add the following test to `wasmtime/crates/fuzzing/tests/regressions.rs`:
```
#[test]
fn my_fuzzing_regression_test() {
let data = include_bytes!("./regressions/my-regression.wasm");
oracles::instantiate(data, CompilationStrategy::Auto)
}
```
```
* Migrate back to `std::` stylistically
This commit moves away from idioms such as `alloc::` and `core::` as
imports of standard data structures and types. Instead it migrates all
crates to uniformly use `std::` for importing standard data structures
and types. This also removes the `std` and `core` features from all
crates to and removes any conditional checking for `feature = "std"`
All of this support was previously added in #407 in an effort to make
wasmtime/cranelift "`no_std` compatible". Unfortunately though this
change comes at a cost:
* The usage of `alloc` and `core` isn't idiomatic. Especially trying to
dual between types like `HashMap` from `std` as well as from
`hashbrown` causes imports to be surprising in some cases.
* Unfortunately there was no CI check that crates were `no_std`, so none
of them actually were. Many crates still imported from `std` or
depended on crates that used `std`.
It's important to note, however, that **this does not mean that wasmtime
will not run in embedded environments**. The style of the code today and
idioms aren't ready in Rust to support this degree of multiplexing and
makes it somewhat difficult to keep up with the style of `wasmtime`.
Instead it's intended that embedded runtime support will be added as
necessary. Currently only `std` is necessary to build `wasmtime`, and
platforms that natively need to execute `wasmtime` will need to use a
Rust target that supports `std`. Note though that not all of `std` needs
to be supported, but instead much of it could be configured off to
return errors, and `wasmtime` would be configured to gracefully handle
errors.
The goal of this PR is to move `wasmtime` back to idiomatic usage of
features/`std`/imports/etc and help development in the short-term.
Long-term when platform concerns arise (if any) they can be addressed by
moving back to `no_std` crates (but fixing the issues mentioned above)
or ensuring that the target in Rust has `std` available.
* Start filling out platform support doc
* Transform DWARF sections into native format for wasm2obj and wasmtime.
Generate DWARF sections based on WASM DWARF.
Ignore some of debug_info/debug_line for dead code.
* Fix test
This adds a feature which allows one to look up an export by name
without knowing what module it's in -- `lookup_global_export` on an
`InstanceContents`.
The main expected use for this is to support APIs where module A
imports a function from module B, and module B needs to access module
A's memory. B can't import it from A in the normal way, because that
would create a dependency cycle. So for now, allow B to look up A's
exported memory dynamically with `lookup_global_export`.
In the future, with reference types and possibly host bindings, we'll be
able to pass references to memory as arguments, which will obviate the
need for this mechanism.
wasmtime-execute is now wasmtime-jit. Move `JITCode` and the TargetIsa
into a new `Compiler` type. `InstancePlus` is no more, with trampoline
functionality now handled by `Compiler`.
Nick Fitzgerald
Alex Crichton
Sergei Pepyakin
Dan Gohman
Josh Triplett
Yury Delendik