cranelift

Commit Graph

Author	SHA1	Message	Date
Nick Fitzgerald	4ac1bedfca	Introduce the `pulley-interpreter` crate (#9008 ) * Introduce the `pulley-interpreter` crate This commit is the first step towards implementing https://github.com/bytecodealliance/rfcs/pull/35 This commit introduces the `pulley-interpreter` crate which contains the Pulley bytecode definition, encoder, decoder, disassembler, and interpreter. This is still very much a work in progress! It is expected that we will tweak encodings and bytecode definitions, that we will overhaul the interpreter (to, for example, optionally support the unstable Rust `explicit_tail_calls` feature), and otherwise make large changes. This is just a starting point to get the ball rolling. Subsequent commits and pull requests will do things like add the Cranelift backend to produce Pulley bytecode from Wasm as well as the runtime integration to run the Pulley interpreter inside Wasmtime. * remove stray fn main * Add small tests for special x registers * Remove now-unused import * always generate 0 pc rel offsets in arbitrary * Add doc_auto_cfg feature for docs.rs * enable all optional features for docs.rs * Consolidate `BytecodeStream::{advance,get1,get2,...}` into `BytecodeStream::read` * fix fuzz targets build * inherit workspace lints in pulley's fuzz crate * Merge fuzz targets into one target; fix a couple small fuzz bugs * Add Pulley to our cargo vet config * Add pulley as a crate to publish * Move Pulley fuzz target into top level fuzz directory	3 months ago
Nick Fitzgerald	9f66134e40	Resolve aliases before inserting values into the live set (#8945 ) * Refactor the internals of `FunctionBuilder::insert_safepoint_spills` into a few smaller methods * Initialize a logger for the `cranelift-fuzzgen` fuzz target * Resolve aliases before inserting values into the live set This fixes a fuzz bug found in the development of https://github.com/bytecodealliance/wasmtime/pull/8941	4 months ago
Nick Fitzgerald	7a37e313d2	Add a fuzz target for exercising bounds checks with various memory configs (#8742 )	5 months ago
Alex Crichton	b4ecea38bc	Add a fuzzer for async wasm (#8440 ) * Add a fuzzer for async wasm This commit revives a very old branch of mine to add a fuzzer for Wasmtime in async mode. This work was originally blocked on llvm/llvm-project#53891 and while that's still an issue it now contains a workaround for that issue. Support for async fuzzing required a good deal of refactorings and changes, and the highlights are: * The main part is that new intrinsics, `__sanitizer_{start,finish}_fiber_switch` are now invoked around the stack-switching routines of fibers. This only works on Unix and is set to only compile when ASAN is enabled (otherwise everything is a noop). This required refactoring of things to get it all in just the right way for ASAN since it appears that these functions not only need to be called but more-or-less need to be adjacent to each other in the code. My guess is that while we're switching ASAN is in a "weird state" and it's not ready to run arbitrary code. * Stacks are a problem. The above issue in LLVM outlines how stacks cannot be deallocated at this time because if the deallocated virtual memory is later used for the heap then ASAN will have a false positive about stack overflow. To handle this stacks are specially handled in asan mode by using a special allocation path that never deallocates stacks. This logic additionally applies to the pooling allocator which uses a different stack allocation strategy with ASAN. With all of the above a new fuzzer is added. This fuzzer generates an arbitrary module, selects an arbitrary means of async (e.g. epochs/fuel), and then tries to execute the exports of the module with various values. In general the fuzzer is looking for crashes/panics as opposed to correct answers as there's no oracle here. This is also intended to stress the code used to switch on and off stacks. * Fix non-async build * Remove unused import * Review comments * Fix compile on MIRI * Fix Windows build	7 months ago
Alex Crichton	1898b8c771	Run all `.wast` tests in fuzzing (#8121 ) Run all `.wast` tests in fuzzing Currently we have a `spectest` fuzzer which uses fuzz input to generate an arbitrary configuration for Wasmtime and then executes the spec test. This ensures that no matter the configuration Wasmtime can pass spec tests. This commit expands this testing to include all `.wast` tests we have in this repository. While we don't have a ton we still have some significant ones like in #8118 which will only reproduce when turning knobs on CPU features. * Fix CLI build * Fix wast testing	8 months ago
Alex Crichton	04c03b31b7	Update the wasm-tools family of crates (#7921 ) * Update the wasm-tools family of crates Pulling in some updates to improve how WIT is managed in this repository. No changes just yet, however, just pulling in the updates first. * Fix tests * Fix fuzzer build	9 months ago
Alex Crichton	5856590fae	Configure workspace lints, enable running some Clippy lints on CI (#7561 ) * Configure Rust lints at the workspace level This commit adds necessary configuration knobs to have lints configured at the workspace level in Wasmtime rather than the crate level. This uses a feature of Cargo first released with 1.74.0 (last week) of the `[workspace.lints]` table. This should help create a more consistent set of lints applied across all crates in our workspace in addition to possibly running select clippy lints on CI as well. * Move `unused_extern_crates` to the workspace level This commit configures a `deny` lint level for the `unused_extern_crates` lint to the workspace level rather than the previous configuration at the individual crate level. * Move `trivial_numeric_casts` to workspace level * Change workspace lint levels to `warn` CI will ensure that these don't get checked into the codebase and otherwise provide fewer speed bumps for in-process development. * Move `unstable_features` lint to workspace level * Move `unused_import_braces` lint to workspace level * Start running Clippy on CI This commit configures our CI to run `cargo clippy --workspace` for all merged PRs. Historically this hasn't been all the feasible due to the amount of configuration required to control the number of warnings on CI, but with Cargo's new `[lint]` table it's possible to have a one-liner to silence all lints from Clippy by default. This commit by default sets the `all` lint in Clippy to `allow` to by-default disable warnings from Clippy. The goal of this PR is to enable selective access to Clippy lints for Wasmtime on CI. * Selectively enable `clippy::cast_sign_loss` This would have fixed #7558 so try to head off future issues with that by warning against this situation in a few crates. This lint is still quite noisy though for Cranelift for example so it's not worthwhile at this time to enable it for the whole workspace. * Fix CI error prtest:full	12 months ago
Nick Fitzgerald	c16540ed2a	Update `arbitrary` to 1.3.1 (#7236 ) * Update to arbitrary 1.3.1 And use workspace dependencies for arbitrary. * Prune cargo vet's supply-chain files This is the mechanical changes made by running `cargo vet prune` which was suggested to me when I ran `cargo vet`.	1 year ago
Saúl Cabrera	a61be19d88	winch(fuzz): Refactor Winch's fuzzing (#6432 ) * winch(fuzz): Refactor Winch's fuzzing This change is a follow-up to the discussion in https://github.com/bytecodealliance/wasmtime/pull/6281. The most notable characteristic of this change is that it enables `winch` by default in the fuzzers. If compilation time is a big enough concern I can add the cargo feature back. I opted to enable `winch` by default for several reasons: * It substantially reduces the `cfg` complexity -- at first I thought I had covered all the places in which a `cfg` check would be needed, but then I realized that I missed the Cranelift specific compiler flags. * It's the fastest route to enable winch by default in the fuzzers, which we want to do eventually -- the only change we'd need at that point would be to get rid of the winch-specific environment variable. * We can get rid of the winch-specific checks in CI for fuzzing * Implement Arbitraty for CompilerStrategy Unconditionally return `Cranelift` for the `Arbitrary` implementation of `CompilerStrategy`. This ensures that `Cranelift` is used as the compiler for all the targets unless explicitly requested otherwise. As of this change, only the differential target overrides the `CompilerStrategy`	1 year ago
Alex Crichton	42f8ab1493	Improve longevity for fuzzing corpus of wasm modules (#6322 ) * Improve longevity for fuzzing corpus of wasm modules This commit is an improvement to the longevity of Wasmtime's corpus of fuzz inputs to the `instantiate` fuzzer. Currently the input to this fuzzers is arbitrary binary data which is a "DNA" of sorts of what to do. This DNA changes over time as we update the fuzzer and add configuration options, for example. When this happens though the meaning of all existing inputs in the corpus changes because they all have slightly different meanings now. The goal of this commit is to improve the usefulness of a historical corpus, with respect to the WebAssembly modules generated, across changes to the DNA. A custom mutator is now provided for the `instantiate` fuzzer. This mutator will not only perform libfuzzer's default mutation for the input but will additionally place an "envelope" around the fuzz input. Namely, the fuzz input is encoded as a valid WebAssembly module where the actual input to the fuzzer is a trailing custom section. When the fuzzer runs over this input it will read the custom section, perform any configuration generation necessary, and then use the envelope module as the actual input to the fuzzer instead of whatever was generated from the fuzz input. This means that when a future update is made to the DNA of a module the interpretation of the fuzz input section will change but the module in question will not change. This means that any interesting shapes of modules with respect to instructions should be preserved over time in theory. Some consequences of this strategy, however, are: * If the DNA changes then it's difficult to produce minor mutations of the original module. This is because mutations generate a module based on the new DNA which is likely much different than the preexisting module. This mainly just means that libFuzzer will have to rediscover how to mutate up into interesting shapes on DNA changes but it'll still be able to retain all the existing interesting modules. Additionally this can be mitigate with the integration of `wasm-mutate` perhaps into these fuzzers as well. * Protection is necessary against libFuzzer itself with respect to the module. The existing fuzzers only expect valid modules to be created, but libFuzzer can now create mutations which leave the trailing section in place, meaning the module is no longer valid. One option is to record a cryptographic hash in the fuzz input section of the previous module, only using the module if the hashes match. This approach will not work over time in the face of binary format changes, however. For example the multi-memory proposal changed binary encodings a year or so ago meaning that any previous fuzz-generated cases would no longer be guaranteed to be valid. The strategy settled by this PR is to pass a flag to the execution function indicating if the module is "known valid" and gracefully handle error if it isn't (for example if it's a prior test case). I'll note that this new strategy of fuzzing is not applied to the `differential` fuzzer. This could theoretically use the same strategy but it relies much more strictly on being able to produce a module with properties like NaN canonicalization, resource limits, fuel to limit execution, etc. While it may be possible to integrate this with `differential` in the future I figured it'd be better to start with the `instantiate` fuzzer and go from there. * Fix doc build	2 years ago
Saúl Cabrera	a1732b2906	winch(fuzz): Initial support for differential fuzzing (#6281 ) * winch(fuzz): Initial support for differential fuzzing This commit introduces initial support for differential fuzzing for Winch. In order to fuzz winch, this change introduces the `winch` cargo feature. When the `winch` cargo feature is enabled the differential fuzz target uses `wasmi` as the differential engine and `wasm-smith` and `single-inst` as the module sources. The intention behind this change is to have a local approach for fuzzing and verifying programs generated by Winch and to have an initial implementation that will allow us to eventually enable this change by default. Currently it's not worth it to enable this change by default given all the filtering that needs to happen to ensure that the generated modules are supported by Winch. It's worth noting that the Wasm filtering code will be temporary, until Winch reaches feature parity in terms of Wasm operators. * Check build targets with the `winch` feature flag * Rename fuzz target feature to `fuzz-winch`	2 years ago
Remo Senekowitsch	7eb8914090	Chaos mode MVP: Skip branch optimization in MachBuffer (#6039 ) * fuzz: Add chaos mode control plane Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Moritz Waser <mzrw.dev@pm.me> * fuzz: Skip branch optimization with chaos mode Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Moritz Waser <mzrw.dev@pm.me> * fuzz: Rename chaos engine -> control plane Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Moritz Waser <mzrw.dev@pm.me> * chaos mode: refactoring ControlPlane to be passed through the call stack by reference Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Remo Senekowitsch <contact@remsle.dev> * fuzz: annotate chaos todos Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Moritz Waser <mzrw.dev@pm.me> * fuzz: cleanup control plane Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Moritz Waser <mzrw.dev@pm.me> * fuzz: remove control plane from compiler context Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Moritz Waser <mzrw.dev@pm.me> * fuzz: move control plane into emit state Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Moritz Waser <mzrw.dev@pm.me> * fuzz: fix remaining compiler errors Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Moritz Waser <mzrw.dev@pm.me> * fix tests * refactor emission state ctrl plane accessors Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Moritz Waser <mzrw.dev@pm.me> * centralize conditional compilation of chaos mode Also cleanup a few straggling dependencies on cranelift-control that aren't needed anymore. Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Moritz Waser <mzrw.dev@pm.me> * add cranelift-control to published crates prtest:full Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Moritz Waser <mzrw.dev@pm.me> * add cranelift-control to public crates Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Moritz Waser <mzrw.dev@pm.me> --------- Co-authored-by: Falk Zwimpfer <24669719+FalkZ@users.noreply.github.com> Co-authored-by: Moritz Waser <mzrw.dev@pm.me> Co-authored-by: Remo Senekowitsch <contact@remsle.dev>	2 years ago
Afonso Bordado	e96214968c	fuzzgen: Move `Arbitrary` structs into the fuzzers (#5820 ) * fuzzgen: Move `FunctionWithIsa` to icache fuzzer * fuzzgen: Move `Testcase` to fuzzgen fuzzer * fuzzgen: Move allowed libcalls to fuzzers * fuzzgen: Centralize printing of testcases	2 years ago
Ulrich Weigand	a2e9a608c1	fuzzgen: Enable s390x and disable unimplemented ops (#5596 ) Also fix assertion failure when using "i128 uext" or "i128 sext" arguments or return values, as discovered by the fuzzer.	2 years ago
Afonso Bordado	d3e6b7bd2a	fuzzgen: Enable riscv64 and disable unimplemented ops (#5502 )	2 years ago
Trevor Elliott	36e5bdfd0e	Fuzz multiple targets in cranelift-icache (#5482 ) Fuzz additional targets in the cranelift-icache target. The list of targets fuzzed is controlled by the targets enabled in fuzz/Cargo.toml. This PR also reworks how instruction disabling is done in function generator, moving the deny-list to a function to make the decision at runtime instead of compile time.	2 years ago
Afonso Bordado	65a3af72c7	fuzzgen: Statistics framework (#4868 ) * cranelift: Add non user trap codes function * cranelift: Add Fuzzgen stats * cranelift: Use `once_cell` and cleanup some stuff * fuzzgen: Remove total_inputs metric * fuzzgen: Filter empty trap codes	2 years ago
Alex Crichton	7b311004b5	Leverage Cargo's workspace inheritance feature (#4905 ) * Leverage Cargo's workspace inheritance feature This commit is an attempt to reduce the complexity of the Cargo manifests in this repository with Cargo's workspace-inheritance feature becoming stable in Rust 1.64.0. This feature allows specifying fields in the root workspace `Cargo.toml` which are then reused throughout the workspace. For example this PR shares definitions such as: * All of the Wasmtime-family of crates now use `version.workspace = true` to have a single location which defines the version number. * All crates use `edition.workspace = true` to have one default edition for the entire workspace. * Common dependencies are listed in `[workspace.dependencies]` to avoid typing the same version number in a lot of different places (e.g. the `wasmparser = "0.89.0"` is now in just one spot. Currently the workspace-inheritance feature doesn't allow having two different versions to inherit, so all of the Cranelift-family of crates still manually specify their version. The inter-crate dependencies, however, are shared amongst the root workspace. This feature can be seen as a method of "preprocessing" of sorts for Cargo manifests. This will help us develop Wasmtime but shouldn't have any actual impact on the published artifacts -- everything's dependency lists are still the same. * Fix wasi-crypto tests	2 years ago
Jamey Sharp	bd870a9d6c	Shrink all SmallVecs by 8 bytes (#4951 ) We weren't using the "union" cargo feature for the smallvec crate, which reduces the size of a SmallVec by one machine word. This feature requires Rust 1.49 but we already require much newer versions. When using Wasmtime to compile pulldown-cmark from Sightglass, this saves a decent amount of memory allocations and writes. According to `valgrind --tool=dhat`: - 6.2MiB (3.69%) less memory allocated over the program's lifetime - 0.5MiB (4.13%) less memory allocated at maximum heap size - 5.5MiB (1.88%) fewer bytes written to - 0.44% fewer instructions executed Sightglass reports a statistically significant runtime improvement too: compilation :: cycles :: benchmarks/pulldown-cmark/benchmark.wasm Δ = 24379323.60 ± 20051394.04 (confidence = 99%) shrink-abiarg-0406da67c.so is 1.01x to 1.13x faster than main-be690a468.so! [227506364 355007998.78 423280514] main-be690a468.so [227686018 330628675.18 406025344] shrink-abiarg-0406da67c.so compilation :: cycles :: benchmarks/spidermonkey/benchmark.wasm Δ = 360151622.56 ± 278294316.90 (confidence = 99%) shrink-abiarg-0406da67c.so is 1.01x to 1.07x faster than main-be690a468.so! [8709162212 8911001926.44 9535111576] main-be690a468.so [5058015392 8550850303.88 9282148438] shrink-abiarg-0406da67c.so compilation :: cycles :: benchmarks/bz2/benchmark.wasm Δ = 6936570.28 ± 6897696.38 (confidence = 99%) shrink-abiarg-0406da67c.so is 1.00x to 1.08x faster than main-be690a468.so! [155810934 175260571.20 234737344] main-be690a468.so [119128240 168324000.92 257451074] shrink-abiarg-0406da67c.so	2 years ago
Jamey Sharp	dd81e5a64f	Don't let fuzz targets import `arbitrary` directly (#4806 ) The version of the `arbitrary` crate used in fuzz targets needs to be the same as the version used in `libfuzzer-sys`. That's why the latter crate re-exports the former. But we need to make sure to consistently use the re-exported version. That's most easily done if that's the only version we have available. However, `fuzz/Cargo.toml` declared a direct dependency on `arbitrary`, making it available for import, and leading to that version being used in a couple places. There were two copies of `arbitrary` built before, even though they were the same version: one with the `derive` feature turned on, through the direct dependency, and one with it turned off when imported through `libfuzzer-sys`. So I haven't specifically tested this but fuzzer builds might be slightly faster now. I have not removed the build-dep on `arbitrary`, because `build.rs` is not invoked by libFuzzer and so it doesn't matter what version of `arbitrary` it uses. Our other crates, like `cranelift-fuzzgen` and `wasmtime-fuzzing`, can still accidentally use a different version of `arbitrary` than the fuzz targets which rely on them. This commit only fixes the direct cases within `fuzz/**`.	2 years ago
Afonso Bordado	9a8bd5be02	cranelift: Add LibCalls to the interpreter (#4782 ) * cranelift: Add libcall handlers to interpreter * cranelift: Fuzz IshlI64 libcall * cranelift: Revert back to fuzzing udivi64 * cranelift: Use sdiv as a fuzz libcall * cranelift: Register Sdiv in fuzzgen * cranelift: Add multiple libcalls to fuzzer * cranelift: Register a single libcall handler * cranelift: Simplify args checking in interpreter * cranelift: Remove unused LibCalls * cranelift: Cleanup interpreter libcall types * cranelift: Fix Interpreter Docs	2 years ago
Alex Crichton	fd98814b96	Port v8 fuzzer to the new framework (#4739 ) * Port v8 fuzzer to the new framework This commit aims to improve the support for the new "meta" differential fuzzer added in #4515 by ensuring that all existing differential fuzzing is migrated to this new fuzzer. This PR includes features such as: * The V8 differential execution is migrated to the new framework. * `Config::set_differential_config` no longer force-disables wasm features, instead allowing them to be enabled as per the fuzz input. * `DiffInstance::{hash, hash}` was replaced with `DiffInstance::get_{memory,global}` to allow more fine-grained assertions. * Support for `FuncRef` and `ExternRef` have been added to `DiffValue` and `DiffValueType`. For now though generating an arbitrary `ExternRef` and `FuncRef` simply generates a null value. * Arbitrary `DiffValue::{F32,F64}` values are guaranteed to use canonical NaN representations to fix an issue with v8 where with the v8 engine we can't communicate non-canonical NaN values through JS. * `DiffEngine::evaluate` allows "successful failure" for cases where engines can't support that particular invocation, for example v8 can't support `v128` arguments or return values. * Smoke tests were added for each engine to ensure that a simple wasm module works at PR-time. * Statistics printed from the main fuzzer now include percentage-rates for chosen engines as well as percentage rates for styles-of-module. There's also a few small refactorings here and there but mostly just things I saw along the way. * Update the fuzzing README	2 years ago
Andrew Brown	9758f5420e	[fuzz] Remove more fuzz targets (#4737 ) * [fuzz] Remove the `differential` fuzz target This functionality is already covered by the `differential_meta` target. * [fuzz] Rename `differential_meta` to `differential` Now that the `differential_meta` fuzz target does everything that the existing `differential` target did and more, it can take over the original name.	2 years ago
Andrew Brown	8b7fb19b1d	[fuzz] Remove some differential fuzz targets (#4735 ) * [fuzz] Remove some differential fuzz targets The changes in #4515 do everything the `differential_spec` and `differential_wasmi` fuzz target already do. These fuzz targets are now redundant and this PR removes them. It also updates the fuzz documentation slightly.	2 years ago
Andrew Brown	5ec92d59d2	[fuzz] Add a meta-differential fuzz target (#4515 ) * [fuzz] Add `Module` enum, refactor `ModuleConfig` This change adds a way to create either a single-instruction module or a regular (big) `wasm-smith` module. It has some slight refactorings in preparation for the use of this new code. * [fuzz] Add `DiffValue` for differential evaluation In order to evaluate functions with randomly-generated values, we needed a common way to generate these values. Using the Wasmtime `Val` type is not great because we would like to be able to implement various traits on the new value type, e.g., to convert `Into` and `From` boxed values of other engines we differentially fuzz against. This new type, `DiffValue`, gives us a common ground for all the conversions and comparisons between the other engine types. * [fuzz] Add interface for differential engines In order to randomly choose an engine to fuzz against, we expect all of the engines to meet a common interface. The traits in this commit allow us to instantiate a module from its binary form, evaluate exported functions, and (possibly) hash the exported items of the instance. This change has some missing pieces, though: - the `wasm-spec-interpreter` needs some work to be able to create instances, evaluate a function by name, and expose exported items - the `v8` engine is not implemented yet due to the complexity of its Rust lifetimes * [fuzz] Use `ModuleFeatures` instead of existing configuration When attempting to use both wasm-smith and single-instruction modules, there is a mismatch in how we communicate what an engine must be able to support. In the first case, we could use the `ModuleConfig`, a wrapper for wasm-smith's `SwarmConfig`, but single-instruction modules do not have a `SwarmConfig`--the many options simply don't apply. Here, we instead add `ModuleFeatures` and adapt a `ModuleConfig` to that. `ModuleFeatures` then becomes the way to communicate what features an engine must support to evaluate functions in a module. * [fuzz] Add a new fuzz target using the meta-differential oracle This change adds the `differential_meta` target to the list of fuzz targets. I expect that sometime soon this could replace the other `differential` targets, as it almost checks all the things those check. The major missing piece is that currently it only chooses single-instruction modules instead of also generating arbitrary modules using `wasm-smith`. Also, this change adds the concept of an ignorable error: some differential engines will choke with certain inputs (e.g., `wasmi` might have an old opcode mapping) which we do not want to flag as fuzz bugs. Here we wrap those errors in `DiffIgnoreError` and then use a new helper trait, `DiffIgnorable`, to downcast and inspect the `anyhow` error to only panic on non-ignorable errors; the ignorable errors are converted to one of the `arbitrary::Error` variants, which we already ignore. [fuzz] Compare `DiffValue` NaNs more leniently Because arithmetic NaNs can contain arbitrary payload bits, checking that two differential executions should produce the same result should relax the comparison of the `F32` and `F64` types (and eventually `V128` as well... TODO). This change adds several considerations, however, so that in the future we make the comparison a bit stricter, e.g., re: canonical NaNs. This change, however, just matches the current logic used by other fuzz targets. * review: allow hashing mutate the instance state @alexcrichton requested that the interface be adapted to accommodate Wasmtime's API, in which even reading from an instance could trigger mutation of the store. * review: refactor where configurations are made compatible See @alexcrichton's [suggestion](https://github.com/bytecodealliance/wasmtime/pull/4515#discussion_r928974376). * review: convert `DiffValueType` using `TryFrom` See @alexcrichton's [comment](https://github.com/bytecodealliance/wasmtime/pull/4515#discussion_r928962394). * review: adapt target implementation to Wasmtime-specific RHS This change is joint work with @alexcrichton to adapt the structure of the fuzz target to his comments [here](https://github.com/bytecodealliance/wasmtime/pull/4515#pullrequestreview-1073247791). This change: - removes `ModuleFeatures` and the `Module` enum (for big and small modules) - upgrades `SingleInstModule` to filter out cases that are not valid for a given `ModuleConfig` - adds `DiffEngine::name()` - constructs each `DiffEngine` using a `ModuleConfig`, eliminating `DiffIgnoreError` completely - prints an execution rate to the `differential_meta` target Still TODO: - `get_exported_function_signatures` could be re-written in terms of the Wasmtime API instead `wasmparser` - the fuzzer crashes eventually, we think due to the signal handler interference between OCaml and Wasmtime - the spec interpreter has several cases that we skip for now but could be fuzzed with further work Co-authored-by: Alex Crichton <alex@alexcrichton.com> * fix: avoid SIGSEGV by explicitly initializing OCaml runtime first * review: use Wasmtime's API to retrieve exported functions Co-authored-by: Alex Crichton <alex@alexcrichton.com>	2 years ago
Benjamin Bouvier	8a9b1a9025	Implement an incremental compilation cache for Cranelift (#4551 ) This is the implementation of https://github.com/bytecodealliance/wasmtime/issues/4155, using the "inverted API" approach suggested by @cfallin (thanks!) in Cranelift, and trait object to provide a backend for an all-included experience in Wasmtime. After the suggestion of Chris, `Function` has been split into mostly two parts: - on the one hand, `FunctionStencil` contains all the fields required during compilation, and that act as a compilation cache key: if two function stencils are the same, then the result of their compilation (`CompiledCodeBase<Stencil>`) will be the same. This makes caching trivial, as the only thing to cache is the `FunctionStencil`. - on the other hand, `FunctionParameters` contain the... function parameters that are required to finalize the result of compilation into a `CompiledCode` (aka `CompiledCodeBase<Final>`) with proper final relocations etc., by applying fixups and so on. Most changes are here to accomodate those requirements, in particular that `FunctionStencil` should be `Hash`able to be used as a key in the cache: - most source locations are now relative to a base source location in the function, and as such they're encoded as `RelSourceLoc` in the `FunctionStencil`. This required changes so that there's no need to explicitly mark a `SourceLoc` as the base source location, it's automatically detected instead the first time a non-default `SourceLoc` is set. - user-defined external names in the `FunctionStencil` (aka before this patch `ExternalName::User { namespace, index }`) are now references into an external table of `UserExternalNameRef -> UserExternalName`, present in the `FunctionParameters`, and must be explicitly declared using `Function::declare_imported_user_function`. - some refactorings have been made for function names: - `ExternalName` was used as the type for a `Function`'s name; while it thus allowed `ExternalName::Libcall` in this place, this would have been quite confusing to use it there. Instead, a new enum `UserFuncName` is introduced for this name, that's either a user-defined function name (the above `UserExternalName`) or a test case name. - The future of `ExternalName` is likely to become a full reference into the `FunctionParameters`'s mapping, instead of being "either a handle for user-defined external names, or the thing itself for other variants". I'm running out of time to do this, and this is not trivial as it implies touching ISLE which I'm less familiar with. The cache computes a sha256 hash of the `FunctionStencil`, and uses this as the cache key. No equality check (using `PartialEq`) is performed in addition to the hash being the same, as we hope that this is sufficient data to avoid collisions. A basic fuzz target has been introduced that tries to do the bare minimum: - check that a function successfully compiled and cached will be also successfully reloaded from the cache, and returns the exact same function. - check that a trivial modification in the external mapping of `UserExternalNameRef -> UserExternalName` hits the cache, and that other modifications don't hit the cache. - This last check is less efficient and less likely to happen, so probably should be rethought a bit. Thanks to both @alexcrichton and @cfallin for your very useful feedback on Zulip. Some numbers show that for a large wasm module we're using internally, this is a 20% compile-time speedup, because so many `FunctionStencil`s are the same, even within a single module. For a group of modules that have a lot of code in common, we get hit rates up to 70% when they're used together. When a single function changes in a wasm module, every other function is reloaded; that's still slower than I expect (between 10% and 50% of the overall compile time), so there's likely room for improvement. Fixes #4155.	2 years ago
Joel Dice	ed8908efcf	implement fuzzing for component types (#4537 ) This addresses #4307. For the static API we generate 100 arbitrary test cases at build time, each of which includes 0-5 parameter types, a result type, and a WAT fragment containing an imported function and an exported function. The exported function calls the imported function, which is implemented by the host. At runtime, the fuzz test selects a test case at random and feeds it zero or more sets of arbitrary parameters and results, checking that values which flow host-to-guest and guest-to-host make the transition unchanged. The fuzz test for the dynamic API follows a similar pattern, the only difference being that test cases are generated at runtime. Signed-off-by: Joel Dice <joel.dice@fermyon.com>	2 years ago
Nick Fitzgerald	46782b18c2	`wasmtime`: Implement fast Wasm stack walking (#4431 ) * Always preserve frame pointers in Wasmtime This allows us to efficiently and simply capture Wasm stacks without maintaining and synchronizing any safety-critical side tables between the compiler and the runtime. * wasmtime: Implement fast Wasm stack walking Why do we want Wasm stack walking to be fast? Because we capture stacks whenever there is a trap and traps actually happen fairly frequently with short-lived programs and WASI's `exit`. Previously, we would rely on generating the system unwind info (e.g. `.eh_frame`) and using the system unwinder (via the `backtrace`crate) to walk the full stack and filter out any non-Wasm stack frames. This can, unfortunately, be slow for two primary reasons: 1. The system unwinder is doing `O(all-kinds-of-frames)` work rather than `O(wasm-frames)` work. 2. System unwind info and the system unwinder need to be much more general than a purpose-built stack walker for Wasm needs to be. It has to handle any kind of stack frame that any compiler might emit where as our Wasm frames are emitted by Cranelift and always have frame pointers. This translates into implementation complexity and general overhead. There can also be unnecessary-for-our-use-cases global synchronization and locks involved, further slowing down stack walking in the presence of multiple threads trying to capture stacks in parallel. This commit introduces a purpose-built stack walker for traversing just our Wasm frames. To find all the sequences of Wasm-to-Wasm stack frames, and ignore non-Wasm stack frames, we keep a linked list of `(entry stack pointer, exit frame pointer)` pairs. This linked list is maintained via Wasm-to-host and host-to-Wasm trampolines. Within a sequence of Wasm-to-Wasm calls, we can use frame pointers (which Cranelift preserves) to find the next older Wasm frame on the stack, and we keep doing this until we reach the entry stack pointer, meaning that the next older frame will be a host frame. The trampolines need to avoid a couple stumbling blocks. First, they need to be compiled ahead of time, since we may not have access to a compiler at runtime (e.g. if the `cranelift` feature is disabled) but still want to be able to call functions that have already been compiled and get stack traces for those functions. Usually this means we would compile the appropriate trampolines inside `Module::new` and the compiled module object would hold the trampolines. However, we also need to support calling host functions that are wrapped into `wasmtime::Func`s and there doesn't exist any ahead-of-time compiled module object to hold the appropriate trampolines: ```rust // Define a host function. let func_type = wasmtime::FuncType::new( vec![wasmtime::ValType::I32], vec![wasmtime::ValType::I32], ); let func = Func::new(&mut store, func_type, \|_, params, results\| { // ... Ok(()) }); // Call that host function. let mut results = vec![wasmtime::Val::I32(0)]; func.call(&[wasmtime::Val::I32(0)], &mut results)?; ``` Therefore, we define one host-to-Wasm trampoline and one Wasm-to-host trampoline in assembly that work for all Wasm and host function signatures. These trampolines are careful to only use volatile registers, avoid touching any register that is an argument in the calling convention ABI, and tail call to the target callee function. This allows forwarding any set of arguments and any returns to and from the callee, while also allowing us to maintain our linked list of Wasm stack and frame pointers before transferring control to the callee. These trampolines are not used in Wasm-to-Wasm calls, only when crossing the host-Wasm boundary, so they do not impose overhead on regular calls. (And if using one trampoline for all host-Wasm boundary crossing ever breaks branch prediction enough in the CPU to become any kind of bottleneck, we can do fun things like have multiple copies of the same trampoline and choose a random copy for each function, sharding the functions across branch predictor entries.) Finally, this commit also ends the use of a synthetic `Module` and allocating a stubbed out `VMContext` for host functions. Instead, we define a `VMHostFuncContext` with its own magic value, similar to `VMComponentContext`, specifically for host functions. <h2>Benchmarks</h2> <h3>Traps and Stack Traces</h3> Large improvements to taking stack traces on traps, ranging from shaving off 64% to 99.95% of the time it used to take. <details> ``` multi-threaded-traps/0 time: [2.5686 us 2.5808 us 2.5934 us] thrpt: [0.0000 elem/s 0.0000 elem/s 0.0000 elem/s] change: time: [-85.419% -85.153% -84.869%] (p = 0.00 < 0.05) thrpt: [+560.90% +573.56% +585.84%] Performance has improved. Found 8 outliers among 100 measurements (8.00%) 4 (4.00%) high mild 4 (4.00%) high severe multi-threaded-traps/1 time: [2.9021 us 2.9167 us 2.9322 us] thrpt: [341.04 Kelem/s 342.86 Kelem/s 344.58 Kelem/s] change: time: [-91.455% -91.294% -91.096%] (p = 0.00 < 0.05) thrpt: [+1023.1% +1048.6% +1070.3%] Performance has improved. Found 6 outliers among 100 measurements (6.00%) 1 (1.00%) high mild 5 (5.00%) high severe multi-threaded-traps/2 time: [2.9996 us 3.0145 us 3.0295 us] thrpt: [660.18 Kelem/s 663.47 Kelem/s 666.76 Kelem/s] change: time: [-94.040% -93.910% -93.762%] (p = 0.00 < 0.05) thrpt: [+1503.1% +1542.0% +1578.0%] Performance has improved. Found 5 outliers among 100 measurements (5.00%) 5 (5.00%) high severe multi-threaded-traps/4 time: [5.5768 us 5.6052 us 5.6364 us] thrpt: [709.68 Kelem/s 713.63 Kelem/s 717.25 Kelem/s] change: time: [-93.193% -93.121% -93.052%] (p = 0.00 < 0.05) thrpt: [+1339.2% +1353.6% +1369.1%] Performance has improved. multi-threaded-traps/8 time: [8.6408 us 9.1212 us 9.5438 us] thrpt: [838.24 Kelem/s 877.08 Kelem/s 925.84 Kelem/s] change: time: [-94.754% -94.473% -94.202%] (p = 0.00 < 0.05) thrpt: [+1624.7% +1709.2% +1806.1%] Performance has improved. multi-threaded-traps/16 time: [10.152 us 10.840 us 11.545 us] thrpt: [1.3858 Melem/s 1.4760 Melem/s 1.5761 Melem/s] change: time: [-97.042% -96.823% -96.577%] (p = 0.00 < 0.05) thrpt: [+2821.5% +3048.1% +3281.1%] Performance has improved. Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) high mild many-modules-registered-traps/1 time: [2.6278 us 2.6361 us 2.6447 us] thrpt: [378.11 Kelem/s 379.35 Kelem/s 380.55 Kelem/s] change: time: [-85.311% -85.108% -84.909%] (p = 0.00 < 0.05) thrpt: [+562.65% +571.51% +580.76%] Performance has improved. Found 9 outliers among 100 measurements (9.00%) 3 (3.00%) high mild 6 (6.00%) high severe many-modules-registered-traps/8 time: [2.6294 us 2.6460 us 2.6623 us] thrpt: [3.0049 Melem/s 3.0235 Melem/s 3.0425 Melem/s] change: time: [-85.895% -85.485% -85.022%] (p = 0.00 < 0.05) thrpt: [+567.63% +588.95% +608.95%] Performance has improved. Found 8 outliers among 100 measurements (8.00%) 3 (3.00%) high mild 5 (5.00%) high severe many-modules-registered-traps/64 time: [2.6218 us 2.6329 us 2.6452 us] thrpt: [24.195 Melem/s 24.308 Melem/s 24.411 Melem/s] change: time: [-93.629% -93.551% -93.470%] (p = 0.00 < 0.05) thrpt: [+1431.4% +1450.6% +1469.5%] Performance has improved. Found 3 outliers among 100 measurements (3.00%) 3 (3.00%) high mild many-modules-registered-traps/512 time: [2.6569 us 2.6737 us 2.6923 us] thrpt: [190.17 Melem/s 191.50 Melem/s 192.71 Melem/s] change: time: [-99.277% -99.268% -99.260%] (p = 0.00 < 0.05) thrpt: [+13417% +13566% +13731%] Performance has improved. Found 4 outliers among 100 measurements (4.00%) 4 (4.00%) high mild many-modules-registered-traps/4096 time: [2.7258 us 2.7390 us 2.7535 us] thrpt: [1.4876 Gelem/s 1.4955 Gelem/s 1.5027 Gelem/s] change: time: [-99.956% -99.955% -99.955%] (p = 0.00 < 0.05) thrpt: [+221417% +223380% +224881%] Performance has improved. Found 2 outliers among 100 measurements (2.00%) 1 (1.00%) high mild 1 (1.00%) high severe many-stack-frames-traps/1 time: [1.4658 us 1.4719 us 1.4784 us] thrpt: [676.39 Kelem/s 679.38 Kelem/s 682.21 Kelem/s] change: time: [-90.368% -89.947% -89.586%] (p = 0.00 < 0.05) thrpt: [+860.23% +894.72% +938.21%] Performance has improved. Found 8 outliers among 100 measurements (8.00%) 5 (5.00%) high mild 3 (3.00%) high severe many-stack-frames-traps/8 time: [2.4772 us 2.4870 us 2.4973 us] thrpt: [3.2034 Melem/s 3.2167 Melem/s 3.2294 Melem/s] change: time: [-85.550% -85.370% -85.199%] (p = 0.00 < 0.05) thrpt: [+575.65% +583.51% +592.03%] Performance has improved. Found 8 outliers among 100 measurements (8.00%) 4 (4.00%) high mild 4 (4.00%) high severe many-stack-frames-traps/64 time: [10.109 us 10.171 us 10.236 us] thrpt: [6.2525 Melem/s 6.2925 Melem/s 6.3309 Melem/s] change: time: [-78.144% -77.797% -77.336%] (p = 0.00 < 0.05) thrpt: [+341.22% +350.38% +357.55%] Performance has improved. Found 7 outliers among 100 measurements (7.00%) 5 (5.00%) high mild 2 (2.00%) high severe many-stack-frames-traps/512 time: [126.16 us 126.54 us 126.96 us] thrpt: [4.0329 Melem/s 4.0461 Melem/s 4.0583 Melem/s] change: time: [-65.364% -64.933% -64.453%] (p = 0.00 < 0.05) thrpt: [+181.32% +185.17% +188.71%] Performance has improved. Found 4 outliers among 100 measurements (4.00%) 4 (4.00%) high severe ``` </details> <h3>Calls</h3> There is, however, a small regression in raw Wasm-to-host and host-to-Wasm call performance due the new trampolines. It seems to be on the order of about 2-10 nanoseconds per call, depending on the benchmark. I believe this regression is ultimately acceptable because 1. this overhead will be vastly dominated by whatever work a non-nop callee actually does, 2. we will need these trampolines, or something like them, when implementing the Wasm exceptions proposal to do things like translate Wasm's exceptions into Rust's `Result`s, 3. and because the performance improvements to trapping and capturing stack traces are of such a larger magnitude than this call regressions. <details> ``` sync/no-hook/host-to-wasm - typed - nop time: [28.683 ns 28.757 ns 28.844 ns] change: [+16.472% +17.183% +17.904%] (p = 0.00 < 0.05) Performance has regressed. Found 10 outliers among 100 measurements (10.00%) 1 (1.00%) low mild 4 (4.00%) high mild 5 (5.00%) high severe sync/no-hook/host-to-wasm - untyped - nop time: [42.515 ns 42.652 ns 42.841 ns] change: [+12.371% +14.614% +17.462%] (p = 0.00 < 0.05) Performance has regressed. Found 11 outliers among 100 measurements (11.00%) 1 (1.00%) high mild 10 (10.00%) high severe sync/no-hook/host-to-wasm - unchecked - nop time: [33.936 ns 34.052 ns 34.179 ns] change: [+25.478% +26.938% +28.369%] (p = 0.00 < 0.05) Performance has regressed. Found 9 outliers among 100 measurements (9.00%) 7 (7.00%) high mild 2 (2.00%) high severe sync/no-hook/host-to-wasm - typed - nop-params-and-results time: [34.290 ns 34.388 ns 34.502 ns] change: [+40.802% +42.706% +44.526%] (p = 0.00 < 0.05) Performance has regressed. Found 13 outliers among 100 measurements (13.00%) 5 (5.00%) high mild 8 (8.00%) high severe sync/no-hook/host-to-wasm - untyped - nop-params-and-results time: [62.546 ns 62.721 ns 62.919 ns] change: [+2.5014% +3.6319% +4.8078%] (p = 0.00 < 0.05) Performance has regressed. Found 12 outliers among 100 measurements (12.00%) 2 (2.00%) high mild 10 (10.00%) high severe sync/no-hook/host-to-wasm - unchecked - nop-params-and-results time: [42.609 ns 42.710 ns 42.831 ns] change: [+20.966% +22.282% +23.475%] (p = 0.00 < 0.05) Performance has regressed. Found 11 outliers among 100 measurements (11.00%) 4 (4.00%) high mild 7 (7.00%) high severe sync/hook-sync/host-to-wasm - typed - nop time: [29.546 ns 29.675 ns 29.818 ns] change: [+20.693% +21.794% +22.836%] (p = 0.00 < 0.05) Performance has regressed. Found 5 outliers among 100 measurements (5.00%) 3 (3.00%) high mild 2 (2.00%) high severe sync/hook-sync/host-to-wasm - untyped - nop time: [45.448 ns 45.699 ns 45.961 ns] change: [+17.204% +18.514% +19.590%] (p = 0.00 < 0.05) Performance has regressed. Found 14 outliers among 100 measurements (14.00%) 4 (4.00%) high mild 10 (10.00%) high severe sync/hook-sync/host-to-wasm - unchecked - nop time: [34.334 ns 34.437 ns 34.558 ns] change: [+23.225% +24.477% +25.886%] (p = 0.00 < 0.05) Performance has regressed. Found 12 outliers among 100 measurements (12.00%) 5 (5.00%) high mild 7 (7.00%) high severe sync/hook-sync/host-to-wasm - typed - nop-params-and-results time: [36.594 ns 36.763 ns 36.974 ns] change: [+41.967% +47.261% +52.086%] (p = 0.00 < 0.05) Performance has regressed. Found 12 outliers among 100 measurements (12.00%) 3 (3.00%) high mild 9 (9.00%) high severe sync/hook-sync/host-to-wasm - untyped - nop-params-and-results time: [63.541 ns 63.831 ns 64.194 ns] change: [-4.4337% -0.6855% +2.7134%] (p = 0.73 > 0.05) No change in performance detected. Found 8 outliers among 100 measurements (8.00%) 6 (6.00%) high mild 2 (2.00%) high severe sync/hook-sync/host-to-wasm - unchecked - nop-params-and-results time: [43.968 ns 44.169 ns 44.437 ns] change: [+18.772% +21.802% +24.623%] (p = 0.00 < 0.05) Performance has regressed. Found 15 outliers among 100 measurements (15.00%) 3 (3.00%) high mild 12 (12.00%) high severe async/no-hook/host-to-wasm - typed - nop time: [4.9612 us 4.9743 us 4.9889 us] change: [+9.9493% +11.911% +13.502%] (p = 0.00 < 0.05) Performance has regressed. Found 10 outliers among 100 measurements (10.00%) 6 (6.00%) high mild 4 (4.00%) high severe async/no-hook/host-to-wasm - untyped - nop time: [5.0030 us 5.0211 us 5.0439 us] change: [+10.841% +11.873% +12.977%] (p = 0.00 < 0.05) Performance has regressed. Found 10 outliers among 100 measurements (10.00%) 3 (3.00%) high mild 7 (7.00%) high severe async/no-hook/host-to-wasm - typed - nop-params-and-results time: [4.9273 us 4.9468 us 4.9700 us] change: [+4.7381% +6.8445% +8.8238%] (p = 0.00 < 0.05) Performance has regressed. Found 14 outliers among 100 measurements (14.00%) 5 (5.00%) high mild 9 (9.00%) high severe async/no-hook/host-to-wasm - untyped - nop-params-and-results time: [5.1151 us 5.1338 us 5.1555 us] change: [+9.5335% +11.290% +13.044%] (p = 0.00 < 0.05) Performance has regressed. Found 16 outliers among 100 measurements (16.00%) 3 (3.00%) high mild 13 (13.00%) high severe async/hook-sync/host-to-wasm - typed - nop time: [4.9330 us 4.9394 us 4.9467 us] change: [+10.046% +11.038% +12.035%] (p = 0.00 < 0.05) Performance has regressed. Found 12 outliers among 100 measurements (12.00%) 5 (5.00%) high mild 7 (7.00%) high severe async/hook-sync/host-to-wasm - untyped - nop time: [5.0073 us 5.0183 us 5.0310 us] change: [+9.3828% +10.565% +11.752%] (p = 0.00 < 0.05) Performance has regressed. Found 8 outliers among 100 measurements (8.00%) 3 (3.00%) high mild 5 (5.00%) high severe async/hook-sync/host-to-wasm - typed - nop-params-and-results time: [4.9610 us 4.9839 us 5.0097 us] change: [+9.0857% +11.513% +14.359%] (p = 0.00 < 0.05) Performance has regressed. Found 13 outliers among 100 measurements (13.00%) 7 (7.00%) high mild 6 (6.00%) high severe async/hook-sync/host-to-wasm - untyped - nop-params-and-results time: [5.0995 us 5.1272 us 5.1617 us] change: [+9.3600% +11.506% +13.809%] (p = 0.00 < 0.05) Performance has regressed. Found 10 outliers among 100 measurements (10.00%) 6 (6.00%) high mild 4 (4.00%) high severe async-pool/no-hook/host-to-wasm - typed - nop time: [2.4242 us 2.4316 us 2.4396 us] change: [+7.8756% +8.8803% +9.8346%] (p = 0.00 < 0.05) Performance has regressed. Found 8 outliers among 100 measurements (8.00%) 5 (5.00%) high mild 3 (3.00%) high severe async-pool/no-hook/host-to-wasm - untyped - nop time: [2.5102 us 2.5155 us 2.5210 us] change: [+12.130% +13.194% +14.270%] (p = 0.00 < 0.05) Performance has regressed. Found 12 outliers among 100 measurements (12.00%) 4 (4.00%) high mild 8 (8.00%) high severe async-pool/no-hook/host-to-wasm - typed - nop-params-and-results time: [2.4203 us 2.4310 us 2.4440 us] change: [+4.0380% +6.3623% +8.7534%] (p = 0.00 < 0.05) Performance has regressed. Found 14 outliers among 100 measurements (14.00%) 5 (5.00%) high mild 9 (9.00%) high severe async-pool/no-hook/host-to-wasm - untyped - nop-params-and-results time: [2.5501 us 2.5593 us 2.5700 us] change: [+8.8802% +10.976% +12.937%] (p = 0.00 < 0.05) Performance has regressed. Found 16 outliers among 100 measurements (16.00%) 5 (5.00%) high mild 11 (11.00%) high severe async-pool/hook-sync/host-to-wasm - typed - nop time: [2.4135 us 2.4190 us 2.4254 us] change: [+8.3640% +9.3774% +10.435%] (p = 0.00 < 0.05) Performance has regressed. Found 11 outliers among 100 measurements (11.00%) 6 (6.00%) high mild 5 (5.00%) high severe async-pool/hook-sync/host-to-wasm - untyped - nop time: [2.5172 us 2.5248 us 2.5357 us] change: [+11.543% +12.750% +13.982%] (p = 0.00 < 0.05) Performance has regressed. Found 8 outliers among 100 measurements (8.00%) 1 (1.00%) high mild 7 (7.00%) high severe async-pool/hook-sync/host-to-wasm - typed - nop-params-and-results time: [2.4214 us 2.4353 us 2.4532 us] change: [+1.5158% +5.0872% +8.6765%] (p = 0.00 < 0.05) Performance has regressed. Found 15 outliers among 100 measurements (15.00%) 2 (2.00%) high mild 13 (13.00%) high severe async-pool/hook-sync/host-to-wasm - untyped - nop-params-and-results time: [2.5499 us 2.5607 us 2.5748 us] change: [+10.146% +12.459% +14.919%] (p = 0.00 < 0.05) Performance has regressed. Found 18 outliers among 100 measurements (18.00%) 3 (3.00%) high mild 15 (15.00%) high severe sync/no-hook/wasm-to-host - nop - typed time: [6.6135 ns 6.6288 ns 6.6452 ns] change: [+37.927% +38.837% +39.869%] (p = 0.00 < 0.05) Performance has regressed. Found 7 outliers among 100 measurements (7.00%) 2 (2.00%) high mild 5 (5.00%) high severe sync/no-hook/wasm-to-host - nop-params-and-results - typed time: [15.930 ns 15.993 ns 16.067 ns] change: [+3.9583% +5.6286% +7.2430%] (p = 0.00 < 0.05) Performance has regressed. Found 12 outliers among 100 measurements (12.00%) 11 (11.00%) high mild 1 (1.00%) high severe sync/no-hook/wasm-to-host - nop - untyped time: [20.596 ns 20.640 ns 20.690 ns] change: [+4.3293% +5.2047% +6.0935%] (p = 0.00 < 0.05) Performance has regressed. Found 10 outliers among 100 measurements (10.00%) 5 (5.00%) high mild 5 (5.00%) high severe sync/no-hook/wasm-to-host - nop-params-and-results - untyped time: [42.659 ns 42.882 ns 43.159 ns] change: [-2.1466% -0.5079% +1.2554%] (p = 0.58 > 0.05) No change in performance detected. Found 15 outliers among 100 measurements (15.00%) 1 (1.00%) high mild 14 (14.00%) high severe sync/no-hook/wasm-to-host - nop - unchecked time: [10.671 ns 10.691 ns 10.713 ns] change: [+83.911% +87.620% +92.062%] (p = 0.00 < 0.05) Performance has regressed. Found 9 outliers among 100 measurements (9.00%) 2 (2.00%) high mild 7 (7.00%) high severe sync/no-hook/wasm-to-host - nop-params-and-results - unchecked time: [11.136 ns 11.190 ns 11.263 ns] change: [-29.719% -28.446% -27.029%] (p = 0.00 < 0.05) Performance has improved. Found 14 outliers among 100 measurements (14.00%) 4 (4.00%) high mild 10 (10.00%) high severe sync/hook-sync/wasm-to-host - nop - typed time: [6.7964 ns 6.8087 ns 6.8226 ns] change: [+21.531% +24.206% +27.331%] (p = 0.00 < 0.05) Performance has regressed. Found 14 outliers among 100 measurements (14.00%) 4 (4.00%) high mild 10 (10.00%) high severe sync/hook-sync/wasm-to-host - nop-params-and-results - typed time: [15.865 ns 15.921 ns 15.985 ns] change: [+4.8466% +6.3330% +7.8317%] (p = 0.00 < 0.05) Performance has regressed. Found 16 outliers among 100 measurements (16.00%) 3 (3.00%) high mild 13 (13.00%) high severe sync/hook-sync/wasm-to-host - nop - untyped time: [21.505 ns 21.587 ns 21.677 ns] change: [+8.0908% +9.1943% +10.254%] (p = 0.00 < 0.05) Performance has regressed. Found 8 outliers among 100 measurements (8.00%) 4 (4.00%) high mild 4 (4.00%) high severe sync/hook-sync/wasm-to-host - nop-params-and-results - untyped time: [44.018 ns 44.128 ns 44.261 ns] change: [-1.4671% -0.0458% +1.2443%] (p = 0.94 > 0.05) No change in performance detected. Found 14 outliers among 100 measurements (14.00%) 5 (5.00%) high mild 9 (9.00%) high severe sync/hook-sync/wasm-to-host - nop - unchecked time: [11.264 ns 11.326 ns 11.387 ns] change: [+80.225% +81.659% +83.068%] (p = 0.00 < 0.05) Performance has regressed. Found 6 outliers among 100 measurements (6.00%) 3 (3.00%) high mild 3 (3.00%) high severe sync/hook-sync/wasm-to-host - nop-params-and-results - unchecked time: [11.816 ns 11.865 ns 11.920 ns] change: [-29.152% -28.040% -26.957%] (p = 0.00 < 0.05) Performance has improved. Found 14 outliers among 100 measurements (14.00%) 8 (8.00%) high mild 6 (6.00%) high severe async/no-hook/wasm-to-host - nop - typed time: [6.6221 ns 6.6385 ns 6.6569 ns] change: [+43.618% +44.755% +45.965%] (p = 0.00 < 0.05) Performance has regressed. Found 13 outliers among 100 measurements (13.00%) 6 (6.00%) high mild 7 (7.00%) high severe async/no-hook/wasm-to-host - nop-params-and-results - typed time: [15.884 ns 15.929 ns 15.983 ns] change: [+3.5987% +5.2053% +6.7846%] (p = 0.00 < 0.05) Performance has regressed. Found 16 outliers among 100 measurements (16.00%) 3 (3.00%) high mild 13 (13.00%) high severe async/no-hook/wasm-to-host - nop - untyped time: [20.615 ns 20.702 ns 20.821 ns] change: [+6.9799% +8.1212% +9.2819%] (p = 0.00 < 0.05) Performance has regressed. Found 10 outliers among 100 measurements (10.00%) 2 (2.00%) high mild 8 (8.00%) high severe async/no-hook/wasm-to-host - nop-params-and-results - untyped time: [41.956 ns 42.207 ns 42.521 ns] change: [-4.3057% -2.7730% -1.2428%] (p = 0.00 < 0.05) Performance has improved. Found 14 outliers among 100 measurements (14.00%) 3 (3.00%) high mild 11 (11.00%) high severe async/no-hook/wasm-to-host - nop - unchecked time: [10.440 ns 10.474 ns 10.513 ns] change: [+83.959% +85.826% +87.541%] (p = 0.00 < 0.05) Performance has regressed. Found 11 outliers among 100 measurements (11.00%) 5 (5.00%) high mild 6 (6.00%) high severe async/no-hook/wasm-to-host - nop-params-and-results - unchecked time: [11.476 ns 11.512 ns 11.554 ns] change: [-29.857% -28.383% -26.978%] (p = 0.00 < 0.05) Performance has improved. Found 12 outliers among 100 measurements (12.00%) 1 (1.00%) low mild 6 (6.00%) high mild 5 (5.00%) high severe async/no-hook/wasm-to-host - nop - async-typed time: [26.427 ns 26.478 ns 26.532 ns] change: [+6.5730% +7.4676% +8.3983%] (p = 0.00 < 0.05) Performance has regressed. Found 9 outliers among 100 measurements (9.00%) 2 (2.00%) high mild 7 (7.00%) high severe async/no-hook/wasm-to-host - nop-params-and-results - async-typed time: [28.557 ns 28.693 ns 28.880 ns] change: [+1.9099% +3.7332% +5.9731%] (p = 0.00 < 0.05) Performance has regressed. Found 15 outliers among 100 measurements (15.00%) 1 (1.00%) high mild 14 (14.00%) high severe async/hook-sync/wasm-to-host - nop - typed time: [6.7488 ns 6.7630 ns 6.7784 ns] change: [+19.935% +22.080% +23.683%] (p = 0.00 < 0.05) Performance has regressed. Found 9 outliers among 100 measurements (9.00%) 4 (4.00%) high mild 5 (5.00%) high severe async/hook-sync/wasm-to-host - nop-params-and-results - typed time: [15.928 ns 16.031 ns 16.149 ns] change: [+5.5188% +6.9567% +8.3839%] (p = 0.00 < 0.05) Performance has regressed. Found 11 outliers among 100 measurements (11.00%) 9 (9.00%) high mild 2 (2.00%) high severe async/hook-sync/wasm-to-host - nop - untyped time: [21.930 ns 22.114 ns 22.296 ns] change: [+4.6674% +7.7588% +10.375%] (p = 0.00 < 0.05) Performance has regressed. Found 4 outliers among 100 measurements (4.00%) 3 (3.00%) high mild 1 (1.00%) high severe async/hook-sync/wasm-to-host - nop-params-and-results - untyped time: [42.684 ns 42.858 ns 43.081 ns] change: [-5.2957% -3.4693% -1.6217%] (p = 0.00 < 0.05) Performance has improved. Found 14 outliers among 100 measurements (14.00%) 2 (2.00%) high mild 12 (12.00%) high severe async/hook-sync/wasm-to-host - nop - unchecked time: [11.026 ns 11.053 ns 11.086 ns] change: [+70.751% +72.378% +73.961%] (p = 0.00 < 0.05) Performance has regressed. Found 10 outliers among 100 measurements (10.00%) 5 (5.00%) high mild 5 (5.00%) high severe async/hook-sync/wasm-to-host - nop-params-and-results - unchecked time: [11.840 ns 11.900 ns 11.982 ns] change: [-27.977% -26.584% -24.887%] (p = 0.00 < 0.05) Performance has improved. Found 18 outliers among 100 measurements (18.00%) 3 (3.00%) high mild 15 (15.00%) high severe async/hook-sync/wasm-to-host - nop - async-typed time: [27.601 ns 27.709 ns 27.882 ns] change: [+8.1781% +9.1102% +10.030%] (p = 0.00 < 0.05) Performance has regressed. Found 11 outliers among 100 measurements (11.00%) 2 (2.00%) low mild 3 (3.00%) high mild 6 (6.00%) high severe async/hook-sync/wasm-to-host - nop-params-and-results - async-typed time: [28.955 ns 29.174 ns 29.413 ns] change: [+1.1226% +3.0366% +5.1126%] (p = 0.00 < 0.05) Performance has regressed. Found 13 outliers among 100 measurements (13.00%) 7 (7.00%) high mild 6 (6.00%) high severe async-pool/no-hook/wasm-to-host - nop - typed time: [6.5626 ns 6.5733 ns 6.5851 ns] change: [+40.561% +42.307% +44.514%] (p = 0.00 < 0.05) Performance has regressed. Found 9 outliers among 100 measurements (9.00%) 5 (5.00%) high mild 4 (4.00%) high severe async-pool/no-hook/wasm-to-host - nop-params-and-results - typed time: [15.820 ns 15.886 ns 15.969 ns] change: [+4.1044% +5.7928% +7.7122%] (p = 0.00 < 0.05) Performance has regressed. Found 17 outliers among 100 measurements (17.00%) 4 (4.00%) high mild 13 (13.00%) high severe async-pool/no-hook/wasm-to-host - nop - untyped time: [20.481 ns 20.521 ns 20.566 ns] change: [+6.7962% +7.6950% +8.7612%] (p = 0.00 < 0.05) Performance has regressed. Found 11 outliers among 100 measurements (11.00%) 6 (6.00%) high mild 5 (5.00%) high severe async-pool/no-hook/wasm-to-host - nop-params-and-results - untyped time: [41.834 ns 41.998 ns 42.189 ns] change: [-3.8185% -2.2687% -0.7541%] (p = 0.01 < 0.05) Change within noise threshold. Found 13 outliers among 100 measurements (13.00%) 3 (3.00%) high mild 10 (10.00%) high severe async-pool/no-hook/wasm-to-host - nop - unchecked time: [10.353 ns 10.380 ns 10.414 ns] change: [+82.042% +84.591% +87.205%] (p = 0.00 < 0.05) Performance has regressed. Found 7 outliers among 100 measurements (7.00%) 4 (4.00%) high mild 3 (3.00%) high severe async-pool/no-hook/wasm-to-host - nop-params-and-results - unchecked time: [11.123 ns 11.168 ns 11.228 ns] change: [-30.813% -29.285% -27.874%] (p = 0.00 < 0.05) Performance has improved. Found 12 outliers among 100 measurements (12.00%) 11 (11.00%) high mild 1 (1.00%) high severe async-pool/no-hook/wasm-to-host - nop - async-typed time: [27.442 ns 27.528 ns 27.638 ns] change: [+7.5215% +9.9795% +12.266%] (p = 0.00 < 0.05) Performance has regressed. Found 18 outliers among 100 measurements (18.00%) 3 (3.00%) high mild 15 (15.00%) high severe async-pool/no-hook/wasm-to-host - nop-params-and-results - async-typed time: [29.014 ns 29.148 ns 29.312 ns] change: [+2.0227% +3.4722% +4.9047%] (p = 0.00 < 0.05) Performance has regressed. Found 7 outliers among 100 measurements (7.00%) 6 (6.00%) high mild 1 (1.00%) high severe async-pool/hook-sync/wasm-to-host - nop - typed time: [6.7916 ns 6.8116 ns 6.8325 ns] change: [+20.937% +22.050% +23.281%] (p = 0.00 < 0.05) Performance has regressed. Found 11 outliers among 100 measurements (11.00%) 5 (5.00%) high mild 6 (6.00%) high severe async-pool/hook-sync/wasm-to-host - nop-params-and-results - typed time: [15.917 ns 15.975 ns 16.051 ns] change: [+4.6404% +6.4217% +8.3075%] (p = 0.00 < 0.05) Performance has regressed. Found 16 outliers among 100 measurements (16.00%) 5 (5.00%) high mild 11 (11.00%) high severe async-pool/hook-sync/wasm-to-host - nop - untyped time: [21.558 ns 21.612 ns 21.679 ns] change: [+8.1158% +9.1409% +10.217%] (p = 0.00 < 0.05) Performance has regressed. Found 9 outliers among 100 measurements (9.00%) 2 (2.00%) high mild 7 (7.00%) high severe async-pool/hook-sync/wasm-to-host - nop-params-and-results - untyped time: [42.475 ns 42.614 ns 42.775 ns] change: [-6.3613% -4.4709% -2.7647%] (p = 0.00 < 0.05) Performance has improved. Found 18 outliers among 100 measurements (18.00%) 3 (3.00%) high mild 15 (15.00%) high severe async-pool/hook-sync/wasm-to-host - nop - unchecked time: [11.150 ns 11.195 ns 11.247 ns] change: [+74.424% +77.056% +79.811%] (p = 0.00 < 0.05) Performance has regressed. Found 14 outliers among 100 measurements (14.00%) 3 (3.00%) high mild 11 (11.00%) high severe async-pool/hook-sync/wasm-to-host - nop-params-and-results - unchecked time: [11.639 ns 11.695 ns 11.760 ns] change: [-30.212% -29.023% -27.954%] (p = 0.00 < 0.05) Performance has improved. Found 15 outliers among 100 measurements (15.00%) 7 (7.00%) high mild 8 (8.00%) high severe async-pool/hook-sync/wasm-to-host - nop - async-typed time: [27.480 ns 27.712 ns 27.984 ns] change: [+2.9764% +6.5061% +9.8914%] (p = 0.00 < 0.05) Performance has regressed. Found 8 outliers among 100 measurements (8.00%) 6 (6.00%) high mild 2 (2.00%) high severe async-pool/hook-sync/wasm-to-host - nop-params-and-results - async-typed time: [29.218 ns 29.380 ns 29.600 ns] change: [+5.2283% +7.7247% +10.822%] (p = 0.00 < 0.05) Performance has regressed. Found 16 outliers among 100 measurements (16.00%) 2 (2.00%) high mild 14 (14.00%) high severe ``` </details> * Add s390x support for frame pointer-based stack walking * wasmtime: Allow `Caller::get_export` to get all exports * fuzzing: Add a fuzz target to check that our stack traces are correct We generate Wasm modules that keep track of their own stack as they call and return between functions, and then we periodically check that if the host captures a backtrace, it matches what the Wasm module has recorded. * Remove VM offsets for `VMHostFuncContext` since it isn't used by JIT code * Add doc comment with stack walking implementation notes * Document the extra state that can be passed to `wasmtime_runtime::Backtrace` methods * Add extensive comments for stack walking function * Factor architecture-specific bits of stack walking out into modules * Initialize store-related fields in a vmctx to null when there is no store yet Rather than leaving them as uninitialized data. * Use `set_callee` instead of manually setting the vmctx field * Use a more informative compile error message for unsupported architectures * Document unsafety of `prepare_host_to_wasm_trampoline` * Use `bti c` instead of `hint #34` in inline aarch64 assembly * Remove outdated TODO comment * Remove setting of `last_wasm_exit_fp` in `set_jit_trap` This is no longer needed as the value is plumbed through to the backtrace code directly now. * Only set the stack limit once, in the face of re-entrancy into Wasm * Add comments for s390x-specific stack walking bits * Use the helper macro for all libcalls If we forget to use it, and then trigger a GC from the libcall, that means we could miss stack frames when walking the stack, fail to find live GC refs, and then get use after free bugs. Much less risky to always use the helper macro that takes care of all of that for us. * Use the `asm_sym!` macro in Wasm-to-libcall trampolines This macro handles the macOS-specific underscore prefix stuff for us. * wasmtime: add size and align to `externref` assertion error message * Extend the `stacks` fuzzer to have host frames in between Wasm frames This way we get one or more contiguous sequences of Wasm frames on the stack, instead of exactly one. * Add documentation for aarch64-specific backtrace helpers * Clarify that we only support little-endian aarch64 in trampoline comment * Use `.machine z13` in s390x assembly file Since apparently our CI machines have pretty old assemblers that don't have `.machine z14`. This should be fine though since these trampolines don't make use of anything that is introduced in z14. * Fix aarch64 build * Fix macOS build * Document the `asm_sym!` macro * Add windows support to the `wasmtime-asm-macros` crate * Add windows support to host<--->Wasm trampolines * Fix trap handler build on windows * Run `rustfmt` on s390x trampoline source file * Temporarily disable some assertions about a trap's backtrace in the component model tests Follow up to re-enable this and fix the associated issue: https://github.com/bytecodealliance/wasmtime/issues/4535 * Refactor libcall definitions with less macros This refactors the `libcall!` macro to use the `foreach_builtin_function!` macro to define all of the trampolines. Additionally the macro surrounding each libcall itself is no longer necessary and helps avoid too many macros. * Use `VMOpaqueContext::from_vm_host_func_context` in `VMHostFuncContext::new` * Move `backtrace` module to be submodule of `traphandlers` This avoids making some things `pub(crate)` in `traphandlers` that really shouldn't be. * Fix macOS aarch64 build * Use "i64" instead of "word" in aarch64-specific file * Save/restore entry SP and exit FP/return pointer in the face of panicking imported host functions Also clean up assertions surrounding our saved entry/exit registers. * Put "typed" vs "untyped" in the same position of call benchmark names Regardless if we are doing wasm-to-host or host-to-wasm * Fix stacks test case generator build for new `wasm-encoder` * Fix build for s390x * Expand libcalls in s390x asm * Disable more parts of component tests now that backtrace assertions are a bit tighter * Remove assertion that can maybe fail on s390x Co-authored-by: Ulrich Weigand <ulrich.weigand@de.ibm.com> Co-authored-by: Alex Crichton <alex@alexcrichton.com>	2 years ago
Jamey Sharp	f242975c49	cranelift-frontend: Allow jump table reuse (#4429 ) * Allow using jump-tables multiple times (fixes #3347) If there are multiple `br_table` instructions using the same jump table, then `append_jump_argument` must not modify the jump table in-place. When this function is called, we don't know if more `br_table` instructions might be added later. So this patch conservatively assumes that all jump tables might be reused. If Cranelift needs to add a block argument to a block that's the target of some jump table, then the jump table will be unconditionally cloned. I'm not sure if having duplicated and unused jump tables will turn out to be a compile-time performance issue. If it is, there's discussion in issue #3347 about ways to determine that there can't be any more uses of a jump table, so that it's safe to modify in-place. * Re-enable cranelift-fuzzgen fuzz target I've been running this fuzz target for an hour without finding new bugs. Let's see if oss-fuzz finds anything now.	2 years ago
Jamey Sharp	ff8c26b7ef	cranelift-fuzzgen-verify is redundant with cranelift-fuzzgen (#4411 ) The compile step that cranelift-fuzzgen does also triggers IR verification. So all bugs that cranelift-fuzzgen-verify could catch are also caught by cranelift-fuzzgen. Removing redundant fuzzers lets us spend limited fuzz-testing CPU time budgets better.	2 years ago
Alex Crichton	b522171a68	Disable the cranelift-fuzzgen fuzz targets (#4391 ) Consulting oss-fuzz it looks like these fuzz targets are crashing 100% of the time partly due to #3347 I believe. Until that's fixed this hopes to reclaim the time used on oss-fuzz for other fuzzers to make progress.	2 years ago
Nick Fitzgerald	988d6ef9ac	fuzzing: Combine the `compile` and `compile-mutate` fuzz targets (#4048 ) We should still get the same amount of fuzzing using libfuzzer's mutators and using `wasm-mutate` as a mutator now, but they can share the same corpus, allowing mutations that one performed but the other didn't to reach new areas.	3 years ago
Alex Crichton	7b5176baea	Upgrade all crates to the Rust 2021 edition (#3991 ) * Upgrade all crates to the Rust 2021 edition I've personally started using the new format strings for things like `panic!("some message {foo}")` or similar and have been upgrading crates on a case-by-case basis, but I think it probably makes more sense to go ahead and blanket upgrade everything so 2021 features are always available. * Fix compile of the C API * Fix a warning * Fix another warning	3 years ago
Alex Crichton	7d1bc7d808	Move spec interpreter fuzzing behind a Cargo feature (#3871 ) * Move spec interpreter fuzzing behind a Cargo feature Building the spec interpreter requires a local installation of Ocaml and now libgmp which isn't always available, so this enables the ability to disable building the spec interpreter by using `cargo +nightly fuzz build --no-default-features`. The spec interpreter is still built by default but if fuzzers are being built locally and the spec interpreter isn't needed then this should enable it to be relatively easily opted-out of. * Tweak manifest directives	3 years ago
Conrad Watt	98ef18a22a	Fuzzing against verified fork of spec interpreter (#3843 ) * Revert "Remove spec interpreter fuzz target temporarily (#3399)" This reverts commit `25d3fa4d7b`. * add support for differential fuzzing against verified OCaml interpreter * formatting * comments * fix missing dep case * fix build error * fix unit tests? * restore previous differential_v8 max_table config * attempt: add OCaml deps * fix interpeter github repo * fix spec repo url * fix zarith package * fix unit test	3 years ago
Nick Fitzgerald	bad9a35418	`wasm-mutate` fuzz targets (#3836 ) * fuzzing: Add a custom mutator based on `wasm-mutate` * fuzz: Add a version of the `compile` fuzz target that uses `wasm-mutate` * Update `wasmparser` dependencies	3 years ago
Peter Huene	41eb225765	Add the instance allocation strategy to generated fuzzing configs. (#3780 ) * Add the instance allocation strategy to generated fuzzing configs. This commit adds support for generating configs with arbitrary instance allocation strategies. With this, the pooling allocator will be fuzzed as part of the existing fuzz targets. * Refine maximum constants for arbitrary module limits. * Add an `instantiate-many` fuzz target. This commit adds a new `instantiate-many` fuzz target that will attempt to instantiate and terminate modules in an arbitrary order. It generates up to 5 modules, from which a random sequence of instances will be created. The primary benefactor of this fuzz target is the pooling instance allocator. * Allow no aliasing in generated modules when using the pooling allocator. This commit prevents aliases in the generated modules as they might count against the configured import limits of the pooling allocator. As the existing module linking proposal implementation will eventually be deprecated in favor of the component model proposal, it isn't very important that we test aliases in generated modules with the pooling allocator. * Improve distribution of memory config in fuzzing. The previous commit attempted to provide a 32-bit upper bound to 64-bit arbitrary values, which skewed the distribution heavily in favor of the upper bound. This commit removes the constraint and instead uses arbitrary 32-bit values that are converted to 64-bit values in the `Arbitrary` implementation.	3 years ago
Alex Crichton	ab1d845ac1	Refactor fuzzing configuration and sometimes disable debug verifier. (#3664 ) * fuzz: Refactor Wasmtime's fuzz targets A recent fuzz bug found is related to timing out when compiling a module. This timeout, however, is predominately because Cranelift's debug verifier is enabled and taking up over half the compilation time. I wanted to fix this by disabling the verifier when input modules might have a lot of functions, but this was pretty difficult to implement. Over time we've grown a number of various fuzzers. Most are `wasm-smith`-based at this point but there's various entry points for configuring the wasm-smith module, the wasmtime configuration, etc. I've historically gotten quite lost in trying to change defaults and feeling like I have to touch a lot of different places. This is the motivation for this commit, simplifying fuzzer default configuration. This commit removes the ability to create a default `Config` for fuzzing, instead only supporting generating a configuration via `Arbitrary`. This then involved refactoring all targets and fuzzers to ensure that configuration is generated through `Arbitrary`. This should actually expand the coverage of some existing fuzz targets since `Arbitrary for Config` will tweak options that don't affect runtime, such as memory configuration or jump veneers. All existing fuzz targets are refactored to use this new method of configuration. Some fuzz targets were also shuffled around or reimplemented: * `compile` - this now directly calls `Module::new` to skip all the fuzzing infrastructure. This is mostly done because this fuzz target isn't too interesting and is largely just seeing what happens when things are thrown at the wall for Wasmtime. * `instantiate-maybe-invalid` - this fuzz target now skips instantiation and instead simply goes into `Module::new` like the `compile` target. The rationale behind this is that most modules won't instantiate anyway and this fuzz target is primarily fuzzing the compiler. This skips having to generate arbitrary configuration since wasm-smith-generated-modules (or valid ones at least) aren't used here. * `instantiate` - this fuzz target was removed. In general this fuzz target isn't too interesting in isolation. Almost everything it deals with likely won't pass compilation and is covered by the `compile` fuzz target, and otherwise interesting modules being instantiated can all theoretically be created by `wasm-smith` anyway. * `instantiate-wasm-smith` and `instantiate-swarm` - these were both merged into a new `instantiate` target (replacing the old one from above). There wasn't really much need to keep these separate since they really only differed at this point in methods of timeout. Otherwise we much more heavily use `SwarmConfig` than wasm-smith's built-in options. The intention is that we should still have basically the same coverage of fuzzing as before, if not better because configuration is now possible on some targets. Additionally there is one centralized point of configuration for fuzzing for wasmtime, `Arbitrary for ModuleConfig`. This internally creates an arbitrary `SwarmConfig` from `wasm-smith` and then further tweaks it for Wasmtime's needs, such as enabling various wasm proposals by default. In the future enabling a wasm proposal on fuzzing should largely just be modifying this one trait implementation. * fuzz: Sometimes disable the cranelift debug verifier This commit disables the cranelift debug verifier if the input wasm module might be "large" for the definition of "more than 10 functions". While fuzzing we disable threads (set them to 1) and enable the cranelift debug verifier. Coupled with a 20-30x slowdown this means that a module with the maximum number of functions, 100, gives: 60x / 100 functions / 30x slowdown = 20ms With only 20 milliseconds per function this is even further halved by the `differential` fuzz target compiling a module twice, which means that, when compiling with a normal release mode Wasmtime, if any function takes more than 10ms to compile then it's a candidate for timing out while fuzzing. Given that the cranelift debug verifier can more than double compilation time in fuzzing mode this actually means that the real time budget for function compilation is more like 4ms. The `wasm-smith` crate can pretty easily generate a large function that takes 4ms to compile, and then when that function is multiplied 100x in the `differential` fuzz target we trivially time out the fuzz target. The hope of this commit is to buy back half our budget by disabling the debug verifier for modules that may have many functions. Further refinements can be implemented in the future such as limiting functions for just the differential target as well. * Fix the single-function-module fuzz configuration * Tweak how features work in differential fuzzing * Disable everything for baseline differential fuzzing * Enable selectively for each engine afterwards * Also forcibly enable reference types and bulk memory for spec tests * Log wasms when compiling * Add reference types support to v8 fuzzer * Fix timeouts via fuel The default store has "infinite" fuel so that needs to be consumed before fuel is added back in. * Remove fuzzing-specific tests These no longer compile and also haven't been added to in a long time. Most of the time a reduced form of original the fuzz test case is added when a fuzz bug is fixed.	3 years ago
Nick Fitzgerald	d2d0a0f36b	Remove Peepmatic!!! Peepmatic was an early attempt at a DSL for peephole optimizations, with the idea that maybe sometime in the future we could user it for instruction selection as well. It didn't really pan out, however: * Peepmatic wasn't quite flexible enough, and adding new operators or snippets of code implemented externally in Rust was a bit of a pain. * The performance was never competitive with the hand-written peephole optimizers. It was very size efficient, but that came at the cost of run-time efficiency. Everything was table-based and interpreted, rather than generating any Rust code. Ultimately, because of these reasons, we never turned Peepmatic on by default. These days, we just landed the ISLE domain-specific language, and it is better suited than Peepmatic for all the things that Peepmatic was originally designed to do. It is more flexible and easy to integrate with external Rust code. It is has better time efficiency, meeting or even beating hand-written code. I think a small part of the reason why ISLE excels in these things is because its design was informed by Peepmatic's failures. I still plan on continuing Peepmatic's mission to make Cranelift's peephole optimizer passes generated from DSL rewrite rules, but using ISLE instead of Peepmatic. Thank you Peepmatic, rest in peace!	3 years ago
Alex Crichton	25d3fa4d7b	Remove spec interpreter fuzz target temporarily (#3399 ) This commit removes the `differential_spec` fuzz target for now, although this removal is intended to be temporary. We have #3251 to track re-enabling the spec interpreter in a way that it won't time out, and additionally the spec interpreter is also failing to build with ocaml on oss-fuzz so that will also need to be investigated when re-enabling.	3 years ago
Alex Crichton	4376cf2609	Add differential fuzzing against V8 (#3264 ) * Add differential fuzzing against V8 This commit adds a differential fuzzing target to Wasmtime along the lines of the wasmi and spec interpreters we already have, but with V8 instead. The intention here is that wasmi is unlikely to receive updates over time (e.g. for SIMD), and the spec interpreter is not suitable for fuzzing against in general due to its performance characteristics. The hope is that V8 is indeed appropriate to fuzz against because it's naturally receiving updates and it also is expected to have good performance. Here the `rusty_v8` crate is used which provides bindings to V8 as well as precompiled binaries by default. This matches exactly the use case we need and at least for now I think the `rusty_v8` crate will be maintained by the Deno folks as they continue to develop it. If it becomes an issue though maintaining we can evaluate other options to have differential fuzzing against. For now this commit enables the SIMD and bulk-memory feature of fuzz-target-generation which should enable them to get differentially-fuzzed with V8 in addition to the compilation fuzzing we're already getting. * Use weak linkage for GDB jit helpers This should help us deduplicate our symbol with other JIT runtimes, if any. For now this leans on some C helpers to define the weak linkage since Rust doesn't support that on stable yet. * Don't use rusty_v8 on MinGW They don't have precompiled libraries there. * Fix msvc build * Comment about execution	3 years ago
Alex Crichton	0642e62f16	Use wasm-smith to canonicalize NaN in differential fuzzing (#3195 ) * Update wasm-smith to 0.7.0 * Canonicalize NaN with wasm-smith for differential fuzzing This then also enables floating point executing in wasmi in addition to the spec interpreter. With NaN canonicalization at the wasm level this means that we should be producing deterministic results between Wasmtime and these alternative implementations.	3 years ago
Andrew Brown	651a321f1a	fuzz: add differential_spec fuzzing target This new target compares the outputs of executing the first exported function of a Wasm module in Wasmtime and in the official Wasm spec interpreter (using the `wasm-spec-interpreter` crate). This is an initial step towards more fully-featured fuzzing (e.g. compare memories, add `v128`, add references, add other proposals, etc.)	3 years ago
Alex Crichton	bb85366a3b	Enable simd fuzzing on oss-fuzz (#3152 ) * Enable simd fuzzing on oss-fuzz This commit generally enables the simd feature while fuzzing, which should affect almost all fuzzers. For fuzzers that just throw random data at the wall and see what sticks, this means that they'll now be able to throw simd-shaped data at the wall and have it stick. For wasm-smith-based fuzzers this commit also updates wasm-smith to 0.6.0 which allows further configuring the `SwarmConfig` after generation, notably allowing `instantiate-swarm` to generate modules using simd using `wasm-smith`. This should much more reliably feed simd-related things into the fuzzers. Finally, this commit updates wasmtime to avoid usage of the general `wasm_smith::Module` generator to instead use a Wasmtime-specific custom default configuration which enables various features we have implemented. * Allow dummy table creation to fail Tables might creation for imports may exceed the memory limit on the store, which we'll want to gracefully recover from and not fail the fuzzers.	3 years ago
Afonso Bordado	7453bd5f0d	Cranelift CLIF-level differential fuzzer (#3038 ) * cranelift: Initial fuzzer implementation * cranelift: Generate multiple test cases in fuzzer * cranelift: Separate function generator in fuzzer * cranelift: Insert random instructions in fuzzer * cranelift: Rename gen_testcase * cranelift: Implement div for unsigned values in interpreter * cranelift: Run all test cases in fuzzer * cranelift: Comment options in function_runner * cranelift: Improve fuzzgen README.md * cranelift: Fuzzgen remove unused variable * cranelift: Fuzzer code style fixes Thanks! @bjorn3 * cranelift: Fix nits in CLIF fuzzer Thanks @cfallin! * cranelift: Implement Arbitrary for TestCase * cranelift: Remove gen_testcase * cranelift: Move fuzzers to wasmtime fuzz directory * cranelift: CLIF-Fuzzer ignore tests that produce traps * cranelift: CLIF-Fuzzer create new fuzz target to validate generated testcases * cranelift: Store clif-fuzzer config in a separate struct * cranelift: Generate variables upfront per function * cranelift: Prevent publishing of fuzzgen crate	3 years ago
Alex Crichton	5140fd251a	Update wasm-tools crates (#2989 ) * Update wasm-tools crates This brings in recent updates, notably including more improvements to wasm-smith which will hopefully help exercise non-trapping wasm more. * Fix some wat	3 years ago
Chris Fallin	cb48ea406e	Switch default to new x86_64 backend. This PR switches the default backend on x86, for both the `cranelift-codegen` crate and for Wasmtime, to the new (`MachInst`-style, `VCode`-based) backend that has been under development and testing for some time now. The old backend is still available by default in builds with the `old-x86-backend` feature, or by requesting `BackendVariant::Legacy` from the appropriate APIs. As part of that switch, it adds some more runtime-configurable plumbing to the testing infrastructure so that tests can be run using the appropriate backend. `clif-util test` is now capable of parsing a backend selector option from filetests and instantiating the correct backend. CI has been updated so that the old x86 backend continues to run its tests, just as we used to run the new x64 backend separately. At some point, we will remove the old x86 backend entirely, once we are satisfied that the new backend has not caused any unforeseen issues and we do not need to revert.	4 years ago
Benjamin Bouvier	6e6713ae0b	cranelift: add support for the Mac aarch64 calling convention This bumps target-lexicon and adds support for the AppleAarch64 calling convention. Specifically for WebAssembly support, we only have to worry about the new stack slots convention. Stack slots don't need to be at least 8-bytes, they can be as small as the data type's size. For instance, if we need stack slots for (i32, i32), they can be located at offsets (+0, +4). Note that they still need to be properly aligned on the data type they're containing, though, so if we need stack slots for (i32, i64), we can't start the i64 slot at the +4 offset (it must start at the +8 offset). Added one test that was failing on the Mac M1, as well as other tests stressing different yet similar situations.	4 years ago
Nick Fitzgerald	c361485c9e	fuzz: Remove peepmatic fuzz targets There are occasional timeouts in type checking where Z3 hangs. This is a known issue[0] with the implementation of type checking in Peepmatic, and getting these timeouts in the fuzzers is just annoying and adds noise to our fuzzing results. When we fix [0] we can reintroduce these fuzz targets. [0]: https://github.com/bytecodealliance/wasmtime/issues/2695	4 years ago
Nick Fitzgerald	824ce7bf89	deps: Update `Arbitrary` to 1.0; `libfuzzer-sys` to 0.4.0; `wasm-smith` to 0.4.0	4 years ago

1 2 3

130 Commits (69078680785df7330dd967d00f8d82cf590b5179)