* Enable jitdump profiling support by default
This the result of some of the investigation I was doing for #1017. I've
done a number of refactorings here which culminated in a number of
changes that all amount to what I think should result in jitdump support being
enabled by default:
* Pass in a list of finished functions instead of just a range to
ensure that we're emitting jit dump data for a specific module rather
than a whole `CodeMemory` which may have other modules.
* Define `ProfilingStrategy` in the `wasmtime` crate to have everything
locally-defined
* Add support to the C API to enable profiling
* Documentation added for profiling with jitdump to the book
* Split out supported/unsupported files in `jitdump.rs` to avoid having
lots of `#[cfg]`.
* Make dependencies optional that are only used for `jitdump`.
* Move initialization up-front to `JitDumpAgent::new()` instead of
deferring it to the first module.
* Pass around `Arc<dyn ProfilingAgent>` instead of
`Option<Arc<Mutex<Box<dyn ProfilingAgent>>>>`
The `jitdump` Cargo feature is now enabled by default which means that
our published binaries, C API artifacts, and crates will support
profiling at runtime by default. The support I don't think is fully
fleshed out and working but I think it's probably in a good enough spot
we can get users playing around with it!
Removes an edge towards the `synstructure` dependency in our dependency
graph which, if removed entirely, is hoped to reduce build times locally
and on CI as you switch between `cargo build` and `cargo test` because
deep dependencies like `syn` won't have their features alternating back
and forth.
* Temporarily remove support for interface types
This commit temporarily removes support for interface types from the
`wasmtime` CLI and removes the `wasmtime-interface-types` crate. An
error is now printed for any input wasm modules that have wasm interface
types sections to indicate that support has been removed and references
to two issues are printed as well:
* #677 - tracking work for re-adding interface types support
* #1271 - rationale for removal and links to other discussions
Closes#1271
* Update the python extension
* Move all examples to a top-level directory
This commit moves all API examples (Rust and C) to a top-level
`examples` directory. This is intended to make it more discoverable and
conventional as to where examples are located. Additionally all examples
are now available in both Rust and C to see how to execute the example
in the language you're familiar with. The intention is that as more
languages are supported we'd add more languages as examples here too.
Each example is also accompanied by either a `*.wat` file which is
parsed as input, or a Rust project in a `wasm` folder which is compiled
as input.
A simple driver crate was also added to `crates/misc` which executes all
the examples on CI, ensuring the C and Rust examples all execute
successfully.
Update the documentation for the merger, and also for various changes in
Cranelift. Remove some old obsolete documentation, and convert the remaining
Sphinx files to Markdown. Some of the remaining content is still out of
date, but this is a step forward.
this allows us to reuse the code in wiggle-generate elsewhere, because
a proc-macro=true lib can only export a #[proc_macro] and not ordinary
functions.
In lucet, I will depend on wiggle-generate to define a proc macro that
glues wiggle to the specifics of the runtime.
* Cargo.lock: Update, to no longer use multiple versions of autocfg
* Update wasmtime-debug and wasmtime-profiling to current gimli 0.20.0
This also eliminates duplicate versions of gimli and arrayvec, and
eliminates the nodrop dependency entirely.
* Update wasmtime-profiling to goblin 0.1.3 and object 0.17.0
This also eliminates two duplicate versions of goblin, and duplicate
versions of proc-macro2, quote, syn, scroll_derive, and unicode-xid.
* Update wasmtime-profiling to current scroll 0.10.1
This eliminates duplicate versions of scroll.
* Update wasmtime-profiling to current target-lexicon 0.10.0
This eliminates duplicate versions of target-lexicon.
* Update wasmtime-interface-types to current walrus and wasm-webidl-bindings
This also eliminates the oldest of the three duplicate versions of
wasmparser.
* Update wasmtime-wast to current wast 8.0.0
This eliminates one of the duplicate versions of wast.
Patch adds support for the perf jitdump file specification.
With this patch it should be possible to see profile data for code
generated and maped at runtime. Specifically the patch adds support
for the JIT_CODE_LOAD and the JIT_DEBUG_INFO record as described in
the specification. Dumping jitfiles is enabled with the --jitdump
flag. When the -g flag is also used there is an attempt to dump file
and line number information where this option would be most useful
when the WASM file already includes DWARF debug information.
The generation of the jitdump files has been tested on only a few wasm
files. This patch is expected to be useful/serviceable where currently
there is no means for jit profiling, but future patches may benefit
line mapping and add support for additional jitdump record types.
Usage Example:
Record
sudo perf record -k 1 -e instructions:u target/debug/wasmtime -g
--jitdump test.wasm
Combine
sudo perf inject -v -j -i perf.data -o perf.jit.data
Report
sudo perf report -i perf.jit.data -F+period,srcline
* Refactor naming and crates info
This commit:
* changes workspace crates to have a `wiggle_` prefix in names
* rename `memory` module of `wiggle-memory` crate to `runtime`
* fixes authors of all crates
* Rename wiggle memory crate to runtime
* generator: take an &mut GuestMemory
rather than pass the owned GuestMemory in, just give exclusive access
to it. Makes testing easier.
* tests: start transforming tests to check abi-level generated code as well
* finish lowering of test funcs
* tests: rename variables to more sensible names
* proptesting: reliably finds that we dont allow stuff to be right against end of memory!
* memory: fix off-by-one calc in GuestMemory::contains(&self, Region)
ty proptest!
also, refactored the Region::overlaps to be the same code but easier to
read.
* generator: better location information in GuestError
* testing: proptest generates memory areas, tests everything
Crates used exclusively at build time, such as proc-macro crates and
their dependencies, don't benefit substantially from optimization; they
take far longer to optimize than time saved when running them during the
build. No machine code from these crates will appear in the final
compiled binary.
Use the new profile-overrides mechanism in Rust 1.41
(https://doc.rust-lang.org/cargo/reference/profiles.html#overrides) to
build such crates with opt-level 0.
On a 4-thread laptop, this brings build time from 6m27s to 4m21s, and
CPU time spent from 23m43s to 16m23s.
This commit deletes the old C implementation of the original
`wasi_unstable` module, instead only leaving around our single
`wasmtime-wasi` crate as the implementation for both
`wasi_snapshot_preview1` and `wasi_unstable`.
This hasn't been discussed (AFAIK) up until now, so this is also a
proposal! Some thoughts in favor of this deletion I would have are:
* This has been off-by-default for ages
* We don't build or test any of this on CI
* Published binaries with `wasmtime` do not have this possibility
enabled
* Future refactorings to the `wasmtime-wasi` crate will either need to
work around how the C implementation is different or bring it up to
speed.
This is motivated by the last bullet point where I was working on
getting `wasmtime-wasi` working purely as an implementation detail on
top of the `wasmtime` crate itself, but quickly ran into a case where
the CLI would need to multiplex all sorts of wasi implementations. In
any case I'm curious what others think, is this too soon? Is there
something remaining blocking this? (etc)
Currently the wasmtime binary (src/bin/wasmtime) and the wasmtime API
crate (crates/api) share the same output filename. This causes the
output files of the wasmtime binary to clobber the output files of the
wasmtime API crate. So running `cargo doc --all` will often not build
the wasmtime API docs, which is the more useful of the two.
This is a rustdoc bug, and can be worked around by disabling
documentation for the wasmtime binary.
* Move the C API to a separate crate
This commit moves the C API from `crates/api/src/wasm.rs` to
`crates/capi/src/lib.rs` to be located in a separate crate. There's a
number of reasons for this:
* When a Rust program depends on the `wasmtime` crate, there's no need
to compile in the C API.
* This should improve compile times of the `wasmtime` crate since it's
not producing artifacts which aren't always used.
* The development of the C API can be guaranteed to only use the public
API of the `wasmtime` crate itself.
Some CI pieces are tweaked and this overall shouldn't have much impact
on users, it's intended that it's a cleanup/speedup for developers!
* Disable rustdoc/tests for capi
* Review feedback
* Add back in accidentally deleted comment
* More renamings
* Try to fix dotnet build
* Update to the latest spec_testsuite and dependencies.
Update to target-lexicon 0.10, cranelift 0.54, wast 0.6, faerie 0.14,
and the latest spec_testsuite.
For wast and cranelift-wasm, update the code for API changes.
* Factor out the code for matching f32, f64, and v128.
This takes the idea from #802 to split out `f32_matches`, `f64_matches`,
and `v128_matches` functions, which better factor out the matching
functionality between scalar and vector.
This commit refactors the Wasmtime CLI tools to use `structopt` instead of
`docopt`.
The `wasmtime` tool now has the following subcommands:
* `config new` - creates a new Wasmtime configuration file.
* `run` - runs a WebAssembly module.
* `wasm2obj` - translates a Wasm module to native object file.
* `wast` - runs a test script file.
If no subcommand is specified, the `run` subcommand is used. Thus,
`wasmtime foo.wasm` should continue to function as expected.
The `wasm2obj` and `wast` tools still exist, but delegate to the same
implementation as the `wasmtime` subcommands. The standalone `wasm2obj` and
`wast` tools may be removed in the future in favor of simply using `wasmtime`.
Included in this commit is a breaking change to the default Wasmtime
configuration file: it has been renamed from `wasmtime-cache-config.toml` to
simply `config.toml`. The new name is less specific which will allow for
additional (non-cache-related) settings in the future.
There are some breaking changes to improve command line UX:
* The `--cache-config` option has been renamed to `--config`.
* The `--create-config-file` option has moved to the `config new` subcommand.
As a result, the `wasm2obj` and `wast` tools cannot be used to create a new
config file.
* The short form of the `--optimize` option has changed from
`-o` to `-O` for consistency.
* The `wasm2obj` command takes the output object file as a
required positional argument rather than the former required output *option*
(e.g. `wasmtime wasm2obj foo.wasm foo.obj`).
* Remove the `Flags` type from `Config` API
This commit removes the final foreign type from the `Config` API in the
`wasmtime` crate. The cranelift `Flags` type is now expanded into
various options on the `Config` structure itself, all prefixed with
`cranelift_` since they're only relevant to the Cranelift backend. The
various changes here were:
* The `avoid_div_traps` feature is enabled by default since it seemed
that was done anywhere anyway.
* Enabling the wasm SIMD feature enables the requisite features in
Cranelift as well.
* A method for enabling the debug verifier has been added.
* A method for configuring the Cranelift optimization level, as well as
a corresponding enumeration, has been added.
* Assert that `Config` is both `Send` and `Sync`
* Remove usage of `CompilationStrategy` from `Config`
This commit removes the public API usage of the internal
`CompilationStrategy` enumeration from the `Config` type in the
`wasmtime` crate. To do this the `enum` was copied locally into the
crate and renamed `Strategy`. The high-level description of this change
is:
* The `Config::strategy` method now takes a locally-defined `Strategy`
enumeration instead of an internal type.
* The contents of `Strategy` are always the same, not relying on Cargo
features to indicate which variants are present. This avoids
unnecessary downstream `#[cfg]`.
* A `lightbeam` feature was added to the `wasmtime` crate itself to
lightbeam compilation support.
* The `Config::strategy` method is now fallible. It returns a runtime
error if support for the selected strategy wasn't compiled in.
* The `Strategy` enum is listed as `#[non_exhaustive]` so we can safely
add variants over time to it.
This reduces the public crate dependencies of the `wasmtime` crate
itself, removing the need to reach into internal crates even more!
cc #708
* Fix fuzz targets
* Update nightly used to build releases
* Run rustfmt
This crate is intended to hold all of our various test case generators and
oracles. The fuzz targets we have at `wasmtime/fuzz/fuzz_targets/*` will
eventually be ~one-liner glue code calling into this crate.
Part of #611
* Fix feature-gating of test-programs
This commit fixes bugs in enabling feature-gating of `test-programs`
which was introduced in #600. It turns out, #600 accidentally
disabled `test-programs` from ever running, and this commit fixes
that.
* Fix the CI
This commit removes the usage of the `failure` crate and finishes up the
final pieces of the migration to `std::error::Error` and `anyhow`. The
`faerie` crate was updated to pull in its migration from `failure` to
`anyhow` as well.
* Fix fuzz target compilation.
* Bump version to 0.7.0
* Temporarily disable fuzz tests
Temporarily disable fuzz tests until https://github.com/bytecodealliance/cranelift/issues/1216 is resolved.
* Fix publish-all.sh to not modify the witx crate.
* Remove the "publish = false" attribute from Lightbeam.
* Add a README.md for wasmtime-interface-types.
* Remove the "rust" category.
This fixes the following warning:
warning: the following are not valid category slugs and were ignored: rust. Please see https://crates.io/category_slugs for the list of all category slugs.
* Mark wasmtime-cli as "publish = false".
* Sort the publishing rules in topological order.
Also, publish nightly-only crates with cargo +nightly.
* Reorganize wasi-misc-tests.
Move wasi-misc-tests out of wasi-common, to break a dependency cycle;
previously, wasmtime-* depended on wasi-common, but wasi-common
dev-dependended on wasmtime-*.
Now, wasi-common no longer dev-depends on wasmtime-*; instead, the
tests are in their own crate which depends on wasi-common and on
wasmtime-*.
Also, rename wasi-misc-tests to wasi-tests for simplicity.
This also removes the "wasm_tests" feature; it's replaced by the
"test-programs" feature.
* Update the CI script to use the new feature name.
* Update the CI script to use the new feature name in one more place.
* Change a `write!` to a `writeln!`.
* General Cargo.toml cleanup.
- Remove travis-ci attributes.
- Remove "experimental" badges from actively-developed crates.
- Reflow some long lines.
- Use dependency features consistently.
- Add readme attributes
* Update WASI to the latest trunk.
This notably adds a .gitignore file for the WASI directory.
Alex Crichton
Pat Hickey
Jakub Konka
Nick Fitzgerald
Josh Triplett
Johnnie Birch
Yury Delendik
Jakub Konka
Pat Hickey
Andrew Chin
Dan Gohman
Peter Huene