* Update wasm proposal support docs
Rename `--enable` flags to simply names and additionally replace module
linking with the component model.
* Fix a typo
* Improve the `wasmtime` crate's README
This commit is me finally getting back to #2688 and improving the README
of the `wasmtime` crate. Currently we have a [pretty drab README][drab]
that doesn't really convey what we want about Wasmtime.
While I was doing this I opted to update the feature list of Wasmtime as
well in the main README (which is mirrored into the crate readme),
namely adding a bullet point for "secure" which I felt was missing
relative to how we think about Wasmtime.
Naturally there's a lot of ways to paint this shed, so feedback is of
course welcome on this! (I'm not the best writer myself)
[drab]: https://crates.io/crates/wasmtime/0.37.0
* Expand the "Fast" bullet a bit more
* Reference the book from the wasmtime crate
* Update more security docs
Also merge the sandboxing security page with the main security page to
avoid the empty security page.
* Upgrade all crates to the Rust 2021 edition
I've personally started using the new format strings for things like
`panic!("some message {foo}")` or similar and have been upgrading crates
on a case-by-case basis, but I think it probably makes more sense to go
ahead and blanket upgrade everything so 2021 features are always
available.
* Fix compile of the C API
* Fix a warning
* Fix another warning
* Bump to 0.36.0
* Add a two-week delay to Wasmtime's release process
This commit is a proposal to update Wasmtime's release process with a
two-week delay from branching a release until it's actually officially
released. We've had two issues lately that came up which led to this proposal:
* In #3915 it was realized that changes just before the 0.35.0 release
weren't enough for an embedding use case, but the PR didn't meet the
expectations for a full patch release.
* At Fastly we were about to start rolling out a new version of Wasmtime
when over the weekend the fuzz bug #3951 was found. This led to the
desire internally to have a "must have been fuzzed for this long"
period of time for Wasmtime changes which we felt were better
reflected in the release process itself rather than something about
Fastly's own integration with Wasmtime.
This commit updates the automation for releases to unconditionally
create a `release-X.Y.Z` branch on the 5th of every month. The actual
release from this branch is then performed on the 20th of every month,
roughly two weeks later. This should provide a period of time to ensure
that all changes in a release are fuzzed for at least two weeks and
avoid any further surprises. This should also help with any last-minute
changes made just before a release if they need tweaking since
backporting to a not-yet-released branch is much easier.
Overall there are some new properties about Wasmtime with this proposal
as well:
* The `main` branch will always have a section in `RELEASES.md` which is
listed as "Unreleased" for us to fill out.
* The `main` branch will always be a version ahead of the latest
release. For example it will be bump pre-emptively as part of the
release process on the 5th where if `release-2.0.0` was created then
the `main` branch will have 3.0.0 Wasmtime.
* Dates for major versions are automatically updated in the
`RELEASES.md` notes.
The associated documentation for our release process is updated and the
various scripts should all be updated now as well with this commit.
* Add notes on a security patch
* Clarify security fixes shouldn't be previewed early on CI
In today's Wasmtime meeting we discussed the acceptance criteria for
patch releases for Wasmtime and Cranelift. The criteria we came up with
were:
* Cranelift will get a patch release for any miscompilation, whether or
not it affects Wasmtime.
* Wasmtime will get a patch release for security issues and bugs which
seriously hinder usability.
The consensus at the time was that due to Wasmtime's monthly release
schedule we want to be pretty strict about what generates a patch
release, hence the threshold being at serious bugs as opposed to any
bugs found.
This commit attempts to update the `stability-release.md` document with
our documented release process. The release cadence section is brought
up to date, the Wasmtime section was edited slightly (it largely already
said this which I only just realized), and a Cranelift section was
added.
* x64: enable VTune support by default
After significant work in the `ittapi-rs` crate, this dependency should
build without issue on Wasmtime's supported operating systems: Windows,
Linux, and macOS. The difference in the release binary is <20KB, so this
change makes `vtune` a default build feature. This change upgrades
`ittapi-rs` to v0.2.0 and updates the documentation.
* review: add configuration for defaults in more places
* review: remove OS conditional compilation, add architecture
* review: do not default vtune feature in wasmtime-jit
* Document the invoke argument of the run command.
* Update docs/cli-options.md
Co-authored-by: Kyle Brown <kyleb@liquidrocketry.com>
Co-authored-by: Nick Fitzgerald <fitzgen@gmail.com>
While using VTune, it seemed a good idea to check that the VTune
documentation for Wasmtime was still correct. It is and VTune support
still works (improvements: click-through to x86 assembly is not
available). These changes simply re-organize the documentation and add a
section for running VTune from a GUI.
* Change the bump-version workflow's schedule
Either I don't understand cron or GitHub doesn't understand cron. It's
not clear which. I think that
https://github.com/bytecodealliance/wasmtime/pull/3511 may have fallen
within our schedule but it was supposed to be on a weekday. Otherwise
https://github.com/bytecodealliance/wasmtime/pull/3499 was certainly
spurious. This commit moves to a simpler "just do it on the same day
each month" and we can manually figure out weekdays and such. Hopefully
this should reduce the number of spurious PRs we're getting to bump
versions.
This also removes the script to force a version bump since I found a
button on the GitHub UI to do the same thing. Additionally I've updated
the patch-release documentation to use this button. Note that this
button takes inputs as well which means we can further automate patch
releases to look even more like normal release process, differing only
in one part of the argument used to trigger the workflow.
* Fix a typo
* Automate more of Wasmtime's release process
This change revamps the release process for Wasmtime and intends to make
it nearly 100% automated for major release and hopefully still pretty
simple for patch releases. New workflows are introduced as part of
this commit:
* Once a month a PR is created with major version bumps
* Specifically hinted commit messages to the `main` branch will get
tagged and pushed to the main repository.
* On tags we'll now not only build releases after running CI but
additionally crates will be published to crates.io.
In conjunction with other changes this means that the release process
for a new major version of Wasmtime is simply merging a PR. Patch
releases will involve running some steps locally but most of the
nitty-gritty should be simply merging the PR that's generated.
* Use an anchor in a regex
Before this commit we actually have two builders checking for security
advisories on CI, one is `cargo audit` and one is `cargo deny`. The
`cargo deny` builder is slightly different in that it checks a few other
things about our dependency tree such as licenses, duplicates, etc. This
commit removes the advisory check from `cargo deny` on CI and then moves
the `cargo audit` check to a separate workflow.
The `cargo audit` check will now run nightly and will open an issue on
the Wasmtime repository when an advisory is found. This should help make
it such that our CI is never broken by the publication of an advisory
but we're still promptly notified whenever an advisory is made. I've
updated the release process notes to indicate that the open issues
should be double-checked to ensure that there are no open advisories
that we need to take care of.
This commit removes the Lightbeam backend from Wasmtime as per [RFC 14].
This backend hasn't received maintenance in quite some time, and as [RFC
14] indicates this doesn't meet the threshold for keeping the code
in-tree, so this commit removes it.
A fast "baseline" compiler may still be added in the future. The
addition of such a backend should be in line with [RFC 14], though, with
the principles we now have for stable releases of Wasmtime. I'll close
out Lightbeam-related issues once this is merged.
[RFC 14]: https://github.com/bytecodealliance/rfcs/pull/14
* Remove the `wasmtime wasm2obj` command
This commit removes the `wasm2obj` subcommand of the `wasmtime` CLI.
This subcommand has a very long history and dates back quite far. While
it's existed, however, it's never been documented in terms of the output
it's produced. AFAIK it's only ever been used for debugging to see the
machine code output of Wasmtime on some modules. With recent changes to
the module serialization output the output of `wasmtime compile`, the
`*.cwasm` file, is now a native ELF file which can be fed to standard
tools like `objdump`. Consequently I dont think there's any remaining
need to keep `wasm2obj` around itself, so this commit removes the
subcommand.
* More code to delete
* Try to fix debuginfo tests
* Finished the Markdown Parser Example for Wasmtime
* Made requested changes
* Tiny change to explanation of `--dir` CLI arg
* Add `bash` annotations to shell script code blocks
* Trying to fix the markdown example bug
* Figured out rustdoc, and what needed to be done
* Made requested changes
Co-authored-by: Till Schneidereit <till@tillschneidereit.net>
The previous documentation only covers how to enable debug info when
embedding Wasmtime. This change should cover the commonly-asked
question: how do I debug in Wasmtime?
* Start a high-level architecture document for Wasmtime
This commit cleands up some existing documentation by removing a number
of "noop README files" and starting a high-level overview of the
architecture of Wasmtime. I've placed this documentation under the
contributing section of the book since it seems most useful for possible
contributors.
I've surely left some things out in this pass, and am happy to add more!
* Review comments
* More rewording
* typos
Implement Wasmtime's new API as designed by RFC 11. This is quite a large commit which has had lots of discussion externally, so for more information it's best to read the RFC thread and the PR thread.
* Bring back per-thread lazy initialization
Platforms Wasmtime supports may have per-thread initialization that
needs to run before WebAssembly. For example Unix needs to setup a
sigaltstack and macOS needs to set up mach ports. In #2757 this
per-thread setup was moved out of the invocation of a wasm function,
relying on the lack of Send for Store to initialize the thread at Store
creation time and never worry about it later.
This conflicted with [wasmtime's desired multithreading
story](https://github.com/bytecodealliance/wasmtime/pull/2812) so a new
[`Store::notify_switched_thread` was
added](https://github.com/bytecodealliance/wasmtime/pull/2822) to
explicitly indicate a Store has moved to another thread (if it unsafely
did so).
It turns out though that it's not always easy to determine when a
`Store` moves to a new thread. For example the Go bindings for Wasmtime
are generally unaware when a goroutine switches OS threads. This led to
https://github.com/bytecodealliance/wasmtime-go/issues/74 where a SIGILL
was left uncaught, making it appear that traps aren't working properly.
This commit revisits the decision in #2757 and moves per-thread
initialization back into the path of calling into WebAssembly. This is
differently from before, though, where there's still only one TLS access
on the path of calling into WebAssembly, unlike before where it was a
separate access. This allows us to get the speed benefits of #2757 as
well as the flexibility benefits of not having to explicitly move a
store between threads.
With this new ability this commit deletes the recently added
`Store::notify_switched_thread` method since it's no longer necessary.
* Fix a test compiling
* Introduce a new API that allows notifying that a Store has moved to a new thread
* Add backlink to documentation, and mention the new API in the multithreading doc;
* Document guidance around multithreading and Wasmtime
This commit writes a page of documentation for the Wasmtime book to
serve as guidance for embedders looking to add multithreading with
Wasmtime support. As always with any safe Rust API this reading is
optional because you can't mis-use Wasmtime without `unsafe`, but I'm
hoping that this documentation can serve as a point of reference for
folks who want to add multithreading but are confused/annoyed that
Wasmtime's types do not implement the `Send` and `Sync` traits.
Closes#793
* I can type
This commit adds a `compile` command to the Wasmtime CLI.
The command can be used to Ahead-Of-Time (AOT) compile WebAssembly modules.
With the `all-arch` feature enabled, AOT compilation can be performed for
non-native architectures (i.e. cross-compilation).
The `Module::compile` method has been added to perform AOT compilation.
A few of the CLI flags relating to "on by default" Wasm features have been
changed to be "--disable-XYZ" flags.
A simple example of using the `wasmtime compile` command:
```text
$ wasmtime compile input.wasm
$ wasmtime input.cwasm
```
* Redo the statically typed `Func` API
This commit reimplements the `Func` API with respect to statically typed
dispatch. Previously `Func` had a `getN` and `getN_async` family of
methods which were implemented for 0 to 16 parameters. The return value
of these functions was an `impl Fn(..)` closure with the appropriate
parameters and return values.
There are a number of downsides with this approach that have become
apparent over time:
* The addition of `*_async` doubled the API surface area (which is quite
large here due to one-method-per-number-of-parameters).
* The [documentation of `Func`][old-docs] are quite verbose and feel
"polluted" with all these getters, making it harder to understand the
other methods that can be used to interact with a `Func`.
* These methods unconditionally pay the cost of returning an owned `impl
Fn` with a `'static` lifetime. While cheap, this is still paying the
cost for cloning the `Store` effectively and moving data into the
closed-over environment.
* Storage of the return value into a struct, for example, always
requires `Box`-ing the returned closure since it otherwise cannot be
named.
* Recently I had the desire to implement an "unchecked" path for
invoking wasm where you unsafely assert the type signature of a wasm
function. Doing this with today's scheme would require doubling
(again) the API surface area for both async and synchronous calls,
further polluting the documentation.
The main benefit of the previous scheme is that by returning a `impl Fn`
it was quite easy and ergonomic to actually invoke the function. In
practice, though, examples would often have something akin to
`.get0::<()>()?()?` which is a lot of things to interpret all at once.
Note that `get0` means "0 parameters" yet a type parameter is passed.
There's also a double function invocation which looks like a lot of
characters all lined up in a row.
Overall, I think that the previous design is starting to show too many
cracks and deserves a rewrite. This commit is that rewrite.
The new design in this commit is to delete the `getN{,_async}` family of
functions and instead have a new API:
impl Func {
fn typed<P, R>(&self) -> Result<&Typed<P, R>>;
}
impl Typed<P, R> {
fn call(&self, params: P) -> Result<R, Trap>;
async fn call_async(&self, params: P) -> Result<R, Trap>;
}
This should entirely replace the current scheme, albeit by slightly
losing ergonomics use cases. The idea behind the API is that the
existence of `Typed<P, R>` is a "proof" that the underlying function
takes `P` and returns `R`. The `Func::typed` method peforms a runtime
type-check to ensure that types all match up, and if successful you get
a `Typed` value. Otherwise an error is returned.
Once you have a `Typed` then, like `Func`, you can either `call` or
`call_async`. The difference with a `Typed`, however, is that the
params/results are statically known and hence these calls can be much
more efficient.
This is a much smaller API surface area from before and should greatly
simplify the `Func` documentation. There's still a problem where
`Func::wrapN_async` produces a lot of functions to document, but that's
now the sole offender. It's a nice benefit that the
statically-typed-async verisons are now expressed with an `async`
function rather than a function-returning-a-future which makes it both
more efficient and easier to understand.
The type `P` and `R` are intended to either be bare types (e.g. `i32`)
or tuples of any length (including 0). At this time `R` is only allowed
to be `()` or a bare `i32`-style type because multi-value is not
supported with a native ABI (yet). The `P`, however, can be any size of
tuples of parameters. This is also where some ergonomics are lost
because instead of `f(1, 2)` you now have to write `f.call((1, 2))`
(note the double-parens). Similarly `f()` becomes `f.call(())`.
Overall I feel that this is a better tradeoff than before. While not
universally better due to the loss in ergonomics I feel that this design
is much more flexible in terms of what you can do with the return value
and also understanding the API surface area (just less to take in).
[old-docs]: https://docs.rs/wasmtime/0.24.0/wasmtime/struct.Func.html#method.get0
* Rename Typed to TypedFunc
* Implement multi-value returns through `Func::typed`
* Fix examples in docs
* Fix some more errors
* More test fixes
* Rebasing and adding `get_typed_func`
* Updating tests
* Fix typo
* More doc tweaks
* Tweak visibility on `Func::invoke`
* Fix tests again
Jairus Tanaka
Alex Crichton
Piotr Sikora
Kk Shinkai
Andrew Brown
Kyle Brown
Tim Park
Nathan Froyd
Shinobu Hayashi
Aaron Turner
Pat Hickey
Olivier Lemasle
Benjamin Bouvier
Tshepang Lekhonkhobe
Till Schneidereit
Peter Huene
Dan Gohman
Pat Hickey
Nick Fitzgerald
hayasshi
Ivan Zvonimir Horvat