* Update to latest wasm-tools crates
This commit pushes through the full update of the wasm-tools crates
through Wasmtime. There are two major features which changed, both
related to components, which required updates in Wasmtime:
* Resource types are now implemented in wasm-tools and they're not yet
implemented in Wasmtime so I've stubbed out the integration point with
panics as reminders to come back and implement them.
* There are new validation rules about how aggregate types must be
named. This doesn't affect runtime internals at all but was done on
behalf of code generators. This did however affect a number of tests
which have to ensure that types are exported.
* Fix more tests
* Add vet entries
* Update coredump generation in the cli to use wasm_encoder
* Add deny.toml exception for wasm-encoder 0.25.0
* add missing newline
* update custom section in fuzzing crate
* wasmtime: In-process sampling profiler
Unlike the existing profiling options, this works on all platforms and
does not rely on any external profiling tools like perf or VTune. On the
other hand, it can only profile time spent in the WebAssembly guest, not
in Wasmtime itself or other host code. Also it can't measure time as
precisely as platform-native tools can.
The profile is saved in the Firefox processed format, which can be
viewed using https://profiler.firefox.com/.
* Ensure func_offset is populated
* Refactor
* Review comments
* Move GuestProfiler to the wasmtime crate
* Document the new GuestProfiler API
* Add TODO comments for future work
* Use module_offset, not func_offset, as fallback PC
* Minimize work done during `sample()`
Use fxprof_processed_profile's support for looking up symbols to avoid
looking up the same PC more than once per profile.
* Keep profiler state in the store
Also extend the documentation based on review comments.
* Import debugid audit from Mozilla again
This trims down the `[exemptions]` list ever-so-slightly by following
the suggestions of `cargo vet suggest` and updating a few crates across
some minor versions.
This commit splits `VMCallerCheckedFuncRef::func_ptr` into three new function
pointers: `VMCallerCheckedFuncRef::{wasm,array,native}_call`. Each one has a
dedicated calling convention, so callers just choose the version that works for
them. This is as opposed to the previous behavior where we would chain together
many trampolines that converted between calling conventions, sometimes up to
four on the way into Wasm and four more on the way back out. See [0] for
details.
[0] https://github.com/bytecodealliance/rfcs/blob/main/accepted/tail-calls.md#a-review-of-our-existing-trampolines-calling-conventions-and-call-paths
Thanks to @bjorn3 for the initial idea of having multiple function pointers for
different calling conventions.
This is generally a nice ~5-10% speed up to our call benchmarks across the
board: both Wasm-to-host and host-to-Wasm. The one exception is typed calls from
Wasm to the host, which have a minor regression. We hypothesize that this is
because the old hand-written assembly trampolines did not maintain a call frame
and do a tail call, but the new Cranelift-generated trampolines do maintain a
call frame and do a regular call. The regression is only a couple nanoseconds,
which seems well-explained by these differences explain, and ultimately is not a
big deal.
However, this does lead to a ~5% code size regression for compiled modules.
Before, we compiled a trampoline per escaping function's signature and we
deduplicated these trampolines by signature. Now we compile two trampolines per
escaping function: one for if the host calls via the array calling convention
and one for it the host calls via the native calling convention. Additionally,
we compile a trampoline for every type in the module, in case there is a native
calling convention function from the host that we `call_indirect` of that
type. Much of this is in the `.eh_frame` section in the compiled module, because
each of our trampolines needs an entry there. Note that the `.eh_frame` section
is not required for Wasmtime's correctness, and you can disable its generation
to shrink compiled module code size; we just emit it to play nice with external
unwinders and profilers. We believe there are code size gains available for
follow up work to offset this code size regression in the future.
Backing up a bit: the reason each Wasm module needs to provide these
Wasm-to-native trampolines is because `wasmtime::Func::wrap` and friends allow
embedders to create functions even when there is no compiler available, so they
cannot bring their own trampoline. Instead the Wasm module has to supply
it. This in turn means that we need to look up and patch in these Wasm-to-native
trampolines during roughly instantiation time. But instantiation is super hot,
and we don't want to add more passes over imports or any extra work on this
path. So we integrate with `wasmtime::InstancePre` to patch these trampolines in
ahead of time.
Co-Authored-By: Jamey Sharp <jsharp@fastly.com>
Co-Authored-By: Alex Crichton <alex@alexcrichton.com>
prtest:full
This updates to rustix 0.37.13, which contains some features we can use to
implement more features in wasi-common for the wasi-sockets API. This also
pulls in several other updates to avoid having multiple versions of rustix.
This does introduce multiple versions of windows-sys, as the errno and tokio
crates are currently using 0.45 while rustix and other dependencies have
updated to 0.48; PRs updating these are already in flight so this will
hopefully be resolved soon.
It also includes cap-std 1.0.14, which disables the use of `openat2` and
`statx` on Android, fixing a bug where some Android devices crash the
process when those syscalls are executed.
While bringing in no major updates for Wasmtime I've taken this
opportunity to list myself for `cargo vet` with wildcard audits of this
family of crates. That means I shouldn't need to further add any more
entries in the future for updating these crates and additionally any
other organizations using these audits will automatically be able to
have audits for version that I publish.
While here I also ran `cargo vet prune` which was able to remove a
number of our exemptions.
The `ring` crate needed to be exempted: it contains a large quantity of asm and native binary implementations of crypto primitives. It is a major undertaking to certify the safety of those implementations.
ring also pulled in the wasm-bindgen family of crates for its wasm32-unknown-unknown target, which this project will not be using. Because we don't care about that platform, I added exemptions for all of these crates, so we don't have to audit them.
The actual supply chain audits for rusttls, rustls-webpki, sct, and tokio-rustls were unremarkable. I also audited a small diff on wasm-bindgen-shared because it was trivial.
* add cargo-deny exception for duplicate versions of windows-sys
* cargo vetting for all new deps introduced by https://github.com/bytecodealliance/wasmtime/pull/5929
The audits are straightforward. The exemptions, as always, need to be justified:
* core-foundation, core-foundation-sys, security-framework, security-framework-sys: these are large crates which are FFI bindings to Mac OS frameworks. As such they contain tons of unsafe code to make these FFI calls and manage memory. These crates are too big to audit.
* schannel: same as the above, except this is a windows component, which I'm also unfamiliar with.
* openssl, openssl-sys: also large FFI bindings which are impractical to audit.
* futures-macro, futures-task: while not as complex as futures-util, these are beyond my personal understanding of futures to vet practically. I've asked Alex to look at auditing these, and he will after he returns from vacation next week.
* futures-util: 25kloc of code, over 149 instances of the substring "unsafe" (case insensitive), this is impractical to audit in the extreme.
* h2, http, httparse, hyper, mio, tokio: this so-called tokio/hyper family are very large and challenging to audit. Bobby Holley has indicated that he is working to get the AWS engineers who maintain these crates to publish their own audits, which we can then import. We expect to exempt these until those imports are available.
Aside from a few new features (notably automatic registry suggestions), this
release removes the need to import description for criteria that are not
directly used, and adds an explicit version to the cargo-vet instance.
* Import Wasmtime support from the `wit-bindgen` repo
This commit imports the `wit-bindgen-gen-host-wasmtime-rust` crate from
the `wit-bindgen` repository into the upstream Wasmtime repository. I've
chosen to not import the full history here since the crate is relatively
small and doesn't have a ton of complexity. While the history of the
crate is quite long the current iteration of the crate's history is
relatively short so there's not a ton of import there anyway. The
thinking is that this can now continue to evolve in-tree.
* Refactor `wasmtime-component-macro` a bit
Make room for a `wit_bindgen` macro to slot in.
* Add initial support for a `bindgen` macro
* Add tests for `wasmtime::component::bindgen!`
* Improve error forgetting `async` feature
* Add end-to-end tests for bindgen
* Add an audit of `unicase`
* Add a license to the test-helpers crate
* Add vet entry for `pulldown-cmark`
* Update publish script with new crate
* Try to fix publish script
* Update audits
* Update lock file
A minor update of a few other crates drops `semver` and `rustc_version`
from `Cargo.lock`. I've audited the deltas in versions for the other
crates here as well and they all look good.
* Update wasm-tools dependencies
This update brings in a number of features such as:
* The component model binary format and AST has been slightly adjusted
in a few locations. Names are dropped from parameters/results now in
the internal representation since they were not used anyway. At this
time the ability to bind a multi-return function has not been exposed.
* The `wasmparser` validator pass will now share allocations with prior
functions, providing what's probably a very minor speedup for Wasmtime
itself.
* The text format for many component-related tests now requires named
parameters.
* Some new relaxed-simd instructions are updated to be ignored.
I hope to have a follow-up to expose the multi-return ability to the
embedding API of components.
* Update audit information for new crates
Alex Crichton
Pat Hickey
Rainy Sinclair
Afonso Bordado
Chris Fallin
Trevor Elliott
Remo Senekowitsch
Jamey Sharp
Nick Fitzgerald
Dan Gohman
Bobby Holley
Benjamin Bouvier
Bobby Holley