* Adjust dependency directives between crates
This commit is a preparation for the release process for Wasmtime. The
specific changes here are to delineate which crates are "public", and
all version requirements on non-public crates will now be done with
`=A.B.C` version requirements instead of today's `A.B.C` version
requirements.
The purpose for doing this is to assist with patch releases that might
happen in the future. Patch releases of wasmtime are already required to
not break the APIs of "public" crates, but no such guarantee is given
about "internal" crates. This means that a patch release runs the risk,
for example, of breaking an internal API. In doing so though we would
also need to release a new major version of the internal crate, but we
wouldn't have a great hole in the number scheme of major versions to do
so. By using `=A.B.C` requirements for internal crates it means we can
safely ignore strict semver-compatibility between releases of internal
crates for patch releases, since the only consumers of the crate will be
the corresponding patch release of the `wasmtime` crate itself (or other
public crates).
The `publish.rs` script has been updated with a check to verify that
dependencies on internal crates are all specified with an `=`
dependency, and dependnecies on all public crates are without a `=`
dependency. This will hopefully make it so we don't have to worry about
what to use where, we just let CI tell us what to do. Using this
modification all version dependency declarations have been updated.
Note that some crates were adjusted to simply remove their `version`
requirement in cases such as the crate wasn't published anyway (`publish
= false` was specified) or it's in the `dev-dependencies` section which
doesn't need version specifiers for path dependencies.
* Switch to normal sever deps for cranelift dependencies
These crates will now all be considered "public" where in patch releases
they will be guaranteed to not have breaking changes.
* Optimize `Func::call` and its C API
This commit is an alternative to #3298 which achieves effectively the
same goal of optimizing the `Func::call` API as well as its C API
sibling of `wasmtime_func_call`. The strategy taken here is different
than #3298 though where a new API isn't created, rather a small tweak to
an existing API is done. Specifically this commit handles the major
sources of slowness with `Func::call` with:
* Looking up the type of a function, to typecheck the arguments with and
use to guide how the results should be loaded, no longer hits the
rwlock in the `Engine` but instead each `Func` contains its own
`FuncType`. This can be an unnecessary allocation for funcs not used
with `Func::call`, so this is a downside of this implementation
relative to #3298. A mitigating factor, though, is that instance
exports are loaded lazily into the `Store` and in theory not too many
funcs are active in the store as `Func` objects.
* Temporary storage is amortized with a long-lived `Vec` in the `Store`
rather than allocating a new vector on each call. This is basically
the same strategy as #3294 only applied to different types in
different places. Specifically `wasmtime::Store` now retains a
`Vec<u128>` for `Func::call`, and the C API retains a `Vec<Val>` for
calling `Func::call`.
* Finally, an API breaking change is made to `Func::call` and its type
signature (as well as `Func::call_async`). Instead of returning
`Box<[Val]>` as it did before this function now takes a
`results: &mut [Val]` parameter. This allows the caller to manage the
allocation and we can amortize-remove it in `wasmtime_func_call` by
using space after the parameters in the `Vec<Val>` we're passing in.
This change is naturally a breaking change and we'll want to consider
it carefully, but mitigating factors are that most embeddings are
likely using `TypedFunc::call` instead and this signature taking a
mutable slice better aligns with `Func::new` which receives a mutable
slice for the results.
Overall this change, in the benchmark of "call a nop function from the C
API" is not quite as good as #3298. It's still a bit slower, on the
order of 15ns, because there's lots of capacity checks around vectors
and the type checks are slightly less optimized than before. Overall
though this is still significantly better than today because allocations
and the rwlock to acquire the type information are both avoided. I
personally feel that this change is the best to do because it has less
of an API impact than #3298.
* Rebase issues
* Fix wiggle code generation for correct span usage
Up to this point when using wiggle to generate functions we could end up
with two types of functions an async or sync one with this proc macro
```
#[allow(unreachable_code)] // deals with warnings in noreturn functions
pub #asyncness fn #ident(
ctx: &mut (impl #(#bounds)+*),
memory: &dyn #rt::GuestMemory,
#(#abi_params),*
) -> Result<#abi_ret, #rt::Trap> {
use std::convert::TryFrom as _;
let _span = #rt::tracing::span!(
#rt::tracing::Level::TRACE,
"wiggle abi",
module = #mod_name,
function = #func_name
);
let _enter = _span.enter();
#body
}
```
Now this might seem fine, we just create a span and enter it and run the
body code and we get async versions as well. However, this is where the
source of our problem lies. The impetus for this fix was seeing multiple
request IDs output in the logs for a single function call of a generated
function. Something was clearly happening that shouldn't have been. If
we take a look at the tracing docs here we can see why the above code
will not work in asynchronous code.
https://docs.rs/tracing/0.1.26/tracing/span/struct.Span.html#in-asynchronous-code
> Warning: in asynchronous code that uses async/await syntax,
> Span::enter should be used very carefully or avoided entirely.
> Holding the drop guard returned by Span::enter across .await points
> will result in incorrect traces.
The above documentation provides some more information, but what could
happen is that the `#body` itself could contain code that would await
and mess up the tracing that occurred and causing output that would be
completely nonsensical. The code itself should work fine in the
synchronous case though and in cases where await was not called again
inside the body as the future would poll to completion as if it was a
synchronous function.
The solution then is to use the newer `Instrument` trait which can make
sure that the span will be entered on every poll of the future. In order
to make sure that we have the same behavior as before we generate
synchronous functions and the ones that were async instead return a
future that uses the instrument trait. This way we can guarantee that
the span is created in synchronous code before being passed into a
future. This does change the function signature, but the functionality
itself is exactly as before and so we should see no actual difference in
how it's used by others. We also just to be safe call the synchronous
version's body with `in_scope` now as per the docs recommendation even
though it's more intended for calling sync code inside async functions.
Functionally it's the same as before with the call to enter. We also
bump the version of tracing uses so that wiggle can reexport tracing
with the instrument changes.
* Move function span generation out of if statement
We were duplicating the span creation code in our function generation in
wiggle. This commit moves it out into one spot so that we can reuse it
in both branches of the async/sync function generation.
* Make formatting consistent
sync test: show the dummy executor will trap (rather than panic) when a
future inside it pends.
async test: show that the executor is hooked up to a future that pends
for a trivial amount of time.
this adds tokio to the dev-dependencies of wiggle, it shouldn't end up
increasing the build burden for the project as a whole since its already
a dev-dependency.
* wasi-common: update wasi submodule
This updates the WASI submodule, pulling in changes to the witx crate,
now that there is a 0.9.1 version including some bug fixes. See
WebAssembly/WASI#434 for more information.
* wiggle: update witx dependencies
* publish: verify and vendor witx-cli
* adjust root workspace members
This commit removes some items from the root manifest's workspace
members array, and adds `witx-cli` to the root `workspace.exclude`
array.
The motivation for this stems from a cargo bug described in
rust-lang/cargo#6745: `workspace.exclude` does not work if it is nested
under a `workspace.members` path.
See WebAssembly/WASI#438 for the underlying change to the WASI submodule
which reorganized the `witx-cli` crate, and WebAssembly/WASI#398 for the
original PR introducing `witx-cli`.
See [this
comment](https://github.com/bytecodealliance/wasmtime/pull/3025#issuecomment-867741175)
for more details about the compilation errors, and failed alternative
approaches that necessitated this change.
N.B. This is not a functional change, these crates are still implicitly
workspace members as transitive dependencies, but this will allow us to
side-step the aforementioned cargo bug.
Co-Authored-By: Alex Crichton <alex@alexcrichton.com>
Co-authored-by: Alex Crichton <alex@alexcrichton.com>
* Update wasm-tools crates
This brings in recent updates, notably including more improvements to
wasm-smith which will hopefully help exercise non-trapping wasm more.
* Fix some wat
This is currently a very common operation in host bindings where if wasm
gives a host function a relative pointer you'll want to simulataneously
work with the host state and the wasm memory. These two regions are
distinct and safe to borrow mutably simulataneously but it's not obvious
in the Rust type system that this is so, so add a helper method here to
assist in doing so.
In commit 33c791e1f5 (PR #2944), I added LICENSE files to all published
crates. However, since then PR #2897 has been merged and remove 3 crates,
resulting in license files in empty directories.
Implement Wasmtime's new API as designed by RFC 11. This is quite a large commit which has had lots of discussion externally, so for more information it's best to read the RFC thread and the PR thread.
This commit adds LICENSE files to all **published** crates which do
not have it already (most of the crates have it).
Providing the license files is a requiment of the Apache 2.0 License.
async methods used by wiggle currently need to Not have the Send
constraint, so rather than make all use sites pass the argument
to the re-exported async_trait macro, define a new macro that
applies the argument.
The code I wrote here prior was incorrect: a span is present at the
level specified and below; previously I thought it was present at the
level specified and above. So, previously, a TRACE-level event inside
this span would be associated with the module and function name provided
here. Now all events inside this span should be associated with it.
* Implement defining host functions at the Config level.
This commit introduces defining host functions at the `Config` rather than with
`Func` tied to a `Store`.
The intention here is to enable a host to define all of the functions once
with a `Config` and then use a `Linker` (or directly with
`Store::get_host_func`) to use the functions when instantiating a module.
This should help improve the performance of use cases where a `Store` is
short-lived and redefining the functions at every module instantiation is a
noticeable performance hit.
This commit adds `add_to_config` to the code generation for Wasmtime's `Wasi`
type.
The new method adds the WASI functions to the given config as host functions.
This commit adds context functions to `Store`: `get` to get a context of a
particular type and `set` to set the context on the store.
For safety, `set` cannot replace an existing context value of the same type.
`Wasi::set_context` was added to set the WASI context for a `Store` when using
`Wasi::add_to_config`.
* Add `Config::define_host_func_async`.
* Make config "async" rather than store.
This commit moves the concept of "async-ness" to `Config` rather than `Store`.
Note: this is a breaking API change for anyone that's already adopted the new
async support in Wasmtime.
Now `Config::new_async` is used to create an "async" config and any `Store`
associated with that config is inherently "async".
This is needed for async shared host functions to have some sanity check during their
execution (async host functions, like "async" `Func`, need to be called with
the "async" variants).
* Update async function tests to smoke async shared host functions.
This commit updates the async function tests to also smoke the shared host
functions, plus `Func::wrap0_async`.
This also changes the "wrap async" method names on `Config` to
`wrap$N_host_func_async` to slightly better match what is on `Func`.
* Move the instance allocator into `Engine`.
This commit moves the instantiated instance allocator from `Config` into
`Engine`.
This makes certain settings in `Config` no longer order-dependent, which is how
`Config` should ideally be.
This also removes the confusing concept of the "default" instance allocator,
instead opting to construct the on-demand instance allocator when needed.
This does alter the semantics of the instance allocator as now each `Engine`
gets its own instance allocator rather than sharing a single one between all
engines created from a configuration.
* Make `Engine::new` return `Result`.
This is a breaking API change for anyone using `Engine::new`.
As creating the pooling instance allocator may fail (likely cause is not enough
memory for the provided limits), instead of panicking when creating an
`Engine`, `Engine::new` now returns a `Result`.
* Remove `Config::new_async`.
This commit removes `Config::new_async` in favor of treating "async support" as
any other setting on `Config`.
The setting is `Config::async_support`.
* Remove order dependency when defining async host functions in `Config`.
This commit removes the order dependency where async support must be enabled on
the `Config` prior to defining async host functions.
The check is now delayed to when an `Engine` is created from the config.
* Update WASI example to use shared `Wasi::add_to_config`.
This commit updates the WASI example to use `Wasi::add_to_config`.
As only a single store and instance are used in the example, it has no semantic
difference from the previous example, but the intention is to steer users
towards defining WASI on the config and only using `Wasi::add_to_linker` when
more explicit scoping of the WASI context is required.
* ctx parameter no longer accepted by wiggle::from_witx macro.
* optional async_ parameter specifies which functions are async.
* re-export async_trait::async_trait, so users don't have to take a dep.
wasmtime-publish
Benjamin Bouvier
Pat Hickey
Alex Crichton
Nick Fitzgerald
Andrew Brown
Michael Gattozzi
Chris Fallin
Pat Hickey
katelyn martin
Olivier Lemasle
Chris Fallin
Peter Huene
Dan Gohman