This commit aims to address #8607 by dynamically determining whether the
pooling allocator should be used rather than unconditionally using it.
It looks like some systems don't have enough virtual memory to support
the default configuration settings so this should help `wasmtime serve`
work on those systems.
Closes#8607
* Respect pooling allocation options in `wasmtime serve`
This commit updates the processing of pooling allocator options in
`wasmtime serve`. Previously the pooling allocator was enabled by
default but the options to configure it weren't processed due to how
this default-enable was implemented. The option to enable it by default
for `wasmtime serve`, but only `wasmtime serve`, is now processed
differently in a way that handles various other
pooling-allocator-related options.
Closes#8504
* Fix compile of bench api
* Fix test build
* Ignore newly added test as it's flaky
* Expose `wasmtime-runtime` as `crate::runtime::vm` internally for the `wasmtime` crate
* Rewrite uses of `wasmtime_runtime` to `crate::runtime::vm`
* Remove dep on `wasmtime-runtime` from `wasmtime-cli`
* Move the `wasmtime-runtime` crate into the `wasmtime::runtime::vm` module
* Update labeler for merged crates
* Fix `publish verify`
prtest:full
* Force users to construct WasiHttpCtx through a constructor
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Rename OutgoingRequest to OutgoingRequestConfig and breakout actual request
This rename makes the role of the type easier to understand and draws a
bigger difference between it and HostOutgoingRequest.
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Remove the authority field from the OutgoingRequestConfig
The authority is already encoded in the request itself.
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Add doc comments to the major request/response resource types
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Add more doc comments to resource types
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Rename `IncomingResponseInternal` to `IncomingResponse`
Calling a public type "internal" seems strange. This also adds docs to
the type.
* Small additional changes
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Refactor HttpView::send_request
Don't require implementors to do resource management.
* Make worker task optional
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Fix CI
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
---------
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Support `.wat` files in the `explore` command
Many on the `wasmtime` subcommands work with either `.wasm` or `.wat`
files, but `explore` only works with `.wasm`. This change accepts `.wat`
as well.
* use wat feature
\### The `GcRuntime` and `GcCompiler` Traits
This commit factors out the details of the garbage collector away from the rest
of the runtime and the compiler. It does this by introducing two new traits,
very similar to a subset of [those proposed in the Wasm GC RFC], although not
all equivalent functionality has been added yet because Wasmtime doesn't
support, for example, GC structs yet:
[those proposed in the Wasm GC RFC]: https://github.com/bytecodealliance/rfcs/blob/main/accepted/wasm-gc.md#defining-the-pluggable-gc-interface
1. The `GcRuntime` trait: This trait defines how to create new GC heaps, run
collections within them, and execute the various GC barriers the collector
requires.
Rather than monomorphize all of Wasmtime on this trait, we use it
as a dynamic trait object. This does imply some virtual call overhead and
missing some inlining (and resulting post-inlining) optimization
opportunities. However, it is *much* less disruptive to the existing embedder
API, results in a cleaner embedder API anyways, and we don't believe that VM
runtime/embedder code is on the hot path for working with the GC at this time
anyways (that would be the actual Wasm code, which has inlined GC barriers
and direct calls and all of that). In the future, once we have optimized
enough of the GC that such code is ever hot, we have options we can
investigate at that time to avoid these dynamic virtual calls, like only
enabling one single collector at build time and then creating a static type
alias like `type TheOneGcImpl = ...;` based on the compile time
configuration, and using this type alias in the runtime rather than a dynamic
trait object.
The `GcRuntime` trait additionally defines a method to reset a GC heap, for
use by the pooling allocator. This allows reuse of GC heaps across different
stores. This integration is very rudimentary at the moment, and is missing
all kinds of configuration knobs that we should have before deploying Wasm GC
in production. This commit is large enough as it is already! Ideally, in the
future, I'd like to make it so that GC heaps receive their memory region,
rather than allocate/reserve it themselves, and let each slot in the pooling
allocator's memory pool be *either* a linear memory or a GC heap. This would
unask various capacity planning questions such as "what percent of memory
capacity should we dedicate to linear memories vs GC heaps?". It also seems
like basically all the same configuration knobs we have for linear memories
apply equally to GC heaps (see also the "Indexed Heaps" section below).
2. The `GcCompiler` trait: This trait defines how to emit CLIF that implements
GC barriers for various operations on GC-managed references. The Rust code
calls into this trait dynamically via a trait object, but since it is
customizing the CLIF that is generated for Wasm code, the Wasm code itself is
not making dynamic, indirect calls for GC barriers. The `GcCompiler`
implementation can inline the parts of GC barrier that it believes should be
inline, and leave out-of-line calls to rare slow paths.
All that said, there is still only a single implementation of each of these
traits: the existing deferred reference-counting (DRC) collector. So there is a
bunch of code motion in this commit as the DRC collector was further isolated
from the rest of the runtime and moved to its own submodule. That said, this was
not *purely* code motion (see "Indexed Heaps" below) so it is worth not simply
skipping over the DRC collector's code in review.
\### Indexed Heaps
This commit does bake in a couple assumptions that must be shared across all
collector implementations, such as a shared `VMGcHeader` that all objects
allocated within a GC heap must begin with, but the most notable and
far-reaching of these assumptions is that all collectors will use "indexed
heaps".
What we are calling indexed heaps are basically the three following invariants:
1. All GC heaps will be a single contiguous region of memory, and all GC objects
will be allocated within this region of memory. The collector may ask the
system allocator for additional memory, e.g. to maintain its free lists, but
GC objects themselves will never be allocated via `malloc`.
2. A pointer to a GC-managed object (i.e. a `VMGcRef`) is a 32-bit offset into
the GC heap's contiguous region of memory. We never hold raw pointers to GC
objects (although, of course, we have to compute them and use them
temporarily when actually accessing objects). This means that deref'ing GC
pointers is equivalent to deref'ing linear memory pointers: we need to add a
base and we also check that the GC pointer/index is within the bounds of the
GC heap. Furthermore, compressing 64-bit pointers into 32 bits is a fairly
common technique among high-performance GC
implementations[^compressed-oops][^v8-ptr-compression] so we are in good
company.
3. Anything stored inside the GC heap is untrusted. Even each GC reference that
is an element of an `(array (ref any))` is untrusted, and bounds checked on
access. This means that, for example, we do not store the raw pointer to an
`externref`'s host object inside the GC heap. Instead an `externref` now
stores an ID that can be used to index into a side table in the store that
holds the actual `Box<dyn Any>` host object, and accessing that side table is
always checked.
[^compressed-oops]: See ["Compressed OOPs" in
OpenJDK.](https://wiki.openjdk.org/display/HotSpot/CompressedOops)
[^v8-ptr-compression]: See [V8's pointer
compression](https://v8.dev/blog/pointer-compression).
The good news with regards to all the bounds checking that this scheme implies
is that we can use all the same virtual memory tricks that linear memories use
to omit explicit bounds checks. Additionally, (2) means that the sizes of GC
objects is that much smaller (and therefore that much more cache friendly)
because they are only holding onto 32-bit, rather than 64-bit, references to
other GC objects. (We can, in the future, support GC heaps up to 16GiB in size
without losing 32-bit GC pointers by taking advantage of `VMGcHeader` alignment
and storing aligned indices rather than byte indices, while still leaving the
bottom bit available for tagging as an `i31ref` discriminant. Should we ever
need to support even larger GC heap capacities, we could go to full 64-bit
references, but we would need explicit bounds checks.)
The biggest benefit of indexed heaps is that, because we are (explicitly or
implicitly) bounds checking GC heap accesses, and because we are not otherwise
trusting any data from inside the GC heap, we greatly reduce how badly things
can go wrong in the face of collector bugs and GC heap corruption. We are
essentially sandboxing the GC heap region, the same way that linear memory is a
sandbox. GC bugs could lead to the guest program accessing the wrong GC object,
or getting garbage data from within the GC heap. But only garbage data from
within the GC heap, never outside it. The worse that could happen would be if we
decided not to zero out GC heaps between reuse across stores (which is a valid
trade off to make, since zeroing a GC heap is a defense-in-depth technique
similar to zeroing a Wasm stack and not semantically visible in the absence of
GC bugs) and then a GC bug would allow the current Wasm guest to read old GC
data from the old Wasm guest that previously used this GC heap. But again, it
could never access host data.
Taken altogether, this allows for collector implementations that are nearly free
from `unsafe` code, and unsafety can otherwise be targeted and limited in scope,
such as interactions with JIT code. Most importantly, we do not have to maintain
critical invariants across the whole system -- invariants which can't be nicely
encapsulated or abstracted -- to preserve memory safety. Such holistic
invariants that refuse encapsulation are otherwise generally a huge safety
problem with GC implementations.
\### `VMGcRef` is *NOT* `Clone` or `Copy` Anymore
`VMGcRef` used to be `Clone` and `Copy`. It is not anymore. The motivation here
was to be sure that I was actually calling GC barriers at all the correct
places. I couldn't be sure before. Now, you can still explicitly copy a raw GC
reference without running GC barriers if you need to and understand why that's
okay (aka you are implementing the collector), but that is something you have to
opt into explicitly by calling `unchecked_copy`. The default now is that you
can't just copy the reference, and instead call an explicit `clone` method (not
*the* `Clone` trait, because we need to pass in the GC heap context to run the
GC barriers) and it is hard to forget to do that accidentally. This resulted in
a pretty big amount of churn, but I am wayyyyyy more confident that the correct
GC barriers are called at the correct times now than I was before.
\### `i31ref`
I started this commit by trying to add `i31ref` support. And it grew into the
whole traits interface because I found that I needed to abstract GC barriers
into helpers anyways to avoid running them for `i31ref`s, so I figured that I
might as well add the whole traits interface. In comparison, `i31ref` support is
much easier and smaller than that other part! But it was also difficult to pull
apart from this commit, sorry about that!
---------------------
Overall, I know this is a very large commit. I am super happy to have some
synchronous meetings to walk through this all, give an overview of the
architecture, answer questions directly, etc... to make review easier!
prtest:full
* More flags like `--dir` and `--env` are moved into `RunCommon` to be
shared between `wasmtime serve` and `wasmtime run`, meaning that the
`serve` command can now configure environment variables.
* A small test has been added as well as infrastructure for running
tests with `wasmtime serve` itself. Previously there were no tests
that executed `wasmtime serve`.
* The `test_programs` crate had a small refactoring to avoid
double-generation of http bindings.
* Add documentation and examples for `wasmtime-wasi`
This commit adds lots of missing documentation and examples to top-level
types in `wasmtime-wasi`, mostly related to WASIp2. I've additionally
made a number of small refactorings here to try to make the APIs a bit
more straightforward and symmetric and simplify where I can.
* Remove `bindings::wasi` (reexports are still present under `bindings`)
* Rename `bindings::sync_io` to `bindings::sync`
* Generate fewer bindings in `bindings::sync` that can be pulled in from
the `bindings` module.
* Change `WasiCtxBuilder::preopened_dir` to take a path instead of a
`Dir` argument to avoid the need for another import.
* Synchronize `wasmtime_wasi_http::{add_to_linker, sync::add_to_linker}`
in terms of interfaces added.
* Remove `wasmtime_wasi::command` and move the generated types to the
`bindings` module.
* Move top-level add-to-linker functions to
`wasmtime_wasi::add_to_linker_sync` and
`wasmtime_wasi::add_to_linker_async`.
Closes#8187Closes#8188
* Add documentation for `wasmtime_wasi::preview1` and refactor
This commit adds documentation for the `wasmtime_wasi::preview1` module
and additionally refactors it as well. Previously this was based on a
similar design as WASIp2 with a "view trait" and various bits and
pieces, but the design constraints of WASIp1 lends itself to a simpler
solution of "just" passing around `WasiP1Ctx` instead. This goes back to
what `wasi-common` did of sorts where the `add_to_linker_*` functions
only need a projection from `&mut T` to `&mut WasiP1Ctx`, a concrete
type, which simplifies the module and usage.
* Small refactorings to `preopened_dir`
* Add `WasiCtx::builder`.
* Fix typo
* Review comments
* Add a `ModuleBuilder` type to the `wasmtime` crate
This commit is extracted from #8055 and adds a new `ModuleBuilder` type
which is intended to be able to further configure compilation beyond
what the constructors of `Module` already provide. For example in #8055
knobs will be added to process `*.dwp` files for more debuginfo
processing.
Co-authored-by: yowl00 <scott.waye@hubse.com>
* Fix build
* Review feedback
* Rename ModuleBuilder to CodeBuilder
* Fix doc errors
---------
Co-authored-by: yowl00 <scott.waye@hubse.com>
* Implement opt-in for enabling WASI to block the current thread
Currently all of Wasmtime's implementation of WASI is built on Tokio,
but some operations are currently not asynchronous such as opening a
file or reading a directory. Internally these use `spawn_blocking` to
satisfy the requirements of async users of WASI to avoid blocking the
current thread. This use of `spawn_blocking`, however, ends up causing
mostly just performance overhead in the use case of the CLI, for
example, where async wasm is not used. That then leads to this commit,
implementing an opt-in mechanism to be able to block the current thread.
A `WasiCtx` now has a flag indicating whether it's ok to block the
current thread and that's carried to various filesystem operations that
use `spawn_blocking`. The call to `spawn_blocking` is now conditional
and skipped if this flag is set.
Additionally the invocation point in the CLI for wasm modules is wrapped
in a Tokio runtime to avoid entering/exiting Tokio in the "leaves" when
wasm calls the host, as happens today. This hits a better fast path in
Tokio that appears to be more efficient.
Semantically this should not result in any change for CLI programs
except in one case: file writes. By default writes on `output-stream` in
WASI are asynchronous meaning that only one write can be in flight at a
time. That being said all current users are immediately blocking waiting
for this write to finish anyway, so this commit won't end up changing
much. It's already the case that file reads are always blocking, for
example. If necessary in the future though this can be further
special-cased at the preview1 layer.
* Thread around allow_blocking_current_thread less
Pass around `&File` instead.
* Rename `-S common` to `-S cli`.
The "common" in `-S common` came from "wasi-common" which came from the
idea of having code in common between Wasmtime, Lucet, and others. It
doesn't have a clear meaning for end users, and has a risk of being
interpreted as "common" functionality that's generally available
everywhere.
This PR renames `-S common` to `-S cli`, and documents it as including
the WASI CLI APIs, to clarify its purpose `-S common` is still accepted,
with a warning.
* Fix formatting in RELEASES.md.
* Disable the warning about `-S common` for now.
* Run all `*.wast` tests in fuzzing
Currently we have a `spectest` fuzzer which uses fuzz input to generate
an arbitrary configuration for Wasmtime and then executes the spec test.
This ensures that no matter the configuration Wasmtime can pass spec
tests. This commit expands this testing to include all `*.wast` tests we
have in this repository. While we don't have a ton we still have some
significant ones like in #8118 which will only reproduce when turning
knobs on CPU features.
* Fix CLI build
* Fix wast testing
* Fix `wasmtime settings` command.
Currently the `settings` command panics because the tunables are not set in the
compiler builder.
This commit creates a default tunables based on either the target triple passed
on the command line or uses the default for the host.
Fixes#8058.
* Additional clean up.
* Show true error reason in `wasmtime serve`
The `wasmtime serve` command prints failed HTTP handling to `stderr`
but, when the reason was due to failure inside the HTTP handler itself,
the printed error message is not descriptive enough. After looking at
long dumps of `guest never invoked `response-outparam::set` method`, I
realized that errors inside the Tokio `task` are never propagated
outwards. This change captures those errors and appends them to the
printed error message.
* review: remove TODO comment
* review: improve doc comment
Co-authored-by: Trevor Elliott <telliott@fastly.com>
* review: add comment about immediate resolution
---------
Co-authored-by: Trevor Elliott <telliott@fastly.com>
* Enable compiling the Wasmtime CLI to Wasm
While not the most useful thing to do in the world it's kind of neat to
play around with. This builds on the previous work to exclude the
runtime from the `wasmtime` crate so it's now possible to compile the
Wasmtime CLI's `compile` command, and only the `compile` command, to
wasm itself. This means you can run Wasmtime in Wasmtime!
* Fix warning on wasm
* Fix some feature combos
* better top matter
* eliminate wasi-common deprecations from wasmtime-wasi
* wasmtime-wasi: preview2 feature always enabled, so just eliminate it
* rename preview1-on-preview2 feature to just preview1
* wasi-http fix
* dep fixes. change some log::debug! to tracing::debug!
* mv preview2 up to root
and `sed -i 's/crate::preview2/crate/g' **/*.rs`
* fix tests too
* fixes throughout wasmtime-wasi-http
* cli: s/wasmtime_wasi::preview2/wasmtime_wasi
* rustfmt
* fix wasi-async example
* fix docs build (needs wasi-common built to compile examples)
* c-api: use wasi-common directly
i guess we didnt build the c-api in CI with deprecation warnings denied
prtest:full
* fix example required-features
* fix examples CMakeLists build
* Wasmtime: Finish support for the typed function references proposal
While we supported the function references proposal inside Wasm, we didn't
support it on the "outside" in the Wasmtime embedder APIs. So much of the work
here is exposing typed function references, and their type system updates, in
the embedder API. These changes include:
* `ValType::FuncRef` and `ValType::ExternRef` are gone, replaced with the
introduction of the `RefType` and `HeapType` types and a
`ValType::Ref(RefType)` variant.
* `ValType` and `FuncType` no longer implement `Eq` and `PartialEq`. Instead
there are `ValType::matches` and `FuncType::matches` methods which check
directional subtyping. I also added `ValType::eq` and `FuncType::eq` static
methods for the rare case where someone needs to check precise equality, but
that is almost never actually the case, 99.99% of the time you want to check
subtyping.
* There are also public `Val::matches_ty` predicates for checking if a value is
an instance of a type, as well as internal helpers like
`Val::ensure_matches_ty` that return a formatted error if the value does not
match the given type. These helpers are used throughout Wasmtime internals
now.
* There is now a dedicated `wasmtime::Ref` type that represents reference
values. Table operations have been updated to take and return `Ref`s rather
than `Val`s.
Furthermore, this commit also includes type registry changes to correctly manage
lifetimes of types that reference other types. This wasn't previously an issue
because the only thing that could reference types that reference other types was
a Wasm module that added all the types that could reference each other at the
same time and removed them all at the same time. But now that the previously
discussed work to expose these things in the embedder API is done, type lifetime
management in the registry becomes a little trickier because the embedder might
grab a reference to a type that references another type, and then unload the
Wasm module that originally defined that type, but then the user should still be
able use that type and the other types it transtively references. Before, we
were refcounting individual registry entries. Now, we still are refcounting
individual entries, but now we are also accounting for type-to-type references
and adding a new type to the registry will increment the refcounts of each of
the types that it references, and removing a type from the registry will
decrement the refcounts of each of the types it references, and then recursively
(logically, not literally) remove any types whose refcount has now reached zero.
Additionally, this PR adds support for subtyping to `Func::typed`- and
`Func::wrap`-style APIs. For result types, you can always use a supertype of the
WebAssembly function's actual declared return type in `Func::typed`. And for
param types, you can always use a subtype of the Wasm function's actual declared
param type. Doing these things essentially erases information but is always
correct. But additionally, for functions which take a reference to a concrete
type as a parameter, you can also use the concrete type's supertype. Consider a
WebAssembly function that takes a reference to a function with a concrete type:
`(ref null <func type index>)`. In this scenario, there is no static
`wasmtime::Foo` Rust type that corresponds to that particular Wasm-defined
concrete reference type because Wasm modules are loaded dynamically at
runtime. You *could* do `f.typed::<Option<NoFunc>, ()>()`, and while that is
correctly typed and valid, it is often overly restrictive. The only value you
could call the resulting typed function with is the null function reference, but
we'd like to call it with non-null function references that happen to be of the
correct type. Therefore, `f.typed<Option<Func>, ()>()` is also allowed in this
case, even though `Option<Func>` represents `(ref null func)` which is the
supertype, not subtype, of `(ref null <func type index>)`. This does imply some
minimal dynamic type checks in this case, but it is supported for better
ergonomics, to enable passing non-null references into the function.
We can investigate whether it is possible to use generic type parameters and
combinators to define Rust types that precisely match concrete reference types
in future, follow-up pull requests. But for now, we've made things usable, at
least.
Finally, this also takes the first baby step towards adding support for the Wasm
GC proposal. Right now the only thing that is supported is `nofunc` references,
and this was mainly to make testing function reference subtyping easier. But
that does mean that supporting `nofunc` references entailed also adding a
`wasmtime::NoFunc` type as well as the `Config::wasm_gc(enabled)` knob. So we
officially have an in-progress implementation of Wasm GC in Wasmtime after this
PR lands!
Fixes https://github.com/bytecodealliance/wasmtime/issues/6455
* Fix WAT in test to be valid
* Check that dependent features are enabled for function-references and GC
* Remove unnecessary engine parameters from a few methods
Ever since `FuncType`'s internal `RegisteredType` holds onto its own `Engine`,
we don't need these anymore.
Still useful to keep the `Engine` parameter around for the `ensure_matches`
methods because that can be used to check correct store/engine usage for
embedders.
* Add missing dependent feature enabling for some tests
* Remove copy-paste bit from test
* match self to show it is uninhabited
* Add a missing `is_v128` method
* Short circuit a few func type comparisons
* Turn comment into part of doc comment
* Add test for `Global::new` and subtyping
* Add tests for embedder API, tables, and subtyping
* Add an embedder API test for setting globals and subtyping
* Construct realloc's type from its index, rather than from scratch
* Help LLVM better optimize our dynamic type checks in `TypedFunc::call_raw`
* Fix call benchmark compilation
* Change `WasmParams::into_abi` to take the whole func type instead of iter of params
* Fix doc links
prtest:full
* Fix size assertion on s390x
* WIP: try to make wasi-common self contained.
* rebase: cargo.lock
* remove all dependencies between wasi-common and wasmtime-wasi
* use wasi-common directly throughout tests, benches, examples, cli run
* wasi-threads: use wasi-common's maybe_exit_on_error in spawned thread
not a very modular design, but at this point wasi-common and
wasi-threads are forever wed
* fix wasmtime's docs
* re-introduce wasmtime-wasi's exports of wasi-common definitions behind deprecated
* factor out determining i32 process exit code
and remove libc dep because rustix provides the same constant
* commands/run: inline the logic about aborting on trap
since this is the sole place in the codebase its used
* Add high-level summary to wasi-common's top-level doc comment.
* c-api: fix use of wasi_cap_std_sync => wasi_common::sync, wasmtime_wasi => wasi_common
* fix tokio example
* think better of combining downcast and masking into one method
* fix references to wasmtime_wasi in docs
prtest:full
* benches: use wasi-common
* cfg-if around use of rustix::process because that doesnt exist on windows
* wasi-common: include tests, caught by verify-publish
* fix another bench
* exit requires wasmtime dep. caught by verify-publish.
* Add function accepting explicit CPU delta time to GuestProfiler
* Remove unused import
* Modify existing signature instead of adding new
* Use qualified Duration instead of import to prevent unused warnings when building without features
Using an epoch of `1` and setting its duration to the full timeout
configured does not work for `wasmtime serve`, as the epoch is counted
globally per engine. This commit switches to dividing the timeout by 10,
and expiring an instance after 11 epochs have been observed, giving a
maximum overshoot of 10%.
Co-authored-by: Jamey Sharp <jsharp@fastly.com>
Simplify the implementation of the serve command by removing the
explicit Service trait impl, and using `service_fn` instead.
Co-authored-by: Jamey Sharp <jsharp@fastly.com>
* Add `runtime` feature to `wasmtime` crate
This feature can be disabled to build `wasmtime` only for compilation.
This can be useful when cross-compiling, especially on a target that
can't run wasmtime itself (e.g. `wasm32`).
* prtest:full
* don't round pages without runtime feature
* fix async assertions
* move profiling into runtime
* enable runtime for wasmtime-wasi
* enable runtime for c-api
* fix build_artifacts in non-cache case
* fix miri extensions
* enable runtime for wast
* enable runtime for explorer
* support cranelift all-arch on wasm32
* add doc links for `WeakEngine`
* simplify lib runtime cfgs
* move limits and resources to runtime
* move stack to runtime
* move coredump and debug to runtime
* add runtime to coredump and async features
* add wasm32 build job
* combine engine modules
* single compile mod
* remove allow for macro paths
* add comments
* Enable the component model by default
This commit enables the component model by default in the embedding API
and the CLI. This means that an opt-in of `-W component-model` is no
longer required and additionally `.wasm_component_model(true)` is no
longer required. Note that this won't impact existing embeddings since
the component model feature doesn't do much less `wasmtime::component`
is used, and if that's being used this is probably good news for you.
* Fix non-component model build
* Remove `+ Sync` constraints from preview2 implementation
* Only use mutable references in WasiView to guarantee unique access to Preview2 resources. Removed the _mut suffixes to align with WasiHttpView.
* Always use Descriptor::Directory for directories, regardless of whether they were preopened or opened using open_at. This fixes build errors regarding overlapping mutable lifetimes introduced in the previous commit.
* Remove some more `+ Sync`s
* Remove one more
* typo
* Fix build errors on Rust <= 1.73. Code already compiled fine on >= 1.74
* Add API to WasiCtx for adding dynamic SocketAddr checks
The user can now specify a closure which gets run any time a connection
to an external address is attempted. If the closure returns true, the
address is allowed, but if it returns false, the underlying pool is then
checked. In otherwords, false does not imply denied, but rather, check
the underlyin pool.
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Get rid of Pool in favor of just a closure
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Add a SocketAddrUse arg to the SocketAddrCheck
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
---------
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Ensure remote_address is allowed when creating UDP stream
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Check addr validity after UDP state check and wrap pool in Arc
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Check provided addr in outgoing stream
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Add test for turning off udp
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Model pool on UDP types as optional to catch bad state bugs
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
---------
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* rename to ResourceTable, docs, and export
* wasi: use the table from wasmtime::component
* wasi-http: use wasmtime::component's resource table now
* rename file (somehow got lost)
* wasmtime-cli: fixes for resourcetable
* fix wasi-http tests
* more test fixes
* error messages
* fix warning
* cli: fix min build
prtest:full
* Add options to WasiCtx for toggling TCP and UDP on and off
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Address PR feedback
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
* Add test for turning off tcp
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
---------
Signed-off-by: Ryan Levick <ryan.levick@fermyon.com>
This commit adds some more information to `wasmtime --version` which
includes the git commit plus the git commit's date. This matches `rustc
-V` for example which was additionally copied to `wasm-tools` and
mirrored as `wasm-tools -V`.
Personally I've found this useful since it can help point to exact
commits and additionally quickly get a sense of how old a version is
based on its commit date presented.
This commit aims to address a discrepancy in Wasmtime where the world
supported by `wasmtime serve` is too large today. This includes
WIT interfaces which are not specified in `wasi:http/proxy` such as
`wasi:filesystem/types`, aka access to a filesystem.
This commit slims down `wasmtime serve` to, by default, only supporting
the `wasi:http/proxy` world. Like with `wasmtime run` various CLI flags
can be passed to enable more interfaces, however:
* `-Scommon` - this enables "common" interfaces such as
`wasi:filesystem`, `wasi:sockets`, and `wasi:cli/*`.
* `-Snn` - this enables wasi-nn
It's expected that more will get extended here over time too.
This change is enabled by a third build of the adapter, a "proxy" mode.
In this mode most functions are cfg'd to return `ERRNO_NOTSUP` to
indicate that the runtime does not support it. Notably this includes the
filesystem, arguments, and environment variables.
This change is tested by updating all `api_proxy*` tests to use this new
adapter which is now required that many previous interfaces are no
longer present by default in the proxy world.
* Configure Rust lints at the workspace level
This commit adds necessary configuration knobs to have lints configured
at the workspace level in Wasmtime rather than the crate level. This
uses a feature of Cargo first released with 1.74.0 (last week) of the
`[workspace.lints]` table. This should help create a more consistent set
of lints applied across all crates in our workspace in addition to
possibly running select clippy lints on CI as well.
* Move `unused_extern_crates` to the workspace level
This commit configures a `deny` lint level for the
`unused_extern_crates` lint to the workspace level rather than the
previous configuration at the individual crate level.
* Move `trivial_numeric_casts` to workspace level
* Change workspace lint levels to `warn`
CI will ensure that these don't get checked into the codebase and
otherwise provide fewer speed bumps for in-process development.
* Move `unstable_features` lint to workspace level
* Move `unused_import_braces` lint to workspace level
* Start running Clippy on CI
This commit configures our CI to run `cargo clippy --workspace` for all
merged PRs. Historically this hasn't been all the feasible due to the
amount of configuration required to control the number of warnings on
CI, but with Cargo's new `[lint]` table it's possible to have a
one-liner to silence all lints from Clippy by default. This commit by
default sets the `all` lint in Clippy to `allow` to by-default disable
warnings from Clippy. The goal of this PR is to enable selective access
to Clippy lints for Wasmtime on CI.
* Selectively enable `clippy::cast_sign_loss`
This would have fixed#7558 so try to head off future issues with that
by warning against this situation in a few crates. This lint is still
quite noisy though for Cranelift for example so it's not worthwhile at
this time to enable it for the whole workspace.
* Fix CI error
prtest:full
This commit implements the `wasi_unstable` module, sometimes referred to
as "preview0", in the `wasmtime-wasi` crate. Previously this was only
implemented by the `wasi-common` crate but now this is implemented for
both meaning that the switch to preview2 won't lose this functionality.
The preview0 WITX files are vendored like the preview1 files and the
implementation of preview0 is exclusively implemented by delegating to
the preview1 implementation.
* Implement the missing http-error-code function
* Add concrete error conversion functions
* Test the error returned when writing too much
* Remove an unused import
* Only log errors when they will get the default representation
* Use `source` instead of `into_cause`
* Migrate to a more specific error-code variant in wasi-http
Co-authored-by: Pat Hickey <phickey@fastly.com>
* Optional fields, and align with upstream pr
* Update for upstream changes to the error-code variant
* Sync with the upstream implementation
* Missed updating an error for riscv64 and s390x
* More debuggable error
prtest:full
* Try to stabilize the test on windows
---------
Co-authored-by: Pat Hickey <phickey@fastly.com>
* Print the error from the wasi-http incoming handler to stderr
Prior to this change, the error was never printed out, making it a challenge to figure out what caused the error
* use log::error instead of eprintln
Co-authored-by: Trevor Elliott <awesomelyawesome@gmail.com>
* Update src/commands/serve.rs
---------
Co-authored-by: Trevor Elliott <awesomelyawesome@gmail.com>
In Wasmtime 13 and prior the `--dir` argument was unconditionally used
to open a host dir as the same name and in the guest. In Wasmtime 14+
though this argument is being repurposed with an optional trailing
`::GUEST` to configure the guest directory. This means that
`--dir`-with-remapping behavior is actually unusable without the
environment variable configuration from #7385 due to it parsing
differently in the old and the new.
This commit updates the situation by adding `::`-parsing to the old CLI
meaning that both the old and the new parse this the same way. This will
break any scripts that open host directories with two colons in their
path, but that seems niche enough we can handle that later.
Print a message on `wasmtime serve` startup showing the address that
`wasmtime` is listening for requests on, so that users know what it's
doing, and can copy+past the URL into curl or a browser to start
using it.
For example:
```
$ wasmtime serve target/wasm32-wasi/debug/hello_wasi_http.wasm
Serving HTTP on http://0.0.0.0:8080/
```
This commit fixes a panic in the `wasmtime serve` CLI command when the
`response-outparam::set` method was never invoked by the guest. In this
situation return a more first-class error than a panic explaining what
the guest should be doing.
FrankReh
Alex Crichton
Nick Fitzgerald
Ryan Levick
Adam Bratschi-Kaye
Dan Gohman
Pat Hickey
Peter Huene
Andrew Brown
Milek7
Trevor Elliott
Dave Bakker
Xinzhao Xu
Jake Champion