cranelift

Commit Graph

Author	SHA1	Message	Date
Alex Crichton	5a6ed0fbd2	Implement component model resources in Wasmtime (#6691 ) * Fix signatures registered with modules-in-components This commit fixes a minor issue in `FunctionIndices::link_and_append_code` which previously ended up only filling out the `wasm_to_native_trampolines` field for the first module rather than all the modules. Additionally the first module might have too many entries that encompass all modules instead of just its own entries. The fix in this commit is to refactor this logic to ensure that the necessary maps are present for all translations. While technically a bug that can be surfaced through the embedder API it's pretty obscure. The given test here panics beforehand but succeeds afterwards, but this is moreso prep for some future resource-related work where this map will need persisting into the component metadata side of things. * Initial support for resources Lots of bits and pieces squashed into this commit. Much to be done still. * Start supporting destructors * Get some basic drop tests working Also add a test which requires host-defined drop to be called which isn't working. * Fix rebase issue * Fix a failing test * I am zorthax, destroyer of resources * Remove a branch in compiled code No need to check for a null funcref when we already know ahead of time if it's ever going to be null or not. * Fix the test suite * Add embedder API to destroy resources * Add TODO for factc * Fix a warning and leave a comment * Integrate resources into `Type` Plumb around dynamic information about resource types. * Implement `Val::Own` * Implement reentrance check for destructors Implemented both in the raw wasm intrinsic as well as the host. * Use cast instead of transmute * Fill out some cranelift-shared comments * Update codegen for resource.drop shim The MAY_ENTER flag must always be checked, regardless of whether there's an actual destructor or not. * Update wasm-tools crates to latest `main` * Update resource.drop binary format * Add some docs * Implement dynamic tracking for borrow resources Not actually hooked up anywhere but this should at least be a first stab at an implementation of the spec. * Remove git overrides * Remove no-longer-needed arms in wit-bindgen * Prepare for mutability in `LiftContext` * Change `&LiftContext` to `&mut LiftContext` * Remove `store: &'a StoreOpaque` from `LiftContext`, instead storing just `memory: &'a [u8]` * Refactor methods to avoid needing the entire `StoreOpaque` This'll enable `LiftContext` to store `&'a mut ResourceTable` in an upcoming commit to refer to the host's resources. * Lowering a borrow is infallible * Use `ResourceAny` for both own/borrow Rename `Val::Own` to `Val::Resource` accordingly. * Initial implementation of borrowed resources Lots of juggling of contexts here and there to try and get everything working but this is hopefully a faithful implementation. Tests not implemented yet and will come next and additionally likely update implementation details as issues are weeded out. * Add a suite of tests for borrowing resources Code coverage was used to ensure that almost all of the various paths through the code are taken to ensure all the basic bases are covered. There's probably still lurking bugs, but this should be a solid enough base to start from hopefully. * Fill in an issue for bindgen todo * Add docs, still more to go * Fill out more documentation * Fill out a test TODO * Update the host `Resource<T>` type * Add docs everywhere * Don't require a `Store` for creating the resource or getting the representation. The latter point is the main refactoring in this commit. This is done in preparation for `bindgen!` to use this type where host bindings generally do not have access to the store. * Document `ResourceAny` * Debug assert dtor is non-null * Review comments on loading libcalls * Update some comments * Update a comment * Fix some typos * Add a test that host types are the same when guest types differ * Fix some typos * Thread things through a bit less * Undo CompileKey-related changes * Gate an async function on the async feature * Fix doc links * Skip resources tests in miri They all involve compilation which takes too long and doesn't currently work	1 year ago
Alex Crichton	6d7bb360bd	Dependency gardening for Wasmtime (#6731 ) * Remove deny.toml exception for wasm-coredump-builder This isn't used any more so no need to continue to list this. * Update Wasmtime's pretty_env_logger dependency This removes a `deny.toml` exception for that crate, but `openvino-sys` still depends on `pretty_env_logger 0.4.0` so a new exception is added for that. * Update criterion and clap dependencies This commit started out by updating the `criterion` dependency to remove an entry in `deny.toml`, but that ended up transitively requiring a `clap` dependency upgrade from 3.x to 4.x because `criterion` uses pieces of clap 4.x. Most of this commit is then dedicated to updating clap 3.x to 4.x which was relatively simple, mostly renaming attributes here and there. * Update gimli-related dependencies I originally wanted to remove the `indexmap` clause in `deny.toml` but enough dependencies haven't updated from 1.9 to 2.0 that it wasn't possible. In the meantime though this updates some various dependencies to bring them to the latest and a few of them now use `indexmap` 2.0. * Update deps to remove `windows-sys 0.45.0` This involved updating tokio/mio and then providing new audits for new crates. The tokio exemption was updated from its old version to the new version and tokio remains un-audited. * Update `syn` to 2.x.x This required a bit of rewriting for the component-macro related bits but otherwise was pretty straightforward. The `syn` 1.x.x track is still present in the wasi-crypto tree at this time. I've additionally added some trusted audits for my own publications of `wasm-bindgen` * Update bitflags to 2.x.x This updates Wasmtime's dependency on the `bitflags` crate to the 2.x.x track to keep it up-to-date. * Update the cap-std family of crates This bumps them all to the next major version to keep up with updates. I've additionally added trusted entries for publishes of cap-std crates from Dan. There's still lingering references to rustix 0.37.x which will need to get weeded out over time. * Update memoffset dependency to latest Avoids having two versions in our crate graph. * Fix tests * Update try_from for wiggle flags * Fix build on AArch64 Linux * Enable `event` for rustix on Windows too	1 year ago
Alex Crichton	80e68c336b	Update the wasm-tools family of crates (#6710 ) * Update wasm-tools dependencies * Get tests passing after wasm-tools update Mostly dealing with updates to `wasmparser`'s API. * Update `cargo vet` for new crates * Add `equivalent`, `hashbrown`, and `quote` to the list of trusted authors. We already trust these authors for other crates. * Pull in some upstream audits for various deps. * I've audited the `pulldown-cmark` dependency upgrade myself.	1 year ago
Alex Crichton	c34567cddd	Fix some warnings on nightly Rust (#6702 ) * Fix some warnings on nightly Rust * One is to avoid using `drop(&mut T)` * Another is to avoid using `()` in FFI functions since apparently that's not recommended. Working around this in the macro was pretty tricky, however. * Fix a typo * Review feedback	1 year ago
Adam Petro	ca90650c12	Support multiple versions of `wasmtime` in the same crate (#6673 ) * Attempt versioned exports to facilitate having multiple versions in the same crate * Modify approach to use `export_name` and `link_name` * Only apply version to names in assembly and foreign item fns * Attempt to handle the s390x case * Fix alignment of backslashes in assembly file * Pretend I understand the preprocessor * Version symbols in `crates/runtime/src/helpers.c` * Stop versioning `__jit_debug_register_code` because gdb relies on it and it is uses `weak` linkage * Version symbol in `crates/fiber/src/windows.c` * Consolidate `LitStr` creation in macro * Add new crate to publish script and supply-chain config * Fix order in supply chain config * Set `audit-as-crates-io` to false * Missing `versioned_link` for Windows * Version strings used in debug * Formatting * Get rid of `versioned_str` and bring back `versioned_suffix` --------- Co-authored-by: Alex Crichton <alex@alexcrichton.com>	1 year ago
Alex Crichton	e8f4f86209	Shepherd along type information in lifting/lowering (#6667 ) * Shepherd along type information in lifting/lowering This commit is a large refactor to the component runtime of Wasmtime to shepherd along type information when lifting and lowering values. Previously lifting and lowering would assume type information given context such as "surely lowering a `u32` must lower into the type `InterfaceType::U32`" or "lowering a `Val` works as it knows its own type". This is still true, and this commit isn't changing these features. The rationale for this commit instead stems from the upcoming implementation of resources in Wasmtime. Resources are trickier than all existing types in Wasmtime because what exactly is the type of a resource depends on who you're asking. For example the host might have one type called `http::Headers` but a component could import it as two distinct types: (component (import "headers1" (type $h1 (sub resource))) (import "headers2" (type $h2 (sub resource))) ;; ... ) in the above component the `$h1` and `$h2` types will each get their own table at runtime for managing their state. This means that if the host instantiates the component with `http::Headers` supplied as both types then the same type on the outside maps to two different types inside. This means that the lowering of a host-defined type into the component is now dependent on the "name" that the component has for the type, basically if the function used `$h1` or `$h2`. This overall means that the type that the component assigned for a function is significant as part of lifting and lowering. Hence the rationale for this commit, threading around this type information. The major change in this commit is updates to the `Lift` and `Lower` traits. Previously they took a mishmash of parameters and now they needed to take more parameters, so I've updated them with: * `Lift` operations take a `&LiftContext<'_>` and an `InterfaceType` as contextual information. The context stores the store, the options, and type information. The `InterfaceType` is the type that's being lifted, which would indicate which resource table to load from for example. * `Lower` operations now take a `&mut LowerContext<'_, T>` and an `InterfaceType`. The `LowerContext` is similar to its lift cousin where it stores the store, options, and type information. The different context passed in to `lift` and `load`, for example, is no longer distinguished and both simply take a `&LiftContext<'_>` which simplifies things a bit. This refactoring was pretty far reaching and touches quite a bit of the component model implementation. This is because basically everything deals with type information as types can be recursively nested in one another. I've taken the liberty to make code continue to be ergonomic/understandable where appropriate so some "shapes" of code are now different to continue to accommodate readability and maintainability. Finally it's worth noting that this should not have any actual function impact on components running today (or tomorrow). User-facing APIs haven't changed at all here and it's just the guts that are refactored. One unfortunate aspect, though, is that this is going to be a small perf hit on lifting/lowering due to the fact that type information essentially needs to be "iterated" over during the lifting/lowering process. This iteration involves index lookups in `&ComponentTypes` along with assertions that when you lower `Vec<T>` that the type is `InterfaceType::List(i)`. These assertions should always succeed, and in theory could become some sort of `unreachable_unchecked` in the future, but for now it's all left as safe checks/panics for us to optimize at a later date if necessary. * Fill out a TODO comment * Fill out more comments	1 year ago
Alex Crichton	3dfbfb61a9	x64: Add non-SSSE3 lowerings of `pshufb` (#6606 ) * x64: Add non-SSSE3 lowerings of `pshufb` Or, more accurately, add lowerings which don't use `pshufb`'s functionality at all where possible or otherwise fall back to a new libcall. This particular instruction seemed uniquely difficult to implement in the backend so I decided to "cop out" and use libcall instead. The libcall will be used for `popcnt`, `shuffle`, and `swizzle` instructions when SSSE3 isn't available. * Implemente SSE2 popcnt with Hacker's Delight * x64: Implement passing vector arguments in the fastcall convention Windows says that vector arguments are passed indirectly so handle that here through the `ABIArg::ImplicitPtrArg` variant. Some additional handling is added to the general machinst backend. * Update `gen_load_base_offset` for x64 * Fill out remaining bits of fastcall and vector parameters * Remove now-unnecessary `Clone` bound	1 year ago
Alex Crichton	550a16f539	Fix a soundness issue with the component model and async (#6509 ) * Force `execute_across_threads` to use multiple threads Currently this uses tokio's `spawn_blocking` but that will reuse threads in its thread pool. Instead spawn a thread and perform a single poll on that thread to force lots of fresh threads to be used and ideally stress TLS management further. * Add a guard against using stale stacks This commit adds a guard to Wasmtime's async support to double-check that when a call to `poll` is finished that the currently active TLS activation pointer does not point to the stack that is being switched off of. This is attempting to be a bit of a defense-in-depth measure to prevent stale pointers from sticking around in TLS. This is currently happening and causing #6493 which can result in unsoundness but currently is manifesting as a crash. * Fix a soundness issue with the component model and async This commit addresses #6493 by fixing a soundness issue with the async implementation of the component model. This issue has been presence since the inception of the addition of async support to the component model and doesn't represent a recent regression. The underlying problem is that one of the base assumptions of the trap handling code is that there's only one single activation in TLS that needs to be pushed/popped when a stack is switched (e.g. a fiber is switched to or from). In the case of the component model there might be two activations: one for an invocation of a component function and then a second for an invocation of a `realloc` function to return results back to wasm (e.g. in the case an imported function returns a list). This problem is fixed by changing how TLS is managed in the presence of fibers. Previously when a fiber was suspended it would pop a single activation from the top of the stack and save that to get pushed when the fiber was resumed. This has the benefit of maintaining an entire linked list of activations for the current thread but has the problem above where it doesn't handle a fiber with multiple activations on it. Instead now TLS management is done when a fiber is resumed instead of suspended. Instead of pushing/popping a single activation the entire linked list of activations is tracked for a particular fiber and stored within the fiber itself. In this manner resuming a fiber will push all activations onto the current thread and suspending a fiber will pop all activations for the fiber (and store them as a new linked list in the fiber's state itself). This end result is that all activations on a fiber should now be managed correctly, regardless of how many there are. The main downside of this commit is that fiber suspension and resumption is more complicated, but the hope there is that fiber suspension typically corresponds with I/O not being ready or similar so the order of magnitude of TLS operations isn't too significant compared to the I/O overhead. Closes #6493 * Review comments * Fix restoration during panic	1 year ago
Luna P-C	92024ad117	Function references (#5288 ) * Make wasmtime-types type check * Make wasmtime-environ type check. * Make wasmtime-runtime type check * Make cranelift-wasm type check * Make wasmtime-cranelift type check * Make wasmtime type check * Make wasmtime-wast type check * Make testsuite compile * Address Luna's comments * Restore compatibility with effect-handlers/wasm-tools#func-ref-2 * Add function refs feature flag; support testing * Provide function references support in helpers - Always support Index in blocktypes - Support Index as table type by pretending to be Func - Etc * Implement ref.as_non_null * Add br_on_null * Update Cargo.lock to use wasm-tools with peek This will ultimately be reverted when we refer to wasm-tools#function-references, which doesn't have peek, but does have type annotations on CallRef * Add call_ref * Support typed function references in ref.null * Implement br_on_non_null * Remove extraneous flag; default func refs false * Use IndirectCallToNull trap code for call_ref * Factor common call_indirect / call_ref into a fn * Remove copypasta clippy attribute / format * Add a some more tests for typed table instructions There certainly need to be many more, but this at least catches the bugs fixed in the next commit * Fix missing typed cases for table_grow, table_fill * Document trap code; remove answered question * Mark wasm-tools to wasmtime reftype infallible * Fix reversed conditional * Scope externref/funcref shorthands within WasmRefType * Merge with upstream * Make wasmtime compile again * Fix warnings * Remove Bot from the type algebra * Fix table tests. `wast::Cranelift::spec::function_references::table` `wast::Cranelift::spec::function_references::table_pooling` * Fix table{get,set} tests. ``` wast::Cranelift::misc::function_references::table_get wast::Cranelift::misc::function_references::table_get_pooling wast::Cranelift::misc::function_references::table_set wast::Cranelift::misc::function_references::table_set_pooling ``` * Insert subtype check to fix local_get tests. ``` wast::Cranelift::spec::function_references::local_get wast::Cranelift::spec::function_references::local_get_pooling ``` * Fix compilation of `br_on_non_null`. The branch destinations were the other way round... :-) Fixes the following test failures: ``` wast::Cranelift::spec::function_references::br_on_non_null wast::Cranelift::spec::function_references::br_on_non_null_pooling ``` * Fix ref_as_non_null tests. The test was failing due to the wrong error message being printed. As per upstream folks' suggest we were using the trap code `IndirectCallToNull`, but it produces an unexpected error message. This commit reinstates the `NullReference` trap code. It produces the expected error message. We will have to chat with the maintainers upstream about how to handle these "test failures". Fixes the following test failures: ``` wast::Cranelift::spec::function_references::ref_as_non_null wast::Cranelift::spec::function_references::ref_as_non_null_pooling ``` * Fix a call_ref regression. * Fix global tests. Extend `is_matching_assert_invalid_error_message` to circumvent the textual error message failure. Fixes the following test failures: ``` wast::Cranelift::spec::function_references::global wast::Cranelift::spec::function_references::global_pooling ``` * Cargo update * Update * Spell out some cases in match_val * Disgusting hack to subvert limitations of type reconstruction. In the function `wasmtime::values::Val::ty()` attempts to reconstruct the type of its underlying value purely based on the shape of the value. With function references proposal this sort of reconstruction is no longer complete as a source reference type may have been nullable. Nullability is not inferrable by looking at the shape of the runtime object alone. Consequently, the runtime cannot reconstruct the type for `Val::FuncRef` and `Val::ExternRef` by looking at their respective shapes. * Address workflows comments. * null reference => null_reference for CLIF parsing compliance. * Delete duplicate-loads-dynamic-memory-egraph (again) * Idiomatic code change. * Nullability subtyping + fix non-null storage check. This commit removes the `hacky_eq` check in `func.rs`. Instead it is replaced by a subtype check. This subtype check occurs in `externals.rs` too. This commit also fixes a bug. Previously, it was possible to store a null reference into a non-null table cell. I have added to new test cases for this bug: one for funcrefs and another for externrefs. * Trigger unimplemented for typed function references. Format values.rs * run cargo fmt * Explicitly match on HeapType::Extern. * Address cranelift-related feedback * Remove PartialEq,Eq from ValType, RefType, HeapType. * Pin wasmparser to a fairly recent commit. * Run cargo fmt * Ignore tail call tests. * Remove garbage * Revert changes to wasmtime public API. * Run cargo fmt * Get more CI passing (#19) * Undo Cargo.lock changes * Fix build of cranelift tests * Implement link-time matches relation. Disable tests failing due to lack of public API support. * Run cargo fmt * Run cargo fmt * Initial implementation of eager table initialization * Tidy up eager table initialisation * Cargo fmt * Ignore type-equivalence test * Replace TODOs with descriptive comments. * Various changes found during review (#21) * Clarify a comment This isn't only used for null references * Resolve a TODO in local init Don't initialize non-nullable locals to null, instead skip initialization entirely and wasm validation will ensure it's always initialized in the scope where it's used. * Clarify a comment and skipping the null check. * Remove a stray comment * Change representation of `WasmHeapType` Use a `SignatureIndex` instead of a `u32` which while not 100% correct should be more correct. This additionally renames the `Index` variant to `TypedFunc` to leave space for future types which aren't functions to not all go into an `Index` variant. This required updates to Winch because `wasmtime_environ` types can no longer be converted back to their `wasmparser` equivalents. Additionally this means that all type translation needs to go through some form of context to resolve indices which is now encapsulated in a `TypeConvert` trait implemented in various locations. * Refactor table initialization Reduce some duplication and simplify some data structures to have a more direct form of table initialization and a bit more graceful handling of element-initialized tables. Additionally element-initialize tables are now treated the same as if there's a large element segment initializing them. * Clean up some unrelated chagnes * Simplify Table bindings slightly * Remove a no-longer-needed TODO * Add a FIXME for `SignatureIndex` in `WasmHeapType` * Add a FIXME for panicking on exposing function-references types * Fix a warning on nightly * Fix tests for winch and cranelift * Cargo fmt * Fix arity mismatch in aarch64/abi --------- Co-authored-by: Daniel Hillerström <daniel.hillerstrom@ed.ac.uk> Co-authored-by: Daniel Hillerström <daniel.hillerstrom@huawei.com> Co-authored-by: Alex Crichton <alex@alexcrichton.com>	1 year ago
Pat Hickey	338b535a3a	Refactor test-programs to build modules and components (#6385 ) * wasi-tests and wasi-http-tests no longer have their own workspace * wasi-tests: fix warnings * rewrite the test-programs build.rs to generate {package}_modules.rs and _components.rs The style is cribbed from preview2-prototying repo, but I ended up refactoring it a bit. * better escaping should help with windows? * long form cap-std-sync and tokio test suites * convert wasi-http test * fixes, comments * apply cargo fmt to whole workspace * bump test-programs and wasi-http-tests to all use common dependency versions wit-bindgen 0.6.0 and wit-component 0.7.4 * add new audits * cargo vet prune * package and supply chain updates to fix vulnerabilities h2 upgraded from 0.3.16 -> 0.3.19 to fix vulnerability tempfile upgraded from 0.3.3 -> 0.3.5 to eliminate dep on vulnerable remove_dir_all * deny: temporarily allow duplicate wasm-encoder, wasmparser, wit-parser prtest:full * convert more dependencies to { workspace = true } Alex asked me to do thsi for wit-component and wit-bindgen, and I found a few more (cfg-if, tempfile, filecheck, anyhow... I also reorganized the workspace dependencies section to make the ones our team maintains more clearly separated from our external dependencies. * test-programs build: ensure that the user writes a #[test] for each module, component * fix build of wasi-tests on windows * misspelled macos * mark wasi-tests crate test=false so we dont try building it natively... * mark wasi-http-tests test=false as well * try getting the cargo keys right * just exclude wasi-tests and wasi-http-tests in run-tests.sh * interesting paths fails on windows * misspelling so nice i did it twice * new cargo deny exception: ignore all of wit-bindgen's dependencies * auto-import wildcard vets	2 years ago
Alex Crichton	7f0228c967	Fix some warnings on nightly Rust (#6388 ) Upstream rust has decided that ignoring a value is not spelled `drop(foo)` but instead it's spelled `let _ = foo`	2 years ago
Alex Crichton	614665688d	Make libcalls slightly safer (#6360 ) Bake in the `*mut VMContext` to `&mut Instance` translation into the macro-generated trampolines to avoid the need to use `Instance::from_vmctx` with an extra level of indentation everywhere. Additionally some libcalls are now entirely safe code as their one unsafe operation was the `VMContext` to `Instance` translation.	2 years ago
Alex Crichton	ec92f8e480	Make Wasmtime compatible with Stacked Borrows in MIRI (#6338 ) * Make Wasmtime compatible with Stacked Borrows in MIRI The fact that Wasmtime executes correctly under Tree Borrows but not Stacked Borrows is a bit suspect and given what I've since learned about the aliasing models I wanted to give it a stab to get things working with Stacked Borrows. It turns out that this wasn't all that difficult, but required two underlying changes: * First the implementation of `Instance::vmctx` is now specially crafted in an intentional way to preserve the provenance of the returned pointer. This way all `&Instance` pointers will return a `VMContext` pointer with the same provenance and acquiring the pointer won't accidentally invalidate all prior pointers. * Second the conversion from `VMContext` to `Instance` has been updated to work with provenance and such. Previously the conversion looked like `&mut VMContext -> &mut Instance`, but I think this didn't play well with MIRI because `&mut VMContext` has no provenance over any data since it's zero-sized. Instead now the conversion is from `mut VMContext` to `&mut Instance` where we know that `mut VMContext` has provenance over the entire instance allocation. This shuffled a fair bit around to handle the new closure-based API to prevent escaping pointers, but otherwise no major change other than the structure and the types in play. This commit additionally picks up a dependency on the `sptr` crate which is a crate for prototyping strict-provenance APIs in Rust. This is I believe intended to be upstreamed into Rust one day (it's in the standard library as a Nightly-only API right now) but in the meantime this is a stable alternative. * Clean up manual `unsafe impl Send` impls This commit adds a new wrapper type `SendSyncPtr<T>` which automatically impls the `Send` and `Sync` traits based on the `T` type contained. Otherwise it works similarly to `NonNull<T>`. This helps clean up a number of manual annotations of `unsafe impl {Send,Sync} for ...` throughout the runtime. * Remove pointer-to-integer casts with tables In an effort to enable MIRI's "strict provenance" mode this commit removes the integer-to-pointer casts in the runtime `Table` implementation for Wasmtime. Most of the bits were already there to track all this, so this commit plumbed around the various pointer types and with the help of the `sptr` crate preserves the provenance of all related pointers. * Remove integer-to-pointer casts in CoW management The `MemoryImageSlot` type stored a `base: usize` field mostly because I was too lazy to have a `Send`/`Sync` type as a pointer, so this commit updates it to use `SendSyncPtr<u8>` and then plumbs the pointer-ness throughout the implementation. This removes all integer-to-pointer casts and has pointers stores as actual pointers when they're at rest. * Remove pointer-to-integer casts in "raw" representations This commit changes the "raw" representation of `Func` and `ExternRef` to a `mut c_void` instead of the previous `usize`. This is done to satisfy MIRI's requirements with strict provenance, properly marking the intermediate value as a pointer rather than round-tripping through integers. Minor remaining cleanups * Switch to Stacked Borrows for MIRI on CI Additionally enable the strict-provenance features to force warnings emitted today to become errors. * Fix a typo * Replace a negative offset with `sub` * Comment the sentinel value * Use NonNull::dangling	2 years ago
Alex Crichton	d9cad60be9	Remove the initializer from a global's type information (#6349 ) * Remove the initializer from a global's type information This commit removes the `Global::initializer` field to instead only store type information about the global rather than its initialization state. Instead now the initializer is stored separately from the type of the global, and only for defined globals. This removes the need in a few locations to synthesize dummy initializers. * Actually delete what I intended to delete * Simplify global initializer loop	2 years ago
Dante Broggi	4b3c50b147	Fix typo 'oeprations' ⇒ 'operations' (#6339 ) Fix typo in crates/runtime/src/mmap/miri.rs on line 4 'oeprations' ⇒ 'operations'.	2 years ago
Alex Crichton	c0bb341d95	Run some tests in MIRI on CI (#6332 ) * Run some tests in MIRI on CI This commit is an implementation of getting at least chunks of Wasmtime to run in MIRI on CI. The full test suite is not possible to run in MIRI because MIRI cannot run Cranelift-produced code at runtime (aka it doesn't support JITs). Running MIRI, however, is still quite valuable if we can manage it because it would have trivially detected GHSA-ch89-5g45-qwc7, our most recent security advisory. The goal of this PR is to select a subset of the test suite to execute in CI under MIRI and boost our confidence in the copious amount of `unsafe` code in Wasmtime's runtime. Under MIRI's default settings, which is to use the [Stacked Borrows][stacked] model, much of the code in `Instance` and `VMContext` is considered invalid. Under the optional [Tree Borrows][tree] model, however, this same code is accepted. After some [extremely helpful discussion][discuss] on the Rust Zulip my current conclusion is that what we're doing is not fundamentally un-sound but we need to model it in a different way. This PR, however, uses the Tree Borrows model for MIRI to get something onto CI sooner rather than later, and I hope to follow this up with something that passed Stacked Borrows. Additionally that'll hopefully make this diff smaller and easier to digest. Given all that, the end result of this PR is to get 131 separate unit tests executing on CI. These unit tests largely exercise the embedding API where wasm function compilation is not involved. Some tests compile wasm functions but don't run them, but compiling wasm through Cranelift in MIRI is so slow that it doesn't seem worth it at this time. This does mean that there's a pretty big hole in MIRI's test coverage, but that's to be expected as we're a JIT compiler after all. To get tests working in MIRI this PR uses a number of strategies: * When platform-specific code is involved there's now `#[cfg(miri)]` for MIRI's version. For example there's a custom-built "mmap" just for MIRI now. Many of these are simple noops, some are `unimplemented!()` as they shouldn't be reached, and some are slightly nontrivial implementations such as mmaps and trap handling (for native-to-native function calls). * Many test modules are simply excluded via `#![cfg(not(miri))]` at the top of the file. This excludes the entire module's worth of tests from MIRI. Other modules have `#[cfg_attr(miri, ignore)]` annotations to ignore tests by default on MIRI. The latter form is used in modules where some tests work and some don't. This means that all future test additions will need to be effectively annotated whether they work in MIRI or not. My hope though is that there's enough precedent in the test suite of what to do to not cause too much burden. * A number of locations are fixed with respect to MIRI's analysis. For example `ComponentInstance`, the component equivalent of `wasmtime_runtime::Instance`, was actually left out from the fix for the CVE by accident. MIRI dutifully highlighted the issues here and I've fixed them locally. Some locations fixed for MIRI are changed to something that looks similar but is subtly different. For example retaining items in a `Store<T>` is now done with a Wasmtime-specific `StoreBox<T>` type. This is because, with MIRI's analyses, moving a `Box<T>` invalidates all pointers derived from this `Box<T>`. We don't want these semantics, so we effectively have a custom `Box<T>` to suit our needs in this regard. * Some default configuration is different under MIRI. For example most linear memories are dynamic with no guards and no space reserved for growth. Settings such as parallel compilation are disabled. These are applied to make MIRI "work by default" in more places ideally. Some tests which perform N iterations of something perform fewer iterations on MIRI to not take quite so long. This PR is not intended to be a one-and-done-we-never-look-at-it-again kind of thing. Instead this is intended to lay the groundwork to continuously run MIRI in CI to catch any soundness issues. This feels, to me, overdue given the amount of `unsafe` code inside of Wasmtime. My hope is that over time we can figure out how to run Wasm in MIRI but that may take quite some time. Regardless this will be adding nontrivial maintenance work to contributors to Wasmtime. MIRI will be run on CI for merges, MIRI will have test failures when everything else passes, MIRI's errors will need to be deciphered by those who have probably never run MIRI before, things like that. Despite all this to me it seems worth the cost at this time. Just getting this running caught two possible soundness bugs in the component implementation that could have had a real-world impact in the future! [stacked]: https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md [tree]: https://perso.crans.org/vanille/treebor/ [discuss]: https://rust-lang.zulipchat.com/#narrow/stream/269128-miri/topic/Tree.20vs.20Stacked.20Borrows.20.26.20a.20debugging.20question * Update alignment comment	2 years ago
Nick Fitzgerald	60ce6f5d5c	`wasmtime`: Fix resetting stack-walking registers when entering/exiting Wasm (#6321 ) * wasmtime: Fix resetting stack-walking registers when entering/exiting Wasm Fixes a regression from #6262, originally reported in https://github.com/bytecodealliance/wasmtime-dotnet/pull/245 The issue was that we would enter Wasm and save the stack-walking registers but never clear them after Wasm returns. Then if a host-to-host call tried to capture a stack, we would mistakenly attempt to use those stale registers to start the stack walk. This mistake would be caught by an assertion, triggering a panic. This commit fixes the issue by managing the save/restore in the `CallThreadState` construction/drop, rather than in the old `set_prev` method. Co-Authored-By: Alex Crichton <alex@alexcrichton.com> * Plumb through `VMRuntimeLimits` when capturing stack traces This way we can differentiate between the same module loaded in different stores and avoid leaking other stores' frames into our backtraces. Co-Authored-By: Jamey Sharp <jsharp@fastly.com> --------- Co-authored-by: Alex Crichton <alex@alexcrichton.com> Co-authored-by: Jamey Sharp <jsharp@fastly.com>	2 years ago
Jamey Sharp	fea6ab9a2b	wasmtime: In-process sampling profiler (#6282 ) * wasmtime: In-process sampling profiler Unlike the existing profiling options, this works on all platforms and does not rely on any external profiling tools like perf or VTune. On the other hand, it can only profile time spent in the WebAssembly guest, not in Wasmtime itself or other host code. Also it can't measure time as precisely as platform-native tools can. The profile is saved in the Firefox processed format, which can be viewed using https://profiler.firefox.com/. * Ensure func_offset is populated * Refactor * Review comments * Move GuestProfiler to the wasmtime crate * Document the new GuestProfiler API * Add TODO comments for future work * Use module_offset, not func_offset, as fallback PC * Minimize work done during `sample()` Use fxprof_processed_profile's support for looking up symbols to avoid looking up the same PC more than once per profile. * Keep profiler state in the store Also extend the documentation based on review comments. * Import debugid audit from Mozilla again	2 years ago
Alex Crichton	a7b58c41cb	Split out platform-specific logic for `Mmap` (#6318 ) * Split out platform-specific logic for `Mmap` This commit refactors the implementation of the `wasmtime_runtime::Mmap` structure to have the platform-specific bits separated by file rather than interspersed throughout `mmap.rs`. I plan in the near future to add a faux implementation for `cfg(miri)` to get some tests running with miri on CI. At the same time this additionally updates the interface of `Mmap` to be more miri-friendly in the sense of ensuring that mutability is all in the right place and we don't eagerly mark items as safe too soon. For example it seems questionable that previously you could get a mutable slice to readonly memory. Probably not going to cause any issues, but this interface should hopefully be more verification-friendly. * Fix tests on Windows	2 years ago
Nick Fitzgerald	b453c70d7a	wasmtime-runtime: Rename `VMCallerCheckedFuncRef` to `VMFuncRef` (#6299 ) We only have caller-checked function references, and it is unlikely we will have any other kind for quite a long time. Also remove all old `anyfunc`s and replace them with some variation of `func_ref`. Also consolidate style to use `func_ref` instead of `funcref` except when that would be a breaking change to the public API or a comment is using the `funcref` shorthand from WAT.	2 years ago
Nick Fitzgerald	913efdf24d	wasmtime: Overhaul trampolines (#6262 ) This commit splits `VMCallerCheckedFuncRef::func_ptr` into three new function pointers: `VMCallerCheckedFuncRef::{wasm,array,native}_call`. Each one has a dedicated calling convention, so callers just choose the version that works for them. This is as opposed to the previous behavior where we would chain together many trampolines that converted between calling conventions, sometimes up to four on the way into Wasm and four more on the way back out. See [0] for details. [0] https://github.com/bytecodealliance/rfcs/blob/main/accepted/tail-calls.md#a-review-of-our-existing-trampolines-calling-conventions-and-call-paths Thanks to @bjorn3 for the initial idea of having multiple function pointers for different calling conventions. This is generally a nice ~5-10% speed up to our call benchmarks across the board: both Wasm-to-host and host-to-Wasm. The one exception is typed calls from Wasm to the host, which have a minor regression. We hypothesize that this is because the old hand-written assembly trampolines did not maintain a call frame and do a tail call, but the new Cranelift-generated trampolines do maintain a call frame and do a regular call. The regression is only a couple nanoseconds, which seems well-explained by these differences explain, and ultimately is not a big deal. However, this does lead to a ~5% code size regression for compiled modules. Before, we compiled a trampoline per escaping function's signature and we deduplicated these trampolines by signature. Now we compile two trampolines per escaping function: one for if the host calls via the array calling convention and one for it the host calls via the native calling convention. Additionally, we compile a trampoline for every type in the module, in case there is a native calling convention function from the host that we `call_indirect` of that type. Much of this is in the `.eh_frame` section in the compiled module, because each of our trampolines needs an entry there. Note that the `.eh_frame` section is not required for Wasmtime's correctness, and you can disable its generation to shrink compiled module code size; we just emit it to play nice with external unwinders and profilers. We believe there are code size gains available for follow up work to offset this code size regression in the future. Backing up a bit: the reason each Wasm module needs to provide these Wasm-to-native trampolines is because `wasmtime::Func::wrap` and friends allow embedders to create functions even when there is no compiler available, so they cannot bring their own trampoline. Instead the Wasm module has to supply it. This in turn means that we need to look up and patch in these Wasm-to-native trampolines during roughly instantiation time. But instantiation is super hot, and we don't want to add more passes over imports or any extra work on this path. So we integrate with `wasmtime::InstancePre` to patch these trampolines in ahead of time. Co-Authored-By: Jamey Sharp <jsharp@fastly.com> Co-Authored-By: Alex Crichton <alex@alexcrichton.com> prtest:full	2 years ago
Alex Crichton	0977952dcd	Merge pull request from GHSA-ch89-5g45-qwc7 * Fix miscompile from functions mutating `VMContext` This commit fixes a miscompilation in Wasmtime on LLVM 16 where methods on `Instance` which mutated the state of the internal `VMContext` were optimized to not actually mutate the state. The root cause of this issue is a change in LLVM which takes advantage of `noalias readonly` pointers which is how `&self` methods are translated. This means that `Instance` methods which take `&self` but actually mutate the `VMContext` end up being undefined behavior from LLVM's point of view, meaning that the writes are candidate for removal. The fix applied here is intended to be a temporary one while a more formal fix, ideally backed by `cargo miri` verification, is implemented on `main`. The fix here is to change the return value of `vmctx_plus_offset` to return `const T` instead of `mut T`. This caused lots of portions of the runtime code to stop compiling because mutations were indeed happening. To cover these a new `vmctx_plus_offset_mut` method was added which notably takes `&mut self` instead of `&self`. This forced all callers which may mutate to reflect the `&mut self` requirement, propagating that outwards. This fixes the miscompilation with LLVM 16 in the immediate future and should be at least a meager line of defense against issues like this in the future. This is not a long-term fix, though, since `cargo miri` still does not like what's being done in `Instance` and with `VMContext`. That fix is likely to be more invasive, though, so it's being deferred to later. * Update release notes * Fix dates and fill out more notes	2 years ago
Dan Gohman	ec6755512f	Remove `maintenance` badges from the Cargo.toml files. (#6286 ) Several of these badges were out of date, with some crates in wide production use marked as "experimental". Insted of trying to keep them up to date, just remove them, since they are [no longer displayed on crates.io]. [no longer displayed on crates.io]: https://doc.rust-lang.org/cargo/reference/manifest.html#the-badges-section	2 years ago
Jamey Sharp	bd473dba31	wasmtime-runtime: Re-export backtrace `Frame` type (#6280 ) This crate re-exports the `Backtrace` type at top-level from a nested module. `Backtrace` in turn has `Frame` in its public API, which is not re-exported anywhere. This is legal and external users can call methods on `Frame`, but it doesn't appear in the rustdocs, making it unnecessarily difficult to figure out how to use this API. Re-exporting `Frame` fixes that, and also allows naming the type directly if needed.	2 years ago
Dan Gohman	c59bb8db39	Update several dependencies. (#6171 ) This updates to rustix 0.37.13, which contains some features we can use to implement more features in wasi-common for the wasi-sockets API. This also pulls in several other updates to avoid having multiple versions of rustix. This does introduce multiple versions of windows-sys, as the errno and tokio crates are currently using 0.45 while rustix and other dependencies have updated to 0.48; PRs updating these are already in flight so this will hopefully be resolved soon. It also includes cap-std 1.0.14, which disables the use of `openat2` and `statx` on Android, fixing a bug where some Android devices crash the process when those syscalls are executed.	2 years ago
Alex Crichton	299131ae2d	Gracefully handle ENOTSUP for `memfd_create` (#6230 ) * Gracefully handle ENOTSUP for `memfd_create` Looks like in #6229 older kernels may not support memfd in which case we can fall back to manual memory initialization gracefully. Closes #6229 * Fix tests	2 years ago
Alex Crichton	dbd000c1ce	Change `asm` to `__asm__` in `helpers.c` (#6188 ) This is an attempt to fix #6177 since according to [this reference][1] some modes of compilation require `__asm__` instead of `asm`. [1]: https://en.cppreference.com/w/c/language/asm	2 years ago
Nick Fitzgerald	2e48babf23	cranelift-wasm: Add a bounds-checking optimization for dynamic memories and guard pages (#6031 ) * cranelift-wasm: Add a bounds-checking optimization for dynamic memories and guard pages This is a new special case for when we know that there are enough guard pages to cover the memory access's offset and access size. The precise should-we-trap condition is index + offset + access_size > bound However, if we instead check only the partial condition index > bound then the most out of bounds that the access can be, while that partial check still succeeds, is `offset + access_size`. However, when we have a guard region that is at least as large as `offset + access_size`, we can rely on the virtual memory subsystem handling these out-of-bounds errors at runtime. Therefore, the partial `index > bound` check is sufficient for this heap configuration. Additionally, this has the advantage that a series of Wasm loads that use the same dynamic index operand but different static offset immediates -- which is a common code pattern when accessing multiple fields in the same struct that is in linear memory -- will all emit the same `index > bound` check, which we can GVN. * cranelift: Add WAT tests for accessing dynamic memories with the same index but different offsets The bounds check comparison is GVN'd but we still branch on values we should know will always be true if we get this far in the code. This is actual `br_if`s in the non-Spectre code and `select_spectre_guard`s that we should know will always go a certain way if we have Spectre mitigations enabled. Improving the non-Spectre case is pretty straightforward: walk the dominator tree and remember which values we've already branched on at this point, and therefore we can simplify any further conditional branches on those same values into direct jumps. Improving the Spectre case requires something that is morally the same, but has a few snags: * We don't have actual `br_if`s to determine whether the bounds checking condition succeeded or not. We need to instead reason about dominating `select_spectre_guard; {load, store}` instruction pairs. * We have to be SUPER careful about reasoning "through" `select_spectre_guard`s. Our general rule is never to do that, since it could break the speculative execution sandboxing that the instruction is designed for.	2 years ago
Alex Crichton	28371bfd40	Validate faulting addresses are valid to fault on (#6028 ) * Validate faulting addresses are valid to fault on This commit adds a defense-in-depth measure to Wasmtime which is intended to mitigate the impact of CVEs such as GHSA-ff4p-7xrq-q5r8. Currently Wasmtime will catch `SIGSEGV` signals for WebAssembly code so long as the instruction which faulted is an allow-listed instruction (aka has a trap code listed for it). With the recent security issue, however, the problem was that a wasm guest could exploit a compiler bug to access memory outside of its sandbox. If the access was successful there's no real way to detect that, but if the access was unsuccessful then Wasmtime would happily swallow the `SIGSEGV` and report a nominal trap. To embedders, this might look like nothing is going awry. The new strategy implemented here in this commit is to attempt to be more robust towards these sorts of failures. When a `SIGSEGV` is raised the faulting pc is recorded but additionally the address of the inaccessible location is also record. After the WebAssembly stack is unwound and control returns to Wasmtime which has access to a `Store` Wasmtime will now use this inaccessible faulting address to translate it to a wasm address. This process should be guaranteed to succeed as WebAssembly should only be able to access a well-defined region of memory for all linear memories in a `Store`. If no linear memory in a `Store` could contain the faulting address, then Wasmtime now prints a scary message and aborts the process. The purpose of this is to catch these sorts of bugs, make them very loud errors, and hopefully mitigate impact. This would continue to not mitigate the impact of a guest successfully loading data outside of its sandbox, but if a guest was doing a sort of probing strategy trying to find valid addresses then any invalid access would turn into a process crash which would immediately be noticed by embedders. While I was here I went ahead and additionally took a stab at #3120. Traps due to `SIGSEGV` will now report the size of linear memory and the address that was being accessed in addition to the bland "access out of bounds" error. While this is still somewhat bland in the context of a high level source language it's hopefully at least a little bit more actionable for some. I'll note though that this isn't a guaranteed contextual message since only the default configuration for Wasmtime generates `SIGSEGV` on out-of-bounds memory accesses. Dynamically bounds-checked configurations, for example, don't do this. Testing-wise I unfortunately am not aware of a great way to test this. The closet equivalent would be something like an `unsafe` method `Config::allow_wasm_sandbox_escape`. In lieu of adding tests, though, I can confirm that during development the crashing messages works just fine as it took awhile on macOS to figure out where the faulting address was recorded in the exception information which meant I had lots of instances of recording an address of a trap not accessible from wasm. * Fix tests * Review comments * Fix compile after refactor * Fix compile on macOS * Fix trap test for s390x s390x rounds faulting addresses to 4k boundaries.	2 years ago
Alex Crichton	5ae8575296	x64: Take SIGFPE signals for divide traps (#6026 ) * x64: Take SIGFPE signals for divide traps Prior to this commit Wasmtime would configure `avoid_div_traps=true` unconditionally for Cranelift. This, for the division-based instructions, would change emitted code to explicitly trap on trap conditions instead of letting the `div` x86 instruction trap. There's no specific reason for Wasmtime, however, to specifically avoid traps in the `div` instruction. This means that the extra generated branches on x86 aren't necessary since the `div` and `idiv` instructions already trap for similar conditions as wasm requires. This commit instead disables the `avoid_div_traps` setting for Wasmtime's usage of Cranelift. Subsequently the codegen rules were updated slightly: * When `avoid_div_traps=true`, traps are no longer emitted for `div` instructions. * The `udiv`/`urem` instructions now list their trap as divide-by-zero instead of integer overflow. * The lowering for `sdiv` was updated to still explicitly check for zero but the integer overflow case is deferred to the instruction itself. * The lowering of `srem` no longer checks for zero and the listed trap for the `div` instruction is a divide-by-zero. This means that the codegen for `udiv` and `urem` no longer have any branches. The codegen for `sdiv` removes one branch but keeps the zero-check to differentiate the two kinds of traps. The codegen for `srem` removes one branch but keeps the -1 check since the semantics of `srem` mismatch with the semantics of `idiv` with a -1 divisor (specifically for INT_MIN). This is unlikely to have really all that much of a speedup but was something I noticed during #6008 which seemed like it'd be good to clean up. Plus Wasmtime's signal handling was already set up to catch `SIGFPE`, it was just never firing. * Remove the `avoid_div_traps` cranelift setting With no known users currently removing this should be possible and helps simplify the x64 backend. * x64: GC more support for avoid_div_traps Remove the `validate_sdiv_divisor` pseudo-instructions and clean up some of the ISLE rules now that `div` is allowed to itself trap unconditionally. x64: Store div trap code in instruction itself * Keep divisors in registers, not in memory Don't accidentally fold multiple traps together * Handle EXC_ARITHMETIC on macos * Update emit tests * Update winch and tests	2 years ago
Alex Crichton	8bb183f16e	Implement the relaxed SIMD proposal (#5892 ) * Initial support for the Relaxed SIMD proposal This commit adds initial scaffolding and support for the Relaxed SIMD proposal for WebAssembly. Codegen support is supported on the x64 and AArch64 backends on this time. The purpose of this commit is to get all the boilerplate out of the way in terms of plumbing through a new feature, adding tests, etc. The tests are copied from the upstream repository at this time while the WebAssembly/testsuite repository hasn't been updated. A summary of changes made in this commit are: * Lowerings for all relaxed simd opcodes have been added, currently all exhibiting deterministic behavior. This means that few lowerings are optimal on the x86 backend, but on the AArch64 backend, for example, all lowerings should be optimal. * Support is added to codegen to, eventually, conditionally generate different code based on input codegen flags. This is intended to enable codegen to more efficient instructions on x86 by default, for example, while still allowing embedders to force architecture-independent semantics and behavior. One good example of this is the `f32x4.relaxed_fmadd` instruction which when deterministic forces the `fma` instruction, but otherwise if the backend doesn't have support for `fma` then intermediate operations are performed instead. * Lowerings of `iadd_pairwise` for `i16x8` and `i32x4` were added to the x86 backend as they're now exercised by the deterministic lowerings of relaxed simd instructions. * Sample codegen tests for added for x86 and aarch64 for some relaxed simd instructions. * Wasmtime embedder support for the relaxed-simd proposal and forcing determinism have been added to `Config` and the CLI. * Support has been added to the `.wast` runtime execution for the `(either ...)` matcher used in the relaxed-simd proposal. Tests for relaxed-simd are run both with a default `Engine` as well as a "force deterministic" `Engine` to test both configurations. * All tests from the upstream repository were copied into Wasmtime. These tests should be deleted when WebAssembly/testsuite is updated. * x64: Add x86-specific lowerings for relaxed simd This commit builds on the prior commit and adds an array of `x86_` instructions to Cranelift which have semantics that match their corresponding x86 equivalents. Translation for relaxed simd is then additionally updated to conditionally generate different CLIF for relaxed simd instructions depending on whether the target is x86 or not. This means that for AArch64 no changes are made but for x86 most relaxed instructions now lower to some x86-equivalent with slightly different semantics than the "deterministic" lowering. Add libcall support for fma to Wasmtime This will be required to implement the `f32x4.relaxed_madd` instruction (and others) when an x86 host doesn't specify the `has_fma` feature. * Ignore relaxed-simd tests on s390x and riscv64 * Enable relaxed-simd tests on s390x * Update cranelift/codegen/meta/src/shared/instructions.rs Co-authored-by: Andrew Brown <andrew.brown@intel.com> * Add a FIXME from review * Add notes about deterministic semantics * Don't default `has_native_fma` to `true` * Review comments and rebase fixes --------- Co-authored-by: Andrew Brown <andrew.brown@intel.com>	2 years ago
Alex Crichton	f91640ffab	Fix a panic due to a race in unpark and park (#5871 ) * Remove globals from parking spot tests Use `std:🧵:scope` to keep everything local to just the tests. * Fix a panic due to a race in `unpark` and `park` This commit fixes a panic in the `ParkingSpot` implementation where an `unpark` signal may not get acknowledged when a waiter times out, causing the waiter to remove itself from the internal map but panic thinking that it missed an unpark signal. The fix in this commit is to consume unpark signals when a timeout happens. This can lead to another possible race I've detailed in the comments which I believe is allowed by the specification of park/unpark in wasm. * Update crates/runtime/src/parking_spot.rs Co-authored-by: Andrew Brown <andrew.brown@intel.com> --------- Co-authored-by: Andrew Brown <andrew.brown@intel.com>	2 years ago
Alphyr	cb150d37ce	Update dependencies (#5513 )	2 years ago
Koute	e40a838beb	Prevent trampoline entrypoints from being stripped out during LTO (#5773 ) This works around a `rustc` bug where compiling with LTO will sometimes strip out some of the trampoline entrypoint symbols resulting in a linking failure.	2 years ago
Nick Fitzgerald	317cc51337	Rename `VMCallerCheckedAnyfunc` to `VMCallerCheckedFuncRef` (#5738 ) At some point what is now `funcref` was called `anyfunc` and the spec changed, but we didn't update our internal names. This does that. Co-authored-by: Jamey Sharp <jsharp@fastly.com>	2 years ago
Alex Crichton	91b8a2c527	Always allocate `Instance` memory with `malloc` (#5656 ) This commit removes the pooling of `Instance` allocations from the pooling instance allocator. This means that the allocation of `Instance` (and `VMContext`) memory, now always happens through the system `malloc` and `free` instead of optionally being part of the pooling instance allocator. Along the way this refactors the `InstanceAllocator` trait so the pooling and on-demand allocators can share more structure with this new property of the implementation. The main rationale for this commit is to reduce the RSS of long-lived programs which allocate instances with the pooling instance allocator and aren't using the "next available" allocation strategy. In this situation the memory for an instance is never decommitted until the end of the program, meaning that eventually all instance slots will become occupied and resident. This has the effect of Wasmtime slowly eating more and more memory over time as each slot gets an instance allocated. By switching to the system allocator this should reduce the current RSS workload from O(used slots) to O(active slots), which is more in line with expectations.	2 years ago
Alex Crichton	8ffbb9cfd7	Reimplement the pooling instance allocation strategy (#5661 ) * Reimplement the pooling instance allocation strategy This commit is a reimplementation of the strategy by which the pooling instance allocator selects a slot for a module. Previously there was a choice amongst three different algorithms: "reuse affinity", "next available", and "random". The default was "reuse affinity" but some new data has come to light which shows that this may not always be a good default. Notably the pooling allocator will retain some memory per-slot in the pooling instance allocator, for example instance data or memory data if-so-configured. This means that a currently unused, but previously used, slot can contribute to the RSS usage of a program using Wasmtime. Consequently the RSS impact here is O(max slots) which can be counter-intuitive for embedders. This particularly affects "reuse affinity" because the algorithm for picking a slot when there are no affine slots is "pick a random slot", which means eventually all slots will get used. In discussions about possible ways to tackle this, an alternative to "pick a strategy" arose and is now implemented in this commit. Concretely the new allocation algorithm for a slot is now: * First pick the most recently used affine slot, if one exists. * Otherwise if the number of affine slots to other modules is above some threshold N then pick the least-recently used affine slot. * Otherwise pick a slot that's affine to nothing. The "N" in this algorithm is configurable and setting it to 0 is the same as the old "next available" strategy while setting it to infinity is the same as the "reuse affinity" algorithm. Setting it to something in the middle provides a knob to allow a modest "cache" of affine slots while not allowing the total set of slots used to grow too much beyond the maximal concurrent set of modules. The "random" strategy is now no longer possible and was removed to help simplify the allocator. * Resolve rustdoc warnings in `wasmtime-runtime` crate * Remove `max_cold` as it duplicates the `slot_state.len()` * More descriptive names * Add a comment and debug assertion * Add some list assertions	2 years ago
Alex Crichton	4ad86752de	Fix libcall relocations for precompiled modules (#5608 ) * Fix libcall relocations for precompiled modules This commit fixes some asserts and support for relocation libcalls in precompiled modules loaded from disk. In doing so this reworks how mmaps are managed for files from disk. All non-file-backed `Mmap` entries are read/write but file-backed versions were readonly. This commit changes this such that all `Mmap` objects, even if they're file-backed, start as read/write. The file-based versions all use copy-on-write to preserve the private-ness of the mapping. This is not functionally intended to change anything. Instead this should have some more memory writable after a module is loaded but the text section, for example, is still left as read/execute when loading is finished. Additionally this makes modules compiled in memory more consistent with modules loaded from disk. * Update a comment * Force images to become readonly during publish This marks compiled images as entirely readonly during the `CodeMemory::publish` step which happens just before the text section becomes executable. This ensures that all images, no matter where they come from, are guaranteed frozen before they start executing.	2 years ago
Szczepan Ćwikliński	86790d36df	Fix compile errors on FreeBSD x64/arm64 (#5606 ) * Fix compile error on FreeBSD x64 * Fix compile on FreeBSD arm64 * Update Cargo.lock for ittapi * vet: certify diff for ittapi libraries Co-authored-by: Andrew Brown <andrew.brown@intel.com>	2 years ago
Alex Crichton	9b896d2a70	Resolve libcall relocations for older CPUs (#5567 ) * Resolve libcall relocations for older CPUs Long ago Wasmtime used to have logic for resolving relocations post-compilation for libcalls which I ended up removing during refactorings last year. As #5563 points out, however, it's possible to get Wasmtime to panic by disabling SSE features which forces Cranelift to use libcalls for some floating-point operations instead. Note that this also requires disabling SIMD because SIMD support has a baseline of SSE 4.2. This commit pulls back the old implementations of various libcalls and reimplements logic necessary to have them work on CPUs without SSE 4.2 Closes #5563 * Fix log message in `wast` support * Fix offset listed in relocations Be sure to factor in the offset of the function itself * Review comments	2 years ago
Alex Crichton	d9fdbfd50e	Use the `sym` operator for inline assembly (#5459 ) * Use the `sym` operator for inline assembly Avoids extra `#[no_mangle]` functions and undue symbols being exposed from Wasmtime. This is a newly stabilized feature in Rust 1.66.0. I've also added a `rust-version` entry to the `wasmtime` crate to try to head off possible reports in the future about odd error messages or usage of unstable features if the rustc version is too old. * Fix a s390x warning * Add `rust-version` annotation to Wasmtime crate As the other main entrypoint for embedders.	2 years ago
Alex Crichton	03715dda9d	Tidy up some internals of instance allocation (#5346 ) * Simplify the `ModuleRuntimeInfo` trait slightly Fold two functions into one as they're only called from one location anyway. * Remove ModuleRuntimeInfo::signature This is redundant as the array mapping is already stored within the `VMContext` so that can be consulted rather than having a separate trait function for it. This required altering the `Global` creation slightly to work correctly in this situation. * Remove a now-dead constant * Shared `VMOffsets` across instances This commit removes the computation of `VMOffsets` to being per-module instead of per-instance. The `VMOffsets` structure is also quite large so this shaves off 112 bytes per instance which isn't a huge impact but should help lower the cost of instantiating small modules. * Remove `InstanceAllocator::adjust_tunables` This is no longer needed or necessary with the pooling allocator. * Fix compile warning * Fix a vtune warning * Fix pooling tests * Fix another test warning	2 years ago
Alex Crichton	ed6769084b	Add a `WasmBacktrace::new()` constructor (#5341 ) * Add a `WasmBacktrace::new()` constructor This commit adds a method of manually capturing a backtrace of WebAssembly frames within a `Store`. The new constructor can be called with any `AsContext` values, primarily `&Store` and `&Caller`, during host functions to inspect the calling state. For now this does not respect the `Config::wasm_backtrace` option and instead unconditionally captures the backtrace. It's hoped that this can continue to adapt to needs of embedders by making it more configurable int he future if necessary. Closes #5339 * Split `new` into `capture` and `force_capture`	2 years ago
Alex Crichton	e0b9663e44	Remove some custom error types in Wasmtime (#5347 ) * Remove some custom error types in Wasmtime These types are mostly cumbersome to work with nowadays that `anyhow` is used everywhere else. This commit removes `InstantiationError` and `SetupError` in favor of using `anyhow::Error` throughout. This can eventually culminate in creation of specific errors for embedders to downcast to but for now this should be general enough. * Fix Windows build	2 years ago
Alex Crichton	86acb9a438	Use workspace inheritance for some more dependencies (#5349 ) Deduplicate some dependency directives through `[workspace.dependencies]`	2 years ago
Dan Gohman	d6d3c49972	Update to cap-std 1.0, io-lifetimes 1.0. (#5330 ) The main change here is that io-lifetimes 1.0 switches to use the I/O safety feature in the standard library rather than providing its own copy. This also updates to windows-sys 0.42.0 and rustix 0.36.	2 years ago
Alex Crichton	951bdcb2cf	Clear affine slots when dropping a `Module` (#5321 ) * Clear affine slots when dropping a `Module` This commit implements a resource usage optimization for Wasmtime with the pooling instance allocator by ensuring that when a `Module` is dropped its backing virtual memory mappings are all removed. Currently when a `Module` is dropped it releases a strong reference to its internal memory image but the memory image may stick around in individual pooling instance allocator slots. When using the `Random` allocation strategy, for example, this means that the memory images could stick around for a long time. While not a pressing issue this has resource usage implications for Wasmtime. Namely removing a `Module` does not guarantee the memfd, if in use for a memory image, is closed and deallocated within the kernel. Unfortunately simply closing the memfd is not sufficient as well as the mappings into the address space additionally all need to be removed for the kernel to release the resources for the memfd. This means that to release all kernel-level resources for a `Module` all slots which have the memory image mapped in must have the slot reset. This problem isn't particularly present when using the `NextAvailable` allocation strategy since the number of lingering memfds is proportional to the maximum concurrent size of wasm instances. With the `Random` and `ReuseAffinity` strategies, however, it's much more prominent because the number of lingering memfds can reach the total number of slots available. This can appear as a leak of kernel-level memory which can cause other system instability. To fix this issue this commit adds necessary instrumentation to `Drop for Module` to purge all references to the module in the pooling instance allocator. All index allocation strategies now maintain affinity tracking to ensure that regardless of the strategy in use a module that is dropped will remove all its memory mappings. A new allocation method was added to the index allocator for allocating an index without setting affinity and only allocating affine slots. This is used to iterate over all the affine slots without holding the global index lock for an unnecessarily long time while mappings are removed. * Review comments	2 years ago
Alex Crichton	6ce2ac19b8	Refactor shared memory internals, expose embedder methods (#5311 ) This commit refactors the internals of `wasmtime_runtime::SharedMemory` a bit to expose the necessary functions to invoke from the `wasmtime::SharedMemory` layer. Notably some items are moved out of the `RwLock` from prior, such as the type and the `VMMemoryDefinition`. Additionally the organization around the `atomic_*` methods has been redone to ensure that the `wasmtime`-layer abstraction has a single method to call into which everything else uses as well.	2 years ago
Harald Hoyer	8ce98e3c12	fix: atomit wait does not sleep long enough (#5315 ) From the documentation of `CondVar::wait_timeout`: > The semantics of this function are equivalent to wait except that the thread > will be blocked for roughly no longer than `dur`. This method should not be > used for precise timing due to anomalies such as preemption or platform > differences that might not cause the maximum amount of time waited to be > precisely `dur`. Therefore, go to sleep again, if the thread has not slept long enough. Signed-off-by: Harald Hoyer <harald@profian.com> Signed-off-by: Harald Hoyer <harald@profian.com>	2 years ago
Harald Hoyer	c74706aa59	feat: implement memory.atomic.notify,wait32,wait64 (#5255 ) * feat: implement memory.atomic.notify,wait32,wait64 Added the parking_spot crate, which provides the needed registry for the operations. Signed-off-by: Harald Hoyer <harald@profian.com> * fix: change trap message for HeapMisaligned The threads spec test wants "unaligned atomic" instead of "misaligned memory access". Signed-off-by: Harald Hoyer <harald@profian.com> * tests: add test for atomic wait on non-shared memory Signed-off-by: Harald Hoyer <harald@profian.com> * tests: add tests/spec_testsuite/proposals/threads without pooling and reference types. Also "shared_memory" is added to the "spectest" interface. Signed-off-by: Harald Hoyer <harald@profian.com> * tests: add atomics_notify.wast checking that notify with 0 waiters returns 0 on shared and non-shared memory. Signed-off-by: Harald Hoyer <harald@profian.com> * tests: add tests for atomic wait on shared memory - return 2 - timeout for 0 - return 2 - timeout for 1000ns - return 1 - invalid value Signed-off-by: Harald Hoyer <harald@profian.com> * fixup! feat: implement memory.atomic.notify,wait32,wait64 Signed-off-by: Harald Hoyer <harald@profian.com> * fixup! feat: implement memory.atomic.notify,wait32,wait64 Signed-off-by: Harald Hoyer <harald@profian.com> Signed-off-by: Harald Hoyer <harald@profian.com>	2 years ago

1 2 3 4 5 ...

448 Commits (0f9ac11bfbb8c6c876813781af484e07f998a25c)