github/cranelift - cranelift - Gitea: Git with home nas server

Author	SHA1	Message	Date
Nick Fitzgerald	bd2ea901d3	Define garbage collection rooting APIs (#8011 ) * Define garbage collection rooting APIs Rooting prevents GC objects from being collected while they are actively being used. We have a few sometimes-conflicting goals with our GC rooting APIs: 1. Safety: It should never be possible to get a use-after-free bug because the user misused the rooting APIs, the collector "mistakenly" determined an object was unreachable and collected it, and then the user tried to access the object. This is our highest priority. 2. Moving GC: Our rooting APIs should moving collectors (such as generational and compacting collectors) where an object might get relocated after a collection and we need to update the GC root's pointer to the moved object. This means we either need cooperation and internal mutability from individual GC roots as well as the ability to enumerate all GC roots on the native Rust stack, or we need a level of indirection. 3. Performance: Our rooting APIs should generally be as low-overhead as possible. They definitely shouldn't require synchronization and locking to create, access, and drop GC roots. 4. Ergonomics: Our rooting APIs should be, if not a pleasure, then at least not a burden for users. Additionally, the API's types should be `Sync` and `Send` so that they work well with async Rust. For example, goals (3) and (4) are in conflict when we think about how to support (2). Ideally, for ergonomics, a root would automatically unroot itself when dropped. But in the general case that requires holding a reference to the store's root set, and that root set needs to be held simultaneously by all GC roots, and they each need to mutate the set to unroot themselves. That implies `Rc<RefCell<...>>` or `Arc<Mutex<...>>`! The former makes the store and GC root types not `Send` and not `Sync`. The latter imposes synchronization and locking overhead. So we instead make GC roots indirect and require passing in a store context explicitly to unroot in the general case. This trades worse ergonomics for better performance and support for moving GC and async Rust. Okay, with that out of the way, this module provides two flavors of rooting API. One for the common, scoped lifetime case, and another for the rare case where we really need a GC root with an arbitrary, non-LIFO/non-scoped lifetime: 1. `RootScope` and `Rooted<T>`: These are used for temporarily rooting GC objects for the duration of a scope. Upon exiting the scope, they are automatically unrooted. The internal implementation takes advantage of the LIFO property inherent in scopes, making creating and dropping `Rooted<T>`s and `RootScope`s super fast and roughly equivalent to bump allocation. This type is vaguely similar to V8's [`HandleScope`]. [`HandleScope`]: https://v8.github.io/api/head/classv8_1_1HandleScope.html Note that `Rooted<T>` can't be statically tied to its context scope via a lifetime parameter, unfortunately, as that would allow the creation and use of only one `Rooted<T>` at a time, since the `Rooted<T>` would take a borrow of the whole context. This supports the common use case for rooting and provides good ergonomics. 2. `ManuallyRooted<T>`: This is the fully general rooting API used for holding onto non-LIFO GC roots with arbitrary lifetimes. However, users must manually unroot them. Failure to manually unroot a `ManuallyRooted<T>` before it is dropped will result in the GC object (and everything it transitively references) leaking for the duration of the `Store`'s lifetime. This type is roughly similar to SpiderMonkey's [`PersistentRooted<T>`], although they avoid the manual-unrooting with internal mutation and shared references. (Our constraints mean we can't do those things, as mentioned explained above.) [`PersistentRooted<T>`]: http://devdoc.net/web/developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/JSAPI_reference/JS::PersistentRooted.html At the end of the day, both `Rooted<T>` and `ManuallyRooted<T>` are just tagged indices into the store's `RootSet`. This indirection allows working with Rust's borrowing discipline (we use `&mut Store` to represent mutable access to the GC heap) while still allowing rooted references to be moved around without tying up the whole store in borrows. Additionally, and crucially, this indirection allows us to update the actual GC pointers in the `RootSet` and support moving GCs (again, as mentioned above). * Reorganize GC-related submodules in `wasmtime-runtime` * Reorganize GC-related submodules in `wasmtime` * Use `Into<StoreContext[Mut]<'a, T>` for `Externref::data[_mut]` methods * Run rooting tests under MIRI * Make `into_abi` take an `AutoAssertNoGc` * Don't use atomics to update externref ref counts anymore * Try to make lifetimes/safety more-obviously correct Remove some transmute methods, assert that `VMExternRef`s are the only valid `VMGcRef`, etc. * Update extenref constructor examples * Make `GcRefImpl::transmute_ref` a non-default trait method * Make inline fast paths for GC LIFO scopes * Make `RootSet::unroot_gc_ref` an `unsafe` function * Move Hash and Eq for Rooted, move to impl methods * Remove type parameter from `AutoAssertNoGc` Just wrap a `&mut StoreOpaque` directly. * Make a bunch of internal `ExternRef` methods that deal with raw `VMGcRef`s take `AutoAssertNoGc` instead of `StoreOpaque` * Fix compile after rebase * rustfmt * revert unrelated egraph changes * Fix non-gc build * Mark `AutoAssertNoGc` methods inline * review feedback * Temporarily remove externref support from the C API Until we can add proper GC rooting. * Remove doxygen reference to temp deleted function * Remove need to `allow(private_interfaces)` * Fix call benchmark compilation	8 months ago
Alex Crichton	9ce3ffe15e	Update some CI dependencies (#7983 ) * Update some CI dependencies * Update to the latest nightly toolchain * Update mdbook * Update QEMU for cross-compiled testing * Update `cargo nextest` for usage with MIRI prtest:full * Remove lots of unnecessary imports * Downgrade qemu as 8.2.1 seems to segfault * Remove more imports * Remove unused winch trait method * Fix warnings about unused trait methods * More unused imports * More unused imports	9 months ago
Nick Fitzgerald	ff93bce067	Wasmtime: Finish support for the typed function references proposal (#7943 ) * Wasmtime: Finish support for the typed function references proposal While we supported the function references proposal inside Wasm, we didn't support it on the "outside" in the Wasmtime embedder APIs. So much of the work here is exposing typed function references, and their type system updates, in the embedder API. These changes include: * `ValType::FuncRef` and `ValType::ExternRef` are gone, replaced with the introduction of the `RefType` and `HeapType` types and a `ValType::Ref(RefType)` variant. * `ValType` and `FuncType` no longer implement `Eq` and `PartialEq`. Instead there are `ValType::matches` and `FuncType::matches` methods which check directional subtyping. I also added `ValType::eq` and `FuncType::eq` static methods for the rare case where someone needs to check precise equality, but that is almost never actually the case, 99.99% of the time you want to check subtyping. * There are also public `Val::matches_ty` predicates for checking if a value is an instance of a type, as well as internal helpers like `Val::ensure_matches_ty` that return a formatted error if the value does not match the given type. These helpers are used throughout Wasmtime internals now. * There is now a dedicated `wasmtime::Ref` type that represents reference values. Table operations have been updated to take and return `Ref`s rather than `Val`s. Furthermore, this commit also includes type registry changes to correctly manage lifetimes of types that reference other types. This wasn't previously an issue because the only thing that could reference types that reference other types was a Wasm module that added all the types that could reference each other at the same time and removed them all at the same time. But now that the previously discussed work to expose these things in the embedder API is done, type lifetime management in the registry becomes a little trickier because the embedder might grab a reference to a type that references another type, and then unload the Wasm module that originally defined that type, but then the user should still be able use that type and the other types it transtively references. Before, we were refcounting individual registry entries. Now, we still are refcounting individual entries, but now we are also accounting for type-to-type references and adding a new type to the registry will increment the refcounts of each of the types that it references, and removing a type from the registry will decrement the refcounts of each of the types it references, and then recursively (logically, not literally) remove any types whose refcount has now reached zero. Additionally, this PR adds support for subtyping to `Func::typed`- and `Func::wrap`-style APIs. For result types, you can always use a supertype of the WebAssembly function's actual declared return type in `Func::typed`. And for param types, you can always use a subtype of the Wasm function's actual declared param type. Doing these things essentially erases information but is always correct. But additionally, for functions which take a reference to a concrete type as a parameter, you can also use the concrete type's supertype. Consider a WebAssembly function that takes a reference to a function with a concrete type: `(ref null <func type index>)`. In this scenario, there is no static `wasmtime::Foo` Rust type that corresponds to that particular Wasm-defined concrete reference type because Wasm modules are loaded dynamically at runtime. You could do `f.typed::<Option<NoFunc>, ()>()`, and while that is correctly typed and valid, it is often overly restrictive. The only value you could call the resulting typed function with is the null function reference, but we'd like to call it with non-null function references that happen to be of the correct type. Therefore, `f.typed<Option<Func>, ()>()` is also allowed in this case, even though `Option<Func>` represents `(ref null func)` which is the supertype, not subtype, of `(ref null <func type index>)`. This does imply some minimal dynamic type checks in this case, but it is supported for better ergonomics, to enable passing non-null references into the function. We can investigate whether it is possible to use generic type parameters and combinators to define Rust types that precisely match concrete reference types in future, follow-up pull requests. But for now, we've made things usable, at least. Finally, this also takes the first baby step towards adding support for the Wasm GC proposal. Right now the only thing that is supported is `nofunc` references, and this was mainly to make testing function reference subtyping easier. But that does mean that supporting `nofunc` references entailed also adding a `wasmtime::NoFunc` type as well as the `Config::wasm_gc(enabled)` knob. So we officially have an in-progress implementation of Wasm GC in Wasmtime after this PR lands! Fixes https://github.com/bytecodealliance/wasmtime/issues/6455 * Fix WAT in test to be valid * Check that dependent features are enabled for function-references and GC * Remove unnecessary engine parameters from a few methods Ever since `FuncType`'s internal `RegisteredType` holds onto its own `Engine`, we don't need these anymore. Still useful to keep the `Engine` parameter around for the `ensure_matches` methods because that can be used to check correct store/engine usage for embedders. * Add missing dependent feature enabling for some tests * Remove copy-paste bit from test * match self to show it is uninhabited * Add a missing `is_v128` method * Short circuit a few func type comparisons * Turn comment into part of doc comment * Add test for `Global::new` and subtyping * Add tests for embedder API, tables, and subtyping * Add an embedder API test for setting globals and subtyping * Construct realloc's type from its index, rather than from scratch * Help LLVM better optimize our dynamic type checks in `TypedFunc::call_raw` * Fix call benchmark compilation * Change `WasmParams::into_abi` to take the whole func type instead of iter of params * Fix doc links prtest:full * Fix size assertion on s390x	9 months ago
Alex Crichton	b0939f6626	Remove explicit `S` type parameters (#5275 ) * Remove explicit `S` type parameters This commit removes the explicit `S` type parameter on `Func::typed` and `Instance::get_typed_func`. Historical versions of Rust required that this be a type parameter but recent rustcs support a mixture of explicit type parameters and `impl Trait`. This removes, at callsites, a superfluous `, _` argument which otherwise never needs specification. * Fix mdbook examples	2 years ago
Alex Crichton	7a1b7cdf92	Implement RFC 11: Redesigning Wasmtime's APIs (#2897 ) Implement Wasmtime's new API as designed by RFC 11. This is quite a large commit which has had lots of discussion externally, so for more information it's best to read the RFC thread and the PR thread.	3 years ago
Alex Crichton	2697a18d2f	Redo the statically typed `Func` API (#2719 ) * Redo the statically typed `Func` API This commit reimplements the `Func` API with respect to statically typed dispatch. Previously `Func` had a `getN` and `getN_async` family of methods which were implemented for 0 to 16 parameters. The return value of these functions was an `impl Fn(..)` closure with the appropriate parameters and return values. There are a number of downsides with this approach that have become apparent over time: * The addition of `_async` doubled the API surface area (which is quite large here due to one-method-per-number-of-parameters). The [documentation of `Func`][old-docs] are quite verbose and feel "polluted" with all these getters, making it harder to understand the other methods that can be used to interact with a `Func`. * These methods unconditionally pay the cost of returning an owned `impl Fn` with a `'static` lifetime. While cheap, this is still paying the cost for cloning the `Store` effectively and moving data into the closed-over environment. * Storage of the return value into a struct, for example, always requires `Box`-ing the returned closure since it otherwise cannot be named. * Recently I had the desire to implement an "unchecked" path for invoking wasm where you unsafely assert the type signature of a wasm function. Doing this with today's scheme would require doubling (again) the API surface area for both async and synchronous calls, further polluting the documentation. The main benefit of the previous scheme is that by returning a `impl Fn` it was quite easy and ergonomic to actually invoke the function. In practice, though, examples would often have something akin to `.get0::<()>()?()?` which is a lot of things to interpret all at once. Note that `get0` means "0 parameters" yet a type parameter is passed. There's also a double function invocation which looks like a lot of characters all lined up in a row. Overall, I think that the previous design is starting to show too many cracks and deserves a rewrite. This commit is that rewrite. The new design in this commit is to delete the `getN{,_async}` family of functions and instead have a new API: impl Func { fn typed<P, R>(&self) -> Result<&Typed<P, R>>; } impl Typed<P, R> { fn call(&self, params: P) -> Result<R, Trap>; async fn call_async(&self, params: P) -> Result<R, Trap>; } This should entirely replace the current scheme, albeit by slightly losing ergonomics use cases. The idea behind the API is that the existence of `Typed<P, R>` is a "proof" that the underlying function takes `P` and returns `R`. The `Func::typed` method peforms a runtime type-check to ensure that types all match up, and if successful you get a `Typed` value. Otherwise an error is returned. Once you have a `Typed` then, like `Func`, you can either `call` or `call_async`. The difference with a `Typed`, however, is that the params/results are statically known and hence these calls can be much more efficient. This is a much smaller API surface area from before and should greatly simplify the `Func` documentation. There's still a problem where `Func::wrapN_async` produces a lot of functions to document, but that's now the sole offender. It's a nice benefit that the statically-typed-async verisons are now expressed with an `async` function rather than a function-returning-a-future which makes it both more efficient and easier to understand. The type `P` and `R` are intended to either be bare types (e.g. `i32`) or tuples of any length (including 0). At this time `R` is only allowed to be `()` or a bare `i32`-style type because multi-value is not supported with a native ABI (yet). The `P`, however, can be any size of tuples of parameters. This is also where some ergonomics are lost because instead of `f(1, 2)` you now have to write `f.call((1, 2))` (note the double-parens). Similarly `f()` becomes `f.call(())`. Overall I feel that this is a better tradeoff than before. While not universally better due to the loss in ergonomics I feel that this design is much more flexible in terms of what you can do with the return value and also understanding the API surface area (just less to take in). [old-docs]: https://docs.rs/wasmtime/0.24.0/wasmtime/struct.Func.html#method.get0 * Rename Typed to TypedFunc * Implement multi-value returns through `Func::typed` * Fix examples in docs * Fix some more errors * More test fixes * Rebasing and adding `get_typed_func` * Updating tests * Fix typo * More doc tweaks * Tweak visibility on `Func::invoke` * Fix tests again	4 years ago
Peter Huene	54c07d8f16	Implement shared host functions. (#2625 ) * Implement defining host functions at the Config level. This commit introduces defining host functions at the `Config` rather than with `Func` tied to a `Store`. The intention here is to enable a host to define all of the functions once with a `Config` and then use a `Linker` (or directly with `Store::get_host_func`) to use the functions when instantiating a module. This should help improve the performance of use cases where a `Store` is short-lived and redefining the functions at every module instantiation is a noticeable performance hit. This commit adds `add_to_config` to the code generation for Wasmtime's `Wasi` type. The new method adds the WASI functions to the given config as host functions. This commit adds context functions to `Store`: `get` to get a context of a particular type and `set` to set the context on the store. For safety, `set` cannot replace an existing context value of the same type. `Wasi::set_context` was added to set the WASI context for a `Store` when using `Wasi::add_to_config`. * Add `Config::define_host_func_async`. * Make config "async" rather than store. This commit moves the concept of "async-ness" to `Config` rather than `Store`. Note: this is a breaking API change for anyone that's already adopted the new async support in Wasmtime. Now `Config::new_async` is used to create an "async" config and any `Store` associated with that config is inherently "async". This is needed for async shared host functions to have some sanity check during their execution (async host functions, like "async" `Func`, need to be called with the "async" variants). * Update async function tests to smoke async shared host functions. This commit updates the async function tests to also smoke the shared host functions, plus `Func::wrap0_async`. This also changes the "wrap async" method names on `Config` to `wrap$N_host_func_async` to slightly better match what is on `Func`. * Move the instance allocator into `Engine`. This commit moves the instantiated instance allocator from `Config` into `Engine`. This makes certain settings in `Config` no longer order-dependent, which is how `Config` should ideally be. This also removes the confusing concept of the "default" instance allocator, instead opting to construct the on-demand instance allocator when needed. This does alter the semantics of the instance allocator as now each `Engine` gets its own instance allocator rather than sharing a single one between all engines created from a configuration. * Make `Engine::new` return `Result`. This is a breaking API change for anyone using `Engine::new`. As creating the pooling instance allocator may fail (likely cause is not enough memory for the provided limits), instead of panicking when creating an `Engine`, `Engine::new` now returns a `Result`. * Remove `Config::new_async`. This commit removes `Config::new_async` in favor of treating "async support" as any other setting on `Config`. The setting is `Config::async_support`. * Remove order dependency when defining async host functions in `Config`. This commit removes the order dependency where async support must be enabled on the `Config` prior to defining async host functions. The check is now delayed to when an `Engine` is created from the config. * Update WASI example to use shared `Wasi::add_to_config`. This commit updates the WASI example to use `Wasi::add_to_config`. As only a single store and instance are used in the example, it has no semantic difference from the previous example, but the intention is to steer users towards defining WASI on the config and only using `Wasi::add_to_linker` when more explicit scoping of the WASI context is required.	4 years ago
Nick Fitzgerald	17b99cc9c8	examples: Add a GC call to the externref Rust example	4 years ago
Nick Fitzgerald	4a349ee299	wasmtime: Add `externref` Rust example	4 years ago