cranelift

Commit Graph

Author	SHA1	Message	Date
Alex Crichton	2f4419cc6c	Implement runtime checks for compilation settings (#3899 ) * Implement runtime checks for compilation settings This commit fills out a few FIXME annotations by implementing run-time checks that when a `Module` is created it has compatible codegen settings for the current host (as `Module` is proof of "this code can run"). This is done by implementing new `Engine`-level methods which validate compiler settings. These settings are validated on `Module::new` as well as when loading serialized modules. Settings are split into two categories, one for "shared" top-level settings and one for ISA-specific settings. Both categories now have allow-lists hardcoded into `Engine` which indicate the acceptable values for each setting (if applicable). ISA-specific settings are checked with the Rust standard library's `std::is_x86_feature_detected!` macro. Other macros for other platforms are not stable at this time but can be added here if necessary. Closes #3897 * Fix fall-through logic to actually be correct * Use a `OnceCell`, not an `AtomicBool` * Fix some broken tests	3 years ago
Pat Hickey	63b7120a00	fix the tests	3 years ago
Alex Crichton	9e0c910023	Add a `Module::deserialize_file` method (#3266 ) * Add a `Module::deserialize_file` method This commit adds a new method to the `wasmtime::Module` type, `deserialize_file`. This is intended to be the same as the `deserialize` method except for the serialized module is present as an on-disk file. This enables Wasmtime to internally use `mmap` to avoid copying bytes around and generally makes loading a module much faster. A C API is added in this commit as well for various bindings to use this accelerated path now as well. Another option perhaps for a Rust-based API is to have an API taking a `File` itself to allow for a custom file descriptor in one way or another, but for now that's left for a possible future refactoring if we find a use case. * Fix compat with main - handle readdonly mmap * wip * Try to fix Windows support	3 years ago
Alex Crichton	c73be1f13a	Use an mmap-friendly serialization format (#3257 ) * Use an mmap-friendly serialization format This commit reimplements the main serialization format for Wasmtime's precompiled artifacts. Previously they were generally a binary blob of `bincode`-encoded metadata prefixed with some versioning information. The downside of this format, though, is that loading a precompiled artifact required pushing all information through `bincode`. This is inefficient when some data, such as trap/address tables, are rarely accessed. The new format added in this commit is one which is designed to be `mmap`-friendly. This means that the relevant parts of the precompiled artifact are already page-aligned for updating permissions of pieces here and there. Additionally the artifact is optimized so that if data is rarely read then we can delay reading it until necessary. The new artifact format for serialized modules is an ELF file. This is not a public API guarantee, so it cannot be relied upon. In the meantime though this is quite useful for exploring precompiled modules with standard tooling like `objdump`. The ELF file is already constructed as part of module compilation, and this is the main contents of the serialized artifact. THere is some extra information, though, not encoded in each module's individual ELF file such as type information. This information continues to be `bincode`-encoded, but it's intended to be much smaller and much faster to deserialize. This extra information is appended to the end of the ELF file. This means that the original ELF file is still a valid ELF file, we just get to have extra bits at the end. More information on the new format can be found in the module docs of the serialization module of Wasmtime. Another refatoring implemented as part of this commit is to deserialize and store object files directly in `mmap`-backed storage. This avoids the need to copy bytes after the artifact is loaded into memory for each compiled module, and in a future commit it opens up the door to avoiding copying the text section into a `CodeMemory`. For now, though, the main change is that copies are not necessary when loading from a precompiled compilation artifact once the artifact is itself in mmap-based memory. To assist with managing `mmap`-based memory a new `MmapVec` type was added to `wasmtime_jit` which acts as a form of `Vec<T>` backed by a `wasmtime_runtime::Mmap`. This type notably supports `drain(..N)` to slice the buffer into disjoint regions that are all separately owned, such as having a separately owned window into one artifact for all object files contained within. Finally this commit implements a small refactoring in `wasmtime-cache` to use the standard artifact format for cache entries rather than a bincode-encoded version. This required some more hooks for serializing/deserializing but otherwise the crate still performs as before. * Review comments	3 years ago
Pat Hickey	895ee2b85f	make Module::deserialize's version check optional via Config (#2945 ) * make Module::deserialize's version check optional via Config A SerializedModule contains the CARGO_PKG_VERSION string, which is checked for equality when loading. This is a great guard-rail but some users may want to disable this check (e.g. so they can implement their own versioning scheme) * rename config to deserialize_check_wasmtime_version * add test * fix doc links * fix * thank you rustdoc	3 years ago
Alex Crichton	7a1b7cdf92	Implement RFC 11: Redesigning Wasmtime's APIs (#2897 ) Implement Wasmtime's new API as designed by RFC 11. This is quite a large commit which has had lots of discussion externally, so for more information it's best to read the RFC thread and the PR thread.	3 years ago
Alex Crichton	8384f3a347	Bring back `Module::deserialize` (#2858 ) * Bring back `Module::deserialize` I thought I was being clever suggesting that `Module::deserialize` was removed from #2791 by funneling all module constructors into `Module::new`. As our studious fuzzers have found, though, this means that `Module::new` is not safe currently to pass arbitrary user-defined input into. Now one might pretty reasonable expect to be able to do that, however, being a WebAssembly engine and all. This PR as a result separates the `deserialize` part of `Module::new` back into `Module::deserialize`. This means that binary blobs created with `Module::serialize` and `Engine::precompile_module` will need to be passed to `Module::deserialize` to "rehydrate" them back into a `Module`. This restores the property that it should be safe to pass arbitrary input to `Module::new` since it's always expected to be a wasm module. This also means that fuzzing will no longer attempt to fuzz `Module::deserialize` which isn't something we want to do anyway. * Fix an example * Mark `Module::deserialize` as `unsafe`	4 years ago
Chris Fallin	6bec13da04	Bump versions: Wasmtime to 0.26.0, Cranelift to 0.73.0.	4 years ago
Peter Huene	d1313b1291	Code review feedback. * Move `Module::compile` to `Engine::precompile_module`. * Remove `Module::deserialize` method. * Make `Module::serialize` the same format as `Engine::precompile_module`. * Make `Engine::precompile_module` return a `Vec<u8>`. * Move the remaining serialization-related code to `serialization.rs`.	4 years ago
Peter Huene	1ce2a87149	Code review feedback. * Remove `Config::for_target` in favor of setter `Config::target`. * Remove explicit setting of Cranelift flags in `Config::new` in favor of calling the `Config` methods that do the same thing. * Serialize the package version independently of the data when serializing a module. * Use struct deconstructing in module serialization to ensure tunables and features aren't missed. * Move common log initialization in the CLI into `CommonOptions`.	4 years ago
Alex Crichton	2697a18d2f	Redo the statically typed `Func` API (#2719 ) * Redo the statically typed `Func` API This commit reimplements the `Func` API with respect to statically typed dispatch. Previously `Func` had a `getN` and `getN_async` family of methods which were implemented for 0 to 16 parameters. The return value of these functions was an `impl Fn(..)` closure with the appropriate parameters and return values. There are a number of downsides with this approach that have become apparent over time: * The addition of `_async` doubled the API surface area (which is quite large here due to one-method-per-number-of-parameters). The [documentation of `Func`][old-docs] are quite verbose and feel "polluted" with all these getters, making it harder to understand the other methods that can be used to interact with a `Func`. * These methods unconditionally pay the cost of returning an owned `impl Fn` with a `'static` lifetime. While cheap, this is still paying the cost for cloning the `Store` effectively and moving data into the closed-over environment. * Storage of the return value into a struct, for example, always requires `Box`-ing the returned closure since it otherwise cannot be named. * Recently I had the desire to implement an "unchecked" path for invoking wasm where you unsafely assert the type signature of a wasm function. Doing this with today's scheme would require doubling (again) the API surface area for both async and synchronous calls, further polluting the documentation. The main benefit of the previous scheme is that by returning a `impl Fn` it was quite easy and ergonomic to actually invoke the function. In practice, though, examples would often have something akin to `.get0::<()>()?()?` which is a lot of things to interpret all at once. Note that `get0` means "0 parameters" yet a type parameter is passed. There's also a double function invocation which looks like a lot of characters all lined up in a row. Overall, I think that the previous design is starting to show too many cracks and deserves a rewrite. This commit is that rewrite. The new design in this commit is to delete the `getN{,_async}` family of functions and instead have a new API: impl Func { fn typed<P, R>(&self) -> Result<&Typed<P, R>>; } impl Typed<P, R> { fn call(&self, params: P) -> Result<R, Trap>; async fn call_async(&self, params: P) -> Result<R, Trap>; } This should entirely replace the current scheme, albeit by slightly losing ergonomics use cases. The idea behind the API is that the existence of `Typed<P, R>` is a "proof" that the underlying function takes `P` and returns `R`. The `Func::typed` method peforms a runtime type-check to ensure that types all match up, and if successful you get a `Typed` value. Otherwise an error is returned. Once you have a `Typed` then, like `Func`, you can either `call` or `call_async`. The difference with a `Typed`, however, is that the params/results are statically known and hence these calls can be much more efficient. This is a much smaller API surface area from before and should greatly simplify the `Func` documentation. There's still a problem where `Func::wrapN_async` produces a lot of functions to document, but that's now the sole offender. It's a nice benefit that the statically-typed-async verisons are now expressed with an `async` function rather than a function-returning-a-future which makes it both more efficient and easier to understand. The type `P` and `R` are intended to either be bare types (e.g. `i32`) or tuples of any length (including 0). At this time `R` is only allowed to be `()` or a bare `i32`-style type because multi-value is not supported with a native ABI (yet). The `P`, however, can be any size of tuples of parameters. This is also where some ergonomics are lost because instead of `f(1, 2)` you now have to write `f.call((1, 2))` (note the double-parens). Similarly `f()` becomes `f.call(())`. Overall I feel that this is a better tradeoff than before. While not universally better due to the loss in ergonomics I feel that this design is much more flexible in terms of what you can do with the return value and also understanding the API surface area (just less to take in). [old-docs]: https://docs.rs/wasmtime/0.24.0/wasmtime/struct.Func.html#method.get0 * Rename Typed to TypedFunc * Implement multi-value returns through `Func::typed` * Fix examples in docs * Fix some more errors * More test fixes * Rebasing and adding `get_typed_func` * Updating tests * Fix typo * More doc tweaks * Tweak visibility on `Func::invoke` * Fix tests again	4 years ago
Peter Huene	54c07d8f16	Implement shared host functions. (#2625 ) * Implement defining host functions at the Config level. This commit introduces defining host functions at the `Config` rather than with `Func` tied to a `Store`. The intention here is to enable a host to define all of the functions once with a `Config` and then use a `Linker` (or directly with `Store::get_host_func`) to use the functions when instantiating a module. This should help improve the performance of use cases where a `Store` is short-lived and redefining the functions at every module instantiation is a noticeable performance hit. This commit adds `add_to_config` to the code generation for Wasmtime's `Wasi` type. The new method adds the WASI functions to the given config as host functions. This commit adds context functions to `Store`: `get` to get a context of a particular type and `set` to set the context on the store. For safety, `set` cannot replace an existing context value of the same type. `Wasi::set_context` was added to set the WASI context for a `Store` when using `Wasi::add_to_config`. * Add `Config::define_host_func_async`. * Make config "async" rather than store. This commit moves the concept of "async-ness" to `Config` rather than `Store`. Note: this is a breaking API change for anyone that's already adopted the new async support in Wasmtime. Now `Config::new_async` is used to create an "async" config and any `Store` associated with that config is inherently "async". This is needed for async shared host functions to have some sanity check during their execution (async host functions, like "async" `Func`, need to be called with the "async" variants). * Update async function tests to smoke async shared host functions. This commit updates the async function tests to also smoke the shared host functions, plus `Func::wrap0_async`. This also changes the "wrap async" method names on `Config` to `wrap$N_host_func_async` to slightly better match what is on `Func`. * Move the instance allocator into `Engine`. This commit moves the instantiated instance allocator from `Config` into `Engine`. This makes certain settings in `Config` no longer order-dependent, which is how `Config` should ideally be. This also removes the confusing concept of the "default" instance allocator, instead opting to construct the on-demand instance allocator when needed. This does alter the semantics of the instance allocator as now each `Engine` gets its own instance allocator rather than sharing a single one between all engines created from a configuration. * Make `Engine::new` return `Result`. This is a breaking API change for anyone using `Engine::new`. As creating the pooling instance allocator may fail (likely cause is not enough memory for the provided limits), instead of panicking when creating an `Engine`, `Engine::new` now returns a `Result`. * Remove `Config::new_async`. This commit removes `Config::new_async` in favor of treating "async support" as any other setting on `Config`. The setting is `Config::async_support`. * Remove order dependency when defining async host functions in `Config`. This commit removes the order dependency where async support must be enabled on the `Config` prior to defining async host functions. The check is now delayed to when an `Engine` is created from the config. * Update WASI example to use shared `Wasi::add_to_config`. This commit updates the WASI example to use `Wasi::add_to_config`. As only a single store and instance are used in the example, it has no semantic difference from the previous example, but the intention is to steer users towards defining WASI on the config and only using `Wasi::add_to_linker` when more explicit scoping of the WASI context is required.	4 years ago
Yury Delendik	399ee0a54c	Serialize and deserialize compilation artifacts. (#2020 ) * Serialize and deserialize Module * Use bincode to serialize * Add wasm_module_serialize; docs * Simple tests	4 years ago

13 Commits (871a9d93f24cfc50c42b5b832cda84be387626b3)