* Add an example for wasi-nn WinML backend.
This example is similar as classification-component-onnx, but it's not a wasm
component, and it's for WinML backend. It also includes a step by step
instruction for running this example.
* Touch up the documentation for this example
This change removes some duplicated information and tweaks some of the
wording.
---------
Co-authored-by: Andrew Brown <andrew.brown@intel.com>
* add an initial implemenation for onnxruntime backend of wasi-nn
Signed-off-by: David Justice <david@devigned.com>
* vet: audit ONNX dependencies
This change is the result of a long slog through the dependencies of the
`ort` library. The only missing dependency is `compact_str`, which needs
further discussion.
* vet: add ONNX audit entry for compact_str 0.7.1
Signed-off-by: David Justice <david@devigned.com>
* refactor tests to break out onnx and openvino
Signed-off-by: David Justice <david@devigned.com>
* mark wasi-nn onnx example as publish false
Signed-off-by: David Justice <david@devigned.com>
* update the ONNX classification example
* do not use wasi-nn onnx feature if riskv or s390
Signed-off-by: David Justice <david@devigned.com>
* prtest:full fix running WASI-NN ONNX tests across arch os
Signed-off-by: David Justice <david@devigned.com>
---------
Signed-off-by: David Justice <david@devigned.com>
Co-authored-by: Andrew Brown <andrew.brown@intel.com>
* wasi-nn: add [named models]
This change adds a way to retrieve preloaded ML models (i.e., "graphs"
in wasi-nn terms) from a registry. The wasi-nn specification includes a
new function, `load_by_name`, that can be used to access these models
more efficiently than before; previously, a user's only option was to
read/download/etc. all of the bytes of an ML model and pass them to the
`load` function.
[named models]: https://github.com/WebAssembly/wasi-nn/issues/36
In Wasmtime's implementation of wasi-nn, we call the registry that holds
the models a `GraphRegistry`. We include a simplistic `InMemoryRegistry`
for use in the Wasmtime CLI (more on this later) but the idea is that
production use will involve some more complex caching and thus a new
implementation of a registry--a `Box<dyn GraphRegistry>`--passed into
the wasi-nn context. Note that, because we now must be able to `clone` a
graph out of the registry and into the "used graphs" table, the OpenVINO
`BackendGraph` is updated to be easier to copy around.
To allow experimentation with this "preload a named model"
functionality, this change also adds a new Wasmtime CLI flag: `--graph
<encoding>:<host dir>`. Wasmtime CLI users can now preload a model from
a directory; the directory `basename` is used as the model name. Loading
models from a directory is probably not desired in Wasmtime embeddings
so it is cordoned off into a separate `BackendFromDir` extension trait.
* wasi-nn: add "named model" test
Add a new example crate which loads a model by name and performs image
classification. It uses the same MobileNet model as the existing test
but a new version of the Rust bindings. The new crate is built and run
with the new CLI flag in the `ci/run-wasi-nn-example.sh` script.
prtest:full
* review: rename `--graph` to `--wasi-nn-graph`
* wasi-nn: refactor to allow `preview2` access
This change refactors the `wasmtime-wasi-nn` crate to allow access from
both `preview1` and `preview2` ABIs. Though the `wasi-nn` specification
has included a WIT description for some time, here we use some in-tree
files until https://github.com/WebAssembly/wasi-nn/pull/38 is landed.
The `preview2` code is not exercised anywhere yet: ideally this would be
wired up once component model `resource`s are fully implemented in
Wasmtime.
prtest:full
* wasi-nn: use `preview1` linkage
prtest:full
* review: rename `preview*` to `wit*`
This is based on @pchickey's [comments] on ABI naming.
[comments]: https://bytecodealliance.zulipchat.com/#narrow/stream/266558-wasi-nn/topic/wasi-nn.20.2B.20preview2/near/383368292
* review: update README
* fix: remove broken doc links
* fix: replace typo `wit` with `gen`
* review: use `wit` types everywhere
This removes the crate-specific types in order to use the WIT-generated
types throughout the crate. The main effect of this is that the crate
no longer optionally includes `wasmtime` with the `component-model`
feature--now that is required.
* review: move `BackendKind` conversion into `witx.rs`
* review: remove `<'a>`
* review: use `tracing` crate instead of `eprintln!`
* Add release notes for 3.0.1
* Update some version directives for crates in Wasmtime
* Mark anything with `publish = false` as version 0.0.0
* Mark the icache coherence crate with the same version as Wasmtime
* Fix manifest directives
* Upgrade all crates to the Rust 2021 edition
I've personally started using the new format strings for things like
`panic!("some message {foo}")` or similar and have been upgrading crates
on a case-by-case basis, but I think it probably makes more sense to go
ahead and blanket upgrade everything so 2021 features are always
available.
* Fix compile of the C API
* Fix a warning
* Fix another warning
* Use stable Rust on CI to test the x64 backend
This commit leverages the newly-released 1.51.0 compiler to test the
new backend on Windows and Linux with a stable compiler instead of a
nightly compiler. This isolates the nightly build to just the nightly
documentation generation and fuzzing, both of which rely on nightly for
the best results right now.
* Use updated stable in book build job
* Run rustfmt for new stable
* Silence new warnings for wasi-nn
* Allow some dead code in the x64 backend
Looks like new rustc is better about emitting some dead-code warnings
* Update rust in peepmatic job
* Fix a test in the pooling allocator
* Remove `package.metdata.docs.rs` temporarily
Needs resolution of https://github.com/rust-lang/cargo/pull/9300 first
* Fix a warning in a wasi-nn example
* Add an initial wasi-nn implementation for Wasmtime
This change adds a crate, `wasmtime-wasi-nn`, that uses `wiggle` to expose the current state of the wasi-nn API and `openvino` to implement the exposed functions. It includes an end-to-end test demonstrating how to do classification using wasi-nn:
- `crates/wasi-nn/tests/classification-example` contains Rust code that is compiled to the `wasm32-wasi` target and run with a Wasmtime embedding that exposes the wasi-nn calls
- the example uses Rust bindings for wasi-nn contained in `crates/wasi-nn/tests/wasi-nn-rust-bindings`; this crate contains code generated by `witx-bindgen` and eventually should be its own standalone crate
* Test wasi-nn as a CI step
This change adds:
- a GitHub action for installing OpenVINO
- a script, `ci/run-wasi-nn-example.sh`, to run the classification example
Bruce Mitchener
FrankReh
jianjunz
David Justice
Andrew Brown
Alex Crichton
Nick Fitzgerald