* Remove wasm-c-api submodule
This submodule hasn't been updated in ~3 years at this point and we
additionally don't need most of the submodule. Instead add a script to
copy the files we need and verify in CI that the files are up-to-date.
This also makes using the C API a bit nicer where you don't have to have
two `include` directories with a Wasmtime source tree, just one
suffices.
* Don't format wasm.h{,h} vendored files
* Remove the `WASI` submodule
Historically the `WASI` submodule in this repository was used to pull
the sources of the `*.witx` files for preview1. This has never been used
by the `wasmtime-wasi` crate (which has its own copy in the
`crates/wasi/witx` folder) and was only ever used by `wasi-common`. This
submodule also served as a location for the `witx` crate itself.
Neither of these are really needed any more as the `*.witx` files are
unlikely to ever really change again. This commit removes the submodule
entirely, as well as the `path` dependency on `witx`, and copies the
`*.witx` files in the same manner as the `wasmtime-wasi` crate.
* Updated vet entries for witx
* Remove witx special cases in publish script
* Change how the wasi-common package is built
This was removed in #6195 but re-added in #6877, I believe by accident,
so this re-deletes it. I've also edited `.gitmodules` a bit while I was
here to remove it and additionally keep other entries up-to-date with
matching paths.
* Remove the implementation of wasi-crypto
This commit is a follow-up to the discussion on #6732. This removes
Wasmtime's implementation of the wasi-crypto proposal from in-tree along
with its various support in CI, configuration, etc. See the discussion
on #6732 for the full information but at a high level the main reasons
for removing the implementation at this time are:
* There is not currently an active maintainer of the Wasmtime
integration here for wasi-crypto.
* There are known issues with the code quality of the implementation
such as transmutes of guest-owned memory to `&'static mut [u8]` and
known unsafety in dependencies.
* The size and breadth of the dependency tree brings maintenance burden
and overhead to managing Wasmtime's dependency tree.
As mentioned on the issue this commit does not mean that Wasmtime
doesn't want to implement the wasi-crypto proposal. Instead the "tier 3"
status of wasi-crypto needs to be re-attained to be included back
in-tree, which would mean resolving the above issues.
Note that this commit is intentionally just after the 13.0.0 branch
point which means that this is slated for Wasmtime 14 to be released on
September 20.
* Remove some cfgs
* Remove wasi-crypto CI
* wasi: add the `wasi-testsuite` tests for wasi-common
As described [here], this uses the `prod/testsuite-base` branch in which
the tests are built as `.wasm` files.
[here]: https://github.com/WebAssembly/wasi-testsuite/#getting-started
* chore: update `walkdir` everywhere to its latest version
This is done in order to use it for `wasi_testsuite` testing.
* vet: extend `walkdir`'s exemption
* test: factor out `get_wasmtime_command`
This will be helpful for `wasi_testsuite` testing.
* test: use all `wasi-testsuite` test cases
This change alters the `wasi_testsuite` test to run all of the available
test cases in [wasi-testsuite]. This involved making the test runner a
bit more robust to the various shapes of JSON specifications in that
project. Unfortunately, the `wasi_testsuite` test fails some of the
cases, so I added a `WASI_COMMON_IGNORE_LIST` to avoid these
temporarily. (This may remind some of the Wasm testsuite ignore lists in
Cranelift; those relied on `build.rs` to create a `#[test]` for each
test case, which I felt is not yet needed here).
It's unclear to me why the tests are failing. It could be because:
- wasi-common has a bug
- wasi-testsuite overspecifies (or incorrectly specifies) a test
- the test runner incorrectly configures Wasmtime's CLI execution.
But this change makes it easier to resolve this. Remove the file from
`WASI_COMMON_IGNORE_LIST` and run `cargo test wasi_testsuite --
--nocapture`. The printed output will show the expected result, the
actual result, and a command to replicate the failure from the command
line.
[wasi-testsuite]: https://github.com/WebAssembly/wasi-testsuite
* review: add "shrinking" comment
* Integrate experimental HTTP into wasmtime.
* Reset Cargo.lock
* Switch to bail!, plumb options partially.
* Implement timeouts.
* Remove generated files & wasm, add Makefile
* Remove generated code textfile
* Update crates/wasi-http/Cargo.toml
Co-authored-by: Eduardo de Moura Rodrigues <16357187+eduardomourar@users.noreply.github.com>
* Update crates/wasi-http/Cargo.toml
Co-authored-by: Eduardo de Moura Rodrigues <16357187+eduardomourar@users.noreply.github.com>
* Extract streams from request/response.
* Fix read for len < buffer length.
* Formatting.
* types impl: swap todos for traps
* streams_impl: idioms, and swap todos for traps
* component impl: idioms, swap all unwraps for traps, swap all todos for traps
* http impl: idiom
* Remove an unnecessary mut.
* Remove an unsupported function.
* Switch to the tokio runtime for the HTTP request.
* Add a rust example.
* Update to latest wit definition
* Remove example code.
* wip: start writing a http test...
* finish writing the outbound request example
havent executed it yet
* better debug output
* wasi-http: some stubs required for rust rewrite of the example
* add wasi_http tests to test-programs
* CI: run the http tests
* Fix some warnings.
* bump new deps to latest releases (#3)
* Add tests for wasi-http to test-programs (#2)
* wip: start writing a http test...
* finish writing the outbound request example
havent executed it yet
* better debug output
* wasi-http: some stubs required for rust rewrite of the example
* add wasi_http tests to test-programs
* CI: run the http tests
* bump new deps to latest releases
h2 0.3.16
http 0.2.9
mio 0.8.6
openssl 0.10.48
openssl-sys 0.9.83
tokio 1.26.0
---------
Co-authored-by: Brendan Burns <bburns@microsoft.com>
* Update crates/test-programs/tests/http_tests/runtime/wasi_http_tests.rs
* Update crates/test-programs/tests/http_tests/runtime/wasi_http_tests.rs
* Update crates/test-programs/tests/http_tests/runtime/wasi_http_tests.rs
* wasi-http: fix cargo.toml file and publish script to work together (#4)
unfortunately, the publish script doesn't use a proper toml parser (in
order to not have any dependencies), so the whitespace has to be the
trivial expected case.
then, add wasi-http to the list of crates to publish.
* Update crates/test-programs/build.rs
* Switch to rustls
* Cleanups.
* Merge switch to rustls.
* Formatting
* Remove libssl install
* Fix tests.
* Rename wasi-http -> wasmtime-wasi-http
* prtest:full
Conditionalize TLS on riscv64gc.
* prtest:full
Fix formatting, also disable tls on s390x
* prtest:full
Add a path parameter to wit-bindgen, remove symlink.
* prtest:full
Fix tests for places where SSL isn't supported.
* Update crates/wasi-http/Cargo.toml
---------
Co-authored-by: Eduardo de Moura Rodrigues <16357187+eduardomourar@users.noreply.github.com>
Co-authored-by: Pat Hickey <phickey@fastly.com>
Co-authored-by: Pat Hickey <pat@moreproductive.org>
* wasi-threads: run test suite
This change enables the running of the wasi-threads [test suite]. It
relies on a Wasmtime CLI binary being available and runs all `*.wasm`
and `*.wat` files present in the test suite directory. The results of
each execution are compared against a JSON spec file with the same base
name as the WebAssembly module. The spec file defines the expected exit
code, e.g.
This commit does not yet build any `*.c` or `*.s` files from the test
suite. That could be done later, perhaps upstream; in the meantime, this
work is still valuable as it lays the foundation for running other WASI
tests from the in-progress [wasi-testsuite] which share the same JSON
spec infrastructure.
[test suite]: https://github.com/WebAssembly/wasi-threads/tree/main/test/testsuite
[wasi-testsuite]: https://github.com/WebAssembly/wasi-testsuite
* review: move testsuite to top-level tests
* fix: remove now-unnecessary wasi-threads test
* fix: update testsuite submodule name
* fix: ignore tests on Windows
prtest:full
* fix: `cfg_attr` syntax
prtest:full
The WebAssembly spec interpreter is written in OCaml and the new crate
uses `ocaml-interop` along with a small OCaml wrapper to interpret Wasm
modules in-process. The build process for this crate is currently
Linux-specific: it requires several OCaml packages (e.g. `apt install -y
ocaml-nox ocamlbuild`) as well as `make`, `cp`, and `ar`.
This commit updates to the 0.9 version of the witx crate implemented in
WebAssembly/wasi#395. This new version drastically changes code
generation and how we interface with the crate. The intention is to
abstract the code generation aspects and allow code generators to
implement much more low-level instructions to enable more flexible APIs
in the future. Additionally a bunch of `*.witx` files were updated in
the WASI repository.
It's worth pointing out, however, that `wasi-common` does not change as
a result of this change. The shape of the APIs that we need to implement
are effectively the same and the only difference is that the shim
functions generated by wiggle are a bit different.
* Add support for the experimental wasi-crypto APIs
The sole purpose of the implementation is to allow bindings and
application developers to test the proposed APIs.
Rust and AssemblyScript bindings are also available as examples.
Like `wasi-nn`, it is currently disabled by default, and requires
the `wasi-crypto` feature flag to be compiled in.
* Rename the wasi-crypto/spec submodule
* Add a path dependency into the submodule for wasi-crypto
* Tell the publish script to vendor wasi-crypto
* Add an initial wasi-nn implementation for Wasmtime
This change adds a crate, `wasmtime-wasi-nn`, that uses `wiggle` to expose the current state of the wasi-nn API and `openvino` to implement the exposed functions. It includes an end-to-end test demonstrating how to do classification using wasi-nn:
- `crates/wasi-nn/tests/classification-example` contains Rust code that is compiled to the `wasm32-wasi` target and run with a Wasmtime embedding that exposes the wasi-nn calls
- the example uses Rust bindings for wasi-nn contained in `crates/wasi-nn/tests/wasi-nn-rust-bindings`; this crate contains code generated by `witx-bindgen` and eventually should be its own standalone crate
* Test wasi-nn as a CI step
This change adds:
- a GitHub action for installing OpenVINO
- a script, `ci/run-wasi-nn-example.sh`, to run the classification example
* Move back to only one WASI submodule
This commit fixes the issue where we have two WASI submodules for build
reasons in this repository. The fix was to place the submodule in the
`wasi-common` crate, and then anyone using the `wig` crate has to be
sure to define a `WASI_ROOT` env var in a build script to be able to
parse witx files.
With all that in place `wasi-common` becomes the source of truth for the
witx files we're parsing, and crates like `wasmtime-wasi` use
build-scripts shenanigans to read the same witx files. This should
hopefully get us so we're compatible with publishing and still only have
one submodule!
* rustfmt
* Publishing fixes.
* Make WASI a symlink.
* More fixes.
* Cargo doesn't allow dev-dependencies to have optional features.
* Remove the symlink.
* Add WASI as another git submodule.
* Move all examples to a top-level directory
This commit moves all API examples (Rust and C) to a top-level
`examples` directory. This is intended to make it more discoverable and
conventional as to where examples are located. Additionally all examples
are now available in both Rust and C to see how to execute the example
in the language you're familiar with. The intention is that as more
languages are supported we'd add more languages as examples here too.
Each example is also accompanied by either a `*.wat` file which is
parsed as input, or a Rust project in a `wasm` folder which is compiled
as input.
A simple driver crate was also added to `crates/misc` which executes all
the examples on CI, ensuring the C and Rust examples all execute
successfully.
* Move the C API to a separate crate
This commit moves the C API from `crates/api/src/wasm.rs` to
`crates/capi/src/lib.rs` to be located in a separate crate. There's a
number of reasons for this:
* When a Rust program depends on the `wasmtime` crate, there's no need
to compile in the C API.
* This should improve compile times of the `wasmtime` crate since it's
not producing artifacts which aren't always used.
* The development of the C API can be guaranteed to only use the public
API of the `wasmtime` crate itself.
Some CI pieces are tweaked and this overall shouldn't have much impact
on users, it's intended that it's a cleanup/speedup for developers!
* Disable rustdoc/tests for capi
* Review feedback
* Add back in accidentally deleted comment
* More renamings
* Try to fix dotnet build
* Initial checkin.
* Update to rust-lang libc.
* Add a .gitignore file.
* Factor out functions for cleaning up files and directories.
* Fix a typo in a comment.
* Print a "Success!" message if all tests passed.
* Factor out code for creating directories.
* Add wrappers around WASI functions.
These wrappers handle converting from &str to pointer+length and handle
unsafe.
* More refactoring.
* Refactor a fd_close helper.
* Move utility functions into a separate file.
* cargo update
* Add a basic test for random_get.
* Test that directories aren't resizable.
* Test clearing __WASI_RIGHT_PATH_FILESTAT_SET_SIZE.
Ensure that clearing __WASI_RIGHT_PATH_FILESTAT_SET_SIZE succeeds before
testing file truncation.
* cargo update
* Modularise tests for easier use with wasi-common crate
* Add a Code of Conduct and CONTRIBUTING.md.
* Fix typo
* Add testcase for fd_allocate
* Add positive test for fd_renumber
* Assert bufused in readlink_no_buffer testcase
* Add positive readlink testcase
* Add testcase for fd_seek and fd_tell
* Add fd_p{read, write} test
* Add README
* Add cases with trailing slashes to interesting_paths
* Split nofollow_errors testcase into two
* nofollow_errors now operators on symlinks to existing resources
* dangling_symlink covers danling symlinks tests
* Factor out a `create_file` helper function.
* Switch from the error crate to `std::io::Error::last_os_error()`.
* Use `create_file` in the readlink test too.
* Add a test for fd_filestat_set_*
* Minor refactoring
Add missing cleanup_file calls to file_pread_pwrite and
file_seek_tell.
* Add testcase for unbuffered fd_write; fixes#11
* Add testcase for path_rename
* Use the wasi crate.
Switch from depending on libc to depending on the new wasi crate to provide
the low-level WASI interfaces.
See also https://github.com/rust-lang/libc/pull/1461.
* Add a test for path_filestat_*
* Add a test for fd_readdir
* Use expect instead of unwrap
* Add a check for ino.
* Fix the build
* Don't assume a specific order of dirents
* Better test
* Test cookie value
* Fix file types
* Fix the test
* Fix the test
* Fix the test
* Cleanup
* Minor formatting tidying in README.md.
* Fix miscellaneous clippy warnings.
* Rename the crate to wasi-misc-tests.
* Update to wasi 0.7.0.
This switches from using the libc wasi bindings to using the wasi
crate's bindings. This eliminates a git dependency on libc, updates
to the new-style bindings which use Result where possible, and treats
functions that operate on raw file descriptors as unsafe.
* Add various tests for trailing-slash behavior.
* Sync new testcases with latest upstream
* Fix path_filestat testcase
* Add smoke test for fd_advise
This test is a true smoke test as it only tests whether issuing
an advise call to the host's kernel doesn't yield an error. The
consequence of issuing such a syscall is not tested.
* Check if CLOCK_MONOTONIC is actually monotonic
* Refactor the inequality assertions for more debuggable errors.
* Bump libc from 0.2.62 to 0.2.65
Bumps [libc](https://github.com/rust-lang/libc) from 0.2.62 to 0.2.65.
- [Release notes](https://github.com/rust-lang/libc/releases)
- [Commits](https://github.com/rust-lang/libc/compare/0.2.62...0.2.65)
Signed-off-by: dependabot-preview[bot] <support@dependabot.com>
* Fix compilation error
* Enable Actions and add rust.yml (#35)
* Enable Actions and add rust.yml
This commit enables Github Actions and adds corresponding configuration in rust.yml file.
* Update rust.yml
* Fix formatting
* Add empty .rustfmt.toml config file
* Add badge to README
* Update README
* Clean up Github Actions and README
* Add test case for `poll_oneoff` syscall (#38)
* Add test case for `poll_oneoff` syscall
This commit adds a test case for `poll_oneoff` syscall. In particular,
it builds on the excellent test use case provided by @dunnock in their
repo [poll_oneoff_tests] (thanks!), and tests:
* simple timeout
* stdin read with timeout
* fd read and fd write polls
[poll_oneoff_tests]: https://github.com/dunnock/poll_oneoff_tests
* Apply suggestions and negative test for bad fd
Co-authored-by: Maxim Vorobjov <maxim.vorobjov@gmail.com>
* Add smoke test for STDOUT/ERR readwrite poll
* Add comment on stdin/out/err
* Add a test for `*at`-style functions returning `ENOTDIR` when `dirfd` is not a dir.
* Remove misc_testsuite submodule
* Add "publish=false" to Cargo.toml; remove LICENSE
* Use generated type bindings.
Use the witx API descriptions to generate the bulk of the contents of
host.rs, wasi.rs, and wasi32.rs.
This also prunes out many of the miscellaneous libc definitions from
those files which aren't currently in use by wasi-common. If there's
anything removed that's still needed by someone, it's easy to add things
back in.
* Remove unneeded iovec conversion routines.
Now, test binaries are bundled with the repo, and
just like in CraneStation/wasmtime, the test cases
are generated automatically using build.rs. So all
it takes is to drop a new test binary in the
testsuite dir to get the test case for it generated
(with some caveats to do with handling preopens).
And lots of other miscellaneous changes. Rename InstanceWorld to
InstancePlus and reorganize its contents. This still isn't a great name,
but at least now it has a clear purpose.
Alex Crichton
Andrew Brown
Brendan Burns
Frank Denis
Dan Gohman
Jakub Konka
Pat Hickey
Jakub Konka
Sergey Pepyakin
Marcin Mielniczuk
Yury Delendik
Jef