rust jit wasm webassembly runtime wasi cranelift sandbox wasmtime

History

Pat Hickey 0f9ac11bfb WASI Preview 2: rewrite streams and pollable implementation (#6556 ) * preview2: make everything but streams/io and poll/poll synchronous * streams: get rid of as_any method, which is no longer used * delete legacy sched and pollable concepts * more code motion and renaming * make tokio a workspace dep, because we need it directly in wasmtime-wasi * HostPollable exists * more fixes * pollable can trap, and implement clock properly * HostPollable is now a generator of futures because we need to be able to poll a pollable many times * explain various todo!s * Synchronous version of the wasi-preview2-components tests * Change with_tokio to accept the future as an argument * Store futures in the PollOneoff struct instead, to avoid dropping them * Remove TODO for HostOutputStream impl for WritePipe * Implement pollable for ReadPipe * Use a Notify when ReadPipe is ready * wip * wip * Read/write pipe ends with tokio channels * Empty reader/writer wrappers * EmptyStream, and warning cleanup * Wrapped reader/writer structs * Rework stdio in terms of wrapped read/write * Add MemoryOutputPipe and update tests * Remove todo * rewrite nearly everything * implement the pipe stuff * wibble * fix MemoryOutputPipe just enough to make the tests compile * Move the table iteration into a helper function * AsyncFd stream implementation to fix stdin on unix * Rename Wrapped{Read,Write} streams to Async{Read,Write}Stream * Move async io wrappers into stream.rs * Fix the sync tests * fix test uses of pipes, juggle tokio context for stdin construction * add some fixmes * the future i named Never is defined in futures-util as pending which is a better name * i believe this is a correct implementation of one global stdin resource * move unix stdin to its own file * make most of the mods private * fix build - we are skipping rust 1.70 due to llvm regressions in s390x and riscv64 which are fixed in 1.71 and will not be backported * preview1-in-preview2: use async funcs for io, and the async io interface prtest:full * windows stdin support * done! * table ext functions: fix tests * tests: expect poll_oneoff_{files,stdio} to pass on all platforms * export the bindings under wasmtime_wasi::preview2::bindings rather than preview2::wasi. and command moves to wasmtime_wasi::preview2::command as well. * fix renaming of wasi to bindings in tests * use block_in_place throughout filesystem and move block_on and block_in_place to be pub crate at the root * AsyncFdStream: ensure file is nonblocking * tests: block_in_place requires multi-threaded runtime * actually, use fcntl_setfl to make the asyncfd file nonblocking * fix windows block_on * docs, remove unnecessary methods * more docs * Add a workspace dependency on bytes-1.4 * Remove vectored stream operations * Rework the read/write stream traits * Add a size parameter to `read`, and switch to usize for traits * Pipe through the bool -> stream-status change in wit * Plumb stream-status through write operations in wit * write host trait also gives streamstate * hook new stream host read/write back up to the wit bindgen * sketchy AsyncReadStream impl * Fill out implementations for AsyncReadStream and AsyncWriteStream * some reasonable read tests * more * first smoke test for AsyncWriteStream * bunch of AsyncWriteStream tests * half-baked idea that the output-stream interface will need a flush mechanism * adapter: fixes for changes to stream wit * fix new rust 1.71 warnings * make stdin work on unix without using AsyncFdStream inline the tokio docs example of how to impl AsyncRead for an AsyncFd, except theres some "minor" changes because stdin doesnt impl Read on &Stdin whereas tcpstream from the example does * delete AsyncFdStream for now it turns out to be kinda hard and we can always work on adding it back in later. * Implement some memory pipe operations, and move async wrappers to the pipe mod * Make blocking_write actually block until everything is written * Remove debug print * Adapter stdio should use blocking write Rust guests will panic if the write returns less than the number of bytes sent with stdio. * Clean up implementations of {blocking_}write_zeros and skip * Remove debug macro usage * Move EmptyStream to pipe, and split it into four variants Use EmptyInputStream and SinkOutputStream as the defaults for stdin and stdout/stderr respectively. * Add a big warning about resource lifetime tracking in pollables * Start working through changes to the filesystem implementation * Remove todos in the filesystem implementation * Avoid lifetime errors by moving blocking operations to File and Dir * Fix more lifetime issues with `block` * Finish filling out translation impl * fix warnings * we can likely eliminate block_in_place in the stdin implementations * sync command uses sync filesystem, start of translation layer * symc filesystem: all the trait boilerplate is in place just need to finish the from impl boilerplate * finish type conversion boilerplate * Revert "half-baked idea that the output-stream interface will need a flush mechanism" This reverts commit `3eb762e333`. * cargo fmt * test type fixes * renames and comments * refactor stream table internals so we can have a blocking variant... * preview1 host adapter: stdout/stderr use blocking_write here too * filesystem streams are blocking now * fixes * satisfy cargo doc * cargo vet: dep upgrades taken care of by imports from mozilla * unix stdio: eliminate block_in_place * replace private in_tokio with spawn, since its only used for spawning * comments * worker thread stdin implementation can be tested on linux, i guess and start outlining a test plan * eliminate tokio boilerplate - no longer using tokios lock * rename our private block_on to in_tokio * fill in missing file input skip * code review: fix MemoryInputPipe. Closed status is always available immediately. * code review: empty input stream is not essential, closed input stream is a better fi for stdin * code review: unreachable * turn worker thread (windows) stdin off * expect preview2-based poll_oneoff_stdio to fail on windows * command directory_list test: no need to inherit stdin * preview1 in preview2: turn off inherit_stdio except for poll_oneoff_stdio * wasi-preview2-components: apparently inherit_stdio was on everywhere here as well. turn it off except for poll_oneoff_stdio * extend timeout for riscv64 i suppose --------- Co-authored-by: Trevor Elliott <telliott@fastly.com>		1 year ago
..
WASI@0ba0c5e2e3	Implement `sock_accept`	3 years ago
cap-std-sync	Dependency gardening for Wasmtime (#6731)	1 year ago
src	Dependency gardening for Wasmtime (#6731)	1 year ago
tokio	WASI Preview 2: rewrite streams and pollable implementation (#6556)	1 year ago
Cargo.toml	Dependency gardening for Wasmtime (#6731)	1 year ago
LICENSE	Initial reorg.	5 years ago
README.md	Implement the remaining socket-related WASI functions. (#4776)	2 years ago
build.rs	Add "cargo:rerun-if-changed=build.rs" to some build.rs files.	3 years ago

README.md

`wasi-common`

A Bytecode Alliance project

A library providing a common implementation of WASI hostcalls for re-use in any WASI-enabled runtime.

The wasi-common crate will ultimately serve as a library providing a common implementation of WASI hostcalls for re-use in any WASI (and potentially non-WASI) runtimes such as Wasmtime and Lucet.

The library is an adaption of lucet-wasi crate from the Lucet project, and it is currently based on 40ae1df git revision.

Please note that the library requires Rust compiler version at least 1.37.0.

Supported syscalls

*nix

In our *nix implementation, we currently support the entire WASI API with the exception of the proc_raise hostcall, as it is expected to be dropped entirely from WASI.

Windows

In our Windows implementation, we currently support the minimal subset of WASI API which allows for running the very basic "Hello world!" style WASM apps. More coming shortly, so stay tuned!

Development hints

When testing the crate, you may want to enable and run full wasm32 integration testsuite. This requires wasm32-wasi target installed which can be done as follows using rustup

rustup target add wasm32-wasi

Now, you should be able to run the integration testsuite by running cargo test on the test-programs package with test-programs/test_programs feature enabled:

cargo test --features test-programs/test_programs --package test-programs