This commit disables the Emscripten CI job until a regression
in rustc for `wasm32-unknown-emscripten` target is fixed.
Tracking issue: [rust-lang/rust 66308](https://github.com/rust-lang/rust/issues/66308).
Our corpora are too large to run in full on every pull request, they just take
too long. Instead, we sample some of them and make sure that running our fuzzers
over the sampled inputs still works OK.
This uses `--locked` on CI to ensure that if the lock file needs
changing it's reflected in the PR instead of letting CI accidentally and
silently update the lock file for us.
* Compile wasi-common to Emscripten
This commit enables cross-compiling of `wasi-common` to Emscripten. To achieve
this, this commit does quite a bit reshuffling in the existing codebase. Namely,
* rename `linux` modules in `wasi-common` and `yanix` to `linux_like` -- this is
needed so that we can separate out logic specific to Linux and Emscripten out
* tweak `dir` module in `yanix` to support Emscripten -- in particular, the main
change involves `SeekLoc::from_raw` which has to be now host-specific, and is now
fallible
* tweak `filetime` so that in Emscripten we never check for existence of `utimensat`
at runtime since we are guaranteed for it to exist by design
* since `utimes` and `futimes` are not present in Emscripten, move them into a separate
module, `utimesat`, and tag it cfg-non-emscripten only
* finally, `to_timespec` is now fallible since on Emscripten we have to cast number of
seconds, `FileTime::seconds` from `i64` to `libc::c_long` which resolves to `i32`
unlike on other nixes
* Fix macos build
* Verify wasi-common compiles to Emscripten
This commit adds `emscripten` job to Github Actions which installs
`wasm32-unknown-emscripten` target, and builds `wasi-common` crate.
* Use #[path] to cherry-pick mods for Emscripten
This commit effectively reverses the reorg introduced in 145f4a5
in that it ditches `linux_like` mod for separate mods `linux` and
`emscripten` which are now on the same crate level, and instead,
pulls in common bits from `linux` using the `#[path = ..]` proc
macro.
* Refactor the `types.rs` types and structures
A few changes applied along the way:
* Documentation added to most methods and types.
* Limits are now stored with the maximum as optional rather than a
sentinel u32 value for `None`.
* The `Name` type was removed in favor of just using a bare `String`.
* The `Extern` prefix in the varaints of `ExternType` has been removed
since it was redundant.
* Accessors of `ExternType` variants no longer panic, and unwrapping
versions were added with "unwrap" in the name.
* Fields and methods named `r#type` were renamed to `ty` to avoid
requiring a raw identifier to use them.
* Remove `fail-fast: false`
This was left around since the development of GitHub Actions for
wasmtime, but they're no longer needed!
* Fix compilation of the test-programs code
* Fix compilation of wasmtime-py package
* Run rustfmt
For Windows release builds, the `wasm_valtype_kind` C API return value
is being returned as a single byte.
The .NET interop signature for this function was expecting an
integer-sized return, resulting in three extra bytes being used on
Windows.
The fix is to limit the corresponding C# enum to a byte representation,
which will properly mask the return value from `wasm_valtype_kind`.
CI has also been updated to test both debug and release configurations
(previously it was only testing debug, hence why this was missed).
Also fixed a cast bug in the `declare_vec!` macro in the C API when the
element types were pointers to values. The `as_slice` implementation
was incorrectly casting away a level of pointer indirection, resulting
in corrupted data when accessing the slice's elements.
Rust's recent update to libstd of the wasm32-wasi target turned out to
be buggy with respect to fetching the process arguments, so we'll need
to wait on a fix there before we can run these tests with nightly again.
* Deny warnings in the CI by default
This commit passes `-D warnings` flag to `rustc` in the Test stage
of the CI in order to deny warnings on _all_ tested platforms.
* Fix warnings
* Fix warnings
This doesn't have libFuzzer generate new inputs in CI, only verifies that we can
still successfully process our existing seed inputs in each of our targets'
corpora.
This commit fixes the CI by setting `actions/checkout` at version `v1`.
The current master of `actions/checkout` now obsoleted the
`with: submodules: true` input.
See [actions/checkout/releases/tag/v2-beta] for more info.
[actions/checkout/releases/tag/v2-beta]: https://github.com/actions/checkout/releases/tag/v2-beta
* use setuptools_scm for python version management
* add git dep for python wheel building
* if no tag is defined, default to dev
* any untagged version default to 0.0.1
* Fix feature-gating of test-programs
This commit fixes bugs in enabling feature-gating of `test-programs`
which was introduced in #600. It turns out, #600 accidentally
disabled `test-programs` from ever running, and this commit fixes
that.
* Fix the CI
* Rename the `wasmtime_api` library to match the containing `wasmtime` crate
Commit d9ca508f80 renamed the
`wasmtime-api` crate to `wasmtime`, but left the name of the library it
contains as `wasmtime_api`.
It's fairly unusual for a crate to contain a library with a different
name, and it results in rather confusing error messages for a user; if
you list `wasmtime = "0.7"` in `Cargo.toml`, you can't `use
wasmtime::*`, you have to `use wasmtime_api::*;`.
Rename the `wasmtime_api` library to `wasmtime`.
* Stop renaming wasmtime to api on imports
Various users renamed the crate formerly known as wasmtime_api to api,
and then used api:: prefixes everywhere; change those all to wasmtime::
and drop the renaming.
* Reorganize wasi-misc-tests.
Move wasi-misc-tests out of wasi-common, to break a dependency cycle;
previously, wasmtime-* depended on wasi-common, but wasi-common
dev-dependended on wasmtime-*.
Now, wasi-common no longer dev-depends on wasmtime-*; instead, the
tests are in their own crate which depends on wasi-common and on
wasmtime-*.
Also, rename wasi-misc-tests to wasi-tests for simplicity.
This also removes the "wasm_tests" feature; it's replaced by the
"test-programs" feature.
* Update the CI script to use the new feature name.
* Update the CI script to use the new feature name in one more place.
* Change a `write!` to a `writeln!`.
A small amount of cleanup for the top-level of this repository, where
the `installer` directory just had one misc file for the Windows
installer which we can put elsewhere inside of `ci`, another misc folder
but one that already has a few files in it.
* Update the top-level README.md and embedding documentation.
wasmtime-api is now the primary external API crate, so recommend that
instead of wasmtime-jit.
Also, enable wasmtime-api's C API by default, so that it shows up on
docs.rs, and to make it easier to use.
And, add basic embedding documentation and link to it from the
README.md. Credit to @yurydelendik for the content.
* Use the new wasm-c-api URL.
* Don't pass --features wasm-c-api, as it is now on by default.
This commit switches the beta/nightly tests to happen on Ubuntu instead
of macOS. Turns out GitHub Actions has scheduling limitations on macOS
that limit repositories to 5 concurrent jobs per repository, so let's
reduce the load a bit by running more builds on Linux than mac.
This commit migrates wasmtime's CI infrastructure from Azure Pipelines
to Github Actions. Using Github Actions has a few benefits over other
offerings:
* Being natively integrated with Github means that there's no degree of
user account configuration or access control management, it's all
inherent via already existing Github permissions.
* Github Actions gives 20 parallel builders instead of Azure's 10 by
default, which is a nice boost to have!
Overall I've found Github Actions to feel a bit cleaner than Azure
Pipelines as well. Subjectively I've found the configuration to be more
readable and more pleasant to work with, although they're both just as
"powerful" I think. Additionally Github Actions has been pretty solid in
my own personal testing for a number of other projects.
The main trickiness with wasmtime's CI is the rolling `dev` release of
the master branch as well as binary releases for tags. Github Actions
doesn't have quite as much built in functionality as Azure Pipelines,
but Github Actions does have a nice feature where you can define the
code for an action locally rather than only using built-in actions.
This migration adds three local actions with some associated JS code to
run the action (currently it looks like it basically requires JS)
* An `install-rust` action papers over the gotchas about installing
Rust, allowing Rust installation to be a one-liner in the configuration.
* A `binary-compatible-builds` action allows easily configuring the
wheels and the binaries to be "more binary compatible" and handles
things like compilation flags on OSX and Windows while handling the
`centos:6` container on Linux.
* The `github-release` action is the logic using the `@actions/github`
JS package to orchestrate the custom way we manage rolling releases,
ensuring that a new release is made for the master branch under `dev`
(deleting the previous tag/release ahead of time) and then also
manages tagged releases by uploading them there.
I'm hoping that most of the inline actions here will largely go away.
For example `install-rust` should be simply `rustup update $toolchain`
once various environment issues are fixed on Github Actions runner
images. Additionally `github-release` will ideally migrate to something
like https://github.com/actions/create-release or similar once it has
enough functionality. I'm also hoping that the maintenance in the
meantime of these actions is pretty low-cost, but if it becomes an issue
we can look into other solutions!
* Use generated type bindings.
Use the witx API descriptions to generate the bulk of the contents of
host.rs, wasi.rs, and wasi32.rs.
This also prunes out many of the miscellaneous libc definitions from
those files which aren't currently in use by wasi-common. If there's
anything removed that's still needed by someone, it's easy to add things
back in.
* Remove unneeded iovec conversion routines.
Jakub Konka
Nick Fitzgerald
Alex Crichton
Peter Huene
Luiz Irber
Josh Triplett
Dan Gohman
Jakub Konka