You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

63 lines
2.0 KiB

Migrate from Azure Pipelines to Github Actions (#474) This commit migrates wasmtime's CI infrastructure from Azure Pipelines to Github Actions. Using Github Actions has a few benefits over other offerings: * Being natively integrated with Github means that there's no degree of user account configuration or access control management, it's all inherent via already existing Github permissions. * Github Actions gives 20 parallel builders instead of Azure's 10 by default, which is a nice boost to have! Overall I've found Github Actions to feel a bit cleaner than Azure Pipelines as well. Subjectively I've found the configuration to be more readable and more pleasant to work with, although they're both just as "powerful" I think. Additionally Github Actions has been pretty solid in my own personal testing for a number of other projects. The main trickiness with wasmtime's CI is the rolling `dev` release of the master branch as well as binary releases for tags. Github Actions doesn't have quite as much built in functionality as Azure Pipelines, but Github Actions does have a nice feature where you can define the code for an action locally rather than only using built-in actions. This migration adds three local actions with some associated JS code to run the action (currently it looks like it basically requires JS) * An `install-rust` action papers over the gotchas about installing Rust, allowing Rust installation to be a one-liner in the configuration. * A `binary-compatible-builds` action allows easily configuring the wheels and the binaries to be "more binary compatible" and handles things like compilation flags on OSX and Windows while handling the `centos:6` container on Linux. * The `github-release` action is the logic using the `@actions/github` JS package to orchestrate the custom way we manage rolling releases, ensuring that a new release is made for the master branch under `dev` (deleting the previous tag/release ahead of time) and then also manages tagged releases by uploading them there. I'm hoping that most of the inline actions here will largely go away. For example `install-rust` should be simply `rustup update $toolchain` once various environment issues are fixed on Github Actions runner images. Additionally `github-release` will ideally migrate to something like https://github.com/actions/create-release or similar once it has enough functionality. I'm also hoping that the maintenance in the meantime of these actions is pretty low-cost, but if it becomes an issue we can look into other solutions!
5 years ago
name: 'Install Rust toolchain'
description: 'Install a rust toolchain'
Migrate from Azure Pipelines to Github Actions (#474) This commit migrates wasmtime's CI infrastructure from Azure Pipelines to Github Actions. Using Github Actions has a few benefits over other offerings: * Being natively integrated with Github means that there's no degree of user account configuration or access control management, it's all inherent via already existing Github permissions. * Github Actions gives 20 parallel builders instead of Azure's 10 by default, which is a nice boost to have! Overall I've found Github Actions to feel a bit cleaner than Azure Pipelines as well. Subjectively I've found the configuration to be more readable and more pleasant to work with, although they're both just as "powerful" I think. Additionally Github Actions has been pretty solid in my own personal testing for a number of other projects. The main trickiness with wasmtime's CI is the rolling `dev` release of the master branch as well as binary releases for tags. Github Actions doesn't have quite as much built in functionality as Azure Pipelines, but Github Actions does have a nice feature where you can define the code for an action locally rather than only using built-in actions. This migration adds three local actions with some associated JS code to run the action (currently it looks like it basically requires JS) * An `install-rust` action papers over the gotchas about installing Rust, allowing Rust installation to be a one-liner in the configuration. * A `binary-compatible-builds` action allows easily configuring the wheels and the binaries to be "more binary compatible" and handles things like compilation flags on OSX and Windows while handling the `centos:6` container on Linux. * The `github-release` action is the logic using the `@actions/github` JS package to orchestrate the custom way we manage rolling releases, ensuring that a new release is made for the master branch under `dev` (deleting the previous tag/release ahead of time) and then also manages tagged releases by uploading them there. I'm hoping that most of the inline actions here will largely go away. For example `install-rust` should be simply `rustup update $toolchain` once various environment issues are fixed on Github Actions runner images. Additionally `github-release` will ideally migrate to something like https://github.com/actions/create-release or similar once it has enough functionality. I'm also hoping that the maintenance in the meantime of these actions is pretty low-cost, but if it becomes an issue we can look into other solutions!
5 years ago
inputs:
toolchain:
description: 'Default toolchan to install'
required: false
default: 'default'
Migrate from Azure Pipelines to Github Actions (#474) This commit migrates wasmtime's CI infrastructure from Azure Pipelines to Github Actions. Using Github Actions has a few benefits over other offerings: * Being natively integrated with Github means that there's no degree of user account configuration or access control management, it's all inherent via already existing Github permissions. * Github Actions gives 20 parallel builders instead of Azure's 10 by default, which is a nice boost to have! Overall I've found Github Actions to feel a bit cleaner than Azure Pipelines as well. Subjectively I've found the configuration to be more readable and more pleasant to work with, although they're both just as "powerful" I think. Additionally Github Actions has been pretty solid in my own personal testing for a number of other projects. The main trickiness with wasmtime's CI is the rolling `dev` release of the master branch as well as binary releases for tags. Github Actions doesn't have quite as much built in functionality as Azure Pipelines, but Github Actions does have a nice feature where you can define the code for an action locally rather than only using built-in actions. This migration adds three local actions with some associated JS code to run the action (currently it looks like it basically requires JS) * An `install-rust` action papers over the gotchas about installing Rust, allowing Rust installation to be a one-liner in the configuration. * A `binary-compatible-builds` action allows easily configuring the wheels and the binaries to be "more binary compatible" and handles things like compilation flags on OSX and Windows while handling the `centos:6` container on Linux. * The `github-release` action is the logic using the `@actions/github` JS package to orchestrate the custom way we manage rolling releases, ensuring that a new release is made for the master branch under `dev` (deleting the previous tag/release ahead of time) and then also manages tagged releases by uploading them there. I'm hoping that most of the inline actions here will largely go away. For example `install-rust` should be simply `rustup update $toolchain` once various environment issues are fixed on Github Actions runner images. Additionally `github-release` will ideally migrate to something like https://github.com/actions/create-release or similar once it has enough functionality. I'm also hoping that the maintenance in the meantime of these actions is pretty low-cost, but if it becomes an issue we can look into other solutions!
5 years ago
runs:
using: composite
steps:
- name: Install Rust
shell: bash
id: select
run: |
# Determine MSRV as N in `1.N.0` by looking at the `rust-version`
# located in the root `Cargo.toml`.
msrv=$(grep 'rust-version.*1' Cargo.toml | sed 's/.*\.\([0-9]*\)\..*/\1/')
if [ "${{ inputs.toolchain }}" = "default" ]; then
echo "version=1.$((msrv+2)).0" >> "$GITHUB_OUTPUT"
elif [ "${{ inputs.toolchain }}" = "msrv" ]; then
echo "version=1.$msrv.0" >> "$GITHUB_OUTPUT"
elif [ "${{ inputs.toolchain }}" = "wasmtime-ci-pinned-nightly" ]; then
echo "version=nightly-2024-07-25" >> "$GITHUB_OUTPUT"
else
echo "version=${{ inputs.toolchain }}" >> "$GITHUB_OUTPUT"
fi
- name: Install Rust
shell: bash
run: |
rustup set profile minimal
rustup update "${{ steps.select.outputs.version }}" --no-self-update
rustup default "${{ steps.select.outputs.version }}"
# Save disk space by avoiding incremental compilation. Also turn down
# debuginfo from 2 to 0 to help save disk space.
cat >> "$GITHUB_ENV" <<EOF
CARGO_INCREMENTAL=0
CARGO_PROFILE_DEV_DEBUG=0
CARGO_PROFILE_TEST_DEBUG=0
EOF
# Deny warnings on CI to keep our code warning-free as it lands in-tree.
echo RUSTFLAGS="-D warnings" >> "$GITHUB_ENV"
if [[ "${{ runner.os }}" = "macOS" ]]; then
cat >> "$GITHUB_ENV" <<EOF
CARGO_PROFILE_DEV_SPLIT_DEBUGINFO=unpacked
CARGO_PROFILE_TEST_SPLIT_DEBUGINFO=unpacked
EOF
fi
- name: Require semicolons in WIT
shell: bash
run: echo WIT_REQUIRE_SEMICOLONS=1 >> "$GITHUB_ENV"
Refactor the test-programs test suite (#7182) * Refactor the test-programs test suite This commit is a large refactoring that reorganizes `test-programs` and how we tests wasms in Wasmtime. Often writing tests requires complicated interactions with the guest which can&#39;t be done via hand-written `*.wat` syntax and requires a compiler to get engaged. For this purpose Wasmtime currently has the `crates/test-programs/*` test suite which builds files from source and then runs the tests. This has been somewhat cumbersome in the past though and it&#39;s not been easy to extend this over time, so this commit attempts to address this. The scheme implemented in this PR looks like: * All wasm test programs live in `crates/test-programs/src/bin/*.rs`. All of them, no exceptions. * Wasm tests have shared support located at `crates/test-programs/src/lib.rs` and its submodules, such as bindings generation for WASI. * Wasm tests are built by a new `crates/test-programs/artifacts` crate. This crate compiles modules and additionally creates components for all test programs. The crate itself only records the path to these outputs and a small amount of testing support, but otherwise doesn&#39;t interact with `wasmtime`-the-crate itself. * All tests in `crates/test-programs/tests/*.rs` have moved. For example wasi-http tests now live at `crates/wasi-http/tests/*.rs`. Legacy tests of wasi-common now live at `crates/wasi-common/tests/*.rs`. Modern tests for preview2 live at `crates/wasi/tests/*.rs`. * Wasm tests are bucketed based on their filename prefix. For example `preview1_*` is tested in wasi-common and wasmtime-wasi. The `preview2_*` prefix is only tested with wasmtime-wasi, however. * A new `cli_*` prefix is used to execute tests as part of `tests/all/main.rs`. This is a new submodule in `tests/all/cli_tests.rs` which executes these components on the command line. Many old &#34;command&#34; tests were migrated here. * Helper macros are generated to assert that a test suite is run in its entirety. This way if a `preview1_*` test is added it&#39;s asserted to get added to both wasi-common and wasmtime-wasi in the various modes they run tests. Overall this moved a number of tests around and refactored some edges of the tests, but this should not lose any tests (except one that wasn&#39;t actually testing anything). Additionally the hope is that it&#39;s much easier to add tests in the future. The process is to add a new file in `crates/test-programs/src/bin/*.rs` named appropriately. For example a preview2 executable is `preview2_*` and a CLI tests is `cli_*`. When building the test suite an error is generated in the appropriate module then of &#34;please write a test here&#34;, and then a test is written in the same manner as the other tests in the module. * Remove no-longer-needed fetches prtest:full * I&#39;m worried wasi is running low on semicolons * Add the WASI target in all CI actions * Add unknown-unknown target on all CI builders too * Fix building test artifacts under miri Need to avoid wrappers for these cross-compiled targets * Break circular dependency for packaging Don&#39;t use the workspace dep for `wasmtime-wasi` since it injects a version, instead use a `path = &#39;..&#39;` dependency to fool Cargo into dropping the dependency during the package phase. * Fix some merge conflicts with tests * Fix rebase for new tests * Remove stray comment * Fix some flaky tests * Fix network tests in synchronous mode This commit is an attempt to fix some networking tests in synchronous mode in our test suite. Currently networking tests don&#39;t actually run in synchronous mode on CI which is why no failures have been surfaced yet, but the refactoring in #7182 is going to start doing this. Currently the `udp_sample_application.rs` test blocks infinitely in synchronous mode for me locally, most of the time. This appears to be an interaction between how Tokio handles readiness and how we&#39;re entering the event loop. We&#39;re effectively entering the Tokio event loop with a future that&#39;s always ready which ends up starving Tokio of otherwise performing its background work such as updating flags for readiness of reading/writing. The fix here is to add a yield at the start of an `in_tokio` block which is used in synchronous mode. This is a kludge fix but the intention is to enable Tokio to have a chance to update readiness flags and process events from epoll/kqueue/etc. An additional fix to this issue is WebAssembly/wasi-sockets#64 where the test is waiting on `READABLE` or `WRITABLE`, but in this specific case it should only wait on `READABLE`. If it waited on just this then that would also fix this issue. Nevertheless having a `yield_now` is expected to have little-to-no overhead and otherwise fix this edge case of an always-ready future. * Fix passing empty arguments on the CLI * Add another blocking accept * Update crates/test-programs/src/bin/api_proxy.rs Co-authored-by: Trevor Elliott &lt;awesomelyawesome@gmail.com&gt; --------- Co-authored-by: Trevor Elliott &lt;awesomelyawesome@gmail.com&gt;
1 year ago
- name: Install the WASI target
shell: bash
run: rustup target add wasm32-wasip1 wasm32-unknown-unknown