A small amount of cleanup for the top-level of this repository, where
the `installer` directory just had one misc file for the Windows
installer which we can put elsewhere inside of `ci`, another misc folder
but one that already has a few files in it.
This commit migrates wasmtime's CI infrastructure from Azure Pipelines
to Github Actions. Using Github Actions has a few benefits over other
offerings:
* Being natively integrated with Github means that there's no degree of
user account configuration or access control management, it's all
inherent via already existing Github permissions.
* Github Actions gives 20 parallel builders instead of Azure's 10 by
default, which is a nice boost to have!
Overall I've found Github Actions to feel a bit cleaner than Azure
Pipelines as well. Subjectively I've found the configuration to be more
readable and more pleasant to work with, although they're both just as
"powerful" I think. Additionally Github Actions has been pretty solid in
my own personal testing for a number of other projects.
The main trickiness with wasmtime's CI is the rolling `dev` release of
the master branch as well as binary releases for tags. Github Actions
doesn't have quite as much built in functionality as Azure Pipelines,
but Github Actions does have a nice feature where you can define the
code for an action locally rather than only using built-in actions.
This migration adds three local actions with some associated JS code to
run the action (currently it looks like it basically requires JS)
* An `install-rust` action papers over the gotchas about installing
Rust, allowing Rust installation to be a one-liner in the configuration.
* A `binary-compatible-builds` action allows easily configuring the
wheels and the binaries to be "more binary compatible" and handles
things like compilation flags on OSX and Windows while handling the
`centos:6` container on Linux.
* The `github-release` action is the logic using the `@actions/github`
JS package to orchestrate the custom way we manage rolling releases,
ensuring that a new release is made for the master branch under `dev`
(deleting the previous tag/release ahead of time) and then also
manages tagged releases by uploading them there.
I'm hoping that most of the inline actions here will largely go away.
For example `install-rust` should be simply `rustup update $toolchain`
once various environment issues are fixed on Github Actions runner
images. Additionally `github-release` will ideally migrate to something
like https://github.com/actions/create-release or similar once it has
enough functionality. I'm also hoping that the maintenance in the
meantime of these actions is pretty low-cost, but if it becomes an issue
we can look into other solutions!
This commit adds initial support for [WebAssembly Interface
Types][proposal] to wasmtime. This is all intended to be quite
experimental, so experimental in fact that even the name of the
[proposal] is still in flux. (this has otherwise been known as "host
bindings" or "webidl bindings" or "wasm bindings").
The goal of this commit is to start adding support the wasmtime set of
crates for WebAssembly Interface Types. A new `wasmtime-interface-types`
crate has been added with very basic support for dynamically invoking
and inspecting the various bindings of a module. This is in turn powered
by the `wasm-webidl-bindings` crate which is shared with the
`wasm-bindgen` CLI tool as a producer of this section.
Currently the only integration in `wasmtime`-the-binary itself is that
when passed the `--invoke` argument the CLI will now attempt to invoke
the target function with arguments as parsed from the command line
itself. For example if you export a function like:
fn render(&str) -> String
Then passing `--invoke render` will require one argument on the command
line, which is the first argument as a string, and the return value is
printed to the console. This differs from today's interpretation of
`--invoke` where it is a failure if the invoked function takes more than
one argument and the return values are currently ignored.
This is intended to also be the basis of embedding wasmtime in other
contexts which also want to consume WebAssembly interface types. A
Python extension is also added to this repository which implements the
`wasmtime` package on PyPI. This Python extension is intended to make it
as easy as `pip3 install wasmtime` to load a WebAssembly file with
WebAssembly Interface Types into Python. Extensions for other languages
is of course possible as well!
One of the major missing pieces from this is handling imported functions
with interface bindings. Currently the embedding support doesn't have
much ability to support handling imports ergonomically, so it's intended
that this will be included in a follow-up patch.
[proposal]: https://github.com/webassembly/webidl-bindings
Co-authored-by: Yury Delendik <ydelendik@mozilla.com>
Currently our Linux binaries aren't quite as portable as they otherwise could
be. There's two primary reasons for this, and one of them is that the binary is
produced in a relatively recent Linux distribution (Ubuntu 16.04) which means
it has a relatively recent requirement in terms of glibc versions. On
OSX/Windows we can set some flags to rely on older libc implementations, but on
Linux we have to actually build against an older version.
This commit switches the container for the build to CentOS 6 instead of the
default Ubuntu 16.04. The main trick here is also finding a C++11-capable
compiler to compile wabt. Turns out though there's a helpful tutorial for this
at https://edwards.sdsu.edu/research/c11-on-centos-6/ and it was as easy as
installing a few packages.
The second portability concern of our Linux binaries is that they link
dynamically to `libstdc++.so` which isn't always installed on target systems,
or even if it is it may be too old or have a different ABI. This is solved by
statically linking to `libstdc++.a` in the build on Azure by doing a bit of
trickery with libraries and what's available.
After these results the glibc requirements drops from 2.18 (released in 2013)
to 2.6 (released in 2007) and avoids the need for users to have libstdc++.so
installed. We may eventually want to investigate fully-static musl binaries,
but finding a musl compiler for C++ is something I'm not that good at, so I
figure this is probably good enough for now.
* Refactor Azure Pipelines config and tweak releases
* Extract out doc/rustfmt jobs into their own separate builders. Helps
avoiding having to skip tons of steps and can get failing results more
quickly.
* Extract out Rust installation logic to a dedicated template.
* Separate out the build/test job matrices, where one matrix runs tests
and another runs a full build
* Refactor release directory structure to follow a convention where
`foo.tar.gz` extracts to a folder called `foo` and follow unix-like
conventions and copy over the license/readme files into the release
tarballs.
* Swap order of build/test
Alex Crichton
Dan Gohman
Artur Jamro
Yury Delendik
Till Schneidereit