On WebAssembly it is possible to grow the heap with the memory.grow
instruction. This commit implements this feature and with that also
removes the -heap-size flag that was reportedly broken (I haven't
verified that). This should make it easier to use TinyGo for
WebAssembly, where there was no good reason to use a fixed heap size.
This commit has no effect on baremetal targets with optimizations
enabled.
The only architecture that actually needs special support for scanning
the stack is WebAssembly. All others allow raw access to the stack with
a small bit of assembly. Therefore, don't manually keep track of all
these objects on the stack manually and instead just use conservative
stack scanning.
This results in a massive code size decrease in the affected targets
(only tested linux/amd64 for code size) - sometimes around 33%. It also
allows for future improvements such as using proper stackful goroutines.
This commit allows debugging like the following:
GOARCH=arm tinygo gdb ./testdata/alias.go
This can be very useful to debug issues on a different instruction set
architecture but still on a host system.
I tested the following 7 configurations to make sure it works and I
didn't break anything:
GOOS=amd64
GOOS=386
GOOS=arm
GOOS=arm64
tinygo gdb -target=hifive1-qemu
tinygo gdb -target=cortex-m-qemu
tinygo gdb -target=microbit
To avoid breaking this, make sure we actually test x86-32 (aka i386 aka
GOARCH=386) support in CI.
Also remove the now-unnecessary binutils-arm-none-eabi package to speed
up CI a bit.
By modifying the linker script a bit and adding the NRO0 header directly
in the assembly, it's possible to craft an ELF file that can be
converted straight to a binary (using objcopy or similar) that is a NRO
file. This avoids custom code for NRO files or an extra build step.
With another change, .nro files are recognized by TinyGo so that this
will create a ready-to-run NRO file:
tinygo build -o test.nro -target=nintendoswitch examples/serial
This is a big change that will determine the stack size for many
goroutines automatically. Functions that aren't recursive and don't call
function pointers can in many cases have an automatically determined
worst case stack size. This is useful, as the stack size is usually much
lower than the previous hardcoded default of 1024 bytes: somewhere
around 200-500 bytes is common.
A side effect of this change is that the default stack sizes (including
the stack size for other architectures such as AVR) can now be changed
in the config JSON file, making it tunable per application.
For now, this is just an extra flag that can be used to print stack
frame information, but this is intended to provide a way to determine
stack sizes for goroutines at compile time in many cases.
Stack sizes are often somewhere around 350 bytes so are in fact not all
that big usually. Once this can be determined at compile time in many
cases, it is possible to use this information when available and as a
result increase the fallback stack size if the size cannot be determined
at compile time. This should reduce stack overflows while at the same
time reducing RAM consumption in many cases.
Interesting output for testdata/channel.go:
function stack usage (in bytes)
Reset_Handler 332
.Lcommand-line-arguments.fastreceiver 220
.Lcommand-line-arguments.fastsender 192
.Lcommand-line-arguments.iterator 192
.Lcommand-line-arguments.main$1 184
.Lcommand-line-arguments.main$2 200
.Lcommand-line-arguments.main$3 200
.Lcommand-line-arguments.main$4 328
.Lcommand-line-arguments.receive 176
.Lcommand-line-arguments.selectDeadlock 72
.Lcommand-line-arguments.selectNoOp 72
.Lcommand-line-arguments.send 184
.Lcommand-line-arguments.sendComplex 192
.Lcommand-line-arguments.sender 192
.Lruntime.run$1 548
This shows that the stack size (if these numbers are correct) can in
fact be determined automatically in many cases, especially for small
goroutines. One of the great things about Go is lightweight goroutines,
and reducing stack sizes is very important to make goroutines
lightweight on microcontrollers.
Debug information is often useful and there is no reason to include it
for Go code but not for C code. Also, disabling debug information should
disable it entirely, not just for Go code.
Previously we used --sysroot to set the sysroot explicitly.
Unfortunately, this flag is not used directly by Clang to set the
include path (<sysroot>/include) but is instead interpreted by the
toolchain code. This means that even when the toolchain is explicitly
set (using the --sysroot parameter), it may still decide to use a
different include path such as <sysroot>/usr/include (such as on
baremetal aarch64).
This commit uses the Clang-internal -internal-isystem flag which sets
the include directory directly (as a system include path). This should
be more robust.
The reason the --sysroot parameter has so far worked is that all
existing targets happened to add <sysroot>/include as an include path.
The relevant Clang code is here:
https://github.com/llvm/llvm-project/blob/release/9.x/clang/lib/Driver/Driver.cpp#L4693-L4739
So far, RISC-V is handled by RISCVToolchain, Cortex-M targets by
BareMetal (which seems to be specific to ARM unlike what the name says)
and aarch64 fell back to Generic_ELF.
This is necessary for better CGo support on bare metal. Existing
libraries expect to be able to include parts of libc and expect to be
able to link to those symbols.
Because with this all targets have a working libc, it is now possible to
add tests to check that a libc in fact works basically.
Not all parts of picolibc are included, such as the math or stdio parts.
These should be added later, when needed.
This commit also avoids the need for the custom memcpy/memset/memcmp
symbols that are sometimes emitted by LLVM. The C library will take care
of that.
This allows packages other than the compiler to know (from a single
source of truth) which implemenation is used for Go func values.
This refactor is necessary to be able to move the Optimize function to
the transform package.
Use the cross compiling toolchains for compiling/linking. This fixes CGo
support, and therefore allows CGo to be used when cross compiling to
Linux on a different architecture.
This commit also removes some redundant testing code.
Add a target for the Adafruit Circuit Playground Bluefruit, which is
based on the nRF52840. Adds the necessary code for the machine
package and the json and linker script files in the targets directory.
The machine package code is based on board_circuitplay_express.go,
with modifications made by consulting the wiring diagram on the
adafruit website here:
https://learn.adafruit.com/adafruit-circuit-playground-bluefruit/downloads
Also adds support to the uf2 conversion packacge to set the familyID
field. The Circuit Playground Bluefruit firmware rejects uf2 files
without the family id set to 0xADA52840 (and without the flag specifying
that the family id is present).
This code is required by transformation passes which are being moved
into a separate package, but is too complicated to simply copy.
Therefore, I decided to move them into a new package.
This flag is overloaded. It can be used in two ways:
* Choosing the flash method to use (openocd, msd, command).
* Choosing the OpenOCD programmer name.
For example, you can use one of these to use OpenOCD instead of the
mass-storage device programmer:
tinygo flash -target=microbit -programmer=openocd
tinygo flash -target=microbit -programmer=cmsis-dap
This is a large commit that moves all code directly related to
compiling/linking into a new builder package. This has a number of
advantages:
* It cleanly separates the API between the command line and the full
compilation (with a very small API surface).
* When the compiler finally compiles one package at a time (instead of
everything at once as it does now), something will have to invoke it
once per package. This builder package will be the natural place to
do that, and also be the place where the whole process can be
parallelized.
* It allows the TinyGo compiler to be used as a package. A client can
simply import the builder package and compile code using it.
As part of this refactor, the following additional things changed:
* Exported symbols have been made unexported when they weren't needed.
* The compilation target has been moved into the compileopts.Options
struct. This is done because the target really is just another
compiler option, and the API is simplified by moving it in there.
* The moveFile function has been duplicated. It does not really belong
in the builder API but is used both by the builder and the command
line. Moving it into a separate package didn't seem useful either
for what is essentially an utility function.
* Some doc strings have been improved.
Some future changes/refactors I'd like to make after this commit:
* Clean up the API between the builder and the compiler package.
* Perhaps move the test files (in testdata/) into the builder package.
* Perhaps move the loader package into the builder package.
Setting the linker script as one property (instead of as part of the
generic ldflags property) allows it to be overriden.
This is important for the SoftDevice on Nordic chips, because the
SoftDevice takes up a fixed part of the flash/RAM and the application
must be flashed at a different position. With this linkerscript option,
it is possible to create (for example) a pca10040-s132v6 that overrides
the default linker script.
Move most of the logic of determining which compiler configuration to
use (such as GOOS/GOARCH, build tags, whether to include debug symbols,
panic strategy, etc.) into the compileopts package. This makes it a
single source of truth for anything related to compiler configuration.
It has a few advantages:
* The compile configuration is independent of the compiler package.
This makes it possible to move optimization passes out of the
compiler, as they don't rely on compiler.Config anymore.
* There is only one place to look if an incorrect compile option is
used.
* The compileopts provides some resistance against unintentionally
picking the wrong option, such as with c.selectGC() vs c.GC() in the
compiler.
* It is now a lot easier to change compile options, as most options
are getters now.
Fauchon
Ayke van Laethem
Jacques Supcik
Takeshi Yoneda
Jacques Supcik
Lucas Teske
Yannis Huber
Yannis Huber
cebernardi
Jaden Weiss
Michael Matloob