This was triggered by the following code:
var smallPrimesProduct = new(big.Int).SetUint64(16294579238595022365)
It is part of the new TinyGo version of the crypto/rand package.
The markExternal function is used when a global (function or global
variable) is somehow run at runtime. All the other globals it refers to
are from then on no longer known at compile time, so can't be used by
the interp package anymore.
This can also include inline assembly. While it is possible to modify
globals that way, it is only possible to modify exported globals:
similar to calling an undefined function (in C for example).
The interp package is in many cases able to execute map functions in the
runtime directly. This is probably slower than adding special support
for them in the interp package and also doesn't cover all cases (most
importantly, map keys that contain pointers) but removing this code also
removes a large amount of code that needs to be maintained and is
susceptible to hard-to-find bugs.
As a side effect, this resulted in different output of the
testdata/map.go test because the test relied on the existing iteration
order of TinyGo maps. I've updated the test to not rely on this test,
making the output compatible with what the Go toolchain would output.
I've discovered a bug in the implementation of the PHI instruction in
the interp package. This commit fixes the bug.
I've found this issue while investigating an issue with maps after
running interp per package.
A switch statement is not normally emitted by the compiler package, but
LLVM function passes may convert a series of if/else pairs to a switch
statement. A future change will run function passes in the package
compile phase, so the interp package (which is also run after all
modules are merged together) will need to deal with these new switch
statements.
Sometimes, LLVM may rename named structs when merging modules.
Therefore, we can't rely on typecodeID structs to retain their struct
names.
This commit changes the interface lowering pass to not rely on these
names. The interp package does however still rely on this name, but I
hope to fix that in the future.
This results in a significant speedup in some cases. For example, this
runs over twice as fast with a warm cache:
tinygo build -o test.elf ./testdata/stdlib.go
This should help a lot with edit-compile-test cycles, that typically
only modify a single package.
This required some changes to the interp package to deal with globals
created in a previous run of the interp package and to deal with
external globals (that can't be loaded from or stored to).
This commit replaces a number of panics with returning an error value as
a result of changing the toLLVMValue method signature. This should make
it easier to diagnose issues.
This matches the main Go implementation and (among others) fixes a
compatibility issue with the encoding/json package. The encoding/json
package compares reflect.Type variables against nil, which does not work
as long as reflect.Type is of integer type.
This also adds a reflect.RawType() function (like reflect.Type()) that
makes it easier to avoid working with interfaces in the runtime package.
It is internal only, but exported to let the runtime package use it.
This change introduces a small code size increase when working with the
reflect package, but I've tried to keep it to a minimum. Most programs
that don't make extensive use of the reflect package (and don't use
package like fmt) should not be impacted by this.
This is necessary so that when reflect.Type is converted from a concrete
type to an interface type, the errors package can still be interpreted.
Without this change, basically every program would grow in size by a few
bytes.
The errors package has a call like this in the package initializer. This
commit adds support for running it at compile time, avoiding the call at
runtime.
This doesn't always help (the call is already optimized away in many
small programs) but it does help to shave off some binary size in larger
programs. Perhaps more importantly, it will avoid a penalty in code size
when the reflect package will convert reflect.Type from a regular type
to an interface type.
Previously there was code to avoid impossible type asserts but it wasn't
great and in fact was too aggressive when combined with reflection.
This commit improves this by checking all types that exist in the
program that may appear in an interface (even struct fields and the
like) but without creating runtime.typecodeID objects with the type
assert. This has two advantages:
* As mentioned, it optimizes impossible type asserts away.
* It allows methods on types that were only asserted on (in
runtime.typeAssert) but never used in an interface to be optimized
away using GlobalDCE. This may have a cascading effect so that other
parts of the code can be further optimized.
This sometimes massively improves code size and mostly negates the code
size regression of the previous commit.
This distinction was useful before when reflect wasn't properly
supported. Back then it made sense to only include method sets that were
actually used in an interface. But now that it is possible to get to
other values (for example, by extracting fields from structs) and it is
possible to turn them back into interfaces, it is necessary to preserve
all method sets that can possibly be used in the program in a type
assert, interface assert or interface method call.
In the future, this logic will need to be revisited again when
reflect.New or reflect.Zero gets implemented.
Code size increases a bit in some cases, but usually in a very limited
way (except for one outlier in the drivers smoke tests). The next commit
will improve the situation significantly.
GetElementPtr would not work on values that weren't pointers. Because
fixed addresses (often used in memory-mapped I/O) are integers rather
than pointers in interp, it would return an error.
This resulted in the teensy40 target not compiling correctly since the
interp package rewrite. This commit should fix that.
During a run of interp, some memory (for example, memory allocated
through runtime.alloc) may not have a known LLVM type. This memory is
alllocated by creating an i8 array.
This does not necessarily work, as i8 has no alignment requirements
while the allocated object may have allocation requirements. Therefore,
the resulting global may have an alignment that is too loose.
This works on some microcontrollers but notably does not work on a
Cortex-M0 or Cortex-M0+, as all load/store operations must be aligned.
This commit fixes this by setting the alignment of untyped memory to the
maximum alignment. The determination of "maximum alignment" is not
great but should get the job done on most architectures.
For a full explanation, see interp/README.md. In short, this rewrite is
a redesign of the partial evaluator which improves it over the previous
partial evaluator. The main functional difference is that when
interpreting a function, the interpretation can be rolled back when an
unsupported instruction is encountered (for example, an actual unknown
instruction or a branch on a value that's only known at runtime). This
also means that it is no longer necessary to scan functions to see
whether they can be interpreted: instead, this package now just tries to
interpret it and reverts when it can't go further.
This new design has several benefits:
* Most errors coming from the interp package are avoided, as it can
simply skip the code it can't handle. This has long been an issue.
* The memory model has been improved, which means some packages now
pass all tests that previously didn't pass them.
* Because of a better design, it is in fact a bit faster than the
previous version.
This means the following packages now pass tests with `tinygo test`:
* hash/adler32: previously it would hang in an infinite loop
* math/cmplx: previously it resulted in errors
This also means that the math/big package can be imported. It would
previously fail with a "interp: branch on a non-constant" error.
This can be useful to test improvements in LLVM master and to make it
possible to support LLVM 11 for the most part already before the next
release. That also allows catching LLVM bugs early to fix them upstream.
Note that tests do not yet pass for this LLVM version, but the TinyGo
compiler can be built with the binaries from apt.llvm.org (at the time
of making this commit).
A number of functions now return errors instead of panicking, which
should help greatly when investigating interp errors. It at least shows
the package responsible for it.
These methods do some unsafe pointer casting but can be assumed to not
have significant side effects. Simply call these functions at runtime
instead of compile time.
This is a partial fix for importing image/png.
This option was broken for a long time, in part because we didn't test
for it. This commit fixes that and adds a test to make sure it won't
break again unnoticed.
Previously, the compiler used LLVM's shift instructions directly, which have UB whenever the shifts are large or negative.
This commit adds runtime checks for negative shifts, and handles oversized shifts.
This commit teaches the interp scanner that supported interface
operations (type assertions, interface assertions) are supported.
This fixes a problem with math/rand in Go 1.14.
Non-constant type asserts are not yet implemented, but should be
relatively easy to add at a later time. They should result in a clear
error message for now.
This commit removes a panic and replaces it with a proper source
location. The message still isn't very helpful, but at least it points
to a location in the source code.
I'm not very happy with all the `err.Error()` calls, but that's the way
to fit this in a `scanner.Error`. Eventually we should make a
replacement for `scanner.Error` that does proper wrapping of the
original error message.
Some mapassign operations cannot (yet) be done by the interp package.
Implement a fallback mechanism so that these operations can still be
performed at runtime.
This commit replaces most panics in interp/frame.go and interp/scan.go
with real error messages. The remaining ones are panics that should not
happen when working with valid IR.
This kind of code might be generated by the switch implementation of
func values. The func value is represented as a ptrtoint, and before
calling it, it is compared against 0.
This commit improves error reporting in several ways:
* Location information is read from the intruction that causes the
error, as far as that's available.
* The package that is being interpreted is included in the error
message. This may be the most useful part of the improvements.
* The hashmap update intrinsics now doesn't panic, instead it logs a
clear error (with location information, as in the above two bullet
points).
This is possible thanks to improvements in LLVM 9. This means that after
this change, TinyGo will depend on LLVM 9.
This is really just a simple workaround. When such an instruction is
encountered, it will just fall back to marking the entire function as
having side effects. Ideally it should trace all affected instructions
and check if they would have any side effects, but this at least fixes a
number of compile errors.
This commit gets the following packages to compile:
* context
* database/sql/driver
* image/jpeg
* image/png
Declarations would enter an infinite loop when trying to loop over basic
blocks. That was probably an undefined operation, but still somehow
didn't crash the compiler.
Make sure that scanning declarations works as expected.
This implements the copy() built-in function. It may not work in all
cases, but should work in most cases.
This commit gets the following 3 packages to compile, according to
tinygo-site/imports/main.go:
* encoding/base32
* encoding/base64
* encoding/pem (was blocked by encoding/base64)
The IR builder does not appear to fold comparisons of constant inttoptr
instructions to const null pointer. So do it manually during
interpretatin, as a somewhat ugly hack.
This fixes https://github.com/tinygo-org/tinygo/issues/363
Ayke van Laethem
Jaden Weiss
Jaden Weiss