This is done at a later time anyway, so doesn't need to be done in the
cgo package. In fact, _not_ doing it there makes it easier to print
correct relative packages.
Go code might sometimes want to use preprocessor macros that were passed
on the command line. This wasn't working before and resulted in the
following error:
internal error: could not find file where macro is defined
This is now supported, though location information isn't available
(which makes sense: the command line is not a file).
I had to use the `clang_tokenize` API for this and reconstruct the
original source location. Apparently this is the only way to do it:
https://stackoverflow.com/a/19074846/559350
In the future we could consider replacing our own tokenization with the
tokenizer that's built into Clang directly. This should reduce the
possibility of bugs a bit.
This commit adds support for LLVM 16 and switches to it by default. That
means three LLVM versions are supported at the same time: LLVM 14, 15,
and 16.
This commit includes work by QuLogic:
* Part of this work was based on a PR by QuLogic:
https://github.com/tinygo-org/tinygo/pull/3649
But I also had parts of this already implemented in an old branch I
already made for LLVM 16.
* QuLogic also provided a CGo fix here, which is also incorporated in
this commit:
https://github.com/tinygo-org/tinygo/pull/3869
The difference with the original PR by QuLogic is that this commit is
more complete:
* It switches to LLVM 16 by default.
* It updates some things to also make it work with a self-built LLVM.
* It fixes the CGo bug in a slightly different way, and also fixes
another one not included in the original PR.
* It does not keep compiler tests passing on older LLVM versions. I
have found this to be quite burdensome and therefore don't generally
do this - the smoke tests should hopefully catch most regressions.
Anonymous enums (often used in typedefs) triggered a problem that was
already solved for structs but wasn't yet solved for enums. So this
patch generalizes the code to work for both structs and enums, and adds
testing for both.
This is a large refactor of the cgo package. It should fix a number of
smaller problems and be a bit more strict (like upstream CGo): it for
example requires every Go file in a package to include the header files
it needs instead of piggybacking on imports in earlier files.
The main benefit is that it should be a bit more maintainable and easier
to add new features in the future (like static functions).
This breaks the tinygo.org/x/bluetooth package, which should be updated
before this change lands.
Updating them to libclang-13-dev was a good change, but we can go even
further:
* The suggestion didn't apply to MacOS.
* The suggestion would need to be updated with every LLVM release,
which is a maintenance burden.
* The suggestion is wrong when compiling with `-tags=llvm12` for
example to choose a different LLVM version.
Therefore, link to the build documentation instead.
Previously, libclang was run on each fragment (import "C") separately.
However, in regular Go it's possible for later fragments to refer to
types in earlier fragments so they must have been parsed as one.
This commit changes the behavior to run only one C parser invocation for
each Go file.
This is just a first step. It's not complete, but it gets some real
world C code to parse.
This signature, from the ESP-IDF:
esp_err_t esp_wifi_get_mac(wifi_interface_t ifx, uint8_t mac[6]);
Was previously converted to something like this (pseudocode):
C.esp_err_t esp_wifi_get_mac(ifx C.wifi_interface_t, mac [6]uint8)
But this is not correct. C array parameters will decay. The array is
passed by reference instead of by value. Instead, this would be the
correct signature:
C.esp_err_t esp_wifi_get_mac(ifx C.wifi_interface_t, mac *uint8)
So that it can be called like this (using CGo):
var mac [6]byte
errCode := C.esp_wifi_get_mac(C.ESP_IF_WIFI_AP, &mac[0])
This stores the result in the 6-element array mac.
This is a loose collection of small fixes flagged by staticcheck:
- dead code
- regexp expressions not using backticks (`foobar` / "foobar")
- redundant types of slice and map initializers
- misc other fixes
Not all of these seem very useful to me, but in particular dead code is
nice to fix. I've fixed them all just so that if there are problems,
they aren't hidden in the noise of less useful issues.
This commit switches from the previous behavior of compiling the whole
program at once, to compiling every package in parallel and linking the
LLVM bitcode files together for further whole-program optimization.
This is a small performance win, but it has several advantages in the
future:
- There are many more things that can be done per package in parallel,
avoiding the bottleneck at the end of the compiler phase. This
should speed up the compiler futher.
- This change is a necessary step towards a non-LTO build mode for
fast incremental builds that only rebuild the changed package, when
compiler speed is more important than binary size.
- This change refactors the compiler in such a way that it will be
easier to inspect the IR for one package only. Inspecting this IR
will be very helpful for compiler developers.
This doesn't yet add support for actually making use of variadic
functions, but at least allows (unintended) variadic functions like the
following to work:
void foo();
Previously it would return a `*scanner.Error`, which is not supported in
the error printer of the main package. This can easily be fixed by
making it a regular object (instead of a pointer).
This commit also adds a bit of version independence, in particular for
external commands. It also adds the LLVM version to the `tinygo version`
command, which might help while debugging.
This implementation is still very limited but provides a base to build
upon. Limitations:
* CGO_CFLAGS etc is not taken into account.
* These CFLAGS are not used in C files compiled with the package.
* Other flags (CPPFLAGS, LDFAGS, ...) are not yet implemented.
This commit adds tests for CGo preprocessing. There are various errors
that can be reported while preprocessing, and they should integrate well
with the compiler (including accurate source location tracking).
Also allow CGo preprocessing to continue after Clang encountered an
error, for a better view of what happened.
Instead of putting the magic in the AST, generate regular accessor
methods. This avoids a number of special cases in the compiler, and
avoids missing any of them.
The resulting union accesses are somewhat clunkier to use, but the
compiler implementation has far less coupling between the CGo
implementation and the IR generator.
Not all enums may be used as a type anywhere, which was previously the
only way to include an enum in the AST. This commit makes sure all enums
are included.
This commit renames reserved field names like `type` to `_type`, and in
turn renames those fields as well (recursively). This avoids name
clashes when a C struct contains a field named `type`, which is a
reserved keyword in Go.
For some details, see:
https://golang.org/cmd/cgo/#hdr-Go_references_to_C
Previously it was just a combination of heuristics to try to fit a
constant in an *ast.BasicLit. For more complex expressions, this is not
enough.
This change also introduces proper syntax error with locations, if
parsing a constant failed. For example, this will print a real error
message with source location:
#define FOO 5)
This commit improves diagnostics in a few ways:
* All panics apart from panics with no (easy) recovery are converted to
regular errors with source location.
* Some errors were improved slightly to give more information. For
example, show the libclang type kind as a string instead of a
number.
* Improve source location by respecting line directives in the C
preprocessor.
* Refactor to unify error handling between libclang diagnostics and
failures to parse the libclang AST.
Types used in a program may not be implemented. Print a nice error
message explaining the situation, instead of just prepending C. to the
type spelling (and hoping the user knows what that undefined reference
means).
Enum types are implemented as named types (with possible accompanying
typedefs as type aliases). The constants inside the enums are treated as
Go constants like in the Go toolchain.
This is a big commit that does a few things:
* It moves CGo processing into a separate package. It never really
belonged in the loader package, and certainly not now that the
loader package may be refactored into a driver package.
* It adds support for multiple CGo files (files that import package
"C") in a single package. Previously, this led to multiple
definition errors in the Go typecheck phase because certain C
symbols were defined multiple times in all the files. Now it
generates a new fake AST that defines these, to avoid multiple
definition errors.
* It improves debug info in a few edge cases that are probably not
relevant outside of bugs in cgo itself.
Only try to convert the C symbols to their Go equivalents that are
actually referenced by the Go code with C.<somesymbol>. This avoids
having to support all possible C types, which is difficult because of
oddities like `typedef void` or `__builtin_va_list`. Especially
__builtin_va_list, which varies between targets.
These types (called elaborated types in C) are used as part of linked
lists, among others.
This is part an extra feature (to be compatible with CGo C.struct_
types) and part a bugfix: linked lists would result in endless recursion
leading to a stack overflow.
Sometimes when a GC happens while processing a C fragment with libclang,
a pointer-typed integer with value 0x1 ends up on the Go stack and the
GC will trip over it. This commit changes the offending struct type to
be uintptr_t instead of void*.
See https://go-review.googlesource.com/c/go/+/66332 for a similar
change.
Unions are somewhat hard to implement in Go because they are not a
native type. But it is actually possible with some compiler magic.
This commit inserts a special "C union" field at the start of a struct
to indicate that it is a union. As such a field cannot be written
directly in Go, this is a useful to distinguish structs and unions.
This makes CGo-emitted diagnostics very similar to regular errors
emitted while parsing/typechecking a package.
It's not complete, but after introducing some errors in testdata/cgo,
this is the resulting output:
# ./testdata/cgo/
testdata/cgo/main.h:18:11: error: a parameter list without types is only allowed in a function definition
testdata/cgo/main.go:5:10: note: in file included from testdata/cgo/main.go!cgo.c:2:
testdata/cgo/main.go:6:19: error: expected identifier or '('
Previously, this was the output:
/home/ayke/src/github.com/tinygo-org/tinygo/testdata/cgo/main.h:18:11: error: a parameter list without types is only allowed in a function definition
cgo-fake.c:3:19: error: expected identifier or '('
# ./testdata/cgo/
cgo: libclang cannot parse fragment
Ayke van Laethem
ZauberNerd