You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
712 lines
30 KiB
712 lines
30 KiB
// -*- mode:doc; -*-
|
|
// vim: set syntax=asciidoc:
|
|
|
|
== Contributing to Buildroot
|
|
|
|
There are many ways in which you can contribute to Buildroot: analyzing
|
|
and fixing bugs, analyzing and fixing package build failures detected by
|
|
the autobuilders, testing and reviewing patches sent by other
|
|
developers, working on the items in our TODO list and sending your own
|
|
improvements to Buildroot or its manual. The following sections give a
|
|
little more detail on each of these items.
|
|
|
|
If you are interested in contributing to Buildroot, the first thing you
|
|
should do is to subscribe to the Buildroot mailing list. This list is
|
|
the main way of interacting with other Buildroot developers and to send
|
|
contributions to. If you aren't subscribed yet, then refer to
|
|
xref:community-resources[] for the subscription link.
|
|
|
|
If you are going to touch the code, it is highly recommended to use a
|
|
git repository of Buildroot, rather than starting from an extracted
|
|
source code tarball. Git is the easiest way to develop from and directly
|
|
send your patches to the mailing list. Refer to xref:getting-buildroot[]
|
|
for more information on obtaining a Buildroot git tree.
|
|
|
|
=== Reproducing, analyzing and fixing bugs
|
|
|
|
A first way of contributing is to have a look at the open bug reports in
|
|
the https://bugs.buildroot.org/buglist.cgi?product=buildroot[Buildroot bug
|
|
tracker]. As we strive to keep the bug count as small as possible, all
|
|
help in reproducing, analyzing and fixing reported bugs is more than
|
|
welcome. Don't hesitate to add a comment to bug reports reporting your
|
|
findings, even if you don't yet see the full picture.
|
|
|
|
=== Analyzing and fixing autobuild failures
|
|
|
|
The Buildroot autobuilders are a set of build machines that continuously
|
|
run Buildroot builds based on random configurations. This is done for
|
|
all architectures supported by Buildroot, with various toolchains, and
|
|
with a random selection of packages. With the large commit activity on
|
|
Buildroot, these autobuilders are a great help in detecting problems
|
|
very early after commit.
|
|
|
|
All build results are available at http://autobuild.buildroot.org[],
|
|
statistics are at http://autobuild.buildroot.org/stats.php[]. Every day,
|
|
an overview of all failed packages is sent to the mailing list.
|
|
|
|
Detecting problems is great, but obviously these problems have to be
|
|
fixed as well. Your contribution is very welcome here! There are
|
|
basically two things that can be done:
|
|
|
|
- Analyzing the problems. The daily summary mails do not contain details
|
|
about the actual failures: in order to see what's going on you have to
|
|
open the build log and check the last output. Having someone doing
|
|
this for all packages in the mail is very useful for other developers,
|
|
as they can make a quick initial analysis based on this output alone.
|
|
|
|
- Fixing a problem. When fixing autobuild failures, you should follow
|
|
these steps:
|
|
. Check if you can reproduce the problem by building with the same
|
|
configuration. You can do this manually, or use the
|
|
http://git.buildroot.org/buildroot-test/tree/utils/br-reproduce-build[br-reproduce-build]
|
|
script that will automatically clone a Buildroot git repository,
|
|
checkout the correct revision, download and set the right
|
|
configuration, and start the build.
|
|
. Analyze the problem and create a fix.
|
|
. Verify that the problem is really fixed by starting from a clean
|
|
Buildroot tree and only applying your fix.
|
|
. Send the fix to the Buildroot mailing list (see
|
|
xref:submitting-patches[]). In case you created a patch against the
|
|
package sources, you should also send the patch upstream so that the
|
|
problem will be fixed in a later release, and the patch in Buildroot
|
|
can be removed.
|
|
In the commit message of a patch fixing an autobuild failure, add a
|
|
reference to the build result directory, as follows:
|
|
---------------------
|
|
Fixes: http://autobuild.buildroot.org/results/51000a9d4656afe9e0ea6f07b9f8ed374c2e4069
|
|
---------------------
|
|
|
|
=== Reviewing and testing patches
|
|
|
|
With the amount of patches sent to the mailing list each day, the
|
|
maintainer has a very hard job to judge which patches are ready to apply
|
|
and which ones aren't. Contributors can greatly help here by reviewing
|
|
and testing these patches.
|
|
|
|
In the review process, do not hesitate to respond to patch submissions
|
|
for remarks, suggestions or anything that will help everyone to
|
|
understand the patches and make them better. Please use internet
|
|
style replies in plain text emails when responding to patch
|
|
submissions.
|
|
|
|
To indicate approval of a patch, there are three formal tags that keep
|
|
track of this approval. To add your tag to a patch, reply to it with the
|
|
approval tag below the original author's Signed-off-by line. These tags
|
|
will be picked up automatically by patchwork (see
|
|
xref:apply-patches-patchwork[]) and will be part of the commit log when
|
|
the patch is accepted.
|
|
|
|
Tested-by:: Indicates that the patch has been tested successfully.
|
|
You are encouraged to specify what kind of testing you performed
|
|
(compile-test on architecture X and Y, runtime test on target A,
|
|
...). This additional information helps other testers and the
|
|
maintainer.
|
|
|
|
Reviewed-by:: Indicates that you code-reviewed the patch and did your
|
|
best in spotting problems, but you are not sufficiently familiar with
|
|
the area touched to provide an Acked-by tag. This means that there
|
|
may be remaining problems in the patch that would be spotted by
|
|
someone with more experience in that area. Should such problems be
|
|
detected, your Reviewed-by tag remains appropriate and you cannot
|
|
be blamed.
|
|
|
|
Acked-by:: Indicates that you code-reviewed the patch and you are
|
|
familiar enough with the area touched to feel that the patch can be
|
|
committed as-is (no additional changes required). In case it later
|
|
turns out that something is wrong with the patch, your Acked-by could
|
|
be considered inappropriate. The difference between Acked-by and
|
|
Reviewed-by is thus mainly that you are prepared to take the blame on
|
|
Acked patches, but not on Reviewed ones.
|
|
|
|
If you reviewed a patch and have comments on it, you should simply reply
|
|
to the patch stating these comments, without providing a Reviewed-by or
|
|
Acked-by tag. These tags should only be provided if you judge the patch
|
|
to be good as it is.
|
|
|
|
It is important to note that neither Reviewed-by nor Acked-by imply
|
|
that testing has been performed. To indicate that you both reviewed and
|
|
tested the patch, provide two separate tags (Reviewed/Acked-by and
|
|
Tested-by).
|
|
|
|
Note also that _any developer_ can provide Tested/Reviewed/Acked-by
|
|
tags, without exception, and we encourage everyone to do this. Buildroot
|
|
does not have a defined group of _core_ developers, it just so happens
|
|
that some developers are more active than others. The maintainer will
|
|
value tags according to the track record of their submitter. Tags
|
|
provided by a regular contributor will naturally be trusted more than
|
|
tags provided by a newcomer. As you provide tags more regularly, your
|
|
'trustworthiness' (in the eyes of the maintainer) will go up, but _any_
|
|
tag provided is valuable.
|
|
|
|
Buildroot's Patchwork website can be used to pull in patches for testing
|
|
purposes. Please see xref:apply-patches-patchwork[] for more
|
|
information on using Buildroot's Patchwork website to apply patches.
|
|
|
|
[[apply-patches-patchwork]]
|
|
==== Applying Patches from Patchwork
|
|
|
|
The main use of Buildroot's Patchwork website for a developer is for
|
|
pulling in patches into their local git repository for testing
|
|
purposes.
|
|
|
|
When browsing patches in the patchwork management interface, an +mbox+
|
|
link is provided at the top of the page. Copy this link address and
|
|
run the following commands:
|
|
|
|
---------------------
|
|
$ git checkout -b <test-branch-name>
|
|
$ wget -O - <mbox-url> | git am
|
|
---------------------
|
|
|
|
Another option for applying patches is to create a bundle. A bundle is
|
|
a set of patches that you can group together using the patchwork
|
|
interface. Once the bundle is created and the bundle is made public,
|
|
you can copy the +mbox+ link for the bundle and apply the bundle
|
|
using the above commands.
|
|
|
|
|
|
=== Work on items from the TODO list
|
|
|
|
If you want to contribute to Buildroot but don't know where to start,
|
|
and you don't like any of the above topics, you can always work on items
|
|
from the http://elinux.org/Buildroot#Todo_list[Buildroot TODO list].
|
|
Don't hesitate to discuss an item first on the mailing list or on IRC.
|
|
Do edit the wiki to indicate when you start working on an item, so we
|
|
avoid duplicate efforts.
|
|
|
|
[[submitting-patches]]
|
|
=== Submitting patches
|
|
|
|
[NOTE]
|
|
_Please, do not attach patches to bugs, send them to the mailing list
|
|
instead_.
|
|
|
|
If you made some changes to Buildroot and you would like to contribute
|
|
them to the Buildroot project, proceed as follows.
|
|
|
|
==== The formatting of a patch
|
|
|
|
We expect patches to be formatted in a specific way. This is necessary
|
|
to make it easy to review patches, to be able to apply them easily to
|
|
the git repository, to make it easy to find back in the history how
|
|
and why things have changed, and to make it possible to use +git
|
|
bisect+ to locate the origin of a problem.
|
|
|
|
First of all, it is essential that the patch has a good commit
|
|
message. The commit message should start with a separate line with a
|
|
brief summary of the change, prefixed by the area touched by the
|
|
patch. A few examples of good commit titles:
|
|
|
|
* +package/linuxptp: bump version to 2.0+
|
|
|
|
* +configs/imx23evk: bump Linux version to 4.19+
|
|
|
|
* +package/pkg-generic: postpone evaluation of dependency conditions+
|
|
|
|
* +boot/uboot: needs host-{flex,bison}+
|
|
|
|
* +support/testing: add python-ubjson tests+
|
|
|
|
The description that follows the prefix should start with a lower case
|
|
letter (i.e "bump", "needs", "postpone", "add" in the above examples).
|
|
|
|
Second, the body of the commit message should describe _why_ this
|
|
change is needed, and if necessary also give details about _how_ it
|
|
was done. When writing the commit message, think of how the reviewers
|
|
will read it, but also think about how you will read it when you look
|
|
at this change again a few years down the line.
|
|
|
|
Third, the patch itself should do only one change, but do it
|
|
completely. Two unrelated or weakly related changes should usually be
|
|
done in two separate patches. This usually means that a patch affects
|
|
only a single package. If several changes are related, it is often
|
|
still possible to split them up in small patches and apply them in a
|
|
specific order. Small patches make it easier to review, and often
|
|
make it easier to understand afterwards why a change was done.
|
|
However, each patch must be complete. It is not allowed that the
|
|
build is broken when only the first but not the second patch is
|
|
applied. This is necessary to be able to use +git bisect+ afterwards.
|
|
|
|
Of course, while you're doing your development, you're probably going
|
|
back and forth between packages, and certainly not committing things
|
|
immediately in a way that is clean enough for submission. So most
|
|
developers rewrite the history of commits to produce a clean set of
|
|
commits that is appropriate for submission. To do this, you need to
|
|
use _interactive rebasing_. You can learn about it
|
|
https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History[in the Pro
|
|
Git book]. Sometimes, it is even easier to discard you history with
|
|
+git reset --soft origin/master+ and select individual changes with
|
|
+git add -i+ or +git add -p+.
|
|
|
|
Finally, the patch should be signed off. This is done by adding
|
|
+Signed-off-by: Your Real Name <your@email.address>+ at the end of the
|
|
commit message. +git commit -s+ does that for you, if configured
|
|
properly. The +Signed-off-by+ tag means that you publish the patch
|
|
under the Buildroot license (i.e. GPL-2.0+, except for package patches,
|
|
which have the upstream license), and that you are allowed to do so.
|
|
See http://developercertificate.org/[the Developer Certificate of
|
|
Origin] for details.
|
|
|
|
When adding new packages, you should submit every package in a
|
|
separate patch. This patch should have the update to
|
|
+package/Config.in+, the package +Config.in+ file, the +.mk+ file, the
|
|
+.hash+ file, any init script, and all package patches. If the package
|
|
has many sub-options, these are sometimes better added as separate
|
|
follow-up patches. The summary line should be something like
|
|
+<packagename>: new package+. The body of the commit message can be
|
|
empty for simple packages, or it can contain the description of the
|
|
package (like the Config.in help text). If anything special has to be
|
|
done to build the package, this should also be explained explicitly in
|
|
the commit message body.
|
|
|
|
When you bump a package to a new version, you should also submit a
|
|
separate patch for each package. Don't forget to update the +.hash+
|
|
file, or add it if it doesn't exist yet. Also don't forget to check if
|
|
the +_LICENSE+ and +_LICENSE_FILES+ are still valid. The summary line
|
|
should be something like +<packagename>: bump to version <new
|
|
version>+. If the new version only contains security updates compared
|
|
to the existing one, the summary should be +<packagename>: security
|
|
bump to version <new version>+ and the commit message body should show
|
|
the CVE numbers that are fixed. If some package patches can be removed
|
|
in the new version, it should be explained explicitly why they can be
|
|
removed, preferably with the upstream commit ID. Also any other
|
|
required changes should be explained explicitly, like configure
|
|
options that no longer exist or are no longer needed.
|
|
|
|
If you are interested in getting notified of build failures and of
|
|
further changes in the packages you added or modified, please add
|
|
yourself to the DEVELOPERS file. This should be done in the same patch
|
|
creating or modifying the package. See xref:DEVELOPERS[the DEVELOPERS file]
|
|
for more information.
|
|
|
|
Buildroot provides a handy tool to check for common coding style
|
|
mistakes on files you created or modified, called +check-package+ (see
|
|
xref:check-package[] for more information).
|
|
|
|
==== Preparing a patch series
|
|
|
|
Starting from the changes committed in your local git view, _rebase_
|
|
your development branch on top of the upstream tree before generating
|
|
a patch set. To do so, run:
|
|
|
|
---------------------
|
|
$ git fetch --all --tags
|
|
$ git rebase origin/master
|
|
---------------------
|
|
|
|
Now, you are ready to generate then submit your patch set.
|
|
|
|
To generate it, run:
|
|
|
|
---------------------
|
|
$ git format-patch -M -n -s -o outgoing origin/master
|
|
---------------------
|
|
|
|
This will generate patch files in the +outgoing+ subdirectory,
|
|
automatically adding the +Signed-off-by+ line.
|
|
|
|
Once patch files are generated, you can review/edit the commit message
|
|
before submitting them, using your favorite text editor.
|
|
|
|
Buildroot provides a handy tool to know to whom your patches should be
|
|
sent, called +get-developers+ (see xref:DEVELOPERS[] for more
|
|
information). This tool reads your patches and outputs the appropriate
|
|
+git send-email+ command to use:
|
|
|
|
---------------------
|
|
$ ./utils/get-developers outgoing/*
|
|
---------------------
|
|
|
|
Use the output of +get-developers+ to send your patches:
|
|
|
|
---------------------
|
|
$ git send-email --to buildroot@buildroot.org --cc bob --cc alice outgoing/*
|
|
---------------------
|
|
|
|
Alternatively, +get-developers -e+ can be used directly with the
|
|
+--cc-cmd+ argument to +git send-email+ to automatically CC the
|
|
affected developers:
|
|
|
|
---------------------
|
|
$ git send-email --to buildroot@buildroot.org \
|
|
--cc-cmd './utils/get-developers -e' origin/master
|
|
---------------------
|
|
|
|
+git+ can be configured to automatically do this out of the box with:
|
|
|
|
---------------------
|
|
$ git config sendemail.to buildroot@buildroot.org
|
|
$ git config sendemail.ccCmd "$(pwd)/utils/get-developers -e"
|
|
---------------------
|
|
|
|
And then just do:
|
|
|
|
---------------------
|
|
$ git send-email origin/master
|
|
---------------------
|
|
|
|
Note that +git+ should be configured to use your mail account.
|
|
To configure +git+, see +man git-send-email+ or google it.
|
|
|
|
If you do not use +git send-email+, make sure posted *patches are not
|
|
line-wrapped*, otherwise they cannot easily be applied. In such a case,
|
|
fix your e-mail client, or better yet, learn to use +git send-email+.
|
|
|
|
==== Cover letter
|
|
|
|
If you want to present the whole patch set in a separate mail, add
|
|
+--cover-letter+ to the +git format-patch+ command (see +man
|
|
git-format-patch+ for further information). This will generate a
|
|
template for an introduction e-mail to your patch series.
|
|
|
|
A 'cover letter' may be useful to introduce the changes you propose
|
|
in the following cases:
|
|
|
|
* large number of commits in the series;
|
|
|
|
* deep impact of the changes in the rest of the project;
|
|
|
|
* RFC footnote:[RFC: (Request for comments) change proposal];
|
|
|
|
* whenever you feel it will help presenting your work, your choices,
|
|
the review process, etc.
|
|
|
|
==== Patches for maintenance branches
|
|
|
|
When fixing bugs on a maintenance branch, bugs should be fixed on the
|
|
master branch first. The commit log for such a patch may then contain a
|
|
post-commit note specifying what branches are affected:
|
|
|
|
----
|
|
package/foo: fix stuff
|
|
|
|
Signed-off-by: Your Real Name <your@email.address>
|
|
---
|
|
Backport to: 2020.02.x, 2020.05.x
|
|
(2020.08.x not affected as the version was bumped)
|
|
----
|
|
|
|
Those changes will then be backported by a maintainer to the affected
|
|
branches.
|
|
|
|
However, some bugs may apply only to a specific release, for example
|
|
because it is using an older version of a package. In that case, patches
|
|
should be based off the maintenance branch, and the patch subject prefix
|
|
must include the maintenance branch name (for example "[PATCH 2020.02.x]").
|
|
This can be done with the +git format-patch+ flag +--subject-prefix+:
|
|
|
|
---------------------
|
|
$ git format-patch --subject-prefix "PATCH 2020.02.x" \
|
|
-M -s -o outgoing origin/2020.02.x
|
|
---------------------
|
|
|
|
Then send the patches with +git send-email+, as described above.
|
|
|
|
==== Patch revision changelog
|
|
|
|
When improvements are requested, the new revision of each commit
|
|
should include a changelog of the modifications between each
|
|
submission. Note that when your patch series is introduced by a cover
|
|
letter, an overall changelog may be added to the cover letter in
|
|
addition to the changelog in the individual commits.
|
|
The best thing to rework a patch series is by interactive rebasing:
|
|
+git rebase -i origin/master+. Consult the git manual for more
|
|
information.
|
|
|
|
When added to the individual commits, this changelog is added when
|
|
editing the commit message. Below the +Signed-off-by+ section, add
|
|
+---+ and your changelog.
|
|
|
|
Although the changelog will be visible for the reviewers in the mail
|
|
thread, as well as in http://patchwork.buildroot.org[patchwork], +git+
|
|
will automatically ignores lines below +---+ when the patch will be
|
|
merged. This is the intended behavior: the changelog is not meant to
|
|
be preserved forever in the +git+ history of the project.
|
|
|
|
Hereafter the recommended layout:
|
|
|
|
---------------
|
|
Patch title: short explanation, max 72 chars
|
|
|
|
A paragraph that explains the problem, and how it manifests itself. If
|
|
the problem is complex, it is OK to add more paragraphs. All paragraphs
|
|
should be wrapped at 72 characters.
|
|
|
|
A paragraph that explains the root cause of the problem. Again, more
|
|
than one paragraph is OK.
|
|
|
|
Finally, one or more paragraphs that explain how the problem is solved.
|
|
Don't hesitate to explain complex solutions in detail.
|
|
|
|
Signed-off-by: John DOE <john.doe@example.net>
|
|
|
|
---
|
|
Changes v2 -> v3:
|
|
- foo bar (suggested by Jane)
|
|
- bar buz
|
|
|
|
Changes v1 -> v2:
|
|
- alpha bravo (suggested by John)
|
|
- charly delta
|
|
---------------
|
|
|
|
Any patch revision should include the version number. The version number
|
|
is simply composed of the letter +v+ followed by an +integer+ greater or
|
|
equal to two (i.e. "PATCH v2", "PATCH v3" ...).
|
|
|
|
This can be easily handled with +git format-patch+ by using the option
|
|
+--subject-prefix+:
|
|
|
|
---------------------
|
|
$ git format-patch --subject-prefix "PATCH v4" \
|
|
-M -s -o outgoing origin/master
|
|
---------------------
|
|
|
|
Since git version 1.8.1, you can also use +-v <n>+ (where <n> is the
|
|
version number):
|
|
|
|
---------------------
|
|
$ git format-patch -v4 -M -s -o outgoing origin/master
|
|
---------------------
|
|
|
|
When you provide a new version of a patch, please mark the old one as
|
|
superseded in http://patchwork.buildroot.org[patchwork]. You need to
|
|
create an account on http://patchwork.buildroot.org[patchwork] to be
|
|
able to modify the status of your patches. Note that you can only change
|
|
the status of patches you submitted yourself, which means the email
|
|
address you register in http://patchwork.buildroot.org[patchwork] should
|
|
match the one you use for sending patches to the mailing list.
|
|
|
|
You can also add the +--in-reply-to <message-id>+ option when
|
|
submitting a patch to the mailing list. The id of the mail to reply to
|
|
can be found under the "Message Id" tag on
|
|
http://patchwork.buildroot.org[patchwork]. The advantage of
|
|
*in-reply-to* is that patchwork will automatically mark the previous
|
|
version of the patch as superseded.
|
|
|
|
[[reporting-bugs]]
|
|
=== Reporting issues/bugs or getting help
|
|
|
|
Before reporting any issue, please check in
|
|
xref:community-resources[the mailing list archive] whether someone has
|
|
already reported and/or fixed a similar problem.
|
|
|
|
However you choose to report bugs or get help, either by
|
|
opening a bug in the xref:community-resources[bug tracker] or by
|
|
xref:community-resources[sending a mail to the mailing list], there are
|
|
a number of details to provide in order to help people reproduce and
|
|
find a solution to the issue.
|
|
|
|
Try to think as if you were trying to help someone else; in
|
|
that case, what would you need?
|
|
|
|
Here is a short list of details to provide in such case:
|
|
|
|
* host machine (OS/release)
|
|
* version of Buildroot
|
|
* target for which the build fails
|
|
* package(s) for which the build fails
|
|
* the command that fails and its output
|
|
* any information you think that may be relevant
|
|
|
|
Additionally, you should add the +.config+ file (or if you know how, a
|
|
+defconfig+; see xref:customize-store-buildroot-config[]).
|
|
|
|
If some of these details are too large, do not hesitate to use a
|
|
pastebin service. Note that not all available pastebin services will
|
|
preserve Unix-style line terminators when downloading raw pastes.
|
|
Following pastebin services are known to work correctly:
|
|
- https://gist.github.com/
|
|
- http://code.bulix.org/
|
|
|
|
=== Using the runtime tests framework
|
|
|
|
Buildroot includes a run-time testing framework built upon Python
|
|
scripting and QEMU runtime execution. The goals of the framework are
|
|
the following:
|
|
|
|
* build a well defined Buildroot configuration
|
|
* optionally, verify some properties of the build output
|
|
* optionally, boot the build results under Qemu, and verify that a
|
|
given feature is working as expected
|
|
|
|
The entry point to use the runtime tests framework is the
|
|
+support/testing/run-tests+ tool, which has a series of options
|
|
documented in the tool's help '-h' description. Some common options
|
|
include setting the download folder, the output folder, keeping build
|
|
output, and for multiple test cases, you can set the JLEVEL for each.
|
|
|
|
Here is an example walk through of running a test case.
|
|
|
|
* For a first step, let us see what all the test case options are. The test
|
|
cases can be listed by executing +support/testing/run-tests -l+. These tests
|
|
can all be run individually during test development from the console. Both
|
|
one at a time and selectively as a group of a subset of tests.
|
|
|
|
---------------------
|
|
$ support/testing/run-tests -l
|
|
List of tests
|
|
test_run (tests.utils.test_check_package.TestCheckPackage)
|
|
test_run (tests.toolchain.test_external.TestExternalToolchainBuildrootMusl) ... ok
|
|
test_run (tests.toolchain.test_external.TestExternalToolchainBuildrootuClibc) ... ok
|
|
test_run (tests.toolchain.test_external.TestExternalToolchainCCache) ... ok
|
|
test_run (tests.toolchain.test_external.TestExternalToolchainCtngMusl) ... ok
|
|
test_run (tests.toolchain.test_external.TestExternalToolchainLinaroArm) ... ok
|
|
test_run (tests.toolchain.test_external.TestExternalToolchainSourceryArmv4) ... ok
|
|
test_run (tests.toolchain.test_external.TestExternalToolchainSourceryArmv5) ... ok
|
|
test_run (tests.toolchain.test_external.TestExternalToolchainSourceryArmv7) ... ok
|
|
[snip]
|
|
test_run (tests.init.test_systemd.TestInitSystemSystemdRoFull) ... ok
|
|
test_run (tests.init.test_systemd.TestInitSystemSystemdRoIfupdown) ... ok
|
|
test_run (tests.init.test_systemd.TestInitSystemSystemdRoNetworkd) ... ok
|
|
test_run (tests.init.test_systemd.TestInitSystemSystemdRwFull) ... ok
|
|
test_run (tests.init.test_systemd.TestInitSystemSystemdRwIfupdown) ... ok
|
|
test_run (tests.init.test_systemd.TestInitSystemSystemdRwNetworkd) ... ok
|
|
test_run (tests.init.test_busybox.TestInitSystemBusyboxRo) ... ok
|
|
test_run (tests.init.test_busybox.TestInitSystemBusyboxRoNet) ... ok
|
|
test_run (tests.init.test_busybox.TestInitSystemBusyboxRw) ... ok
|
|
test_run (tests.init.test_busybox.TestInitSystemBusyboxRwNet) ... ok
|
|
|
|
Ran 157 tests in 0.021s
|
|
|
|
OK
|
|
---------------------
|
|
|
|
* Then, to run one test case:
|
|
|
|
---------------------
|
|
$ support/testing/run-tests -d dl -o output_folder -k tests.init.test_busybox.TestInitSystemBusyboxRw
|
|
15:03:26 TestInitSystemBusyboxRw Starting
|
|
15:03:28 TestInitSystemBusyboxRw Building
|
|
15:08:18 TestInitSystemBusyboxRw Building done
|
|
15:08:27 TestInitSystemBusyboxRw Cleaning up
|
|
.
|
|
Ran 1 test in 301.140s
|
|
|
|
OK
|
|
---------------------
|
|
|
|
The standard output indicates if the test is successful or not. By
|
|
default, the output folder for the test is deleted automatically
|
|
unless the option +-k+ is passed to *keep* the output directory.
|
|
|
|
==== Creating a test case
|
|
|
|
Within the Buildroot repository, the testing framework is organized at the
|
|
top level in +support/testing/+ by folders of +conf+, +infra+ and +tests+.
|
|
All the test cases live under the +tests+ folder and are organized in various
|
|
folders representing the category of test.
|
|
|
|
The best way to get familiar with how to create a test case is to look
|
|
at a few of the basic file system +support/testing/tests/fs/+ and init
|
|
+support/testing/tests/init/+ test scripts. Those tests give good
|
|
examples of a basic tests that include both checking the build
|
|
results, and doing runtime tests. There are other more advanced cases
|
|
that use things like nested +br2-external+ folders to provide
|
|
skeletons and additional packages.
|
|
|
|
Creating a basic test case involves:
|
|
|
|
* Defining a test class that inherits from +infra.basetest.BRTest+
|
|
|
|
* Defining the +config+ member of the test class, to the Buildroot
|
|
configuration to build for this test case. It can optionally rely on
|
|
configuration snippets provided by the runtime test infrastructure:
|
|
+infra.basetest.BASIC_TOOLCHAIN_CONFIG+ to get a basic
|
|
architecture/toolchain configuration, and
|
|
+infra.basetest.MINIMAL_CONFIG+ to not build any filesystem. The
|
|
advantage of using +infra.basetest.BASIC_TOOLCHAIN_CONFIG+ is that a
|
|
matching Linux kernel image is provided, which allows to boot the
|
|
resulting image in Qemu without having to build a Linux kernel image
|
|
as part of the test case, therefore significant decreasing the build
|
|
time required for the test case.
|
|
|
|
* Implementing a +def test_run(self):+ function to implement the
|
|
actual tests to run after the build has completed. They may be tests
|
|
that verify the build output, by running command on the host using
|
|
the +run_cmd_on_host()+ helper function. Or they may boot the
|
|
generated system in Qemu using the +Emulator+ object available as
|
|
+self.emulator+ in the test case. For example +self.emulator.boot()+
|
|
allows to boot the system in Qemu, +self.emulator.login()+ allows to
|
|
login, +self.emulator.run()+ allows to run shell commands inside
|
|
Qemu.
|
|
|
|
After creating the test script, add yourself to the +DEVELOPERS+ file to
|
|
be the maintainer of that test case.
|
|
|
|
==== Debugging a test case
|
|
|
|
When a test case runs, the +output_folder+ will contain the following:
|
|
|
|
---------------------
|
|
$ ls output_folder/
|
|
TestInitSystemBusyboxRw/
|
|
TestInitSystemBusyboxRw-build.log
|
|
TestInitSystemBusyboxRw-run.log
|
|
---------------------
|
|
|
|
+TestInitSystemBusyboxRw/+ is the Buildroot output directory, and it
|
|
is preserved only if the +-k+ option is passed.
|
|
|
|
+TestInitSystemBusyboxRw-build.log+ is the log of the Buildroot build.
|
|
|
|
+TestInitSystemBusyboxRw-run.log+ is the log of the Qemu boot and
|
|
test. This file will only exist if the build was successful and the
|
|
test case involves booting under Qemu.
|
|
|
|
If you want to manually run Qemu to do manual tests of the build
|
|
result, the first few lines of +TestInitSystemBusyboxRw-run.log+
|
|
contain the Qemu command line to use.
|
|
|
|
You can also make modifications to the current sources inside the
|
|
+output_folder+ (e.g. for debug purposes) and rerun the standard
|
|
Buildroot make targets (in order to regenerate the complete image with
|
|
the new modifications) and then rerun the test.
|
|
|
|
==== Runtime tests and Gitlab CI
|
|
|
|
All runtime tests are regularly executed by Buildroot Gitlab CI
|
|
infrastructure, see .gitlab.yml and
|
|
https://gitlab.com/buildroot.org/buildroot/-/jobs.
|
|
|
|
You can also use Gitlab CI to test your new test cases, or verify that
|
|
existing tests continue to work after making changes in Buildroot.
|
|
|
|
In order to achieve this, you need to create a fork of the Buildroot
|
|
project on Gitlab, and be able to push branches to your Buildroot fork
|
|
on Gitlab.
|
|
|
|
The name of the branch that you push will determine if a Gitlab CI
|
|
pipeline will be triggered or not, and for which test cases.
|
|
|
|
In the examples below, the <name> component of the branch name is an
|
|
arbitrary string you choose.
|
|
|
|
* To trigger all run-test test case jobs, push a branch that ends with
|
|
+-runtime-tests+:
|
|
|
|
---------------------
|
|
$ git push gitlab HEAD:<name>-runtime-tests
|
|
---------------------
|
|
|
|
* To trigger one or several test case jobs, push a branch that ends
|
|
with the complete test case name
|
|
(+tests.init.test_busybox.TestInitSystemBusyboxRo+) or with the name
|
|
of a category of tests (+tests.init.test_busybox+):
|
|
|
|
---------------------
|
|
$ git push gitlab HEAD:<name>-<test case name>
|
|
---------------------
|
|
|
|
Example to run one test:
|
|
|
|
---------------------
|
|
$ git push gitlab HEAD:foo-tests.init.test_busybox.TestInitSystemBusyboxRo
|
|
---------------------
|
|
|
|
Examples to run several tests part of the same group:
|
|
|
|
---------------------
|
|
$ git push gitlab HEAD:foo-tests.init.test_busybox
|
|
$ git push gitlab HEAD:foo-tests.init
|
|
---------------------
|
|
|