U-Boot host tools use the tools configuration namespace when code calls
CONFIG_IS_ENABLED(). With USE_HOSTCC, CONFIG_IS_ENABLED(FIT_SIGNATURE)
resolves to CONFIG_TOOLS_FIT_SIGNATURE, while
CONFIG_VAL(FIT_SIGNATURE_MAX_SIZE) resolves to
CONFIG_TOOLS_FIT_SIGNATURE_MAX_SIZE.
The host-uboot-tools package only generates the old CONFIG_FIT_SIGNATURE
define. This makes tools/Makefile include fit_check_sign, but the host code
sees FIT signature support as disabled and the OpenSSL-backed signing and
verification objects are omitted. As a result, mkimage accepts a signature
node without writing its value or injecting a required public key.
fit_check_sign then has no required key and reports success without
checking the configuration signature.
A FIT-support-only configuration never exercises this path, which is why
the existing hash-only runtime test still passes.
Generate the tools FIT signature options needed by the host code and pass
CONFIG_TOOLS_LIBCRYPTO=y so the OpenSSL-backed signing, verification and
cipher objects are selected. Keep CONFIG_FIT_SIGNATURE=y in the make
options because U-Boot tools/Makefile still uses it to build
fit_check_sign.
Extend TestHostUbootTools to create an RSA-signed FIT, require a 256-byte
configuration signature and a required public key, verify the FIT, corrupt
the signature, and require verification to fail.
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
[Julien: add comments in runtime test]
Signed-off-by: Julien Olivain <ju.o@free.fr>
Boot an armv5 image running the mdnsd responder alongside the mquery
client, then check that service discovery works: mquery browses for the
bundled _http._tcp service and mdnsd, on the same host, answers over the
loopback of the eth0 multicast group.
Signed-off-by: Joachim Wiberg <troglobit@gmail.com>
[Julien: add "mdnsd -v" invocation to show version]
Signed-off-by: Julien Olivain <ju.o@free.fr>
The Debian 12 Bookworm snapshot we use, 20250203, uses the new deb822
format [0] for source list files; it carries a source list file that
points to the current repository, not to the snapshot.
Even though we do inject an old-style source list that points to the
snapshot repository, the packages in the current repository are more
recent than the one in the snapshot, so when we install our packages,
they get retrieved mostly from the current repository rather than from
the snapshot. The image is not reproducible.
Switch to using the new deb822-style source list file.
Note: we do not need to carry the "Latest just before" trick: the
snapshot repository will use the most recent actual snapshot before the
requested dated, so we can just use the date of the image we use (stuck
at midnight because we don't have better).
Since the snapshot repository can be really slow, while at the same time
the remote http server not dropping connections, it can take a very long
while to build the image; add a timeout so that stale connections are
detected and re-attemped early-ish (the timeout applies to both the
connect and the actual download, so it should be large enough to
accomodate slowish network connections).
Fixes: c95d5b8e1e (support/docker: move to current bookworm (Debian
12) snapshot)
[0] https://manpages.debian.org/trixie/apt/sources.list.5.en.html#DEB822-STYLE_FORMAT
Signed-off-by: Yann E. MORIN <yann.morin@orange.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Agec is a simple file encryption tool that implements the
age format in C with minimal dependencies. The tool supports
asymmetric encryption based on X25519, and a passphrase
encryption based on scrypt.
https://git.sr.ht/~min/agechttps://age-encryption.org
Encryption is silently broken for files <35 bytes, so add a patch submitted
upstream to fix that.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
In the precedent patch "support/testing: test_gnupg2.py: use
assertRunNotOk()" [1] a small typo (OK instead of Ok) was introduced
in the test_gnupg2.py file, which caused a runtime failure.
This patch simply replace "self.assertRunNotOK(cmd)" with
"self.assertRunNotOk(cmd)" to correct this typo.
[1] cb791850ad
Signed-off-by: Dowan Gullient <dowan.gullient@smile.fr>
[Fiona: fix commit message formatting]
Signed-off-by: Fiona Klute <fiona.klute@gmx.de>
The sample script would've passed even if the connections list was
empty. Check that there is at least one connection, and that the
sample script can read loopback device information.
Signed-off-by: Fiona Klute <fiona.klute@gmx.de>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
The new test inherits from TestIptables, with only minimal changes to
use the nftables backend instead.
Signed-off-by: Fiona Klute <fiona.klute@gmx.de>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
TestIptables expect netfilter legacy tables enabled in the kernel.
Make sure we use iptables-legacy:
iptables --version
iptables v1.8.11 (legacy)
The test would fail if nftables compat (iptables-nft) is enabled.
iptables --version
iptables v1.8.11 (nf_tables)
Add the following configuration fragment to enable iptables-nft:
BR2_PACKAGE_IPTABLES_NFTABLES=y
BR2_PACKAGE_IPTABLES_NFTABLES_DEFAULT=y
Acked-by: Fiona Klute <fiona.klute@gmx.de>
Tested-by: Fiona Klute <fiona.klute@gmx.de>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Fiona Klute <fiona.klute@gmx.de>
Add basic smoke test for glmark2 running in a nested qemu guest with
virtio-gpu-gl-pci.
The outer machine uses the Mesa softpipe driver for GLES support with
minimal dependencies. Just test virgl for now, as lavapipe
(software Vulkan driver) requires LLVM, and this validates
virglrenderer.
Signed-off-by: Joseph Kogut <joseph.kogut@gmail.com>
[Romain: add gitlab-runner "2xlarge" tag]
Signed-off-by: Romain Naour <romain.naour@smile.fr>
There are a number of runtime tests that checks that a command fails as
expected, so add an assertRunNotOk() similar to the existing assertRunOk()
to handle that instead of open coding it everywhere.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Reviewed-by: Fiona Klute <fiona.klute@gmx.de>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Our docker image was finally updated to Debian 12 and now provide Qemu
7.2.15 [1]. We can safely revert this workaround for older Qemu version.
[1] e5bbe020ca
This reverts commit 37e23e0dd7.
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
defconfigs are very unlikely to successfully build on a small runner
because they build a kernel (and often a toolchain). They're also likely
to benefit a lot from the additional parallelism on larger runners.
For now, always build them on large runners. There may be some for which
even the large runners don't have sufficient disk space or memory, but
we'll solve that when it happens.
Signed-off-by: Arnout Vandecappelle <arnout@rnout.be>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Since the ARM AArch64 toolchain toolchain bump [1], the TestPppd fail
at runtime since ttyppp0 and ttyppp1 interfaces were not created
properly:
pppd[87]: pppd 2.5.2 started by root, uid 0
pppd[87]: Baud rate for /dev/ttyppp1 is 0; need explicit baud rate
The new toolchain uses glibc 2.42 that include a rework in termios
"implement arbitrary and split speeds in termios" to support BOTHER
[2]. This rework seems to conflict with pppd workaround [3] introduced
when BOTHER was not supported in Glibc.
Fix this by setting the baudrate on pppd command line (we already set
this baudrate on socat command line).
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/14607335920 (TestPppd)
[1] 8d413d6179
[2] https://sourceware.org/git/?p=glibc.git;a=commit;h=5cf101a85aae0d703cdd8ed7b25fe288e41fdacb
[3] d65f6713c8
Fix-suggested-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
We have our own GitLab-CI runner, but with only one we can't run many
jobs in parallel so it takes a very long time before all the tests have
completed. In addition, if that runner goes down, we have nothing at
all.
GitLab offers the following machine types for hosted runners on Linux
x86-64 [1]. The default is the "small" runner. Using a larger runner
increases the "Cost factor" [2].
For opensource projects, the Cost factor is reduced to 0.5 (1 minute per
2 minutes of job time) whatever the runner type.
Runner Tag vCPUs Memory Storage Cost factor (OSS)
saas-linux-small-amd64 (default) 2 8 GB 30 GB 1 (0.5)
saas-linux-medium-amd64 4 16 GB 50 GB 2 (0.5)
saas-linux-large-amd64 (Premium and Ultimate only) 8 32 GB 100 GB 3 (0.5)
saas-linux-xlarge-amd64 (Premium and Ultimate only) 16 64 GB 200 GB 6 (0.5)
saas-linux-2xlarge-amd64 (Premium and Ultimate only) 32 128 GB 200 GB 12 (0.5)
Compute minutes consumed by a job is calculated by:
Job duration / 60 * Cost factor
(Job duration: The time, in seconds, that a job took to run, not
including time spent in the created or pending statuses.)
Thanks to the GitLab OSS program [3], Buildroot benefits from a free
Ultimate subscription and can use GitLab shared runners tagged with
saas-linux-{large, xlarge, 2xlarge}-amd64. In addition, we receive
50,000 free runner minutes per month.
In order to use one of those tags in Buildroot GitLab-CI jobs, we have
to classify all tests by resource requirement, to make sure the job
doesn't fail because it times out or has insufficient memory or disk
space. While we usually shouldn't use the largest runner for
everything, we can use larger runner without cost penalty thanks to
the cost factor reduced to 0.5 for opensource projects. This will
reduce the CI minutes consumed by a CPU intensive job.
First we introduce some new templates used to add the corresponding
runner tag to a runtime test job (reusing the GitLab terminology).
.runner-{small,medium,large,xlarge,2xlarge}
Most of our tests are fast (checkpackage, test_external_bootlin...), so
saas-linux-small-amd64 runner tag is enough. Default to this tag if
nothing else is specified.
Add a comment next to the test class to provide the runner tag.
This runner tag is retrieved when generating the
generated-gitlab-ci.yml file used to create the child pipeline where
the runtime test jobs are executed.
We use the list of runtime tests returned by node2:
"tests.boot.test_edk2.TestEdk2.test_run"
We convert each element of this list to get the path to the test source
file and the name of the test:
"support/testing/tests/boot/test_edk2.py"
TestEdk2
With that, we can grep into the test source file to retrieve the runner
tag placed one line above the test class:
# GitLab-runner: large
class TestEdk2(infra.basetest.BRTest):
Once the runner tag is retrieved, it's used to use the corresponding
runner template to the runtime test job:
tests.boot.test_edk2.TestEdk2.test_run: { extends: [ .runtime_test_base, .runner-large ]}
GitLab runners hosted by the Buildroot project should be able to run
any jobs, so they should be tagged with Gitlab runner tags
(saas-linux-{small,medium,large,xlarge,2xlarge}-amd64).
A specific runner tag "buildroot-runner" can be used to allow running
a job only on such runners.
If a test can't be executed by any shared GitLab-CI runners, we have
to use a runner owned by the Buildroot project. In this case we have
to use a specific template ".runner-buildroot-runner-only" in order to
add the specific runner tag "buildroot-runner" to the job running the
test. There is no such runtime test at the moment.
The proposed classification is based on a previous pipeline analysis
[5]:
- Tests lasting more than 3 hours will use 2xlarge runners.
- Tests lasting more than 2 hours will use xlarge runners.
- Tests lasting more than 1 hours will use large runners.
- Tests building a kernel or a toolchain will use medium runners.
- All other tests will use small runners when possible.
CI minute cost estimate:
tests.package.test_clang.TestClangCompilerRT.test_run lasts 4h25 on the
Buildroot runner. If we this duration for 2xlarge runners, the CI
minute consumed would be:
(15900 / 60) * 0.5 = 133
With 6 jobs using a 2xlarge runners we used ~795 CI minutes.
tests.package.test_kmscube.TestKmsCube.test_run list 2h04 on the
Buildroot runner. If we this duration for xlarge runners, the CI
minute consumed would be:
(7440 / 60) * 0.5 = 62
With 4 jobs using a xlarge runners we used ~248 CI minutes.
tests.package.test_weston.TestWeston.test_run last 1h15 on th
Buildroot runner. If we this duration for large runners, the CI
minute consumed would be:
(4500 / 60) * 0.5 = 37.5
With 28 jobs using a large runners we used ~1050 CI minutes.
tests.package.test_gstreamer1.TestGstreamer1.test_run last 46min on the
Buildroot runner. If we this duration for large runners, the CI
minute consumed would be:
(2760 / 60) * 0.5 = 23
With 31 jobs using a medium runners we used ~713 CI minutes.
tests.package.test_python.TestPython3Py.test_run last 13min on the
Buildroot runner. If we this duration for large runners, the CI
minute consumed would be:
(780 / 60) * 0.5 = 6.5
With 691 jobs using a medium runners we used ~4491 CI minutes.
In total, one pipeline for the runtime tests cost ~7297 CI minutes.
After a first try [6], we are actually using 8000 CI minutes per
pipeline.
We run such pipeline once a week (on Monday), one for each Buildroot
releases every 3 month, one for each stable and LTS release per month,
and one for each release candidate (3).
Worst case (release month):
(4 weeks + 1 release + 1 stable + 1 LTS + 3 release candidate) * 8000 CI
minutes: 80000 CI minutes / 50000.
So we would spend the minutes very quickly in the worst case scenario.
We have to keep one pipeline under 5000 CI minutes.
[1] https://docs.gitlab.com/ci/runners/hosted_runners/linux/#machine-types-available-for-linux---x86-64
[2] https://docs.gitlab.com/ci/pipelines/compute_minutes/#cost-factorshttps://docs.gitlab.com/ci/pipelines/compute_minutes/#compute-usage-calculationhttps://docs.gitlab.com/ci/pipelines/compute_minutes/#cost-factors-of-hosted-runners-for-gitlabcom
[3] https://gitlab.com/buildroot.org/gitlab-oss
[4] https://docs.gitlab.com/ci/runners/hosted_runners/#gitlabcom-hosted-runner-workflow
[5] https://gitlab.com/buildroot.org/buildroot/-/pipelines/2416603721
[6] https://gitlab.com/buildroot.org/buildroot/-/pipelines/2562421098
Signed-off-by: Romain Naour <romain.naour@smile.fr>
[Arnout:
- simplify parsing of test_file and test_name;
- match the entire test_name instead of substring;
- assume "small" by default;
- remove the "small" tags;
- use "gitlab-runner" instead of "Gitlab-runner".
]
Signed-off-by: Arnout Vandecappelle <arnout@rnout.be>
Signed-off-by: Arnout Vandecappelle <arnout@rnout.be>
