We have our own GitLab-CI runner, but with only one we can't run many
jobs in parallel so it takes a very long time before all the tests have
completed. In addition, if that runner goes down, we have nothing at
all.
GitLab offers the following machine types for hosted runners on Linux
x86-64 [1]. The default is the "small" runner. Using a larger runner
increases the "Cost factor" [2].
For opensource projects, the Cost factor is reduced to 0.5 (1 minute per
2 minutes of job time) whatever the runner type.
Runner Tag vCPUs Memory Storage Cost factor (OSS)
saas-linux-small-amd64 (default) 2 8 GB 30 GB 1 (0.5)
saas-linux-medium-amd64 4 16 GB 50 GB 2 (0.5)
saas-linux-large-amd64 (Premium and Ultimate only) 8 32 GB 100 GB 3 (0.5)
saas-linux-xlarge-amd64 (Premium and Ultimate only) 16 64 GB 200 GB 6 (0.5)
saas-linux-2xlarge-amd64 (Premium and Ultimate only) 32 128 GB 200 GB 12 (0.5)
Compute minutes consumed by a job is calculated by:
Job duration / 60 * Cost factor
(Job duration: The time, in seconds, that a job took to run, not
including time spent in the created or pending statuses.)
Thanks to the GitLab OSS program [3], Buildroot benefits from a free
Ultimate subscription and can use GitLab shared runners tagged with
saas-linux-{large, xlarge, 2xlarge}-amd64. In addition, we receive
50,000 free runner minutes per month.
In order to use one of those tags in Buildroot GitLab-CI jobs, we have
to classify all tests by resource requirement, to make sure the job
doesn't fail because it times out or has insufficient memory or disk
space. While we usually shouldn't use the largest runner for
everything, we can use larger runner without cost penalty thanks to
the cost factor reduced to 0.5 for opensource projects. This will
reduce the CI minutes consumed by a CPU intensive job.
First we introduce some new templates used to add the corresponding
runner tag to a runtime test job (reusing the GitLab terminology).
.runner-{small,medium,large,xlarge,2xlarge}
Most of our tests are fast (checkpackage, test_external_bootlin...), so
saas-linux-small-amd64 runner tag is enough. Default to this tag if
nothing else is specified.
Add a comment next to the test class to provide the runner tag.
This runner tag is retrieved when generating the
generated-gitlab-ci.yml file used to create the child pipeline where
the runtime test jobs are executed.
We use the list of runtime tests returned by node2:
"tests.boot.test_edk2.TestEdk2.test_run"
We convert each element of this list to get the path to the test source
file and the name of the test:
"support/testing/tests/boot/test_edk2.py"
TestEdk2
With that, we can grep into the test source file to retrieve the runner
tag placed one line above the test class:
# GitLab-runner: large
class TestEdk2(infra.basetest.BRTest):
Once the runner tag is retrieved, it's used to use the corresponding
runner template to the runtime test job:
tests.boot.test_edk2.TestEdk2.test_run: { extends: [ .runtime_test_base, .runner-large ]}
GitLab runners hosted by the Buildroot project should be able to run
any jobs, so they should be tagged with Gitlab runner tags
(saas-linux-{small,medium,large,xlarge,2xlarge}-amd64).
A specific runner tag "buildroot-runner" can be used to allow running
a job only on such runners.
If a test can't be executed by any shared GitLab-CI runners, we have
to use a runner owned by the Buildroot project. In this case we have
to use a specific template ".runner-buildroot-runner-only" in order to
add the specific runner tag "buildroot-runner" to the job running the
test. There is no such runtime test at the moment.
The proposed classification is based on a previous pipeline analysis
[5]:
- Tests lasting more than 3 hours will use 2xlarge runners.
- Tests lasting more than 2 hours will use xlarge runners.
- Tests lasting more than 1 hours will use large runners.
- Tests building a kernel or a toolchain will use medium runners.
- All other tests will use small runners when possible.
CI minute cost estimate:
tests.package.test_clang.TestClangCompilerRT.test_run lasts 4h25 on the
Buildroot runner. If we this duration for 2xlarge runners, the CI
minute consumed would be:
(15900 / 60) * 0.5 = 133
With 6 jobs using a 2xlarge runners we used ~795 CI minutes.
tests.package.test_kmscube.TestKmsCube.test_run list 2h04 on the
Buildroot runner. If we this duration for xlarge runners, the CI
minute consumed would be:
(7440 / 60) * 0.5 = 62
With 4 jobs using a xlarge runners we used ~248 CI minutes.
tests.package.test_weston.TestWeston.test_run last 1h15 on th
Buildroot runner. If we this duration for large runners, the CI
minute consumed would be:
(4500 / 60) * 0.5 = 37.5
With 28 jobs using a large runners we used ~1050 CI minutes.
tests.package.test_gstreamer1.TestGstreamer1.test_run last 46min on the
Buildroot runner. If we this duration for large runners, the CI
minute consumed would be:
(2760 / 60) * 0.5 = 23
With 31 jobs using a medium runners we used ~713 CI minutes.
tests.package.test_python.TestPython3Py.test_run last 13min on the
Buildroot runner. If we this duration for large runners, the CI
minute consumed would be:
(780 / 60) * 0.5 = 6.5
With 691 jobs using a medium runners we used ~4491 CI minutes.
In total, one pipeline for the runtime tests cost ~7297 CI minutes.
After a first try [6], we are actually using 8000 CI minutes per
pipeline.
We run such pipeline once a week (on Monday), one for each Buildroot
releases every 3 month, one for each stable and LTS release per month,
and one for each release candidate (3).
Worst case (release month):
(4 weeks + 1 release + 1 stable + 1 LTS + 3 release candidate) * 8000 CI
minutes: 80000 CI minutes / 50000.
So we would spend the minutes very quickly in the worst case scenario.
We have to keep one pipeline under 5000 CI minutes.
[1] https://docs.gitlab.com/ci/runners/hosted_runners/linux/#machine-types-available-for-linux---x86-64
[2] https://docs.gitlab.com/ci/pipelines/compute_minutes/#cost-factorshttps://docs.gitlab.com/ci/pipelines/compute_minutes/#compute-usage-calculationhttps://docs.gitlab.com/ci/pipelines/compute_minutes/#cost-factors-of-hosted-runners-for-gitlabcom
[3] https://gitlab.com/buildroot.org/gitlab-oss
[4] https://docs.gitlab.com/ci/runners/hosted_runners/#gitlabcom-hosted-runner-workflow
[5] https://gitlab.com/buildroot.org/buildroot/-/pipelines/2416603721
[6] https://gitlab.com/buildroot.org/buildroot/-/pipelines/2562421098
Signed-off-by: Romain Naour <romain.naour@smile.fr>
[Arnout:
- simplify parsing of test_file and test_name;
- match the entire test_name instead of substring;
- assume "small" by default;
- remove the "small" tags;
- use "gitlab-runner" instead of "Gitlab-runner".
]
Signed-off-by: Arnout Vandecappelle <arnout@rnout.be>
Signed-off-by: Arnout Vandecappelle <arnout@rnout.be>
Passing runtime configuration to tests in BRConfigTest class variables
requires the "fork" start method for multiprocessing. With
"forkserver" (default for POSIX platforms since Python 3.14) class
variables modified in the parent process are lost, because the child
process does not inherit the full Python state. This broke the way
support/testing/run-tests sets configuration for BRConfigTest, so a
start method override was added in commit
3d2141bcee.
Instead this patch adds a settings dataclass (requires Python >= 3.7)
which is serialized into an environment variable from run-tests and
read during BRConfigTest.__init__(). Reading is skipped if the
environment variable is not set so test discovery (not execution)
works without configuration (e.g. utils/get-developers uses this).
With that, run-tests no longer relies on a specific multiprocessing
start method.
Signed-off-by: Fiona Klute <fiona.klute@gmx.de>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Some packages do not have a http/https download URL for a source tarball,
but are acquired over a version control system like git. If so, add
externalReferences of type "vcs" for such URLs.
As most git repositories use a https:// transport that may not indicated the
repository type, add a "comment" due to the lack of a better mechanism in
CycloneDX.
While the hashes are calculated over a tarball created locally, it still may
be useful, so add them for "vcs" externalReferences as well.
Signed-off-by: Martin Willi <martin@strongswan.org>
Acked-By: Thomas Perale <thomas.perale@mind.be>
Signed-off-by: Arnout Vandecappelle <arnout@rnout.be>
Commit dc4af8bfa9 ("utils/generate-cyclonedx: use direct dependencies")
removes indirect dependencies from any listed component, as required by
CycloneDX. The root component, however, still includes indirect dependencies,
as it just takes the components from the show-info output.
Fix this by collecting all component dependencies, and then filter the root
component dependencies to include direct dependencies only.
Signed-off-by: Martin Willi <martin@strongswan.org>
Acked-By: Thomas Perale <thomas.perale@mind.be>
Signed-off-by: Arnout Vandecappelle <arnout@rnout.be>
As Thomas stated in 3bb260cf38:
The br-arm-internal-glibc.config is generally used as a configuration
to test the bleeding edge versions of components. However, it has been
lagging behind somewhat, so let's bring it up-to-date:
- Binutils 2.46.x
- GCC 16.x
Let the fun begin in the autobuilders!
Signed-off-by: Bernd Kuhls <bernd@kuhls.net>
Signed-off-by: Julien Olivain <ju.o@free.fr>
OpenJDK 25 is the latest release.
See: https://endoflife.date/oracle-jdk
- BR2_OPENJDK_VERSION_LATEST is now set to 25.
- Add version-specific patches for 25.0.2+10:
- 0001: Add ARCv2 ISA processors support to Zero
- 0002: Compile OpenJDK in headless mode without requirements
- 0003: Fix ambiguous cmp() overload in aarch64 macro assembler
(older GCC 6.x compatibility)
- 0004: Fix constexpr on non-literal type in Shenandoah GC
(older GCC 6.x compatibility)
- Add -fpermissive to fix template definition error with older GCC.
- Update HOST_OPENJDK_BIN_VERSION for OpenJDK 25.
- Update the expectedVersion variable in JniTest.java from 0x00150000
(JNI 21) to 0x00180000 (JNI 24/25).
Tested with:
$ ./support/testing/run-tests -o ~/br-test-py/ -d ~/br-test-dl/ \
tests.package.test_openjdk.TestOpenJdk
Signed-off-by: Thomas Devoogdt <thomas@devoogdt.com>
[Bernd:
- rebased on bump of versions 17 & 21
- build-tested with gcc 16-snapshot]
Signed-off-by: Bernd Kuhls <bernd@kuhls.net>
[Julien:
- add .checkpackageignore entry to fix check-package error
- reformat patches with without numbering to fix check-package error
]
Signed-off-by: Julien Olivain <ju.o@free.fr>
Buildroot commit aac3d2b402 added a hidden
boolean option that packages which depend on java on the host can select.
Since buildroot commit 5366b8f734 we can
provide our own host-openjdk-bin package.
Kodi previously used BR2_NEEDS_HOST_JAVA but was switched to host-open-
jdk-bin in this series.
The option BR2_NEEDS_HOST_JAVA is now unused and can be removed.
Signed-off-by: Bernd Kuhls <bernd@kuhls.net>
Signed-off-by: Arnout Vandecappelle <arnout@rnout.be>
When running 'make show-info-all' without a '.config', it is possible to
trigger this script without passing a version number.
The 'show-info-all' target is special because it forces the reading of
all packages without requiring a .config, so BR2_HOST_CMAKE_AT_LEAST is
unset and the script is called as:
check-host-cmake.sh cmake cmake3
Without validation, the integer comparisons below would produce errors
like:
check-host-cmake.sh: line 37: [: cmake: integer expected
It's possible to trigger this by adding the following file somewhere in
you path:
cat >/bin/cmake3 <<EOF
echo "cmake version 4.3.3 CMake suite maintained and supported by Kitware (kitware.com/cmake)."
EOF
make show-info-all
The same issue can also occur with pkg-stats.
Signed-off-by: Thomas Perale <thomas.perale@mind.be>
Signed-off-by: Arnout Vandecappelle <arnout@rnout.be>
uutils install 0.8.0 as used in Ubuntu 26.04 has a bug in the install
applet, breaking a number of packages:
https://github.com/uutils/coreutils/pull/11505https://bugs.launchpad.net/ubuntu/+source/rust-coreutils/+bug/2151454
The fix has been merged upstream but not yet released or packaged in Ubuntu,
so detect and reject the buggy version and explain how to change to the
coreutils version. Once fixed the version output will hopefully change.
For simplicity, only check for the exact 0.8.0 version string. Hopefully
when it is fixed in Ubuntu, they also update the version string. Note
that earlier versions of uutils have the same issue of course, but those
versions were never the default "install" on Ubuntu (or anywhere else).
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Arnout Vandecappelle <arnout@rnout.be>
The tags specified by a job are ANDed, not ORed. So with
.runner-small:
tags:
- buildroot-runner
- saas-linux-small-amd64
the job will only run on a runner that has _both_ tags - so only on the
buildroot runner. That makes it pretty pointless.
Remove all the buildroot-runner tags, except in the
.runner-buildroot-runner-only template.
Signed-off-by: Arnout Vandecappelle <arnout@rnout.be>
Currently the weekly pipelines are triggered from a cron job on the
Buildroot server, so generate-gitlab-ci-yml filters on the "trigger"
source. However, we'd like to schedule it on gitlab itself, which makes
managing it easier.
We could filter on "schedule" in addition to "trigger", but there's not
really a reason to. We can simply rely on the BR_SCHEDULE_JOBS variable
- if it is set, we use its information.
Signed-off-by: Arnout Vandecappelle <arnout@rnout.be>
We have our own GitLab-CI runner, but with only one we can't run many
jobs in parallel so it takes a very long time before all the tests have
completed. In addition, if that runner goes down, we have nothing at
all.
GitLab offers the following machine types for hosted runners on Linux
x86-64 [1]. The default is the "small" runner. Using a larger runner
increases the "Cost factor" [2]:
Runner Tag vCPUs Memory Storage Cost factor
saas-linux-small-amd64 (default) 2 8 GB 30 GB 1
saas-linux-medium-amd64 4 16 GB 50 GB 2
saas-linux-large-amd64 (Premium and Ultimate only) 8 32 GB 100 GB 3
saas-linux-xlarge-amd64 (Premium and Ultimate only) 16 64 GB 200 GB 6
saas-linux-2xlarge-amd64 (Premium and Ultimate only) 32 128 GB 200 GB 12
Compute minutes consumed by a job is calculated by:
Job duration / 60 * Cost factor
(Job duration: The time, in seconds, that a job took to run, not
including time spent in the created or pending statuses.)
Thanks to the GitLab OSS program [3], Buildroot benefits from a free
Ultimate subscription and can use GitLab shared runners tagged with
saas-linux-{large, xlarge, 2xlarge}-amd64. In addition, we receive
50,000 free runner minutes per month.
For opensource projects, the Cost factor is reduced by 0,5 (1 minute per
2 minutes of job time).
In order to use one of those tags in Buildroot GitLab-CI jobs, we have
to classify all tests by resource requirement, to make sure the job
doesn't fail because it times out or has insufficient memory or disk
space. We also don't want to use the largest runner for everything
because that would spend the minutes very quickly while the resources
aren't used efficiently (mostly serialized).
First we introduce some new templates used to add the corresponding
runner tag to a runtime test job (reusing the GitLab terminology).
.runner-{small,medium,large,xlarge,2xlarge}
Most of our tests are fast (checkpackage, test_external_bootlin...), so
saas-linux-small-amd64 runner tag is enough. Default to this tag if
nothing else is specified.
Add a comment next to the test class to provide the runner tag.
This runner tag is retrieved when generating the
generated-gitlab-ci.yml file used to create the child pipeline where
the runtime test jobs are executed.
We use the list of runtime tests returned by node2:
"tests.boot.test_edk2.TestEdk2.test_run"
We convert each element of this list to get the path to the test source
file and the name of the test:
"support/testing/tests/boot/test_edk2.py"
TestEdk2
With that, we can grep into the test source file to retrieve the runner
tag placed one line above the test class:
# GitLab-runner: large
class TestEdk2(infra.basetest.BRTest):
Once the runner tag is retrieved, it's used to use the corresponding
runner template to the runtime test job:
tests.boot.test_edk2.TestEdk2.test_run: { extends: [ .runtime_test_base, .runner-large ]}
GitLab runners hosted by the Buildroot project should be able to run any
jobs, so a specific runner tag "buildroot-runner" is used to allow
running a job on such runners. The "buildroot-runner" tag is used in
addition to GitLab ones to run a job on the Buildroot runner or on a
shared GitLab-CI runner.
If a test can't be executed by any shared GitLab-CI runners, we have
to use a runner owned by the Buildroot project. In this case we have
to use a specific template ".runner-buildroot-runner-only". There is no
such runtime test at the moment.
The proposed classification is based on a previous pipeline analysis
[5]:
- Tests lasting more than 3 hours will use 2xlarge runners.
- Tests lasting more than 2 hours will use xlarge runners.
- Tests lasting more than 1 hours will use large runners.
- Tests building a kernel or a toolchain will use medium runners.
- All other tests will use small runners when possible.
CI minute cost estimate:
tests.package.test_clang.TestClangCompilerRT.test_run lasts 4h25 on the
Buildroot runner. If we this duration for 2xlarge runners, the CI
minute consumed would be:
(265 / 60) * 12 * 0.5 = 26,5
With 6 jobs using a 2xlarge runners we used ~160 CI minutes.
tests.package.test_kmscube.TestKmsCube.test_run list 2h04 on the
Buildroot runner. If we this duration for xlarge runners, the CI
minute consumed would be:
(124 / 60) * 6 * 0.5 = 6,2
With 4 jobs using a xlarge runners we used ~37 CI minutes.
tests.package.test_weston.TestWeston.test_run last 1h15 on th
Buildroot runner. If we this duration for large runners, the CI
minute consumed would be:
(75 / 60) * 3 * 0.5 = 1.87
With 28 jobs using a large runners we used ~53 CI minutes.
tests.package.test_gstreamer1.TestGstreamer1.test_run last 46min on the
Buildroot runner. If we this duration for large runners, the CI
minute consumed would be:
(46 / 60) * 2 * 0.5 = 0.76
With 31 jobs using a medium runners we used ~24 CI minutes.
tests.package.test_python.TestPython3Py.test_run last 13min on the
Buildroot runner. If we this duration for large runners, the CI
minute consumed would be:
(13 / 60) * 1 * 0.5 = 0.1
With 691 jobs using a medium runners we used ~69 CI minutes.
In total, one pipeline for the runtime tests cost ~350 CI minutes.
We run such pipeline once a week (on Monday), one for each Buildroot
releases every 3 month, one for each stable and LTS release per month,
and one for each release candidate (3).
Worst case (realse month):
(4 weeks + 1 release + 1 stable + 1 LTS + 3 release candidate) * 350 CI
minutes: 3500 CI minutes / 50000.
So we should be able to run our pipeline without exhausting our CI
minutes credit.
[1] https://docs.gitlab.com/ci/runners/hosted_runners/linux/#machine-types-available-for-linux---x86-64
[2] https://docs.gitlab.com/ci/pipelines/compute_minutes/#cost-factorshttps://docs.gitlab.com/ci/pipelines/compute_minutes/#compute-usage-calculationhttps://docs.gitlab.com/ci/pipelines/compute_minutes/#cost-factors-of-hosted-runners-for-gitlabcom
[3] https://gitlab.com/buildroot.org/gitlab-oss
[4] https://docs.gitlab.com/ci/runners/hosted_runners/#gitlabcom-hosted-runner-workflow
[5] https://gitlab.com/buildroot.org/buildroot/-/pipelines/2416603721
Signed-off-by: Romain Naour <romain.naour@smile.fr>
[Arnout:
- simplify parsing of test_file and test_name;
- match the entire test_name instead of substring;
- assume "small" by default;
- remove the "small" tags;
- use "gitlab-runner" instead of "Gitlab-runner".
]
Signed-off-by: Arnout Vandecappelle <arnout@rnout.be>
Before this commit, only one entry per vulnerability ID was added to the
output. In CycloneDX, if you need to provide different analyses for
different affected components with the same vulnerability ID, you must
create multiple entries with the same ID.
When running `cve-check` with the `--include-resolved` argument, the
analysis of some vulnerabilities would get overwritten, which led to
undefined analysis results.
This is especially true when running the analysis on multiple components
with the same name but different versions. For instance, if the input
SBOM includes both the `gnupg` and `gnupg2` packages, CVE-2025-68973
could be included. This CVE might be exploitable for the `gnupg` package
but resolved for `gnupg2`. Therefore, a single analysis entry cannot
cover both cases.
This commit fixes the logic for adding vulnerabilities to the output
SBOM. A vulnerability is now added as a new entry if:
1. A vulnerability with the same ID doesn't exist yet.
2. The affect of the new vulnerability is not the same as the one
already present.
For the CVE-2025-68973 example this would result in the following
output:
```json
[
{
"id": "CVE-2025-68973",
"analysis": {
"state": "exploitable"
}
"affects": [
{"ref": "gnupg"}
]
},
{
"id": "CVE-2025-68973",
"analysis": {
"state": "resolved"
}
"affects": [
{"ref": "gnupg2"}
]
}
]
```
45 vulnerabilities were concerned by this bug over the Buildroot tree.
Co-Authored-By: Tim Soubry <tim.soubry@mind.be>
Signed-off-by: Thomas Perale <thomas.perale@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The 'bom-ref' are optional and since we don't reference the
vulnerabilities from anywhere else in the SBOM they are not necessary in
this case.
In the following commit, cve-check will potentially emit multiple
vulnerabilities that have the same id. So using the vulnerability id
as 'bom-ref' won't be correct as the 'bom-ref' needs to be unique
unlike the id property.
Signed-off-by: Thomas Perale <thomas.perale@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Always run this script from the output of 'generate-cyclonedx'. Do not re-run
this script over an already analysed SBOM.
Signed-off-by: Thomas Perale <thomas.perale@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Normalize vulnerability timestamps to RFC 3339 format with explicit UTC
timezone suffix for CycloneDX 1.6 compliance.
This fixes validation errors in sbom-utility and makes the generated
SBOM with vulnerabilities compatible with DependencyTrack VEX parsers.
The NVD JSON data feeds provide timestamps in ISO 8601 format without timezone
information (e.g., "1999-01-01T05:00:00.000"), but CycloneDX 1.6 requires
RFC 3339 format with explicit timezone designation (e.g.,
"1999-01-01T05:00:00.000Z").
Add nvd_datetime_to_rfc3339() helper function to convert timestamps before
serialization.
Validation results:
Before fix:
$ sbom-utility validate -i cve/cve_report_current.json
[INFO] BOM valid against JSON schema: 'false'
[INFO] (234) schema errors detected.
Error example:
{
"type": "format",
"field": "vulnerabilities.0.updated",
"context": "(root).vulnerabilities.0.updated",
"description": "Does not match format 'date-time'",
"value": "2025-04-03T01:03:51.193"
}
After fix:
$ sbom-utility validate -i cve/cve_report_update.json
[INFO] BOM valid against JSON schema: 'true'
Tested-with: sbom-utility v0.18.1
Co-authored-by: Fabien Lehoussel <fabien.lehoussel@smile.fr>
Signed-off-by: Thomas Perale <thomas.perale@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Commit 324612d68e fixed several compiler warnings,
but actually introduced a new one by increasing the buffer size in confdata.c that gets passed
along to file_write_dep in util.c, because buf2's size wasn't increased along with it.
./util.c: In function ‘file_write_dep’:
./util.c:86:26: warning: ‘%s’ directive writing 10 or more bytes into a region of size between 1 and 4097 [-Wformat-overflow=]
86 | sprintf(buf2, "%s%s", dir, name);
| ^~
./util.c:86:9: note: ‘sprintf’ output 11 or more bytes (assuming 4107) into a destination of size 4097
86 | sprintf(buf2, "%s%s", dir, name);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Fix this by increasing the size of buf2 to match the passed buffer size.
Signed-off-by: Devreese Jorik <jorik.devreese@barco.com>
Signed-off-by: Thomas Devoogdt <thomas.devoogdt@barco.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Buildroot commit [1] updated the Kernel for this test, but forgot to
add the "arm/" prefix for the dtb path. Since Linux 6.5, .dts files
have been moved to "arch/arm/boot/dts/" and the test was not able to
find the file anymore, which caused the build to fail.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/14297955524
[1] 74e7e07f83
Signed-off-by: Dowan Gullient <dowan.gullient@smile.fr>
[Julien: reword commit log]
Signed-off-by: Julien Olivain <ju.o@free.fr>
And updates the kernel version used for the test_openjdk test to
the last LTS version (6.18.21). instead of 5.10.3, which is quite
old and may not be compatible with the latest OpenJDK versions.
This change was tested by running the openjdk test.
Signed-off-by: Dowan Gullient <dowan.gullient@smile.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Update the Device Tree name for the vexpress platform to match the
new directory structure introduced in Linux 6.5.
Since kernel 6.5, the 'arch/arm/boot/dts/' directory was restructured
to avoid having thousands of files in a single folder. Device Trees
for ARM (32-bit) are now organized into subdirectories. For the
Versatile Express platform, the DTS file moved to the 'arm/'
subdirectory.
The build was failing with:
make[3]: *** No rule to make target 'arch/arm/boot/dts/vexpress-v2p-ca9.dtb'. Stop.
Adjust BR2_LINUX_KERNEL_INTREE_DTS_NAME from "vexpress-v2p-ca9" to
"arm/vexpress-v2p-ca9".
Signed-off-by: Dowan Gullient <dowan.gullient@smile.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Update the Device Tree name for the vexpress platform to match the
new directory structure introduced in Linux 6.5.
Since kernel 6.5, the 'arch/arm/boot/dts/' directory was restructured
to avoid having thousands of files in a single folder. Device Trees
for ARM (32-bit) are now organized into subdirectories. For the
Versatile Express platform, the DTS file moved to the 'arm/'
subdirectory.
The build was failing with:
make[3]: *** No rule to make target 'arch/arm/boot/dts/vexpress-v2p-ca9.dtb'. Stop.
Adjust BR2_LINUX_KERNEL_INTREE_DTS_NAME from "vexpress-v2p-ca9" to
"arm/vexpress-v2p-ca9".
Signed-off-by: Dowan Gullient <dowan.gullient@smile.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
This commit bump the kernel version of this docker test (5.4->6.18), and thus
fixes several issues preventing the Docker Compose runtime test from
succeeding due to this bump:
Kernel Infrastructure:
The Docker daemon failed to start because the 'nat' table could not
be initialized. This was due to missing legacy Netfilter support.
- Enable CONFIG_NETFILTER_XTABLES_LEGACY to support iptables nat table.
- Enable CONFIG_NAMESPACES and CONFIG_USER_NS for container isolation.
- Enable CONFIG_TUN and CONFIG_VETH for container networking.
- Enable CONFIG_IKCONFIG and CONFIG_IKCONFIG_PROC for easier debugging.
Build Fixes:
While building the kernel tools, the libelf header was missing because
objtool was not handled correctly for this kernel version. This resulted
in a fatal error: "gelf.h: No such file or directory".
- Select BR2_LINUX_KERNEL_NEEDS_HOST_LIBELF=y to build host-libelf.
Resources and Stability:
The previous 512MB of RAM was tight for the stack (Kernel +
Docker + Containerd + Python), potentially leading to Out-Of-Memory kills.
Idem for the disk size for Docker images/layers storage.
- Increase QEMU RAM to 1024MB (-m 1024M).
- Increase rootfs size to 1024MB.
- Switch to EXT4 (BR2_TARGET_ROOTFS_EXT2_4=y) for better stability
and modern feature support required by Docker's storage drivers.
Test Script Adjustments:
- Update rootfs path in the test script to point to rootfs.ext4.
Signed-off-by: Dowan Gullient <dowan.gullient@smile.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Update the kernel configuration to support modern storage stacks
required for ISO booting on x86_64.
The transition from 4.19 to 6.18 requires explicit activation of:
- CONFIG_PCI: To discover the emulated IDE/SATA controllers.
- CONFIG_ATA & CONFIG_ATA_PIIX: Modern libATA drivers for QEMU's
chipset (replacing the legacy IDE subsystem).
- CONFIG_SCSI & CONFIG_BLK_DEV_SR: Necessary to handle the CD-ROM
as a SCSI device (/dev/sr0), which is mandatory for ISO9660.
Without these options, the kernel cannot locate or mount the
rootfs from the ISO image.
Signed-off-by: Dowan Gullient <dowan.gullient@smile.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
BR2_LINUX_KERNEL_NEEDS_HOST_LIBELF=y :
While building the kernel tools, libelf header is missing because objtools
was not compiled by default on older kernel versions. This results in the following error:
[...]tools/objtool/include/objtool/elf.h:10:10: fatal error: gelf.h: No such file or directory
10 | #include <gelf.h>
That is why BR2_LINUX_KERNEL_NEEDS_HOST_LIBELF was selected to build host-libelf.
Signed-off-by: Dowan Gullient <dowan.gullient@smile.fr>
[Julien: updated kernel to 6.18.21]
Signed-off-by: Julien Olivain <ju.o@free.fr>
The __name__ == '__main__' guard allows importing pkg-stats as a
module using importlib, circumventing the normal module filename
requirements. This in turn makes it possible to test/debug individual
functions.
Signed-off-by: Fiona Klute <fiona.klute@gmx.de>
Signed-off-by: Arnout Vandecappelle <arnout@rnout.be>
The file handle can be iterated over directly and each line is used
exactly once, so the only effect of reading all lines into a list
first was higher memory use and complexity.
Signed-off-by: Fiona Klute <fiona.klute@gmx.de>
Signed-off-by: Arnout Vandecappelle <arnout@rnout.be>
The filter is supposed to exclude host/target infra from output if the
respective package is not built with the current
configuration.
However, excluding host packages did not work correctly: If keep_host
is False because the host package is not built, the next branch was
checked and included the host infra in output with "target" type if
the target package is built. For a package that support host and
target build, but gets built only for the target, this leads to output
like (Meson example):
meson (target)
host-meson (target)
Skip host infra in the target branch instead. Also include
Package.infra in Package.__str__() result, which was needed for
debugging this bug.
Signed-off-by: Fiona Klute <fiona.klute@gmx.de>
Signed-off-by: Arnout Vandecappelle <arnout@rnout.be>
Use only one of the classes for "error" or "warning" status so they
look different, and format the error/warning text for both. Do not
make the text a link if the URL is None.
Signed-off-by: Fiona Klute <fiona.klute@gmx.de>
Signed-off-by: Arnout Vandecappelle <arnout@rnout.be>
