Commit f439b47ed "boot/grub2: add RISC-V 64bit EFI support" [1]
introduced the grub2 RISC-V UEFI support.
At that time, it was not possible to have a runtime test for this.
qemu-system-riscv64 is not in the Buildroot Docker reference image,
and the base distribution (Debian 11) does not provide a
sufficiently recent version. A qemu-system-riscv64 >= 8.0.0 is needed.
Commit 0d4177598 "support/testing/infra/emulator.py: add build host
dir to qemu search path" [2] added the capability to use the
Buildroot host-qemu package inside runtime test.
With this support, it is now possible to add a runtime test for a
grub2 UEFI boot on RISC-V 64bit.
Also, due to an upstream edk2 git submodule issue, this commit
includes a workaround to use https://sources.buildroot.net for this
test. See [3].
[1] f439b47ed6
[2] 0d4177598c
[3] 47fc9e5509
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The EDK2 bump to stable202405 introduced an incompatibility with the
Qemu version 5.2.0 present in the reference Docker image. For that
reason, this test was disabled.
This commit enables host-qemu in the build, to be used by the runtime
test infrastructure.
Since this commit switch to a more recent host-qemu, it also removes
firmware options to workaround old qemu bugs.
Those changes are fixing this test, which is why this commit re-enables
it by removing the skipTest() invocation.
Signed-off-by: Julien Olivain <ju.o@free.fr>
Reviewed-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
The EDK2 bump to stable202405 introduced an incompatibility with the
Qemu version 5.2.0 present in the reference Docker image. For that
reason, this test was disabled.
This commit updates ATF to 2.11, and enables host-qemu in the build,
to be used by the runtime test infrastructure.
Those changes are fixing this test, which is why this commit re-enables
it by removing the skipTest() invocation.
Signed-off-by: Julien Olivain <ju.o@free.fr>
Reviewed-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
The EDK2 bump to stable202405 introduced an incompatibility with the
Qemu version 5.2.0 present in the reference Docker image. For that
reason, this test was disabled.
This commit updates ATF to 2.11 and kernel to 6.6.58, and enables
host-qemu in the build, to be used by the runtime test infrastructure.
Those changes are fixing this test, which is why this commit re-enables
it by removing the skipTest() invocation.
Signed-off-by: Julien Olivain <ju.o@free.fr>
Reviewed-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
The relevant qemu emulator is started with pexpect.spawn() with a
default environment (and PATH). This will always use the qemu available
on the host. When running in the reference Docker image, it is using an
old Qemu v5.2.0. This version is getting too old to run with more
recent components, like EDK2 or ATF.
The main reason to use a qemu version preinstalled in the Docker image
was to save some build time. Also, this behavior is opposite to the
"start-qemu.sh" helper script [1], which tries to execute the
Buildroot host-qemu first (and then fallback the the version on the
host system, if available). An option was added in [2] to let the user
have a better control on which version is used.
Also, updating the Buildroot reference Docker image has a larger
impact on the whole project (i.e. updating the image would bring
a newer qemu version, but might break many other things, at a time
which is not always convenient).
We now need some kind of control in runtime tests, to choose between a
"stock" qemu, and a more recent version with all the bells and
whistles.
This commit adds the Buildroot host build directory in the path. This
will make the runtime test infrastructure to use a host-qemu, if
selected by a runtime test configuration.
For now, most of the test can continue to use the qemu version provided
in the Docker image. Only the problematic tests relying on a recent
version will select BR2_PACKAGE_HOST_QEMU=y. The host-qemu package is
usually maintained to a recent version, so this should be sufficient.
[1] https://gitlab.com/buildroot.org/buildroot/-/blob/master/board/qemu/start-qemu.sh.in
[2] 5de78f181a
Signed-off-by: Julien Olivain <ju.o@free.fr>
Reviewed-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
EDK2 version edk2-stable202405 (not yet in Buildroot) has dependencies
on specific Qemu version.
Buildroot guidelines also recommend that "each patch must be complete".
See: [1]. This means, a patch in a series is not allowed to break
something knowingly, even in the same patch series. This rule is
mainly for using git bisect.
We also want to avoid bumping EDK2, update the runtime test infra,
and few tests in a big single commit, for readability. This would mix
too many changes in the same commit.
As a trade off, this commit disables the runtime tests which are known
to be broken by the upcoming edk2 update. Then, those tests will be
individually fixed and re-enabled in followup commits.
This EDK2/Qemu incompatibility only affects runtime tests using EDK2
on the Arm Aarch64 architecture. This commit disable the runtime
tests by adding a "skipTest()" call in:
- tests.boot.test_edk2
- tests.boot.test_grub.TestGrubAArch64EFI
- tests.package.test_fwts
[1] https://buildroot.org/downloads/manual/manual.html#submitting-patches
Signed-off-by: Julien Olivain <ju.o@free.fr>
Reviewed-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Currently the test TestPythonTwisted is not broken [1] since it doesn't
test all modules provided by python-twisted package.
But TestPythonTwisted should have failed due to missing xml in
twisted/web module.
Due to this, TestPythonPy3MagicWormhole runtime test is currently
broken.
Add missing python3-xml dependency and update
sample_python_twisted.py to load twisted/web that uses xml
module.
Fixes (partially due to missing python setuptool in
python-magic-wormhole-mailbox-server):
https://gitlab.com/buildroot.org/buildroot/-/jobs/8199992646
Signed-off-by: Romain Naour <romain.naour@smile.fr>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Generating Bitcoins to an address can take longer than the current
timeout, on slow runners. This commit fixes this issue by increasing
the timeout on specific commands. This issue was also observed more
frequently on newer bitcoin-core version 28.0.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/7782083081
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add a test that runs the dtc commandline tools. To test devicetree
compilation, we use an example devicetree from the dtc project. The
example source is GPL-2.0+ licensed.
Signed-off-by: Brandon Maier <brandon.maier@gmail.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Test the go-bin provider of host-go to build a Go package.
The tests consist of building and installing a Go package in the root
file system of an ARM vexpress QEMU system.
The tests pass if the program runs on the target.
Signed-off-by: Thomas Perale <thomas.perale@mind.be>
[yann.morin.1998@free.fr: use to flannel, which can be vendored today]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The test currently uses tinifier, but it can't be vendored as of today,
so switch to use flannel which can (still?) be...
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Currently, the runtime tests will use the sources.buildroot.org backup
mirror, which is the default setup.
However, in some cases we do not want to use the backup site, because we
want to ensure that the download actually works.
This is the case for vendored packages, like cargo or golang packages,
for whbich we want to check that gthe download still works when we
update the rust or go versions, or when our download helpers change.
So, disable the use of the backup site in all runtime tests, and drop
the no-longer needed special cases.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The mysql virtual package was removed in commit 8708f3a23a
"package/mysql: drop virtual package".
The mariadb runtime test was authored before this mysql virtual
package removal, but was merged after it, in commit 5356754d1e
"support/testing: add mariadb runtime test". Due to this, this test
always failed with the error:
Makefile.legacy:9: *** "You have legacy configuration in your .config! Please check your configuration.". Stop.
This commit fixes the issue by removing the legacy
BR2_PACKAGE_MYSQL=y configuration directive.
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/7540345406
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
This commit adds the -D/--debug command line options to the
support/testing/run-tests script.
Using one of those options will pass "--log-level debug" to the
nose2 invocations. This can be useful to debug nose2 internal
behaviors. See:
https://github.com/nose-devs/nose2/blob/0.15.1/nose2/main.py#L209
For example, calling:
support/testing/run-tests --list --debug
will output:
List of tests
DEBUG:nose2.main:logging initialized debug
...
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Commit 7f0af11cee (support/testing/tests/package/test_flutter.py: new
runtime test) added a simple test that just checked that the systemd
unit launching the application, was active.
It is perfectly possible, from a systemd perspective, that the unit is
active, while the application actually crashes.
Instead, what we need to check, is that the application does actually
render "something"; we don't really care what, as long as we know it is
actually rendering, thus the graphical stack is working and the
aplication stack is running.
Extend the flutter runtime test to also check that the framebuffer is
modified by the application when it is running, similarly to what we do
in the weston test.
We drop the activation of the unit and start it manually, as we want to
check the state of the CRC before, while, and after the application
runs.
We also need to disable the blinking cursor on the console, or we would
not be able to detect whether a change in CRC is due to the application
starting rendering, or to the cursor blinking. We tell the kernel to
disable the cursor with the appropriate kernel command line parameter.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Adam Duskett <aduskett@gmail.com>
Reviewed-by: Adam Duskett <aduskett@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
In 4edb0e3456 (support/testing/tests/package/test_weston.py: new
runtime test), the weston test was introduced, and thus was the first
that needed to test that rendering was happening.
Now we also have a test for a flutter application, and we'll want to
have it test the rendering too.
Move the corresponding code to a helper that can be reused by other
tests, rather than duplicate (or reinvent) it.
Switch weston to using that new helper.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Julien Olivain <ju.o@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
