Parallelizes locale generation based on `BR2_JLEVEL` setting.
Locale generation always runs during the finalize stage and can consume
a significant amount of time. Parallelizing it greatly reduces that time
on multi-core machines.
To parallelize it, we first invoke `localedef` for every locale in
parallel with the `--no-archive` option. This creates the intermediate
locale data instead of writing to the finally archive directly.
Then, we invoke `localedef` again once to create the archive from the
intermediate compiled locale data files.
We have to do it this way because `localedef` does not do any locking
when writing to the archive file, so calling it without `--no-archive`
concurrently could result in a corrupt archive file or an archive file
that is missing some locales.
While we're at it, make two additional improvements:
- Remove locale-archive before adding to it. Otherwise, repeated
applications of target-finalize will keep on growing the file.
- Sort the locales when creating locale-archive so its contents are
reproducible.
We use `find` to collect the installed locales rather than LOCALES. This
makes it possible for something else (skeleton, overlay, custom package)
to create and install additional locales and still have them added to
locale-archive.
Signed-off-by: Gleb Mazovetskiy <glex.spb@gmail.com>
[Arnout:
- Remove -j$(PARALLEL_JOBS), it's already part of $(MAKE)
- Remove HOST_DIR, TARGET_DIR, STAGING_DIR, they're already exported
- Extend commit message
]
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
This test script tests polkit with and without systemd.
The Systemd test does the following:
- The brtest user attempts to restart the systemd-timesyncd service and is
denied.
- A systemd-timesyncd-restart.rules file provided by polkit-rules-test
is copied from /root/ to /etc/polkit-1/rules.d
- The brtest user attempts to restart the systemd-timesyncd service and should
now succeed.
The initd test does the following:
- The brtest user attempts to run the test application "hello-polkit" with the
command "pkexec hello-polkit" and is denied.
- A hello-polkit.rules file provided by polkit-rules-test is copied from /root/
to /etc/polkit-1/rules.d
- The brtest user attempts to re-run the test hello-polkit binary with
"pkexec hello-polkit" and succeeds.
Signed-off-by: Adam Duskett <aduskett@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Commit 2eaa6d0f36 (boot/uboot: fix uboot building host tools on x86
architecture) added use of $(PKG_CONFIG_HOST_BINARY), but forgot to add
the corresponding build-ordr dependency.
Add this missing depenency now.
Additionally, the associated test had an explicit host pkgconf enbled in
its configuration. This is superfluous now that uboot properly depends
on host-pkgconf, so drop that from the test.
Note: it hapenned to work, because host-pkgconf, when explicitly enabled
in the configuration, and without per-package directories, would build
before uboot and thus be available. This would fail with PPD, though,
and thus would break for TLPB.
Reported-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Kory Maincent <kory.maincent@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The make all command run the tools/makefile on the process.
This makefile use "pkg-config" command to support static link.
The issue is the use of pkg-config configured for crosscompiling
to build binaries tools for host architecture.
To fix it, I add pkg-config environment variable to configure it for host.
Add a test to avoid future regress on the build of U-boot.
Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
[yann.morin.1998@free.fr:
- fix mixed space-TAB indentation
- fix check-package
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Commit 3cf2782906 (support/testing/infra/emulator.py: update pre-built
kernels) bumped the default kernels used by the testing infra.
However, the newer armv7 kernel (at least) no longer has support for
lz4-compressed squashfs filesystems.
This breaks the squashfs test:
Filesystem uses "lz4" compression. This is not supported
List of all partitions:
1f00 131072 mtdblock0
(driver?)
1f01 32768 mtdblock1
(driver?)
b300 2048 mmcblk0
driver: mmcblk
No filesystem could mount root, tried:
squashfs
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(179,0)
Updating the kernel again is a little bit cumbersome, while fixing the
actual test is relatively trivial, so this is what we do: we switch
over to lzo, which is supported by the new kernel:
# zcat /proc/config.gz |grep SQUA
CONFIG_SQUASHFS=y
CONFIG_SQUASHFS_ZLIB=y
# CONFIG_SQUASHFS_LZ4 is not set
CONFIG_SQUASHFS_LZO=y
# CONFIG_SQUASHFS_XZ is not set
While at it, also drop the superfluous line disabling gzip compression:
it is part of a choice, so enabling one (lzo here) forcibly disables the
others (of which gzip).
Fixes: 3cf2782906
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Will avoid the following warning:
WARNING: Image format was not specified for
'/home/thomas/projets/outputs/TestExt3/images/rootfs.ext3' and
probing guessed raw. Automatically detecting the format is
dangerous for raw images, write operations on block 0 will be
restricted. Specify the 'raw' format explicitly to remove the
restrictions.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
All the tests that are using if=sd as a Qemu options are changed to
use infra.img_round_power2() instead of simply extending the size of
the image to the next MB boundary, which is not longer sufficient with
Qemu >= 5.1.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: drop now-useless imports]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Since Qemu 5.1, SD card images must have a size that are a power of
two. While some filesystem (such as ext2/3/4) allow to specify the
expected size of the filesystem, others such as SquashFS do not have
this capability.
We were already extending the size of such images to the next 1 MB
boundary using "truncate -s %1M", but that is no longer sufficient. So
instead, we introduce a helper function that extends the size of an
image to the next power of two.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr:
- use f.trunctate() rather than subprocess.call([truncate,...])
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Since Qemu 5.1, the SD card size must be a power of two, so the
default size for ext2/3/4 filesystem images of 60 MB is not
suitable. Since 16 MB is used for the Ext4 test, let's use the same
value for the other tests as well (ext2, ext2r1, ext3). Without this
change, the ext2, ext2r1 and ext3 simply fail to run under Qemu >=
5.1.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The BRTest() class implements an assertRunOk() method that does the
very common work of running a command inside the emulator, and
checking that it is successful.
This commit changes all locations where this .assertRunOk() method can
be used, instead of open-coding the same logic.
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Bump U-Boot version in TestATFAllwinner. Updating U-Boot version to
2021.04 requires the following two changes.
First, after switching to binman, u-boot.itb is no more generated for
64-bit sunxi boards. Combined u-boot-sunxi-with-spl.bin image should
be used instead. This image contains SPL, U-Boot, and FIT image,
where FIT image contains other binaries such as BL31 and SCP.
Second, new U-Boot enables support for System Control Processor (SCP)
firmware. SCP firmware is included by default into FIT image in the
combined u-boot-sunxi-with-spl.bin binary. When SCP is not available
or not needed, it should be explicitly disabled by pointing to an
empty file. Support for Allwinner SCP firmware is not yet available
neither in Buildroot nor in mainline kernel. So disable it for now
using custom U-Boot build options.
Signed-off-by: Sergey Matyukevich <geomatsi@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Switch to mainline TF-A that provides basic support for H5 and A64.
Note that Allwinner platform layer in TF-A does not provide support
for GCC stack protection, so make sure to disable this TF-A feature.
Signed-off-by: Sergey Matyukevich <geomatsi@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
As this version brings support for kernel up to 5.12, we update the
test cases to use the 5.12 kernel.
Signed-off-by: José Luis Salvador Rufo <salvador.joseluis@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: check the two files are identical]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
This patch implements a simple test in which a dummy file system image
is created, then `bmaptool create` and `bmaptool copy` are used to copy
it to another file.
Signed-off-by: Nicolas Carrier <nicolas.carrier@orolia.com>
[Thomas: several reworks, add myself to DEVELOPERS]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: check the two files are identical]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The pre-built vexpress kernel used by the testing infrastructure is a
4.0.0 kernel, which is getting old to be used with reasonably recent
toolchains.
This commit updates the pre-built kernels for both the versatile and
vexpress machines to 5.10.7 (they have already been put online).
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
The br-arm-internal-glibc.config is generally used as a configuration
to test the bleeding edge versions of components. However, it has been
lagging behind somewhat, so let's bring it up-to-date:
- Binutils 2.36.x
- GCC 11.x
Let the fun begin in the autobuilders!
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
This module is only partially compatible with lzlib (which is no longer
maintained).
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
[yann.morin.1998@free.fr: amend commit log about limited compatibility]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
- If a package doesn't have any versioning, ignore and state that
- If a package is virtual, CVE=ignore and CPE state virtual
- For any of these NA cases, don't provide search link and color box
green
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Matthew Weber <matthew.weber@collins.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
has_valid_infra() is incorrectly named; it probably should be named
is_actual_package(), and has_valid_infra() would be changed to
actually represent having an actual infra.
This resolves packages reporting as having no valid package infra and
cleans up reporting cases of CPE and CVEs where there isn't a valid version
or package definition outside Buildroot
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Matthew Weber <matthew.weber@collins.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Currently a verified CPE reports the following if versions are not found
cpe:2.3🅰️qemu:qemu:5.2.0:*:*:*:*:*:*:*
CPE identifier unknown in CPE database (Search)
This patch clarifies the report to state the 'version' is unknown instead
of the 'identifier'.
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Matthew Weber <matthew.weber@collins.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Currently, patches with renames are refused, as they reqire patch 2.7
or newer. So far, we did not require that version because it was too
recent to be widely available.
But patch 2.7 has been released in 2012, almost 9 years ago now; it is
old enough that we can start relying on it.
Add a check that patch is GNU patch 2.7 or newer, and so drop the common
check for patch, and drop the check about renames in apply-patches.sh.
Signed-off-by: Ryota Kinukawa <pojiro.jp@gmail.com>
[yann.morin.1998@free.fr:
- drop common check
- shorten variable names
- drop now-incorrect comment about busybox w/desktop
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Since [1], PIC/PIE is enabled by default but the TestRelroPartial
test expect implicitely PIC/PIE being disabled.
Disable PIC/PIE from the config fragment provided by
TestRelroPartial.
[1] 810ba387be
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/1255661757
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
From [1]
"Xorg does not implement real dynamic linking and requires that its
modules get loaded in the right order."
From /var/log/Xorg.0.0.log:
(II) LoadModule: "modesetting"
(II) Loading /usr/lib/xorg/modules/drivers/modesetting_drv.so
(EE) Failed to load /usr/lib/xorg/modules/drivers/modesetting_drv.so: /usr/lib/xorg/modules/drivers/modesetting_drv.so: undefined symbol: shadowRemove
Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/1255661899
[1] https://forums.gentoo.org/viewtopic-p-8245578.html#8245578
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Since the mesa3d bump to version 21.0.3 [1], the
BR2_PACKAGE_MESA3D_DRI_DRIVER_SWRAST option is not supported anymore
since the mesa DRI swrast driver has been removed upstream
So, switch to Gallium swrast.
[1]15a2f9b819806d38a7d8172a20f80130b1d60e63
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Bernd Kuhls <bernd.kuhls@t-online.de>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Boot a QEMU sbsa-ref machine with ATF, EDK2, GRUB2 and a minimal
kernel. This is a simple but effective test of a compliant setup.
Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
When doing analysis it is helpful to be able to view what CVE have
been patched / diagnosed to not apply to Buildroot. This exposes
that list to the reporting and prevents a step where you have to
dig into the .mk's of a pkg to check for sure what has been
ignored.
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
[yann.morin.1998@free.fr: only set background if there are ignored CVEs]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
For cases of a CPE having a unknown version or when there hasn't
been a CPE verified, proposed a search criteria to help the
user research an update.
(libcurl has NIST dict entries but not this version)
cpe:2.3🅰️haxx:libcurl:7.76.1:*:*:*:*:*:*:*
CPE identifier unknown in CPE database (Search)
(jitterentropy-library package doesn't have any NIST dict entries)
no verified CPE identifier (Search)
Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
[yann.morin.1998@free.fr: fix flake8 issues]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
This script queries the list of CPE IDs for the packages of the
current configuration (based on the "make show-info" output), and:
- for CPE IDs that do not have any matching entry in the CPE
database, it emits a warning
- for CPE IDs that do have a matching entry, but not with the same
version, it generates a snippet of XML that can be used to propose
an updated version to NIST.
Ref: NIST has a group email (cpe_dictionary@nist.gov) used to
recieve these version update and new entry xml files. They do
process the XML and provide feedback. In some cases they will
propose back something different where the vendor or version is
slightly different.
Limitations
- Currently any use of non-number version identifiers isn't
supported by NIST as they use ranges to determine impact
of a CVE
- Any Linux version from a non-upstream is also not supported
without manually adjusting the information as the custom
kernel will more then likely not match the upstream version
used in the dictionary
Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
[yann.morin.1998@free.fr:
- codestyles as spotted by Arnout
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Extend docker_compose_test() to expose /bin on the host to the container
through a volume mount and verify that /bin/busybox can be downloaded and
contains the right data.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Extend docker_test() to expose a random (8888) port to verify that doesn't
fail, and extend the docker-compose test to run the busybox httpd in the
background, expose that as port 80 and verify that /etc/resolv.conf could be
fetched by wget.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The python2 support has been removed since the python-colorzero bump version to 2.0.
[1] 73bf3292e1
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Peter Seiderer <ps.report@gmx.net>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The python2 support has been removed since the python-colorzero bump version to 2.0.
Remove the gpiozero test with python2
[1] 73bf3292e1
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Peter Seiderer <ps.report@gmx.net>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 76b4f9e9b6)
[Peter: drop Makefile change]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Create a new user 'sudotest' to validate that sudo really works (i.e.
properly has setuid).
Creating the user and adding it to sudoers is done at runtime, otherwise
we'd need to add extra files to the config which complicates things a
little bit.
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Kernel 4.16.7 is old enough to produce the "multiple definition of `yylloc'"
error which is fixed in newer versions.
Bump the test kernel version from 4.16.7 to 5.10.34 to prevent this error wwhen
building the test image.
Signed-off-by: Adam Duskett <aduskett@gmail.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Test that the TAICLOCK and TCP servers are working.
Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Arnout: indent config lines more]
Test that s6-rc service database compilation is working.
Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Arnout: indent config lines more]
Test that a few basis utilities are working.
Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Arnout: indent config lines more]
Test that directory scanning and supervision is working.
Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Arnout: properly indent, and use textwrap to dedent again.]
Test that the interpreter can run a basic command.
Signed-off-by: Dick Olsson <hi@senzilla.io>
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
[Arnout: indent config lines more]
Commit 54d3d94b6e broke the 'hg' download
method, in a similar way as it broke the 'git' download method (later fixed
with commit b70ce56651), by introducing extra
output on stdout in a case where the output is redirected.
In the case of 'hg', the 'hg archive' step uses shell redirection rather
than directly letting hg write the output file, since commit
76b51f90c0.
As a result, the extra print added by the _hg function is prepended to the
actual archive, causing an invalid archive.
Fix by using the _plain_hg function instead. The disadvantage is that the
command for 'hg archive' is no longer printed.
Signed-off-by: Thomas De Schampheleire <thomas.de_schampheleire@nokia.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
