Commit Graph

1185 Commits

Author SHA1 Message Date
Fabrice Fontaine
f7922a57f5 Replace LIBFOO_CPE_ID_VERSION_MINOR by LIBFOO_CPE_ID_UPDATE
Replace LIBFOO_CPE_ID_VERSION_MINOR by LIBFOO_CPE_ID_UPDATE to better
"comply" with the official "Well-Formed CPE Name Data Model" parameters:
 - https://csrc.nist.gov/publications/detail/nistir/7695/final
 - https://nvlpubs.nist.gov/nistpubs/Legacy/IR/nistir7695.pdf

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-01-31 23:21:38 +01:00
Thomas Petazzoni
c9a1a1eb91 support/scripts/pkg-stats: properly handle host packages with -c option
In commit 7a607dab33
("support/scripts/pkg-stats: support generating stats based on
configured packages"), we added a -c option to pkg-stats to generate a
report based on the list of packages enabled in the configuration,
rather than for all packages.

This is done based on the list of packages returned in JSON format by
"make show-info". However, we use the keys of the JSON dict returned
by "make show-info", which include the host- prefix of host
packages. Due to this, none of the host packages are currently
matching and therefore they are not reported in the pkg-stats -c
output.

This commit fixes that by using the recently introduced "name"
property in the "make show-info" JSON dict.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
[yann.morin.1998@free.fr: use anonymous '_' for unused variable]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-01-31 23:01:21 +01:00
Peter Korsgaard
a93208ccec Update for 2020.11.2
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 551cb63007)
[Peter: drop Makefile changes]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2021-01-31 22:44:08 +01:00
Thomas Petazzoni
52a53c97e5 support/scripts/pkg-stats: fix flake8 warnings
Fixes:

support/scripts/pkg-stats:148:17: E741 ambiguous variable name 'l'
support/scripts/pkg-stats:379:9: E741 ambiguous variable name 'l'

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-01-31 12:03:12 +01:00
Thomas Petazzoni
67768e97b0 support/scripts/pkg-stats: drop unused --cpeid option
The --cpeid option was mistakenly introduced by commit
92e7089a8c ("support/script/pkg-stats:
show CPE ID in results") but is in fact not necessary.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-01-31 12:03:11 +01:00
Thomas Petazzoni
cad5a69f83 support/scripts/pkg-stats: drop unused cpeid_name() function
The cpeid_name() function is not used anywhere, drop it.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-01-31 12:03:00 +01:00
Thomas Petazzoni
68364a4b58 support/scripts/pkg-stats: fix the status reporting of CVEs
Since commit bd665d182c
("support/scripts/pkg-stats: improve rendering of CVE information"),
we have better reporting of CVE related information, based on
pkg.status['cve']. However, this commit broke pkg-stats when the
--nvd-path option is not passed, and therefore no CVE information is
available.

This commit fixes that, by making use of the is_status_ok(),
is_status_error() and is_status_na() methods recently introduced.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-01-31 12:02:42 +01:00
Thomas Petazzoni
7f83ad7467 support/scripts/pkg-stats: improvements in is_status_*() methods
Make is_status_ok() work when the given status name is not even listed
in the status dict. This will be necessary for following commits.

Introduced similar methods for the error and na status, which will be
used in following commits.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-01-31 12:02:41 +01:00
Fabrice Fontaine
4b6202f721 Replace LIBFOO_CPE_ID_NAME by LIBFOO_CPE_ID_PRODUCT
Replace LIBFOO_CPE_ID_NAME by LIBFOO_CPE_ID_PRODUCT to better "comply"
with the official "Well-Formed CPE Name Data Model" parameters:
 - https://csrc.nist.gov/publications/detail/nistir/7695/final
 - https://nvlpubs.nist.gov/nistpubs/Legacy/IR/nistir7695.pdf

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-01-21 22:43:24 +01:00
Thomas Petazzoni
40bb37bd70 utils/getdeveloperlib.py: use relative paths for files
Using absolute paths within getdeveloperlib isn't very sensible, it
makes a lot more sense to handle everything as relative paths from the
top-level Buildroot source directory.

parse_developers() is changed to no longer take the base path as
argument: it is automatically calculated based on the location of
utils/getdeveloperlib.py. Then, the rest of the logic is adjusted to
use relative paths, and prepend them with the base "brpath" when
needed.

This commit allows pkg-stats to report correct developers information
even when executed from an out of tree directory.

Before this patch:

$ ~/buildroot/support/scripts/pkg-stats -p ipmitool --json out.json
$ cat out.json | jq '.packages.ipmitool.developers'
[]

$ cat out.json | jq '.defconfigs.stm32f469_disco'
{
  "name": "stm32f469_disco",
  "path": "configs/stm32f469_disco_defconfig",
  "developers": []
}

After this patch:

$ ~/buildroot/support/scripts/pkg-stats -p ipmitool --json out.json
$ cat out.json | jq '.packages.ipmitool.developers'
[
  "Floris Bos <bos@je-eigen-domein.nl>",
  "Heiko Thiery <heiko.thiery@gmail.com>"
]
$ cat out.json | jq '.defconfigs.stm32f469_disco'
{
  "name": "stm32f469_disco",
  "path": "configs/stm32f469_disco_defconfig",
  "developers": [
    "Christophe Priouzeau <christophe.priouzeau@st.com>"
  ]
}

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Tested-by: Heiko Thiery <heiko.thiery@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-01-17 15:32:08 +01:00
Thomas Petazzoni
ffb2620405 support/scripts/pkg-stats: fix Python 3.8 deprecation warning
With Python 3.8, the following deprecation warnings are emitted:

/home/thomas/projets/buildroot/./support/scripts/pkg-stats:418: DeprecationWarning: The explicit passing of coroutine objects to asyncio.wait() is deprecated since Python 3.8, and scheduled for removal in Python 3.11.

/home/thomas/projets/buildroot/./support/scripts/pkg-stats:536: DeprecationWarning: The explicit passing of coroutine objects to asyncio.wait() is deprecated since Python 3.8, and scheduled for removal in Python 3.11.

The correct way to pass coroutines is to use asyncio.create_task(),
but this is rather new method (Python 3.7), and using it breaks
compatibility with older Python versions. As suggested at
https://docs.python.org/3/library/asyncio-task.html#asyncio.create_task,
use the more cryptic, but also more compatible asyncio.ensure_future()
method.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-01-17 15:32:05 +01:00
Thomas Petazzoni
824032d168 support/scripts/pkg-stats: import cve module only when needed
The cve module needs ijson, which may not be installed. Since cve
matching is only enabled when --nvd-path is passed, it is a bit silly
to error out about ijson being missing if it's not used.

So instead of unconditionally importing the cve module, only do it
conditionally.

However, instead of doing it right at the point where it is used, we
do it at the beginning of the main() function. Indeed, if the cve
module is needed but cannot be imported, we want to error out
immediately rather than doing a whole bunch of things, and failing on
the user later on in the middle of the pkg-stats execution.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Tested-by: Heiko Thiery <heiko.thiery@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-01-17 15:32:04 +01:00
Thomas Petazzoni
01533244a5 support/config-fragments/autobuild/bootlin-x86-64-glibc: use toolchain-external-bootlin
Until now the bootlin-x86-64-glibc was using a Bootlin toolchain as a
custom external toolchain. However, now that we have the
toolchain-external-bootlin package explicitly supporting Bootlin
toolchains as known toolchain profiles, it makes sense to use
that. Indeed, this will ensure that this autobuilder configuration
will use the latest available version of the Buildroot toolchain for
x86-64 glibc.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-01-17 10:29:17 +01:00
Sergey Matyukevich
5d54214049 support/testing: fix TestATFAllwinner test
Fix U-Boot config snippet in TestATFAllwinner. Bump U-Boot
version to fix DTC build on hosts with gcc 10 and add
pylibfdt dependency.

Signed-off-by: Sergey Matyukevich <geomatsi@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-01-16 22:35:20 +01:00
Sergey Matyukevich
22188ff6e0 support/testing: remove TestATFMarvell
There were only two users of Marvell ATF: SolidRun MacchiatoBin board
and ClearFrog GT 8k board. After mv-ddr-marvell package update both
boards switched to upstream ATF. Remove tests for now unused
Marvell ATF.

Signed-off-by: Sergey Matyukevich <geomatsi@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-01-16 22:35:18 +01:00
Yann E. MORIN
ec50e407be support/dependencies: drop check for maximal tar version
So far, we checked that the tar present on the host was at most tar
1.29, because tar 1.30 changed the way it generates archives.

Having a maximum tar version requirement meant that we would eventually
always have to build our own host-tar, as distributions are updating
the version they use.

But now, we have found a way to generate reproducible archives starting
with tar 1.27 onward, so we no longer need the check for a maximum tar
version, so we can drop that requirement.

Note: this is semantically a revert of b8fa273d50 (check-host-tar.sh:
blacklist tar 1.30+), but keeping the new, mostly-linear code-path.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Vincent Fazio <vfazio@xes-inc.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Vincent Fazio <vfazio@xes-inc.com>
2021-01-10 22:06:58 +01:00
Yann E. MORIN
c043ecb20c support/download: change format of archives generated from svn
Like we recently did for git, switch the archives generated from
subversion to be reproducible whatever the tar version.

We have no in-tree users of the svn backend which also has hashes,
so no hash to update.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Heiko Thiery <heiko.thiery@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Vincent Fazio <vfazio@xes-inc.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Vincent Fazio <vfazio@xes-inc.com>
2021-01-10 22:06:58 +01:00
Yann E. MORIN
ccfae17015 support/download: cleanup svn backend
Commit 89f5e9893 (support/download/svn: generate reproducible svn
archives) did what it said, but can be siplified a bit.

Indeed, we are doing an svn export, so we won't have any of the .svn
directories, neither at the root of the extract, nor in any of the
sub-directories.

As such, we do not need to filter them out  when we generate the list
of files to include in the archive.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Heiko Thiery <heiko.thiery@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Vincent Fazio <vfazio@xes-inc.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Vincent Fazio <vfazio@xes-inc.com>
2021-01-10 22:06:58 +01:00
Yann E. MORIN
5b95a5dc27 support/download: change format of archives generated from git
Switch to using the tarball helper, that can generate reproducible
archives whatever the tar version >= 1.27.

However, those archives are not identical to the previous ones generated
in the (now-broken) gnu format.

To avoid any clashing between old and new archives, and new and old
Buildroot versions, we need to name the new generated archives
differently from the existing ones.

So, we bump the git-specific format-version to -br1.

The %ci date  has been supported by git back to 1.6.0, released August
2008); it is not strictly ISO8601, but is still accepted as a PAX date
header. The strict ISO8601 placeholder, %cI, was only introduced with
2.2.0, release in November 2014, so too recent to be widely available.

As the format and the names of the archives changes, we need to update
all the hash files with the new names and hashes.

Of all the bootloaders that have a git download method, vexpress-firmware
is the only one to have a hash. Others have no hash files, or they have
explicitly set BR_NO_CHECK_HASH_FOR.

For the packages, linux-headers is the special snowflake, as the git
download is only for custom git tree, so it is excluded from the hash
verification with BR_NO_CHECK_HASH_FOR.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Vincent Fazio <vfazio@xes-inc.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Vincent Fazio <vfazio@xes-inc.com>

    ---8<------8<------8<------8<---
    #!/bin/sh
    # Find and download all packages using git as backend.
    # Manually fix hashes for affected packages.

    # Packages that only have a host variant
    HOST_ONLY='imx-mkimage|mxsldr|netsurf-buildsystem|opkg-utils|prelink-cross|qoriq-rcw|vboot-utils'

    # Packages that have a non-git main _SOURCE, and/or which
    # have BR_NO_CHECK_HASH_FOR for the git _SOURCE
    NOT_GIT='aufs|aufs-util|xenomai|linux-headers'

    export BR2_DL_DIR=$(pwd)/temp-dl-dir

    make defconfig
    make $( git grep -l -E 'SITE_METHOD[[:space:]]*:?=[[:space:]]*git\>|_SITE[[:space:]]*:?=[[:space:]]*git:' \
                boot/vexpress-firmware/ package/ \
            |sed -r -e 's,.*/([^/]+)\.mk,\1,' \
            |sed -r -e '/^('"${NOT_GIT}"')$/d;' \
                    -e 's/^('"${HOST_ONLY}"')/host-\1/;' \
                    -e 's/$/-legal-info/;'
          )

    ---8<------8<------8<------8<---
2021-01-10 22:06:58 +01:00
Yann E. MORIN
cbe95b1a45 support/download: add helper to generate a reproducible archive
We currently need to generate reproducible archives in at least two
locations: the git and svn download backends. We also know of some
future potential use (e.g. the other download backends, like cvs, or
in the upcoming download post-processors for vendoring, like cargo
and go).

However, we are currently limited to a narrow range of tar versions
that we support, to create reproducible archives, because the gnu
format we use has changed with tar 1.30.

As a consequence, and as time advances, more and more distros are,
or will eventually start, shipping with tar 1.30 or later, and thus
we need to always build our on host-tar.

Now, thanks to some grunt work by Vincent, we have a set of options
that we can pass tar, to generate reproducible archives back from
tar-1.27 and up through tar-1.32, the latest released version.

However, those options are non-trivial, so we do not want to have
to repeat those (and maintain them) in multiple locations.

Introduce a helper that can generate a reproducible archive from
an input directory.

The --pax-option, to set specific PAX headers, does not accept
RFC2822 timestamps which value are too away from some fixed point
(set atcompile-time?):
    tar: Time stamp is out of allowed range

However, the same timestamps passed as strict compliant ISO 8601 are
accepted, so that's what we expect as a date format.

Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Vincent Fazio <vfazio@xes-inc.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Reviewed-by: Vincent Fazio <vfazio@xes-inc.com>

    ---8<------8<------8<------8<---
    # Here is a Makefile used to test all the versions of tar, with
    # different output formats and different sets of options:
    # Versions prior to 1.27 do not build on recent machines, because
    # 'gets()' got removed (rightfully so), so don't count them as
    # candidates.
    VERSIONS = 1.27 1.27.1 1.28 1.29 1.30 1.31 1.32
    DATE = Thu 21 May 2020 06:44:11 PM CEST

    TARS = \
    	$(patsubst %,test_gnu_%.tar,$(VERSIONS)) \
    	$(patsubst %,test_posix_%.tar,$(VERSIONS)) \
    	$(patsubst %,test_posix_paxoption_%.tar,$(VERSIONS))

    all: $(TARS)
    	sha1sum $(^)

    .INTERMEDIATE: test_%.tar
    test_gnu_%.tar: tar.% list
    	./$(<) cf - -C test \
    		--transform="s#^\./#test-version/#" \
    		--numeric-owner --owner=0 --group=0 \
    		--mtime="$(DATE)" \
    		--format=gnu \
    		-T list \
    	>$(@)
    test_posix_%.tar: tar.% list
    	./$(<) cf - -C test \
    		--transform="s#^\./#test-version/#" \
    		--numeric-owner --owner=0 --group=0 \
    		--mtime="$(DATE)" \
    		--format=posix \
    		-T list \
    	>$(@)
    test_posix_paxoption_%.tar: tar.% list
    	./$(<) cf - -C test \
    		--transform="s#^\./#test-version/#" \
    		--numeric-owner --owner=0 --group=0 \
    		--mtime="$(DATE)" \
    		--format=posix \
    		--pax-option='delete=atime,delete=ctime,delete=mtime' \
    		--pax-option='exthdr.name=%d/PaxHeaders/%f,exthdr.mtime={$(DATE)}' \
    		-T list \
    	>$(@)

    list: .FORCE
    list: test
    	(cd test && find . -not -type d ) |LC_ALL=C sort >$(@)

    LONG = L$$(for i in $$(seq 1 200); do printf 'o'; done)ng
    test: .FORCE
    test:
    	rm -rf test
    	mkdir -p test/bar
    	echo foo >test/Foo
    	echo bar >test/bar/Bar
    	ln -s bar/Bar test/buz
    	echo long >test/Very-$(LONG)-filename
    	ln test/Very-$(LONG)-filename \
    	   test/short

    .PRECIOUS: tar.%
    tar.%: tar-%
    	cd $(<) && ./configure
    	$(MAKE) -C $(<)
    	install -m 0755 $(<)/src/tar $(@)

    .PRECIOUS: tar-%
    tar-%: tar-%.tar.gz
    	tar xzf $(<)

    .PRECIOUS: tar-%.tar.gz
    tar-%.tar.gz:
    	wget "https://ftp.gnu.org/gnu/tar/$(@)"

    .FORCE:

    clean:
    	rm -rf tar-* tar.* test_* test list
    ---8<------8<------8<------8<---
2021-01-10 22:04:33 +01:00
Arnout Vandecappelle (Essensium/Mind)
24dc403be3 support/scripts/pkg-stats: fix flake8 errors
support/scripts/pkg-stats:81:22: E211 whitespace before '('
support/scripts/pkg-stats:404:1: E305 expected 2 blank lines after class or function definition, found 1
support/scripts/pkg-stats:561:12: E713 test for membership should be 'not in'
support/scripts/pkg-stats:567:1: E302 expected 2 blank lines, found 1
support/scripts/pkg-stats:595:1: E302 expected 2 blank lines, found 1
support/scripts/pkg-stats:1051:1: E302 expected 2 blank lines, found 1
support/scripts/pkg-stats:1057:1: E302 expected 2 blank lines, found 1

Also fix:
support/scripts/pkg-stats:1054:5: E722 do not use bare 'except'
found by a more recent flake8 version. The exception may be either
IndexError or AttributeError, so use Exception to catch either.

Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
2021-01-05 23:34:28 +01:00
Thomas Petazzoni
bd665d182c support/scripts/pkg-stats: improve rendering of CVE information
This commit improves pkg-stats to fill in pkg.status['cve'] depending
on the situation for CVEs affecting this package. They are then used
in the HTML rendering.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-01-04 21:39:11 +01:00
Thomas Petazzoni
78d7521f82 support/scripts/pkg-stats: ignore packages with no valid infra and no version for CVE checking
Virtual packages (with in pkg-stats speak have "no valid
infrastructure") and packages that have no version specified cannot be
used for CVE checking. They trigger a bunch of warnings from the CVE
checking code, as it cannot parse their version: they don't have any
version. So instead, we simply skip those packages.

A follow-up commit will improve the reporting to be able to
distinguish those packages from packages that have seen their CVEs
checked and don't have any reported.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-01-04 21:38:35 +01:00
Thomas Petazzoni
e3ef352ef6 support/scripts/{pkg-stats, cve.py}: support CPE ID based matching
This commit modifies cve.py, as well as its users cve-checker and
pkg-stats to support CPE ID based matching, for packages that have CPE
ID information.

One of the non-trivial thing is that we can't simply iterate over all
CVEs, and then iterate over all our packages to see which packages
have CPE ID information that match the CPEs affected by the
CVE. Indeed, this is an O(n^2) operation.

So instead, we do a pre-filtering of packages potentially affected. In
check_package_cves(), we build a cpe_product_pkgs dict that associates
a CPE product name to the packages that have this CPE product
name. The CPE product name is either derived from the CPE information
provided by the package if available, and otherwise we use the package
name, which is what was used prior to this patch.

And then, when we look at CVEs, we only consider the packages that
have a CPE product name matching the CPE products affected by the
CVEs. This is done in check_package_cve_affects().

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-01-04 21:38:20 +01:00
Gregory CLEMENT
92e7089a8c support/script/pkg-stats: show CPE ID in results
This commit improves the pkg-stats script to show the CPE ID of
packages, if available. For now, it doesn't use CPE IDs to match CVEs.

Signed-off-by: Gregory CLEMENT <gregory.clement@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2021-01-04 21:36:50 +01:00
Yann E. MORIN
728484538a support/testing: fix flake8 issue in CPE ID test
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2021-01-02 18:06:24 +01:00
Francois Perrad
2949f423a4 package/lualdap: new package
Signed-off-by: Francois Perrad <francois.perrad@gadz.org>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-12-31 15:13:06 +01:00
Marcin Niestroj
55a6ff34ba support/testing: add pytest-asyncio test
Signed-off-by: Marcin Niestroj <m.niestroj@grinn-global.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-12-31 15:10:16 +01:00
Thomas Petazzoni
8cba5463de support/scripts/gen-bootlin-toolchains: do not assume RPC for glibc toolchains
The logic in gen-bootlin-toolchains was assuming all glibc toolchains
have RPC support, which is no longer true since glibc 2.32 has dropped
RPC support.

It turns out that gen-bootlin-toolchains already had some proper logic
that selects BR2_TOOLCHAIN_HAS_NATIVE_RPC depending on the presence of
BR2_TOOLCHAIN_EXTERNAL_INET_RPC in the toolchain fragment. As such
toolchain fragments have been fixed in https://toolchains.bootlin.com,
we can now rely on this to properly decide if the toolchain has RPC
support or not.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-12-29 22:03:17 +01:00
Peter Korsgaard
e9bf54ab23 Update for 2020.11.1
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
(cherry picked from commit 804a9e1865)
[Peter: drop Makefile changes]
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-12-27 15:39:52 +01:00
Fabrice Fontaine
37f197f863 support/dependencies: set cmake version min to 3.15
quazip requires cmake 3.15 since version 1.0 and
89e7c201f0
818adc8224

The rationale for this requirement is that "default locations for the
install(TARGETS command based on the GNUInstallDirs package were only
added in 3.14" and "3.15 is not that much of a difference from 3.14 and
it introduced a lot of useful UI improvements.":
https://github.com/stachenov/quazip/issues/82

Fixes:
 - http://autobuild.buildroot.org/results/5d848a46109aef448ea1d1b857a500d9461dc2d9

Note: we also have some patches to allow some packages to build with
cmake-3.10, and this will not be tenable over the long run.

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
[yann.morin.1998@free.fr: add the "note"]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
2020-12-25 16:21:27 +01:00
Romain Naour
d673618342 support/scripts/boot-qemu-image.py: don't fail if start-qemu.sh is missing
When boot-qemu-image.py script was added, we wanted to run
each qemu defconfig in gitlab, so we expect that all qemu
defconfig generate the script start-qemu.sh in images
directory.

Don't make it a hard requirement even if we prefer to be
able to do a runtime test for each qemu defconfig.

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-12-05 22:33:27 +01:00
Peter Korsgaard
6eff339082 Merge branch 'next'
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-12-03 10:30:49 +01:00
Peter Korsgaard
9624a001f9 Update for 2020.11
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-12-02 23:21:32 +01:00
Thomas Petazzoni
3950c53cd0 support/testing/tests/core/test_cpeid: new test
This commit adds a number of test cases to verify that the CPE_ID_*
variables are properly handled by the generic package infrastructure
and that the "make show-info" JSON output matches what we expect.

A total of 5 different example packages are used to exercise different
scenarios of CPE_ID_* variables usage.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-11-26 16:36:56 +01:00
Thomas Petazzoni
1ff7f003e1 support/scripts/cve.py: properly match CPEs with version '*'
Currently, when the version encoded in a CPE is '-', we assume all
versions are affected, but when it's '*' with no further range
information, we assume no version is affected.

This doesn't make sense, so instead, we handle '*' and '-' in the same
way. If there's no version information available in the CVE CPE ID, we
assume all versions are affected.

This increases quite a bit the number of CVEs and package affected:

-    "total-cves": 302,
-    "pkg-cves": 100,
+    "total-cves": 597,
+    "pkg-cves": 135,

For example, CVE-2007-4476 has a CPE ID of:

    cpe:2.3🅰️gnu:tar:*:*:*:*:*:*:*:*

So it should be taken into account. In this specific case, it is
combined with an AND with CPE ID
cpe:2.3suse:suse_linux:10:*:enterprise_server:*:*:*:*:* but since
we don't support this kind of matching, we'd better be on the safe
side, and report this CVE as affecting tar, do an analysis of the CVE
impact, and document it in TAR_IGNORE_CVES.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-11-26 16:31:37 +01:00
Baruch Siach
1b1c049af2 support/dependencies: clarify intended use of host bison/flex
We should not rely on host installed bison/flex for target code. This
ensures better reproducibility of generated code.

http://lists.busybox.net/pipermail/buildroot/2020-November/296786.html

Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-11-22 15:24:12 +01:00
Thomas Petazzoni
3de9337781 support/scripts/cve-checker: remove script
Now that pkg-stats is able to generate its output based on the list of
packages enabled in the current configuration, cve-checker doesn't
serve any purpose.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-11-11 12:07:37 +01:00
Thomas Petazzoni
7a607dab33 support/scripts/pkg-stats: support generating stats based on configured packages
pkg-stats was initially a Buildroot maintenance oriented tool: it was
designed to examine all Buildroot packages and provide
statistics/details about them.

However, it turns out that a number of details provided by pkg-stats,
especially CVEs, are relevant also for Buildroot users, who would like
to check regularly if their specific Buildroot configuration is
affected by CVEs or not, and possibly check if all packages have
license information, license files, etc.

The cve-checker script was recently introduced to provide an output
relatively similar to pkg-stats, but focused on CVEs only.

But in fact, its main difference is on the set of packages that we
consider: pkg-stats considers all packages, while cve-checker uses
"make show-info" to only consider packages enabled in the current
configuration.

So, this commit introduces a -c option to pkg-stats, to tell pkg-stats
to generate its output based on the list of configured packages. -c is
mutually exclusive with the -p option (explicit list of packages) and
-n option (a number of packages, picked randomly).

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-11-11 12:03:55 +01:00
Thomas Petazzoni
ae86067a15 support/scripts/pkg-stats: allow to run script outside of the top-level directory
Currently, pkg-stats expects being executed from Buildroot's top-level
source directory. As we are going to extend pkg-stats to cover only
the packages available in the current configuration, it makes sense to
be able to run it from the output directory, which can be anywhere
compared to Buildroot's top-level directory.

This commit adjusts pkg-stats to this, by inferring all Buildroot
paths based on the location of the pkg-stats script itself.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-11-11 12:01:21 +01:00
Romain Naour
1989a49feb support/testing/tests/init/test_systemd: update to BR2_PACKAGE_SYSTEMD_JOURNAL_REMOTE
The commit [1] moved systemd-journal-gatewayd into systemd-journal-remote
option. Update to BR2_PACKAGE_SYSTEMD_JOURNAL_REMOTE in the testsuite
when BR2_PACKAGE_SYSTEMD_JOURNAL_GATEWAY is used.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981805
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981811

[1] e46fe9a6f2

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-11-11 11:49:59 +01:00
Romain Naour
f52a9835fa support/testing: update to the new text representation of capabilities
Since libcap 2.41, the text representation of capabilities now use
the '=' instead of '+' [1].

This break our capabilities tests since we still use the old
representation.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/morgan/libcap.git/commit/?id=177cd418031b1acfcf73fe3b1af9f3279828681c

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981737

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-11-11 10:12:48 +01:00
Romain Naour
7b9762f4ab support/testing/test_hardening: add missing Kconfig symbol
BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y is needed to use the
custom external toolchain x86-i686--glibc--bleeding-edge-2018.11-1.tar.bz2

Otherwise the symbol BR2_TOOLCHAIN_EXTERNAL_URL is lost.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981738
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981739
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981740
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981741
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981742
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981743

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-11-11 10:11:12 +01:00
Romain Naour
ddf40cffea support/testing/test_syslinux: add missing Kconfig symbol
BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y is needed to use the
custom external toolchain x86-i686--glibc--bleeding-edge-2018.11-1.tar.bz2

Otherwise the symbol BR2_TOOLCHAIN_EXTERNAL_URL is lost.

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981734
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981733

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-11-11 09:06:03 +01:00
Romain Naour
14ed65e3a6 support/testing: TestInitSystemSystemdRwIfupdown test expect a RW rootfs
When running the TestInitSystemSystemdRwIfupdown test, the rootfs must
be in read-write mode. The commit log [1] introducing systemd tests say
so:

"basic systemd, read-write, network w/ ifupdown"

With systemd 246.5, the service systemd-update-done return an error code
when it can't write on the filesystem (/etc)

[1] 117835d5fc
[2] 8019995e9a

Fixes:
https://gitlab.com/buildroot.org/buildroot/-/jobs/830981813

Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Yann E. MORIN <yann.morin.1998@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-11-09 21:10:35 +01:00
Thomas Petazzoni
0a5c9f824a support/testing/tests: fix flake8 warnings
Fixes the following flake8 warnings:

support/testing/tests/core/test_selinux.py:21:1: E302 expected 2 blank lines, found 1
support/testing/tests/core/test_selinux.py:38:1: E302 expected 2 blank lines, found 1
support/testing/tests/core/test_selinux.py:51:1: E302 expected 2 blank lines, found 1
support/testing/tests/core/test_selinux.py:62:1: E302 expected 2 blank lines, found 1
support/testing/tests/core/test_selinux.py:65:14: E127 continuation line over-indented for visual indent
support/testing/tests/init/test_systemd_selinux.py:53:1: E302 expected 2 blank lines, found 1
support/testing/tests/init/test_systemd_selinux.py:64:1: E302 expected 2 blank lines, found 1

Interestingly, the "continuation line over-indented for visual indent"
shows up only once, while the same pattern is there at multiple places
in the file. We fix all places with that over-indentation pattern.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
2020-11-06 08:11:25 +01:00
Fabrice Fontaine
35c3a1e693 support/scripts/apply-patches.sh: manage uncompression
Extract from bug report:

"Code line 120 to line 128 is to check whether the patch containing
"rename from" and "rename to". But it directly use grep to find,
ignoring the patch may be a tar file or else. It can only work on patch
of textfile form."

Fixes:
 - https://bugs.buildroot.org/show_bug.cgi?id=11931

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-10-17 14:44:05 +02:00
Thomas Petazzoni
03f556d7d2 support/testing/tests/toolchain/test_external_bootlin: regenerate with new PowerPC toolchains
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-10-15 00:00:57 +02:00
Thomas Petazzoni
1f3e5e6b3d support/scripts/gen-bootlin-toolchains: add support for more PowerPC toolchains
The 2020.08-1 release of Bootlin toolchains has brought support for 3
additional architecture variants, so let's support them.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-10-14 23:57:54 +02:00
Romain Naour
4ba1ce24b9 support/scripts/gen-bootlin-toolchains: add missing MMU dependencies
Signed-off-by: Romain Naour <romain.naour@gmail.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
2020-10-14 23:56:17 +02:00