Currently, the gdbinit is generated and installed during post install
hooks, either from the gdb package, or from the external-toolchain
package. When using per-package directories (PPD), the staging directory
of the either package is stored in the generated gdbinit, which is not
going to be valid when all the PPD staging dirs get merged into the
final staging: it would lack any library installed afterwards, i.e.
mostly everything would be missing (but the libraries from the C
toolchain in the case of an external toolchain).
Similarly, all the RPATH will point to various PPD drectories. This
does not cause any issue when the final host is aggregated, because the
PPD directories still exist when we call programs from there (e.g. from
the fs infra, or from post-image scripts).
However, we knew that would not always be possible to keep the PPD
directories: we have the prepare-sdk rule that runs a cleanup pass on
the RPATH, and also applies the generic PPD fixups.
When we introduced prepare-sdk in c32ad51cbf (core/sdk: generate the
SDK tarball ourselves), we did not yet have support for PPD for the host
directory, and especially, we did not have the host-finalize rule, which
was only introduced in d0f4f95e39 (Makefile: rework main directory
creation logic) which kick-started the introduction of PPD.
At that point, we did not realise that the rpath fixups from
prepare-sdk, would be better moved to the new host-finalize rule,
because that had no impact unless one would need an SDK.
Later, in 25e60fbe1c (Makefile: fix SDK relocation for
per-package-dirs), we eventually introduced the PPD generic fixups in
the prepare-sdk rule. Again, we did not realise that those fixups would
be better placed in the host-finalize rule rather than the prepare-sdk.
While fixing the RPATH in host-finalize is not critical, fixing up the
PPD paths actually is, as the gdbinit case demonstrate.
As such, move the PPD fixups to the host-finalize step, and while at it,
also move the RPATH fixups.
This now does not leave much to do in the prepare-sdk step, and that
could very well be moved to the host-finalize rule as well. However,
some people may have started to rely on prepare-sdk in its 6 years of
existence, and the little script it installs is not needed unless one
really needs an SDK. So leave it as it is for now.
Reported-by: Casey Reeves <casey@xogium.me>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Tested-by: Casey Reeves <casey@xogium.me>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Brandon Maier <Brandon.Maier@collins.com>
Tested-by: Brandon Maier <brandon.maier@collins.com>
Acked-by: TIAN Yuanhao <tianyuanhao3@163.com>
The packages 'python-jsonschema-specifications', 'pydantic-core', and
likely other packages based on Python maturin silently stopped
installing files to the host and target directories after commit
a14c862c08.
A similar issue was reported by the Conda project[1]. It seems some
build tools scan for gitignore files to decide what files to build
and install.
I attempted various combinations of gitignore patterns including
mimicking the Conda project's gitignore[2], but the only thing that has
worked reliably is the original `/output` ignore in the root Buildroot
directory.
This reverts commit a14c862c08.
[1] https://github.com/PyO3/maturin/issues/1911
[2] https://github.com/conda-forge/conda-smithy/blob/main/conda_smithy/feedstock_content/.gitignore
Reported-by: Bryan Brattlof <bb@ti.com>
Signed-off-by: Brandon Maier <brandon.maier@collins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Add missing ' to fix `make help'.
This fixes the following error:
/bin/bash: -c: line 1: unexpected EOF while looking for matching `''
make: *** [Makefile:1180: help] Error 2
Fixes: ca6c896bdd
Signed-off-by: Vincent Stehlé <vincent.stehle@arm.com>
Cc: Steve Hay <me@stevenhay.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
This .gitignore file ignores all files in an output directory and
hence doesn't change the git status.
Remove "/output" entry from the main .gitignore file as it is already
handled by this general approach.
Signed-off-by: Yegor Yefremov <yegorslists@googlemail.com>
Signed-off-by: Brandon Maier <brandon.maier@collins.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Teach the generic package handling code how to extract zstd (.tar.zst)
archives. When zstd is not installed on the host, host-zstd gets built
automatically.
Signed-off-by: Matt Staveley-Taylor <matt.stav.taylor@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
The current implementation of buildroot depe dependency graphing
either does forward- or reverse-dependency traversal.
This patch enables buildroot to graph forward and reverse dependencies
on the graph for the same package: (Diagram Credit: Yann E. MORIN)
$ make pkg-d-graph-both-depends
pkg A -. .-> pkg E
\ /
pkg B ----> pkg D ----> pkg F
/ \
pkg C -' '-> pkg G
In the above example a single graph shows pkg {A,B,C} are needed
by pkg D, and pkg D is a dependency of pkg {E,F,G}.
Makefile help and manual are also updated.
Signed-off-by: Steve Hay <me@stevenhay.com>
[Arnout:
- remove DEPTH and RDEPTH, their functionality is already covered by
BR2_GRAPH_DEPS_OPTS;
- remove --rdepth, it was felt to not add sufficient added value;
- add the new target to the manual;
- fix flake8 errors.
]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Root makefile imposes 'umask 0022', and this may be more restrictive than the
user's original umask - which could have provisions set to share files/dirs
with other users.
As an example, the imposed value makes the per-package download directories not
writeable for the group, but just for the owner - the user that issued the first
build that populated the per-package dl dir for the first time (say user A).
Thus, if a BR package changes its version (e.g. for buildroot update), and
another user (say user B, in the same group of A) starts a build, BR fails the
creation of package-xxx.tar.gz inside the dl dir, because user B has no write
permissions on that path. Furthermore, in the case of the git backend, this
makes the git cache not updatable by a different user. This is disruptive for a
host used by many users, all belonging to a certain group.
So, to allow sharing of a rw BR2_DL_DIR location among users, we save the
original umask value and restore it during the download process.
Signed-off-by: Luca Pesce <luca.pesce@vimar.com>
[Arnout:
- CURR_UMASK -> CUR_UMASK.
- BR2_ORIG_UMASK -> BR_ORIG_UMASK.
- Don't check if the umask is more permissive, apply it regardless. If
the user explicitly doesn't want to make their DL_DIR readable by
others, that's fine.
- Don't export BR_ORIG_UMASK.
- Only set BR_ORIG_UMASK if it we recurse, and only set umask if
BR_ORIG_UMASK is set.
- Add DOWNLOAD_SET_UMASK to simplify the latter.
]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Currently, one may only specify one list of arguments that are passed to
several scripts (BR2_ROOTFS_PRE_BUILD_SCRIPT, BR2_ROOTFS_POST_BUILD_SCRIPT,
BR2_ROOTFS_POST_FAKEROOT_SCRIPT and BR2_ROOTFS_POST_IMAGE_SCRIPT_ARGS).
So one has to be careful that the arguments for these scripts do noti
collide.
To allow specifiying dedicated arguments to each type of scripts, new
config options are introduced. For backward compatibility the value of
BR2_ROOTFS_POST_SCRIPT_ARGS is still passed to the scripts. But now one
can add specific arguments from the new config option.
Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
[yann.morin.1998@free.fr:
- mention common args in help texts
- slight coding style beautification
- slight rewording in commit log
]
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Currently, we expect and only use hash files that lie within the package
directory, alongside the .mk file. Those hash files are thus bundled
with Buildroot.
This implies that only what's known to Buildroot can ever get into those
hash files. For packages where the version is fixed (or a static
choice), then we can carry hashes for those known versions.
However, we do have a few packages for which the version is a free-form
entry, where the user can provide a custom location and/or version. like
a custom VCS tree and revision, or a custom tarball URL. This means that
Buildroot has no way to be able to cary hashes for such custom versions.
This means that there is no integrity check that what was downloaded is
what was expected. For a sha1 in a git tree, this is a minor issue,
because the sha1 by itself is already a hash of the expected content.
But for custom tarballs URLs, or for a tag in a VCS, there is indeed no
integrity check.
Buildroot can't provide such hashes, but interested users may want to
provide those, and currently there is no (easy) way to do so.
So, we need our download helpers to be able to accept more than one hash
file to lookup for hashes.
Extend the dl-wrapper and the check-hash helpers thusly, and update the
legal-info accordingly.
Note that, to be able to pass more than one hash file, we also need to
re-order the arguments passed to support/download/check-hash, which also
impies some shuffling in the three places it is called:
- 2 in dl-wrapper
- 1 in the legal-info infra
That in turn also requires that the legal-license-file macro args get
re-ordered to have the hash file last; we take the opportunity to also
move the HOST/TARGET arg to be first, like in the other legal-info
macros.
Reported-by: "Martin Zeiser (mzeiser)" <mzeiser@cisco.com>
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Peter Korsgaard <peter@korsgaard.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
In 6fa3a239 the gen-missing-cpe support script was removed together with
"make missing-cpe".
Remove the leftover path variable and drop it from "make clean".
Signed-off-by: Daniel Lang <dalang@gmx.at>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
NodeJS 20 requires gcc >= 10.1. Unfortunately, except for
BR2_HOST_GCC_AT_LEAST_4_9, Buildroot only handles host gcc version
with the granularity of the major release, so we will have to round up
to GCC >= 11 for NodeJS 20.
Signed-off-by: Adam Duskett <adam.duskett@amarulasolutions.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
commit 21d52e52d8 (package/pkg-utils.mk: break hardlinks in global
{TARGET, HOST}_DIR on per-package build) was recently reverted, so we
are back to a situation where it is possible for packages and post-build
scripts to modify files in-place, and thus impact files in any arbitrary
per-package directory, which may break things on rebuild for example.
21d52e52d8 was too big a hammer, but we can still apply the reasoning
from it, to the aggregation of the final target and host directories.
This solves the case for post-build scripts at least. We leave the case
of inter-package modification aside, as it is a bigger issue that will
need more than just copying files around.
We use --hard-links, so that hard-links in the source (the PPD), are
kept as new hard-links (i.e. "copy" of hard-links) in the destination.
This contributes to limiting the size of target/.
Signed-off-by: Yann E. MORIN <yann.morin.1998@free.fr>
Cc: Herve Codina <herve.codina@bootlin.com>
Cc: Peter Korsgaard <peter@korsgaard.com>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: Herve Codina <herve.codina@bootlin.com>
The relocate-sdk.sh script does not work correctly when
BR2_PER_PACKAGE_DIRECTORIES is enabled. relocate-sdk.sh expects
everything to point at $HOST_DIR, but each package will be pointing at
its $(O)/per-package/*/host.
Use the same command for scrubing host paths during the build, to scrub
to the final host directory location.
Signed-off-by: Brandon Maier <Brandon.Maier@collins.com>
Acked-by: Charles Hardin <ckhardin@gmail.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
The reinstall, rebuild and reconfigure commands rely on the
left-to-right order of evaluation of the dependencies to make sure that
the stamp files are removed before attempting to rebuild. However, this
order of evaluation is not guaranteed. In particular, if top-level
parallel build is enabled, they are executed in parallel and the stamp
file may not have been removed yet when it is evaluated to decide if
rebuild has to be done.
Since make 4.4, it is possible to reproduce this issue by passing
`--shuffle=reverse` to the make commandline.
To solve this, add a .WAIT directive between the clean and
install/build/configure dependencies. .WAIT was introduced in make 4.4
as well. It makes sure that the dependencies on the left are evaluated
before the dependencies on the right - exactly what we want here.
Earlier versions of make don't know about .WAIT, so we need to add a
.PHONY dependency to effectively ignore it.
Note that this doesn't fix the problem for make versions earlier than
4.4. However, the issue isn't really that important: reinstall, rebuild
and reconfigure are development tools, they're not fully reliable to
begin with, and it's anyway less likely that someone uses `make -j` when
doing a reinstall/rebuild/reconfigure.
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Reported-by: James Hilliard <james.hilliard1@gmail.com>
The intention of this script is to generate the XML that can be sent to
NVD to request a new CPE identifier.
As discussed on the mailing list [0] keeping up with version numbers of
all registered CPE ID won't work.
In addition the feed used to generated the XML files will be retired
[1]. In the future an API needs to be used for fetching the data in
connection with a local database.
All of this works against keeping this script and porting it to the new
API.
As a last blow Matthew, the original author concluded [2]:
> Makes sense to drop it. There never got to be enough momentum in the overall
> software community to make CVE or even the new identifier really accurate.
The intention is to ignore the version part of CPE IDs in the future,
and only look at the version range specified on a CVE. Therefore, a tool
to add new CPE ID versions isn't useful to us. It might still be useful
to have a tool to create the vendor and project parts of a CPE ID.
However, the current gen-missing-cpe tool doesn't support that, and the
API is anyway going to be retired. So there is no reason at all to keep
this around.
Remove gen-missing-cpe and the cpedb module. Remove the Makefile target
to call the script.
Since the cpedb module is removed, the CPEDB_URL definition must be
moved to the place where it is still used, in pkg-stats.
[0]: https://lists.buildroot.org/pipermail/buildroot/2023-August/672620.html
[1]: https://nvd.nist.gov/General/News/change-timeline
[2]: https://lists.buildroot.org/pipermail/buildroot/2023-August/672651.html
Signed-off-by: Daniel Lang <dalang@gmx.at>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Using "xargs" instead of "while read" loop allows for the patching of
files to be parallelized. This significantly reduces the amount of
time it takes to fix all the paths. On a larger RFS(~300MB) this
script was taking 5 minutes, it now only takes about 30s on a 12 core
machine.
Signed-off-by: Victor Dumas <dumasv.dev@gmail.com>
[Thomas: take into account the suggestion of Quentin Schulz to pass
PARALLEL_JOBS through the environment down to the fix-rpath script]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Teach check-package to detect python files by type and check them using
flake8.
Do not use subprocess to call 'python3 -m flake8' in order to avoid too
many spawned shells, which in its turn would slow down the check for
multiple files. (make check-package takes twice the time using a shell
for each flake8 call, when compared of importing the main application)
Expand the runtime test and the unit tests for check-package.
Remove check-flake8 from the makefile and also from the GitLab CI
because the exact same checks become part of check-package.
Suggested-by: Arnout Vandecappelle <arnout@mind.be>
Signed-off-by: Ricardo Martincoski <ricardo.martincoski@gmail.com>
[Arnout: add a comment to x-python to explain its purpose]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
