As more and more packages are required to build the host ccache package
and therefore they has to be filtered out for ccache dependency the list
growed up.
By adding the variable <PKG>_ADD_CCACHE_DEPENDENCY with the default
value to 'yes' we can remove that dependency for each package that is
needed by ccache by setting the value to 'no'. This can be done in the
package makefile instead of tracking the list in the core pkg-generic.mk file.
This behavior is similar to the skeleton and toolchain dependency.
Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
[Arnout: move the BR2_CCACHE condition to the dependency itself]
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
https://ccache.dev/releasenotes.html#_ccache_4_10_1
This version requires use the system dependencies of host-blake3 and
host-xxhash. The bundled version is not able to build/download.
The ZSTD_FROM_INTERNET and HIREDIS_FROM_INTERNET variables are replaced
in the ccache build system with DEP_ZSTD and DEP_HIREDIS in version 4.10.
For the sake of completeness, all DEP_<xxx> variables were set for external
dependencies.
The generation of documentation has also been switched off.
The LICENSE.adoc was updated regarding the bundled libraries version used
like cpp-httplib, BLAKE3, fmt, nonstd-span, cxxurl.
Signed-off-by: Heiko Thiery <heiko.thiery@gmail.com>
Signed-off-by: Arnout Vandecappelle <arnout@mind.be>
Ccache CMake build system has some conditions that automatically
enables a developer mode (sets CCACHE_DEV_MODE=ON). See [1].
More specifically, if CCACHE_DEV_MODE is unset AND the environment
variable "CI" is set, CCACHE_DEV_MODE is set to "ON".
This situation can happen when Buildroot builds are executed in
Jenkins jobs, for example. Since Buildroot does not set
CCACHE_DEV_MODE and Jenkins sets the "CI" environment variable,
this ccache developer mode can be enabled in an unexpected way
for the Buildroot user. For example, it happened that a Jenkins build
breaks, while the build with the same configuration in the user
session is working.
One of the effects of enabling this ccache developer mode, is to treat
compiler warnings as errors, see [3]. This can lead to build error,
depending on the ccache version and the host compiler being used.
This behavior can be reproduced and observed, with commands:
cat > .config <<EOF
BR2_aarch64=y
BR2_TOOLCHAIN_EXTERNAL=y
BR2_CCACHE=y
BR2_PACKAGE_BUSYBOX=y
EOF
make olddefconfig
make clean
make host-ccache
Outputs:
...
-- Ccache version: 4.9.1
-- Ccache dev mode: OFF
-- Setting CMAKE_BUILD_TYPE to Release as none was specified.
...
Whereas:
make clean
CI=true make host-ccache
Outputs:
...
-- Ccache version: 4.9.1
-- Ccache dev mode: ON
-- Setting CMAKE_BUILD_TYPE to Debug as none was specified.
...
For a failure example: on Fedora 40 with host gcc 14.1.1, Buildroot at
tag 2024.02 has ccache 4.8.2. Host ccache can fail, when building with
"CI=true make host-ccache" with output:
/buildroot/output/build/host-ccache-4.8.2/src/third_party/fmt/core.h:3119:44: in 'constexpr' expansion of 'fmt::v8::make_format_args<>(args#0, args#1)'
/buildroot/output/build/host-ccache-4.8.2/src/third_party/fmt/core.h:1706:15: error: possibly dangling reference to a temporary [-Werror=dangling-reference]
1706 | const auto& arg = arg_mapper<Context>().map(std::forward<T>(val));
| ^~~
This commit sets CCACHE_DEV_MODE=OFF to make the ccache behavior more
deterministic in Buildroot, independently of being used in a CI tool
or not.
[1] https://github.com/ccache/ccache/blob/v4.9.1/CMakeLists.txt#L56
[2] https://github.com/jenkinsci/jenkins/blob/jenkins-2.459/core/src/main/java/jenkins/model/CoreEnvironmentContributor.java#L43
[3] https://github.com/ccache/ccache/blob/v4.9.1/cmake/StandardWarnings.cmake#L5
Reported-by: Xavier Roumegue <xroumegue@gmail.com>
Signed-off-by: Julien Olivain <ju.o@free.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Since the bump of ccache from 4.7.4 to 4.8.2 in Buildroot commit
1db3f0f844 ("package/ccache: bump
version to 4.8.2"), it started failing to build on machines that use
gcc 12.3, due to a bug in gcc:
/home/buildroot/autobuild/instance-0/output-1/build/host-ccache-4.8.2/src/storage/local/LocalStorage.cpp:701:24: required from here
/home/buildroot/autobuild/instance-0/output-1/build/host-ccache-4.8.2/src/Logging.hpp:33:7: internal compiler error: Segmentation fault
33 | Logging::log(std::string_view(message_)); \
| ^~~~~~~
Upstream ccache has committed a workaround to avoid this compiler bug,
which we backport in this commit.
Fixes:
http://autobuild.buildroot.net/results/cd341365d1c429bfca505a742e86b3090897db23/
Signed-off-by: Bernd Kuhls <bernd@kuhls.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Migrate to cmake package infrastructure.
Add new host-hiredis host-zstd dependencies.
Add new ccache dependency exclusions to pkg-generic.
Migrate HOST_CCACHE_PATCH_CONFIGURATION to handle updated
source files/format.
License hashes changed due to migrating urls to https:
a0f32f161f
Signed-off-by: James Hilliard <james.hilliard1@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
- Update CCACHE_SITE to github.
- The hash of the license file is updated. There were two changes:
* The reference to the credits.html file changed from
ccache.samba.org to ccache.dev
* The MIT license text for minitrace.[ch] was added, but it doesn't
change the fact that the whole is under GPL-3.0, and we anyway
already had "GPL-3.0, others" in CCACHE_LICENSE
Signed-off-by: Asaf Kahlon <asafka7@gmail.com>
[Thomas: update the license file hash]
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Update LICENSE.adoc: change was zlib and year bump.
Signed-off-by: Asaf Kahlon <asafka7@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Update license info, and remove the patch since it's already on upstream.
Signed-off-by: Asaf Kahlon <asafka7@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
This moves the host-ccache dependency handling from
DEPENDENCIES_HOST_PREREQ to a proper package dependency. When
BR2_CCACHE=y, we add host-ccache as a regular dependency of all
packages except:
- The extractor packages host-tar, host-xz and host-lzip
- host-ccache itself
- host-skeleton, because all packages depend on it
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Tested-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
While building I noticed:
>>> host-ccache 3.3.4 Building
conf.c: In function 'conf_create':
conf.c:314:2: warning: too many arguments for format [-Wformat-extra-args]
conf->cache_dir = format("/home/peko/.buildroot-ccache", get_home_directory());
^
As host-ccache gets installed into $(HOST_DIR) and is part of the SDK,
hardcoding the build user homedir isn't really nice for the relocatable
SDK feature (or simply for a SDK used by multiple users).
As the warning shows, CCache replaces "%s" with the current user home
directory, so rewrite BR_CACHE_DIR to use this feature if it begins with
$HOME.
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
We want to use SPDX identifier for license string as much as possible.
SPDX short identifier for GPLv3/GPLv3+ is GPL-3.0/GPL-3.0+.
This change is done using following command.
find . -name "*.mk" | xargs sed -ri '/LICENSE( )?[\+:]?=/s/\<GPLv3\>/GPL-3.0/g'
Signed-off-by: Rahul Bedarkar <rahulbedarkar89@gmail.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Switch download URL to avoid a redirect.
Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
ccache doesn't rewrite the --sysroot base_dir from toolchain-wrapper
correctly. Resulting in a large number of cache misses when building
from different base directories.
A fix has already been accepted to the ccache upstream[1], but won't be
released until ccache 3.4. (Was noted as an improvement, so unsure
of the 3.4 release timing)
[1] https://github.com/ccache/ccache/pull/132
Signed-off-by: Brandon Maier <brandon.maier@rockwellcollins.com>
Signed-off-by: Matt Weber <matthew.weber@rockwellcollins.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Note, that version 3.3.3 of ccache fixes a bug [1] where the Linux
kernel image is the same even after changing the source code and
rebuilding it.
[1] https://github.com/ccache/ccache/issues/136
Signed-off-by: Jörg Krause <joerg.krause@embedded.rocks>
Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Our current ccache disables hashing of the compiler executable itself,
because using the default 'mtime' doesn't work in buildroot: we always
rebuild the compiler, so the mtime is always different, so the cache
always misses.
However, in the current situation, if a user changes the compiler
configuration (which would result in the compiler generating different
object files than before) and does 'make clean all', ccache may in fact
reuse object files from the previous run. This rarely gives problems,
because
(1) the cache expires quite quickly (it's only 1GB by default),
(2) radically changing compiler options will cause cache misses because
different header files are used,
(3) many compiler changes (e.g. changing -mtune) have little practical
effect because the resulting code is usually still compatible,
(4) we currently don't use CCACHE_BASEDIR, and almost all object files
will contain an absolute path (e.g. in debug info), so when
building in a different directory, most of it will miss,
(5) we do mostly build test, and many of the potential problems only
appear at runtime.
Still, when ccache _does_ use the wrong cached object files, the
effects are really weird and hard to debug. Also, we want reproducible
builds and obviously the above makes builds non-reproducible. So we
have a FAQ entry that warns against using ccache and tells the user to
clear the cache in case of problems.
Now that ccache is called from the toolchain wrapper, it is in fact
possible to at least use the 'mtime' compiler hash for the external
toolchain and for the host-gcc. Indeed, in this case, the compiler
executable comes from a tarball so the mtime will be a good reference
for its state. Therefore, the patch (sed script) that changes the
default from 'mtime' to 'none' is removed.
For the internal toolchain, we can do better by providing a hash of
the relevant toolchain options. We are only interested in things that
affect the compiler itself, because ccache also processes the header
files and it doesn't look at libraries because it doesn't cache the
link step, just compilation. Everything that affects the compiler
itself can nicely be summarised in $(HOST_GCC_FINAL_CONF_OPTS). Of
course, also the compiler source itself is relevant, so the source
tarball and all the patches are included in the hash. For this purpose,
a new HOST_GCC_XTENSA_OVERLAY_TAR is introduced.
The following procedure tests the ccache behaviour:
Use this defconfig:
BR2_arm=y
BR2_CCACHE=y
make
readelf -A output/build/uclibc-1.0.6/libc/signal/signal.os
-> Tag_CPU_name: "ARM926EJ-S"
Now make menuconfig, change variant into BR2_cortex_a9
make clean; make
readelf -A output/build/uclibc-1.0.6/libc/signal/signal.os
-> Tag_CPU_name: "ARM926EJ-S"
should be "Cortex-A9"
After this commit, it is "Cortex-A9".
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Cc: Danomi Manchego <danomimanchego123@gmail.com>
Cc: Károly Kasza <kaszak@gmail.com>
Cc: Samuel Martin <s.martin49@gmail.com>
Cc: Romain Naour <romain.naour@openwide.fr>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Bump ccache to minor version 3.2.2.
Also changed download URL to https.
Signed-off-by: Karoly Kasza <kaszak@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
Bump ccache from v.3.1.x to v.3.2.x.
All functionality remained the same.
- Configure now accepts a parameter to use ccache's internal zlib.
- conf.c was introduced and some settings were moved from ccache.c,
with minor changes.
- The PATCH_CONFIGURATION hook was moved from POST_CONFIGURE to
POST_PATCH, as patching files should be done there, before calling
the configure script.
Signed-off-by: Karoly Kasza <kaszak@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Commit "packages: indentation cleanup" broke the build of master.
This commit reverse the patch on just ccache package and fix that issue.
Signed-off-by: Jerzy Grzegorek <jerzy.grzegorek@trzebnica.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This commit doesn't touch infra packages.
Signed-off-by: Jerzy Grzegorek <jerzy.grzegorek@trzebnica.net>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
The Buildroot coding style defines one space around make assignments and
does not align the assignment symbols.
This patch does a bulk fix of offending packages. The package
infrastructures (or more in general assignments to calculated variable
names, like $(2)_FOO) are not touched.
Alignment of line continuation characters (\) is kept as-is.
The sed command used to do this replacement is:
find * -name "*.mk" | xargs sed -i \
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*$#\1 \2#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\]\+\)$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\s*\([^\\ \t]\+\s*\\\)\s*$#\1 \2 \3#'
-e 's#^\([A-Z0-9a-z_]\+\)\s*\([?:+]\?=\)\(\s*\\\)#\1 \2\3#'
Brief explanation of this command:
^\([A-Z0-9a-z_]\+\) a regular variable at the beginning of the line
\([?:+]\?=\) any assignment character =, :=, ?=, +=
\([^\\]\+\) any string not containing a line continuation
\([^\\ \t]\+\s*\\\) string, optional whitespace, followed by a
line continuation character
\(\s*\\\) optional whitespace, followed by a line
continuation character
Hence, the first subexpression handles empty assignments, the second
handles regular assignments, the third handles regular assignments with
line continuation, and the fourth empty assignments with line
continuation.
This expression was tested on following test text: (initial tab not
included)
FOO = spaces before
FOO = spaces before and after
FOO = tab before
FOO = tab and spaces before
FOO = tab after
FOO = tab and spaces after
FOO = spaces and tab after
FOO = \
FOO = bar \
FOO = bar space \
FOO = \
GENIMAGE_DEPENDENCIES = host-pkgconf libconfuse
FOO += spaces before
FOO ?= spaces before and after
FOO :=
FOO =
FOO =
FOO =
FOO =
$(MAKE1) CROSS_COMPILE=$(TARGET_CROSS) -C
AT91BOOTSTRAP3_DEFCONFIG = \
AXEL_DISABLE_I18N=--i18n=0
After this bulk change, following manual fixups were done:
- fix line continuation alignment in cegui06 and spice (the sed
expression leaves the number of whitespace between the value and line
continuation character intact, but the whitespace before that could have
changed, causing misalignment.
- qt5base was reverted, as this package uses extensive alignment which
actually makes the code more readable.
Finally, the end result was manually reviewed.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Cc: Yann E. Morin <yann.morin.1998@free.fr>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
To be consistent with the recent change of FOO_MAKE_OPT into FOO_MAKE_OPTS,
make the same change for FOO_CONF_OPT.
Sed command used:
find * -type f | xargs sed -i 's#_CONF_OPT\>#&S#g'
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Reviewed-by: "Yann E. MORIN" <yann.morin.1998@free.fr>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
For example, if your project is known to require more space than the
default max cache size, then you might want to increase the cache size
to a suitable amount using the -M (--max-size) option.
The string you specify here is passed verbatim to ccache. Refer to
ccache documentation for more details.
These initial settings are applied after ccache has been compiled.
Signed-off-by: Danomi Manchego <danomimanchego123@gmail.com>
Reviewed-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Tested-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Commit 433290761f changed the hard-coded
ccache directory location to use BR_CACHE_DIR (then BUILDROOT_CACHE_DIR),
which is exported by Makefile based on the BR2_CCACHE_DIR config option.
This allowed the cache location to be changed on-the-fly by setting a
"make" command line variable, but left the default location of ccache's
normal default at "$HOME/.ccache". Since this location does not match the
default for BR2_CCACHE_DIR, it is basically almost never correct, so
direct invocation of ccache outside of the buildroot Makefile, such as for
increasing the cache size, becomes cumbersome.
This patch changes the last-ditch cache location from "$HOME/.ccache" to
the BR_CCACHE_DIR value defined when host-ccache is configured. Note that
the ability to later override the cache location by using a BR_CACHE_DIR
command line variable is left intact.
Signed-off-by: Danomi Manchego <danomimanchego123@gmail.com>
Reviewed-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Tested-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
This ensures that the cache directory (and all of its ancestor
directories) exist. This is a nice thing to do because, if the parent
of the cache directory doesn't exist, then ccache will complain that
it cannot create the cache directory, causing the build to fail.
[Peter: drop BR2_CCACHE conditional, use POST_INSTALL hook]
Signed-off-by: Dan Moulding <dan.moulding@rackwareinc.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
The ccache target package has been deprecated since 2013.02 and thus can be
removed in 2014.02. This does not change anything about host ccache (used
for speeding up builds).
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
To make the naming consistent (qstripped variant of a config option
should be named BR_XXX).
Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
In order to keep better track of when a feature got deprecated, and hence
when it can be removed, a new set of symbols BR2_DEPRECATED_SINCE_xxxx_xx is
introduced. These symbols are automatically selected when BR2_DEPRECATED is
selected, and thus are transparent to the user.
A deprecated feature will no longer depend on BR2_DEPRECATED directly, but
rather on the appropriate BR2_DEPRECATED_SINCE_xxxx_xx. If that symbol does
not yet exist, it has to be created in Config.in.
When removing a deprecated feature, one should also check whether this was
the last feature using the BR2_DEPRECATED_SINCE_xxxx_xx symbol, in which
case the latter can be removed from Config.in.
A followup patch will make sure the overview is added to the list of
deprecated features in the manual, so that a buildroot core developer can
easily determine which features to remove in a given development cycle.
Signed-off-by: Thomas De Schampheleire <thomas.de.schampheleire@gmail.com>
Acked-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
Signed-off-by: Peter Korsgaard <peter@korsgaard.com>
