mirrors/systemd
0
0
Fork 0
mirror of https://github.com/systemd/systemd.git synced 2024-11-23 10:13:34 +08:00
Code Issues Packages Projects Releases Wiki Activity
838235be87
systemd/test/test-functions

3555 lines
119 KiB
Plaintext
Raw Normal View History

treewide: more portable bash shebangs As in 2a5fcfae024ffc370bb780572279f45a1da3f946 and in 3e67e5c9928f8b1e1c5a63def88d53ed1fed12eb using /usr/bin/env allows bash to be looked up in PATH rather than being hard-coded. As with the previous changes the same arguments apply - distributions have scripts to rewrite shebangs on installation and they know what locations to rely on. - For tests/compilation we should rather rely on the user to have setup there PATH correctly. In particular this makes testing from git easier on NixOS where do not provide /bin/bash to improve compose-ability.
2020-03-04 17:35:06 +08:00
#!/usr/bin/env bash
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
# -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*-
licensing: add forgotten spdx headers Those are all "our" files, but we forgot to add the headers, most likely because of non-standard file extensions.
2021-10-01 19:04:32 +08:00
# SPDX-License-Identifier: LGPL-2.1-or-later
#
# shellcheck disable=SC2030,SC2031
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
# ex: ts=8 sw=4 sts=4 et filetype=sh tw=180
# Note: the shellcheck line above disables warning for variables which were
# modified in a subshell. In our case this behavior is expected, but
# `shellcheck` can't distinguish this because of poor variable tracking,
# which results in warning for every instance of such variable used
# throughout this file.
# See:
# * comment in function install_verity_minimal()
# * koalaman/shellcheck#280
set -o pipefail
test: make the LOOKS_LIKE_* variables proper booleans
2023-11-20 19:17:05 +08:00
# Simple wrapper to unify boolean checks.
# Note: this function needs to stay near the top of the file, so we can use it
# in code in the outermost scope.
get_bool() {
# Make the value lowercase to make the regex matching simpler
local _bool="${1,,}"
# Consider empty value as "false"
if [[ -z "$_bool" || "$_bool" =~ ^(0|no|false)$ ]]; then
return 1
elif [[ "$_bool" =~ ^(1|yes|true)$ ]]; then
return 0
else
echo >&2 "Value '$_bool' is not a valid boolean value"
exit 1
fi
}
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
PATH=/sbin:/bin:/usr/sbin:/usr/bin
export PATH
tests/TEST-50: support the case when /etc/os-release is present We have four legal cases: 1. /usr/lib/os-release exists and /etc/os-release is a symlink to it 2. both exist but /etc/os-release is not a symlink to /usr/lib/os-release 3. only /usr/lib/os-release exists 4. only /etc/os-release exists The generic setup code in test-functions and create-busybox-image didn't handle case 3. The test-specific code in TEST-50 didn't handle 2 (because the general setup code would only install /etc/os-release in the image and grep -f /usr/lib/os-release would not work) and 4 (same reason) and would fail in case 3 in generic setup.
2020-08-27 17:59:45 +08:00
os_release=$(test -e /etc/os-release && echo /etc/os-release || echo /usr/lib/os-release)
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
# shellcheck source=/dev/null
source "$os_release"
test: make the LOOKS_LIKE_* variables proper booleans
2023-11-20 19:17:05 +08:00
[[ "$ID" == "debian" || " $ID_LIKE " == *" debian "* ]] && LOOKS_LIKE_DEBIAN=yes || LOOKS_LIKE_DEBIAN=no
[[ "$ID" == "arch" || " $ID_LIKE " == *" arch "* ]] && LOOKS_LIKE_ARCH=yes || LOOKS_LIKE_ARCH=no
test: support NO_BUILD=yes on Fedora
2023-11-20 19:33:58 +08:00
[[ "$ID" == "fedora" ]] && LOOKS_LIKE_FEDORA=yes || LOOKS_LIKE_FEDORA=no
test: make the LOOKS_LIKE_* variables proper booleans
2023-11-20 19:17:05 +08:00
[[ " $ID_LIKE " == *" suse "* ]] && LOOKS_LIKE_SUSE=yes || LOOKS_LIKE_SUSE=no
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
KERNEL_VER="${KERNEL_VER-$(uname -r)}"
test: set the default QEMU and nspawn timeouts to 30 minutes Set both timeouts to some reasonable values instead of just `infinity` to provide some form of a safe-net in case the test goes haywire and the environment didn't set the timeouts itself (like our CIs do).
2022-08-26 22:19:42 +08:00
QEMU_TIMEOUT="${QEMU_TIMEOUT:-1800}"
NSPAWN_TIMEOUT="${NSPAWN_TIMEOUT:-1800}"
tests: track and check for timeouts If run_qemu() exits with non-zero, this either meant that QEMU was not available (which should be a SKIP) or that QEMU timed out if $QEMU_TIMEOUT was set (which then should be a FAIL). Limit the exit code of run_qemu() to QEMU availability only, and track timeouts separately through the new $TIMED_OUT variable, which is then checked in check_result_qemu(). Do the same for $NSPAWN_TIMEOUT and run_nspawn() so that nspawn and QEMU work similarly.
2016-06-24 18:11:19 +08:00
TIMED_OUT= # will be 1 after run_* if *_TIMEOUT is set and test timed out
Revert "test: use btrfs by default on Arch as well" There's something very wrong going on when using btrfs for the test images, namely: - there's a significant performance hit, i.e. the Arch Linux run is ~20% slower, in the coverage run the situation is even worse - intermittent boot failures - intermittent "No space left on device" errors (even though there's enough free space) Since debugging this might take a while, let's temporarily revert back to ext4 to make the CI stable again. This reverts commit 7eb7e3ec4f5dbc13ee729557e1544527f3101187.
2024-02-19 18:23:31 +08:00
get_bool "$LOOKS_LIKE_SUSE" && FSTYPE="${FSTYPE:-btrfs}" || FSTYPE="${FSTYPE:-ext4}"
tests: add UNIFIED_CGROUP_HIERARCHY=[default|hybrid] (#4675) This will simplify testing a bit. Mainly for https://github.com/systemd/systemd/pull/4670
2016-11-16 00:38:04 +08:00
UNIFIED_CGROUP_HIERARCHY="${UNIFIED_CGROUP_HIERARCHY:-default}"
test: allow overriding EFI_MOUNT like other variables
2019-12-10 01:52:35 +08:00
EFI_MOUNT="${EFI_MOUNT:-$(bootctl -x 2>/dev/null || echo /boot)}"
tests: build the image once and then copy/extend it Building custom images for each test takes a lot of time. Build the default one, and if the test needs incompatible changes just copy it and extend it instead.
2021-01-07 05:42:28 +08:00
# Note that defining a different IMAGE_NAME in a test setup script will only result
# in default.img being copied and renamed. It can then be extended by defining
# a test_append_files() function. The $1 parameter will be the root directory.
# To force creating a new image from scratch (eg: to encrypt it), also define
# TEST_FORCE_NEWIMAGE=1 in the test setup script.
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
IMAGE_NAME=${IMAGE_NAME:-default}
test/TEST-01: allow running w/o -Dinstall-tests=true This is useful for sanitizer builds under CI, see https://github.com/systemd/systemd-centos-ci/pull/202#issuecomment-566476592.
2019-12-17 19:52:35 +08:00
TEST_REQUIRE_INSTALL_TESTS="${TEST_REQUIRE_INSTALL_TESTS:-1}"
test: introduce TEST_PARALLELIZE to support running tests in parallel Support running tests in parallel by switching to copying of the base image instead of symlinking it.. This still requires some setup steps, like running `make setup` on tests which have unique $IMAGE_NAME beforehand (and sequentially), otherwise they'll all try to create the same base image when started in parallel, leading to nasty issues. However, as running the integration tests in parallel is such an unusual use case it should be good enough, for now.
2020-05-17 16:48:16 +08:00
TEST_PARALLELIZE="${TEST_PARALLELIZE:-0}"
test: make masking of supplementary services configurable
2022-02-17 03:29:14 +08:00
TEST_SUPPORTING_SERVICES_SHOULD_BE_MASKED="${TEST_SUPPORTING_SERVICES_SHOULD_BE_MASKED:-1}"
test: stop caching loopdev It is more trouble than it is worth. The setup is of a loopback device is very quick, so it's better to always create it when needed and immediately drop afterwards.
2020-03-21 03:17:11 +08:00
LOOPDEV=
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test: allow multiple handlers for the EXIT signal Bash allows only one handler per signal, so let's overcome this limitation by having one dedicated EXIT signal which runs all registered handlers from all over the place.
2022-07-08 02:00:46 +08:00
# Since in Bash we can have only one handler per signal, let's overcome this
# limitation by having one global handler for the EXIT signal which executes
# all registered handlers
_AT_EXIT_HANDLERS=()
_at_exit() {
set +e
# Run the EXIT handlers in reverse order
for ((i = ${#_AT_EXIT_HANDLERS[@]} - 1; i >= 0; i--)); do
ddebug "Running EXIT handler '${_AT_EXIT_HANDLERS[$i]}'"
test: move TPM2-related setup stuff into test-functions And hide it all behind $TEST_SETUP_SWTPM.
2023-06-03 02:28:41 +08:00
eval "${_AT_EXIT_HANDLERS[$i]}"
test: allow multiple handlers for the EXIT signal Bash allows only one handler per signal, so let's overcome this limitation by having one dedicated EXIT signal which runs all registered handlers from all over the place.
2022-07-08 02:00:46 +08:00
done
}
trap _at_exit EXIT
add_at_exit_handler() {
test: move TPM2-related setup stuff into test-functions And hide it all behind $TEST_SETUP_SWTPM.
2023-06-03 02:28:41 +08:00
_AT_EXIT_HANDLERS+=("${1:?}")
test: allow multiple handlers for the EXIT signal Bash allows only one handler per signal, so let's overcome this limitation by having one dedicated EXIT signal which runs all registered handlers from all over the place.
2022-07-08 02:00:46 +08:00
}
docs: use lowercase "qemu" and other minor tweaks to test docs We used both "qemu" and "QEMU", let's use the lower-case version everywhere since it's also the name of the binary and the version that people are most familiar with. The stuff under test/ is not only for the integeration tests, but also for various other test-related stuff, so adjust the docs a bit.
2022-05-04 18:29:31 +08:00
# Decide if we can (and want to) run qemu with KVM acceleration.
test: allow use of nested KVM Introduce TEST_NESTED_KVM variable, which allows use of nested KVM for QEMU VMs (which is disabled by default by KVM autodetection).
2019-09-27 05:29:38 +08:00
# Check if nested KVM is explicitly enabled (TEST_NESTED_KVM). If not,
# check if it's not explicitly disabled (TEST_NO_KVM) and we're not already
# running under KVM. If these conditions are met, enable KVM (and possibly
# nested KVM), otherwise disable it.
test: make the virt detection quiet Follow-up to cde09b07dfdc132a31672693c037bfc0b5879331.
2022-08-27 01:18:20 +08:00
if get_bool "${TEST_NESTED_KVM:=}" || (! get_bool "${TEST_NO_KVM:=}" && ! systemd-detect-virt -qv); then
test: allow use of nested KVM Introduce TEST_NESTED_KVM variable, which allows use of nested KVM for QEMU VMs (which is disabled by default by KVM autodetection).
2019-09-27 05:29:38 +08:00
QEMU_KVM=yes
else
QEMU_KVM=no
fi
tests: determine rootlibdir from pkg-config Prefer asking pkg-config for the rootlibdir, to also support systems with --enable-split-usr. Fall back to the hardcoded /usr/lib/systemd if that fails.
2015-02-02 07:19:29 +08:00
if ! ROOTLIBDIR=$(pkg-config --variable=systemdutildir systemd); then
Drop split-usr and unmerged-usr support As previously announced, execute order 66: https://lists.freedesktop.org/archives/systemd-devel/2022-September/048352.html The meson options split-usr, rootlibdir and rootprefix become no-ops that print a warning if they are set to anything other than the default values. We can remove them in a future release.
2023-06-12 09:15:19 +08:00
echo "WARNING! Cannot determine libdir from pkg-config, assuming /usr/lib/systemd" >&2
tests: determine rootlibdir from pkg-config Prefer asking pkg-config for the rootlibdir, to also support systems with --enable-split-usr. Fall back to the hardcoded /usr/lib/systemd if that fails.
2015-02-02 07:19:29 +08:00
ROOTLIBDIR=/usr/lib/systemd
fi
test/test-functions: add variables for several dir locations
2021-01-27 05:46:10 +08:00
# The calling test.sh scripts have TEST_BASE_DIR set via their Makefile, but we don't need them to provide it
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
TEST_BASE_DIR=${TEST_BASE_DIR:-$(realpath "$(dirname "${BASH_SOURCE[0]}")")}
test: "detect" the test number automagically Specifying the test number manually is tedious and prone to errors (as recently proven). Since we have all the necessary data to work out the test number, let's do it automagically.
2021-04-27 01:20:18 +08:00
TEST_UNITS_DIR="$(realpath "$TEST_BASE_DIR/units")"
test/test-functions: add variables for several dir locations
2021-01-27 05:46:10 +08:00
SOURCE_DIR=$(realpath "$TEST_BASE_DIR/..")
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
# These variables are used by test scripts
test: don't export $TOOLS_DIR It's not (and shouldn't be) used by any test scripts.
2023-03-22 17:33:58 +08:00
export TEST_BASE_DIR TEST_UNITS_DIR SOURCE_DIR
test/test-functions: add variables for several dir locations
2021-01-27 05:46:10 +08:00
test: don't export $TOOLS_DIR It's not (and shouldn't be) used by any test scripts.
2023-03-22 17:33:58 +08:00
TOOLS_DIR="$SOURCE_DIR/tools"
test: find $BUILD_DIR in test-functions, remove from other scripts The $BUILD_DIR is only used in test-functions, and doesn't need to be specified in any other scripts. Additionally, to be able to allow the integration test suite to be run against locally installed binaries, instead of built binaries, moving BUILD_DIR logic completely into test-functions allows later patches to be simpler.
2020-11-18 04:39:37 +08:00
# note that find-build-dir.sh will return $BUILD_DIR if provided, else it will try to find it
test: don't try to find BUILD_DIR when NO_BUILD is set NO_BUILD=1 indicates that we want to test systemd from the local system and not the one from the local build. Hence there should be no need to call find-build-dir.sh when NO_BUID=1 especially since it's likely that the script will fail to find a local build in this case. This avoids find-build-dir.sh to emit 'Specify build directory with $BUILD_DIR' message when NO_BUILD=1 and no local build can be found. This introduces a behavior change though: systemd from the local system will always be preferred when NO_BUILD=1 even if a local build can be found.
2021-08-03 14:44:47 +08:00
if get_bool "${NO_BUILD:=}"; then
BUILD_DIR="$SOURCE_DIR"
elif ! BUILD_DIR="$("$TOOLS_DIR"/find-build-dir.sh)"; then
echo "ERROR: no build found, please set BUILD_DIR or use NO_BUILD" >&2
exit 1
test: find $BUILD_DIR in test-functions, remove from other scripts The $BUILD_DIR is only used in test-functions, and doesn't need to be specified in any other scripts. Additionally, to be able to allow the integration test suite to be run against locally installed binaries, instead of built binaries, moving BUILD_DIR logic completely into test-functions allows later patches to be simpler.
2020-11-18 04:39:37 +08:00
fi
test: avoid using pipes in certain cases `command -v <bin> | grep ...` can under certain conditions cause the `command` to exit with SIGPIPE, which in combination with `set -o pipefail` means that the tests sometimes randomly die during setup. Let's avoid using pipes in such cases.
2021-04-17 00:37:49 +08:00
PATH_TO_INIT="$ROOTLIBDIR/systemd"
SYSTEMD_JOURNALD="${SYSTEMD_JOURNALD:-$(command -v "$BUILD_DIR/systemd-journald" || command -v "$ROOTLIBDIR/systemd-journald")}"
test: use cp for journal copying when systemd-journal-remote non-existent
2022-04-24 15:31:33 +08:00
SYSTEMD_JOURNAL_REMOTE="${SYSTEMD_JOURNAL_REMOTE:-$(command -v "$BUILD_DIR/systemd-journal-remote" || command -v "$ROOTLIBDIR/systemd-journal-remote" || echo "")}"
test: avoid using pipes in certain cases `command -v <bin> | grep ...` can under certain conditions cause the `command` to exit with SIGPIPE, which in combination with `set -o pipefail` means that the tests sometimes randomly die during setup. Let's avoid using pipes in such cases.
2021-04-17 00:37:49 +08:00
SYSTEMD="${SYSTEMD:-$(command -v "$BUILD_DIR/systemd" || command -v "$ROOTLIBDIR/systemd")}"
SYSTEMD_NSPAWN="${SYSTEMD_NSPAWN:-$(command -v "$BUILD_DIR/systemd-nspawn" || command -v systemd-nspawn)}"
JOURNALCTL="${JOURNALCTL:-$(command -v "$BUILD_DIR/journalctl" || command -v journalctl)}"
test: fuzz our dbus interfaces with dfuzzer
2021-12-14 03:50:28 +08:00
SYSTEMCTL="${SYSTEMCTL:-$(command -v "$BUILD_DIR/systemctl" || command -v systemctl)}"
test: use systemd-id128 from the build dir As distro sd-id128 might not have all the options we need. Follow-up for: 378712c Replaces: #30901
2024-01-12 17:27:43 +08:00
SYSTEMD_ID128="${SYSTEMD_ID128:-$(command -v "$BUILD_DIR/systemd-id128" || command -v systemd-id128)}"
tests: use the asan wrapper to boot a VM/container if systemd is built with ASAN
2018-07-05 23:14:07 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
TESTFILE="${BASH_SOURCE[1]}"
test/test-functions: add variables for several dir locations
2021-01-27 05:46:10 +08:00
if [ -z "$TESTFILE" ]; then
echo "ERROR: test-functions must be sourced from one of the TEST-*/test.sh scripts" >&2
exit 1
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
TESTNAME="$(basename "$(dirname "$(realpath "$TESTFILE")")")"
test: $STATEDIR should not point to /usr/lib/systemd/tests when NO_BUILD=1 When testing the binaries from the host, make sure to not store the state data below /usr but use a dedicated directory in /var/tmp/ instead. The working directories of the tests, initially located in /var/tmp, are also moved in a dedicated directory /var/tmp/systemd-tests.
2023-03-21 23:44:41 +08:00
WORKDIR="/var/tmp/systemd-tests"
if get_bool "${NO_BUILD:=}"; then
STATEDIR="$WORKDIR/$TESTNAME"
else
STATEDIR="$BUILD_DIR/test/$TESTNAME"
fi
test/test-functions: move var assignment This makes no code change, only moves a small block of vars higher in the file. This makes the next commit a bit easier to read.
2020-11-18 05:02:14 +08:00
STATEFILE="$STATEDIR/.testdir"
IMAGESTATEDIR="$STATEDIR/.."
TESTLOG="$STATEDIR/test.log"
test: "detect" the test number automagically Specifying the test number manually is tedious and prone to errors (as recently proven). Since we have all the necessary data to work out the test number, let's do it automagically.
2021-04-27 01:20:18 +08:00
if ! [[ "$TESTNAME" =~ ^TEST\-([0-9]+)\-.+$ ]]; then
echo "ERROR: Test name '$TESTNAME' is not in the expected format: TEST-[0-9]+-*" >&2
exit 1
fi
TESTID="${BASH_REMATCH[1]:?}"
if [[ ! -f "$TEST_UNITS_DIR/testsuite-$TESTID.service" ]]; then
echo "ERROR: Test '$TESTNAME' is missing its service file '$TEST_UNITS_DIR/testsuite-$TESTID.service" >&2
exit 1
fi
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
BASICTOOLS=(
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
awk
test: add a testcase for issue #19895
2021-06-15 14:34:41 +08:00
base64
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
basename
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
bash
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
capsh
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
cat
test: set correct group for systemd-journal-upload tests We can't use the systemd-journal-upload user here, since it's created dynamically by DynamicUser=yes. However, we can use the group specified in SupplementaryGroups=, so do exactly that.
2023-12-05 04:33:15 +08:00
chgrp
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
chmod
chown
TEST-65: check cat-config operation in chroot This verifies the fix in 2075b6dd394e09a0f203b9cc7e3253908397f933.
2022-12-05 20:52:28 +08:00
chroot
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
cmp
test: move a couple of binaries to the BASICTOOLS array As they're not optional.
2023-11-20 23:26:49 +08:00
cp
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
cryptsetup
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
cut
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
date
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
dd
test: move a couple of binaries to the BASICTOOLS array As they're not optional.
2023-11-20 23:26:49 +08:00
dhclient
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
diff
dirname
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
dmsetup
echo
env
false
test: move a couple of binaries to the BASICTOOLS array As they're not optional.
2023-11-20 23:26:49 +08:00
find
test: add a couple more tests for systemd-pcrextend
2023-10-25 23:42:07 +08:00
findmnt
test: use flock when calling mkfs.btrfs As stated in https://github.com/systemd/systemd/issues/21819#issuecomment-1064377645 `mkfs.btrfs` doesn't hold the lock for the whole duration of `mkfs.btrfs`, thus causing unexpected races & test fails. Let's wrap the `mkfs.btrfs` calls in an flock wrapper to mitigate this. Hopefully fixes: #21819
2022-03-12 01:15:03 +08:00
flock
test: add basic memory pressure extended test for oomd Simple test to generate a lot of pressure in one unit and assert that systemd-oomd kills the right one.
2020-08-14 16:20:43 +08:00
getconf
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
getent
getfacl
test: unify checking for user xattrs support in cgroupfs Also, run the coredump forwarding test only if user xattrs are supported.
2023-10-19 18:28:37 +08:00
getfattr
test: convert TEST-45 to a normal meson test On my laptop, it now takes 0.23 s. We don't need the overhead of creating an image and spawning a machine to mangle a file in /tmp.
2020-03-21 03:22:42 +08:00
grep
test: move a couple of binaries to the BASICTOOLS array As they're not optional.
2023-11-20 23:26:49 +08:00
grep
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
gunzip
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
gzip
head
test: move a couple of binaries to the BASICTOOLS array As they're not optional.
2023-11-20 23:26:49 +08:00
hostname
id
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
ionice
test: move most of TEST-13-NSPAWN-SMOKE setup to static files
2019-12-10 04:45:46 +08:00
ip
test: install jq into all images
2022-09-11 20:16:34 +08:00
jq
test: kill plymouthd after initrd transition if it's still running Until now using the INTERACTIVE_DEBUG=yes stuff together with sanitizers was almost impossible, since the console kept eating up our inputs or not responding at all. After a painful day of debugging I noticed that if we use a shell script in the initrd -> root transition, we might end up with a plymouthd still running, which kept screwing with the tty. E.g. with initrd -> wrapper -> systemd transition, where the `wrapper` is a simple script: ``` exec -- /usr/lib/systemd/systemd "$@" ``` we'd end up with a stray plymouthd process after the bootup: ``` 1 0 440 2 20 0 0 0 worker I ? 0:00 [kworker/5:2-ata_sff] 1 0 453 2 20 0 0 0 worker I ? 0:00 [kworker/9:2-rcu_gp] 5 0 456 1 20 0 7252 1960 do_epo S ? 0:00 @usr/sbin/plymouthd --mode=boot --pid-file=/run/plymouth/pid --attach-to-session ``` After killing it, the tty works finally as expected.
2022-09-06 03:02:25 +08:00
killall
test-execute: add a testcase for MountAPIVFS=no
2021-08-23 05:45:33 +08:00
ldd
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
ln
test: move a couple of binaries to the BASICTOOLS array As they're not optional.
2023-11-20 23:26:49 +08:00
ln
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
loadkeys
login
test: install losetup by default Preparation for the next commit.
2021-12-20 22:34:17 +08:00
losetup
test: move a couple of binaries to the BASICTOOLS array As they're not optional.
2023-11-20 23:26:49 +08:00
ls
test: add a couple of tests for machinectl
2023-05-12 23:39:41 +08:00
lsattr
test: add a couple more tests for systemd-pcrextend
2023-10-25 23:42:07 +08:00
lsblk
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
lz4cat
test: move a couple of binaries to the BASICTOOLS array As they're not optional.
2023-11-20 23:26:49 +08:00
mkdir
test: move TEST-22-TMPFILES setup to static files
2019-12-10 20:32:20 +08:00
mkfifo
test: add a couple of tests for getty-generator
2023-04-14 18:58:51 +08:00
mknod
test: move TEST-22-TMPFILES setup to static files
2019-12-10 20:32:20 +08:00
mktemp
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
modprobe
mount
test: make TEST-02-CRYPTSETUP a bit more robust Prompted by systemd/systemd#16111. * check if /var is a mountpoint - if not, something went wrong. In case of systemd/systemd#16111 the /failed file was created, because systemd-cryptsetup failed, but it ended up being empty, making the result check incorrectly pass * forward journal messages to console - if we fail to mount /var, journald won't flush logs to the persistent storage and we end up empty handed and with no clue what went wrong For example, without systemd/systemd#16111 and with this patch: ... [FAILED] Failed to start systemd-cryptsetup@varcrypt.service. See 'systemctl status systemd-cryptsetup@varcrypt.service' for details. [DEPEND] Dependency failed for cryptsetup.target. ... [ 3.882451] systemd-cryptsetup[581]: Key file /etc/varkey is world-readable. This is not a good idea! [ 3.883946] systemd-cryptsetup[581]: WARNING: Locking directory /run/cryptsetup is missing! [ 3.884846] systemd-cryptsetup[581]: Failed to load Bitlocker superblock on device /dev/disk/by-uuid/180ba5ef-873b-4018-9968-47c23431f71a: Invalid argument ... [ 4.099451] sh[606]: + mountpoint /var [ 4.100025] sh[603]: + systemctl poweroff --no-block [ 4.101636] systemd[1]: Finished systemd-user-sessions.service. [ 4.102598] sh[608]: /var is not a mountpoint [FAILED] Failed to start testsuite-02.service.
2020-06-09 22:51:55 +08:00
mountpoint
test: move TEST-12-ISSUE-3171 setup to static files
2019-12-10 04:26:59 +08:00
mv
nc
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
nproc
test: move a couple of binaries to the BASICTOOLS array As they're not optional.
2023-11-20 23:26:49 +08:00
ping
test: import logind test from debian/ubuntu test suite
2022-06-01 07:56:08 +08:00
pkill
test: move a couple of binaries to the BASICTOOLS array As they're not optional.
2023-11-20 23:26:49 +08:00
ps
test: move TEST-22-TMPFILES setup to static files
2019-12-10 20:32:20 +08:00
readlink
test: install realpath into the test images
2022-11-08 19:43:57 +08:00
realpath
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
rev
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
rm
test: move most of TEST-13-NSPAWN-SMOKE setup to static files
2019-12-10 04:45:46 +08:00
rmdir
test: import logind test from debian/ubuntu test suite
2022-06-01 07:56:08 +08:00
rmmod
test: always install the script utility
2023-05-12 23:56:22 +08:00
script
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
sed
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
seq
test: cover --bind-user=
2023-05-17 21:35:10 +08:00
setfacl
oom: implement avoid/omit xattr support There may be situations where a cgroup should be protected from killing or deprioritized as a candidate. In FB oomd xattrs are used to bias oomd away from supervisor cgroups and towards worker cgroups in container tasks. On desktops this can be used to protect important units with unpredictable resource consumption. The patch allows systemd-oomd to understand 2 xattrs: "user.oomd_avoid" and "user.oomd_omit". If systemd-oomd sees these xattrs set to 1 on a candidate cgroup (i.e. while attempting to kill something) AND the cgroup is owned by root, it will either deprioritize the cgroup as a candidate (avoid) or remove it completely as a candidate (omit). Usage is restricted to root owned cgroups to prevent situations where an unprivileged user can set their own cgroups lower in the kill priority than another user's (and prevent them from omitting their units from systemd-oomd killing).
2021-02-09 17:47:34 +08:00
setfattr
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
setfont
test: use setpriv instead of su for user switch from root systemd-repart needs to find mkfs.ext4 for the test. This is located in the directory /usr/sbin on openSUSE Tumbleweed. But since the variable ALWAYS_SET_PATH in /etc/login.defs is set to yes, su re-initializes the $PATH variable and removes /usr/sbin. Hence, mkfs.ext4 is not found and the test fails. Using setpriv instead of su fixes this issue and is more appropriate to do the switch user task from root. [zjs: move setpriv to $BASICTOOLS and force-push to retrigger CI]
2023-03-14 22:21:29 +08:00
setpriv
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
setsid
test: convert TEST-45 to a normal meson test On my laptop, it now takes 0.23 s. We don't need the overhead of creating an image and spawning a machine to mangle a file in /tmp.
2020-03-21 03:22:42 +08:00
sfdisk
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
sh
sleep
test: move a couple of binaries to the BASICTOOLS array As they're not optional.
2023-11-20 23:26:49 +08:00
sort
test: move TEST-12-ISSUE-3171 setup to static files
2019-12-10 04:26:59 +08:00
stat
test: check if we skip the full setup on daemon-reexec A simple test case for issue #27106. Resolves: #27139
2023-04-07 01:14:12 +08:00
stty
test: move TEST-43-* setup to static files Setup of lingering is dropped. I don't think it's necessary for anything, because the unit is still pulled in from the testsuite service.
2020-03-21 02:05:36 +08:00
su
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
sulogin
test: move most of TEST-13-NSPAWN-SMOKE setup to static files
2019-12-10 04:45:46 +08:00
sysctl
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
tail
test: move TEST-25-IMPORT setup to static files Also run mask_supporting_services() here.
2019-12-10 16:10:28 +08:00
tar
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
tee
test
test: add test for cgroup v2 freezer support
2020-04-03 15:13:51 +08:00
timeout
test: move TEST-11-ISSUE-3166 setup to static files
2019-12-10 03:53:38 +08:00
touch
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
tr
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
true
test: convert TEST-45 to a normal meson test On my laptop, it now takes 0.23 s. We don't need the overhead of creating an image and spawning a machine to mangle a file in /tmp.
2020-03-21 03:22:42 +08:00
truncate
test: move a couple of binaries to the BASICTOOLS array As they're not optional.
2023-11-20 23:26:49 +08:00
tty
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
umount
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
uname
test: move most of TEST-13-NSPAWN-SMOKE setup to static files
2019-12-10 04:45:46 +08:00
unshare
test: import logind test from debian/ubuntu test suite
2022-06-01 07:56:08 +08:00
useradd
userdel
test: wc is needed by test/units/testsuite-50.sh
2021-09-22 15:04:52 +08:00
wc
test: add a test for #31384
2024-02-16 23:22:05 +08:00
whoami
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
xargs
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
xzcat
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
)
DEBUGTOOLS=(
df
dmesg
du
free
less
strace
vi
test: install /usr/libexec/vi as well since `/bin/vi` (at least on Fedora) is a shell wrapper which runs either `/bin/vim` or `/usr/libexec/vi` based on availability.
2022-04-16 20:43:17 +08:00
/usr/libexec/vi
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
)
test: remove duplicated code
2013-11-06 06:32:56 +08:00
tests: create the asan wrapper automatically if systemd has been built with ASAN
2018-07-05 12:09:30 +08:00
is_built_with_asan() {
test: don't wrap binaries built with ASan since they should handle loading other instrumented libraries without issues.
2022-06-16 01:43:11 +08:00
local _bin="${1:?}"
tests: create the asan wrapper automatically if systemd has been built with ASAN
2018-07-05 12:09:30 +08:00
if ! type -P objdump >/dev/null; then
test: don't use ddebug() before it's defined I'm pretty sure this is not the only case, but it's the one I recently noticed. Even though we call ddebug() from a function, that function is called before ddebug() is defined, resulting in the same issue as if we called just ddebug() in its place, i.e.: ..//test-functions: line 276: ddebug: command not found
2023-11-21 00:41:20 +08:00
echo "Failed to find objdump, assuming systemd hasn't been built with ASAN."
tests: create the asan wrapper automatically if systemd has been built with ASAN
2018-07-05 12:09:30 +08:00
return 1
fi
# Borrowed from https://github.com/google/oss-fuzz/blob/cd9acd02f9d3f6e80011cc1e9549be526ce5f270/infra/base-images/base-runner/bad_build_check#L182
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local _asan_calls
test: don't wrap binaries built with ASan since they should handle loading other instrumented libraries without issues.
2022-06-16 01:43:11 +08:00
_asan_calls="$(objdump -dC "$_bin" | grep -E "(callq?|brasl?|bl)\s.+__asan" -c)"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if ((_asan_calls < 1000)); then
tests: create the asan wrapper automatically if systemd has been built with ASAN
2018-07-05 12:09:30 +08:00
return 1
else
return 0
fi
}
test: make the coverage check safer for non-compiled builds
2021-10-22 00:34:21 +08:00
is_built_with_coverage() {
if get_bool "${NO_BUILD:=}" || ! command -v meson >/dev/null; then
return 1
fi
meson configure "${BUILD_DIR:?}" | grep 'b_coverage' | awk '{ print $2 }' | grep -q 'true'
}
test: don't wrap binaries built with ASan since they should handle loading other instrumented libraries without issues.
2022-06-16 01:43:11 +08:00
IS_BUILT_WITH_ASAN=$(is_built_with_asan "$SYSTEMD_JOURNALD" && echo yes || echo no)
test: make the coverage check safer for non-compiled builds
2021-10-22 00:34:21 +08:00
IS_BUILT_WITH_COVERAGE=$(is_built_with_coverage && echo yes || echo no)
tests: create the asan wrapper automatically if systemd has been built with ASAN
2018-07-05 12:09:30 +08:00
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$IS_BUILT_WITH_ASAN"; then
test: split the ASan wrapper into smaller blocks and tidy it up a bit No functional change (hopefully), just making it easier on the eyes.
2023-05-25 21:31:48 +08:00
PATH_TO_INIT="$ROOTLIBDIR/systemd-under-asan"
test: allow overriding $QEMU_MEM when running w/ ASan
2022-06-21 18:09:35 +08:00
QEMU_MEM="${QEMU_MEM:-2G}"
test: tidy up the ASan-related stuff
2021-03-07 05:21:30 +08:00
QEMU_SMP="${QEMU_SMP:-4}"
test: improve handling of ASan under clang Running integration tests with ASan is somewhat tricky to begin with, as we need to pre-load the ASan runtime DSO for certain services (like dbus), otherwise they won't start or behave as expected. In case of gcc this is pretty easy, as we need the runtime DSO during compilation, so it's already present on the host system. For clang things get more complicated, as ASan is compiled in statically by default, thus to enable the necessary dynamic-ish behavior one needs to compile with -shared-libasan and then correctly set LD_PRELOAD_PATH, as the runtime libraries are not in a standard library path.
2019-05-25 04:35:52 +08:00
# We need to correctly distinguish between gcc's and clang's ASan DSOs.
test: fix a yet another pipefail + pipe race Basically the same scenario as in a33e2692e162671f0d97856ad2f49a2620a1ec10, where `awk` exits as soon as it finds a match, thus sending SIGPIPE to `ldd` if it's not fast enough. That, in combination with `set -o pipefail` causes random & unexpected fails, like: ``` No journal files were found. -rw-r----- 1 root root 16777216 Apr 30 10:31 /var/tmp/TEST-01-BASIC_sanitizers-nspawn/system.journal TEST-01-BASIC RUN: Basic systemd setup [OK] systemd is not linked against the ASan DSO gcc does this by default, for clang compile with -shared-libasan make: *** [Makefile:2: clean-again] Error 1 make: Leaving directory '/build/test/TEST-01-BASIC' ```
2021-04-30 19:26:54 +08:00
if ASAN_RT_NAME="$(awk '/libasan.so/ {x=$1; exit} END {print x; exit x==""}' < <(ldd "$SYSTEMD"))"; then
test: improve handling of ASan under clang Running integration tests with ASan is somewhat tricky to begin with, as we need to pre-load the ASan runtime DSO for certain services (like dbus), otherwise they won't start or behave as expected. In case of gcc this is pretty easy, as we need the runtime DSO during compilation, so it's already present on the host system. For clang things get more complicated, as ASan is compiled in statically by default, thus to enable the necessary dynamic-ish behavior one needs to compile with -shared-libasan and then correctly set LD_PRELOAD_PATH, as the runtime libraries are not in a standard library path.
2019-05-25 04:35:52 +08:00
ASAN_COMPILER=gcc
test: tidy up the ASan-related stuff
2021-03-07 05:21:30 +08:00
ASAN_RT_PATH="$(readlink -f "$(${CC:-gcc} --print-file-name "$ASAN_RT_NAME")")"
test: fix a yet another pipefail + pipe race Basically the same scenario as in a33e2692e162671f0d97856ad2f49a2620a1ec10, where `awk` exits as soon as it finds a match, thus sending SIGPIPE to `ldd` if it's not fast enough. That, in combination with `set -o pipefail` causes random & unexpected fails, like: ``` No journal files were found. -rw-r----- 1 root root 16777216 Apr 30 10:31 /var/tmp/TEST-01-BASIC_sanitizers-nspawn/system.journal TEST-01-BASIC RUN: Basic systemd setup [OK] systemd is not linked against the ASan DSO gcc does this by default, for clang compile with -shared-libasan make: *** [Makefile:2: clean-again] Error 1 make: Leaving directory '/build/test/TEST-01-BASIC' ```
2021-04-30 19:26:54 +08:00
elif ASAN_RT_NAME="$(awk '/libclang_rt.asan/ {x=$1; exit} END {print x; exit x==""}' < <(ldd "$SYSTEMD"))"; then
test: improve handling of ASan under clang Running integration tests with ASan is somewhat tricky to begin with, as we need to pre-load the ASan runtime DSO for certain services (like dbus), otherwise they won't start or behave as expected. In case of gcc this is pretty easy, as we need the runtime DSO during compilation, so it's already present on the host system. For clang things get more complicated, as ASan is compiled in statically by default, thus to enable the necessary dynamic-ish behavior one needs to compile with -shared-libasan and then correctly set LD_PRELOAD_PATH, as the runtime libraries are not in a standard library path.
2019-05-25 04:35:52 +08:00
ASAN_COMPILER=clang
test: tidy up the ASan-related stuff
2021-03-07 05:21:30 +08:00
ASAN_RT_PATH="$(readlink -f "$(${CC:-clang} --print-file-name "$ASAN_RT_NAME")")"
test: improve handling of ASan under clang Running integration tests with ASan is somewhat tricky to begin with, as we need to pre-load the ASan runtime DSO for certain services (like dbus), otherwise they won't start or behave as expected. In case of gcc this is pretty easy, as we need the runtime DSO during compilation, so it's already present on the host system. For clang things get more complicated, as ASan is compiled in statically by default, thus to enable the necessary dynamic-ish behavior one needs to compile with -shared-libasan and then correctly set LD_PRELOAD_PATH, as the runtime libraries are not in a standard library path.
2019-05-25 04:35:52 +08:00
# As clang's ASan DSO is usually in a non-standard path, let's check if
# the environment is set accordingly. If not, warn the user and exit.
# We're not setting the LD_LIBRARY_PATH automagically here, because
# user should encounter (and fix) the same issue when running the unit
# tests (meson test)
test/test-functions: use binaries from $BUILD_DIR or installed system In Ubuntu CI, we test binaries from the installed system, not from $BUILD_DIR, so use the appropriate binary. Most of the calls to the binaries are part of checking/processing asan-built binaries, and so did not apply to Ubuntu CI, except for generating noise in the stderr log like: objdump: '/tmp/autopkgtest.83yGoI/build.fHB/src/test/TEST-01-BASIC/systemd-journald': No such file However this also applies to the call to systemd-nspawn, which the debian upstream test wrapper was sed-adjusting to use the installed binary instead of the binary in $BUILD_DIR. This commit allows removing that sed processing of the test-functions file during Ubuntu CI test.
2019-08-13 19:52:57 +08:00
if ldd "$SYSTEMD" | grep -q "libclang_rt.asan.*not found"; then
test: tidy up the ASan-related stuff
2021-03-07 05:21:30 +08:00
echo >&2 "clang's ASan DSO ($ASAN_RT_NAME) is not present in the runtime library path"
echo >&2 "Consider setting LD_LIBRARY_PATH=${ASAN_RT_PATH%/*}"
test: improve handling of ASan under clang Running integration tests with ASan is somewhat tricky to begin with, as we need to pre-load the ASan runtime DSO for certain services (like dbus), otherwise they won't start or behave as expected. In case of gcc this is pretty easy, as we need the runtime DSO during compilation, so it's already present on the host system. For clang things get more complicated, as ASan is compiled in statically by default, thus to enable the necessary dynamic-ish behavior one needs to compile with -shared-libasan and then correctly set LD_PRELOAD_PATH, as the runtime libraries are not in a standard library path.
2019-05-25 04:35:52 +08:00
exit 1
fi
else
echo >&2 "systemd is not linked against the ASan DSO"
echo >&2 "gcc does this by default, for clang compile with -shared-libasan"
exit 1
fi
test: tidy up the ASan-related stuff
2021-03-07 05:21:30 +08:00
echo "Detected ASan RT '$ASAN_RT_NAME' located at '$ASAN_RT_PATH'"
tests: create the asan wrapper automatically if systemd has been built with ASAN
2018-07-05 12:09:30 +08:00
fi
test: introduce test_require_bin() and use it No functional change.
2023-06-03 03:10:08 +08:00
test_require_bin() {
local bin
for bin in "$@"; do
if ! command -v "$bin" >/dev/null; then
echo "Required binary $bin not available, skipping the test"
exit 0
fi
done
}
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
find_qemu_bin() {
QEMU_BIN="${QEMU_BIN:-""}"
test-functions: don't nest KVM (#8883) Nested KVM is very flaky as we learnt from our CI. Hence, let's avoid KVM whenever we detect we are already running inside of KVM. Maybe one day nested KVM is fixed, at which point we can turn this on again, but for now let's simply avoid nested KVM, since reliable CI is more important than quick CI, I guess. And yes, avoiding KVM for our qemu runs does make things substantially slower, but I think it's not a complete loss. Inspired by @evverx' findings in: https://github.com/systemd/systemd/pull/8701#issuecomment-380213302
2018-05-03 02:06:13 +08:00
# SUSE and Red Hat call the binary qemu-kvm. Debian and Gentoo call it kvm.
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$QEMU_KVM"; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -n "$QEMU_BIN" ]] || QEMU_BIN="$(command -v kvm qemu-kvm 2>/dev/null | grep '^/' -m1)"
test-functions: don't nest KVM (#8883) Nested KVM is very flaky as we learnt from our CI. Hence, let's avoid KVM whenever we detect we are already running inside of KVM. Maybe one day nested KVM is fixed, at which point we can turn this on again, but for now let's simply avoid nested KVM, since reliable CI is more important than quick CI, I guess. And yes, avoiding KVM for our qemu runs does make things substantially slower, but I think it's not a complete loss. Inspired by @evverx' findings in: https://github.com/systemd/systemd/pull/8701#issuecomment-380213302
2018-05-03 02:06:13 +08:00
fi
test: rework run_qemu It tries to find a suitable QEMU binary and will use KVM if present. We can now configure QEMU from outside with 4 variables : - $QEMU_BIN : path to QEMU's binary - $KERNEL_APPEND : arguments appended to kernel cmdline - $KERNEL_BIN : path to a kernel Default /boot/vmlinuz-$KERNEL_VER - $INITRD : path to an initramfs Default /boot/initramfs-${KERNEL_VER}.img - $QEMU_SMP : number of CPU simulated by QEMU. Default 1 (from Alexander Graf's script: http://www.spinics.net/lists/kvm/msg72389.html)
2013-12-08 07:01:53 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -n "$ARCH" ]] || ARCH="$(uname -m)"
test: rework run_qemu It tries to find a suitable QEMU binary and will use KVM if present. We can now configure QEMU from outside with 4 variables : - $QEMU_BIN : path to QEMU's binary - $KERNEL_APPEND : arguments appended to kernel cmdline - $KERNEL_BIN : path to a kernel Default /boot/vmlinuz-$KERNEL_VER - $INITRD : path to an initramfs Default /boot/initramfs-${KERNEL_VER}.img - $QEMU_SMP : number of CPU simulated by QEMU. Default 1 (from Alexander Graf's script: http://www.spinics.net/lists/kvm/msg72389.html)
2013-12-08 07:01:53 +08:00
case $ARCH in
x86_64)
# QEMU's own build system calls it qemu-system-x86_64
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -n "$QEMU_BIN" ]] || QEMU_BIN="$(command -v qemu-system-x86_64 2>/dev/null | grep '^/' -m1)"
test: rework run_qemu It tries to find a suitable QEMU binary and will use KVM if present. We can now configure QEMU from outside with 4 variables : - $QEMU_BIN : path to QEMU's binary - $KERNEL_APPEND : arguments appended to kernel cmdline - $KERNEL_BIN : path to a kernel Default /boot/vmlinuz-$KERNEL_VER - $INITRD : path to an initramfs Default /boot/initramfs-${KERNEL_VER}.img - $QEMU_SMP : number of CPU simulated by QEMU. Default 1 (from Alexander Graf's script: http://www.spinics.net/lists/kvm/msg72389.html)
2013-12-08 07:01:53 +08:00
;;
i*86)
# new i386 version of QEMU
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -n "$QEMU_BIN" ]] || QEMU_BIN="$(command -v qemu-system-i386 2>/dev/null | grep '^/' -m1)"
test: rework run_qemu It tries to find a suitable QEMU binary and will use KVM if present. We can now configure QEMU from outside with 4 variables : - $QEMU_BIN : path to QEMU's binary - $KERNEL_APPEND : arguments appended to kernel cmdline - $KERNEL_BIN : path to a kernel Default /boot/vmlinuz-$KERNEL_VER - $INITRD : path to an initramfs Default /boot/initramfs-${KERNEL_VER}.img - $QEMU_SMP : number of CPU simulated by QEMU. Default 1 (from Alexander Graf's script: http://www.spinics.net/lists/kvm/msg72389.html)
2013-12-08 07:01:53 +08:00
# i386 version of QEMU
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -n "$QEMU_BIN" ]] || QEMU_BIN="$(command -v qemu 2>/dev/null | grep '^/' -m1)"
test: rework run_qemu It tries to find a suitable QEMU binary and will use KVM if present. We can now configure QEMU from outside with 4 variables : - $QEMU_BIN : path to QEMU's binary - $KERNEL_APPEND : arguments appended to kernel cmdline - $KERNEL_BIN : path to a kernel Default /boot/vmlinuz-$KERNEL_VER - $INITRD : path to an initramfs Default /boot/initramfs-${KERNEL_VER}.img - $QEMU_SMP : number of CPU simulated by QEMU. Default 1 (from Alexander Graf's script: http://www.spinics.net/lists/kvm/msg72389.html)
2013-12-08 07:01:53 +08:00
;;
test-functions: also add qemu define for ppc64
2016-11-20 11:45:23 +08:00
ppc64*)
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -n "$QEMU_BIN" ]] || QEMU_BIN="$(command -v qemu-system-ppc64 2>/dev/null | grep '^/' -m1)"
test-functions: also add qemu define for ppc64
2016-11-20 11:45:23 +08:00
;;
test: rework run_qemu It tries to find a suitable QEMU binary and will use KVM if present. We can now configure QEMU from outside with 4 variables : - $QEMU_BIN : path to QEMU's binary - $KERNEL_APPEND : arguments appended to kernel cmdline - $KERNEL_BIN : path to a kernel Default /boot/vmlinuz-$KERNEL_VER - $INITRD : path to an initramfs Default /boot/initramfs-${KERNEL_VER}.img - $QEMU_SMP : number of CPU simulated by QEMU. Default 1 (from Alexander Graf's script: http://www.spinics.net/lists/kvm/msg72389.html)
2013-12-08 07:01:53 +08:00
esac
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [[ ! -e "$QEMU_BIN" ]]; then
docs: use lowercase "qemu" and other minor tweaks to test docs We used both "qemu" and "QEMU", let's use the lower-case version everywhere since it's also the name of the binary and the version that people are most familiar with. The stuff under test/ is not only for the integeration tests, but also for various other test-related stuff, so adjust the docs a bit.
2022-05-04 18:29:31 +08:00
echo "Could not find a suitable qemu binary" >&2
test: rework run_qemu It tries to find a suitable QEMU binary and will use KVM if present. We can now configure QEMU from outside with 4 variables : - $QEMU_BIN : path to QEMU's binary - $KERNEL_APPEND : arguments appended to kernel cmdline - $KERNEL_BIN : path to a kernel Default /boot/vmlinuz-$KERNEL_VER - $INITRD : path to an initramfs Default /boot/initramfs-${KERNEL_VER}.img - $QEMU_SMP : number of CPU simulated by QEMU. Default 1 (from Alexander Graf's script: http://www.spinics.net/lists/kvm/msg72389.html)
2013-12-08 07:01:53 +08:00
return 1
fi
}
test: move TPM2-related setup stuff into test-functions And hide it all behind $TEST_SETUP_SWTPM.
2023-06-03 02:28:41 +08:00
qemu_setup_swtpm_socket() {
local pid state_dir tpm_device
if ! tpm_device="$(qemu_get_tpm_device)"; then
dinfo "Found QEMU version is too old for TPM2 on ppc64le"
exit 0
fi
state_dir="$(mktemp -d)"
swtpm socket --tpm2 --tpmstate dir="$state_dir" --ctrl type=unixio,path="$state_dir/sock" &
pid=$!
if ! kill -0 "$pid"; then
test: wait for the swtpm socket to appear On slower/overloaded systems it may take a bit for the swtpm socket to show up: I: Started swtpm as PID 189419 with state dir /tmp/tmp.pWqUutuGUj I: Configured emulated TPM2 device tpm-spapr + tee /var/tmp/systemd-test-TEST-70-TPM2_1/console.log + timeout --foreground 1200 /bin/qemu-system-ppc64le -smp 4 ... qemu-system-ppc64le: -chardev socket,id=chrtpm,path=/tmp/tmp.pWqUutuGUj/sock: Failed to connect to '/tmp/tmp.pWqUutuGUj/sock': No such file or directory E: qemu failed with exit code 1 Spotted regularly in the ppc64le cron job and in some Ubuntu CI/CentOS CI pr runs [0]. [0] https://github.com/systemd/systemd/pull/29183#issuecomment-1721727927
2023-09-23 15:13:56 +08:00
derror "Failed to start swtpm"
return 1
fi
if ! timeout 5 bash -c "until [[ -S $state_dir/sock ]]; do sleep .5; done"; then
derror "Failed to setup swtpm socket"
test: move TPM2-related setup stuff into test-functions And hide it all behind $TEST_SETUP_SWTPM.
2023-06-03 02:28:41 +08:00
return 1
fi
dinfo "Started swtpm as PID $pid with state dir $state_dir"
add_at_exit_handler "kill -TERM $pid 2>/dev/null; rm -rf '$state_dir'"
QEMU_OPTIONS+=" -chardev socket,id=chrtpm,path=$state_dir/sock -tpmdev emulator,id=tpm0,chardev=chrtpm -device $tpm_device,tpmdev=tpm0"
dinfo "Configured emulated TPM2 device $tpm_device"
return 0
}
qemu_get_tpm_device() {
local tpm_device="tpm-tis"
if [[ "$(uname -m)" == "ppc64le" ]]; then
# tpm-spapr support was introduced in qemu 5.0.0
if ! qemu_min_version "5.0.0"; then
return 1
fi
tpm_device="tpm-spapr"
fi
echo "$tpm_device"
return 0
}
test: use modern qemu numa arguments Upgrading to qemu 5.2 breaks TEST-36-NUMAPOLICY like: qemu-system-x86_64: total memory for NUMA nodes (0x0) should equal RAM size (0x20000000) Use the new (as in >=2014) form of memdev in test 36: -object memory-backend-ram,id=mem0,size=512M -numa node,memdev=mem0,nodeid=0 Since some target systems are as old as qemu 1.5.3 (CentOS7) but the new kind to specify was added in qemu 2.1 this needs to add version parsing and add the argument only when qemu is >=5.2. Fixes #17986. Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
2020-12-15 19:05:14 +08:00
# Compares argument #1=X.Y.Z (X&Y&Z = numeric) to the version of the installed qemu
# returns 0 if newer or equal
# returns 1 if older
# returns 2 if failing
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
qemu_min_version() {
test: use modern qemu numa arguments Upgrading to qemu 5.2 breaks TEST-36-NUMAPOLICY like: qemu-system-x86_64: total memory for NUMA nodes (0x0) should equal RAM size (0x20000000) Use the new (as in >=2014) form of memdev in test 36: -object memory-backend-ram,id=mem0,size=512M -numa node,memdev=mem0,nodeid=0 Since some target systems are as old as qemu 1.5.3 (CentOS7) but the new kind to specify was added in qemu 2.1 this needs to add version parsing and add the argument only when qemu is >=5.2. Fixes #17986. Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
2020-12-15 19:05:14 +08:00
find_qemu_bin || return 2
# get version from binary
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local qemu_ver
qemu_ver="$("$QEMU_BIN" --version | awk '/^QEMU emulator version ([0-9]*\.[0-9]*\.[0-9]*)/ {print $4}')"
test: use modern qemu numa arguments Upgrading to qemu 5.2 breaks TEST-36-NUMAPOLICY like: qemu-system-x86_64: total memory for NUMA nodes (0x0) should equal RAM size (0x20000000) Use the new (as in >=2014) form of memdev in test 36: -object memory-backend-ram,id=mem0,size=512M -numa node,memdev=mem0,nodeid=0 Since some target systems are as old as qemu 1.5.3 (CentOS7) but the new kind to specify was added in qemu 2.1 this needs to add version parsing and add the argument only when qemu is >=5.2. Fixes #17986. Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
2020-12-15 19:05:14 +08:00
# Check version string format
echo "$qemu_ver" | grep -q '^[0-9]*\.[0-9]*\.[0-9]*$' || return 2
echo "$1" | grep -q '^[0-9]*\.[0-9]*\.[0-9]*$' || return 2
# compare as last command to return that value
printf "%s\n%s\n" "$1" "$qemu_ver" | sort -V -C
}
test: add test for initrd credentials This extends the test framework a bit, and allows adding additional initrds to the qemu invocation, which we use here to place credentials in the new /run/systemd/@initrd/ credentials dir which are then passed to the host.
2023-06-30 01:03:08 +08:00
# Pads a file to multiple of 4 bytes
pad4_file() {
local size
size=$(stat -c "%s" "$1")
local padded
padded=$((((size + 3) / 4) * 4))
truncate -s "$padded" "$1"
}
docs: use lowercase "qemu" and other minor tweaks to test docs We used both "qemu" and "QEMU", let's use the lower-case version everywhere since it's also the name of the binary and the version that people are most familiar with. The stuff under test/ is not only for the integeration tests, but also for various other test-related stuff, so adjust the docs a bit.
2022-05-04 18:29:31 +08:00
# Return 0 if qemu did run (then you must check the result state/logs for actual
# success), or 1 if qemu is not available.
test: remove duplicated code
2013-11-06 06:32:56 +08:00
run_qemu() {
test: add test for initrd credentials This extends the test framework a bit, and allows adding additional initrds to the qemu invocation, which we use here to place credentials in the new /run/systemd/@initrd/ credentials dir which are then passed to the host.
2023-06-30 01:03:08 +08:00
if declare -F run_qemu_hook >/dev/null; then
if ! run_qemu_hook "${workspace}"; then
derror "check_qemu_hook() returned with EC > 0"
ret=4
fi
fi
test: generate a custom initrd for TEST-24 if $INITRD is unset Co-Authored-By: Yu Watanabe <watanabe.yu+github@gmail.com>
2022-05-26 19:19:11 +08:00
# If the test provided its own initrd, use it (e.g. TEST-24)
if [[ -z "$INITRD" && -f "${TESTDIR:?}/initrd.img" ]]; then
INITRD="$TESTDIR/initrd.img"
fi
test-functions: search for the initrd and kernel according to the BLS Also search for the initrd and kernel according to the boot loader specification.
2015-02-03 17:43:24 +08:00
if [ -f /etc/machine-id ]; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
read -r MACHINE_ID </etc/machine-id
test: do not hardcode location of EFI partition
2017-11-17 21:09:11 +08:00
[ -z "$INITRD" ] && [ -e "$EFI_MOUNT/$MACHINE_ID/$KERNEL_VER/initrd" ] \
&& INITRD="$EFI_MOUNT/$MACHINE_ID/$KERNEL_VER/initrd"
[ -z "$KERNEL_BIN" ] && [ -e "$EFI_MOUNT/$MACHINE_ID/$KERNEL_VER/linux" ] \
&& KERNEL_BIN="$EFI_MOUNT/$MACHINE_ID/$KERNEL_VER/linux"
test-functions: search for the initrd and kernel according to the BLS Also search for the initrd and kernel according to the boot loader specification.
2015-02-03 17:43:24 +08:00
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local CONSOLE=ttyS0
test/test-functions: on PPC64 use hvc0 console
2018-02-20 20:01:40 +08:00
test: remove the reboot counter just before spawning container/VM Doing that in test_run() is not enough, since it's called only once and spawns all QEMU/nspawn jobs.
2023-11-26 03:22:24 +08:00
# Reset the boot counter, if present
rm -f "${initdir:?}/var/tmp/.systemd_reboot_count"
test: correctly clean up test artifacts Remove the artifact files indicating test result (testok, failed, and skipped) just before running the test so we always get the latest and most relevant result instead of incorrectly consuming previous results. Discovered in https://github.com/systemd/systemd/pull/15378#issuecomment-616801873
2020-04-21 19:18:36 +08:00
rm -f "$initdir"/{testok,failed,skipped}
test: run tests directly from the loopback device Before, we'd copy the test tree into nspawn-root, and run the tests from there. This is OK, and doesn't actually take much extra time. But it uses quite a lot of extra disk space. So let's make things a bit more efficient by running directly from the image file. We still run the unprivileged nspawn tests from a copy. Once the kernel implements fs shift, we can do away with that too.
2019-12-13 21:21:31 +08:00
# make sure the initdir is not mounted to avoid concurrent access
cleanup_initdir
umount_loopback
tests: fix kernel/initrd searching (Arch) We can run sudo make run instead of the sudo make run KERNEL_BIN=/boot/vmlinuz-linux INITRD=/boot/initramfs-linux.img See https://github.com/systemd/systemd/issues/4607
2016-11-11 10:45:38 +08:00
if [[ ! "$KERNEL_BIN" ]]; then
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$LOOKS_LIKE_ARCH"; then
tests: fix kernel/initrd searching (Arch) We can run sudo make run instead of the sudo make run KERNEL_BIN=/boot/vmlinuz-linux INITRD=/boot/initramfs-linux.img See https://github.com/systemd/systemd/issues/4607
2016-11-11 10:45:38 +08:00
KERNEL_BIN=/boot/vmlinuz-linux
else
test/test-functions: on PP64 use vmlinux At least on Ubuntu, ppc64el uses vmlinux-, not vmlinuz. With this, it should be possible to run qemu tests on ppc64el as part of Ubuntu autopkgtests.
2018-02-20 04:47:41 +08:00
[ "$ARCH" ] || ARCH=$(uname -m)
case $ARCH in
ppc64*)
test: use a correct kernel binary on ppc64le machines Ubuntu calls the kernel binary as `vmlinux-*`, but RHEL/CentOS uses the "standard" `vmlinuz-` prefix. Follow-up to eaa602cb14217f2623c9c8c44b6f3012dc4d0b24.
2021-08-09 20:32:43 +08:00
# Ubuntu ppc64* calls the kernel binary as vmlinux-*, RHEL/CentOS
# uses the "standard" vmlinuz- prefix
[[ -e "/boot/vmlinux-$KERNEL_VER" ]] && KERNEL_BIN="/boot/vmlinux-$KERNEL_VER" || KERNEL_BIN="/boot/vmlinuz-$KERNEL_VER"
CONSOLE=hvc0
;;
test/test-functions: on PP64 use vmlinux At least on Ubuntu, ppc64el uses vmlinux-, not vmlinuz. With this, it should be possible to run qemu tests on ppc64el as part of Ubuntu autopkgtests.
2018-02-20 04:47:41 +08:00
*)
test: use a correct kernel binary on ppc64le machines Ubuntu calls the kernel binary as `vmlinux-*`, but RHEL/CentOS uses the "standard" `vmlinuz-` prefix. Follow-up to eaa602cb14217f2623c9c8c44b6f3012dc4d0b24.
2021-08-09 20:32:43 +08:00
KERNEL_BIN="/boot/vmlinuz-$KERNEL_VER"
;;
test/test-functions: on PP64 use vmlinux At least on Ubuntu, ppc64el uses vmlinux-, not vmlinuz. With this, it should be possible to run qemu tests on ppc64el as part of Ubuntu autopkgtests.
2018-02-20 04:47:41 +08:00
esac
tests: fix kernel/initrd searching (Arch) We can run sudo make run instead of the sudo make run KERNEL_BIN=/boot/vmlinuz-linux INITRD=/boot/initramfs-linux.img See https://github.com/systemd/systemd/issues/4607
2016-11-11 10:45:38 +08:00
fi
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local default_fedora_initrd="/boot/initramfs-${KERNEL_VER}.img"
local default_debian_initrd="/boot/initrd.img-${KERNEL_VER}"
local default_arch_initrd="/boot/initramfs-linux-fallback.img"
local default_suse_initrd="/boot/initrd-${KERNEL_VER}"
tests: fix kernel/initrd searching (Arch) We can run sudo make run instead of the sudo make run KERNEL_BIN=/boot/vmlinuz-linux INITRD=/boot/initramfs-linux.img See https://github.com/systemd/systemd/issues/4607
2016-11-11 10:45:38 +08:00
if [[ ! "$INITRD" ]]; then
if [[ -e "$default_fedora_initrd" ]]; then
INITRD="$default_fedora_initrd"
test: make the LOOKS_LIKE_* variables proper booleans
2023-11-20 19:17:05 +08:00
elif get_bool "$LOOKS_LIKE_DEBIAN" && [[ -e "$default_debian_initrd" ]]; then
tests: fix kernel/initrd searching (Arch) We can run sudo make run instead of the sudo make run KERNEL_BIN=/boot/vmlinuz-linux INITRD=/boot/initramfs-linux.img See https://github.com/systemd/systemd/issues/4607
2016-11-11 10:45:38 +08:00
INITRD="$default_debian_initrd"
test: make the LOOKS_LIKE_* variables proper booleans
2023-11-20 19:17:05 +08:00
elif get_bool "$LOOKS_LIKE_ARCH" && [[ -e "$default_arch_initrd" ]]; then
tests: fix kernel/initrd searching (Arch) We can run sudo make run instead of the sudo make run KERNEL_BIN=/boot/vmlinuz-linux INITRD=/boot/initramfs-linux.img See https://github.com/systemd/systemd/issues/4607
2016-11-11 10:45:38 +08:00
INITRD="$default_arch_initrd"
test: make the LOOKS_LIKE_* variables proper booleans
2023-11-20 19:17:05 +08:00
elif get_bool "$LOOKS_LIKE_SUSE" && [[ -e "$default_suse_initrd" ]]; then
tests: adapt test-functions to run tests on SUSE (#6270)
2017-07-07 02:30:14 +08:00
INITRD="$default_suse_initrd"
tests: fix kernel/initrd searching (Arch) We can run sudo make run instead of the sudo make run KERNEL_BIN=/boot/vmlinuz-linux INITRD=/boot/initramfs-linux.img See https://github.com/systemd/systemd/issues/4607
2016-11-11 10:45:38 +08:00
fi
fi
test: try to determine QEMU_SMP dynamically If the QEMU_SMP value has not been explicitly set, try to determine it from the number of online CPUs using the nproc utility. If this approach fails, fall back to the default value QEMU_SMP=1. This change should significantly help when running integration tests under QEMU on multicore systems.
2019-03-05 23:08:00 +08:00
# If QEMU_SMP was not explicitly set, try to determine the value 'dynamically'
# i.e. use the number of online CPUs on the host machine. If the nproc utility
# is not installed or there's some other error when calling it, fall back
# to the original value (QEMU_SMP=1).
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [[ -z "${QEMU_SMP:=}" ]]; then
test: try to determine QEMU_SMP dynamically If the QEMU_SMP value has not been explicitly set, try to determine it from the number of online CPUs using the nproc utility. If this approach fails, fall back to the default value QEMU_SMP=1. This change should significantly help when running integration tests under QEMU on multicore systems.
2019-03-05 23:08:00 +08:00
if ! QEMU_SMP=$(nproc); then
dwarn "nproc utility is not installed, falling back to QEMU_SMP=1"
QEMU_SMP=1
fi
fi
test: rework run_qemu It tries to find a suitable QEMU binary and will use KVM if present. We can now configure QEMU from outside with 4 variables : - $QEMU_BIN : path to QEMU's binary - $KERNEL_APPEND : arguments appended to kernel cmdline - $KERNEL_BIN : path to a kernel Default /boot/vmlinuz-$KERNEL_VER - $INITRD : path to an initramfs Default /boot/initramfs-${KERNEL_VER}.img - $QEMU_SMP : number of CPU simulated by QEMU. Default 1 (from Alexander Graf's script: http://www.spinics.net/lists/kvm/msg72389.html)
2013-12-08 07:01:53 +08:00
find_qemu_bin || return 1
test: move TPM2-related setup stuff into test-functions And hide it all behind $TEST_SETUP_SWTPM.
2023-06-03 02:28:41 +08:00
if get_bool "${TEST_SETUP_SWTPM:-}"; then
qemu_setup_swtpm_socket || return 1
fi
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
# Umount initdir to avoid concurrent access to the filesystem
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
_umount_dir "$initdir"
local kernel_params=()
local qemu_options=()
local qemu_cmd=("$QEMU_BIN")
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
tests: add UNIFIED_CGROUP_HIERARCHY=[default|hybrid] (#4675) This will simplify testing a bit. Mainly for https://github.com/systemd/systemd/pull/4670
2016-11-16 00:38:04 +08:00
if [[ "$UNIFIED_CGROUP_HIERARCHY" = "yes" ]]; then
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
kernel_params+=("systemd.unified_cgroup_hierarchy=yes")
tests: add UNIFIED_CGROUP_HIERARCHY=[default|hybrid] (#4675) This will simplify testing a bit. Mainly for https://github.com/systemd/systemd/pull/4670
2016-11-16 00:38:04 +08:00
elif [[ "$UNIFIED_CGROUP_HIERARCHY" = "no" ]]; then
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
kernel_params+=("systemd.unified_cgroup_hierarchy=no" "systemd.legacy_systemd_cgroup_controller=yes")
tests: add UNIFIED_CGROUP_HIERARCHY=[default|hybrid] (#4675) This will simplify testing a bit. Mainly for https://github.com/systemd/systemd/pull/4670
2016-11-16 00:38:04 +08:00
elif [[ "$UNIFIED_CGROUP_HIERARCHY" = "hybrid" ]]; then
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
kernel_params+=("systemd.unified_cgroup_hierarchy=no" "systemd.legacy_systemd_cgroup_controller=no")
tests: add UNIFIED_CGROUP_HIERARCHY=[default|hybrid] (#4675) This will simplify testing a bit. Mainly for https://github.com/systemd/systemd/pull/4670
2016-11-16 00:38:04 +08:00
elif [[ "$UNIFIED_CGROUP_HIERARCHY" != "default" ]]; then
dfatal "Unknown UNIFIED_CGROUP_HIERARCHY. Got $UNIFIED_CGROUP_HIERARCHY, expected [yes|no|hybrid|default]"
exit 1
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$LOOKS_LIKE_SUSE"; then
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
kernel_params+=("rd.hostonly=0")
scripts: use 4 space indentation We had all kinds of indentation: 2 sp, 3 sp, 4 sp, 8 sp, and mixed. 4 sp was the most common, in particular the majority of scripts under test/ used that. Let's standarize on 4 sp, because many commandlines are long and there's a lot of nesting, and with 8sp indentation less stuff fits. 4 sp also seems to be the default indentation, so this will make it less likely that people will mess up if they don't load the editor config. (I think people often use vi, and vi has no support to load project-wide configuration automatically. We distribute a .vimrc file, but it is not loaded by default, and even the instructions in it seem to discourage its use for security reasons.) Also remove the few vim config lines that were left. We should either have them on all files, or none. Also remove some strange stuff like '#!/bin/env bash', yikes.
2019-04-04 20:10:42 +08:00
fi
tests: adapt test-functions to run tests on SUSE (#6270)
2017-07-07 02:30:14 +08:00
integration test: pass 'noresume' to qemu When running on Debian/Ubuntu, I get a minute delay or so on every boot because the local initramfs tries to resume from hibernation. This is not really useful here, so always skip it
2023-05-18 20:08:56 +08:00
# Debian/Ubuntu's initramfs tries to check if it can resume from hibernation
# and wastes a minute or so probing disks, skip that as it's not useful here
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
kernel_params+=(
test: specify the rootfs by label otherwise the boot might fail when attaching multiple SATA devices to the VM.
2021-09-14 00:14:05 +08:00
"root=LABEL=systemd_boot"
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
"rw"
"raid=noautodetect"
"rd.luks=0"
"loglevel=2"
"init=$PATH_TO_INIT"
"console=$CONSOLE"
"SYSTEMD_UNIT_PATH=/usr/lib/systemd/tests/testdata/testsuite-$1.units:/usr/lib/systemd/tests/testdata/units:"
"systemd.unit=testsuite.target"
"systemd.wants=testsuite-$1.service"
integration test: pass 'noresume' to qemu When running on Debian/Ubuntu, I get a minute delay or so on every boot because the local initramfs tries to resume from hibernation. This is not really useful here, so always skip it
2023-05-18 20:08:56 +08:00
"noresume"
test: introduce $TEST_SKIP_SHUTDOWN To get rid of some boilerplate.
2023-10-24 18:35:03 +08:00
"oops=panic"
test: allow running only specified subtests/testcases Useful when debugging, e.g.: make -C test/TEST-74-AUX-UTILS clean setup run TEST_MATCH_SUBTEST=run Resolves: #27914
2023-06-05 16:47:21 +08:00
${TEST_MATCH_SUBTEST:+"systemd.setenv=TEST_MATCH_SUBTEST=$TEST_MATCH_SUBTEST"}
${TEST_MATCH_TESTCASE:+"systemd.setenv=TEST_MATCH_TESTCASE=$TEST_MATCH_TESTCASE"}
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
)
test: introduce $TEST_SKIP_SHUTDOWN To get rid of some boilerplate.
2023-10-24 18:35:03 +08:00
if ! get_bool "$INTERACTIVE_DEBUG" && ! get_bool "$TEST_SKIP_SHUTDOWN"; then
test: kill the machine on oops/panic/soft_lockup Otherwise the machine will hang on the panic until the timeout happens, which might waste quite a considerable amount of time in certain cases.
2022-09-30 14:48:53 +08:00
kernel_params+=(
"panic=1"
Revert "test: don't panic on soft lockups" Let's put this back in, as it could help with occasional machine lock ups on overloaded systems (and it didn't help with the original issue anyway). This reverts commit 3a89904e45cbbd96fb1c5d0768de5e5fcdaaa508.
2023-11-03 22:15:33 +08:00
"softlockup_panic=1"
test: kill the machine on oops/panic/soft_lockup Otherwise the machine will hang on the panic until the timeout happens, which might waste quite a considerable amount of time in certain cases.
2022-09-30 14:48:53 +08:00
"systemd.wants=end.service"
)
test: move TEST-01-BASIC setup to static files After this commit, only tests that were converted will work, because the interface to test_run() changed.
2019-12-10 01:56:13 +08:00
fi
test: fix public/private image detection and prefer the latter one
2020-05-17 16:46:34 +08:00
[ -e "$IMAGE_PRIVATE" ] && image="$IMAGE_PRIVATE" || image="$IMAGE_PUBLIC"
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
qemu_options+=(
-smp "$QEMU_SMP"
-net none
test: bump the base VM memory to 768M as with 512M some tests occasionally trip off OOM-killer (e.g. TEST-64 + multipath).
2022-08-25 20:52:30 +08:00
-m "${QEMU_MEM:-768M}"
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
-nographic
-kernel "$KERNEL_BIN"
-drive "format=raw,cache=unsafe,file=$image"
test: enable virtio-rng device for QEMU guests If rngd is included in the host initrd, QEMU guests need at least one source of entropy otherwise rngd will refuse to start. Hence this patch enables the virtio RNG device in QEMU guests (exposed as a HW RNG device available at /dev/hwrng). As a safety measure, the patch limits the data sent to the guest to 1KB per second in order to not let the guest starve the host entropy.
2022-06-02 15:31:55 +08:00
-device "virtio-rng-pci,max-bytes=1024,period=1000"
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
)
if [[ -n "${QEMU_OPTIONS:=}" ]]; then
local user_qemu_options
read -ra user_qemu_options <<< "$QEMU_OPTIONS"
qemu_options+=("${user_qemu_options[@]}")
fi
test-64: add tests for compat devlinks for NVMe drive
2023-04-08 02:33:32 +08:00
qemu_options+=(${QEMU_OPTIONS_ARRAY:+"${QEMU_OPTIONS_ARRAY[@]}"})
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
test: drop $SKIP_INITRD It's pointless nowadays, since we always need an initrd for the test VMs as we require modules that are usually not compiled in the kernel.
2023-06-07 21:21:10 +08:00
if [[ -n "$INITRD" ]]; then
test: add test for initrd credentials This extends the test framework a bit, and allows adding additional initrds to the qemu invocation, which we use here to place credentials in the new /run/systemd/@initrd/ credentials dir which are then passed to the host.
2023-06-30 01:03:08 +08:00
if [[ -n "$INITRD_EXTRA" ]]; then
# An addition initrd has been specified, let's combine it with the main one.
local t="$WORKDIR"/initrd.combined."$RANDOM"
# First, show contents of additional initrd
echo "Additional initrd contents:"
cpio -tv < "$INITRD_EXTRA"
# Copy the main initrd
zstd -d -c -f "$INITRD" > "$t"
add_at_exit_handler "rm $t"
# Kernel requires this to be padded to multiple of 4 bytes with zeroes
pad4_file "$t"
# Copy the additional initrd
cat "$INITRD_EXTRA" >> "$t"
pad4_file "$t"
qemu_options+=(-initrd "$t")
else
qemu_options+=(-initrd "$INITRD")
fi
test: rework run_qemu It tries to find a suitable QEMU binary and will use KVM if present. We can now configure QEMU from outside with 4 variables : - $QEMU_BIN : path to QEMU's binary - $KERNEL_APPEND : arguments appended to kernel cmdline - $KERNEL_BIN : path to a kernel Default /boot/vmlinuz-$KERNEL_VER - $INITRD : path to an initramfs Default /boot/initramfs-${KERNEL_VER}.img - $QEMU_SMP : number of CPU simulated by QEMU. Default 1 (from Alexander Graf's script: http://www.spinics.net/lists/kvm/msg72389.html)
2013-12-08 07:01:53 +08:00
fi
test: allow use of nested KVM Introduce TEST_NESTED_KVM variable, which allows use of nested KVM for QEMU VMs (which is disabled by default by KVM autodetection).
2019-09-27 05:29:38 +08:00
# Let's use KVM if possible
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if [[ -c /dev/kvm ]] && get_bool $QEMU_KVM; then
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
qemu_options+=(-machine "accel=kvm" -enable-kvm -cpu host)
test: make qemu use kvm if available If we're running in a KVM virtualized environment (such as Semaphore's Docker enabled platform), pass '-cpu host' to QEMU.
2015-11-19 20:17:15 +08:00
fi
tests: add test for https://github.com/systemd/systemd/issues/2730
2016-02-25 02:22:06 +08:00
if [[ "$QEMU_TIMEOUT" != "infinity" ]]; then
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
qemu_cmd=(timeout --foreground "$QEMU_TIMEOUT" "$QEMU_BIN")
tests: add test for https://github.com/systemd/systemd/issues/2730
2016-02-25 02:22:06 +08:00
fi
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
test: don't pre-process $KERNEL_APPEND Let's just rely on the word splitting done by bash instead of messing with that ourselves, as it's just adding extra complexity to appease one ShellCheck check. Also, this apparently never worked for the nspawn stuff anyway, since I forgot to set $IFS to an appropriate value, so it always put all arguments from $KERNEL_APPEND into a single array item with an extra newline, which then made systemd sad: ~# readarray arr <<< "foo bar baz"; for i in "${arr[@]}"; do echo "'$i'"; done 'foo bar baz ' ~# make -C test/TEST-45-TIMEDATE/ clean setup run BUILD_DIR=$PWD/build TEST_NO_QEMU=1 KERNEL_APPEND="systemd.log_level=console" ... ~# journalctl -o short-monotonic --no-hostname --file /var/tmp/systemd-tests/systemd-test.XaDX67/system.journal --grep "Failed to parse" -p info --no-pager [551138.986882] systemd-tmpfiles[21]: Failed to parse log level 'console [551138.987179] systemd-remount-fs[20]: Failed to parse log level 'console [551138.993125] systemd-sysusers[23]: Failed to parse log level 'console [551138.998685] journalctl[29]: Failed to parse log level 'console Resolves: #29945
2023-11-09 22:33:31 +08:00
(set -x; "${qemu_cmd[@]}" "${qemu_options[@]}" -append "${kernel_params[*]} ${KERNEL_APPEND:-}" |& tee "${TESTDIR:?}/console.log")
tests: track and check for timeouts If run_qemu() exits with non-zero, this either meant that QEMU was not available (which should be a SKIP) or that QEMU timed out if $QEMU_TIMEOUT was set (which then should be a FAIL). Limit the exit code of run_qemu() to QEMU availability only, and track timeouts separately through the new $TIMED_OUT variable, which is then checked in check_result_qemu(). Do the same for $NSPAWN_TIMEOUT and run_nspawn() so that nspawn and QEMU work similarly.
2016-06-24 18:11:19 +08:00
rc=$?
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if [ "$rc" -eq 124 ] && [ "$QEMU_TIMEOUT" != "infinity" ]; then
derror "Test timed out after ${QEMU_TIMEOUT}s"
tests: track and check for timeouts If run_qemu() exits with non-zero, this either meant that QEMU was not available (which should be a SKIP) or that QEMU timed out if $QEMU_TIMEOUT was set (which then should be a FAIL). Limit the exit code of run_qemu() to QEMU availability only, and track timeouts separately through the new $TIMED_OUT variable, which is then checked in check_result_qemu(). Do the same for $NSPAWN_TIMEOUT and run_nspawn() so that nspawn and QEMU work similarly.
2016-06-24 18:11:19 +08:00
TIMED_OUT=1
else
docs: use lowercase "qemu" and other minor tweaks to test docs We used both "qemu" and "QEMU", let's use the lower-case version everywhere since it's also the name of the binary and the version that people are most familiar with. The stuff under test/ is not only for the integeration tests, but also for various other test-related stuff, so adjust the docs a bit.
2022-05-04 18:29:31 +08:00
[ "$rc" != 0 ] && derror "qemu failed with exit code $rc"
tests: track and check for timeouts If run_qemu() exits with non-zero, this either meant that QEMU was not available (which should be a SKIP) or that QEMU timed out if $QEMU_TIMEOUT was set (which then should be a FAIL). Limit the exit code of run_qemu() to QEMU availability only, and track timeouts separately through the new $TIMED_OUT variable, which is then checked in check_result_qemu(). Do the same for $NSPAWN_TIMEOUT and run_nspawn() so that nspawn and QEMU work similarly.
2016-06-24 18:11:19 +08:00
fi
return 0
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
test: merge check_nspawn() into run_nspawn() This makes nspawn tests symmetric with run_qemu() which also exits with 1 if QEMU is not available.
2016-06-24 22:23:39 +08:00
# Return 0 if nspawn did run (then you must check the result state/logs for actual
# success), or 1 if nspawn is not available.
test: remove duplicated code
2013-11-06 06:32:56 +08:00
run_nspawn() {
test: merge check_nspawn() into run_nspawn() This makes nspawn tests symmetric with run_qemu() which also exits with 1 if QEMU is not available.
2016-06-24 22:23:39 +08:00
[[ -d /run/systemd/system ]] || return 1
test: remove the reboot counter just before spawning container/VM Doing that in test_run() is not enough, since it's called only once and spawns all QEMU/nspawn jobs.
2023-11-26 03:22:24 +08:00
# Reset the boot counter, if present
rm -f "${initdir:?}/var/tmp/.systemd_reboot_count"
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
rm -f "${initdir:?}"/{testok,failed,skipped}
local nspawn_cmd=()
local nspawn_options=(
"--register=no"
"--kill-signal=SIGKILL"
"--directory=${1:?}"
"--setenv=SYSTEMD_UNIT_PATH=/usr/lib/systemd/tests/testdata/testsuite-$2.units:/usr/lib/systemd/tests/testdata/units:"
test: use a unique machine name for each nspawn test to avoid scope clashing in case some previous test crashed and/or didn't clean up properly. Currently all test machines are called `root`, since the name is automagically derived from the container path (in this case `/var/tmp/systemd-test.XXXXXX/root`). E.g. (from Ubuntu CI): ``` [23:10:12] --x-- Running TEST-71-HOSTNAME --x-- make: Entering directory '/tmp/autopkgtest.5LjnBV/build.0mE/systemd/test/TEST-71-HOSTNAME' + make -C TEST-71-HOSTNAME setup run TEST-71-HOSTNAME SETUP: test hostnamed Reusing existing cached image /tmp/autopkgtest.5LjnBV/build.0mE/systemd/test/TEST-71-HOSTNAME/../default.img → /tmp/autopkgtest.5LjnBV/build.0mE/systemd/test/default.img '/var/tmp/systemd-test.1yy2SS/default.img' -> '/tmp/autopkgtest.5LjnBV/build.0mE/systemd/test/default.img' I: Masking supporting services '/var/tmp/systemd-test.1yy2SS/root/etc/systemd/system/systemd-hwdb-update.service' -> '/dev/null' '/var/tmp/systemd-test.1yy2SS/root/etc/systemd/system/systemd-journal-catalog-update.service' -> '/dev/null' '/var/tmp/systemd-test.1yy2SS/root/etc/systemd/system/systemd-networkd.service' -> '/dev/null' '/var/tmp/systemd-test.1yy2SS/root/etc/systemd/system/systemd-networkd.socket' -> '/dev/null' '/var/tmp/systemd-test.1yy2SS/root/etc/systemd/system/systemd-resolved.service' -> '/dev/null' TEST-71-HOSTNAME RUN: test hostnamed + env --unset=UNIFIED_CGROUP_HIERARCHY --unset=SYSTEMD_NSPAWN_UNIFIED_HIERARCHY timeout --foreground 1200 /bin/systemd-nspawn --register=no --kill-signal=SIGKILL --directory=/var/tmp/systemd-test.1yy2SS/root --setenv=SYSTEMD_UNIT_PATH=/usr/lib/systemd/tests/testdata/testsuite-71.units:/usr/lib/systemd/tests/testdata/units: /lib/systemd/systemd systemd.unit=testsuite.target systemd.wants=testsuite-71.service systemd.wants=end.service Spawning container root on /var/tmp/systemd-test.1yy2SS/root. Press ^] three times within 1s to kill container. Failed to allocate scope: Unit root.scope already exists. E: nspawn failed with exit code 1 ```
2022-08-26 21:59:37 +08:00
"--machine=TEST-$TESTID"
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
)
local kernel_params=(
"$PATH_TO_INIT"
"systemd.unit=testsuite.target"
"systemd.wants=testsuite-$2.service"
test: allow running only specified subtests/testcases Useful when debugging, e.g.: make -C test/TEST-74-AUX-UTILS clean setup run TEST_MATCH_SUBTEST=run Resolves: #27914
2023-06-05 16:47:21 +08:00
${TEST_MATCH_SUBTEST:+"systemd.setenv=TEST_MATCH_SUBTEST=$TEST_MATCH_SUBTEST"}
${TEST_MATCH_TESTCASE:+"systemd.setenv=TEST_MATCH_TESTCASE=$TEST_MATCH_TESTCASE"}
test: move TEST-01-BASIC setup to static files After this commit, only tests that were converted will work, because the interface to test_run() changed.
2019-12-10 01:56:13 +08:00
)
test: fix debugging of nspawn tests The pipe stuff introduced in 701e0c2660 causes nspawn to switch the console from 'interactive' into 'read-only' which is a bit useless when debugging. Let's set --console=interactive explicitly in such case. Follow-up to 701e0c2660.
2023-08-29 04:42:36 +08:00
if get_bool "$INTERACTIVE_DEBUG"; then
nspawn_options+=("--console=interactive")
test: introduce $TEST_SKIP_SHUTDOWN To get rid of some boilerplate.
2023-10-24 18:35:03 +08:00
elif ! get_bool "$TEST_SKIP_SHUTDOWN"; then
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
kernel_params+=("systemd.wants=end.service")
test: move TEST-01-BASIC setup to static files After this commit, only tests that were converted will work, because the interface to test_run() changed.
2019-12-10 01:56:13 +08:00
fi
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
if [[ -n "${NSPAWN_ARGUMENTS:=}" ]]; then
local user_nspawn_arguments
read -ra user_nspawn_arguments <<< "$NSPAWN_ARGUMENTS"
nspawn_options+=("${user_nspawn_arguments[@]}")
fi
tests: add UNIFIED_CGROUP_HIERARCHY=[default|hybrid] (#4675) This will simplify testing a bit. Mainly for https://github.com/systemd/systemd/pull/4670
2016-11-16 00:38:04 +08:00
if [[ "$UNIFIED_CGROUP_HIERARCHY" = "hybrid" ]]; then
nspawn: rename UNIFIED_CGROUP_HIERARCHY to SYSTEMD_NSPAWN_UNIFIED_HIERARCHY We should never have used an unprefixed environment variable name. All other systemd-nspawn variables have the "SYSTEMD_NSPAWN_" prefix, and all other systemd variables have the "SYSTEMD_" prefix. The new variable name takes precedence, but we fall back to checking the old one. If only the old one is found, a warning is emitted. In addition, SYSTEMD_NSPAWN_UNIFIED_HIERARCHY="" is accepted as an override to avoid looking for the old variable name. We have a variable with the same name ($UNIFIED_CGROUP_HIERARCHY) in tests, which governs both systemd-nspawn and qemu behaviour. It is not renamed.
2019-09-27 20:17:41 +08:00
dwarn "nspawn doesn't support SYSTEMD_NSPAWN_UNIFIED_HIERARCHY=hybrid, skipping"
tests: add UNIFIED_CGROUP_HIERARCHY=[default|hybrid] (#4675) This will simplify testing a bit. Mainly for https://github.com/systemd/systemd/pull/4670
2016-11-16 00:38:04 +08:00
exit
elif [[ "$UNIFIED_CGROUP_HIERARCHY" = "yes" || "$UNIFIED_CGROUP_HIERARCHY" = "no" ]]; then
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
nspawn_cmd+=(env "SYSTEMD_NSPAWN_UNIFIED_HIERARCHY=$UNIFIED_CGROUP_HIERARCHY")
tests: add UNIFIED_CGROUP_HIERARCHY=[default|hybrid] (#4675) This will simplify testing a bit. Mainly for https://github.com/systemd/systemd/pull/4670
2016-11-16 00:38:04 +08:00
elif [[ "$UNIFIED_CGROUP_HIERARCHY" = "default" ]]; then
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
nspawn_cmd+=(env "--unset=UNIFIED_CGROUP_HIERARCHY" "--unset=SYSTEMD_NSPAWN_UNIFIED_HIERARCHY")
tests: add UNIFIED_CGROUP_HIERARCHY=[default|hybrid] (#4675) This will simplify testing a bit. Mainly for https://github.com/systemd/systemd/pull/4670
2016-11-16 00:38:04 +08:00
else
dfatal "Unknown UNIFIED_CGROUP_HIERARCHY. Got $UNIFIED_CGROUP_HIERARCHY, expected [yes|no|hybrid|default]"
exit 1
fi
tests: introduce UNIFIED_CGROUP_HIERARCHY (#3419) There are many cgroups-related changes (thanks, @htejun!) This commit will simplify testing a bit. Use: make run UNIFIED_CGROUP_HIERARCHY=yes to enable cgroup-v2 make run UNIFIED_CGROUP_HIERARCHY=no to enable cgroup-v1
2016-06-03 17:17:00 +08:00
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
if [[ "$NSPAWN_TIMEOUT" != "infinity" ]]; then
nspawn_cmd+=(timeout --foreground "$NSPAWN_TIMEOUT" "$SYSTEMD_NSPAWN")
else
nspawn_cmd+=("$SYSTEMD_NSPAWN")
fi
test: don't pre-process $KERNEL_APPEND Let's just rely on the word splitting done by bash instead of messing with that ourselves, as it's just adding extra complexity to appease one ShellCheck check. Also, this apparently never worked for the nspawn stuff anyway, since I forgot to set $IFS to an appropriate value, so it always put all arguments from $KERNEL_APPEND into a single array item with an extra newline, which then made systemd sad: ~# readarray arr <<< "foo bar baz"; for i in "${arr[@]}"; do echo "'$i'"; done 'foo bar baz ' ~# make -C test/TEST-45-TIMEDATE/ clean setup run BUILD_DIR=$PWD/build TEST_NO_QEMU=1 KERNEL_APPEND="systemd.log_level=console" ... ~# journalctl -o short-monotonic --no-hostname --file /var/tmp/systemd-tests/systemd-test.XaDX67/system.journal --grep "Failed to parse" -p info --no-pager [551138.986882] systemd-tmpfiles[21]: Failed to parse log level 'console [551138.987179] systemd-remount-fs[20]: Failed to parse log level 'console [551138.993125] systemd-sysusers[23]: Failed to parse log level 'console [551138.998685] journalctl[29]: Failed to parse log level 'console Resolves: #29945
2023-11-09 22:33:31 +08:00
# Word splitting here is intentional
# shellcheck disable=SC2086
(set -x; "${nspawn_cmd[@]}" "${nspawn_options[@]}" "${kernel_params[@]}" ${KERNEL_APPEND:-} |& tee "${TESTDIR:?}/console.log")
tests: track and check for timeouts If run_qemu() exits with non-zero, this either meant that QEMU was not available (which should be a SKIP) or that QEMU timed out if $QEMU_TIMEOUT was set (which then should be a FAIL). Limit the exit code of run_qemu() to QEMU availability only, and track timeouts separately through the new $TIMED_OUT variable, which is then checked in check_result_qemu(). Do the same for $NSPAWN_TIMEOUT and run_nspawn() so that nspawn and QEMU work similarly.
2016-06-24 18:11:19 +08:00
rc=$?
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if [ "$rc" -eq 124 ] && [ "$NSPAWN_TIMEOUT" != "infinity" ]; then
derror "Test timed out after ${NSPAWN_TIMEOUT}s"
tests: track and check for timeouts If run_qemu() exits with non-zero, this either meant that QEMU was not available (which should be a SKIP) or that QEMU timed out if $QEMU_TIMEOUT was set (which then should be a FAIL). Limit the exit code of run_qemu() to QEMU availability only, and track timeouts separately through the new $TIMED_OUT variable, which is then checked in check_result_qemu(). Do the same for $NSPAWN_TIMEOUT and run_nspawn() so that nspawn and QEMU work similarly.
2016-06-24 18:11:19 +08:00
TIMED_OUT=1
else
[ "$rc" != 0 ] && derror "nspawn failed with exit code $rc"
fi
return 0
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
test-50-dissect: move minimal image setup to common setup function So that it can be re-used for other tests by simply setting TEST_INSTALL_VERITY_MINIMAL=1 in test.sh, without having to replicate the setup commands.
2020-08-28 21:17:31 +08:00
# Build two very minimal root images, with two units, one is the same and one is different across them
install_verity_minimal() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Set up a set of minimal images for verity verification"
if [ -e "$initdir/usr/share/minimal.raw" ]; then
test-50-dissect: move minimal image setup to common setup function So that it can be re-used for other tests by simply setting TEST_INSTALL_VERITY_MINIMAL=1 in test.sh, without having to replicate the setup commands.
2020-08-28 21:17:31 +08:00
return
fi
if ! command -v mksquashfs >/dev/null 2>&1; then
dfatal "mksquashfs not found"
exit 1
fi
if ! command -v veritysetup >/dev/null 2>&1; then
dfatal "veritysetup not found"
exit 1
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
# Local modifications of some global variables is intentional in this
# subshell (SC2030)
# shellcheck disable=SC2030
test-50-dissect: move minimal image setup to common setup function So that it can be re-used for other tests by simply setting TEST_INSTALL_VERITY_MINIMAL=1 in test.sh, without having to replicate the setup commands.
2020-08-28 21:17:31 +08:00
(
BASICTOOLS=(
bash
cat
Add ExtensionImages directive to form overlays Add support for overlaying images for services on top of their root fs, using a read-only overlay.
2021-02-22 20:20:33 +08:00
grep
test-50-dissect: move minimal image setup to common setup function So that it can be re-used for other tests by simply setting TEST_INSTALL_VERITY_MINIMAL=1 in test.sh, without having to replicate the setup commands.
2020-08-28 21:17:31 +08:00
mount
sleep
test: Add touch into minimal verity test image
2023-06-20 19:53:45 +08:00
touch
test-50-dissect: move minimal image setup to common setup function So that it can be re-used for other tests by simply setting TEST_INSTALL_VERITY_MINIMAL=1 in test.sh, without having to replicate the setup commands.
2020-08-28 21:17:31 +08:00
)
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
oldinitdir="$initdir"
rm -rfv "$TESTDIR/minimal"
export initdir="$TESTDIR/minimal"
core: do not attempt to add 'private' symlinks when RootImage/RootDirectory are used A bind mount is added directly from private on the host to the actual destination directory, no need for the symlinks (which cannot be created as the bind mount happens first and creates the target as an actual directory) Fixes https://github.com/systemd/systemd/issues/22264
2022-01-27 22:10:34 +08:00
# app0 will use TemporaryFileSystem=/var/lib, app1 will need the mount point in the base image
mkdir -p "$initdir/usr/lib/systemd/system" "$initdir/usr/lib/extension-release.d" "$initdir/etc" "$initdir/var/tmp" "$initdir/opt" "$initdir/var/lib/app1"
test-50-dissect: move minimal image setup to common setup function So that it can be re-used for other tests by simply setting TEST_INSTALL_VERITY_MINIMAL=1 in test.sh, without having to replicate the setup commands.
2020-08-28 21:17:31 +08:00
setup_basic_dirs
install_basic_tools
test: configure ldconfig's cache in the minimal verity images The glibc stuff on ppc64le C8S is a little bit wild, as there are two versions: ``` $ ldconfig -p | grep libc.so libc.so.6 (libc6,64bit, hwcap: "power9", OS ABI: Linux 3.10.0) => /lib64/glibc-hwcaps/power9/libc-2.28.so libc.so.6 (libc6,64bit, OS ABI: Linux 3.10.0) => /lib64/libc.so.6 ``` and with `/etc/ld.so.cache` present all binaries use the first one: ``` $ ldd /bin/cat linux-vdso64.so.1 (0x00007fffa8070000) libc.so.6 => /lib64/glibc-hwcaps/power9/libc-2.28.so (0x00007fffa7e20000) /lib64/ld64.so.2 (0x00007fffa8090000) ``` However, without the cache the binaries will fall back to `/lib64/libc.so.6` which breaks tests that use the minimal verity images (like TEST-29), because we install only the first version (that's shown by `ldd` at the time the images are created): ``` [ 91.595343] testsuite-29.sh[747]: + portablectl --profile=trusted attach --now --runtime /usr/share/minimal_0.raw minimal-app0 Starting systemd-portabled.service... [ OK ] Started systemd-portabled.service. Starting minimal-app0-foo.service... Starting minimal-app0.service... [ 104.432217] cat[858]: cat: error while loading shared libraries: libc.so.6: cannot open shared object file: No such file or directory [ 104.435080] cat[857]: cat: error while loading shared libraries: libc.so.6: cannot open shared object file: No such file or directory [FAILED] Failed to start minimal-app0.service. See 'systemctl status minimal-app0.service' for details. ``` ``` $ chroot /var/tmp/systemd-test.nMHPfc/minimal/ /bin/bash: error while loading shared libraries: libc.so.6: cannot open shared object file: No such file or directory ``` With the ldconfig's cache it seems to work as expected: ``` $ chroot /var/tmp/systemd-test.gVtYLg/minimal bash-4.4# cat --version cat (GNU coreutils) 8.30 ... ```
2022-10-05 21:26:16 +08:00
install_ld_so_conf
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
# Shellcheck treats [[ -v VAR ]] as an assignment to avoid a different
# issue, thus falsely triggering SC2030 in this case
# See: koalaman/shellcheck#1409
test: fix TEST-50-DISSECT under sanitizers This test would normally get stuck when trying to mount the verity image due to: systemd-udevd[299]: dm-0: '/usr/sbin/dmsetup udevflags 6293812'(err) '==371==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD.' systemd-udevd[299]: dm-0: Process '/usr/sbin/dmsetup udevflags 6293812' failed with exit code 1 ... systemd-udevd[299]: dm-0: '/usr/sbin/dmsetup udevcomplete 6293812'(err) '==372==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD.' systemd-udevd[299]: dm-0: Process '/usr/sbin/dmsetup udevcomplete 6293812' failed with exit code 1. systemd-udevd[299]: dm-0: Command "/usr/sbin/dmsetup udevcomplete 6293812" returned 1 (error), ignoring. so let's add a simple udev rule which sets $LD_PRELOAD for the block subsystem. Also, install the ASan library along with necessary dependencies into the verity minimal image, to get rid of the annoying (yet harmless) errors about missing library from $LD_LIBRARY.
2021-03-07 05:22:32 +08:00
if [[ -v ASAN_RT_PATH ]]; then
# If we're compiled with ASan, install the ASan RT (and its dependencies)
# into the verity images to get rid of the annoying errors about
# missing $LD_PRELOAD libraries.
inst_libs "$ASAN_RT_PATH"
inst_library "$ASAN_RT_PATH"
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
cp "$os_release" "$initdir/usr/lib/os-release"
ln -s ../usr/lib/os-release "$initdir/etc/os-release"
touch "$initdir/etc/machine-id" "$initdir/etc/resolv.conf"
touch "$initdir/opt/some_file"
echo MARKER=1 >>"$initdir/usr/lib/os-release"
test: rename service used in TEST-29-PORTABLE to avoid conflict There's an app0.service in the extension app0.raw, so don't use the same name for a unit in minimal.raw
2022-01-27 22:07:20 +08:00
echo "PORTABLE_PREFIXES=app0 minimal minimal-app0" >>"$initdir/usr/lib/os-release"
cat >"$initdir/usr/lib/systemd/system/minimal-app0.service" <<EOF
[Service]
ExecStartPre=cat /usr/lib/os-release
ExecStart=sleep 120
EOF
cp "$initdir/usr/lib/systemd/system/minimal-app0.service" "$initdir/usr/lib/systemd/system/minimal-app0-foo.service"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
TEST-29: do not pass -q to mk/unsquashfs, not supported on CentOS 8
2022-03-08 00:20:42 +08:00
mksquashfs "$initdir" "$oldinitdir/usr/share/minimal_0.raw" -noappend
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
veritysetup format "$oldinitdir/usr/share/minimal_0.raw" "$oldinitdir/usr/share/minimal_0.verity" | \
grep '^Root hash:' | cut -f2 | tr -d '\n' >"$oldinitdir/usr/share/minimal_0.roothash"
sed -i "s/MARKER=1/MARKER=2/g" "$initdir/usr/lib/os-release"
test: rename service used in TEST-29-PORTABLE to avoid conflict There's an app0.service in the extension app0.raw, so don't use the same name for a unit in minimal.raw
2022-01-27 22:07:20 +08:00
rm "$initdir/usr/lib/systemd/system/minimal-app0-foo.service"
cp "$initdir/usr/lib/systemd/system/minimal-app0.service" "$initdir/usr/lib/systemd/system/minimal-app0-bar.service"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
TEST-29: do not pass -q to mk/unsquashfs, not supported on CentOS 8
2022-03-08 00:20:42 +08:00
mksquashfs "$initdir" "$oldinitdir/usr/share/minimal_1.raw" -noappend
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
veritysetup format "$oldinitdir/usr/share/minimal_1.raw" "$oldinitdir/usr/share/minimal_1.verity" | \
grep '^Root hash:' | cut -f2 | tr -d '\n' >"$oldinitdir/usr/share/minimal_1.roothash"
Add ExtensionImages directive to form overlays Add support for overlaying images for services on top of their root fs, using a read-only overlay.
2021-02-22 20:20:33 +08:00
# Rolling distros like Arch do not set VERSION_ID
local version_id=""
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if grep -q "^VERSION_ID=" "$os_release"; then
version_id="$(grep "^VERSION_ID=" "$os_release")"
Add ExtensionImages directive to form overlays Add support for overlaying images for services on top of their root fs, using a read-only overlay.
2021-02-22 20:20:33 +08:00
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
export initdir="$TESTDIR/app0"
mkdir -p "$initdir/usr/lib/extension-release.d" "$initdir/usr/lib/systemd/system" "$initdir/opt"
grep "^ID=" "$os_release" >"$initdir/usr/lib/extension-release.d/extension-release.app0"
echo "${version_id}" >>"$initdir/usr/lib/extension-release.d/extension-release.app0"
portable: add PORTABLE_NAME_AND_VERSION= and other metadata to LogsExtraFields= This is useful to identify log messages with metadata from the images they run on. Look for ID/VERSION_ID/IMAGE_ID/IMAGE_VERSION/BUILD_ID, with a SYSEXT_ prefix if we are looking at an extension, and append via LogExtraFields= as respectively PORTABLE_NAME_AND_VERSION= in case of a single image. In case of extensions, append as PORTABLE_ROOT_NAME_AND_VERSION= for the base and one PORTABLE_EXTENSION_AND_VERSION= for each extension. Example with a base and two extensions, with the unit coming from the first extension: [Service] RootImage=/home/bluca/git/systemd/base.raw Environment=PORTABLE=app0.raw BindReadOnlyPaths=/etc/os-release:/run/host/os-release LogExtraFields=PORTABLE=app0.raw Environment=PORTABLE_ROOT=base.raw LogExtraFields=PORTABLE_ROOT=base.raw LogExtraFields=PORTABLE_ROOT_NAME_AND_VERSION=debian_10 ExtensionImages=/home/bluca/git/systemd/app0.raw LogExtraFields=PORTABLE_EXTENSION=app0.raw LogExtraFields=PORTABLE_EXTENSION_NAME_AND_VERSION=app_0 ExtensionImages=/home/bluca/git/systemd/app1.raw LogExtraFields=PORTABLE_EXTENSION=app1.raw LogExtraFields=PORTABLE_EXTENSION_NAME_AND_VERSION=app_1
2023-03-23 09:23:04 +08:00
( echo "${version_id}"
echo "SYSEXT_IMAGE_ID=app" ) >>"$initdir/usr/lib/extension-release.d/extension-release.app0"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
cat >"$initdir/usr/lib/systemd/system/app0.service" <<EOF
Add ExtensionImages directive to form overlays Add support for overlaying images for services on top of their root fs, using a read-only overlay.
2021-02-22 20:20:33 +08:00
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/opt/script0.sh
core: do not attempt to add 'private' symlinks when RootImage/RootDirectory are used A bind mount is added directly from private on the host to the actual destination directory, no need for the symlinks (which cannot be created as the bind mount happens first and creates the target as an actual directory) Fixes https://github.com/systemd/systemd/issues/22264
2022-01-27 22:10:34 +08:00
TemporaryFileSystem=/var/lib
StateDirectory=app0
RuntimeDirectory=app0
Add ExtensionImages directive to form overlays Add support for overlaying images for services on top of their root fs, using a read-only overlay.
2021-02-22 20:20:33 +08:00
EOF
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
cat >"$initdir/opt/script0.sh" <<EOF
Add ExtensionImages directive to form overlays Add support for overlaying images for services on top of their root fs, using a read-only overlay.
2021-02-22 20:20:33 +08:00
#!/bin/bash
set -e
test -e /usr/lib/os-release
test: drop whitespace after shell redirection operators (The one case that is left unchanged is '< <(subcommand)'.) This way, the style with no gap was already dominant. This way, the reader immediately knows that ' < ' is a comparison operator and ' << ' is a shift. In a few cases, replace custom EOF replacement by just EOF. There is no point in using someting like "_EOL" unless "EOF" appears in the text.
2023-02-06 04:41:24 +08:00
echo bar >\${STATE_DIRECTORY}/foo
Add ExtensionImages directive to form overlays Add support for overlaying images for services on top of their root fs, using a read-only overlay.
2021-02-22 20:20:33 +08:00
cat /usr/lib/extension-release.d/extension-release.app0
EOF
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
chmod +x "$initdir/opt/script0.sh"
echo MARKER=1 >"$initdir/usr/lib/systemd/system/some_file"
TEST-29: do not pass -q to mk/unsquashfs, not supported on CentOS 8
2022-03-08 00:20:42 +08:00
mksquashfs "$initdir" "$oldinitdir/usr/share/app0.raw" -noappend
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
portable: add support for confext Support confexts for portable services
2023-08-17 02:43:06 +08:00
export initdir="$TESTDIR/conf0"
mkdir -p "$initdir/etc/extension-release.d" "$initdir/etc/systemd/system" "$initdir/opt"
grep "^ID=" "$os_release" >"$initdir/etc/extension-release.d/extension-release.conf0"
echo "${version_id}" >>"$initdir/etc/extension-release.d/extension-release.conf0"
( echo "${version_id}"
echo "CONFEXT_IMAGE_ID=app" ) >>"$initdir/etc/extension-release.d/extension-release.conf0"
echo MARKER_1 >"$initdir/etc/systemd/system/some_file"
mksquashfs "$initdir" "$oldinitdir/usr/share/conf0.raw" -noappend
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
export initdir="$TESTDIR/app1"
mkdir -p "$initdir/usr/lib/extension-release.d" "$initdir/usr/lib/systemd/system" "$initdir/opt"
extension-release: search for other files if expected name not found In some cases image names are unpredictable - some orchestrators/deployment tools like to mangle names to suit their internal formats. In these cases, the requirement that the extension-release file matches exactly the image name where it's contained cannot work. Allow falling back to loading the first regular file which name starts with 'extension-release' located in /usr/lib/extension-release.d/ and tagged with a user.extension-release.strict extended attribute with a true value, if the one with the expected name cannot be found.
2021-07-23 03:41:34 +08:00
grep "^ID=" "$os_release" >"$initdir/usr/lib/extension-release.d/extension-release.app2"
test: test new SYSEXT_SCOPE=/PORTABLE_PREFIXES= fields in TEST-29
2021-11-23 23:34:40 +08:00
( echo "${version_id}"
echo "SYSEXT_SCOPE=portable"
portable: add PORTABLE_NAME_AND_VERSION= and other metadata to LogsExtraFields= This is useful to identify log messages with metadata from the images they run on. Look for ID/VERSION_ID/IMAGE_ID/IMAGE_VERSION/BUILD_ID, with a SYSEXT_ prefix if we are looking at an extension, and append via LogExtraFields= as respectively PORTABLE_NAME_AND_VERSION= in case of a single image. In case of extensions, append as PORTABLE_ROOT_NAME_AND_VERSION= for the base and one PORTABLE_EXTENSION_AND_VERSION= for each extension. Example with a base and two extensions, with the unit coming from the first extension: [Service] RootImage=/home/bluca/git/systemd/base.raw Environment=PORTABLE=app0.raw BindReadOnlyPaths=/etc/os-release:/run/host/os-release LogExtraFields=PORTABLE=app0.raw Environment=PORTABLE_ROOT=base.raw LogExtraFields=PORTABLE_ROOT=base.raw LogExtraFields=PORTABLE_ROOT_NAME_AND_VERSION=debian_10 ExtensionImages=/home/bluca/git/systemd/app0.raw LogExtraFields=PORTABLE_EXTENSION=app0.raw LogExtraFields=PORTABLE_EXTENSION_NAME_AND_VERSION=app_0 ExtensionImages=/home/bluca/git/systemd/app1.raw LogExtraFields=PORTABLE_EXTENSION=app1.raw LogExtraFields=PORTABLE_EXTENSION_NAME_AND_VERSION=app_1
2023-03-23 09:23:04 +08:00
echo "SYSEXT_IMAGE_ID=app"
echo "SYSEXT_IMAGE_VERSION=1"
test: test new SYSEXT_SCOPE=/PORTABLE_PREFIXES= fields in TEST-29
2021-11-23 23:34:40 +08:00
echo "PORTABLE_PREFIXES=app1" ) >>"$initdir/usr/lib/extension-release.d/extension-release.app2"
extension-release: search for other files if expected name not found In some cases image names are unpredictable - some orchestrators/deployment tools like to mangle names to suit their internal formats. In these cases, the requirement that the extension-release file matches exactly the image name where it's contained cannot work. Allow falling back to loading the first regular file which name starts with 'extension-release' located in /usr/lib/extension-release.d/ and tagged with a user.extension-release.strict extended attribute with a true value, if the one with the expected name cannot be found.
2021-07-23 03:41:34 +08:00
setfattr -n user.extension-release.strict -v false "$initdir/usr/lib/extension-release.d/extension-release.app2"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
cat >"$initdir/usr/lib/systemd/system/app1.service" <<EOF
Add ExtensionImages directive to form overlays Add support for overlaying images for services on top of their root fs, using a read-only overlay.
2021-02-22 20:20:33 +08:00
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/opt/script1.sh
core: do not attempt to add 'private' symlinks when RootImage/RootDirectory are used A bind mount is added directly from private on the host to the actual destination directory, no need for the symlinks (which cannot be created as the bind mount happens first and creates the target as an actual directory) Fixes https://github.com/systemd/systemd/issues/22264
2022-01-27 22:10:34 +08:00
StateDirectory=app1
RuntimeDirectory=app1
Add ExtensionImages directive to form overlays Add support for overlaying images for services on top of their root fs, using a read-only overlay.
2021-02-22 20:20:33 +08:00
EOF
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
cat >"$initdir/opt/script1.sh" <<EOF
Add ExtensionImages directive to form overlays Add support for overlaying images for services on top of their root fs, using a read-only overlay.
2021-02-22 20:20:33 +08:00
#!/bin/bash
set -e
test -e /usr/lib/os-release
test: drop whitespace after shell redirection operators (The one case that is left unchanged is '< <(subcommand)'.) This way, the style with no gap was already dominant. This way, the reader immediately knows that ' < ' is a comparison operator and ' << ' is a shift. In a few cases, replace custom EOF replacement by just EOF. There is no point in using someting like "_EOL" unless "EOF" appears in the text.
2023-02-06 04:41:24 +08:00
echo baz >\${STATE_DIRECTORY}/foo
extension-release: search for other files if expected name not found In some cases image names are unpredictable - some orchestrators/deployment tools like to mangle names to suit their internal formats. In these cases, the requirement that the extension-release file matches exactly the image name where it's contained cannot work. Allow falling back to loading the first regular file which name starts with 'extension-release' located in /usr/lib/extension-release.d/ and tagged with a user.extension-release.strict extended attribute with a true value, if the one with the expected name cannot be found.
2021-07-23 03:41:34 +08:00
cat /usr/lib/extension-release.d/extension-release.app2
Add ExtensionImages directive to form overlays Add support for overlaying images for services on top of their root fs, using a read-only overlay.
2021-02-22 20:20:33 +08:00
EOF
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
chmod +x "$initdir/opt/script1.sh"
echo MARKER=1 >"$initdir/usr/lib/systemd/system/other_file"
TEST-29: do not pass -q to mk/unsquashfs, not supported on CentOS 8
2022-03-08 00:20:42 +08:00
mksquashfs "$initdir" "$oldinitdir/usr/share/app1.raw" -noappend
sysext: support distribution-independent extensions using ID=_any A sysext image that merely contains static binaries has no dependency on the host distribution and should be able to be used anywhere. Support the special '_any' value for the ID field in the extension to opt-out of ID and VERSION_ID/SYSEXT_LEVEL matching. See https://github.com/systemd/systemd/issues/24061
2022-08-09 19:34:18 +08:00
export initdir="$TESTDIR/app-nodistro"
mkdir -p "$initdir/usr/lib/extension-release.d" "$initdir/usr/lib/systemd/system"
sysext: introduce ARCHITECTURE field to match host architecture When an extension image has binaries they should match the host architecture. Currently there is no way to specify this requirement. Introduce an ARCHITECTURE field in the extension's release file that may be set to prevent loading on the wrong host architecture. Since this new field is introduced late, we don't want to make specifying it mandatory as it would break existing sysext images. See https://github.com/systemd/systemd/issues/24061
2022-08-09 23:07:20 +08:00
( echo "ID=_any"
echo "ARCHITECTURE=_any" ) >"$initdir/usr/lib/extension-release.d/extension-release.app-nodistro"
sysext: support distribution-independent extensions using ID=_any A sysext image that merely contains static binaries has no dependency on the host distribution and should be able to be used anywhere. Support the special '_any' value for the ID field in the extension to opt-out of ID and VERSION_ID/SYSEXT_LEVEL matching. See https://github.com/systemd/systemd/issues/24061
2022-08-09 19:34:18 +08:00
echo MARKER=1 >"$initdir/usr/lib/systemd/system/some_file"
mksquashfs "$initdir" "$oldinitdir/usr/share/app-nodistro.raw" -noappend
test: add test for confext service-scoped support
2023-08-15 02:29:13 +08:00
export initdir="$TESTDIR/service-scoped-test"
mkdir -p "$initdir/etc/extension-release.d" "$initdir/etc/systemd/system"
( echo "ID=_any"
echo "ARCHITECTURE=_any" ) >"$initdir/etc/extension-release.d/extension-release.service-scoped-test"
echo MARKER_CONFEXT_123 >"$initdir/etc/systemd/system/some_file"
mksquashfs "$initdir" "$oldinitdir/etc/service-scoped-test.raw" -noappend
sysext: support EXTENSION_RELOAD_MANAGER metadata This metadata (EXTENSION_RELOAD_MANAGER) can be set to "1" to reload the manager when merging/refreshing/unmerging a system extension image. This can be useful in case the sysext image provides systemd units that need to be loaded. With `--no-reload`, one can deactivate the EXTENSION_RELOAD_MANAGER metadata interpretation. Signed-off-by: Mathieu Tortuyaux <mtortuyaux@microsoft.com>
2023-04-25 20:06:59 +08:00
# We need to create a dedicated sysext image to test the reload mechanism. If we share an image to install the
# 'foo.service' it will be loaded from another test run, which will impact the targeted test.
export initdir="$TESTDIR/app-reload"
mkdir -p "$initdir/usr/lib/extension-release.d" "$initdir/usr/lib/systemd/system"
( echo "ID=_any"
echo "ARCHITECTURE=_any"
echo "EXTENSION_RELOAD_MANAGER=1" ) >"$initdir/usr/lib/extension-release.d/extension-release.app-reload"
mkdir -p "$initdir/usr/lib/systemd/system/multi-user.target.d"
cat >"${initdir}/usr/lib/systemd/system/foo.service" <<EOF
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=echo foo
[Install]
WantedBy=multi-user.target
EOF
{ echo "[Unit]"; echo "Upholds=foo.service"; } > "$initdir/usr/lib/systemd/system/multi-user.target.d/10-foo-service.conf"
mksquashfs "$initdir" "$oldinitdir/usr/share/app-reload.raw" -noappend
test-50-dissect: move minimal image setup to common setup function So that it can be re-used for other tests by simply setting TEST_INSTALL_VERITY_MINIMAL=1 in test.sh, without having to replicate the setup commands.
2020-08-28 21:17:31 +08:00
)
}
test: remove duplicated code
2013-11-06 06:32:56 +08:00
setup_basic_environment() {
# create the basic filesystem layout
setup_basic_dirs
install_systemd
install_missing_libraries
install_config_files
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
install_zoneinfo
test: remove duplicated code
2013-11-06 06:32:56 +08:00
create_rc_local
install_basic_tools
install_libnss
install_pam
install_dbus
install_fonts
test: install C.UTF-8 and English locales
2022-06-14 08:13:00 +08:00
install_locales
test: remove duplicated code
2013-11-06 06:32:56 +08:00
install_keymaps
test: install libxkbcommon and x11 keymaps
2022-06-14 08:12:00 +08:00
install_x11_keymaps
test: remove duplicated code
2013-11-06 06:32:56 +08:00
install_terminfo
install_execs
tests: install mkfs.ext4, mkfs.vfat and modules into the test image This allows TEST-58-REPART to at least start. It fails later with with loopback device errors.
2021-04-21 15:58:26 +08:00
install_fs_tools
install_modules
test: remove duplicated code
2013-11-06 06:32:56 +08:00
install_plymouth
test: if haveged is part of initrd it needs to be installed in the image too Otherwise haveged won't survive when switching root from initrd to host making haveged service in host fail.
2021-08-06 22:47:32 +08:00
install_haveged
test: remove duplicated code
2013-11-06 06:32:56 +08:00
install_debug_tools
install_ld_so_conf
test: move TEST-43-* setup to static files Setup of lingering is dropped. I don't think it's necessary for anything, because the unit is still pulled in from the testsuite service.
2020-03-21 02:05:36 +08:00
install_testuser
has_user_dbus_socket && install_user_dbus
tests: add setup_selinux
2016-01-25 10:45:43 +08:00
setup_selinux
test: remove duplicated code
2013-11-06 06:32:56 +08:00
install_depmod_files
generate_module_dependencies
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$IS_BUILT_WITH_ASAN"; then
test: improve handling of ASan under clang Running integration tests with ASan is somewhat tricky to begin with, as we need to pre-load the ASan runtime DSO for certain services (like dbus), otherwise they won't start or behave as expected. In case of gcc this is pretty easy, as we need the runtime DSO during compilation, so it's already present on the host system. For clang things get more complicated, as ASan is compiled in statically by default, thus to enable the necessary dynamic-ish behavior one needs to compile with -shared-libasan and then correctly set LD_PRELOAD_PATH, as the runtime libraries are not in a standard library path.
2019-05-25 04:35:52 +08:00
create_asan_wrapper
tests: create the asan wrapper automatically if systemd has been built with ASAN
2018-07-05 12:09:30 +08:00
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$TEST_INSTALL_VERITY_MINIMAL"; then
test-50-dissect: move minimal image setup to common setup function So that it can be re-used for other tests by simply setting TEST_INSTALL_VERITY_MINIMAL=1 in test.sh, without having to replicate the setup commands.
2020-08-28 21:17:31 +08:00
install_verity_minimal
fi
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
tests: add setup_selinux
2016-01-25 10:45:43 +08:00
setup_selinux() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Setup SELinux"
tests: add setup_selinux
2016-01-25 10:45:43 +08:00
# don't forget KERNEL_APPEND='... selinux=1 ...'
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if ! get_bool "$SETUP_SELINUX"; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "SETUP_SELINUX != yes, skipping SELinux configuration"
tests: add setup_selinux
2016-01-25 10:45:43 +08:00
return 0
fi
test: make TEST-06-SELINUX work with the refpolicy and beef it up a bit Currently the test works only with policy shipped by Fedora, which makes it pretty much useless in most of our CIs. Let's drop the custom module and make the test more generic, so it works with the refpolicy as well, which should allow us to run it on Arch and probably even in Ubuntu CI.
2023-11-14 19:53:51 +08:00
for dir in /etc/selinux /usr/share/selinux; do
rm -rf "${initdir:?}/$dir"
if ! cp -ar "$dir" "$initdir/$dir"; then
dfatal "Failed to copy $dir"
exit 1
fi
done
tests: add setup_selinux
2016-01-25 10:45:43 +08:00
test: add a reasoning why we don't use the SELinux-provided units
2023-06-06 04:32:27 +08:00
# We use a custom autorelabel service instead of the SELinux provided set
# of units & a generator, since the generator overrides the default target
# to the SELinux one when it detects /.autorelabel. However, we use
# systemd.unit= on the kernel command cmdline which always takes precedence,
# rendering all SELinux efforts useless. Also, pulling in selinux-autorelabel.service
# explicitly doesn't work either, as it doesn't check for the presence of /.autorelabel
# and does the relabeling unconditionally which always ends with a reboot, so
# we end up in a reboot loop (and it also spews quite a lot of errors as it
# wants /etc/fstab and dracut-initramfs-restore).
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
touch "$initdir/.autorelabel"
mkdir -p "$initdir/usr/lib/systemd/tests/testdata/units/basic.target.wants"
ln -sf ../autorelabel.service "$initdir/usr/lib/systemd/tests/testdata/units/basic.target.wants/"
tests: add setup_selinux
2016-01-25 10:45:43 +08:00
test: make TEST-06-SELINUX work with the refpolicy and beef it up a bit Currently the test works only with policy shipped by Fedora, which makes it pretty much useless in most of our CIs. Let's drop the custom module and make the test more generic, so it works with the refpolicy as well, which should allow us to run it on Arch and probably even in Ubuntu CI.
2023-11-14 19:53:51 +08:00
# Tools requires by fixfiles
image_install awk bash cat chcon expr egrep find grep head secon setfiles rm sort uname uniq
image_install fixfiles getenforce load_policy selinuxenabled sestatus
tests: add setup_selinux
2016-01-25 10:45:43 +08:00
}
tests: add function for valgrind installation I used it for d9814c76ec35e53a6b6448c0 Very handy:)
2016-01-14 16:09:09 +08:00
install_valgrind() {
if ! type -p valgrind; then
dfatal "Failed to install valgrind"
exit 1
fi
test: use sed and grep instead of perl This gets rid of the all-but-one remaining uses of perl. I tested the new code on my machine, so I'm fairly confident that it works as expected. install_iscsi() has one majestic perl invocation, but we can't get rid of it easily: it extends the code of tgt-admin to print some list of files. Obviously this only works because tgt-admin is written in perl, and perl will be installed if tgt-admin is installed. install_iscsi() is used in TEST-64-UDEV-STORAGE conditionally if tgtadm is installed, so this can stay as is.
2023-05-08 19:43:03 +08:00
local valgrind_bins valgrind_libs valgrind_supp
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
test: use sed and grep instead of perl This gets rid of the all-but-one remaining uses of perl. I tested the new code on my machine, so I'm fairly confident that it works as expected. install_iscsi() has one majestic perl invocation, but we can't get rid of it easily: it extends the code of tgt-admin to print some list of files. Obviously this only works because tgt-admin is written in perl, and perl will be installed if tgt-admin is installed. install_iscsi() is used in TEST-64-UDEV-STORAGE conditionally if tgtadm is installed, so this can stay as is.
2023-05-08 19:43:03 +08:00
readarray -t valgrind_bins < <(strace -e execve valgrind /bin/true 2>&1 >/dev/null |
sed -r -n 's/execve\("([^"]*)".*/\1/p')
tests: make it possible to install valgrind
2022-05-04 04:50:39 +08:00
image_install "${valgrind_bins[@]}"
tests: add function for valgrind installation I used it for d9814c76ec35e53a6b6448c0 Very handy:)
2016-01-14 16:09:09 +08:00
test: use sed and grep instead of perl This gets rid of the all-but-one remaining uses of perl. I tested the new code on my machine, so I'm fairly confident that it works as expected. install_iscsi() has one majestic perl invocation, but we can't get rid of it easily: it extends the code of tgt-admin to print some list of files. Obviously this only works because tgt-admin is written in perl, and perl will be installed if tgt-admin is installed. install_iscsi() is used in TEST-64-UDEV-STORAGE conditionally if tgtadm is installed, so this can stay as is.
2023-05-08 19:43:03 +08:00
readarray -t valgrind_libs < <(LD_DEBUG=files valgrind /bin/true 2>&1 >/dev/null |
sed -r -n 's|.*calling init: (/.*vgpreload_.*)|\1|p')
tests: make it possible to install valgrind
2022-05-04 04:50:39 +08:00
image_install "${valgrind_libs[@]}"
tests: add function for valgrind installation I used it for d9814c76ec35e53a6b6448c0 Very handy:)
2016-01-14 16:09:09 +08:00
test: use sed and grep instead of perl This gets rid of the all-but-one remaining uses of perl. I tested the new code on my machine, so I'm fairly confident that it works as expected. install_iscsi() has one majestic perl invocation, but we can't get rid of it easily: it extends the code of tgt-admin to print some list of files. Obviously this only works because tgt-admin is written in perl, and perl will be installed if tgt-admin is installed. install_iscsi() is used in TEST-64-UDEV-STORAGE conditionally if tgtadm is installed, so this can stay as is.
2023-05-08 19:43:03 +08:00
readarray -t valgrind_supp < <(strace -e open valgrind /bin/true 2>&1 >/dev/null |
sed -r -n 's,open\("([^"]*(/debug[^"]*|\.supp))".*= [0-9].*,\1,p')
image_install "${valgrind_supp[@]}"
tests: add function for valgrind installation I used it for d9814c76ec35e53a6b6448c0 Very handy:)
2016-01-14 16:09:09 +08:00
}
tests: add function for valgrind wrapper creation I used it for d9814c7 and bffd87b
2016-01-15 10:52:12 +08:00
create_valgrind_wrapper() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local valgrind_wrapper="$initdir/$ROOTLIBDIR/systemd-under-valgrind"
ddebug "Create $valgrind_wrapper"
cat >"$valgrind_wrapper" <<EOF
treewide: more portable bash shebangs As in 2a5fcfae024ffc370bb780572279f45a1da3f946 and in 3e67e5c9928f8b1e1c5a63def88d53ed1fed12eb using /usr/bin/env allows bash to be looked up in PATH rather than being hard-coded. As with the previous changes the same arguments apply - distributions have scripts to rewrite shebangs on installation and they know what locations to rely on. - For tests/compilation we should rather rely on the user to have setup there PATH correctly. In particular this makes testing from git easier on NixOS where do not provide /bin/bash to improve compose-ability.
2020-03-04 17:35:06 +08:00
#!/usr/bin/env bash
tests: add function for valgrind wrapper creation I used it for d9814c7 and bffd87b
2016-01-15 10:52:12 +08:00
tests: mount proc before `exec valgrind` (#4044) Fixes: $ sudo make run QEMU_BIN=/usr/bin/qemu-system-x86_64 KERNEL_APPEND=systemd.unit=multi-user.target SKIP_INITRD=yes TEST RUN: Basic systemd setup + /usr/bin/qemu-system-x86_64 -smp 1 -net none -m 512M -nographic -kernel /boot/vmlinuz-4.6.7-200.fc23.x86_64 -drive format=raw,cache=unsafe,file=/var/tmp/systemd-test.izx99J/rootdisk.img -append 'root=/dev/sda1 raid=noautodetect loglevel=2 init=/usr/lib/systemd/systemd-under-valgrind rw console=ttyS0 selinux=0 systemd.unified_cgroup_hierarchy=no systemd.unit=multi-user.target ' valgrind: warning (non-fatal): readlink("/proc/self/exe") failed. valgrind: continuing, however --trace-children=yes will not work. --1:0: aspacem Valgrind: FATAL: can't open /proc/self/maps --1:0: aspacem Exiting now. [ 3.152367] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000100 [ 3.152367] [ 3.152367] CPU: 0 PID: 1 Comm: memcheck-amd64- Not tainted 4.6.7-200.fc23.x86_64 #1 [ 3.152367] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.8.2-20150714_191134- 04/01/2014 [ 3.152367] 0000000000000086 0000000047ca1666 ffff88001ea43e00 ffffffff813d954e [ 3.152367] ffffffff81a205a0 ffff88001ea43e98 ffff88001ea43e88 ffffffff811b5557 [ 3.152367] ffffffff00000010 ffff88001ea43e98 ffff88001ea43e30 0000000047ca1666 [ 3.152367] Call Trace: [ 3.152367] [<ffffffff813d954e>] dump_stack+0x63/0x85 [ 3.152367] [<ffffffff811b5557>] panic+0xde/0x220 [ 3.152367] [<ffffffff810ab9c3>] do_exit+0xb43/0xb50 [ 3.152367] [<ffffffff810aba57>] do_group_exit+0x47/0xb0 [ 3.152367] [<ffffffff810abad4>] SyS_exit_group+0x14/0x20 [ 3.152367] [<ffffffff817dae72>] entry_SYSCALL_64_fastpath+0x1a/0xa4 [ 3.152367] Kernel Offset: disabled [ 3.152367] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000100 [ 3.152367] QEMU: Terminated
2016-08-26 23:49:56 +08:00
mount -t proc proc /proc
tests: make valgrind_wrapper track file descriptors
2022-05-04 04:51:56 +08:00
exec valgrind --leak-check=full --track-fds=yes --log-file=/valgrind.out $ROOTLIBDIR/systemd "\$@"
tests: add function for valgrind wrapper creation I used it for d9814c7 and bffd87b
2016-01-15 10:52:12 +08:00
EOF
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
chmod 0755 "$valgrind_wrapper"
tests: add function for valgrind wrapper creation I used it for d9814c7 and bffd87b
2016-01-15 10:52:12 +08:00
}
tests: add a wrapper for when systemd is built with ASAN
2018-07-03 11:25:53 +08:00
create_asan_wrapper() {
test: make sure we get PID1's stack trace on ASan/UBSan errors As hitting an ASan/UBSan error in PID1 results in a crash (and a kernel panic when running under qemu), we usually lose the stack trace which makes debugging quite painful. Let's mitigate this by forwarding the stack trace to multiple places - namely to a file and the syslog.
2023-06-23 20:28:30 +08:00
local asan_wrapper default_asan_options default_ubsan_options default_environment manager_environment
test: improve handling of ASan under clang Running integration tests with ASan is somewhat tricky to begin with, as we need to pre-load the ASan runtime DSO for certain services (like dbus), otherwise they won't start or behave as expected. In case of gcc this is pretty easy, as we need the runtime DSO during compilation, so it's already present on the host system. For clang things get more complicated, as ASan is compiled in statically by default, thus to enable the necessary dynamic-ish behavior one needs to compile with -shared-libasan and then correctly set LD_PRELOAD_PATH, as the runtime libraries are not in a standard library path.
2019-05-25 04:35:52 +08:00
test: tidy up the ASan-related stuff
2021-03-07 05:21:30 +08:00
[[ -z "$ASAN_RT_PATH" ]] && dfatal "ASAN_RT_PATH is empty, but it shouldn't be"
test: allow sanitized binaries to dump a core If a binary built with ASan crashes for a reason unrelated to ASan stuff, we're left with pretty much nothing, as there is neither an ASan trace nor a coredump. Let's make this slightly more debug-able by allowing such binaries to dump a core, but without the huge shadow map (we should be actually fine by just setting disable_coredump=0, since use_madv_dontdump defaults to true, but let's play it safe and not potentially dump a 16+ TB core file).
2024-01-05 03:31:32 +08:00
default_asan_options="${ASAN_OPTIONS:-strict_string_checks=1:detect_stack_use_after_return=1:check_initialization_order=1:strict_init_order=1:disable_coredump=0:use_madv_dontdump=1}"
test: split the ASan wrapper into smaller blocks and tidy it up a bit No functional change (hopefully), just making it easier on the eyes.
2023-05-25 21:31:48 +08:00
default_ubsan_options="${UBSAN_OPTIONS:-print_stacktrace=1:print_summary=1:halt_on_error=1}"
test: improve handling of ASan under clang Running integration tests with ASan is somewhat tricky to begin with, as we need to pre-load the ASan runtime DSO for certain services (like dbus), otherwise they won't start or behave as expected. In case of gcc this is pretty easy, as we need the runtime DSO during compilation, so it's already present on the host system. For clang things get more complicated, as ASan is compiled in statically by default, thus to enable the necessary dynamic-ish behavior one needs to compile with -shared-libasan and then correctly set LD_PRELOAD_PATH, as the runtime libraries are not in a standard library path.
2019-05-25 04:35:52 +08:00
test: split the ASan wrapper into smaller blocks and tidy it up a bit No functional change (hopefully), just making it easier on the eyes.
2023-05-25 21:31:48 +08:00
if [[ "$ASAN_COMPILER" == "clang" ]]; then
# clang: install llvm-symbolizer to generate useful reports
# See: https://clang.llvm.org/docs/AddressSanitizer.html#symbolizing-the-reports
image_install "llvm-symbolizer"
tests: add a wrapper for when systemd is built with ASAN
2018-07-03 11:25:53 +08:00
test: split the ASan wrapper into smaller blocks and tidy it up a bit No functional change (hopefully), just making it easier on the eyes.
2023-05-25 21:31:48 +08:00
# Let's add the ASan DSO's path to the dynamic linker's cache. This is pretty
# unnecessary for gcc & libasan, however, for clang this is crucial, as its
# runtime ASan DSO is in a non-standard (library) path.
mkdir -p "${initdir:?}/etc/ld.so.conf.d/"
echo "${ASAN_RT_PATH%/*}" >"${initdir:?}/etc/ld.so.conf.d/asan-path-override.conf"
test: rebuild the ldconfig cache after changing its configuration Follow-up to ba79e8c2cc.
2023-06-07 21:19:44 +08:00
ldconfig -r "$initdir"
test: split the ASan wrapper into smaller blocks and tidy it up a bit No functional change (hopefully), just making it easier on the eyes.
2023-05-25 21:31:48 +08:00
fi
test: introduce a simple environment file for test service
2022-09-24 06:00:36 +08:00
test: split the ASan wrapper into smaller blocks and tidy it up a bit No functional change (hopefully), just making it easier on the eyes.
2023-05-25 21:31:48 +08:00
# Create a simple environment file which can be included by systemd services
# that need it (i.e. services that utilize DynamicUser=true and bash, etc.)
cat >"${initdir:?}/usr/lib/systemd/systemd-asan-env" <<EOF
test: introduce a simple environment file for test service
2022-09-24 06:00:36 +08:00
LD_PRELOAD=$ASAN_RT_PATH
test: split the ASan wrapper into smaller blocks and tidy it up a bit No functional change (hopefully), just making it easier on the eyes.
2023-05-25 21:31:48 +08:00
ASAN_OPTIONS=$default_asan_options
test: disable LSan in the ASan env wrapper This wrapper is used in situations where we don't care about *San reports, we just want to make things work. However, with enabled LSan we might trigger some bogus reports we're definitely not interested in, causing unexpected test fails. Spotted on C8S in TEST-34-DYNAMICUSERMIGRATE: ``` [10654.804162] testsuite-34.sh[56]: + systemctl start testservice-34-check-writable.service Starting testservice-34-check-writable.service... [10655.055969] bash[546]: + set -o pipefail [10655.056127] bash[546]: + declare -a writable_dirs [10655.056234] bash[546]: + readarray -t writable_dirs [10655.060838] bash[548]: ++ find / '(' -path /var/tmp -o -path /tmp -o -path /proc -o -path /dev/mqueue -o -path /dev/shm -o -path /sys/fs/bpf -o -path /dev/.lxc -o -path /sys/devices/system/cpu ')' -prune -o -type d -writable -print [10655.061534] bash[549]: ++ sort -u [10655.688740] bash[547]: ================================================================= [10655.689075] bash[547]: ==547==ERROR: LeakSanitizer: detected memory leaks [10655.689246] bash[547]: Direct leak of 112 byte(s) in 1 object(s) allocated from: [10655.743851] bash[547]: #0 0x7ffff752d364 (/usr/lib64/clang/14.0.0/lib/libclang_rt.asan-powerpc64le.so+0x13d364) (BuildId: 321f4ed1caea6a1a4c37f9272e07275cf16f034d) [10655.744060] bash[547]: #1 0x1000b5d20 in xmalloc (/usr/bin/bash+0xb5d20) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.744224] bash[547]: #2 0x100083338 (/usr/bin/bash+0x83338) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.744393] bash[547]: #3 0x10008847c (/usr/bin/bash+0x8847c) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.744552] bash[547]: #4 0x1000af6ec in redirection_expand (/usr/bin/bash+0xaf6ec) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.744728] bash[547]: #5 0x1000b005c (/usr/bin/bash+0xb005c) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.744886] bash[547]: #6 0x1000b1388 in do_redirections (/usr/bin/bash+0xb1388) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.745051] bash[547]: #7 0x100050484 (/usr/bin/bash+0x50484) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.745208] bash[547]: #8 0x100052160 in execute_command_internal (/usr/bin/bash+0x52160) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.745376] bash[547]: #9 0x100052a10 in execute_command_internal (/usr/bin/bash+0x52a10) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.745536] bash[547]: #10 0x100053e38 in execute_command (/usr/bin/bash+0x53e38) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.745711] bash[547]: #11 0x1000529d8 in execute_command_internal (/usr/bin/bash+0x529d8) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.745870] bash[547]: #12 0x100053e38 in execute_command (/usr/bin/bash+0x53e38) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.746038] bash[547]: #13 0x1000529d8 in execute_command_internal (/usr/bin/bash+0x529d8) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.746198] bash[547]: #14 0x100053e38 in execute_command (/usr/bin/bash+0x53e38) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.746367] bash[547]: #15 0x1000529d8 in execute_command_internal (/usr/bin/bash+0x529d8) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.746548] bash[547]: #16 0x100053e38 in execute_command (/usr/bin/bash+0x53e38) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.746741] bash[547]: #17 0x1000529d8 in execute_command_internal (/usr/bin/bash+0x529d8) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.746897] bash[547]: #18 0x100053e38 in execute_command (/usr/bin/bash+0x53e38) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.747067] bash[547]: #19 0x1000529d8 in execute_command_internal (/usr/bin/bash+0x529d8) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.747227] bash[547]: #20 0x100053e38 in execute_command (/usr/bin/bash+0x53e38) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.747414] bash[547]: #21 0x1000529d8 in execute_command_internal (/usr/bin/bash+0x529d8) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.747573] bash[547]: #22 0x100053e38 in execute_command (/usr/bin/bash+0x53e38) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.747741] bash[547]: #23 0x1000529d8 in execute_command_internal (/usr/bin/bash+0x529d8) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.747896] bash[547]: #24 0x100053e38 in execute_command (/usr/bin/bash+0x53e38) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.748064] bash[547]: #25 0x1000529d8 in execute_command_internal (/usr/bin/bash+0x529d8) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.748225] bash[547]: #26 0x100053e38 in execute_command (/usr/bin/bash+0x53e38) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.748390] bash[547]: #27 0x1000529d8 in execute_command_internal (/usr/bin/bash+0x529d8) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.748553] bash[547]: #28 0x1000bf91c in parse_and_execute (/usr/bin/bash+0xbf91c) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.748717] bash[547]: #29 0x1000311ec (/usr/bin/bash+0x311ec) (BuildId: da38eb38f6870bdc2a6ef51c52aa6ce20921fe40) [10655.748883] bash[547]: Direct leak of 17 byte(s) in 1 object(s) allocated from: ... ```
2022-10-12 23:28:47 +08:00
LSAN_OPTIONS=detect_leaks=0
test: split the ASan wrapper into smaller blocks and tidy it up a bit No functional change (hopefully), just making it easier on the eyes.
2023-05-25 21:31:48 +08:00
UBSAN_OPTIONS=$default_ubsan_options
EOF
tests: add a wrapper for when systemd is built with ASAN
2018-07-03 11:25:53 +08:00
test: split the ASan wrapper into smaller blocks and tidy it up a bit No functional change (hopefully), just making it easier on the eyes.
2023-05-25 21:31:48 +08:00
default_environment=(
"ASAN_OPTIONS='$default_asan_options'"
"UBSAN_OPTIONS='$default_ubsan_options'"
"ASAN_RT_PATH='$ASAN_RT_PATH'"
)
test: make sure we get PID1's stack trace on ASan/UBSan errors As hitting an ASan/UBSan error in PID1 results in a crash (and a kernel panic when running under qemu), we usually lose the stack trace which makes debugging quite painful. Let's mitigate this by forwarding the stack trace to multiple places - namely to a file and the syslog.
2023-06-23 20:28:30 +08:00
manager_environment=(
"ASAN_OPTIONS='$default_asan_options:log_path=/systemd-pid1.asan.log:log_to_syslog=1'"
"UBSAN_OPTIONS='$default_ubsan_options:log_path=/systemd-pid1.ubsan.log:log_to_syslog=1'"
"ASAN_RT_PATH='$ASAN_RT_PATH'"
)
test: tidy up the ASan-related stuff
2021-03-07 05:21:30 +08:00
test: split the ASan wrapper into smaller blocks and tidy it up a bit No functional change (hopefully), just making it easier on the eyes.
2023-05-25 21:31:48 +08:00
mkdir -p "${initdir:?}/etc/systemd/system.conf.d/"
cat >"${initdir:?}/etc/systemd/system.conf.d/asan.conf" <<EOF
[Manager]
DefaultEnvironment=${default_environment[*]}
test: make sure we get PID1's stack trace on ASan/UBSan errors As hitting an ASan/UBSan error in PID1 results in a crash (and a kernel panic when running under qemu), we usually lose the stack trace which makes debugging quite painful. Let's mitigate this by forwarding the stack trace to multiple places - namely to a file and the syslog.
2023-06-23 20:28:30 +08:00
ManagerEnvironment=${manager_environment[*]}
test: split the ASan wrapper into smaller blocks and tidy it up a bit No functional change (hopefully), just making it easier on the eyes.
2023-05-25 21:31:48 +08:00
DefaultTimeoutStartSec=180s
EOF
tests: add a wrapper for when systemd is built with ASAN
2018-07-03 11:25:53 +08:00
test: split the ASan wrapper into smaller blocks and tidy it up a bit No functional change (hopefully), just making it easier on the eyes.
2023-05-25 21:31:48 +08:00
# ASAN and syscall filters aren't compatible with each other.
find "${initdir:?}" -name '*.service' -type f -print0 | xargs -0 sed -i 's/^\(MemoryDeny\|SystemCall\)/#\1/'
tests: add a wrapper for when systemd is built with ASAN
2018-07-03 11:25:53 +08:00
test: split the ASan wrapper into smaller blocks and tidy it up a bit No functional change (hopefully), just making it easier on the eyes.
2023-05-25 21:31:48 +08:00
mkdir -p "${initdir:?}/etc/systemd/system/systemd-journald.service.d/"
cat >"${initdir:?}/etc/systemd/system/systemd-journald.service.d/asan-env.conf" <<EOF
[Service]
tests: redirect ASAN reports on journald to a file Otherwise, they will end up in /dev/null.
2018-07-04 03:29:42 +08:00
# The redirection of ASAN reports to a file prevents them from ending up in /dev/null.
# But, apparently, sometimes it doesn't work: https://github.com/google/sanitizers/issues/886.
test: split the ASan wrapper into smaller blocks and tidy it up a bit No functional change (hopefully), just making it easier on the eyes.
2023-05-25 21:31:48 +08:00
Environment=ASAN_OPTIONS=$default_asan_options:log_path=/systemd-journald.asan.log UBSAN_OPTIONS=$default_ubsan_options:log_path=/systemd-journald.ubsan.log
tests: redirect ASAN reports on journald to a file Otherwise, they will end up in /dev/null.
2018-07-04 03:29:42 +08:00
tests: redirect the stdout/stderr of journald to a file (under ASan+UBSan) Sometimes UBSan sends its reports to stderr regardless of what is specified in log_path Let's try to catch them by redirecting stderr (and stdout just in case) to a file See https://github.com/systemd/systemd/pull/12524#issuecomment-491108821
2019-05-10 08:47:03 +08:00
# Sometimes UBSan sends its reports to stderr regardless of what is specified in log_path
# Let's try to catch them by redirecting stderr (and stdout just in case) to a file
# See https://github.com/systemd/systemd/pull/12524#issuecomment-491108821
test: split the ASan wrapper into smaller blocks and tidy it up a bit No functional change (hopefully), just making it easier on the eyes.
2023-05-25 21:31:48 +08:00
StandardOutput=file:/systemd-journald.out
EOF
tests: redirect the stdout/stderr of journald to a file (under ASan+UBSan) Sometimes UBSan sends its reports to stderr regardless of what is specified in log_path Let's try to catch them by redirecting stderr (and stdout just in case) to a file See https://github.com/systemd/systemd/pull/12524#issuecomment-491108821
2019-05-10 08:47:03 +08:00
test: split the ASan wrapper into smaller blocks and tidy it up a bit No functional change (hopefully), just making it easier on the eyes.
2023-05-25 21:31:48 +08:00
# 90s isn't enough for some services to finish when literally everything is run
# under ASan+UBSan in containers, which, in turn, are run in VMs.
# Let's limit which environments such services should be executed in.
mkdir -p "${initdir:?}/etc/systemd/system/systemd-hwdb-update.service.d/"
cat >"${initdir:?}/etc/systemd/system/systemd-hwdb-update.service.d/asan.conf" <<EOF
[Unit]
ConditionVirtualization=container
tests: overwrite a hard-coded timeout in systemd-hwdb-update.service
2018-12-24 08:15:43 +08:00
test: split the ASan wrapper into smaller blocks and tidy it up a bit No functional change (hopefully), just making it easier on the eyes.
2023-05-25 21:31:48 +08:00
[Service]
TimeoutSec=240s
EOF
tests: override a hard-coded timeout that kicks in too early (under ASan+UBSan) The test has failed two times today: https://travis-ci.org/systemd/systemd/jobs/530750626 and https://travis-ci.org/systemd/systemd/jobs/530759465.
2019-05-11 00:33:35 +08:00
test: split the ASan wrapper into smaller blocks and tidy it up a bit No functional change (hopefully), just making it easier on the eyes.
2023-05-25 21:31:48 +08:00
# Let's override another hard-coded timeout that kicks in too early
mkdir -p "${initdir:?}/etc/systemd/system/systemd-journal-flush.service.d/"
cat >"${initdir:?}/etc/systemd/system/systemd-journal-flush.service.d/asan.conf" <<EOF
[Service]
TimeoutSec=180s
tests: add a wrapper for when systemd is built with ASAN
2018-07-03 11:25:53 +08:00
EOF
test: split the ASan wrapper into smaller blocks and tidy it up a bit No functional change (hopefully), just making it easier on the eyes.
2023-05-25 21:31:48 +08:00
asan_wrapper="${initdir:?}/${PATH_TO_INIT:?}"
# Sanity check to make sure we don't overwrite something we shouldn't.
[[ "$asan_wrapper" =~ systemd-under-asan$ ]]
cat >"$asan_wrapper" <<EOF
#!/usr/bin/env bash
set -eux
test: set $PATH to something predictable in the ASan wrapper This is still necessary when running with clang's ASan, as it uses an external binary to symbolize the stack traces. Follow-up to ba79e8c2ccd.
2023-06-15 19:11:24 +08:00
export PATH="/sbin:/bin:/usr/sbin:/usr/bin"
test: make sure we get PID1's stack trace on ASan/UBSan errors As hitting an ASan/UBSan error in PID1 results in a crash (and a kernel panic when running under qemu), we usually lose the stack trace which makes debugging quite painful. Let's mitigate this by forwarding the stack trace to multiple places - namely to a file and the syslog.
2023-06-23 20:28:30 +08:00
export ${manager_environment[@]}
test: split the ASan wrapper into smaller blocks and tidy it up a bit No functional change (hopefully), just making it easier on the eyes.
2023-05-25 21:31:48 +08:00
[[ -n "\$ASAN_OPTIONS" && -n "\$UBSAN_OPTIONS" ]]
test: set $PATH to something predictable in the ASan wrapper This is still necessary when running with clang's ASan, as it uses an external binary to symbolize the stack traces. Follow-up to ba79e8c2ccd.
2023-06-15 19:11:24 +08:00
test: split the ASan wrapper into smaller blocks and tidy it up a bit No functional change (hopefully), just making it easier on the eyes.
2023-05-25 21:31:48 +08:00
exec "$ROOTLIBDIR/systemd" "\$@"
EOF
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
chmod 0755 "$asan_wrapper"
tests: add a wrapper for when systemd is built with ASAN
2018-07-03 11:25:53 +08:00
}
tests: add create_strace_wrapper Very handy for early-boot debugging See https://github.com/systemd/systemd/pull/2781#discussion_r54782628
2016-03-04 09:55:46 +08:00
create_strace_wrapper() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local strace_wrapper="$initdir/$ROOTLIBDIR/systemd-under-strace"
ddebug "Create $strace_wrapper"
cat >"$strace_wrapper" <<EOF
treewide: more portable bash shebangs As in 2a5fcfae024ffc370bb780572279f45a1da3f946 and in 3e67e5c9928f8b1e1c5a63def88d53ed1fed12eb using /usr/bin/env allows bash to be looked up in PATH rather than being hard-coded. As with the previous changes the same arguments apply - distributions have scripts to rewrite shebangs on installation and they know what locations to rely on. - For tests/compilation we should rather rely on the user to have setup there PATH correctly. In particular this makes testing from git easier on NixOS where do not provide /bin/bash to improve compose-ability.
2020-03-04 17:35:06 +08:00
#!/usr/bin/env bash
tests: add create_strace_wrapper Very handy for early-boot debugging See https://github.com/systemd/systemd/pull/2781#discussion_r54782628
2016-03-04 09:55:46 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
exec strace -f -D -o /strace.out "$ROOTLIBDIR/systemd" "\$@"
tests: add create_strace_wrapper Very handy for early-boot debugging See https://github.com/systemd/systemd/pull/2781#discussion_r54782628
2016-03-04 09:55:46 +08:00
EOF
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
chmod 0755 "$strace_wrapper"
tests: add create_strace_wrapper Very handy for early-boot debugging See https://github.com/systemd/systemd/pull/2781#discussion_r54782628
2016-03-04 09:55:46 +08:00
}
tests: install mkfs.ext4, mkfs.vfat and modules into the test image This allows TEST-58-REPART to at least start. It fails later with with loopback device errors.
2021-04-21 15:58:26 +08:00
install_fs_tools() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install fsck"
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
image_install /sbin/fsck*
image_install -o /bin/fsck*
basic: fallback to fstatat if entry->d_type is DT_UNKNOWN * tests-functions: improve FSTYPE-support make clean setup FSTYPE=reiserfs is working fine now :) * basic: fallback to fstatat if entry->d_type is DT_UNKNOWN Fixes localectl on reiserfs: -bash-4.3# mkdir -p /usr/lib/locale -bash-4.3# stat -f /usr/lib/locale/ File: "/usr/lib/locale/" ID: bdb0322715b5366e Namelen: 255 Type: reiserfs Block size: 4096 Blocks: Total: 99835 Free: 60262 Available: 60262 Inodes: Total: 0 Free: 0 -bash-4.3# mkdir /usr/lib/locale/HeyHo -bash-4.3# localectl list-locales --no-pager -bash-4.3# mount -t tmpfs tmpfs /usr/lib/locale -bash-4.3# mkdir /usr/lib/locale/HeyHo -bash-4.3# localectl list-locales --no-pager HeyHo
2016-04-04 19:07:56 +08:00
# fskc.reiserfs calls reiserfsck. so, install it
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
image_install -o reiserfsck
tests: install mkfs.ext4, mkfs.vfat and modules into the test image This allows TEST-58-REPART to at least start. It fails later with with loopback device errors.
2021-04-21 15:58:26 +08:00
# we use mkfs in system-repart tests
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
image_install /sbin/mkfs.ext4
image_install /sbin/mkfs.vfat
tests: install mkfs.ext4, mkfs.vfat and modules into the test image This allows TEST-58-REPART to at least start. It fails later with with loopback device errors.
2021-04-21 15:58:26 +08:00
}
install_modules() {
dinfo "Install modules"
test: always try to install the ext4 module So the tests work even if the base image filesystem is not ext4.
2024-02-14 18:24:05 +08:00
instmods bridge dummy ext4 ipvlan macvlan vfat veth
test: install the overlayfs/loop modules unconditionally
2023-05-14 19:13:24 +08:00
instmods loop =block
test: install nls modules, required by vfat kernel: FAT-fs (loop0p2): codepage cp437 not found kernel: FAT-fs (loop0p2): IO charset ascii not found
2021-06-18 23:10:30 +08:00
instmods nls_ascii =nls
test: install the overlayfs/loop modules unconditionally
2023-05-14 19:13:24 +08:00
instmods overlay =overlayfs
test: install test specific modules in test_append_files()
2023-05-20 04:01:05 +08:00
instmods scsi_debug
tests: install mkfs.ext4, mkfs.vfat and modules into the test image This allows TEST-58-REPART to at least start. It fails later with with loopback device errors.
2021-04-21 15:58:26 +08:00
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$LOOKS_LIKE_SUSE"; then
test: always try to install the ext4 module So the tests work even if the base image filesystem is not ext4.
2024-02-14 18:24:05 +08:00
instmods af_packet
tests: install mkfs.ext4, mkfs.vfat and modules into the test image This allows TEST-58-REPART to at least start. It fails later with with loopback device errors.
2021-04-21 15:58:26 +08:00
fi
tests: install fsck* systemd-fsck depends on /sbin/fsck*
2016-01-16 16:06:59 +08:00
}
test: remove duplicated code
2013-11-06 06:32:56 +08:00
install_dmevent() {
instmods dm_crypt =crypto
test/test-functions: fix install_dmeventd to correctly install bin/libs
2019-07-14 11:03:26 +08:00
inst_binary dmeventd
test: install dmevent service and socket
2021-09-14 00:39:00 +08:00
image_install "${ROOTLIBDIR:?}"/system/dm-event.{service,socket}
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$LOOKS_LIKE_DEBIAN"; then
tests: fix TEST-02-CRYPTSETUP on Debian/Ubuntu
2016-01-17 11:32:37 +08:00
# dmsetup installs 55-dm and 60-persistent-storage-dm on Debian/Ubuntu
test/test-functions: Debian/Ubuntu, now ship 95-dm-notify.rules, copy it This fixes cryptsetup tests on recent Ubuntu/Debian systems (current development series). Bug-Launchpad: https://launchpad.net/bugs/1749432
2018-02-15 04:34:55 +08:00
# and since buster/bionic 95-dm-notify.rules
# see https://gitlab.com/debian-lvm/lvm2/blob/master/debian/patches/udev.patch
inst_rules 55-dm.rules 60-persistent-storage-dm.rules 95-dm-notify.rules
tests: fix TEST-02-CRYPTSETUP on Debian/Ubuntu
2016-01-17 11:32:37 +08:00
else
inst_rules 10-dm.rules 13-dm-disk.rules 95-dm-notify.rules
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$LOOKS_LIKE_SUSE"; then
test: adapt test-functions for SUSE
2020-07-22 04:14:53 +08:00
inst_rules 60-persistent-storage.rules 61-persistent-storage-compat.rules 99-systemd.rules
fi
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
test: add a multipath helper
2021-09-09 00:17:46 +08:00
install_multipath() {
instmods "=md" multipath
image_install kpartx /lib/udev/kpartx_id lsmod mpathpersist multipath multipathd partx
image_install "${ROOTLIBDIR:?}"/system/multipathd.{service,socket}
if get_bool "$LOOKS_LIKE_DEBIAN"; then
test: install correct kpartx udev rules on Ubuntu Follow-up for 519f0074cf.
2024-01-05 03:28:37 +08:00
# Note: try both 60-kpartx.rules (as seen on Debian Sid with 0.9.4-7) and 95-kpartx.rules (as seen on
test: install correct kpartx udev rules on Debian Resolves: #30703
2024-01-04 00:24:03 +08:00
# Ubuntu Jammy with 0.8.8-1ubuntu1.22.04.4)
test: install correct kpartx udev rules on Ubuntu Follow-up for 519f0074cf.
2024-01-05 03:28:37 +08:00
inst_rules 56-dm-parts.rules 56-dm-mpath.rules 60-kpartx.rules 60-multipath.rules 68-del-part-nodes.rules 95-kpartx.rules
test: add a multipath helper
2021-09-09 00:17:46 +08:00
else
inst_rules 11-dm-mpath.rules 11-dm-parts.rules 62-multipath.rules 66-kpartx.rules 68-del-part-nodes.rules
fi
mkdir -p "${initdir:?}/etc/multipath"
local file
while read -r file; do
# Install libraries required by the given library
inst_libs "$file"
# Install the library itself and create necessary symlinks
inst_library "$file"
done < <(find /lib*/multipath -type f)
}
test: add an LVM helper
2021-09-14 00:39:22 +08:00
install_lvm() {
test: install all necessary units & generators for LVM on Debian And derivates. Replaces: #30458
2023-12-13 19:27:17 +08:00
local lvm_rules rule_prefix
test: add an LVM helper
2021-09-14 00:39:22 +08:00
image_install lvm
image_install "${ROOTLIBDIR:?}"/system/lvm2-lvmpolld.{service,socket}
test: support the new lvm udev autoactivation Introduced in lvm 2.03.14. See: https://sourceware.org/git/?p=lvm2.git;a=commit;h=67722b312390cdab29c076c912e14bd739c5c0f6
2021-11-02 03:55:38 +08:00
image_install "${ROOTLIBDIR:?}"/system/{blk-availability,lvm2-monitor}.service
test: add an LVM helper
2021-09-14 00:39:22 +08:00
image_install -o "/lib/tmpfiles.d/lvm2.conf"
test: install all necessary units & generators for LVM on Debian And derivates. Replaces: #30458
2023-12-13 19:27:17 +08:00
test: add an LVM helper
2021-09-14 00:39:22 +08:00
if get_bool "$LOOKS_LIKE_DEBIAN"; then
test: install all necessary units & generators for LVM on Debian And derivates. Replaces: #30458
2023-12-13 19:27:17 +08:00
lvm_rules="56-lvm.rules"
rule_prefix=""
test: add an LVM helper
2021-09-14 00:39:22 +08:00
else
test: install all necessary units & generators for LVM on Debian And derivates. Replaces: #30458
2023-12-13 19:27:17 +08:00
lvm_rules="11-dm-lvm.rules"
rule_prefix="dm-"
test: add an LVM helper
2021-09-14 00:39:22 +08:00
fi
test: install all necessary units & generators for LVM on Debian And derivates. Replaces: #30458
2023-12-13 19:27:17 +08:00
# Support the new udev autoactivation introduced in lvm 2.03.14
# https://sourceware.org/git/?p=lvm2.git;a=commit;h=67722b312390cdab29c076c912e14bd739c5c0f6
# Static autoactivation (via lvm2-activation-generator) was dropped
# in lvm 2.03.15
# https://sourceware.org/git/?p=lvm2.git;a=commit;h=ee8fb0310c53ed003a43b324c99cdfd891dd1a7c
if [[ -f "/lib/udev/rules.d/69-${rule_prefix}lvm.rules" ]]; then
inst_rules "$lvm_rules" "69-${rule_prefix}lvm.rules"
else
image_install "${ROOTLIBDIR:?}"/system-generators/lvm2-activation-generator
image_install "${ROOTLIBDIR:?}"/system/lvm2-pvscan@.service
inst_rules "$lvm_rules" "69-${rule_prefix}lvm-metad.rules"
fi
test: add an LVM helper
2021-09-14 00:39:22 +08:00
mkdir -p "${initdir:?}/etc/lvm"
}
test: split out host_has_{btrfs,mdadm}() from TEST-64-UDEV-STORAGE
2023-12-06 12:29:36 +08:00
host_has_btrfs() (
set -e
modprobe -nv btrfs && command -v mkfs.btrfs && command -v btrfs || return $?
)
test: add a btrfs helper
2021-09-18 01:19:23 +08:00
install_btrfs() {
instmods btrfs
# Not all utilities provided by btrfs-progs are listed here; extend the list
# if necessary
image_install btrfs btrfstune mkfs.btrfs
inst_rules 64-btrfs-dm.rules
}
test: add an iSCSI helper
2021-09-22 03:32:30 +08:00
install_iscsi() {
# Install both client and server side stuff by default
local inst="${1:-}"
local file
# Install client-side stuff ("initiator" in iSCSI jargon) - Open-iSCSI in this case
# (open-iscsi on Debian, iscsi-initiator-utils on Fedora, etc.)
if [[ -z "$inst" || "$inst" =~ (client|initiator) ]]; then
image_install iscsi-iname iscsiadm iscsid iscsistart
image_install -o "${ROOTLIBDIR:?}"/system/iscsi-{init,onboot,shutdown}.service
image_install "${ROOTLIBDIR:?}"/system/iscsid.{service,socket}
image_install "${ROOTLIBDIR:?}"/system/iscsi.service
mkdir -p "${initdir:?}"/var/lib/iscsi/{ifaces,isns,nodes,send_targets,slp,static}
mkdir -p "${initdir:?}/etc/iscsi"
echo "iscsid.startup = /bin/systemctl start iscsid.socket" >"${initdir:?}/etc/iscsi/iscsid.conf"
test: support open-iscsi >= 2.1.2 Since open-iscsi 2.1.2 [0] the initiator name should be generated via a one-time service instead of distro package's post-install scripts. However, some distros still use this approach even after this patch, so prefer the already existing initiatorname.iscsi file if it exists. [0] https://github.com/open-iscsi/open-iscsi/commit/f37d5b653f9f251845db3f29b1a3dcb90ec89731
2022-09-27 05:14:18 +08:00
# Since open-iscsi 2.1.2 [0] the initiator name should be generated via
# a one-time service instead of distro package's post-install scripts.
# However, some distros still use this approach even after this patch,
# so prefer the already existing initiatorname.iscsi file if it exists.
#
# [0] https://github.com/open-iscsi/open-iscsi/commit/f37d5b653f9f251845db3f29b1a3dcb90ec89731
if [[ ! -e /etc/iscsi/initiatorname.iscsi ]]; then
image_install "${ROOTLIBDIR:?}"/system/iscsi-init.service
test: pre-load ASan's DSO for iscsi-init.service The iscsi-init.service calls `sh` which might, in certain circumstances, pull in instrumented systemd NSS modules causing `sh` to fail. Let's mitigate this by pulling in an env file crafted by `create_asan_wrapper()` that (among others) pre-loads ASan's DSO.
2022-10-03 04:51:56 +08:00
if get_bool "$IS_BUILT_WITH_ASAN"; then
# The iscsi-init.service calls `sh` which might, in certain circumstances,
# pull in instrumented systemd NSS modules causing `sh` to fail. Let's mitigate
# this by pulling in an env file crafted by `create_asan_wrapper()` that
# (among others) pre-loads ASan's DSO.
mkdir -p "${initdir:?}/etc/systemd/system/iscsi-init.service.d/"
printf "[Service]\nEnvironmentFile=/usr/lib/systemd/systemd-asan-env" >"${initdir:?}/etc/systemd/system/iscsi-init.service.d/asan-env.conf"
fi
test: support open-iscsi >= 2.1.2 Since open-iscsi 2.1.2 [0] the initiator name should be generated via a one-time service instead of distro package's post-install scripts. However, some distros still use this approach even after this patch, so prefer the already existing initiatorname.iscsi file if it exists. [0] https://github.com/open-iscsi/open-iscsi/commit/f37d5b653f9f251845db3f29b1a3dcb90ec89731
2022-09-27 05:14:18 +08:00
else
inst_simple "/etc/iscsi/initiatorname.iscsi"
fi
test: add an iSCSI helper
2021-09-22 03:32:30 +08:00
fi
# Install server-side stuff ("target" in iSCSI jargon) - TGT in this case
# (tgt on Debian, scsi-target-utils on Fedora, etc.)
if [[ -z "$inst" || "$inst" =~ (server|target) ]]; then
image_install tgt-admin tgt-setup-lun tgtadm tgtd tgtimg
image_install -o /etc/sysconfig/tgtd
image_install "${ROOTLIBDIR:?}"/system/tgtd.service
mkdir -p "${initdir:?}/etc/tgt"
touch "${initdir:?}"/etc/tgt/{tgtd,targets}.conf
# Install perl modules required by tgt-admin
#
# Forgive me father for I have sinned. The monstrosity below appends
# a perl snippet to the `tgt-admin` perl script on the fly, which
# dumps a list of files (perl modules) required by `tgt-admin` at
# the runtime plus any DSOs loaded via DynaLoader. This list is then
# passed to `inst_simple` which installs the necessary files into the image
test: shellcheck-ify test scripts
2021-09-30 02:30:08 +08:00
#
# shellcheck disable=SC2016
test: add an iSCSI helper
2021-09-22 03:32:30 +08:00
while read -r file; do
inst_simple "$file"
test: shellcheck-ify test scripts
2021-09-30 02:30:08 +08:00
done < <(perl -- <(cat "$(command -v tgt-admin)" <(echo -e 'use DynaLoader; print map { "$_\n" } values %INC; print join("\n", @DynaLoader::dl_shared_objects)')) -p | awk '/^\// { print $1 }')
test: add an iSCSI helper
2021-09-22 03:32:30 +08:00
fi
}
test: split out host_has_{btrfs,mdadm}() from TEST-64-UDEV-STORAGE
2023-12-06 12:29:36 +08:00
host_has_mdadm() (
set -e
command -v mdadm || return $?
)
test: add MD-related tests to TEST-64
2022-04-08 00:03:17 +08:00
install_mdadm() {
local unit
local mdadm_units=(
system/mdadm-grow-continue@.service
system/mdadm-last-resort@.service
system/mdadm-last-resort@.timer
system/mdmon@.service
system/mdmonitor-oneshot.service
system/mdmonitor-oneshot.timer
system/mdmonitor.service
system-shutdown/mdadm.shutdown
)
test: make install_mdadm() also install relevant kernel modules Installing mdadm without kernel modules is mostly meaningless.
2023-12-06 12:32:15 +08:00
instmods "=md"
test: add MD-related tests to TEST-64
2022-04-08 00:03:17 +08:00
image_install mdadm mdmon
inst_rules 01-md-raid-creating.rules 63-md-raid-arrays.rules 64-md-raid-assembly.rules 69-md-clustered-confirm-device.rules
# Fedora/CentOS/RHEL ships this rule file
[[ -f /lib/udev/rules.d/65-md-incremental.rules ]] && inst_rules 65-md-incremental.rules
for unit in "${mdadm_units[@]}"; do
image_install "${ROOTLIBDIR:?}/$unit"
done
test: mask mdmonitor when building image Follow-up for 22e31655f3f9f54d932d4f48b92b36698e701729.
2023-12-13 13:28:00 +08:00
# Disable the mdmonitor service, since it fails if there's no valid email address
# configured in /etc/mdadm.conf, which just unnecessarily pollutes the logs
"${SYSTEMCTL:?}" mask --root "${initdir:?}" mdmonitor.service || :
test: add MD-related tests to TEST-64
2022-04-08 00:03:17 +08:00
}
test/test-functions: allow installing systemd files from local system Add NO_BUILD var to allow testing with no local build, by installing local systemd files into the image. This only works for debian-like distros currently, that use the tools 'apt' and 'dpkg' for package management.
2020-11-18 06:20:22 +08:00
install_compiled_systemd() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install compiled systemd"
test/test-functions: do not execute strip_binaries twice It is called from setup_basic_environment(), which also calls install_symtemd() a bit earlier, so in effect it would be called twice.
2020-08-27 19:10:30 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local ninja_bin
ninja_bin="$(type -P ninja || type -P ninja-build)"
if [[ -z "$ninja_bin" ]]; then
tests: use ninja-build if ninja is not available (#6544) This makes the tests work on CentOS, which currently has ninja-build only.
2017-08-07 17:06:07 +08:00
dfatal "ninja was not found"
exit 1
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
(set -x; DESTDIR="$initdir" "$ninja_bin" -C "$BUILD_DIR" install)
test: create and merge code coverage reports in integration tests If -Db_coverage=true is used at build time, then ARTIFACT_DIRECTORY/TEST-XX-FOO.coverage-info files are created with code coverage data, and run-integration-test.sh also merges them into ARTIFACT_DIRECTORY/merged.coverage-info since the coveralls.io helpers accept only a single file.
2021-10-03 23:50:38 +08:00
# If we are doing coverage runs, copy over the binary notes files, as lcov expects to
# find them in the same directory as the runtime data counts
test: loosen sandbox restrictions for integration tests as well Otherwise we miss quite a lot of coverage (mainly from logind, hostnamed, networkd, and possibly others), since they can't write their reports with `ProtectSystem=strict`.
2021-10-22 00:06:36 +08:00
if get_bool "$IS_BUILT_WITH_COVERAGE"; then
test: create and merge code coverage reports in integration tests If -Db_coverage=true is used at build time, then ARTIFACT_DIRECTORY/TEST-XX-FOO.coverage-info files are created with code coverage data, and run-integration-test.sh also merges them into ARTIFACT_DIRECTORY/merged.coverage-info since the coveralls.io helpers accept only a single file.
2021-10-03 23:50:38 +08:00
mkdir -p "${initdir}/${BUILD_DIR:?}/"
rsync -am --include='*/' --include='*.gcno' --exclude='*' "${BUILD_DIR:?}/" "${initdir}/${BUILD_DIR:?}/"
test: set ACLs for the build dir when built with coverage Otherwise unprivileged processes (like systemd-resolved) can't write coverage data.
2022-02-14 18:58:54 +08:00
# Set effective & default ACLs for the build dir so unprivileged
# processes can write gcda files with coverage stats
setfacl -R -m 'd:o:rwX' -m 'o:rwX' "${initdir}/${BUILD_DIR:?}/"
test: create and merge code coverage reports in integration tests If -Db_coverage=true is used at build time, then ARTIFACT_DIRECTORY/TEST-XX-FOO.coverage-info files are created with code coverage data, and run-integration-test.sh also merges them into ARTIFACT_DIRECTORY/merged.coverage-info since the coveralls.io helpers accept only a single file.
2021-10-03 23:50:38 +08:00
fi
test/test-functions: allow installing systemd files from local system Add NO_BUILD var to allow testing with no local build, by installing local systemd files into the image. This only works for debian-like distros currently, that use the tools 'apt' and 'dpkg' for package management.
2020-11-18 06:20:22 +08:00
}
test: install empty directories with NO_BUILD=1 Resolves: #30478
2023-12-14 22:06:12 +08:00
install_package_file() {
local file="${1:?}"
# Skip missing files (like /etc/machine-info)
[[ ! -e "$file" ]] && return 0
# Skip python unit tests, since the image_install machinery will try to pull
# in the whole python stack in a very questionable state, making the tests fail.
# And given we're trying to transition to mkosi-based images anyway I'm not even
# going to bother
[[ "$file" =~ /tests/unit-tests/.*.py$ ]] && return 0
# If the current file is a directory, create it with the original
# mode; if it's a symlink to a directory, copy it as-is
if [[ -d "$file" ]]; then
inst_dir "$file"
else
inst "$file"
fi
}
test/test-functions: allow installing systemd files from local system Add NO_BUILD var to allow testing with no local build, by installing local systemd files into the image. This only works for debian-like distros currently, that use the tools 'apt' and 'dpkg' for package management.
2020-11-18 06:20:22 +08:00
install_debian_systemd() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install debian systemd"
test/test-functions: allow installing systemd files from local system Add NO_BUILD var to allow testing with no local build, by installing local systemd files into the image. This only works for debian-like distros currently, that use the tools 'apt' and 'dpkg' for package management.
2020-11-18 06:20:22 +08:00
test: install empty directories with NO_BUILD=1 Resolves: #30478
2023-12-14 22:06:12 +08:00
local deb file
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
while read -r deb; do
test/test-functions: allow installing systemd files from local system Add NO_BUILD var to allow testing with no local build, by installing local systemd files into the image. This only works for debian-like distros currently, that use the tools 'apt' and 'dpkg' for package management.
2020-11-18 06:20:22 +08:00
ddebug "Install debian files from package $deb"
test: install empty directories with NO_BUILD=1 Resolves: #30478
2023-12-14 22:06:12 +08:00
while read -r file; do
install_package_file "$file"
done < <(dpkg-query -L "$deb" 2>/dev/null)
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
done < <(grep -E '^Package:' "${SOURCE_DIR}/debian/control" | cut -d ':' -f 2)
test/test-functions: allow installing systemd files from local system Add NO_BUILD var to allow testing with no local build, by installing local systemd files into the image. This only works for debian-like distros currently, that use the tools 'apt' and 'dpkg' for package management.
2020-11-18 06:20:22 +08:00
}
test: support NO_BUILD=yes on Fedora
2023-11-20 19:33:58 +08:00
install_rpm() {
local rpm="${1:?}"
local file
if ! rpm -q "$rpm" >/dev/null; then
derror "RPM $rpm is not installed"
return 1
fi
dinfo "Installing contents of RPM $rpm"
while read -r file; do
test: install empty directories with NO_BUILD=1 Resolves: #30478
2023-12-14 22:06:12 +08:00
install_package_file "$file"
test: support NO_BUILD=yes on Fedora
2023-11-20 19:33:58 +08:00
done < <(rpm -ql "$rpm")
}
test: add support for NO_BUILD=1 on openSUSE
2021-08-03 14:18:13 +08:00
install_suse_systemd() {
local pkgs
test: make sure to install the filesystem package in the test image on SUSE Othewise test images are missing the tmpfiles snippets used to create the very basic files at boot, which can be useful when a test wants to reuse the OS tree (is already running in) for spawning a new container in pristine state.
2024-02-14 17:01:57 +08:00
dinfo "Install basic filesystem structure"
install_rpm filesystem
test: add support for NO_BUILD=1 on openSUSE
2021-08-03 14:18:13 +08:00
dinfo "Install SUSE systemd"
pkgs=(
systemd
test: install systemd-boot in openSUSE test images Needed since 87282a337d1ba7dc7d755f53b46c64b43718dcf7.
2024-02-14 01:12:42 +08:00
systemd-boot
test: add support for NO_BUILD=1 on openSUSE
2021-08-03 14:18:13 +08:00
systemd-container
systemd-coredump
systemd-experimental
test: install systemd-homed for openSUSE This new sub-package has been recently introduced.
2023-08-02 01:40:34 +08:00
systemd-homed
test: add support for NO_BUILD=1 on openSUSE
2021-08-03 14:18:13 +08:00
systemd-journal-remote
tests: install systemd-resolved on openSUSE It's needed since systemd-resolved has its own test suite (commit fb6f25d7b979134adf57).
2022-11-08 23:01:59 +08:00
# Since commit fb6f25d7b979134a, systemd-resolved, which is shipped by
# systemd-network sub-package on openSUSE, has its own testsuite.
systemd-network
test: add support for NO_BUILD=1 on openSUSE
2021-08-03 14:18:13 +08:00
systemd-portable
udev
)
for p in "${pkgs[@]}"; do
rpm -q "$p" &>/dev/null || continue
test: support NO_BUILD=yes on Fedora
2023-11-20 19:33:58 +08:00
install_rpm "$p"
test: add support for NO_BUILD=1 on openSUSE
2021-08-03 14:18:13 +08:00
done
test: on openSUSE install the collection of unit test binaries in the target only for TEST-02-UNITTESTS
2023-03-21 01:23:33 +08:00
dinfo "Install the data needed by the tests at runtime"
tests: update install_suse_systemd() - Use inst_recursive() and image_install() helpers where appropriate - Update comments to explain why we need to install the test data manually in $initdir - Install manual/ in $initdir as TEST-35-LOGIN relies on manual/test-session-properties
2022-11-18 17:18:19 +08:00
inst_recursive "${SOURCE_DIR}/testdata"
test: testsuite-35.sh needs manual/test-session-properties to be installed on SUSE
2023-08-08 17:40:36 +08:00
inst_recursive "${SOURCE_DIR}/unit-tests/manual"
tests: update install_suse_systemd() - Use inst_recursive() and image_install() helpers where appropriate - Update comments to explain why we need to install the test data manually in $initdir - Install manual/ in $initdir as TEST-35-LOGIN relies on manual/test-session-properties
2022-11-18 17:18:19 +08:00
# On openSUSE, this directory is not created at package install, at least
# for now.
test: add support for NO_BUILD=1 on openSUSE
2021-08-03 14:18:13 +08:00
mkdir -p "$initdir/var/log/journal/remote"
}
test: support NO_BUILD=yes on Fedora
2023-11-20 19:33:58 +08:00
install_fedora_systemd() {
local required_packages=(
systemd
systemd-container
systemd-libs
systemd-pam
systemd-tests
systemd-udev
)
local optional_packages=(
systemd-boot-unsigned
systemd-bootchart
systemd-journal-remote
systemd-networkd
systemd-oomd-defaults
systemd-resolved
)
local package
for package in "${required_packages[@]}"; do
install_rpm "$package"
done
for package in "${optional_packages[@]}"; do
rpm -q "$package" >/dev/null || continue
install_rpm "$package"
done
}
test/test-functions: allow installing systemd files from local system Add NO_BUILD var to allow testing with no local build, by installing local systemd files into the image. This only works for debian-like distros currently, that use the tools 'apt' and 'dpkg' for package management.
2020-11-18 06:20:22 +08:00
install_distro_systemd() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install distro systemd"
test/test-functions: allow installing systemd files from local system Add NO_BUILD var to allow testing with no local build, by installing local systemd files into the image. This only works for debian-like distros currently, that use the tools 'apt' and 'dpkg' for package management.
2020-11-18 06:20:22 +08:00
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$LOOKS_LIKE_DEBIAN"; then
test/test-functions: allow installing systemd files from local system Add NO_BUILD var to allow testing with no local build, by installing local systemd files into the image. This only works for debian-like distros currently, that use the tools 'apt' and 'dpkg' for package management.
2020-11-18 06:20:22 +08:00
install_debian_systemd
test: add support for NO_BUILD=1 on openSUSE
2021-08-03 14:18:13 +08:00
elif get_bool "$LOOKS_LIKE_SUSE"; then
install_suse_systemd
test: support NO_BUILD=yes on Fedora
2023-11-20 19:33:58 +08:00
elif get_bool "$LOOKS_LIKE_FEDORA"; then
install_fedora_systemd
test/test-functions: allow installing systemd files from local system Add NO_BUILD var to allow testing with no local build, by installing local systemd files into the image. This only works for debian-like distros currently, that use the tools 'apt' and 'dpkg' for package management.
2020-11-18 06:20:22 +08:00
else
dfatal "NO_BUILD not supported for this distro"
exit 1
fi
}
install_systemd() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install systemd"
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$NO_BUILD"; then
test/test-functions: allow installing systemd files from local system Add NO_BUILD var to allow testing with no local build, by installing local systemd files into the image. This only works for debian-like distros currently, that use the tools 'apt' and 'dpkg' for package management.
2020-11-18 06:20:22 +08:00
install_distro_systemd
else
install_compiled_systemd
fi
test: use dropins to override system configuration Instead of touching the configuration files directly.
2023-05-14 14:15:33 +08:00
# Remove unneeded documentation
test: loosen sandbox restrictions for integration tests as well Otherwise we miss quite a lot of coverage (mainly from logind, hostnamed, networkd, and possibly others), since they can't write their reports with `ProtectSystem=strict`.
2021-10-22 00:06:36 +08:00
rm -fr "${initdir:?}"/usr/share/{man,doc}
tests: enable logging for pid1, disable for other systemd services systemd-udev generated an insane amount of log output at debug level. It would break TEST-02-CRYPTSETUP by filling the overflowing the disk (which seems to be a bug in itself!).
2016-04-26 02:20:57 +08:00
test: use dropins to override system configuration Instead of touching the configuration files directly.
2023-05-14 14:15:33 +08:00
# Enable debug logging in PID1
mkdir -p "$initdir/etc/systemd/system.conf.d/"
echo -ne "[Manager]\nLogLevel=debug\n" >"$initdir/etc/systemd/system.conf.d/10-log-level.conf"
tests: allow running all the services with SYSTEMD_LOG_LEVEL It should make it easier to figure out what exactly services do there. For example, with SYSTEMD_LOG_LEVEL=debug userdbd (v249) prints ``` varlink-5: New incoming message: {"method":"io.systemd.UserDatabase.GetUserRecord","parameters":{}} ``` before it crashes and systemd-resolved prints ``` varlink-21: New incoming message: {"method":"io.systemd.Resolve.ResolveAddress","parameters":{"address":[127,0,0,1],"flags":0,"ifindex":1000000,"family":0}} ``` and those messages are helpful (especially when scripts causing them aren't clever enough to keep track of random stuff they send to systemd :-))
2022-02-15 09:13:10 +08:00
if [[ -n "$TEST_SYSTEMD_LOG_LEVEL" ]]; then
test: use dropins to override system configuration Instead of touching the configuration files directly.
2023-05-14 14:15:33 +08:00
echo DefaultEnvironment=SYSTEMD_LOG_LEVEL="$TEST_SYSTEMD_LOG_LEVEL" >>"$initdir/etc/systemd/system.conf.d/99-log-level.conf"
tests: allow running all the services with SYSTEMD_LOG_LEVEL It should make it easier to figure out what exactly services do there. For example, with SYSTEMD_LOG_LEVEL=debug userdbd (v249) prints ``` varlink-5: New incoming message: {"method":"io.systemd.UserDatabase.GetUserRecord","parameters":{}} ``` before it crashes and systemd-resolved prints ``` varlink-21: New incoming message: {"method":"io.systemd.Resolve.ResolveAddress","parameters":{"address":[127,0,0,1],"flags":0,"ifindex":1000000,"family":0}} ``` and those messages are helpful (especially when scripts causing them aren't clever enough to keep track of random stuff they send to systemd :-))
2022-02-15 09:13:10 +08:00
fi
test: enable debug logs for the user instances as well To make debugging [0], hopefully, easier. [0] https://github.com/systemd/systemd/issues/28504
2023-09-25 19:25:27 +08:00
# Enable debug logging for user instances as well
mkdir -p "$initdir/etc/systemd/user.conf.d/"
echo -ne "[Manager]\nLogLevel=debug\n" >"$initdir/etc/systemd/user.conf.d/10-log-level.conf"
test: use dropins to override system configuration Instead of touching the configuration files directly.
2023-05-14 14:15:33 +08:00
# Store coredumps in journal
mkdir -p "$initdir/etc/systemd/coredump.conf.d/"
echo -ne "[Coredump]\nStorage=journal\n" >"$initdir/etc/systemd/coredump.conf.d/10-storage-journal.conf"
oom: Add support for user unit ManagedOOM property updates Compared to PID1 where systemd-oomd has to be the client to PID1 because PID1 is a more privileged process than systemd-oomd, systemd-oomd is the more privileged process compared to a user manager so we have user managers be the client whereas systemd-oomd is now the server. The same varlink protocol is used between user managers and systemd-oomd to deliver ManagedOOM property updates. systemd-oomd now sets up a varlink server that user managers connect to to send ManagedOOM property updates. We also add extra validation to make sure that non-root senders don't send updates for cgroups they don't own. The integration test was extended to repeat the chill/bloat test using a user manager instead of PID1.
2021-09-09 23:12:55 +08:00
# Propagate SYSTEMD_UNIT_PATH to user systemd managers
test: use dropins to override system configuration Instead of touching the configuration files directly.
2023-05-14 14:15:33 +08:00
mkdir -p "$initdir/etc/systemd/system/user@.service.d/"
echo -ne "[Service]\nPassEnvironment=SYSTEMD_UNIT_PATH\n" >"$initdir/etc/systemd/system/user@.service.d/99-SYSTEMD_UNIT_PATH.conf"
test: loosen sandbox restrictions for integration tests as well Otherwise we miss quite a lot of coverage (mainly from logind, hostnamed, networkd, and possibly others), since they can't write their reports with `ProtectSystem=strict`.
2021-10-22 00:06:36 +08:00
test: set `ReadWritePaths=$BUILD_DIR` when running w/ coverage otherwise units using `DynamicUser=yes` won't be able to write the coverage stats (currently affecting TEST-20-MAINPIDGAMES). `DynamicUser=yes` implies `ProtectSystem=strict` and `ProtectHome=read-only` and can't be overridden hence we need to utilize `ReadWritePaths=` to work around that.
2022-02-15 03:19:51 +08:00
# When built with gcov, disable ProtectSystem= and ProtectHome= in the test
# images, since it prevents gcov to write the coverage reports (*.gcda
test: set `ReadWritePaths=` only for units from TEST-20 as setting it system-wide has some undesirable side-effects. Follow-up to b6fc524.
2022-02-16 22:55:09 +08:00
# files)
test: loosen sandbox restrictions for integration tests as well Otherwise we miss quite a lot of coverage (mainly from logind, hostnamed, networkd, and possibly others), since they can't write their reports with `ProtectSystem=strict`.
2021-10-22 00:06:36 +08:00
if get_bool "$IS_BUILT_WITH_COVERAGE"; then
mkdir -p "$initdir/etc/systemd/system/service.d/"
test: use dropins to override system configuration Instead of touching the configuration files directly.
2023-05-14 14:15:33 +08:00
echo -ne "[Service]\nProtectSystem=no\nProtectHome=no\n" >"$initdir/etc/systemd/system/service.d/99-gcov-override.conf"
test: extend the DynamicUser=yes coverage workaround to a couple more services
2023-06-19 05:08:24 +08:00
# Similarly, set ReadWritePaths= to the $BUILD_DIR in the test image to make the coverage work with
# units using DynamicUser=yes. Do this only for services with test- prefix and a couple of
# known-to-use DynamicUser=yes services, as setting this system-wide has many undesirable
# side-effects, as it creates its own namespace.
for service in test- systemd-journal-{gatewayd,upload}; do
mkdir -p "$initdir/etc/systemd/system/$service.service.d/"
echo -ne "[Service]\nReadWritePaths=${BUILD_DIR:?}\n" >"$initdir/etc/systemd/system/$service.service.d/99-gcov-rwpaths-override.conf"
done
test: set ReadWritePaths= for test-.services when built w/ coverage Let's make the dropin, to make the build dir writable for gcov, a bit more generic, so it can be used by all units starting with prefix test-. This should help with a bunch of recent reports about missing coverage I got, as well as with existing test units using DynamicUser=true. This might feel a bit like a magic trick from behind the curtains, but I want to touch the actual tests as little as possible, since it makes them unnecessarily messy (see the various workarounds for sanitizers), and the coverage reports are generated only in a specific CI job anyway.
2023-04-01 00:42:38 +08:00
# Ditto, but for the user daemon
mkdir -p "$initdir/etc/systemd/user/test-.service.d/"
test: use dropins to override system configuration Instead of touching the configuration files directly.
2023-05-14 14:15:33 +08:00
echo -ne "[Service]\nReadWritePaths=${BUILD_DIR:?}\n" >"$initdir/etc/systemd/user/test-.service.d/99-gcov-rwpaths-override.conf"
test: bind mount the $BUILD_DIR into nspawn containers when collecting coverage. This applies only to containers started via machinectl (or directly via the systemd-nspawn@.service unit).
2023-05-14 17:28:30 +08:00
# Bind the $BUILD_DIR into nspawn containers that are executed using
# machinectl. Unfortunately, the .nspawn files don't support drop-ins
# so we have to inject the bind mount directly into
# the systemd-nspawn@.service unit.
cp "$initdir/usr/lib/systemd/system/systemd-nspawn@.service" "$initdir/etc/systemd/system/systemd-nspawn@.service"
sed -ri "s/^ExecStart=.+$/& --bind=${BUILD_DIR//\//\\\/}/" "$initdir/etc/systemd/system/systemd-nspawn@.service"
test: pass $BUILD_DIR as $COVERAGE_BUILD_DIR to the system manager To make additional coverage-related tweaks slightly easier.
2023-05-15 16:06:18 +08:00
# Pass the $BUILD_DIR as $COVERAGE_BUILD_DIR env variable to the system
# manager, similarly to what we do with $ASAN_RT_PATH during sanitized
# builds
mkdir -p "$initdir/etc/systemd/system.conf.d/"
echo -ne "[Manager]\nDefaultEnvironment=COVERAGE_BUILD_DIR=$BUILD_DIR\n" >"$initdir/etc/systemd/system.conf.d/99-COVERAGE_BUILD_DIR.conf"
test: loosen sandbox restrictions for integration tests as well Otherwise we miss quite a lot of coverage (mainly from logind, hostnamed, networkd, and possibly others), since they can't write their reports with `ProtectSystem=strict`.
2021-10-22 00:06:36 +08:00
fi
test: copy portable profiles into the image if they don't exist there If we're built with `-Dportable=false`, the portable profiles won't get installed into the image. Since we need only the profile files and nothing else, let's copy them into the image explicitly in such case.
2022-01-31 21:19:09 +08:00
# If we're built with -Dportabled=false, tests with systemd-analyze
# --profile will fail. Since we need just the profile (text) files, let's
# copy them into the image if they don't exist there.
local portable_dir="${initdir:?}${ROOTLIBDIR:?}/portable"
if [[ ! -d "$portable_dir/profile/strict" ]]; then
dinfo "Couldn't find portable profiles in the test image"
dinfo "Copying them directly from the source tree"
mkdir -p "$portable_dir"
cp -frv "${SOURCE_DIR:?}/src/portable/profile" "$portable_dir"
fi
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
tests: follow RUNPATH when installing missing libraries Fixes #3630
2016-07-03 23:17:13 +08:00
get_ldpath() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local rpath
rpath="$(objdump -p "${1:?}" 2>/dev/null | awk "/R(UN)?PATH/ { print \"$initdir\" \$2 }" | paste -sd :)"
test-functions: make sure we use the right library path for binaries without RPATH Meson appears to set the rpath only for some binaries it builds, but not all. (The rules are not clear to me, but that's besides the point of this commit). Let's make sure if our test script operates on a binary that has no rpath set we fall back preferably to the BUILD_DIR rather than directly to the host. This matters if a test uses a libsystemd symbol introduced in a version newer than the one on the host. In that case "ldd" will not work on the test binary if rpath is not set. With this fix that behaviour is corrected, and "ldd" works correctly even in this case. (Or in other words: before this fix on binaries lacking rpath we'd base dependency info on the libraries of the host, not the buidl tree, if they exist in both.)
2019-11-28 01:25:52 +08:00
if [ -z "$rpath" ] ; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
echo "$BUILD_DIR"
test-functions: make sure we use the right library path for binaries without RPATH Meson appears to set the rpath only for some binaries it builds, but not all. (The rules are not clear to me, but that's besides the point of this commit). Let's make sure if our test script operates on a binary that has no rpath set we fall back preferably to the BUILD_DIR rather than directly to the host. This matters if a test uses a libsystemd symbol introduced in a version newer than the one on the host. In that case "ldd" will not work on the test binary if rpath is not set. With this fix that behaviour is corrected, and "ldd" works correctly even in this case. (Or in other words: before this fix on binaries lacking rpath we'd base dependency info on the libraries of the host, not the buidl tree, if they exist in both.)
2019-11-28 01:25:52 +08:00
else
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
echo "$rpath"
test-functions: make sure we use the right library path for binaries without RPATH Meson appears to set the rpath only for some binaries it builds, but not all. (The rules are not clear to me, but that's besides the point of this commit). Let's make sure if our test script operates on a binary that has no rpath set we fall back preferably to the BUILD_DIR rather than directly to the host. This matters if a test uses a libsystemd symbol introduced in a version newer than the one on the host. In that case "ldd" will not work on the test binary if rpath is not set. With this fix that behaviour is corrected, and "ldd" works correctly even in this case. (Or in other words: before this fix on binaries lacking rpath we'd base dependency info on the libraries of the host, not the buidl tree, if they exist in both.)
2019-11-28 01:25:52 +08:00
fi
tests: follow RUNPATH when installing missing libraries Fixes #3630
2016-07-03 23:17:13 +08:00
}
test: remove duplicated code
2013-11-06 06:32:56 +08:00
install_missing_libraries() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install missing libraries"
test: remove duplicated code
2013-11-06 06:32:56 +08:00
# install possible missing libraries
test: install unit tests in a dedicated subdirectory below '$testsdir' /usr/lib/systemd/tests may contain more than the unit tests. For example on SUSE we also install the integration tests there. Putting the unit tests in a dedicated directory named 'unit-tests' makes the layout cleaner. Note that `run-unit-tests.py` has not been moved so we don't need to adjust (Fedora) packaging and users also don't need to descend into the subdirectory.
2023-01-26 23:07:17 +08:00
for i in "${initdir:?}"{,/usr}/{sbin,bin}/* "$initdir"{,/usr}/lib/systemd/{,tests/unit-tests/{,manual/,unsafe/}}*; do
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
LD_LIBRARY_PATH="${LD_LIBRARY_PATH:+$LD_LIBRARY_PATH:}$(get_ldpath "$i")" inst_libs "$i"
test: remove duplicated code
2013-11-06 06:32:56 +08:00
done
test: pull in weak deps into tests (Pulled from @bluca's comment here: https://github.com/systemd/systemd/pull/17884#issuecomment-740005624 and turned into a commit by @poettering)
2020-12-09 17:15:36 +08:00
test: install libgcc_s.so.1 explicitly if available Since the library is dlopen()ed by libpthread and required during pthread_exit()/pthread_cancel(), let's install it explicitly if available to avoid unexpected fails in tests. This also consolidates all related workarounds for this library across the test scripts.
2022-08-02 19:54:57 +08:00
# Install libgcc_s.so if available, since it's dlopen()ed by libpthread
# and might cause unexpected failures during pthread_exit()/pthread_cancel()
# if not present
# See: https://github.com/systemd/systemd/pull/23858
while read -r libgcc_s; do
[[ -e "$libgcc_s" ]] && inst_library "$libgcc_s"
done < <(ldconfig -p | awk '/\/libgcc_s.so.1$/ { print $4 }')
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local lib path
test: pull in weak deps into tests (Pulled from @bluca's comment here: https://github.com/systemd/systemd/pull/17884#issuecomment-740005624 and turned into a commit by @poettering)
2020-12-09 17:15:36 +08:00
# A number of dependencies is now optional via dlopen, so the install
# script will not pick them up, since it looks at linkage.
dissect: add --make-archive option to convert DDI to tarball
2024-01-24 19:28:41 +08:00
for lib in libcryptsetup libidn libidn2 pwquality libqrencode tss2-esys tss2-rc tss2-mu tss2-tcti-device libfido2 libbpf libelf libdw xkbcommon p11-kit-1 libarchive; do
test: skip missing optional libraries in image install Not all optional libraries might be available on developers machines, so log and skip. Also some pkg-config files are broken (eg: tss2 on Debian Stable) so skip if the required variables are missing, and improve logs.
2021-01-15 00:48:13 +08:00
ddebug "Searching for $lib via pkg-config"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if pkg-config --exists "$lib"; then
path="$(pkg-config --variable=libdir "$lib")"
test: skip missing optional libraries in image install Not all optional libraries might be available on developers machines, so log and skip. Also some pkg-config files are broken (eg: tss2 on Debian Stable) so skip if the required variables are missing, and improve logs.
2021-01-15 00:48:13 +08:00
if [ -z "${path}" ]; then
ddebug "$lib.pc does not contain a libdir variable, skipping"
continue
fi
if ! [[ ${lib} =~ ^lib ]]; then
lib="lib${lib}"
fi
p11kit: switch to dlopen()
2022-12-17 09:33:46 +08:00
# p11-kit-1's .so doesn't have the API level in the name
if [[ ${lib} =~ p11-kit-1$ ]]; then
lib="libp11-kit"
fi
test: skip missing optional libraries in image install Not all optional libraries might be available on developers machines, so log and skip. Also some pkg-config files are broken (eg: tss2 on Debian Stable) so skip if the required variables are missing, and improve logs.
2021-01-15 00:48:13 +08:00
# Some pkg-config files are broken and give out the wrong paths
# (eg: libcryptsetup), so just ignore them
inst_libs "${path}/${lib}.so" || true
inst_library "${path}/${lib}.so" || true
test: install libxkbcommon and x11 keymaps
2022-06-14 08:12:00 +08:00
if [[ "$lib" == "libxkbcommon" ]]; then
install_x11_keymaps full
fi
test: skip missing optional libraries in image install Not all optional libraries might be available on developers machines, so log and skip. Also some pkg-config files are broken (eg: tss2 on Debian Stable) so skip if the required variables are missing, and improve logs.
2021-01-15 00:48:13 +08:00
else
ddebug "$lib.pc not found, skipping"
continue
fi
test: pull in weak deps into tests (Pulled from @bluca's comment here: https://github.com/systemd/systemd/pull/17884#issuecomment-740005624 and turned into a commit by @poettering)
2020-12-09 17:15:36 +08:00
done
test: Install openssl 3 extra library dependencies
2022-09-18 04:35:19 +08:00
# Install extra openssl 3 stuff
path="$(pkg-config --variable=libdir libcrypto)"
inst_simple "${path}/ossl-modules/legacy.so" || true
inst_simple "${path}/ossl-modules/fips.so" || true
inst_simple "${path}/engines-3/afalg.so" || true
inst_simple "${path}/engines-3/capi.so" || true
inst_simple "${path}/engines-3/loader_attic.so" || true
inst_simple "${path}/engines-3/padlock.so" || true
mkfs-util: Add support to populate vfat without mounting using mcopy mkfs.vfat doesn't support specifying a root directory to bootstrap the filesystem from (see https://github.com/dosfstools/dosfstools/issues/183). Instead, we can use the mcopy tool from the mtools package to copy files into the vfat filesystem after creating it without needing to mount the vfat filesystem.
2022-10-11 16:50:58 +08:00
# Binaries from mtools depend on the gconv modules to translate between codepages. Because there's no
# pkg-config file for these, we copy every gconv/ directory we can find in /usr/lib and /usr/lib64.
# shellcheck disable=SC2046
inst_recursive $(find /usr/lib* -name gconv 2>/dev/null)
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
test: stop caching loopdev It is more trouble than it is worth. The setup is of a loopback device is very quick, so it's better to always create it when needed and immediately drop afterwards.
2020-03-21 03:17:11 +08:00
cleanup_loopdev() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [ -n "${LOOPDEV:=}" ]; then
test: stop caching loopdev It is more trouble than it is worth. The setup is of a loopback device is very quick, so it's better to always create it when needed and immediately drop afterwards.
2020-03-21 03:17:11 +08:00
ddebug "losetup -d $LOOPDEV"
losetup -d "${LOOPDEV}"
test: create public images by default Let's create new images public by default and then symlink/copy them into the respective private directories afterwards, not the other way around. This should fix a nasty race condition in parallel runs where one tests attempts to copy the backing public image at the same moment another test is already modifying it.
2020-05-26 04:48:01 +08:00
unset LOOPDEV
test: stop caching loopdev It is more trouble than it is worth. The setup is of a loopback device is very quick, so it's better to always create it when needed and immediately drop afterwards.
2020-03-21 03:17:11 +08:00
fi
}
test: allow multiple handlers for the EXIT signal Bash allows only one handler per signal, so let's overcome this limitation by having one dedicated EXIT signal which runs all registered handlers from all over the place.
2022-07-08 02:00:46 +08:00
add_at_exit_handler cleanup_loopdev
test: stop caching loopdev It is more trouble than it is worth. The setup is of a loopback device is very quick, so it's better to always create it when needed and immediately drop afterwards.
2020-03-21 03:17:11 +08:00
test: remove duplicated code
2013-11-06 06:32:56 +08:00
create_empty_image() {
test: minor cleanup No functional change.
2023-06-04 21:33:24 +08:00
if [[ -z "${IMAGE_NAME:=}" ]]; then
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
echo "create_empty_image: \$IMAGE_NAME not set"
exit 1
fi
test: bump the data partition size if we don't strip binaries so we can run TEST-24 under sanitizers as well. Also, when at it, use the 'named-fields' sfdisk format to make the code a bit more descriptive without needing a manual.
2022-07-07 20:13:32 +08:00
# Partition sizes are in MiBs
test: drop binary stripping stuff Stripping the binaries in the test images makes potential stack straces quite useless, so let's drop the stripping stuff to make test fails a bit more developer friendly. Related: https://github.com/systemd/systemd-centos-ci/pull/616
2023-04-11 23:23:23 +08:00
local root_size=768
local data_size=100
test: create ESP and xbootldr partitions
2023-12-05 15:31:53 +08:00
local esp_size=128
local boot_size=128
local total=
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if ! get_bool "$NO_BUILD"; then
test: do not run 'meson configure' if NO_BUILD is set There is no build tree and packages are used, so it cannot work. Unlikely that static linking has been set for those builds anyway. Fixes https://github.com/systemd/systemd/issues/19955
2021-06-17 04:50:23 +08:00
if meson configure "${BUILD_DIR:?}" | grep 'static-lib\|standalone-binaries' | awk '{ print $2 }' | grep -q 'true'; then
test: minor cleanup No functional change.
2023-06-04 21:33:24 +08:00
root_size=$((root_size + 200))
test: do not run 'meson configure' if NO_BUILD is set There is no build tree and packages are used, so it cannot work. Unlikely that static linking has been set for those builds anyway. Fixes https://github.com/systemd/systemd/issues/19955
2021-06-17 04:50:23 +08:00
fi
if meson configure "${BUILD_DIR:?}" | grep 'link-.*-shared' | awk '{ print $2 }' | grep -q 'false'; then
test: minor cleanup No functional change.
2023-06-04 21:33:24 +08:00
root_size=$((root_size + 200))
test: do not run 'meson configure' if NO_BUILD is set There is no build tree and packages are used, so it cannot work. Unlikely that static linking has been set for those builds anyway. Fixes https://github.com/systemd/systemd/issues/19955
2021-06-17 04:50:23 +08:00
fi
test: loosen sandbox restrictions for integration tests as well Otherwise we miss quite a lot of coverage (mainly from logind, hostnamed, networkd, and possibly others), since they can't write their reports with `ProtectSystem=strict`.
2021-10-22 00:06:36 +08:00
if get_bool "$IS_BUILT_WITH_COVERAGE"; then
test: minor cleanup No functional change.
2023-06-04 21:33:24 +08:00
root_size=$((root_size + 250))
test: create and merge code coverage reports in integration tests If -Db_coverage=true is used at build time, then ARTIFACT_DIRECTORY/TEST-XX-FOO.coverage-info files are created with code coverage data, and run-integration-test.sh also merges them into ARTIFACT_DIRECTORY/merged.coverage-info since the coveralls.io helpers accept only a single file.
2021-10-03 23:50:38 +08:00
fi
test: drop binary stripping stuff Stripping the binaries in the test images makes potential stack straces quite useless, so let's drop the stripping stuff to make test fails a bit more developer friendly. Related: https://github.com/systemd/systemd-centos-ci/pull/616
2023-04-11 23:23:23 +08:00
if get_bool "$IS_BUILT_WITH_ASAN"; then
root_size=$((root_size * 2))
fi
test: increase image size when static library or standalone binaries are installed
2021-05-06 12:28:19 +08:00
fi
test: drop binary stripping stuff Stripping the binaries in the test images makes potential stack straces quite useless, so let's drop the stripping stuff to make test fails a bit more developer friendly. Related: https://github.com/systemd/systemd-centos-ci/pull/616
2023-04-11 23:23:23 +08:00
test: make the root/data partition size configurable per test
2023-06-05 15:39:51 +08:00
if [[ "${IMAGE_ADDITIONAL_ROOT_SIZE:-0}" -gt 0 ]]; then
root_size=$((root_size + IMAGE_ADDITIONAL_ROOT_SIZE))
fi
if [[ "${IMAGE_ADDITIONAL_DATA_SIZE:-0}" -gt 0 ]]; then
data_size=$((data_size + IMAGE_ADDITIONAL_DATA_SIZE))
repart: Add Minimize= integration test
2022-11-10 19:33:02 +08:00
fi
test: improve reporting a bit
2019-12-11 00:33:58 +08:00
test: create ESP and xbootldr partitions
2023-12-05 15:31:53 +08:00
total=$((root_size + data_size + esp_size + boot_size))
echo "Setting up ${IMAGE_PUBLIC:?} (${total} MB)"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
rm -f "${IMAGE_PRIVATE:?}" "$IMAGE_PUBLIC"
test: run tests directly from the loopback device Before, we'd copy the test tree into nspawn-root, and run the tests from there. This is OK, and doesn't actually take much extra time. But it uses quite a lot of extra disk space. So let's make things a bit more efficient by running directly from the image file. We still run the unprivileged nspawn tests from a copy. Once the kernel implements fs shift, we can do away with that too.
2019-12-13 21:21:31 +08:00
test: remove duplicated code
2013-11-06 06:32:56 +08:00
# Create the blank file to use as a root filesystem
test: create ESP and xbootldr partitions
2023-12-05 15:31:53 +08:00
truncate -s "${total}M" "$IMAGE_PUBLIC"
test: run tests directly from the loopback device Before, we'd copy the test tree into nspawn-root, and run the tests from there. This is OK, and doesn't actually take much extra time. But it uses quite a lot of extra disk space. So let's make things a bit more efficient by running directly from the image file. We still run the unprivileged nspawn tests from a copy. Once the kernel implements fs shift, we can do away with that too.
2019-12-13 21:21:31 +08:00
test: minor cleanup No functional change.
2023-06-04 21:33:24 +08:00
LOOPDEV="$(losetup --show -P -f "$IMAGE_PUBLIC")"
[[ -b "$LOOPDEV" ]] || return 1
test: bump the data partition size if we don't strip binaries so we can run TEST-24 under sanitizers as well. Also, when at it, use the 'named-fields' sfdisk format to make the code a bit more descriptive without needing a manual.
2022-07-07 20:13:32 +08:00
# Create two partitions - a root one and a data one (utilized by some tests)
test-functions: use sfdisk without cyl,head,sectors newer sfdisk implementations do not work with these anymore
2015-02-03 17:44:27 +08:00
sfdisk "$LOOPDEV" <<EOF
test: bump the data partition size if we don't strip binaries so we can run TEST-24 under sanitizers as well. Also, when at it, use the 'named-fields' sfdisk format to make the code a bit more descriptive without needing a manual.
2022-07-07 20:13:32 +08:00
label: gpt
test: create ESP and xbootldr partitions
2023-12-05 15:31:53 +08:00
type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B name=esp size=${esp_size}M
test: use systemd-id128 from the build dir As distro sd-id128 might not have all the options we need. Follow-up for: 378712c Replaces: #30901
2024-01-12 17:27:43 +08:00
type=$("${SYSTEMD_ID128:?}" show root -Pu) name=root size=${root_size}M bootable
test: create ESP and xbootldr partitions
2023-12-05 15:31:53 +08:00
type=BC13C2FF-59E6-4262-A352-B275FD6F7172 name=boot size=${boot_size}M
test: bump the data partition size if we don't strip binaries so we can run TEST-24 under sanitizers as well. Also, when at it, use the 'named-fields' sfdisk format to make the code a bit more descriptive without needing a manual.
2022-07-07 20:13:32 +08:00
type=0FC63DAF-8483-4772-8E79-3D69D8477DE4 name=data
test: remove duplicated code
2013-11-06 06:32:56 +08:00
EOF
tests: run `udevadm settle` after `sfdisk` (#8610) This makes the script wait for the newly created partition to show up before trying to put a filesystem on it, which should prevent the tests from failing with the following error: ``` New situation: Disklabel type: dos Disk identifier: 0x3541a0ec Device Boot Start End Sectors Size Id Type /dev/loop6p1 2048 800767 798720 390M 83 Linux /dev/loop6p2 800768 819199 18432 9M 83 Linux The partition table has been altered. Calling ioctl() to re-read partition table. Syncing disks. The file /dev/loop6p1 does not exist and no size was specified. make: *** [setup] Error 1 F: Failed to mkfs -t ext4 Makefile:4: recipe for target 'setup' failed ```
2018-03-29 12:20:57 +08:00
udevadm settle
test: create ESP and xbootldr partitions
2023-12-05 15:31:53 +08:00
if ! mkfs -t vfat "${LOOPDEV}p1"; then
dfatal "Failed to mkfs -t vfat ${LOOPDEV}p1"
exit 1
fi
test: specify the rootfs by label otherwise the boot might fail when attaching multiple SATA devices to the VM.
2021-09-14 00:14:05 +08:00
local label=(-L systemd_boot)
basic: fallback to fstatat if entry->d_type is DT_UNKNOWN * tests-functions: improve FSTYPE-support make clean setup FSTYPE=reiserfs is working fine now :) * basic: fallback to fstatat if entry->d_type is DT_UNKNOWN Fixes localectl on reiserfs: -bash-4.3# mkdir -p /usr/lib/locale -bash-4.3# stat -f /usr/lib/locale/ File: "/usr/lib/locale/" ID: bdb0322715b5366e Namelen: 255 Type: reiserfs Block size: 4096 Blocks: Total: 99835 Free: 60262 Available: 60262 Inodes: Total: 0 Free: 0 -bash-4.3# mkdir /usr/lib/locale/HeyHo -bash-4.3# localectl list-locales --no-pager -bash-4.3# mount -t tmpfs tmpfs /usr/lib/locale -bash-4.3# mkdir /usr/lib/locale/HeyHo -bash-4.3# localectl list-locales --no-pager HeyHo
2016-04-04 19:07:56 +08:00
# mkfs.reiserfs doesn't know -L. so, use --label instead
test: specify the rootfs by label otherwise the boot might fail when attaching multiple SATA devices to the VM.
2021-09-14 00:14:05 +08:00
[[ "$FSTYPE" == "reiserfs" ]] && label=(--label systemd_boot)
test: create ESP and xbootldr partitions
2023-12-05 15:31:53 +08:00
if ! mkfs -t "${FSTYPE}" "${label[@]}" "${LOOPDEV}p2" -q; then
dfatal "Failed to mkfs -t ${FSTYPE} ${label[*]} ${LOOPDEV}p2 -q"
exit 1
fi
local label=(-L xbootldr)
[[ "$FSTYPE" == "reiserfs" ]] && label=(--label xbootldr)
if ! mkfs -t "${FSTYPE}" "${label[@]}" "${LOOPDEV}p3" -q; then
dfatal "Failed to mkfs -t ${FSTYPE} ${label[*]} ${LOOPDEV}p3 -q"
basic: fallback to fstatat if entry->d_type is DT_UNKNOWN * tests-functions: improve FSTYPE-support make clean setup FSTYPE=reiserfs is working fine now :) * basic: fallback to fstatat if entry->d_type is DT_UNKNOWN Fixes localectl on reiserfs: -bash-4.3# mkdir -p /usr/lib/locale -bash-4.3# stat -f /usr/lib/locale/ File: "/usr/lib/locale/" ID: bdb0322715b5366e Namelen: 255 Type: reiserfs Block size: 4096 Blocks: Total: 99835 Free: 60262 Available: 60262 Inodes: Total: 0 Free: 0 -bash-4.3# mkdir /usr/lib/locale/HeyHo -bash-4.3# localectl list-locales --no-pager -bash-4.3# mount -t tmpfs tmpfs /usr/lib/locale -bash-4.3# mkdir /usr/lib/locale/HeyHo -bash-4.3# localectl list-locales --no-pager HeyHo
2016-04-04 19:07:56 +08:00
exit 1
fi
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
mount_initdir() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [ -z "${LOOPDEV:=}" ]; then
[ -e "${IMAGE_PRIVATE:?}" ] && image="$IMAGE_PRIVATE" || image="${IMAGE_PUBLIC:?}"
LOOPDEV="$(losetup --show -P -f "$image")"
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
[ -b "$LOOPDEV" ] || return 1
test: stop caching loopdev It is more trouble than it is worth. The setup is of a loopback device is very quick, so it's better to always create it when needed and immediately drop afterwards.
2020-03-21 03:17:11 +08:00
udevadm settle
fi
test: add create_empty_image_rootdir() to simplify testcase setup Almost all tests were manually mounting/unmounting $TESTDIR/root from the loopback image; this moves all that into test-functions so the test setup functions are simplier. Also add test_setup_cleanup() function, to cleanup what is mounted by create_empty_image_rootdir()
2019-07-12 23:47:26 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if ! mountpoint -q "${initdir:?}"; then
mkdir -p "$initdir"
test: create ESP and xbootldr partitions
2023-12-05 15:31:53 +08:00
mount "${LOOPDEV}p2" "$initdir"
test: stop caching loopdev It is more trouble than it is worth. The setup is of a loopback device is very quick, so it's better to always create it when needed and immediately drop afterwards.
2020-03-21 03:17:11 +08:00
TEST_SETUP_CLEANUP_ROOTDIR=1
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
fi
}
test: run tests directly from the loopback device Before, we'd copy the test tree into nspawn-root, and run the tests from there. This is OK, and doesn't actually take much extra time. But it uses quite a lot of extra disk space. So let's make things a bit more efficient by running directly from the image file. We still run the unprivileged nspawn tests from a copy. Once the kernel implements fs shift, we can do away with that too.
2019-12-13 21:21:31 +08:00
cleanup_initdir() {
# only umount if create_empty_image_rootdir() was called to mount it
test: expand the expression in `cleanup_initdir()` Otherwise we might unexpectedly return 1 if the `get_bool` call fails. If the `get_bool` part in `get_bool "$TEST_SETUP_CLEANUP_ROOTDIR" && _umount_dir "${initdir:?}"` fails, the whole expression will short-circuit evaluate to 1, and since it's the last expression in the function it's also it's return value, which doesn't reflect the original intent of the expression: ``` # BUILD_DIR=$PWD/build make -C test/TEST-64-UDEV-STORAGE/ setup run TESTCASES=testcase_always_skip make: Entering directory '/home/fsumsal/repos/@systemd/systemd/test/TEST-64-UDEV-STORAGE' TEST-64-UDEV-STORAGE SETUP: systemd-udev storage tests Reusing existing image /var/tmp/systemd-test.uPbJZ9/default.img → /var/tmp/systemd-test.uPbJZ9/default.img TEST-64-UDEV-STORAGE RUN: systemd-udev storage tests ------ testcase_always_skip: BEGIN ------ Skipping... ------ testcase_always_skip: END (SKIP) ------ Passed tests: 0 * Skipped tests: 1 * testcase_always_skip Failed tests: 0 * TEST-64-UDEV-STORAGE RUN: systemd-udev storage tests [OK] make: Leaving directory '/home/fsumsal/repos/@systemd/systemd/test/TEST-64-UDEV-STORAGE' # BUILD_DIR=$PWD/build make -C test/TEST-64-UDEV-STORAGE/ setup run TESTCASES=testcase_always_skip make: Entering directory '/home/fsumsal/repos/@systemd/systemd/test/TEST-64-UDEV-STORAGE' TEST-64-UDEV-STORAGE SETUP: systemd-udev storage tests Reusing existing image /var/tmp/systemd-test.uPbJZ9/default.img → /var/tmp/systemd-test.uPbJZ9/default.img make: *** [Makefile:4: setup] Error 1 make: Leaving directory '/home/fsumsal/repos/@systemd/systemd/test/TEST-64-UDEV-STORAGE' ```
2022-10-02 03:56:08 +08:00
if get_bool "$TEST_SETUP_CLEANUP_ROOTDIR"; then
_umount_dir "${initdir:?}"
fi
test: run tests directly from the loopback device Before, we'd copy the test tree into nspawn-root, and run the tests from there. This is OK, and doesn't actually take much extra time. But it uses quite a lot of extra disk space. So let's make things a bit more efficient by running directly from the image file. We still run the unprivileged nspawn tests from a copy. Once the kernel implements fs shift, we can do away with that too.
2019-12-13 21:21:31 +08:00
}
umount_loopback() {
# unmount the loopback device from all places. Otherwise we risk file
# system corruption.
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
for device in $(losetup -l | awk '$6=="'"${IMAGE_PUBLIC:?}"'" {print $1}'); do
test: run tests directly from the loopback device Before, we'd copy the test tree into nspawn-root, and run the tests from there. This is OK, and doesn't actually take much extra time. But it uses quite a lot of extra disk space. So let's make things a bit more efficient by running directly from the image file. We still run the unprivileged nspawn tests from a copy. Once the kernel implements fs shift, we can do away with that too.
2019-12-13 21:21:31 +08:00
ddebug "Unmounting all uses of $device"
mount | awk '/^'"${device}"'p/{print $1}' | xargs --no-run-if-empty umount -v
done
}
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
create_empty_image_rootdir() {
create_empty_image
mount_initdir
}
tests: introduce check_asan_reports and use it in check_result_{qemu|nspawn}
2018-12-25 15:31:50 +08:00
check_asan_reports() {
local ret=0
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local root="${1:?}"
test: make sure we get PID1's stack trace on ASan/UBSan errors As hitting an ASan/UBSan error in PID1 results in a crash (and a kernel panic when running under qemu), we usually lose the stack trace which makes debugging quite painful. Let's mitigate this by forwarding the stack trace to multiple places - namely to a file and the syslog.
2023-06-23 20:28:30 +08:00
local log report
tests: introduce check_asan_reports and use it in check_result_{qemu|nspawn}
2018-12-25 15:31:50 +08:00
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$IS_BUILT_WITH_ASAN"; then
tests: introduce check_asan_reports and use it in check_result_{qemu|nspawn}
2018-12-25 15:31:50 +08:00
ls -l "$root"
if [[ -e "$root/systemd.asan.log.1" ]]; then
cat "$root/systemd.asan.log.1"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
ret=$((ret+1))
tests: reproduce https://github.com/systemd/systemd/issues/11251
2018-12-23 21:46:00 +08:00
fi
tests: fail if asan has found issues in journald
2018-12-23 22:30:41 +08:00
test: make sure we get PID1's stack trace on ASan/UBSan errors As hitting an ASan/UBSan error in PID1 results in a crash (and a kernel panic when running under qemu), we usually lose the stack trace which makes debugging quite painful. Let's mitigate this by forwarding the stack trace to multiple places - namely to a file and the syslog.
2023-06-23 20:28:30 +08:00
for log in pid1 journald; do
report="$(find "$root" -name "systemd-$log.*san.log*" -exec cat {} \;)"
if [[ -n "$report" ]]; then
printf "%s\n" "$report"
# shellcheck disable=SC2015
[[ "$log" == journald ]] && cat "$root/systemd-journald.out" || :
ret=$((ret+1))
fi
done
tests: look for ASan+UBSan reports in the journal
2018-12-23 22:48:18 +08:00
test: use sed and grep instead of perl This gets rid of the all-but-one remaining uses of perl. I tested the new code on my machine, so I'm fairly confident that it works as expected. install_iscsi() has one majestic perl invocation, but we can't get rid of it easily: it extends the code of tgt-admin to print some list of files. Obviously this only works because tgt-admin is written in perl, and perl will be installed if tgt-admin is installed. install_iscsi() is used in TEST-64-UDEV-STORAGE conditionally if tgtadm is installed, so this can stay as is.
2023-05-08 19:43:03 +08:00
# May 08 13:23:31 H testleak[2907148]: SUMMARY: AddressSanitizer: 4 byte(s) leaked in 1 allocation(s).
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
pids="$(
test: use sed and grep instead of perl This gets rid of the all-but-one remaining uses of perl. I tested the new code on my machine, so I'm fairly confident that it works as expected. install_iscsi() has one majestic perl invocation, but we can't get rid of it easily: it extends the code of tgt-admin to print some list of files. Obviously this only works because tgt-admin is written in perl, and perl will be installed if tgt-admin is installed. install_iscsi() is used in TEST-64-UDEV-STORAGE conditionally if tgtadm is installed, so this can stay as is.
2023-05-08 19:43:03 +08:00
"$JOURNALCTL" -D "$root/var/log/journal" --grep 'SUMMARY: .*Sanitizer:' |
grep -v -E 'dbus-daemon|dbus-broker-launch' |
sed -r -n 's/.* .+\[([0-9]+)\]: SUMMARY:.*/\1/p'
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
)"
test: use sed and grep instead of perl This gets rid of the all-but-one remaining uses of perl. I tested the new code on my machine, so I'm fairly confident that it works as expected. install_iscsi() has one majestic perl invocation, but we can't get rid of it easily: it extends the code of tgt-admin to print some list of files. Obviously this only works because tgt-admin is written in perl, and perl will be installed if tgt-admin is installed. install_iscsi() is used in TEST-64-UDEV-STORAGE conditionally if tgtadm is installed, so this can stay as is.
2023-05-08 19:43:03 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [[ -n "$pids" ]]; then
ret=$((ret+1))
tests: look for ASan+UBSan reports in the journal
2018-12-23 22:48:18 +08:00
for pid in $pids; do
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
"$JOURNALCTL" -D "$root/var/log/journal" _PID="$pid" --no-pager
tests: look for ASan+UBSan reports in the journal
2018-12-23 22:48:18 +08:00
done
tests: ignore memory leaks in dbus-daemon Otherwise, the test fails on Fedora 28 with ``` Jan 30 01:42:35 systemd-testsuite dbus-daemon[61]: [system] Successfully activated service 'org.freedesktop.systemd1' Jan 30 01:42:35 systemd-testsuite systemd[61]: dbus.service: Kernel keyring access prohibited, ignoring. Jan 30 01:42:35 systemd-testsuite systemd[61]: dbus.service: Executing: /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: ================================================================= Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: ==61==ERROR: LeakSanitizer: detected memory leaks Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: Direct leak of 72 byte(s) in 1 object(s) allocated from: Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #0 0x7f21f9e29088 in __interceptor_realloc (/usr/lib64/libasan.so.5+0xef088) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #1 0x7f21f9b1b23c (/lib64/libdbus-1.so.3+0x3323c) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #2 0x240000001b (<unknown module>) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: Direct leak of 16 byte(s) in 1 object(s) allocated from: Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #0 0x7f21f9e29088 in __interceptor_realloc (/usr/lib64/libasan.so.5+0xef088) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #1 0x7f21f9b1b23c (/lib64/libdbus-1.so.3+0x3323c) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #2 0x7ffffffff (<unknown module>) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: SUMMARY: AddressSanitizer: 88 byte(s) leaked in 2 allocation(s). ``` The leaks were reported and fixed in https://bugs.freedesktop.org/show_bug.cgi?id=107320.
2019-01-30 09:19:45 +08:00
fi
tests: reproduce https://github.com/systemd/systemd/issues/11251
2018-12-23 21:46:00 +08:00
fi
test: remove duplicated code
2013-11-06 06:32:56 +08:00
return $ret
}
test: create and merge code coverage reports in integration tests If -Db_coverage=true is used at build time, then ARTIFACT_DIRECTORY/TEST-XX-FOO.coverage-info files are created with code coverage data, and run-integration-test.sh also merges them into ARTIFACT_DIRECTORY/merged.coverage-info since the coveralls.io helpers accept only a single file.
2021-10-03 23:50:38 +08:00
check_coverage_reports() {
local root="${1:?}"
if get_bool "$NO_BUILD"; then
return 0
fi
test: loosen sandbox restrictions for integration tests as well Otherwise we miss quite a lot of coverage (mainly from logind, hostnamed, networkd, and possibly others), since they can't write their reports with `ProtectSystem=strict`.
2021-10-22 00:06:36 +08:00
if ! get_bool "$IS_BUILT_WITH_COVERAGE"; then
test: create and merge code coverage reports in integration tests If -Db_coverage=true is used at build time, then ARTIFACT_DIRECTORY/TEST-XX-FOO.coverage-info files are created with code coverage data, and run-integration-test.sh also merges them into ARTIFACT_DIRECTORY/merged.coverage-info since the coveralls.io helpers accept only a single file.
2021-10-03 23:50:38 +08:00
return 0
fi
if [ -n "${ARTIFACT_DIRECTORY}" ]; then
dest="${ARTIFACT_DIRECTORY}/${testname:?}.coverage-info"
else
dest="${TESTDIR:?}/coverage-info"
fi
test: do an initial coverage capture I noticed that our coverage reports miss some files completely - this happens when the test doesn't touch the code in them at all, so the generated coverage data (and resulting reports) have no information about them. Let's fix this by doing an initial zero coverage capture that contains a zeroed counter for every instrumented line in every file, so when we later merge it with a capture from the test, it shows up with a missing coverage instead of not showing at all.
2023-03-25 04:15:28 +08:00
if [[ ! -e "${TESTDIR:?}/coverage-base" ]]; then
# This shouldn't happen, as the report is generated during the setup
# phase (test_setup()).
derror "Missing base coverage report"
return 1
fi
test: drop generated stuff from the final coverage report Let's drop stuff from the current $BUILD_DIR from the final coverage report, as it's all generated files (mostly gperf) which we don't really care about and it makes the Coveralls report confusing, since it reports "source not available" for all such files.
2023-05-19 17:07:07 +08:00
# Create a coverage report that will later be uploaded. Remove info about system
# libraries/headers and generated files, as we don't really care about them.
test: do an initial coverage capture I noticed that our coverage reports miss some files completely - this happens when the test doesn't touch the code in them at all, so the generated coverage data (and resulting reports) have no information about them. Let's fix this by doing an initial zero coverage capture that contains a zeroed counter for every instrumented line in every file, so when we later merge it with a capture from the test, it shows up with a missing coverage instead of not showing at all.
2023-03-25 04:15:28 +08:00
lcov --directory "${root}/${BUILD_DIR:?}" --capture --output-file "${dest}.new"
test: merge coverage reports from previous test runs Relevant mainly for tests which utilize both QEMU and nspawn.
2021-10-25 17:35:47 +08:00
if [[ -f "$dest" ]]; then
# If the destination report file already exists, don't overwrite it, but
test: do an initial coverage capture I noticed that our coverage reports miss some files completely - this happens when the test doesn't touch the code in them at all, so the generated coverage data (and resulting reports) have no information about them. Let's fix this by doing an initial zero coverage capture that contains a zeroed counter for every instrumented line in every file, so when we later merge it with a capture from the test, it shows up with a missing coverage instead of not showing at all.
2023-03-25 04:15:28 +08:00
# merge it with the already present one - this usually happens when
# running both "parts" of a test in one run (the qemu and the nspawn part).
test: merge coverage reports from previous test runs Relevant mainly for tests which utilize both QEMU and nspawn.
2021-10-25 17:35:47 +08:00
lcov --add-tracefile "${dest}" --add-tracefile "${dest}.new" -o "${dest}"
else
test: do an initial coverage capture I noticed that our coverage reports miss some files completely - this happens when the test doesn't touch the code in them at all, so the generated coverage data (and resulting reports) have no information about them. Let's fix this by doing an initial zero coverage capture that contains a zeroed counter for every instrumented line in every file, so when we later merge it with a capture from the test, it shows up with a missing coverage instead of not showing at all.
2023-03-25 04:15:28 +08:00
# If there's no prior coverage report, merge the new one with the base
# report we did during the setup phase (see test_setup()).
lcov --add-tracefile "${TESTDIR:?}/coverage-base" --add-tracefile "${dest}.new" -o "${dest}"
test: merge coverage reports from previous test runs Relevant mainly for tests which utilize both QEMU and nspawn.
2021-10-25 17:35:47 +08:00
fi
test: drop generated stuff from the final coverage report Let's drop stuff from the current $BUILD_DIR from the final coverage report, as it's all generated files (mostly gperf) which we don't really care about and it makes the Coveralls report confusing, since it reports "source not available" for all such files.
2023-05-19 17:07:07 +08:00
lcov --remove "$dest" -o "$dest" '/usr/include/*' '/usr/lib/*' "${BUILD_DIR:?}/*"
test: do an initial coverage capture I noticed that our coverage reports miss some files completely - this happens when the test doesn't touch the code in them at all, so the generated coverage data (and resulting reports) have no information about them. Let's fix this by doing an initial zero coverage capture that contains a zeroed counter for every instrumented line in every file, so when we later merge it with a capture from the test, it shows up with a missing coverage instead of not showing at all.
2023-03-25 04:15:28 +08:00
rm -f "${dest}.new"
test: create and merge code coverage reports in integration tests If -Db_coverage=true is used at build time, then ARTIFACT_DIRECTORY/TEST-XX-FOO.coverage-info files are created with code coverage data, and run-integration-test.sh also merges them into ARTIFACT_DIRECTORY/merged.coverage-info since the coveralls.io helpers accept only a single file.
2021-10-03 23:50:38 +08:00
test: check for (possibly) missing test coverage If the test logs contain lines like: ``` ...systemd-resolved[735885]: profiling:/systemd-meson-build/src/shared/libsystemd-shared-250.a.p/base-filesystem.c.gcda:Cannot open ``` it means we're possibly missing some coverage since gcov can't write the stats, usually due to the sandbox being too restrictive (e.g. ProtectSystem=yes, ProtectHome=yes) or the $BUILD_DIR being inaccessible to non-root users.
2022-02-14 19:20:02 +08:00
# If the test logs contain lines like:
#
# ...systemd-resolved[735885]: profiling:/systemd-meson-build/src/shared/libsystemd-shared-250.a.p/base-filesystem.c.gcda:Cannot open
#
# it means we're possibly missing some coverage since gcov can't write the stats,
# usually due to the sandbox being too restrictive (e.g. ProtectSystem=yes,
# ProtectHome=yes) or the $BUILD_DIR being inaccessible to non-root users - see
# `setfacl` stuff in install_compiled_systemd().
test: do an initial coverage capture I noticed that our coverage reports miss some files completely - this happens when the test doesn't touch the code in them at all, so the generated coverage data (and resulting reports) have no information about them. Let's fix this by doing an initial zero coverage capture that contains a zeroed counter for every instrumented line in every file, so when we later merge it with a capture from the test, it shows up with a missing coverage instead of not showing at all.
2023-03-25 04:15:28 +08:00
#
# Also, a note: some tests, like TEST-46, overmount /home with tmpfs, which
# means if your build dir is under /home/your-user (which is usually the
tree-wide: code spelling fixes As reported by Fossies.
2023-04-21 03:54:59 +08:00
# case) you might get bogus errors and missing coverage.
test: ignore missing coverage in TEST-02 Since c6552ad we now try to collect coverage even in situations where it's basically impossible (like in test-mount-util where the whole / is mounted as read-only). As dealing with this is not worth the trouble, let's ignore the missing coverage errors thrown by gcov in such cases.
2022-04-07 18:45:13 +08:00
if ! get_bool "${IGNORE_MISSING_COVERAGE:=}" && \
"${JOURNALCTL:?}" -q --no-pager -D "${root:?}/var/log/journal" --grep "profiling:.+?gcda:[Cc]annot open"; then
test: check for (possibly) missing test coverage If the test logs contain lines like: ``` ...systemd-resolved[735885]: profiling:/systemd-meson-build/src/shared/libsystemd-shared-250.a.p/base-filesystem.c.gcda:Cannot open ``` it means we're possibly missing some coverage since gcov can't write the stats, usually due to the sandbox being too restrictive (e.g. ProtectSystem=yes, ProtectHome=yes) or the $BUILD_DIR being inaccessible to non-root users.
2022-02-14 19:20:02 +08:00
derror "Detected possibly missing coverage, check the journal"
return 1
fi
test: create and merge code coverage reports in integration tests If -Db_coverage=true is used at build time, then ARTIFACT_DIRECTORY/TEST-XX-FOO.coverage-info files are created with code coverage data, and run-integration-test.sh also merges them into ARTIFACT_DIRECTORY/merged.coverage-info since the coveralls.io helpers accept only a single file.
2021-10-03 23:50:38 +08:00
return 0
}
test: use a helper function to move the journal files
2020-03-30 22:39:31 +08:00
save_journal() {
test: clean up the save_journal() stuff a bit Let's save all journals from the test machine instead of calling export on each journal file separately, which makes the code less complicated (and probably faster).
2023-11-21 06:34:09 +08:00
local source_dir="${1:?}"
local state="${2:?}"
test: optionally, only save test journal for failing tests Saving the journal for passing tests creates a huge amount of unneeded data stored for each full test run. Add a env var to allow saving the journal only for failed tests.
2021-07-02 22:38:14 +08:00
# Default to always saving journal
local save="yes"
test: clean up the save_journal() stuff a bit Let's save all journals from the test machine instead of calling export on each journal file separately, which makes the code less complicated (and probably faster).
2023-11-21 06:34:09 +08:00
local dest_dir dest_name dest
test: optionally, only save test journal for failing tests Saving the journal for passing tests creates a huge amount of unneeded data stored for each full test run. Add a env var to allow saving the journal only for failed tests.
2021-07-02 22:38:14 +08:00
test: clean up the save_journal() stuff a bit Let's save all journals from the test machine instead of calling export on each journal file separately, which makes the code less complicated (and probably faster).
2023-11-21 06:34:09 +08:00
if [[ "${TEST_SAVE_JOURNAL:-}" == "no" ]]; then
test: optionally, only save test journal for failing tests Saving the journal for passing tests creates a huge amount of unneeded data stored for each full test run. Add a env var to allow saving the journal only for failed tests.
2021-07-02 22:38:14 +08:00
save="no"
test: clean up the save_journal() stuff a bit Let's save all journals from the test machine instead of calling export on each journal file separately, which makes the code less complicated (and probably faster).
2023-11-21 06:34:09 +08:00
elif [[ "${TEST_SAVE_JOURNAL:-}" == "fail" && "$state" -eq 0 ]]; then
test: optionally, only save test journal for failing tests Saving the journal for passing tests creates a huge amount of unneeded data stored for each full test run. Add a env var to allow saving the journal only for failed tests.
2021-07-02 22:38:14 +08:00
save="no"
fi
test: clean up the save_journal() stuff a bit Let's save all journals from the test machine instead of calling export on each journal file separately, which makes the code less complicated (and probably faster).
2023-11-21 06:34:09 +08:00
if [[ -n "${ARTIFACT_DIRECTORY:-}" ]]; then
dest_dir="$ARTIFACT_DIRECTORY"
dest_name="${testname:?}.journal"
test: use systemd-journal-remote for journal copying, support $ARTIFACT_DIRECTORY Using s-j-remote fixes the following issue: when coalescing files from multiple inputs, simply copying all files with into the the same directory might potentially mess things up, because a newer system.journal might overwrite an older journal. This happens because we run multiple tests from the same image, and need to clean out the directory after each run. By using systemd-journal-remote, we nicely coalesce all files. This has the advantage that if there aren't too many logs, we end up with just one journal file. ARTIFACT_DIRECTORY is for ubuntuautopackagetests, where the journal files are copied to a separate directory to preserve after tests have been run. This functionality can now be recreated by setting ARTIFACT_DIRECTORY=$AUTOPKGTEST_ARTIFACTS.
2020-03-31 03:29:52 +08:00
else
test: clean up the save_journal() stuff a bit Let's save all journals from the test machine instead of calling export on each journal file separately, which makes the code less complicated (and probably faster).
2023-11-21 06:34:09 +08:00
dest_dir="${TESTDIR:?}"
dest_name="system.journal"
test: use systemd-journal-remote for journal copying, support $ARTIFACT_DIRECTORY Using s-j-remote fixes the following issue: when coalescing files from multiple inputs, simply copying all files with into the the same directory might potentially mess things up, because a newer system.journal might overwrite an older journal. This happens because we run multiple tests from the same image, and need to clean out the directory after each run. By using systemd-journal-remote, we nicely coalesce all files. This has the advantage that if there aren't too many logs, we end up with just one journal file. ARTIFACT_DIRECTORY is for ubuntuautopackagetests, where the journal files are copied to a separate directory to preserve after tests have been run. This functionality can now be recreated by setting ARTIFACT_DIRECTORY=$AUTOPKGTEST_ARTIFACTS.
2020-03-31 03:29:52 +08:00
fi
test: use a helper function to move the journal files
2020-03-30 22:39:31 +08:00
test: don't redirect stdout/stderr to console Turns out that redirecting a lot of output to the console can have some funny effects, like random kernel soft lockups. I spotted this in various CIs, but it remained almost entirely hidden thanks to `softlockup_panic=1`, until 1a36d2672f which introduced a couple of tests that log quite a lot in a short amount of time. This, in combination with newer kernel version, which, for some reason, seem to be more susceptible to such soft lockups, made the Arch Linux jobs soft lockup quite a lot, see [0]. While debugging this I also noticed that runs which don't redirect stdout/stderr to the console are noticeably faster, e.g.: # TEST-71 nspawn + QEMU (KVM), StandardOutput=journal+console Elapsed (wall clock) time (h:mm:ss or m:ss): 0:24.64 # TEST-71 nspawn + QEMU (KVM), StandardOutput=journal Elapsed (wall clock) time (h:mm:ss or m:ss): 0:17.95 # TEST-71 nspawn + QEMU, StandardOutput=journal+console Elapsed (wall clock) time (h:mm:ss or m:ss): 2:04.70 # TEST-71 nspawn + QEMU, StandardOutput=journal Elapsed (wall clock) time (h:mm:ss or m:ss): 1:44.48 # TEST-04 QEMU, StandardOutput=journal+console Elapsed (wall clock) time (h:mm:ss or m:ss): 4:22.70 # TEST-04 QEMU, StandardOutput=console Elapsed (wall clock) time (h:mm:ss or m:ss): 5:04.67 Given all this, let's effectively revert ba7abf79a5, and dump the testsuite-related journal messages only after the test finishes, so they don't go through the slow console. Resolves: systemd/systemd-centos-ci#660 [0] https://github.com/systemd/systemd-centos-ci/issues/660
2023-11-21 16:37:21 +08:00
# Show messages from the testsuite-XX.service or messages with priority "warning" and higher
echo " --- $source_dir ---"
test: don't abbreviate log messages when dumping the test journal To make debugging test fails easier.
2024-02-16 20:49:50 +08:00
"$JOURNALCTL" --all --no-pager --no-hostname -o short-monotonic -D "$source_dir" \
test: show journal entries matching the test's syslog identifier as well
2023-12-29 20:28:05 +08:00
_SYSTEMD_UNIT="testsuite-${TESTID:?}.service" + SYSLOG_IDENTIFIER="testsuite-$TESTID.sh" + \
PRIORITY=4 + PRIORITY=3 + PRIORITY=2 + PRIORITY=1 + PRIORITY=0
test: optionally print the journal after tests This is hopefully the last bit of functionality from https://salsa.debian.org/systemd-team/systemd/-/blob/debian/master/debian/tests/upstream.
2020-03-31 03:45:21 +08:00
test: clean up the save_journal() stuff a bit Let's save all journals from the test machine instead of calling export on each journal file separately, which makes the code less complicated (and probably faster).
2023-11-21 06:34:09 +08:00
if get_bool "$save"; then
# If we don't have systemd-journal-remote copy all journals from /var/log/journal/
# to $dest_dir/journals/ as is, otherwise merge all journals into a single .journal
# file
if [[ -z "${SYSTEMD_JOURNAL_REMOTE:-}" ]]; then
dest="$dest_dir/journals"
mkdir -p "$dest"
cp -a "$source_dir/*" "$dest/"
else
dest="$dest_dir/$dest_name"
"$SYSTEMD_JOURNAL_REMOTE" -o "$dest" --getter="$JOURNALCTL -o export -D $source_dir"
test: optionally print the journal after tests This is hopefully the last bit of functionality from https://salsa.debian.org/systemd-team/systemd/-/blob/debian/master/debian/tests/upstream.
2020-03-31 03:45:21 +08:00
fi
test: clean up the save_journal() stuff a bit Let's save all journals from the test machine instead of calling export on each journal file separately, which makes the code less complicated (and probably faster).
2023-11-21 06:34:09 +08:00
if [[ -n "${SUDO_USER:-}" ]]; then
setfacl -R -m "user:$SUDO_USER:r-X" "$dest"
fi
test: optionally, only save test journal for failing tests Saving the journal for passing tests creates a huge amount of unneeded data stored for each full test run. Add a env var to allow saving the journal only for failed tests.
2021-07-02 22:38:14 +08:00
test: clean up the save_journal() stuff a bit Let's save all journals from the test machine instead of calling export on each journal file separately, which makes the code less complicated (and probably faster).
2023-11-21 06:34:09 +08:00
# we want to print this sometime later, so save this in a variable
JOURNAL_LIST="$(ls -lR "$dest")"
test: drop default ACL from $TESTDIR This fixes an issue introduced by the commit 954c77c2510c0328fd98354a59f380945752c38c. For some reasons, setting default ACL on $TESTDIR makes TEST-29-PORTABLE fail. Let's drop the default ACL, and set ACL on saved results instead. Fixes #19519.
2021-05-06 14:47:01 +08:00
fi
test: clean up the save_journal() stuff a bit Let's save all journals from the test machine instead of calling export on each journal file separately, which makes the code less complicated (and probably faster).
2023-11-21 06:34:09 +08:00
rm -rf "${source_dir:?}"/*
test: use a helper function to move the journal files
2020-03-30 22:39:31 +08:00
}
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
check_result_common() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local workspace="${1:?}"
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
local ret
if [ -s "$workspace/failed" ]; then
test: properly catch tests error with no /testok or empty /failed When editing this function in 7bf20e48bd7d641a39a14a7feb749b7e8, I couldn't decide whether to initialize ret at the top and only reset it on success, or whether to assign a value in each branch. In the end I did neither ;( So if the test finished without creating any of the result files, we would echo a message, but return "success". But there was bigger confusion with /failed: some tests create it empty, some don't. I think we may want to do away pre-creation of /failed completely, and assume the test failed unless /testok is found. But I'm leaving that for later rework. For now let's just make sure we report return success only if /testok or /skipped is found.
2021-04-29 15:13:12 +08:00
# Non-empty …/failed has highest priority
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
cp -a "$workspace/failed" "${TESTDIR:?}/"
test: drop default ACL from $TESTDIR This fixes an issue introduced by the commit 954c77c2510c0328fd98354a59f380945752c38c. For some reasons, setting default ACL on $TESTDIR makes TEST-29-PORTABLE fail. Let's drop the default ACL, and set ACL on saved results instead. Fixes #19519.
2021-05-06 14:47:01 +08:00
if [ -n "${SUDO_USER}" ]; then
setfacl -m "user:${SUDO_USER:?}:r-X" "${TESTDIR:?}/"failed
fi
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
ret=1
test: make sure test timeout has a higher priority than a pass otherwise we might mark tests where something crashes during shutdown as successful, as happened in one of the recent TEST-01-BASIC runs: ``` testsuite-01.service: About to execute rm -f /failed /testok testsuite-01.service: Forked rm as 606 testsuite-01.service: Executing: rm -f /failed /testoktestsuite-01.service: Changed dead -> start-pre Starting TEST-01-BASIC... ... Child 606 (rm) died (code=exited, status=0/SUCCESS) testsuite-01.service: Child 606 belongs to testsuite-01.service. testsuite-01.service: Control process exited, code=exited, status=0/SUCCESS (success) testsuite-01.service: Got final SIGCHLD for state start-pre. testsuite-01.service: Passing 0 fds to service testsuite-01.service: About to execute sh -e -x -c "systemctl --state=failed --no-legend --no-pager >/failed ; systemctl daemon-reload ; echo OK >/testok" testsuite-01.service: Forked sh as 607 testsuite-01.service: Changed start-pre -> start testsuite-01.service: Executing: sh -e -x -c "systemctl --state=failed --no-legend --no-pager >/failed ; systemctl daemon-reload ; echo OK >/testok"systemd-journald.service: Got notification message from PID 560 (FDSTORE=1)S ... testsuite-01.service: Child 607 belongs to testsuite-01.service. testsuite-01.service: Main process exited, code=exited, status=0/SUCCESS (success) testsuite-01.service: Deactivated successfully. testsuite-01.service: Service will not restart (restart setting) testsuite-01.service: Changed start -> dead testsuite-01.service: Job 207 testsuite-01.service/start finished, result=done [ OK ] Finished TEST-01-BASIC. ... end.service: About to execute /bin/sh -x -c "systemctl poweroff --no-block" end.service: Forked /bin/sh as 623end.service: Executing: /bin/sh -x -c "systemctl poweroff --no-block" ... end.service: Job 213 end.service/start finished, result=canceled Caught <SEGV>, dumped core as pid 624. Freezing execution. CentOS Linux 8 Kernel 4.18.0-305.12.1.el8_4.x86_64 on an x86_64 (ttyS0) H login: qemu-kvm: terminating on signal 15 from pid 80134 (timeout) E: Test timed out after 600s Spawning getter /root/systemd/build/journalctl -o export -D /var/tmp/systemd-test.0UYjAS/root/var/log/journal/ca6031c2491543fe8286c748258df8d1... Finishing after writing 15125 entries Spawning getter /root/systemd/build/journalctl -o export -D /var/tmp/systemd-test.0UYjAS/root/var/log/journal/remote... Finishing after writing 0 entries -rw-r-----. 1 root root 25165824 Aug 20 12:26 /var/tmp/systemd-test.0UYjAS/system.journal TEST-01-BASIC RUN: Basic systemd setup [OK] ...
2021-08-20 20:08:18 +08:00
elif get_bool "$TIMED_OUT"; then
echo "(timeout)" >"${TESTDIR:?}/failed"
ret=2
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
elif [ -e "$workspace/testok" ]; then
# …/testok always counts (but with lower priority than …/failed)
ret=0
elif [ -e "$workspace/skipped" ]; then
# …/skipped always counts (a message is expected)
echo "${TESTNAME:?} was skipped:"
cat "$workspace/skipped"
ret=0
else
test: properly catch tests error with no /testok or empty /failed When editing this function in 7bf20e48bd7d641a39a14a7feb749b7e8, I couldn't decide whether to initialize ret at the top and only reset it on success, or whether to assign a value in each branch. In the end I did neither ;( So if the test finished without creating any of the result files, we would echo a message, but return "success". But there was bigger confusion with /failed: some tests create it empty, some don't. I think we may want to do away pre-creation of /failed completely, and assume the test failed unless /testok is found. But I'm leaving that for later rework. For now let's just make sure we report return success only if /testok or /skipped is found.
2021-04-29 15:13:12 +08:00
echo "(failed; see logs)" >"${TESTDIR:?}/failed"
ret=3
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
fi
test: properly catch tests error with no /testok or empty /failed When editing this function in 7bf20e48bd7d641a39a14a7feb749b7e8, I couldn't decide whether to initialize ret at the top and only reset it on success, or whether to assign a value in each branch. In the end I did neither ;( So if the test finished without creating any of the result files, we would echo a message, but return "success". But there was bigger confusion with /failed: some tests create it empty, some don't. I think we may want to do away pre-creation of /failed completely, and assume the test failed unless /testok is found. But I'm leaving that for later rework. For now let's just make sure we report return success only if /testok or /skipped is found.
2021-04-29 15:13:12 +08:00
check_asan_reports "$workspace" || ret=4
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
test: create and merge code coverage reports in integration tests If -Db_coverage=true is used at build time, then ARTIFACT_DIRECTORY/TEST-XX-FOO.coverage-info files are created with code coverage data, and run-integration-test.sh also merges them into ARTIFACT_DIRECTORY/merged.coverage-info since the coveralls.io helpers accept only a single file.
2021-10-03 23:50:38 +08:00
check_coverage_reports "$workspace" || ret=5
test: optionally, only save test journal for failing tests Saving the journal for passing tests creates a huge amount of unneeded data stored for each full test run. Add a env var to allow saving the journal only for failed tests.
2021-07-02 22:38:14 +08:00
save_journal "$workspace/var/log/journal" $ret
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [ -d "${ARTIFACT_DIRECTORY}" ] && [ -f "$workspace/strace.out" ]; then
cp "$workspace/strace.out" "${ARTIFACT_DIRECTORY}/"
fi
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
test: properly catch tests error with no /testok or empty /failed When editing this function in 7bf20e48bd7d641a39a14a7feb749b7e8, I couldn't decide whether to initialize ret at the top and only reset it on success, or whether to assign a value in each branch. In the end I did neither ;( So if the test finished without creating any of the result files, we would echo a message, but return "success". But there was bigger confusion with /failed: some tests create it empty, some don't. I think we may want to do away pre-creation of /failed completely, and assume the test failed unless /testok is found. But I'm leaving that for later rework. For now let's just make sure we report return success only if /testok or /skipped is found.
2021-04-29 15:13:12 +08:00
if [ ${ret:?} != 0 ] && [ -f "$TESTDIR/failed" ]; then
echo -n "${TESTNAME:?}: "
cat "$TESTDIR/failed"
fi
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
echo "${JOURNAL_LIST:-"No journals were saved"}"
test: properly catch tests error with no /testok or empty /failed When editing this function in 7bf20e48bd7d641a39a14a7feb749b7e8, I couldn't decide whether to initialize ret at the top and only reset it on success, or whether to assign a value in each branch. In the end I did neither ;( So if the test finished without creating any of the result files, we would echo a message, but return "success". But there was bigger confusion with /failed: some tests create it empty, some don't. I think we may want to do away pre-creation of /failed completely, and assume the test failed unless /testok is found. But I'm leaving that for later rework. For now let's just make sure we report return success only if /testok or /skipped is found.
2021-04-29 15:13:12 +08:00
return ${ret:?}
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
}
check_result_nspawn() {
local workspace="${1:?}"
test: correctly propagate return code from check_result hooks
2023-08-11 22:39:30 +08:00
local ret=0
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
test: run the custom check hooks before common checks since we delete the guest journals as part of the save_journal() step in check_result_common(), making journal inaccessible from the custom check hooks.
2022-06-17 23:28:13 +08:00
# Run a test-specific checks if defined by check_result_nspawn_hook()
test: add a couple of hooks for the check_result_*() functions So we don't have to duplicate the whole functions if we need to inject some test-specific checks.
2021-04-27 01:22:42 +08:00
if declare -F check_result_nspawn_hook >/dev/null; then
test: pass the initdir to check_result_{qemu,nspawn} hooks
2022-02-18 21:10:15 +08:00
if ! check_result_nspawn_hook "${workspace}"; then
test: add a couple of hooks for the check_result_*() functions So we don't have to duplicate the whole functions if we need to inject some test-specific checks.
2021-04-27 01:22:42 +08:00
derror "check_result_nspawn_hook() returned with EC > 0"
ret=4
fi
fi
test: correctly propagate return code from check_result hooks
2023-08-11 22:39:30 +08:00
check_result_common "${workspace}" || ret=$?
test: run the custom check hooks before common checks since we delete the guest journals as part of the save_journal() step in check_result_common(), making journal inaccessible from the custom check hooks.
2022-06-17 23:28:13 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
_umount_dir "${initdir:?}"
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
tests: introduce check_asan_reports and use it in check_result_{qemu|nspawn}
2018-12-25 15:31:50 +08:00
return $ret
}
test: Factorize common integration test functions (#6540) All test/TEST* but TEST-02-CRYPTSETUP share the same check_result_qemu() and test_cleanup(), so move them into test_functions and only override them in TEST-02-CRYPTSETUP. Also provide a common test_run() which by default assumes that both QEMU and nspawn tests are run. Particular tests which don't support either need to explicitly opt out by setting $TEST_NO_{QEMU,NSPAWN}. Do it this way around to avoid accidentally forgetting to opt in, and to encourage test authors to at least always support nspawn.
2017-08-04 20:34:14 +08:00
# can be overridden in specific test
check_result_qemu() {
test: correctly propagate return code from check_result hooks
2023-08-11 22:39:30 +08:00
local ret=0
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
mount_initdir
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
test: run the custom check hooks before common checks since we delete the guest journals as part of the save_journal() step in check_result_common(), making journal inaccessible from the custom check hooks.
2022-06-17 23:28:13 +08:00
# Run a test-specific checks if defined by check_result_qemu_hook()
test: add a couple of hooks for the check_result_*() functions So we don't have to duplicate the whole functions if we need to inject some test-specific checks.
2021-04-27 01:22:42 +08:00
if declare -F check_result_qemu_hook >/dev/null; then
test: pass the initdir to check_result_{qemu,nspawn} hooks
2022-02-18 21:10:15 +08:00
if ! check_result_qemu_hook "${initdir:?}"; then
test: add a couple of hooks for the check_result_*() functions So we don't have to duplicate the whole functions if we need to inject some test-specific checks.
2021-04-27 01:22:42 +08:00
derror "check_result_qemu_hook() returned with EC > 0"
ret=4
fi
fi
test: correctly propagate return code from check_result hooks
2023-08-11 22:39:30 +08:00
check_result_common "${initdir:?}" || ret=$?
test: run the custom check hooks before common checks since we delete the guest journals as part of the save_journal() step in check_result_common(), making journal inaccessible from the custom check hooks.
2022-06-17 23:28:13 +08:00
test: pass the initdir to check_result_{qemu,nspawn} hooks
2022-02-18 21:10:15 +08:00
_umount_dir "${initdir:?}"
test: Factorize common integration test functions (#6540) All test/TEST* but TEST-02-CRYPTSETUP share the same check_result_qemu() and test_cleanup(), so move them into test_functions and only override them in TEST-02-CRYPTSETUP. Also provide a common test_run() which by default assumes that both QEMU and nspawn tests are run. Particular tests which don't support either need to explicitly opt out by setting $TEST_NO_{QEMU,NSPAWN}. Do it this way around to avoid accidentally forgetting to opt in, and to encourage test authors to at least always support nspawn.
2017-08-04 20:34:14 +08:00
return $ret
}
test: move custom result checks from TEST-02-UNITTESTS to test-functions
2021-06-22 03:05:47 +08:00
check_result_nspawn_unittests() {
local workspace="${1:?}"
local ret=1
[[ -e "$workspace/testok" ]] && ret=0
if [[ -s "$workspace/failed" ]]; then
ret=$((ret + 1))
echo "=== Failed test log ==="
cat "$workspace/failed"
else
if [[ -s "$workspace/skipped" ]]; then
echo "=== Skipped test log =="
cat "$workspace/skipped"
# We might have only skipped tests - that should not fail the job
ret=0
fi
if [[ -s "$workspace/testok" ]]; then
echo "=== Passed tests ==="
cat "$workspace/testok"
fi
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
get_bool "${TIMED_OUT:=}" && ret=1
test: collect coverage in a couple more codepaths Otherwise we'd miss coverage from TEST-{02,24,61}, since they don't go through the "standard" `check_result_common` hook. Follow-up to 7bf20e48bd7d641a39a14a7feb749b7e8b0fc0f6.
2021-10-20 19:00:37 +08:00
check_coverage_reports "$workspace" || ret=5
test: optionally, only save test journal for failing tests Saving the journal for passing tests creates a huge amount of unneeded data stored for each full test run. Add a env var to allow saving the journal only for failed tests.
2021-07-02 22:38:14 +08:00
save_journal "$workspace/var/log/journal" $ret
test: show saved journal files after TEST-02-UNITTESTS
2023-11-27 08:42:15 +08:00
echo "${JOURNAL_LIST:-"No journals were saved"}"
test: optionally, only save test journal for failing tests Saving the journal for passing tests creates a huge amount of unneeded data stored for each full test run. Add a env var to allow saving the journal only for failed tests.
2021-07-02 22:38:14 +08:00
test: move custom result checks from TEST-02-UNITTESTS to test-functions
2021-06-22 03:05:47 +08:00
_umount_dir "${initdir:?}"
return $ret
}
check_result_qemu_unittests() {
local ret=1
mount_initdir
[[ -e "${initdir:?}/testok" ]] && ret=0
if [[ -s "$initdir/failed" ]]; then
ret=$((ret + 1))
echo "=== Failed test log ==="
cat "$initdir/failed"
else
if [[ -s "$initdir/skipped" ]]; then
echo "=== Skipped test log =="
cat "$initdir/skipped"
# We might have only skipped tests - that should not fail the job
ret=0
fi
if [[ -s "$initdir/testok" ]]; then
echo "=== Passed tests ==="
cat "$initdir/testok"
fi
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
get_bool "${TIMED_OUT:=}" && ret=1
test: collect coverage in a couple more codepaths Otherwise we'd miss coverage from TEST-{02,24,61}, since they don't go through the "standard" `check_result_common` hook. Follow-up to 7bf20e48bd7d641a39a14a7feb749b7e8b0fc0f6.
2021-10-20 19:00:37 +08:00
check_coverage_reports "$initdir" || ret=5
test: optionally, only save test journal for failing tests Saving the journal for passing tests creates a huge amount of unneeded data stored for each full test run. Add a env var to allow saving the journal only for failed tests.
2021-07-02 22:38:14 +08:00
save_journal "$initdir/var/log/journal" $ret
test: show saved journal files after TEST-02-UNITTESTS
2023-11-27 08:42:15 +08:00
echo "${JOURNAL_LIST:-"No journals were saved"}"
test: optionally, only save test journal for failing tests Saving the journal for passing tests creates a huge amount of unneeded data stored for each full test run. Add a env var to allow saving the journal only for failed tests.
2021-07-02 22:38:14 +08:00
test: move custom result checks from TEST-02-UNITTESTS to test-functions
2021-06-22 03:05:47 +08:00
_umount_dir "$initdir"
return $ret
}
test: remove duplicated code
2013-11-06 06:32:56 +08:00
create_rc_local() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Create rc.local"
mkdir -p "${initdir:?}/etc/rc.d"
cat >"$initdir/etc/rc.d/rc.local" <<EOF
treewide: more portable bash shebangs As in 2a5fcfae024ffc370bb780572279f45a1da3f946 and in 3e67e5c9928f8b1e1c5a63def88d53ed1fed12eb using /usr/bin/env allows bash to be looked up in PATH rather than being hard-coded. As with the previous changes the same arguments apply - distributions have scripts to rewrite shebangs on installation and they know what locations to rely on. - For tests/compilation we should rather rely on the user to have setup there PATH correctly. In particular this makes testing from git easier on NixOS where do not provide /bin/bash to improve compose-ability.
2020-03-04 17:35:06 +08:00
#!/usr/bin/env bash
test: remove duplicated code
2013-11-06 06:32:56 +08:00
exit 0
EOF
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
chmod 0755 "$initdir/etc/rc.d/rc.local"
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
install_execs() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
ddebug "Install executables from the service files"
local pkg_config_path="${BUILD_DIR:?}/src/core/"
local systemunitdir userunitdir exe
systemunitdir="$(PKG_CONFIG_PATH="$pkg_config_path" pkg-config --variable=systemdsystemunitdir systemd)"
userunitdir="$(PKG_CONFIG_PATH="$pkg_config_path" pkg-config --variable=systemduserunitdir systemd)"
while read -r exe; do
# some {rc,halt}.local scripts and programs are okay to not exist, the rest should
# also, plymouth is pulled in by rescue.service, but even there the exit code
# is ignored; as it's not present on some distros, don't fail if it doesn't exist
dinfo "Attempting to install $exe (based on unit file reference)"
inst "$exe" || [ "${exe%.local}" != "$exe" ] || [ "${exe%systemd-update-done}" != "$exe" ] || [ "${exe##*/}" == "plymouth" ]
done < <(sed -r -n 's|^Exec[a-zA-Z]*=[@+!-]*([^ ]+).*|\1|gp' "${initdir:?}"/{"$systemunitdir","$userunitdir"}/*.service | sort -u)
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
generate_module_dependencies() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Generate modules dependencies"
if [[ -d "${initdir:?}/lib/modules/${KERNEL_VER:?}" ]] && \
! depmod -a -b "$initdir" "$KERNEL_VER"; then
test: remove duplicated code
2013-11-06 06:32:56 +08:00
dfatal "\"depmod -a $KERNEL_VER\" failed."
exit 1
fi
}
install_depmod_files() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install depmod files"
inst "/lib/modules/${KERNEL_VER:?}/modules.order"
inst "/lib/modules/$KERNEL_VER/modules.builtin"
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
install_plymouth() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install plymouth"
test: remove duplicated code
2013-11-06 06:32:56 +08:00
# install plymouth, if found... else remove plymouth service files
# if [ -x /usr/libexec/plymouth/plymouth-populate-initrd ]; then
# PLYMOUTH_POPULATE_SOURCE_FUNCTIONS="$TEST_BASE_DIR/test-functions" \
# /usr/libexec/plymouth/plymouth-populate-initrd -t $initdir
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
# image_install plymouth plymouthd
test: remove duplicated code
2013-11-06 06:32:56 +08:00
# else
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
rm -f "${initdir:?}"/{usr/lib,lib,etc}/systemd/system/plymouth* "$initdir"/{usr/lib,lib,etc}/systemd/system/*/plymouth*
test: remove duplicated code
2013-11-06 06:32:56 +08:00
# fi
}
test: if haveged is part of initrd it needs to be installed in the image too Otherwise haveged won't survive when switching root from initrd to host making haveged service in host fail.
2021-08-06 22:47:32 +08:00
install_haveged() {
test: make sure to include all haveged unit files Recent versions of haveged relies on haveged-switch-root.service too.
2021-08-31 21:49:43 +08:00
# If haveged is installed, it's probably included in initrd and needs to be
test: if haveged is part of initrd it needs to be installed in the image too Otherwise haveged won't survive when switching root from initrd to host making haveged service in host fail.
2021-08-06 22:47:32 +08:00
# installed in the image too.
if [ -x /usr/sbin/haveged ]; then
dinfo "Install haveged files"
inst /usr/sbin/haveged
test: make sure to include all haveged unit files Recent versions of haveged relies on haveged-switch-root.service too.
2021-08-31 21:49:43 +08:00
for u in /usr/lib/systemd/system/haveged*; do
test: shellcheck-ify test scripts
2021-09-30 02:30:08 +08:00
inst "$u"
test: make sure to include all haveged unit files Recent versions of haveged relies on haveged-switch-root.service too.
2021-08-31 21:49:43 +08:00
done
test: if haveged is part of initrd it needs to be installed in the image too Otherwise haveged won't survive when switching root from initrd to host making haveged service in host fail.
2021-08-06 22:47:32 +08:00
fi
}
test: remove duplicated code
2013-11-06 06:32:56 +08:00
install_ld_so_conf() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install /etc/ld.so.conf*"
cp -a /etc/ld.so.conf* "${initdir:?}/etc"
test: remove duplicated code
2013-11-06 06:32:56 +08:00
ldconfig -r "$initdir"
}
test: move TEST-43-* setup to static files Setup of lingering is dropped. I don't think it's necessary for anything, because the unit is still pulled in from the testsuite service.
2020-03-21 02:05:36 +08:00
install_testuser() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Set up a test user"
test: move TEST-43-* setup to static files Setup of lingering is dropped. I don't think it's necessary for anything, because the unit is still pulled in from the testsuite service.
2020-03-21 02:05:36 +08:00
# create unprivileged user for user manager tests
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
mkdir -p "${initdir:?}/etc/sysusers.d"
cat >"$initdir/etc/sysusers.d/testuser.conf" <<EOF
test: add test cases for journal corruption on btrfs For issue #24150 and #31222.
2024-02-10 15:24:10 +08:00
u testuser 4711 "Test User" /home/testuser /bin/bash
test: move TEST-43-* setup to static files Setup of lingering is dropped. I don't think it's necessary for anything, because the unit is still pulled in from the testsuite service.
2020-03-21 02:05:36 +08:00
EOF
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
mkdir -p "$initdir/home/testuser"
chmod 0700 "$initdir/home/testuser"
chown 4711:4711 "$initdir/home/testuser"
test: move TEST-43-* setup to static files Setup of lingering is dropped. I don't think it's necessary for anything, because the unit is still pulled in from the testsuite service.
2020-03-21 02:05:36 +08:00
}
test: remove duplicated code
2013-11-06 06:32:56 +08:00
install_config_files() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install config files"
test: convert all uses of '|| true' into '|| :' No change in functionality; just use the shorter || :
2019-07-19 08:34:57 +08:00
inst /etc/sysconfig/init || :
test: remove duplicated code
2013-11-06 06:32:56 +08:00
inst /etc/passwd
inst /etc/shadow
test: adapt test-functions for SUSE
2020-07-22 04:14:53 +08:00
inst_any /etc/login.defs /usr/etc/login.defs
test: remove duplicated code
2013-11-06 06:32:56 +08:00
inst /etc/group
inst /etc/shells
test-functions: read /usr/etc/nsswitch.conf if /etc/nsswitch.conf does not exist (#16195) See this for more info why and since when this change is needed: https://build.opensuse.org/request/show/807179.
2020-06-23 13:42:15 +08:00
inst_any /etc/nsswitch.conf /usr/etc/nsswitch.conf
test: convert all uses of '|| true' into '|| :' No change in functionality; just use the shorter || :
2019-07-19 08:34:57 +08:00
inst /etc/pam.conf || :
tests/TEST-50: support the case when /etc/os-release is present We have four legal cases: 1. /usr/lib/os-release exists and /etc/os-release is a symlink to it 2. both exist but /etc/os-release is not a symlink to /usr/lib/os-release 3. only /usr/lib/os-release exists 4. only /etc/os-release exists The generic setup code in test-functions and create-busybox-image didn't handle case 3. The test-specific code in TEST-50 didn't handle 2 (because the general setup code would only install /etc/os-release in the image and grep -f /usr/lib/os-release would not work) and 4 (same reason) and would fail in case 3 in generic setup.
2020-08-27 17:59:45 +08:00
inst_any /etc/os-release /usr/lib/os-release
test: remove duplicated code
2013-11-06 06:32:56 +08:00
inst /etc/localtime
# we want an empty environment
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
: >"${initdir:?}/etc/environment"
: >"$initdir/etc/machine-id"
: >"$initdir/etc/resolv.conf"
test: move TEST-30-ONCLOCKCHANGE setup to static files The two timezone files are now installed in the global setup. I am not too happy about this, but it still seems better than to create a completely separate image just for this.
2019-12-10 20:45:48 +08:00
test: remove duplicated code
2013-11-06 06:32:56 +08:00
# set the hostname
tests: use "H" as the hostname "systemd-testsuite" gets in the way when grepping for "testsuite-*.sh". Also, the name doesn't matter for anything, so let's just use something very short to save space.
2021-05-04 19:54:33 +08:00
echo 'H' >"$initdir/etc/hostname"
test: use systemd.status-unit-format=name in all but one test We want to test the traditional output, but the non-default one is generally much nicer for debugging (and takes less space...).
2020-03-25 20:43:20 +08:00
# let's set up just one image with the traditional verbose output
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [ "${IMAGE_NAME:?}" != "basic" ]; then
mkdir -p "$initdir/etc/systemd/system.conf.d"
echo -e '[Manager]\nStatusUnitFormat=name' >"$initdir/etc/systemd/system.conf.d/status.conf"
test: use systemd.status-unit-format=name in all but one test We want to test the traditional output, but the non-default one is generally much nicer for debugging (and takes less space...).
2020-03-25 20:43:20 +08:00
fi
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
install_basic_tools() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install basic tools"
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
image_install "${BASICTOOLS[@]}"
image_install -o sushell
tests: install ldconfig.real for Debian based distros In Debian based distributions ldconfig is just a shell wrapper around ldconfig.real, for deferring it during package installation.
2015-02-02 07:19:33 +08:00
# in Debian ldconfig is just a shell script wrapper around ldconfig.real
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
image_install -o ldconfig.real
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
install_debug_tools() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install debug tools"
test: make the installation of the debug tools optional in the image
2021-09-22 14:37:34 +08:00
image_install -o "${DEBUGTOOLS[@]}"
test: introduce INTERACTIVE_DEBUG Make the interactive debugging of (particularly QEMU) machines less painful, by replacing the default vt220 TERM with linux one, and by not shutting down the machine after running the test itself.
2019-03-17 00:46:08 +08:00
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$INTERACTIVE_DEBUG"; then
test: introduce INTERACTIVE_DEBUG Make the interactive debugging of (particularly QEMU) machines less painful, by replacing the default vt220 TERM with linux one, and by not shutting down the machine after running the test itself.
2019-03-17 00:46:08 +08:00
# Set default TERM from vt220 to linux, so at least basic key shortcuts work
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local getty_override="${initdir:?}/etc/systemd/system/serial-getty@.service.d"
mkdir -p "$getty_override"
echo -e "[Service]\nEnvironment=TERM=linux" >"$getty_override/default-TERM.conf"
test: resize the terminal automagically with INTERACTIVE_DEBUG=yes
2022-04-18 01:49:17 +08:00
echo 'export TERM=linux' >>"$initdir/etc/profile"
test: introduce INTERACTIVE_DEBUG Make the interactive debugging of (particularly QEMU) machines less painful, by replacing the default vt220 TERM with linux one, and by not shutting down the machine after running the test itself.
2019-03-17 00:46:08 +08:00
test: resize the terminal automagically with INTERACTIVE_DEBUG=yes
2022-04-18 01:49:17 +08:00
if command -v resize >/dev/null; then
image_install resize
echo "resize" >>"$initdir/etc/profile"
fi
test: kill plymouthd after initrd transition if it's still running Until now using the INTERACTIVE_DEBUG=yes stuff together with sanitizers was almost impossible, since the console kept eating up our inputs or not responding at all. After a painful day of debugging I noticed that if we use a shell script in the initrd -> root transition, we might end up with a plymouthd still running, which kept screwing with the tty. E.g. with initrd -> wrapper -> systemd transition, where the `wrapper` is a simple script: ``` exec -- /usr/lib/systemd/systemd "$@" ``` we'd end up with a stray plymouthd process after the bootup: ``` 1 0 440 2 20 0 0 0 worker I ? 0:00 [kworker/5:2-ata_sff] 1 0 453 2 20 0 0 0 worker I ? 0:00 [kworker/9:2-rcu_gp] 5 0 456 1 20 0 7252 1960 do_epo S ? 0:00 @usr/sbin/plymouthd --mode=boot --pid-file=/run/plymouth/pid --attach-to-session ``` After killing it, the tty works finally as expected.
2022-09-06 03:02:25 +08:00
# Sometimes we might end up with plymouthd still running (especially
# with the initrd -> asan_wrapper -> systemd transition), which will eat
# our inputs and make debugging via tty impossible. Let's fix this by
# killing plymouthd explicitly for the interactive sessions.
# Note: we can't use pkill/pidof/etc. here due to a bug in libasan, see:
# - https://github.com/llvm/llvm-project/issues/49223
# - https://bugzilla.redhat.com/show_bug.cgi?id=2098125
local plymouth_unit="${initdir:?}/etc/systemd/system/kill-plymouth.service"
cat >"$plymouth_unit" <<EOF
[Unit]
After=multi-user.target
[Service]
ExecStart=sh -c 'killall --verbose plymouthd || :'
[Install]
WantedBy=multi-user.target
EOF
"${SYSTEMCTL:?}" enable --root "${initdir:?}" kill-plymouth.service
test: introduce INTERACTIVE_DEBUG Make the interactive debugging of (particularly QEMU) machines less painful, by replacing the default vt220 TERM with linux one, and by not shutting down the machine after running the test itself.
2019-03-17 00:46:08 +08:00
fi
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
install_libnss() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install libnss"
test: remove duplicated code
2013-11-06 06:32:56 +08:00
# install libnss_files for login
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
local NSS_LIBS
mapfile -t NSS_LIBS < <(LD_DEBUG=files getent passwd 2>&1 >/dev/null | sed -n '/calling init: .*libnss_/ {s!^.* /!/!; p}')
test: don't fail if we don't need any external nss libs On certain systems the `install_libnss()` function might end up with an empty list of libraries to install, which triggers an assertion in `image_install()`: ``` I: Install libnss ..//test-functions: line 2721: 1: parameter null or not set make: *** [Makefile:4: setup] Error 1 ``` E.g.: ``` # LD_DEBUG=files getent passwd 2>&1 >/dev/null | sed -n '/calling init: .*libnss_/ {s!^.* /!/!; p}' /lib64/libnss_sss.so.2 /lib64/libnss_systemd.so.2 # dnf -y remove sssd-client systemd-libs # LD_DEBUG=files getent passwd 2>&1 >/dev/null | sed -n '/calling init: .*libnss_/ {s!^.* /!/!; p}' <no output> ``` Let's handle this case gracefully.
2022-09-06 20:04:37 +08:00
if [[ ${#NSS_LIBS[@]} -gt 0 ]]; then
image_install "${NSS_LIBS[@]}"
fi
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
install_dbus() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install dbus"
inst "${ROOTLIBDIR:?}/system/dbus.socket"
tests: dbus.service was renamed to dbus-daemon.service in Rawhide Fedora Rawhide renamed dbus.service to dbus-daemon.service - that breaks tests which require working DBus (e.g. TEST-03-JOBS) Excerpt from the dbus.spec: The 'dbus' package is only retained for compatibility purposes. It will eventually be removed and then replaced by 'Provides: dbus' in the dbus-daemon package. It will then exclusively be used for other packages to describe their dependency on a system and user bus. It does not pull in any particular dbus *implementation*, nor any libraries. These should be pulled in, if required, via explicit dependencies.
2018-11-11 19:13:48 +08:00
core: add test case for PrivateUsers=true in user manager The test exercises that PrivateTmp=yes and ProtectHome={read-only,tmpfs} directives work as expected when PrivateUsers=yes in a user manager. Some code is also added to test-functions to help set up test cases that exercise the user manager.
2019-11-14 02:32:24 +08:00
# Newer Fedora versions use dbus-broker by default. Let's install it if it's available.
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [ -f "$ROOTLIBDIR/system/dbus-broker.service" ]; then
inst "$ROOTLIBDIR/system/dbus-broker.service"
test-functions: follow fedora dbus-daemon/dbus-broker rearrangements Apparently dbus.service is now a symlink to either dbus-broker.service or dbus-daemon.service. Copy in the right target service in each case.
2019-04-29 22:09:36 +08:00
inst /usr/bin/dbus-broker
inst /usr/bin/dbus-broker-launch
test: fix dbus installation on Arch Arch finally made dbus-broker the default dbus daemon [0], but unlike Fedora they don't use Alias=dbus.service to make the dbus.symlink under /etc, instead they create the symlink manually under /usr/lib, so let's account for that. [0] https://gitlab.archlinux.org/archlinux/packaging/packages/dbus-broker/-/commit/b24d15795addeb15f9532f28deae9475fad8b9fa
2024-01-11 18:02:05 +08:00
image_install -o {/etc,/usr/lib}/systemd/system/dbus.service
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
elif [ -f "$ROOTLIBDIR/system/dbus-daemon.service" ]; then
test-functions: follow fedora dbus-daemon/dbus-broker rearrangements Apparently dbus.service is now a symlink to either dbus-broker.service or dbus-daemon.service. Copy in the right target service in each case.
2019-04-29 22:09:36 +08:00
# Fedora rawhide replaced dbus.service with dbus-daemon.service
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
inst "$ROOTLIBDIR/system/dbus-daemon.service"
tests: dbus.service was renamed to dbus-daemon.service in Rawhide Fedora Rawhide renamed dbus.service to dbus-daemon.service - that breaks tests which require working DBus (e.g. TEST-03-JOBS) Excerpt from the dbus.spec: The 'dbus' package is only retained for compatibility purposes. It will eventually be removed and then replaced by 'Provides: dbus' in the dbus-daemon package. It will then exclusively be used for other packages to describe their dependency on a system and user bus. It does not pull in any particular dbus *implementation*, nor any libraries. These should be pulled in, if required, via explicit dependencies.
2018-11-11 19:13:48 +08:00
# Alias symlink
test: fix dbus installation on Arch Arch finally made dbus-broker the default dbus daemon [0], but unlike Fedora they don't use Alias=dbus.service to make the dbus.symlink under /etc, instead they create the symlink manually under /usr/lib, so let's account for that. [0] https://gitlab.archlinux.org/archlinux/packaging/packages/dbus-broker/-/commit/b24d15795addeb15f9532f28deae9475fad8b9fa
2024-01-11 18:02:05 +08:00
image_install -o {/etc,/usr/lib}/systemd/system/dbus.service
tests: dbus.service was renamed to dbus-daemon.service in Rawhide Fedora Rawhide renamed dbus.service to dbus-daemon.service - that breaks tests which require working DBus (e.g. TEST-03-JOBS) Excerpt from the dbus.spec: The 'dbus' package is only retained for compatibility purposes. It will eventually be removed and then replaced by 'Provides: dbus' in the dbus-daemon package. It will then exclusively be used for other packages to describe their dependency on a system and user bus. It does not pull in any particular dbus *implementation*, nor any libraries. These should be pulled in, if required, via explicit dependencies.
2018-11-11 19:13:48 +08:00
else
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
inst "$ROOTLIBDIR/system/dbus.service"
tests: dbus.service was renamed to dbus-daemon.service in Rawhide Fedora Rawhide renamed dbus.service to dbus-daemon.service - that breaks tests which require working DBus (e.g. TEST-03-JOBS) Excerpt from the dbus.spec: The 'dbus' package is only retained for compatibility purposes. It will eventually be removed and then replaced by 'Provides: dbus' in the dbus-daemon package. It will then exclusively be used for other packages to describe their dependency on a system and user bus. It does not pull in any particular dbus *implementation*, nor any libraries. These should be pulled in, if required, via explicit dependencies.
2018-11-11 19:13:48 +08:00
fi
test: remove duplicated code
2013-11-06 06:32:56 +08:00
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
while read -r file; do
inst "$file"
done < <(find /etc/dbus-1 /usr/share/dbus-1 -xtype f 2>/dev/null)
test: move TEST-42-EXECSTOPPOST setup to static files
2019-12-10 18:49:39 +08:00
# setup policy for Type=dbus test
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
mkdir -p "${initdir:?}/etc/dbus-1/system.d"
cat >"$initdir/etc/dbus-1/system.d/systemd.test.ExecStopPost.conf" <<EOF
test: move TEST-42-EXECSTOPPOST setup to static files
2019-12-10 18:49:39 +08:00
<?xml version="1.0"?>
<!DOCTYPE busconfig PUBLIC "-//freedesktop//DTD D-BUS Bus Configuration 1.0//EN"
Use https for freedesktop.org grep -l -r http:// | xargs sed -E -i s'#http://(.*).freedesktop.org#https://\1.freedesktop.org#'
2022-06-28 18:00:47 +08:00
"https://www.freedesktop.org/standards/dbus/1.0/busconfig.dtd">
test: move TEST-42-EXECSTOPPOST setup to static files
2019-12-10 18:49:39 +08:00
<busconfig>
<policy user="root">
<allow own="systemd.test.ExecStopPost"/>
</policy>
</busconfig>
EOF
test: bump D-Bus service start timeout if we run without accel The default (25s) doesn't seem to be enough in some cases (especially in VMs without acceleration), causing spurious timeouts: [ 174.297658] dbus-daemon[647]: [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service' requested by ':1.0' (uid=0 pid=645 comm="hostnamectl " label="kernel") [ 184.202313] systemd[1]: systemd-update-utmp-runlevel.service: Consumed 1.253s CPU time. [ 197.335422] systemd[1]: Started dbus.service. [ 199.211468] testsuite-71.sh[639]: + assert_in 'Static hostname: H' '' [ 199.347192] dbus-daemon[647]: [system] Failed to activate service 'org.freedesktop.hostname1': timed out (service_start_timeout=25000ms) [ 199.394879] testsuite-71.sh[657]: + set +ex [ 199.438918] testsuite-71.sh[657]: FAIL: 'Static hostname: H' not found in: [ 200.966006] systemd-logind[631]: Watching system buttons on /dev/input/event0 (Power Button) [ 201.008178] systemd-logind[631]: Watching system buttons on /dev/input/event1 (AT Translated Set 2 keyboard) [ 201.034106] systemd-logind[631]: New seat seat0. [ 201.238267] sh[658]: + systemctl poweroff --no-block [ 201.329890] systemd[1]: Starting systemd-hostnamed.service... [ 202.156622] systemd[1]: systemd-update-utmp-runlevel.service: Deactivated successfully. [ 204.818913] hostnamectl[645]: Failed to query system properties: Connection timed out [ 205.195583] systemd[1]: testsuite-71.service: Main process exited, code=exited, status=1/FAILURE [ 205.227237] systemd[1]: testsuite-71.service: Failed with result 'exit-code'. [ 205.712780] systemd[1]: Failed to start testsuite-71.service.
2023-01-18 01:04:30 +08:00
# If we run without KVM, bump the service start timeout
if ! get_bool "$QEMU_KVM"; then
cat >"$initdir/etc/dbus-1/system.d/service.timeout.conf" <<EOF
<?xml version="1.0"?>
<!DOCTYPE busconfig PUBLIC "-//freedesktop//DTD D-BUS Bus Configuration 1.0//EN"
"https://www.freedesktop.org/standards/dbus/1.0/busconfig.dtd">
<busconfig>
test: bump the D-Bus related timeouts to 120s Let's attempt to reduce the amount of flakes further when the AWS region we run in is under heavy load and the hypervisor stars stealing our CPU time. Follow-up to e0cbb73911 and c78d18215b.
2023-03-09 20:27:57 +08:00
<limit name="service_start_timeout">120000</limit>
test: bump D-Bus service start timeout if we run without accel The default (25s) doesn't seem to be enough in some cases (especially in VMs without acceleration), causing spurious timeouts: [ 174.297658] dbus-daemon[647]: [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service' requested by ':1.0' (uid=0 pid=645 comm="hostnamectl " label="kernel") [ 184.202313] systemd[1]: systemd-update-utmp-runlevel.service: Consumed 1.253s CPU time. [ 197.335422] systemd[1]: Started dbus.service. [ 199.211468] testsuite-71.sh[639]: + assert_in 'Static hostname: H' '' [ 199.347192] dbus-daemon[647]: [system] Failed to activate service 'org.freedesktop.hostname1': timed out (service_start_timeout=25000ms) [ 199.394879] testsuite-71.sh[657]: + set +ex [ 199.438918] testsuite-71.sh[657]: FAIL: 'Static hostname: H' not found in: [ 200.966006] systemd-logind[631]: Watching system buttons on /dev/input/event0 (Power Button) [ 201.008178] systemd-logind[631]: Watching system buttons on /dev/input/event1 (AT Translated Set 2 keyboard) [ 201.034106] systemd-logind[631]: New seat seat0. [ 201.238267] sh[658]: + systemctl poweroff --no-block [ 201.329890] systemd[1]: Starting systemd-hostnamed.service... [ 202.156622] systemd[1]: systemd-update-utmp-runlevel.service: Deactivated successfully. [ 204.818913] hostnamectl[645]: Failed to query system properties: Connection timed out [ 205.195583] systemd[1]: testsuite-71.service: Main process exited, code=exited, status=1/FAILURE [ 205.227237] systemd[1]: testsuite-71.service: Failed with result 'exit-code'. [ 205.712780] systemd[1]: Failed to start testsuite-71.service.
2023-01-18 01:04:30 +08:00
</busconfig>
EOF
test: bump the client-side timeout in sd-bus as well Since c78d18215b D-Bus services now have 60s to start, but the client side (sd-bus) still waits only for 25s before giving up: ``` [ 226.196380] testsuite-71.sh[556]: + assert_in 'Static hostname: H' '' [ 226.332965] testsuite-71.sh[576]: + set +ex [ 226.332965] testsuite-71.sh[576]: FAIL: 'Static hostname: H' not found in: [ 228.910782] sh[577]: + systemctl poweroff --no-block [ 232.255584] hostnamectl[565]: Failed to query system properties: Connection timed out [ 236.827514] systemd[1]: end.service: Consumed 2.131s CPU time. [ 237.476969] dbus-daemon[566]: [system] Successfully activated service 'org.freedesktop.hostname1' [ 237.516308] systemd[1]: system-modprobe.slice: Consumed 1.533s CPU time. [ 237.794635] systemd[1]: testsuite-71.service: Main process exited, code=exited, status=1/FAILURE [ 237.818469] systemd[1]: testsuite-71.service: Failed with result 'exit-code'. [ 237.931415] systemd[1]: Failed to start testsuite-71.service. [ 238.000833] systemd[1]: testsuite-71.service: Consumed 5.651s CPU time. [ 238.181030] systemd[1]: Reached target testsuite.target. ``` Let's override the timeout in sd-bus as well to mitigate this. Follow-up to c78d18215b3e5b0f0896ddb1d0d72c666b5e830b.
2023-01-24 01:40:38 +08:00
# Bump the client-side timeout in sd-bus as well
mkdir -p "$initdir/etc/systemd/system.conf.d"
test: bump the D-Bus related timeouts to 120s Let's attempt to reduce the amount of flakes further when the AWS region we run in is under heavy load and the hypervisor stars stealing our CPU time. Follow-up to e0cbb73911 and c78d18215b.
2023-03-09 20:27:57 +08:00
echo -e '[Manager]\nDefaultEnvironment=SYSTEMD_BUS_TIMEOUT=120' >"$initdir/etc/systemd/system.conf.d/bus-timeout.conf"
test: bump D-Bus service start timeout if we run without accel The default (25s) doesn't seem to be enough in some cases (especially in VMs without acceleration), causing spurious timeouts: [ 174.297658] dbus-daemon[647]: [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service' requested by ':1.0' (uid=0 pid=645 comm="hostnamectl " label="kernel") [ 184.202313] systemd[1]: systemd-update-utmp-runlevel.service: Consumed 1.253s CPU time. [ 197.335422] systemd[1]: Started dbus.service. [ 199.211468] testsuite-71.sh[639]: + assert_in 'Static hostname: H' '' [ 199.347192] dbus-daemon[647]: [system] Failed to activate service 'org.freedesktop.hostname1': timed out (service_start_timeout=25000ms) [ 199.394879] testsuite-71.sh[657]: + set +ex [ 199.438918] testsuite-71.sh[657]: FAIL: 'Static hostname: H' not found in: [ 200.966006] systemd-logind[631]: Watching system buttons on /dev/input/event0 (Power Button) [ 201.008178] systemd-logind[631]: Watching system buttons on /dev/input/event1 (AT Translated Set 2 keyboard) [ 201.034106] systemd-logind[631]: New seat seat0. [ 201.238267] sh[658]: + systemctl poweroff --no-block [ 201.329890] systemd[1]: Starting systemd-hostnamed.service... [ 202.156622] systemd[1]: systemd-update-utmp-runlevel.service: Deactivated successfully. [ 204.818913] hostnamectl[645]: Failed to query system properties: Connection timed out [ 205.195583] systemd[1]: testsuite-71.service: Main process exited, code=exited, status=1/FAILURE [ 205.227237] systemd[1]: testsuite-71.service: Failed with result 'exit-code'. [ 205.712780] systemd[1]: Failed to start testsuite-71.service.
2023-01-18 01:04:30 +08:00
fi
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
core: add test case for PrivateUsers=true in user manager The test exercises that PrivateTmp=yes and ProtectHome={read-only,tmpfs} directives work as expected when PrivateUsers=yes in a user manager. Some code is also added to test-functions to help set up test cases that exercise the user manager.
2019-11-14 02:32:24 +08:00
install_user_dbus() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install user dbus"
test: use pkg-config to get user unit dir when installing dbus user socket Usually on Debian ROOTLIBDIR is /lib/<arch triplet>, which is not the right place. Use pkg-config since we define it, and then fallback to /usr/lib/systemd/user which is the canonical location. On both Debian&friends and Fedora dbus/dbus-broker install the user socket/service under /usr/lib/systemd/user, not /lib/systemd/systemd/user.
2020-12-23 01:47:39 +08:00
local userunitdir
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if ! userunitdir="$(pkg-config --variable=systemduserunitdir systemd)"; then
dwarn "WARNING! Cannot determine userunitdir from pkg-config, assuming /usr/lib/systemd/user"
userunitdir=/usr/lib/systemd/user
test: use pkg-config to get user unit dir when installing dbus user socket Usually on Debian ROOTLIBDIR is /lib/<arch triplet>, which is not the right place. Use pkg-config since we define it, and then fallback to /usr/lib/systemd/user which is the canonical location. On both Debian&friends and Fedora dbus/dbus-broker install the user socket/service under /usr/lib/systemd/user, not /lib/systemd/systemd/user.
2020-12-23 01:47:39 +08:00
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
inst "$userunitdir/dbus.socket"
inst_symlink "$userunitdir/sockets.target.wants/dbus.socket" || inst_symlink /etc/systemd/user/sockets.target.wants/dbus.socket
core: add test case for PrivateUsers=true in user manager The test exercises that PrivateTmp=yes and ProtectHome={read-only,tmpfs} directives work as expected when PrivateUsers=yes in a user manager. Some code is also added to test-functions to help set up test cases that exercise the user manager.
2019-11-14 02:32:24 +08:00
# Append the After= dependency on dbus in case it isn't already set up
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
mkdir -p "${initdir:?}/etc/systemd/system/user@.service.d/"
cat >"$initdir/etc/systemd/system/user@.service.d/dbus.conf" <<EOF
core: add test case for PrivateUsers=true in user manager The test exercises that PrivateTmp=yes and ProtectHome={read-only,tmpfs} directives work as expected when PrivateUsers=yes in a user manager. Some code is also added to test-functions to help set up test cases that exercise the user manager.
2019-11-14 02:32:24 +08:00
[Unit]
After=dbus.service
EOF
# Newer Fedora versions use dbus-broker by default. Let's install it if it's available.
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [ -f "$userunitdir/dbus-broker.service" ]; then
inst "$userunitdir/dbus-broker.service"
test: fix dbus installation on Arch Arch finally made dbus-broker the default dbus daemon [0], but unlike Fedora they don't use Alias=dbus.service to make the dbus.symlink under /etc, instead they create the symlink manually under /usr/lib, so let's account for that. [0] https://gitlab.archlinux.org/archlinux/packaging/packages/dbus-broker/-/commit/b24d15795addeb15f9532f28deae9475fad8b9fa
2024-01-11 18:02:05 +08:00
image_install -o {/etc,/usr/lib}/systemd/user/dbus.service
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
elif [ -f "${ROOTLIBDIR:?}/system/dbus-daemon.service" ]; then
core: add test case for PrivateUsers=true in user manager The test exercises that PrivateTmp=yes and ProtectHome={read-only,tmpfs} directives work as expected when PrivateUsers=yes in a user manager. Some code is also added to test-functions to help set up test cases that exercise the user manager.
2019-11-14 02:32:24 +08:00
# Fedora rawhide replaced dbus.service with dbus-daemon.service
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
inst "$userunitdir/dbus-daemon.service"
core: add test case for PrivateUsers=true in user manager The test exercises that PrivateTmp=yes and ProtectHome={read-only,tmpfs} directives work as expected when PrivateUsers=yes in a user manager. Some code is also added to test-functions to help set up test cases that exercise the user manager.
2019-11-14 02:32:24 +08:00
# Alias symlink
test: fix dbus installation on Arch Arch finally made dbus-broker the default dbus daemon [0], but unlike Fedora they don't use Alias=dbus.service to make the dbus.symlink under /etc, instead they create the symlink manually under /usr/lib, so let's account for that. [0] https://gitlab.archlinux.org/archlinux/packaging/packages/dbus-broker/-/commit/b24d15795addeb15f9532f28deae9475fad8b9fa
2024-01-11 18:02:05 +08:00
image_install -o {/etc,/usr/lib}/systemd/user/dbus.service
core: add test case for PrivateUsers=true in user manager The test exercises that PrivateTmp=yes and ProtectHome={read-only,tmpfs} directives work as expected when PrivateUsers=yes in a user manager. Some code is also added to test-functions to help set up test cases that exercise the user manager.
2019-11-14 02:32:24 +08:00
else
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
inst "$userunitdir/dbus.service"
core: add test case for PrivateUsers=true in user manager The test exercises that PrivateTmp=yes and ProtectHome={read-only,tmpfs} directives work as expected when PrivateUsers=yes in a user manager. Some code is also added to test-functions to help set up test cases that exercise the user manager.
2019-11-14 02:32:24 +08:00
fi
}
test: remove duplicated code
2013-11-06 06:32:56 +08:00
install_pam() {
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
dinfo "Install PAM"
local paths=()
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$LOOKS_LIKE_DEBIAN" && type -p dpkg-architecture &>/dev/null; then
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
paths+=("/lib/$(dpkg-architecture -qDEB_HOST_MULTIARCH)/security")
test: Run qemu/nspawn tests with "set -e" This catches errors like "ninja not found", missing programs etc. early, instead of silently ignoring them and trying to boot a broken VM. In install_config_files(), allow some distro specific files to be absent (such as /etc/sysconfig/init).
2017-08-08 03:09:21 +08:00
else
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
paths+=(/lib*/security)
test: Run qemu/nspawn tests with "set -e" This catches errors like "ninja not found", missing programs etc. early, instead of silently ignoring them and trying to boot a broken VM. In install_config_files(), allow some distro specific files to be absent (such as /etc/sysconfig/init).
2017-08-08 03:09:21 +08:00
fi
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
test: adapt install_pam() for openSUSE limits.conf is installed in /usr/etc/security for openSUSE.
2022-04-11 17:00:07 +08:00
for d in /etc/pam.d /{usr/,}etc/security /usr/{etc,lib}/pam.d; do
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
[ -d "$d" ] && paths+=("$d")
test: remove duplicated code
2013-11-06 06:32:56 +08:00
done
tests: copy /etc/pam.d/systemd-user from the host on Debian, Ubuntu Fixes: systemd-testsuite systemd[34]: PAM _pam_load_conf_file: unable to open /etc/pam.d/system-auth systemd-testsuite systemd[34]: PAM _pam_load_conf_file: unable to open /etc/pam.d/system-auth systemd-testsuite systemd[34]: user@0.service: Failed at step PAM spawning /lib/systemd/systemd: Operation not permitted ... on Debian, Ubuntu
2015-11-17 07:58:20 +08:00
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
while read -r file; do
inst "$file"
done < <(find "${paths[@]}" -xtype f)
tests: fix install_pam * pam_unix depends on unix_chkpwd * selinux=1 -> pam_unix runs unix_chkpwd for root too https://git.fedorahosted.org/cgit/linux-pam.git/tree/modules/pam_unix/passverify.c?id=e89d4c97385ff8180e6e81e84c5aa745daf28a79#n202
2016-01-25 13:00:34 +08:00
# pam_unix depends on unix_chkpwd.
# see http://www.linux-pam.org/Linux-PAM-html/sag-pam_unix.html
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
image_install -o unix_chkpwd
tests: fix install_pam * pam_unix depends on unix_chkpwd * selinux=1 -> pam_unix runs unix_chkpwd for root too https://git.fedorahosted.org/cgit/linux-pam.git/tree/modules/pam_unix/passverify.c?id=e89d4c97385ff8180e6e81e84c5aa745daf28a79#n202
2016-01-25 13:00:34 +08:00
tests: allow root to login with empty password to test images
2016-04-26 02:03:27 +08:00
# set empty root password for easy debugging
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
sed -i 's/^root:x:/root::/' "${initdir:?}/etc/passwd"
test: adapt install_pam() for openSUSE On openSUSE the default pam config files are shipped in /usr/etc/pam.d. Also empty password is not allowed by default.
2021-08-05 21:34:37 +08:00
# And make sure pam_unix will accept it by making sure that
# the PAM module has the nullok option.
for d in /etc/pam.d /usr/{etc,lib}/pam.d; do
[ -d "$initdir/$d" ] || continue
sed -i '/^auth.*pam_unix.so/s/$/ nullok/' "$initdir/$d"/*
done
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
test: install C.UTF-8 and English locales
2022-06-14 08:13:00 +08:00
install_locales() {
# install only C.UTF-8 and English locales
dinfo "Install locales"
if command -v meson >/dev/null \
&& (meson configure "${BUILD_DIR:?}" | grep 'localegen-path */') \
|| get_bool "$LOOKS_LIKE_DEBIAN"; then
# locale-gen support
image_install -o locale-gen localedef
inst /etc/locale.gen || :
inst /usr/share/i18n/SUPPORTED || :
inst_recursive /usr/share/i18n/charmaps
inst_recursive /usr/share/i18n/locales
test: fix locale installation when locale-gen is used locale-gen might merge all compiled locales into a simple archive, so we need to install it as well if necessary.
2022-11-07 18:57:59 +08:00
inst_recursive /usr/share/locale/en*
inst_recursive /usr/share/locale/de*
image_install /usr/share/locale/locale.alias
# locale-gen might either generate each locale separately or merge them
# into a single archive
if ! (inst_recursive /usr/lib/locale/C.*8 /usr/lib/locale/en_*8 ||
image_install /usr/lib/locale/locale-archive); then
dfatal "Failed to install required locales"
exit 1
fi
else
inst_recursive /usr/lib/locale/C.*8 /usr/lib/locale/en_*8
test: install C.UTF-8 and English locales
2022-06-14 08:13:00 +08:00
fi
}
test: shellcheck-ify test scripts
2021-09-30 02:30:08 +08:00
# shellcheck disable=SC2120
test: remove duplicated code
2013-11-06 06:32:56 +08:00
install_keymaps() {
test: add one more path to search keymaps Now it also supports split-usr.
2022-06-14 08:10:00 +08:00
local i p
local -a prefix=(
test: fix keymaps installation on Arch Where the keymaps live under /usr/share/kbd/keymaps/.
2022-11-07 19:07:27 +08:00
"/usr/lib"
"/usr/share"
test: add one more path to search keymaps Now it also supports split-usr.
2022-06-14 08:10:00 +08:00
)
test: make install_keymaps() optionally install more keymaps
2018-09-19 09:54:28 +08:00
test: add one more path to search keymaps Now it also supports split-usr.
2022-06-14 08:10:00 +08:00
dinfo "Install console keymaps"
if (( $# == 0 )); then
for p in "${prefix[@]}"; do
# The first three paths may be deprecated.
# It seems now the last three paths are used by many distributions.
for i in \
test: fix keymaps installation on Arch Where the keymaps live under /usr/share/kbd/keymaps/.
2022-11-07 19:07:27 +08:00
"$p"/kbd/keymaps/include/* \
"$p"/kbd/keymaps/i386/include/* \
"$p"/kbd/keymaps/i386/qwerty/us.* \
"$p"/kbd/keymaps/legacy/include/* \
"$p"/kbd/keymaps/legacy/i386/qwerty/us.* \
"$p"/kbd/keymaps/xkb/us*; do
test: add one more path to search keymaps Now it also supports split-usr.
2022-06-14 08:10:00 +08:00
[[ -f "$i" ]] || continue
inst "$i"
done
done
else
# When it takes any argument, then install more keymaps.
for p in "${prefix[@]}"; do
for i in \
test: fix keymaps installation on Arch Where the keymaps live under /usr/share/kbd/keymaps/.
2022-11-07 19:07:27 +08:00
"$p"/kbd/keymaps/include/* \
"$p"/kbd/keymaps/i386/*/* \
"$p"/kbd/keymaps/legacy/i386/*/* \
"$p"/kbd/keymaps/xkb/*; do
test: add one more path to search keymaps Now it also supports split-usr.
2022-06-14 08:10:00 +08:00
[[ -f "$i" ]] || continue
inst "$i"
done
test: make install_keymaps() optionally install more keymaps
2018-09-19 09:54:28 +08:00
done
fi
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
test: install libxkbcommon and x11 keymaps
2022-06-14 08:12:00 +08:00
install_x11_keymaps() {
dinfo "Install x11 keymaps"
if (( $# == 0 )); then
# Install only keymap list.
inst /usr/share/X11/xkb/rules/base.lst
else
# When it takes any argument, then install all keymaps.
inst_recursive /usr/share/X11/xkb
fi
}
test: introduce install_zoneinfo() But it is not called by default.
2018-09-12 17:20:31 +08:00
install_zoneinfo() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install time zones"
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
inst_any /usr/share/zoneinfo/Asia/Seoul
inst_any /usr/share/zoneinfo/Asia/Vladivostok
inst_any /usr/share/zoneinfo/Australia/Sydney
inst_any /usr/share/zoneinfo/Europe/Berlin
shared/calendarspec: when mktime() moves us backwards, jump forward When trying to calculate the next firing of 'Sun *-*-* 01:00:00', we'd fall into an infinite loop, because mktime() moves us "backwards": Before this patch: tm_within_bounds: good=0 2021-03-29 01:00:00 → 2021-03-29 00:00:00 tm_within_bounds: good=0 2021-03-29 01:00:00 → 2021-03-29 00:00:00 tm_within_bounds: good=0 2021-03-29 01:00:00 → 2021-03-29 00:00:00 ... We rely on mktime() normalizing the time. The man page does not say that it'll move the time forward, but our algorithm relies on this. So let's catch this case explicitly. With this patch: $ TZ=Europe/Dublin faketime 2021-03-21 build/systemd-analyze calendar --iterations=5 'Sun *-*-* 01:00:00' Normalized form: Sun *-*-* 01:00:00 Next elapse: Sun 2021-03-21 01:00:00 GMT (in UTC): Sun 2021-03-21 01:00:00 UTC From now: 59min left Iter. #2: Sun 2021-04-04 01:00:00 IST (in UTC): Sun 2021-04-04 00:00:00 UTC From now: 1 weeks 6 days left <---- note the 2 week jump here Iter. #3: Sun 2021-04-11 01:00:00 IST (in UTC): Sun 2021-04-11 00:00:00 UTC From now: 2 weeks 6 days left Iter. #4: Sun 2021-04-18 01:00:00 IST (in UTC): Sun 2021-04-18 00:00:00 UTC From now: 3 weeks 6 days left Iter. #5: Sun 2021-04-25 01:00:00 IST (in UTC): Sun 2021-04-25 00:00:00 UTC From now: 1 months 4 days left Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1941335.
2021-03-22 19:51:47 +08:00
inst_any /usr/share/zoneinfo/Europe/Dublin
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
inst_any /usr/share/zoneinfo/Europe/Kiev
inst_any /usr/share/zoneinfo/Pacific/Auckland
inst_any /usr/share/zoneinfo/Pacific/Honolulu
inst_any /usr/share/zoneinfo/CET
inst_any /usr/share/zoneinfo/EET
inst_any /usr/share/zoneinfo/UTC
test: introduce install_zoneinfo() But it is not called by default.
2018-09-12 17:20:31 +08:00
}
test: remove duplicated code
2013-11-06 06:32:56 +08:00
install_fonts() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install system fonts"
test: remove duplicated code
2013-11-06 06:32:56 +08:00
for i in \
test: console fonts are located in /usr/share on openSUSE
2023-08-02 01:38:13 +08:00
/usr/{lib,share}/kbd/consolefonts/eurlatgr* \
/usr/{lib,share}/kbd/consolefonts/latarcyrheb-sun16*; do
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -f "$i" ]] || continue
inst "$i"
test: remove duplicated code
2013-11-06 06:32:56 +08:00
done
}
install_terminfo() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install terminfo files"
local terminfodir
for terminfodir in /lib/terminfo /etc/terminfo /usr/share/terminfo; do
[ -f "${terminfodir}/l/linux" ] && break
test: remove duplicated code
2013-11-06 06:32:56 +08:00
done
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
image_install -o "${terminfodir}/l/linux"
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
core: add test case for PrivateUsers=true in user manager The test exercises that PrivateTmp=yes and ProtectHome={read-only,tmpfs} directives work as expected when PrivateUsers=yes in a user manager. Some code is also added to test-functions to help set up test cases that exercise the user manager.
2019-11-14 02:32:24 +08:00
has_user_dbus_socket() {
if [ -f /usr/lib/systemd/user/dbus.socket ] || [ -f /etc/systemd/user/dbus.socket ]; then
return 0
else
echo "Per-user instances are not supported. Skipping..."
return 1
fi
}
test: add shutdown test Wraps nspawn to be able to use pexpect. The test logs in on the console and runs screen. In one screen window it types in shutdown commands and checks whether a wall message was sent to the other.
2021-12-22 18:50:08 +08:00
setup_nspawn_root_hook() { :;}
test: remove duplicated code
2013-11-06 06:32:56 +08:00
setup_nspawn_root() {
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
if [ -z "${initdir}" ]; then
dfatal "\$initdir not defined"
exit 1
fi
test: run tests directly from the loopback device Before, we'd copy the test tree into nspawn-root, and run the tests from there. This is OK, and doesn't actually take much extra time. But it uses quite a lot of extra disk space. So let's make things a bit more efficient by running directly from the image file. We still run the unprivileged nspawn tests from a copy. Once the kernel implements fs shift, we can do away with that too.
2019-12-13 21:21:31 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
rm -rf "${TESTDIR:?}/unprivileged-nspawn-root"
test: perform partial cleanup after each test is run This causes the unprivileged-nspawn-root directory to be removed after running one test. The advantage is that we reduce the maximum disk-space use quite a bit (47*400 MB → about 18GB).
2020-03-21 02:09:35 +08:00
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$RUN_IN_UNPRIVILEGED_CONTAINER"; then
test: run tests directly from the loopback device Before, we'd copy the test tree into nspawn-root, and run the tests from there. This is OK, and doesn't actually take much extra time. But it uses quite a lot of extra disk space. So let's make things a bit more efficient by running directly from the image file. We still run the unprivileged nspawn tests from a copy. Once the kernel implements fs shift, we can do away with that too.
2019-12-13 21:21:31 +08:00
ddebug "cp -ar $initdir $TESTDIR/unprivileged-nspawn-root"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
cp -ar "$initdir" "$TESTDIR/unprivileged-nspawn-root"
tests: also run TEST-01-BASIC in an unprivileged container (#9957) This should make it much easier to catch regressions like https://github.com/systemd/systemd/issues/9914 and https://github.com/systemd/systemd/issues/8535.
2018-08-30 12:01:18 +08:00
fi
test: add shutdown test Wraps nspawn to be able to use pexpect. The test logs in on the console and runs screen. In one screen window it types in shutdown commands and checks whether a wall message was sent to the other.
2021-12-22 18:50:08 +08:00
setup_nspawn_root_hook
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
setup_basic_dirs() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
mkdir -p "${initdir:?}/run"
mkdir -p "$initdir/etc/systemd/system"
mkdir -p "$initdir/var/log/journal"
test: remove duplicated code
2013-11-06 06:32:56 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
for d in usr/bin usr/sbin bin etc lib "${libdir:?}" sbin tmp usr var var/log var/tmp dev proc sys sysroot root run run/lock run/initramfs; do
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
if [ -L "/$d" ]; then
inst_symlink "/$d"
else
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
inst_dir "/$d"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
fi
done
ln -sfn /run "$initdir/var/run"
ln -sfn /run/lock "$initdir/var/lock"
}
test: add function to reduce copied setup boilerplate Many tests were also masking systemd-machined.service. But machined should only start when activated, so having it not masked shouldn't be noticable. TEST-25-IMPORT needs it.
2019-10-08 15:10:12 +08:00
mask_supporting_services() {
# mask some services that we do not want to run in these tests
test: don't mask "supporting" services in TEST-01-BASIC This got lost during one of the code de-duplication attempts.
2021-05-01 03:02:41 +08:00
ln -fsv /dev/null "${initdir:?}/etc/systemd/system/systemd-hwdb-update.service"
ln -fsv /dev/null "$initdir/etc/systemd/system/systemd-journal-catalog-update.service"
ln -fsv /dev/null "$initdir/etc/systemd/system/systemd-networkd.service"
ln -fsv /dev/null "$initdir/etc/systemd/system/systemd-networkd.socket"
ln -fsv /dev/null "$initdir/etc/systemd/system/systemd-resolved.service"
test: add function to reduce copied setup boilerplate Many tests were also masking systemd-machined.service. But machined should only start when activated, so having it not masked shouldn't be noticable. TEST-25-IMPORT needs it.
2019-10-08 15:10:12 +08:00
}
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
inst_libs() {
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
local bin="${1:?}"
local so_regex='([^ ]*/lib[^/]*/[^ ]*\.so[^ ]*)'
local file line
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
while read -r line; do
[[ "$line" = 'not a dynamic executable' ]] && break
test: ignore the error about our own libraries missing during image creation 19:50:59 F: Missing a shared library required by /var/tmp/systemd-test.NIPT2q/root/lib/systemd/libsystemd-core-250.so. 19:50:59 F: Run "ldd /var/tmp/systemd-test.NIPT2q/root/lib/systemd/libsystemd-core-250.so" to find out what it is. 19:50:59 F: libsystemd-shared-250.so => not found 19:50:59 F: Cannot create a test image.
2021-12-15 16:14:44 +08:00
# Ignore errors about our own stuff missing. This is most likely caused
# by ldd attempting to use the unprefixed RPATH.
test: ignore missing libudev when creating the test image Same scenario as with libsystemd - ldd might use unprefixed RPATH, and we install our own stuff into the image unconditionally anyway. Also, bail out early if we hit a missing DSO with a possibly helpful message.
2023-08-05 00:02:01 +08:00
[[ "$line" =~ (libsystemd|libudev).*\ not\ found ]] && continue
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
if [[ "$line" =~ not\ found ]]; then
dfatal "Missing a shared library required by $bin."
dfatal "Run \"ldd $bin\" to find out what it is."
dfatal "$line"
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
dfatal "Cannot create a test image."
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
exit 1
fi
test: ignore missing libudev when creating the test image Same scenario as with libsystemd - ldd might use unprefixed RPATH, and we install our own stuff into the image unconditionally anyway. Also, bail out early if we hit a missing DSO with a possibly helpful message.
2023-08-05 00:02:01 +08:00
if [[ "$line" =~ $so_regex ]]; then
file="${BASH_REMATCH[1]}"
[[ -e "${initdir:?}/$file" ]] && continue
inst_library "$file"
fi
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
done < <(LC_ALL=C ldd "$bin" 2>/dev/null)
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
}
import_testdir() {
test: stop caching loopdev It is more trouble than it is worth. The setup is of a loopback device is very quick, so it's better to always create it when needed and immediately drop afterwards.
2020-03-21 03:17:11 +08:00
# make sure we don't get a stale LOOPDEV value from old times
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local _LOOPDEV="${LOOPDEV:=}"
# We don't want shellcheck to follow & check the $STATEFILE
# shellcheck source=/dev/null
[[ -e "$STATEFILE" ]] && . "$STATEFILE"
LOOPDEV="$_LOOPDEV"
test: don't overwrite TESTDIR if already set
2019-03-13 05:44:25 +08:00
if [[ ! -d "$TESTDIR" ]]; then
if [[ -z "$TESTDIR" ]]; then
test: $STATEDIR should not point to /usr/lib/systemd/tests when NO_BUILD=1 When testing the binaries from the host, make sure to not store the state data below /usr but use a dedicated directory in /var/tmp/ instead. The working directories of the tests, initially located in /var/tmp, are also moved in a dedicated directory /var/tmp/systemd-tests.
2023-03-21 23:44:41 +08:00
TESTDIR="$(mktemp --tmpdir=$WORKDIR -d -t systemd-test.XXXXXX)"
test: don't overwrite TESTDIR if already set
2019-03-13 05:44:25 +08:00
else
mkdir -p "$TESTDIR"
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
cat >"$STATEFILE" <<EOF
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
TESTDIR="$TESTDIR"
EOF
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
export TESTDIR
fi
test: fix public/private image detection and prefer the latter one
2020-05-17 16:46:34 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
IMAGE_PRIVATE="${TESTDIR}/${IMAGE_NAME:?}.img"
IMAGE_PUBLIC="${IMAGESTATEDIR:?}/${IMAGE_NAME}.img"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
}
test: remove duplicated code
2013-11-06 06:32:56 +08:00
import_initdir() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
initdir="${TESTDIR:?}/root"
mkdir -p "$initdir"
test: remove duplicated code
2013-11-06 06:32:56 +08:00
export initdir
}
test: introduce `get_cgroup_hierarchy() helper which returns the host's cgroup hierarchy (unified, hybrid, or legacy).
2022-02-02 03:18:29 +08:00
get_cgroup_hierarchy() {
case "$(stat -c '%T' -f /sys/fs/cgroup)" in
cgroup2fs)
echo "unified"
;;
tmpfs)
if [[ -d /sys/fs/cgroup/unified && "$(stat -c '%T' -f /sys/fs/cgroup/unified)" == cgroup2fs ]]; then
echo "hybrid"
else
echo "legacy"
fi
;;
*)
dfatal "Failed to determine host's cgroup hierarchy"
exit 1
esac
}
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
## @brief Converts numeric logging level to the first letter of level name.
#
# @param lvl Numeric logging level in range from 1 to 6.
# @retval 1 if @a lvl is out of range.
# @retval 0 if @a lvl is correct.
# @result Echoes first letter of level name.
_lvl2char() {
case "$1" in
1) echo F;;
2) echo E;;
3) echo W;;
4) echo I;;
5) echo D;;
6) echo T;;
*) return 1;;
esac
}
## @brief Internal helper function for _do_dlog()
#
# @param lvl Numeric logging level.
# @param msg Message.
# @retval 0 It's always returned, even if logging failed.
#
# @note This function is not supposed to be called manually. Please use
# dtrace(), ddebug(), or others instead which wrap this one.
#
# This function calls _do_dlog() either with parameter msg, or if
# none is given, it will read standard input and will use every line as
# a message.
#
# This enables:
# dwarn "This is a warning"
# echo "This is a warning" | dwarn
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
LOG_LEVEL="${LOG_LEVEL:-4}"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
dlog() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local lvl lvlc
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
[ -z "$LOG_LEVEL" ] && return 0
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
lvl="${1:?}"; shift
[ "$lvl" -le "$LOG_LEVEL" ] || return 0
lvlc="$(_lvl2char "$lvl")" || return 0
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
if [ $# -ge 1 ]; then
echo "$lvlc: $*"
else
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
while read -r line; do
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
echo "$lvlc: " "$line"
done
fi
}
## @brief Logs message at TRACE level (6)
#
# @param msg Message.
# @retval 0 It's always returned, even if logging failed.
dtrace() {
set +x
dlog 6 "$@"
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "${debug:=}"; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
set -x
fi
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
}
## @brief Logs message at DEBUG level (5)
#
# @param msg Message.
# @retval 0 It's always returned, even if logging failed.
ddebug() {
dlog 5 "$@"
}
## @brief Logs message at INFO level (4)
#
# @param msg Message.
# @retval 0 It's always returned, even if logging failed.
dinfo() {
set +x
dlog 4 "$@"
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "${debug:=}"; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
set -x
fi
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
}
## @brief Logs message at WARN level (3)
#
# @param msg Message.
# @retval 0 It's always returned, even if logging failed.
dwarn() {
set +x
dlog 3 "$@"
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "${debug:=}"; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
set -x
fi
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
}
## @brief Logs message at ERROR level (2)
#
# @param msg Message.
# @retval 0 It's always returned, even if logging failed.
derror() {
dlog 2 "$@"
}
## @brief Logs message at FATAL level (1)
#
# @param msg Message.
# @retval 0 It's always returned, even if logging failed.
dfatal() {
set +x
dlog 1 "$@"
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "${debug:=}"; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
set -x
fi
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
}
# Generic substring function. If $2 is in $1, return 0.
test: fix a couple of "new" shellcheck-related issues related to https://github.com/koalaman/shellcheck/wiki/SC2295
2021-11-23 03:51:15 +08:00
strstr() { [ "${1#*"$2"*}" != "$1" ]; }
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
# normalize_path <path>
# Prints the normalized path, where it removes any duplicated
# and trailing slashes.
# Example:
# $ normalize_path ///test/test//
# /test/test
normalize_path() {
shopt -q -s extglob
set -- "${1//+(\/)//}"
shopt -q -u extglob
echo "${1%/}"
}
# convert_abs_rel <from> <to>
# Prints the relative path, when creating a symlink to <to> from <from>.
# Example:
# $ convert_abs_rel /usr/bin/test /bin/test-2
# ../../bin/test-2
# $ ln -s $(convert_abs_rel /usr/bin/test /bin/test-2) /usr/bin/test
convert_abs_rel() {
local __current __absolute __abssize __cursize __newpath
local -i __i __level
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
set -- "$(normalize_path "${1:?}")" "$(normalize_path "${2:?}")"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
# corner case #1 - self looping link
[[ "$1" == "$2" ]] && { echo "${1##*/}"; return; }
# corner case #2 - own dir link
[[ "${1%/*}" == "$2" ]] && { echo "."; return; }
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
IFS="/" read -ra __current <<< "$1"
IFS="/" read -ra __absolute <<< "$2"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
__abssize=${#__absolute[@]}
__cursize=${#__current[@]}
test: move "do" at the end of line
2022-06-14 20:06:06 +08:00
while [[ "${__absolute[__level]}" == "${__current[__level]}" ]]; do
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
(( __level++ ))
if (( __level > __abssize || __level > __cursize ))
then
break
fi
done
test: move "do" at the end of line
2022-06-14 20:06:06 +08:00
for ((__i = __level; __i < __cursize-1; __i++)); do
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
if ((__i > __level))
then
__newpath=$__newpath"/"
fi
__newpath=$__newpath".."
done
test: move "do" at the end of line
2022-06-14 20:06:06 +08:00
for ((__i = __level; __i < __abssize; __i++)); do
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
if [[ -n $__newpath ]]
then
__newpath=$__newpath"/"
fi
__newpath=$__newpath${__absolute[__i]}
done
echo "$__newpath"
}
# Install a directory, keeping symlinks as on the original system.
# Example: if /lib points to /lib64 on the host, "inst_dir /lib/file"
# will create ${initdir}/lib64, ${initdir}/lib64/file,
# and a symlink ${initdir}/lib -> lib64.
inst_dir() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local dir="${1:?}"
local part="${dir%/*}"
local file
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -e "${initdir:?}/${dir}" ]] && return 0 # already there
while [[ "$part" != "${part%/*}" ]] && ! [[ -e "${initdir}/${part}" ]]; do
dir="$part $dir"
part="${part%/*}"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
done
# iterate over parent directories
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
for file in $dir; do
[[ -e "${initdir}/$file" ]] && continue
if [[ -L $file ]]; then
inst_symlink "$file"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
else
# create directory
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
mkdir -m 0755 "${initdir}/$file" || return 1
[[ -e "$file" ]] && chmod --reference="$file" "${initdir}/$file"
chmod u+w "${initdir}/$file"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
fi
done
}
# $1 = file to copy to ramdisk
# $2 (optional) Name for the file on the ramdisk
# Location of the image dir is assumed to be $initdir
# We never overwrite the target if it exists.
inst_simple() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -f "${1:?}" ]] || return 1
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
strstr "$1" "/" || return 1
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local src="$1"
local target="${2:-$1}"
if ! [[ -d ${initdir:?}/$target ]]; then
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
[[ -e ${initdir}/$target ]] && return 0
[[ -L ${initdir}/$target ]] && return 0
[[ -d "${initdir}/${target%/*}" ]] || inst_dir "${target%/*}"
fi
# install checksum files also
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [[ -e "${src%/*}/.${src##*/}.hmac" ]]; then
inst "${src%/*}/.${src##*/}.hmac" "${target%/*}/.${target##*/}.hmac"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
ddebug "Installing $src"
test: copy file xattrs (if any) as well
2023-06-12 19:21:11 +08:00
cp --sparse=always --force --dereference --preserve=all "$src" "${initdir}/$target"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
}
# find symlinks linked to given library file
# $1 = library file
# Function searches for symlinks by stripping version numbers appended to
# library filename, checks if it points to the same target and finally
# prints the list of symlinks to stdout.
#
# Example:
# rev_lib_symlinks libfoo.so.8.1
# output: libfoo.so.8 libfoo.so
# (Only if libfoo.so.8 and libfoo.so exists on host system.)
rev_lib_symlinks() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local fn="${1:?}"
local links=""
local orig
orig="$(readlink -f "$1")"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ "${fn}" =~ .*\.so\..* ]] || return 1
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
until [[ "${fn##*.}" == so ]]; do
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
fn="${fn%.*}"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -L "${fn}" && "$(readlink -f "${fn}")" == "${orig}" ]] && links+=" ${fn}"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
done
echo "${links}"
}
# Same as above, but specialized to handle dynamic libraries.
# It handles making symlinks according to how the original library
# is referenced.
inst_library() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local src="${1:?}"
local dest="${2:-$1}"
local reallib symlink
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
strstr "$1" "/" || return 1
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -e ${initdir:?}/$dest ]] && return 0
if [[ -L $src ]]; then
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
# install checksum files also
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [[ -e "${src%/*}/.${src##*/}.hmac" ]]; then
inst "${src%/*}/.${src##*/}.hmac" "${dest%/*}/.${dest##*/}.hmac"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
reallib="$(readlink -f "$src")"
inst_simple "$reallib" "$reallib"
inst_dir "${dest%/*}"
[[ -d "${dest%/*}" ]] && dest="$(readlink -f "${dest%/*}")/${dest##*/}"
test: be a bit more verbose about shared library symlinks For example: ``` D: Installing /usr/lib64/libz.so.1.2.11 D: Creating symlink /usr/lib64/libz.so.1.2.11 -> /usr/lib64/libz.so.1 D: Creating extra symlink: /lib64/libz.so D: Installing /usr/lib64/libgcc_s-12-20220507.so.1 D: Creating symlink /usr/lib64/libgcc_s-12-20220507.so.1 -> /usr/lib64/libgcc_s.so.1 ```
2022-08-03 00:28:43 +08:00
ddebug "Creating symlink $reallib -> $dest"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
ln -sfn -- "$(convert_abs_rel "${dest}" "${reallib}")" "${initdir}/${dest}"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
else
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
inst_simple "$src" "$dest"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
fi
# Create additional symlinks. See rev_symlinks description.
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
for symlink in $(rev_lib_symlinks "$src") ${reallib:+$(rev_lib_symlinks "$reallib")}; do
if [[ ! -e "$initdir/$symlink" ]]; then
ddebug "Creating extra symlink: $symlink"
inst_symlink "$symlink"
fi
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
done
}
# find a binary. If we were not passed the full path directly,
# search in the usual places to find the binary.
find_binary() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local bin="${1:?}"
if [[ -z ${bin##/*} ]]; then
if [[ -x "$bin" ]] || { strstr "$bin" ".so" && ldd "$bin" &>/dev/null; }; then
echo "$bin"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
return 0
fi
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
type -P "$bin"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
}
# Same as above, but specialized to install binary executables.
# Install binary executable, and all shared library dependencies, if any.
inst_binary() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local bin="${1:?}"
local path target
test: correctly handle installation of newly introduced binaries In certain cases we might attempt to install a binary which is already present in the test image, yet it's missing from the host system. In such cases, let's check if the binary indeed exists in the image before doing any other chcecks. If it does, immediately return with success. This was discovered during installation of /usr/lib/systemd/systemd-bless-boot, which was not present in Ubuntu CI (as the installed systemd was from the Ubuntu repositories), and the binary itself was already in the image thanks to `ninja install`. However, during extraction of binaries from the systemd service files, another attempt to install this binary was made, which failed due to `find_binary` being unable to find it.
2019-07-09 03:05:51 +08:00
# In certain cases we might attempt to install a binary which is already
# present in the test image, yet it's missing from the host system.
# In such cases, let's check if the binary indeed exists in the image
test: ignore the error about our own libraries missing during image creation 19:50:59 F: Missing a shared library required by /var/tmp/systemd-test.NIPT2q/root/lib/systemd/libsystemd-core-250.so. 19:50:59 F: Run "ldd /var/tmp/systemd-test.NIPT2q/root/lib/systemd/libsystemd-core-250.so" to find out what it is. 19:50:59 F: libsystemd-shared-250.so => not found 19:50:59 F: Cannot create a test image.
2021-12-15 16:14:44 +08:00
# before doing any other checks. If it does, immediately return with
test: correctly handle installation of newly introduced binaries In certain cases we might attempt to install a binary which is already present in the test image, yet it's missing from the host system. In such cases, let's check if the binary indeed exists in the image before doing any other chcecks. If it does, immediately return with success. This was discovered during installation of /usr/lib/systemd/systemd-bless-boot, which was not present in Ubuntu CI (as the installed systemd was from the Ubuntu repositories), and the binary itself was already in the image thanks to `ninja install`. However, during extraction of binaries from the systemd service files, another attempt to install this binary was made, which failed due to `find_binary` being unable to find it.
2019-07-09 03:05:51 +08:00
# success.
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [[ $# -eq 1 ]]; then
for path in "" bin sbin usr/bin usr/sbin; do
[[ -e "${initdir:?}${path:+/$path}/${bin}" ]] && return 0
done
fi
test: correctly handle installation of newly introduced binaries In certain cases we might attempt to install a binary which is already present in the test image, yet it's missing from the host system. In such cases, let's check if the binary indeed exists in the image before doing any other chcecks. If it does, immediately return with success. This was discovered during installation of /usr/lib/systemd/systemd-bless-boot, which was not present in Ubuntu CI (as the installed systemd was from the Ubuntu repositories), and the binary itself was already in the image thanks to `ninja install`. However, during extraction of binaries from the systemd service files, another attempt to install this binary was made, which failed due to `find_binary` being unable to find it.
2019-07-09 03:05:51 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
bin="$(find_binary "$bin")" || return 1
target="${2:-$bin}"
[[ -e "${initdir:?}/$target" ]] && return 0
[[ -L "$bin" ]] && inst_symlink "$bin" "$target" && return 0
local file line
local so_regex='([^ ]*/lib[^/]*/[^ ]*\.so[^ ]*)'
test: wrap binaries using systemd DSOs when running w/ ASan Let's detect & wrap binaries which are linked against systemd DSOs and we're running under ASan, since otherwise running such binaries ends with: ``` ==633==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD. ```
2022-06-15 04:54:39 +08:00
# DSOs provided by systemd
local systemd_so_regex='/(libudev|libsystemd.*|.+[\-_]systemd([\-_].+)?|libnss_(mymachines|myhostname|resolve)).so'
local wrap_binary=0
test: enable LSan for certain wrapped binaries So we're able to detect memory leaks in our NSS modules. An example after introducing a memory leak in nss-myhostname.c: testsuite-71.sh[2881]: ================================================================= testsuite-71.sh[2881]: ==2880==ERROR: LeakSanitizer: detected memory leaks testsuite-71.sh[2881]: Direct leak of 2 byte(s) in 1 object(s) allocated from: testsuite-71.sh[2881]: #0 0x7fa28907243b in strdup (/usr/lib64/libasan.so.8.0.0+0x7243b) testsuite-71.sh[2881]: #1 0x7fa286a7bc10 in gethostname_full ../src/basic/hostname-util.c:67 testsuite-71.sh[2881]: #2 0x7fa286a74af9 in gethostname_malloc ../src/basic/hostname-util.h:24 testsuite-71.sh[2881]: #3 0x7fa286a756f4 in _nss_myhostname_gethostbyname4_r ../src/nss-myhostname/nss-myhostname.c:79 testsuite-71.sh[2881]: #4 0x7fa288f17588 in getaddrinfo (/lib64/libc.so.6+0xf4588) testsuite-71.sh[2881]: #5 0x7fa2890a4d93 in __interceptor_getaddrinfo.part.0 (/usr/lib64/libasan.so.8.0.0+0xa4d93) testsuite-71.sh[2881]: #6 0x55a54b2b7159 in ahosts_keys_int.part.0 (/usr/bin/getent.orig+0x4159) testsuite-71.sh[2881]: SUMMARY: AddressSanitizer: 2 byte(s) leaked in 1 allocation(s).
2023-06-28 15:53:13 +08:00
local enable_lsan=0
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
# I love bash!
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
while read -r line; do
[[ "$line" = 'not a dynamic executable' ]] && break
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test: ignore the error about our own libraries missing during image creation 19:50:59 F: Missing a shared library required by /var/tmp/systemd-test.NIPT2q/root/lib/systemd/libsystemd-core-250.so. 19:50:59 F: Run "ldd /var/tmp/systemd-test.NIPT2q/root/lib/systemd/libsystemd-core-250.so" to find out what it is. 19:50:59 F: libsystemd-shared-250.so => not found 19:50:59 F: Cannot create a test image.
2021-12-15 16:14:44 +08:00
# Ignore errors about our own stuff missing. This is most likely caused
# by ldd attempting to use the unprefixed RPATH.
[[ "$line" =~ libsystemd.*\ not\ found ]] && continue
test: wrap binaries using systemd DSOs when running w/ ASan Let's detect & wrap binaries which are linked against systemd DSOs and we're running under ASan, since otherwise running such binaries ends with: ``` ==633==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD. ```
2022-06-15 04:54:39 +08:00
# We're built with ASan and the target binary loads one of the systemd's
# DSOs, so we need to tweak the environment before executing the binary
if get_bool "$IS_BUILT_WITH_ASAN" && [[ "$line" =~ $systemd_so_regex ]]; then
wrap_binary=1
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [[ "$line" =~ $so_regex ]]; then
file="${BASH_REMATCH[1]}"
[[ -e "${initdir}/$file" ]] && continue
inst_library "$file"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
continue
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [[ "$line" =~ not\ found ]]; then
dfatal "Missing a shared library required by $bin."
dfatal "Run \"ldd $bin\" to find out what it is."
dfatal "$line"
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
dfatal "Cannot create a test image."
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
exit 1
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
done < <(LC_ALL=C ldd "$bin" 2>/dev/null)
test: wrap binaries using systemd DSOs when running w/ ASan Let's detect & wrap binaries which are linked against systemd DSOs and we're running under ASan, since otherwise running such binaries ends with: ``` ==633==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD. ```
2022-06-15 04:54:39 +08:00
# Same as above, but we need to wrap certain libraries unconditionally
#
test: set correct group for systemd-journal-upload tests We can't use the systemd-journal-upload user here, since it's created dynamically by DynamicUser=yes. However, we can use the group specified in SupplementaryGroups=, so do exactly that.
2023-12-05 04:33:15 +08:00
# chgrp, chown, getent, login, setfacl, su, useradd, userdel
test: wrap setfacl when running with ASan
2023-05-18 23:30:36 +08:00
# - dlopen() (not only) systemd's PAM modules
test: use setpriv instead of su for user switch from root systemd-repart needs to find mkfs.ext4 for the test. This is located in the directory /usr/sbin on openSUSE Tumbleweed. But since the variable ALWAYS_SET_PATH in /etc/login.defs is set to yes, su re-initializes the $PATH variable and removes /usr/sbin. Hence, mkfs.ext4 is not found and the test fails. Using setpriv instead of su fixes this issue and is more appropriate to do the switch user task from root. [zjs: move setpriv to $BASICTOOLS and force-push to retrigger CI]
2023-03-14 22:21:29 +08:00
# ls, mkfs.*, mksquashfs, mkswap, setpriv, stat
test: wrap setfacl when running with ASan
2023-05-18 23:30:36 +08:00
# - pull in nss_systemd with certain options (like ls -l) when
# nsswitch.conf uses [SUCCESS=merge] (like on Arch Linux)
test: wrap mkfs.*/mksquashfs/mkswap binaries when running w/ ASan
2023-03-16 18:42:12 +08:00
# delv, dig - pull in nss_resolve if `resolve` is in nsswitch.conf
test: wrap binaries using systemd DSOs when running w/ ASan Let's detect & wrap binaries which are linked against systemd DSOs and we're running under ASan, since otherwise running such binaries ends with: ``` ==633==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD. ```
2022-06-15 04:54:39 +08:00
# tar - called by machinectl in TEST-25
test: set correct group for systemd-journal-upload tests We can't use the systemd-journal-upload user here, since it's created dynamically by DynamicUser=yes. However, we can use the group specified in SupplementaryGroups=, so do exactly that.
2023-12-05 04:33:15 +08:00
bin_rx='/(agetty|chgrp|chown|curl|delv|dig|getfacl|getent|id|login|ls|mkfs\.[a-z0-9]+|mksquashfs|mkswap|setfacl|setpriv|stat|su|tar|useradd|userdel)$'
test: wrap mkfs.*/mksquashfs/mkswap binaries when running w/ ASan
2023-03-16 18:42:12 +08:00
if get_bool "$IS_BUILT_WITH_ASAN" && [[ "$bin" =~ $bin_rx ]]; then
test: wrap binaries using systemd DSOs when running w/ ASan Let's detect & wrap binaries which are linked against systemd DSOs and we're running under ASan, since otherwise running such binaries ends with: ``` ==633==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD. ```
2022-06-15 04:54:39 +08:00
wrap_binary=1
test: enable LSan for certain wrapped binaries So we're able to detect memory leaks in our NSS modules. An example after introducing a memory leak in nss-myhostname.c: testsuite-71.sh[2881]: ================================================================= testsuite-71.sh[2881]: ==2880==ERROR: LeakSanitizer: detected memory leaks testsuite-71.sh[2881]: Direct leak of 2 byte(s) in 1 object(s) allocated from: testsuite-71.sh[2881]: #0 0x7fa28907243b in strdup (/usr/lib64/libasan.so.8.0.0+0x7243b) testsuite-71.sh[2881]: #1 0x7fa286a7bc10 in gethostname_full ../src/basic/hostname-util.c:67 testsuite-71.sh[2881]: #2 0x7fa286a74af9 in gethostname_malloc ../src/basic/hostname-util.h:24 testsuite-71.sh[2881]: #3 0x7fa286a756f4 in _nss_myhostname_gethostbyname4_r ../src/nss-myhostname/nss-myhostname.c:79 testsuite-71.sh[2881]: #4 0x7fa288f17588 in getaddrinfo (/lib64/libc.so.6+0xf4588) testsuite-71.sh[2881]: #5 0x7fa2890a4d93 in __interceptor_getaddrinfo.part.0 (/usr/lib64/libasan.so.8.0.0+0xa4d93) testsuite-71.sh[2881]: #6 0x55a54b2b7159 in ahosts_keys_int.part.0 (/usr/bin/getent.orig+0x4159) testsuite-71.sh[2881]: SUMMARY: AddressSanitizer: 2 byte(s) leaked in 1 allocation(s).
2023-06-28 15:53:13 +08:00
# Ugh, so we want to disable LSan in most cases for the wrapped binaries, since
# we don't care about memory leaks in such binaries. However, in certain cases
# the external binary is the only interface for the systemd code, like for
# the systemd NSS modules, where we want to detect memory leaks. So let's
# do another check to decide if we want to enable LSan for given binary.
if [[ "$bin" =~ /getent$ ]]; then
enable_lsan=1
fi
test: wrap binaries using systemd DSOs when running w/ ASan Let's detect & wrap binaries which are linked against systemd DSOs and we're running under ASan, since otherwise running such binaries ends with: ``` ==633==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD. ```
2022-06-15 04:54:39 +08:00
fi
test: don't wrap binaries built with ASan since they should handle loading other instrumented libraries without issues.
2022-06-16 01:43:11 +08:00
# If the target binary is built with ASan support, we don't need to wrap
# it, as it should handle everything by itself
if get_bool "$wrap_binary" && ! is_built_with_asan "$bin"; then
test: wrap binaries using systemd DSOs when running w/ ASan Let's detect & wrap binaries which are linked against systemd DSOs and we're running under ASan, since otherwise running such binaries ends with: ``` ==633==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD. ```
2022-06-15 04:54:39 +08:00
dinfo "Creating ASan-compatible wrapper for binary '$target'"
# Install the target binary with a ".orig" suffix
inst_simple "$bin" "${target}.orig"
# Create a simple shell wrapper in place of the target binary, which
# sets necessary ASan-related env variables and then exec()s the
# suffixed target binary
cat >"$initdir/$target" <<EOF
#!/bin/bash
# Preload the ASan runtime DSO, otherwise ASAn will complain
export LD_PRELOAD="$ASAN_RT_PATH"
# Disable LSan to speed things up, since we don't care about leak reports
# from 'external' binaries
test: enable LSan for certain wrapped binaries So we're able to detect memory leaks in our NSS modules. An example after introducing a memory leak in nss-myhostname.c: testsuite-71.sh[2881]: ================================================================= testsuite-71.sh[2881]: ==2880==ERROR: LeakSanitizer: detected memory leaks testsuite-71.sh[2881]: Direct leak of 2 byte(s) in 1 object(s) allocated from: testsuite-71.sh[2881]: #0 0x7fa28907243b in strdup (/usr/lib64/libasan.so.8.0.0+0x7243b) testsuite-71.sh[2881]: #1 0x7fa286a7bc10 in gethostname_full ../src/basic/hostname-util.c:67 testsuite-71.sh[2881]: #2 0x7fa286a74af9 in gethostname_malloc ../src/basic/hostname-util.h:24 testsuite-71.sh[2881]: #3 0x7fa286a756f4 in _nss_myhostname_gethostbyname4_r ../src/nss-myhostname/nss-myhostname.c:79 testsuite-71.sh[2881]: #4 0x7fa288f17588 in getaddrinfo (/lib64/libc.so.6+0xf4588) testsuite-71.sh[2881]: #5 0x7fa2890a4d93 in __interceptor_getaddrinfo.part.0 (/usr/lib64/libasan.so.8.0.0+0xa4d93) testsuite-71.sh[2881]: #6 0x55a54b2b7159 in ahosts_keys_int.part.0 (/usr/bin/getent.orig+0x4159) testsuite-71.sh[2881]: SUMMARY: AddressSanitizer: 2 byte(s) leaked in 1 allocation(s).
2023-06-28 15:53:13 +08:00
export ASAN_OPTIONS=detect_leaks=$enable_lsan
test: wrap binaries using systemd DSOs when running w/ ASan Let's detect & wrap binaries which are linked against systemd DSOs and we're running under ASan, since otherwise running such binaries ends with: ``` ==633==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD. ```
2022-06-15 04:54:39 +08:00
# Set argv[0] to the original binary name without the ".orig" suffix
exec -a "\$0" -- "${target}.orig" "\$@"
EOF
chmod +x "$initdir/$target"
else
inst_simple "$bin" "$target"
fi
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
}
# same as above, except for shell scripts.
# If your shell script does not start with shebang, it is not a shell script.
inst_script() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local bin line shebang_regex
bin="$(find_binary "${1:?}")" || return 1
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
shift
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
read -r -n 80 line <"$bin"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
# If debug is set, clean unprintable chars to prevent messing up the term
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
get_bool "${debug:=}" && line="$(echo -n "$line" | tr -c -d '[:print:][:space:]')"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
shebang_regex='(#! *)(/[^ ]+).*'
[[ "$line" =~ $shebang_regex ]] || return 1
inst "${BASH_REMATCH[2]}" && inst_simple "$bin" "$@"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
}
# same as above, but specialized for symlinks
inst_symlink() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local src="${1:?}"
local target="${2:-$src}"
local realsrc
strstr "$src" "/" || return 1
[[ -L "$src" ]] || return 1
[[ -L "${initdir:?}/$target" ]] && return 0
realsrc="$(readlink -f "$src")"
if ! [[ -e "$initdir/$realsrc" ]]; then
if [[ -d "$realsrc" ]]; then
inst_dir "$realsrc"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
else
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
inst "$realsrc"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
fi
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ ! -e "$initdir/${target%/*}" ]] && inst_dir "${target%/*}"
[[ -d "${target%/*}" ]] && target="$(readlink -f "${target%/*}")/${target##*/}"
ln -sfn -- "$(convert_abs_rel "${target}" "${realsrc}")" "$initdir/$target"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
}
# attempt to install any programs specified in a udev rule
inst_rule_programs() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local rule="${1:?}"
local prog bin
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test: drop the mawk-incompatible expression The three-argument match() is a GNU AWK extension, thus breaking the compatibility with mawk (used on Ubuntu/Debian, for example). Let's replace it with a (hopefully) more portable sed expression to drop the inadvertently introduced gawk dependency. Fixes: #19957
2021-06-17 20:38:21 +08:00
sed -rn 's/^.*?PROGRAM==?"([^ "]+).*$/\1/p' "$rule" | while read -r prog; do
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [ -x "/lib/udev/$prog" ]; then
bin="/lib/udev/$prog"
else
if ! bin="$(find_binary "$prog")"; then
dinfo "Skipping program $prog used in udev rule $(basename "$rule") as it cannot be found"
continue
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
fi
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
#dinfo "Installing $_bin due to it's use in the udev rule $(basename $1)"
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
image_install "$bin"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
done
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
}
# udev rules always get installed in the same place, so
# create a function to install them to make life simpler.
inst_rules() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local target=/etc/udev/rules.d
local found rule
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
inst_dir "/lib/udev/rules.d"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
inst_dir "$target"
for rule in "$@"; do
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
if [ "${rule#/}" = "$rule" ]; then
for r in /lib/udev/rules.d /etc/udev/rules.d; do
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [[ -f "$r/$rule" ]]; then
found="$r/$rule"
inst_simple "$found"
inst_rule_programs "$found"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
fi
done
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
for r in '' ./; do
if [[ -f "${r}${rule}" ]]; then
found="${r}${rule}"
inst_simple "$found" "$target/${found##*/}"
inst_rule_programs "$found"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
fi
done
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ $found ]] || dinfo "Skipping udev rule: $rule"
found=
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
done
}
# general purpose installation function
# Same args as above.
inst() {
case $# in
1) ;;
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
2)
[[ ! "$initdir" && -d "$2" ]] && export initdir="$2"
[[ "$initdir" = "$2" ]] && set "$1"
;;
3)
[[ -z "$initdir" ]] && export initdir="$2"
set "$1" "$3"
;;
*)
dfatal "inst only takes 1 or 2 or 3 arguments"
exit 1
;;
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
esac
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local fun
for fun in inst_symlink inst_script inst_binary inst_simple; do
"$fun" "$@" && return 0
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
done
test: explicitly report if we fail to install a file into the image
2021-09-22 16:59:52 +08:00
dwarn "Failed to install '$1'"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
return 1
}
# install any of listed files
#
# If first argument is '-d' and second some destination path, first accessible
# source is installed into this path, otherwise it will installed in the same
# path as source. If none of listed files was installed, function return 1.
# On first successful installation it returns with 0 status.
#
# Example:
#
# inst_any -d /bin/foo /bin/bar /bin/baz
#
# Lets assume that /bin/baz exists, so it will be installed as /bin/foo in
tree-wide: use the term "initrd" at most places we so far used "initramfs" In most cases we refernced the concept as "initrd". Let's convert most remaining uses of "initramfs" to "initrd" too, to stay internally consistent. This leaves "initramfs" only where it's relevant to explain historical concepts or where "initramfs" is part of the API (i.e. in /run/initramfs). Follow-up for: b66a6e1a5838b874b789820c090dd6850cf10513
2022-09-23 20:59:02 +08:00
# initrd.
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
inst_any() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local dest file
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ "${1:?}" = '-d' ]] && dest="${2:?}" && shift 2
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
for file in "$@"; do
if [[ -e "$file" ]]; then
[[ -n "$dest" ]] && inst "$file" "$dest" && return 0
inst "$file" && return 0
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
fi
done
return 1
}
test: introduce inst_recursive() helper function
2022-06-14 08:11:00 +08:00
inst_recursive() {
local p item
for p in "$@"; do
test: don't ignore non-existent paths in inst_recursive() The process substitution in the while loop hides errors raised by the find utility, which might (and did), in turn, hide errors in test setup.
2022-11-07 18:55:29 +08:00
# Make sure the source exists, as the process substitution below
# suppresses errors
stat "$p" >/dev/null || return 1
test: introduce inst_recursive() helper function
2022-06-14 08:11:00 +08:00
while read -r item; do
if [[ -d "$item" ]]; then
inst_dir "$item"
elif [[ -f "$item" ]]; then
inst_simple "$item"
fi
done < <(find "$p" 2>/dev/null)
done
}
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
# image_install [-o ] <file> [<file> ... ]
# Install <file> to the test image
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
# -o optionally install the <file> and don't fail, if it is not there
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
image_install() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local optional=no
local prog="${1:?}"
if [[ "$prog" = '-o' ]]; then
optional=yes
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
shift
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
for prog in "$@"; do
if ! inst "$prog" ; then
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$optional"; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Skipping program $prog as it cannot be found and is" \
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
"flagged to be optional"
else
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dfatal "Failed to install $prog"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
exit 1
fi
fi
done
}
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
# Install a single kernel module along with any firmware it may require.
# $1 = full path to kernel module to install
install_kmod_with_fw() {
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
local module="${1:?}"
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
# no need to go further if the module is already installed
test: fix a couple of "new" shellcheck-related issues related to https://github.com/koalaman/shellcheck/wiki/SC2295
2021-11-23 03:51:15 +08:00
[[ -e "${initdir:?}/lib/modules/${KERNEL_VER:?}/${module##*"/lib/modules/$KERNEL_VER/"}" ]] && return 0
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
[[ -e "$initdir/.kernelmodseen/${module##*/}" ]] && return 0
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
[ -d "$initdir/.kernelmodseen" ] && : >"$initdir/.kernelmodseen/${module##*/}"
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
test: fix a couple of "new" shellcheck-related issues related to https://github.com/koalaman/shellcheck/wiki/SC2295
2021-11-23 03:51:15 +08:00
inst_simple "$module" "/lib/modules/$KERNEL_VER/${module##*"/lib/modules/$KERNEL_VER/"}" || return $?
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
local modname="${module##*/}"
local fwdir found fw
modname="${modname%.ko*}"
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
while read -r fw; do
found=
for fwdir in /lib/firmware/updates /lib/firmware; do
if [[ -d "$fwdir" && -f "$fwdir/$fw" ]]; then
inst_simple "$fwdir/$fw" "/lib/firmware/$fw"
found=yes
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
fi
done
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if ! get_bool "$found"; then
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
if ! grep -qe "\<${modname//-/_}\>" /proc/modules; then
dinfo "Possible missing firmware \"${fw}\" for kernel module" \
"\"${modname}.ko\""
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
else
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
dwarn "Possible missing firmware \"${fw}\" for kernel module" \
"\"${modname}.ko\""
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
fi
fi
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
done < <(modinfo -k "$KERNEL_VER" -F firmware "$module" 2>/dev/null)
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
return 0
}
# Do something with all the dependencies of a kernel module.
# Note that kernel modules depend on themselves using the technique we use
# $1 = function to call for each dependency we find
# It will be passed the full path to the found kernel module
# $2 = module to get dependencies for
# rest of args = arguments to modprobe
for_each_kmod_dep() {
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
local func="${1:?}"
local kmod="${2:?}"
local found=0
local cmd modpath
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
shift 2
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
while read -r cmd modpath _; do
[[ "$cmd" = insmod ]] || continue
"$func" "$modpath" || return $?
found=1
done < <(modprobe "$@" --ignore-install --show-depends "$kmod")
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
! get_bool "$found" && return 1
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
return 0
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
}
# instmods [-c] <kernel module> [<kernel module> ... ]
# instmods [-c] <kernel subsystem>
# install kernel modules along with all their dependencies.
# <kernel subsystem> can be e.g. "=block" or "=drivers/usb/storage"
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
# FIXME(?): dracutdevs/dracut@f4e38c0da8d6bf3764c1ad753d9d52aef63050e5
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
instmods() {
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
local check=no
if [[ $# -ge 0 && "$1" = '-c' ]]; then
check=yes
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
shift
fi
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
inst1mod() {
local mod="${1:?}"
local ret=0
local mod_dir="/lib/modules/${KERNEL_VER:?}/"
case "$mod" in
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
=*)
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
if [ -f "${mod_dir}/modules.${mod#=}" ]; then
(
[[ "$mpargs" ]] && echo "$mpargs"
cat "${mod_dir}/modules.${mod#=}"
) | instmods
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
else
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
(
[[ "$mpargs" ]] && echo "$mpargs"
test: attempt to install only kernel modules for each subsystem When `linux-headers` is installed on Arch Linux, it stores the module source tree in the kernel module directory, which is then picked up by `find` and we get a lot of harmless but annoying errors: ``` ... modprobe: FATAL: Module Kconfig.iosched not found in directory /lib/modules/5.13.7-arch1-1 modprobe: FATAL: Module Kconfig not found in directory /lib/modules/5.13.7-arch1-1 modprobe: FATAL: Module Kconfig not found in directory /lib/modules/5.13.7-arch1-1 modprobe: FATAL: Module dm-mpath.h not found in directory /lib/modules/5.13.7-arch1-1 modprobe: FATAL: Module dm-bio-prison-v2.h not found in directory /lib/modules/5.13.7-arch1-1 modprobe: FATAL: Module raid0.h not found in directory /lib/modules/5.13.7-arch1-1 ... ``` Let's fix this by trying to install only kernel modules (*.ko files with an optional compression).
2021-08-08 19:18:18 +08:00
find "$mod_dir" -path "*/${mod#=}/*" -name "*.ko*" -type f -printf '%f\n'
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
) | instmods
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
fi
;;
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
--*)
mpargs+=" $mod"
;;
i2o_scsi)
# Do not load this diagnostic-only module
return
;;
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
*)
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
mod=${mod##*/}
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
# if we are already installed, skip this module and go on
# to the next one.
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
[[ -f "${initdir:?}/.kernelmodseen/${mod%.ko}.ko" ]] && return
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
# We use '-d' option in modprobe only if modules prefix path
# differs from default '/'. This allows us to use Dracut with
# old version of modprobe which doesn't have '-d' option.
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
local mod_dirname=${mod_dir%%/lib/modules/*}
[[ -n ${mod_dirname} ]] && mod_dirname="-d ${mod_dirname}/"
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
# ok, load the module, all its dependencies, and any firmware
# it may require
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
for_each_kmod_dep install_kmod_with_fw "$mod" \
--set-version "$KERNEL_VER" \
${mod_dirname:+"$mod_dirname"} \
${mpargs:+"$mpargs"}
((ret+=$?))
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
;;
esac
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
return "$ret"
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
}
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
local mod mpargs
if [[ $# -eq 0 ]]; then # filenames from stdin
while read -r mod; do
if ! inst1mod "${mod%.ko*}" && [ "$check" = "yes" ]; then
dfatal "Failed to install $mod"
return 1
fi
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
done
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
fi
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
for mod in "$@"; do # filenames as arguments
if ! inst1mod "${mod%.ko*}" && [ "$check" = "yes" ]; then
dfatal "Failed to install $mod"
return 1
fi
done
return 0
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
}
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test/test-functions: avoid stderr noise, only umount on cleanup if mountpoint Only umount it during cleanup if the $TESTDIR/root dir is a mountpoint. This avoids adding noise to the stderr log such as: mountpoint: /var/tmp/systemd-test.waLOFT/root: No such file or directory
2019-08-13 19:50:59 +08:00
_umount_dir() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local mountpoint="${1:?}"
if mountpoint -q "$mountpoint"; then
ddebug "umount $mountpoint"
umount "$mountpoint"
test/test-functions: avoid stderr noise, only umount on cleanup if mountpoint Only umount it during cleanup if the $TESTDIR/root dir is a mountpoint. This avoids adding noise to the stderr log such as: mountpoint: /var/tmp/systemd-test.waLOFT/root: No such file or directory
2019-08-13 19:50:59 +08:00
fi
}
test: add create_empty_image_rootdir() to simplify testcase setup Almost all tests were manually mounting/unmounting $TESTDIR/root from the loopback image; this moves all that into test-functions so the test setup functions are simplier. Also add test_setup_cleanup() function, to cleanup what is mounted by create_empty_image_rootdir()
2019-07-12 23:47:26 +08:00
# can be overridden in specific test
test_setup_cleanup() {
test: run tests directly from the loopback device Before, we'd copy the test tree into nspawn-root, and run the tests from there. This is OK, and doesn't actually take much extra time. But it uses quite a lot of extra disk space. So let's make things a bit more efficient by running directly from the image file. We still run the unprivileged nspawn tests from a copy. Once the kernel implements fs shift, we can do away with that too.
2019-12-13 21:21:31 +08:00
cleanup_initdir
test: add create_empty_image_rootdir() to simplify testcase setup Almost all tests were manually mounting/unmounting $TESTDIR/root from the loopback image; this moves all that into test-functions so the test setup functions are simplier. Also add test_setup_cleanup() function, to cleanup what is mounted by create_empty_image_rootdir()
2019-07-12 23:47:26 +08:00
}
_test_cleanup() {
test: ignore errors during test cleanup, so cleanup can finish Also move TESTDIR and STATEFILE removal into test_cleanup
2019-07-13 02:09:48 +08:00
# (post-test) cleanup should always ignore failure and cleanup as much as possible
(
set +e
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -n "$initdir" ]] && _umount_dir "$initdir"
[[ -n "$IMAGE_PUBLIC" ]] && rm -vf "$IMAGE_PUBLIC"
test: fix 'make clean' not removing shared image
2021-01-28 21:26:35 +08:00
# If multiple setups/cleans are ran in parallel, this can cause a race
test: shellcheck-ify test scripts
2021-09-30 02:30:08 +08:00
if [[ -n "$IMAGESTATEDIR" && $TEST_PARALLELIZE -ne 1 ]]; then
test: fix 'make clean' not removing shared image
2021-01-28 21:26:35 +08:00
rm -vf "${IMAGESTATEDIR}/default.img"
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -n "$TESTDIR" ]] && rm -vfr "$TESTDIR"
[[ -n "$STATEFILE" ]] && rm -vf "$STATEFILE"
test: convert all uses of '|| true' into '|| :' No change in functionality; just use the shorter || :
2019-07-19 08:34:57 +08:00
) || :
test: add create_empty_image_rootdir() to simplify testcase setup Almost all tests were manually mounting/unmounting $TESTDIR/root from the loopback image; this moves all that into test-functions so the test setup functions are simplier. Also add test_setup_cleanup() function, to cleanup what is mounted by create_empty_image_rootdir()
2019-07-12 23:47:26 +08:00
}
test: Factorize common integration test functions (#6540) All test/TEST* but TEST-02-CRYPTSETUP share the same check_result_qemu() and test_cleanup(), so move them into test_functions and only override them in TEST-02-CRYPTSETUP. Also provide a common test_run() which by default assumes that both QEMU and nspawn tests are run. Particular tests which don't support either need to explicitly opt out by setting $TEST_NO_{QEMU,NSPAWN}. Do it this way around to avoid accidentally forgetting to opt in, and to encourage test authors to at least always support nspawn.
2017-08-04 20:34:14 +08:00
# can be overridden in specific test
test_cleanup() {
test: add create_empty_image_rootdir() to simplify testcase setup Almost all tests were manually mounting/unmounting $TESTDIR/root from the loopback image; this moves all that into test-functions so the test setup functions are simplier. Also add test_setup_cleanup() function, to cleanup what is mounted by create_empty_image_rootdir()
2019-07-12 23:47:26 +08:00
_test_cleanup
test: Factorize common integration test functions (#6540) All test/TEST* but TEST-02-CRYPTSETUP share the same check_result_qemu() and test_cleanup(), so move them into test_functions and only override them in TEST-02-CRYPTSETUP. Also provide a common test_run() which by default assumes that both QEMU and nspawn tests are run. Particular tests which don't support either need to explicitly opt out by setting $TEST_NO_{QEMU,NSPAWN}. Do it this way around to avoid accidentally forgetting to opt in, and to encourage test authors to at least always support nspawn.
2017-08-04 20:34:14 +08:00
}
test: perform partial cleanup after each test is run This causes the unprivileged-nspawn-root directory to be removed after running one test. The advantage is that we reduce the maximum disk-space use quite a bit (47*400 MB → about 18GB).
2020-03-21 02:09:35 +08:00
test_cleanup_again() {
[ -n "$TESTDIR" ] || return
rm -rf "$TESTDIR/unprivileged-nspawn-root"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -n "$initdir" ]] && _umount_dir "$initdir"
test: delete private images on clean-again Private images are not reused, they are unique to tests, so delete them as they take a lot of disk space, and we are starting to run in /var/tmp space issues on the Ubuntu CI
2024-03-20 20:55:02 +08:00
# Test specific images are not reused, so delete them or we run out of disk space
if [[ -n "$IMAGE_PUBLIC" ]] && [ "$(basename "$IMAGE_PUBLIC")" != "default.img" ]; then
rm -vf "$IMAGE_PUBLIC"
fi
if [[ -n "$IMAGE_PRIVATE" ]] && [ "$(basename "$IMAGE_PRIVATE")" != "default.img" ]; then
rm -vf "$IMAGE_PRIVATE"
fi
test: perform partial cleanup after each test is run This causes the unprivileged-nspawn-root directory to be removed after running one test. The advantage is that we reduce the maximum disk-space use quite a bit (47*400 MB → about 18GB).
2020-03-21 02:09:35 +08:00
}
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
test_create_image() {
test: de-duplicate test_setup() functions Now we will run setup_nspawn_root() in some case where do don't need to.
2019-12-10 23:40:31 +08:00
create_empty_image_rootdir
# Create what will eventually be our root filesystem onto an overlay
(
LOG_LEVEL=5
setup_basic_environment
)
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
}
test_setup() {
test: don't check for -Dinstall-tests=true with NO_BUILD=1
2023-12-14 22:06:46 +08:00
if ! get_bool "$NO_BUILD" && \
get_bool "${TEST_REQUIRE_INSTALL_TESTS:?}" && \
command -v meson >/dev/null && \
[[ "$(meson configure "${BUILD_DIR:?}" | grep install-tests | awk '{ print $2 }')" != "true" ]]; then
test: print path to build directory on misconfiguration This way it's easier to see when a wrong $BUILD_DIR was passed.
2020-03-31 21:13:13 +08:00
dfatal "$BUILD_DIR needs to be built with -Dinstall-tests=true"
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
exit 1
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [ -e "${IMAGE_PRIVATE:?}" ]; then
echo "Reusing existing image $IMAGE_PRIVATE → $(realpath "$IMAGE_PRIVATE")"
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
mount_initdir
test: create public images by default Let's create new images public by default and then symlink/copy them into the respective private directories afterwards, not the other way around. This should fix a nasty race condition in parallel runs where one tests attempts to copy the backing public image at the same moment another test is already modifying it.
2020-05-26 04:48:01 +08:00
else
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [ ! -e "${IMAGE_PUBLIC:?}" ]; then
tests: build the image once and then copy/extend it Building custom images for each test takes a lot of time. Build the default one, and if the test needs incompatible changes just copy it and extend it instead.
2021-01-07 05:42:28 +08:00
# default.img is the base that every test uses and optionally appends to
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [ ! -e "${IMAGESTATEDIR:?}/default.img" ] || [ -n "${TEST_FORCE_NEWIMAGE:=}" ]; then
tests: build the image once and then copy/extend it Building custom images for each test takes a lot of time. Build the default one, and if the test needs incompatible changes just copy it and extend it instead.
2021-01-07 05:42:28 +08:00
# Create the backing public image, but then completely unmount
# it and drop the loopback device responsible for it, since we're
# going to symlink/copy the image and mount it again from
# elsewhere.
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local image_old="${IMAGE_PUBLIC}"
tests: build the image once and then copy/extend it Building custom images for each test takes a lot of time. Build the default one, and if the test needs incompatible changes just copy it and extend it instead.
2021-01-07 05:42:28 +08:00
if [ -z "${TEST_FORCE_NEWIMAGE}" ]; then
IMAGE_PUBLIC="${IMAGESTATEDIR}/default.img"
fi
test_create_image
test_setup_cleanup
umount_loopback
cleanup_loopdev
IMAGE_PUBLIC="${image_old}"
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if [ "${IMAGE_NAME:?}" != "default" ] && ! get_bool "${TEST_FORCE_NEWIMAGE}"; then
tests: build the image once and then copy/extend it Building custom images for each test takes a lot of time. Build the default one, and if the test needs incompatible changes just copy it and extend it instead.
2021-01-07 05:42:28 +08:00
cp -v "$(realpath "${IMAGESTATEDIR}/default.img")" "$IMAGE_PUBLIC"
fi
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
local hook_defined
declare -f -F test_append_files >/dev/null && hook_defined=yes || hook_defined=no
test: create public images by default Let's create new images public by default and then symlink/copy them into the respective private directories afterwards, not the other way around. This should fix a nasty race condition in parallel runs where one tests attempts to copy the backing public image at the same moment another test is already modifying it.
2020-05-26 04:48:01 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
echo "Reusing existing cached image $IMAGE_PUBLIC → $(realpath "$IMAGE_PUBLIC")"
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$TEST_PARALLELIZE" || get_bool "$hook_defined"; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
cp -v -- "$(realpath "$IMAGE_PUBLIC")" "$IMAGE_PRIVATE"
test: introduce TEST_PARALLELIZE to support running tests in parallel Support running tests in parallel by switching to copying of the base image instead of symlinking it.. This still requires some setup steps, like running `make setup` on tests which have unique $IMAGE_NAME beforehand (and sequentially), otherwise they'll all try to create the same base image when started in parallel, leading to nasty issues. However, as running the integration tests in parallel is such an unusual use case it should be good enough, for now.
2020-05-17 16:48:16 +08:00
else
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
ln -sv -- "$(realpath "$IMAGE_PUBLIC")" "$IMAGE_PRIVATE"
test: introduce TEST_PARALLELIZE to support running tests in parallel Support running tests in parallel by switching to copying of the base image instead of symlinking it.. This still requires some setup steps, like running `make setup` on tests which have unique $IMAGE_NAME beforehand (and sequentially), otherwise they'll all try to create the same base image when started in parallel, leading to nasty issues. However, as running the integration tests in parallel is such an unusual use case it should be good enough, for now.
2020-05-17 16:48:16 +08:00
fi
test: create public images by default Let's create new images public by default and then symlink/copy them into the respective private directories afterwards, not the other way around. This should fix a nasty race condition in parallel runs where one tests attempts to copy the backing public image at the same moment another test is already modifying it.
2020-05-26 04:48:01 +08:00
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
mount_initdir
test: make masking of supplementary services configurable
2022-02-17 03:29:14 +08:00
if get_bool "${TEST_SUPPORTING_SERVICES_SHOULD_BE_MASKED}"; then
test: correctly mask supporting services in tests, take #2 Due to a little misunderstanding the last patch doesn't work as expected, since test_create_image() is called only for the first image (usually TEST-01-BASIC), and all subsequent images are then (possibly) modified with test_append_files(). Follow-up to 179ca4d2b1b5579014773a128462475f99b7a91b.
2021-06-24 18:13:52 +08:00
dinfo "Masking supporting services"
mask_supporting_services
fi
test: do an initial coverage capture I noticed that our coverage reports miss some files completely - this happens when the test doesn't touch the code in them at all, so the generated coverage data (and resulting reports) have no information about them. Let's fix this by doing an initial zero coverage capture that contains a zeroed counter for every instrumented line in every file, so when we later merge it with a capture from the test, it shows up with a missing coverage instead of not showing at all.
2023-03-25 04:15:28 +08:00
if get_bool "$IS_BUILT_WITH_COVERAGE"; then
# Do an initial coverage capture, to make sure the final report includes
# files that the tests didn't touch at all
lcov --initial --capture --directory "${initdir}/${BUILD_DIR:?}" --output-file "${TESTDIR:?}/coverage-base"
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$hook_defined"; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
test_append_files "${initdir:?}"
tests: build the image once and then copy/extend it Building custom images for each test takes a lot of time. Build the default one, and if the test needs incompatible changes just copy it and extend it instead.
2021-01-07 05:42:28 +08:00
fi
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
fi
test: de-duplicate test_setup() functions Now we will run setup_nspawn_root() in some case where do don't need to.
2019-12-10 23:40:31 +08:00
setup_nspawn_root
}
test: Factorize common integration test functions (#6540) All test/TEST* but TEST-02-CRYPTSETUP share the same check_result_qemu() and test_cleanup(), so move them into test_functions and only override them in TEST-02-CRYPTSETUP. Also provide a common test_run() which by default assumes that both QEMU and nspawn tests are run. Particular tests which don't support either need to explicitly opt out by setting $TEST_NO_{QEMU,NSPAWN}. Do it this way around to avoid accidentally forgetting to opt in, and to encourage test authors to at least always support nspawn.
2017-08-04 20:34:14 +08:00
test_run() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local test_id="${1:?}"
test: make sure to reset state before starting integration tests
2019-12-13 19:34:41 +08:00
mount_initdir
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if ! get_bool "${TEST_NO_QEMU:=}"; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if run_qemu "$test_id"; then
docs: use lowercase "qemu" and other minor tweaks to test docs We used both "qemu" and "QEMU", let's use the lower-case version everywhere since it's also the name of the binary and the version that people are most familiar with. The stuff under test/ is not only for the integeration tests, but also for various other test-related stuff, so adjust the docs a bit.
2022-05-04 18:29:31 +08:00
check_result_qemu || { echo "qemu test failed"; return 1; }
test: Factorize common integration test functions (#6540) All test/TEST* but TEST-02-CRYPTSETUP share the same check_result_qemu() and test_cleanup(), so move them into test_functions and only override them in TEST-02-CRYPTSETUP. Also provide a common test_run() which by default assumes that both QEMU and nspawn tests are run. Particular tests which don't support either need to explicitly opt out by setting $TEST_NO_{QEMU,NSPAWN}. Do it this way around to avoid accidentally forgetting to opt in, and to encourage test authors to at least always support nspawn.
2017-08-04 20:34:14 +08:00
else
docs: use lowercase "qemu" and other minor tweaks to test docs We used both "qemu" and "QEMU", let's use the lower-case version everywhere since it's also the name of the binary and the version that people are most familiar with. The stuff under test/ is not only for the integeration tests, but also for various other test-related stuff, so adjust the docs a bit.
2022-05-04 18:29:31 +08:00
dwarn "can't run qemu, skipping"
test: Factorize common integration test functions (#6540) All test/TEST* but TEST-02-CRYPTSETUP share the same check_result_qemu() and test_cleanup(), so move them into test_functions and only override them in TEST-02-CRYPTSETUP. Also provide a common test_run() which by default assumes that both QEMU and nspawn tests are run. Particular tests which don't support either need to explicitly opt out by setting $TEST_NO_{QEMU,NSPAWN}. Do it this way around to avoid accidentally forgetting to opt in, and to encourage test authors to at least always support nspawn.
2017-08-04 20:34:14 +08:00
fi
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if ! get_bool "${TEST_NO_NSPAWN:=}"; then
test: run tests directly from the loopback device Before, we'd copy the test tree into nspawn-root, and run the tests from there. This is OK, and doesn't actually take much extra time. But it uses quite a lot of extra disk space. So let's make things a bit more efficient by running directly from the image file. We still run the unprivileged nspawn tests from a copy. Once the kernel implements fs shift, we can do away with that too.
2019-12-13 21:21:31 +08:00
mount_initdir
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if run_nspawn "${initdir:?}" "$test_id"; then
test: run tests directly from the loopback device Before, we'd copy the test tree into nspawn-root, and run the tests from there. This is OK, and doesn't actually take much extra time. But it uses quite a lot of extra disk space. So let's make things a bit more efficient by running directly from the image file. We still run the unprivileged nspawn tests from a copy. Once the kernel implements fs shift, we can do away with that too.
2019-12-13 21:21:31 +08:00
check_result_nspawn "$initdir" || { echo "nspawn-root test failed"; return 1; }
test: Factorize common integration test functions (#6540) All test/TEST* but TEST-02-CRYPTSETUP share the same check_result_qemu() and test_cleanup(), so move them into test_functions and only override them in TEST-02-CRYPTSETUP. Also provide a common test_run() which by default assumes that both QEMU and nspawn tests are run. Particular tests which don't support either need to explicitly opt out by setting $TEST_NO_{QEMU,NSPAWN}. Do it this way around to avoid accidentally forgetting to opt in, and to encourage test authors to at least always support nspawn.
2017-08-04 20:34:14 +08:00
else
dwarn "can't run systemd-nspawn, skipping"
fi
tests: also run TEST-01-BASIC in an unprivileged container (#9957) This should make it much easier to catch regressions like https://github.com/systemd/systemd/issues/9914 and https://github.com/systemd/systemd/issues/8535.
2018-08-30 12:01:18 +08:00
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "${RUN_IN_UNPRIVILEGED_CONTAINER:=}"; then
test: run tests directly from the loopback device Before, we'd copy the test tree into nspawn-root, and run the tests from there. This is OK, and doesn't actually take much extra time. But it uses quite a lot of extra disk space. So let's make things a bit more efficient by running directly from the image file. We still run the unprivileged nspawn tests from a copy. Once the kernel implements fs shift, we can do away with that too.
2019-12-13 21:21:31 +08:00
dir="$TESTDIR/unprivileged-nspawn-root"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if NSPAWN_ARGUMENTS="-U --private-network ${NSPAWN_ARGUMENTS:-}" run_nspawn "$dir" "$test_id"; then
test: run tests directly from the loopback device Before, we'd copy the test tree into nspawn-root, and run the tests from there. This is OK, and doesn't actually take much extra time. But it uses quite a lot of extra disk space. So let's make things a bit more efficient by running directly from the image file. We still run the unprivileged nspawn tests from a copy. Once the kernel implements fs shift, we can do away with that too.
2019-12-13 21:21:31 +08:00
check_result_nspawn "$dir" || { echo "unprivileged-nspawn-root test failed"; return 1; }
tests: also run TEST-01-BASIC in an unprivileged container (#9957) This should make it much easier to catch regressions like https://github.com/systemd/systemd/issues/9914 and https://github.com/systemd/systemd/issues/8535.
2018-08-30 12:01:18 +08:00
else
dwarn "can't run systemd-nspawn, skipping"
fi
tests: ignore memory leaks in dbus-daemon Otherwise, the test fails on Fedora 28 with ``` Jan 30 01:42:35 systemd-testsuite dbus-daemon[61]: [system] Successfully activated service 'org.freedesktop.systemd1' Jan 30 01:42:35 systemd-testsuite systemd[61]: dbus.service: Kernel keyring access prohibited, ignoring. Jan 30 01:42:35 systemd-testsuite systemd[61]: dbus.service: Executing: /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: ================================================================= Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: ==61==ERROR: LeakSanitizer: detected memory leaks Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: Direct leak of 72 byte(s) in 1 object(s) allocated from: Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #0 0x7f21f9e29088 in __interceptor_realloc (/usr/lib64/libasan.so.5+0xef088) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #1 0x7f21f9b1b23c (/lib64/libdbus-1.so.3+0x3323c) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #2 0x240000001b (<unknown module>) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: Direct leak of 16 byte(s) in 1 object(s) allocated from: Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #0 0x7f21f9e29088 in __interceptor_realloc (/usr/lib64/libasan.so.5+0xef088) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #1 0x7f21f9b1b23c (/lib64/libdbus-1.so.3+0x3323c) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #2 0x7ffffffff (<unknown module>) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: SUMMARY: AddressSanitizer: 88 byte(s) leaked in 2 allocation(s). ``` The leaks were reported and fixed in https://bugs.freedesktop.org/show_bug.cgi?id=107320.
2019-01-30 09:19:45 +08:00
fi
test: Factorize common integration test functions (#6540) All test/TEST* but TEST-02-CRYPTSETUP share the same check_result_qemu() and test_cleanup(), so move them into test_functions and only override them in TEST-02-CRYPTSETUP. Also provide a common test_run() which by default assumes that both QEMU and nspawn tests are run. Particular tests which don't support either need to explicitly opt out by setting $TEST_NO_{QEMU,NSPAWN}. Do it this way around to avoid accidentally forgetting to opt in, and to encourage test authors to at least always support nspawn.
2017-08-04 20:34:14 +08:00
fi
return 0
}
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
do_test() {
test/test-functions: be more verbose about skipping tests
2012-05-25 20:53:29 +08:00
if [[ $UID != "0" ]]; then
echo "TEST: $TEST_DESCRIPTION [SKIPPED]: not root" >&2
exit 0
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "${TEST_NO_QEMU:=}" && get_bool "${TEST_NO_NSPAWN:=}"; then
docs: use lowercase "qemu" and other minor tweaks to test docs We used both "qemu" and "QEMU", let's use the lower-case version everywhere since it's also the name of the binary and the version that people are most familiar with. The stuff under test/ is not only for the integeration tests, but also for various other test-related stuff, so adjust the docs a bit.
2022-05-04 18:29:31 +08:00
echo "TEST: $TEST_DESCRIPTION [SKIPPED]: both qemu and nspawn disabled" >&2
test: shortcut skip if both TEST_NO_QEMU and TEST_NO_NSPAWN are set Allows to run all tests in bulk with TEST_NO_QEMU, skipping those where it is mandatory, without wasting time building the image.
2020-12-23 01:51:40 +08:00
exit 0
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "${TEST_QEMU_ONLY:=}" && ! get_bool "$TEST_NO_NSPAWN"; then
docs: use lowercase "qemu" and other minor tweaks to test docs We used both "qemu" and "QEMU", let's use the lower-case version everywhere since it's also the name of the binary and the version that people are most familiar with. The stuff under test/ is not only for the integeration tests, but also for various other test-related stuff, so adjust the docs a bit.
2022-05-04 18:29:31 +08:00
echo "TEST: $TEST_DESCRIPTION [SKIPPED]: qemu-only tests requested" >&2
tests: add TEST_QEMU_ONLY variable to run only tests where qemu is mandatory Allows to split the test run in two parts. Most tests can run under nspawn which is much faster, and they can be ran in one chunk with TEST_NO_QEMU=1. The qemu-only tests, which are just a handful, can be ran in another chunk with TEST_QEMU_ONLY=1. Allows autopkgtest to be split in two parts.
2020-12-27 04:11:55 +08:00
exit 0
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "${TEST_PREFER_NSPAWN:=}" && ! get_bool "$TEST_NO_NSPAWN"; then
tests: add TEST_PREFER_NSPAWN variable to run as many as possible under nspawn By default the test suite prefers qemu, and uses nspawn only if a test specifically says it doesn't support qemu. Add a variable to allow flipping the default, and run as many tests under nspawn as possible.
2021-01-01 01:29:58 +08:00
TEST_NO_QEMU=1
fi
scripts: use 4 space indentation We had all kinds of indentation: 2 sp, 3 sp, 4 sp, 8 sp, and mixed. 4 sp was the most common, in particular the majority of scripts under test/ used that. Let's standarize on 4 sp, because many commandlines are long and there's a lot of nesting, and with 8sp indentation less stuff fits. 4 sp also seems to be the default indentation, so this will make it less likely that people will mess up if they don't load the editor config. (I think people often use vi, and vi has no support to load project-wide configuration automatically. We distribute a .vimrc file, but it is not loaded by default, and even the instructions in it seem to discourage its use for security reasons.) Also remove the few vim config lines that were left. We should either have them on all files, or none. Also remove some strange stuff like '#!/bin/env bash', yikes.
2019-04-04 20:10:42 +08:00
# Detect lib paths
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ "$libdir" ]] || for libdir in /lib64 /lib; do
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
[[ -d $libdir ]] && libdirs+=" $libdir" && break
done
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ "$usrlibdir" ]] || for usrlibdir in /usr/lib64 /usr/lib; do
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
[[ -d $usrlibdir ]] && libdirs+=" $usrlibdir" && break
done
test: $STATEDIR should not point to /usr/lib/systemd/tests when NO_BUILD=1 When testing the binaries from the host, make sure to not store the state data below /usr but use a dedicated directory in /var/tmp/ instead. The working directories of the tests, initially located in /var/tmp, are also moved in a dedicated directory /var/tmp/systemd-tests.
2023-03-21 23:44:41 +08:00
mkdir -p "$WORKDIR"
test: Write state and log files into $BUILD_DIR This avoids clobbering the source tree with .testdir/test.log files and makes the tests work in situations where the source tree is read-only.
2017-08-07 23:30:11 +08:00
mkdir -p "$STATEDIR"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
import_testdir
test: remove duplicated code
2013-11-06 06:32:56 +08:00
import_initdir
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
tests: use setfacl to give $SUDO_USER read permissions on artifacts We have to invoke the tests as superuser, and not being able to read the journal as the invoking user is annoying. I don't think there are any security considerations here, since the invoking user can already put arbitrary code in the Makefile and test scripts which get executed with root privileges.
2021-04-20 23:47:50 +08:00
if [ -n "${SUDO_USER}" ]; then
ddebug "Making ${TESTDIR:?} readable for ${SUDO_USER} (acquired from sudo)"
setfacl -m "user:${SUDO_USER:?}:r-X" "${TESTDIR:?}"
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
testname="$(basename "$PWD")"
test: print test of name in output
2020-03-25 20:15:37 +08:00
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
while (($# > 0)); do
case $1 in
--run)
test: print test of name in output
2020-03-25 20:15:37 +08:00
echo "${testname} RUN: $TEST_DESCRIPTION"
test: "detect" the test number automagically Specifying the test number manually is tedious and prone to errors (as recently proven). Since we have all the necessary data to work out the test number, let's do it automagically.
2021-04-27 01:20:18 +08:00
test_run "$TESTID"
test: do not use global variable to pass error
2018-09-14 12:25:02 +08:00
ret=$?
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [ $ret -eq 0 ]; then
test: print test of name in output
2020-03-25 20:15:37 +08:00
echo "${testname} RUN: $TEST_DESCRIPTION [OK]"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
else
test: print test of name in output
2020-03-25 20:15:37 +08:00
echo "${testname} RUN: $TEST_DESCRIPTION [FAILED]"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
exit $ret
;;
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
--setup)
test: print test of name in output
2020-03-25 20:15:37 +08:00
echo "${testname} SETUP: $TEST_DESCRIPTION"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test_setup
test: add create_empty_image_rootdir() to simplify testcase setup Almost all tests were manually mounting/unmounting $TESTDIR/root from the loopback image; this moves all that into test-functions so the test setup functions are simplier. Also add test_setup_cleanup() function, to cleanup what is mounted by create_empty_image_rootdir()
2019-07-12 23:47:26 +08:00
test_setup_cleanup
test: Run qemu/nspawn tests with "set -e" This catches errors like "ninja not found", missing programs etc. early, instead of silently ignoring them and trying to boot a broken VM. In install_config_files(), allow some distro specific files to be absent (such as /etc/sysconfig/init).
2017-08-08 03:09:21 +08:00
;;
test: perform partial cleanup after each test is run This causes the unprivileged-nspawn-root directory to be removed after running one test. The advantage is that we reduce the maximum disk-space use quite a bit (47*400 MB → about 18GB).
2020-03-21 02:09:35 +08:00
--clean)
test: print test of name in output
2020-03-25 20:15:37 +08:00
echo "${testname} CLEANUP: $TEST_DESCRIPTION"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test_cleanup
test: Run qemu/nspawn tests with "set -e" This catches errors like "ninja not found", missing programs etc. early, instead of silently ignoring them and trying to boot a broken VM. In install_config_files(), allow some distro specific files to be absent (such as /etc/sysconfig/init).
2017-08-08 03:09:21 +08:00
;;
test: perform partial cleanup after each test is run This causes the unprivileged-nspawn-root directory to be removed after running one test. The advantage is that we reduce the maximum disk-space use quite a bit (47*400 MB → about 18GB).
2020-03-21 02:09:35 +08:00
--clean-again)
test: print test of name in output
2020-03-25 20:15:37 +08:00
echo "${testname} CLEANUP AGAIN: $TEST_DESCRIPTION"
test: perform partial cleanup after each test is run This causes the unprivileged-nspawn-root directory to be removed after running one test. The advantage is that we reduce the maximum disk-space use quite a bit (47*400 MB → about 18GB).
2020-03-21 02:09:35 +08:00
test_cleanup_again
;;
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
--all)
test: Run qemu/nspawn tests with "set -e" This catches errors like "ninja not found", missing programs etc. early, instead of silently ignoring them and trying to boot a broken VM. In install_config_files(), allow some distro specific files to be absent (such as /etc/sysconfig/init).
2017-08-08 03:09:21 +08:00
ret=0
test: print test of name in output
2020-03-25 20:15:37 +08:00
echo -n "${testname}: $TEST_DESCRIPTION "
test: avoid leaking open loop devices When a subshell is used ('make' or 'make all') the LOOPDEV environment variable, which is used to store the opened loop device, is lost. So the cleanup on trap/exit doesn't do anything, and the loop device used to mount the test image is left around. Avoid using a subshell to fix the issue.
2021-02-17 07:47:34 +08:00
# Do not use a subshell, otherwise cleanup variables (LOOPDEV) will be lost
# and loop devices will leak
test_setup </dev/null >"$TESTLOG" 2>&1 || ret=$?
if [ $ret -eq 0 ]; then
test_setup_cleanup </dev/null >>"$TESTLOG" 2>&1 || ret=$?
fi
if [ $ret -eq 0 ]; then
test: "detect" the test number automagically Specifying the test number manually is tedious and prone to errors (as recently proven). Since we have all the necessary data to work out the test number, let's do it automagically.
2021-04-27 01:20:18 +08:00
test_run "$TESTID" </dev/null >>"$TESTLOG" 2>&1 || ret=$?
test: avoid leaking open loop devices When a subshell is used ('make' or 'make all') the LOOPDEV environment variable, which is used to store the opened loop device, is lost. So the cleanup on trap/exit doesn't do anything, and the loop device used to mount the test image is left around. Avoid using a subshell to fix the issue.
2021-02-17 07:47:34 +08:00
fi
test: ignore errors during test cleanup, so cleanup can finish Also move TESTDIR and STATEFILE removal into test_cleanup
2019-07-13 02:09:48 +08:00
test_cleanup
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
if [ $ret -eq 0 ]; then
test: do not remove state directory on failure The test log is in the state directory, and we want to keep it around when a test fails. Follow-up for: 256a835f1c6c646a55039659aa2db186fbeb5c5d
2023-03-29 07:07:45 +08:00
# $TESTLOG is in $STATEDIR, so clean it up only on success
[[ -n "$STATEDIR" ]] && rm -vfr "$STATEDIR"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
echo "[OK]"
else
echo "[FAILED]"
test: Write state and log files into $BUILD_DIR This avoids clobbering the source tree with .testdir/test.log files and makes the tests work in situations where the source tree is read-only.
2017-08-07 23:30:11 +08:00
echo "see $TESTLOG"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
exit $ret
;;
*)
break
;;
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
esac
shift
done
}
Reference in New Issue Copy Permalink