mirrors/systemd
0
0
Fork 0
mirror of https://github.com/systemd/systemd.git synced 2024-11-23 18:23:32 +08:00
Code Issues Packages Projects Releases Wiki Activity
994c9c70a1
systemd/test/test-functions

2889 lines
92 KiB
Plaintext
Raw Normal View History

treewide: more portable bash shebangs As in 2a5fcfae024ffc370bb780572279f45a1da3f946 and in 3e67e5c9928f8b1e1c5a63def88d53ed1fed12eb using /usr/bin/env allows bash to be looked up in PATH rather than being hard-coded. As with the previous changes the same arguments apply - distributions have scripts to rewrite shebangs on installation and they know what locations to rely on. - For tests/compilation we should rather rely on the user to have setup there PATH correctly. In particular this makes testing from git easier on NixOS where do not provide /bin/bash to improve compose-ability.
2020-03-04 17:35:06 +08:00
#!/usr/bin/env bash
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
# -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*-
licensing: add forgotten spdx headers Those are all "our" files, but we forgot to add the headers, most likely because of non-standard file extensions.
2021-10-01 19:04:32 +08:00
# SPDX-License-Identifier: LGPL-2.1-or-later
#
# shellcheck disable=SC2030,SC2031
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
# ex: ts=8 sw=4 sts=4 et filetype=sh tw=180
# Note: the shellcheck line above disables warning for variables which were
# modified in a subshell. In our case this behavior is expected, but
# `shellcheck` can't distinguish this because of poor variable tracking,
# which results in warning for every instance of such variable used
# throughout this file.
# See:
# * comment in function install_verity_minimal()
# * koalaman/shellcheck#280
set -o pipefail
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
PATH=/sbin:/bin:/usr/sbin:/usr/bin
export PATH
tests/TEST-50: support the case when /etc/os-release is present We have four legal cases: 1. /usr/lib/os-release exists and /etc/os-release is a symlink to it 2. both exist but /etc/os-release is not a symlink to /usr/lib/os-release 3. only /usr/lib/os-release exists 4. only /etc/os-release exists The generic setup code in test-functions and create-busybox-image didn't handle case 3. The test-specific code in TEST-50 didn't handle 2 (because the general setup code would only install /etc/os-release in the image and grep -f /usr/lib/os-release would not work) and 4 (same reason) and would fail in case 3 in generic setup.
2020-08-27 17:59:45 +08:00
os_release=$(test -e /etc/os-release && echo /etc/os-release || echo /usr/lib/os-release)
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
# shellcheck source=/dev/null
source "$os_release"
[[ "$ID" = "debian" || " $ID_LIKE " = *" debian "* ]] && LOOKS_LIKE_DEBIAN=yes || LOOKS_LIKE_DEBIAN=""
[[ "$ID" = "arch" || " $ID_LIKE " = *" arch "* ]] && LOOKS_LIKE_ARCH=yes || LOOKS_LIKE_ARCH=""
[[ " $ID_LIKE " = *" suse "* ]] && LOOKS_LIKE_SUSE=yes || LOOKS_LIKE_SUSE=""
KERNEL_VER="${KERNEL_VER-$(uname -r)}"
tests: add test for https://github.com/systemd/systemd/issues/2730
2016-02-25 02:22:06 +08:00
QEMU_TIMEOUT="${QEMU_TIMEOUT:-infinity}"
tests: don't rely on the host's systemd-nspawn, don't register test machine (#3023) Fixes: $ cd test/TEST-07-ISSUE-1981/ $ sudo make clean setup run ... timeout: failed to run command ‘systemd-nspawn’: No such file or directory ... TEST RUN: https://github.com/systemd/systemd/issues/1981 [FAILED] Makefile:10: recipe for target 'run' failed make: *** [run] Error 1
2016-04-13 21:57:06 +08:00
NSPAWN_TIMEOUT="${NSPAWN_TIMEOUT:-infinity}"
tests: track and check for timeouts If run_qemu() exits with non-zero, this either meant that QEMU was not available (which should be a SKIP) or that QEMU timed out if $QEMU_TIMEOUT was set (which then should be a FAIL). Limit the exit code of run_qemu() to QEMU availability only, and track timeouts separately through the new $TIMED_OUT variable, which is then checked in check_result_qemu(). Do the same for $NSPAWN_TIMEOUT and run_nspawn() so that nspawn and QEMU work similarly.
2016-06-24 18:11:19 +08:00
TIMED_OUT= # will be 1 after run_* if *_TIMEOUT is set and test timed out
test: switch to using ext4 instead of ext3 as default fallback fs (#7265) Using ext3 is not representative anymore, and Ubuntu has stopped shipping fsck.ext3 in the initramfs.
2017-11-07 22:51:30 +08:00
[[ "$LOOKS_LIKE_SUSE" ]] && FSTYPE="${FSTYPE:-btrfs}" || FSTYPE="${FSTYPE:-ext4}"
tests: add UNIFIED_CGROUP_HIERARCHY=[default|hybrid] (#4675) This will simplify testing a bit. Mainly for https://github.com/systemd/systemd/pull/4670
2016-11-16 00:38:04 +08:00
UNIFIED_CGROUP_HIERARCHY="${UNIFIED_CGROUP_HIERARCHY:-default}"
test: allow overriding EFI_MOUNT like other variables
2019-12-10 01:52:35 +08:00
EFI_MOUNT="${EFI_MOUNT:-$(bootctl -x 2>/dev/null || echo /boot)}"
test: introduce QEMU_MEM Some tests (mainly under ASan/UBSan) require more than 512M of memory, so let's make it configurable (but still default to 512M).
2019-01-11 22:51:07 +08:00
QEMU_MEM="${QEMU_MEM:-512M}"
tests: build the image once and then copy/extend it Building custom images for each test takes a lot of time. Build the default one, and if the test needs incompatible changes just copy it and extend it instead.
2021-01-07 05:42:28 +08:00
# Note that defining a different IMAGE_NAME in a test setup script will only result
# in default.img being copied and renamed. It can then be extended by defining
# a test_append_files() function. The $1 parameter will be the root directory.
# To force creating a new image from scratch (eg: to encrypt it), also define
# TEST_FORCE_NEWIMAGE=1 in the test setup script.
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
IMAGE_NAME=${IMAGE_NAME:-default}
test: strip binaries by default Since 23f8e01 we always kept binaries unstripped, since $STRIP_BINARIES is unset by default.
2021-07-09 20:59:11 +08:00
STRIP_BINARIES="${STRIP_BINARIES:-yes}"
test/TEST-01: allow running w/o -Dinstall-tests=true This is useful for sanitizer builds under CI, see https://github.com/systemd/systemd-centos-ci/pull/202#issuecomment-566476592.
2019-12-17 19:52:35 +08:00
TEST_REQUIRE_INSTALL_TESTS="${TEST_REQUIRE_INSTALL_TESTS:-1}"
test: introduce TEST_PARALLELIZE to support running tests in parallel Support running tests in parallel by switching to copying of the base image instead of symlinking it.. This still requires some setup steps, like running `make setup` on tests which have unique $IMAGE_NAME beforehand (and sequentially), otherwise they'll all try to create the same base image when started in parallel, leading to nasty issues. However, as running the integration tests in parallel is such an unusual use case it should be good enough, for now.
2020-05-17 16:48:16 +08:00
TEST_PARALLELIZE="${TEST_PARALLELIZE:-0}"
test: stop caching loopdev It is more trouble than it is worth. The setup is of a loopback device is very quick, so it's better to always create it when needed and immediately drop afterwards.
2020-03-21 03:17:11 +08:00
LOOPDEV=
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
# Simple wrapper to unify boolean checks.
# Note: this function needs to stay near the top of the file, so we can use it
# in code in the outermost scope.
get_bool() {
# Make the value lowercase to make the regex matching simpler
local _bool="${1,,}"
# Consider empty value as "false"
if [[ -z "$_bool" || "$_bool" =~ ^(0|no|false)$ ]]; then
return 1
elif [[ "$_bool" =~ ^(1|yes|true)$ ]]; then
return 0
else
echo >&2 "Value '$_bool' is not a valid boolean value"
exit 1
fi
}
test: allow use of nested KVM Introduce TEST_NESTED_KVM variable, which allows use of nested KVM for QEMU VMs (which is disabled by default by KVM autodetection).
2019-09-27 05:29:38 +08:00
# Decide if we can (and want to) run QEMU with KVM acceleration.
# Check if nested KVM is explicitly enabled (TEST_NESTED_KVM). If not,
# check if it's not explicitly disabled (TEST_NO_KVM) and we're not already
# running under KVM. If these conditions are met, enable KVM (and possibly
# nested KVM), otherwise disable it.
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "${TEST_NESTED_KVM:=}" || (! get_bool "${TEST_NO_KVM:=}" && [[ "$(systemd-detect-virt -v)" != kvm ]]); then
test: allow use of nested KVM Introduce TEST_NESTED_KVM variable, which allows use of nested KVM for QEMU VMs (which is disabled by default by KVM autodetection).
2019-09-27 05:29:38 +08:00
QEMU_KVM=yes
else
QEMU_KVM=no
fi
tests: determine rootlibdir from pkg-config Prefer asking pkg-config for the rootlibdir, to also support systems with --enable-split-usr. Fall back to the hardcoded /usr/lib/systemd if that fails.
2015-02-02 07:19:29 +08:00
if ! ROOTLIBDIR=$(pkg-config --variable=systemdutildir systemd); then
echo "WARNING! Cannot determine rootlibdir from pkg-config, assuming /usr/lib/systemd" >&2
ROOTLIBDIR=/usr/lib/systemd
fi
test/test-functions: add variables for several dir locations
2021-01-27 05:46:10 +08:00
# The calling test.sh scripts have TEST_BASE_DIR set via their Makefile, but we don't need them to provide it
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
TEST_BASE_DIR=${TEST_BASE_DIR:-$(realpath "$(dirname "${BASH_SOURCE[0]}")")}
test: "detect" the test number automagically Specifying the test number manually is tedious and prone to errors (as recently proven). Since we have all the necessary data to work out the test number, let's do it automagically.
2021-04-27 01:20:18 +08:00
TEST_UNITS_DIR="$(realpath "$TEST_BASE_DIR/units")"
test/test-functions: add variables for several dir locations
2021-01-27 05:46:10 +08:00
SOURCE_DIR=$(realpath "$TEST_BASE_DIR/..")
TOOLS_DIR="$SOURCE_DIR/tools"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
# These variables are used by test scripts
export TEST_BASE_DIR TEST_UNITS_DIR SOURCE_DIR TOOLS_DIR
test/test-functions: add variables for several dir locations
2021-01-27 05:46:10 +08:00
test: find $BUILD_DIR in test-functions, remove from other scripts The $BUILD_DIR is only used in test-functions, and doesn't need to be specified in any other scripts. Additionally, to be able to allow the integration test suite to be run against locally installed binaries, instead of built binaries, moving BUILD_DIR logic completely into test-functions allows later patches to be simpler.
2020-11-18 04:39:37 +08:00
# note that find-build-dir.sh will return $BUILD_DIR if provided, else it will try to find it
test: don't try to find BUILD_DIR when NO_BUILD is set NO_BUILD=1 indicates that we want to test systemd from the local system and not the one from the local build. Hence there should be no need to call find-build-dir.sh when NO_BUID=1 especially since it's likely that the script will fail to find a local build in this case. This avoids find-build-dir.sh to emit 'Specify build directory with $BUILD_DIR' message when NO_BUILD=1 and no local build can be found. This introduces a behavior change though: systemd from the local system will always be preferred when NO_BUILD=1 even if a local build can be found.
2021-08-03 14:44:47 +08:00
if get_bool "${NO_BUILD:=}"; then
BUILD_DIR="$SOURCE_DIR"
elif ! BUILD_DIR="$("$TOOLS_DIR"/find-build-dir.sh)"; then
echo "ERROR: no build found, please set BUILD_DIR or use NO_BUILD" >&2
exit 1
test: find $BUILD_DIR in test-functions, remove from other scripts The $BUILD_DIR is only used in test-functions, and doesn't need to be specified in any other scripts. Additionally, to be able to allow the integration test suite to be run against locally installed binaries, instead of built binaries, moving BUILD_DIR logic completely into test-functions allows later patches to be simpler.
2020-11-18 04:39:37 +08:00
fi
test: avoid using pipes in certain cases `command -v <bin> | grep ...` can under certain conditions cause the `command` to exit with SIGPIPE, which in combination with `set -o pipefail` means that the tests sometimes randomly die during setup. Let's avoid using pipes in such cases.
2021-04-17 00:37:49 +08:00
PATH_TO_INIT="$ROOTLIBDIR/systemd"
SYSTEMD_JOURNALD="${SYSTEMD_JOURNALD:-$(command -v "$BUILD_DIR/systemd-journald" || command -v "$ROOTLIBDIR/systemd-journald")}"
SYSTEMD_JOURNAL_REMOTE="${SYSTEMD_JOURNAL_REMOTE:-$(command -v "$BUILD_DIR/systemd-journal-remote" || command -v "$ROOTLIBDIR/systemd-journal-remote")}"
SYSTEMD="${SYSTEMD:-$(command -v "$BUILD_DIR/systemd" || command -v "$ROOTLIBDIR/systemd")}"
SYSTEMD_NSPAWN="${SYSTEMD_NSPAWN:-$(command -v "$BUILD_DIR/systemd-nspawn" || command -v systemd-nspawn)}"
JOURNALCTL="${JOURNALCTL:-$(command -v "$BUILD_DIR/journalctl" || command -v journalctl)}"
tests: use the asan wrapper to boot a VM/container if systemd is built with ASAN
2018-07-05 23:14:07 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
TESTFILE="${BASH_SOURCE[1]}"
test/test-functions: add variables for several dir locations
2021-01-27 05:46:10 +08:00
if [ -z "$TESTFILE" ]; then
echo "ERROR: test-functions must be sourced from one of the TEST-*/test.sh scripts" >&2
exit 1
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
TESTNAME="$(basename "$(dirname "$(realpath "$TESTFILE")")")"
test/test-functions: add variables for several dir locations
2021-01-27 05:46:10 +08:00
STATEDIR="$BUILD_DIR/test/$TESTNAME"
test/test-functions: move var assignment This makes no code change, only moves a small block of vars higher in the file. This makes the next commit a bit easier to read.
2020-11-18 05:02:14 +08:00
STATEFILE="$STATEDIR/.testdir"
IMAGESTATEDIR="$STATEDIR/.."
TESTLOG="$STATEDIR/test.log"
test: "detect" the test number automagically Specifying the test number manually is tedious and prone to errors (as recently proven). Since we have all the necessary data to work out the test number, let's do it automagically.
2021-04-27 01:20:18 +08:00
if ! [[ "$TESTNAME" =~ ^TEST\-([0-9]+)\-.+$ ]]; then
echo "ERROR: Test name '$TESTNAME' is not in the expected format: TEST-[0-9]+-*" >&2
exit 1
fi
TESTID="${BASH_REMATCH[1]:?}"
if [[ ! -f "$TEST_UNITS_DIR/testsuite-$TESTID.service" ]]; then
echo "ERROR: Test '$TESTNAME' is missing its service file '$TEST_UNITS_DIR/testsuite-$TESTID.service" >&2
exit 1
fi
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
BASICTOOLS=(
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
awk
test: add a testcase for issue #19895
2021-06-15 14:34:41 +08:00
base64
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
basename
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
bash
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
capsh
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
cat
chmod
chown
cmp
cryptsetup
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
cut
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
date
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
dd
diff
dirname
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
dmsetup
echo
env
false
test: add basic memory pressure extended test for oomd Simple test to generate a lot of pressure in one unit and assert that systemd-oomd kills the right one.
2020-08-14 16:20:43 +08:00
getconf
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
getent
getfacl
test: convert TEST-45 to a normal meson test On my laptop, it now takes 0.23 s. We don't need the overhead of creating an image and spawning a machine to mangle a file in /tmp.
2020-03-21 03:22:42 +08:00
grep
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
gunzip
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
gzip
head
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
ionice
test: move most of TEST-13-NSPAWN-SMOKE setup to static files
2019-12-10 04:45:46 +08:00
ip
test-execute: add a testcase for MountAPIVFS=no
2021-08-23 05:45:33 +08:00
ldd
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
ln
loadkeys
login
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
lz4cat
test: move TEST-22-TMPFILES setup to static files
2019-12-10 20:32:20 +08:00
mkfifo
mktemp
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
modprobe
mount
test: make TEST-02-CRYPTSETUP a bit more robust Prompted by systemd/systemd#16111. * check if /var is a mountpoint - if not, something went wrong. In case of systemd/systemd#16111 the /failed file was created, because systemd-cryptsetup failed, but it ended up being empty, making the result check incorrectly pass * forward journal messages to console - if we fail to mount /var, journald won't flush logs to the persistent storage and we end up empty handed and with no clue what went wrong For example, without systemd/systemd#16111 and with this patch: ... [FAILED] Failed to start systemd-cryptsetup@varcrypt.service. See 'systemctl status systemd-cryptsetup@varcrypt.service' for details. [DEPEND] Dependency failed for cryptsetup.target. ... [ 3.882451] systemd-cryptsetup[581]: Key file /etc/varkey is world-readable. This is not a good idea! [ 3.883946] systemd-cryptsetup[581]: WARNING: Locking directory /run/cryptsetup is missing! [ 3.884846] systemd-cryptsetup[581]: Failed to load Bitlocker superblock on device /dev/disk/by-uuid/180ba5ef-873b-4018-9968-47c23431f71a: Invalid argument ... [ 4.099451] sh[606]: + mountpoint /var [ 4.100025] sh[603]: + systemctl poweroff --no-block [ 4.101636] systemd[1]: Finished systemd-user-sessions.service. [ 4.102598] sh[608]: /var is not a mountpoint [FAILED] Failed to start testsuite-02.service.
2020-06-09 22:51:55 +08:00
mountpoint
test: move TEST-12-ISSUE-3171 setup to static files
2019-12-10 04:26:59 +08:00
mv
nc
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
nproc
test: move TEST-22-TMPFILES setup to static files
2019-12-10 20:32:20 +08:00
readlink
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
rev
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
rm
test: move most of TEST-13-NSPAWN-SMOKE setup to static files
2019-12-10 04:45:46 +08:00
rmdir
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
sed
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
seq
oom: implement avoid/omit xattr support There may be situations where a cgroup should be protected from killing or deprioritized as a candidate. In FB oomd xattrs are used to bias oomd away from supervisor cgroups and towards worker cgroups in container tasks. On desktops this can be used to protect important units with unpredictable resource consumption. The patch allows systemd-oomd to understand 2 xattrs: "user.oomd_avoid" and "user.oomd_omit". If systemd-oomd sees these xattrs set to 1 on a candidate cgroup (i.e. while attempting to kill something) AND the cgroup is owned by root, it will either deprioritize the cgroup as a candidate (avoid) or remove it completely as a candidate (omit). Usage is restricted to root owned cgroups to prevent situations where an unprivileged user can set their own cgroups lower in the kill priority than another user's (and prevent them from omitting their units from systemd-oomd killing).
2021-02-09 17:47:34 +08:00
setfattr
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
setfont
setsid
test: convert TEST-45 to a normal meson test On my laptop, it now takes 0.23 s. We don't need the overhead of creating an image and spawning a machine to mangle a file in /tmp.
2020-03-21 03:22:42 +08:00
sfdisk
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
sh
sleep
test: move TEST-10-ISSUE-2467 setup to static files
2019-12-10 03:42:48 +08:00
socat
test: move TEST-12-ISSUE-3171 setup to static files
2019-12-10 04:26:59 +08:00
stat
test: move TEST-43-* setup to static files Setup of lingering is dropped. I don't think it's necessary for anything, because the unit is still pulled in from the testsuite service.
2020-03-21 02:05:36 +08:00
su
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
sulogin
test: move most of TEST-13-NSPAWN-SMOKE setup to static files
2019-12-10 04:45:46 +08:00
sysctl
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
tail
test: move TEST-25-IMPORT setup to static files Also run mask_supporting_services() here.
2019-12-10 16:10:28 +08:00
tar
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
tee
test
test: add test for cgroup v2 freezer support
2020-04-03 15:13:51 +08:00
timeout
test: move TEST-11-ISSUE-3166 setup to static files
2019-12-10 03:53:38 +08:00
touch
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
tr
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
true
test: convert TEST-45 to a normal meson test On my laptop, it now takes 0.23 s. We don't need the overhead of creating an image and spawning a machine to mangle a file in /tmp.
2020-03-21 03:22:42 +08:00
truncate
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
umount
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
uname
test: move most of TEST-13-NSPAWN-SMOKE setup to static files
2019-12-10 04:45:46 +08:00
unshare
test: wc is needed by test/units/testsuite-50.sh
2021-09-22 15:04:52 +08:00
wc
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
xargs
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
xzcat
test: use shell arrays and remove duplicates The tool lists are obviously not empty, so let's remove the test if they are.
2019-12-10 03:35:51 +08:00
)
DEBUGTOOLS=(
cp
df
dhclient
dmesg
du
find
free
grep
hostname
id
less
ln
ls
mkdir
ping
ps
route
sort
strace
stty
tty
vi
)
test: remove duplicated code
2013-11-06 06:32:56 +08:00
tests: create the asan wrapper automatically if systemd has been built with ASAN
2018-07-05 12:09:30 +08:00
is_built_with_asan() {
if ! type -P objdump >/dev/null; then
ddebug "Failed to find objdump. Assuming systemd hasn't been built with ASAN."
return 1
fi
# Borrowed from https://github.com/google/oss-fuzz/blob/cd9acd02f9d3f6e80011cc1e9549be526ce5f270/infra/base-images/base-runner/bad_build_check#L182
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local _asan_calls
test: yet another fix for ASan detection This time for ppc64le: ``` 100737c4: 5d 55 f9 4b bl 10008d20 <00000024.plt_call.__asan_report_load8> 100737f4: 4d 55 f9 4b bl 10008d40 <00000024.plt_call.__asan_handle_no_return> 10073884: 5d 50 f9 4b bl 100088e0 <00000024.plt_call.__asan_init> 1007388c: 75 54 f9 4b bl 10008d00 <00000024.plt_call.__asan_version_mismatch_check_v8> 100738a0: 41 36 f9 4b bl 10006ee0 <00000024.plt_call.__asan_register_globals> 100738f0: 71 4c f9 4b bl 10008560 <00000024.plt_call.__asan_unregister_globals> ```
2021-08-09 01:33:10 +08:00
_asan_calls="$(objdump -dC "$SYSTEMD_JOURNALD" | grep -E "(callq?|brasl?|bl)\s.+__asan" -c)"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if ((_asan_calls < 1000)); then
tests: create the asan wrapper automatically if systemd has been built with ASAN
2018-07-05 12:09:30 +08:00
return 1
else
return 0
fi
}
test: make the coverage check safer for non-compiled builds
2021-10-22 00:34:21 +08:00
is_built_with_coverage() {
if get_bool "${NO_BUILD:=}" || ! command -v meson >/dev/null; then
return 1
fi
meson configure "${BUILD_DIR:?}" | grep 'b_coverage' | awk '{ print $2 }' | grep -q 'true'
}
tests: create the asan wrapper automatically if systemd has been built with ASAN
2018-07-05 12:09:30 +08:00
IS_BUILT_WITH_ASAN=$(is_built_with_asan && echo yes || echo no)
test: make the coverage check safer for non-compiled builds
2021-10-22 00:34:21 +08:00
IS_BUILT_WITH_COVERAGE=$(is_built_with_coverage && echo yes || echo no)
tests: create the asan wrapper automatically if systemd has been built with ASAN
2018-07-05 12:09:30 +08:00
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$IS_BUILT_WITH_ASAN"; then
tests: create the asan wrapper automatically if systemd has been built with ASAN
2018-07-05 12:09:30 +08:00
STRIP_BINARIES=no
test: allow SKIP_INITRD override Certain distributions (e.g. Arch Linux) require booting with initrd, as they lack support for commonly used filesystems in the kernel (i.e. the support is compiled in as modules)
2019-05-10 01:30:13 +08:00
SKIP_INITRD="${SKIP_INITRD:-yes}"
tests: use the asan wrapper to boot a VM/container if systemd is built with ASAN
2018-07-05 23:14:07 +08:00
PATH_TO_INIT=$ROOTLIBDIR/systemd-under-asan
tests: bump up QEMU_MEM
2019-07-16 23:01:26 +08:00
QEMU_MEM="2048M"
test: tidy up the ASan-related stuff
2021-03-07 05:21:30 +08:00
QEMU_SMP="${QEMU_SMP:-4}"
test: improve handling of ASan under clang Running integration tests with ASan is somewhat tricky to begin with, as we need to pre-load the ASan runtime DSO for certain services (like dbus), otherwise they won't start or behave as expected. In case of gcc this is pretty easy, as we need the runtime DSO during compilation, so it's already present on the host system. For clang things get more complicated, as ASan is compiled in statically by default, thus to enable the necessary dynamic-ish behavior one needs to compile with -shared-libasan and then correctly set LD_PRELOAD_PATH, as the runtime libraries are not in a standard library path.
2019-05-25 04:35:52 +08:00
# We need to correctly distinguish between gcc's and clang's ASan DSOs.
test: fix a yet another pipefail + pipe race Basically the same scenario as in a33e2692e162671f0d97856ad2f49a2620a1ec10, where `awk` exits as soon as it finds a match, thus sending SIGPIPE to `ldd` if it's not fast enough. That, in combination with `set -o pipefail` causes random & unexpected fails, like: ``` No journal files were found. -rw-r----- 1 root root 16777216 Apr 30 10:31 /var/tmp/TEST-01-BASIC_sanitizers-nspawn/system.journal TEST-01-BASIC RUN: Basic systemd setup [OK] systemd is not linked against the ASan DSO gcc does this by default, for clang compile with -shared-libasan make: *** [Makefile:2: clean-again] Error 1 make: Leaving directory '/build/test/TEST-01-BASIC' ```
2021-04-30 19:26:54 +08:00
if ASAN_RT_NAME="$(awk '/libasan.so/ {x=$1; exit} END {print x; exit x==""}' < <(ldd "$SYSTEMD"))"; then
test: improve handling of ASan under clang Running integration tests with ASan is somewhat tricky to begin with, as we need to pre-load the ASan runtime DSO for certain services (like dbus), otherwise they won't start or behave as expected. In case of gcc this is pretty easy, as we need the runtime DSO during compilation, so it's already present on the host system. For clang things get more complicated, as ASan is compiled in statically by default, thus to enable the necessary dynamic-ish behavior one needs to compile with -shared-libasan and then correctly set LD_PRELOAD_PATH, as the runtime libraries are not in a standard library path.
2019-05-25 04:35:52 +08:00
ASAN_COMPILER=gcc
test: tidy up the ASan-related stuff
2021-03-07 05:21:30 +08:00
ASAN_RT_PATH="$(readlink -f "$(${CC:-gcc} --print-file-name "$ASAN_RT_NAME")")"
test: fix a yet another pipefail + pipe race Basically the same scenario as in a33e2692e162671f0d97856ad2f49a2620a1ec10, where `awk` exits as soon as it finds a match, thus sending SIGPIPE to `ldd` if it's not fast enough. That, in combination with `set -o pipefail` causes random & unexpected fails, like: ``` No journal files were found. -rw-r----- 1 root root 16777216 Apr 30 10:31 /var/tmp/TEST-01-BASIC_sanitizers-nspawn/system.journal TEST-01-BASIC RUN: Basic systemd setup [OK] systemd is not linked against the ASan DSO gcc does this by default, for clang compile with -shared-libasan make: *** [Makefile:2: clean-again] Error 1 make: Leaving directory '/build/test/TEST-01-BASIC' ```
2021-04-30 19:26:54 +08:00
elif ASAN_RT_NAME="$(awk '/libclang_rt.asan/ {x=$1; exit} END {print x; exit x==""}' < <(ldd "$SYSTEMD"))"; then
test: improve handling of ASan under clang Running integration tests with ASan is somewhat tricky to begin with, as we need to pre-load the ASan runtime DSO for certain services (like dbus), otherwise they won't start or behave as expected. In case of gcc this is pretty easy, as we need the runtime DSO during compilation, so it's already present on the host system. For clang things get more complicated, as ASan is compiled in statically by default, thus to enable the necessary dynamic-ish behavior one needs to compile with -shared-libasan and then correctly set LD_PRELOAD_PATH, as the runtime libraries are not in a standard library path.
2019-05-25 04:35:52 +08:00
ASAN_COMPILER=clang
test: tidy up the ASan-related stuff
2021-03-07 05:21:30 +08:00
ASAN_RT_PATH="$(readlink -f "$(${CC:-clang} --print-file-name "$ASAN_RT_NAME")")"
test: improve handling of ASan under clang Running integration tests with ASan is somewhat tricky to begin with, as we need to pre-load the ASan runtime DSO for certain services (like dbus), otherwise they won't start or behave as expected. In case of gcc this is pretty easy, as we need the runtime DSO during compilation, so it's already present on the host system. For clang things get more complicated, as ASan is compiled in statically by default, thus to enable the necessary dynamic-ish behavior one needs to compile with -shared-libasan and then correctly set LD_PRELOAD_PATH, as the runtime libraries are not in a standard library path.
2019-05-25 04:35:52 +08:00
# As clang's ASan DSO is usually in a non-standard path, let's check if
# the environment is set accordingly. If not, warn the user and exit.
# We're not setting the LD_LIBRARY_PATH automagically here, because
# user should encounter (and fix) the same issue when running the unit
# tests (meson test)
test/test-functions: use binaries from $BUILD_DIR or installed system In Ubuntu CI, we test binaries from the installed system, not from $BUILD_DIR, so use the appropriate binary. Most of the calls to the binaries are part of checking/processing asan-built binaries, and so did not apply to Ubuntu CI, except for generating noise in the stderr log like: objdump: '/tmp/autopkgtest.83yGoI/build.fHB/src/test/TEST-01-BASIC/systemd-journald': No such file However this also applies to the call to systemd-nspawn, which the debian upstream test wrapper was sed-adjusting to use the installed binary instead of the binary in $BUILD_DIR. This commit allows removing that sed processing of the test-functions file during Ubuntu CI test.
2019-08-13 19:52:57 +08:00
if ldd "$SYSTEMD" | grep -q "libclang_rt.asan.*not found"; then
test: tidy up the ASan-related stuff
2021-03-07 05:21:30 +08:00
echo >&2 "clang's ASan DSO ($ASAN_RT_NAME) is not present in the runtime library path"
echo >&2 "Consider setting LD_LIBRARY_PATH=${ASAN_RT_PATH%/*}"
test: improve handling of ASan under clang Running integration tests with ASan is somewhat tricky to begin with, as we need to pre-load the ASan runtime DSO for certain services (like dbus), otherwise they won't start or behave as expected. In case of gcc this is pretty easy, as we need the runtime DSO during compilation, so it's already present on the host system. For clang things get more complicated, as ASan is compiled in statically by default, thus to enable the necessary dynamic-ish behavior one needs to compile with -shared-libasan and then correctly set LD_PRELOAD_PATH, as the runtime libraries are not in a standard library path.
2019-05-25 04:35:52 +08:00
exit 1
fi
else
echo >&2 "systemd is not linked against the ASan DSO"
echo >&2 "gcc does this by default, for clang compile with -shared-libasan"
exit 1
fi
test: tidy up the ASan-related stuff
2021-03-07 05:21:30 +08:00
echo "Detected ASan RT '$ASAN_RT_NAME' located at '$ASAN_RT_PATH'"
tests: create the asan wrapper automatically if systemd has been built with ASAN
2018-07-05 12:09:30 +08:00
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
find_qemu_bin() {
QEMU_BIN="${QEMU_BIN:-""}"
test-functions: don't nest KVM (#8883) Nested KVM is very flaky as we learnt from our CI. Hence, let's avoid KVM whenever we detect we are already running inside of KVM. Maybe one day nested KVM is fixed, at which point we can turn this on again, but for now let's simply avoid nested KVM, since reliable CI is more important than quick CI, I guess. And yes, avoiding KVM for our qemu runs does make things substantially slower, but I think it's not a complete loss. Inspired by @evverx' findings in: https://github.com/systemd/systemd/pull/8701#issuecomment-380213302
2018-05-03 02:06:13 +08:00
# SUSE and Red Hat call the binary qemu-kvm. Debian and Gentoo call it kvm.
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$QEMU_KVM"; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -n "$QEMU_BIN" ]] || QEMU_BIN="$(command -v kvm qemu-kvm 2>/dev/null | grep '^/' -m1)"
test-functions: don't nest KVM (#8883) Nested KVM is very flaky as we learnt from our CI. Hence, let's avoid KVM whenever we detect we are already running inside of KVM. Maybe one day nested KVM is fixed, at which point we can turn this on again, but for now let's simply avoid nested KVM, since reliable CI is more important than quick CI, I guess. And yes, avoiding KVM for our qemu runs does make things substantially slower, but I think it's not a complete loss. Inspired by @evverx' findings in: https://github.com/systemd/systemd/pull/8701#issuecomment-380213302
2018-05-03 02:06:13 +08:00
fi
test: rework run_qemu It tries to find a suitable QEMU binary and will use KVM if present. We can now configure QEMU from outside with 4 variables : - $QEMU_BIN : path to QEMU's binary - $KERNEL_APPEND : arguments appended to kernel cmdline - $KERNEL_BIN : path to a kernel Default /boot/vmlinuz-$KERNEL_VER - $INITRD : path to an initramfs Default /boot/initramfs-${KERNEL_VER}.img - $QEMU_SMP : number of CPU simulated by QEMU. Default 1 (from Alexander Graf's script: http://www.spinics.net/lists/kvm/msg72389.html)
2013-12-08 07:01:53 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -n "$ARCH" ]] || ARCH="$(uname -m)"
test: rework run_qemu It tries to find a suitable QEMU binary and will use KVM if present. We can now configure QEMU from outside with 4 variables : - $QEMU_BIN : path to QEMU's binary - $KERNEL_APPEND : arguments appended to kernel cmdline - $KERNEL_BIN : path to a kernel Default /boot/vmlinuz-$KERNEL_VER - $INITRD : path to an initramfs Default /boot/initramfs-${KERNEL_VER}.img - $QEMU_SMP : number of CPU simulated by QEMU. Default 1 (from Alexander Graf's script: http://www.spinics.net/lists/kvm/msg72389.html)
2013-12-08 07:01:53 +08:00
case $ARCH in
x86_64)
# QEMU's own build system calls it qemu-system-x86_64
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -n "$QEMU_BIN" ]] || QEMU_BIN="$(command -v qemu-system-x86_64 2>/dev/null | grep '^/' -m1)"
test: rework run_qemu It tries to find a suitable QEMU binary and will use KVM if present. We can now configure QEMU from outside with 4 variables : - $QEMU_BIN : path to QEMU's binary - $KERNEL_APPEND : arguments appended to kernel cmdline - $KERNEL_BIN : path to a kernel Default /boot/vmlinuz-$KERNEL_VER - $INITRD : path to an initramfs Default /boot/initramfs-${KERNEL_VER}.img - $QEMU_SMP : number of CPU simulated by QEMU. Default 1 (from Alexander Graf's script: http://www.spinics.net/lists/kvm/msg72389.html)
2013-12-08 07:01:53 +08:00
;;
i*86)
# new i386 version of QEMU
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -n "$QEMU_BIN" ]] || QEMU_BIN="$(command -v qemu-system-i386 2>/dev/null | grep '^/' -m1)"
test: rework run_qemu It tries to find a suitable QEMU binary and will use KVM if present. We can now configure QEMU from outside with 4 variables : - $QEMU_BIN : path to QEMU's binary - $KERNEL_APPEND : arguments appended to kernel cmdline - $KERNEL_BIN : path to a kernel Default /boot/vmlinuz-$KERNEL_VER - $INITRD : path to an initramfs Default /boot/initramfs-${KERNEL_VER}.img - $QEMU_SMP : number of CPU simulated by QEMU. Default 1 (from Alexander Graf's script: http://www.spinics.net/lists/kvm/msg72389.html)
2013-12-08 07:01:53 +08:00
# i386 version of QEMU
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -n "$QEMU_BIN" ]] || QEMU_BIN="$(command -v qemu 2>/dev/null | grep '^/' -m1)"
test: rework run_qemu It tries to find a suitable QEMU binary and will use KVM if present. We can now configure QEMU from outside with 4 variables : - $QEMU_BIN : path to QEMU's binary - $KERNEL_APPEND : arguments appended to kernel cmdline - $KERNEL_BIN : path to a kernel Default /boot/vmlinuz-$KERNEL_VER - $INITRD : path to an initramfs Default /boot/initramfs-${KERNEL_VER}.img - $QEMU_SMP : number of CPU simulated by QEMU. Default 1 (from Alexander Graf's script: http://www.spinics.net/lists/kvm/msg72389.html)
2013-12-08 07:01:53 +08:00
;;
test-functions: also add qemu define for ppc64
2016-11-20 11:45:23 +08:00
ppc64*)
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -n "$QEMU_BIN" ]] || QEMU_BIN="$(command -v qemu-system-ppc64 2>/dev/null | grep '^/' -m1)"
test-functions: also add qemu define for ppc64
2016-11-20 11:45:23 +08:00
;;
test: rework run_qemu It tries to find a suitable QEMU binary and will use KVM if present. We can now configure QEMU from outside with 4 variables : - $QEMU_BIN : path to QEMU's binary - $KERNEL_APPEND : arguments appended to kernel cmdline - $KERNEL_BIN : path to a kernel Default /boot/vmlinuz-$KERNEL_VER - $INITRD : path to an initramfs Default /boot/initramfs-${KERNEL_VER}.img - $QEMU_SMP : number of CPU simulated by QEMU. Default 1 (from Alexander Graf's script: http://www.spinics.net/lists/kvm/msg72389.html)
2013-12-08 07:01:53 +08:00
esac
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [[ ! -e "$QEMU_BIN" ]]; then
test: rework run_qemu It tries to find a suitable QEMU binary and will use KVM if present. We can now configure QEMU from outside with 4 variables : - $QEMU_BIN : path to QEMU's binary - $KERNEL_APPEND : arguments appended to kernel cmdline - $KERNEL_BIN : path to a kernel Default /boot/vmlinuz-$KERNEL_VER - $INITRD : path to an initramfs Default /boot/initramfs-${KERNEL_VER}.img - $QEMU_SMP : number of CPU simulated by QEMU. Default 1 (from Alexander Graf's script: http://www.spinics.net/lists/kvm/msg72389.html)
2013-12-08 07:01:53 +08:00
echo "Could not find a suitable QEMU binary" >&2
return 1
fi
}
test: use modern qemu numa arguments Upgrading to qemu 5.2 breaks TEST-36-NUMAPOLICY like: qemu-system-x86_64: total memory for NUMA nodes (0x0) should equal RAM size (0x20000000) Use the new (as in >=2014) form of memdev in test 36: -object memory-backend-ram,id=mem0,size=512M -numa node,memdev=mem0,nodeid=0 Since some target systems are as old as qemu 1.5.3 (CentOS7) but the new kind to specify was added in qemu 2.1 this needs to add version parsing and add the argument only when qemu is >=5.2. Fixes #17986. Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
2020-12-15 19:05:14 +08:00
# Compares argument #1=X.Y.Z (X&Y&Z = numeric) to the version of the installed qemu
# returns 0 if newer or equal
# returns 1 if older
# returns 2 if failing
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
qemu_min_version() {
test: use modern qemu numa arguments Upgrading to qemu 5.2 breaks TEST-36-NUMAPOLICY like: qemu-system-x86_64: total memory for NUMA nodes (0x0) should equal RAM size (0x20000000) Use the new (as in >=2014) form of memdev in test 36: -object memory-backend-ram,id=mem0,size=512M -numa node,memdev=mem0,nodeid=0 Since some target systems are as old as qemu 1.5.3 (CentOS7) but the new kind to specify was added in qemu 2.1 this needs to add version parsing and add the argument only when qemu is >=5.2. Fixes #17986. Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
2020-12-15 19:05:14 +08:00
find_qemu_bin || return 2
# get version from binary
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local qemu_ver
qemu_ver="$("$QEMU_BIN" --version | awk '/^QEMU emulator version ([0-9]*\.[0-9]*\.[0-9]*)/ {print $4}')"
test: use modern qemu numa arguments Upgrading to qemu 5.2 breaks TEST-36-NUMAPOLICY like: qemu-system-x86_64: total memory for NUMA nodes (0x0) should equal RAM size (0x20000000) Use the new (as in >=2014) form of memdev in test 36: -object memory-backend-ram,id=mem0,size=512M -numa node,memdev=mem0,nodeid=0 Since some target systems are as old as qemu 1.5.3 (CentOS7) but the new kind to specify was added in qemu 2.1 this needs to add version parsing and add the argument only when qemu is >=5.2. Fixes #17986. Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
2020-12-15 19:05:14 +08:00
# Check version string format
echo "$qemu_ver" | grep -q '^[0-9]*\.[0-9]*\.[0-9]*$' || return 2
echo "$1" | grep -q '^[0-9]*\.[0-9]*\.[0-9]*$' || return 2
# compare as last command to return that value
printf "%s\n%s\n" "$1" "$qemu_ver" | sort -V -C
}
tests: track and check for timeouts If run_qemu() exits with non-zero, this either meant that QEMU was not available (which should be a SKIP) or that QEMU timed out if $QEMU_TIMEOUT was set (which then should be a FAIL). Limit the exit code of run_qemu() to QEMU availability only, and track timeouts separately through the new $TIMED_OUT variable, which is then checked in check_result_qemu(). Do the same for $NSPAWN_TIMEOUT and run_nspawn() so that nspawn and QEMU work similarly.
2016-06-24 18:11:19 +08:00
# Return 0 if QEMU did run (then you must check the result state/logs for actual
# success), or 1 if QEMU is not available.
test: remove duplicated code
2013-11-06 06:32:56 +08:00
run_qemu() {
test-functions: search for the initrd and kernel according to the BLS Also search for the initrd and kernel according to the boot loader specification.
2015-02-03 17:43:24 +08:00
if [ -f /etc/machine-id ]; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
read -r MACHINE_ID </etc/machine-id
test: do not hardcode location of EFI partition
2017-11-17 21:09:11 +08:00
[ -z "$INITRD" ] && [ -e "$EFI_MOUNT/$MACHINE_ID/$KERNEL_VER/initrd" ] \
&& INITRD="$EFI_MOUNT/$MACHINE_ID/$KERNEL_VER/initrd"
[ -z "$KERNEL_BIN" ] && [ -e "$EFI_MOUNT/$MACHINE_ID/$KERNEL_VER/linux" ] \
&& KERNEL_BIN="$EFI_MOUNT/$MACHINE_ID/$KERNEL_VER/linux"
test-functions: search for the initrd and kernel according to the BLS Also search for the initrd and kernel according to the boot loader specification.
2015-02-03 17:43:24 +08:00
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local CONSOLE=ttyS0
test/test-functions: on PPC64 use hvc0 console
2018-02-20 20:01:40 +08:00
test: correctly clean up test artifacts Remove the artifact files indicating test result (testok, failed, and skipped) just before running the test so we always get the latest and most relevant result instead of incorrectly consuming previous results. Discovered in https://github.com/systemd/systemd/pull/15378#issuecomment-616801873
2020-04-21 19:18:36 +08:00
rm -f "$initdir"/{testok,failed,skipped}
test: run tests directly from the loopback device Before, we'd copy the test tree into nspawn-root, and run the tests from there. This is OK, and doesn't actually take much extra time. But it uses quite a lot of extra disk space. So let's make things a bit more efficient by running directly from the image file. We still run the unprivileged nspawn tests from a copy. Once the kernel implements fs shift, we can do away with that too.
2019-12-13 21:21:31 +08:00
# make sure the initdir is not mounted to avoid concurrent access
cleanup_initdir
umount_loopback
tests: fix kernel/initrd searching (Arch) We can run sudo make run instead of the sudo make run KERNEL_BIN=/boot/vmlinuz-linux INITRD=/boot/initramfs-linux.img See https://github.com/systemd/systemd/issues/4607
2016-11-11 10:45:38 +08:00
if [[ ! "$KERNEL_BIN" ]]; then
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$LOOKS_LIKE_ARCH"; then
tests: fix kernel/initrd searching (Arch) We can run sudo make run instead of the sudo make run KERNEL_BIN=/boot/vmlinuz-linux INITRD=/boot/initramfs-linux.img See https://github.com/systemd/systemd/issues/4607
2016-11-11 10:45:38 +08:00
KERNEL_BIN=/boot/vmlinuz-linux
else
test/test-functions: on PP64 use vmlinux At least on Ubuntu, ppc64el uses vmlinux-, not vmlinuz. With this, it should be possible to run qemu tests on ppc64el as part of Ubuntu autopkgtests.
2018-02-20 04:47:41 +08:00
[ "$ARCH" ] || ARCH=$(uname -m)
case $ARCH in
ppc64*)
test: use a correct kernel binary on ppc64le machines Ubuntu calls the kernel binary as `vmlinux-*`, but RHEL/CentOS uses the "standard" `vmlinuz-` prefix. Follow-up to eaa602cb14217f2623c9c8c44b6f3012dc4d0b24.
2021-08-09 20:32:43 +08:00
# Ubuntu ppc64* calls the kernel binary as vmlinux-*, RHEL/CentOS
# uses the "standard" vmlinuz- prefix
[[ -e "/boot/vmlinux-$KERNEL_VER" ]] && KERNEL_BIN="/boot/vmlinux-$KERNEL_VER" || KERNEL_BIN="/boot/vmlinuz-$KERNEL_VER"
CONSOLE=hvc0
;;
test/test-functions: on PP64 use vmlinux At least on Ubuntu, ppc64el uses vmlinux-, not vmlinuz. With this, it should be possible to run qemu tests on ppc64el as part of Ubuntu autopkgtests.
2018-02-20 04:47:41 +08:00
*)
test: use a correct kernel binary on ppc64le machines Ubuntu calls the kernel binary as `vmlinux-*`, but RHEL/CentOS uses the "standard" `vmlinuz-` prefix. Follow-up to eaa602cb14217f2623c9c8c44b6f3012dc4d0b24.
2021-08-09 20:32:43 +08:00
KERNEL_BIN="/boot/vmlinuz-$KERNEL_VER"
;;
test/test-functions: on PP64 use vmlinux At least on Ubuntu, ppc64el uses vmlinux-, not vmlinuz. With this, it should be possible to run qemu tests on ppc64el as part of Ubuntu autopkgtests.
2018-02-20 04:47:41 +08:00
esac
tests: fix kernel/initrd searching (Arch) We can run sudo make run instead of the sudo make run KERNEL_BIN=/boot/vmlinuz-linux INITRD=/boot/initramfs-linux.img See https://github.com/systemd/systemd/issues/4607
2016-11-11 10:45:38 +08:00
fi
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local default_fedora_initrd="/boot/initramfs-${KERNEL_VER}.img"
local default_debian_initrd="/boot/initrd.img-${KERNEL_VER}"
local default_arch_initrd="/boot/initramfs-linux-fallback.img"
local default_suse_initrd="/boot/initrd-${KERNEL_VER}"
tests: fix kernel/initrd searching (Arch) We can run sudo make run instead of the sudo make run KERNEL_BIN=/boot/vmlinuz-linux INITRD=/boot/initramfs-linux.img See https://github.com/systemd/systemd/issues/4607
2016-11-11 10:45:38 +08:00
if [[ ! "$INITRD" ]]; then
if [[ -e "$default_fedora_initrd" ]]; then
INITRD="$default_fedora_initrd"
elif [[ "$LOOKS_LIKE_DEBIAN" && -e "$default_debian_initrd" ]]; then
INITRD="$default_debian_initrd"
elif [[ "$LOOKS_LIKE_ARCH" && -e "$default_arch_initrd" ]]; then
INITRD="$default_arch_initrd"
tests: adapt test-functions to run tests on SUSE (#6270)
2017-07-07 02:30:14 +08:00
elif [[ "$LOOKS_LIKE_SUSE" && -e "$default_suse_initrd" ]]; then
INITRD="$default_suse_initrd"
tests: fix kernel/initrd searching (Arch) We can run sudo make run instead of the sudo make run KERNEL_BIN=/boot/vmlinuz-linux INITRD=/boot/initramfs-linux.img See https://github.com/systemd/systemd/issues/4607
2016-11-11 10:45:38 +08:00
fi
fi
test: try to determine QEMU_SMP dynamically If the QEMU_SMP value has not been explicitly set, try to determine it from the number of online CPUs using the nproc utility. If this approach fails, fall back to the default value QEMU_SMP=1. This change should significantly help when running integration tests under QEMU on multicore systems.
2019-03-05 23:08:00 +08:00
# If QEMU_SMP was not explicitly set, try to determine the value 'dynamically'
# i.e. use the number of online CPUs on the host machine. If the nproc utility
# is not installed or there's some other error when calling it, fall back
# to the original value (QEMU_SMP=1).
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [[ -z "${QEMU_SMP:=}" ]]; then
test: try to determine QEMU_SMP dynamically If the QEMU_SMP value has not been explicitly set, try to determine it from the number of online CPUs using the nproc utility. If this approach fails, fall back to the default value QEMU_SMP=1. This change should significantly help when running integration tests under QEMU on multicore systems.
2019-03-05 23:08:00 +08:00
if ! QEMU_SMP=$(nproc); then
dwarn "nproc utility is not installed, falling back to QEMU_SMP=1"
QEMU_SMP=1
fi
fi
test: rework run_qemu It tries to find a suitable QEMU binary and will use KVM if present. We can now configure QEMU from outside with 4 variables : - $QEMU_BIN : path to QEMU's binary - $KERNEL_APPEND : arguments appended to kernel cmdline - $KERNEL_BIN : path to a kernel Default /boot/vmlinuz-$KERNEL_VER - $INITRD : path to an initramfs Default /boot/initramfs-${KERNEL_VER}.img - $QEMU_SMP : number of CPU simulated by QEMU. Default 1 (from Alexander Graf's script: http://www.spinics.net/lists/kvm/msg72389.html)
2013-12-08 07:01:53 +08:00
find_qemu_bin || return 1
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
# Umount initdir to avoid concurrent access to the filesystem
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
_umount_dir "$initdir"
local kernel_params=()
local qemu_options=()
local qemu_cmd=("$QEMU_BIN")
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
tests: add UNIFIED_CGROUP_HIERARCHY=[default|hybrid] (#4675) This will simplify testing a bit. Mainly for https://github.com/systemd/systemd/pull/4670
2016-11-16 00:38:04 +08:00
if [[ "$UNIFIED_CGROUP_HIERARCHY" = "yes" ]]; then
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
kernel_params+=("systemd.unified_cgroup_hierarchy=yes")
tests: add UNIFIED_CGROUP_HIERARCHY=[default|hybrid] (#4675) This will simplify testing a bit. Mainly for https://github.com/systemd/systemd/pull/4670
2016-11-16 00:38:04 +08:00
elif [[ "$UNIFIED_CGROUP_HIERARCHY" = "no" ]]; then
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
kernel_params+=("systemd.unified_cgroup_hierarchy=no" "systemd.legacy_systemd_cgroup_controller=yes")
tests: add UNIFIED_CGROUP_HIERARCHY=[default|hybrid] (#4675) This will simplify testing a bit. Mainly for https://github.com/systemd/systemd/pull/4670
2016-11-16 00:38:04 +08:00
elif [[ "$UNIFIED_CGROUP_HIERARCHY" = "hybrid" ]]; then
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
kernel_params+=("systemd.unified_cgroup_hierarchy=no" "systemd.legacy_systemd_cgroup_controller=no")
tests: add UNIFIED_CGROUP_HIERARCHY=[default|hybrid] (#4675) This will simplify testing a bit. Mainly for https://github.com/systemd/systemd/pull/4670
2016-11-16 00:38:04 +08:00
elif [[ "$UNIFIED_CGROUP_HIERARCHY" != "default" ]]; then
dfatal "Unknown UNIFIED_CGROUP_HIERARCHY. Got $UNIFIED_CGROUP_HIERARCHY, expected [yes|no|hybrid|default]"
exit 1
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$LOOKS_LIKE_SUSE"; then
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
kernel_params+=("rd.hostonly=0")
scripts: use 4 space indentation We had all kinds of indentation: 2 sp, 3 sp, 4 sp, 8 sp, and mixed. 4 sp was the most common, in particular the majority of scripts under test/ used that. Let's standarize on 4 sp, because many commandlines are long and there's a lot of nesting, and with 8sp indentation less stuff fits. 4 sp also seems to be the default indentation, so this will make it less likely that people will mess up if they don't load the editor config. (I think people often use vi, and vi has no support to load project-wide configuration automatically. We distribute a .vimrc file, but it is not loaded by default, and even the instructions in it seem to discourage its use for security reasons.) Also remove the few vim config lines that were left. We should either have them on all files, or none. Also remove some strange stuff like '#!/bin/env bash', yikes.
2019-04-04 20:10:42 +08:00
fi
tests: adapt test-functions to run tests on SUSE (#6270)
2017-07-07 02:30:14 +08:00
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
kernel_params+=(
test: specify the rootfs by label otherwise the boot might fail when attaching multiple SATA devices to the VM.
2021-09-14 00:14:05 +08:00
"root=LABEL=systemd_boot"
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
"rw"
"raid=noautodetect"
"rd.luks=0"
"loglevel=2"
"init=$PATH_TO_INIT"
"console=$CONSOLE"
"SYSTEMD_UNIT_PATH=/usr/lib/systemd/tests/testdata/testsuite-$1.units:/usr/lib/systemd/tests/testdata/units:"
"systemd.unit=testsuite.target"
"systemd.wants=testsuite-$1.service"
)
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if ! get_bool "$INTERACTIVE_DEBUG"; then
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
kernel_params+=("systemd.wants=end.service")
test: move TEST-01-BASIC setup to static files After this commit, only tests that were converted will work, because the interface to test_run() changed.
2019-12-10 01:56:13 +08:00
fi
test: fix public/private image detection and prefer the latter one
2020-05-17 16:46:34 +08:00
[ -e "$IMAGE_PRIVATE" ] && image="$IMAGE_PRIVATE" || image="$IMAGE_PUBLIC"
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
qemu_options+=(
-smp "$QEMU_SMP"
-net none
-m "$QEMU_MEM"
-nographic
-kernel "$KERNEL_BIN"
-drive "format=raw,cache=unsafe,file=$image"
)
if [[ -n "${QEMU_OPTIONS:=}" ]]; then
local user_qemu_options
read -ra user_qemu_options <<< "$QEMU_OPTIONS"
qemu_options+=("${user_qemu_options[@]}")
fi
if [[ -n "${KERNEL_APPEND:=}" ]]; then
local user_kernel_append
read -ra user_kernel_append <<< "$KERNEL_APPEND"
kernel_params+=("${user_kernel_append[@]}")
fi
test: rework run_qemu It tries to find a suitable QEMU binary and will use KVM if present. We can now configure QEMU from outside with 4 variables : - $QEMU_BIN : path to QEMU's binary - $KERNEL_APPEND : arguments appended to kernel cmdline - $KERNEL_BIN : path to a kernel Default /boot/vmlinuz-$KERNEL_VER - $INITRD : path to an initramfs Default /boot/initramfs-${KERNEL_VER}.img - $QEMU_SMP : number of CPU simulated by QEMU. Default 1 (from Alexander Graf's script: http://www.spinics.net/lists/kvm/msg72389.html)
2013-12-08 07:01:53 +08:00
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if [[ "$INITRD" ]] && ! get_bool "$SKIP_INITRD"; then
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
qemu_options+=(-initrd "$INITRD")
test: rework run_qemu It tries to find a suitable QEMU binary and will use KVM if present. We can now configure QEMU from outside with 4 variables : - $QEMU_BIN : path to QEMU's binary - $KERNEL_APPEND : arguments appended to kernel cmdline - $KERNEL_BIN : path to a kernel Default /boot/vmlinuz-$KERNEL_VER - $INITRD : path to an initramfs Default /boot/initramfs-${KERNEL_VER}.img - $QEMU_SMP : number of CPU simulated by QEMU. Default 1 (from Alexander Graf's script: http://www.spinics.net/lists/kvm/msg72389.html)
2013-12-08 07:01:53 +08:00
fi
test: allow use of nested KVM Introduce TEST_NESTED_KVM variable, which allows use of nested KVM for QEMU VMs (which is disabled by default by KVM autodetection).
2019-09-27 05:29:38 +08:00
# Let's use KVM if possible
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if [[ -c /dev/kvm ]] && get_bool $QEMU_KVM; then
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
qemu_options+=(-machine "accel=kvm" -enable-kvm -cpu host)
test: make qemu use kvm if available If we're running in a KVM virtualized environment (such as Semaphore's Docker enabled platform), pass '-cpu host' to QEMU.
2015-11-19 20:17:15 +08:00
fi
tests: add test for https://github.com/systemd/systemd/issues/2730
2016-02-25 02:22:06 +08:00
if [[ "$QEMU_TIMEOUT" != "infinity" ]]; then
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
qemu_cmd=(timeout --foreground "$QEMU_TIMEOUT" "$QEMU_BIN")
tests: add test for https://github.com/systemd/systemd/issues/2730
2016-02-25 02:22:06 +08:00
fi
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
(set -x; "${qemu_cmd[@]}" "${qemu_options[@]}" -append "${kernel_params[*]}")
tests: track and check for timeouts If run_qemu() exits with non-zero, this either meant that QEMU was not available (which should be a SKIP) or that QEMU timed out if $QEMU_TIMEOUT was set (which then should be a FAIL). Limit the exit code of run_qemu() to QEMU availability only, and track timeouts separately through the new $TIMED_OUT variable, which is then checked in check_result_qemu(). Do the same for $NSPAWN_TIMEOUT and run_nspawn() so that nspawn and QEMU work similarly.
2016-06-24 18:11:19 +08:00
rc=$?
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if [ "$rc" -eq 124 ] && [ "$QEMU_TIMEOUT" != "infinity" ]; then
derror "Test timed out after ${QEMU_TIMEOUT}s"
tests: track and check for timeouts If run_qemu() exits with non-zero, this either meant that QEMU was not available (which should be a SKIP) or that QEMU timed out if $QEMU_TIMEOUT was set (which then should be a FAIL). Limit the exit code of run_qemu() to QEMU availability only, and track timeouts separately through the new $TIMED_OUT variable, which is then checked in check_result_qemu(). Do the same for $NSPAWN_TIMEOUT and run_nspawn() so that nspawn and QEMU work similarly.
2016-06-24 18:11:19 +08:00
TIMED_OUT=1
else
[ "$rc" != 0 ] && derror "QEMU failed with exit code $rc"
fi
return 0
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
test: merge check_nspawn() into run_nspawn() This makes nspawn tests symmetric with run_qemu() which also exits with 1 if QEMU is not available.
2016-06-24 22:23:39 +08:00
# Return 0 if nspawn did run (then you must check the result state/logs for actual
# success), or 1 if nspawn is not available.
test: remove duplicated code
2013-11-06 06:32:56 +08:00
run_nspawn() {
test: merge check_nspawn() into run_nspawn() This makes nspawn tests symmetric with run_qemu() which also exits with 1 if QEMU is not available.
2016-06-24 22:23:39 +08:00
[[ -d /run/systemd/system ]] || return 1
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
rm -f "${initdir:?}"/{testok,failed,skipped}
local nspawn_cmd=()
local nspawn_options=(
"--register=no"
"--kill-signal=SIGKILL"
"--directory=${1:?}"
"--setenv=SYSTEMD_UNIT_PATH=/usr/lib/systemd/tests/testdata/testsuite-$2.units:/usr/lib/systemd/tests/testdata/units:"
)
local kernel_params=(
"$PATH_TO_INIT"
"systemd.unit=testsuite.target"
"systemd.wants=testsuite-$2.service"
test: move TEST-01-BASIC setup to static files After this commit, only tests that were converted will work, because the interface to test_run() changed.
2019-12-10 01:56:13 +08:00
)
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if ! get_bool "$INTERACTIVE_DEBUG"; then
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
kernel_params+=("systemd.wants=end.service")
test: move TEST-01-BASIC setup to static files After this commit, only tests that were converted will work, because the interface to test_run() changed.
2019-12-10 01:56:13 +08:00
fi
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
if [[ -n "${NSPAWN_ARGUMENTS:=}" ]]; then
local user_nspawn_arguments
read -ra user_nspawn_arguments <<< "$NSPAWN_ARGUMENTS"
nspawn_options+=("${user_nspawn_arguments[@]}")
fi
if [[ -n "${KERNEL_APPEND:=}" ]]; then
local user_kernel_append
read -ra user_kernel_append <<< "$KERNEL_APPEND"
kernel_params+=("${user_kernel_append[@]}")
tests: don't rely on the host's systemd-nspawn, don't register test machine (#3023) Fixes: $ cd test/TEST-07-ISSUE-1981/ $ sudo make clean setup run ... timeout: failed to run command ‘systemd-nspawn’: No such file or directory ... TEST RUN: https://github.com/systemd/systemd/issues/1981 [FAILED] Makefile:10: recipe for target 'run' failed make: *** [run] Error 1
2016-04-13 21:57:06 +08:00
fi
tests: introduce UNIFIED_CGROUP_HIERARCHY (#3419) There are many cgroups-related changes (thanks, @htejun!) This commit will simplify testing a bit. Use: make run UNIFIED_CGROUP_HIERARCHY=yes to enable cgroup-v2 make run UNIFIED_CGROUP_HIERARCHY=no to enable cgroup-v1
2016-06-03 17:17:00 +08:00
tests: add UNIFIED_CGROUP_HIERARCHY=[default|hybrid] (#4675) This will simplify testing a bit. Mainly for https://github.com/systemd/systemd/pull/4670
2016-11-16 00:38:04 +08:00
if [[ "$UNIFIED_CGROUP_HIERARCHY" = "hybrid" ]]; then
nspawn: rename UNIFIED_CGROUP_HIERARCHY to SYSTEMD_NSPAWN_UNIFIED_HIERARCHY We should never have used an unprefixed environment variable name. All other systemd-nspawn variables have the "SYSTEMD_NSPAWN_" prefix, and all other systemd variables have the "SYSTEMD_" prefix. The new variable name takes precedence, but we fall back to checking the old one. If only the old one is found, a warning is emitted. In addition, SYSTEMD_NSPAWN_UNIFIED_HIERARCHY="" is accepted as an override to avoid looking for the old variable name. We have a variable with the same name ($UNIFIED_CGROUP_HIERARCHY) in tests, which governs both systemd-nspawn and qemu behaviour. It is not renamed.
2019-09-27 20:17:41 +08:00
dwarn "nspawn doesn't support SYSTEMD_NSPAWN_UNIFIED_HIERARCHY=hybrid, skipping"
tests: add UNIFIED_CGROUP_HIERARCHY=[default|hybrid] (#4675) This will simplify testing a bit. Mainly for https://github.com/systemd/systemd/pull/4670
2016-11-16 00:38:04 +08:00
exit
elif [[ "$UNIFIED_CGROUP_HIERARCHY" = "yes" || "$UNIFIED_CGROUP_HIERARCHY" = "no" ]]; then
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
nspawn_cmd+=(env "SYSTEMD_NSPAWN_UNIFIED_HIERARCHY=$UNIFIED_CGROUP_HIERARCHY")
tests: add UNIFIED_CGROUP_HIERARCHY=[default|hybrid] (#4675) This will simplify testing a bit. Mainly for https://github.com/systemd/systemd/pull/4670
2016-11-16 00:38:04 +08:00
elif [[ "$UNIFIED_CGROUP_HIERARCHY" = "default" ]]; then
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
nspawn_cmd+=(env "--unset=UNIFIED_CGROUP_HIERARCHY" "--unset=SYSTEMD_NSPAWN_UNIFIED_HIERARCHY")
tests: add UNIFIED_CGROUP_HIERARCHY=[default|hybrid] (#4675) This will simplify testing a bit. Mainly for https://github.com/systemd/systemd/pull/4670
2016-11-16 00:38:04 +08:00
else
dfatal "Unknown UNIFIED_CGROUP_HIERARCHY. Got $UNIFIED_CGROUP_HIERARCHY, expected [yes|no|hybrid|default]"
exit 1
fi
tests: introduce UNIFIED_CGROUP_HIERARCHY (#3419) There are many cgroups-related changes (thanks, @htejun!) This commit will simplify testing a bit. Use: make run UNIFIED_CGROUP_HIERARCHY=yes to enable cgroup-v2 make run UNIFIED_CGROUP_HIERARCHY=no to enable cgroup-v1
2016-06-03 17:17:00 +08:00
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
if [[ "$NSPAWN_TIMEOUT" != "infinity" ]]; then
nspawn_cmd+=(timeout --foreground "$NSPAWN_TIMEOUT" "$SYSTEMD_NSPAWN")
else
nspawn_cmd+=("$SYSTEMD_NSPAWN")
fi
(set -x; "${nspawn_cmd[@]}" "${nspawn_options[@]}" "${kernel_params[@]}")
tests: track and check for timeouts If run_qemu() exits with non-zero, this either meant that QEMU was not available (which should be a SKIP) or that QEMU timed out if $QEMU_TIMEOUT was set (which then should be a FAIL). Limit the exit code of run_qemu() to QEMU availability only, and track timeouts separately through the new $TIMED_OUT variable, which is then checked in check_result_qemu(). Do the same for $NSPAWN_TIMEOUT and run_nspawn() so that nspawn and QEMU work similarly.
2016-06-24 18:11:19 +08:00
rc=$?
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if [ "$rc" -eq 124 ] && [ "$NSPAWN_TIMEOUT" != "infinity" ]; then
derror "Test timed out after ${NSPAWN_TIMEOUT}s"
tests: track and check for timeouts If run_qemu() exits with non-zero, this either meant that QEMU was not available (which should be a SKIP) or that QEMU timed out if $QEMU_TIMEOUT was set (which then should be a FAIL). Limit the exit code of run_qemu() to QEMU availability only, and track timeouts separately through the new $TIMED_OUT variable, which is then checked in check_result_qemu(). Do the same for $NSPAWN_TIMEOUT and run_nspawn() so that nspawn and QEMU work similarly.
2016-06-24 18:11:19 +08:00
TIMED_OUT=1
else
[ "$rc" != 0 ] && derror "nspawn failed with exit code $rc"
fi
return 0
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
test-50-dissect: move minimal image setup to common setup function So that it can be re-used for other tests by simply setting TEST_INSTALL_VERITY_MINIMAL=1 in test.sh, without having to replicate the setup commands.
2020-08-28 21:17:31 +08:00
# Build two very minimal root images, with two units, one is the same and one is different across them
install_verity_minimal() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Set up a set of minimal images for verity verification"
if [ -e "$initdir/usr/share/minimal.raw" ]; then
test-50-dissect: move minimal image setup to common setup function So that it can be re-used for other tests by simply setting TEST_INSTALL_VERITY_MINIMAL=1 in test.sh, without having to replicate the setup commands.
2020-08-28 21:17:31 +08:00
return
fi
if ! command -v mksquashfs >/dev/null 2>&1; then
dfatal "mksquashfs not found"
exit 1
fi
if ! command -v veritysetup >/dev/null 2>&1; then
dfatal "veritysetup not found"
exit 1
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
# Local modifications of some global variables is intentional in this
# subshell (SC2030)
# shellcheck disable=SC2030
test-50-dissect: move minimal image setup to common setup function So that it can be re-used for other tests by simply setting TEST_INSTALL_VERITY_MINIMAL=1 in test.sh, without having to replicate the setup commands.
2020-08-28 21:17:31 +08:00
(
BASICTOOLS=(
bash
cat
Add ExtensionImages directive to form overlays Add support for overlaying images for services on top of their root fs, using a read-only overlay.
2021-02-22 20:20:33 +08:00
grep
test-50-dissect: move minimal image setup to common setup function So that it can be re-used for other tests by simply setting TEST_INSTALL_VERITY_MINIMAL=1 in test.sh, without having to replicate the setup commands.
2020-08-28 21:17:31 +08:00
mount
sleep
)
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
oldinitdir="$initdir"
rm -rfv "$TESTDIR/minimal"
export initdir="$TESTDIR/minimal"
mkdir -p "$initdir/usr/lib/systemd/system" "$initdir/usr/lib/extension-release.d" "$initdir/etc" "$initdir/var/tmp" "$initdir/opt"
test-50-dissect: move minimal image setup to common setup function So that it can be re-used for other tests by simply setting TEST_INSTALL_VERITY_MINIMAL=1 in test.sh, without having to replicate the setup commands.
2020-08-28 21:17:31 +08:00
setup_basic_dirs
install_basic_tools
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
# Shellcheck treats [[ -v VAR ]] as an assignment to avoid a different
# issue, thus falsely triggering SC2030 in this case
# See: koalaman/shellcheck#1409
test: fix TEST-50-DISSECT under sanitizers This test would normally get stuck when trying to mount the verity image due to: systemd-udevd[299]: dm-0: '/usr/sbin/dmsetup udevflags 6293812'(err) '==371==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD.' systemd-udevd[299]: dm-0: Process '/usr/sbin/dmsetup udevflags 6293812' failed with exit code 1 ... systemd-udevd[299]: dm-0: '/usr/sbin/dmsetup udevcomplete 6293812'(err) '==372==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD.' systemd-udevd[299]: dm-0: Process '/usr/sbin/dmsetup udevcomplete 6293812' failed with exit code 1. systemd-udevd[299]: dm-0: Command "/usr/sbin/dmsetup udevcomplete 6293812" returned 1 (error), ignoring. so let's add a simple udev rule which sets $LD_PRELOAD for the block subsystem. Also, install the ASan library along with necessary dependencies into the verity minimal image, to get rid of the annoying (yet harmless) errors about missing library from $LD_LIBRARY.
2021-03-07 05:22:32 +08:00
if [[ -v ASAN_RT_PATH ]]; then
# If we're compiled with ASan, install the ASan RT (and its dependencies)
# into the verity images to get rid of the annoying errors about
# missing $LD_PRELOAD libraries.
inst_libs "$ASAN_RT_PATH"
inst_library "$ASAN_RT_PATH"
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
cp "$os_release" "$initdir/usr/lib/os-release"
ln -s ../usr/lib/os-release "$initdir/etc/os-release"
touch "$initdir/etc/machine-id" "$initdir/etc/resolv.conf"
touch "$initdir/opt/some_file"
echo MARKER=1 >>"$initdir/usr/lib/os-release"
echo -e "[Service]\nExecStartPre=cat /usr/lib/os-release\nExecStart=sleep 120" >"$initdir/usr/lib/systemd/system/app0.service"
cp "$initdir/usr/lib/systemd/system/app0.service" "$initdir/usr/lib/systemd/system/app0-foo.service"
mksquashfs "$initdir" "$oldinitdir/usr/share/minimal_0.raw"
veritysetup format "$oldinitdir/usr/share/minimal_0.raw" "$oldinitdir/usr/share/minimal_0.verity" | \
grep '^Root hash:' | cut -f2 | tr -d '\n' >"$oldinitdir/usr/share/minimal_0.roothash"
sed -i "s/MARKER=1/MARKER=2/g" "$initdir/usr/lib/os-release"
rm "$initdir/usr/lib/systemd/system/app0-foo.service"
cp "$initdir/usr/lib/systemd/system/app0.service" "$initdir/usr/lib/systemd/system/app0-bar.service"
mksquashfs "$initdir" "$oldinitdir/usr/share/minimal_1.raw"
veritysetup format "$oldinitdir/usr/share/minimal_1.raw" "$oldinitdir/usr/share/minimal_1.verity" | \
grep '^Root hash:' | cut -f2 | tr -d '\n' >"$oldinitdir/usr/share/minimal_1.roothash"
Add ExtensionImages directive to form overlays Add support for overlaying images for services on top of their root fs, using a read-only overlay.
2021-02-22 20:20:33 +08:00
# Rolling distros like Arch do not set VERSION_ID
local version_id=""
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if grep -q "^VERSION_ID=" "$os_release"; then
version_id="$(grep "^VERSION_ID=" "$os_release")"
Add ExtensionImages directive to form overlays Add support for overlaying images for services on top of their root fs, using a read-only overlay.
2021-02-22 20:20:33 +08:00
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
export initdir="$TESTDIR/app0"
mkdir -p "$initdir/usr/lib/extension-release.d" "$initdir/usr/lib/systemd/system" "$initdir/opt"
grep "^ID=" "$os_release" >"$initdir/usr/lib/extension-release.d/extension-release.app0"
echo "${version_id}" >>"$initdir/usr/lib/extension-release.d/extension-release.app0"
cat >"$initdir/usr/lib/systemd/system/app0.service" <<EOF
Add ExtensionImages directive to form overlays Add support for overlaying images for services on top of their root fs, using a read-only overlay.
2021-02-22 20:20:33 +08:00
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/opt/script0.sh
EOF
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
cat >"$initdir/opt/script0.sh" <<EOF
Add ExtensionImages directive to form overlays Add support for overlaying images for services on top of their root fs, using a read-only overlay.
2021-02-22 20:20:33 +08:00
#!/bin/bash
set -e
test -e /usr/lib/os-release
cat /usr/lib/extension-release.d/extension-release.app0
EOF
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
chmod +x "$initdir/opt/script0.sh"
echo MARKER=1 >"$initdir/usr/lib/systemd/system/some_file"
mksquashfs "$initdir" "$oldinitdir/usr/share/app0.raw"
export initdir="$TESTDIR/app1"
mkdir -p "$initdir/usr/lib/extension-release.d" "$initdir/usr/lib/systemd/system" "$initdir/opt"
extension-release: search for other files if expected name not found In some cases image names are unpredictable - some orchestrators/deployment tools like to mangle names to suit their internal formats. In these cases, the requirement that the extension-release file matches exactly the image name where it's contained cannot work. Allow falling back to loading the first regular file which name starts with 'extension-release' located in /usr/lib/extension-release.d/ and tagged with a user.extension-release.strict extended attribute with a true value, if the one with the expected name cannot be found.
2021-07-23 03:41:34 +08:00
grep "^ID=" "$os_release" >"$initdir/usr/lib/extension-release.d/extension-release.app2"
echo "${version_id}" >>"$initdir/usr/lib/extension-release.d/extension-release.app2"
setfattr -n user.extension-release.strict -v false "$initdir/usr/lib/extension-release.d/extension-release.app2"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
cat >"$initdir/usr/lib/systemd/system/app1.service" <<EOF
Add ExtensionImages directive to form overlays Add support for overlaying images for services on top of their root fs, using a read-only overlay.
2021-02-22 20:20:33 +08:00
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/opt/script1.sh
EOF
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
cat >"$initdir/opt/script1.sh" <<EOF
Add ExtensionImages directive to form overlays Add support for overlaying images for services on top of their root fs, using a read-only overlay.
2021-02-22 20:20:33 +08:00
#!/bin/bash
set -e
test -e /usr/lib/os-release
extension-release: search for other files if expected name not found In some cases image names are unpredictable - some orchestrators/deployment tools like to mangle names to suit their internal formats. In these cases, the requirement that the extension-release file matches exactly the image name where it's contained cannot work. Allow falling back to loading the first regular file which name starts with 'extension-release' located in /usr/lib/extension-release.d/ and tagged with a user.extension-release.strict extended attribute with a true value, if the one with the expected name cannot be found.
2021-07-23 03:41:34 +08:00
cat /usr/lib/extension-release.d/extension-release.app2
Add ExtensionImages directive to form overlays Add support for overlaying images for services on top of their root fs, using a read-only overlay.
2021-02-22 20:20:33 +08:00
EOF
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
chmod +x "$initdir/opt/script1.sh"
echo MARKER=1 >"$initdir/usr/lib/systemd/system/other_file"
mksquashfs "$initdir" "$oldinitdir/usr/share/app1.raw"
test-50-dissect: move minimal image setup to common setup function So that it can be re-used for other tests by simply setting TEST_INSTALL_VERITY_MINIMAL=1 in test.sh, without having to replicate the setup commands.
2020-08-28 21:17:31 +08:00
)
}
test: remove duplicated code
2013-11-06 06:32:56 +08:00
setup_basic_environment() {
# create the basic filesystem layout
setup_basic_dirs
install_systemd
install_missing_libraries
install_config_files
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
install_zoneinfo
test: remove duplicated code
2013-11-06 06:32:56 +08:00
create_rc_local
install_basic_tools
install_libnss
install_pam
install_dbus
install_fonts
install_keymaps
install_terminfo
install_execs
tests: install mkfs.ext4, mkfs.vfat and modules into the test image This allows TEST-58-REPART to at least start. It fails later with with loopback device errors.
2021-04-21 15:58:26 +08:00
install_fs_tools
install_modules
test: remove duplicated code
2013-11-06 06:32:56 +08:00
install_plymouth
test: if haveged is part of initrd it needs to be installed in the image too Otherwise haveged won't survive when switching root from initrd to host making haveged service in host fail.
2021-08-06 22:47:32 +08:00
install_haveged
test: remove duplicated code
2013-11-06 06:32:56 +08:00
install_debug_tools
install_ld_so_conf
test: move TEST-43-* setup to static files Setup of lingering is dropped. I don't think it's necessary for anything, because the unit is still pulled in from the testsuite service.
2020-03-21 02:05:36 +08:00
install_testuser
has_user_dbus_socket && install_user_dbus
tests: add setup_selinux
2016-01-25 10:45:43 +08:00
setup_selinux
test: remove duplicated code
2013-11-06 06:32:56 +08:00
strip_binaries
tests: add integration test for RestrictNetworkInterfaces= Signed-off-by: Mauricio Vásquez <mauricio@kinvolk.io>
2021-02-26 08:59:36 +08:00
instmods veth
test: remove duplicated code
2013-11-06 06:32:56 +08:00
install_depmod_files
generate_module_dependencies
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$IS_BUILT_WITH_ASAN"; then
test: improve handling of ASan under clang Running integration tests with ASan is somewhat tricky to begin with, as we need to pre-load the ASan runtime DSO for certain services (like dbus), otherwise they won't start or behave as expected. In case of gcc this is pretty easy, as we need the runtime DSO during compilation, so it's already present on the host system. For clang things get more complicated, as ASan is compiled in statically by default, thus to enable the necessary dynamic-ish behavior one needs to compile with -shared-libasan and then correctly set LD_PRELOAD_PATH, as the runtime libraries are not in a standard library path.
2019-05-25 04:35:52 +08:00
create_asan_wrapper
tests: create the asan wrapper automatically if systemd has been built with ASAN
2018-07-05 12:09:30 +08:00
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$TEST_INSTALL_VERITY_MINIMAL"; then
test-50-dissect: move minimal image setup to common setup function So that it can be re-used for other tests by simply setting TEST_INSTALL_VERITY_MINIMAL=1 in test.sh, without having to replicate the setup commands.
2020-08-28 21:17:31 +08:00
install_verity_minimal
fi
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
tests: add setup_selinux
2016-01-25 10:45:43 +08:00
setup_selinux() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Setup SELinux"
tests: add setup_selinux
2016-01-25 10:45:43 +08:00
# don't forget KERNEL_APPEND='... selinux=1 ...'
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if ! get_bool "$SETUP_SELINUX"; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "SETUP_SELINUX != yes, skipping SELinux configuration"
tests: add setup_selinux
2016-01-25 10:45:43 +08:00
return 0
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local conf_dir=/etc/selinux
local fixfiles_tools=(bash uname cat sort uniq awk grep egrep head expr find rm secon setfiles)
# Make sure the following statement can't expand to "/" to prevent
# a potential where-are-my-backups situation
rm -rf "${initdir:?}/$conf_dir"
if ! cp -ar "$conf_dir" "$initdir/$conf_dir"; then
dfatal "Failed to copy $conf_dir"
tests: add setup_selinux
2016-01-25 10:45:43 +08:00
exit 1
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
touch "$initdir/.autorelabel"
mkdir -p "$initdir/usr/lib/systemd/tests/testdata/units/basic.target.wants"
ln -sf ../autorelabel.service "$initdir/usr/lib/systemd/tests/testdata/units/basic.target.wants/"
tests: add setup_selinux
2016-01-25 10:45:43 +08:00
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
image_install "${fixfiles_tools[@]}"
image_install fixfiles
image_install sestatus
tests: add setup_selinux
2016-01-25 10:45:43 +08:00
}
tests: add function for valgrind installation I used it for d9814c76ec35e53a6b6448c0 Very handy:)
2016-01-14 16:09:09 +08:00
install_valgrind() {
if ! type -p valgrind; then
dfatal "Failed to install valgrind"
exit 1
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local valgrind_bins valgrind_libs valgrind_dbg_and_supp
valgrind_bins="$(strace -e execve valgrind /bin/true 2>&1 >/dev/null | perl -lne 'print $1 if /^execve\("([^"]+)"/')"
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
image_install "$valgrind_bins"
tests: add function for valgrind installation I used it for d9814c76ec35e53a6b6448c0 Very handy:)
2016-01-14 16:09:09 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
valgrind_libs="$(LD_DEBUG=files valgrind /bin/true 2>&1 >/dev/null | perl -lne 'print $1 if m{calling init: (/.*vgpreload_.*)}')"
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
image_install "$valgrind_libs"
tests: add function for valgrind installation I used it for d9814c76ec35e53a6b6448c0 Very handy:)
2016-01-14 16:09:09 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
valgrind_dbg_and_supp="$(
tests: add function for valgrind installation I used it for d9814c76ec35e53a6b6448c0 Very handy:)
2016-01-14 16:09:09 +08:00
strace -e open valgrind /bin/true 2>&1 >/dev/null |
perl -lne 'if (my ($fname) = /^open\("([^"]+).*= (?!-)\d+/) { print $fname if $fname =~ /debug|\.supp$/ }'
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
)"
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
image_install "$valgrind_dbg_and_supp"
tests: add function for valgrind installation I used it for d9814c76ec35e53a6b6448c0 Very handy:)
2016-01-14 16:09:09 +08:00
}
tests: add function for valgrind wrapper creation I used it for d9814c7 and bffd87b
2016-01-15 10:52:12 +08:00
create_valgrind_wrapper() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local valgrind_wrapper="$initdir/$ROOTLIBDIR/systemd-under-valgrind"
ddebug "Create $valgrind_wrapper"
cat >"$valgrind_wrapper" <<EOF
treewide: more portable bash shebangs As in 2a5fcfae024ffc370bb780572279f45a1da3f946 and in 3e67e5c9928f8b1e1c5a63def88d53ed1fed12eb using /usr/bin/env allows bash to be looked up in PATH rather than being hard-coded. As with the previous changes the same arguments apply - distributions have scripts to rewrite shebangs on installation and they know what locations to rely on. - For tests/compilation we should rather rely on the user to have setup there PATH correctly. In particular this makes testing from git easier on NixOS where do not provide /bin/bash to improve compose-ability.
2020-03-04 17:35:06 +08:00
#!/usr/bin/env bash
tests: add function for valgrind wrapper creation I used it for d9814c7 and bffd87b
2016-01-15 10:52:12 +08:00
tests: mount proc before `exec valgrind` (#4044) Fixes: $ sudo make run QEMU_BIN=/usr/bin/qemu-system-x86_64 KERNEL_APPEND=systemd.unit=multi-user.target SKIP_INITRD=yes TEST RUN: Basic systemd setup + /usr/bin/qemu-system-x86_64 -smp 1 -net none -m 512M -nographic -kernel /boot/vmlinuz-4.6.7-200.fc23.x86_64 -drive format=raw,cache=unsafe,file=/var/tmp/systemd-test.izx99J/rootdisk.img -append 'root=/dev/sda1 raid=noautodetect loglevel=2 init=/usr/lib/systemd/systemd-under-valgrind rw console=ttyS0 selinux=0 systemd.unified_cgroup_hierarchy=no systemd.unit=multi-user.target ' valgrind: warning (non-fatal): readlink("/proc/self/exe") failed. valgrind: continuing, however --trace-children=yes will not work. --1:0: aspacem Valgrind: FATAL: can't open /proc/self/maps --1:0: aspacem Exiting now. [ 3.152367] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000100 [ 3.152367] [ 3.152367] CPU: 0 PID: 1 Comm: memcheck-amd64- Not tainted 4.6.7-200.fc23.x86_64 #1 [ 3.152367] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.8.2-20150714_191134- 04/01/2014 [ 3.152367] 0000000000000086 0000000047ca1666 ffff88001ea43e00 ffffffff813d954e [ 3.152367] ffffffff81a205a0 ffff88001ea43e98 ffff88001ea43e88 ffffffff811b5557 [ 3.152367] ffffffff00000010 ffff88001ea43e98 ffff88001ea43e30 0000000047ca1666 [ 3.152367] Call Trace: [ 3.152367] [<ffffffff813d954e>] dump_stack+0x63/0x85 [ 3.152367] [<ffffffff811b5557>] panic+0xde/0x220 [ 3.152367] [<ffffffff810ab9c3>] do_exit+0xb43/0xb50 [ 3.152367] [<ffffffff810aba57>] do_group_exit+0x47/0xb0 [ 3.152367] [<ffffffff810abad4>] SyS_exit_group+0x14/0x20 [ 3.152367] [<ffffffff817dae72>] entry_SYSCALL_64_fastpath+0x1a/0xa4 [ 3.152367] Kernel Offset: disabled [ 3.152367] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000100 [ 3.152367] QEMU: Terminated
2016-08-26 23:49:56 +08:00
mount -t proc proc /proc
tests: add function for valgrind wrapper creation I used it for d9814c7 and bffd87b
2016-01-15 10:52:12 +08:00
exec valgrind --leak-check=full --log-file=/valgrind.out $ROOTLIBDIR/systemd "\$@"
EOF
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
chmod 0755 "$valgrind_wrapper"
tests: add function for valgrind wrapper creation I used it for d9814c7 and bffd87b
2016-01-15 10:52:12 +08:00
}
tests: add a wrapper for when systemd is built with ASAN
2018-07-03 11:25:53 +08:00
create_asan_wrapper() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local asan_wrapper="$initdir/$ROOTLIBDIR/systemd-under-asan"
dinfo "Create ASan wrapper as '$asan_wrapper'"
test: improve handling of ASan under clang Running integration tests with ASan is somewhat tricky to begin with, as we need to pre-load the ASan runtime DSO for certain services (like dbus), otherwise they won't start or behave as expected. In case of gcc this is pretty easy, as we need the runtime DSO during compilation, so it's already present on the host system. For clang things get more complicated, as ASan is compiled in statically by default, thus to enable the necessary dynamic-ish behavior one needs to compile with -shared-libasan and then correctly set LD_PRELOAD_PATH, as the runtime libraries are not in a standard library path.
2019-05-25 04:35:52 +08:00
test: tidy up the ASan-related stuff
2021-03-07 05:21:30 +08:00
[[ -z "$ASAN_RT_PATH" ]] && dfatal "ASAN_RT_PATH is empty, but it shouldn't be"
# clang: install llvm-symbolizer to generate useful reports
# See: https://clang.llvm.org/docs/AddressSanitizer.html#symbolizing-the-reports
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
[[ "$ASAN_COMPILER" == "clang" ]] && image_install "llvm-symbolizer"
test: improve handling of ASan under clang Running integration tests with ASan is somewhat tricky to begin with, as we need to pre-load the ASan runtime DSO for certain services (like dbus), otherwise they won't start or behave as expected. In case of gcc this is pretty easy, as we need the runtime DSO during compilation, so it's already present on the host system. For clang things get more complicated, as ASan is compiled in statically by default, thus to enable the necessary dynamic-ish behavior one needs to compile with -shared-libasan and then correctly set LD_PRELOAD_PATH, as the runtime libraries are not in a standard library path.
2019-05-25 04:35:52 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
cat >"$asan_wrapper" <<EOF
treewide: more portable bash shebangs As in 2a5fcfae024ffc370bb780572279f45a1da3f946 and in 3e67e5c9928f8b1e1c5a63def88d53ed1fed12eb using /usr/bin/env allows bash to be looked up in PATH rather than being hard-coded. As with the previous changes the same arguments apply - distributions have scripts to rewrite shebangs on installation and they know what locations to rely on. - For tests/compilation we should rather rely on the user to have setup there PATH correctly. In particular this makes testing from git easier on NixOS where do not provide /bin/bash to improve compose-ability.
2020-03-04 17:35:06 +08:00
#!/usr/bin/env bash
tests: add a wrapper for when systemd is built with ASAN
2018-07-03 11:25:53 +08:00
set -x
test: tidy up the ASan-related stuff
2021-03-07 05:21:30 +08:00
echo "ASan RT: $ASAN_RT_PATH"
if [[ ! -e "$ASAN_RT_PATH" ]]; then
echo >&2 "Couldn't find ASan RT at '$ASAN_RT_PATH', can't continue"
exit 1
fi
test: suppress certain leaks reported by LSan so we can run TEST-46 under sanitizers once again. `systemd-homed` runs fsck on home directories, which reports a memory leak we're not interested in. Let's introduce an LSan suppression file to get around this. Since the patterns in the suppression file are matched using basic substring match[0], they're a bit cumbersome, but should get the work one. [0] https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer#suppressions Example leaks (as reported by TEST-46): ``` systemd-homed[1333]: ================================================================= systemd-homed[1333]: ==1333==ERROR: LeakSanitizer: detected memory leaks systemd-homed[1333]: Direct leak of 24 byte(s) in 1 object(s) allocated from: systemd-homed[1333]: #0 0x7f0c8facccd1 in calloc (/usr/lib/clang/12.0.1/lib/linux/libclang_rt.asan-x86_64.so+0xf4cd1) systemd-homed[1333]: #1 0x558d9494ff67 (/usr/bin/fsck+0x3f67) systemd-homed[1333]: Direct leak of 6 byte(s) in 1 object(s) allocated from: systemd-homed[1333]: #0 0x7f0c8fa906c1 in strdup (/usr/lib/clang/12.0.1/lib/linux/libclang_rt.asan-x86_64.so+0xb86c1) systemd-homed[1333]: #1 0x558d949518fd (/usr/bin/fsck+0x58fd) systemd-homed[1333]: SUMMARY: AddressSanitizer: 30 byte(s) leaked in 2 allocation(s). systemd-homed[1337]: ==1337==WARNING: Symbolizer was blocked from starting itself! systemd-homed[1337]: ================================================================= systemd-homed[1337]: ==1337==ERROR: LeakSanitizer: detected memory leaks systemd-homed[1337]: Direct leak of 67584 byte(s) in 1 object(s) allocated from: systemd-homed[1337]: #0 0x7f01edb84b19 (/usr/lib/clang/12.0.1/lib/linux/libclang_rt.asan-x86_64.so+0xf4b19) systemd-homed[1337]: #1 0x7f01e8326829 (/usr/bin/../lib/libLLVM-12.so+0xb46829) systemd-homed[1337]: SUMMARY: AddressSanitizer: 67584 byte(s) leaked in 1 allocation(s). ``` With the suppression file: ``` systemd-homed[1339]: ----------------------------------------------------- systemd-homed[1339]: Suppressions used: systemd-homed[1339]: count bytes template systemd-homed[1339]: 2 30 /bin/fsck$ systemd-homed[1339]: ----------------------------------------------------- systemd-homed[1343]: ==1343==WARNING: Symbolizer was blocked from starting itself! systemd-homed[1343]: ----------------------------------------------------- systemd-homed[1343]: Suppressions used: systemd-homed[1343]: count bytes template systemd-homed[1343]: 1 67584 /lib/libLLVM systemd-homed[1343]: ----------------------------------------------------- ```
2021-11-23 03:13:51 +08:00
# Suppress certain leaks reported by LSan (either in external tools or bogus
# ones)
# Docs: # https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer#suppressions
#
# - fsck is called by systemd-homed and is reporting a leak we're not interested
# in
# - libLLVM is a "side effect" caused by the previous fsck leak
cat >/systemd-lsan.supp <<INNER_EOF
leak:/bin/fsck$
leak:/sbin/fsck$
leak:/lib/libLLVM
INNER_EOF
DEFAULT_LSAN_OPTIONS=${LSAN_OPTIONS:-}:suppressions=/systemd-lsan.supp
test: make ASAN/UBSAN_OPTIONS overridable from the outside This should allow us to tweak the ASAN_OPTIONS and UBSAN_OPTIONS env variables for integration tests as well
2019-07-08 19:47:46 +08:00
DEFAULT_ASAN_OPTIONS=${ASAN_OPTIONS:-strict_string_checks=1:detect_stack_use_after_return=1:check_initialization_order=1:strict_init_order=1}
DEFAULT_UBSAN_OPTIONS=${UBSAN_OPTIONS:-print_stacktrace=1:print_summary=1:halt_on_error=1}
test: suppress certain leaks reported by LSan so we can run TEST-46 under sanitizers once again. `systemd-homed` runs fsck on home directories, which reports a memory leak we're not interested in. Let's introduce an LSan suppression file to get around this. Since the patterns in the suppression file are matched using basic substring match[0], they're a bit cumbersome, but should get the work one. [0] https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer#suppressions Example leaks (as reported by TEST-46): ``` systemd-homed[1333]: ================================================================= systemd-homed[1333]: ==1333==ERROR: LeakSanitizer: detected memory leaks systemd-homed[1333]: Direct leak of 24 byte(s) in 1 object(s) allocated from: systemd-homed[1333]: #0 0x7f0c8facccd1 in calloc (/usr/lib/clang/12.0.1/lib/linux/libclang_rt.asan-x86_64.so+0xf4cd1) systemd-homed[1333]: #1 0x558d9494ff67 (/usr/bin/fsck+0x3f67) systemd-homed[1333]: Direct leak of 6 byte(s) in 1 object(s) allocated from: systemd-homed[1333]: #0 0x7f0c8fa906c1 in strdup (/usr/lib/clang/12.0.1/lib/linux/libclang_rt.asan-x86_64.so+0xb86c1) systemd-homed[1333]: #1 0x558d949518fd (/usr/bin/fsck+0x58fd) systemd-homed[1333]: SUMMARY: AddressSanitizer: 30 byte(s) leaked in 2 allocation(s). systemd-homed[1337]: ==1337==WARNING: Symbolizer was blocked from starting itself! systemd-homed[1337]: ================================================================= systemd-homed[1337]: ==1337==ERROR: LeakSanitizer: detected memory leaks systemd-homed[1337]: Direct leak of 67584 byte(s) in 1 object(s) allocated from: systemd-homed[1337]: #0 0x7f01edb84b19 (/usr/lib/clang/12.0.1/lib/linux/libclang_rt.asan-x86_64.so+0xf4b19) systemd-homed[1337]: #1 0x7f01e8326829 (/usr/bin/../lib/libLLVM-12.so+0xb46829) systemd-homed[1337]: SUMMARY: AddressSanitizer: 67584 byte(s) leaked in 1 allocation(s). ``` With the suppression file: ``` systemd-homed[1339]: ----------------------------------------------------- systemd-homed[1339]: Suppressions used: systemd-homed[1339]: count bytes template systemd-homed[1339]: 2 30 /bin/fsck$ systemd-homed[1339]: ----------------------------------------------------- systemd-homed[1343]: ==1343==WARNING: Symbolizer was blocked from starting itself! systemd-homed[1343]: ----------------------------------------------------- systemd-homed[1343]: Suppressions used: systemd-homed[1343]: count bytes template systemd-homed[1343]: 1 67584 /lib/libLLVM systemd-homed[1343]: ----------------------------------------------------- ```
2021-11-23 03:13:51 +08:00
DEFAULT_ENVIRONMENT="ASAN_OPTIONS=\$DEFAULT_ASAN_OPTIONS UBSAN_OPTIONS=\$DEFAULT_UBSAN_OPTIONS LSAN_OPTIONS=\$DEFAULT_LSAN_OPTIONS"
tests: add a wrapper for when systemd is built with ASAN
2018-07-03 11:25:53 +08:00
test: improve handling of ASan under clang Running integration tests with ASan is somewhat tricky to begin with, as we need to pre-load the ASan runtime DSO for certain services (like dbus), otherwise they won't start or behave as expected. In case of gcc this is pretty easy, as we need the runtime DSO during compilation, so it's already present on the host system. For clang things get more complicated, as ASan is compiled in statically by default, thus to enable the necessary dynamic-ish behavior one needs to compile with -shared-libasan and then correctly set LD_PRELOAD_PATH, as the runtime libraries are not in a standard library path.
2019-05-25 04:35:52 +08:00
# As right now bash is the PID 1, we can't expect PATH to have a sane value.
# Let's make one to prevent unexpected "<bin> not found" issues in the future
export PATH="/sbin:/bin:/usr/sbin:/usr/bin"
tests: add a wrapper for when systemd is built with ASAN
2018-07-03 11:25:53 +08:00
mount -t proc proc /proc
mount -t sysfs sysfs /sys
mount -o remount,rw /
test: tidy up the ASan-related stuff
2021-03-07 05:21:30 +08:00
# A lot of services (most notably dbus) won't start without preloading libasan
# See https://github.com/systemd/systemd/issues/5004
DEFAULT_ENVIRONMENT="\$DEFAULT_ENVIRONMENT LD_PRELOAD=$ASAN_RT_PATH"
if [[ "$ASAN_COMPILER" == "clang" ]]; then
test: improve handling of ASan under clang Running integration tests with ASan is somewhat tricky to begin with, as we need to pre-load the ASan runtime DSO for certain services (like dbus), otherwise they won't start or behave as expected. In case of gcc this is pretty easy, as we need the runtime DSO during compilation, so it's already present on the host system. For clang things get more complicated, as ASan is compiled in statically by default, thus to enable the necessary dynamic-ish behavior one needs to compile with -shared-libasan and then correctly set LD_PRELOAD_PATH, as the runtime libraries are not in a standard library path.
2019-05-25 04:35:52 +08:00
# Let's add the ASan DSO's path to the dynamic linker's cache. This is pretty
# unnecessary for gcc & libasan, however, for clang this is crucial, as its
# runtime ASan DSO is in a non-standard (library) path.
TEST-*: use spacing before redirection operator, but not after << EOF → <<EOF > foo < bar → >foo <bar
2021-04-08 06:09:55 +08:00
echo "${ASAN_RT_PATH%/*}" >/etc/ld.so.conf.d/asan-path-override.conf
test: improve handling of ASan under clang Running integration tests with ASan is somewhat tricky to begin with, as we need to pre-load the ASan runtime DSO for certain services (like dbus), otherwise they won't start or behave as expected. In case of gcc this is pretty easy, as we need the runtime DSO during compilation, so it's already present on the host system. For clang things get more complicated, as ASan is compiled in statically by default, thus to enable the necessary dynamic-ish behavior one needs to compile with -shared-libasan and then correctly set LD_PRELOAD_PATH, as the runtime libraries are not in a standard library path.
2019-05-25 04:35:52 +08:00
ldconfig
tests: add a wrapper for when systemd is built with ASAN
2018-07-03 11:25:53 +08:00
fi
echo DefaultEnvironment=\$DEFAULT_ENVIRONMENT >>/etc/systemd/system.conf
tests: double DefaultTimeoutStartSec when systemd is run under ASan+UBSan
2018-12-25 13:13:45 +08:00
echo DefaultTimeoutStartSec=180s >>/etc/systemd/system.conf
test: improve handling of ASan under clang Running integration tests with ASan is somewhat tricky to begin with, as we need to pre-load the ASan runtime DSO for certain services (like dbus), otherwise they won't start or behave as expected. In case of gcc this is pretty easy, as we need the runtime DSO during compilation, so it's already present on the host system. For clang things get more complicated, as ASan is compiled in statically by default, thus to enable the necessary dynamic-ish behavior one needs to compile with -shared-libasan and then correctly set LD_PRELOAD_PATH, as the runtime libraries are not in a standard library path.
2019-05-25 04:35:52 +08:00
echo DefaultStandardOutput=journal+console >>/etc/systemd/system.conf
tests: add a wrapper for when systemd is built with ASAN
2018-07-03 11:25:53 +08:00
# ASAN and syscall filters aren't compatible with each other.
find / -name '*.service' -type f | xargs sed -i 's/^\\(MemoryDeny\\|SystemCall\\)/#\\1/'
tests: redirect ASAN reports on journald to a file Otherwise, they will end up in /dev/null.
2018-07-04 03:29:42 +08:00
# The redirection of ASAN reports to a file prevents them from ending up in /dev/null.
# But, apparently, sometimes it doesn't work: https://github.com/google/sanitizers/issues/886.
JOURNALD_CONF_DIR=/etc/systemd/system/systemd-journald.service.d
mkdir -p "\$JOURNALD_CONF_DIR"
tests: redirect UBsan reports to a file so that they won't end up in /dev/null
2019-05-10 07:00:29 +08:00
printf "[Service]\nEnvironment=ASAN_OPTIONS=\$DEFAULT_ASAN_OPTIONS:log_path=/systemd-journald.asan.log UBSAN_OPTIONS=\$DEFAULT_UBSAN_OPTIONS:log_path=/systemd-journald.ubsan.log\n" >"\$JOURNALD_CONF_DIR/env.conf"
tests: redirect ASAN reports on journald to a file Otherwise, they will end up in /dev/null.
2018-07-04 03:29:42 +08:00
tests: redirect the stdout/stderr of journald to a file (under ASan+UBSan) Sometimes UBSan sends its reports to stderr regardless of what is specified in log_path Let's try to catch them by redirecting stderr (and stdout just in case) to a file See https://github.com/systemd/systemd/pull/12524#issuecomment-491108821
2019-05-10 08:47:03 +08:00
# Sometimes UBSan sends its reports to stderr regardless of what is specified in log_path
# Let's try to catch them by redirecting stderr (and stdout just in case) to a file
# See https://github.com/systemd/systemd/pull/12524#issuecomment-491108821
printf "[Service]\nStandardOutput=file:/systemd-journald.out\n" >"\$JOURNALD_CONF_DIR/out.conf"
tests: overwrite a hard-coded timeout in systemd-hwdb-update.service
2018-12-24 08:15:43 +08:00
# 90s isn't enough for some services to finish when literally everything is run
# under ASan+UBSan in containers, which, in turn, are run in VMs.
test: limit environments for systemd-hwdb-update under ASan
2019-01-12 17:48:13 +08:00
# Let's limit which environments such services should be executed in.
tests: overwrite a hard-coded timeout in systemd-hwdb-update.service
2018-12-24 08:15:43 +08:00
mkdir -p /etc/systemd/system/systemd-hwdb-update.service.d
test: bump the timeout for systemd-hwdb-update.service under ASan Since the hwdb update from a79be2f80777eb80e0d8177f6bccd7615de7ec1a the systemd-hwdb-update service started timing out under ASan when compiled with gcc, as we started tripping over the 3 minutes timeout. This affects only gcc runs, since the current gcc on Arch still suffers from the detect_stack_use_after_return performance penalty[0]. Until the fixed gcc is present in the respective repositories, let's bump the timeout to 4 minutes, as we might not be able to upgrade right away, due to systemd/systemd#16199. Before the hwdb update: [ 7958.292540] systemd[63]: systemd-hwdb-update.service: Executing: /usr/bin/time systemd-hwdb update [ 7958.304005] systemd[1]: systemd-journald.service: Got notification message from PID 44 (FDSTORE=1) [ 7958.314434] systemd[1]: systemd-journald.service: Added fd 3 (n/a) to fd store. [ 8008.520082] systemd[1]: systemd-journald.service: Got notification message from PID 44 (WATCHDOG=1) [ 8068.520151] systemd[1]: systemd-journald.service: Got notification message from PID 44 (WATCHDOG=1) [ 8125.682843] time[63]: 84.47user 82.92system 2:47.50elapsed 99%CPU (0avgtext+0avgdata 811512maxresident)k [ 8125.682843] time[63]: 0inputs+19680outputs (0major+25000853minor)pagefaults 0swaps After the hwdb update: [ 6215.491958] systemd[63]: systemd-hwdb-update.service: Executing: /usr/bin/time systemd-hwdb update [ 6215.503380] systemd[1]: systemd-journald.service: Got notification message from PID 44 (FDSTORE=1) [ 6215.514172] systemd[1]: systemd-journald.service: Added fd 3 (n/a) to fd store. [ 6329.392918] systemd[1]: systemd-journald.service: Got notification message from PID 44 (WATCHDOG=1) [ 6394.920205] time[63]: 89.48user 89.98system 2:59.55elapsed 99%CPU (0avgtext+0avgdata 812764maxresident)k [ 6394.920205] time[63]: 0inputs+20568outputs (0major+27318354minor)pagefaults 0swaps [0] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94910
2020-06-29 00:53:28 +08:00
printf "[Unit]\nConditionVirtualization=container\n\n[Service]\nTimeoutSec=240s\n" >/etc/systemd/system/systemd-hwdb-update.service.d/env-override.conf
tests: overwrite a hard-coded timeout in systemd-hwdb-update.service
2018-12-24 08:15:43 +08:00
tests: override a hard-coded timeout that kicks in too early (under ASan+UBSan) The test has failed two times today: https://travis-ci.org/systemd/systemd/jobs/530750626 and https://travis-ci.org/systemd/systemd/jobs/530759465.
2019-05-11 00:33:35 +08:00
# Let's override another hard-coded timeout that kicks in too early
mkdir -p /etc/systemd/system/systemd-journal-flush.service.d
printf "[Service]\nTimeoutSec=180s\n" >/etc/systemd/system/systemd-journal-flush.service.d/timeout.conf
test: disable at_exit LSan check for dbus.service When running integration tests under sanitizers D-Bus fails to shutdown cleanly, causing unnecessary noise in the logs: ``` dbus-daemon[272]: ==272==LeakSanitizer has encountered a fatal error. dbus-daemon[272]: ==272==HINT: For debugging, try setting environment variable LSAN_OPTIONS=verbosity=1:log_threads=1 dbus-daemon[272]: ==272==HINT: LeakSanitizer does not work under ptrace (strace, gdb, etc) ``` Since we're not "sanitizing" D-Bus anyway let's disable LSan's at_exit check for the dbus.service to get rid of this error.
2021-03-06 06:20:45 +08:00
# D-Bus has troubles during system shutdown causing it to fail. Although it's
# harmless, it causes unnecessary noise in the logs, so let's disable LSan's
# at_exit check just for the dbus.service
mkdir -p /etc/systemd/system/dbus.service.d
printf "[Service]\nEnvironment=ASAN_OPTIONS=leak_check_at_exit=false\n" >/etc/systemd/system/dbus.service.d/disable-lsan.conf
test: fix TEST-50-DISSECT under sanitizers This test would normally get stuck when trying to mount the verity image due to: systemd-udevd[299]: dm-0: '/usr/sbin/dmsetup udevflags 6293812'(err) '==371==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD.' systemd-udevd[299]: dm-0: Process '/usr/sbin/dmsetup udevflags 6293812' failed with exit code 1 ... systemd-udevd[299]: dm-0: '/usr/sbin/dmsetup udevcomplete 6293812'(err) '==372==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD.' systemd-udevd[299]: dm-0: Process '/usr/sbin/dmsetup udevcomplete 6293812' failed with exit code 1. systemd-udevd[299]: dm-0: Command "/usr/sbin/dmsetup udevcomplete 6293812" returned 1 (error), ignoring. so let's add a simple udev rule which sets $LD_PRELOAD for the block subsystem. Also, install the ASan library along with necessary dependencies into the verity minimal image, to get rid of the annoying (yet harmless) errors about missing library from $LD_LIBRARY.
2021-03-07 05:22:32 +08:00
# Some utilities run via IMPORT/RUN/PROGRAM udev directives fail because
# they're uninstrumented (like dmsetup). Let's add a simple rule which sets
# LD_PRELOAD to the ASan RT library to fix this.
mkdir -p /etc/udev/rules.d
TEST-*: use spacing before redirection operator, but not after << EOF → <<EOF > foo < bar → >foo <bar
2021-04-08 06:09:55 +08:00
cat >/etc/udev/rules.d/00-set-LD_PRELOAD.rules <<INNER_EOF
test: fix TEST-50-DISSECT under sanitizers This test would normally get stuck when trying to mount the verity image due to: systemd-udevd[299]: dm-0: '/usr/sbin/dmsetup udevflags 6293812'(err) '==371==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD.' systemd-udevd[299]: dm-0: Process '/usr/sbin/dmsetup udevflags 6293812' failed with exit code 1 ... systemd-udevd[299]: dm-0: '/usr/sbin/dmsetup udevcomplete 6293812'(err) '==372==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD.' systemd-udevd[299]: dm-0: Process '/usr/sbin/dmsetup udevcomplete 6293812' failed with exit code 1. systemd-udevd[299]: dm-0: Command "/usr/sbin/dmsetup udevcomplete 6293812" returned 1 (error), ignoring. so let's add a simple udev rule which sets $LD_PRELOAD for the block subsystem. Also, install the ASan library along with necessary dependencies into the verity minimal image, to get rid of the annoying (yet harmless) errors about missing library from $LD_LIBRARY.
2021-03-07 05:22:32 +08:00
SUBSYSTEM=="block", ENV{LD_PRELOAD}="$ASAN_RT_PATH"
INNER_EOF
test: fix permissions of the ASan udev workaround otherwise udev complains about the file being world-writable: systemd-udevd[228]: Configuration file /etc/udev/rules.d/00-set-LD_PRELOAD.rules is marked world-writable. Please remove world writability permission bits. Proceeding anyway. Fixes: systemd/systemd-centos-ci#354
2021-03-10 19:30:04 +08:00
chmod 0644 /etc/udev/rules.d/00-set-LD_PRELOAD.rules
test: fix TEST-50-DISSECT under sanitizers This test would normally get stuck when trying to mount the verity image due to: systemd-udevd[299]: dm-0: '/usr/sbin/dmsetup udevflags 6293812'(err) '==371==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD.' systemd-udevd[299]: dm-0: Process '/usr/sbin/dmsetup udevflags 6293812' failed with exit code 1 ... systemd-udevd[299]: dm-0: '/usr/sbin/dmsetup udevcomplete 6293812'(err) '==372==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD.' systemd-udevd[299]: dm-0: Process '/usr/sbin/dmsetup udevcomplete 6293812' failed with exit code 1. systemd-udevd[299]: dm-0: Command "/usr/sbin/dmsetup udevcomplete 6293812" returned 1 (error), ignoring. so let's add a simple udev rule which sets $LD_PRELOAD for the block subsystem. Also, install the ASan library along with necessary dependencies into the verity minimal image, to get rid of the annoying (yet harmless) errors about missing library from $LD_LIBRARY.
2021-03-07 05:22:32 +08:00
test: don't preload libasan for systemd-remount-fs.service The `mount` utility has an unexpected behavior when run with libasan, causing false-positives during the integration testing. For example, on Arch Linux with LD_PRELOAD pointing to libasan: ``` bash-5.0# mount -o remount,rw -v / mount: /dev/sda1 mounted on /. bash-5.0# echo $? 1 ``` However: ``` bash-5.0# LD_PRELOAD= mount -o remount,rw -v / mount: /dev/sda1 mounted on /. bash-5.0# echo $? 0 ``` Further investigation with strace shows a LeakSanitizer error: ``` bash-5.0# strace -s 512 mount -o remount,rw -v / ... write(2, "==355==LeakSanitizer has encountered a fatal error.\n", 52) = -1 EBADF (Bad file descriptor) write(2, "ReportFile::Write() can't output requested buffer!\n", 51) = -1 EBADF (Bad file descriptor) exit_group(1) = ? +++ exited with 1 +++ ``` Let's workaround this by clearing the LD_PRELOAD variable for systemd-remount-fs.service
2019-05-10 01:20:34 +08:00
# The 'mount' utility doesn't behave well under libasan, causing unexpected
# fails during boot and subsequent test results check:
# bash-5.0# mount -o remount,rw -v /
# mount: /dev/sda1 mounted on /.
# bash-5.0# echo \$?
# 1
# Let's workaround this by clearing the previously set LD_PRELOAD env variable,
# so the libasan library is not loaded for this particular service
tests: unset LD_PRELOAD in testsuite.service when it's run under ASan This should help to address the "grep" issue we ran into in https://github.com/systemd/systemd-centos-ci/pull/217.
2020-01-31 14:13:14 +08:00
unset_ld_preload() {
local _dropin_dir="/etc/systemd/system/\$1.service.d"
mkdir -p "\$_dropin_dir"
printf "[Service]\nUnsetEnvironment=LD_PRELOAD\n" >"\$_dropin_dir/unset_ld_preload.conf"
}
unset_ld_preload systemd-remount-fs
test: move TEST-01-BASIC setup to static files After this commit, only tests that were converted will work, because the interface to test_run() changed.
2019-12-10 01:56:13 +08:00
unset_ld_preload testsuite-
test: don't preload libasan for systemd-remount-fs.service The `mount` utility has an unexpected behavior when run with libasan, causing false-positives during the integration testing. For example, on Arch Linux with LD_PRELOAD pointing to libasan: ``` bash-5.0# mount -o remount,rw -v / mount: /dev/sda1 mounted on /. bash-5.0# echo $? 1 ``` However: ``` bash-5.0# LD_PRELOAD= mount -o remount,rw -v / mount: /dev/sda1 mounted on /. bash-5.0# echo $? 0 ``` Further investigation with strace shows a LeakSanitizer error: ``` bash-5.0# strace -s 512 mount -o remount,rw -v / ... write(2, "==355==LeakSanitizer has encountered a fatal error.\n", 52) = -1 EBADF (Bad file descriptor) write(2, "ReportFile::Write() can't output requested buffer!\n", 51) = -1 EBADF (Bad file descriptor) exit_group(1) = ? +++ exited with 1 +++ ``` Let's workaround this by clearing the LD_PRELOAD variable for systemd-remount-fs.service
2019-05-10 01:20:34 +08:00
tests: make UBSan print full backtraces This should make it easier to get backtraces that can be pasted into issues like https://github.com/systemd/systemd/issues/10096.
2018-09-15 22:21:33 +08:00
export ASAN_OPTIONS=\$DEFAULT_ASAN_OPTIONS:log_path=/systemd.asan.log UBSAN_OPTIONS=\$DEFAULT_UBSAN_OPTIONS
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
exec "$ROOTLIBDIR/systemd" "\$@"
tests: add a wrapper for when systemd is built with ASAN
2018-07-03 11:25:53 +08:00
EOF
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
chmod 0755 "$asan_wrapper"
tests: add a wrapper for when systemd is built with ASAN
2018-07-03 11:25:53 +08:00
}
tests: add create_strace_wrapper Very handy for early-boot debugging See https://github.com/systemd/systemd/pull/2781#discussion_r54782628
2016-03-04 09:55:46 +08:00
create_strace_wrapper() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local strace_wrapper="$initdir/$ROOTLIBDIR/systemd-under-strace"
ddebug "Create $strace_wrapper"
cat >"$strace_wrapper" <<EOF
treewide: more portable bash shebangs As in 2a5fcfae024ffc370bb780572279f45a1da3f946 and in 3e67e5c9928f8b1e1c5a63def88d53ed1fed12eb using /usr/bin/env allows bash to be looked up in PATH rather than being hard-coded. As with the previous changes the same arguments apply - distributions have scripts to rewrite shebangs on installation and they know what locations to rely on. - For tests/compilation we should rather rely on the user to have setup there PATH correctly. In particular this makes testing from git easier on NixOS where do not provide /bin/bash to improve compose-ability.
2020-03-04 17:35:06 +08:00
#!/usr/bin/env bash
tests: add create_strace_wrapper Very handy for early-boot debugging See https://github.com/systemd/systemd/pull/2781#discussion_r54782628
2016-03-04 09:55:46 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
exec strace -f -D -o /strace.out "$ROOTLIBDIR/systemd" "\$@"
tests: add create_strace_wrapper Very handy for early-boot debugging See https://github.com/systemd/systemd/pull/2781#discussion_r54782628
2016-03-04 09:55:46 +08:00
EOF
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
chmod 0755 "$strace_wrapper"
tests: add create_strace_wrapper Very handy for early-boot debugging See https://github.com/systemd/systemd/pull/2781#discussion_r54782628
2016-03-04 09:55:46 +08:00
}
tests: install mkfs.ext4, mkfs.vfat and modules into the test image This allows TEST-58-REPART to at least start. It fails later with with loopback device errors.
2021-04-21 15:58:26 +08:00
install_fs_tools() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install fsck"
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
image_install /sbin/fsck*
image_install -o /bin/fsck*
basic: fallback to fstatat if entry->d_type is DT_UNKNOWN * tests-functions: improve FSTYPE-support make clean setup FSTYPE=reiserfs is working fine now :) * basic: fallback to fstatat if entry->d_type is DT_UNKNOWN Fixes localectl on reiserfs: -bash-4.3# mkdir -p /usr/lib/locale -bash-4.3# stat -f /usr/lib/locale/ File: "/usr/lib/locale/" ID: bdb0322715b5366e Namelen: 255 Type: reiserfs Block size: 4096 Blocks: Total: 99835 Free: 60262 Available: 60262 Inodes: Total: 0 Free: 0 -bash-4.3# mkdir /usr/lib/locale/HeyHo -bash-4.3# localectl list-locales --no-pager -bash-4.3# mount -t tmpfs tmpfs /usr/lib/locale -bash-4.3# mkdir /usr/lib/locale/HeyHo -bash-4.3# localectl list-locales --no-pager HeyHo
2016-04-04 19:07:56 +08:00
# fskc.reiserfs calls reiserfsck. so, install it
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
image_install -o reiserfsck
tests: install mkfs.ext4, mkfs.vfat and modules into the test image This allows TEST-58-REPART to at least start. It fails later with with loopback device errors.
2021-04-21 15:58:26 +08:00
# we use mkfs in system-repart tests
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
image_install /sbin/mkfs.ext4
image_install /sbin/mkfs.vfat
tests: install mkfs.ext4, mkfs.vfat and modules into the test image This allows TEST-58-REPART to at least start. It fails later with with loopback device errors.
2021-04-21 15:58:26 +08:00
}
install_modules() {
dinfo "Install modules"
instmods loop
instmods vfat
test: install nls modules, required by vfat kernel: FAT-fs (loop0p2): codepage cp437 not found kernel: FAT-fs (loop0p2): IO charset ascii not found
2021-06-18 23:10:30 +08:00
instmods nls_ascii =nls
test/TEST-17-UDEV: use default image No need to build a separate 700MB image to save 3080 bytes in the default image.
2021-06-11 14:42:04 +08:00
instmods dummy
tests: install mkfs.ext4, mkfs.vfat and modules into the test image This allows TEST-58-REPART to at least start. It fails later with with loopback device errors.
2021-04-21 15:58:26 +08:00
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$LOOKS_LIKE_SUSE"; then
tests: install mkfs.ext4, mkfs.vfat and modules into the test image This allows TEST-58-REPART to at least start. It fails later with with loopback device errors.
2021-04-21 15:58:26 +08:00
instmods ext4
fi
tests: install fsck* systemd-fsck depends on /sbin/fsck*
2016-01-16 16:06:59 +08:00
}
test: remove duplicated code
2013-11-06 06:32:56 +08:00
install_dmevent() {
instmods dm_crypt =crypto
test/test-functions: fix install_dmeventd to correctly install bin/libs
2019-07-14 11:03:26 +08:00
inst_binary dmeventd
test: install dmevent service and socket
2021-09-14 00:39:00 +08:00
image_install "${ROOTLIBDIR:?}"/system/dm-event.{service,socket}
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$LOOKS_LIKE_DEBIAN"; then
tests: fix TEST-02-CRYPTSETUP on Debian/Ubuntu
2016-01-17 11:32:37 +08:00
# dmsetup installs 55-dm and 60-persistent-storage-dm on Debian/Ubuntu
test/test-functions: Debian/Ubuntu, now ship 95-dm-notify.rules, copy it This fixes cryptsetup tests on recent Ubuntu/Debian systems (current development series). Bug-Launchpad: https://launchpad.net/bugs/1749432
2018-02-15 04:34:55 +08:00
# and since buster/bionic 95-dm-notify.rules
# see https://gitlab.com/debian-lvm/lvm2/blob/master/debian/patches/udev.patch
inst_rules 55-dm.rules 60-persistent-storage-dm.rules 95-dm-notify.rules
tests: fix TEST-02-CRYPTSETUP on Debian/Ubuntu
2016-01-17 11:32:37 +08:00
else
inst_rules 10-dm.rules 13-dm-disk.rules 95-dm-notify.rules
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$LOOKS_LIKE_SUSE"; then
test: adapt test-functions for SUSE
2020-07-22 04:14:53 +08:00
inst_rules 60-persistent-storage.rules 61-persistent-storage-compat.rules 99-systemd.rules
fi
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
test: add a multipath helper
2021-09-09 00:17:46 +08:00
install_multipath() {
instmods "=md" multipath
image_install kpartx /lib/udev/kpartx_id lsmod mpathpersist multipath multipathd partx
image_install "${ROOTLIBDIR:?}"/system/multipathd.{service,socket}
if get_bool "$LOOKS_LIKE_DEBIAN"; then
inst_rules 56-dm-parts.rules 56-dm-mpath.rules 60-multipath.rules 68-del-part-nodes.rules 95-kpartx.rules
else
inst_rules 11-dm-mpath.rules 11-dm-parts.rules 62-multipath.rules 66-kpartx.rules 68-del-part-nodes.rules
fi
mkdir -p "${initdir:?}/etc/multipath"
local file
while read -r file; do
# Install libraries required by the given library
inst_libs "$file"
# Install the library itself and create necessary symlinks
inst_library "$file"
done < <(find /lib*/multipath -type f)
if get_bool "$LOOKS_LIKE_ARCH"; then
# On Arch the multipath libraries are not linked against libgcc_s.so.1,
# but it's still required at runtime
inst_library "/lib64/libgcc_s.so.1"
fi
}
test: add an LVM helper
2021-09-14 00:39:22 +08:00
install_lvm() {
image_install lvm
image_install "${ROOTLIBDIR:?}"/system/lvm2-lvmpolld.{service,socket}
test: support the new lvm udev autoactivation Introduced in lvm 2.03.14. See: https://sourceware.org/git/?p=lvm2.git;a=commit;h=67722b312390cdab29c076c912e14bd739c5c0f6
2021-11-02 03:55:38 +08:00
image_install "${ROOTLIBDIR:?}"/system/{blk-availability,lvm2-monitor}.service
test: add an LVM helper
2021-09-14 00:39:22 +08:00
image_install "${ROOTLIBDIR:?}"/system-generators/lvm2-activation-generator
image_install -o "/lib/tmpfiles.d/lvm2.conf"
if get_bool "$LOOKS_LIKE_DEBIAN"; then
inst_rules 56-lvm.rules 69-lvm-metad.rules
else
test: support the new lvm udev autoactivation Introduced in lvm 2.03.14. See: https://sourceware.org/git/?p=lvm2.git;a=commit;h=67722b312390cdab29c076c912e14bd739c5c0f6
2021-11-02 03:55:38 +08:00
# Support the new udev autoactivation introduced in lvm 2.03.14
# https://sourceware.org/git/?p=lvm2.git;a=commit;h=67722b312390cdab29c076c912e14bd739c5c0f6
if [[ -f /lib/udev/rules.d/69-dm-lvm.rules ]]; then
inst_rules 11-dm-lvm.rules 69-dm-lvm.rules
else
image_install "${ROOTLIBDIR:?}"/system/lvm2-pvscan@.service
inst_rules 11-dm-lvm.rules 69-dm-lvm-metad.rules
fi
test: add an LVM helper
2021-09-14 00:39:22 +08:00
fi
mkdir -p "${initdir:?}/etc/lvm"
}
test: add a btrfs helper
2021-09-18 01:19:23 +08:00
install_btrfs() {
instmods btrfs
# Not all utilities provided by btrfs-progs are listed here; extend the list
# if necessary
image_install btrfs btrfstune mkfs.btrfs
inst_rules 64-btrfs-dm.rules
}
test: add an iSCSI helper
2021-09-22 03:32:30 +08:00
install_iscsi() {
# Install both client and server side stuff by default
local inst="${1:-}"
local file
# Install client-side stuff ("initiator" in iSCSI jargon) - Open-iSCSI in this case
# (open-iscsi on Debian, iscsi-initiator-utils on Fedora, etc.)
if [[ -z "$inst" || "$inst" =~ (client|initiator) ]]; then
image_install iscsi-iname iscsiadm iscsid iscsistart
image_install -o "${ROOTLIBDIR:?}"/system/iscsi-{init,onboot,shutdown}.service
image_install "${ROOTLIBDIR:?}"/system/iscsid.{service,socket}
image_install "${ROOTLIBDIR:?}"/system/iscsi.service
mkdir -p "${initdir:?}"/var/lib/iscsi/{ifaces,isns,nodes,send_targets,slp,static}
mkdir -p "${initdir:?}/etc/iscsi"
echo "iscsid.startup = /bin/systemctl start iscsid.socket" >"${initdir:?}/etc/iscsi/iscsid.conf"
inst_simple "/etc/iscsi/initiatorname.iscsi"
fi
# Install server-side stuff ("target" in iSCSI jargon) - TGT in this case
# (tgt on Debian, scsi-target-utils on Fedora, etc.)
if [[ -z "$inst" || "$inst" =~ (server|target) ]]; then
image_install tgt-admin tgt-setup-lun tgtadm tgtd tgtimg
image_install -o /etc/sysconfig/tgtd
image_install "${ROOTLIBDIR:?}"/system/tgtd.service
mkdir -p "${initdir:?}/etc/tgt"
touch "${initdir:?}"/etc/tgt/{tgtd,targets}.conf
# Install perl modules required by tgt-admin
#
# Forgive me father for I have sinned. The monstrosity below appends
# a perl snippet to the `tgt-admin` perl script on the fly, which
# dumps a list of files (perl modules) required by `tgt-admin` at
# the runtime plus any DSOs loaded via DynaLoader. This list is then
# passed to `inst_simple` which installs the necessary files into the image
test: shellcheck-ify test scripts
2021-09-30 02:30:08 +08:00
#
# shellcheck disable=SC2016
test: add an iSCSI helper
2021-09-22 03:32:30 +08:00
while read -r file; do
inst_simple "$file"
test: shellcheck-ify test scripts
2021-09-30 02:30:08 +08:00
done < <(perl -- <(cat "$(command -v tgt-admin)" <(echo -e 'use DynaLoader; print map { "$_\n" } values %INC; print join("\n", @DynaLoader::dl_shared_objects)')) -p | awk '/^\// { print $1 }')
test: add an iSCSI helper
2021-09-22 03:32:30 +08:00
fi
}
test/test-functions: allow installing systemd files from local system Add NO_BUILD var to allow testing with no local build, by installing local systemd files into the image. This only works for debian-like distros currently, that use the tools 'apt' and 'dpkg' for package management.
2020-11-18 06:20:22 +08:00
install_compiled_systemd() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install compiled systemd"
test/test-functions: do not execute strip_binaries twice It is called from setup_basic_environment(), which also calls install_symtemd() a bit earlier, so in effect it would be called twice.
2020-08-27 19:10:30 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local ninja_bin
ninja_bin="$(type -P ninja || type -P ninja-build)"
if [[ -z "$ninja_bin" ]]; then
tests: use ninja-build if ninja is not available (#6544) This makes the tests work on CentOS, which currently has ninja-build only.
2017-08-07 17:06:07 +08:00
dfatal "ninja was not found"
exit 1
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
(set -x; DESTDIR="$initdir" "$ninja_bin" -C "$BUILD_DIR" install)
test: create and merge code coverage reports in integration tests If -Db_coverage=true is used at build time, then ARTIFACT_DIRECTORY/TEST-XX-FOO.coverage-info files are created with code coverage data, and run-integration-test.sh also merges them into ARTIFACT_DIRECTORY/merged.coverage-info since the coveralls.io helpers accept only a single file.
2021-10-03 23:50:38 +08:00
# If we are doing coverage runs, copy over the binary notes files, as lcov expects to
# find them in the same directory as the runtime data counts
test: loosen sandbox restrictions for integration tests as well Otherwise we miss quite a lot of coverage (mainly from logind, hostnamed, networkd, and possibly others), since they can't write their reports with `ProtectSystem=strict`.
2021-10-22 00:06:36 +08:00
if get_bool "$IS_BUILT_WITH_COVERAGE"; then
test: create and merge code coverage reports in integration tests If -Db_coverage=true is used at build time, then ARTIFACT_DIRECTORY/TEST-XX-FOO.coverage-info files are created with code coverage data, and run-integration-test.sh also merges them into ARTIFACT_DIRECTORY/merged.coverage-info since the coveralls.io helpers accept only a single file.
2021-10-03 23:50:38 +08:00
mkdir -p "${initdir}/${BUILD_DIR:?}/"
rsync -am --include='*/' --include='*.gcno' --exclude='*' "${BUILD_DIR:?}/" "${initdir}/${BUILD_DIR:?}/"
fi
test/test-functions: allow installing systemd files from local system Add NO_BUILD var to allow testing with no local build, by installing local systemd files into the image. This only works for debian-like distros currently, that use the tools 'apt' and 'dpkg' for package management.
2020-11-18 06:20:22 +08:00
}
install_debian_systemd() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install debian systemd"
test/test-functions: allow installing systemd files from local system Add NO_BUILD var to allow testing with no local build, by installing local systemd files into the image. This only works for debian-like distros currently, that use the tools 'apt' and 'dpkg' for package management.
2020-11-18 06:20:22 +08:00
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
local files
while read -r deb; do
files="$(dpkg-query -L "$deb" 2>/dev/null)" || continue
test/test-functions: allow installing systemd files from local system Add NO_BUILD var to allow testing with no local build, by installing local systemd files into the image. This only works for debian-like distros currently, that use the tools 'apt' and 'dpkg' for package management.
2020-11-18 06:20:22 +08:00
ddebug "Install debian files from package $deb"
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
for file in $files; do
test/test-functions: allow installing systemd files from local system Add NO_BUILD var to allow testing with no local build, by installing local systemd files into the image. This only works for debian-like distros currently, that use the tools 'apt' and 'dpkg' for package management.
2020-11-18 06:20:22 +08:00
[ -e "$file" ] || continue
[ -d "$file" ] && continue
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
inst "$file"
test/test-functions: allow installing systemd files from local system Add NO_BUILD var to allow testing with no local build, by installing local systemd files into the image. This only works for debian-like distros currently, that use the tools 'apt' and 'dpkg' for package management.
2020-11-18 06:20:22 +08:00
done
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
done < <(grep -E '^Package:' "${SOURCE_DIR}/debian/control" | cut -d ':' -f 2)
test/test-functions: allow installing systemd files from local system Add NO_BUILD var to allow testing with no local build, by installing local systemd files into the image. This only works for debian-like distros currently, that use the tools 'apt' and 'dpkg' for package management.
2020-11-18 06:20:22 +08:00
}
test: add support for NO_BUILD=1 on openSUSE
2021-08-03 14:18:13 +08:00
install_suse_systemd() {
local testsdir=/usr/lib/systemd/tests
local pkgs
dinfo "Install SUSE systemd"
pkgs=(
systemd
systemd-container
systemd-coredump
systemd-experimental
systemd-journal-remote
systemd-portable
udev
)
for p in "${pkgs[@]}"; do
rpm -q "$p" &>/dev/null || continue
ddebug "Install files from package $p"
while read -r f; do
[ -e "$f" ] || continue
[ -d "$f" ] && continue
inst "$f"
done < <(rpm -ql "$p")
done
# we only need testsdata dir as well as the unit tests (for
# TEST-02-UNITTESTS) in the image.
dinfo "Install unit tests and testdata directory"
mkdir -p "$initdir/$testsdir"
cp "$testsdir"/test-* "$initdir/$testsdir/"
cp -a "$testsdir/testdata" "$initdir/$testsdir/"
# On openSUSE, these dirs are not created at package install for now on.
mkdir -p "$initdir/var/log/journal/remote"
}
test/test-functions: allow installing systemd files from local system Add NO_BUILD var to allow testing with no local build, by installing local systemd files into the image. This only works for debian-like distros currently, that use the tools 'apt' and 'dpkg' for package management.
2020-11-18 06:20:22 +08:00
install_distro_systemd() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install distro systemd"
test/test-functions: allow installing systemd files from local system Add NO_BUILD var to allow testing with no local build, by installing local systemd files into the image. This only works for debian-like distros currently, that use the tools 'apt' and 'dpkg' for package management.
2020-11-18 06:20:22 +08:00
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$LOOKS_LIKE_DEBIAN"; then
test/test-functions: allow installing systemd files from local system Add NO_BUILD var to allow testing with no local build, by installing local systemd files into the image. This only works for debian-like distros currently, that use the tools 'apt' and 'dpkg' for package management.
2020-11-18 06:20:22 +08:00
install_debian_systemd
test: add support for NO_BUILD=1 on openSUSE
2021-08-03 14:18:13 +08:00
elif get_bool "$LOOKS_LIKE_SUSE"; then
install_suse_systemd
test/test-functions: allow installing systemd files from local system Add NO_BUILD var to allow testing with no local build, by installing local systemd files into the image. This only works for debian-like distros currently, that use the tools 'apt' and 'dpkg' for package management.
2020-11-18 06:20:22 +08:00
else
dfatal "NO_BUILD not supported for this distro"
exit 1
fi
}
install_systemd() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install systemd"
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$NO_BUILD"; then
test/test-functions: allow installing systemd files from local system Add NO_BUILD var to allow testing with no local build, by installing local systemd files into the image. This only works for debian-like distros currently, that use the tools 'apt' and 'dpkg' for package management.
2020-11-18 06:20:22 +08:00
install_distro_systemd
else
install_compiled_systemd
fi
test: remove duplicated code
2013-11-06 06:32:56 +08:00
# remove unneeded documentation
test: loosen sandbox restrictions for integration tests as well Otherwise we miss quite a lot of coverage (mainly from logind, hostnamed, networkd, and possibly others), since they can't write their reports with `ProtectSystem=strict`.
2021-10-22 00:06:36 +08:00
rm -fr "${initdir:?}"/usr/share/{man,doc}
tests: enable logging for pid1, disable for other systemd services systemd-udev generated an insane amount of log output at debug level. It would break TEST-02-CRYPTSETUP by filling the overflowing the disk (which seems to be a bug in itself!).
2016-04-26 02:20:57 +08:00
# enable debug logging in PID1
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
echo LogLevel=debug >>"$initdir/etc/systemd/system.conf"
test: store coredumps in journal To make debugging much easier, especially for crashes in tests under QEMU, let's store the entire coredump bundle in the systemd journal, which is usually kept around by various CIs. Right now, we usually end up with a journal, but without the coredump itself, which is pretty useless.
2019-08-13 06:14:54 +08:00
# store coredumps in journal
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
echo Storage=journal >>"$initdir/etc/systemd/coredump.conf"
oom: Add support for user unit ManagedOOM property updates Compared to PID1 where systemd-oomd has to be the client to PID1 because PID1 is a more privileged process than systemd-oomd, systemd-oomd is the more privileged process compared to a user manager so we have user managers be the client whereas systemd-oomd is now the server. The same varlink protocol is used between user managers and systemd-oomd to deliver ManagedOOM property updates. systemd-oomd now sets up a varlink server that user managers connect to to send ManagedOOM property updates. We also add extra validation to make sure that non-root senders don't send updates for cgroups they don't own. The integration test was extended to repeat the chill/bloat test using a user manager instead of PID1.
2021-09-09 23:12:55 +08:00
# Propagate SYSTEMD_UNIT_PATH to user systemd managers
mkdir "$initdir/etc/systemd/system/user@.service.d/"
echo -e "[Service]\nPassEnvironment=SYSTEMD_UNIT_PATH\n" >"$initdir/etc/systemd/system/user@.service.d/override.conf"
test: loosen sandbox restrictions for integration tests as well Otherwise we miss quite a lot of coverage (mainly from logind, hostnamed, networkd, and possibly others), since they can't write their reports with `ProtectSystem=strict`.
2021-10-22 00:06:36 +08:00
test: disable ProtectHome= when collecting coverage as well Depending on the location of the original build dir, either ProtectHome= or ProtectSystem= may get in the way when creating the gcov metadata files. Follow-up to: * 02d7e73013a92c5580023e4e548d19adbc254dbe * 6c9efba67715cd1ced170ac46c04d47934ad276a
2021-10-24 03:20:37 +08:00
# When built with gcov, disable ProtectSystem= and ProtectHome in the test
# images, since it prevents gcov to write the coverage reports (*.gcda files)
test: loosen sandbox restrictions for integration tests as well Otherwise we miss quite a lot of coverage (mainly from logind, hostnamed, networkd, and possibly others), since they can't write their reports with `ProtectSystem=strict`.
2021-10-22 00:06:36 +08:00
if get_bool "$IS_BUILT_WITH_COVERAGE"; then
mkdir -p "$initdir/etc/systemd/system/service.d/"
test: rename the global service override file for coverage runs Otherwise we break TEST-15-DROPIN, since it uses /usr/lib/systemd/system/service.d/override.conf in some of its sub-tests.
2021-10-24 04:10:24 +08:00
echo -e "[Service]\nProtectSystem=no\nProtectHome=no\n" >"$initdir/etc/systemd/system/service.d/gcov-override.conf"
test: loosen sandbox restrictions for integration tests as well Otherwise we miss quite a lot of coverage (mainly from logind, hostnamed, networkd, and possibly others), since they can't write their reports with `ProtectSystem=strict`.
2021-10-22 00:06:36 +08:00
fi
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
tests: follow RUNPATH when installing missing libraries Fixes #3630
2016-07-03 23:17:13 +08:00
get_ldpath() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local rpath
rpath="$(objdump -p "${1:?}" 2>/dev/null | awk "/R(UN)?PATH/ { print \"$initdir\" \$2 }" | paste -sd :)"
test-functions: make sure we use the right library path for binaries without RPATH Meson appears to set the rpath only for some binaries it builds, but not all. (The rules are not clear to me, but that's besides the point of this commit). Let's make sure if our test script operates on a binary that has no rpath set we fall back preferably to the BUILD_DIR rather than directly to the host. This matters if a test uses a libsystemd symbol introduced in a version newer than the one on the host. In that case "ldd" will not work on the test binary if rpath is not set. With this fix that behaviour is corrected, and "ldd" works correctly even in this case. (Or in other words: before this fix on binaries lacking rpath we'd base dependency info on the libraries of the host, not the buidl tree, if they exist in both.)
2019-11-28 01:25:52 +08:00
if [ -z "$rpath" ] ; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
echo "$BUILD_DIR"
test-functions: make sure we use the right library path for binaries without RPATH Meson appears to set the rpath only for some binaries it builds, but not all. (The rules are not clear to me, but that's besides the point of this commit). Let's make sure if our test script operates on a binary that has no rpath set we fall back preferably to the BUILD_DIR rather than directly to the host. This matters if a test uses a libsystemd symbol introduced in a version newer than the one on the host. In that case "ldd" will not work on the test binary if rpath is not set. With this fix that behaviour is corrected, and "ldd" works correctly even in this case. (Or in other words: before this fix on binaries lacking rpath we'd base dependency info on the libraries of the host, not the buidl tree, if they exist in both.)
2019-11-28 01:25:52 +08:00
else
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
echo "$rpath"
test-functions: make sure we use the right library path for binaries without RPATH Meson appears to set the rpath only for some binaries it builds, but not all. (The rules are not clear to me, but that's besides the point of this commit). Let's make sure if our test script operates on a binary that has no rpath set we fall back preferably to the BUILD_DIR rather than directly to the host. This matters if a test uses a libsystemd symbol introduced in a version newer than the one on the host. In that case "ldd" will not work on the test binary if rpath is not set. With this fix that behaviour is corrected, and "ldd" works correctly even in this case. (Or in other words: before this fix on binaries lacking rpath we'd base dependency info on the libraries of the host, not the buidl tree, if they exist in both.)
2019-11-28 01:25:52 +08:00
fi
tests: follow RUNPATH when installing missing libraries Fixes #3630
2016-07-03 23:17:13 +08:00
}
test: remove duplicated code
2013-11-06 06:32:56 +08:00
install_missing_libraries() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install missing libraries"
test: remove duplicated code
2013-11-06 06:32:56 +08:00
# install possible missing libraries
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
for i in "${initdir:?}"{,/usr}/{sbin,bin}/* "$initdir"{,/usr}/lib/systemd/{,tests/{,manual/,unsafe/}}*; do
LD_LIBRARY_PATH="${LD_LIBRARY_PATH:+$LD_LIBRARY_PATH:}$(get_ldpath "$i")" inst_libs "$i"
test: remove duplicated code
2013-11-06 06:32:56 +08:00
done
test: pull in weak deps into tests (Pulled from @bluca's comment here: https://github.com/systemd/systemd/pull/17884#issuecomment-740005624 and turned into a commit by @poettering)
2020-12-09 17:15:36 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local lib path
test: pull in weak deps into tests (Pulled from @bluca's comment here: https://github.com/systemd/systemd/pull/17884#issuecomment-740005624 and turned into a commit by @poettering)
2020-12-09 17:15:36 +08:00
# A number of dependencies is now optional via dlopen, so the install
# script will not pick them up, since it looks at linkage.
core: make libbpf a dlopen() dependency
2021-05-23 20:04:53 +08:00
for lib in libcryptsetup libidn libidn2 pwquality libqrencode tss2-esys tss2-rc tss2-mu libfido2 libbpf; do
test: skip missing optional libraries in image install Not all optional libraries might be available on developers machines, so log and skip. Also some pkg-config files are broken (eg: tss2 on Debian Stable) so skip if the required variables are missing, and improve logs.
2021-01-15 00:48:13 +08:00
ddebug "Searching for $lib via pkg-config"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if pkg-config --exists "$lib"; then
path="$(pkg-config --variable=libdir "$lib")"
test: skip missing optional libraries in image install Not all optional libraries might be available on developers machines, so log and skip. Also some pkg-config files are broken (eg: tss2 on Debian Stable) so skip if the required variables are missing, and improve logs.
2021-01-15 00:48:13 +08:00
if [ -z "${path}" ]; then
ddebug "$lib.pc does not contain a libdir variable, skipping"
continue
fi
if ! [[ ${lib} =~ ^lib ]]; then
lib="lib${lib}"
fi
# Some pkg-config files are broken and give out the wrong paths
# (eg: libcryptsetup), so just ignore them
inst_libs "${path}/${lib}.so" || true
inst_library "${path}/${lib}.so" || true
else
ddebug "$lib.pc not found, skipping"
continue
fi
test: pull in weak deps into tests (Pulled from @bluca's comment here: https://github.com/systemd/systemd/pull/17884#issuecomment-740005624 and turned into a commit by @poettering)
2020-12-09 17:15:36 +08:00
done
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
test: stop caching loopdev It is more trouble than it is worth. The setup is of a loopback device is very quick, so it's better to always create it when needed and immediately drop afterwards.
2020-03-21 03:17:11 +08:00
cleanup_loopdev() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [ -n "${LOOPDEV:=}" ]; then
test: stop caching loopdev It is more trouble than it is worth. The setup is of a loopback device is very quick, so it's better to always create it when needed and immediately drop afterwards.
2020-03-21 03:17:11 +08:00
ddebug "losetup -d $LOOPDEV"
losetup -d "${LOOPDEV}"
test: create public images by default Let's create new images public by default and then symlink/copy them into the respective private directories afterwards, not the other way around. This should fix a nasty race condition in parallel runs where one tests attempts to copy the backing public image at the same moment another test is already modifying it.
2020-05-26 04:48:01 +08:00
unset LOOPDEV
test: stop caching loopdev It is more trouble than it is worth. The setup is of a loopback device is very quick, so it's better to always create it when needed and immediately drop afterwards.
2020-03-21 03:17:11 +08:00
fi
}
test: run loop device cleanup in more cases
2020-03-31 17:46:00 +08:00
trap cleanup_loopdev EXIT INT QUIT PIPE
test: stop caching loopdev It is more trouble than it is worth. The setup is of a loopback device is very quick, so it's better to always create it when needed and immediately drop afterwards.
2020-03-21 03:17:11 +08:00
test: remove duplicated code
2013-11-06 06:32:56 +08:00
create_empty_image() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [ -z "${IMAGE_NAME:=}" ]; then
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
echo "create_empty_image: \$IMAGE_NAME not set"
exit 1
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local size=500
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if ! get_bool "$NO_BUILD"; then
test: do not run 'meson configure' if NO_BUILD is set There is no build tree and packages are used, so it cannot work. Unlikely that static linking has been set for those builds anyway. Fixes https://github.com/systemd/systemd/issues/19955
2021-06-17 04:50:23 +08:00
if meson configure "${BUILD_DIR:?}" | grep 'static-lib\|standalone-binaries' | awk '{ print $2 }' | grep -q 'true'; then
size=$((size+=200))
fi
if meson configure "${BUILD_DIR:?}" | grep 'link-.*-shared' | awk '{ print $2 }' | grep -q 'false'; then
size=$((size+=200))
fi
test: loosen sandbox restrictions for integration tests as well Otherwise we miss quite a lot of coverage (mainly from logind, hostnamed, networkd, and possibly others), since they can't write their reports with `ProtectSystem=strict`.
2021-10-22 00:06:36 +08:00
if get_bool "$IS_BUILT_WITH_COVERAGE"; then
test: create and merge code coverage reports in integration tests If -Db_coverage=true is used at build time, then ARTIFACT_DIRECTORY/TEST-XX-FOO.coverage-info files are created with code coverage data, and run-integration-test.sh also merges them into ARTIFACT_DIRECTORY/merged.coverage-info since the coveralls.io helpers accept only a single file.
2021-10-03 23:50:38 +08:00
size=$((size+=250))
fi
test: increase image size when static library or standalone binaries are installed
2021-05-06 12:28:19 +08:00
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if ! get_bool "$STRIP_BINARIES"; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
size=$((4 * size))
tests: up the image size a little, use twice that size with STRIP_BINARIES=no to make sure that there's enough room for unstripped binaries.
2018-06-30 13:19:01 +08:00
fi
test: improve reporting a bit
2019-12-11 00:33:58 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
echo "Setting up ${IMAGE_PUBLIC:?} (${size} MB)"
rm -f "${IMAGE_PRIVATE:?}" "$IMAGE_PUBLIC"
test: run tests directly from the loopback device Before, we'd copy the test tree into nspawn-root, and run the tests from there. This is OK, and doesn't actually take much extra time. But it uses quite a lot of extra disk space. So let's make things a bit more efficient by running directly from the image file. We still run the unprivileged nspawn tests from a copy. Once the kernel implements fs shift, we can do away with that too.
2019-12-13 21:21:31 +08:00
test: remove duplicated code
2013-11-06 06:32:56 +08:00
# Create the blank file to use as a root filesystem
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
truncate -s "${size}M" "$IMAGE_PUBLIC"
test: run tests directly from the loopback device Before, we'd copy the test tree into nspawn-root, and run the tests from there. This is OK, and doesn't actually take much extra time. But it uses quite a lot of extra disk space. So let's make things a bit more efficient by running directly from the image file. We still run the unprivileged nspawn tests from a copy. Once the kernel implements fs shift, we can do away with that too.
2019-12-13 21:21:31 +08:00
test: fix public/private image detection and prefer the latter one
2020-05-17 16:46:34 +08:00
LOOPDEV=$(losetup --show -P -f "$IMAGE_PUBLIC")
test: wrap $LOOPDEV in quotes Otherwise things go awry if it is not set ('[ -b ]' is not the same as '[ -b "" ]'!).
2013-12-10 20:39:42 +08:00
[ -b "$LOOPDEV" ] || return 1
test-functions: use sfdisk without cyl,head,sectors newer sfdisk implementations do not work with these anymore
2015-02-03 17:44:27 +08:00
sfdisk "$LOOPDEV" <<EOF
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
,$((size - 50))M
test: remove duplicated code
2013-11-06 06:32:56 +08:00
,
EOF
tests: run `udevadm settle` after `sfdisk` (#8610) This makes the script wait for the newly created partition to show up before trying to put a filesystem on it, which should prevent the tests from failing with the following error: ``` New situation: Disklabel type: dos Disk identifier: 0x3541a0ec Device Boot Start End Sectors Size Id Type /dev/loop6p1 2048 800767 798720 390M 83 Linux /dev/loop6p2 800768 819199 18432 9M 83 Linux The partition table has been altered. Calling ioctl() to re-read partition table. Syncing disks. The file /dev/loop6p1 does not exist and no size was specified. make: *** [setup] Error 1 F: Failed to mkfs -t ext4 Makefile:4: recipe for target 'setup' failed ```
2018-03-29 12:20:57 +08:00
udevadm settle
test: specify the rootfs by label otherwise the boot might fail when attaching multiple SATA devices to the VM.
2021-09-14 00:14:05 +08:00
local label=(-L systemd_boot)
basic: fallback to fstatat if entry->d_type is DT_UNKNOWN * tests-functions: improve FSTYPE-support make clean setup FSTYPE=reiserfs is working fine now :) * basic: fallback to fstatat if entry->d_type is DT_UNKNOWN Fixes localectl on reiserfs: -bash-4.3# mkdir -p /usr/lib/locale -bash-4.3# stat -f /usr/lib/locale/ File: "/usr/lib/locale/" ID: bdb0322715b5366e Namelen: 255 Type: reiserfs Block size: 4096 Blocks: Total: 99835 Free: 60262 Available: 60262 Inodes: Total: 0 Free: 0 -bash-4.3# mkdir /usr/lib/locale/HeyHo -bash-4.3# localectl list-locales --no-pager -bash-4.3# mount -t tmpfs tmpfs /usr/lib/locale -bash-4.3# mkdir /usr/lib/locale/HeyHo -bash-4.3# localectl list-locales --no-pager HeyHo
2016-04-04 19:07:56 +08:00
# mkfs.reiserfs doesn't know -L. so, use --label instead
test: specify the rootfs by label otherwise the boot might fail when attaching multiple SATA devices to the VM.
2021-09-14 00:14:05 +08:00
[[ "$FSTYPE" == "reiserfs" ]] && label=(--label systemd_boot)
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if ! mkfs -t "${FSTYPE}" "${label[@]}" "${LOOPDEV}p1" -q; then
basic: fallback to fstatat if entry->d_type is DT_UNKNOWN * tests-functions: improve FSTYPE-support make clean setup FSTYPE=reiserfs is working fine now :) * basic: fallback to fstatat if entry->d_type is DT_UNKNOWN Fixes localectl on reiserfs: -bash-4.3# mkdir -p /usr/lib/locale -bash-4.3# stat -f /usr/lib/locale/ File: "/usr/lib/locale/" ID: bdb0322715b5366e Namelen: 255 Type: reiserfs Block size: 4096 Blocks: Total: 99835 Free: 60262 Available: 60262 Inodes: Total: 0 Free: 0 -bash-4.3# mkdir /usr/lib/locale/HeyHo -bash-4.3# localectl list-locales --no-pager -bash-4.3# mount -t tmpfs tmpfs /usr/lib/locale -bash-4.3# mkdir /usr/lib/locale/HeyHo -bash-4.3# localectl list-locales --no-pager HeyHo
2016-04-04 19:07:56 +08:00
dfatal "Failed to mkfs -t ${FSTYPE}"
exit 1
fi
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
mount_initdir() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [ -z "${LOOPDEV:=}" ]; then
[ -e "${IMAGE_PRIVATE:?}" ] && image="$IMAGE_PRIVATE" || image="${IMAGE_PUBLIC:?}"
LOOPDEV="$(losetup --show -P -f "$image")"
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
[ -b "$LOOPDEV" ] || return 1
test: stop caching loopdev It is more trouble than it is worth. The setup is of a loopback device is very quick, so it's better to always create it when needed and immediately drop afterwards.
2020-03-21 03:17:11 +08:00
udevadm settle
fi
test: add create_empty_image_rootdir() to simplify testcase setup Almost all tests were manually mounting/unmounting $TESTDIR/root from the loopback image; this moves all that into test-functions so the test setup functions are simplier. Also add test_setup_cleanup() function, to cleanup what is mounted by create_empty_image_rootdir()
2019-07-12 23:47:26 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if ! mountpoint -q "${initdir:?}"; then
mkdir -p "$initdir"
mount "${LOOPDEV}p1" "$initdir"
test: stop caching loopdev It is more trouble than it is worth. The setup is of a loopback device is very quick, so it's better to always create it when needed and immediately drop afterwards.
2020-03-21 03:17:11 +08:00
TEST_SETUP_CLEANUP_ROOTDIR=1
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
fi
}
test: run tests directly from the loopback device Before, we'd copy the test tree into nspawn-root, and run the tests from there. This is OK, and doesn't actually take much extra time. But it uses quite a lot of extra disk space. So let's make things a bit more efficient by running directly from the image file. We still run the unprivileged nspawn tests from a copy. Once the kernel implements fs shift, we can do away with that too.
2019-12-13 21:21:31 +08:00
cleanup_initdir() {
# only umount if create_empty_image_rootdir() was called to mount it
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
get_bool "$TEST_SETUP_CLEANUP_ROOTDIR" && _umount_dir "${initdir:?}"
test: run tests directly from the loopback device Before, we'd copy the test tree into nspawn-root, and run the tests from there. This is OK, and doesn't actually take much extra time. But it uses quite a lot of extra disk space. So let's make things a bit more efficient by running directly from the image file. We still run the unprivileged nspawn tests from a copy. Once the kernel implements fs shift, we can do away with that too.
2019-12-13 21:21:31 +08:00
}
umount_loopback() {
# unmount the loopback device from all places. Otherwise we risk file
# system corruption.
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
for device in $(losetup -l | awk '$6=="'"${IMAGE_PUBLIC:?}"'" {print $1}'); do
test: run tests directly from the loopback device Before, we'd copy the test tree into nspawn-root, and run the tests from there. This is OK, and doesn't actually take much extra time. But it uses quite a lot of extra disk space. So let's make things a bit more efficient by running directly from the image file. We still run the unprivileged nspawn tests from a copy. Once the kernel implements fs shift, we can do away with that too.
2019-12-13 21:21:31 +08:00
ddebug "Unmounting all uses of $device"
mount | awk '/^'"${device}"'p/{print $1}' | xargs --no-run-if-empty umount -v
done
}
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
create_empty_image_rootdir() {
create_empty_image
mount_initdir
}
tests: introduce check_asan_reports and use it in check_result_{qemu|nspawn}
2018-12-25 15:31:50 +08:00
check_asan_reports() {
local ret=0
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local root="${1:?}"
tests: introduce check_asan_reports and use it in check_result_{qemu|nspawn}
2018-12-25 15:31:50 +08:00
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$IS_BUILT_WITH_ASAN"; then
tests: introduce check_asan_reports and use it in check_result_{qemu|nspawn}
2018-12-25 15:31:50 +08:00
ls -l "$root"
if [[ -e "$root/systemd.asan.log.1" ]]; then
cat "$root/systemd.asan.log.1"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
ret=$((ret+1))
tests: reproduce https://github.com/systemd/systemd/issues/11251
2018-12-23 21:46:00 +08:00
fi
tests: fail if asan has found issues in journald
2018-12-23 22:30:41 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
journald_report="$(find "$root" -name "systemd-journald.*san.log*" -exec cat {} \;)"
if [[ -n "$journald_report" ]]; then
tests: redirect the stdout/stderr of journald to a file (under ASan+UBSan) Sometimes UBSan sends its reports to stderr regardless of what is specified in log_path Let's try to catch them by redirecting stderr (and stdout just in case) to a file See https://github.com/systemd/systemd/pull/12524#issuecomment-491108821
2019-05-10 08:47:03 +08:00
printf "%s\n" "$journald_report"
test: convert all uses of '|| true' into '|| :' No change in functionality; just use the shorter || :
2019-07-19 08:34:57 +08:00
cat "$root/systemd-journald.out" || :
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
ret=$((ret+1))
tests: ignore memory leaks in dbus-daemon Otherwise, the test fails on Fedora 28 with ``` Jan 30 01:42:35 systemd-testsuite dbus-daemon[61]: [system] Successfully activated service 'org.freedesktop.systemd1' Jan 30 01:42:35 systemd-testsuite systemd[61]: dbus.service: Kernel keyring access prohibited, ignoring. Jan 30 01:42:35 systemd-testsuite systemd[61]: dbus.service: Executing: /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: ================================================================= Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: ==61==ERROR: LeakSanitizer: detected memory leaks Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: Direct leak of 72 byte(s) in 1 object(s) allocated from: Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #0 0x7f21f9e29088 in __interceptor_realloc (/usr/lib64/libasan.so.5+0xef088) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #1 0x7f21f9b1b23c (/lib64/libdbus-1.so.3+0x3323c) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #2 0x240000001b (<unknown module>) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: Direct leak of 16 byte(s) in 1 object(s) allocated from: Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #0 0x7f21f9e29088 in __interceptor_realloc (/usr/lib64/libasan.so.5+0xef088) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #1 0x7f21f9b1b23c (/lib64/libdbus-1.so.3+0x3323c) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #2 0x7ffffffff (<unknown module>) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: SUMMARY: AddressSanitizer: 88 byte(s) leaked in 2 allocation(s). ``` The leaks were reported and fixed in https://bugs.freedesktop.org/show_bug.cgi?id=107320.
2019-01-30 09:19:45 +08:00
fi
tests: look for ASan+UBSan reports in the journal
2018-12-23 22:48:18 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
pids="$(
test/test-functions: use binaries from $BUILD_DIR or installed system In Ubuntu CI, we test binaries from the installed system, not from $BUILD_DIR, so use the appropriate binary. Most of the calls to the binaries are part of checking/processing asan-built binaries, and so did not apply to Ubuntu CI, except for generating noise in the stderr log like: objdump: '/tmp/autopkgtest.83yGoI/build.fHB/src/test/TEST-01-BASIC/systemd-journald': No such file However this also applies to the call to systemd-nspawn, which the debian upstream test wrapper was sed-adjusting to use the installed binary instead of the binary in $BUILD_DIR. This commit allows removing that sed processing of the test-functions file during Ubuntu CI test.
2019-08-13 19:52:57 +08:00
"$JOURNALCTL" -D "$root/var/log/journal" | perl -alne '
tests: ignore memory leaks in dbus-daemon Otherwise, the test fails on Fedora 28 with ``` Jan 30 01:42:35 systemd-testsuite dbus-daemon[61]: [system] Successfully activated service 'org.freedesktop.systemd1' Jan 30 01:42:35 systemd-testsuite systemd[61]: dbus.service: Kernel keyring access prohibited, ignoring. Jan 30 01:42:35 systemd-testsuite systemd[61]: dbus.service: Executing: /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: ================================================================= Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: ==61==ERROR: LeakSanitizer: detected memory leaks Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: Direct leak of 72 byte(s) in 1 object(s) allocated from: Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #0 0x7f21f9e29088 in __interceptor_realloc (/usr/lib64/libasan.so.5+0xef088) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #1 0x7f21f9b1b23c (/lib64/libdbus-1.so.3+0x3323c) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #2 0x240000001b (<unknown module>) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: Direct leak of 16 byte(s) in 1 object(s) allocated from: Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #0 0x7f21f9e29088 in __interceptor_realloc (/usr/lib64/libasan.so.5+0xef088) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #1 0x7f21f9b1b23c (/lib64/libdbus-1.so.3+0x3323c) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #2 0x7ffffffff (<unknown module>) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: SUMMARY: AddressSanitizer: 88 byte(s) leaked in 2 allocation(s). ``` The leaks were reported and fixed in https://bugs.freedesktop.org/show_bug.cgi?id=107320.
2019-01-30 09:19:45 +08:00
BEGIN {
%services_to_ignore = (
"dbus-daemon" => undef,
);
}
test: unify indentation
2019-03-16 22:49:43 +08:00
print $2 if /\s(\S*)\[(\d+)\]:\s*SUMMARY:\s+\w+Sanitizer/ && !exists $services_to_ignore{$1}'
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
)"
if [[ -n "$pids" ]]; then
ret=$((ret+1))
tests: look for ASan+UBSan reports in the journal
2018-12-23 22:48:18 +08:00
for pid in $pids; do
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
"$JOURNALCTL" -D "$root/var/log/journal" _PID="$pid" --no-pager
tests: look for ASan+UBSan reports in the journal
2018-12-23 22:48:18 +08:00
done
tests: ignore memory leaks in dbus-daemon Otherwise, the test fails on Fedora 28 with ``` Jan 30 01:42:35 systemd-testsuite dbus-daemon[61]: [system] Successfully activated service 'org.freedesktop.systemd1' Jan 30 01:42:35 systemd-testsuite systemd[61]: dbus.service: Kernel keyring access prohibited, ignoring. Jan 30 01:42:35 systemd-testsuite systemd[61]: dbus.service: Executing: /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: ================================================================= Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: ==61==ERROR: LeakSanitizer: detected memory leaks Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: Direct leak of 72 byte(s) in 1 object(s) allocated from: Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #0 0x7f21f9e29088 in __interceptor_realloc (/usr/lib64/libasan.so.5+0xef088) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #1 0x7f21f9b1b23c (/lib64/libdbus-1.so.3+0x3323c) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #2 0x240000001b (<unknown module>) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: Direct leak of 16 byte(s) in 1 object(s) allocated from: Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #0 0x7f21f9e29088 in __interceptor_realloc (/usr/lib64/libasan.so.5+0xef088) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #1 0x7f21f9b1b23c (/lib64/libdbus-1.so.3+0x3323c) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #2 0x7ffffffff (<unknown module>) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: SUMMARY: AddressSanitizer: 88 byte(s) leaked in 2 allocation(s). ``` The leaks were reported and fixed in https://bugs.freedesktop.org/show_bug.cgi?id=107320.
2019-01-30 09:19:45 +08:00
fi
tests: reproduce https://github.com/systemd/systemd/issues/11251
2018-12-23 21:46:00 +08:00
fi
test: remove duplicated code
2013-11-06 06:32:56 +08:00
return $ret
}
test: create and merge code coverage reports in integration tests If -Db_coverage=true is used at build time, then ARTIFACT_DIRECTORY/TEST-XX-FOO.coverage-info files are created with code coverage data, and run-integration-test.sh also merges them into ARTIFACT_DIRECTORY/merged.coverage-info since the coveralls.io helpers accept only a single file.
2021-10-03 23:50:38 +08:00
check_coverage_reports() {
local root="${1:?}"
if get_bool "$NO_BUILD"; then
return 0
fi
test: loosen sandbox restrictions for integration tests as well Otherwise we miss quite a lot of coverage (mainly from logind, hostnamed, networkd, and possibly others), since they can't write their reports with `ProtectSystem=strict`.
2021-10-22 00:06:36 +08:00
if ! get_bool "$IS_BUILT_WITH_COVERAGE"; then
test: create and merge code coverage reports in integration tests If -Db_coverage=true is used at build time, then ARTIFACT_DIRECTORY/TEST-XX-FOO.coverage-info files are created with code coverage data, and run-integration-test.sh also merges them into ARTIFACT_DIRECTORY/merged.coverage-info since the coveralls.io helpers accept only a single file.
2021-10-03 23:50:38 +08:00
return 0
fi
if [ -n "${ARTIFACT_DIRECTORY}" ]; then
dest="${ARTIFACT_DIRECTORY}/${testname:?}.coverage-info"
else
dest="${TESTDIR:?}/coverage-info"
fi
# Create a coverage report that will later be uploaded. Remove info about
# system libraries/headers, as we don't really care about them.
test: merge coverage reports from previous test runs Relevant mainly for tests which utilize both QEMU and nspawn.
2021-10-25 17:35:47 +08:00
if [[ -f "$dest" ]]; then
# If the destination report file already exists, don't overwrite it, but
# dump the new report in a temporary file and then merge it with the already
# present one - this usually happens when running both "parts" of a test
# in one run (the qemu and the nspawn part).
lcov --directory "${root}/${BUILD_DIR:?}" --capture --output-file "${dest}.new"
lcov --remove "${dest}.new" -o "${dest}.new" '/usr/include/*' '/usr/lib/*'
lcov --add-tracefile "${dest}" --add-tracefile "${dest}.new" -o "${dest}"
rm -f "${dest}.new"
else
lcov --directory "${root}/${BUILD_DIR:?}" --capture --output-file "${dest}"
lcov --remove "${dest}" -o "${dest}" '/usr/include/*' '/usr/lib/*'
fi
test: create and merge code coverage reports in integration tests If -Db_coverage=true is used at build time, then ARTIFACT_DIRECTORY/TEST-XX-FOO.coverage-info files are created with code coverage data, and run-integration-test.sh also merges them into ARTIFACT_DIRECTORY/merged.coverage-info since the coveralls.io helpers accept only a single file.
2021-10-03 23:50:38 +08:00
return 0
}
test: use a helper function to move the journal files
2020-03-30 22:39:31 +08:00
save_journal() {
test: optionally, only save test journal for failing tests Saving the journal for passing tests creates a huge amount of unneeded data stored for each full test run. Add a env var to allow saving the journal only for failed tests.
2021-07-02 22:38:14 +08:00
# Default to always saving journal
local save="yes"
if [ "${TEST_SAVE_JOURNAL}" = "no" ]; then
save="no"
elif [ "${TEST_SAVE_JOURNAL}" = "fail" ] && [ "$2" = "0" ]; then
save="no"
fi
test: use systemd-journal-remote for journal copying, support $ARTIFACT_DIRECTORY Using s-j-remote fixes the following issue: when coalescing files from multiple inputs, simply copying all files with into the the same directory might potentially mess things up, because a newer system.journal might overwrite an older journal. This happens because we run multiple tests from the same image, and need to clean out the directory after each run. By using systemd-journal-remote, we nicely coalesce all files. This has the advantage that if there aren't too many logs, we end up with just one journal file. ARTIFACT_DIRECTORY is for ubuntuautopackagetests, where the journal files are copied to a separate directory to preserve after tests have been run. This functionality can now be recreated by setting ARTIFACT_DIRECTORY=$AUTOPKGTEST_ARTIFACTS.
2020-03-31 03:29:52 +08:00
if [ -n "${ARTIFACT_DIRECTORY}" ]; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dest="${ARTIFACT_DIRECTORY}/${testname:?}.journal"
test: use systemd-journal-remote for journal copying, support $ARTIFACT_DIRECTORY Using s-j-remote fixes the following issue: when coalescing files from multiple inputs, simply copying all files with into the the same directory might potentially mess things up, because a newer system.journal might overwrite an older journal. This happens because we run multiple tests from the same image, and need to clean out the directory after each run. By using systemd-journal-remote, we nicely coalesce all files. This has the advantage that if there aren't too many logs, we end up with just one journal file. ARTIFACT_DIRECTORY is for ubuntuautopackagetests, where the journal files are copied to a separate directory to preserve after tests have been run. This functionality can now be recreated by setting ARTIFACT_DIRECTORY=$AUTOPKGTEST_ARTIFACTS.
2020-03-31 03:29:52 +08:00
else
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dest="${TESTDIR:?}/system.journal"
test: use systemd-journal-remote for journal copying, support $ARTIFACT_DIRECTORY Using s-j-remote fixes the following issue: when coalescing files from multiple inputs, simply copying all files with into the the same directory might potentially mess things up, because a newer system.journal might overwrite an older journal. This happens because we run multiple tests from the same image, and need to clean out the directory after each run. By using systemd-journal-remote, we nicely coalesce all files. This has the advantage that if there aren't too many logs, we end up with just one journal file. ARTIFACT_DIRECTORY is for ubuntuautopackagetests, where the journal files are copied to a separate directory to preserve after tests have been run. This functionality can now be recreated by setting ARTIFACT_DIRECTORY=$AUTOPKGTEST_ARTIFACTS.
2020-03-31 03:29:52 +08:00
fi
test: use a helper function to move the journal files
2020-03-30 22:39:31 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
for j in "${1:?}"/*; do
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$save"; then
test: optionally, only save test journal for failing tests Saving the journal for passing tests creates a huge amount of unneeded data stored for each full test run. Add a env var to allow saving the journal only for failed tests.
2021-07-02 22:38:14 +08:00
"$SYSTEMD_JOURNAL_REMOTE" -o "$dest" --getter="$JOURNALCTL -o export -D $j"
fi
test: optionally print the journal after tests This is hopefully the last bit of functionality from https://salsa.debian.org/systemd-team/systemd/-/blob/debian/master/debian/tests/upstream.
2020-03-31 03:45:21 +08:00
if [ -n "${TEST_SHOW_JOURNAL}" ]; then
echo "---- $j ----"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
"$JOURNALCTL" --no-pager -o short-monotonic --no-hostname --priority="${TEST_SHOW_JOURNAL}" -D "$j"
test: optionally print the journal after tests This is hopefully the last bit of functionality from https://salsa.debian.org/systemd-team/systemd/-/blob/debian/master/debian/tests/upstream.
2020-03-31 03:45:21 +08:00
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
rm -r "$j"
test: use systemd-journal-remote for journal copying, support $ARTIFACT_DIRECTORY Using s-j-remote fixes the following issue: when coalescing files from multiple inputs, simply copying all files with into the the same directory might potentially mess things up, because a newer system.journal might overwrite an older journal. This happens because we run multiple tests from the same image, and need to clean out the directory after each run. By using systemd-journal-remote, we nicely coalesce all files. This has the advantage that if there aren't too many logs, we end up with just one journal file. ARTIFACT_DIRECTORY is for ubuntuautopackagetests, where the journal files are copied to a separate directory to preserve after tests have been run. This functionality can now be recreated by setting ARTIFACT_DIRECTORY=$AUTOPKGTEST_ARTIFACTS.
2020-03-31 03:29:52 +08:00
done
test: use a helper function to move the journal files
2020-03-30 22:39:31 +08:00
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if ! get_bool "$save"; then
test: optionally, only save test journal for failing tests Saving the journal for passing tests creates a huge amount of unneeded data stored for each full test run. Add a env var to allow saving the journal only for failed tests.
2021-07-02 22:38:14 +08:00
return 0
fi
test: drop default ACL from $TESTDIR This fixes an issue introduced by the commit 954c77c2510c0328fd98354a59f380945752c38c. For some reasons, setting default ACL on $TESTDIR makes TEST-29-PORTABLE fail. Let's drop the default ACL, and set ACL on saved results instead. Fixes #19519.
2021-05-06 14:47:01 +08:00
if [ -n "${SUDO_USER}" ]; then
setfacl -m "user:${SUDO_USER:?}:r-X" "$dest"*
fi
test: use a helper function to move the journal files
2020-03-30 22:39:31 +08:00
# we want to print this sometime later, so save this in a variable
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
JOURNAL_LIST="$(ls -l "$dest"*)"
test: use a helper function to move the journal files
2020-03-30 22:39:31 +08:00
}
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
check_result_common() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local workspace="${1:?}"
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
local ret
if [ -s "$workspace/failed" ]; then
test: properly catch tests error with no /testok or empty /failed When editing this function in 7bf20e48bd7d641a39a14a7feb749b7e8, I couldn't decide whether to initialize ret at the top and only reset it on success, or whether to assign a value in each branch. In the end I did neither ;( So if the test finished without creating any of the result files, we would echo a message, but return "success". But there was bigger confusion with /failed: some tests create it empty, some don't. I think we may want to do away pre-creation of /failed completely, and assume the test failed unless /testok is found. But I'm leaving that for later rework. For now let's just make sure we report return success only if /testok or /skipped is found.
2021-04-29 15:13:12 +08:00
# Non-empty …/failed has highest priority
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
cp -a "$workspace/failed" "${TESTDIR:?}/"
test: drop default ACL from $TESTDIR This fixes an issue introduced by the commit 954c77c2510c0328fd98354a59f380945752c38c. For some reasons, setting default ACL on $TESTDIR makes TEST-29-PORTABLE fail. Let's drop the default ACL, and set ACL on saved results instead. Fixes #19519.
2021-05-06 14:47:01 +08:00
if [ -n "${SUDO_USER}" ]; then
setfacl -m "user:${SUDO_USER:?}:r-X" "${TESTDIR:?}/"failed
fi
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
ret=1
test: make sure test timeout has a higher priority than a pass otherwise we might mark tests where something crashes during shutdown as successful, as happened in one of the recent TEST-01-BASIC runs: ``` testsuite-01.service: About to execute rm -f /failed /testok testsuite-01.service: Forked rm as 606 testsuite-01.service: Executing: rm -f /failed /testoktestsuite-01.service: Changed dead -> start-pre Starting TEST-01-BASIC... ... Child 606 (rm) died (code=exited, status=0/SUCCESS) testsuite-01.service: Child 606 belongs to testsuite-01.service. testsuite-01.service: Control process exited, code=exited, status=0/SUCCESS (success) testsuite-01.service: Got final SIGCHLD for state start-pre. testsuite-01.service: Passing 0 fds to service testsuite-01.service: About to execute sh -e -x -c "systemctl --state=failed --no-legend --no-pager >/failed ; systemctl daemon-reload ; echo OK >/testok" testsuite-01.service: Forked sh as 607 testsuite-01.service: Changed start-pre -> start testsuite-01.service: Executing: sh -e -x -c "systemctl --state=failed --no-legend --no-pager >/failed ; systemctl daemon-reload ; echo OK >/testok"systemd-journald.service: Got notification message from PID 560 (FDSTORE=1)S ... testsuite-01.service: Child 607 belongs to testsuite-01.service. testsuite-01.service: Main process exited, code=exited, status=0/SUCCESS (success) testsuite-01.service: Deactivated successfully. testsuite-01.service: Service will not restart (restart setting) testsuite-01.service: Changed start -> dead testsuite-01.service: Job 207 testsuite-01.service/start finished, result=done [ OK ] Finished TEST-01-BASIC. ... end.service: About to execute /bin/sh -x -c "systemctl poweroff --no-block" end.service: Forked /bin/sh as 623end.service: Executing: /bin/sh -x -c "systemctl poweroff --no-block" ... end.service: Job 213 end.service/start finished, result=canceled Caught <SEGV>, dumped core as pid 624. Freezing execution. CentOS Linux 8 Kernel 4.18.0-305.12.1.el8_4.x86_64 on an x86_64 (ttyS0) H login: qemu-kvm: terminating on signal 15 from pid 80134 (timeout) E: Test timed out after 600s Spawning getter /root/systemd/build/journalctl -o export -D /var/tmp/systemd-test.0UYjAS/root/var/log/journal/ca6031c2491543fe8286c748258df8d1... Finishing after writing 15125 entries Spawning getter /root/systemd/build/journalctl -o export -D /var/tmp/systemd-test.0UYjAS/root/var/log/journal/remote... Finishing after writing 0 entries -rw-r-----. 1 root root 25165824 Aug 20 12:26 /var/tmp/systemd-test.0UYjAS/system.journal TEST-01-BASIC RUN: Basic systemd setup [OK] ...
2021-08-20 20:08:18 +08:00
elif get_bool "$TIMED_OUT"; then
echo "(timeout)" >"${TESTDIR:?}/failed"
ret=2
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
elif [ -e "$workspace/testok" ]; then
# …/testok always counts (but with lower priority than …/failed)
ret=0
elif [ -e "$workspace/skipped" ]; then
# …/skipped always counts (a message is expected)
echo "${TESTNAME:?} was skipped:"
cat "$workspace/skipped"
ret=0
else
test: properly catch tests error with no /testok or empty /failed When editing this function in 7bf20e48bd7d641a39a14a7feb749b7e8, I couldn't decide whether to initialize ret at the top and only reset it on success, or whether to assign a value in each branch. In the end I did neither ;( So if the test finished without creating any of the result files, we would echo a message, but return "success". But there was bigger confusion with /failed: some tests create it empty, some don't. I think we may want to do away pre-creation of /failed completely, and assume the test failed unless /testok is found. But I'm leaving that for later rework. For now let's just make sure we report return success only if /testok or /skipped is found.
2021-04-29 15:13:12 +08:00
echo "(failed; see logs)" >"${TESTDIR:?}/failed"
ret=3
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
fi
test: properly catch tests error with no /testok or empty /failed When editing this function in 7bf20e48bd7d641a39a14a7feb749b7e8, I couldn't decide whether to initialize ret at the top and only reset it on success, or whether to assign a value in each branch. In the end I did neither ;( So if the test finished without creating any of the result files, we would echo a message, but return "success". But there was bigger confusion with /failed: some tests create it empty, some don't. I think we may want to do away pre-creation of /failed completely, and assume the test failed unless /testok is found. But I'm leaving that for later rework. For now let's just make sure we report return success only if /testok or /skipped is found.
2021-04-29 15:13:12 +08:00
check_asan_reports "$workspace" || ret=4
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
test: create and merge code coverage reports in integration tests If -Db_coverage=true is used at build time, then ARTIFACT_DIRECTORY/TEST-XX-FOO.coverage-info files are created with code coverage data, and run-integration-test.sh also merges them into ARTIFACT_DIRECTORY/merged.coverage-info since the coveralls.io helpers accept only a single file.
2021-10-03 23:50:38 +08:00
check_coverage_reports "$workspace" || ret=5
test: optionally, only save test journal for failing tests Saving the journal for passing tests creates a huge amount of unneeded data stored for each full test run. Add a env var to allow saving the journal only for failed tests.
2021-07-02 22:38:14 +08:00
save_journal "$workspace/var/log/journal" $ret
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [ -d "${ARTIFACT_DIRECTORY}" ] && [ -f "$workspace/strace.out" ]; then
cp "$workspace/strace.out" "${ARTIFACT_DIRECTORY}/"
fi
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
test: properly catch tests error with no /testok or empty /failed When editing this function in 7bf20e48bd7d641a39a14a7feb749b7e8, I couldn't decide whether to initialize ret at the top and only reset it on success, or whether to assign a value in each branch. In the end I did neither ;( So if the test finished without creating any of the result files, we would echo a message, but return "success". But there was bigger confusion with /failed: some tests create it empty, some don't. I think we may want to do away pre-creation of /failed completely, and assume the test failed unless /testok is found. But I'm leaving that for later rework. For now let's just make sure we report return success only if /testok or /skipped is found.
2021-04-29 15:13:12 +08:00
if [ ${ret:?} != 0 ] && [ -f "$TESTDIR/failed" ]; then
echo -n "${TESTNAME:?}: "
cat "$TESTDIR/failed"
fi
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
echo "${JOURNAL_LIST:-"No journals were saved"}"
test: properly catch tests error with no /testok or empty /failed When editing this function in 7bf20e48bd7d641a39a14a7feb749b7e8, I couldn't decide whether to initialize ret at the top and only reset it on success, or whether to assign a value in each branch. In the end I did neither ;( So if the test finished without creating any of the result files, we would echo a message, but return "success". But there was bigger confusion with /failed: some tests create it empty, some don't. I think we may want to do away pre-creation of /failed completely, and assume the test failed unless /testok is found. But I'm leaving that for later rework. For now let's just make sure we report return success only if /testok or /skipped is found.
2021-04-29 15:13:12 +08:00
return ${ret:?}
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
}
check_result_nspawn() {
local workspace="${1:?}"
local ret
check_result_common "${workspace}"
ret=$?
test: add a couple of hooks for the check_result_*() functions So we don't have to duplicate the whole functions if we need to inject some test-specific checks.
2021-04-27 01:22:42 +08:00
# Run additional test-specific checks if defined by check_result_nspawn_hook()
if declare -F check_result_nspawn_hook >/dev/null; then
if ! check_result_nspawn_hook; then
derror "check_result_nspawn_hook() returned with EC > 0"
ret=4
fi
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
_umount_dir "${initdir:?}"
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
tests: introduce check_asan_reports and use it in check_result_{qemu|nspawn}
2018-12-25 15:31:50 +08:00
return $ret
}
test: Factorize common integration test functions (#6540) All test/TEST* but TEST-02-CRYPTSETUP share the same check_result_qemu() and test_cleanup(), so move them into test_functions and only override them in TEST-02-CRYPTSETUP. Also provide a common test_run() which by default assumes that both QEMU and nspawn tests are run. Particular tests which don't support either need to explicitly opt out by setting $TEST_NO_{QEMU,NSPAWN}. Do it this way around to avoid accidentally forgetting to opt in, and to encourage test authors to at least always support nspawn.
2017-08-04 20:34:14 +08:00
# can be overridden in specific test
check_result_qemu() {
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
local ret
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
mount_initdir
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
check_result_common "${initdir:?}"
ret=$?
_umount_dir "${initdir:?}"
test: add a couple of hooks for the check_result_*() functions So we don't have to duplicate the whole functions if we need to inject some test-specific checks.
2021-04-27 01:22:42 +08:00
# Run additional test-specific checks if defined by check_result_qemu_hook()
if declare -F check_result_qemu_hook >/dev/null; then
if ! check_result_qemu_hook; then
derror "check_result_qemu_hook() returned with EC > 0"
ret=4
fi
fi
test: Factorize common integration test functions (#6540) All test/TEST* but TEST-02-CRYPTSETUP share the same check_result_qemu() and test_cleanup(), so move them into test_functions and only override them in TEST-02-CRYPTSETUP. Also provide a common test_run() which by default assumes that both QEMU and nspawn tests are run. Particular tests which don't support either need to explicitly opt out by setting $TEST_NO_{QEMU,NSPAWN}. Do it this way around to avoid accidentally forgetting to opt in, and to encourage test authors to at least always support nspawn.
2017-08-04 20:34:14 +08:00
return $ret
}
test: move custom result checks from TEST-02-UNITTESTS to test-functions
2021-06-22 03:05:47 +08:00
check_result_nspawn_unittests() {
local workspace="${1:?}"
local ret=1
[[ -e "$workspace/testok" ]] && ret=0
if [[ -s "$workspace/failed" ]]; then
ret=$((ret + 1))
echo "=== Failed test log ==="
cat "$workspace/failed"
else
if [[ -s "$workspace/skipped" ]]; then
echo "=== Skipped test log =="
cat "$workspace/skipped"
# We might have only skipped tests - that should not fail the job
ret=0
fi
if [[ -s "$workspace/testok" ]]; then
echo "=== Passed tests ==="
cat "$workspace/testok"
fi
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
get_bool "${TIMED_OUT:=}" && ret=1
test: collect coverage in a couple more codepaths Otherwise we'd miss coverage from TEST-{02,24,61}, since they don't go through the "standard" `check_result_common` hook. Follow-up to 7bf20e48bd7d641a39a14a7feb749b7e8b0fc0f6.
2021-10-20 19:00:37 +08:00
check_coverage_reports "$workspace" || ret=5
test: optionally, only save test journal for failing tests Saving the journal for passing tests creates a huge amount of unneeded data stored for each full test run. Add a env var to allow saving the journal only for failed tests.
2021-07-02 22:38:14 +08:00
save_journal "$workspace/var/log/journal" $ret
test: move custom result checks from TEST-02-UNITTESTS to test-functions
2021-06-22 03:05:47 +08:00
_umount_dir "${initdir:?}"
return $ret
}
check_result_qemu_unittests() {
local ret=1
mount_initdir
[[ -e "${initdir:?}/testok" ]] && ret=0
if [[ -s "$initdir/failed" ]]; then
ret=$((ret + 1))
echo "=== Failed test log ==="
cat "$initdir/failed"
else
if [[ -s "$initdir/skipped" ]]; then
echo "=== Skipped test log =="
cat "$initdir/skipped"
# We might have only skipped tests - that should not fail the job
ret=0
fi
if [[ -s "$initdir/testok" ]]; then
echo "=== Passed tests ==="
cat "$initdir/testok"
fi
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
get_bool "${TIMED_OUT:=}" && ret=1
test: collect coverage in a couple more codepaths Otherwise we'd miss coverage from TEST-{02,24,61}, since they don't go through the "standard" `check_result_common` hook. Follow-up to 7bf20e48bd7d641a39a14a7feb749b7e8b0fc0f6.
2021-10-20 19:00:37 +08:00
check_coverage_reports "$initdir" || ret=5
test: optionally, only save test journal for failing tests Saving the journal for passing tests creates a huge amount of unneeded data stored for each full test run. Add a env var to allow saving the journal only for failed tests.
2021-07-02 22:38:14 +08:00
save_journal "$initdir/var/log/journal" $ret
test: move custom result checks from TEST-02-UNITTESTS to test-functions
2021-06-22 03:05:47 +08:00
_umount_dir "$initdir"
return $ret
}
test: remove duplicated code
2013-11-06 06:32:56 +08:00
strip_binaries() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Strip binaries"
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if ! get_bool "$STRIP_BINARIES"; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "STRIP_BINARIES == no, keeping binaries unstripped"
tests: add STRIP_BINARIES We need a beautiful stacktraces sometimes For example https://github.com/systemd/systemd/pull/2328
2016-01-18 14:45:20 +08:00
return 0
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
while read -r bin; do
strip --strip-unneeded "$bin" |& grep -vi 'file format not recognized' | ddebug || :
done < <(find "${initdir:?}" -executable -not -path '*/lib/modules/*.ko' -type f)
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
create_rc_local() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Create rc.local"
mkdir -p "${initdir:?}/etc/rc.d"
cat >"$initdir/etc/rc.d/rc.local" <<EOF
treewide: more portable bash shebangs As in 2a5fcfae024ffc370bb780572279f45a1da3f946 and in 3e67e5c9928f8b1e1c5a63def88d53ed1fed12eb using /usr/bin/env allows bash to be looked up in PATH rather than being hard-coded. As with the previous changes the same arguments apply - distributions have scripts to rewrite shebangs on installation and they know what locations to rely on. - For tests/compilation we should rather rely on the user to have setup there PATH correctly. In particular this makes testing from git easier on NixOS where do not provide /bin/bash to improve compose-ability.
2020-03-04 17:35:06 +08:00
#!/usr/bin/env bash
test: remove duplicated code
2013-11-06 06:32:56 +08:00
exit 0
EOF
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
chmod 0755 "$initdir/etc/rc.d/rc.local"
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
install_execs() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
ddebug "Install executables from the service files"
local pkg_config_path="${BUILD_DIR:?}/src/core/"
local systemunitdir userunitdir exe
systemunitdir="$(PKG_CONFIG_PATH="$pkg_config_path" pkg-config --variable=systemdsystemunitdir systemd)"
userunitdir="$(PKG_CONFIG_PATH="$pkg_config_path" pkg-config --variable=systemduserunitdir systemd)"
while read -r exe; do
# some {rc,halt}.local scripts and programs are okay to not exist, the rest should
# also, plymouth is pulled in by rescue.service, but even there the exit code
# is ignored; as it's not present on some distros, don't fail if it doesn't exist
dinfo "Attempting to install $exe (based on unit file reference)"
inst "$exe" || [ "${exe%.local}" != "$exe" ] || [ "${exe%systemd-update-done}" != "$exe" ] || [ "${exe##*/}" == "plymouth" ]
done < <(sed -r -n 's|^Exec[a-zA-Z]*=[@+!-]*([^ ]+).*|\1|gp' "${initdir:?}"/{"$systemunitdir","$userunitdir"}/*.service | sort -u)
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
generate_module_dependencies() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Generate modules dependencies"
if [[ -d "${initdir:?}/lib/modules/${KERNEL_VER:?}" ]] && \
! depmod -a -b "$initdir" "$KERNEL_VER"; then
test: remove duplicated code
2013-11-06 06:32:56 +08:00
dfatal "\"depmod -a $KERNEL_VER\" failed."
exit 1
fi
}
install_depmod_files() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install depmod files"
inst "/lib/modules/${KERNEL_VER:?}/modules.order"
inst "/lib/modules/$KERNEL_VER/modules.builtin"
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
install_plymouth() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install plymouth"
test: remove duplicated code
2013-11-06 06:32:56 +08:00
# install plymouth, if found... else remove plymouth service files
# if [ -x /usr/libexec/plymouth/plymouth-populate-initrd ]; then
# PLYMOUTH_POPULATE_SOURCE_FUNCTIONS="$TEST_BASE_DIR/test-functions" \
# /usr/libexec/plymouth/plymouth-populate-initrd -t $initdir
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
# image_install plymouth plymouthd
test: remove duplicated code
2013-11-06 06:32:56 +08:00
# else
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
rm -f "${initdir:?}"/{usr/lib,lib,etc}/systemd/system/plymouth* "$initdir"/{usr/lib,lib,etc}/systemd/system/*/plymouth*
test: remove duplicated code
2013-11-06 06:32:56 +08:00
# fi
}
test: if haveged is part of initrd it needs to be installed in the image too Otherwise haveged won't survive when switching root from initrd to host making haveged service in host fail.
2021-08-06 22:47:32 +08:00
install_haveged() {
test: make sure to include all haveged unit files Recent versions of haveged relies on haveged-switch-root.service too.
2021-08-31 21:49:43 +08:00
# If haveged is installed, it's probably included in initrd and needs to be
test: if haveged is part of initrd it needs to be installed in the image too Otherwise haveged won't survive when switching root from initrd to host making haveged service in host fail.
2021-08-06 22:47:32 +08:00
# installed in the image too.
if [ -x /usr/sbin/haveged ]; then
dinfo "Install haveged files"
inst /usr/sbin/haveged
test: make sure to include all haveged unit files Recent versions of haveged relies on haveged-switch-root.service too.
2021-08-31 21:49:43 +08:00
for u in /usr/lib/systemd/system/haveged*; do
test: shellcheck-ify test scripts
2021-09-30 02:30:08 +08:00
inst "$u"
test: make sure to include all haveged unit files Recent versions of haveged relies on haveged-switch-root.service too.
2021-08-31 21:49:43 +08:00
done
test: if haveged is part of initrd it needs to be installed in the image too Otherwise haveged won't survive when switching root from initrd to host making haveged service in host fail.
2021-08-06 22:47:32 +08:00
fi
}
test: remove duplicated code
2013-11-06 06:32:56 +08:00
install_ld_so_conf() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install /etc/ld.so.conf*"
cp -a /etc/ld.so.conf* "${initdir:?}/etc"
test: remove duplicated code
2013-11-06 06:32:56 +08:00
ldconfig -r "$initdir"
}
test: move TEST-43-* setup to static files Setup of lingering is dropped. I don't think it's necessary for anything, because the unit is still pulled in from the testsuite service.
2020-03-21 02:05:36 +08:00
install_testuser() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Set up a test user"
test: move TEST-43-* setup to static files Setup of lingering is dropped. I don't think it's necessary for anything, because the unit is still pulled in from the testsuite service.
2020-03-21 02:05:36 +08:00
# create unprivileged user for user manager tests
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
mkdir -p "${initdir:?}/etc/sysusers.d"
cat >"$initdir/etc/sysusers.d/testuser.conf" <<EOF
test: move TEST-43-* setup to static files Setup of lingering is dropped. I don't think it's necessary for anything, because the unit is still pulled in from the testsuite service.
2020-03-21 02:05:36 +08:00
u testuser 4711 "Test User" /home/testuser
EOF
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
mkdir -p "$initdir/home/testuser"
chmod 0700 "$initdir/home/testuser"
chown 4711:4711 "$initdir/home/testuser"
test: move TEST-43-* setup to static files Setup of lingering is dropped. I don't think it's necessary for anything, because the unit is still pulled in from the testsuite service.
2020-03-21 02:05:36 +08:00
}
test: remove duplicated code
2013-11-06 06:32:56 +08:00
install_config_files() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install config files"
test: convert all uses of '|| true' into '|| :' No change in functionality; just use the shorter || :
2019-07-19 08:34:57 +08:00
inst /etc/sysconfig/init || :
test: remove duplicated code
2013-11-06 06:32:56 +08:00
inst /etc/passwd
inst /etc/shadow
test: adapt test-functions for SUSE
2020-07-22 04:14:53 +08:00
inst_any /etc/login.defs /usr/etc/login.defs
test: remove duplicated code
2013-11-06 06:32:56 +08:00
inst /etc/group
inst /etc/shells
test-functions: read /usr/etc/nsswitch.conf if /etc/nsswitch.conf does not exist (#16195) See this for more info why and since when this change is needed: https://build.opensuse.org/request/show/807179.
2020-06-23 13:42:15 +08:00
inst_any /etc/nsswitch.conf /usr/etc/nsswitch.conf
test: convert all uses of '|| true' into '|| :' No change in functionality; just use the shorter || :
2019-07-19 08:34:57 +08:00
inst /etc/pam.conf || :
tests/TEST-50: support the case when /etc/os-release is present We have four legal cases: 1. /usr/lib/os-release exists and /etc/os-release is a symlink to it 2. both exist but /etc/os-release is not a symlink to /usr/lib/os-release 3. only /usr/lib/os-release exists 4. only /etc/os-release exists The generic setup code in test-functions and create-busybox-image didn't handle case 3. The test-specific code in TEST-50 didn't handle 2 (because the general setup code would only install /etc/os-release in the image and grep -f /usr/lib/os-release would not work) and 4 (same reason) and would fail in case 3 in generic setup.
2020-08-27 17:59:45 +08:00
inst_any /etc/os-release /usr/lib/os-release
test: remove duplicated code
2013-11-06 06:32:56 +08:00
inst /etc/localtime
# we want an empty environment
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
: >"${initdir:?}/etc/environment"
: >"$initdir/etc/machine-id"
: >"$initdir/etc/resolv.conf"
test: move TEST-30-ONCLOCKCHANGE setup to static files The two timezone files are now installed in the global setup. I am not too happy about this, but it still seems better than to create a completely separate image just for this.
2019-12-10 20:45:48 +08:00
test: remove duplicated code
2013-11-06 06:32:56 +08:00
# set the hostname
tests: use "H" as the hostname "systemd-testsuite" gets in the way when grepping for "testsuite-*.sh". Also, the name doesn't matter for anything, so let's just use something very short to save space.
2021-05-04 19:54:33 +08:00
echo 'H' >"$initdir/etc/hostname"
test: use systemd.status-unit-format=name in all but one test We want to test the traditional output, but the non-default one is generally much nicer for debugging (and takes less space...).
2020-03-25 20:43:20 +08:00
# let's set up just one image with the traditional verbose output
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [ "${IMAGE_NAME:?}" != "basic" ]; then
mkdir -p "$initdir/etc/systemd/system.conf.d"
echo -e '[Manager]\nStatusUnitFormat=name' >"$initdir/etc/systemd/system.conf.d/status.conf"
test: use systemd.status-unit-format=name in all but one test We want to test the traditional output, but the non-default one is generally much nicer for debugging (and takes less space...).
2020-03-25 20:43:20 +08:00
fi
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
install_basic_tools() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install basic tools"
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
image_install "${BASICTOOLS[@]}"
image_install -o sushell
tests: install ldconfig.real for Debian based distros In Debian based distributions ldconfig is just a shell wrapper around ldconfig.real, for deferring it during package installation.
2015-02-02 07:19:33 +08:00
# in Debian ldconfig is just a shell script wrapper around ldconfig.real
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
image_install -o ldconfig.real
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
install_debug_tools() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install debug tools"
test: make the installation of the debug tools optional in the image
2021-09-22 14:37:34 +08:00
image_install -o "${DEBUGTOOLS[@]}"
test: introduce INTERACTIVE_DEBUG Make the interactive debugging of (particularly QEMU) machines less painful, by replacing the default vt220 TERM with linux one, and by not shutting down the machine after running the test itself.
2019-03-17 00:46:08 +08:00
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$INTERACTIVE_DEBUG"; then
test: introduce INTERACTIVE_DEBUG Make the interactive debugging of (particularly QEMU) machines less painful, by replacing the default vt220 TERM with linux one, and by not shutting down the machine after running the test itself.
2019-03-17 00:46:08 +08:00
# Set default TERM from vt220 to linux, so at least basic key shortcuts work
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local getty_override="${initdir:?}/etc/systemd/system/serial-getty@.service.d"
mkdir -p "$getty_override"
echo -e "[Service]\nEnvironment=TERM=linux" >"$getty_override/default-TERM.conf"
test: introduce INTERACTIVE_DEBUG Make the interactive debugging of (particularly QEMU) machines less painful, by replacing the default vt220 TERM with linux one, and by not shutting down the machine after running the test itself.
2019-03-17 00:46:08 +08:00
TEST-*: use spacing before redirection operator, but not after << EOF → <<EOF > foo < bar → >foo <bar
2021-04-08 06:09:55 +08:00
cat >"$initdir/etc/motd" <<EOF
test: introduce INTERACTIVE_DEBUG Make the interactive debugging of (particularly QEMU) machines less painful, by replacing the default vt220 TERM with linux one, and by not shutting down the machine after running the test itself.
2019-03-17 00:46:08 +08:00
To adjust the terminal size use:
export COLUMNS=xx
export LINES=yy
or
stty cols xx rows yy
EOF
fi
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
install_libnss() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install libnss"
test: remove duplicated code
2013-11-06 06:32:56 +08:00
# install libnss_files for login
test: use arrays when applicable
2021-04-17 00:40:30 +08:00
local NSS_LIBS
mapfile -t NSS_LIBS < <(LD_DEBUG=files getent passwd 2>&1 >/dev/null | sed -n '/calling init: .*libnss_/ {s!^.* /!/!; p}')
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
image_install "${NSS_LIBS[@]}"
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
install_dbus() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install dbus"
inst "${ROOTLIBDIR:?}/system/dbus.socket"
tests: dbus.service was renamed to dbus-daemon.service in Rawhide Fedora Rawhide renamed dbus.service to dbus-daemon.service - that breaks tests which require working DBus (e.g. TEST-03-JOBS) Excerpt from the dbus.spec: The 'dbus' package is only retained for compatibility purposes. It will eventually be removed and then replaced by 'Provides: dbus' in the dbus-daemon package. It will then exclusively be used for other packages to describe their dependency on a system and user bus. It does not pull in any particular dbus *implementation*, nor any libraries. These should be pulled in, if required, via explicit dependencies.
2018-11-11 19:13:48 +08:00
core: add test case for PrivateUsers=true in user manager The test exercises that PrivateTmp=yes and ProtectHome={read-only,tmpfs} directives work as expected when PrivateUsers=yes in a user manager. Some code is also added to test-functions to help set up test cases that exercise the user manager.
2019-11-14 02:32:24 +08:00
# Newer Fedora versions use dbus-broker by default. Let's install it if it's available.
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [ -f "$ROOTLIBDIR/system/dbus-broker.service" ]; then
inst "$ROOTLIBDIR/system/dbus-broker.service"
test-functions: follow fedora dbus-daemon/dbus-broker rearrangements Apparently dbus.service is now a symlink to either dbus-broker.service or dbus-daemon.service. Copy in the right target service in each case.
2019-04-29 22:09:36 +08:00
inst_symlink /etc/systemd/system/dbus.service
inst /usr/bin/dbus-broker
inst /usr/bin/dbus-broker-launch
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
elif [ -f "$ROOTLIBDIR/system/dbus-daemon.service" ]; then
test-functions: follow fedora dbus-daemon/dbus-broker rearrangements Apparently dbus.service is now a symlink to either dbus-broker.service or dbus-daemon.service. Copy in the right target service in each case.
2019-04-29 22:09:36 +08:00
# Fedora rawhide replaced dbus.service with dbus-daemon.service
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
inst "$ROOTLIBDIR/system/dbus-daemon.service"
tests: dbus.service was renamed to dbus-daemon.service in Rawhide Fedora Rawhide renamed dbus.service to dbus-daemon.service - that breaks tests which require working DBus (e.g. TEST-03-JOBS) Excerpt from the dbus.spec: The 'dbus' package is only retained for compatibility purposes. It will eventually be removed and then replaced by 'Provides: dbus' in the dbus-daemon package. It will then exclusively be used for other packages to describe their dependency on a system and user bus. It does not pull in any particular dbus *implementation*, nor any libraries. These should be pulled in, if required, via explicit dependencies.
2018-11-11 19:13:48 +08:00
# Alias symlink
inst_symlink /etc/systemd/system/dbus.service
else
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
inst "$ROOTLIBDIR/system/dbus.service"
tests: dbus.service was renamed to dbus-daemon.service in Rawhide Fedora Rawhide renamed dbus.service to dbus-daemon.service - that breaks tests which require working DBus (e.g. TEST-03-JOBS) Excerpt from the dbus.spec: The 'dbus' package is only retained for compatibility purposes. It will eventually be removed and then replaced by 'Provides: dbus' in the dbus-daemon package. It will then exclusively be used for other packages to describe their dependency on a system and user bus. It does not pull in any particular dbus *implementation*, nor any libraries. These should be pulled in, if required, via explicit dependencies.
2018-11-11 19:13:48 +08:00
fi
test: remove duplicated code
2013-11-06 06:32:56 +08:00
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
while read -r file; do
inst "$file"
done < <(find /etc/dbus-1 /usr/share/dbus-1 -xtype f 2>/dev/null)
test: move TEST-42-EXECSTOPPOST setup to static files
2019-12-10 18:49:39 +08:00
# setup policy for Type=dbus test
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
mkdir -p "${initdir:?}/etc/dbus-1/system.d"
cat >"$initdir/etc/dbus-1/system.d/systemd.test.ExecStopPost.conf" <<EOF
test: move TEST-42-EXECSTOPPOST setup to static files
2019-12-10 18:49:39 +08:00
<?xml version="1.0"?>
<!DOCTYPE busconfig PUBLIC "-//freedesktop//DTD D-BUS Bus Configuration 1.0//EN"
"http://www.freedesktop.org/standards/dbus/1.0/busconfig.dtd">
<busconfig>
<policy user="root">
<allow own="systemd.test.ExecStopPost"/>
</policy>
</busconfig>
EOF
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
core: add test case for PrivateUsers=true in user manager The test exercises that PrivateTmp=yes and ProtectHome={read-only,tmpfs} directives work as expected when PrivateUsers=yes in a user manager. Some code is also added to test-functions to help set up test cases that exercise the user manager.
2019-11-14 02:32:24 +08:00
install_user_dbus() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install user dbus"
test: use pkg-config to get user unit dir when installing dbus user socket Usually on Debian ROOTLIBDIR is /lib/<arch triplet>, which is not the right place. Use pkg-config since we define it, and then fallback to /usr/lib/systemd/user which is the canonical location. On both Debian&friends and Fedora dbus/dbus-broker install the user socket/service under /usr/lib/systemd/user, not /lib/systemd/systemd/user.
2020-12-23 01:47:39 +08:00
local userunitdir
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if ! userunitdir="$(pkg-config --variable=systemduserunitdir systemd)"; then
dwarn "WARNING! Cannot determine userunitdir from pkg-config, assuming /usr/lib/systemd/user"
userunitdir=/usr/lib/systemd/user
test: use pkg-config to get user unit dir when installing dbus user socket Usually on Debian ROOTLIBDIR is /lib/<arch triplet>, which is not the right place. Use pkg-config since we define it, and then fallback to /usr/lib/systemd/user which is the canonical location. On both Debian&friends and Fedora dbus/dbus-broker install the user socket/service under /usr/lib/systemd/user, not /lib/systemd/systemd/user.
2020-12-23 01:47:39 +08:00
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
inst "$userunitdir/dbus.socket"
inst_symlink "$userunitdir/sockets.target.wants/dbus.socket" || inst_symlink /etc/systemd/user/sockets.target.wants/dbus.socket
core: add test case for PrivateUsers=true in user manager The test exercises that PrivateTmp=yes and ProtectHome={read-only,tmpfs} directives work as expected when PrivateUsers=yes in a user manager. Some code is also added to test-functions to help set up test cases that exercise the user manager.
2019-11-14 02:32:24 +08:00
# Append the After= dependency on dbus in case it isn't already set up
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
mkdir -p "${initdir:?}/etc/systemd/system/user@.service.d/"
cat >"$initdir/etc/systemd/system/user@.service.d/dbus.conf" <<EOF
core: add test case for PrivateUsers=true in user manager The test exercises that PrivateTmp=yes and ProtectHome={read-only,tmpfs} directives work as expected when PrivateUsers=yes in a user manager. Some code is also added to test-functions to help set up test cases that exercise the user manager.
2019-11-14 02:32:24 +08:00
[Unit]
After=dbus.service
EOF
# Newer Fedora versions use dbus-broker by default. Let's install it if it's available.
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [ -f "$userunitdir/dbus-broker.service" ]; then
inst "$userunitdir/dbus-broker.service"
core: add test case for PrivateUsers=true in user manager The test exercises that PrivateTmp=yes and ProtectHome={read-only,tmpfs} directives work as expected when PrivateUsers=yes in a user manager. Some code is also added to test-functions to help set up test cases that exercise the user manager.
2019-11-14 02:32:24 +08:00
inst_symlink /etc/systemd/user/dbus.service
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
elif [ -f "${ROOTLIBDIR:?}/system/dbus-daemon.service" ]; then
core: add test case for PrivateUsers=true in user manager The test exercises that PrivateTmp=yes and ProtectHome={read-only,tmpfs} directives work as expected when PrivateUsers=yes in a user manager. Some code is also added to test-functions to help set up test cases that exercise the user manager.
2019-11-14 02:32:24 +08:00
# Fedora rawhide replaced dbus.service with dbus-daemon.service
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
inst "$userunitdir/dbus-daemon.service"
core: add test case for PrivateUsers=true in user manager The test exercises that PrivateTmp=yes and ProtectHome={read-only,tmpfs} directives work as expected when PrivateUsers=yes in a user manager. Some code is also added to test-functions to help set up test cases that exercise the user manager.
2019-11-14 02:32:24 +08:00
# Alias symlink
inst_symlink /etc/systemd/user/dbus.service
else
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
inst "$userunitdir/dbus.service"
core: add test case for PrivateUsers=true in user manager The test exercises that PrivateTmp=yes and ProtectHome={read-only,tmpfs} directives work as expected when PrivateUsers=yes in a user manager. Some code is also added to test-functions to help set up test cases that exercise the user manager.
2019-11-14 02:32:24 +08:00
fi
}
test: remove duplicated code
2013-11-06 06:32:56 +08:00
install_pam() {
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
dinfo "Install PAM"
local paths=()
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$LOOKS_LIKE_DEBIAN" && type -p dpkg-architecture &>/dev/null; then
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
paths+=("/lib/$(dpkg-architecture -qDEB_HOST_MULTIARCH)/security")
test: Run qemu/nspawn tests with "set -e" This catches errors like "ninja not found", missing programs etc. early, instead of silently ignoring them and trying to boot a broken VM. In install_config_files(), allow some distro specific files to be absent (such as /etc/sysconfig/init).
2017-08-08 03:09:21 +08:00
else
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
paths+=(/lib*/security)
test: Run qemu/nspawn tests with "set -e" This catches errors like "ninja not found", missing programs etc. early, instead of silently ignoring them and trying to boot a broken VM. In install_config_files(), allow some distro specific files to be absent (such as /etc/sysconfig/init).
2017-08-08 03:09:21 +08:00
fi
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
test: adapt install_pam() for openSUSE On openSUSE the default pam config files are shipped in /usr/etc/pam.d. Also empty password is not allowed by default.
2021-08-05 21:34:37 +08:00
for d in /etc/pam.d /etc/security /usr/{etc,lib}/pam.d; do
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
[ -d "$d" ] && paths+=("$d")
test: remove duplicated code
2013-11-06 06:32:56 +08:00
done
tests: copy /etc/pam.d/systemd-user from the host on Debian, Ubuntu Fixes: systemd-testsuite systemd[34]: PAM _pam_load_conf_file: unable to open /etc/pam.d/system-auth systemd-testsuite systemd[34]: PAM _pam_load_conf_file: unable to open /etc/pam.d/system-auth systemd-testsuite systemd[34]: user@0.service: Failed at step PAM spawning /lib/systemd/systemd: Operation not permitted ... on Debian, Ubuntu
2015-11-17 07:58:20 +08:00
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
while read -r file; do
inst "$file"
done < <(find "${paths[@]}" -xtype f)
tests: fix install_pam * pam_unix depends on unix_chkpwd * selinux=1 -> pam_unix runs unix_chkpwd for root too https://git.fedorahosted.org/cgit/linux-pam.git/tree/modules/pam_unix/passverify.c?id=e89d4c97385ff8180e6e81e84c5aa745daf28a79#n202
2016-01-25 13:00:34 +08:00
# pam_unix depends on unix_chkpwd.
# see http://www.linux-pam.org/Linux-PAM-html/sag-pam_unix.html
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
image_install -o unix_chkpwd
tests: fix install_pam * pam_unix depends on unix_chkpwd * selinux=1 -> pam_unix runs unix_chkpwd for root too https://git.fedorahosted.org/cgit/linux-pam.git/tree/modules/pam_unix/passverify.c?id=e89d4c97385ff8180e6e81e84c5aa745daf28a79#n202
2016-01-25 13:00:34 +08:00
tests: allow root to login with empty password to test images
2016-04-26 02:03:27 +08:00
# set empty root password for easy debugging
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
sed -i 's/^root:x:/root::/' "${initdir:?}/etc/passwd"
test: adapt install_pam() for openSUSE On openSUSE the default pam config files are shipped in /usr/etc/pam.d. Also empty password is not allowed by default.
2021-08-05 21:34:37 +08:00
# And make sure pam_unix will accept it by making sure that
# the PAM module has the nullok option.
for d in /etc/pam.d /usr/{etc,lib}/pam.d; do
[ -d "$initdir/$d" ] || continue
sed -i '/^auth.*pam_unix.so/s/$/ nullok/' "$initdir/$d"/*
done
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
test: shellcheck-ify test scripts
2021-09-30 02:30:08 +08:00
# shellcheck disable=SC2120
test: remove duplicated code
2013-11-06 06:32:56 +08:00
install_keymaps() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install keymaps"
test: add paths of keymaps in install_keymaps() It seems that the paths of directories storing keymaps are changed.
2018-09-19 09:54:16 +08:00
# The first three paths may be deprecated.
# It seems now the last two paths are used by many distributions.
test: remove duplicated code
2013-11-06 06:32:56 +08:00
for i in \
/usr/lib/kbd/keymaps/include/* \
/usr/lib/kbd/keymaps/i386/include/* \
test: add paths of keymaps in install_keymaps() It seems that the paths of directories storing keymaps are changed.
2018-09-19 09:54:16 +08:00
/usr/lib/kbd/keymaps/i386/qwerty/us.* \
/usr/lib/kbd/keymaps/legacy/include/* \
/usr/lib/kbd/keymaps/legacy/i386/qwerty/us.*; do
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -f "$i" ]] || continue
inst "$i"
test: remove duplicated code
2013-11-06 06:32:56 +08:00
done
test: make install_keymaps() optionally install more keymaps
2018-09-19 09:54:28 +08:00
# When it takes any argument, then install more keymaps.
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [[ $# -gt 1 ]]; then
test: make install_keymaps() optionally install more keymaps
2018-09-19 09:54:28 +08:00
for i in \
/usr/lib/kbd/keymaps/i386/*/* \
/usr/lib/kbd/keymaps/legacy/i386/*/*; do
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -f "$i" ]] || continue
inst "$i"
test: make install_keymaps() optionally install more keymaps
2018-09-19 09:54:28 +08:00
done
fi
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
test: introduce install_zoneinfo() But it is not called by default.
2018-09-12 17:20:31 +08:00
install_zoneinfo() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install time zones"
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
inst_any /usr/share/zoneinfo/Asia/Seoul
inst_any /usr/share/zoneinfo/Asia/Vladivostok
inst_any /usr/share/zoneinfo/Australia/Sydney
inst_any /usr/share/zoneinfo/Europe/Berlin
shared/calendarspec: when mktime() moves us backwards, jump forward When trying to calculate the next firing of 'Sun *-*-* 01:00:00', we'd fall into an infinite loop, because mktime() moves us "backwards": Before this patch: tm_within_bounds: good=0 2021-03-29 01:00:00 → 2021-03-29 00:00:00 tm_within_bounds: good=0 2021-03-29 01:00:00 → 2021-03-29 00:00:00 tm_within_bounds: good=0 2021-03-29 01:00:00 → 2021-03-29 00:00:00 ... We rely on mktime() normalizing the time. The man page does not say that it'll move the time forward, but our algorithm relies on this. So let's catch this case explicitly. With this patch: $ TZ=Europe/Dublin faketime 2021-03-21 build/systemd-analyze calendar --iterations=5 'Sun *-*-* 01:00:00' Normalized form: Sun *-*-* 01:00:00 Next elapse: Sun 2021-03-21 01:00:00 GMT (in UTC): Sun 2021-03-21 01:00:00 UTC From now: 59min left Iter. #2: Sun 2021-04-04 01:00:00 IST (in UTC): Sun 2021-04-04 00:00:00 UTC From now: 1 weeks 6 days left <---- note the 2 week jump here Iter. #3: Sun 2021-04-11 01:00:00 IST (in UTC): Sun 2021-04-11 00:00:00 UTC From now: 2 weeks 6 days left Iter. #4: Sun 2021-04-18 01:00:00 IST (in UTC): Sun 2021-04-18 00:00:00 UTC From now: 3 weeks 6 days left Iter. #5: Sun 2021-04-25 01:00:00 IST (in UTC): Sun 2021-04-25 00:00:00 UTC From now: 1 months 4 days left Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1941335.
2021-03-22 19:51:47 +08:00
inst_any /usr/share/zoneinfo/Europe/Dublin
test: move TEST-24-UNIT-TESTS setup to static files I picked the list of zone files to install by grepping through the code. This is is a bit brittle, but installing all of them takes a while, and more importantly, writes a lot of lines to the log.
2019-12-10 22:31:09 +08:00
inst_any /usr/share/zoneinfo/Europe/Kiev
inst_any /usr/share/zoneinfo/Pacific/Auckland
inst_any /usr/share/zoneinfo/Pacific/Honolulu
inst_any /usr/share/zoneinfo/CET
inst_any /usr/share/zoneinfo/EET
inst_any /usr/share/zoneinfo/UTC
test: introduce install_zoneinfo() But it is not called by default.
2018-09-12 17:20:31 +08:00
}
test: remove duplicated code
2013-11-06 06:32:56 +08:00
install_fonts() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install system fonts"
test: remove duplicated code
2013-11-06 06:32:56 +08:00
for i in \
Use eurlatgr as the example console font See https://fedoraproject.org/wiki/Changes/NewDefaultConsoleFont https://bugzilla.redhat.com/show_bug.cgi?id=1182529
2015-01-15 20:44:17 +08:00
/usr/lib/kbd/consolefonts/eurlatgr* \
test: remove duplicated code
2013-11-06 06:32:56 +08:00
/usr/lib/kbd/consolefonts/latarcyrheb-sun16*; do
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -f "$i" ]] || continue
inst "$i"
test: remove duplicated code
2013-11-06 06:32:56 +08:00
done
}
install_terminfo() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Install terminfo files"
local terminfodir
for terminfodir in /lib/terminfo /etc/terminfo /usr/share/terminfo; do
[ -f "${terminfodir}/l/linux" ] && break
test: remove duplicated code
2013-11-06 06:32:56 +08:00
done
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
image_install -o "${terminfodir}/l/linux"
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
core: add test case for PrivateUsers=true in user manager The test exercises that PrivateTmp=yes and ProtectHome={read-only,tmpfs} directives work as expected when PrivateUsers=yes in a user manager. Some code is also added to test-functions to help set up test cases that exercise the user manager.
2019-11-14 02:32:24 +08:00
has_user_dbus_socket() {
if [ -f /usr/lib/systemd/user/dbus.socket ] || [ -f /etc/systemd/user/dbus.socket ]; then
return 0
else
echo "Per-user instances are not supported. Skipping..."
return 1
fi
}
test: remove duplicated code
2013-11-06 06:32:56 +08:00
setup_nspawn_root() {
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
if [ -z "${initdir}" ]; then
dfatal "\$initdir not defined"
exit 1
fi
test: run tests directly from the loopback device Before, we'd copy the test tree into nspawn-root, and run the tests from there. This is OK, and doesn't actually take much extra time. But it uses quite a lot of extra disk space. So let's make things a bit more efficient by running directly from the image file. We still run the unprivileged nspawn tests from a copy. Once the kernel implements fs shift, we can do away with that too.
2019-12-13 21:21:31 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
rm -rf "${TESTDIR:?}/unprivileged-nspawn-root"
test: perform partial cleanup after each test is run This causes the unprivileged-nspawn-root directory to be removed after running one test. The advantage is that we reduce the maximum disk-space use quite a bit (47*400 MB → about 18GB).
2020-03-21 02:09:35 +08:00
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$RUN_IN_UNPRIVILEGED_CONTAINER"; then
test: run tests directly from the loopback device Before, we'd copy the test tree into nspawn-root, and run the tests from there. This is OK, and doesn't actually take much extra time. But it uses quite a lot of extra disk space. So let's make things a bit more efficient by running directly from the image file. We still run the unprivileged nspawn tests from a copy. Once the kernel implements fs shift, we can do away with that too.
2019-12-13 21:21:31 +08:00
ddebug "cp -ar $initdir $TESTDIR/unprivileged-nspawn-root"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
cp -ar "$initdir" "$TESTDIR/unprivileged-nspawn-root"
tests: also run TEST-01-BASIC in an unprivileged container (#9957) This should make it much easier to catch regressions like https://github.com/systemd/systemd/issues/9914 and https://github.com/systemd/systemd/issues/8535.
2018-08-30 12:01:18 +08:00
fi
test: remove duplicated code
2013-11-06 06:32:56 +08:00
}
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
setup_basic_dirs() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
mkdir -p "${initdir:?}/run"
mkdir -p "$initdir/etc/systemd/system"
mkdir -p "$initdir/var/log/journal"
test: remove duplicated code
2013-11-06 06:32:56 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
for d in usr/bin usr/sbin bin etc lib "${libdir:?}" sbin tmp usr var var/log var/tmp dev proc sys sysroot root run run/lock run/initramfs; do
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
if [ -L "/$d" ]; then
inst_symlink "/$d"
else
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
inst_dir "/$d"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
fi
done
ln -sfn /run "$initdir/var/run"
ln -sfn /run/lock "$initdir/var/lock"
}
test: add function to reduce copied setup boilerplate Many tests were also masking systemd-machined.service. But machined should only start when activated, so having it not masked shouldn't be noticable. TEST-25-IMPORT needs it.
2019-10-08 15:10:12 +08:00
mask_supporting_services() {
# mask some services that we do not want to run in these tests
test: don't mask "supporting" services in TEST-01-BASIC This got lost during one of the code de-duplication attempts.
2021-05-01 03:02:41 +08:00
ln -fsv /dev/null "${initdir:?}/etc/systemd/system/systemd-hwdb-update.service"
ln -fsv /dev/null "$initdir/etc/systemd/system/systemd-journal-catalog-update.service"
ln -fsv /dev/null "$initdir/etc/systemd/system/systemd-networkd.service"
ln -fsv /dev/null "$initdir/etc/systemd/system/systemd-networkd.socket"
ln -fsv /dev/null "$initdir/etc/systemd/system/systemd-resolved.service"
test: add function to reduce copied setup boilerplate Many tests were also masking systemd-machined.service. But machined should only start when activated, so having it not masked shouldn't be noticable. TEST-25-IMPORT needs it.
2019-10-08 15:10:12 +08:00
}
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
inst_libs() {
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
local bin="${1:?}"
local so_regex='([^ ]*/lib[^/]*/[^ ]*\.so[^ ]*)'
local file line
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
while read -r line; do
[[ "$line" = 'not a dynamic executable' ]] && break
test: skip a harmless ldd error when installing test libs Skip a harmless error when running the tests on a system with a significantly older systemd version (ldd tries to resolve the unprefixed RPATH for libsystemd.so.0, which is in this case older than the already installed libsystemd.so.0 in $initdir). The issue is triggered by installing test dependencies in install_missing_libraries(). Spotted on CentOS 8. ``` $ ldd /var/tmp/systemd-test.nZO11F/root/lib/systemd/tests/test-sd-device-thread /var/tmp/systemd-test.nZO11F/root/lib/systemd/tests/test-sd-device-thread: /lib64/libsystemd.so.0: version `LIBSYSTEMD_240' not found (required by /var/tmp/systemd-test.nZO11F/root/lib/systemd/tests/test-sd-device-thread) linux-vdso64.so.1 (0x00007fffb79d0000) libclang_rt.asan-powerpc64le.so => /usr/lib64/clang/11.0.0/lib/linux/libclang_rt.asan-powerpc64le.so (0x00007fffb6ef0000) libsystemd.so.0 => /lib64/libsystemd.so.0 (0x00007fffb6d20000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fffb6cd0000) libc.so.6 => /lib64/libc.so.6 (0x00007fffb6ab0000) $ LD_LIBRARY_PATH=/var/tmp/systemd-test.nZO11F/root/lib64/ ldd /var/tmp/systemd-test.nZO11F/root/lib/systemd/tests/test-sd-device-thread linux-vdso64.so.1 (0x00007fffaba80000) libclang_rt.asan-powerpc64le.so => /usr/lib64/clang/11.0.0/lib/linux/libclang_rt.asan-powerpc64le.so (0x00007fffaafa0000) libsystemd.so.0 => /var/tmp/systemd-test.nZO11F/root/lib64/libsystemd.so.0 (0x00007fffaa5f0000) libpthread.so.0 => /var/tmp/systemd-test.nZO11F/root/lib64/libpthread.so.0 (0x00007fffaa5a0000) libc.so.6 => /var/tmp/systemd-test.nZO11F/root/lib64/libc.so.6 (0x00007fffaa380000) ```
2021-08-09 05:47:31 +08:00
# Skip a harmless error when running the tests on a system with a significantly
# older systemd version (ldd tries to resolve the unprefixed RPATH for libsystemd.so.0,
# which is in this case older than the already installed libsystemd.so.0 in $initdir).
# The issue is triggered by installing test dependencies in install_missing_libraries().
[[ "$line" =~ libsystemd.so.*:\ version\ .*\ not\ found ]] && continue
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
if [[ "$line" =~ $so_regex ]]; then
file="${BASH_REMATCH[1]}"
[[ -e "${initdir:?}/$file" ]] && continue
inst_library "$file"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
continue
fi
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
if [[ "$line" =~ not\ found ]]; then
dfatal "Missing a shared library required by $bin."
dfatal "Run \"ldd $bin\" to find out what it is."
dfatal "$line"
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
dfatal "Cannot create a test image."
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
exit 1
fi
test: use set -o pipefail This breaks some existing loops which previously ignored if the piped program exited with EC >0. Rewrite them to mitigate this (and also make them more robust in some cases).
2021-04-17 00:36:27 +08:00
done < <(LC_ALL=C ldd "$bin" 2>/dev/null)
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
}
import_testdir() {
test: stop caching loopdev It is more trouble than it is worth. The setup is of a loopback device is very quick, so it's better to always create it when needed and immediately drop afterwards.
2020-03-21 03:17:11 +08:00
# make sure we don't get a stale LOOPDEV value from old times
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local _LOOPDEV="${LOOPDEV:=}"
# We don't want shellcheck to follow & check the $STATEFILE
# shellcheck source=/dev/null
[[ -e "$STATEFILE" ]] && . "$STATEFILE"
LOOPDEV="$_LOOPDEV"
test: don't overwrite TESTDIR if already set
2019-03-13 05:44:25 +08:00
if [[ ! -d "$TESTDIR" ]]; then
if [[ -z "$TESTDIR" ]]; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
TESTDIR="$(mktemp --tmpdir=/var/tmp -d -t systemd-test.XXXXXX)"
test: don't overwrite TESTDIR if already set
2019-03-13 05:44:25 +08:00
else
mkdir -p "$TESTDIR"
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
cat >"$STATEFILE" <<EOF
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
TESTDIR="$TESTDIR"
EOF
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
export TESTDIR
fi
test: fix public/private image detection and prefer the latter one
2020-05-17 16:46:34 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
IMAGE_PRIVATE="${TESTDIR}/${IMAGE_NAME:?}.img"
IMAGE_PUBLIC="${IMAGESTATEDIR:?}/${IMAGE_NAME}.img"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
}
test: remove duplicated code
2013-11-06 06:32:56 +08:00
import_initdir() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
initdir="${TESTDIR:?}/root"
mkdir -p "$initdir"
test: remove duplicated code
2013-11-06 06:32:56 +08:00
export initdir
}
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
## @brief Converts numeric logging level to the first letter of level name.
#
# @param lvl Numeric logging level in range from 1 to 6.
# @retval 1 if @a lvl is out of range.
# @retval 0 if @a lvl is correct.
# @result Echoes first letter of level name.
_lvl2char() {
case "$1" in
1) echo F;;
2) echo E;;
3) echo W;;
4) echo I;;
5) echo D;;
6) echo T;;
*) return 1;;
esac
}
## @brief Internal helper function for _do_dlog()
#
# @param lvl Numeric logging level.
# @param msg Message.
# @retval 0 It's always returned, even if logging failed.
#
# @note This function is not supposed to be called manually. Please use
# dtrace(), ddebug(), or others instead which wrap this one.
#
# This function calls _do_dlog() either with parameter msg, or if
# none is given, it will read standard input and will use every line as
# a message.
#
# This enables:
# dwarn "This is a warning"
# echo "This is a warning" | dwarn
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
LOG_LEVEL="${LOG_LEVEL:-4}"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
dlog() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local lvl lvlc
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
[ -z "$LOG_LEVEL" ] && return 0
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
lvl="${1:?}"; shift
[ "$lvl" -le "$LOG_LEVEL" ] || return 0
lvlc="$(_lvl2char "$lvl")" || return 0
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
if [ $# -ge 1 ]; then
echo "$lvlc: $*"
else
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
while read -r line; do
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
echo "$lvlc: " "$line"
done
fi
}
## @brief Logs message at TRACE level (6)
#
# @param msg Message.
# @retval 0 It's always returned, even if logging failed.
dtrace() {
set +x
dlog 6 "$@"
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "${debug:=}"; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
set -x
fi
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
}
## @brief Logs message at DEBUG level (5)
#
# @param msg Message.
# @retval 0 It's always returned, even if logging failed.
ddebug() {
dlog 5 "$@"
}
## @brief Logs message at INFO level (4)
#
# @param msg Message.
# @retval 0 It's always returned, even if logging failed.
dinfo() {
set +x
dlog 4 "$@"
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "${debug:=}"; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
set -x
fi
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
}
## @brief Logs message at WARN level (3)
#
# @param msg Message.
# @retval 0 It's always returned, even if logging failed.
dwarn() {
set +x
dlog 3 "$@"
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "${debug:=}"; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
set -x
fi
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
}
## @brief Logs message at ERROR level (2)
#
# @param msg Message.
# @retval 0 It's always returned, even if logging failed.
derror() {
dlog 2 "$@"
}
## @brief Logs message at FATAL level (1)
#
# @param msg Message.
# @retval 0 It's always returned, even if logging failed.
dfatal() {
set +x
dlog 1 "$@"
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "${debug:=}"; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
set -x
fi
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
}
# Generic substring function. If $2 is in $1, return 0.
test: fix a couple of "new" shellcheck-related issues related to https://github.com/koalaman/shellcheck/wiki/SC2295
2021-11-23 03:51:15 +08:00
strstr() { [ "${1#*"$2"*}" != "$1" ]; }
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
# normalize_path <path>
# Prints the normalized path, where it removes any duplicated
# and trailing slashes.
# Example:
# $ normalize_path ///test/test//
# /test/test
normalize_path() {
shopt -q -s extglob
set -- "${1//+(\/)//}"
shopt -q -u extglob
echo "${1%/}"
}
# convert_abs_rel <from> <to>
# Prints the relative path, when creating a symlink to <to> from <from>.
# Example:
# $ convert_abs_rel /usr/bin/test /bin/test-2
# ../../bin/test-2
# $ ln -s $(convert_abs_rel /usr/bin/test /bin/test-2) /usr/bin/test
convert_abs_rel() {
local __current __absolute __abssize __cursize __newpath
local -i __i __level
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
set -- "$(normalize_path "${1:?}")" "$(normalize_path "${2:?}")"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
# corner case #1 - self looping link
[[ "$1" == "$2" ]] && { echo "${1##*/}"; return; }
# corner case #2 - own dir link
[[ "${1%/*}" == "$2" ]] && { echo "."; return; }
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
IFS="/" read -ra __current <<< "$1"
IFS="/" read -ra __absolute <<< "$2"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
__abssize=${#__absolute[@]}
__cursize=${#__current[@]}
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
while [[ "${__absolute[__level]}" == "${__current[__level]}" ]]
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
do
(( __level++ ))
if (( __level > __abssize || __level > __cursize ))
then
break
fi
done
for ((__i = __level; __i < __cursize-1; __i++))
do
if ((__i > __level))
then
__newpath=$__newpath"/"
fi
__newpath=$__newpath".."
done
for ((__i = __level; __i < __abssize; __i++))
do
if [[ -n $__newpath ]]
then
__newpath=$__newpath"/"
fi
__newpath=$__newpath${__absolute[__i]}
done
echo "$__newpath"
}
# Install a directory, keeping symlinks as on the original system.
# Example: if /lib points to /lib64 on the host, "inst_dir /lib/file"
# will create ${initdir}/lib64, ${initdir}/lib64/file,
# and a symlink ${initdir}/lib -> lib64.
inst_dir() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local dir="${1:?}"
local part="${dir%/*}"
local file
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -e "${initdir:?}/${dir}" ]] && return 0 # already there
while [[ "$part" != "${part%/*}" ]] && ! [[ -e "${initdir}/${part}" ]]; do
dir="$part $dir"
part="${part%/*}"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
done
# iterate over parent directories
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
for file in $dir; do
[[ -e "${initdir}/$file" ]] && continue
if [[ -L $file ]]; then
inst_symlink "$file"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
else
# create directory
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
mkdir -m 0755 "${initdir}/$file" || return 1
[[ -e "$file" ]] && chmod --reference="$file" "${initdir}/$file"
chmod u+w "${initdir}/$file"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
fi
done
}
# $1 = file to copy to ramdisk
# $2 (optional) Name for the file on the ramdisk
# Location of the image dir is assumed to be $initdir
# We never overwrite the target if it exists.
inst_simple() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -f "${1:?}" ]] || return 1
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
strstr "$1" "/" || return 1
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local src="$1"
local target="${2:-$1}"
if ! [[ -d ${initdir:?}/$target ]]; then
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
[[ -e ${initdir}/$target ]] && return 0
[[ -L ${initdir}/$target ]] && return 0
[[ -d "${initdir}/${target%/*}" ]] || inst_dir "${target%/*}"
fi
# install checksum files also
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [[ -e "${src%/*}/.${src##*/}.hmac" ]]; then
inst "${src%/*}/.${src##*/}.hmac" "${target%/*}/.${target##*/}.hmac"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
ddebug "Installing $src"
cp --sparse=always -pfL "$src" "${initdir}/$target"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
}
# find symlinks linked to given library file
# $1 = library file
# Function searches for symlinks by stripping version numbers appended to
# library filename, checks if it points to the same target and finally
# prints the list of symlinks to stdout.
#
# Example:
# rev_lib_symlinks libfoo.so.8.1
# output: libfoo.so.8 libfoo.so
# (Only if libfoo.so.8 and libfoo.so exists on host system.)
rev_lib_symlinks() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local fn="${1:?}"
local links=""
local orig
orig="$(readlink -f "$1")"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ "${fn}" =~ .*\.so\..* ]] || return 1
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
until [[ "${fn##*.}" == so ]]; do
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
fn="${fn%.*}"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -L "${fn}" && "$(readlink -f "${fn}")" == "${orig}" ]] && links+=" ${fn}"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
done
echo "${links}"
}
# Same as above, but specialized to handle dynamic libraries.
# It handles making symlinks according to how the original library
# is referenced.
inst_library() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local src="${1:?}"
local dest="${2:-$1}"
local reallib symlink
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
strstr "$1" "/" || return 1
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -e ${initdir:?}/$dest ]] && return 0
if [[ -L $src ]]; then
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
# install checksum files also
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [[ -e "${src%/*}/.${src##*/}.hmac" ]]; then
inst "${src%/*}/.${src##*/}.hmac" "${dest%/*}/.${dest##*/}.hmac"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
reallib="$(readlink -f "$src")"
inst_simple "$reallib" "$reallib"
inst_dir "${dest%/*}"
[[ -d "${dest%/*}" ]] && dest="$(readlink -f "${dest%/*}")/${dest##*/}"
ln -sfn -- "$(convert_abs_rel "${dest}" "${reallib}")" "${initdir}/${dest}"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
else
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
inst_simple "$src" "$dest"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
fi
# Create additional symlinks. See rev_symlinks description.
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
for symlink in $(rev_lib_symlinks "$src") ${reallib:+$(rev_lib_symlinks "$reallib")}; do
if [[ ! -e "$initdir/$symlink" ]]; then
ddebug "Creating extra symlink: $symlink"
inst_symlink "$symlink"
fi
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
done
}
# find a binary. If we were not passed the full path directly,
# search in the usual places to find the binary.
find_binary() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local bin="${1:?}"
if [[ -z ${bin##/*} ]]; then
if [[ -x "$bin" ]] || { strstr "$bin" ".so" && ldd "$bin" &>/dev/null; }; then
echo "$bin"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
return 0
fi
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
type -P "$bin"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
}
# Same as above, but specialized to install binary executables.
# Install binary executable, and all shared library dependencies, if any.
inst_binary() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local bin="${1:?}"
local path target
test: correctly handle installation of newly introduced binaries In certain cases we might attempt to install a binary which is already present in the test image, yet it's missing from the host system. In such cases, let's check if the binary indeed exists in the image before doing any other chcecks. If it does, immediately return with success. This was discovered during installation of /usr/lib/systemd/systemd-bless-boot, which was not present in Ubuntu CI (as the installed systemd was from the Ubuntu repositories), and the binary itself was already in the image thanks to `ninja install`. However, during extraction of binaries from the systemd service files, another attempt to install this binary was made, which failed due to `find_binary` being unable to find it.
2019-07-09 03:05:51 +08:00
# In certain cases we might attempt to install a binary which is already
# present in the test image, yet it's missing from the host system.
# In such cases, let's check if the binary indeed exists in the image
# before doing any other chcecks. If it does, immediately return with
# success.
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [[ $# -eq 1 ]]; then
for path in "" bin sbin usr/bin usr/sbin; do
[[ -e "${initdir:?}${path:+/$path}/${bin}" ]] && return 0
done
fi
test: correctly handle installation of newly introduced binaries In certain cases we might attempt to install a binary which is already present in the test image, yet it's missing from the host system. In such cases, let's check if the binary indeed exists in the image before doing any other chcecks. If it does, immediately return with success. This was discovered during installation of /usr/lib/systemd/systemd-bless-boot, which was not present in Ubuntu CI (as the installed systemd was from the Ubuntu repositories), and the binary itself was already in the image thanks to `ninja install`. However, during extraction of binaries from the systemd service files, another attempt to install this binary was made, which failed due to `find_binary` being unable to find it.
2019-07-09 03:05:51 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
bin="$(find_binary "$bin")" || return 1
target="${2:-$bin}"
[[ -e "${initdir:?}/$target" ]] && return 0
[[ -L "$bin" ]] && inst_symlink "$bin" "$target" && return 0
local file line
local so_regex='([^ ]*/lib[^/]*/[^ ]*\.so[^ ]*)'
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
# I love bash!
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
while read -r line; do
[[ "$line" = 'not a dynamic executable' ]] && break
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [[ "$line" =~ $so_regex ]]; then
file="${BASH_REMATCH[1]}"
[[ -e "${initdir}/$file" ]] && continue
inst_library "$file"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
continue
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [[ "$line" =~ not\ found ]]; then
dfatal "Missing a shared library required by $bin."
dfatal "Run \"ldd $bin\" to find out what it is."
dfatal "$line"
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
dfatal "Cannot create a test image."
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
exit 1
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
done < <(LC_ALL=C ldd "$bin" 2>/dev/null)
inst_simple "$bin" "$target"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
}
# same as above, except for shell scripts.
# If your shell script does not start with shebang, it is not a shell script.
inst_script() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local bin line shebang_regex
bin="$(find_binary "${1:?}")" || return 1
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
shift
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
read -r -n 80 line <"$bin"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
# If debug is set, clean unprintable chars to prevent messing up the term
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
get_bool "${debug:=}" && line="$(echo -n "$line" | tr -c -d '[:print:][:space:]')"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
shebang_regex='(#! *)(/[^ ]+).*'
[[ "$line" =~ $shebang_regex ]] || return 1
inst "${BASH_REMATCH[2]}" && inst_simple "$bin" "$@"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
}
# same as above, but specialized for symlinks
inst_symlink() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local src="${1:?}"
local target="${2:-$src}"
local realsrc
strstr "$src" "/" || return 1
[[ -L "$src" ]] || return 1
[[ -L "${initdir:?}/$target" ]] && return 0
realsrc="$(readlink -f "$src")"
if ! [[ -e "$initdir/$realsrc" ]]; then
if [[ -d "$realsrc" ]]; then
inst_dir "$realsrc"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
else
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
inst "$realsrc"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
fi
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ ! -e "$initdir/${target%/*}" ]] && inst_dir "${target%/*}"
[[ -d "${target%/*}" ]] && target="$(readlink -f "${target%/*}")/${target##*/}"
ln -sfn -- "$(convert_abs_rel "${target}" "${realsrc}")" "$initdir/$target"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
}
# attempt to install any programs specified in a udev rule
inst_rule_programs() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local rule="${1:?}"
local prog bin
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test: drop the mawk-incompatible expression The three-argument match() is a GNU AWK extension, thus breaking the compatibility with mawk (used on Ubuntu/Debian, for example). Let's replace it with a (hopefully) more portable sed expression to drop the inadvertently introduced gawk dependency. Fixes: #19957
2021-06-17 20:38:21 +08:00
sed -rn 's/^.*?PROGRAM==?"([^ "]+).*$/\1/p' "$rule" | while read -r prog; do
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [ -x "/lib/udev/$prog" ]; then
bin="/lib/udev/$prog"
else
if ! bin="$(find_binary "$prog")"; then
dinfo "Skipping program $prog used in udev rule $(basename "$rule") as it cannot be found"
continue
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
fi
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
#dinfo "Installing $_bin due to it's use in the udev rule $(basename $1)"
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
image_install "$bin"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
done
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
}
# udev rules always get installed in the same place, so
# create a function to install them to make life simpler.
inst_rules() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local target=/etc/udev/rules.d
local found rule
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
inst_dir "/lib/udev/rules.d"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
inst_dir "$target"
for rule in "$@"; do
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
if [ "${rule#/}" = "$rule" ]; then
for r in /lib/udev/rules.d /etc/udev/rules.d; do
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [[ -f "$r/$rule" ]]; then
found="$r/$rule"
inst_simple "$found"
inst_rule_programs "$found"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
fi
done
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
for r in '' ./; do
if [[ -f "${r}${rule}" ]]; then
found="${r}${rule}"
inst_simple "$found" "$target/${found##*/}"
inst_rule_programs "$found"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
fi
done
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ $found ]] || dinfo "Skipping udev rule: $rule"
found=
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
done
}
# general purpose installation function
# Same args as above.
inst() {
case $# in
1) ;;
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
2)
[[ ! "$initdir" && -d "$2" ]] && export initdir="$2"
[[ "$initdir" = "$2" ]] && set "$1"
;;
3)
[[ -z "$initdir" ]] && export initdir="$2"
set "$1" "$3"
;;
*)
dfatal "inst only takes 1 or 2 or 3 arguments"
exit 1
;;
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
esac
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local fun
for fun in inst_symlink inst_script inst_binary inst_simple; do
"$fun" "$@" && return 0
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
done
test: explicitly report if we fail to install a file into the image
2021-09-22 16:59:52 +08:00
dwarn "Failed to install '$1'"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
return 1
}
# install any of listed files
#
# If first argument is '-d' and second some destination path, first accessible
# source is installed into this path, otherwise it will installed in the same
# path as source. If none of listed files was installed, function return 1.
# On first successful installation it returns with 0 status.
#
# Example:
#
# inst_any -d /bin/foo /bin/bar /bin/baz
#
# Lets assume that /bin/baz exists, so it will be installed as /bin/foo in
# initramfs.
inst_any() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local dest file
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ "${1:?}" = '-d' ]] && dest="${2:?}" && shift 2
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
for file in "$@"; do
if [[ -e "$file" ]]; then
[[ -n "$dest" ]] && inst "$file" "$dest" && return 0
inst "$file" && return 0
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
fi
done
return 1
}
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
# image_install [-o ] <file> [<file> ... ]
# Install <file> to the test image
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
# -o optionally install the <file> and don't fail, if it is not there
test: rename `dracut_install` to `image_install` The `dracut_install` is a misnomer, since the systemd integration test suite is based on the original dracut's test suite, and not all the references to dracut has been edited out. Let's fix that.
2021-09-08 19:33:34 +08:00
image_install() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local optional=no
local prog="${1:?}"
if [[ "$prog" = '-o' ]]; then
optional=yes
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
shift
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
for prog in "$@"; do
if ! inst "$prog" ; then
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$optional"; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dinfo "Skipping program $prog as it cannot be found and is" \
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
"flagged to be optional"
else
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
dfatal "Failed to install $prog"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
exit 1
fi
fi
done
}
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
# Install a single kernel module along with any firmware it may require.
# $1 = full path to kernel module to install
install_kmod_with_fw() {
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
local module="${1:?}"
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
# no need to go further if the module is already installed
test: fix a couple of "new" shellcheck-related issues related to https://github.com/koalaman/shellcheck/wiki/SC2295
2021-11-23 03:51:15 +08:00
[[ -e "${initdir:?}/lib/modules/${KERNEL_VER:?}/${module##*"/lib/modules/$KERNEL_VER/"}" ]] && return 0
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
[[ -e "$initdir/.kernelmodseen/${module##*/}" ]] && return 0
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
[ -d "$initdir/.kernelmodseen" ] && : >"$initdir/.kernelmodseen/${module##*/}"
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
test: fix a couple of "new" shellcheck-related issues related to https://github.com/koalaman/shellcheck/wiki/SC2295
2021-11-23 03:51:15 +08:00
inst_simple "$module" "/lib/modules/$KERNEL_VER/${module##*"/lib/modules/$KERNEL_VER/"}" || return $?
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
local modname="${module##*/}"
local fwdir found fw
modname="${modname%.ko*}"
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
while read -r fw; do
found=
for fwdir in /lib/firmware/updates /lib/firmware; do
if [[ -d "$fwdir" && -f "$fwdir/$fw" ]]; then
inst_simple "$fwdir/$fw" "/lib/firmware/$fw"
found=yes
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
fi
done
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if ! get_bool "$found"; then
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
if ! grep -qe "\<${modname//-/_}\>" /proc/modules; then
dinfo "Possible missing firmware \"${fw}\" for kernel module" \
"\"${modname}.ko\""
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
else
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
dwarn "Possible missing firmware \"${fw}\" for kernel module" \
"\"${modname}.ko\""
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
fi
fi
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
done < <(modinfo -k "$KERNEL_VER" -F firmware "$module" 2>/dev/null)
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
return 0
}
# Do something with all the dependencies of a kernel module.
# Note that kernel modules depend on themselves using the technique we use
# $1 = function to call for each dependency we find
# It will be passed the full path to the found kernel module
# $2 = module to get dependencies for
# rest of args = arguments to modprobe
for_each_kmod_dep() {
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
local func="${1:?}"
local kmod="${2:?}"
local found=0
local cmd modpath
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
shift 2
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
while read -r cmd modpath _; do
[[ "$cmd" = insmod ]] || continue
"$func" "$modpath" || return $?
found=1
done < <(modprobe "$@" --ignore-install --show-depends "$kmod")
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
! get_bool "$found" && return 1
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
return 0
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
}
# instmods [-c] <kernel module> [<kernel module> ... ]
# instmods [-c] <kernel subsystem>
# install kernel modules along with all their dependencies.
# <kernel subsystem> can be e.g. "=block" or "=drivers/usb/storage"
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
# FIXME(?): dracutdevs/dracut@f4e38c0da8d6bf3764c1ad753d9d52aef63050e5
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
instmods() {
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
local check=no
if [[ $# -ge 0 && "$1" = '-c' ]]; then
check=yes
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
shift
fi
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
inst1mod() {
local mod="${1:?}"
local ret=0
local mod_dir="/lib/modules/${KERNEL_VER:?}/"
case "$mod" in
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
=*)
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
if [ -f "${mod_dir}/modules.${mod#=}" ]; then
(
[[ "$mpargs" ]] && echo "$mpargs"
cat "${mod_dir}/modules.${mod#=}"
) | instmods
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
else
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
(
[[ "$mpargs" ]] && echo "$mpargs"
test: attempt to install only kernel modules for each subsystem When `linux-headers` is installed on Arch Linux, it stores the module source tree in the kernel module directory, which is then picked up by `find` and we get a lot of harmless but annoying errors: ``` ... modprobe: FATAL: Module Kconfig.iosched not found in directory /lib/modules/5.13.7-arch1-1 modprobe: FATAL: Module Kconfig not found in directory /lib/modules/5.13.7-arch1-1 modprobe: FATAL: Module Kconfig not found in directory /lib/modules/5.13.7-arch1-1 modprobe: FATAL: Module dm-mpath.h not found in directory /lib/modules/5.13.7-arch1-1 modprobe: FATAL: Module dm-bio-prison-v2.h not found in directory /lib/modules/5.13.7-arch1-1 modprobe: FATAL: Module raid0.h not found in directory /lib/modules/5.13.7-arch1-1 ... ``` Let's fix this by trying to install only kernel modules (*.ko files with an optional compression).
2021-08-08 19:18:18 +08:00
find "$mod_dir" -path "*/${mod#=}/*" -name "*.ko*" -type f -printf '%f\n'
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
) | instmods
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
fi
;;
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
--*)
mpargs+=" $mod"
;;
i2o_scsi)
# Do not load this diagnostic-only module
return
;;
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
*)
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
mod=${mod##*/}
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
# if we are already installed, skip this module and go on
# to the next one.
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
[[ -f "${initdir:?}/.kernelmodseen/${mod%.ko}.ko" ]] && return
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
# We use '-d' option in modprobe only if modules prefix path
# differs from default '/'. This allows us to use Dracut with
# old version of modprobe which doesn't have '-d' option.
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
local mod_dirname=${mod_dir%%/lib/modules/*}
[[ -n ${mod_dirname} ]] && mod_dirname="-d ${mod_dirname}/"
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
# ok, load the module, all its dependencies, and any firmware
# it may require
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
for_each_kmod_dep install_kmod_with_fw "$mod" \
--set-version "$KERNEL_VER" \
${mod_dirname:+"$mod_dirname"} \
${mpargs:+"$mpargs"}
((ret+=$?))
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
;;
esac
test: move the logic to support /skipped into shared logic The logic to query test state was rather complex. I don't quite grok the point of ret=$((ret+1))… But afaics, the precise result was always ignored by the caller anyway.
2021-04-23 16:55:49 +08:00
return "$ret"
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
}
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
local mod mpargs
if [[ $# -eq 0 ]]; then # filenames from stdin
while read -r mod; do
if ! inst1mod "${mod%.ko*}" && [ "$check" = "yes" ]; then
dfatal "Failed to install $mod"
return 1
fi
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
done
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
fi
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
test: rewrite kernel module handling This code was partially broken, since the firmware directory was undefined. Also, some of the parts were a dead code, since they relied on code from the original dracut test suite.
2021-04-17 00:42:00 +08:00
for mod in "$@"; do # filenames as arguments
if ! inst1mod "${mod%.ko*}" && [ "$check" = "yes" ]; then
dfatal "Failed to install $mod"
return 1
fi
done
return 0
test/TEST-01-BASIC: add /var on cryptoluks and wait for the journal
2012-05-26 00:32:55 +08:00
}
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test/test-functions: avoid stderr noise, only umount on cleanup if mountpoint Only umount it during cleanup if the $TESTDIR/root dir is a mountpoint. This avoids adding noise to the stderr log such as: mountpoint: /var/tmp/systemd-test.waLOFT/root: No such file or directory
2019-08-13 19:50:59 +08:00
_umount_dir() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local mountpoint="${1:?}"
if mountpoint -q "$mountpoint"; then
ddebug "umount $mountpoint"
umount "$mountpoint"
test/test-functions: avoid stderr noise, only umount on cleanup if mountpoint Only umount it during cleanup if the $TESTDIR/root dir is a mountpoint. This avoids adding noise to the stderr log such as: mountpoint: /var/tmp/systemd-test.waLOFT/root: No such file or directory
2019-08-13 19:50:59 +08:00
fi
}
test: add create_empty_image_rootdir() to simplify testcase setup Almost all tests were manually mounting/unmounting $TESTDIR/root from the loopback image; this moves all that into test-functions so the test setup functions are simplier. Also add test_setup_cleanup() function, to cleanup what is mounted by create_empty_image_rootdir()
2019-07-12 23:47:26 +08:00
# can be overridden in specific test
test_setup_cleanup() {
test: run tests directly from the loopback device Before, we'd copy the test tree into nspawn-root, and run the tests from there. This is OK, and doesn't actually take much extra time. But it uses quite a lot of extra disk space. So let's make things a bit more efficient by running directly from the image file. We still run the unprivileged nspawn tests from a copy. Once the kernel implements fs shift, we can do away with that too.
2019-12-13 21:21:31 +08:00
cleanup_initdir
test: add create_empty_image_rootdir() to simplify testcase setup Almost all tests were manually mounting/unmounting $TESTDIR/root from the loopback image; this moves all that into test-functions so the test setup functions are simplier. Also add test_setup_cleanup() function, to cleanup what is mounted by create_empty_image_rootdir()
2019-07-12 23:47:26 +08:00
}
_test_cleanup() {
test: ignore errors during test cleanup, so cleanup can finish Also move TESTDIR and STATEFILE removal into test_cleanup
2019-07-13 02:09:48 +08:00
# (post-test) cleanup should always ignore failure and cleanup as much as possible
(
set +e
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -n "$initdir" ]] && _umount_dir "$initdir"
[[ -n "$IMAGE_PUBLIC" ]] && rm -vf "$IMAGE_PUBLIC"
test: fix 'make clean' not removing shared image
2021-01-28 21:26:35 +08:00
# If multiple setups/cleans are ran in parallel, this can cause a race
test: shellcheck-ify test scripts
2021-09-30 02:30:08 +08:00
if [[ -n "$IMAGESTATEDIR" && $TEST_PARALLELIZE -ne 1 ]]; then
test: fix 'make clean' not removing shared image
2021-01-28 21:26:35 +08:00
rm -vf "${IMAGESTATEDIR}/default.img"
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -n "$TESTDIR" ]] && rm -vfr "$TESTDIR"
[[ -n "$STATEFILE" ]] && rm -vf "$STATEFILE"
test: convert all uses of '|| true' into '|| :' No change in functionality; just use the shorter || :
2019-07-19 08:34:57 +08:00
) || :
test: add create_empty_image_rootdir() to simplify testcase setup Almost all tests were manually mounting/unmounting $TESTDIR/root from the loopback image; this moves all that into test-functions so the test setup functions are simplier. Also add test_setup_cleanup() function, to cleanup what is mounted by create_empty_image_rootdir()
2019-07-12 23:47:26 +08:00
}
test: Factorize common integration test functions (#6540) All test/TEST* but TEST-02-CRYPTSETUP share the same check_result_qemu() and test_cleanup(), so move them into test_functions and only override them in TEST-02-CRYPTSETUP. Also provide a common test_run() which by default assumes that both QEMU and nspawn tests are run. Particular tests which don't support either need to explicitly opt out by setting $TEST_NO_{QEMU,NSPAWN}. Do it this way around to avoid accidentally forgetting to opt in, and to encourage test authors to at least always support nspawn.
2017-08-04 20:34:14 +08:00
# can be overridden in specific test
test_cleanup() {
test: add create_empty_image_rootdir() to simplify testcase setup Almost all tests were manually mounting/unmounting $TESTDIR/root from the loopback image; this moves all that into test-functions so the test setup functions are simplier. Also add test_setup_cleanup() function, to cleanup what is mounted by create_empty_image_rootdir()
2019-07-12 23:47:26 +08:00
_test_cleanup
test: Factorize common integration test functions (#6540) All test/TEST* but TEST-02-CRYPTSETUP share the same check_result_qemu() and test_cleanup(), so move them into test_functions and only override them in TEST-02-CRYPTSETUP. Also provide a common test_run() which by default assumes that both QEMU and nspawn tests are run. Particular tests which don't support either need to explicitly opt out by setting $TEST_NO_{QEMU,NSPAWN}. Do it this way around to avoid accidentally forgetting to opt in, and to encourage test authors to at least always support nspawn.
2017-08-04 20:34:14 +08:00
}
test: perform partial cleanup after each test is run This causes the unprivileged-nspawn-root directory to be removed after running one test. The advantage is that we reduce the maximum disk-space use quite a bit (47*400 MB → about 18GB).
2020-03-21 02:09:35 +08:00
test_cleanup_again() {
[ -n "$TESTDIR" ] || return
rm -rf "$TESTDIR/unprivileged-nspawn-root"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ -n "$initdir" ]] && _umount_dir "$initdir"
test: perform partial cleanup after each test is run This causes the unprivileged-nspawn-root directory to be removed after running one test. The advantage is that we reduce the maximum disk-space use quite a bit (47*400 MB → about 18GB).
2020-03-21 02:09:35 +08:00
}
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
test_create_image() {
test: de-duplicate test_setup() functions Now we will run setup_nspawn_root() in some case where do don't need to.
2019-12-10 23:40:31 +08:00
create_empty_image_rootdir
# Create what will eventually be our root filesystem onto an overlay
(
LOG_LEVEL=5
setup_basic_environment
)
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
}
test_setup() {
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "${TEST_REQUIRE_INSTALL_TESTS:?}" && \
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
command -v meson >/dev/null && \
[[ "$(meson configure "${BUILD_DIR:?}" | grep install-tests | awk '{ print $2 }')" != "true" ]]; then
test: print path to build directory on misconfiguration This way it's easier to see when a wrong $BUILD_DIR was passed.
2020-03-31 21:13:13 +08:00
dfatal "$BUILD_DIR needs to be built with -Dinstall-tests=true"
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
exit 1
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [ -e "${IMAGE_PRIVATE:?}" ]; then
echo "Reusing existing image $IMAGE_PRIVATE → $(realpath "$IMAGE_PRIVATE")"
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
mount_initdir
test: create public images by default Let's create new images public by default and then symlink/copy them into the respective private directories afterwards, not the other way around. This should fix a nasty race condition in parallel runs where one tests attempts to copy the backing public image at the same moment another test is already modifying it.
2020-05-26 04:48:01 +08:00
else
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [ ! -e "${IMAGE_PUBLIC:?}" ]; then
tests: build the image once and then copy/extend it Building custom images for each test takes a lot of time. Build the default one, and if the test needs incompatible changes just copy it and extend it instead.
2021-01-07 05:42:28 +08:00
# default.img is the base that every test uses and optionally appends to
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [ ! -e "${IMAGESTATEDIR:?}/default.img" ] || [ -n "${TEST_FORCE_NEWIMAGE:=}" ]; then
tests: build the image once and then copy/extend it Building custom images for each test takes a lot of time. Build the default one, and if the test needs incompatible changes just copy it and extend it instead.
2021-01-07 05:42:28 +08:00
# Create the backing public image, but then completely unmount
# it and drop the loopback device responsible for it, since we're
# going to symlink/copy the image and mount it again from
# elsewhere.
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local image_old="${IMAGE_PUBLIC}"
tests: build the image once and then copy/extend it Building custom images for each test takes a lot of time. Build the default one, and if the test needs incompatible changes just copy it and extend it instead.
2021-01-07 05:42:28 +08:00
if [ -z "${TEST_FORCE_NEWIMAGE}" ]; then
IMAGE_PUBLIC="${IMAGESTATEDIR}/default.img"
fi
test_create_image
test_setup_cleanup
umount_loopback
cleanup_loopdev
IMAGE_PUBLIC="${image_old}"
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if [ "${IMAGE_NAME:?}" != "default" ] && ! get_bool "${TEST_FORCE_NEWIMAGE}"; then
tests: build the image once and then copy/extend it Building custom images for each test takes a lot of time. Build the default one, and if the test needs incompatible changes just copy it and extend it instead.
2021-01-07 05:42:28 +08:00
cp -v "$(realpath "${IMAGESTATEDIR}/default.img")" "$IMAGE_PUBLIC"
fi
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
local hook_defined
declare -f -F test_append_files >/dev/null && hook_defined=yes || hook_defined=no
test: create public images by default Let's create new images public by default and then symlink/copy them into the respective private directories afterwards, not the other way around. This should fix a nasty race condition in parallel runs where one tests attempts to copy the backing public image at the same moment another test is already modifying it.
2020-05-26 04:48:01 +08:00
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
echo "Reusing existing cached image $IMAGE_PUBLIC → $(realpath "$IMAGE_PUBLIC")"
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$TEST_PARALLELIZE" || get_bool "$hook_defined"; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
cp -v -- "$(realpath "$IMAGE_PUBLIC")" "$IMAGE_PRIVATE"
test: introduce TEST_PARALLELIZE to support running tests in parallel Support running tests in parallel by switching to copying of the base image instead of symlinking it.. This still requires some setup steps, like running `make setup` on tests which have unique $IMAGE_NAME beforehand (and sequentially), otherwise they'll all try to create the same base image when started in parallel, leading to nasty issues. However, as running the integration tests in parallel is such an unusual use case it should be good enough, for now.
2020-05-17 16:48:16 +08:00
else
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
ln -sv -- "$(realpath "$IMAGE_PUBLIC")" "$IMAGE_PRIVATE"
test: introduce TEST_PARALLELIZE to support running tests in parallel Support running tests in parallel by switching to copying of the base image instead of symlinking it.. This still requires some setup steps, like running `make setup` on tests which have unique $IMAGE_NAME beforehand (and sequentially), otherwise they'll all try to create the same base image when started in parallel, leading to nasty issues. However, as running the integration tests in parallel is such an unusual use case it should be good enough, for now.
2020-05-17 16:48:16 +08:00
fi
test: create public images by default Let's create new images public by default and then symlink/copy them into the respective private directories afterwards, not the other way around. This should fix a nasty race condition in parallel runs where one tests attempts to copy the backing public image at the same moment another test is already modifying it.
2020-05-26 04:48:01 +08:00
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
mount_initdir
test: correctly mask supporting services in tests, take #2 Due to a little misunderstanding the last patch doesn't work as expected, since test_create_image() is called only for the first image (usually TEST-01-BASIC), and all subsequent images are then (possibly) modified with test_append_files(). Follow-up to 179ca4d2b1b5579014773a128462475f99b7a91b.
2021-06-24 18:13:52 +08:00
# We want to test all services in TEST-01-BASIC, but mask them in
# all other tests
if [[ "${TESTID:?}" != "01" ]]; then
dinfo "Masking supporting services"
mask_supporting_services
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "$hook_defined"; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
test_append_files "${initdir:?}"
tests: build the image once and then copy/extend it Building custom images for each test takes a lot of time. Build the default one, and if the test needs incompatible changes just copy it and extend it instead.
2021-01-07 05:42:28 +08:00
fi
test: rework how images are created Before, we'd create a separate image for each test, in /var/tmp/systemd-test.XXXXX/rootdisk.img. Most of the images where very similar, except that each one had some unit files installed specifically for the test. The installation of those custom unit files was removed in previous commits (all the unit files are always installed). The new approach is to only create as few distinct images as possible. We have: default.img: the "normal" image suitable for almost all the tests basic.img: the same as default image but doesn't mask any services cryptsetup.img: p2 is used for encrypted /var badid.img: /etc/machine-id is overwritten with stuff selinux.img: with selinux added for fun and fun and a few others: ls -l build/test/*img lrwxrwxrwx 1 root root 38 Mar 21 21:23 build/test/badid.img -> /var/tmp/systemd-test.PJFFeo/badid.img lrwxrwxrwx 1 root root 38 Mar 21 21:17 build/test/basic.img -> /var/tmp/systemd-test.na0xOI/basic.img lrwxrwxrwx 1 root root 43 Mar 21 21:18 build/test/cryptsetup.img -> /var/tmp/systemd-test.Tzjv06/cryptsetup.img lrwxrwxrwx 1 root root 40 Mar 21 21:19 build/test/default.img -> /var/tmp/systemd-test.EscAsS/default.img lrwxrwxrwx 1 root root 39 Mar 21 21:22 build/test/nspawn.img -> /var/tmp/systemd-test.HSebKo/nspawn.img lrwxrwxrwx 1 root root 40 Mar 21 21:20 build/test/selinux.img -> /var/tmp/systemd-test.daBjbx/selinux.img lrwxrwxrwx 1 root root 39 Mar 21 21:21 build/test/test08.img -> /var/tmp/systemd-test.OgnN8Z/test08.img I considered trying to use the same image everywhere. It would probably be possible, but it would be very brittle. By using separate images where it is necessary we keep various orthogonal modifications independent. The way that images are cached is complicated by the fact that we still want to keep them in /var/tmp. Thus, an image is created on first use and linked to from build/test/ so it can be found by other tests. Tests cannot be run in parallel. I think that is an acceptable limitation. Creation of the images was probably taking more resources then the actual tests, so we should be better off anyway.
2019-12-12 16:37:19 +08:00
fi
test: de-duplicate test_setup() functions Now we will run setup_nspawn_root() in some case where do don't need to.
2019-12-10 23:40:31 +08:00
setup_nspawn_root
}
test: Factorize common integration test functions (#6540) All test/TEST* but TEST-02-CRYPTSETUP share the same check_result_qemu() and test_cleanup(), so move them into test_functions and only override them in TEST-02-CRYPTSETUP. Also provide a common test_run() which by default assumes that both QEMU and nspawn tests are run. Particular tests which don't support either need to explicitly opt out by setting $TEST_NO_{QEMU,NSPAWN}. Do it this way around to avoid accidentally forgetting to opt in, and to encourage test authors to at least always support nspawn.
2017-08-04 20:34:14 +08:00
test_run() {
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
local test_id="${1:?}"
test: make sure to reset state before starting integration tests
2019-12-13 19:34:41 +08:00
mount_initdir
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if ! get_bool "${TEST_NO_QEMU:=}"; then
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if run_qemu "$test_id"; then
test: echo a message when the tests fails Sometimes it is not clear from the preceding output that there was a failure.
2020-03-21 00:16:30 +08:00
check_result_qemu || { echo "QEMU test failed"; return 1; }
test: Factorize common integration test functions (#6540) All test/TEST* but TEST-02-CRYPTSETUP share the same check_result_qemu() and test_cleanup(), so move them into test_functions and only override them in TEST-02-CRYPTSETUP. Also provide a common test_run() which by default assumes that both QEMU and nspawn tests are run. Particular tests which don't support either need to explicitly opt out by setting $TEST_NO_{QEMU,NSPAWN}. Do it this way around to avoid accidentally forgetting to opt in, and to encourage test authors to at least always support nspawn.
2017-08-04 20:34:14 +08:00
else
dwarn "can't run QEMU, skipping"
fi
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if ! get_bool "${TEST_NO_NSPAWN:=}"; then
test: run tests directly from the loopback device Before, we'd copy the test tree into nspawn-root, and run the tests from there. This is OK, and doesn't actually take much extra time. But it uses quite a lot of extra disk space. So let's make things a bit more efficient by running directly from the image file. We still run the unprivileged nspawn tests from a copy. Once the kernel implements fs shift, we can do away with that too.
2019-12-13 21:21:31 +08:00
mount_initdir
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if run_nspawn "${initdir:?}" "$test_id"; then
test: run tests directly from the loopback device Before, we'd copy the test tree into nspawn-root, and run the tests from there. This is OK, and doesn't actually take much extra time. But it uses quite a lot of extra disk space. So let's make things a bit more efficient by running directly from the image file. We still run the unprivileged nspawn tests from a copy. Once the kernel implements fs shift, we can do away with that too.
2019-12-13 21:21:31 +08:00
check_result_nspawn "$initdir" || { echo "nspawn-root test failed"; return 1; }
test: Factorize common integration test functions (#6540) All test/TEST* but TEST-02-CRYPTSETUP share the same check_result_qemu() and test_cleanup(), so move them into test_functions and only override them in TEST-02-CRYPTSETUP. Also provide a common test_run() which by default assumes that both QEMU and nspawn tests are run. Particular tests which don't support either need to explicitly opt out by setting $TEST_NO_{QEMU,NSPAWN}. Do it this way around to avoid accidentally forgetting to opt in, and to encourage test authors to at least always support nspawn.
2017-08-04 20:34:14 +08:00
else
dwarn "can't run systemd-nspawn, skipping"
fi
tests: also run TEST-01-BASIC in an unprivileged container (#9957) This should make it much easier to catch regressions like https://github.com/systemd/systemd/issues/9914 and https://github.com/systemd/systemd/issues/8535.
2018-08-30 12:01:18 +08:00
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "${RUN_IN_UNPRIVILEGED_CONTAINER:=}"; then
test: run tests directly from the loopback device Before, we'd copy the test tree into nspawn-root, and run the tests from there. This is OK, and doesn't actually take much extra time. But it uses quite a lot of extra disk space. So let's make things a bit more efficient by running directly from the image file. We still run the unprivileged nspawn tests from a copy. Once the kernel implements fs shift, we can do away with that too.
2019-12-13 21:21:31 +08:00
dir="$TESTDIR/unprivileged-nspawn-root"
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if NSPAWN_ARGUMENTS="-U --private-network ${NSPAWN_ARGUMENTS:-}" run_nspawn "$dir" "$test_id"; then
test: run tests directly from the loopback device Before, we'd copy the test tree into nspawn-root, and run the tests from there. This is OK, and doesn't actually take much extra time. But it uses quite a lot of extra disk space. So let's make things a bit more efficient by running directly from the image file. We still run the unprivileged nspawn tests from a copy. Once the kernel implements fs shift, we can do away with that too.
2019-12-13 21:21:31 +08:00
check_result_nspawn "$dir" || { echo "unprivileged-nspawn-root test failed"; return 1; }
tests: also run TEST-01-BASIC in an unprivileged container (#9957) This should make it much easier to catch regressions like https://github.com/systemd/systemd/issues/9914 and https://github.com/systemd/systemd/issues/8535.
2018-08-30 12:01:18 +08:00
else
dwarn "can't run systemd-nspawn, skipping"
fi
tests: ignore memory leaks in dbus-daemon Otherwise, the test fails on Fedora 28 with ``` Jan 30 01:42:35 systemd-testsuite dbus-daemon[61]: [system] Successfully activated service 'org.freedesktop.systemd1' Jan 30 01:42:35 systemd-testsuite systemd[61]: dbus.service: Kernel keyring access prohibited, ignoring. Jan 30 01:42:35 systemd-testsuite systemd[61]: dbus.service: Executing: /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: ================================================================= Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: ==61==ERROR: LeakSanitizer: detected memory leaks Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: Direct leak of 72 byte(s) in 1 object(s) allocated from: Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #0 0x7f21f9e29088 in __interceptor_realloc (/usr/lib64/libasan.so.5+0xef088) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #1 0x7f21f9b1b23c (/lib64/libdbus-1.so.3+0x3323c) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #2 0x240000001b (<unknown module>) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: Direct leak of 16 byte(s) in 1 object(s) allocated from: Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #0 0x7f21f9e29088 in __interceptor_realloc (/usr/lib64/libasan.so.5+0xef088) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #1 0x7f21f9b1b23c (/lib64/libdbus-1.so.3+0x3323c) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: #2 0x7ffffffff (<unknown module>) Jan 30 01:42:50 systemd-testsuite dbus-daemon[61]: SUMMARY: AddressSanitizer: 88 byte(s) leaked in 2 allocation(s). ``` The leaks were reported and fixed in https://bugs.freedesktop.org/show_bug.cgi?id=107320.
2019-01-30 09:19:45 +08:00
fi
test: Factorize common integration test functions (#6540) All test/TEST* but TEST-02-CRYPTSETUP share the same check_result_qemu() and test_cleanup(), so move them into test_functions and only override them in TEST-02-CRYPTSETUP. Also provide a common test_run() which by default assumes that both QEMU and nspawn tests are run. Particular tests which don't support either need to explicitly opt out by setting $TEST_NO_{QEMU,NSPAWN}. Do it this way around to avoid accidentally forgetting to opt in, and to encourage test authors to at least always support nspawn.
2017-08-04 20:34:14 +08:00
fi
return 0
}
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
do_test() {
test/test-functions: be more verbose about skipping tests
2012-05-25 20:53:29 +08:00
if [[ $UID != "0" ]]; then
echo "TEST: $TEST_DESCRIPTION [SKIPPED]: not root" >&2
exit 0
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "${TEST_NO_QEMU:=}" && get_bool "${TEST_NO_NSPAWN:=}"; then
test: shortcut skip if both TEST_NO_QEMU and TEST_NO_NSPAWN are set Allows to run all tests in bulk with TEST_NO_QEMU, skipping those where it is mandatory, without wasting time building the image.
2020-12-23 01:51:40 +08:00
echo "TEST: $TEST_DESCRIPTION [SKIPPED]: both QEMU and nspawn disabled" >&2
exit 0
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "${TEST_QEMU_ONLY:=}" && ! get_bool "$TEST_NO_NSPAWN"; then
tests: add TEST_QEMU_ONLY variable to run only tests where qemu is mandatory Allows to split the test run in two parts. Most tests can run under nspawn which is much faster, and they can be ran in one chunk with TEST_NO_QEMU=1. The qemu-only tests, which are just a handful, can be ran in another chunk with TEST_QEMU_ONLY=1. Allows autopkgtest to be split in two parts.
2020-12-27 04:11:55 +08:00
echo "TEST: $TEST_DESCRIPTION [SKIPPED]: QEMU-only tests requested" >&2
exit 0
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
if get_bool "${TEST_PREFER_NSPAWN:=}" && ! get_bool "$TEST_NO_NSPAWN"; then
tests: add TEST_PREFER_NSPAWN variable to run as many as possible under nspawn By default the test suite prefers qemu, and uses nspawn only if a test specifically says it doesn't support qemu. Add a variable to allow flipping the default, and run as many tests under nspawn as possible.
2021-01-01 01:29:58 +08:00
TEST_NO_QEMU=1
fi
scripts: use 4 space indentation We had all kinds of indentation: 2 sp, 3 sp, 4 sp, 8 sp, and mixed. 4 sp was the most common, in particular the majority of scripts under test/ used that. Let's standarize on 4 sp, because many commandlines are long and there's a lot of nesting, and with 8sp indentation less stuff fits. 4 sp also seems to be the default indentation, so this will make it less likely that people will mess up if they don't load the editor config. (I think people often use vi, and vi has no support to load project-wide configuration automatically. We distribute a .vimrc file, but it is not loaded by default, and even the instructions in it seem to discourage its use for security reasons.) Also remove the few vim config lines that were left. We should either have them on all files, or none. Also remove some strange stuff like '#!/bin/env bash', yikes.
2019-04-04 20:10:42 +08:00
# Detect lib paths
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ "$libdir" ]] || for libdir in /lib64 /lib; do
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
[[ -d $libdir ]] && libdirs+=" $libdir" && break
done
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
[[ "$usrlibdir" ]] || for usrlibdir in /usr/lib64 /usr/lib; do
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
[[ -d $usrlibdir ]] && libdirs+=" $usrlibdir" && break
done
test: Write state and log files into $BUILD_DIR This avoids clobbering the source tree with .testdir/test.log files and makes the tests work in situations where the source tree is read-only.
2017-08-07 23:30:11 +08:00
mkdir -p "$STATEDIR"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
import_testdir
test: remove duplicated code
2013-11-06 06:32:56 +08:00
import_initdir
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
tests: use setfacl to give $SUDO_USER read permissions on artifacts We have to invoke the tests as superuser, and not being able to read the journal as the invoking user is annoying. I don't think there are any security considerations here, since the invoking user can already put arbitrary code in the Makefile and test scripts which get executed with root privileges.
2021-04-20 23:47:50 +08:00
if [ -n "${SUDO_USER}" ]; then
ddebug "Making ${TESTDIR:?} readable for ${SUDO_USER} (acquired from sudo)"
setfacl -m "user:${SUDO_USER:?}:r-X" "${TESTDIR:?}"
fi
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
testname="$(basename "$PWD")"
test: print test of name in output
2020-03-25 20:15:37 +08:00
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
while (($# > 0)); do
case $1 in
--run)
test: print test of name in output
2020-03-25 20:15:37 +08:00
echo "${testname} RUN: $TEST_DESCRIPTION"
test: "detect" the test number automagically Specifying the test number manually is tedious and prone to errors (as recently proven). Since we have all the necessary data to work out the test number, let's do it automagically.
2021-04-27 01:20:18 +08:00
test_run "$TESTID"
test: do not use global variable to pass error
2018-09-14 12:25:02 +08:00
ret=$?
test: shell code cleanup Just some basic stuff, like double quotes around strings, using predictable values for possibly unset variables, formatting, etc.
2021-04-17 00:33:44 +08:00
if [ $ret -eq 0 ]; then
test: print test of name in output
2020-03-25 20:15:37 +08:00
echo "${testname} RUN: $TEST_DESCRIPTION [OK]"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
else
test: print test of name in output
2020-03-25 20:15:37 +08:00
echo "${testname} RUN: $TEST_DESCRIPTION [FAILED]"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
exit $ret
;;
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
--setup)
test: print test of name in output
2020-03-25 20:15:37 +08:00
echo "${testname} SETUP: $TEST_DESCRIPTION"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test_setup
test: add create_empty_image_rootdir() to simplify testcase setup Almost all tests were manually mounting/unmounting $TESTDIR/root from the loopback image; this moves all that into test-functions so the test setup functions are simplier. Also add test_setup_cleanup() function, to cleanup what is mounted by create_empty_image_rootdir()
2019-07-12 23:47:26 +08:00
test_setup_cleanup
test: Run qemu/nspawn tests with "set -e" This catches errors like "ninja not found", missing programs etc. early, instead of silently ignoring them and trying to boot a broken VM. In install_config_files(), allow some distro specific files to be absent (such as /etc/sysconfig/init).
2017-08-08 03:09:21 +08:00
;;
test: perform partial cleanup after each test is run This causes the unprivileged-nspawn-root directory to be removed after running one test. The advantage is that we reduce the maximum disk-space use quite a bit (47*400 MB → about 18GB).
2020-03-21 02:09:35 +08:00
--clean)
test: print test of name in output
2020-03-25 20:15:37 +08:00
echo "${testname} CLEANUP: $TEST_DESCRIPTION"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
test_cleanup
test: Run qemu/nspawn tests with "set -e" This catches errors like "ninja not found", missing programs etc. early, instead of silently ignoring them and trying to boot a broken VM. In install_config_files(), allow some distro specific files to be absent (such as /etc/sysconfig/init).
2017-08-08 03:09:21 +08:00
;;
test: perform partial cleanup after each test is run This causes the unprivileged-nspawn-root directory to be removed after running one test. The advantage is that we reduce the maximum disk-space use quite a bit (47*400 MB → about 18GB).
2020-03-21 02:09:35 +08:00
--clean-again)
test: print test of name in output
2020-03-25 20:15:37 +08:00
echo "${testname} CLEANUP AGAIN: $TEST_DESCRIPTION"
test: perform partial cleanup after each test is run This causes the unprivileged-nspawn-root directory to be removed after running one test. The advantage is that we reduce the maximum disk-space use quite a bit (47*400 MB → about 18GB).
2020-03-21 02:09:35 +08:00
test_cleanup_again
;;
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
--all)
test: Run qemu/nspawn tests with "set -e" This catches errors like "ninja not found", missing programs etc. early, instead of silently ignoring them and trying to boot a broken VM. In install_config_files(), allow some distro specific files to be absent (such as /etc/sysconfig/init).
2017-08-08 03:09:21 +08:00
ret=0
test: print test of name in output
2020-03-25 20:15:37 +08:00
echo -n "${testname}: $TEST_DESCRIPTION "
test: avoid leaking open loop devices When a subshell is used ('make' or 'make all') the LOOPDEV environment variable, which is used to store the opened loop device, is lost. So the cleanup on trap/exit doesn't do anything, and the loop device used to mount the test image is left around. Avoid using a subshell to fix the issue.
2021-02-17 07:47:34 +08:00
# Do not use a subshell, otherwise cleanup variables (LOOPDEV) will be lost
# and loop devices will leak
test_setup </dev/null >"$TESTLOG" 2>&1 || ret=$?
if [ $ret -eq 0 ]; then
test_setup_cleanup </dev/null >>"$TESTLOG" 2>&1 || ret=$?
fi
if [ $ret -eq 0 ]; then
test: "detect" the test number automagically Specifying the test number manually is tedious and prone to errors (as recently proven). Since we have all the necessary data to work out the test number, let's do it automagically.
2021-04-27 01:20:18 +08:00
test_run "$TESTID" </dev/null >>"$TESTLOG" 2>&1 || ret=$?
test: avoid leaking open loop devices When a subshell is used ('make' or 'make all') the LOOPDEV environment variable, which is used to store the opened loop device, is lost. So the cleanup on trap/exit doesn't do anything, and the loop device used to mount the test image is left around. Avoid using a subshell to fix the issue.
2021-02-17 07:47:34 +08:00
fi
test: ignore errors during test cleanup, so cleanup can finish Also move TESTDIR and STATEFILE removal into test_cleanup
2019-07-13 02:09:48 +08:00
test_cleanup
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
if [ $ret -eq 0 ]; then
test: Write state and log files into $BUILD_DIR This avoids clobbering the source tree with .testdir/test.log files and makes the tests work in situations where the source tree is read-only.
2017-08-07 23:30:11 +08:00
rm "$TESTLOG"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
echo "[OK]"
else
echo "[FAILED]"
test: Write state and log files into $BUILD_DIR This avoids clobbering the source tree with .testdir/test.log files and makes the tests work in situations where the source tree is read-only.
2017-08-07 23:30:11 +08:00
echo "see $TESTLOG"
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
fi
test: unify handling of boolean values Let's unify handling of the boolean values throughout the test-functions code, since we use 0/1, true/false, and yes/no almost randomly in many places, so picking the right values during CI configuration can be a real pain.
2021-07-04 18:21:23 +08:00
exit $ret
;;
*)
break
;;
test: introduce a basic testsuite framework $ cd test $ sudo make check will run all tests in the TEST-* subdirectories $ cd test/TEST-01-BASIC $ sudo make clean setup run will run the different stages of the test for debugging purposes
2012-05-22 00:55:48 +08:00
esac
shift
done
}
Reference in New Issue Copy Permalink