Commit Graph

119 Commits

Author SHA1 Message Date
Daan De Meyer
c859b310ed mkosi: Add github CLI to tools 2024-11-24 16:47:20 +01:00
Daan De Meyer
6fd5df6005 mkosi: Add shellcheck to tools 2024-11-24 16:47:04 +01:00
Davide Cavalca
fa8a55a914 mkosi: ruff is not available on all distros
Refactor to only install ruff where it is available
2024-11-12 18:05:17 +00:00
Davide Cavalca
f2672f2c5d mkosi: Install tpm2-tss-devel to tools for CentOS and Fedora instead of tss2-devel
tss2-devel is the IBM TPM stack, we want the Intel TPM stack, so let's
use the correct package.
2024-11-12 22:45:25 +09:00
Daan De Meyer
5cca978dae mkosi: Add pytest to tools 2024-11-07 20:24:59 +01:00
Daan De Meyer
48c5a4cd67 mkosi: Add ruff and mypy to tools tree packages 2024-11-05 14:26:21 +01:00
Daan De Meyer
d980aee1e8 mkosi: Add extra tools tree packages required to run integration tests
With https://github.com/systemd/mkosi/pull/3164, we'll be able to run
arbitrary commands in the mkosi sandbox, which has /usr from the tools
tree if one is configured. Let's add the required packages to be able to
run meson to setup the integration tests. This allows running the integration
tests without having to install meson or other build dependencies on the
host system.

"""
mkosi sandbox meson setup build
mkosi sandbox meson compile -C build mkosi
mkosi sandbox env SYSTEMD_INTEGRATION_TESTS=1 meson test -C build ...
"""
2024-11-02 23:18:41 +01:00
Yu Watanabe
4e3f16b5ef mkosi: replace PackageManagerTrees= with SandboxTrees= 2024-10-13 05:43:32 +09:00
Daan De Meyer
963157ca78 mkosi: Fix sections for settings
Upstream we moved settings around a bit to different sections, let's
adapt to those changes in the systemd repo.
2024-10-09 20:36:57 +02:00
Daan De Meyer
d615d4e0de mkosi: Remove particle profile
We have https://github.com/systemd/particleos for testing the particle
stuff so let's drop it from the systemd repo as it's bit rotting.
2024-10-09 20:36:57 +02:00
Daan De Meyer
4eae2be0d7 mkosi: Install util-linux-script on Rawhide
It's now subpackaged so we can build images without pulling in
libutempter but we use script in the testsuite so let's install the
subpackage.
2024-09-05 18:38:33 +02:00
Daan De Meyer
89c579788d mkosi: Use apt patterns to install dependencies on Debian/Ubuntu
Instead of parsing the human readable output of apt-cache, let's
use apt patterns to figure out the dependencies.

We also filter out virtual packages as apt will fail and say we need
to install an implementation of the virtual package even if a package
that provides the virtual package is already installed.
2024-09-03 16:03:27 +02:00
Daan De Meyer
70ecdbfa23 mkosi: Make systemd package filtering more robust
Let's not just filter everything with systemd in the name, but instead
use the same list of volatile packages that we install to do the
filtering.
2024-09-03 14:42:09 +02:00
Luca Boccassi
4b54da2e37
Merge pull request #34142 from DaanDeMeyer/update-distributions
mkosi: Update distribution packaging commits
2024-08-28 17:17:12 +01:00
Luca Boccassi
ec54029017 mkosi: use util-linux's autologin
login is now from util-linux so credentials are supported.
It also needs to be pulled in as it's Protected: yes rather than
Essential: yes.

Keep the old setting for Ubuntu as that still uses login from shadow.
2024-08-28 16:32:17 +01:00
Daan De Meyer
3e09a3eac2 mkosi: Include noarch in dnf repoquery architectures
ukify is noarch so we should include noarch to get all results.
2024-08-28 16:15:42 +02:00
Daan De Meyer
edc6592e53 mkosi: Switch back to btrfs
The next commit will introduce a way to iterate on integration
tests which depends on btrfs specific features.

We leave CentOS on ext4 as its kernel does not support btrfs.
2024-08-05 15:00:24 +02:00
Daan De Meyer
ea696c5f53 mkosi: Enable Autologin= again on Debian
Debian uses /usr/bin/login from the shadow package instead of util-linux
which doesn't support credentials. Let's enable autologin the old
fashioned way for now.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=833256
2024-08-05 15:00:24 +02:00
Yu Watanabe
f38aac5e01 mkosi: fix typo
Follow-up for 7205fc7dc3.
2024-08-03 05:47:55 +09:00
Daan De Meyer
3fe25d4530 mkosi: Beef up testuser a bit
Give it a password and add it to some common groups.
2024-07-31 18:32:53 +02:00
Daan De Meyer
ac31804db7 mkosi: Fix typo
Our config parsing is flexible enough that this kind of worked surprisingly
enough.
2024-07-18 12:53:53 +02:00
Daan De Meyer
91cc65c5ad mkosi: Drop util-linux from centos/fedora packages
It's already included in the packages list in mkosi.conf.
2024-07-18 12:51:44 +02:00
Daan De Meyer
773f2e9633 mkosi: Build CentOS Stream 10 images by default
CentOS Stream 10 has a newer util-linux which means the terminal
gets correctly resized to the size specified by mkosi. This is a
much nicer experience than CentOS Stream 9 where you're stuck on
80x24 so let's make CentOS Stream 10 the default release to build.
2024-07-18 11:52:01 +02:00
Daan De Meyer
d279ec4a50 mkosi: Streamline running the integration tests without building systemd
Let's document in detail how to build the integration test image and run
the integration tests without building systemd. To streamline the process,
we stop automatically using binaries from build/ when invoking mkosi directly
and don't automatically use a tools tree anymore if systemd on the host is too
old. Instead, we document these options in HACKING.md and change the mkosi meson
target to automatically use the current build directory as an extra binary search
path for mkosi.
2024-07-18 11:39:07 +02:00
Daan De Meyer
68ee977114 mkosi: Build initrd as a subimage
Let's make things a little more consistent and build the initrd
explicitly as a subimage as well instead of relying on mkosi building
it as part of the main image build.

We drop the opensuse initrd postinst script as we don't use erofs by
default anymore. We can always reintroduce it again later if needed.
2024-07-16 13:59:47 +02:00
Daan De Meyer
0e4a7ab6d5 mkosi: Make epel repositories optional for CentOS Stream 9
This allows us to add CI for CentOS Stream 10 as EPEL 10 doesn't
exist yet and won't exist for quite some time.

CentOS Stream 10 will be enabled later as soon as
https://issues.redhat.com/browse/RHEL-46604 is resolved.
2024-07-15 16:17:33 +02:00
Daan De Meyer
7205fc7dc3 mkosi: Introduce build image
We want the exitrd image to be built with the latest systemd as well.
As the exitrd image is built as part of mkosi.images, and all subimages
are built before the main image, this implies the packages must be built
as a subimage in mkosi.images/ as well. So we introduce the build image and
move all logic related to building distribution packages there.

This also has the nice side effect of slimming down the main image as the
build dependencies are not installed into the main image anymore. It also
makes sure the packages are built in a "clean" chroot without any of the
other packages which we install in the main image available.
2024-07-15 16:17:33 +02:00
Daan De Meyer
72a8f508e5 mkosi: update fedora commit reference
* a3524fc837 Use a more precise Recommends for libkxbcommon
* 980ede8c0f Drop machined revert
* d569018a92 Rebuilt for the bin-sbin merge
* 8881fa94ee Version 256.2
* 1cc4f83002 Link systemd-executor statically
* 0319e62d9c Update dracut workaround
* c96f54de22 Fix ELN build
* 3f68c5d802 Only exclude dracut conflicts on non-fedora on upstream builds
* 7db154308b Conditionalize dracut Conflicts more
2024-07-15 16:17:33 +02:00
Daan De Meyer
a86211b7ca mkosi: Switch back to code.opensuse.org for opensuse
The commits on src.opensuse.org keep disappearing so let's switch
back to code.opensuse.org.
2024-07-11 14:12:59 +02:00
Daan De Meyer
14fb6354f1
Merge pull request #33636 from DaanDeMeyer/ext4
Various integration test improvements
2024-07-10 21:33:23 +02:00
Daan De Meyer
eb6f177ae4 mkosi: update opensuse commit reference
*   8c025c3bdf Accepting request 1184267 from Base:System
|\
| * 735f8c4ba4 - Import commit 5a8eadd0c021758337a020c423f25a353bdb9b3c (merge of v255.8)   For a complete list of changes, visit:   603cd1d4d8...5a8eadd0c0 - Drop 5003-Revert-run-pass-the-pty-slave-fd-to-transient-servic.patch as v255.8   contains the workaround (commit 639c922ede9485) for the broken commit   28459ba1f4.
* | 37853fecc3 Accepting request 1183029 from Base:System
|/
* 638de11012 - Don't automatically clean unmodified config files up (bsc#1226415)
* 369c023c24 reorder one more time...
* ffa9f0ac80 reorder the runtime deps of the testsuite package so the format_spec_file thingy stop screwing up the spec file...
* 12c1190a79 fix rev 1529: the devel packages are really needed by the testsuite script to install the dlopened libs into the image
* ca8e7f54ce - systemd.spec: move a misplaced %endif in the testsuite sub-package.
* b7944f5b14 - Merge systemd-coredump back into the main package (bsc#1091684)
* 3fa0dea84a - Don't pull the devel packages in when installing the testsuite package.
2024-07-10 18:52:29 +02:00
Daan De Meyer
dedd712dd9 TEST-06-SELINUX: Various fixes
- Stop installing the policy in the initramfs as it's not really
supported anyway (https://github.com/fedora-selinux/selinux-policy/issues/2221)
- Stop relabeling on first boot and prefer to do it at image build time
- Disable mkosi relabeling by default but enable it in CI
- Build image as root in CI so the SELinux relabeling works properly
2024-07-10 18:52:29 +02:00
Daan De Meyer
e842deeac6 test: Switch to ncat instead of nc
ncat is available in CentOS Stream 9 without having to enable EPEL.
2024-07-10 18:52:29 +02:00
Daan De Meyer
3e3a15c06e TEST-55-OOMD: Switch to stress-ng
stress-ng is available in OpenSUSE and in CentOS Stream without needing
EPEL so let's switch to it instead of stress.
2024-07-10 18:52:26 +02:00
Daan De Meyer
0443f7e722
Merge pull request #33683 from DaanDeMeyer/sysext
mkosi: Build a sysext if SYSEXT=1 is specified
2024-07-10 13:21:12 +02:00
Daan De Meyer
612a98a065 mkosi: Build a sysext if SYSEXT=1 is specified 2024-07-10 10:55:33 +02:00
Daan De Meyer
73b3179a53 mkosi: Check for configured build directory if WIPE=1
Otherwise trying to use --wipe might fail if the build directory
has not been configured but is not empty either.
2024-07-10 10:08:20 +02:00
Daan De Meyer
20345a86b7 mkosi: Adapt configuration to take into account configuration rework
In https://github.com/systemd/mkosi/pull/2847, the '@' specifier is
removed, CLI arguments take priority over configuration files again
and the "main" image is defined at the top level instead of in
mkosi.images/. Additionally, not every setting from the top level
configuration is inherited by the images in mkosi.images/ anymore,
only settings which make sense to be inherited are inherited.

This commit gets rid of all the usages of '@', moves the "main" image
configuration from mkosi.images/system to the top level and gets rid
of various hacks we had in place to deal with quirks of the old
configuration parsing logic.

We also remove usages of Images= and --append as these options are
removed by the mentioned PR.
2024-07-09 08:07:09 +02:00
Daan De Meyer
6f1f13ca9f mkosi: Enable hyperscale-packages-experimental for CentOS
This gets us a kernel with btrfs support.
2024-06-27 12:50:41 +02:00
Daan De Meyer
8919f86f57 mkosi: Sanitizer improvements
- Let's set the environment on the kernel command line so it applies
to initrd and main system.
- Let's add the necessary wrappers that are also added in test-functions.
Unlike test-functions we don't use gcc/clang to get the library path as
that requires installing gcc/clang in the initrd.
- Let's drop the hack to get journald writing to the console and have
it write to kmsg instead. We'll get the output either way.
- Stop removing libstdc++ and sanitizer libraries from Arch Linux
initrds and other images as it's required by the sanitizer libraries.
- Add a workaround for specifying extra meson options for opensuse
- Add a leak sanitizer suppression file as a workaround for a false
positive leak in verify_selinuxmnt() in libselinux. We do a soname match
because the stacktrace can't be properly symbolized on Debian.
2024-05-31 17:26:13 +02:00
Daan De Meyer
074ac66e88 Revert "mkosi: Sanitizer improvements"
This reverts commit aef13ad029.
2024-05-30 14:50:21 +02:00
Daan De Meyer
aef13ad029 mkosi: Sanitizer improvements
- Let's set the environment on the kernel command line so it applies
to initrd and main system.
- Let's add the necessary wrappers that are also added in test-functions.
Unlike test-functions we don't use gcc/clang to get the library path as
that requires installing gcc/clang in the initrd.
- Let's drop the hack to get journald writing to the console and have
it write to kmsg instead. We'll get the output either way.
- Stop removing libstdc++ and sanitizer libraries from Arch Linux
initrds and other images as it's required by the sanitizer libraries.
- Add a workaround for specifying extra meson options for opensuse
- Add a leak sanitizer suppression file as a workaround for a false
positive leak in verify_selinuxmnt() in libselinux. We do a soname match
because the stacktrace can't be properly symbolized on Debian.
2024-05-30 12:47:45 +02:00
Daan De Meyer
3c367df7c6 mkosi: Switch from btrfs to ext4
Mounting multiple btrfs filesystems with the same fsid only works
properly from kernel 6.7 onwards. Let's switch to ext4 for now which
does support this.
2024-05-29 14:10:50 +02:00
Daan De Meyer
ecf8468dd4 mkosi: Update fedora to latest
We drop the Fedora 40 job as the latest rawhide spec introduced
dependencies that are not available in Fedora 40.
2024-05-14 20:44:39 +02:00
Daan De Meyer
1781de18ab mkosi: Drop usage of updates-testing for Fedora 40
The util-linux update we need has migrated to the stable repositories.
2024-05-09 11:03:39 +02:00
Daan De Meyer
b1670c52ad mkosi: Switch to fedora 40
Enable updates-testing and use the most recent mirror to make sure
we get util-linux 2.40.1 which contains a crucial fix to make sure
the serial terminal in virtual machines works properly.
2024-05-07 11:51:29 +02:00
Daan De Meyer
ffda3c3de9 mkosi: Disable ext4's orphan_file feature for centos images
Not supported by e2fsck from centos. We also disable building repart
from source in CI as running it from the build directory means repart
will run mkfs.ext4 from the host which doesn't know about the orphan_file
feature causing it to fail.
2024-05-06 10:56:45 +02:00
Daan De Meyer
4d0f1451b5 Build distribution packages in mkosi
Instead of running meson install and hoping for the best, let's build
distribution packages from the downstream packaging specs. This gets
us the following:

- Vastly simplified mkosi scripts since we don't need a separate initrd
  image anymore but can just reuse the default mkosi initrd.
- Almost everything can move to the base image as its not the basis
  anymore for the initrd and as such we don't need to care about the
  size anymore.
- The systemd packages that get pulled in as dependencies of other
  packages get properly uninstalled and replaced with our packages that
  we built instead of just installing on top of an existing systemd
  installation with no guarantee that everything from that previous
  installation was removed.
- Much better testing coverage as what we're testing is much closer
  to what will actually be deployed in distributions.
- Immediate feedback if something we change breaks distribution packaging
- We get integration with the distribution for free as we'll automatically
  use the proper directories and such instead of having to hack this
  into a mkosi build script.
- ...
2024-03-07 10:47:19 +01:00
Daan De Meyer
bcb335ac68 Update to mkosi v19
- Use mkosi.images/ instead of mkosi.presets/
- Use the .chroot suffix to run scripts in the image
- Use BuildSources= match for the kernel build
- Move 10-systemd.conf to mkosi.conf and rely on mkosi.local.conf
  for local configuration
2023-11-28 19:54:58 +01:00
Daan De Meyer
74b45fe43f Set default value for tools tree
This allows overriding it from CLI or previous config file if
required.
2023-11-09 12:10:11 +01:00