Currently, ExcludeFiles= supports excluding directories on the host
from being copied. Let's extend this to also support preventing files
from being copied into specific directories in the partition by adding
a new option ExcludeFilesTarget=. An example where this is useful is
when setting up btrfs subvolumes in the top level that are intended to
be mounted into specific locations, so /usr would be stored in @usr,
/home in @home, .... To accomplish this, we need to copy /usr to @usr
and prevent any files from being copied into /usr in the partition,
which with this commit, we'd be able to do as follows:
```
[Partition]
CopyFiles=/usr:@usr
ExcludeFilesTarget=/usr
```
In some cases, we want to exclude a directory's contents but not
the directory itself. In other cases, we want to exclude a directory
and its contents. Let's extend the denylist logic in copy.h to support
both by changing the denylist from a set to hashmap so we can store the
deny type as the value.
We also modify the repart ExcludeFiles= option to make use of this. If
a directory to exclude ends with a "/", we'll only exclude its contents.
Otherwise, we'll exclude the full directory.
Fixes#25780.
> Man page: crypttab.5
> Issue 1: Missing fullstop
> Issue 2: I<cipher=>, I<hash=>, I<size=> → B<cipher=>, B<hash=>, B<size=>
>
> "Force LUKS mode\\&. When this mode is used, the following options are "
> "ignored since they are provided by the LUKS header on the device: "
> "I<cipher=>, I<hash=>, I<size=>"
Seems OK to me. The full stop is there and has been for at least a few years. And we use <option> for the markup, which is appropriate here.
> Man page: crypttab.5
> Issue 1: Missing fullstop
> Issue 2: I<cipher=>, I<hash=>, I<keyfile-offset=>, I<keyfile-size=>, I<size=> → B<cipher=>, B<hash=>, B<keyfile-offset=>, B<keyfile-size=>, B<size=>
>
> "Use TrueCrypt encryption mode\\&. When this mode is used, the following "
> "options are ignored since they are provided by the TrueCrypt header on the "
> "device or do not apply: I<cipher=>, I<hash=>, I<keyfile-offset=>, I<keyfile-"
> "size=>, I<size=>"
Same.
> Man page: journalctl.1
> Issue 1: make be → may be
Fixed.
> Issue 2: below\\&. → below:
Fixed.
> Man page: journalctl.1
> Issue: Colon at the end?
>
> "The following commands are understood\\&. If none is specified the default "
> "is to display journal records\\&."
> msgstr ""
> "Die folgenden Befehle werden verstanden\\&. Falls keiner festgelegt ist, ist "
> "die Anzeige von Journal-Datensätzen die Vorgabe\\&."
This is a bit awkward, but I'm not sure how to fix it.
> Man page: kernel-install.8
> Issue: methods a fallback → methods fallback
It was correct, but I added a comma to make the sense clearer.
> Man page: loader.conf.5
> Issue 1: secure boot variables → Secure Boot variables
> Issue 2: one → one for (multiple times)
>
> "Supported secure boot variables are one database for authorized images, one "
> "key exchange key (KEK) and one platform key (PK)\\&. For more information, "
> "refer to the \\m[blue]B<UEFI specification>\\m[]\\&\\s-2\\u[2]\\d\\s+2, "
> "under Secure Boot and Driver Signing\\&. Another resource that describe the "
> "interplay of the different variables is the \\m[blue]B<EDK2 "
> "documentation>\\m[]\\&\\s-2\\u[3]\\d\\s+2\\&."
"one of" would sound strange. "One this and one that" is OK.
> Man page: loader.conf.5
> Issue: systemd-boot → B<systemd-boot>(7)
Fixed.
> Man page: logind.conf.5
> Issue: systemd-logind → B<systemd-logind>(8)
We use <filename>systemd-logind</> on subsequent references… I think that's good enough.
> Man page: nss-myhostname.8
> Issue: B<getent> → B<getent>(1)
Fixed.
> Man page: nss-resolve.8
> Issue: B<systemd-resolved> → B<systemd-resolved>(8)
The first reference does this, subsequent are shorter.
> Man page: os-release.5
> Issue: Portable Services → Portable Services Documentation?
Updated.
> Man page: pam_systemd_home.8
> Issue: auth and account use "reason", while session and password do not?
Reworded.
> Man page: portablectl.1
> Issue: In systemd-portabled.service(8): Portable Services Documentation
Updated.
> Man page: repart.d.5
> Issue: The partition → the partition
Fixed.
> Man page: repart.d.5
> Issue: B<systemd-repart> → B<systemd-repart>(8)
The first reference does this. I also change this one, because it's pretty far down in the text.
> Man page: systemd.1
> Issue: kernel command line twice?
>
> "Takes a boolean argument\\&. If false disables importing credentials from "
> "the kernel command line, qemu_fw_cfg subsystem or the kernel command line\\&."
Apparently this was fixed already.
> Man page: systemd-boot.7
> Issue: enrollement → enrollment
Fixed.
> Man page: systemd-cryptenroll.1
> Issue: multiple cases: any specified → the specified
Reworded.
> Man page: systemd-cryptenroll.1
> Issue: If this this → If this
Fixed tree-wide.
> Man page: systemd-cryptsetup-generator.8
> Issue: and the initrd → and in the initrd
"Is honoured by the initrd" is OK, because we often speak about the initrd as a single unit. But in the same paragraph we also used "in the initrd", which makes the other use look sloppy. I changed it to "in the initrd" everywhere in that file.
> Man page: systemd.directives.7
> Issue: Why are these two quoted (but not others)?
>
> "B<\\*(Aqh\\*(Aq>"
>
> B<\\*(Aqs\\*(Aq>"
>
> "B<\\*(Aqy\\*(Aq>"
This is autogenerated from files… We use slightly different markup in different files, and it's just too hard to make it consistent. We gave up on this.
> Man page: systemd.exec.5
> Issue 1: B<at>(1p) → B<at>(1)
> Issue 2: B<crontab>(1p) → B<crontab>(1)
Fixed.
> Man page: systemd.exec.5
> Issue: B<select()> → B<select>(2)
Fixed.
> Man page: systemd.exec.5
> Issue: qemu → B<qemu>(1)
The man page doesn't seem to be in any of the canonical places on the web.
I added a link to online docs.
> Man page: systemd.exec.5
> Issue: variable → variables
Seems to be fixed already.
> Man page: systemd-integritysetup-generator.8
> Issue: systemd-integritysetup-generator → B<systemd-integritysetup-generator>
I changed <filename> to <command>.
> Man page: systemd-integritysetup-generator.8
> Issue: superfluous comma at the end
Already fixed.
> Man page: systemd-measure.1
> Issue: (see B<--pcr-bank=>) below → (see B<--pcr-bank=> below)
Reworded.
> Man page: systemd-measure.1
> Issue: =PATH> → =>I<PATH>
Fixed.
> Man page: systemd-measure.1.po
> Issue: B<--bank=DIGEST> → B<--bank=>I<DIGEST>
Fixed.
> Man page: systemd.netdev.5
> Issue: os the → on the
Appears to have been fixed already.
> Man page: systemd.netdev.5
> Issue: Onboard → On-board (as in previous string)
Updated.
> Man page: systemd.network.5
> Issue: B<systemd-networkd> -> B<systemd-networkd>(8)
First reference does this, subsequent do not.
> Man page: systemd.network.5
> Issue: B<netlabelctl> → B<netlabelctl>(8)
First reference does this, subsequent do not.
> Man page: systemd.network.5
> Issue: Missing verb (aquired? configured?) in the half sentence starting with "or by a "
I dropped the comma.
> Man page: systemd-nspawn.1
> Issue: All host users outside of that range → All other host users
Reworded.
> # FIXME no effect → no effect\\&.
> #. type: Plain text
> #: archlinux debian-unstable fedora-rawhide mageia-cauldron opensuse-tumbleweed
> msgid ""
> "Whichever ID mapping option is used, the same mapping will be used for users "
> "and groups IDs\\&. If B<rootidmap> is used, the group owning the bind "
> "mounted directory will have no effect"
A period is added. Not sure if there's some other issue.
> Man page: systemd-oomd.service.8
> Issue: B<systemd> → B<systemd>(1)
Done.
> Man page: systemd.path.5
> Issue 1: B<systemd.exec>(1) → B<systemd.exec>(5)
> Issue 2: This section does not (yet?) exist
Fixed.
> Man page: systemd-pcrphase.service.8
> Issue 1: indicate phases into TPM2 PCR 11 ??
> Issue 2: Colon at the end of the paragraph?
Fixed.
> Man page: systemd-pcrphase.service.8
> Issue: final boot phase → final shutdown phase?
Updated.
> Man page: systemd-pcrphase.service.8
> Issue: for the the → for the
Fixed tree-wide.
> Man page: systemd-portabled.service.8
> Issue: In systemd-portabled.service(8): Portable Services Documentation
Updated.
> Man page: systemd-pstore.service.8
> Issue: Here and the following paragraphs: . → \\&. // Upstream: What does this comment mean? // You normally write \\&. for a full dot (full stop etc.); here you write only "." (i.e. a plain dot).
>
> "and we look up \"localhost\", nss-dns will send the following queries to "
> "systemd-resolved listening on 127.0.0.53:53: first \"localhost.foobar.com\", "
> "then \"localhost.barbar.com\", and finally \"localhost\". If (hopefully) the "
> "first two queries fail, systemd-resolved will synthesize an answer for the "
> "third query."
Looks all OK to me.
> Man page: systemd.resource-control.5
> Issue: Missing closing bracket after link to Control Groups version 1
Fixed.
> Man page: systemd-sysext.8
> Issue: In systemd-portabled.service(8): Portable Services Documentation
Updated.
> Man page: systemd.timer.5
> Issue 1: B<systemd.exec>(1) → B<systemd.exec>(5)
> Issue 2: This section does not (yet?) exist
Fixed.
> Man page: systemd.unit.5
> Issue: that is → that are
Fixed.
> Man page: systemd-veritysetup-generator.8
> Issue: systemd-veritysetup-generator → B<systemd-veritysetup-generator>
>
> "systemd-veritysetup-generator implements B<systemd.generator>(7)\\&."
>
> "systemd-veritysetup-generator understands the following kernel command line "
> "parameters:"
Updated.
> Man page: systemd-volatile-root.service.8
> Issue: initrdyes → Initrd
Fixed.
> Man page: sysupdate.d.5
> Issue: : → \\&. (As above in TRANSFER)
Updated.
> Man page: sysupdate.d.5
> Issue: some → certain
Updated.
> Man page: sysupdate.d.5
> Issue 1: i\\&.e\\& → I\\&.e\\&
Fixed.
> Issue 2: the image → the system
"image" seems correct.
> Man page: tmpfiles.d.5
> Issue: systemd-tmpfiles → B<systemd-tmpfiles>(8)
Updated.
Instead of having Minimize= take a boolean let's allow for two
different ways to enable it. "best" means we want the most minimal
image possible, which currently is only possible for read-only
filesystems but can be extended in the future with bisection
to find the most minimal possible size.
We also add "guess", which is the current behavior, where we
populate once and use the sparse size to make a reasonable guess
on a size that fits all the sources without needing to O(log(n))
tries to find the most minimal size.
So, i think "erofs" is probably the better, more modern alternative to
"squashfs". Many of the benefits don't matter too much to us I guess,
but there's one thing that stands out: erofs has a UUID in the
superblock, squashfs has not. Having an UUID in the superblock matters
if the file systems are used in an overlayfs stack, as overlayfs uses
the UUIDs to robustly and persistently reference inodes on layers in
case of metadata copy-up.
Since we probably want to allow such uses in overlayfs as emplyoed by
sysext (and the future syscfg) we probably should ramp up our erofs game
early on. Hence let's natively support erofs, test it, and in fact
mention it in the docs before squashfs even.
e.g. vfat doesn't support symlinks, sockets, fifos, etc so let's ignore
any copy failures related to unsupported file types when populating
filesystems.
Instead of requiring users to guess the required space for partitions
populated with CopyFiles=, let's make an educated guess ourselves. We
can populate the filesystem once in a very large sparse file and see
how much data is actually used as a good indicator of the required size.
For use with sysupdate or other systemd tooling, it's useful to be
able to generate split artifacts from disk images, where each
partition is written to a separate file. Let's support this with
a --split switch for repart and a SplitName= configuration option.
--split enables split artifacts generation, and SplitName= configures
for which partition to generate split artifacts, and which suffix to
add to the split artifact name.
For SplitName=, we add support for some extra specifiers, more specifically
the partition Type UUID and the partition UUID.
To make this work, we have to set up everything in a temporary
directory tree that we can pass to mksquashfs as a single directory.
To make the most common scenario more efficient, we skip the temporary
setup directory if we only get a single source tree destined to root
in the squashfs filesystem.
This commit adds a new Verity= setting to repart definition files
with two possible values: "data" and "hash".
If Verity= is set to "data", repart works as before, and populates
the partition with the content from CopyBlocks= or CopyFiles=.
If Verity= is set to "hash", repart will try to find a matching
data partition with Verity=data and equal values for CopyBlocks=
or CopyFiles=, Format= and MakeDirectories=. If a matching data
partition is found, repart will generate verity hashes for that
data partition in the verity partition. The UUID of the data
partition is set to the first 128 bits of the verity root hash. The
UUID of the hashes partition is set to the final 128 bits of the
verity root hash.
Fixes#24559
Documentation is licensed under LGPL-2.1-or-later.
Scripts are MIT to facilitate reuse.
Examples are relicensed to CC0-1.0 to maximise copy-and-paste
for users, with permission from authors.
This is useful for provisioning initially empty secondary A/B root file
systems. We don't want those to ever be considered for automatic
mounting, for example in "systemd-nspawn --image=", hence we should
create them with the No-Auto flag turned on. Once a file system image is
dropped into the partition the flag may be turned off by the updater
tool, so that it is considered from then on.
Thew new option for this is called NoAuto. I dislike negated options
like this, but this is taken from the naming in the spec, which in turn
inherited the name from the same flag for Microsoft Data Partitions. To
minimize confusion, let's stick to the name hence.
This might be useful for CopyFiles=, to reference some subdir of $TMP in
a generic way. This allows us to use the new common
system_and_tmp_specifier_table[].
Let's make the GPT partition flags configurable when creating new
partitions. This is primarily useful for the read-only flag (which we
want to set for verity enabled partitions).
This adds two settings for this: Flags= and ReadOnly=, which strictly
speaking are redundant. The main reason to have both is that usually the
ReadOnly= setting is the one wants to control, and it' more generic.
Moreover we might later on introduce inherting of flags from CopyBlocks=
partitions, where one might want to control most flags as is except for
the RO flag and similar, hence let's keep them separate.
When using systemd-repart as an installer that replicates the install
medium on another medium it is useful to reference the root
partition/usr partition or verity data that is currently booted, in
particular in A/B scenarios where we have two copies and want to
reference the one we currently use. Let's add a CopyBlocks=auto for this
case: for a partition that uses that we'll copy a suitable partition
from the host.
CopyBlocks=auto finds the partition to copy like this: based on the
configured partition type uuid we determine the usual mount point (i.e.
for the /usr partition type we determine /usr/, and so on). We then
figure out the block device behind that path, through dm-verity and
dm-crypt if necessary. Finally, we compare the partition type uuid of
the partition found that way with the one we are supposed to fill and
only use it if it matches (the latter is primarily important on
dm-verity setups where a volume is likely backed by two partitions and
we need to find the right one).
This is particularly fun to use in conjunction with --image= (where
we'll restrict the device search onto the specify device, for security
reasons), as this allows "duplicating" an image like this:
# systemd-repart --image=source.raw --empty=create --size=auto target.raw
If the right repart data is embedded into "source.raw" this will be able
to create and initialize a partition table on target.raw that carrries
all needed partitions, and will stream the source's file systems onto it
as configured.
So far we already had the CopyFiles= option in systemd-repart drop-in
files, as a mechanism for populating freshly formatted file systems with
files and directories. This adds MakeDirectories= in similar style, and
creates simple directories as listed. The option is of course entirely
redundant, since the same can be done with CopyFiles= simply by copying
in a directory. It's kinda nice to encode the dirs to create directly in
the drop-in files however, instead of providing a directory subtree to
copy in somehere, to make the files more self-contained — since often
just creating dirs is entirely sufficient.
The main usecase for this are GPT OS images that carry only a /usr/
tree, and for which a root file system is only formatted on first boot
via repart. Without any additional CopyFiles=/MakeDirectories=
configuration these root file systems are entirely empty of course
initially. To mount in the /usr/ tree, a directory inode for /usr/ to
mount over needs to be created. systemd-nspawn will do so automatically
when booting up the image, as will the initrd during boot. However, this
requires the image to be writable – which is OK for npawn and
initrd-based boots, but there are plenty tools where read-only operation
is desirable after repart ran, before the image was booted for the first
time. Specifically, "systemd-dissect" opens the image in read-only to
inspect its contents, and this will only work of /usr/ can be properly
mounted. Moreover systemd-dissect --mount --read-only won't succeed
either if the fs is read-only.
Via MakeDirectories= we now provide a way that ensures that the image
can be mounted/inspected in a fully read-only way immediately after
systemd-repart completed. Specifically, let's consider a GPT disk image
shipping with a file usr/lib/repart.d/50-root.conf:
[Partition]
Type=root
Format=btrfs
MakeDirectories=/usr
MakeDirectories=/efi
With this in place systemd-repart will create a root partition when run,
and add /usr and /efi into it as directory inods. This ensures that the
whole image can then be mounted truly read-only anf /usr and /efi can be
overmounted by the /usr partition and the ESP.
This is similar to the --image= switch in the other tools, like
systemd-sysusers or systemd-tmpfiles, i.e. it apply the configuration
from the image to the image.
This is particularly useful for downloading minimized GPT image, and
then extending it to the desired size via:
# systemd-repart --image=foo.image --size=5G
This specifes two new optional fields for /etc/os-release:
IMAGE_VERSION= and IMAGE_ID= that are supposed to identify the image of
the current booted system by name and version.
This is inspired by the versioning stuff in
https://github.com/systemd/mkosi/pull/683.
In environments where pre-built images are installed and updated as a
whole the existing os-release version/distro identifier are not
sufficient to describe the system's version, as they describe only the
distro an image is built from, but not the image itself, even if that
image is deployed many times on many systems, and even if that image
contains more resources than just the RPMs/DEBs.
In particular, "mkosi" is a tool for building disk images based on
distro RPMs with additional resources dropped in. The combination of all
of these together with their versions should also carry an identifier
and version, and that's what IMAGE_VERSION= and IMAGE_ID= is supposed to
be.
This allows copying in arbitrary file systems on the block level into
newly created partitions.
Usecase: simple replicating OS installers or OS image builders.
Strictly speaking this is a compat breakage, but given the tool was
added only in the last release, let's try to sail under the radar, and
fix this early before anyone notices it wasn't supported always.
