mirrors/systemd
0
0
Fork 0
mirror of https://github.com/systemd/systemd.git synced 2024-11-23 02:03:37 +08:00
Code Issues Packages Projects Releases Wiki Activity

sysupdate: Introduce optional features

Browse Source 
Optional features allow distros to define sets of transfers that can
be enabled or disabled by the system administrator. This is useful for
situations where a distro may want to ship some resources version-locked
to the core OS, but many people have no need for the resource, such as:
development tools/compilers, drivers for specialized hardware, language
packs, etc

We also rename sysupdate.d/*.conf -> sysupdate.d/*.transfer, because
now there are more than one type of definition in sysupdate.d/. For
backwards compat, we still load *.conf files as long as no *.transfer
files are found and the *.conf files don't try to declare themselves
as part of any features

Fixes https://github.com/systemd/systemd/issues/33343
Fixes https://github.com/systemd/systemd/issues/33344
This commit is contained in:
Adrian Vovk 2024-06-22 14:48:50 -04:00
parent 3e18762123
commit e1384cfb09
No known key found for this signature in database
GPG Key ID: 90A7B546533E15FB
11 changed files with 779 additions and 100 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
TODO
View File

@ -1429,9 +1429,6 @@ Features:
- "systemd-sysupdate update --all" support, that iterates through all components
defined on the host, plus all images installed into /var/lib/machines/,
/var/lib/portable/ and so on.
- figure out what to do about system extensions (i.e. they need to imply an
update component, since otherwise sysupdate.d/ files would override the
host's update files.)
- Allow invocation with a single transfer definition, i.e. with
--definitions= pointing to a file rather than a dir.
- add ability to disable implicit decompression of downloaded artifacts,

1
man/rules/meson.build
View File

@ -1192,6 +1192,7 @@ manpages = [
['systemd.unit', '5', [], ''],
['systemd.v', '7', [], ''],
['sysupdate.d', '5', [], 'ENABLE_SYSUPDATE'],
['sysupdate.features', '5', [], 'ENABLE_SYSUPDATE'],
['sysusers.d', '5', [], 'ENABLE_SYSUSERS'],
['telinit', '8', [], 'HAVE_SYSV_COMPAT'],
['timedatectl', '1', [], 'ENABLE_TIMEDATECTL'],

83
man/sysupdate.d.xml
View File

@ -23,21 +23,23 @@
<refsynopsisdiv>
<para><simplelist>
<member><filename>/etc/sysupdate.d/*.conf</filename></member>
<member><filename>/run/sysupdate.d/*.conf</filename></member>
<member><filename>/usr/local/lib/sysupdate.d/*.conf</filename></member>
<member><filename>/usr/lib/sysupdate.d/*.conf</filename></member>
<member><filename>/etc/sysupdate.d/*.transfer</filename></member>
<member><filename>/run/sysupdate.d/*.transfer</filename></member>
<member><filename>/usr/local/lib/sysupdate.d/*.transfer</filename></member>
<member><filename>/usr/lib/sysupdate.d/*.transfer</filename></member>
</simplelist></para>
</refsynopsisdiv>
<refsect1>
<title>Description</title>
<para><filename>sysupdate.d/*.conf</filename> files describe how specific resources on the local system
shall be updated from a remote source. Each such file defines one such transfer: typically a remote
HTTP/HTTPS resource as source; and a local file, directory or partition as target. This may be used as a
simple, automatic, atomic update mechanism for the OS itself, for containers, portable services or system
extension images — but in fact may be used to update any kind of file from a remote source.</para>
<para>These files describe how specific resources on the local system shall be updated from a remote
source.
Each such file defines one such transfer: typically a remote HTTP/HTTPS resource as source; and a local
file, directory or partition as target.
This may be used as a simple, automatic, atomic update mechanism for the OS itself, for containers,
portable services or system extension images — but in fact may be used to update any kind of file from a
remote source.</para>
<para>The
<citerefentry><refentrytitle>systemd-sysupdate</refentrytitle><manvolnum>8</manvolnum></citerefentry>
@ -48,14 +50,22 @@
versions, in order to implement flexible update schemes, e.g. A/B updating (or a superset thereof,
e.g. A/B/C, A/B/C/D, …).</para>
<para>Each <filename>*.conf</filename> file defines one transfer, i.e. describes one resource to
update. Typically, multiple of these files (i.e. multiple of such transfers) are defined together, and
<para>Each <filename>*.transfer</filename> file defines one transfer, i.e. describes one resource to
update.
Typically, multiple of these files (i.e. multiple of such transfers) are defined together, and
are bound together by a common version identifier in order to update multiple resources at once on each
update operation, for example to update a kernel, a root file system and a Verity partition in a single,
combined, synchronized operation, so that only a combined update of all three together constitutes a
complete update.</para>
complete update.
We'll call such a collection of transfers a target.
<command>systemd-sysupdate</command> always operates on a single target.</para>
<para>Each <filename>*.conf</filename> file contains three sections: [Transfer], [Source] and [Target].</para>
<para>Transfers may be grouped together into sets that can be individually enabled or disabled by the
system administrator, called "Optional Features":
<citerefentry><refentrytitle>sysupdate.features</refentrytitle><manvolnum>5</manvolnum></citerefentry>.
</para>
<para>Each <filename>*.transfer</filename> file contains three sections: [Transfer], [Source] and [Target].</para>
</refsect1>
<refsect1>
@ -106,8 +116,8 @@
<para>An update can only complete if the relevant URLs provide their resources for the same version,
i.e. for the same value of <literal>@v</literal>.</para>
<para>The above may be translated into three <filename>*.conf</filename> files in
<filename>sysupdate.d/</filename>, one for each resource to transfer. The <filename>*.conf</filename>
<para>The above may be translated into three <filename>*.transfer</filename> files in
<filename>sysupdate.d/</filename>, one for each resource to transfer. The <filename>*.transfer</filename>
files configure the type of download, and what place to write the download to (i.e. whether to a
partition or a file in the file system). Most importantly these files contain the URL, partition name and
filename patterns shown above that describe how these resources are called on the source and how they
@ -130,11 +140,11 @@
lists file names and their SHA256 hashes.</para></listitem>
</itemizedlist>
<para>Transfers are done in the alphabetical order of the <filename>.conf</filename> file names they are
<para>Transfers are done in the alphabetical order of the <filename>.transfer</filename> file names they are
defined in. First, the resource data is downloaded directly into a target file/directory/partition. Once
this is completed for all defined transfers, in a second step the files/directories/partitions are
renamed to their final names as defined by the target <varname>MatchPattern=</varname>, again in the
order the <filename>.conf</filename> transfer file names dictate. This step is not atomic, however it is
order the <filename>.transfer</filename> transfer file names dictate. This step is not atomic, however it is
guaranteed to be executed strictly in order with suitable disk synchronization in place. Typically, when
updating an OS one of the transfers defines the entry point when booting. Thus it is generally a good idea
to order the resources via the transfer configuration file names so that the entry point is written
@ -519,6 +529,39 @@
<xi:include href="version-info.xml" xpointer="v257"/></listitem>
</varlistentry>
<varlistentry>
<term><varname>Features=</varname></term>
<listitem><para>A space-separated list of
<citerefentry><refentrytitle>sysupdate.features</refentrytitle><manvolnum>5</manvolnum></citerefentry>
that this transfer belongs to, by name.
This option may be specified more than once, in which case the specified list of features is merged.
If the empty string is assigned to this option, the list is reset and all prior assignments will have
no effect.
For example: <literal>Features=foo bar</literal> specifies that the transfer belongs to
<literal>foo.feature</literal> and <literal>bar.feature</literal>.</para>
<para>If the list of features is empty, then this transfer is always used.
If this transfer belongs to more than one feature, then it will be used if <emphasis>any</emphasis>
one of the listed features is enabled.
A name that does not correspond to a defined feature will resolve to an implicit feature that is
always disabled.</para>
<xi:include href="version-info.xml" xpointer="v257"/></listitem>
</varlistentry>
<varlistentry>
<term><varname>RequisiteFeatures=</varname></term>
<listitem><para>This is like <varname>Features=</varname>, except that <emphasis>all</emphasis>
features listed here must be enabled for this transfer to be enabled.
If both options are specified, then they both apply: the transfer will be enabled only if all
features specified here are enabled, and at least one feature listed in <varname>Features=</varname>
is enabled.</para>
<xi:include href="version-info.xml" xpointer="v257"/></listitem>
</varlistentry>
</variablelist>
</refsect1>
@ -869,7 +912,7 @@
partition, and a unified kernel image to update as one. This example is an extension of the example
discussed earlier in this man page.</para>
<para><programlisting># /usr/lib/sysupdate.d/50-verity.conf
<para><programlisting># /usr/lib/sysupdate.d/50-verity.transfer
[Transfer]
ProtectVersion=%A
@ -899,7 +942,7 @@ ReadOnly=1</programlisting></para>
<citerefentry><refentrytitle>systemd-gpt-auto-generator</refentrytitle><manvolnum>8</manvolnum></citerefentry>
suggests).</para>
<para><programlisting># /usr/lib/sysupdate.d/60-root.conf
<para><programlisting># /usr/lib/sysupdate.d/60-root.transfer
[Transfer]
ProtectVersion=%A
@ -918,7 +961,7 @@ ReadOnly=1</programlisting></para>
<para>The above defines a matching transfer definition for the root file system.</para>
<para><programlisting># /usr/lib/sysupdate.d/70-kernel.conf
<para><programlisting># /usr/lib/sysupdate.d/70-kernel.transfer
[Transfer]
ProtectVersion=%A

338
man/sysupdate.features.xml Normal file
View File

@ -0,0 +1,338 @@
<?xml version='1.0'?>
<!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd">
<!-- SPDX-License-Identifier: LGPL-2.1-or-later -->
<refentry id="sysupdate.features" conditional='ENABLE_SYSUPDATE'
xmlns:xi="http://www.w3.org/2001/XInclude">
<refentryinfo>
<title>sysupdate.features</title>
<productname>systemd</productname>
</refentryinfo>
<refmeta>
<refentrytitle>sysupdate.features</refentrytitle>
<manvolnum>5</manvolnum>
</refmeta>
<refnamediv>
<refname>sysupdate.features</refname>
<refpurpose>Definition Files for Optional Features</refpurpose>
</refnamediv>
<refsynopsisdiv>
<para><simplelist>
<member><filename>/etc/sysupdate.d/*.feature</filename></member>
<member><filename>/run/sysupdate.d/*.feature</filename></member>
<member><filename>/usr/local/lib/sysupdate.d/*.feature</filename></member>
<member><filename>/usr/lib/sysupdate.d/*.feature</filename></member>
</simplelist></para>
</refsynopsisdiv>
<refsect1>
<title>Description</title>
<para>"Optional Features" are functionality provided by
<citerefentry><refentrytitle>systemd-sysupdate</refentrytitle><manvolnum>8</manvolnum></citerefentry>,
that allow a distribution to define sets of
<citerefentry><refentrytitle>sysupdate.d</refentrytitle><manvolnum>5</manvolnum></citerefentry>
transfer definitions that are intended to be enabled or disabled by the system administrator.</para>
<para>When a feature is enabled, transfers belonging to it will be considered when checking for and
downloading updates, when vacuuming, and in all other situations.
In effect, transfers belonging to a feature will always be updated in lock-step with the rest of their
target.
This is the primary difference an Optional Feature and a <command>systemd-sysupdate</command> component.
When a feature is disabled, its transfers will not be considered when checking for and downloading updates,
but <command>systemd-sysupdate</command> will still consider them while vacuuming and in other situations
where it needs to determine ownership over previously downloaded system resources.
<command>systemd-sysupdate</command> will clean up all instances of the feature's transfers whenever it
is disabled, effectively uninstalling it.</para>
<para>Optional Features are described by <filename>sysupdate.d/*.feature</filename> files, which are
defined below.
Transfers can declare that they belong to a feature via the <varname>Features=</varname> setting.
Feature definitions support drop-in files, which are most commonly used to override the
<varname>Enabled=</varname> setting).
They can also be masked out to hide the availability of the feature entirely.</para>
<para>Each <filename>*.feature</filename> file contains one section: [Feature].</para>
</refsect1>
<refsect1>
<title>[Feature] Section Options</title>
<para>This section defines general properties of this feature.</para>
<variablelist>
<varlistentry>
<term><varname>Description=</varname></term>
<listitem><para>A short human readable description of this feature.
This may be used as a label for this feature, so the string should meaningfully identify the feature
among the features available in <filename>sysupdate.d/</filename>.</para>
<xi:include href="version-info.xml" xpointer="v257"/></listitem>
</varlistentry>
<varlistentry>
<term><varname>Documentation=</varname></term>
<listitem><para>A user-presentable URL to documentation about this feature.
This setting supports specifier expansion; see below for details on supported specifiers.</para>
<xi:include href="version-info.xml" xpointer="v257"/></listitem>
</varlistentry>
<varlistentry>
<term><varname>AppStream=</varname></term>
<listitem><para>A URL to an
<ulink url="https://www.freedesktop.org/software/appstream/docs/chap-CatalogData.html">AppStream catalog</ulink>
XML file.
This may be used by software centers (such as GNOME Software or KDE Discover) to present rich
metadata about this feature.
This includes display names, chagnelogs, icons, and more.
This setting supports specifier expansion; see below for details on supported specifiers.</para>
<xi:include href="version-info.xml" xpointer="v257"/></listitem>
</varlistentry>
<varlistentry>
<term><varname>Enabled=</varname></term>
<listitem><para>Whether or not this feature is enabled. If unspecified, the feature is disabled
by default.</para>
<xi:include href="version-info.xml" xpointer="v257"/></listitem>
</varlistentry>
</variablelist>
</refsect1>
<refsect1>
<title>Specifiers</title>
<para>Specifiers may be used in the <varname>Documentation=</varname> and <varname>AppStream=</varname>
settings. The following expansions are understood:</para>
<table class='specifiers'>
<title>Specifiers available</title>
<tgroup cols='3' align='left' colsep='1' rowsep='1'>
<colspec colname="spec" />
<colspec colname="mean" />
<colspec colname="detail" />
<thead>
<row>
<entry>Specifier</entry>
<entry>Meaning</entry>
<entry>Details</entry>
</row>
</thead>
<tbody>
<xi:include href="standard-specifiers.xml" xpointer="a"/>
<xi:include href="standard-specifiers.xml" xpointer="A"/>
<xi:include href="standard-specifiers.xml" xpointer="b"/>
<xi:include href="standard-specifiers.xml" xpointer="B"/>
<xi:include href="standard-specifiers.xml" xpointer="H"/>
<xi:include href="standard-specifiers.xml" xpointer="l"/>
<xi:include href="standard-specifiers.xml" xpointer="m"/>
<xi:include href="standard-specifiers.xml" xpointer="M"/>
<xi:include href="standard-specifiers.xml" xpointer="o"/>
<xi:include href="standard-specifiers.xml" xpointer="v"/>
<xi:include href="standard-specifiers.xml" xpointer="w"/>
<xi:include href="standard-specifiers.xml" xpointer="W"/>
<xi:include href="standard-specifiers.xml" xpointer="T"/>
<xi:include href="standard-specifiers.xml" xpointer="V"/>
<xi:include href="standard-specifiers.xml" xpointer="percent"/>
</tbody>
</tgroup>
</table>
</refsect1>
<refsect1>
<title>Examples</title>
<example>
<title>Development Tools for Image-Based OS</title>
<para>We'll use the hypothetical "foobarOS" described in
<citerefentry><refentrytitle>sysupdate.d</refentrytitle><manvolnum>5</manvolnum></citerefentry> as our
example base OS.
The vast majority of foobarOS's users have no need for a compiler, build system, debugger, and other
such development tools to be part of their OS.
However, the developers of foobarOS itself need this build tooling to be available.
So, foobarOS needs to provide a system extension image (see
<citerefentry><refentrytitle>systemd-sysext</refentrytitle><manvolnum>8</manvolnum></citerefentry>)
containing these development tools, and this image must be updated in lock-step with the underlying
base OS.
This is a great use case for an optional OS feature, so let's define one:
</para>
<para><programlisting># /usr/lib/sysupdate.d/devel.feature
[Feature]
Description=Development Tools
Documentation=https://developer.example.com/foobarOS/getting-started
Enabled=false
</programlisting></para>
<para>The above defines the <literal>devel</literal> feature, and disables it by default.
Now let's a define a transfer that's associated with this feature:</para>
<para><programlisting># /usr/lib/sysupdate.d/50-devel.transfer
[Transfer]
Features=devel
ProtectVersion=%A
[Source]
Type=url-file
Path=https://download.example.com/
MatchPattern=foobarOS_@v_devel.raw.xz
[Target]
Type=regular-file
Path=/var/lib/extensions
MatchPattern=foobarOS_@v_devel.raw
Mode=0444
InstancesMax=2
</programlisting></para>
<para>With these two files, we have created a feature called <literal>devel</literal> that, when
enabled, will download and decompress the appropriate version of
<literal>https://download.example.com/foobarOS_@v_devel.raw.xz</literal> into
<literal>/var/lib/extensions/foobarOS_@v_devel.raw</literal> during each OS update.</para>
<para>The developers of foobarOS can enable the <literal>devel</literal> feature on their workstations
by creating the following drop-in:</para>
<para><programlisting># /etc/sysupdate.d/devel.feature.d/enable.conf
[Feature]
Enabled=true
</programlisting></para>
</example>
<example>
<title>Proprietary Drivers</title>
<para>Suppose that many of foobarOS's users have a GPU manufactured by the MVISUAL corporation.
Due to lack of documentation and difficulty in reverse-engineering the hardware, the open-source
drivers for MVISUAL GPUs are unable to make proper use of available graphics and compute performance.
MVISUAL provides a redistributable proprietary driver for their cards, and foobarOS's developers
distribute them to address their users' needs.</para>
<para>MVISUAL's driver has a couple different parts that must be installed for it to function: a UKI
addon to configure the kernel command-line, an initrd system extension image to add the MVISUAL kernel
module into the initrd, and a regular system extension image to add the proprietary OpenGL and Vulkan
userspace drivers.
All of these should be version-locked to the core OS.</para>
<para>Let's start by defining an optional feature named <literal>mvisual-driver</literal>:</para>
<para><programlisting># /usr/lib/sysupdate.d/mvisual-driver.feature
[Feature]
Description=MVISUAL Proprietary GPU Driver
Documentation=https://support.example.com/foobarOS/mvisual
AppStream=https://metadata.example.com/mvisual-driver-%A.xml.gz
</programlisting></para>
<para>Note that we define AppStream metadata for this feature, because we want software centers to
present it to end-users.
Next, let's define the corresponding transfers:</para>
<para><programlisting># /usr/lib/sysupdate.d/50-mvisual-userspace.transfer
[Transfer]
Features=mvisual-driver
ProtectVersion=%A
[Source]
Type=url-file
Path=https://download.example.com/
MatchPattern=foobarOS_@v_mvisual_userspace.raw.xz
[Target]
Type=regular-file
Path=/var/lib/extensions
MatchPattern=foobarOS_@v_mvisual.raw
Mode=0444
InstancesMax=2
</programlisting></para>
<para><programlisting># /usr/lib/sysupdate.d/70-mvisual-initrd.transfer
[Transfer]
Features=mvisual-driver
ProtectVersion=%A
[Source]
Type=url-file
Path=https://download.example.com/
MatchPattern=foobarOS_@v_mvisual_initrd.raw.xz
[Target]
Type=regular-file
Path=/EFI/Linux
PathRelativeTo=boot
MatchPattern=foobarOS_@v.efi.extra.d/foobarOS_mvisual.raw
Mode=0444
InstancesMax=2
</programlisting></para>
<para><programlisting># /usr/lib/sysupdate.d/90-mvisual-addon.transfer
[Transfer]
Features=mvisual-driver
ProtectVersion=%A
[Source]
Type=url-file
Path=https://download.example.com/
MatchPattern=foobarOS_@v_mvisual_addon.efi.xz
[Target]
Type=regular-file
Path=/EFI/Linux
PathRelativeTo=boot
MatchPattern=foobarOS_@v.efi.extra.d/foobarOS_mvisual.addon.efi
Mode=0444
InstancesMax=2
</programlisting></para>
</example>
<example>
<title>Intersecting Features</title>
<para>Suppose that MVISUAL releases special tooling to help a distribution developer troubleshoot
crashes in their proprietary driver.
Let's define a transfer:</para>
<para><programlisting># /usr/lib/sysupdate.d/50-mvisual-debugger.transfer
[Transfer]
RequisiteFeatures=devel mvisual-driver
ProtectVersion=%A
[Source]
Type=url-file
Path=https://download.example.com/
MatchPattern=foobarOS_@v_devel.raw.xz
[Target]
Type=regular-file
Path=/var/lib/extensions
MatchPattern=foobarOS_@v_devel.raw
Mode=0444
InstancesMax=2
</programlisting></para>
<para>This transfer will be used only if both the <literal>devel</literal> and
<literal>mvisual-driver</literal> features are enabled.</para>
</example>
</refsect1>
<refsect1>
<title>See Also</title>
<para><simplelist type="inline">
<member><citerefentry><refentrytitle>systemd</refentrytitle><manvolnum>1</manvolnum></citerefentry></member>
<member><citerefentry><refentrytitle>systemd-sysupdate</refentrytitle><manvolnum>8</manvolnum></citerefentry></member>
<member><citerefentry><refentrytitle>sysupdate.d</refentrytitle><manvolnum>5</manvolnum></citerefentry></member>
</simplelist></para>
</refsect1>
</refentry>

1
src/sysupdate/meson.build
View File

@ -2,6 +2,7 @@
systemd_sysupdate_sources = files(
'sysupdate-cache.c',
'sysupdate-feature.c',
'sysupdate-instance.c',
'sysupdate-partition.c',
'sysupdate-pattern.c',

121
src/sysupdate/sysupdate-feature.c Normal file
View File

@ -0,0 +1,121 @@
/* SPDX-License-Identifier: LGPL-2.1-or-later */
#include "conf-parser.h"
#include "hash-funcs.h"
#include "path-util.h"
#include "sysupdate-feature.h"
#include "sysupdate.h"
#include "web-util.h"
static Feature *feature_free(Feature *f) {
if (!f)
return NULL;
free(f->id);
free(f->description);
free(f->documentation);
free(f->appstream);
return mfree(f);
}
Feature *feature_new(void) {
Feature *f;
f = new(Feature, 1);
if (!f)
return NULL;
*f = (Feature) {
.n_ref = 1,
};
return f;
}
DEFINE_TRIVIAL_REF_UNREF_FUNC(Feature, feature, feature_free);
DEFINE_HASH_OPS_WITH_VALUE_DESTRUCTOR(feature_hash_ops,
char, string_hash_func, string_compare_func,
Feature, feature_unref);
static int config_parse_url_specifiers(
const char *unit,
const char *filename,
unsigned line,
const char *section,
unsigned section_line,
const char *lvalue,
int ltype,
const char *rvalue,
void *data,
void *userdata) {
char **s = ASSERT_PTR(data);
_cleanup_free_ char *resolved = NULL;
int r;
assert(rvalue);
if (isempty(rvalue)) {
*s = mfree(*s);
return 0;
}
r = specifier_printf(rvalue, NAME_MAX, specifier_table, arg_root, NULL, &resolved);
if (r < 0) {
log_syntax(unit, LOG_WARNING, filename, line, r,
"Failed to expand specifiers in %s=, ignoring: %s", lvalue, rvalue);
return 0;
}
if (!http_url_is_valid(resolved)) {
log_syntax(unit, LOG_WARNING, filename, line, 0,
"%s= URL is not valid, ignoring: %s", lvalue, rvalue);
return 0;
}
return free_and_replace(*s, resolved);
}
int feature_read_definition(Feature *f, const char *path, const char *const *dirs) {
assert(f);
ConfigTableItem table[] = {
{ "Feature", "Description", config_parse_string, 0, &f->description },
{ "Feature", "Documentation", config_parse_url_specifiers, 0, &f->documentation },
{ "Feature", "AppStream", config_parse_url_specifiers, 0, &f->appstream },
{ "Feature", "Enabled", config_parse_bool, 0, &f->enabled },
{}
};
_cleanup_free_ char *filename = NULL;
int r;
assert(path);
assert(dirs);
r = path_extract_filename(path, &filename);
if (r < 0)
return log_error_errno(r, "Failed to extract filename from path '%s': %m", path);
r = config_parse_many(
STRV_MAKE_CONST(path),
dirs,
strjoina(filename, ".d"),
arg_root,
"Feature\0",
config_item_table_lookup, table,
CONFIG_PARSE_WARN,
/* userdata= */ NULL,
/* stats_by_path= */ NULL,
/* drop_in_files= */ NULL);
if (r < 0)
return r;
*ASSERT_PTR(endswith(filename, ".feature")) = 0; /* Remove the file extension */
f->id = TAKE_PTR(filename);
return 0;
}

27
src/sysupdate/sysupdate-feature.h Normal file
View File

@ -0,0 +1,27 @@
/* SPDX-License-Identifier: LGPL-2.1-or-later */
#include "hash-funcs.h"
#include "macro.h"
#include "sysupdate-transfer.h"
typedef struct Feature {
unsigned n_ref;
char *id;
char *description;
char *documentation;
char *appstream;
bool enabled;
} Feature;
Feature *feature_new(void);
Feature *feature_ref(Feature *f);
Feature *feature_unref(Feature *f);
DEFINE_TRIVIAL_CLEANUP_FUNC(Feature*, feature_unref);
extern const struct hash_ops feature_hash_ops;
int feature_read_definition(Feature *f, const char *path, const char *const *conf_file_dirs);

83
src/sysupdate/sysupdate-transfer.c
View File

@ -28,6 +28,7 @@
#include "stdio-util.h"
#include "strv.h"
#include "sync-util.h"
#include "sysupdate-feature.h"
#include "sysupdate-pattern.h"
#include "sysupdate-resource.h"
#include "sysupdate-transfer.h"
@ -44,7 +45,6 @@ Transfer* transfer_free(Transfer *t) {
t->temporary_path = rm_rf_subvolume_and_free(t->temporary_path);
free(t->definition_path);
free(t->min_version);
strv_free(t->protected_versions);
free(t->current_symlink);
@ -93,12 +93,6 @@ Transfer* transfer_new(Context *ctx) {
return t;
}
static const Specifier specifier_table[] = {
COMMON_SYSTEM_SPECIFIERS,
COMMON_TMP_SPECIFIERS,
{}
};
static int config_parse_protect_version(
const char *unit,
const char *filename,
@ -466,11 +460,33 @@ static int config_parse_partition_flags(
return 0;
}
int transfer_read_definition(Transfer *t, const char *path) {
int r;
static bool transfer_decide_if_enabled(Transfer *t, Hashmap *known_features) {
assert(t);
/* Requisite feature disabled -> transfer disabled */
STRV_FOREACH(id, t->requisite_features) {
Feature *f = hashmap_get(known_features, *id);
if (!f || !f->enabled) /* missing features are implicitly disabled */
return false;
}
/* No features defined -> transfer implicitly enabled */
if (strv_isempty(t->features))
return true;
/* At least one feature enabled -> transfer enabled */
STRV_FOREACH(id, t->features) {
Feature *f = hashmap_get(known_features, *id);
if (f && f->enabled)
return true;
}
/* All listed features disabled -> transfer disabled */
return false;
}
int transfer_read_definition(Transfer *t, const char *path, const char **dirs, Hashmap *known_features) {
assert(t);
assert(path);
ConfigTableItem table[] = {
{ "Transfer", "MinVersion", config_parse_min_version, 0, &t->min_version },
@ -478,6 +494,8 @@ int transfer_read_definition(Transfer *t, const char *path) {
{ "Transfer", "Verify", config_parse_bool, 0, &t->verify },
{ "Transfer", "ChangeLog", config_parse_url_specifiers, 0, &t->changelog },
{ "Transfer", "AppStream", config_parse_url_specifiers, 0, &t->appstream },
{ "Transfer", "Features", config_parse_strv, 0, &t->features },
{ "Transfer", "RequisiteFeatures", config_parse_strv, 0, &t->requisite_features },
{ "Source", "Type", config_parse_resource_type, 0, &t->source.type },
{ "Source", "Path", config_parse_resource_path, 0, &t->source },
{ "Source", "PathRelativeTo", config_parse_resource_path_relto, 0, &t->source.path_relative_to },
@ -501,17 +519,34 @@ int transfer_read_definition(Transfer *t, const char *path) {
{}
};
r = config_parse(NULL, path, NULL,
"Transfer\0"
"Source\0"
"Target\0",
config_item_table_lookup, table,
CONFIG_PARSE_WARN,
t,
NULL);
_cleanup_free_ char *filename = NULL;
int r;
assert(path);
assert(dirs);
r = path_extract_filename(path, &filename);
if (r < 0)
return log_error_errno(r, "Failed to extract filename from path '%s': %m", path);
r = config_parse_many(
STRV_MAKE_CONST(path),
dirs,
strjoina(filename, ".d"),
arg_root,
"Transfer\0"
"Source\0"
"Target\0",
config_item_table_lookup, table,
CONFIG_PARSE_WARN,
/* userdata= */ NULL,
/* stats_by_path= */ NULL,
/* drop_in_files= */ NULL);
if (r < 0)
return r;
t->enabled = transfer_decide_if_enabled(t, known_features);
if (!RESOURCE_IS_SOURCE(t->source.type))
return log_syntax(NULL, LOG_ERR, path, 1, SYNTHETIC_ERRNO(EINVAL),
"Source Type= must be one of url-file, url-tar, tar, regular-file, directory, subvolume.");
@ -704,6 +739,8 @@ int transfer_vacuum(
assert(instances_max >= 1);
if (instances_max == UINT64_MAX) /* Keep infinite instances? */
limit = UINT64_MAX;
else if (space == UINT64_MAX) /* forcibly delete all instances? */
limit = 0;
else if (space > instances_max)
return log_error_errno(SYNTHETIC_ERRNO(ENOSPC),
"Asked to delete more instances than total maximum allowed number of instances, refusing.");
@ -713,7 +750,7 @@ int transfer_vacuum(
else
limit = instances_max - space;
if (t->target.type == RESOURCE_PARTITION) {
if (t->target.type == RESOURCE_PARTITION && space != UINT64_MAX) {
uint64_t rm, remain;
/* If we are looking at a partition table, we also have to take into account how many
@ -767,7 +804,11 @@ int transfer_vacuum(
assert(oldest->resource);
log_info("%s Removing old '%s' (%s).", special_glyph(SPECIAL_GLYPH_RECYCLING), oldest->path, resource_type_to_string(oldest->resource->type));
log_info("%s Removing %s '%s' (%s).",
special_glyph(SPECIAL_GLYPH_RECYCLING),
space == UINT64_MAX ? "disabled" : "old",
oldest->path,
resource_type_to_string(oldest->resource->type));
switch (t->target.type) {

6
src/sysupdate/sysupdate-transfer.h
View File

@ -15,12 +15,13 @@ typedef struct Transfer Transfer;
#include "sysupdate.h"
struct Transfer {
char *definition_path;
char *min_version;
char **protected_versions;
char *current_symlink;
bool verify;
bool enabled;
Resource source, target;
uint64_t instances_max;
@ -55,11 +56,10 @@ struct Transfer {
typedef int (*TransferProgress)(const Transfer *t, const Instance *inst, unsigned percentage);
Transfer* transfer_new(Context *ctx);
Transfer* transfer_free(Transfer *t);
DEFINE_TRIVIAL_CLEANUP_FUNC(Transfer*, transfer_free);
int transfer_read_definition(Transfer *t, const char *path);
int transfer_read_definition(Transfer *t, const char *path, const char **dirs, Hashmap *features);
int transfer_resolve_paths(Transfer *t, const char *root, const char *node);

212
src/sysupdate/sysupdate.c
View File

@ -25,9 +25,11 @@
#include "set.h"
#include "signal-util.h"
#include "sort-util.h"
#include "specifier.h"
#include "string-util.h"
#include "strv.h"
#include "sysupdate.h"
#include "sysupdate-feature.h"
#include "sysupdate-transfer.h"
#include "sysupdate-update-set.h"
#include "sysupdate-util.h"
@ -57,10 +59,21 @@ STATIC_DESTRUCTOR_REGISTER(arg_component, freep);
STATIC_DESTRUCTOR_REGISTER(arg_image_policy, image_policy_freep);
STATIC_DESTRUCTOR_REGISTER(arg_transfer_source, freep);
const Specifier specifier_table[] = {
COMMON_SYSTEM_SPECIFIERS,
COMMON_TMP_SPECIFIERS,
{}
};
typedef struct Context {
Transfer **transfers;
size_t n_transfers;
Transfer **disabled_transfers;
size_t n_disabled_transfers;
Hashmap *features; /* Defined features, keyed by ID */
UpdateSet **update_sets;
size_t n_update_sets;
@ -77,6 +90,12 @@ static Context* context_free(Context *c) {
transfer_free(*tr);
free(c->transfers);
FOREACH_ARRAY(tr, c->disabled_transfers, c->n_disabled_transfers)
transfer_free(*tr);
free(c->disabled_transfers);
hashmap_free(c->features);
FOREACH_ARRAY(us, c->update_sets, c->n_update_sets)
update_set_free(*us);
free(c->update_sets);
@ -93,83 +112,145 @@ static Context* context_new(void) {
return new0(Context, 1);
}
static int context_read_definitions(
static void free_transfers(Transfer **array, size_t n) {
FOREACH_ARRAY(t, array, n)
transfer_free(*t);
free(array);
}
static int read_definitions(
Context *c,
const char *directory,
const char *component,
const char *root,
const char **dirs,
const char *suffix,
const char *node) {
_cleanup_strv_free_ char **files = NULL;
Transfer **transfers = NULL, **disabled = NULL;
size_t n_transfers = 0, n_disabled = 0;
int r;
CLEANUP_ARRAY(transfers, n_transfers, free_transfers);
CLEANUP_ARRAY(disabled, n_disabled, free_transfers);
assert(c);
assert(dirs);
assert(suffix);
if (directory)
r = conf_files_list_strv(&files, ".conf", NULL, CONF_FILES_REGULAR|CONF_FILES_FILTER_MASKED, (const char**) STRV_MAKE(directory));
else if (component) {
_cleanup_strv_free_ char **n = NULL;
char **l = CONF_PATHS_STRV("");
size_t k = 0;
n = new0(char*, strv_length(l) + 1);
if (!n)
return log_oom();
STRV_FOREACH(i, l) {
char *j;
j = strjoin(*i, "sysupdate.", component, ".d");
if (!j)
return log_oom();
n[k++] = j;
}
r = conf_files_list_strv(&files, ".conf", root, CONF_FILES_REGULAR|CONF_FILES_FILTER_MASKED, (const char**) n);
} else
r = conf_files_list_strv(&files, ".conf", root, CONF_FILES_REGULAR|CONF_FILES_FILTER_MASKED, (const char**) CONF_PATHS_STRV("sysupdate.d"));
r = conf_files_list_strv(&files, suffix, arg_root, CONF_FILES_REGULAR|CONF_FILES_FILTER_MASKED, dirs);
if (r < 0)
return log_error_errno(r, "Failed to enumerate *.conf files: %m");
return log_error_errno(r, "Failed to enumerate sysupdate.d/*%s definitions: %m", suffix);
STRV_FOREACH(f, files) {
STRV_FOREACH(p, files) {
_cleanup_(transfer_freep) Transfer *t = NULL;
if (!GREEDY_REALLOC(c->transfers, c->n_transfers + 1))
return log_oom();
Transfer **appended;
t = transfer_new(c);
if (!t)
return log_oom();
t->definition_path = strdup(*f);
if (!t->definition_path)
return log_oom();
r = transfer_read_definition(t, *f);
r = transfer_read_definition(t, *p, dirs, c->features);
if (r < 0)
return r;
c->transfers[c->n_transfers++] = TAKE_PTR(t);
r = transfer_resolve_paths(t, arg_root, node);
if (r < 0)
return r;
if (t->enabled)
appended = GREEDY_REALLOC_APPEND(transfers, n_transfers, &t, 1);
else
appended = GREEDY_REALLOC_APPEND(disabled, n_disabled, &t, 1);
if (!appended)
return log_oom();
TAKE_PTR(t);
}
c->transfers = TAKE_PTR(transfers);
c->n_transfers = n_transfers;
c->disabled_transfers = TAKE_PTR(disabled);
c->n_disabled_transfers = n_disabled;
return 0;
}
static int context_read_definitions(Context *c, const char* node) {
_cleanup_strv_free_ char **dirs = NULL, **files = NULL;
int r;
assert(c);
if (arg_definitions)
dirs = strv_new(arg_definitions);
else if (arg_component) {
char **l = CONF_PATHS_STRV("");
size_t i = 0;
dirs = new0(char*, strv_length(l) + 1);
if (!dirs)
return log_oom();
STRV_FOREACH(dir, l) {
char *j;
j = strjoin(*dir, "sysupdate.", arg_component, ".d");
if (!j)
return log_oom();
dirs[i++] = j;
}
} else
dirs = strv_new(CONF_PATHS("sysupdate.d"));
if (!dirs)
return log_oom();
r = conf_files_list_strv(&files,
".feature",
arg_root,
CONF_FILES_REGULAR|CONF_FILES_FILTER_MASKED,
(const char**) dirs);
if (r < 0)
return log_error_errno(r, "Failed to enumerate sysupdate.d/*.feature definitions: %m");
STRV_FOREACH(p, files) {
_cleanup_(feature_unrefp) Feature *f = NULL;
f = feature_new();
if (!f)
return log_oom();
r = feature_read_definition(f, *p, (const char**) dirs);
if (r < 0)
return r;
r = hashmap_ensure_put(&c->features, &feature_hash_ops, f->id, f);
if (r < 0)
return log_error_errno(r, "Failed to insert feature '%s' into map: %m", f->id);
TAKE_PTR(f);
}
r = read_definitions(c, (const char**) dirs, ".transfer", node);
if (r < 0)
return r;
if (c->n_transfers + c->n_disabled_transfers == 0) {
/* Backwards-compat: If no .transfer defs are found, fall back to trying .conf! */
r = read_definitions(c, (const char**) dirs, ".conf", node);
if (r < 0)
return r;
if (c->n_transfers + c->n_disabled_transfers > 0)
log_warning("As of v257, transfer definitions should have the '.transfer' extension.");
}
if (c->n_transfers == 0) {
if (arg_component)
return log_error_errno(SYNTHETIC_ERRNO(ENOENT),
"No transfer definitions for component '%s' found.", arg_component);
"No transfer definitions for component '%s' found.",
arg_component);
return log_error_errno(SYNTHETIC_ERRNO(ENOENT),
"No transfer definitions found.");
}
FOREACH_ARRAY(tr, c->transfers, c->n_transfers) {
Transfer *t = *tr;
r = transfer_resolve_paths(t, root, node);
if (r < 0)
return r;
}
return 0;
}
@ -191,6 +272,17 @@ static int context_load_installed_instances(Context *c) {
return r;
}
FOREACH_ARRAY(tr, c->disabled_transfers, c->n_disabled_transfers) {
Transfer *t = *tr;
r = resource_load_instances(
&t->target,
arg_verify >= 0 ? arg_verify : t->verify,
&c->web_cache);
if (r < 0)
return r;
}
return 0;
}
@ -203,7 +295,6 @@ static int context_load_available_instances(Context *c) {
FOREACH_ARRAY(tr, c->transfers, c->n_transfers) {
Transfer *t = *tr;
assert(t);
r = resource_load_instances(
&t->source,
@ -750,6 +841,7 @@ static int context_vacuum(
uint64_t space,
const char *extra_protected_version) {
size_t disabled_count = 0;
int r, count = 0;
assert(c);
@ -773,15 +865,29 @@ static int context_vacuum(
count = MAX(count, r);
}
FOREACH_ARRAY(tr, c->disabled_transfers, c->n_disabled_transfers) {
r = transfer_vacuum(*tr, UINT64_MAX /* wipe all instances */, NULL);
if (r < 0)
return r;
if (r > 0)
disabled_count++;
}
if (FLAGS_SET(arg_json_format_flags, SD_JSON_FORMAT_OFF)) {
if (count > 0)
if (count > 0 && disabled_count > 0)
log_info("Removed %i instances, and %zu disabled transfers.", count, disabled_count);
else if (count > 0)
log_info("Removed %i instances.", count);
else if (disabled_count > 0)
log_info("Removed %zu disabled transfers.", disabled_count);
else
log_info("Removed no instances.");
log_info("Found nothing to remove.");
} else {
_cleanup_(sd_json_variant_unrefp) sd_json_variant *json = NULL;
r = sd_json_buildo(&json, SD_JSON_BUILD_PAIR_INTEGER("removed", count));
r = sd_json_buildo(&json,
SD_JSON_BUILD_PAIR_INTEGER("removed", count),
SD_JSON_BUILD_PAIR_UNSIGNED("disabledTransfers", disabled_count));
if (r < 0)
return log_error_errno(r, "Failed to create JSON: %m");
@ -806,7 +912,7 @@ static int context_make_offline(Context **ret, const char *node) {
if (!context)
return log_oom();
r = context_read_definitions(context, arg_definitions, arg_component, arg_root, node);
r = context_read_definitions(context, node);
if (r < 0)
return r;

4
src/sysupdate/sysupdate.h
View File

@ -4,6 +4,8 @@
#include <inttypes.h>
#include <stdbool.h>
#include "specifier.h"
/* Forward declare this type so that Transfers can point at it */
typedef struct Context Context;
@ -11,3 +13,5 @@ extern bool arg_sync;
extern uint64_t arg_instances_max;
extern char *arg_root;
extern char *arg_transfer_source;
extern const Specifier specifier_table[];