Commit Graph

6696 Commits

Author SHA1 Message Date
Anand Jain
fd3c4c4df1 btrfs-progs: tune: pass metadata_uuid in check_unfinished_fsid_change
In preparation to use check_unfinished_fsid_change() to support the
ability to reunite devices after a failed 'btrfstune -m|M' command,
rename %unused2 to %metadata_uuid as the function
check_unfinished_fsid_change() write the metadata_uuid from the ctree to
it.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain
aff1f16deb btrfs-progs: tune: pass fsid in check_unfinished_fsid_change arg2
In preparation to use check_unfinished_fsid_change() to support the
ability to reunite devices after a failed 'btrfstune -m|M' command,
delete unused1 argument instead reuse %fsid as the function
check_unfinished_fsid_change() returns the fsid.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain
70bedfa68a btrfs-progs: tune: rename uuid_changed to fsid_changed in set_metadata_uuid
We never change the metadata_uuid; we only change the fsid.  So
'%fsid_changed' flows more appropriately than '%uuid_changed'.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain
c6ecbe54bf btrfs-progs: tune: rename new_uuid to new_fsid in set_metadata_uuid
%new_uuid is being used to say there is a new fsid. So why not just call
it %new_fsid.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain
21440a64f7 btrfs-progs: tune: rename new_fsid to fsid in set_metadata_uuid
The %new_fsid is not only new it can be the fsid from the passed disk
so just rename it to %fsid. Also, in the next patch the %new_fsid will
be a bool variable to indicate if the %fsid is new from the fsid in the
disk.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain
9e9ac97902 btrfs-progs: tune: rename arg to new_fsid_str in set_metadata_uuid
In preparation to use check_unfinished_fsid_change() to support the
ability to reunite devices after a failed 'btrfstune -m|M' command,
%uuid_string arg is actually carries new fsid to be used. So just name
it to %new_fsid_str.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:58 +02:00
Anand Jain
3aa3342959 btrfs-progs: tune: cache local variable of fs_info
Since the root pointer dereferences for the fs_info several times,
it is rational to save the fs_info.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:56 +02:00
Anand Jain
4a253f713e btrfs-progs: track num_devices per fs_devices
Similar to the kernel we need to track the number of devices scanned
per fs_devices. A preparation patch to fix incomplete fsid changing.

Signed-off-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:25 +02:00
David Sterba
5c7f36b141 btrfs-progs: tests: fix typo in README
The correct helper name is check_global_prereq, this was once
seen accidentally used in a test.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:25 +02:00
Josef Bacik
0ba50a27f4 btrfs-progs: fix improper error handling in btrfs filesystem usage
I was seeing test-cli/016 failures because it claimed we were getting
EPERM from the TREE_SEARCH ioctl to get the chunk info out of the file
system.  This turned out to be because errno was already set going into
this function, the ioctl itself wasn't actually failing.  Fix this by
checking for a return value from the ioctl first, and then returning
-EPERM if appropriate.  This fixed the failures in my setup.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:25 +02:00
Josef Bacik
db7157d912 btrfs-progs: clear root dirty when we update the root
We don't currently use the bit to track whether or not the root is
dirty, but when we sync ctree.c it uses this bit to determine if we
should add the root to the dirty list.  Clear this bit when we update
the root so that the dirty tracking works properly when we sync ctree.c.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:25 +02:00
Josef Bacik
ba01aadff1 btrfs-progs: take a ref in the root locking code
This code in the kernel not only returns the locked root, but also takes
a reference on the node.  This is important for when we sync ctree.c
into btrfs-progs, it expects that references are held on the root node
after calling these helpers.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:25 +02:00
David Sterba
ce93d20fd5 btrfs-progs: docs: update kernel changes
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:25 +02:00
David Sterba
8001e37409 btrfs-progs: docs: updates
- reformatting
- new documents
- enhancements
- status updates

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:25 +02:00
David Sterba
a81a3d771b btrfs-progs: image: factor out the restore part from main.c
The remaining part of restore functionality starting from
restore_metadump() has been factored out. Same incremental set of
changes so the diff is not clean.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:25 +02:00
David Sterba
add18714b8 btrfs-progs: image: factor out the create part from main.c
The functionality of create and restore is all in main.c, split the
create functionality first. This is not the cleaniest diff to do it, the
functions are entangled and the final result is from several compile and
edit cycles.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:24 +02:00
David Sterba
00df7245f4 btrfs-progs: image: move defintions from main.c to header
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:24 +02:00
David Sterba
55bed0a8db btrfs-progs: ci: add cleanup script for coverage tests
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:24 +02:00
David Sterba
5b9206727b btrfs-progs: rename and move get_device_info
The helper belongs to the device-utils, move it there and use the common
prefix.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:24 +02:00
David Sterba
588f82f333 btrfs-progs: hash-vectest: make test_hash static to fix build warning
The warning:

crypto/hash-vectest.c:559:5: warning: no previous prototype for ‘test_hash’ [-Wmissing-prototypes]
  559 | int test_hash(const struct hash_testspec *sp

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:24 +02:00
David Sterba
83ac6e0a72 btrfs-progs: crypto: make the PCL implementation default for crc32c
Drop the old native intel implementation and use the PCL one. Remove the
artifical CPU flags.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:24 +02:00
David Sterba
992be8b50a btrfs-progs: crypto: add PCL based implementation for crc32c
Copy faster implementation of crc32c from linux kernel as of 6.5-rc7
(x86_64, arch/x86/crypto/crc32c-pcl-intel-asm_64.S). This needs
assembler build support, so detect target architecture so
cross-compilation still works.

Add a special CPU flag so the old and new implementations can be
benchmarked and verified separately.

Sample benchmark:

CPU flags: 0x1ff
CPU features: SSE2 SSSE3 SSE41 SSE42 SHA AVX AVX2 CRC32C_PCL
Block size:     4096
Iterations:     1000000
Implementation: builtin
Units:          CPU cycles

      NULL-NOP: cycles:     77177218, cycles/i       77
   NULL-MEMCPY: cycles:    226313072, cycles/i      226,    62133.395 MiB/s
    CRC32C-ref: cycles:  24418596066, cycles/i    24418,      575.859 MiB/s
     CRC32C-NI: cycles:   1188335920, cycles/i     1188,    11833.073 MiB/s
    CRC32C-PCL: cycles:    463193456, cycles/i      463,    30358.037 MiB/s
        XXHASH: cycles:    851606646, cycles/i      851,    16511.916 MiB/s
    SHA256-ref: cycles:  74476234956, cycles/i    74476,      188.808 MiB/s
     SHA256-NI: cycles:  34198637428, cycles/i    34198,      411.177 MiB/s
    BLAKE2-ref: cycles:  14761411664, cycles/i    14761,      952.597 MiB/s
   BLAKE2-SSE2: cycles:  18101896796, cycles/i    18101,      776.807 MiB/s
  BLAKE2-SSE41: cycles:  12599091062, cycles/i    12599,     1116.087 MiB/s
   BLAKE2-AVX2: cycles:   9668247506, cycles/i     9668,     1454.418 MiB/s

The new implementation is about 2.5x faster.

Note: there new version does not work on musl because of linkage
problems (relocations in .rodata), so it's still using the old
implementation.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:24 +02:00
David Sterba
8461513b9a btrfs-progs: rename prefixcmp to string_has_prefix
We already have the string_* namespace for such helpers.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:24 +02:00
David Sterba
5aa2ea7db0 btrfs-progs: add more helpers to read values from sysfs
In some places we want to read a single u64 value from a sysfs path, or
from fsid directory. Add helpers that do that in one go.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:24 +02:00
David Sterba
4a96a935ad btrfs-progs: move sysfs related helpers to own file
The sysfs could use more convenience helpers so move the current code to
own file before adding more helpers.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:24 +02:00
Qu Wenruo
6abbad4d1b btrfs-progs: docs: enhance the error handling guidelines
Currently we only have a very brief explanation on the unexpected error
handling (only ASSERT()/WARN_ON()/BUG_ON()), and no further
recommendation on the proper usage of them.

This patch would improve the guideline by:

- Add btrfs_abort_transaction() usage
  Which is the recommended way when possible.

- More detailed explanation on the usage of ASSERT()
  Which is only a fail-fast option mostly designed for developers, thus
  is only recommended to rule out some invalid function usage.

- More detailed explanation on the usage of WARN_ON()
  Mostly for call sites which need a call trace strongly, and is not
  applicable for a btrfs_abort_transaction() call.

- Completely discourage the usage of BUG_ON()

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:24 +02:00
David Sterba
85c841f961 btrfs-progs: docs: add developer docs for internal APIs
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:24 +02:00
David Sterba
00e401752f btrfs-progs: list-chunks: port to sorting API
Use the sorting API. This is 1:1 transformation of previous single key
sorting and needs to be updated so there are multiple accepted instead.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:24 +02:00
David Sterba
7f9bcab309 btrfs-progs: add sorting API
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:24 +02:00
David Sterba
a5d95a73f9 btrfs-progs: add an extensible pointer array struct
API for extensible array of pointers for covenience. A simple wrapper
around a (void *) array with length.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:24 +02:00
David Sterba
2b35741a39 btrfs-progs: fix more typos found by codespell
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:24 +02:00
Francesco Yoshi Gobbo
240f518441 btrfs-progs: balance start: fix typo in help text
Pull-request: #660
Author: Francesco Yoshi Gobbo <yoshi@fgobbo.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:24 +02:00
David Sterba
7ed0172bf6 btrfs-progs: docs: add dev docs about json
Programmer documentation, to be continued.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:24 +02:00
David Sterba
ebd046eb8d btrfs-progs: tests: renumber json test cases so they match the parameter
The test utility takes a numeric parameter from 1 to max tests but this
is off by one to the test case function names. Unify that so it's clear
which test fails.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:24 +02:00
David Sterba
ede538f98d btrfs-progs: print null value for empty uuid in json
This is a potentially breaking change to json output. An all zeros uuid
was printed as "-" but we can utilize native json type null for that.
Note the va_copy must be used as va_arg advances the pointer.

{
	"nulluuid": null
}

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:23 +02:00
David Sterba
7ed79f2b3b btrfs-progs: use escaped format for subvolume path strings in json
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:23 +02:00
David Sterba
70ae158ec8 btrfs-progs: rename time-long format name to date-time
Make the timestamp format more descriptive what is actually printed. We
may need separate date or time in the future.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:23 +02:00
David Sterba
386765be15 btrfs-progs: tests: fix build warnings in json tests (-Wmissing-prototypes)
The test cases should be static, otherwise it leads to a warning like

    [LD]     json-formatter-test
tests/json-formatter-test.c:40:6: warning: no previous prototype for ‘test_simple_empty’ [-Wmissing-prototypes]
   40 | void test_simple_empty()
      |      ^~~~~~~~~~~~~~~~~

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:23 +02:00
David Sterba
25fc470252 btrfs-progs: print numeric values unquoted in json
The json spec allows numeric values and it's recommended to use them
instead of the stringified numbers. This is a potentially breaking change
if some tools relied on the string value.

As most formats we now have are '%llu' and it's convenient to just pass
it to vprintf, don't add a special type for ints. Any new int type must
be added to the list.

{
	"number": 1234
}

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:23 +02:00
David Sterba
155eb56417 btrfs-progs: document json format str
The 'str' type was added in ecbb6a7fcd ("btrfs-progs: add json
formatter for escaped string") but not documented. It should be used
e.g. for paths or strings from unknown origin.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:23 +02:00
David Sterba
be5cb7dfe9 btrfs-progs: add bool type to json
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:23 +02:00
David Sterba
b890183072 btrfs-progs: support unquoted values in json
For null or boolean values the "..." quoting must not be done, add
support for that. This is detected internally for each printed value.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:23 +02:00
David Sterba
29060ec176 btrfs-progs: subvolume show: print all items unconditionally in json
The json output could be easily filtered on the user side so we'll print
everything unconditionally.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:23 +02:00
David Sterba
42d0861e31 btrfs-progs: subvolume: rename some json key names
Don't abbreviate generation and use qgroup where it's related to the
qgroup itself and not quotas in general.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:23 +02:00
David Sterba
6120fdadc2 btrfs-progs: subvolume: move json format output under experimental
The format of the json data needs some time to be finalized so it'll
be under experimental.

Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:23 +02:00
Christoph Heiss
328333f9cf btrfs-progs: subvol show: implement json format output
Implements JSON-formatted output for the `subvolume list` command using
the `--format json` global option, much like it is implemented for other
commands.

Signed-off-by: Christoph Heiss <christoph@c8h4.io>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:23 +02:00
Christoph Heiss
f134c86411 btrfs-progs: subvol get-default: implement json format output
Implements JSON-formatted output for the `subvolume get-default` command
using the `--format json` global option, much like it is implemented for
other commands.

Signed-off-by: Christoph Heiss <christoph@c8h4.io>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:23 +02:00
Christoph Heiss
ee233db769 btrfs-progs: subvol list: implement json format output
Implements JSON-formatted output for the `subvolume list` command using
the `--format json` global option, much like it is implemented for other
commands.

Re-uses the `btrfs_list_layout` infrastructure to nicely fit it into the
existing formatting code.

A notable difference to the normal, text-based output is that in the
JSON output, timestamps include the timezone offset as well.

Signed-off-by: Christoph Heiss <christoph@c8h4.io>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:23 +02:00
Christoph Heiss
da440e66fb btrfs-progs: subvol: introduce rowspec definition for json output
List and export all fields that may be needed for any subvolume related
json output.

Signed-off-by: Christoph Heiss <christoph@c8h4.io>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:23 +02:00
Christoph Heiss
7c983d2352 btrfs-progs: subvol show: factor out text printing to own function
Prepare for switching the plain and json output. The format is slightly
different so we can't utilize the unified fmt_* helpers and two separate
printer functions make more sense.

Signed-off-by: Christoph Heiss <christoph@c8h4.io>
Signed-off-by: David Sterba <dsterba@suse.com>
2023-08-28 17:24:23 +02:00