Commit Graph

551 Commits

Author SHA1 Message Date
David Sterba
7ce3ae3966 btrfs-progs: libbtrfs: drop crc32c.h from exported headers
The usage of the crc32 helpers in ctree.h has been removed and there's
no other reason to keep crc32c.h exported.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:35 +02:00
David Sterba
2e2c7c0fc6 btrfs-progs: libbtrfs: drop sizes.h from exported headers
The SZ_ macros have been removed from the other exported headers, we can
now safely drop seizes.h too.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:35 +02:00
David Sterba
38356d456b btrfs-progs: libbtrfs: drop radix-tree.h from exported headers
The header is only included from ctree.h but not actually used, we can
drop it from the exported files.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:35 +02:00
David Sterba
cb9bd8e326 btrfs-progs: subvol show: print a warning with rw and receive_uuid
Add a slightly more convenient way to identify the subvolumes with bad
combination of flags and received uuid.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-07 18:40:34 +02:00
David Sterba
287d472b63 btrfs-progs: tests: subvolume ro->rw switch and received_uuid
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-07 18:40:29 +02:00
David Sterba
49bd94cc14 btrfs-progs: tests: make misc/038 work with free-space-tree
The test misc-tests/038-backup-root-corruption expects a particular
layout of the backup roots but this gets slightly changed due to free
space tree.

  $ make TEST=038-backup-root-corruption TEST_ARGS_MKFS='-Rfree-space-tree' TEST_ENABLE_OVERRIDE=true test-misc

Will result in test failure as the expected root is in slot 3 and not 2.
Update the test to try both.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-07 18:40:21 +02:00
David Sterba
27207d651a btrfs-progs: dump-tree: print complete root_item
The output of root_item in the 'inspect dump-tree' command lacks some
items and some of them are printed conditionally. As the dump utility
is for debugging, it's better to print all the items, with names
matching the structure members and order.

Some values will inevitably be all zeros like uuids or various
timestamps, but that's a minor issue and affecting only a few trees.

Example:

  item 0 key (EXTENT_TREE ROOT_ITEM 0) itemoff 15844 itemsize 439
	  generation 5 root_dirid 0 bytenr 30523392 byte_limit 0 bytes_used 16384
	  last_snapshot 0 flags 0x0(none) refs 1
	  drop_progress key (0 UNKNOWN.0 0) drop_level 0
	  level 0 generation_v2 5
	  uuid 00000000-0000-0000-0000-000000000000
	  parent_uuid 00000000-0000-0000-0000-000000000000
	  received_uuid 00000000-0000-0000-0000-000000000000
	  ctransid 0 otransid 0 stransid 0 rtransid 0
	  ctime 0.0 (1970-01-01 01:00:00)
	  otime 0.0 (1970-01-01 01:00:00)
	  stime 0.0 (1970-01-01 01:00:00)
	  rtime 0.0 (1970-01-01 01:00:00)

  item 3 key (FS_TREE ROOT_ITEM 0) itemoff 14949 itemsize 439
	  generation 4 root_dirid 256 bytenr 30408704 byte_limit 0 bytes_used 16384
	  last_snapshot 0 flags 0x0(none) refs 1
	  drop_progress key (0 UNKNOWN.0 0) drop_level 0
	  level 0 generation_v2 4
	  uuid ec4669b6-6d21-46ab-857e-d60cafde45b3
	  parent_uuid 00000000-0000-0000-0000-000000000000
	  received_uuid 00000000-0000-0000-0000-000000000000
	  ctransid 0 otransid 0 stransid 0 rtransid 0
	  ctime 1633021823.0 (2021-09-30 19:10:23)
	  otime 1633021823.0 (2021-09-30 19:10:23)
	  stime 0.0 (1970-01-01 01:00:00)
	  rtime 0.0 (1970-01-01 01:00:00)

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-07 18:39:44 +02:00
David Sterba
6887a3192f btrfs-progs: tests: fix fsck/024 to work with free-space-tree
The test looks for a pattern FREE_SPACE that also matches
FREE_SPACE_TREE when it's enabled. Use word regexp to match the whole
word only.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:49:25 +02:00
David Sterba
88de623dd2 btrfs-progs: convert: allow to set a duplicate uuid
Relax the condition about a unique uuid for convert, only print a
warning. In case we copy the uuid, it's expected that at the time the
conversion starts the uuid is not unique as it sill exists on the source
filesystem.

In case user sets the uuid manually but it's still the same one as on
the source filesystem we should also allow that, so it warns in this
case as well.

Update the test so it creates a block device where the uuid would be
also cached by blkid and lets the non-unique check succeed.

Issue: #404
Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-20 16:58:13 +02:00
Qu Wenruo
3c7e75eefb btrfs-progs: tests: test case to make sure btrfstune rejects corrupted fs
Although btrfstune will already warn users to make sure the fs is not
corrupted, we can never trust end users.

If the target fs has transid error, btrfstune can cause further damage,
thus we need to make sure btrfstune can safely reject fs with transid
error, other than ignoring the problem.

The image is copied from fsck-tests/002, just override check_image() to
run "btrfstune -u" instead.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-20 12:26:20 +02:00
David Sterba
7d16785ab4 btrfs-progs: tests: test options for defrag -c
Regression test for "btrfs-progs: fix defrag -c option parsing"

Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-20 11:46:24 +02:00
David Sterba
53c9618ce0 btrfs-progs: libbtrfs: drop btrfs-list.h, raid56.h and btrfsck.h
libbtrfs isn't a proper library and exports internal headers that aren't
included from other headers and their use in public API does not make
sense. There are no known applications using them so don't install them.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-08 18:17:29 +02:00
David Sterba
7b2d6cd778 btrfs-progs: tests: add more API coverage to library-test
Update the library test stub to cover all the API functions we care
about (send stream, subvol search) and drop the rootid lookup that's
trivial to do via raw ioctls.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-08 18:16:16 +02:00
David Sterba
e86425242f btrfs-progs: move send.h to kernel-shared/
The header contains the protocol definitions and is almost exactly the
same as the kernel version, move it to the proper directory.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-07 19:26:46 +02:00
David Sterba
9d98ad1db5 btrfs-progs: tests: add templates for most common test types
Create a ready-to-use templates for the most common tests, copied from
readme.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-07 18:10:56 +02:00
David Sterba
5b0e18ce6c btrfs-progs: tests: properly load dm-thin in mkfs/017
The test started to fail some time ago when dm-thin was changed to
dm-thin-pool that loads 2 targets dm-thin and dm-thin-pool. We can now
properly detect both and run the test.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-07 16:58:43 +02:00
David Sterba
37e0aa576c btrfs-progs: tests: allow alternate name for dm target detection
Some dm target name and module do not match exacly, extend the helper to
take optional 2nd parameter that will be checked in case loading by the
first parameter fails.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-07 16:57:06 +02:00
David Sterba
6694fe5de9 btrfs-progs: tests: do dm target detection by one
Switch the helper to take only one parameter, the target name. This can
be used to extend the helper with additional parameters for the target.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-07 16:48:42 +02:00
David Sterba
0534441e2e btrfs-progs: tests: add mkfs test for raid0/1 and raid10/2
Extend basic tests with the degenerate raid0 and raid10, coming in 5.15.
Mount of a freshly created filesystem works even on older kernels too.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-08-27 15:40:53 +02:00
Josef Bacik
4d57632e2f btrfs-progs: tests: add image with an invalid super bytes_used
This is used to validate the detection and correction code in both fsck
modes for an invalid bytes_used value in the super block.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-08-25 15:43:13 +02:00
Josef Bacik
71404c50e8 btrfs-progs: tests: fix running check mode lowmem tests
When I added the invalid super image I saw that the lowmem tests were
passing, despite not having the detection code yet.  Turns out this is
because we weren't using a run command helper which does the proper
expansion and adds the --mode=lowmem option.  Fix this to use the proper
handler, and now the lowmem test fails properly without my patch to add
this support to the lowmem mode.

Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-08-25 15:38:54 +02:00
Josef Bacik
72a37dca27 btrfs-progs: tests: add image with a corrupt block group item
This image has a broken used field of a block group item to validate
fsck does the correct thing.

Reviewed-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-08-25 15:38:54 +02:00
Qu Wenruo
9df3619a9d btrfs-progs: require full nodesize alignement for subpage support
For the incoming extra page size support for subpage (sectorsize <
PAGE_SIZE) cases, the support for metadata will be a critical point.

Currently for subpage support, we require 64K page size, so that no
matter whatever the nodesize is, it will be contained inside one page.
And we will reject any tree block which crosses page boundary.

But for other page size, especially 16K page size, we must support
nodesize differently.

For nodesize < PAGE_SIZE, we will have the same requirement (tree blocks
can't cross page boundary).
While for nodesize >= PAGE_SIZE, we will require the tree blocks to be
page aligned.

To support such feature, we will make btrfs-check to reports more
subpage related warnings for metadata.

This patch will report any tree block which is not nodesize aligned as a
warning.

Existing mkfs/convert has already make sure all new tree blocks are
nodesize aligned, this is just for older converted filesystems.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-08-20 14:45:58 +02:00
Qu Wenruo
701c7d58db btrfs-progs: tests: don't check subpage related warnings for simple fsck tests
For fsck tests, we check the subpage warnings for each type 1 test, but
such type 1 tests are mostly read-only tests, and one of the test will
trigger new subpage related warnings (fsck/018).

For subpage related warnings, what we really care are write operations,
including mkfs, btrfs-convert and repair, not those read-only tests.

So skip the subpage warning check for fsck type 1 tests to prevent false
alert of later more strict subpage warnings.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-08-20 14:44:09 +02:00
Qu Wenruo
82e0720312 btrfs-progs: tests: also check subpage warning for check_image cases
There are two types of test cases:

- Type 1 (without test.sh)
- Type 2 (test.sh, mostly will override check_image())

For Type 2 tests, we check subpage related warnings of btrfs-check, but
didn't check it for Type 1 test cases.

In fact, Type 1 test cases are more important, as they involve repair,
which can generate new tree blocks, and we want to make sure such new
tree blocks won't cause subpage related warnings.

This patch will add the extra check for Type 1 test cases.

And it will make sure the subpage related warnings are really from this
test case, to prevent false alerts.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-08-20 14:42:47 +02:00
David Sterba
03a9dbf784 btrfs-progs: tests: add test for convert --uuid option
Signed-off-by: David Sterba <dsterba@suse.com>
2021-08-20 14:24:55 +02:00
Qu Wenruo
af62875784 btrfs-progs: tests: check nlinks for directories
Make sure btrfs check can detect such problem.  Right now we have no way
to fix it yet.

Reviewed-by: Anand Jain <anand.jain@oracle.com>
Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-07-30 15:52:57 +02:00
David Sterba
1eb7b11303 btrfs-progs: remove stale user transaction ioctl definitions
The user transaction ioctls have been removed in kernel 4.17 by commit
7a5a07a81062 ("btrfs: Remove userspace transaction ioctls"), the
definitions are not relevant and can be removed.

The numbers could be reused in the future, eg. when there are no
maintained LTS kernels older than 4.19.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-06-19 22:07:50 +02:00
Qu Wenruo
ab0d369525 btrfs-progs: tests/fsck: add test case to make sure btrfs check can reset btrfs_dev_item::bytes_used
The test image is manually crafted with 1MiB offset in the device item
of devid 1.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-06-19 22:07:49 +02:00
Qu Wenruo
77aca786ba btrfs-progs: tests: add test to ensure the restored image can be mounted
This new test case is to make sure the restored image file has been
properly enlarged so that newer kernel won't complain.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-06 16:41:47 +02:00
David Sterba
f0d226fa48 btrfs-progs: tests: add missing variable setup to test-console
The paths are not adapted to the TEST_TOP added long time ago in
e44f595dd7 ("btrfs-progs: tests: unify test drivers, make ready for
extenral testsuite").

Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-24 22:20:19 +01:00
David Sterba
05a92e6424 btrfs-progs: crypto: add test vectors
Add test vectors, a subset without keys as found in linux kernel sources
in crypto/test-mgr.h for all supported hash algorithms.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-24 22:20:19 +01:00
David Sterba
0c6e50dddd btrfs-progs: tests: verify that ext4 supports extra_isize
To properly check the 64bit timestamp conversion, the filesystem must
support it. Check that a freshly created filesystem contains the
feature.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-23 17:04:22 +01:00
David Sterba
3a22f0b25b btrfs-progs: balance: fix raid56 warning for other profiles
The warning is printed for profiles where it's not intended (like raid0
or raid1c4). Check the correct variable for the target profiles.

Issue: #355
Fixes: 1ed5db8db4 ("btrfs-progs: balance convert: add a warning and countdown for RAID56 conversion")
Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-23 17:04:22 +01:00
David Sterba
5d80df6065 btrfs-progs: tests: add crafted image to test needs_recovery
Image of ext4 with needs_recovery incompat bit set. This bit cannot be
set by regular tune2fs so was created on an empty 4M image by patched
tune2fs that set the bit unconditionally (the image still passed e2fsck,
with journal recovery).

Issue: #348
Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-23 17:04:21 +01:00
Pierre Labastie
ea1e4c8a7b btrfs-progs: tests: fix quoting of sudo helper in misc/046-seed-multi-mount
When running the tests as root SUDO_HELPER is empty, so that
`run_check "" dd ...` is run, and it fails because the empty command is
not found.

Pull-request: #351
Issue: #352
Author: Pierre Labastie <pierre.labastie@neuf.fr>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-15 15:40:54 +01:00
David Sterba
833bf64762 btrfs-progs: tests: extend mkfs/001 to look for warnings after mkfs
Patch "btrfs-progs: fix false alert on tree block crossing 64K page
boundary" fixes a false alert (warning) when 'btrfs check' is called
right after mkfs.

As we have an extensive mkfs coverage in test mkfs/001, add the check for
the warning there instead of a separate test.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-15 15:40:54 +01:00
David Sterba
06bf4d0e8b btrfs-progs: tests: use force when decompressing images
Some tests report that decompressing the image failed, which did not
fail the test but could lead to wrong errors in case the image is not
overwritten and leaves some old state. Use --force parameter.

    [TEST]   fuzz-tests.sh
    [TEST/fuzz]   001-simple-check-unmounted
xz: btrfs-progs/tests/fuzz-tests/images/bko-97021-invalid-chunk-sectorsize.raw: File exists
failed to decompress image btrfs-progs/tests/fuzz-tests/images/bko-97021-invalid-chunk-sectorsize.raw.xz
    [TEST/fuzz]   002-simple-image
xz: btrfs-progs/tests/fuzz-tests/images/bko-97021-invalid-chunk-sectorsize.raw: File exists
failed to decompress image btrfs-progs/tests/fuzz-tests/images/bko-97021-invalid-chunk-sectorsize.raw.xz

Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-01 16:47:15 +01:00
David Sterba
464e73ff0d btrfs-progs: tests: add test for seeding device mounts
Test multiple mounts of a single device.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-03-01 16:47:15 +01:00
Boris Burkov
feb8f56ba2 btrfs-progs: receive: fix btrfs_mount_root substring bug
The current mount detection code in btrfs receive is not quite perfect.
For example, suppose /tmp is mounted as a tmpfs. In that case,
btrfs receive /tmp2 will find /tmp as the longest mount that matches a
prefix of /tmp2 and blow up because it is not a btrfs filesystem, even
if /tmp2 is just a directory in / mounted as btrfs.

Fix this by replacing the substring check with a dirname recursion to
only check the directories in the path of the dir, rather than every
substring.

Add a new test for this case.

Signed-off-by: Boris Burkov <boris@bur.io>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-02-19 16:29:40 +01:00
Qu Wenruo
5c9a753ba0 btrfs-progs: tests: check the result log for critical warnings
Introduce a new function, check_test_results(), for
misc/fsck/convert/mkfs test cases.

This function is currently to catch warning message for subpage support,
but can be later expanded for other usages.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-02-19 15:24:42 +01:00
Nikolay Borisov
b39d02a0c0 btrfs-progs: tests: cli/003 verify that the path is not an image
Add a test case which ensures that when resize is tried on an image
instead of a directory appropriate warning is produced and the command
fails.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-02-19 15:24:42 +01:00
David Sterba
1804dcaa4b btrfs-progs: tests: wait for dm device in test mkfs/005
When the test environment is in 'docker', there's some delay before the
device mapper nodes appear in /dev/mapper. Add a delay before the test
continues, usually 1 second was enough but give it more just in case.
Add ad fallback to skip the test if the device node does not show up,
this is a problem in the running environment and the testsuite should
continue.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-02-19 15:18:39 +01:00
David Sterba
a3aa4c0717 btrfs-progs: tests: randomize device name in mkfs/005-long-device-name-for-ssd
Add some randomization to the long device name in case the testsuite
runs multiple times on the same host.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-02-05 16:19:24 +01:00
David Sterba
01c1bf8d8e btrfs-progs: tests: replace which by type -p
In some environments the which utility might not be available and the
shell builtin 'type -p' is readily available.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-02-05 14:23:11 +01:00
David Sterba
911741e854 btrfs-progs: tests: add support to test the .static binaries
Testing the statically built binaries is not straightforward, add a
convenient way to do that:

  $ make TEST_FLAVOR=static

There should be no difference in the test results.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-01-29 14:24:33 +01:00
David Sterba
bfc7ab7c39 btrfs-progs: tests: set toplevel subvolume as default when specified as 0
Testcase for commit "btrfs-progs: subvol set-default: change id to 5 if
specified as 0".

Issue: #327
Signed-off-by: David Sterba <dsterba@suse.com>
2021-01-18 17:49:23 +01:00
Marcos Paulo de Souza
6a6a7c4698 btrfs-progs: tests: test logical-resolve in various scenarios
Signed-off-by: Marcos Paulo de Souza <mpdesouza@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-01-18 17:49:22 +01:00
David Sterba
df2bdf97d2 btrfs-progs: tests: test full 64bit timestamp conversion on ext4
Verify that [acm]time are copied from the ext4 source without loss of ns
precision.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-01-13 22:33:10 +01:00
Adam Borowski
c615287cc0 btrfs-progs: a bunch of typo fixes
Signed-off-by: Adam Borowski <kilobyte@angband.pl>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-01-13 22:33:10 +01:00