Commit Graph

152 Commits

Author SHA1 Message Date
David Sterba
17aab2e428 btrfs-progs: receive: add missing unused inode number reads from the stream
Kernel emits inode number for all mkfile/mkdir/... commands but the
receive part does not pass it to the callbacks. At least document that
and read it from the stream in case we'd like to use it in the future.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-29 17:29:57 +02:00
David Sterba
11fcdbc35e btrfs-progs: introduce helper to get allowed profiles for a given device number
Use the raid table helper to avoid hard coding profiles for the given
number of devices in test_num_disk_vs_raid.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-20 18:59:24 +02:00
David Sterba
b29f1603b0 btrfs-progs: use raid table for devs_min and replace local helper
Another duplication of the raid table, in this case missing the changes
to raid10 and raid0 minimum devices changed in a177ef7dd4
("btrfs-progs: mkfs: allow degenerate raid0/raid10").

Define and use a helper using the table value.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-20 18:59:23 +02:00
Naohiro Aota
ae0dfb246d btrfs-progs: introduce btrfs_pread wrapper for pread
Wrap pread with btrfs_pread as well.

Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-20 18:59:23 +02:00
Naohiro Aota
c821e5545f btrfs-progs: introduce btrfs_pwrite wrapper for pwrite
Wrap pwrite with btrfs_pwrite(). It simply calls pwrite() on non-zoned
btrfs (opened without O_DIRECT). On zoned mode (opened with O_DIRECT),
it allocates an aligned bounce buffer, copies the contents and uses it
for direct-IO writing.

Writes in device_zero_blocks() and btrfs_wipe_existing_sb() are a little
tricky. We don't have fs_info on our hands, so use zinfo to determine it
is a zoned device or not.

Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-20 18:59:23 +02:00
Naohiro Aota
2a550d4ccd btrfs-progs: drop ZONED flag from BTRFS_CONVERT_ALLOWED_FEATURES
Since we cannot create ext*/reiserfs on a zoned device, it is useless to
allow ZONED feature when converting a file system. Drop ZONED flag from
BTRFS_CONVERT_ALLOWED_FEATURES.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:47:04 +02:00
David Sterba
b2dc5d0037 btrfs-progs: remove c++ protection from internal headers
We don't need the c++ name mangling protection in headers that are not
part of libbtrfs.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:47:04 +02:00
David Sterba
0fc4bb4d91 btrfs-progs: remove unused subvol_uuid_search_add
After removal of the fallback uuid search code the helper is unused in
the internal code.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:47:04 +02:00
David Sterba
ac5954ee36 btrfs-progs: merge subvol_uuid_search helpers
Due to ambiguity of error values in the public API subvol_uuid_search,
there was second version added. There's a separate copy in libbtrfs and
we don't have to distinguish in the internal code. All callers check for
IS_ERR and NULL, so we can safely merge the helpers into one.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:47:04 +02:00
David Sterba
e491d9cf25 btrfs-progs: simplify struct subvol_uuid_search use
After removing uuid search fallback code the structure has become
trivial and copies the fd that all callers have in their context.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:47:04 +02:00
David Sterba
22f5600649 btrfs-progs: open code subvol_uuid_search_init
The helper is trivial after removing the uuid search fallback code,
open code it.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:47:04 +02:00
David Sterba
f18dbe2bc6 btrfs-progs: remove empty subvol_uuid_search_finit
After the uuid search fallback code has been removed, the finit helper
has become empty and can be removed.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:47:04 +02:00
David Sterba
522945efc8 btrfs-progs: remove unused prototypes from send-utils.h
The functions are part of path-utils.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:47:04 +02:00
David Sterba
a8dceec381 btrfs-progs: drop slow subvol uuid search
There's a lot of code under BTRFS_COMPAT_SEND_NO_UUID_TREE to support
kernels < 3.12 that don't have uuid tree and the subvolume uuids are
searched in a slow way, building the uuid tree on the userspace side.

As the uuid tree is always created, the fallback code was not exercised
anyway due to 'uuid_tree_existed' check in subvol_uuid_search2.

Delete the code from the internal copy of send-utils. The support still
stays for libbtrfs and will be removed in the future.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:47:04 +02:00
David Sterba
a1f2dd12ad btrfs-progs: use btrfs_subvolid_resolve instead of btrfs_list_path_for_root
The btrfs_list_* functions come with some overhead and for simple path
resolution we can use btrfs_subvolid_resolve.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:47:03 +02:00
David Sterba
8bb13015bd btrfs-progs: don't include btrfs-list.h unless necessary
We don't need to include this besides btrfs-list.c itself and
subvolume.c that does use the btrfs_list_* API.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:47:03 +02:00
David Sterba
a3efed2adb btrfs-progs: libbtrfs: merge utils-lib.c back to utils.c
The separate file was needed for libbtrfs in the past to avoid pulling
utils.c in, but this is not needed after recent cleanups.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:47:03 +02:00
David Sterba
00610f5853 btrfs-progs: libbtrfs: remove unneeded BTRFS_FLAT_INCLUDES protections
Remove the switch for local and system-wide headers in headers that are
not part of libbtrfs anymore.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:47:03 +02:00
David Sterba
72260bc3b9 btrfs-progs: mkfs: enable space_cache=v2 (free-space-tree) by default
The free space tree is a better way to track the free space and has been
tested in the wild for a long time. The backward compatibility is
sufficient, several long term kernels. On-line conversion from v1 to v2
can be done by mount, switching from v2 to v1 can be done by 'btrfs
check'.

Issue: 
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:47:03 +02:00
David Sterba
1f918def34 btrfs-progs: mkfs: enable NO_HOLES by default
The no-holes feature reduces consumption of metadata by not representing
file holes. Reducing metadata is a good thing in general, this is the
main goal to enable this by default.

There's a drawback, related to the missing information about holes. The
'check' tool cannot use it to cross-reference extent information and in
some cases may not be able to detect a problem.

The no-hole feature can be also enabled by 'btrfstune -n' on an
unmounted filesystem.

Issue: 
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:47:02 +02:00
David Sterba
6d2bf4cbce btrfs-progs: refactor feature table initializations to C99-style
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:47:02 +02:00
David Sterba
35882be40a btrfs-progs: remove BTRFS_FLAT_INCLUDES from internal send headers
There's a separate copy of the headers, we can remove the include switch
from the internal copies.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:34 +02:00
David Sterba
dad03fac3b btrfs-progs: switch btrfs_group_profile_str to use raid table
Use the profile names from the table, preserving the whole semantics of
unknown flags detection.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:34 +02:00
David Sterba
5664631b5b btrfs-progs: clean up test_uuid_unique
Move the declaration to the right header, constify argument and document
the function.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-08 20:46:33 +02:00
David Sterba
c6c2e3da0a btrfs-progs: device scan: rename is_path_device
The term 'path' is confusing as we normally use it for filesystem paths,
while for multipath it's more related to the physical path by which the
devices are connected (though it also shows up as another path in the
filesystem).

Rename the helper doing the multipath detection so it's clear what path
is meant by that.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:50:42 +02:00
Nikolay Borisov
1b7128c152 btrfs-progs: add fallback code for multipath device detection for static build
Since libudev doesn't provide a static version of the library for static
build btrfs-progs will have to provide manual fallback. This change does
this by parsing the udev database files hosted at /run/udev/data/.
Under that directory every block device should have a file with the
following name: bMAJ:MIN. So implement the bare minimum code necessary
to parse this file and search for the presence of DM_MULTIPATH_DEVICE_PATH
udev attribute. This could likely be racy since access to the udev
database is done outside of libudev but that's the best that can be
done when implementing this manually and is only for a limited usecase
where static build has to be used.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:50:37 +02:00
Nikolay Borisov
681b4bd5e1 btrfs-progs: ignore devices representing paths in multipath
Currently btrfs-progs will happily enumerate any device which has a
btrfs filesystem on it irrespective of its type. For the majority of
use cases that's fine and there haven't been any problems with that.
However, there was a recent report that in multipath scenario when
running "btrfs fi show" after a path flap (path going down and then
coming back up) instead of the multipath device being show the device
which represents the flapped path is shown. So a multipath filesystem
might look like:

  Label: none  uuid: d3c1261f-18be-4015-9fef-6b35759dfdba
	  Total devices 1 FS bytes used 192.00KiB
	  devid    1 size 10.00GiB used 536.00MiB path /dev/mapper/3600140501cc1f49e5364f0093869c763

/dev/mapper/xxx is actually backed by an arbitrary number of paths,
which in turn are presented to the system as ordinary SCSI devices i.e
/dev/sdX. If a path flaps and a user re-runs 'btrfs fi show' the output
would look like:

  Label: none  uuid: d3c1261f-18be-4015-9fef-6b35759dfdba
	  Total devices 1 FS bytes used 192.00KiB
	  devid    1 size 10.00GiB used 536.00MiB path /dev/sdd

This only occurs on unmounted filesystems as those are enumerated by
btrfs-progs, for mounted filesystem the kernel properly deals only with
the actual multipath device.

Turns out the output of this command is consumed by libraries and the
presence of a path device rather than the actual multipath causes
issues.

Fix this by checking for the presence of DM_MULTIPATH_DEVICE_PATH
udev attribute as multipath path devices are tagged with this attribute
by the multipath udev scripts.

Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:50:35 +02:00
David Sterba
c50c448518 btrfs-progs: do sysfs detection of device discard capability
The detection of the discard status of a device is done by issuing a
real discard request but on an empty range. This works in most cases.
However there's a case of a VirtualBox driver that returns 'Operation
not supported' in that case, and then discard is skipped during mkfs.

The other tools like fstrim check the sysfs queue file
discard_granularity which is the recommended way. Do that as well.

Issue: 
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:49:30 +02:00
Naohiro Aota
53ec59ead0 btrfs-progs: do not zone reset on emulated zoned mode
We cannot zone reset a regular file with emulated zones. So, mkfs.btrfs
on such a file causes the following error.

  ERROR: zoned: failed to reset device '/home/naota/tmp/btrfs.img' zones: Inappropriate ioctl for device

Introduce btrfs_zoned_device_info->emulated to distinguish the zones are
emulated or not. And, use it to decide it needs zone reset or not.

Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-10-06 16:48:56 +02:00
David Sterba
4980de3a5f btrfs-progs: read partition size from sysfs as fallback
Reading partition size using an ioctl requires the device open, but that
does not work for unprivileged users. This leads to 0 size in device
info structures filled by device_get_partition_size.

As a consequence, this also misreports such devices as missing in 'fi
us' overview:

 $ btrfs fi us /
 WARNING: cannot read detailed chunk info, per-device usage will not be shown, run as root
 Overall:
     Device size:		 411.35GiB
     Device allocated:		  53.01GiB
     Device unallocated:	 358.34GiB
     Device missing:		 411.35GiB
     Used:			  31.99GiB
     Free (estimated):		 379.16GiB	(min: 379.16GiB)
     Free (statfs, df):		 379.35GiB
     Data ratio:		      1.00
     Metadata ratio:		      1.00
     Global reserve:		 194.77MiB	(used: 0.00B)
     Multiple profiles:		        no

There should be 0 for 'Device missing'.

Add a fallback to read the device size from sysfs in case the ioctl is
not available.

Issue: 
Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-10 13:09:05 +02:00
David Sterba
419cb3011c btrfs-progs: open code btrfs_list_get_path_rootid
The function btrfs_list_get_path_rootid is exported to libbtrfs so it
needs to stay, but we can inline the implementation.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-08 16:35:47 +02:00
David Sterba
e86425242f btrfs-progs: move send.h to kernel-shared/
The header contains the protocol definitions and is almost exactly the
same as the kernel version, move it to the proper directory.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-07 19:26:46 +02:00
David Sterba
76ab1fa364 btrfs-progs: rename and move group_profile_max_safe_loss
The helper belongs to the others that translate bg flags to the raid
attr table member.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-07 16:38:56 +02:00
David Sterba
a5becbde35 btrfs-progs: use raid attr table in group_profile_max_safe_loss
The helper open codes what we already have in the raid attr table, so
use it. We assume a valid flags so there's no error value.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-07 16:32:55 +02:00
David Sterba
45d034d774 btrfs-progs: move btrfs_tree_search2_ioctl_supported to fsfeatures.c
The helper detects a feature support, so put it to the right file.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-07 16:20:17 +02:00
David Sterba
47b04b747d btrfs-progs: move parse_qgroupid to parse utils
Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-07 14:20:42 +02:00
David Sterba
cb5a542871 btrfs-progs: factor out plain qgroupid parsing
We'll use plain qgroupid parsing function elsewhere so split that part
from parse_qgroupid_or_path. The parsing is slightly reworked and goes
from start to end, while previously it looked up the slash and worked
from there. In case a valid qgroupid is also a valid path, the path must
be specified as absolute.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-07 14:20:42 +02:00
David Sterba
ebf500b43d btrfs-progs: rename parse_qgroupid
This helper can parse a qgroupid or a path, so rename it accordingly, so
a plain qgroupid parsing can be factored out as a standalone helper.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-07 14:20:42 +02:00
David Sterba
c3ee6a8a09 btrfs-progs: unify GPL header comments
Add the GPL v2 header to files where it was missing and is not from an
external source, update to the most recent version with the address.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-07 13:58:44 +02:00
David Sterba
f3a132fa1b btrfs-progs: factor out compression type name parsing to common utils
Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-07 13:58:44 +02:00
David Sterba
31df7dc295 btrfs-progs: factor out profile parsing to common utils
There are some duplicate parsers of the profile names, factor out the
one from balance to the common code.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-07 13:58:44 +02:00
David Sterba
6b93a7336c btrfs-progs: move number and range parsing helpers to parse-utils.c
Some of the parsers in cmds/balance.c are generic enough for the common
part.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-07 13:58:43 +02:00
David Sterba
af56460de8 btrfs-progs: split parsing helpers from utils.c
There are various parsing helpers scattered everywhere, unify them to
one file and start with helpers already in utils.c.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-09-06 17:15:51 +02:00
David Sterba
a177ef7dd4 btrfs-progs: mkfs: allow degenerate raid0/raid10
Kernel patch b2f78e88052bc0bee ("btrfs: allow degenerate raid0/raid10")
in
5.15 will allow mounting and converting to single device raid0 or two
device raid10.  Let mkfs create such filesystem.

"The motivation is to allow to preserve the profile type as long as it
 possible for some intermediate state (device removal, conversion), or
 when there are disks of different size, with raid0 the otherwise
 unusable space of the last device will be used too.  Similarly for
 raid10, though the two largest devices would need to be the same."

Signed-off-by: David Sterba <dsterba@suse.com>
2021-08-27 15:40:53 +02:00
Qu Wenruo
773afad3e6 btrfs-progs: slightly enhance btrfs_format_csum()
- Change it void
  The old one always return csum_size.

- Use snprintf()

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-08-26 14:27:01 +02:00
Qu Wenruo
991a598f53 btrfs-progs: move btrfs_format_csum() to common/utils.[ch]
Function btrfs_format_csum() is a special helper only used in
btrfs-progs.

Move it to common/utils.[ch] other than leaving it in
kernel-shared/disk-io.c.

Since we're moving the code, also introduce a macro,
BTRFS_CSUM_STRING_LEN, to replace open-coded string length calculation.

Signed-off-by: Qu Wenruo <wqu@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-08-26 14:26:13 +02:00
Goldwyn Rodrigues
a7ed5b0ced btrfs-progs: correct check_running_fs_exclop() return value
check_running_fs_exclop() can return 1 when exclop is changed to "none"
The ret is set by the return value of the select() operation. Checking
the exclusive op changes just the exclop variable while ret is still
set to 1.

Set ret = 0 if exclop is set to BTRFS_EXCL_NONE or BTRFS_EXCL_UNKNOWN.
Remove unnecessary continue statement at the end of the block.

The command appears to have executed, but does not. This was found when
balance which typically reports chunks relocated did not print anything
on screen.

Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-07-02 17:27:53 +02:00
David Sterba
3d52313591 btrfs-progs: add helper to read zone size from sysfs
Sysfs hides the zone size of a block device in the queue/chunk_sectors
file, so add a helper that will read it for us when given the short
device name (that can be found in FSID/devices).

Signed-off-by: David Sterba <dsterba@suse.com>
2021-07-02 17:27:53 +02:00
David Sterba
e25c8f00a3 btrfs-progs: add helper for opening sysfs fsid directory
There are several directories in /sys/fs/btrfs/FSID that contain more
than one file/directory. Add a helper to open the directory so that the
file descriptor can be used for fdopendir.

Signed-off-by: David Sterba <dsterba@suse.com>
2021-07-02 17:27:53 +02:00
Su Yue
80a86f1b47 btrfs-progs: do not BUG_ON if btrfs_add_to_fsid succeeded to write superblock
Commit 8ef9313cf2 ("btrfs-progs: zoned: implement log-structured
superblock") changed to write BTRFS_SUPER_INFO_SIZE bytes to device.
The before num of bytes to be written is sectorsize.
It causes mkfs.btrfs failed on my 16k pagesize kvm:

  $ /usr/bin/mkfs.btrfs -s 16k -f -mraid0 /dev/vdb2 /dev/vdb3
  btrfs-progs v5.12
  See http://btrfs.wiki.kernel.org for more information.

  ERROR: superblock magic doesn't match
  ERROR: superblock magic doesn't match
  common/device-scan.c:195: btrfs_add_to_fsid: BUG_ON `ret != sectorsize`
  triggered, value 1
  /usr/bin/mkfs.btrfs(btrfs_add_to_fsid+0x274)[0xaaab4fe8a5fc]
  /usr/bin/mkfs.btrfs(main+0x1188)[0xaaab4fe4dc8c]
  /usr/lib/libc.so.6(__libc_start_main+0xe8)[0xffff7223c538]
  /usr/bin/mkfs.btrfs(+0xc558)[0xaaab4fe4c558]

  [1]    225842 abort (core dumped)  /usr/bin/mkfs.btrfs -s 16k -f -mraid0
  /dev/vdb2 /dev/vdb3

btrfs_add_to_fsid() now always calls sbwrite() to write
BTRFS_SUPER_INFO_SIZE bytes to device, so change condition of
the BUG_ON().
Also add comments for sbread() and sbwrite().

Signed-off-by: Su Yue <l@damenly.su>
Signed-off-by: David Sterba <dsterba@suse.com>
2021-05-12 16:00:14 +02:00