The newly added zoned mode constants can utilize the const ilog2
version. Copy it from kernel include/linux/log2.h.
Signed-off-by: David Sterba <dsterba@suse.com>
Implement a zoned chunk and device extent allocator. One device zone
becomes a device extent so that a zone reset affects only this device
extent and does not change the state of blocks in the neighbor device
extents.
To implement the allocator, we need to extend the following functions for
a zoned filesystem:
- init_alloc_chunk_ctl
- dev_extent_search_start
- dev_extent_hole_check
- decide_stripe_size
Here, dev_extent_hole_check() is newly introduced to check the validity of
a hole found.
init_alloc_chunk_ctl_zoned() is mostly the same as regular one. It always
set the stripe_size to the zone size and aligns the parameters to the zone
size.
dev_extent_search_start() only aligns the start offset to zone boundaries.
We don't care about the first 1MB like in regular filesystem because we
anyway reserve the first two zones for superblock logging.
dev_extent_hole_check_zoned() checks if zones in given hole are either
conventional or empty sequential zones. Also, it skips zones reserved for
superblock logging.
With the change to the hole, the new hole may now contain pending extents.
So, in this case, loop again to check that.
Finally, decide_stripe_size_zoned() should shrink the number of devices
instead of stripe size because we need to honor stripe_size == zone_size.
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Superblock (and its copies) is the only data structure in btrfs which has a
fixed location on a device. Since we cannot overwrite in a sequential write
required zone, we cannot place superblock in the zone. One easy solution
is limiting superblock and copies to be placed only in conventional zones.
However, this method has two downsides: one is reduced number of superblock
copies. The location of the second copy of superblock is 256GB, which is in
a sequential write required zone on typical devices in the market today.
So, the number of superblock and copies is limited to be two. Second
downside is that we cannot support devices which have no conventional zones
at all.
To solve these two problems, we employ superblock log writing. It uses two
adjacent zones as a circular buffer to write updated superblocks. Once the
first zone is filled up, start writing into the second one. Then, when
both zones are filled up and before starting to write to the first zone
again, reset the first zone.
We can determine the position of the latest superblock by reading write
pointer information from a device. One corner case is when both zones are
full. For this situation, we read out the last superblock of each zone, and
compare them to determine which zone is older.
The following zones are reserved as the circular buffer on ZONED btrfs.
- primary superblock: offset 0B (and the following zone)
- first copy: offset 512G (and the following zone)
- Second copy: offset 4T (4096G, and the following zone)
If these reserved zones are conventional, superblock is written fixed at
the start of the zone without logging.
Currently, superblock reading/writing is done by pread/pwrite. This
commit replace the call sites with sbread/sbwrite to wrap the functions.
For zoned btrfs, btrfs_sb_io which is called from sbread/sbwrite
reverses the IO position back to a mirror number, maps the mirror number
into the superblock logging position, and do the IO.
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Run a zoned filesystem on non-zoned devices. This is done by "slicing
up" the block device into fixed-sized chunks and emulate a conventional
zone on each of them. The emulated zone size is determined from the size
of device extent.
This is mainly aimed at testing of zoned filesystems, i.e. the zoned
chunk allocator, on regular block devices.
Currently, we always use EMULATED_ZONE_SIZE (256MiB) for the emulated
zone size. In the future, this will be customized by mkfs option.
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Get the zone information (number of zones and zone size) from all the
devices, if the volume contains a zoned block device. To avoid costly
run-time zone report commands to test the device zones type during block
allocation, it also records all the zone status (zone type, write
pointer position, etc.).
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Marking BUG() unreachable helps us silence unnecessary warnings e.g.
"warning: control reaches end of non-void function [-Wreturn-type]" like
the code below.
int foo()
{
...
if (XXX)
return 0;
else if (YYY)
return 1;
else
BUG();
}
Signed-off-by: Naohiro Aota <naohiro.aota@wdc.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Discovered with cppcheck. Fix signed/unsigned int mismatches, sizeof and
long formats.
Pull-request: #197
Signed-off-by: Rosen Penev <rosenp@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
uClibc does not provide backtrace() nor <execinfo.h>. When building
btrfs-progs, passing --disable-backtrace is enough to make it build with
uClibc. But once btrfs-progs is installed and another program/library
includes kerncompat.h, it fails to build with uClibc, because
BTRFS_DISABLE_BACKTRACE is not defined.
The most correct fix for this would be to have kerncompat.h generated
from kerncompat.h.in during the btrfs-progs build process, and tuned
depending on autoconf/automake variables. But as a quick fix that
follows the current strategy, we simply tweak the existing __GLIBC__
conditional. Indeed, uClibc pretends to be glibc and defines __GLIBC__,
but it does not replace it completely, hence the need to define
BTRFS_DISABLE_BACKTRACE when __GLIBC__ is not defined *or* when
__UCLIBC__ is defined.
Pull-request: #206
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
[Retrieved from: https://git.buildroot.net/buildroot/tree/package/btrfs-progs/0002-kerncompat.h-define-BTRFS_DISABLE_BACKTRACE-when-bui.patch]
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The CI build prints a lot of warnings
[CC] btrfs.o
In file included from volumes.h:22,
from btrfs.c:22:
kerncompat.h:39: warning: "__always_inline" redefined
#define __always_inline __inline __attribute__ ((__always_inline__))
so define the macro conditionally.
Signed-off-by: David Sterba <dsterba@suse.com>
The 'swap' is too generic and clashes with some userspace tools that
compile against btrfs, eg. snapper when including
boost::smart_ptr::scoped_array after kerncompat.h:
/usr/include/boost/smart_ptr/scoped_array.hpp:127:13: error: macro "swap" requires 2 arguments, but only 1 given
a.swap(b);
Signed-off-by: David Sterba <dsterba@suse.com>
To help implement free space tree checker in user space some kernel
function are necessary, namely iterating/deleting/adding freespace
items, some internal search functions. Functions to populate a block
group based on the extent tree. The code is largely copy/paste from
the kernel with locking eliminated (i.e free_space_lock). It supports
reading/writing of both bitmap and extent based FST trees.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
This commit pulls those portions of the kernel implementation of
delayed refs which are necessary to have them working in user-space.
I've done the following modifications:
1. Replaced all kmem_cache_alloc calls to kmalloc.
2. Removed all locking-related code, since we are single threaded in
userspace.
3. Removed code which deals with data refs - delayed refs in user space
are going to be used only for cowonly trees.
Signed-off-by: Nikolay Borisov <nborisov@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
In latest linux api headers, __bitwise is already defined in
/usr/include/linux/types.h.
So kerncompat.h will re-define __bitwise, and cause gcc warning.
Fix it by checking if __bitwise is already define.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Commit bc2d4ccc46 "btrfs-progs:
kerncompat: disconnect assert and warning messages" forgot to add the
print_trace call to assert_trace.
Signed-off-by: David Sterba <dsterba@suse.com>
The assertion and warn/bug have reversed condition checks, using the
same helpers drops the exact value, so we'll print the message directly
without the helpers.
Signed-off-by: David Sterba <dsterba@suse.com>
We want to pass unmodified condition down to the handlers so we can't
use assert_trace for that purpose.
Signed-off-by: David Sterba <dsterba@suse.com>
Due to commit 00e769d04c2c83029d6c71(btrfs-progs: Correct value printed
by assertions/BUG_ON/WARN_ON), which changed the assert_trace()
parameter, the condition passed to assert/WARN_ON/BUG_ON are logical
notted for backtrace enabled and disabled case.
Such behavior makes us easier to pass value wrong, and in fact it did
cause us to pass wrong condition for ASSERT().
Instead of passing different conditions for ASSERT/WARN_ON/BUG_ON()
manually, this patch will use ASSERT() to implement the resting
ASSERT/WARN_ON/BUG() for disable backtrace case, and use assert_trace()
to implement ASSERT() and BUG_ON(), to allow them to print correct
value.
Also, move WARN_ON() out of the ifdef branch, as it's completely the
same for both branches.
Cc: Goldwyn Rodrigues <rgoldwyn@suse.de>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Regression introduced by a2f7af94ab
"btrfs-progs: subvol_uuid_search: return error encoded pointer"
IS_ERR() will only check if it's an error code, won't check if it's
NULL. And for all the caller the commit modifies, it can return NULL
and makes cause segfault.
Fix it by introducing new IS_ERR_OR_NULL() macro, and for NULL pointer
and needs to return int case, convert NULL pointer to -ENOENT.
Reported-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Tested-by: Tsutomu Itoh <t-itoh@jp.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Code reduction. Call warning_trace from assert_trace in order to
reduce the printf's used. Also, trace variable in warning_trace()
is not required because it is already handled by BTRFS_DISABLE_BACKTRACE.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
The values passed to BUG_ON/WARN_ON are negated(!) and printed, which
results in printing the value zero for each bug/warning. For example:
volumes.c:988: btrfs_alloc_chunk: Assertion `ret` failed, value 0
This is not useful. Instead changed to print the value of the parameter
passed to BUG_ON()/WARN_ON(). The value needed to be changed to long
to accomodate pointers being passed.
Also, consolidated assert() and BUG() into ifndef.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Temporarily make the build checks optional. The structure sizes could
change on arches due to alignment requirements or padding inserted into
the structures. We need more extensive tests to make sure we'd not break
ioctls.
Signed-off-by: David Sterba <dsterba@suse.com>
If we disable backtrace, btrfs-progs can't be compiled since we don't
have warning_trace defined.
Fix by move it out of #ifndef BTRFS_DISABLE_BACKTRACE block.
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
[ move warning_trace to the right place ]
Signed-off-by: David Sterba <dsterba@suse.com>
Curretnly WARN_ON would crash but that's not it's purpose. Add helper
that prints the warning, optionally with trace.
Signed-off-by: David Sterba <dsterba@suse.com>
Copy le_test_bit() from the kernel and use that for the free space tree
bitmaps.
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Sterba <dsterba@suse.com>
kerncompat.h header file is part of libbtrfs API. min/max macros cause
conflict while building projects dependant on libbtrfs. Moving those
macros to btrfs-progs internal header file fixes the conflict.
Signed-off-by: Ondrej Kozina <okozina@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Enhance chunk validation:
1) Num_stripes
We already have such check but it's only in super block sys chunk
array.
Now check all on-disk chunks.
2) Chunk logical
It should be aligned to sector size.
This behavior should be *DOUBLE CHECKED* for 64K sector size like
PPC64 or AArch64.
Maybe we can found some hidden bugs.
3) Chunk length
Same as chunk logical, should be aligned to sector size.
4) Stripe length
It should be power of 2.
5) Chunk type
Any bit out of TYPE_MAS | PROFILE_MASK is invalid.
With all these much restrict rules, several fuzzed image reported in
mail list should no longer cause btrfsck error.
Reported-by: Vegard Nossum <vegard.nossum@oracle.com>
Signed-off-by: Qu Wenruo <quwenruo@cn.fujitsu.com>
Signed-off-by: David Sterba <dsterba@suse.com>
- limits.h must be included to pick up PATH_MAX.
- remove double declaration of BTRFS_DISABLE_BACKTRACE
kerncompat.h assumed that if __GLIBC__ was not defined,
it could safely define BTRFS_DISABLE_BACKTRACE, however this can be
defined by the configure script. Added a check to ensure it is not
defined first.
Signed-off-by: Brendan Heading <brendanheading@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.com>
Originally broke in commit c2691f807d
__glibc__ should have been __GLIBC__
We also include features.h ; although most includes (at least stdlib.h)
typically already include it -- at least on glibc, where it matters.
Signed-off-by: Merlijn Wajer <merlijn@wizzup.org>
Reported-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Disable backtrace and define __always_inline when glibc is not used as
libc. This, together with some header changes allows btrfs-progs to
compile with musl-libc.
Signed-off-by: Merlijn Wajer <merlijn@wizzup.org>
Signed-off-by: David Sterba <dsterba@suse.cz>
While debugging a broken fs we were seeing hangs in the rb_erase loops. The
rbtree was simple and wasn't corrupted so it appeared to be a bug in our rbtree
library. Updating to the kernels latest rbtree code made the infinite loop go
away, so pull it back. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
This patch pulls back backref.c, adds a couple of helpers everywhere that it
needs, and cleans up backref.c to fit in btrfs-progs. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
[removed free_some_buffers after "do not reclaim extent buffer"]
Signed-off-by: David Sterba <dsterba@suse.cz>
This commit adds the support for a make variable named
"DISABLE_BACKTRACE" which allows to disable the support for backtrace()
usage on ASSERT(), BUG() and BUG_ON() calls.
This is useful because some alternative C libraries like uClibc have
optional support for backtrace() which is rarely built when debugging
isn't taking place.
Signed-off-by: Gustavo Zacarias <gustavo@zacarias.com.ar>
Signed-off-by: David Sterba <dsterba@suse.cz>
Currently these macros just tie to assert(), which gives us line number and such
but no backtrace so no actual context. This patch adds support for spitting out
a backtrace so we can see how we got to the given assert. Thanks,
Signed-off-by: Josef Bacik <jbacik@fb.com>
[backtrace_symbols_fd]
Signed-off-by: Naohiro Aota <naota@elisp.net>
[minor fixups]
Signed-off-by: David Sterba <dsterba@suse.cz>
Fix following build warnings on 32bit platform:
...
utils.c:1708:3: warning: left shift count >= width of
type [enabled by default]
if (x << i & (1UL << 63))
^
qgroup-verify.c:393:9: warning: cast to pointer from integer
of different size [-Wint-to-pointer-cast]
return (struct tree_block *)unode->aux;
^
qgroup-verify.c:407:38: warning: cast from pointer to integer
of different size [-Wpointer-to-int-cast]
if (ulist_add(tree_blocks, bytenr, (unsigned long long)block, 0) >= 0)
^
cmds-restore.c:120:4: warning: format %lu expects argument of type
long unsigned int, but argument 3 has type size_t [-Wformat=]
fprintf(stderr, "bad compress length %lu\n", in_len);
...
BTW, this patch also switches other castings with new helpers.
Signed-off-by: Wang Shilong <wangshilong1991@gmail.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Change the definition of BUG() to use assert instead of abort to
provide information about the location of the issue.
Signed-off-by: Mitch Harder <mitch.harder@sabayonlinux.org>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <clm@fb.com>
There were a few problems that were breaking sparse checking:
- We were defining CHECK_ENDIAN late in the environment, after
linux/fs.h has been included which defines __force and __bitwise in
confusing ways that conflict with ours. Define it up with __CHECKER__
so that linux/fs.h and our copy are acting on the same input.
- We had manually set a few of gcc's internal defines to give to sparse.
It's easier to just ask gcc for all the defines it sets and hand those
to sparse.
- We weren't passing the same *FLAGS to sparse as we were to CC.
- glibc has so many errors with FORTIFY turned on that sparse gives up
and doesn't show us any errors from our code. It's a questionable
hack to always turn on FORTIFY ourselves, so we'll just not do that
when building with sparse.
And add a nice '[SP]' quiet output line for sparse checks.
Signed-off-by: Zach Brown <zab@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
The _una_ struct's entire job is to pass an argument to le*_to_cpu. So
it's a little embarassing that it uses a native cpu types and generates
endian warnings.
ctree.h:1616:1: warning: incorrect type in assignment (different base types)
ctree.h:1616:1: expected unsigned long long [unsigned] [usertype] x
ctree.h:1616:1: got restricted __le64 [usertype] <noident>
Signed-off-by: Zach Brown <zab@redhat.com>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
The <ext2fs/ext2_types.h> header does attempt to avoid conflicts with
<linux/types.h>, but on ppc64, <asm-generic/int-ll64.h> gets somehow
included by other headers.
Include <linux/types.h> explicitly, so that <ext2fs/ext2_types.h>
notices it. The proper fix would be to fix <ext2fs/ext2_types.h> to not
use its own typedefs.
Originally observed in btrfs-convert, put the include into kerncompat.h
to avoid future problems.
Signed-off-by: Michal Marek <mmarek@suse.cz>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
Must use the version provided by the compiler in stddef.h header
Signed-off-by: Cristian Rodríguez <crrodriguez@opensuse.org>
Signed-off-by: David Sterba <dsterba@suse.cz>
Signed-off-by: Chris Mason <chris.mason@fusionio.com>
