This patch enables support for write hints by segment type.
Signed-off-by: Daejun Park <daejun7.park@samsung.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Should finish zones if they are open.
Reviewed-by: Daeho Jeong <daehojeong@google.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Introduce a new help addrs_per_page() to wrap common code
from addrs_per_inode() and addrs_per_block() for cleanup.
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
BLKFINISHZONE may not defined in old blkzoned.h, and cause build error.
Signed-off-by: Wu Bo <bo.wu@vivo.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Previously, dumped symlinks would always create regular files instead.
This allows symlinks to be dumped as symlinks with the -L option.
The i_name field's name may not be the same as the actual name from the
dirent, so we use the dirent name when available.
Currently hardlinks aren't detected, so print a warning if we notice a
nondirectory with a link count over 1.
Signed-off-by: Daniel Rosenberg <drosen@google.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This fixes some android build failures due to the missing permission when
checking the loop device. Until we get a better solution, let's ignore
the error with warnings.
Reviewed-by: Huang Jianan <huangjianan@xiaomi.com>
Reviewed-by: Daeho Jeong <daehojeong@google.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
When we turn on sparse_mode, stat() will be failed, as the file will be
created afterwards. Let's fix.
Fixes: 14197d546b ("f2fs-tools: fix to check loop device")
Reviewed-by: Daeho Jeong <daehojeong@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
If sparse files is set along with multiple devices, we initialize
sparse file data multiple times without freeing the previously allocated
data. This skips the initialization for subsequent devices, as sparse
file mode currently only deals with device 0 anyways.
Signed-off-by: Daniel Rosenberg <drosen@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Since we may not know the block size when initializing sparse files, we
should assume that the sparse file's blocksize is correct.
Signed-off-by: Daniel Rosenberg <drosen@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
generic/019 50s ... _check_generic_filesystem: filesystem on /dev/vdc is inconsistent
(see /media/fstests/results//generic/019.full for details)
[FSCK] Max image size: 16196 MB, Free space: 188 MB
[FSCK] Unreachable nat entries [Ok..] [0x0]
[FSCK] SIT valid block bitmap checking [Ok..]
[FSCK] Hard link checking for regular file [Ok..] [0x166]
[FSCK] valid_block_count matching with CP [Ok..] [0x3ecfe7]
[FSCK] valid_node_count matching with CP (de lookup) [Ok..] [0x4c79]
[FSCK] valid_node_count matching with CP (nat lookup) [Ok..] [0x4c79]
[FSCK] valid_inode_count matched with CP [Ok..] [0xb46]
[FSCK] free segment_count matched with CP [Ok..] [0x9d]
[FSCK] next block offset is free [Ok..]
[FSCK] fixing SIT types
[FSCK] other corrupted bugs [Ok..]
Error: Could not conduct fsync!!!
Generic/019 will trigger fsync() on a clean image, but it will fail
due to simulated failure on disk, result in testcase failure.
Let's add c.need_fsync to record dirty status of image, and only trigger
fsync() when there is dirty data in image.
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This issue comes from a static code scanning tool.
When c.sparse_mode is 1, stat_buf will not be initialized,
but it will be used next.
If this issue does not require modification, please ignore this commit.
Signed-off-by: zangyangyang1 <zangyangyang1@xiaomi.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Need to initialize user cache valid values to prevent from providing
false cache entries.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
We need to make sure to finish all the zones except six open zones. In
a case of that the previous mount wasn't successfully unmounted, we have
to change all the current segments.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
We don't have lots of benefits from this, since block devices already
have page caches in the kernel. Moreover, this feature has some
stability issues corrupting metadata. So, we better make this deprecated.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This adds support for 4K and 16K block size using the same binary.
mkfs can choose block size via the -b option, with other tools getting
the blocksize from the superblock.
On mount time, we can't rely on block size for the location for the
superblock, since that information is in the superblock. If the first
superblock is corrupt, we will attempt to find the second superblock at
a 4K block offset, and then a 16K block offset if that fails.
Signed-off-by: Daniel Rosenberg <drosen@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This converts inodes, direct nodes, and indirect nodes over to not being
based on a 4K page size. f2fs_inode's i_nids field should now be
accessed via a macro, as it's location depends on block size.
Access to direct nodes and indirect nodes is unchanged.
The node footer's location is also based on block sized, and accessed
via a macro. With these changes, you should use F2FS_BLKSIZE in place of
sizeof(struct f2fs_node)
Signed-off-by: Daniel Rosenberg <drosen@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch replaces lseek64 with lseek() having #define _FILE_OFFSET_BITS 64.
Signed-off-by: Waldemar Brodkorb <wbx@openadk.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
f2fs_dentry_hash() may return -ENOMEM due to malloc() failure, however
caller won't check validaty of return value, result assign or check
wrong hash value, fix to call assert() for such case.
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Overview
========
This option allows zoned block device users to configure GC reserved and
overprovision area manually according to their demands on performance of
sustained write latency and WAF.
Problem
=======
The overprovision segments that mkfs generates are mostly occupied by GC
reserved. This degrades WAF performance.
Experiment
==========
The following experiment evaluated the application of configurable reserved.
The experimental environment is as follows.
System info
- 4.2Ghz, 8 core CPU
- 64GiB Memory
Device info
- a conventional null_blk with 448MiB capacity(meta area) and
- a sequential null_blk with 953 zones of 64MiB
Format
- as-is (find out ovp ratio): mkfs.f2fs <conv null_blk> -c <seq null_blk> -m
Info: Overprovision ratio = 3.700%
Info: Overprovision segments = 1152 (GC reserved = 1088)
- config rsvd: mkfs.f2fs <conv null_blk> -c <seq null_blk> -m -Z 8 -o 2.965
Info: Overprovision ratio = 2.965%
Info: Overprovision segments = 1152 (GC reserved = 256)
Mount
- mount <conv null_blk> <mount point>
Fio script
- fio --rw=randwrite --bs=4k --ba=4k --filesize=58630m --norandommap --overwrite=1 --name=job1 --filename=<mount point>/sustain --time_based --runtime=2h
WAF calculation
- (IOs on conv. null_blk + IOs on seq. null_blk) / random write IOs
Conclusion
==========
In the experiment, it can be shown that reducing the reserved segments
decreases WAF to 10% (from 222 to 23) although it triggers checkpoint more
frequently during gc. With direct IO, the WAF of as-is gets much higher.
In other words, a user can configure more reserved segments for lower GC
latency or allocate less reserved segments for lower WAF on the same number
of OP segments.
Signed-off-by: Yonggil Song <yonggil.song@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Change to calculate and export max image size with valid blocks
used of image rather than last used block of image.
[FSCK] Max image size: xx MB, Free space: xx MB
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Make the argument list match the format string. Use PRIu64 for uint64_t
and %zu for size_t.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Change the type of the third argument of f2fs_report_zone() from void *
into struct blk_zone * to enable type checking.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
The definition of struct blk_zone_report is as follows:
struct blk_zone_report {
__u64 sector;
__u32 nr_zones;
__u32 flags;
struct blk_zone zones[0];
};
Since f2fs_report_zone() allocates the above data structure with
malloc() and since f2fs_report_zone() only initializes the sector and
nr_zones members, the flags member is not initialized. Modify
f2fs_report_zone() such that 0 is passed as flags to the
BLKREPORTZONE ioctl instead of a random value. This has been
discovered by reading the source code.
Cc: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Fixes: 6d7c7b785f ("libf2fs_zoned: Introduce f2fs_report_zone() helper function")
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/os/base.h:128:23: error: missing ')' after 'do'
^~~~~~~~~~~
external/f2fs-tools/include/f2fs_fs.h:81:24: note: expanded from macro 'fallthrough'
~~ ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk/usr/include/os/base.h:128:22: note: to match this '('
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
It is recommended to test a HAVE_* macro instead of directly testing the
host type in source code. Hence this patch.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
The <sparse/sparse.h> header is available in Android but not in the
Android NDK. Hence this patch that only includes the sparse header file
if it is available.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This makes it easier to build f2fs-tools for Windows.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Many format strings use one of the PRI* macros. These macros are compatible
with the uint types but not with the u_int types. Hence this patch
that switches from the u_int to the uint types.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
f2fs currently only work with zoned storage devices with a zone
size which is a power of 2 (PO2). So check if a non-power of 2
zoned device is found, and if so disallow its use. This prevents
users from incorrectly using these devices.
This is a non-issue today given today's kernel does not allow NPO2
zoned devices to exist as a block device. But NPO2 zoned devices do exist
so proactively put a stop-gap measure in place to prevent it from being
incorrectly used.
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>
Signed-off-by: Pankaj Raghav <p.raghav@samsung.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
In previous, we did a full scan right after kernel version is changed
and it triggered many unnecessary full scan whenever a newly built
kernel is downloaded. Plus, we are lack of a periodic health check of
on-disk filesystem images. So, I added a monthly periodic check after
a new vesion is updated.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
