The existing bool definition is broken for c23, where bool is now a keyword.
Signed-off-by: Elliott Hughes <enh@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
We can add a device aliasing file which can map the whole device with an
extent, not using node blocks. This mapped area should be pinned and
normally used for read-only usages. After finished using it, we can
deallocate the whole area and return it back to use it for other files.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch enables support for write hints by segment type.
Signed-off-by: Daejun Park <daejun7.park@samsung.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
If there are double '.' or '..' dirents in directory, fsck.f2fs won't
detect and repaire the issue correctly, fix it.
Reviewed-by: Sheng Yong <shengyong@oppo.com>
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Should finish zones if they are open.
Reviewed-by: Daeho Jeong <daehojeong@google.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch introduces a new tool inject.f2fs to modify metadata or data
(directory entry) of f2fs image offline flexibly.
With inject.f2fs, it is easier to generate a corrupted f2fs image, which
can help verify fsck or reproduce userspace behaviors of some a fault.
If option `--dry-run' is used, nothing really gets changed, and that
could be used to get the value of a specified field.
The following patches enable injecting each part of f2fs.
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Sheng Yong <shengyong@oppo.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
If a superblock failed in sanity check, it should be fixed. This patch
add a new state `sb_invalid' to tell fsck needs to update superblock
at the end of all checkings.
This patch also cleans up force_stop, abnormal_stop, fs_errors and
sb_invalid by merging them into an `invalid_sb' flags, and each of
them is indicated using one bit.
Signed-off-by: Sheng Yong <shengyong@oppo.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Introduce a new help addrs_per_page() to wrap common code
from addrs_per_inode() and addrs_per_block() for cleanup.
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch fixes some cases in where we missed to consider
endianness when we access/update inode.i_flags.
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Previously, dumped symlinks would always create regular files instead.
This allows symlinks to be dumped as symlinks with the -L option.
The i_name field's name may not be the same as the actual name from the
dirent, so we use the dirent name when available.
Currently hardlinks aren't detected, so print a warning if we notice a
nondirectory with a link count over 1.
Signed-off-by: Daniel Rosenberg <drosen@google.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
When using the f2fs_io tool to set_verity, it will fail as follows:
unisc:/data # ./f2fs_io set_verity file
FS_IOC_ENABLE_VERITY: Inappropriate ioctl for device
this is because commit: 95ae251fe828 ("f2fs: add fs-verity support"),
the passed parameters do not match the latest kernel version.
After patch:
unisoc:/data # ./f2fs_io set_verity file
Set fsverity bit to file
unisoc:/data # ./f2fs_io getflags file
get a flag on file ret=0, flags=verity
Fixes: 95ae251fe828 ("f2fs: add fs-verity support")
Signed-off-by: Xiuhong Wang <xiuhong.wang@unisoc.com>
Signed-off-by: Zhiguo Niu <zhiguo.niu@unisoc.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
The reserved segments should be aligned to the section boundary.
Reviewed-by: Daeho Jeong <daehojeong@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Add a new stop cp reason STOP_CP_REASON_NO_SEGMENT for keeping
consistent with kernel codes.
Signed-off-by: Zhiguo Niu <zhiguo.niu@unisoc.com>
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
generic/019 50s ... _check_generic_filesystem: filesystem on /dev/vdc is inconsistent
(see /media/fstests/results//generic/019.full for details)
[FSCK] Max image size: 16196 MB, Free space: 188 MB
[FSCK] Unreachable nat entries [Ok..] [0x0]
[FSCK] SIT valid block bitmap checking [Ok..]
[FSCK] Hard link checking for regular file [Ok..] [0x166]
[FSCK] valid_block_count matching with CP [Ok..] [0x3ecfe7]
[FSCK] valid_node_count matching with CP (de lookup) [Ok..] [0x4c79]
[FSCK] valid_node_count matching with CP (nat lookup) [Ok..] [0x4c79]
[FSCK] valid_inode_count matched with CP [Ok..] [0xb46]
[FSCK] free segment_count matched with CP [Ok..] [0x9d]
[FSCK] next block offset is free [Ok..]
[FSCK] fixing SIT types
[FSCK] other corrupted bugs [Ok..]
Error: Could not conduct fsync!!!
Generic/019 will trigger fsync() on a clean image, but it will fail
due to simulated failure on disk, result in testcase failure.
Let's add c.need_fsync to record dirty status of image, and only trigger
fsync() when there is dirty data in image.
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
If we write CP_UMOUNT_FLAG in fsck, f2fs will not do foll forward recovery
even though it has to do.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
We need to make sure to finish all the zones except six open zones. In
a case of that the previous mount wasn't successfully unmounted, we have
to change all the current segments.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
We don't have lots of benefits from this, since block devices already
have page caches in the kernel. Moreover, this feature has some
stability issues corrupting metadata. So, we better make this deprecated.
Signed-off-by: Daeho Jeong <daehojeong@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This adds support for 4K and 16K block size using the same binary.
mkfs can choose block size via the -b option, with other tools getting
the blocksize from the superblock.
On mount time, we can't rely on block size for the location for the
superblock, since that information is in the superblock. If the first
superblock is corrupt, we will attempt to find the second superblock at
a 4K block offset, and then a 16K block offset if that fails.
Signed-off-by: Daniel Rosenberg <drosen@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This moves access to f2fs_dentry_block's dentry list and filename list
behind a macro, as their locations depend on block size.
Since struct f2fs_dentry_block no longer represents the full block, use
F2FS_BLKSIZE instead of sizeof(struct f2fs_dentry_block)
Signed-off-by: Daniel Rosenberg <drosen@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This splits off access to the summary block's journal and footer into a
macro call, as their location is dependent on block size. Because of
this you should use F2FS_BLKSIZE instead of sizeof(struct summary_block)
Signed-off-by: Daniel Rosenberg <drosen@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This converts f2fs_nat_block and f2fs_sit_block to be variable length
arrays. This does not change the way they are accessed, but removes a
misleading statment that these sizes are fixed, as opposed to deriving
from F2FS_BLKSIZE
Signed-off-by: Daniel Rosenberg <drosen@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This converts inodes, direct nodes, and indirect nodes over to not being
based on a 4K page size. f2fs_inode's i_nids field should now be
accessed via a macro, as it's location depends on block size.
Access to direct nodes and indirect nodes is unchanged.
The node footer's location is also based on block sized, and accessed
via a macro. With these changes, you should use F2FS_BLKSIZE in place of
sizeof(struct f2fs_node)
Signed-off-by: Daniel Rosenberg <drosen@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This splits off access to the orphan block's footer into a macro call
as its location is dependent on block size. Because of this, you should
use F2FS_BLKSIZE instead of sizeof(struct f2fs_orphan_block)
Signed-off-by: Daniel Rosenberg <drosen@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This converts the various block size based constants to be defined in
terms of the block size. This makes it possible to change the block size
by changing only F2FS_BLKSIZE_BITS in f2fs_fs.h
Signed-off-by: Daniel Rosenberg <drosen@google.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
fsck.f2fs prints i_gc_failures reset message w/ [FIX] tags, it's not
appropriate due to reset i_gc_failures is not a fix.
Let's add INFO_MSG() macro to print such important non-fix message.
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch replaces lseek64 with lseek() having #define _FILE_OFFSET_BITS 64.
Signed-off-by: Waldemar Brodkorb <wbx@openadk.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
In file included from external/f2fs-tools/lib/nls_utf8.c:29:
external/f2fs-tools/include/f2fs_fs.h:1781:44: error: call to undeclared function 'S_ISDIR'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
raw_node->i.i_current_depth = cpu_to_le32(S_ISDIR(mode) ? 1 : 0);
Reviewed-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
This patch supports to recognize ERROR_INVALID_NODE_REFERENCE error
recorded in superblock during fsck, and force fsck to fix related
issue.
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
During formatting device, node/data block and filesystem metatdata related
to same inode were updated in separated functions, e.g. for root inode:
- f2fs_write_root_inode() updates root inode's block
- f2fs_write_check_point_pack updates its nat/sit journal, summary block
Result in:
- there are some redundant codes in between f2fs_write_check_point_pack()
and functions in f2fs_create_root_dir().
- readability is bad due to filesystem meta and data/node updates in
separated places.
- bad scalability if we want to add more default inode.
So that, this patch introduces below functions:
- update_sit_journal
- update_nat_jounral
- update_summary_entry
Then use them to update inode related metadata into cache in below
functions:
- f2fs_write_root_inode
- f2fs_add_default_dentry_root
- f2fs_write_qf_inode
- f2fs_write_default_quota
- f2fs_write_lpf_inode
- f2fs_write_lpf_inode
Finally, f2fs_write_check_point_pack() can write back cached metadata
into CP area directly.
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Introduce alloc_next_free_block() to wrap below openned codes:
blkaddr = get_sb(main_blkaddr) +
c.cur_seg[curseg_type] * c.blks_per_seg +
c.curseg_offset[curseg_type];
Meanwhile add curseg_offset field in f2fs_configuration to record
last blkaddr in each log.
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Disk quota file is controlled by filesystem quota subsystem, access time
of quota file is almost random, and should be meanless to user, so let's
add noatime to quota file.
Meanwhile, set quota file w/ F2FS_IMMUTABLE_FL instead of FS_IMMUTABLE_FL,
since F2FS_IMMUTABLE_FL is on-disk flag, however FS_IMMUTABLE_FL is in-memory
one.
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Introduce DISP_u8(), and use DISP_u16() and DISP_u8() to print below fields:
- i_compress_algorithm
- i_log_cluster_size
- i_compress_flag
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Otherwise, it may trigger static assert below when the structure
updates.
static_assert(sizeof(struct f2fs_super_block) == 3072, "")
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
Overview
========
This option allows zoned block device users to configure GC reserved and
overprovision area manually according to their demands on performance of
sustained write latency and WAF.
Problem
=======
The overprovision segments that mkfs generates are mostly occupied by GC
reserved. This degrades WAF performance.
Experiment
==========
The following experiment evaluated the application of configurable reserved.
The experimental environment is as follows.
System info
- 4.2Ghz, 8 core CPU
- 64GiB Memory
Device info
- a conventional null_blk with 448MiB capacity(meta area) and
- a sequential null_blk with 953 zones of 64MiB
Format
- as-is (find out ovp ratio): mkfs.f2fs <conv null_blk> -c <seq null_blk> -m
Info: Overprovision ratio = 3.700%
Info: Overprovision segments = 1152 (GC reserved = 1088)
- config rsvd: mkfs.f2fs <conv null_blk> -c <seq null_blk> -m -Z 8 -o 2.965
Info: Overprovision ratio = 2.965%
Info: Overprovision segments = 1152 (GC reserved = 256)
Mount
- mount <conv null_blk> <mount point>
Fio script
- fio --rw=randwrite --bs=4k --ba=4k --filesize=58630m --norandommap --overwrite=1 --name=job1 --filename=<mount point>/sustain --time_based --runtime=2h
WAF calculation
- (IOs on conv. null_blk + IOs on seq. null_blk) / random write IOs
Conclusion
==========
In the experiment, it can be shown that reducing the reserved segments
decreases WAF to 10% (from 222 to 23) although it triggers checkpoint more
frequently during gc. With direct IO, the WAF of as-is gets much higher.
In other words, a user can configure more reserved segments for lower GC
latency or allocate less reserved segments for lower WAF on the same number
of OP segments.
Signed-off-by: Yonggil Song <yonggil.song@samsung.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
cp_payload is set differently [1] when extended node bitmap feature is
enabled. Commit b79c3ba4ea broke fsck on f2fs file systems created on
2+ TB device with extended node bitmap feature enabled.
As the sanity check is for checking overflows, fix this to assume the max
possible cp_payload size under the extended node bitmap.
Link: https://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-tools.git/tree/mkfs/f2fs_format.c?h=v1.15.0#n372 [1]
Fixes: b79c3ba4ea ("fsck.f2fs: sanity check cp_payload before reading checkpoint")
Reported-by: Alexander Koskovich <akoskovich@pm.me>
Signed-off-by: Juhyung Park <qkrwngud825@gmail.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
As f2fs becomes more resilient for GCs, let's give the marginal overprovision
space back to user.
Fix an issue where reserved_space > ovp_space, reported by Shinichiro.
Signed-off-by: Shinichiro Kawasaki <shinichiro.kawasaki@wdc.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
In auto/preen mode, let's trigger repairing if filesystem has
inconsistent errors.
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
In auto/preen mode, let's trigger repairing if filesystem was forced
to stop.
Signed-off-by: Chao Yu <chao@kernel.org>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
