On a filesystem with more than 2^32 blocks, the block group checksum test will
fail because "i" (the group number) is a 32-bit quantity that is used to
calculate the group's block bitmap block number. Unfortunately, "i" is not
automatically promoted to 64-bit for this calculation and overflows. When this
happens, e2fsck will incorrectly report bitmap checksum errors.
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Commit 53e3120c18 introduced a regression which would case e2fsck to
overrun an array boundary for bigalloc file systems, and most likely
crash. Fix this by correctly using blocks instead of clusters when
incrementing the loop counter in the fast path optimization case.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Add a fast path optimization in e2fsck's pass 5 for the common case
where the block bitmap is correct. The optimization works by
extracting each block group's block allocation bitmap into a memory
buffer, and comparing it with the expected allocation bitmap using
memcmp(). If it matches, then we can just update the free block
counts and be on our way, and skip checking each bit individually.
Addresses-Google-Bug: #7534813
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Reviewed-by: Lukas Czerner <lczerner@redhat.com>
Use the helper function to determine if group descriptors have a
checksum. Ensure that metadata_csum and uninit_bg flags are not set
simultaneously, as part of pass 0.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Check block bitmap checksum and write a new checksum if the
verification fails. This is ok because e2fsck has already computed
the correct block bitmap.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Rewrite the block bitmap when the checksum doesn't match. This is
ok since e2fsck will have already computed the correct inode bitmap.
Signed-off-by: Darrick J. Wong <djwong@us.ibm.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Don't consider only an error in the superblock summary as incorrect.
The kernel does not update this field except at unmount time, so
don't print errors during a "-n" run if there is nothing else wrong.
Any other unfixed errors will themselves mark the filesystem invalid.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
We do not want to discard inode table if the underlying device does not
return zeros when reading non-provisioned blocks. The reason is that if
the inode table is not zeroed yet, then discard would not help us since
we would have to zero it anyway. In the case that inode table was
already zeroed, then the discard would cause subsequent reads to contain
non-deterministic data so we would not be able to assume that the inode
table was zeroed and we would need to zero it again, which does not
really make sense.
This commit adds check to prevent inode table from being discarded if
the discard does not zero data.
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
When argument '-n' was specified and should run in read-only mode, we
should not attempt to discard anything. In order to do that we have to
check for E2F_OPT_NO flag and clear E2F_OPT_DISCARD flag if E2F_OPT_NO
is set.
This commit fixes the problem when we would mark inode tables as zeroed
(EXT2_BG_INODE_ZEROED) even when e2fsck is running in read-only mode. We
also move the check for E2F_OPT_NO so we can clear E2F_OPT_DISCARD as
early as possible.
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The boolean expression (!skip_group || csum_flag) is always true,
since if csum_flag is FALSE, skip_group must also be FALSE. Hence, we
can just remove the expression from the conditional altogether, thus
simplifying the code and making it easier to read/understand.
Also, in the case where the bit is set in the bitmap, there's no point
repeatedly setting first_free to be ext2fs_block_count(fs->super).
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Previously when running e2fsck with '-E discard' argument the end of
the last group has not been discarded. This patch fixes it so we
always discard the end of the last group if needed.
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
When calling e2fsck with '-E discard' option it might happen that
valid inodes are discarded accidentally. This is because we just
discard the part of inode table which lies past the free inode count.
This is terribly wrong (sorry!).
This patch fixes it so only the free parts of an inode table
is discarded, leaving used inodes intact. This was tested with highly
fragmented inode tables with block size 4k and 1k.
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Reported-by: Phillip Susi <psusi@ubuntu.com>
Reviewed-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
The pass5 checks would fail if the expected and current {inode,block}
bitmaps used different back ends that returned different non-zero
values from the test_*_bitmap() functions. Fix this by changing
"(actual == bitmap)" to "(!actual == !bitmap)".
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The DEFS line in MCONFIG had gotten so long that it exceeded 4k, and
this was starting to cause some tools heartburn. It also made "make
V=1" almost useless, since trying to following the individual commands
run by make was lost in the noise of all of the defines.
So fix this by putting the configure-generated defines in lib/config.h
and the directory pathnames to lib/dirpaths.h.
In addition, clean up some vestigal defines in configure.in and in the
Makefiles to further shorten the cc command lines.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
If the blocks of a filesystem is a multiple of blocks_per_group,
blocks of the ending group is computed wrongly. Use the
new ext2fs_group_blocks_count() helper instead.
Eric Sandeen: Converted to use new blocks per group helper
Signed-off-by: Yongqiang Yang <xiaoqiangnk@gmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
If '-n' option is specified there should be no changes made to the file
system hence we should not attempt to discard the file system. This
commit adds a check into the e2fsck_discard_blocks() condition so it skip
discard if E2F_OPT_NO flag is set.
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
Treat the s_blocks_count field in the superblock as a free block count
(instead of the number of free clusters) for bigalloc file systems.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The code which simulated handling uninitialized block bitmaps didn't
take bigalloc file systems into account correctly. Fix it.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Fix several types of compiler warnings (unused variables/labels),
uninitialized variables, etc that are hit with gcc -Wall.
Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
In Pass 5 when we are checking block and inode bitmaps we have great
opportunity to discard free space and unused inodes on the device,
because bitmaps has just been verified as valid. This commit takes
advantage of this opportunity and discards both, all free space and
unused inodes.
I have added new set of options, 'nodiscard' and 'discard'. When the
underlying devices does not support discard, or discard ends with an
error, or when any kind of error occurs on the filesystem, no further
discard attempt will be made and the e2fsck will behave as it would
with nodiscard option provided.
As an addition, when there is any not-yet-zeroed inode table and
discard zeroes data, then inode table is marked as zeroed.
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
After cleaning up ext2fs_bg_flag_set() and ext2fs_bg_flag_clear(),
we're left with ext2fs_bg_flag_test(). Convert it to
ext2fs_bg_flags_test().
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The ext2fs_bg_flag* functions were confusing.
Currently we have this:
void ext2fs_bg_flags_set(ext2_filsys fs, dgrp_t group, __u16 bg_flags);
void ext2fs_bg_flags_clear(ext2_filsys fs, dgrp_t group,__u16 bg_flags);
(_set (unused) sets exactly bg_flags; _clear clears all and ignores bg_flags)
and these, which can twiddle individual bits in bg_flags:
void ext2fs_bg_flag_set(ext2_filsys fs, dgrp_t group, __u16 bg_flag);
void ext2fs_bg_flag_clear(ext2_filsys fs, dgrp_t group, __u16 bg_flag);
A better interface, after the patch below, is just:
ext2fs_bg_flags_zap(fs, group) /* zeros bg_flags */
ext2fs_bg_flags_set(fs, group, flags) /* adds flags to bg_flags */
ext2fs_bg_flags_clear(fs, group, flags) /* clears flags in bg_flags */
and remove the original ext2fs_bg_flags_set / ext2fs_bg_flags_clear.
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Blocks per group and group desc count are both 32-bit; multiplied they
produce a 32-bit quantity which overflowed.
Signed-off-by: Valerie Aurora Henson <vaurora@redhat.com>
Signed-off-by: Eric Sandeen <sandeen@redhat.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
The new ext2fs_super_and_bgd_loc2() function has some changes aside
from just blk64_t support. Lets make sure that the interfaces are
sane by adding libext2fs support early to get the new API tested here.
Signed-off-by: Jose R. Santos <jrs@us.ibm.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
e2fsck_pass5() checks whether the inode and block allocation bitmaps
are consistent. However, if EXT2_BG_[INODE/BLOCK]_BITMAP is set to a
ext4's block group, most of its bitmap is uninitialized (0). In that
case, we can optimize e2fsck's pass 5 by checking the entire range of
an uninitalized block group instead of checking bit by bit.
This can speed up e2fsck pass 5 by up to 80%:
+-----+--------------------+--------------------+
| | old e2fsck | new e2fsck |
|Pass | time(s) | time(s) |
| | real | user |system| real | user |system|
+-----+------+------+------+------+------+------+
| 1 | 5.70| 3.29| 0.50| 5.66| 3.21| 0.54|
| 2 | 3.33| 0.80| 0.19| 3.40| 0.82| 0.23|
| 3 | 0.01| 0.00| 0.00| 0.01| 0.00| 0.00|
| 4 | 1.04| 1.04| 0.00| 1.05| 1.04| 0.00|
| 5 | 19.60| 17.27| 0.06| 3.53| 1.21| 0.05|
+-----+------+------+------+------+------+------+
|Total| 29.94| 22.57| 0.80| 13.90| 6.47| 0.86|
+-----+------+------+------+------+------+------+
Comparison of e2fsck time on an ext4 500GB partition (20% blocks used)
Machine environment:
CPU: Intel(R) Xeon(TM) CPU 3.00GHz
Memory: 1GB
Kernel: linux-2.6.29-git2
Signed-off-by: Kazuya Mio <k-mio@sx.jp.nec.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
check_block_bitmap() calculates the block number of superblock in the current
block group but it's not used.
Signed-off-by: Kazuya Mio <k-mio@sx.jp.nec.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
If unused range of the bitmap has an unmarked bit, check_[inode/block]_end()
marks all bits in the range. However, we know that the checked bits are marked.
So this patch fixes loop counter to mark from the unmarked bit.
Signed-off-by: Kazuya Mio <k-mio@sx.jp.nec.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
in the case of ! defined RESOURCE_TRACK, so that we can clean up #ifdef
throughout e2fsck source.
Signed-off-by: Ken Chen <kenchen@google.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This simplifies the code, and using the uninit_bg with the inode table
lazily initialized is just as good.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This patch has all the necesary pieces to open and fix filesystems created
with the uninit block group feature.
Signed-off-by: Jose R. Santos <jrs@us.ibm.com>
Signed-off-by: Andreas Dilger <adilger@clusterfs.com>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
This patch instruments the libext2fs unix I/O manager and adds bytes
read/written and data rate to e2fsck -tt pass/overall timing output.
Signed-off-by: Jim Garlick <garlick@llnl.gov>
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
Create new functions ext2fs_{set,get}_{inode,block}_bitmap_range()
which allow programs like e2fsck, dumpe2fs, etc. to get and set chunks
of the bitmap at a time.
Move the representation details of the 32-bit old-style bitmaps into
gen_bitmap.c.
Change calls in dumpe2fs, mke2s, et. al to use the new abstractions.
Signed-off-by: "Theodore Ts'o" <tytso@mit.edu>
