Commit Graph

1723 Commits

Author SHA1 Message Date
Alessio Balsini
d4197585fa AOSP: Build e2freefrag
Enable the build of e2freefrag to monitor the free space fragmentation
in ext2/3/4 filesystems.

Bug: 146078546
Test: m + e2freefrag on device
Change-Id: Ia56e443a789ae881a03bdaeae1093567e1736c4c
Signed-off-by: Alessio Balsini <balsini@google.com>
From AOSP commit: ab77f6c79f3dab697cd56ad3b793e7d555ac9415
2020-03-20 23:13:52 -04:00
Elliott Hughes
470e78ae21 AOSP: Add -e2fsprogs to the e2fsprogs chattr and lsattr.
We want to start shipping the toybox chattr and lsattr on the device all
the time, so the build system rightly complains that then we'd have two
modules with the same name.

I went with a suffix rather than a prefix so that tab completion works
for folks still wanting to use the e2fsprogs versions.

Bug: http://b/147769529
Test: builds
Change-Id: Ib904fa6c709d29ce709302c61e452383c02cb9a3
From AOSP commit: 8525a455e7410461560a99a42feb0dbfabab5c8e
2020-03-20 23:12:24 -04:00
Theodore Ts'o
4346f675bf misc: add text describing the impact of an inode size < 128 bytes in man pages
Addresses-Debian-Bug: #953493
Addresses-Debian-Bug: #953494
Addresses-Debian-Bug: #951808
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2020-03-20 21:35:03 -04:00
Sawood Alam
6e315c931f misc: fix typos in chattr's man page
Plural form "directories" should be used along with "files".
"id's" should be "ids" (i.e., plural form, not apostrophe).
"much" should "must".

Signed-off-by: Sawood Alam <ibnesayeed@gmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2020-03-10 11:52:27 -04:00
Pino Toscano
693e05942a mke2fs.conf: remove options.fname_encoding
Introduced with commit e7236a9476,
it was later renamed to encoding, and turned into a fs_type-only
option with commit 28887533bb.

Hence, remove an option that does not exist in the default
configuration.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2020-03-09 12:36:10 -04:00
Theodore Ts'o
f106b01c98 mke2fs: fix permissions setting with "mke2fs -d /path/files"
Set the directory for directories in cases where the owner permissions
is not rwx.  This was reported[1] by Robert Yang but we are using a
different approach to fixing the issue.

[1] https://lore.kernel.org/r/1582542522-97508-1-git-send-email-liezhi.yang@windriver.com

Also set the permissions in a more portable way by making a
distinction between the host OS's permissions stats and Linux's
permissions.  We still assume the low 12 bits are the historical Unix
assignments, but we don't assume ST_IFMT bits are the same as Linux's.

Reported-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2020-03-07 12:42:39 -05:00
Andreas Dilger
e08226c708 misc: handle very large files with filefrag
Avoid overflowing the column-width calc printing files over 4B blocks.

Document the [KMG] suffixes for the "-b <blocksize>" option.

The blocksize is limited to at most 1GiB blocksize to avoid shifting
all extents down to zero GB in size.  Even the use of 1GB blocksize
is unlikely, but non-ext4 filesystems may use multi-GB extents.

Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Lustre-bug-id: https://jira.whamcloud.com/browse/LU-13197
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2020-03-04 18:26:09 -05:00
Andreas Dilger
102993ec1c e2fsck: fix overflow if more than 4B inodes
Even though we don't have support for filesystems with over 4B inodes
in the current e2fsprogs, this may happen in the future.  There are
latent overflow bugs when calculating the number of inodes in the
filesystem that can trivially be fixed now, rather than waiting for
them to be hit at some point in the future.  The block number calcs
are already correct in this code.

Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Lustre-bug-id: https://jira.whamcloud.com/browse/LU-13197
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2020-02-29 18:34:54 -05:00
Andreas Dilger
382ed4a1c2 e2fsck: use proper types for variables
Use ext2_ino_t instead of ino_t for referencing inode numbers.
Use loff_t for for file offsets, and dgrp_t for group numbers.

Cast products to ssize_t before multiplication to avoid overflow.

Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Reviewed-by: Shilong Wang <wshilong@ddn.com>
Lustre-bug-id: https://jira.whamcloud.com/browse/LU-13197
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2020-02-29 18:26:38 -05:00
Jeremy Visser
a5676f8da7 chattr.1: improve attribute formatting with labels and indented paragraphs
By convention, lists of options in man pages use a label followed by an
indented description, such as this example from the Options section:

     -R     Recursively change attributes of directories and
            their contents.

But the Attributes section places the available attributes mid-sentence,
which makes it visually more difficult to parse:

     A file with the 'a' attribute set can only be opened
     in append mode for writing.  [...]

     When a file with the 'A' attribute set is accessed, its
     atime record is not modified.  [...]

This patch places a label beside each attribute description, which (in
my opinion) improves readability, especially when visually skimming the
list.  For example:

     a      A file with the 'a' attribute set can only be
            opened in append mode for writing.

     A      When a file with the 'A' attribute set is accessed,
            its atime record is not modified.

Signed-off-by: Jeremy Visser <jeremyvisser@google.com>
Reviewed-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2020-02-24 15:31:16 -05:00
Andreas Dilger
bc56227376 mmp: abstract out repeated 'sizeof(buf), buf' usage
The printf("%.*s") format requires both the buffer size and buffer
pointer to be specified for each use.  Since this is repeatedly given
as "(int)sizeof(buf), (char *)buf" for mmp_nodename and mmp_bdevname
fields, with typecasts to avoid compiler warnings.

Add a helper macro EXT2_LEN_STR() to avoid repeated boilerplate code.

This can also be used for other superblock buffer fields that may not
have NUL-terminated strings (e.g. s_volume_name, s_last_mounted,
s_{first,last}_error_func, s_mount_opts) to simplify code and avoid
the need for temporary buffers for NUL-termination.

Annotate the superblock string fields that may not be NUL-terminated.

Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2020-01-24 23:06:58 -05:00
Andreas Dilger
6b430d60ca mmp: don't assume NUL termination for MMP strings
Don't assume that mmp_nodename and mmp_bdevname are NUL terminated,
since very long node/device names may completely fill the buffers.

Limit string printing to the maximum buffer size for safety, and
change the field definitions to __u8 to make it more clear that
they are not NUL-terminated strings, as is done with other strings
in the superblock that do not have NUL termination.

Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2020-01-24 23:06:58 -05:00
Li Dongyang
d23f88ebaa mke2fs: fix setting bad blocks in the block bitmap
We mark the bad blocks as used on fs->block_map before allocating
group tables.  Don't translate the block number to cluster number when
doing this, the fs->block_map is still a block-granularity allocation
map, it will be coverted later by ext2fs_convert_subcluster_bitmap().

Signed-off-by: Li Dongyang <dongyangli@ddn.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2020-01-01 13:41:35 -05:00
Benno Schulenberg
6607eaad26 e2image: remove redundant -fr from man page and usage message
Also, add a missing dash and two missing brackets and two missing
spaces, and remove three excess spaces.

Signed-off-by: Benno Schulenberg <bensberg@telfort.nl>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2020-01-01 13:41:35 -05:00
Theodore Ts'o
3a15d85bfb ext2fs: add ext2fs_get_stat_i_blocks() function
The function ext2fs_inode_i_blocks() is a bit confusing whether it is
returning the inode's i_blocks value, or whether it is returning the
value ala the stat(2) system call, which returns i_blocks in units of
512 byte sectors.  This caused ext2fs_inode_i_blocks() to be
incorrectly used in fuse2fs and the function quota_compute_usage().

To address this, we add a new function, ext2fs_get_stat_i_blocks()
which is clearly labelled what it is returning, and use it in fuse2fs
and quota_compute_usage().  It's also a bit more convenient to use it
in e2fsck, so use it there too.

Reported-by: Wang Shilong <wangshilong1991@gmail.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2020-01-01 13:41:28 -05:00
Theodore Ts'o
dae1ecc244 fuse2fs: add support for 32-bit uids and gids
Previously, uids were truncated at 16 bits because we weren't properly
handling i_uid_high and i_gid_high.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-12-20 15:16:09 -05:00
Theodore Ts'o
23f94dde25 mke2fs: fix "mke2fs -d /path/to/files" to support 32-bit uids and gids
https://github.com/tytso/e2fsprogs/issues/29

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-12-20 14:58:41 -05:00
Theodore Ts'o
bb7a3dc251 Add a program to test images provided by UBSAN fuzzing reports
This program calls a few ext2fs library functions used by the current
generation of libext2fs fuzzers, and is helpful in reproducing UBSAN
failures reported externally.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-11-22 23:27:47 -05:00
Darrick J. Wong
3f21d82395 tune2fs: prohibit toggling uninit_bg on live filesystems
An internal customer followed an erroneous AskUbuntu article[1] to try to
change the UUID of a live ext4 filesystem.  The article claims that you
can work around tune2fs' "cannot change UUID on live fs" error by
disabling uninit_bg, changing the UUID, and re-enabling the feature.

This led to metadata corruption because tune2fs' journal descriptor
rewrite races with regular filesystem writes.  Therefore, prevent
administrators from turning on or off uninit_bg on a mounted fs.

[1] https://askubuntu.com/questions/132079/how-do-i-change-uuid-of-a-disk-to-whatever-i-want/195839#459097

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-11-20 22:58:55 -05:00
Eric Biggers
446483e398 chattr.1: say "cleared" instead of "reset"
Setting a bit to 0 is normally called "clearing", not "resetting"; and
chattr.1 already says "clear" in some places.  Use "clear" consistently.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-11-18 12:14:33 -05:00
Eric Biggers
f08f1c6492 chattr.1: clarify that ext4 doesn't support tail-merging either
This old text was never updated to mention ext4 in addition to ext2 and
ext3.  Do so now.  Also don't bother to mention old unmerged patches.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-11-18 12:14:33 -05:00
Eric Biggers
26d6e57c5c chattr.1: fix some grammatical errors
- "can only be open" => "can only be opened"
- "is not candidate" => "is not a candidate"
- "written ... on the disk" => "written ... to the disk"

Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-11-18 12:14:33 -05:00
Eric Biggers
6e58704680 chattr.1: add casefold attribute to mode string
When the casefold attribute ('F') was added to the chattr man page, it
was forgotten to add it to the mode string.  Add it.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-11-18 12:14:33 -05:00
Eric Biggers
93cea43534 chattr.1: adjust documentation for encryption attribute
Adjust the documentation for the encryption attribute ('E') to clarify
that encryption isn't experimental anymore and isn't restricted to
regular files, and that the encryption is done by the filesystem.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-11-18 12:14:33 -05:00
Eric Biggers
be335e31e7 chattr.1: document the verity attribute
Document the verity file attribute ('V').

Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-11-18 12:14:33 -05:00
Theodore Ts'o
6dac306792 Fix UBSan when shifting (1LL << 63)
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-11-04 21:22:54 -05:00
Eric Biggers
040cd2479d tune2fs.8: tweak the documentation for the encrypt feature
Try to make it clearer that enabling 'encrypt' just enables *support*
for encryption; it doesn't actually encrypt anything by itself.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-09-23 13:28:36 -04:00
Eric Biggers
5f1e5f5e88 tune2fs.8: document the verity feature
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-09-23 13:28:36 -04:00
Eric Biggers
4ef530d95a ext4.5: tweak the documentation for the encrypt feature
Try to make it clearer that enabling 'encrypt' just enables *support*
for encryption; it doesn't actually encrypt anything by itself.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-09-23 13:28:36 -04:00
Eric Biggers
4cc90574e4 ext4.5: document the verity feature
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-09-23 13:28:36 -04:00
Eric Biggers
4ecbee2cfd ext4.5: document first kernel version to support casefold feature
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-09-23 13:28:36 -04:00
Eric Biggers
fea937ee09 ext4.5: move casefold feature to correct position
The features are listed in alphabetic order, so put the casefold feature
in the right place.

Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-09-23 13:28:36 -04:00
Theodore Ts'o
aa6d326079 fuse2fs: stop using the nonempty option by default
The nonempty option isn't supported by fuse3, and so if fusermount is
from fuse3, having fuse2fs specify nonempty automatically will prevent
fuse2fs from working correctly.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-07-14 12:03:29 -04:00
Theodore Ts'o
b71150355d fuse2fs: install fuse2fs in /usr/bin instead of /usr/sbin
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-07-14 11:05:47 -04:00
Theodore Ts'o
c2eedf431e Fix posix_memalign and posix_fadvise calls.
Almost all posix_ functions return a positive errno value (without
setting errno) rather than -1 and setting errno. Most calls in this
project were correct, but these two weren't.

Reported-by: Hughes <enh@google.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-06-07 13:07:12 -04:00
Theodore Ts'o
0e0dad426a mke2fs: accept the english yes character to the proceed question
In some cases if the translation file is missing some translations,
mke2fs can end up printing an English message, e.g.:

% LANG=it_IT.UTF-8 ./mke2fs /tmp/foo.img 8M
mke2fs 1.45.1 (12-May-2019)
/tmp/foo.img contiene un file system ext4
	created on Mon May 27 19:35:48 2019
Proceed anyway? (y,N)

However, if there is a translation for string to match with "yY"
(e.g., to "sS" for Italian), then 'y' won't work.  Fix this by falling
back to the english 'yY' characters.

Addresses-Debian-Bug: #907034
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-05-27 19:36:15 -04:00
Theodore Ts'o
6cfb145e34 mke2fs: fix memory leak when parsing encoding flags
Also fix extended-options usage string.

Addresses-Coverity-Bug: 1444984
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-05-12 02:23:08 -04:00
Andreas Dilger
f6cf3e6193 mke2fs: fix check for absurdly large devices
The check in mke2fs is intended to be for the number of blocks in the
filesystem exceeding the maximum number of addressable blocks in 2^32
bitmaps, which is (2^32 * 8 bits/byte * blocksize) = 2^47 blocks,
or 2^59 bytes = 512PiB for the common 4KiB blocksize.

However, s_log_blocksize holds log2(blocksize_in_kb), so the current
calculation is a factor of 2^10 too small.  This caused mke2fs to fail
while trying to format a 900TB filesystem.

Fixes: 101ef2e93c ("mke2fs: Avoid crashes / infinite loops for absurdly large devices")
Signed-off-by: Andreas Dilger <adilger@dilger.ca>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-05-06 10:15:41 -04:00
Theodore Ts'o
28887533bb Rename the feature "fname_encoding" to be "casefold".
Also change mke2fs so that the encoding and encoding flags are
specified in mke2fs.conf in the fs_types and defaults stanzas instead
of the options stanza.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-05-03 13:16:29 -04:00
Gabriel Krisman Bertazi
1384050b7a ext4.5.in: document design changes on the casefold attribute
Document the fact that the encoding support is only used by directories
with the +F attribute.

Signed-off-by: Gabriel Krisman Bertazi <krisman@collabora.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-04-28 16:53:08 -04:00
Theodore Ts'o
f680342869 misc: fix spelling error
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-03-06 22:20:31 -05:00
Artem Blagodarenko
3f3e455f02 e2image: add -b and -B options to specify where to find the superblock
e2image has no ability to use superblock backup to copy metadata.
This feature can be useful if someone wants to make partition
image and fix it using e2fsck utility.

New -b option allows to pass superblock number, like e2fsck utility does.
e2image doesn't change primary superblock and store it as is, so
it can be fixed using e2fsck latter. Option -B allows setting
superblock size.

Signed-off-by: Artem Blagodarenko <c17828@cray.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-03-06 11:52:18 -05:00
Theodore Ts'o
731661a739 Merge branch 'maint' into next 2019-03-06 00:06:42 -05:00
Yoshitaka Seto
f6f4b4b39f AOSP: misc: use scandir with alphasort instead of readdir for consistency
During the image generation, When using the e2fsdroid with src_dir,
then dir files are listed with readdir, which will not guarantee
the order across the machines/fielsystems. So instead using the
scandir with alphasort option to list the files in sorted order.

Bug: 122874817

Change-Id: I4b946c737319252b82c74a0e10360843503862a1
From AOSP commit: 4cbe059b48b46d6657e24e4cdef543da7537dba3

Reported-by: Yasuhiro Kubota <yasuhiro.1.kubota@sony.com>
Tested-by: Yasuhiro Kubota <yasuhiro.1.kubota@sony.com>
Signed-off-by: Vikram Dattu Thota <vikram.dattu@sony.com>
Signed-off-by: Takuya Ogawa <takuya.ogawa@sony.com>
Signed-off-by: Yoshitaka Seto <yoshitaka.seto@sony.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-03-05 13:06:12 -05:00
Chen Lin Z
66bae937b9 AOSP: Fix file offset overflow issue when file's size > 4G
fs->blocksize is int(4 bytes), while data is off_t(8 bytes),
'data_blk = data & ~(fs->blocksize - 1)' will cause data_blk
lose high 4 bytes of data if data > 4G and it'll cause file
inconsistent when using -d option to populate ext4 image file.

[ This was also fixed upstream via 1eec741367: "create_inode: fix
  copying large files".  This commit is just to clean up
  whitespace/formatting issues. -- tytso ]

Signed-off-by: Chen Lin Z <lin.z.chen@intel.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
From AOSP commit: 999dd56f2586fadec7bfe846b8cb52c5e528248f
2019-03-05 12:56:45 -05:00
Lukas Czerner
21dde7ba35 create_inode: fix potential memory leak in path_append()
If realloc() fails in path_append() we will lose a memory pointed to by
target->path.  Fix it.

path_append() is used by mke2fs and e2fsdroid.

Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-02-11 12:17:16 -05:00
Lukas Czerner
7d671e66cc e2freefrag: fix memory leak in scan_online()
Signed-off-by: Lukas Czerner <lczerner@redhat.com>
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2019-02-11 11:53:29 -05:00
Theodore Ts'o
a935b93dca Merge branch 'maint' into next 2019-02-10 21:50:26 -05:00
Andreas Gampe
5f6614f32e AOSP: E2fsprogs: fix ODR issue
Do not use whole_static_libs and shared_libs at the same time. Fix
up clients.

Signed-off-by: Theodore Ts'o <tytso@mit.edu>

Bug: 123590866
Test: mmma external/e2fsprogs
Change-Id: I36d184dd878de2beacbb17c306e47076b44d68bb
From AOSP commit: fece156b105719c782351e447298de073d4dab04
2019-02-07 22:41:25 -05:00
Sandeep Patil
3e953c0a03 AOSP: Build filefrag
Signed-off-by: Theodore Ts'o <tytso@mit.edu>

Bug: 121211685
Test: mma -j with lunch aosp_blueline-userdebug

Change-Id: If8cffae8168990a11e8228b9f00d2fee15924935
Signed-off-by: Sandeep Patil <sspatil@google.com>
From AOSP commit: 0393dd541c79dfda1475ef071a31215dd2811162
2019-02-07 22:33:50 -05:00