Adjust to output the file name if any name parameter is passed.
This is consistent with sum -s, cksum, and sum implementations
on other platforms. This should not cause significant compat
issues, as multiple fields are already output, and so already
need to be parsed.
* src/sum.c (bsd_sum_file): Output the file name
if any name parameter is passed.
* tests/misc/sum.pl: Adjust accordingly.
* doc/coreutils.texi (sum invocation): Likewise.
* NEWS: Mention the change in behavior.
Emil Lundberg <lundberg.emil@gmail.com> reports in
https://bugs.gnu.org/49741 about a 'basenc --base64 -d' decoding bug.
The input buffer length was not divisible by 3, resulting in
decoding errors.
* NEWS: Mention fix.
* src/basenc.c (DEC_BLOCKSIZE): Change from 1024*5 to 4200 (35*3*5*8)
which is divisible by 3,4,5,8 - satisfying both base32 and base64;
Use compile-time verify() macro to enforce the above.
* tests/misc/basenc.pl: Add test.
This patch modifies basenc to prefer signed integers to
unsigned, as signed are less error-prone.
This patch also updates Gnulib to to latest, which updates Gnulib’s
base32 and base64 modules to prefer signed to unsigned integers.
* src/basenc.c: Include idx.h.
(struct base2_decode_context): Use unsigned char, not unsigned
for an octet that must fit in an unsigned char.
(base_encode, struct base_decode_context)
(base64_decode_ctx_wrapper, prepare_inbuf, base64url_encode)
(base64url_decode_ctx_wrapper, base32_decode_ctx_wrapper)
(base32hex_encode, base32hex_decode_ctx_wrapper, base16_encode)
(base16_decode_ctx, z85_encode, Z85_HI_CTX_TO_32BIT_VAL)
(z85_decoding, z85_decode_ctx, base2msbf_encode)
(base2lsbf_encode, base2lsbf_decode_ctx, base2msbf_decode_ctx)
(wrap_write, do_encode, do_decode, main):
Prefer signed integers to unsigned.
(main): Treat extremely large wrap columns as if they were
infinite; that’s good enough. Since we’re now using xstrtoimax,
this allows ‘-w -0’ (same as ‘-w 0’).
* tests/misc/base64.pl (gen_tests): -w-0 is no longer an error.
This is so that commands like ‘fmt - -’ read from stdin
both times, even when it is a tty. Fix some other minor
issues that are related.
* src/blake2/b2sum.c (main):
* src/cksum.c (cksum):
* src/cut.c (cut_file):
* src/expand-common.c (next_file):
* src/fmt.c (fmt):
* src/fold.c (fold_file):
* src/md5sum.c (digest_file, digest_check):
* src/nl.c (nl_file):
* src/od.c (check_and_close):
* src/paste.c (paste_parallel, paste_serial):
* src/pr.c (close_file):
* src/sum.c (bsd_sum_file):
Use clearerr on stdin so that stdin can be read multiple times
even if it is a tty. Do not assume that ferror preserves errno as
POSIX does not guarantee this. Coalesce duplicate diagnostic
calls.
* src/blake2/b2sum.c (main):
* src/fmt.c (main, fmt):
Report read error, even if it's merely fclose failure.
* src/fmt.c: Include die.h.
(fmt): New arg FILE. Close input (reporting error) if not stdin.
All callers changed.
* src/ptx.c (swallow_file_in_memory): Clear stdin's EOF flag.
* src/sort.c (xfclose): Remove unnecessary feof call.
Problem reported by Michael Debertol (Bug#50070).
* NEWS: Mention the fix.
* src/chmod.c (struct change_status): New struct, replacing the
old enum Change_status. All uses changed.
(describe_change): Distinguish between cases depending on
whether 'stat' or its equivalent succeeded. Report a line
of output even if 'stat' failed, as that matches the documentation.
Rework to avoid casts.
(process_file): Do not output nonsense modes computed from
uninitialized storage, removing a couple of IF_LINTs. Simplify by
defaulting to CH_NO_STAT.
If the command-line argument is automounted, df would use
stat info that became wrong after the following open.
* NEWS: Mention the fix (bug#50012).
* src/df.c (automount_stat_err): New function.
This fixes the hang on fifos in a better way, by using O_NONBLOCK.
(main): Use it.
We must delay handling when \r is the last character
of the buffer being processed, as the next character
may or may not be \n.
* src/cat.c (pending_cr): A new global to record whether
the last character processed (in -E mode) is '\r'.
(cat): Honor pending_cr when processing the start of the buffer.
(main): Honor pending_cr if no more files to process.
* tests/misc/cat-E.sh: Add test cases.
Fixes https://bugs.gnu.org/49925
Pacify GCC 11.1 -fanalyzer.
* src/uniq.c (check_file): Use simpler test to check whether this
is the first time through the loop. Although the old test was
correct, the new one is easier to understand and perhaps a tiny
bit more efficient.
Caught by GCC 11.1 -fanalyzer.
* src/numfmt.c (simple_strtod_int): Remove unnecessary test of
*endptr vs NULL. Presumably this was a typo and **endptr was
intended instead of *endptr, but an **endptr test is also
unnecessary since c_isdigit (0) returns false.
In documentation and comments, don’t assume that secondary storage
devices are disk devices. Similarly, don’t assume that main memory
uses magnetic cores, which became obsolete in the 1970s.
* src/du.c (usage):
* src/ls.c (usage):
* src/shred.c (usage): Reword to avoid “disk” in usage messages.
* doc/coreutils.texi (ls invocation): Document implementation more
closely. Be more consistent about style. Omit some needless words.
* src/ls.c (usage): Don’t overdocument -f, as the details were wrong.
Omit -1 advice as it’s a bit obsolete now that we have --zero and
is a bit much for --usage output anyway.
* NEWS, doc/coreutils.texi (General output formatting):
* src/ls.c (usage):
Document this.
* src/ls.c (ZERO_OPTION): Rename from NULL_OPTION.
All uses changed.
(long_options): Rename --null to --zero.
(dired_dump_obstack, main, print_dir): Use '\n' instead of
eolbyte where eolbyte must equal '\n'.
(decode_switches): Decode --zero instead of --null.
--zero also implies -1, -N, --color=none, --show-control-chars.
Use easier-to-decipher code to set ‘format’ and ‘dired’.
Reject attempts to combine --dired and --zero.
* tests/local.mk: Adjust to test script renaming.
* tests/ls/zero-option.sh: Rename from tests/ls/null-option.sh,
and test --zero instead of --null.
* src/ls.c (enum time_type, enum sort_type, enum indicator_style)
(enum Dereference_symlink, ignore_mode):
Put ‘= 0’ after default values, since the code relies
on static storage defaulting to zero.
(enum sort_type): Reorder so that -1 can be used to represent unset.
(main): Test print_with_color after parse_ls_color may have reset it.
(decode_line_length): Return the line length instead of setting
static storage. All uses changed. Treat line lengths exceeding
PTRDIFF_MAX as infinite, to avoid pointer-subtraction glitches.
(stdout_isatty): New function, to avoid calling isatty twice.
(decode_switches): Calculate defaults more lazily, to avoid using
syscalls or getenv during startup unless the results are more
likely to be needed. Use -1 to indicate options that haven’t been
set on the command line yet. Move print_with_color test from
here to ‘main’. Suppress bogus GCC warning.
(getenv_quoting_style): Return the quoting style instead of
setting static storage.
(init_column_info): New arg MAX_COLS, to avoid recalculating it.
Caller changed.
* src/ls.c (dired_pos): Now off_t, not size_t, since it counts
output file offsets.
(dired_dump_obstack): This obstack's file offsets are now
off_t, not size_t.
(format_user_or_group, format_user_or_group_width):
ID arg is now uintmax_t, not unsigned long, since uid_t and
gid_t values might exceed ULONG_MAX.
(format_user_or_group_width): Use snprintf with NULL instead of
sprintf with a discarded buffer. This avoids a stack buffer,
and so should be safer.
Prefer functions or constants to macros where either will do.
That’s cleaner, and nowadays there’s no performance reason to
prefer macros. All uses changed.
* src/ls.c (INITIAL_TABLE_SIZE, MIN_COLUMN_WIDTH):
Now constants instead of macros.
(file_or_link_mode): New function, replacing the old macro
FILE_OR_LINK_MODE.
(dired_outbyte): New function, replacing the old macro DIRED_PUTCHAR.
(dired_outbuf): New function, replacing the old macro DIRED_FPUTS.
(dired_outstring): New function, replacing the old macro
DIRED_FPUTS_LITERAL.
(dired_indent): New function, replacing the old macro DIRED_INDENT.
(push_current_dired_pos): New function, replacing the old macro
PUSH_CURRENT_DIRED_POS.
(assert_matching_dev_ino): New function, replacing the old macro
ASSERT_MATCHING_DEV_INO.
(do_stat, do_lstat, stat_for_mode, stat_for_ino, fstat_for_ino)
(signal_init, signal_restore, cmp_ctime, cmp_mtime, cmp_atime)
(cmp_btime, cmp_size, cmp_name, cmp_extension)
(fileinfo_name_width, cmp_width, cmp_version):
No longer inline; compilers can deduce this well enough nowadays.
(main): Protect unused assert with ‘if (false)’ rather than
commenting it out, so that the compiler checks the code.
(print_dir): Output the space and newline in the same buffer
as the human-readable number they surround.
(dirfirst_check): New function, replacing the old macro
DIRFIRST_CHECK. Simplify by using subtraction.
(off_cmp): New function, replacing the old macro longdiff.
(print_long_format): No need to null-terminate the string now.
(format_user_or_group): Let printf count the bytes.
* src/ls.c (format_user_or_group_width, print_long_format):
Use return value from sprintf instead of calling strlen on
the resulting buffer, or inferring the length some other way.
As originally reported in <https://bugzilla.redhat.com/1962515>,
df invoked without -a printed duplicated entries for NFS mounts
of bind mounts. This is a regression from commit v8.25-54-g1c17f61ef99,
which introduced the use of a hash table.
The proposed patch makes sure that the devlist entry seen the last time
is used for comparison when eliminating duplicated mount entries. This
way it worked before introducing the hash table.
Patch co-authored by Roberto Bergantinos.
* src/ls.c (struct devlist): Introduce the seen_last pointer.
(devlist_for_dev): Return the devlist entry seen the last time if found.
(filter_mount_list): Remember the devlist entry seen the last time for
each hashed item.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/49298
This fixes an unlikely stack out-of-bounds write reported by
Stepan Broz via Kamil Dudka (Bug#49209).
* bootstrap.conf (gnulib_modules): Replace select with poll.
* src/tail.c: Do not include <sys/select.h>.
[!_AIX]: Include poll.h.
(check_output_alive) [!_AIX]: Use poll instead of select.
(tail_forever_inotify): Likewise. Simplify logic, as there is no
need for a ‘while (len <= evbuf_off)’ loop.
* src/stat.c (default_format): Use decomposed decimal
representation (major,minor) in the default format.
This is least ambiguous for human interpretation,
and more consistent with ls for example.
Fixes https://bugs.gnu.org/48960
In preparation for changing the default device number
representation (to decomposed decimal), provide more
formatting options for device numbers.
These new (FreeBSD compat) formatting options are added:
%Hd major device number in decimal (st_dev)
%Ld minor device number in decimal (st_dev)
%Hr major device type in decimal (st_rdev)
%Lr minor device type in decimal (st_rdev)
%r (composed) device type in decimal (st_rdev)
%R (composed) device type in hex (st_rdev)
* doc/coreutils.texi (stat invocation): Document new formats.
* src/stat.c (print_it): Handle the new %H and %L modifiers.
(print_statfs): Adjust to passing the format as two chars
rather than an int. Using an int was introduced in commit db42ae78,
but using separate chars is cleaner and more extensible.
(print_stat): Likewise. Handle any modifiers and the new 'r' format.
(usage): Document the new formats.
* tests/misc/stat-fmt.sh: Add a test case for new modifiers.
Addresses https://bugs.gnu.org/48960
Coreutils mistakenly did not list xstrndup as a module
that it depends on directly. When the latest Gnulib removed
the dirname module's dependency on xstrndup, this mistake
caused coreutils to not build. Since all of Coreutils's
uses of xstrndup know the string length, xmemdup0 is a better
match for what's needed. Since the size args are typically
signed or derived from subtracting pointers, the new Gnulib
ximemdup0 function is a better match yet.
So, use ximemdup0 instead of xstrndup.
* src/cut.c, src/dircolors.c, src/expand-common.c, src/expand.c:
* src/numfmt.c, src/set-fields.c, src/unexpand.c:
Do not include xstrndup.h; no longer needed.
* src/dircolors.c (parse_line):
* src/expand-common.c (parse_tab_stops):
* src/numfmt.c (parse_format_string):
* src/set-fields.c (set_fields):
Use ximemdup0 instead of xstrndup.
This is now only used on 10 year old linux kernels,
and performs a sync before each copy.
* src/copy.c (extent_copy): Remove function and all callers.
* src/extent-scan.c: Remove.
* src/extent-scan.h: Remove.
* src/fiemap.h: Remove.
* src/local.mk: Adjust for removed files.
* NEWS: Adjust to say fiemap is removed.
copy_file_range() before Linux kernel release 5.3 had many issues,
as described at https://lwn.net/Articles/789527/, which was
referenced from https://lwn.net/Articles/846403/; a more general
article discussing the generality of copy_file_range().
Linux kernel 5.3 was released in September 2019, which is new enough
that we need to actively avoid older kernels.
* src/copy.c (functional_copy_file_range): A new function
that returns false for Linux kernels before version 5.3.
(sparse_copy): Call this new function to gate use of
copy_file_range().
* src/copy.c (sparse_copy): Upon EPERM from copy_file_range(),
fall back to a standard copy, which will give a more accurate
error as to whether the issue is with the source or destination.
Also this will avoid the issue where seccomp or apparmor are
not configured to handle copy_file_range(), in which case
the fall back standard copy would succeed without issue.
This specific issue with seccomp was noticed for example in:
https://github.com/golang/go/issues/40900
* src/copy.c (infer_scantype): Ensure we don't error out
if SEEK_DATA returns EOPNOTSUPP, on systems where this value
is distinct from ENOTSUP. Generally both of these should be checked.
* src/copy.c (sparse_copy): Ensure we fall back to
a standard copy if copy_file_range() returns ENOTSUP.
This generally is best checked when checking ENOSYS,
but it also seems to be a practical concern on Centos 7,
as a quick search gave https://bugzilla.redhat.com/1840284
Ensure we call hash_free() to avoid valgrind and leak_sanitizer
"definitely lost" warnings. These were not real leaks as
we terminate immediately after, but we should avoid these
"definitely lost" warnings where possible.
* src/copy.c: Add dest_info_free() and src_info_free().
* src/copy.h: Declare the above.
* src/cp-hash.c: Don't define unless "lint" is defined.
* src/install.c: Call dest_info_free() in dev mode.
* src/mv.c: Likewise.
* src/cp.c: Likewise. Also call src_info_free().
* src/ln.c: Call hash_free() in dev mode.
* src/tail.c: Call hash_free() even if about to exit, in dev mode.
Fixes https://bugs.gnu.org/48189
* src/copy.c (sparse_copy): Don't use copy_file_range()
with --reflink=never as copy_file_range() may implicitly
use acceleration techniques like reflinking.
(extent_copy): Pass through whether we allow reflinking.
(lseek_copy): Likewise.
Fixes https://bugs.gnu.org/48164
* src/wc.c: (main): Handle the new --debug option.
Only call avx2_supported if needed.
(avx2_supported): Diagnose various failures and attempts.
* NEWS: Mention the new wc improvement and --debug option.
Use cpuid to detect CPU support for avx2 instructions.
Performance was seen to improve by 5x for a file with only newlines,
while the performance for a file with no such characters is unchanged.
* configure.ac [USE_AVX2_WC_LINECOUNT]: A new conditional,
set when __get_cpuid_count() and avx2 compiler intrinsics are supported.
* src/wc.c (avx2_supported): A new function using __get_cpuid_count()
to determine if avx2 instructions are supported.
(wc_lines): A new function refactored from wc(),
which implements the standard line counting logic,
and provides the fallback implementation for when avx2 is not supported.
* src/wc_avx2.c: A new module to implement using avx2 intrinsics.
* src/local.mk: Reference the new module. Note we build as a separate
lib so that it can be portably built with separate -mavx2 etc. flags.
Problem reported by Roland (Bug#48106).
* src/touch.c (touch): Take more care when deciding whether
to use open_errno or utime_errno in the diagnostic.
Stop worrying about SunOS 4 (which as part of the problem),
as it’s long obsolete. For Solaris 10, verify that EINVAL
really means the file was a directory.
* src/csplit.c (load_buffer):
* src/pinky.c (create_fullname):
Use intprops-based checks rather than xalloc_oversized,
since Gnulib xalloc.h no longer includes xalloc-oversized.h.
* src/copy.c (sparse_copy): Fallback to read() if copy_file_range()
fails with ETXTBSY. Otherwise it would be impossible to copy files
that are being used as swap. This used to work before introducing
the support for copy_file_range() in coreutils. (Bug#48036)
* doc/coreutils.texi (ln invocation): State --symbolic is required.
* src/ln.c (usage): Explicitly state -s is not implied.
Fixes https://bugs.gnu.org/47703
* src/wc.c (usage): State that only printable characters are considered
when counting words. This also disambiguates wether we're talking
about bytes or characters in this context.
* doc/coreutils.texi (wc invocation): Likewise. Also clarify
that --characters counts valid locale aware characters,
and that --lines does not count a trailing "line" unless
it ends with a newline character.
Fixes https://bugs.gnu.org/47702
This is especially important now for --sort=width,
as that can greatly increase how often this
expensive quote_name_width() function is called per file.
This also helps the default invocation of ls,
or specifically the --format={across,vertical} cases
(when --width is not set to 0),
to avoid two calls to this function per file.
Note the only case where we later compute the width,
is for --format=commas. That's only done once though,
so we leave the computation close to use to
maximize hardware caching.
* src/ls.c (struct fileinfo): Add a WIDTH member to cache
the screen width of the file name.
(update_current_files_info): Set the WIDTH members for cases
they're needed multiple times. Note we do this explicitly here,
rather than caching at use, so that the fileinfo
structures can remain const in the sorting and presentation functions.
(sort_files): Call the new update_current_files_info() in this
initialization function.
(fileinfo_name_width): Renamed from fileinfo_width,
and adjusted to return the cached value if available.
This helps identify the outliers for long filenames, and also produces
a more compact display of columns when listing a directory with many
entries of various widths.
* src/ls.c (sort_type, sort_types, sort_width): New sort_width sort
type.
(sort_args): Add "width" sort arg.
(cmp_width, fileinfo_width): New sort function and helper for file name
width.
(quote_name_width): Add function prototype declaration.
(usage): Document --sort=width option.
* doc/coreutils.texi: Document --sort=width option.
* tests/ls/sort-width-option.sh: New test for --sort=width option.
* tests/local.mk: Reference new test.
* NEWS: Mention the new feature.
