* doc/coreutils.texi (md5sum invocation): Describe the new option,
and how it's not supported by --check, and how it disables escaping.
* src/md5sum.c (delim): A new global to parmeterize the out delimiter.
(main): Don't enable file name escaping with -z, and output '\0'.
* tests/misc/md5sum-newline.pl: Add a test case.
* NEWS: Mention the new feature.
===== Benchmark setup (on GNU/Linux) ====
$ yes áááááááááááááááááááá | head -n100000 > mbc.txt
$ yes 12345678901234567890 | head -n100000 > num.txt
===== Before ====
$ time src/wc -Lm < mbc.txt
real 0m0.186s
$ time src/wc -m < mbc.txt
real 0m0.186s
$ time src/wc -Lm < num.txt
real 0m0.055s
$ time src/wc -m < num.txt
real 0m0.056s
==== After ====
$ time src/wc -Lm < mbc.txt
real 0m0.196s
$ time src/wc -m < mbc.txt
real 0m0.173s
$ time src/wc -Lm < num.txt
real 0m0.031s
$ time src/wc -m < num.txt
real 0m0.028s
* src/wc.c (wc): Only call wide variant functions like
iswprint() and wcwidth() for non is_basic() characters.
I.E. non ISO C "basic character set" characters.
This is especially significant on OSX where wcwidth()
is very expensive (about 10x in tests).
* NEWS: Mention the improvement.
Suggested by Eric Fischer.
This issue was introduced in commit v8.19-145-g24ebca6
* src/copy.c (copy_internal): With --no-preserve=mode,
only reset permissions for newly created files.
(copy_reg): Likewise.
* NEWS: Mention the fix.
* tests/cp/preserve-mode.sh: Add a test case.
Fixes https://bugs.gnu.org/31675
* src/copy.c (copy_internal): Don't fail immediately upon
getting ELOOP when running stat() on the destination,
rather proceeding if -f specified, allowing the link
to be removed. If the loop is not in the final component
of the destination path, we still fail but at the
subsequent unlink() stage.
* doc/coreutils.texi (cp invocation): Adjust wording to say
that --force doesn't work with dangling links, rather than
all links that can't be traversed.
* tests/cp/thru-dangling.sh: Add a test case.
* NEWS: Mention the change in behavior.
Discussed in https://bugs.gnu.org/31335
Ensure this _does_ recreate the symlink
Given "path1" and "path2" are on different devices.
$ touch "path1/file"
$ cd path2/; ln -s path1/file
$ cp -dsf path1/file .
Ensure this does _not_ overwrite file
$ touch file
$ ln -s file l1
$ cp -sf l1 file
* src/copy.c (same_file_ok): Remove device ids from consideration,
instead deferring to future EXDEV with --link or allowing
the first case above to work.
Also ensure that we do not exist this function too early,
when the destination file is not a symlink, which protects
against the second case.
* tests/cp/cross-dev-symlink.sh: Add a test for the first case.
* tests/cp/same-file.sh: Add a test for the second case above.
* NEWS: Mention the bug fixes.
* THANKS.in: Mention the reporters who also analyzed the code.
Fixes https://bugs.gnu.org/31364
* src/cp.c (target_directory_operand): Allow through inaccessible
arguments with -f or --remove.
* doc/coreutils.texi (cp invocation): Clarify that -f doesn't directly
impact the removal of non-traversable symlinks.
* tests/cp/dir-rm-dest.sh: Test the new behavior.
* tests/cp/thru-dangling.sh: Enforce -f behavior wrt symlinks.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/31335
* NEWS: Expand on the 8.28 description of how tail more
responsively reacts to closed output, and move from "Improvements"
to "Changed behavior".
* cfg.mk (old_NEWS_hash): Regenerate.
Fixes https://bugs.gnu.org/31225
Increase max range from SIZE_MAX to UINTMAX_MAX, which will
allow cut to support line lengths up to the max file size
on all systems. The inherent SIZE_MAX limitation in cut was
removed with the enhancements in https://bugs.gnu.org/13127.
Also numfmt gets similarly increased --field ranges due to
shared code.
* src/cut.c: s/size_t/uintmax_t/.
* src/numfmt.c: Likewise.
* src/set-fields.c: Likewise.
* src/set-fields.h: Likewise.
* tests/misc/cut-huge-range.sh: Adjust accordingly.
* tests/misc/numfmt.pl: Likewise.
* NEWS: Mention the improvement.
Problem reported by Karl Berry (Bug#30963).
* NEWS: Mention this.
* src/ls.c (decode_switches): Implement this.
* tests/ls/a-option.sh: New file.
* tests/local.mk (all_tests): Add it.
This will impact relatively few languages,
and will make Arabic or Catalan etc.
output unambiguous abbreviated month names.
* src/ls.c (MAX_MON_WIDTH): Increase from 5 to 12.
* NEWS: Mention the bug fix.
* tests/ls/abmon-align.sh: Augment to check for ambiguous output.
Fixes https://bugs.gnu.org/30814
Enhanced XFS (EXFS) is a version of XFS maintained by HPE.
EXFS uses a unique magic number to allow the use of community
XFS, and EXFS filesystems at the same time.
* src/stat.c (human_fstype): Add file system ID definition,
and use "exfs" as the name.
* NEWS: Mention the Improvement.
This issue was introduced in commit v8.19-145-g24ebca6
* src/copy.c (copy_internal): When setting default permissions
to use with --no-preserve=mode, only set executable bits for
directories or sockets.
* NEWS: Mention the fix.
* tests/cp/preserve-mode.sh: Add a test case.
Fixes https://bugs.gnu.org/30534
Problem reported by Kamil Dudka (Bug#29961).
* NEWS: Mention this.
* src/copy.c: Include renameat2.h.
(copy_internal): If mv, try renameat2 first thing, with
RENAME_NOREPLACE. If this works, skip most of the remaining code.
Also, fail quickly if it fails with EEXIST, and we are using -n.
* NEWS: Mention these fixes.
* doc/coreutils.texi (cp invocation, mv invocation):
Mention that -n is silent, and that it overrides -u.
* src/cp.c, src/mv.c (main): -n overrides -u.
Run "make update-copyright" and then...
* gnulib: Update to latest with copyright year adjusted.
* tests/init.sh: Sync with gnulib to pick up copyright year.
* bootstrap: Likewise.
* tests/sample-test: Adjust to use the single most recent year.
* src/tail.c (tail_bytes): On systems were blksize_t is unsigned
and the same size or wider than off_t (android for example),
our initialized (off_t) -1 would be promoted to unsigned before
comparison, and thus fail to follow the appropriate path.
* tests/tail-2/tail-c.sh: Add a test case.
* NEWS: Mention the fix.
This issue was introduced in commit v8.23-47-g2662702
Reported at https://github.com/termux/termux-app/issues/233
* src/dd.c (iread): Handle read error with a non-aligned
file offset in the O_DIRECT case. This is not an issue
on XFS at least, but on EXT4 the final read will return
EINVAL rather than the expected 0 to indicate EOF.
* tests/dd/direct.sh: Test the iflag=direct case also.
* NEWS: Mention the improvement.
This is useful as handling in shell is complicated
with the varying exit status in the --kill-after case.
* src/timeout.c (main): Handle '-v' and store
COMMAND for the diagnostic.
(cleanup): Diagnose the signal name before sending.
(usage): Document -v, --verbose.
* doc/coreutils.texi (timeout invocation): Likewise.
* tests/misc/timeout.sh: Add a test case.
* NEWS: Mention the new feature
Fixes https://bugs.gnu.org/21760
* src/tail.c (tail_bytes): Try lseek(..., SEEK_END) when
we can't determine the file size.
* tests/tail-2/end-of-device.sh: Add a new root only test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the improvement.
Paul Eggert suggested using lseek() (rather than ioctl(BLKGETSIZE64)).
Fixes https://bugs.gnu.org/29259
* src/df.c (main): stat() before open(), and avoid
the optional open when given a fifo argument.
* tests/df/unreadable.sh: Add a test case.
* NEWS: Mention the fix.
Fixes https://bugs.gnu.org/29038
Previously with oflag=direct the call to invalidate_cache()
was not passed to the kernel, as it was less than a page size,
and a subsequent call was not made to invalidate the pending space.
Similarly with oflag=nocache the pending space at EOF was
not invalidated. Even though these amount to only a single page
in the page cache it can be significant. For example on
XFS before kernel patch v4.9-rc1-4-g0ee7a3f, O_DIRECT files
would have been read inefficiently if any pages were cached,
even if they were already synced to storage.
* src/dd.c (i_nocache_eof, o_nocache_eof): New bools used
to control when we want invalidate_cache(,0) to clear to EOF.
(cache_round): Use IO_BUFSIZE (currently 132KiB) to minimize
calls to the relatively expensive advise function, rather
than page_size. This also makes it clear that while the
kernel function operates on pages, this size is chosen for
performance reasons.
(invalidate_cache): Refactor to share more code between
input and output paths. Use i_nocache_eof and o_nocache_eof
rather than proxying off max_records. Ensure we
invalidate full pages when clearing to EOF as the kernel
will ignore any non complete pages. Fix the offset used
for the output path.
(dd_copy): Invalidate the cache of the input after the
offset is updated, for consistency and so we don't try to
invalidate before the start of the file. When we read
EOF on input, set flags so that we invalidate to EOF.
(main): Invalidate to EOF in more cases, by depending
on the i_nocache_eof and o_nocache_eof flags.
* doc/coreutils.texi (dd invocation): Clarify the alignment
and persisted caveats on the example applying "nocache"
to part of a file.
* tests/dd/nocache_eof.sh: A new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
Issue reported by Eric Bergen.
This was a latent issue that became significant with
the addition of the -F option in FILEUTILS-3_16n-56-ge46a424
* src/stty.c (apply_settings): Refactor argument checking
to a function macro. Augment the argument check to ignore
NULLed out arguments (already processed -F).
* NEWS: Mention the fix.
* tests/misc/stty-invalid.sh: Add a test case.
Fixes https://bugs.gnu.org/28859
This fixes a regression from commit v8.26-39-g2f69dba
* src/timeout.c (block_cleanup_and_chld): Rename from block_cleanup
to indicate we also block SIGCHLD to avoid the race where SIGCHLD
fires between waitpid() polling and sigsuspend() waiting for a signal.
* NEWS: Mention the fix.
We inherit the signal mask from our parent process,
therefore ensure SIGCHLD is not blocked.
If SIGCHLD is blocked, sigsuspend() won't be interrupted
when the child process exits and we hang until the timeout (SIGALRM).
This fixes a regression from commit v8.26-39-g2f69dba
* src/timeout.c (install_sigchld): Ensure SIGCHLD is unblocked.
* NEWS: Mention the issue.
man pages change little between systems,
so falling back to distributed pages make sense
when cross compiling or lacking perl.
* man/local.mk: Add all man pages to EXTRA_DIST
so that they're distributed in the generated tarball.
Use the dummy-man page generator if cross compiling.
Set TZ to avoid a distcheck failure where man pages
used a diffent month than those rebuilt (with a .timestamp).
* man/dummy-man: Only fall back to generating a stub
if copying an existing man page fails.
* man/help2man: Sync portable TZ=UTC0 specification
from upstream help2man.
* NEWS: Mention the build-related change.
Fixes https://bugs.gnu.org/28574
This was unintentionally removed in v8.27-60-g2ae1460
* src/shred.c (wipename): Interate through all name lengths.
* tests/misc/shred-remove.sh: Add test cases.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/28507
* NEWS, doc/coreutils.texi (Target directory): Document this.
* src/cp.c, src/install.c, src/ln.c, src/mv.c: Include targetdir.h.
(target_directory_operand): Use the new targetdir_operand_type
function to check for vulnerable target directories.
* src/cp.c (stat_target_operand): New function.
(target_directory_operand, do_copy): Use it.
* src/local.mk (noinst_HEADERS): Add src/targetdir.h.
(src_ginstall_SOURCES, src_cp_SOURCES, src_ln_SOURCES)
(src_mv_SOURCES): Add src/targetdir.c.
* src/targetdir.c, src/targetdir.h: New files.
* tests/mv/vulnerable-target.sh: New test.
* tests/local.mk (all_root_tests): Add it.
* src/ptx.c (find_occurs_in_text): Die with an appropriate error
diagnostic when the given regular expression returns a match of
length 0.
* tests/misc/ptx.pl (S-infloop): Add a test.
* NEWS (Bug fixes): Mention the fix.
Fixes https://bugs.gnu.org/28417 which was detected using
Symbolic Execution techniques developed in the course of the
SYMBIOSYS research project at COMSYS, RWTH Aachen University.
This reverts commit v8.27-97-g8cb06d4 because
the setsid() fallback was not implemented correctly
and disabling the ioctl was not a complete solution
to the security issue of the child being passed
the tty of the parent.
Given runcon is not really a sandbox command,
the advice is to use `runcon ... setsid ...`
to avoid this particular issue.
We only use one of statfs or statvfs for `stat -f`
and on the BSDs we use statfs which doesn't have the
f_namelen member. However on OpenBSD and later FreeBSD
systems statfs does provide f_namemax, so use that.
* NEWS: Mention the improvement for OpenBSD and FreeBSD.
* m4/stat-prog.m4: Check for f_namemax in the statfs struct.
* src/stat.c: Return '?' rather than '*' when we can't
determine the max length of the file system.
* tests/ln/sf-1.sh: This test was failing on all BSDs
due to '*' being returned for the max length which
caused the test to attempt to create 1Mi+1 names.
The test now uses a short name when we can't determine
the max name length to use.
Reported by Assaf Gordon on various BSD based systems.
Similar to the issue with SELinux sandbox (CVE-2016-7545),
children of runcon can inject arbitrary input to the terminal
that would be run at the originating terminal privileges.
The new libseccomp dependency is widely available and used
on modern SELinux systems, but is not available by default
on older systems like RHEL6 etc.
* m4/jm-macros.m4: Check for libseccomp and
warn if unavailable on selinux supporting systems.
* src/local.mk: Link runcon with -lseccomp.
* src/runcon.c (disable_tty_inject): A new function to
disable use of the TIOCSTI using libseccomp, or with setsid()
where libseccomp is unavailable.
* tests/misc/runcon-no-inject.sh: A new test that uses
python to make the TIOCSTI call, and ensure that doesn't succeed.
* tests/local.mk: Reference the new test
* NEWS: Mention the fix.
Addresses http://bugs.gnu.org/24541
Terminals such as iTerm2 and VTE based terminals
(as of version 0.49.1), support hyperlinks when
passed terminals codes as described at:
https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda
* src/ls.c (gobble_file): Allocate an absolute file name to output.
(quote_name): Output the absolute name with the appropriate codes.
(file_escape): A new function to encode file names as per rfc8089.
(main): Handle the new option and call the file_escape_init() helper.
Disable --dired when --hyperlink is specified.
(print_dir): Get the absolute file name here too, so that the
directory name can be linkified.
* NEWS: Mention the new feature.
* tests/ls/hyperlink.sh: Add a new test.
* tests/local.mk: Reference the new test.
* doc/coreutils.texi (ls invocation): Describe --hyperlink.
This is useful when chaining with other commands that run commands in a
different context, while avoiding using the shell to cd, and thus
having to consider shell quoting the chained command.
* NEWS (New features): Document the new option.
* doc/coreutils.texi (env invocation): Likewise.
* src/env.c (usage): Likewise.
(main): Implement the new option.
* tests/misc/env.sh: Test the new option.
When systemd is configured to automount a remote file system - see
'man systemd.automount(5)', then the mount point is initially
mounted by systemd with the file system type "autofs".
When the resource is used later on, then the wanted file system is
mounted over that mount point on demand.
'df -l' triggered systemd to mount the file system because it called
stat() on the mount point.
Instead of single-casing "autofs" targets, we can avoid stat()ing
all dummy file systems (which includes "autofs"), because those are
skipped later on in get_dev() anyway.
*src/df.c (filter_mount_list): Also skip dummy file systems unless
the -a option or a specific target are given.
* NEWS: Mention the fix.
Co-authored-by: Bernhard Voelker <mail@bernhard-voelker.de>
Fixes http://bugzilla.suse.com/show_bug.cgi?id=1043059
* src/ls.c (gobble_file): Disable the optimization to avoid quoting
if the symlink target itself needs quoting. This was introduced
with the quoting alignment adjustments in v8.25-106-g01971c0
* tests/ls/symlink-quote.sh: Add a test.
* tests/local.mk: Reference the test.
* NEWS: Mention the fix.
* src/operand2sig.c (operand2sig): AIX uses a different bit pattern
in the returned status from the wait() functions and from shells.
Therefore hardcode the selection of the lower bits of the number.
* NEWS: Mention the fix.
Suggested by Kamil Dudka in:
http://lists.gnu.org/archive/html/coreutils/2017-07/msg00072.html
* NEWS: Document the changed nature of the fix.
* doc/coreutils.texi, tests/cp/backup-is-src.sh:
* tests/mv/backup-is-src.sh: Revert previous change.
* src/copy.c (source_is_dst_backup): New function.
(copy_internal): Use it. Fail instead of falling back on numbered
backups when it looks like the backup will overwrite the source.
Although this reintroduces a race, it's more compatible with
previous behavior.
* NEWS, doc/coreutils.texi (Backup options): Document the change.
* bootstrap.conf (gnulib_modules): Add backup-rename.
* src/copy.c (copy_internal): Silently switch to numbered backups
if a simple backup might lose data. Use backup_file_rename
to avoid races with numbered backups.
* tests/cp/backup-is-src.sh, tests/mv/backup-is-src.sh:
Adjust to match new behavior.
* src/shred.c (dopass): shred used to read the input file,
and so needed to ensure internal memory was cleared.
This is no longer the case since SH-UTILS-1_16f-260-gf381610
so avoid this redundant clearing.
(do_wipefd): Likewise.
* NEWS: Remove the recent mention of this issue.
