sort -u could omit one or more lines of expected output.
This bug arose because sort recorded the most recently printed line via
reference, and if you were unlucky, the storage for that line would be
reused (overwritten) as additional input was read into memory. If you
were doubly unlucky, the new value of the "saved" line would not only
match the very next line, but if that next line were also the first in
a series of identical, not-yet-printed lines, then the corrupted "saved"
line value would result in the omission of all matching lines.
* src/sort.c (saved_line): New static/global, renamed and moved from...
(write_unique): ...here. Old name was "saved", which was too generic
for its new role as file-scoped global.
(fillbuf): With --unique, when we're about to read into a buffer that
overlaps the saved "preceding" line (saved_line), copy the line's .text
member to a realloc'd-as-needed temporary buffer and adjust the line's
key-defining members if they're set.
(overlap): New function.
* tests/misc/sort: New tests.
* NEWS (Bug fixes): Mention it.
* THANKS.in: Update.
Bug introduced via commit v8.5-89-g9face83.
Reported by Rasmus Borup Hansen in
http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/23173/focus=24647
Add new option to rm (-d/--dir), which allows removal of
empty directories, while still safely disallowing removal
of non-empty ones.
This improves compatibility with Mac OS X and BSD systems,
which honor the -d option.
* src/remove.c (rm_fts): Remove empty directories when requested.
* src/remove.h (rm_options) [remove_empty_directories]: New member.
* src/rm.c (long_opts, usage, main): Update usage and option parsing.
(rm_option_init): Initialize the new member.
* src/mv.c (rm_option_init): Initialize the new member.
* tests/rm/d-1: New test case - successfully delete empty dir.
* tests/rm/d-2: New test case - refuse to delete nonempty dir.
* tests/Makefile.am (TESTS): Add them.
* src/df.c (main): Add conditions to fail when the mount list cannot
be read: this includes the cases when a file name argument is given
and any of -a, -l, -t or -x is used.
* doc/coreutils.texi: Document the additional error conditions.
* tests/df/no-mtab-status: Add a new test.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the fix.
When the combination of the file system options with given files or
devices does not lead to output, "df --total" would exit successfully
although it should not.
Examples:
$ df --total --type=xfs / # when / is not an XFS file system
$ df --total --local -t nfs DIR # nfs is remote per se ...
$ df --total -t qwerty /dev/sdb5 # typo in file system type
Furthermore, "df --total" would not print the error message "no file
systems processed" when the file argument does not exist or is otherwise
not accessible.
Example:
$ df --total __not_exist__
These 2 bugs are present since --total was added by commit
v6.12-166-gea2887b.
* src/df.c (get_dev): Do not set file_systems_processed to true when
force_fsu is true, i.e. when the row for the "total" line is processed.
(main): Don't print totals unless we've processed a file system.
Also only print the "no FS processed" message if there was no
preceding diagnostic.
* tests/df/total-unprocessed: Add a new test.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the fix.
Improved-by: Jim Meyering
Add a test and NEWS entry for a bug inadvertently fixed in
a refactoring in commit v8.9-32-gd4db0cb
* tests/misc/join (v2-format): Add a new test.
* THANKS.in: Add the reporter.
* NEWS: Mention the old bug.
* cfg.mk (old_NEWS_hash): Update.
Reported-by: Jean-Pierre Tosoni
It's awkward to read and problematic for scripts when
control characters like '\n' are output.
Note other fields are already handled with mbsalign,
which converts non printable chars to the replacement char.
A caveat to note with that, is the replacement char takes
a place in the field and so possibly truncates the field
if it was the widest field in the records.
Note a more general replacement function, that
handles all printable, or non white space characters,
would require more sophisticated support for various
encodings, and the complexity vs benefit was not
deemed beneficial enough at present.
Perhaps in future a more general replacement function
could be shared between the various utilities.
Note <space> is unaffected in any field,
which could impact scripts processing the output.
However any of the number fields at least could have
spaces considering `LANG=fr_FR df -B\'1`, so it's
probably best to leave spaces, which also allows
scripts to handle mount points with spaces without change.
* src/df.c (hide_problematic_chars): Replace control chars with '?'.
* tests/df/problematic-chars: Add a new root only test.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the fix.
date -d "$(printf '\xb0')" would print 00:00:00 with today's date
rather than diagnosing the invalid input. Now it reports this:
date: invalid date '\260'
* gnulib: Update submodule to latest for fixed parse-datetime.y.
* tests/misc/date [invalid-high-bit-set]: New test.
* NEWS (Bug fixes): Mention it.
* bootstrap, tests/init.sh: Also update to latest.
Reported by Peter Evans in http://bugs.gnu.org/11843
* src/sort.c (check_inputs): A new function to verify all inputs
are accessible before further processing.
(check_output): A new function to open or create a specified
output file, before futher processing.
(stream_open): Adjust to truncating the previously opened
output file rather than opening directly.
(avoid_trashing_input): Optimize to stat the output file
descriptor, rather than the file name.
(main): Call the new functions to check accessibility of
inputs and output, before processing starts.
* tests/misc/sort: Adjust to the changed error message.
* tests/misc/sort-merge-fdlimit: Account for the earlier opened
file descriptor of the specified output file.
* tests/misc/sort-exit-early: A new test to exercise the improvements.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the improvement.
Suggested-by: Bernhard Voelker
* src/stat.c (human_fstype) [__linux__]: Add a 'case' for the new
remote file system type: aufs (0x61756673).
* NEWS (New features): Mention stat -f.
(Bug fixes): Mention it for tail -f.
Reported by Michael Mol in http://bugs.gnu.org/11823
* src/split.c (create): Check if output file is the
same inode as the input file.
* tests/split/guard-input: New test case.
* tests/Makefile.am: Reference new test case.
* NEWS: Mention the fix.
Improved-by: Jim Meyering
Reported-by: François Pinard
* src/head.c (elide_tail_lines_seekable): Reset file pointer
after printing up to an end-relative line-counted offset.
Anoop Sharma reported the problem and suggested the fix.
* tests/misc/head-pos: Add coverage via a very similar, existing test.
Also add coverage for a previously untested block of code.
* tests/misc/head-elide-tail ($READ_BUFSIZE): Update to 8192, to
match the value of BUFSIZ I see today on Fedora 17/x86_64 (unrelated
to this fix).
* NEWS (Bug fixes): Mention it.
Improved-by: Pádraig Brady
* src/stat.c (human_fstype) [__linux__]: Add a 'case' for the new
remote file system type: panfs (0xAAD7AAEA).
* NEWS (New features): Mention stat -f.
(Bug fixes): Mention it for tail -f.
Reported by Travis Gummels in http://bugzilla.redhat.com/827199
This utility was inadvertently omitted from commit v8.0-34-g710fe41
* src/cksum.c (main): Set stdout to line buffered mode, to ensure
parallel running instances don't intersperse their output.
* NEWS: Mention the fix.
* THANKS.in: Add Anoop.
Reported by Anoop Sharma.
struct statfs has the f_frsize member since Linux 2.6,
so use that rather than f_bsize which can be different.
Note the related df change mentioned in NEWS is handled
in gnulib by using statvfs() rather than statfs()
on Linux > 2.6.36 (where statvfs doesn't hang) and the
same method as stat for Linux 2.6 kernels earlier than that.
stat(1) doesn't use statvfs() on GNU/Linux as the f_type
member isn't available there.
Note the change to not use statvfs() on GNU/Linux was introduced
in gnulib commit eda39b8 16-08-2003.
* m4/stat-prog.m4 (cu_PREREQ_STAT_PROG): Check for the f_frsize
member in the statfs structure.
* src/stat.c: Use (struct statfs).f_frsize if available.
* NEWS (Bug fixes): Mention this stat fix, and the related df fix
coming in the next gnulib update.
* THANKS.in: Add Nikolaus.
Reported and Tested by Nikolaus Rath
In order for ls --color to color each symlink, it must form the name
of each referent and then stat it to see if the link is dangling, to
a directory, to a file, etc. When the symlink is to a relative name,
ls must concatenate the starting directory name and that relative name.
When, in addition, the starting directory was "/" or "/some-name",
the result was ill-formed, and the subsequent stat would usually fail,
making the caller color it as a dangling symlink.
* src/ls.c (make_link_name): Don't botch the case in which
dir_name(NAME) == "/" and LINKNAME is relative.
* tests/ls/root-rel-symlink-color: New file. Test for the above.
* tests/Makefile.am (TESTS): Add it.
* NEWS (Bug fixes): Mention it.
Reported by Mike Frysinger in http://bugs.gnu.org/11453
Bug introduced by commit v8.16-23-gbcb9078.
* src/stat.c (human_fstype) [__linux__]: Add 'case's for these local
file system types: bdevfs (0x62646576), inodefs (0x42494E4D),
qnx6 (0x68191122).
Now that the kernel has a name for S_MAGIC_BINFMTFS, use
theirs in place of our S_MAGIC_BINFMT_MISC.
* NEWS (New features): Mention it.
Problem reported by Samuel Thibault in <http://bugs.gnu.org/11424>.
* NEWS: Document this.
* src/dd.c (skip): Handle skipping past EOF on shared or typed
memory objects the same way as with regular files.
(dd_copy): It's OK to truncate shared memory objects.
* src/du.c (duinfo_add): Check for overflow.
(print_only_size): Report overflow.
(process_file): Ignore negative file sizes in the --apparent-size case.
* src/od.c (skip): Fix comment about st_size.
* src/split.c (main):
* src/truncate.c (do_ftruncate, main):
On files where st_size is not portable, fall back on using lseek
with SEEK_END to determine the size. Although strictly speaking
POSIX says the behavior is implementation-defined, in practice
if lseek returns a nonnegative value it's a reasonable one to
use for the file size.
* src/system.h (usable_st_size): Symlinks have reliable st_size too.
* tests/misc/truncate-dir-fail: Don't assume that getting the size
of a dir is not allowed, as it's now allowed on many platforms,
e.g., GNU/Linux.
* src/split.c (main): Use stat.st_size only for regular files.
Samuel Thibault reported in http://bugs.gnu.org/11424 that the
/dev/zero-splitting tests would appear to infloop on GNU/Hurd,
because /dev/zero's st_size is LONG_MAX. It was only a problem
when using the --number (-n) option.
* NEWS (Bug fixes): Mention it.
This bug was introduced with the --number option, via
commit v8.7-25-gbe10739
* src/copy.c (copy_reg): In a narrow race (stat sees dest, yet
open-without-O_CREAT fails with ENOENT), retry the open with O_CREAT.
* tests/cp/nfs-removal-race: New file.
* tests/Makefile.am (TESTS): Add it.
* NEWS (Bug fixes): Mention it.
Reported by Philipp Thomas and Neil F. Brown in
http://bugs.gnu.org/11100
... i.e., don't use the getpw* functions.
Before this change, running groups or id with no user name argument
would include a group name or ID from /etc/passwd. Thus, under unusual
circumstances (default group is changed, but has not taken effect for a
given session), those programs could print a name or ID that is neither
real nor effective.
To demonstrate, run this:
echo 'for i in 1 2; do id -G; sleep 1.5; done' \
|su -s /bin/sh ftp - &
sleep 1; perl -pi -e 's/^(ftp❌\d+):(\d+)/$1:9876/' /etc/passwd
Those id -G commands printed the following:
50
50 9876
With this change, they print this:
50
50
Similarly, running those programs set-GID could make them
print one ID too many.
* src/group-list.c (print_group_list): When username is NULL, pass
egid, not getpwuid(ruid)->pw_gid), to xgetgroups, per the API
requirements of xgetgroups callee, mgetgroups.
When not using the password database, don't call getpwuid.
* NEWS (Bug fixes): Mention it.
* tests/misc/id-setgid: New file.
* tests/Makefile.am (TESTS): Add it.
(root_tests): It's a root-only test, so add it here, too.
Originally reported by Brynnen Owen as http://bugs.gnu.org/7320.
Raised again by Marc Mengel in http://bugzilla.redhat.com/816708.
Accept -g for BSD/Plan9 compatibility.
* NEWS (New features): Mention it.
* tests/fmt/goal-option: New test.
* tests/fmt/long-line: Rename from tests/fmt-long-line.
* tests/fmt/base: Rename from tests/misc/fmt.
* doc/coreutils.texi: Document it.
* src/fmt.c (main): Accept the new option
(check_for_goals): new function to implement the operands
Based on BSD's and Plan-9's fmt programs.
* src/copy.c (copy_reg): Don't truncate an existing file,
to support copying attributes between existing files.
The original use case only considered creating new files,
and it would be a very unusual use case to be relying
on the truncating behavior.
* doc/coreutils.texi (cp invocation): Mention the non
truncating behavior.
* tests/cp/attr-existing: A new test to ensure O_TRUNC skipped.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the change in behavior.
With the "--relative --symbolic" options, ln computes the relative
symbolic link for the user.
So, ln works just as cp, but creates relative symbolic links instead
of copying the file.
I miss this feature since the beginning of using ln.
$ tree ./
/
`-- usr
|-- bin
`-- lib
`-- foo
`-- foo
4 directories, 1 file
$ ln -s -v --relative usr/lib/foo/foo usr/bin/foo
‘usr/bin/foo’ -> ‘../lib/foo/foo’
$ tree ./
/
`-- usr
|-- bin
| `-- foo -> ../lib/foo/foo
`-- lib
`-- foo
`-- foo
4 directories, 2 files
$ ln -s -v --relative usr/bin/foo usr/lib/foo/link-to-foo
‘usr/lib/foo/link-to-foo’ -> ‘foo’
$ tree ./
/
`-- usr
|-- bin
| `-- foo -> ../lib/foo/foo
`-- lib
`-- foo
|-- link-to-foo -> foo
`-- foo
4 directories, 3 files
* src/Makefile.am: Reference the relpath module.
* src/ln.c (usage): Mention the new option.
(do_link): Call the relative conversion if specified.
(convert_abs_rel): Perform the relative conversion
using the relpath module.
* tests/ln/relative: Add a new test.
* tests/Makefile.am: Reference the new test.
* doc/coreutils.texi: Document the new feature.
* NEWS: Mention the new feature.
Most of the time, if someone wants to filter which paths are
relative while leaving all others absolute, they also want to
to the filtering based on the same --relative-to directory.
Make this easier to specify.
* src/realpath.c (main): Convert error to default.
* doc/coreutils.texi (realpath invocation): Document this.
* tests/misc/realpath: Adjust test to match.
* NEWS: Document it.
When --relative-base is /, all other paths should be treated as
relative (except for // where it matters).
Also, on platforms like Cygwin where / and // are distinct, realpath
was incorrectly collapsing // into /. http://debbugs.gnu.org/10472.
* src/realpath.c (path_prefix, path_common_prefix): Treat /
and // as having no common match.
(relpath): Allow for no match even without --relative-base.
* NEWS: Document this.
* src/dirname.c (main): Handle new -z option and manage more than one
argument.
* doc/coreutils.texi (dirname invocation): Mention it.
* NEWS (New features): Mention it.
* tests/misc/dirname: Add a two arguments test.
* src/split.c (next_file_name): If `suffix_auto' is true and the first
suffix character is 'z', generate a new file file name adding `z' to
the prefix and increasing the suffix length by one.
(set_suffix_length): Disable auto suffix width in various cases.
* tests/split/suffix-auto-length: Test it.
* doc/coreutils.texi (split invocation): Mention it.
* NEWS (Improvements): Likewise.
* NEWS: Document this.
* doc/perm.texi (Operator Numeric Modes): New section.
(Numeric Modes, Directory Setuid and Setgid): Document new behavior.
* src/chmod.c (usage): Document new behavior.
(main): Support new options -0, -1, etc.
* tests/chmod/setgid: Test these new features.
Surprise! "du -x non-DIR" would print nothing.
Note that the problem arises only when processing a non-directory
specified on the command line. Not surprisingly, "du -x" still
works as expected for any directory argument.
When performing its same-file-system check, du may skip an entry
only if it is at fts_level 1 or greater. Command-line arguments
are at fts_level == 0 (FTS_ROOTLEVEL).
* src/du.c (process_file): Don't use the top-level FTS->fts_dev
when testing for --one-file-system (-x). It happens to be valid
for directories, but it is always 0 for a non-directory.
* tests/du/one-file-system: Add tests for this.
* NEWS (Bug fixes): Mention it.
Reported by Daniel Stavrovski in http://bugs.gnu.org/10967.
Introduced by commit v8.14-95-gcfe1040.
* src/basename.c (perform_basename): New function refactored from
main() that performs the basename work on a STRING, optionally
removes a trailing SUFFIX and outputs the result.
(main): Handle new options.
* doc/coreutils.texi (basename invocation): Mention new options.
* test/misc/basename: Add new options test cases.
* NEWS (New features): Mention it.
Notes:
Small seeks are not coalesced to larger ones,
like is done in cache_round() for example.
conv= is used rather then oflag= for FreeBSD compatibility.
* src/dd.c (final_op_was_seek): A new global boolean to flag
whether the final "write" was converted to a seek.
(usage): Describe the new conf=sparse option.
(iwrite): Convert a write of a NUL block to a seek if requested.
(do_copy): Initialize the output buffer to have a sentinel,
to allow for efficient testing for NUL output blocks.
If the last block in the file was converted to a seek,
then convert back to a write so the size is updated.
* NEWS: Mention the new feature.
* tests/dd/sparse: A new test for the feature.
* tests/Makefile.am: Reference the new test.
Add the --additional-suffix option, to append an
additional static suffix to output file names.
* src/split.c (next_file_name): Append suffix to output file names.
(main): Handle new --additional-suffix option.
* NEWS (New features): Mention it.
* doc/coreutils.texi (split invocation): Mention it.
* tests/split/additional-suffix: New file. Test --additional-suffix.
* tests/Makefile.am (TESTS): Add it.
Requested by Peng Yu, in bug 6554
Like the optimization to avoid always-failing getfilecon calls,
this change avoids always-failing queries for whether a file has
a nontrivial ACL and for whether a file has certain "capabilities".
When such a query fails for one file (indicating no support), we know it
will always fail that way for the affected device. With this change, we
have thus eliminated nearly all failing-unsupported getxattr syscalls.
* src/ls.c (has_capability) [!HAVE_CAP]: Set errno to ENOTSUP.
(errno_unsupported): Expand the list of E* errno values to match
that of lib/acl-internal.h's ACL_NOT_WELL_SUPPORTED macro.
(file_has_acl_cache, has_capability_cache): New functions.
(gobble_file): Use them in place of non-caching ones.
* NEWS (Improvements): Mention it.
Suggested by Sven Breuner in
http://thread.gmane.org/gmane.comp.gnu.coreutils.general/2187
While eliminating most getfilecon calls saved about 33%,
eliminating these other calls can save almost all of the
remaining ~67% cost, on some remote file systems.
Allow changing the --numeric-suffixes start number
from the default of 0.
* src/split.c (next_file_name): Initialize the suffix index
and the output filename according to start value.
(main): Check that the suffix length is large enough for the
numerical suffix start value.
* doc/coreutils.texi (split invocation): Mention it.
* NEWS (New features): Mention it.
* tests/split/numeric: New file. Test --numeric-suffixes[=FROM].
* tests/Makefile.am (TESTS): Reference the new test.
dd now accepts the count_bytes and skip_bytes input flag and the
seek_bytes output flag, to more easily allow processing portions of a
file.
* src/dd.c (scanargs): Compute skip_records and skip_bytes when
'skip_bytes' iflag is used. Compute max_records and max_bytes when
'count_bytes' iflag is used. Compute seek_records and seek_bytes
when 'seek_bytes' oflag is used.
(skip_via_lseek): Use new 'bytes' parameter and handle potential
'records' equals to zero. Update the bytes parameter when called with
'fdesc' equal to STDOUT_FILENO. Update the header comments.
(dd_copy): Skip accordingly to skip_records AND skip_bytes. Count
accordingly to max_records AND max_bytes. Seek on output accordingly
to seek_records AND seek_bytes.
* NEWS (New features): Mention it.
* doc/coreutils.texi (dd invocation): Detail new flags and behaviors.
* tests/dd/bytes: New file. Tests for these new flags.
* tests/Makefile.am (TESTS): Add it.
But only when both A and B were hard links to the same symlink.
* src/copy.c (same_file_ok): Handle another special case: the one
in which we are moving a symlink onto a hard link to itself.
In this case, we must explicitly tell the caller to unlink the
source file. Otherwise, at least the linux-3.x kernel rename
function would do nothing, as mandated by POSIX 2008.
* tests/mv/symlink-onto-hardlink-to-self: New test.
* tests/Makefile.am (TESTS): Add it.
* NEWS (Bug fixes): Mention it.
Reported by Bernhard Voelker in http://bugs.gnu.org/10686
Normally, mv detects a few subtle cases in which proceeding with a
same-file rename would, with very high probability, cause data loss.
Here, we have found a corner case in which one of these same-inode
tests makes mv refuse to perform a useful operation. Permit that
corner case.
* src/copy.c (same_file_ok): Detect/exempt this case.
* tests/mv/symlink-onto-hardlink: New test.
* tests/Makefile.am (TESTS): Add it.
* NEWS (Bug fixes): Mention it.
Initially reported by: Matt McCutchen in http://bugs.gnu.org/6960.
Raised again by Anders Kaseorg due to http://bugs.debian.org/654596.
Improved-by: Paul Eggert.
* src/split.c (lines_chunk_split): Fix logic bug that led to
unwarranted failure of "split -n l/2 /dev/zero" on NetBSD 5.1.
The same would happen when splitting a growing file, where
open/lseek-end gives one size, but by the time we read, there
is more data available.
(bytes_chunk_extract): Likewise.
* NEWS (Bug fixes): Mention this.
* tests/split/l-chunk: The latter case was not exercised.
Add code to do that.
Bug introduced with the chunk-selecting feature in v8.7-25-gbe10739.
Co-authored-by: Jim Meyering <meyering@redhat.com>
On systems with recent kernel/tools, a symlink from /etc/mtab to
/proc/mounts, and a by-UUID mount (i.e., soon, nearly everyone),
you will see something like the following when running "df -hT":
(this has been truncated to fit in a width-limited ChangeLog file)
Filesystem Type Siz...
rootfs rootfs 11G
udev devtmpfs 3.8G
tmpfs tmpfs 774M
/dev/disk/by-uuid/828fc648-9f30-43d8-a0b1-f7096a2edb66 ext4 11G
tmpfs tmpfs 1.6G
/dev/sda2 ext3 494M
/dev/sda5 ext4 12G
/dev/sda6 ext4 9.9G
Contrast that with what we're used to seeing (modulo the
two entries mounted on "/", which is a separate problem):
Filesystem Type Size Used Avail Use% Mounted on
rootfs rootfs 11G 1.9G 8.0G 19% /
udev devtmpfs 3.8G 0 3.8G 0% /dev
tmpfs tmpfs 774M 376K 774M 1% /run
/dev/sda3 ext4 11G 1.9G 8.0G 19% /
tmpfs tmpfs 1.6G 8.0K 1.6G 1% /run/shm
/dev/sda2 ext3 494M 78M 392M 17% /boot
/dev/sda5 ext4 12G 7.6G 3.7G 68% /usr
/dev/sda6 ext4 9.9G 6.6G 2.8G 71% /var
When that long /dev/disk/by-uuid/... name is merely a symlink
to a much shorter (and often more useful) device name like
"/dev/sda3", and when it's part of a listing of all file systems,
I would much prefer to see only the latter. Similarly, when using
an encrypted root file system, you would see a name like
/dev/mapper/luks-828fc648-9f30-43d8-a0b1-f7196a2edb66 pointing
to say, /dev/dm-0, I prefer the shorter name.
I.e., if I explicitly run
"df -hT /dev/disk/by-uuid/828fc648-9f30-43d8-a0b1-f7096a2edb66",
then, it's fine -- and expected -- to print to the long name.
It was explicitly given. However, with no non-option argument,
df should print the shorter name. Note that performing this
translation at a lower level (via a change to gnulib's mountlist.c)
would make it impossible to distinguish those two cases.
* src/df.c: Include "canonicalize.h".
(get_dev): Add a parameter, telling when we're in process-all-
mount-points mode; update all callers. When true, resolve
UUID-suffixed symlinks.
* NEWS (Changes in behavior): Mention it.
Reported by Dan Jacobson in http://bugs.gnu.org/10363
This program is compatible with other realpath(1)
implementations, and also incorporates relpath like support,
through the --relative options. The relpath support
was suggested by Peng Yu, who also provided an initial
implemenation of that functionality.
* AUTHORS: Add my name.
* NEWS: Mention the new command.
* README: Likewise.
* doc/coreutils.texi (realpath invocation): Add realpath info.
* man/Makefile.am (realpath.1): Add dependency.
* man/realpath.x: New template.
* man/.gitignore: Ignore generated man page.
* po/POTFILES.in: Add src/realpath.c.
* src/.gitignore: Exclude realpath.
* src/Makefile.am (EXTRA_PROGRAMS): Add realpath.
* src/realpath.c: New file.
* scripts/git-hooks/commit-msg: Add realpath to the list of prefixes.
* tests/Makefile.am (TESTS): Add misc/realpath.
* tests/misc/realpath: New file.
Before, we would use inotify in that case, which would work as long
as updates were taking place locally, but not at all when remote.
Move hard-coded list of known remote FS types into a more
maintainable table in stat.c, alongside the list of FS
names and magic numbers. Generate a new is_local_fs_type function.
* src/Makefile.am (fs-is-local.h): New rule, generated file.
* src/extract-magic: Revamp to parse local/remote keyword after
each magic number in src/stat.c's case statements.
Accept new --local option.
* src/.gitignore: Ignore the generated fs-is-local.h.
* src/tail.c [HAVE_INOTIFY]: Include fs-is-local.h.
(fremote) [HAVE_INOTIFY]: Use the new function in place of
the switch stmt with hard-coded list of FS types.
Emit a warning when processing a file on a file system of unknown type.
* NEWS (Changes in behavior): Mention it.
Suggested by Sven Breuner.
* src/stat.c (human_fstype): Add a case: fhgfs, 0x19830326.
* src/tail.c (fremote): Add S_MAGIC_FHGFS.
* NEWS (Bug fixes): Update the entry for GPFS to mention FhGFS, too.
Reported by Sven Breuner.
This fixes Bug#10293, which I guess was introduced in commit
95c948b06a dated 2003-10-02.
* NEWS: Document fix.
* src/du.c (process_file): Don't count files in different file
systems if -x is given.
* tests/du/one-file-system: Test for this bug.
Starting with commit adc30a83, when using --color, ls inhibited
interrupts to avoid corrupting the state of an output terminal.
However, for very large directories, that inhibition rendered ls
uninterruptible for too long, including a potentially long period
even before any output is generated.
* src/ls.c: Two phases of processing are time-consuming enough that
they can provoke this: the readdir loop and the printing loop. The
printing was supposed to be covered by a call to process_signals in
(print_name_with_quoting): ... but that call was mistakenly guarded
by a condition that might be false for many or even all files being
processed. Call process_signals unconditionally.
(print_dir): Also call process_signals in the readdir loop.
* NEWS (Bug fixes): Mention it.
Reported by Arkadiusz Miśkiewicz in http://bugs.gnu.org/10243
Co-authored-by: Eric Blake <eblake@redhat.com>
Using ls -l on an SELinux-enabled system would leak one SELinux
context string per non-empty-directory command-line argument.
* src/ls.c (free_ent): New function, factored out of...
(clear_files): ...here. Use it.
(extract_dirs_from_files): Call free_ent (f), rather than simply
free (f->name). The latter failed to free the possibly-malloc'd
linkname and scontext members, and thus could leak one of those
strings per command-line argument.
* THANKS.in: Update.
* NEWS (Bug fixes): Mention it.
Reported by Juraj Marko in http://bugzilla.redhat.com/751974.
This fixes an incompatibility with POSIX 2008 and with BSD.
Problem reported by Abdallah Clark (Bug#9939)
via Alan Curry (Bug#10016).
* NEWS: Document this.
* doc/coreutils.texi (General output formatting): Document the
new -k behavior, and --kibibytes.
* src/ls.c (file_human_output_opts): New static var.
(long_options, usage): Add --kibibytes.
(decode_switches, gobble_file, print_long_format):
Implement the new -k behavior.
* tests/ls/block-size: New file.
* tests/Makefile.am (TESTS): Add it.
Because tail's fremote function did not designate GPFS as
a remote file system type, tail -f would mistakenly attempt
to use inotify, which cannot work with a remote file system.
* src/tail.c (fremote): List GPFS as a remote file system type.
* NEWS (Bug fixes): Mention it.
Reported in http://bugs.debian.org/646022.
* src/tac.c (copy_to_temp): Do not reuse the template buffer.
Instead, scribble only on a freshly-xstrdup'd copy each time.
Free that buffer both here, upon failure, and ...
(tac_nonseekable): ...free the buffer in caller, upon success.
* tests/misc/tac-2-nonseekable: New file.
* tests/Makefile.am (TESTS): Add it.
* NEWS (Bug fixes): Mention it.
Reported by Ambrose Feinstein in http://debbugs.gnu.org/9762.
* tests/ls/slink-acl: New file.
* tests/Makefile.am (TESTS): Add it.
* tests/init.cfg (require_setfacl_): New function.
* gnulib: Update to latest, for file-has-acl changes.
* NEWS (Bug fixes): Mention it.
See http://thread.gmane.org/gmane.comp.lib.gnulib.bugs/28538. This
":>k; setfacl -m m::r k; ln -s k s; ls -Log s" should print e.g.,
-rw-r-----+ 1 0 Oct 5 19:22 s
With the ls from coreutils-8.13, it would print this (with "." or
nothing in place of the "+"):
-rw-r-----. 1 0 Oct 5 19:22 s
These commands would fail to terminate:
yes -- -nan | head -156903 | sort -g > /dev/null
echo nan > F; sort -m -g F F
That can happen with any strtold implementation that includes
uninitialized data in its return value. The problem arises in the
mergefps function when bubble-sorting the two or more lines, each
from one of the input streams being merged: compare(a,b) returns 64,
yet compare(b,a) also returns a positive value. With a broken
comparison function like that, the bubble sort never terminates.
Why do the long-double bit strings corresponding to two identical
"nan" strings not compare equal? Because some parts of the result
are uninitialized and thus depend on the state of the stack.
For more details, see http://bugs.gnu.org/9612.
* src/sort.c (nan_compare): New function.
(general_numcompare): Use it rather than bare memcmp.
Reported by Aaron Denney in http://bugs.debian.org/642557.
* NEWS (Bug fixes): Mention it.
* tests/misc/sort-NaN-infloop: New file.
* tests/Makefile.am (TESTS): Add it.
On some systems like glibc on GNU/kFreeBSD, a thread is
implicitly created when timer_settime() is used.
This breaks our scheme to ignore signals we've
sent ourselves.
* src/timeout.c (send_sig): Change the scheme used to
ignore signals we've sent ourselves, to a more robust
but perhaps limited scheme of ignoring all signals of
a certain type after we've sent that signal to the job.
* NEWS: Mention the change in behavior.
This fixes a bug in pwd and all getcwd-using applications (for some
uses: df, readlink, stat) when run from a directory whose absolute name
contains more than PATH_MAX / 3 components. For more details, see
http://git.sv.gnu.org/cgit/gnulib.git/commit/?id=f6fe351fc534ae1
* gnulib: Update.
* NEWS (Improvements): Mention it.
* src/md5sum.c (split_3): Detect and handle BSD reversed
format checksums.
* tests/misc/md5sum-bsd: Add a new test.
* tests/Makefile.am: Reference new test.
* NEWS: Mention the improvement
Suggested by Rimas Kudelis.
* NEWS (Improvements): Mention the cp-vs-NFSv4-ACL improvement we
have inherited via gnulib. For details, see
http://article.gmane.org/gmane.comp.lib.gnulib.bugs/28100
and the following messages in that thread.
* gnulib: Update to latest, for numerous ACL-related improvements.
Thanks to an improvement in gnulib's parse-datetime module,
commands like this now succeed (output manually indented):
$ ./date -u -d 2004-02-29T16:21:42.33+07:00 +%FT%T.%N%z
2004-02-29T09:21:42.330000000+0000
* tests/misc/date: Add a test to exercise the new-in-gnulib
parsing of ISO8601-with-"T" dates.
* NEWS (New features): Mention it.
* gnulib: Update, to pull in this parse-datetime improvement.
Ever since the LAST_PAGE functionality was added
in commit ed0923a1, 1996-12-05,
"Apply big patch (patch-20) from Roland Huebner"
it was ignored when -t or -T were specified or
when -l <= 10.
* src/pr.c (print_page): Increment the current page here.
(print_header): Don't increment the current page here.
* tests/pr/pr-tests: Add a test case.
* NEWS: Mention the fix
Reported at http://debbugs.gnu.org/cgi/bugreport.cgi?bug=9347
For details, see the gnulib commit,
http://git.sv.gnu.org/cgit/gnulib.git/commit/?id=47cb657e
* tests/rm/4-million-entry-dir: New test.
* tests/Makefile.am (TESTS): Add it.
* NEWS (Bug fixes): Mention it.
* gnulib: Update to latest to get the required fts fixes.
* src/join (g_names): New global (was main's "names").
(main): Update all uses of "names".
(line_no[2]): New globals.
(get_line): Increment after reading each line.
(check_order): Print the standard "file name:line_no: " prefix
as well as the offending line when reporting disorder.
Here is a sample old/new comparison:
-join: file 1 is not in sorted order
+join: in:4: is not sorted: contents-of-line-4
* tests/misc/join: Change the two affected tests to expect
the new diagnostic.
Add new tests for more coverage: mismatch in file 2,
two diagnostics, zero-length out-of-order line.
* NEWS (Improvements): Mention it.
Suggested by David Gast in http://debbugs.gnu.org/9236
* src/copy.c (copy_internal): With --update (-u), this function would
return early once it found that the destination is not older than the
source, *without* recording the source-dev/ino--to--dest_name mapping.
That mapping is required in order to preserve src hard links in the
destination tree, so when using cp with --update and --preserve=links
(perhaps via -p or -a), cp could fail to preserve one hard link
per inode when at least one of the hard-linked names already exists
in the destination tree.
Reported by Odd Harry Mannsverk in http://debbugs.gnu.org/8419.
* tests/cp/preserve-link: New file. Exercise the flaw/fix.
* tests/Makefile.am (TESTS): Add it.
* NEWS (Bug fixes): Mention it.
* src/timeout.c (settimeout): A new function to convert
from a floating point duration and call alarm() or
timer_settime() if that's available.
(parse_duration): Return a double rather than unsigned int.
(usage): Mention floating point is supported.
(main): Pass the double to settimeout() rather than
calling alarm() directly with the parsed int.
(cleanup): Likewise.
* doc/coreutils.texi (timeout invocation): Say floating point timeouts
now supported, and mention the caveat with resolution.
* bootstrap.conf: Include the timer-time gnulib module.
* tests/misc/timeout-parameters: Add a test with nanoseconds.
* NEWS: Mention the improvement.
This change derives from improvements to gnulib's fsusage module.
* NEWS (Improvements): df now supports disk partitions larger than
4 TiB on MacOS X 10.5 or newer and on AIX 5.2 or newer.
Alphabetize entries.
* gnulib: Update to latest.
The following dropped the space from the first field
printf "1234567 \t1\n" | unexpand -a
Note POSIX says that spaces should not precede tabs.
Also a single trailing space should not be converted
if the next field starts with non blank characters.
So we enforce those rules too, with this change.
* src/unexpand.c (unexpand): Implement as per POSIX rules.
* tests/misc/unexpand: Add tests, and adjust existing
tests as per POSIX rules.
* NEWS: Mention the fix.
Reported by Hallvard B Furuseth
* src/timeout.c (cleanup): Send signals directly to the child
in case it has started its own process group (like a cascaded
timeout command would for example).
* test/misc/timeout-group: Add a test case.
* NEWS: Mention the fix.
Or more accurately, commands not started from the shell prompt,
that are interactive, or need to receive Ctrl-C etc. from the terminal.
* doc/coreutils.texi (timeout invocation): Document --foreground.
* src/timeout.c (main): Set the foreground flag and don't create
a separate group.
(cleanup): Only send a signal directly to the monitored command
when the foreground flag is set.
(usage): Describe --foreground.
* tests/misc/timeout-group: Add a new test.
* tests/Makefile.am: Reference new test.
NEWS: Mention the new option.
Reported by Shay Shimony
Analysis by Alan Curry
Fix suggested by Paul Eggert
Use this new option with --check when the input is expected to
consist solely of checksum lines. With only --check, an invalid
line evokes a warning, but the program can still exit successfully.
With --strict, any invalid line makes the program exit non-zero.
* src/md5sum.c (strict, STRICT_OPTION): Declare/define.
(long_options): Add "strict".
(usage): Describe --strict.
(digest_check): Count improperly_formatted lines, too, and use
that number and the global "strict" to determine the return value.
(main): Handle STRICT_OPTION.
Reject --strict without --check.
* doc/coreutils.texi: Describe it.
* NEWS (New features): Mention it.
* src/chown-core.c (describe_change): Output the
original owner if possible.
(user_group_str): Handle the case when neither
owner or group are passed.
* NEWS: Mention the change in behavior.
* src/chown_core.c (describe_change): Accept the ownership of
the original file and output that when not changing.
This is significant when --from is specified as then
the original and specified ownership may be different.
(user_group_str): A new helper function refactored from
describe_change().
(change_file_owner): Pass the original user and group
strings to describe_change().
* test/chown/basic: Add a test case.
* NEWS: Mention the fix.
* src/chmod.c (describe_change): Pass in the original mode,
and output this in the messages.
* tests/chmod/c-option: Adjust as per the new message.
* THANKS.in: Remove the now auto-generated name.
* NEWS: Mention the change in behavior.
Signed-off-by: Pádraig Brady <P@draigBrady.com>
* src/split.c (lines_chunk_split): Ensure that data is only
written to stdout when k specified. Also ensure that
extra files are not created when there is more data available
than reported in the file size.
* tests/misc/split-lchunk: Verify that split -n l/k/n doesn't
generate any files, and that -n l/n always generates n files.
* NEWS: Mention the fix.
* NEWS: "an misleading"
* src/expr.c: "a integer
* src/ptx.c (find_occurs_in_text): "a end"
* src/shred.c (do_wipefd): "a infinite"
* src/sort.c (SUBTHREAD_LINES_HEURISTIC): "an dual-core"
(compare_random): "an checksum"
* cfg.mk (old_NEWS_hash): Update, since the typo was in old news.
* src/printf.c (STRTOX): Don't access memory after a
string containing a single quote character.
* tests/misc/printf: Add tests for various combinations
of single quote characters combined with a numeric format.
* THANKS.in: Add bug reporter.
* NEWS: Mention the fix.
Reported-by: Paul Marinescu <paul.marinescu@imperial.ac.uk>
* gl/lib/randperm.c (randperm_new): When the number of items
to return H, is much smaller than the total number of items N,
use a hash to represent the sparse permutations of the set N.
This is currently enabled for N > 128K and N/H > 32.
* tests/misc/shuf: Ensure shuf can quickly return 2 numbers
from a large range.
* gl/modules/randperm: Depend on hash.
* NEWS: Mention the change.
* src/dd.c (O_NOCACHE): Undefine. This symbol is defined
via AIX's <fcntl.h>, yet used as an enum name in dd.c.
Reported by Gary V. Vaughan in http://debbugs.gnu.org/8555
* NEWS (Portability): Mention this.
* src/tail.c (tail_forever_inotify): Just as without inotify,
tail --follow=name now terminates when the last tailed-by-name file
is unlinked or moved aside. This bug was introduced on 2009-06-15
via commit ae494d4b, "tail: use inotify if it is available".
Reported by Tim Underwood in
http://thread.gmane.org/gmane.comp.gnu.coreutils.bugs/22286
* NEWS (Bug fixes): Mention this.
* tests/tail-2/follow-name: Test for this.
* tests/Makefile.am (TESTS): Add it.
to avoid the expense of extent_copy's unconditional use of
FIEMAP_FLAG_SYNC.
* src/copy.c (copy_reg): Do not attempt extent_copy on a file
that appears to have no holes.
* NEWS (Changes in behavior): Document this. At first I labeled this
as a bug fix, but that would be inaccurate, considering there is no
documentation of FIEMAP semantics, nor even consensus among kernel
FS developers. Here's hoping SEEK_HOLE/SEEK_DATA support will soon
make it into the linux kernel.
* NEWS: Slightly obfuscate a line to avoid a false-positive
doubled-word ("is-is") match.
Fix a grammar error in news for 8.2.
* cfg.mk (old_NEWS_hash): Resync.
* src/copy.c (extent_copy): Treat an allocated but empty extent
much like a hole. I.E. don't read data we know is going to be NUL.
Also we convert the empty extent to a hole only when SPARSE_ALWAYS
so that the source and dest have the same allocation. This will
be improved soon, when we use fallocate() to do the allocation.
* tests/cp/fiemap-empty: A new test for efficiency and correctness
of copying empty extents.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the change in behavior.
* src/extent-scan.c (extent_scan_read): Add a more stringent check
for OFF_T overflow, to ensure subsequent code is immune.
Detect overlapping extents and adjust, so as files always copied.
Detection using a single scan with fallback to a standard copy
was thought too expensive in memory or time.
* NEWS: Mention the fix
This bug was introduced in commit ca9e212c, 2009-09-24,
"cp, mv: use linkat to guarantee semantics", which
inadvertently disabled the creation of hardlinks to symlinks.
However rather than implementing the intention of that commit
and relying on gnulib linkat emulation, we'll revert to the
previous emulation as that maintains ownership and timestamps.
* src/copy.c (copy_internal): Use our existing hardlink to
symlink emulation when link() might dereference the symlink.
Also ensure that we copy the timestamps of the original symlink
when we use the emulation.
* tests/cp/link-symlink: Add a test to ensure timestamps copied.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the fix.
Reported by Ruediger Meier
* src/extent-scan.h (struct extent_scan): Add the fm_flags member to
pass to the fiemap scan.
* src/extent-scan.c (extent_need_sync): A new function used to
detect Linux kernels before 2.6.38.
(extent_scan_init): Add FIEMAP_FLAG_SYNC when needed.
* tests/cp/sparse-fiemap: Adjust comment.
* NEWS: Mention the change in behavior.
Indirectly suggested by Mike Frysinger
* src/df.c (alloc_table_row): A new function to allocate storage
for a row of strings.
(print_table): A new function to interate over all stored strings in
the table, and apply alignment honoring the max width of each column.
(get_header): Renamed from print_header, and adjusted accordingly.
(get_dev): Renamed from show_dev. Also we no longer wrap longer
device names over two lines, which can be an unexpected issue for
scripts parsing the output from df.
(get_disk): s/show_/get_/
(get_point): Likewise.
(get_entry): Likewise.
(get_all_entries): Likewise.
* NEWS: Mention the change.
* gnulib: Update to latest, to address http://debbugs.gnu.org/8230.
When built on Solaris 9 and run on Solaris 10, touch would segfault.
Reported by Ben Walton.
* bootstrap: Update from gnulib.
* tests/init.sh: Likewise.
* NEWS (Bug fixes): Mention this.
* src/sort.c (SUBTHREAD_LINES_HEURISTIC): Do not spawn a new thread
for every 4 lines. Increase this from 4 to 128K. 128K lines seems
appropriate for a 5-year-old dual-core laptop, but it is too low for
some common combinations of short lines and/or newer systems.
* NEWS (Bug fixes): Mention it.
* src/dd.c (FFS_MASK): A new macro (Find First Set) refactored
from the following enum as it's now used twice.
(usage): Mention the new 'nocache' flag.
(cache_round): A new function to help ignore requests
to drop cache, that are less than page_size.
(invalidate_cache): A new function to call posix_fadvise()
with the appropriate offset and length. Note we don't
use fdadvise() so we can detect errors when count=0.
(dd_copy): Call invalidate_cache() for the portions read.
(iwrite): Likewise for the portions written.
(main): Call invalidate_cache for page_size slop or
for full file when count=0.
* cfg.mk (sc_dd_O_FLAGS): Adjust to pass.
* doc/coreutils.texi (dd invocation): Describe the 'nocache' flag,
and give some examples of how it can be used.
* tests/dd/nocache: A new test.
* tests/Makefile.am: Reference the new test.
* NEWS: Mention the new feature.
* src/du.c (main): Fail on AI_ERR_READ error, rather than merely
diagnosing and continuing. Based on a patch by Stefan Vargyas.
Also move the handling of AI_ERR_EOF into the case stmt.
Do not report ferror/fclose(stdin) failure when we've
already diagnosed e.g., failure to read the DIR, above.
Bug introduced by 2008-11-24 commit 031e2fb5, "du: read and
process --files0-from= input a name at a time,".
* src/wc.c: Handle read failure as with du: do not exit
immediately, but rather go on to print any total and to clean-up.
As above, move the handling of AI_ERR_EOF into the case stmt.
* tests/du/files0-from-dir: New file, to test both du and wc.
* tests/Makefile.am (TESTS): Add it.
* NEWS (Bug fixes): Mention it.
* src/wc.c (main): Diagnose failed argv_iter_init_* failure,
rather than falling through and dereferencing NULL.
Bug introduced by 2008-11-25 commit c2e56e0d,
"wc: read and process --files0-from= input a name at a time,".
* NEWS (Bug fixes): Mention it.
* src/cut.c (set_fields): When computing the maximum range endpoint,
take into consideration the start of any unbounded range, like "999-".
* NEWS (Bug fixes): Mention it.
* tests/misc/cut (big-unbounded-b,c,f): Add tests.
Reported by Paul Marinescu in http://debbugs.gnu.org/7993
The bug was introduced on 2004-12-04 via commit 7380cf79.
This allows one to use join as a field extractor like:
join -a1 -o 1.3,1.1 - /dev/null
* src/join.c (join): Don't flag unpairable lines when
one of the files is empty.
* tests/misc/join: Add a new test for empty input, and adjust
a previous test that was only checking against empty input.
* doc/coreutils.texi (join invocation): Document the change.
* NEWS: Likewise.
Lines with a different number of fields than the first line,
will be truncated or padded.
* src/join.c (prfields): A new function refactored from prjoin(),
to output all but the join field.
(prjoin): Don't swap line1 and line2 when line1 is blank
so that the padding is applied to the right place.
(main): Handle the -o 'auto' option.
* tests/misc/join: Add 6 new cases to test the auto format.
* NEWS: Mention the change in behavior.
Suggestion from Assaf Gordon
* NEWS (Bug fixes): Mention it.
* src/uniq.c (find_field): Stop processing loop when end of line
is reached. Before this fix, 'uniq -f 10000000000 /etc/passwd'
would run for a very long time.
* NEWS (Bug fixes): Mention it.
* src/du.c (prev_level): Move declaration "up" to file-scope global.
(du_files): Reset prev_level to 0 upon abnormal fts_read termination.
Reported by Johathan Nieder in http://bugs.debian.org/609049
Also, improve a diagnostic.
* tests/du/move-dir-while-traversing: Test for the above.
* tests/Makefile.am (TESTS): Add it.
