* configure.ac: Reenable distribution of gzip-compressed
tarballs, for Guix bootstrapping reasons as discussed at:
https://lists.gnu.org/r/coreutils/2020-02/msg00042.html
* THANKS.in: Remove me, as now a committer.
* NEWS (Build-related): Mention this.
Since v6.10-21-ged5c4e7 `rmdir --ignore-fail-on-non-empty`
had reversed the failure status for directories that failed
to be removed for permissions reasons. I.E. it would have
returned a failure status for such non empty dirs, and vice versa.
* src/rmdir.c (errno_may_be_non_empty): Rename from the
more confusing errno_may_be_empty(), and remove the EEXIST
case (specific to Solaris), which is moot here since
handled in errno_rmdir_non_empty().
(ignorable_failure): Fix the logic error so that
_non_ empty dirs are deemed to have ignorable failures.
(main): Fix clobbering of errno by is_empty_dir().
(remove_parents): Likewise.
* tests/rmdir/ignore.sh: Add a test case.
* THANKS.in: Add reporter who fixed the errno handling.
* NEWS: Mention the bug fix.
Fixes https://bugs.gnu.org/39364
* cfg.mk (sc_prohibit_colon_redirection): Don't expect `|` to denote
the pipe character in git grep.
(sc_tests_executable)
(sc_case_insensitive_file_names)
(sc_some_programs_must_avoid_exit_failure)
(sc_prohibit_test_background_without_cleanup_)
(sc_prohibit_test_calls_print_ver_with_irrelevant_argument)
(sc_prohibit_test_ulimit_without_require_)
(sc_prohibit_test_background_without_cleanup_)
(sc_THANKS_in_duplicates)
*sc_prohibit_test_calls_print_ver_with_irrelevant_argument):
Don't expect builddir to be a descendant of srcdir.
(sc_strftime_check): Don't check file size against 0 when "N\nq\n" was
already put in the file.
* THANKS.in: Remove me.
* src/split.c (set_suffix_length): Use a more standard
zero based logN calculation for the number of units.
* tests/split/suffix-auto-length.sh: Add a test case.
* THANKS.in: Mention the reporter.
* NEWS: Mention the fix.
Fixes https://bugs.gnu.org/35291
Ensure this _does_ recreate the symlink
Given "path1" and "path2" are on different devices.
$ touch "path1/file"
$ cd path2/; ln -s path1/file
$ cp -dsf path1/file .
Ensure this does _not_ overwrite file
$ touch file
$ ln -s file l1
$ cp -sf l1 file
* src/copy.c (same_file_ok): Remove device ids from consideration,
instead deferring to future EXDEV with --link or allowing
the first case above to work.
Also ensure that we do not exist this function too early,
when the destination file is not a symlink, which protects
against the second case.
* tests/cp/cross-dev-symlink.sh: Add a test for the first case.
* tests/cp/same-file.sh: Add a test for the second case above.
* NEWS: Mention the bug fixes.
* THANKS.in: Mention the reporters who also analyzed the code.
Fixes https://bugs.gnu.org/31364
Previously with oflag=direct the call to invalidate_cache()
was not passed to the kernel, as it was less than a page size,
and a subsequent call was not made to invalidate the pending space.
Similarly with oflag=nocache the pending space at EOF was
not invalidated. Even though these amount to only a single page
in the page cache it can be significant. For example on
XFS before kernel patch v4.9-rc1-4-g0ee7a3f, O_DIRECT files
would have been read inefficiently if any pages were cached,
even if they were already synced to storage.
* src/dd.c (i_nocache_eof, o_nocache_eof): New bools used
to control when we want invalidate_cache(,0) to clear to EOF.
(cache_round): Use IO_BUFSIZE (currently 132KiB) to minimize
calls to the relatively expensive advise function, rather
than page_size. This also makes it clear that while the
kernel function operates on pages, this size is chosen for
performance reasons.
(invalidate_cache): Refactor to share more code between
input and output paths. Use i_nocache_eof and o_nocache_eof
rather than proxying off max_records. Ensure we
invalidate full pages when clearing to EOF as the kernel
will ignore any non complete pages. Fix the offset used
for the output path.
(dd_copy): Invalidate the cache of the input after the
offset is updated, for consistency and so we don't try to
invalidate before the start of the file. When we read
EOF on input, set flags so that we invalidate to EOF.
(main): Invalidate to EOF in more cases, by depending
on the i_nocache_eof and o_nocache_eof flags.
* doc/coreutils.texi (dd invocation): Clarify the alignment
and persisted caveats on the example applying "nocache"
to part of a file.
* tests/dd/nocache_eof.sh: A new test.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
Issue reported by Eric Bergen.
* tests/split/filter.sh: Due to an invalid 'FILE = zero.in'
construct trying to initialize a FILE variable, it would
instead try to run the FILE command which is present on
macOS 10.13 with APFS.
We also remove a redundant duplicate test clause introduced
during a rebase, and simplify the piped timeout command,
to avoid requiring a subshell and associated quoting.
* THANKS.in: Add the reporter Jack Howarth.
Fixes https://bugs.gnu.org/28506
When systemd is configured to automount a remote file system - see
'man systemd.automount(5)', then the mount point is initially
mounted by systemd with the file system type "autofs".
When the resource is used later on, then the wanted file system is
mounted over that mount point on demand.
'df -l' triggered systemd to mount the file system because it called
stat() on the mount point.
Instead of single-casing "autofs" targets, we can avoid stat()ing
all dummy file systems (which includes "autofs"), because those are
skipped later on in get_dev() anyway.
*src/df.c (filter_mount_list): Also skip dummy file systems unless
the -a option or a specific target are given.
* NEWS: Mention the fix.
Co-authored-by: Bernhard Voelker <mail@bernhard-voelker.de>
Fixes http://bugzilla.suse.com/show_bug.cgi?id=1043059
Note we don't support the --algorithm option of the
b2sum command in the external BLAKE2 project, as that
was deemed too confusing for users.
"BLAKE2b" was chosen as the default algorithm to use,
which is single threaded but performs well on 64 bit.
* src/blake2: CC0 source copied from external project.
* cfg.mk[VC_LIST_ALWAYS_EXCLUDE_REGEX]: Exclude blake2/
from syntax checks, make update-copyright, etc.
* src/local.mk: Reference the sources for b2sum,
and set the compilation flags.
* doc/coreutils.texi (b2sum invocation): Reference the
md5sum invocation node, and add descriptions of -l.
* tests/misc/b2sum.sh: Add new test.
* tests/local.mk: Reference new test.
* AUTHORS: Add new binary.
* README: Likewise.
* build-aux/gen-lists-of-programs.sh: Likewise.
* man/.gitignore: Likewise.
* scripts/git-hooks/commit-msg: Likewise.
* man/b2sum.x: New man page template.
* man/local.mk: Reference new template.
* src/.gitignore: Ignore new binaries.
* src/blake2/.gitignore: Ignore new build atrifacts.
* src/md5sum.c (usage): Describe the new -l option.
* NEWS: Mention the new program.
which was not the case when inotify was not available.
* src/tail.c (any_live_files): Simplify, since the IGNORE
flag is now only set when a file should be ignored indefinitely.
(recheck): Only output the "giving up on name" message
when that's actually the case. Only set the IGNORE flag
when ignoring a file indefinitely.
(tail_file): Likewise.
* tests/tail-2/retry.sh: Add a test case. Also run
all existing test cases with and without inotify.
NEWS: Mention the fix.
THANKS.in: Add the reporter.
Fixes http://bugs.gnu.org/24495 which was detected
using Symbolic Execution techniques developed in
the course of the SYMBIOSYS research project at
COMSYS, RWTH Aachen University.
Use hash table for seaching in filter_mount_list() and get_dev()
This improves performance for 20K mount entries from:
real 0m1.731s
user 0m0.532s
sys 0m1.188s
to:
real 0m1.066s
user 0m0.028s
sys 0m1.032s
* src/df.c (devlist_table): Define hash table.
(devlist_hash): Add hash function.
(devlist_compare): Add hash comparison function.
(devlist_for_dev): Add lookup function.
(devlist_free): Add cleanup function.
(filter_mount_list): Use the above hash table.
While at it, rename the variable 'devlist' to 'seen_dev' for
better readability.
(me_for_dev): Use the above lookup function.
NEWS: Mention the improvement.
THANKS.in: Remove the committer; add original submitter Josef Cejka.
* src/stty.c (usage): Remove an erroneous call to translate an
empty string, added in commit v8.23-112-g564f84a, which results
in the gettext header being printed for translated languages.
* THANKS.in: Remove the now committer.
* NEWS: Mention the bug fix.
* src/date.c (usage): The colon is used in the timezone offset
since commit v8.24-64-g17bbf6c.
* THANKS.in: Remove committer.
Fixes http://bugs.gnu.org/22491
doc/coreutils.texi (dd invocation): Add oflag=sync to the streaming
example. Also reference the "direct" flag. Mention this is only
a request to the system.
* src/dd.c (usage): Mention the "sync" flag along with "nocache".
Also mention that it's only a request to drop the cache.
* THANKS.in: Add reporter Francois Rigault.
This can be useful if you want to further process data
from process substitutions. For example:
datagen | tee >(md5sum --tag) > >(sha256sum --tag) | sort
* doc/coreutils.texi (tee invocation): Mention that -p is
useful with pipes that may not consume all data.
Add an example, similar to the one above.
* THANKS.in: Add Jirka Hladky.
Suggested in https://bugzilla.redhat.com/1250113
* AUTHORS: Add base32.
* THANKS.in: Add suggester.
* README: Reference the new program.
* NEWS: Mention the new program.
* src/.gitignore: Ignore the new binary.
* bootstrap.conf: Reference the gnulib base32 module.
* build-aux/gen-lists-of-programs.sh: Add base32.
* man/base32.x: A new template.
* man/.gitignore: Ignore the new man page.
* man/local.mk: Reference the new man page.
* doc/coreutils.texi (base32 invocation): Document the new command.
* src/local.mk: Adjust to build base32 based on base64.c.
* src/base64.c: Parameterize to use the correct headers,
functions and buffer sizes, depending on which binary
is being built.
* tests/misc/base64.pl: Adjust to test both base32 and base64.
* tests/misc/tty-eof.pl: Add base32 as a program that
accepts input on stdin without any options specified.
* scripts/git-hooks/commit-msg: Add base32 to the template.
* doc/coreutils.texi (cp invocation): Mention that when copying files
without preserving permissions, the umask or a default ACL affect
the mode of new files.
* THANKS.in: Remove committer.
Related to http://bugs.gnu.org/8527
* THANKS.in: Document the preferred sort order as a comment
at the top of the file. Change "Марк Коренберг" to latin1
("Mark Korenberg"). Sort all entries.
* cfg.mk (sc_THANKS_in_sorted): Add rule to ensure that
'THANKS.in' remains sorted.
Each user has a maximum number of inotify watches,
so handle the cases where we exhaust these resources.
* src/tail.c (tail_forever_inotify): Ensure we inotify_rm_watch()
the watch for an inode, when replacing with a new watch for a name.
Return all used inotify resources when reverting to polling.
Revert to polling upon first indication of inotify resource exhaustion.
Revert to polling on any inotify resource exhaustion.
Diagnose resource exhaustion correctly in all cases.
Avoid redundant reinsertion in the hash for unchanged watches
(where only attributes of the file are changed).
* tests/tail-2/retry.sh: Avoid false failure when reverting to polling.
* tests/tail-2/inotify-rotate.sh: Likewise.
* tests/tail-2/symlink.sh: Likewise.
* tests/tail-2/inotify-rotate-resources.sh: New test to check
that we're calling inotify_rm_watch() for replaced files.
* tests/local.mk: Reference the new test.
* NEWS: Mention the bug fix.
* THANKS.in: Thanks for reporting and problem identification.
Following on from http://bugs.gnu.org/17546
make it more obvious that du may elide specified operands
to avoid double counting in the set.
* src/du.c (usage): Specify that du operates on the set of
operands, rather than each independently.
* doc/coreutils.texi (du invocation): Likewise. Also state
that the number of entries printed may change due to the
order specified. Currently, deeper items specified earlier
will result in them being displayed, but don't mention that
implementation detail in the documentation.
* THANKS.in: Add reporter.
Reported by Stephen Shirley
* THANKS.in: Change the comment at the top to send change requests
regarding this file to the main mailing list rather than referring
to cp's --help output for the mailing list's address - which does
not include that information anymore.
If the hash structures grow sufficiently large so that
the system is actively swapping, then the deallocation
can take a significant amount of time. Details at:
http://lists.gnu.org/archive/html/coreutils/2014-08/msg00012.html
* src/cp.c (main): Only call hash deallocation routines
when in lint checking mode.
* THANKS.in: Remove as now in the git author list.
* src/df.c (get_dev): asssert() on Solaris 10 is not marked as
__noreturn__ and thus the compiler may think V is uninitialized
later on in the function.
* THANKS.in: Remove the now committer.
* tests/dd/sparse.sh: When testing that a hole is created,
use an existing sparse destination file, so that we're
not write extending the file size, and thus avoiding
speculative preallocation which can result in smaller
holes than requested.
Workaround suggested by Brian Foster
This removes an unportable assumption that if lseek succeeds, the
file is capable of seeking. See: http://bugs.gnu.org/17145
* src/head.c (elseek): New function, for consistency in reporting
lseek failures.
(elide_tail_bytes_file, elide_tail_lines_seekable)
(elide_tail_lines_file, head_lines, head): Use it.
(elide_tail_bytes_file, elide_tail_lines_file):
New args CURRENT_POS and SIZE. All uses changed. Don't bother
invoking lseek, since we know the file's pos and size now.
(elide_tail_bytes_file): Change a local from uintmax_t to off_t,
since it fits.
(head): Use lseek only on regular files, since its behavior on
unseekable devices is implementation-defined.
* NEWS: Document this.
Input buffering is best avoided because it introduces
delayed processing of output for intermittent input,
especially when the output size is less than that of
the input buffer. This is significant when output
is being further processed which could happen if split
is writing to precreated fifos, or through --filter.
If input is arriving quickly from a pipe then this will
already be buffered before we read it, so fast arriving
input shouldn't be a performance issue.
* src/split.c (lines_split, lines_bytes_split, bytes_split,
lines_chunk_split, bytes_chunk_extract): s/full_read/safe_read/.
* THANKS.in: Mention the reporter.
* NEWS: Mention the improvement.
Prior to this change, "ln -sr '' F" would segfault, attempting
to read path2[1] in relpath.c's path_common_prefix function.
This problem arises whenever canonicalize_filename_mode returns
NULL.
* src/ln.c (convert_abs_rel): Call relpath only when
both canonicalize_filename_mode calls return non-NULL.
* tests/ln/relative.sh: Add a test to trigger this failure.
* THANKS.in: List reporter's name/address.
* NEWS (Bug fixes): Mention it.
Reported by Erik Bernstein in 739752@bugs.debian.org.
Fixes http://bugs.gnu.org/17010.
- Support arbitrary numbers in --groups, consistent with
what is already done for --userspec
- Avoid look-ups entirely for --groups items with a leading '+'
- Support names that are actually numbers in --groups
- Ignore an empty --groups="" option for consistency with --userspec
- Look up both inside and outside the chroot with inside taking
precedence. The look-up outside may load required libraries
to complete the look-up inside the chroot. This can happen for
example with a 32 bit chroot on a 64 bit system, where the
32 bit NSS plugins within the chroot fail to load.
* src/chroot.c (parse_additional_groups): A new function refactored
from set_addition_groups(), to just do the parsing. The actual
setgroups() call is separated out for calling from the chroot later.
(main): Call parse_user_spec() and parse_additional_groups()
both outside and inside the chroot for the reasons outlined above.
* tests/misc/chroot-credentials.sh: Ensure arbitrary numeric IDs
can be specified without causing look-up errors.
* NEWS: Mention the improvements.
* THANKS.in: Add Norihiro Kamae who initially reported the issue
with a proposed patch.
Also thanks to Dmitry V. Levin for his diagnosis and sample patch.
* src/ln.c (errno_nonexisting): A new function to determine if
the errno implies that a file doesn't or can't (currently) exist.
(target_directory_operand): Use the new function to expand the
set of errors we handle.
* tests/ln/sf-1.sh: Add test cases for the newly handled errors.
* THANKS.in: Mention the reporter.
* NEWS: Mention the bug fix.
* src/copy.c (copy_internal): Use the global process context
to set the context of existing directories before they're populated.
This is more consistent with the new directory case, and fixes
a bug for existing directories where we erroneously set the
context to the last copied descendent, rather than to that of
the source directory itself.
* tests/cp/cp-a-selinux.sh: Add a test for this case.
* NEWS: Mention the fix.
* THANKS.in: Add reporter Michal Trunecka.
A sync operation is very often expensive. For illustration
I timed the following python script which indicated that
each ext4 dir sync was taking about 2ms and 12ms, on an
SSD and traditional disk respectively.
import os
d=os.open(".", os.O_DIRECTORY|os.O_RDONLY)
for i in range(1000):
os.fdatasync(d)
So syncing for each character for each file can result
in significant delays. Often this overhead is redundant,
as only the data is sensitive and not the file name.
Even if the names are sensitive, your file system may
employ synchronous metadata updates, which also makes
explicit syncing redundant.
* tests/misc/shred-remove.sh: Ensure all the new parameters
actually unlink the file.
* doc/coreutils.texi (shred invocation): Describe the new
parameters to the --remove option.
* src/shred.c (Usage): Likewise.
(main): Parse the new options.
(wipename): Inspect the new enum to see which of
the now optional tasks to perform.
* NEWS: Mention the new feature.
* THANKS.in: Add reporter Joseph D. Wagner
* src/md5sum.c (usage): s/three/four/ in the message pertaining
to the --check related options. Also clarify that --strict
is just significant for the formatting of the checksum lines.
Also since we're changing both strings, move the --strict description
in with the description of the other options and order alphabetically.
* THANKS.in: Added reporter: Daniel Mach
* doc/coreutils.texi (df invocation): s/pseude/pseudo/
* THANKS.in (Filipus Klutiero): Remove entry, now that it will be
automatically included in the generated THANKS file.
Fixes http://bugs.gnu.org/15041
* src/od.c (PRINT_FIELDS): Declare "i" to be of type uintmax_t, so that
the numerator in the expression for "next_pad" does not overflow.
(print_named_ascii): Likewise.
(print_ascii): Likewise.
Bug introduced via commit v6.12-42-g20c0b87.
* tests/misc/od.pl: Exercise each of the three affected code paths.
* NEWS (Bug fixes): Mention it.
Reported by Rich Burridge.
This regression was introduced in commit v8.19-132-g3786fb6.
* src/seq.c (seq_fast): Don't use puts() to output the first number,
and instead insert it into the buffer as for other numbers.
Also output the terminator unconditionally.
* tests/misc/seq.pl: Add some basic tests for the -s option.
* NEWS: Mention the fix.
* THANKS.in: Reported by Philipp Gortan.
This allows efficient processing of multiple files,
while also increasing compatibility with BSD's readlink(1).
We also add the -z, --zero option to delimit output items
with the NUL character which disambiguates output in the
presence of '\n' characters.
* src/readlink.c (usage): Add the --zero description,
and also adjust the description of --no-newline accordingly.
(main): Handle the -z option and iterate over multiple arguments.
Also as in commit v8.15-24-g9d46b25 we use fputs() and putchar()
rather than printf() for performance reasons.
* doc/coreutils.texi (readlink invocation): Document the
new --zero option, adjust the --no-newline description, and
tweak the general info to indicate multiple files are supported.
* tests/readlink/multi.sh: A new test for the new functionality.
* tests/local.mk: Reference the new test.
* man/readlink.x: Adjust the summary and also reference realpath.
* NEWS: Mention the improvement.
* THANKS.in: Suggested by Aaron Davies.
Teach tail -f that it must use polling on ceph file systems, and
let stat -f --format=%T report the file system type name, "ceph".
Website: http://ceph.com/
* src/stat.c (human_fstype): Add a case: ceph, 0x00C36400, remote.
* NEWS (Improvements): Mention it.
* THANKS.in: Update.
Reported by Konrad Wróblewski in http://bugs.gnu.org/13172.
The syntax-check sc_THANKS_in_duplicates complained about
that excess entry.
* THANKS.in (Colin Watson): Remove entry, now that it will be
automatically included in the generated THANKS file.
* src/seq.c (scan_arg): Calculate the width more accurately
for numbers specified using scientific notation.
* tests/misc/seq.pl: Add tests for cases that were mishandled.
* NEWS: Mention the fix.
* THANKS.in: Reported by Marcel Böhme.
Fixes http://bugs.gnu.org/12959
* src/local.mk (src_factor_LDADD): Append $(LIBICONV).
* crg.mk (sc_check-I18N-AUTHORS): A new syntax check rule
to ensure we add LIBICONV where appropriate.
* THANKS.in: Add the reporter.
Reported by Christian Jullien
Syntax check suggested by Jim Meyering
Teach tail -f that it must use polling on vmhgfs file systems, and
let stat -f --format=%T report the file system type name, "vmhgfs".
* src/stat.c (human_fstype): Add a case: vmhgfs, 0xbacbacbc, remote.
* NEWS (Improvements): Mention it.
* THANKS.in: Update.
Reported by Daniel Tschinder in http://bugs.gnu.org/12461.
When listing a directory containing dangling symlinks,
and not outputting a long format listing, and orphaned links
are set to no coloring in LS_COLORS, then the symlinks
would get no color rather than reverting to the standard
symlink color. The issue was introduced in v8.13-19-g84457c4
* src/ls.c (print_color_indicator): Use the standard method
to check if coloring is specified for orphaned symlinks.
The existing method would consider 'or=00' or 'or=0' as significant
in LS_COLORS. Even 'or=' was significant as in that case the
string='or=' and the length=0. Also apply the same change
for missing symlinks for consistency.
(gobble_file): Remove the simulation of linkok, which is only
tested in print_color_indicator() which now handles this directly
by keying on the LS_COLORS values correctly.
* tests/misc/ls-misc.pl: Add a test case.
* THANKS: Add the reporter.
* NEWS: Mention the fix.
Reported-by: David Matei
The multiple-precision factoring code (with HAVE_GMP) was copied from
a now-obsolete version of GMP that did not pass proper arguments to
the mpz_probab_prime_p function. It makes that code perform no more
than 3 Miller-Rabin tests only, which is not sufficient.
A Miller-Rabin test will detect composites with at least a probability
of 3/4. For a uniform random composite, the probability will actually
be much higher.
Or put another way, of the N-3 possible Miller-Rabin tests for checking
the composite N, there is no number N for which more than (N-3)/4 of the
tests will fail to detect the number as a composite. For most numbers N
the number of "false witnesses" will be much, much lower.
Problem numbers are of the form N=pq, p,q prime and (p-1)/(q-1) = s,
where s is a small integer. (There are other problem forms too,
involving 3 or more prime factors.) When s = 2, we get the 3/4 factor.
It is easy to find numbers of that form that cause coreutils' factor to
fail:
465658903
2242724851
6635692801
17709149503
17754345703
20889169003
42743470771
54890944111
72047131003
85862644003
98275842811
114654168091
117225546301
...
There are 9008992 composites of the form with s=2 below 2^64. With 3
Miller-Rabin tests, one would expect about 9008992/64 = 140766 to be
invalidly recognized as primes in that range.
* src/factor.c (MR_REPS): Define to 25.
(factor_using_pollard_rho): Use MR_REPS, not 3.
(print_factors_multi): Likewise.
* THANKS.in: Remove my name, now that it will be automatically
included in the generated THANKS file.
This change enables tail -f to use inotify and lets
stat -f --format=%T report the file system type name, "zfs".
* src/stat.c (human_fstype): Add a case: zfs, 0x2fc12fc1.
* NEWS (Improvements): Mention it.
* THANKS.in: Update.
Reported by Raimonds Miltins in http://bugs.gnu.org/12301.
* src/remove.c (prompt): Hoist the computation of is_empty, since we'll
need it slightly earlier.
Before, this function would arrange to fail with EISDIR when processing
a directory without --recursive (-r). Adjust the condition to exempt
an empty directory when --dir has been specified.
Improve comments.
* tests/rm/d-3: New file, to ensure that rm -d -i dir works.
* tests/Makefile.am (TESTS): Add it.
* NEWS (Bug fixes): Mention it.
* THANKS.in: Update.
Reported by Michael Price in http://bugs.gnu.org/12260
