I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 7061 files FOO.
I then removed trailing white space from math/tgmath.h,
support/tst-support-open-dev-null-range.c, and
sysdeps/x86_64/multiarch/strlen-vec.S, to work around the following
obscure pre-commit check failure diagnostics from Savannah. I don't
know why I run into these diagnostics whereas others evidently do not.
remote: *** 912-#endif
remote: *** 913:
remote: *** 914-
remote: *** error: lines with trailing whitespace found
...
remote: *** error: sysdeps/unix/sysv/linux/statx_cp.c: trailing lines
hurd initialization stages use RUN_HOOK to run various initialization
functions. That is however using absolute addresses which need to be
relocated, which is done later by csu. We can however easily make the
linker compute relative addresses which thus don't need a relocation.
The new SET_RELHOOK and RUN_RELHOOK macros implement this.
Since 9cec82de71 ("htl: Initialize later"), we let csu initialize
pthreads. We can thus let it initialize tls later too, to better align
with the generic order. Initialization however accesses ports which
links/unlinks into the sigstate for unwinding. We can however easily
skip that during initialization.
We stopped adding "Contributed by" or similar lines in sources in 2012
in favour of git logs and keeping the Contributors section of the
glibc manual up to date. Removing these lines makes the license
header a bit more consistent across files and also removes the
possibility of error in attribution when license blocks or files are
copied across since the contributed-by lines don't actually reflect
reality in those cases.
Move all "Contributed by" and similar lines (Written by, Test by,
etc.) into a new file CONTRIBUTED-BY to retain record of these
contributions. These contributors are also mentioned in
manual/contrib.texi, so we just maintain this additional record as a
courtesy to the earlier developers.
The following scripts were used to filter a list of files to edit in
place and to clean up the CONTRIBUTED-BY file respectively. These
were not added to the glibc sources because they're not expected to be
of any use in future given that this is a one time task:
https://gist.github.com/siddhesh/b5ecac94eabfd72ed2916d6d8157e7dchttps://gist.github.com/siddhesh/15ea1f5e435ace9774f485030695ee02
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
During critical sections, signal handling is deferred and thus RPCs return
EINTR, even if SA_RESTART is set. We thus have to restart the whole critical
section in that case.
This also adds HURD_CRITICAL_UNLOCK in the cases where one wants to
break the section in the middle.
I used these shell commands:
../glibc/scripts/update-copyrights $PWD/../gnulib/build-aux/update-copyright
(cd ../glibc && git commit -am"[this commit message]")
and then ignored the output, which consisted lines saying "FOO: warning:
copyright statement not found" for each of 6694 files FOO.
I then removed trailing white space from benchtests/bench-pthread-locks.c
and iconvdata/tst-iconv-big5-hkscs-to-2ucs4.c, to work around this
diagnostic from Savannah:
remote: *** pre-commit check failed ...
remote: *** error: lines with trailing whitespace found
remote: error: hook declined to update refs/heads/master
Instead of having the arch-specific trampoline setup code detect whether
preemption happened or not, we'd rather pass it the sigaction. In the
future, this may also allow to change sa_flags from post_signal().
This change is required in order to correctly release per-thread
resources. Directly reusing the threading library reference isn't
possible since the sigstate is also used early in the main thread,
before threading is initialized.
* hurd/hurd/signal.h (_hurd_self_sigstate): Drop thread reference after
calling _hurd_thread_sigstate.
(_hurd_critical_section_lock): Likewise.
* hurd/hurdsig.c (_hurd_thread_sigstate): Add a reference on the thread.
(_hurd_sigstate_delete): Drop thread reference.
SA_SIGINFO is actually just another way of expressing what we were
already passing over with struct sigcontext. This just introduces the
SIGINFO interface and fixes the posix values when that interface is
requested by the application.
We shall not overflow the size of the description parameter. This makes
describe_number and describe_port behave like strpcpy (except for not filling
all the end of buffer with zeroes) and _S_msg_report_wait use series of
stpncpy-like call. If we were to overflow, we can now detect it and
return ENOMEM.
strcpy cannot be used with overlapping buffer, we have to use memmove
instead. strcpy also cannot be safely used when the destination buffer
is smaller that the source, we need to use strncpy to truncate the
source if needed.
Notifying the proc server is an involved task, and unleashes various signal
handling etc. so we have to do this after e.g. ifunc relocations are
completed.
Since htl does not actually need a stack switch, we can initialize it
like nptl is, avoiding all sorts of startup issues with ifunc.
More precisely, htl defines __pthread_initialize_minimal instead of the
elder _cthread_init_routine. We can then drop the stack switching dances.
Making the brk start exactly at the end of the main application binary was
requiring to get it through the _end symbol, which does not work any more
with recent toolchains, and actually produces in libc.so a confusing
external _end symbol that produces odd results, see
https://sourceware.org/bugzilla/show_bug.cgi?id=23499
Trying to do so is quite outdated anyway with the tendency for address
randomization.
Using _end was also allowing to include the main binary data within
the RLIMIT_DATA, but this also seems outdated with dynamic library
loading, and nowadays' memory consumption via malloc and mmap rather than
statically-allocated data.
This adds a BRK_START macro in <vm_param.h> that just tells where we
want to start the brk, and thus removes the _end symbol.
* sysdeps/mach/hurd/i386/vm_param.h: New file.
* sysdeps/mach/hurd/brk.c: Use BRK_START as brk start instead of _end.
Also ignore __data_start.
* hurd/Versions: Remove _end symbol.
* sysdeps/mach/hurd/i386/libc.abilist: Remove _end symbol.
* hurd/hurdselect.c: Include <sysdep-cancel.h>.
(_hurd_select): Surround call to __mach_msg with enabling async cancel.
* sysdeps/mach/hurd/accept4.c: Include <sysdep-cancel.h>.
(__libc_accept4): Surround call to __socket_accept with enabling async cancel,
and use HURD_DPORT_USE_CANCEL instead of HURD_DPORT_USE.
* sysdeps/mach/hurd/connect.c: Include <sysdep-cancel.h>.
(__connect): Surround call to __file_name_lookup and __socket_connect
with enabling async cancel, and use HURD_DPORT_USE_CANCEL instead of
HURD_DPORT_USE.
* sysdeps/mach/hurd/fdatasync.c: Include <sysdep-cancel.h>.
(fdatasync): Surround call to __file_sync with enabling async cancel, and use
HURD_DPORT_USE_CANCEL instead of HURD_DPORT_USE.
* sysdeps/mach/hurd/fsync.c: Include <sysdep-cancel.h>.
(fsync): Surround call to __file_sync with enabling async cancel, and use
HURD_DPORT_USE_CANCEL instead of HURD_DPORT_USE.
* sysdeps/mach/hurd/ioctl.c: Include <sysdep-cancel.h>.
(__ioctl): When request is TIOCDRAIN, surround call to send_rpc with enabling
async cancel, and use HURD_DPORT_USE_CANCEL instead of HURD_DPORT_USE.
* sysdeps/mach/hurd/msync.c: Include <sysdep-cancel.h>.
(msync): Surround call to __vm_object_sync with enabling async cancel.
* sysdeps/mach/hurd/sigsuspend.c: Include <sysdep-cancel.h>.
(__sigsuspend): Surround call to __mach_msg with enabling async cancel.
* sysdeps/mach/hurd/sigwait.c: Include <sysdep-cancel.h>.
(__sigwait): Surround wait code with enabling async cancel.
* sysdeps/mach/msync.c: Include <sysdep-cancel.h>.
(msync): Surround call to __vm_msync with enabling async cancel.
* sysdeps/mach/sleep.c: Include <sysdep-cancel.h>.
(__sleep): Surround call to __mach_msg with enabling async cancel.
* sysdeps/mach/usleep.c: Include <sysdep-cancel.h>.
(usleep): Surround call to __vm_msync with enabling async cancel.
HURD_*PORT_USE link fd and port with a stack-stored structure, so on
thread cancel we need to cleanup this.
* hurd/fd-cleanup.c: New file.
* hurd/port-cleanup.c (_hurd_port_use_cleanup): New function.
* hurd/Makefile (routines): Add fd-cleanup.
* sysdeps/hurd/include/hurd.h (__USEPORT_CANCEL): New macro.
* sysdeps/hurd/include/hurd/fd.h (_hurd_fd_port_use_data): New
structure.
(_hurd_fd_port_use_cleanup): New prototype.
(HURD_DPORT_USE_CANCEL, HURD_FD_PORT_USE_CANCEL): New macros.
* sysdeps/hurd/include/hurd/port.h (_hurd_port_use_data): New structure.
(_hurd_port_use_cleanup): New prototype.
(HURD_PORT_USE_CANCEL): New macro.
* hurd/hurd/fd.h (HURD_FD_PORT_USE): Also refer to HURD_FD_PORT_USE_CANCEL.
* hurd/hurd.h (__USEPORT): Also refer to __USEPORT_CANCEL.
* hurd/hurd/port.h (HURD_PORT_USE): Also refer to HURD_PORT_USE_CANCEL.
* hurd/fd-read.c (_hurd_fd_read): Call HURD_FD_PORT_USE_CANCEL instead
of HURD_FD_PORT_USE.
* hurd/fd-write.c (_hurd_fd_write): Likewise.
* sysdeps/mach/hurd/send.c (__send): Call HURD_DPORT_USE_CANCEL instead
of HURD_DPORT_USE.
* sysdeps/mach/hurd/sendmsg.c (__libc_sendmsg): Likewise.
* sysdeps/mach/hurd/sendto.c (__sendto): Likewise.
* sysdeps/mach/hurd/recv.c (__recv): Likewise.
* sysdeps/mach/hurd/recvfrom.c (__recvfrom): Likewise.
* sysdeps/mach/hurd/recvmsg.c (__libc_recvmsg): Call __USEPORT_CANCEL
instead of __USEPORT, and HURD_DPORT_USE_CANCEL instead of
HURD_DPORT_USE.
_hurdsig_preemptors and _hurdsig_preempted_set are not ABI symbols,
so do not declare them. HURD_PREEMPT_SIGNAL_P is an implementation
detail, so move it as well.
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
This fixes various build errors due to deprecation warnings.
Fixes commit 02802fafcf
("signal: Deprecate additional legacy signal handling functions").
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
In case the signal arrives before the __mach_msg call, we need to catch
between the sigprocmask call and the __mach_msg call. Let's just reuse
the support for sigsuspend to make the signal send a message that
our __mach_msg call will just receive.
* hurd/hurdselect.c (_hurd_select): Add sigport and ss variables. When
sigmask is not NULL, create a sigport port and register as
ss->suspended. Add it to the portset. When we receive a message on it,
set error to EINTR. Clean up sigport and portset appropriately.
* hurd/hurdsig.c (wake_sigsuspend): Note that pselect also uses it.
This adds _hurd_sigstate_set_global_rcv used by libpthread to enable
POSIX-confirming behavior of signals on a per-thread basis.
This also provides a sigstate destructor _hurd_sigstate_delete, and a
global process signal state, which needs to be locked and check when
global disposition is enabled, thus the addition of _hurd_sigstate_lock
_hurd_sigstate_actions _hurd_sigstate_pending _hurd_sigstate_unlock helpers.
This also updates all the glibc code accordingly.
This also drops support for get_int(INIT_SIGMASK), which did not make sense
any more since we do not have a single signal thread any more.
During fork/spawn, this also reinitializes the child global sigstate's
lock. That cures an issue that would very rarely cause a deadlock in the
child in fork, tries to unlock ss' critical section lock at the end of
fork. This will typically (always?) be observed in /bin/sh, which is not
surprising as that is the foremost caller of fork.
To reproduce an intermediate state, add an endless loop if
_hurd_global_sigstate is locked after __proc_dostop (cast through
volatile); that is, while still being in the fork's parent process.
When that triggers (use the libtool testsuite), the signal thread has
already locked ss (which is _hurd_global_sigstate), and is stuck at
hurdsig.c:685 in post_signal, trying to lock _hurd_siglock (which the
main thread already has locked and keeps locked until after
__task_create). This is the case that ss->thread == MACH_PORT_NULL, that
is, a global signal. In the main thread, between __proc_dostop and
__task_create is the __thread_abort call on the signal thread which would
abort any current kernel operation (but leave ss locked). Later in fork,
in the parent, when _hurd_siglock is unlocked in fork, the parent's
signal thread can proceed and will unlock eventually the global sigstate.
In the client, _hurd_siglock will likewise be unlocked, but the global
sigstate never will be, as the client's signal thread has been configured
to restart execution from _hurd_msgport_receive. Thus, when the child
tries to unlock ss' critical section lock at the end of fork, it will
first lock the global sigstate, will spin trying to lock it, which can
never be successful, and we get our deadlock.
Options seem to be:
* Move the locking of _hurd_siglock earlier in post_signal -- but that
may generally impact performance, if this locking isn't generally
needed anyway?
On the other hand, would it actually make sense to wait here until we
are not any longer in a critical section (which is meant to disable
signal delivery anyway (but not for preempted signals?))?
* Clear the global sigstate in the fork's child with the rationale that
we're anyway restarting the signal thread from a clean state. This
has now been implemented.
Why has this problem not been observed before Jérémie's patches? (Or has
it? Perhaps even more rarely?) In _S_msg_sig_post, the signal is now
posted to a *global receiver thread*, whereas previously it was posted to
the *designated signal-receiving thread*. The latter one was in a
critical section in fork, so didn't try to handle the signal until after
leaving the critical section? (Not completely analyzed and verified.)
Another question is what the signal is that is being received
during/around the time __proc_dostop executes.
This should not change the current behavior, although this fixes a few
minor bugs which were made apparent in the process of global signal
disposition work:
- Split into more functions
- Scope variables more restrictively
- Split out inner functions
- refactor check_pending_signals
- make sigsuspend POSIX-conformant.
- fix uninitialized act value.
The valid_nanoseconds () static inline function has been introduced to
check if nanoseconds value is in the correct range - greater or equal to
zero and less than 1000000000.
The explicit #include <time.h> has been added to files where it was
missing.
The __syscall_slong_t type for ns has been used to avoid issues on x32.
Tested with:
- scripts/build-many-glibcs.py
- make PARALLELMFLAGS="-j12" && make PARALLELMFLAGS="-j12" xcheck on x86_64
This fixes the following:
- On error, poll must not return without polling, including EBADF, and instead
report POLLHUP/POLLERR/POLLNVAL
- Select must report EBADF if some set contains an invalid FD.
The idea is to move error management to after all select calls, in the
poll/select final treatment. The error is instead recorded in a new `error'
field, and a new SELECT_ERROR bit set.
Thanks Svante Signell for the initial version of the patch.
* hurd/hurdselect.c (SELECT_ERROR): New macro.
(_hurd_select):
- Add `error' field to `d' structures array.
- If a poll descriptor is bogus, set EBADF, but continue with a zero
timeout.
- Go through the whole fd_set, not only until _hurd_dtablesize. Return
EBADF there is any bit set above _hurd_dtablesize.
- Do not request io_select on bogus descriptors (SELECT_ERROR).
- On io_select request error, record the error.
- On io_select bogus reply, use EIO error code.
- On io_select bogus or error reply, record the error.
- Do not destroy reply port for bogus FDs.
- On error, make poll set POLLHUP in the EPIPE case, POLLNVAL in the
EBADF case, or else POLLERR.
- On error, make select simulated readiness.
Rely on servers to implement timeouts, so that very short values (including
0) don't make mach_msg return before valid replies can be received. The
purpose of this scheme is to guarantee a full client-server round-trip,
whatever the timeout value.
This change depends on the new io_select_timeout RPC being implemented by
servers.
* hurd/Makefile (user-interfaces): Add io_reply and io_request.
* hurd/hurdselect.c: Include <sys/time.h>, <hurd/io_request.h> and
<limits.h>.
(_hurd_select): Replace the call to __io_select with either
__io_select_request or __io_select_timeout_request, depending on the
timeout. Count the number of ready descriptors (replies for which at
least one type bit is set). Implement the timeout locally when there is
no file descriptor.
The function attempts to optimize this case by performing one IPC system
call with the timeout included among the parameters, but in the absence
of a reply, it will call mach_msg again with the same timeout later,
effectively doubling the total timeout of the select/poll call.
Remove this optimization for the time being.
* hurd/hurdselect.c (_hurd_select): Always call __io_select with no
timeout.
