I've updated copyright dates in glibc for 2020. This is the patch for
the changes not generated by scripts/update-copyrights and subsequent
build / regeneration of generated files. As well as the usual annual
updates, mainly dates in --version output (minus libc.texinfo which
previously had to be handled manually but is now successfully updated
by update-copyrights), there is a fix to
sysdeps/unix/sysv/linux/powerpc/bits/termios-c_lflag.h where a typo in
the copyright notice meant it failed to be updated automatically.
Please remember to include 2020 in the dates for any new files added
in future (which means updating any existing uncommitted patches you
have that add new files to use the new copyright dates in them).
In commit 4b7c74179c the nsswitch.conf
file was harmonized with downstream distributions, but this change
included adding "initgroups: files". We should not add initgroups by
default, we can have it, but it should be commented out to allow it
to inherit the settings for group. The problem is principally that
downstream authconfig won't update initgroups and it will get out of
sync with the setting for group.
The current default nsswitch.conf file provided by glibc is not very
distribution friendly. The file contains some minimal directives that no
real distribution uses. This update aims to provide a rich set of
comments which are useful for all distributions, and a broader set of
service defines which should work for all distributions.
Tested defaults on x86_64 and they work. The nsswitch.conf file more
closely matches what we have in Fedora now, and I'll adjust Fedora to
use this version with minor changes to enable Fedora-specific service
providers.
v2
- Add missing databases to manual.
- Add link to manual from default nsswitch.conf.
- Sort nsswitch.conf according to most used database first.
v3
- Only mention implemented services in 'NSS Basics.'
- Mention 'automount' in 'Services in the NSS configuration.'
- Sort services in alphabetical order.
v4
- Project name is 'Samba'.
v5
- Fix typo in manual/nss.texi.
v6
- Fix another typo in manual/nss.texi. Ran spell checker this time.
nss_db allows for getpwent et al to be called without a set*ent,
but it only works once. After the last get*ent a set*ent is
required to restart, because the end*ent did not properly reset
the module. Resetting it to NULL allows for a proper restart.
If the database doesn't exist, however, end*ent erroniously called
munmap which set errno.
The test case runs "makedb" inside the testroot, so needs selinux
DSOs installed.
This patch fixes the following gcc 9 warnings for "make xcheck" / "make bench":
-string/tst-strcasestr.c:
../include/bits/../../misc/bits/error.h:42:5: error: ‘%s’ directive argument is null [-Werror=format-overflow=]
-argp/argp-test.c:
argp-test.c:130:20: error: ‘%d’ directive writing between 1 and 11 bytes into a region of size 10 [-Werror=format-overflow=]
argp-test.c:130:19: note: directive argument in the range [-2147483648, 122]
argp-test.c:130:5: note: ‘sprintf’ output between 2 and 12 bytes into a destination of size 10
-nss/tst-field.c:
tst-field.c:52:7: error: ‘%s’ directive argument is null [-Werror=format-overflow=]
-benchtests/bench-strstr.c:
../include/bits/../../misc/bits/error.h:42:5: error: ‘%s’ directive argument is null [-Werror=format-overflow=]
-benchtests/bench-malloc-simple.c:
bench-malloc-simple.c:93:16: error: iteration 3 invokes undefined behavior [-Werror=aggressive-loop-optimizations]
ChangeLog:
[BZ #24556]
* string/test-strcasestr.c (check_result): Add NULL check.
* nss/tst-field.c (check_rewrite): Likewise.
* benchtests/bench-strstr.c (do_one_test): Likewise.
* string/test-strstr.c (check_result): Likewise.
* argp/argp-test.c (popt): Increase size of buf to 12.
* benchtests/bench-malloc-simple.c (bench):
Do not initialize tests array out of bounds.
The function uses the internal service_user type, so it is not
really usable from the outside of glibc. Rename the function
to __nss_database_lookup2 for internal use, and change
__nss_database_lookup to always indicate failure to the caller.
__nss_next already was a compatibility symbol. The new
implementation always fails and no longer calls __nss_next2.
unscd, the alternative nscd implementation, does not use
__nss_database_lookup, so it is not affected by this change.
Linking to the NSS module directly does not work if the linker defaults
to --as-needed because it will remove the apparently unused DSO
reference and not generate a DT_NEEDED entry. Use an explicit dlopen
call, like in the other chroot tests involving NSS modules.
Add /etc/host.conf file with 'multi on' to tst-nss-files-hosts-long.root.
Ensures the entire file, and all long lines, need to be parsed for the
test.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
When the /etc/hosts file has a line longer than 1028
characters getent ahostsv4 and ahostsv6 will fail.
This test performs a getent call on a /etc/hosts file that contains
a very long line (greater than 1028) using the test-in-container
framework.
If /etc/aliases ends with a continuation line (a line that starts
with whitespace) which does not have a trailing newline character,
the file parser would crash due to a null pointer dereference.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
The IPv4 address parser in the getaddrinfo function is changed so that
it does not ignore trailing whitespace and all characters after it.
For backwards compatibility, the getaddrinfo function still recognizes
legacy name syntax, such as 192.000.002.010 interpreted as 192.0.2.8
(octal).
This commit does not change the behavior of inet_addr and inet_aton.
gethostbyname already had additional sanity checks (but is switched
over to the new __inet_aton_exact function for completeness as well).
To avoid sending the problematic query names over DNS, commit
6ca53a2453 ("resolv: Do not send queries
for non-host-names in nss_dns [BZ #24112]") is needed.
Sometimes tst-nss-test3 fails with:
error: test-container.c:386: unable to open .../nss/libnss_test1.so for reading
The test tst-nss-test3 which runs in a container needs
libnss_test[12].so. (see e.g. tst-nss-test3.script).
Before this test was moved from tests to tests-container variable,
the requirement was met. Thus this patch adds this requirement
also for tests in tests-container.
ChangeLog:
* nss/Makefile (tst-nss-test3.out): New rule.
Increase timeout from the default 20s to 40s. This test makes close to
2 million syscalls with distribution:
1180249 connect
297952 getsockname
144040 lseek
143734 read
14466 close
...
connect can be slow, so the default timeout was not enough on slow
systems.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* nss/tst-nss-files-hosts-multi.c (TIMEOUT): Define.
If the caller-supplied buffer is not large enough, fall back directly
malloc.
The previous __libc_use_alloca check was incorrect because it did not
take into account that extend_alloca may fail to merge allocations, so
it would underestimate the stack space being used by roughly a factor
of two.
Continuing the fixes for linknamespace and localplt test failures with
-Os that arise from functions not being inlined in that case, this
patch fixes such failures for feof_unlocked.
The usual approach is followed of adding __feof_unlocked (inlined when
feof_unlocked is), making calls use it when required for namespace
reasons, and using libc_hidden_proto / libc_hidden_weak for the
feof_unlocked weak alias when only localplt but not namespace issues
are involved. In the case of getaddrinfo.c, use of __feof_unlocked
needs to be conditional since that code is also used in nscd (where
__feof_unlocked is not available).
Tested for x86_64 (both without -Os to make sure that case continues
to work, and with -Os to make sure all the relevant linknamespace and
localplt test failures are resolved). Because of other such failures
that remain after this patch, neither of the bugs can yet be closed.
[BZ #15105]
[BZ #19463]
* libio/feof_u.c (feof_unlocked): Rename to __feof_unlocked and
define as weak alias of __feof_unlocked. Use libc_hidden_weak.
* include/stdio.h (feof_unlocked): Use libc_hidden_proto.
(__feof_unlocked): New declaration, and inline function if
[__USE_EXTERN_INLINES].
* iconv/gconv_conf.c (read_conf_file): Call __feof_unlocked
instead of feof_unlocked.
* intl/localealias.c [_LIBC] (FEOF): Likewise.
* nss/nsswitch.c (nss_parse_file): Likewise.
* sysdeps/unix/sysv/linux/readonly-area.c (__readonly_area):
Likewise.
* time/getdate.c (__getdate_r): Likewise.
* sysdeps/posix/getaddrinfo.c [IS_IN (libc)] (feof_unlocked):
Define as macro to call __feof_unlocked.
The old code uses errno as the primary indicator for success or
failure. This is wrong because errno is only set for specific
combinations of the status return value and the h_errno variable.
Some programs have more than one source files. These non-lib modules
should not be compiled with -DMODULE_NAME=libc. This patch puts these
non-lib modules in $(others-extras) and adds $(others-extras) to
all-nonlib.
[BZ #21864]
* Makerules (all-nonlib): Add $(others-extras).
* catgets/Makefile (others-extras): New.
* elf/Makefile (others-extras): Likewise.
* nss/Makefile (others-extras): Likewise.
Mark internal nss symbols with attribute_hidden to allow direct access
within libc.so and libc.a without using GOT nor PLT.
Tested on x86-64 with and without --disable-nscd.
[BZ #18822]
* grp/initgroups.c (__nss_group_database): Removed.
(__nss_initgroups_database): Likewise.
* nscd/gai.c (__nss_hosts_database): Likewise.
* nss/XXX-lookup.c (DATABASE_NAME_SYMBOL): Likewise.
* posix/tst-rfc3484-2.c (__nss_hosts_database): Likewise.
* posix/tst-rfc3484-3.c (__nss_hosts_database): Likewise.
* posix/tst-rfc3484.c (__nss_hosts_database): Likewise.
* sysdeps/posix/getaddrinfo.c (__nss_hosts_database): Likewise.
* nss/getXXent.c (INTERNAL (REENTRANT_GETNAME)): Add
attribute_hidden.
* nss/nsswitch.c (__nss_database_custom): Define only if
USE_NSCD is defined.
(__nss_configure_lookup): Use __nss_database_custom only if
USE_NSCD is defined.
* nss/nsswitch.h (__nss_database_custom): Declare only if
USE_NSCD is defined. Add attribute_hidden.
(__nss_setent): Add attribute_hidden.
(__nss_endent): Likewise.
(__nss_getent_r): Likewise.
(__nss_getent): Likewise.
(DEFINE_DATABASE): Declare __nss_##arg##_database.
Commit 384ca55174 from 2007 added this to
nss/XXX-lookup.c:
+#ifndef NO_COMPAT
+int
+internal_function attribute_compat_text_section
+DB_COMPAT_FCT (service_user **ni, const char *fct_name, void **fctp)
+{
+ return DB_LOOKUP_FCT (ni, fct_name, NULL, fctp);
+}
+#endif
That is, it adds a pseudo-compat function with an internal_function
attribute. The function it was supposed to replace did not have the
attribute:
extern int DB_LOOKUP_FCT (service_user **ni, const char *fct_name,
- void **fctp) internal_function;
+ const char *fct2_name, void **fctp)
+ internal_function;
This changed the calling convention on i386 for the following
functions in the public ABI:
__nss_passwd_lookup
__nss_group_lookup
__nss_hosts_lookup
This commit replaces the functions with always-failing stubs,
with true compat symbols. Due to a happy accident, the calling
convention of the stub is identical for the internal_function
and non-internal_function case on i386.
In addition, this commit auto-generates the __nss_*_lookup2
function declarations as part of <nsswitch.h>.
struct resolv_context objects provide a temporary resolver context
which does not change during a name lookup operation. Only when the
outmost context is created, the stub resolver configuration is
verified to be current (at present, only against previous res_init
calls). Subsequent attempts to obtain the context will reuse the
result of the initial verification operation.
struct resolv_context can also be extended in the future to store
data which needs to be deallocated during thread cancellation.
Many callers of __res_maybe_init also call _res_hconf_init.
Additional calls to the latter do not hurt because the function
does its work only once. (/etc/hosts.conf is not reloaded or
even checked for changes.) This means that we can simplify the
code by calling _res_hconf_init directly from __res_vinit.
This patch adds a new build module called 'testsuite'.
IS_IN (testsuite) implies _ISOMAC, as do IS_IN_build and __cplusplus
(which means several ad-hoc tests for __cplusplus can go away).
libc-symbols.h now suppresses almost all of *itself* when _ISOMAC is
defined; in particular, _ISOMAC mode does not get config.h
automatically anymore.
There are still quite a few tests that need to see internal gunk of
one variety or another. For them, we now have 'tests-internal' and
'test-internal-extras'; files in this category will still be compiled
with MODULE_NAME=nonlib, and everything proceeds as it always has.
The bulk of this patch is moving tests from 'tests' to
'tests-internal'. There is also 'tests-static-internal', which has
the same effect on files in 'tests-static', and 'modules-names-tests',
which has the *inverse* effect on files in 'modules-names' (it's
inverted because most of the things in modules-names are *not* tests).
For both of these, the file must appear in *both* the new variable and
the old one.
There is also now a special case for when libc-symbols.h is included
without MODULE_NAME being defined at all. (This happens during the
creation of libc-modules.h, and also when preprocessing Versions
files.) When this happens, IS_IN is set to be always false and
_ISOMAC is *not* defined, which was the status quo, but now it's
explicit.
The remaining changes to C source files in this patch seemed likely to
cause problems in the absence of the main change. They should be
relatively self-explanatory. In a few cases I duplicated a definition
from an internal header rather than move the test to tests-internal;
this was a judgement call each time and I'm happy to change those
however reviewers feel is more appropriate.
* Makerules: New subdir configuration variables 'tests-internal'
and 'test-internal-extras'. Test files in these categories will
still be compiled with MODULE_NAME=nonlib. Test files in the
existing categories (tests, xtests, test-srcs, test-extras) are
now compiled with MODULE_NAME=testsuite.
New subdir configuration variable 'modules-names-tests'. Files
which are in both 'modules-names' and 'modules-names-tests' will
be compiled with MODULE_NAME=testsuite instead of
MODULE_NAME=extramodules.
(gen-as-const-headers): Move to tests-internal.
(do-tests-clean, common-mostlyclean): Support tests-internal.
* Makeconfig (built-modules): Add testsuite.
* Makefile: Change libof-check-installed-headers-c and
libof-check-installed-headers-cxx to 'testsuite'.
* Rules: Likewise. Support tests-internal.
* benchtests/strcoll-inputs/filelist#en_US.UTF-8:
Remove extra-modules.mk.
* config.h.in: Don't check for __OPTIMIZE__ or __FAST_MATH__ here.
* include/libc-symbols.h: Move definitions of _GNU_SOURCE,
PASTE_NAME, PASTE_NAME1, IN_MODULE, IS_IN, and IS_IN_LIB to the
very top of the file and rationalize their order.
If MODULE_NAME is not defined at all, define IS_IN to always be
false, and don't define _ISOMAC.
If any of IS_IN (testsuite), IS_IN_build, or __cplusplus are
true, define _ISOMAC and suppress everything else in this file,
starting with the inclusion of config.h.
Do check for inappropriate definitions of __OPTIMIZE__ and
__FAST_MATH__ here, but only if _ISOMAC is not defined.
Correct some out-of-date commentary.
* include/math.h: If _ISOMAC is defined, undefine NO_LONG_DOUBLE
and _Mlong_double_ before including math.h.
* include/string.h: If _ISOMAC is defined, don't expose
_STRING_ARCH_unaligned. Move a comment to a more appropriate
location.
* include/errno.h, include/stdio.h, include/stdlib.h, include/string.h
* include/time.h, include/unistd.h, include/wchar.h: No need to
check __cplusplus nor use __BEGIN_DECLS/__END_DECLS.
* misc/sys/cdefs.h (__NTHNL): New macro.
* sysdeps/m68k/m680x0/fpu/bits/mathinline.h
(__m81_defun): Use __NTHNL to avoid errors with GCC 6.
* elf/tst-env-setuid-tunables.c: Include config.h with _LIBC
defined, for HAVE_TUNABLES.
* inet/tst-checks-posix.c: No need to define _ISOMAC.
* intl/tst-gettext2.c: Provide own definition of N_.
* math/test-signgam-finite-c99.c: No need to define _ISOMAC.
* math/test-signgam-main.c: No need to define _ISOMAC.
* stdlib/tst-strtod.c: Convert to test-driver. Split locale_test to...
* stdlib/tst-strtod1i.c: ...this new file.
* stdlib/tst-strtod5.c: Convert to test-driver and add copyright notice.
Split tests of __strtod_internal to...
* stdlib/tst-strtod5i.c: ...this new file.
* string/test-string.h: Include stdint.h. Duplicate definition of
inhibit_loop_to_libcall here (from libc-symbols.h).
* string/test-strstr.c: Provide dummy definition of
libc_hidden_builtin_def when including strstr.c.
* sysdeps/ia64/fpu/libm-symbols.h: Suppress entire file in _ISOMAC
mode; no need to test __STRICT_ANSI__ nor __cplusplus as well.
* sysdeps/x86_64/fpu/math-tests-arch.h: Include cpu-features.h.
Don't include init-arch.h.
* sysdeps/x86_64/multiarch/test-multiarch.h: Include cpu-features.h.
Don't include init-arch.h.
* elf/Makefile: Move tst-ptrguard1-static, tst-stackguard1-static,
tst-tls1-static, tst-tls2-static, tst-tls3-static, loadtest,
unload, unload2, circleload1, neededtest, neededtest2,
neededtest3, neededtest4, tst-tls1, tst-tls2, tst-tls3,
tst-tls6, tst-tls7, tst-tls8, tst-dlmopen2, tst-ptrguard1,
tst-stackguard1, tst-_dl_addr_inside_object, and all of the
ifunc tests to tests-internal.
Don't add $(modules-names) to test-extras.
* inet/Makefile: Move tst-inet6_scopeid_pton to tests-internal.
Add tst-deadline to tests-static-internal.
* malloc/Makefile: Move tst-mallocstate and tst-scratch_buffer to
tests-internal.
* misc/Makefile: Move tst-atomic and tst-atomic-long to tests-internal.
* nptl/Makefile: Move tst-typesizes, tst-rwlock19, tst-sem11,
tst-sem12, tst-sem13, tst-barrier5, tst-signal7, tst-tls3,
tst-tls3-malloc, tst-tls5, tst-stackguard1, tst-sem11-static,
tst-sem12-static, and tst-stackguard1-static to tests-internal.
Link tests-internal with libpthread also.
Don't add $(modules-names) to test-extras.
* nss/Makefile: Move tst-field to tests-internal.
* posix/Makefile: Move bug-regex5, bug-regex20, bug-regex33,
tst-rfc3484, tst-rfc3484-2, and tst-rfc3484-3 to tests-internal.
* stdlib/Makefile: Move tst-strtod1i, tst-strtod3, tst-strtod4,
tst-strtod5i, tst-tls-atexit, and tst-tls-atexit-nodelete to
tests-internal.
* sunrpc/Makefile: Move tst-svc_register to tests-internal.
* sysdeps/powerpc/Makefile: Move test-get_hwcap and
test-get_hwcap-static to tests-internal.
* sysdeps/unix/sysv/linux/Makefile: Move tst-setgetname to
tests-internal.
* sysdeps/x86_64/fpu/Makefile: Add all libmvec test modules to
modules-names-tests.
These are a grab bag of changes where the testsuite was using internal
symbols of some variety, but this was straightforward to fix, and the
fixed code should work with or without the change to compile the
testsuite under _ISOMAC.
Four of these are just more #include adjustments, but I want to highlight
sysdeps/powerpc/fpu/tst-setcontext-fpscr.c, which appears to have been
written before the advent of sys/auxv.h. I think a big chunk of this file
could be replaced by a simple call to getauxval, but I'll let someone who
actually has a powerpc machine to test on do that.
dlfcn/tst-dladdr.c was including ldsodefs.h just so it could use
DL_LOOKUP_ADDRESS to print an additional diagnostic; as requested by Carlos,
I have removed this.
math/test-misc.c was using #ifndef NO_LONG_DOUBLE, which is an internal
configuration macro, to decide whether to do certain tests involving
'long double'. I changed the test to #if LDBL_MANT_DIG > DBL_MANT_DIG
instead, which uses only public float.h macros and is equivalent on
all supported platforms. (Note that NO_LONG_DOUBLE doesn't mean 'the
compiler doesn't support long double', it means 'long double is the
same as double'.)
tst-writev.c has a configuration macro 'ARTIFICIAL_LIMIT' that the
Makefiles are expected to define, and sysdeps/unix/sysv/linux/Makefile
was using the internal __getpagesize in the definition; changed to
sysconf(_SC_PAGESIZE) which is the POSIX equivalent.
ia64-linux doesn't supply 'clone', only '__clone2', which is not
defined in the public headers(!) All the other clone tests have local
extern declarations of __clone2, but tst-clone.c doesn't; it was
getting away with this because include/sched.h does declare __clone2.
* nss/tst-cancel-getpwuid_r.c: Include nss.h.
* string/strcasestr.c: No need to include config.h.
* sysdeps/powerpc/fpu/tst-setcontext-fpscr.c: Include
sys/auxv.h. Don't include sysdep.h.
* sysdeps/powerpc/tst-set_ppr.c: Don't include dl-procinfo.h.
* dlfcn/tst-dladdr.c: Don't include ldsodefs.h. Don't use
DL_LOOKUP_ADDRESS.
* math/test-misc.c: Instead of testing NO_LONG_DOUBLE, test whether
LDBL_MANT_DIG is greater than DBL_MANT_DIG.
* sysdeps/unix/sysv/linux/Makefile (CFLAGS-tst-writev.c): Use
sysconf (_SC_PAGESIZE) instead of __getpagesize in definition
of ARTIFICIAL_LIMIT.
* sysdeps/unix/sysv/linux/tst-clone.c [__ia64__]: Add extern
declaration of __clone2.
This patch increases timeouts on some tests I've observed timing out.
elf/tst-tls13 and iconvdata/tst-loading both dynamically load many
objects and so are slow when testing over NFS. They had timeouts set
from before the default changed from 2 to 20 seconds; this patch
removes those old settings, so effectively increasing the timeout to
20 seconds (from 3 and 10 seconds respectively).
malloc/tst-malloc-thread-fail.c and malloc/tst-mallocfork2.c are slow
on slow systems and so I set a fairly arbitrary 100 second timeout,
which seems to suffice on the system where I saw them timing out.
nss/tst-cancel-getpwuid_r.c and nss/tst-nss-getpwent.c are slow on
systems with a large passwd file; I set timeouts that empirically
worked for me. (It seems tst-cancel-getpwuid_r.c is hitting the
100000 getpwuid_r call limit in my testing, with each call taking a
bit over 0.007 seconds, so 700 seconds for the test.)
* elf/tst-tls13.c (TIMEOUT): Remove.
* iconvdata/tst-loading.c (TIMEOUT): Likewise.
* malloc/tst-malloc-thread-fail.c (TIMEOUT): Increase to 100.
* malloc/tst-mallocfork2.c (TIMEOUT): Define to 100.
* nss/tst-cancel-getpwuid_r.c (TIMEOUT): Define to 900.
* nss/tst-nss-getpwent.c (TIMEOUT): Define to 300.
The fix in commit 312be3f9f5 resolved
several cancellation issues in several APIs. This regression test is
designed to double check that at least getpwuid_r remainds correctly
implemented and does not provide additional unintended cancellation
points that may leave locks in an inconsistent state.
_res_hconf.initialized was not suitable for use in a multi-threaded
environment due to the lack of atomics and memory barriers. Use of it was
also unnecessary because _res_hconf_init did the right thing by using
__libc_once. This patch fixes the glibc-internal uses by just calling
_res_hconf_init unconditionally, and switches to a release MO atomic store
for _res_hconf.initialized to fix the glibc side of the synchronization
problem (which will maintain backward compatibility, but cannot fix the
lack of acquire MO on any glibc-external loads).
[BZ #20477]
* resolv/res_hconf.c (do_init): Use atomic access.
* resolv/res_hconf.h: Add comments.
* nscd/aicache.c (addhstaiX): Call _res_hconf_init unconditionally.
* nss/getXXbyYY_r.c (REENTRANT_NAME): Likewise.
* sysdeps/posix/getaddrinfo.c (gaih_inet): Likewise.
When get*ent is called without a preceding set*ent, we need
to set the initial iteration position in get*ent.
Reproducer: Add “services: db files” to /etc/nsswitch.conf, then run
“perl -e getservent”. It will segfault before this change, and exit
silently after it.
https://sourceware.org/glibc/wiki/Proposals/GroupMerging
== Justification ==
It is common today for users to rely on centrally-managed user stores for
handling their user accounts. However, much software existing today does
not have an innate understanding of such accounts. Instead, they commonly
rely on membership in known groups for managing access-control (for
example the "wheel" group on Fedora and RHEL systems or the "adm" group
on Debian-derived systems). In the present incarnation of nsswitch, the
only way to have such groups managed by a remote user store such as
FreeIPA or Active Directory would be to manually remove the groups from
/etc/group on the clients so that nsswitch would then move past nss_files
and into the SSSD, nss-ldap or other remote user database.
== Solution ==
With this patch, a new action is introduced for nsswitch:
NSS_ACTION_MERGE. To take advantage of it, one will add [SUCCESS=merge]
between two database entries in the nsswitch.conf file. When a group is
located in the first of the two group entries, processing will continue
on to the next one. If the group is also found in the next entry (and the
group name and GID are an exact match), the member list of the second
entry will be added to the group object to be returned.
== Implementation ==
After each DL_LOOKUP_FN() returns, the next action is checked. If the
function returned NSS_STATUS_SUCCESS and the next action is
NSS_ACTION_MERGE, a copy of the result buffer is saved for the next pass
through the loop. If on this next pass through the loop the database
returns another instance of a group matching both the group name and GID,
the member list is added to the previous list and it is returned as a
single object. If the following database does not contain the same group,
then the original is copied back into the destination buffer.
This patch implements merge functionality only for the group database.
For other databases, there is a default implementation that will return
the EINVAL errno if a merge is requested. The merge functionality can be
implemented for other databases at a later time if such is needed. Each
database must provide a unique implementation of the deep-copy and merge
functions.
If [SUCCESS=merge] is present in nsswitch.conf for a glibc version that
does not support it, glibc will process results up until that operation,
at which time it will return results if it has found them or else will
simply return an error. In practical terms, this ends up behaving like
the remainder of the nsswitch.conf line does not exist.
== Iterators ==
This feature does not modify the iterator functionality from its current
behavior. If getgrnam() or getgrgid() is called, glibc will iterate
through all entries in the `group` line in nsswitch.conf and display the
list of members without attempting to merge them. This is consistent with
the behavior of nss_files where if two separate lines are specified for
the same group in /etc/groups, getgrnam()/getgrgid() will display both.
Clients are already expected to handle this gracefully.
== No Premature Optimizations ==
The following is a list of places that might be eligible for
optimization, but were not overengineered for this initial contribution:
* Any situation where a merge may occur will result in one malloc() of
the same size as the input buffer.
* Any situation where a merge does occur will result in a second
malloc() to hold the list of pointers to member name strings.
* The list of members is simply concatenated together and is not tested
for uniqueness (which is identical to the behavior for nss_files,
which will simply return identical values if they both exist on the
line in the file. This could potentially be optimized to reduce space
usage in the buffer, but it is both complex and computationally
expensive to do so.
== Testing ==
I performed testing by running the getent utility against my newly-built
glibc and configuring /etc/nsswitch.conf with the following entry:
group: group: files [SUCCESS=merge] sss
In /etc/group I included the line:
wheel❌10:sgallagh
I then configured my local SSSD using the id_provider=local to respond
with:
wheel:*:10:localuser,localuser2
I then ran `getent group wheel` against the newly-built glibc in
multiple situations and received the expected output as described
above:
* When SSSD was running.
* When SSSD was configured in nsswitch.conf but the daemon was not
running.
* When SSSD was configured in nsswitch.conf but nss_sss.so.2 was not
installed on the system.
* When the order of 'sss' and 'files' was reversed.
* All of the above with the [SUCCESS=merge] removed (to ensure no
regressions).
* All of the above with `getent group 10`.
* All of the above with `getent group` with and without
`enumerate=true` set in SSSD.
* All of the above with and without nscd enabled on the system.
Reproducer (needs to run as root):
perl -e \
'print "large❌999:" . join(",", map {"user$_"} (1 .. 135))."\n"' \
>> /etc/group
cd /var/db
make
getent -s db group
After the fix, the last command should list the "large" group.
The magic number 135 has been chosen so that the line is shorter than
1024 bytes, but the pointers required to encode the member array will
cross the threshold, triggering the bug.
This mostly automatically-generated patch converts 113 function
definitions in glibc from old-style K&R to prototype-style. Following
my other recent such patches, this one deals with the case of function
definitions in files that either contain assertions or where grep
suggested they might contain assertions - and thus where it isn't
possible to use a simple object code comparison as a sanity check on
the correctness of the patch, because line numbers are changed.
A few such automatically-generated changes needed to be supplemented
by manual changes for the result to compile. openat64 had a prototype
declaration with "..." but an old-style definition in
sysdeps/unix/sysv/linux/dl-openat64.c, and "..." needed adding to the
generated prototype in the definition (I've filed
<https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68024> for diagnosing
such cases in GCC; the old state was undefined behavior not requiring
a diagnostic, but one seems a good idea). In addition, as Florian has
noted regparm attribute mismatches between declaration and definition
are only diagnosed for prototype definitions, and five functions
needed internal_function added to their definitions (in the case of
__pthread_mutex_cond_lock, via the macro definition of
__pthread_mutex_lock) to compile on i386.
After this patch is in, remaining old-style definitions are probably
most readily fixed manually before we can turn on
-Wold-style-definition for all builds.
Tested for x86_64 and x86 (testsuite).
* crypt/md5-crypt.c (__md5_crypt_r): Convert to prototype-style
function definition.
* crypt/sha256-crypt.c (__sha256_crypt_r): Likewise.
* crypt/sha512-crypt.c (__sha512_crypt_r): Likewise.
* debug/backtracesyms.c (__backtrace_symbols): Likewise.
* elf/dl-minimal.c (_itoa): Likewise.
* hurd/hurdmalloc.c (malloc): Likewise.
(free): Likewise.
(realloc): Likewise.
* inet/inet6_option.c (inet6_option_space): Likewise.
(inet6_option_init): Likewise.
(inet6_option_append): Likewise.
(inet6_option_alloc): Likewise.
(inet6_option_next): Likewise.
(inet6_option_find): Likewise.
* io/ftw.c (FTW_NAME): Likewise.
(NFTW_NAME): Likewise.
(NFTW_NEW_NAME): Likewise.
(NFTW_OLD_NAME): Likewise.
* libio/iofwide.c (_IO_fwide): Likewise.
* libio/strops.c (_IO_str_init_static_internal): Likewise.
(_IO_str_init_static): Likewise.
(_IO_str_init_readonly): Likewise.
(_IO_str_overflow): Likewise.
(_IO_str_underflow): Likewise.
(_IO_str_count): Likewise.
(_IO_str_seekoff): Likewise.
(_IO_str_pbackfail): Likewise.
(_IO_str_finish): Likewise.
* libio/wstrops.c (_IO_wstr_init_static): Likewise.
(_IO_wstr_overflow): Likewise.
(_IO_wstr_underflow): Likewise.
(_IO_wstr_count): Likewise.
(_IO_wstr_seekoff): Likewise.
(_IO_wstr_pbackfail): Likewise.
(_IO_wstr_finish): Likewise.
* locale/programs/localedef.c (normalize_codeset): Likewise.
* locale/programs/locarchive.c (add_locale_to_archive): Likewise.
(add_locales_to_archive): Likewise.
(delete_locales_from_archive): Likewise.
* malloc/malloc.c (__libc_mallinfo): Likewise.
* math/gen-auto-libm-tests.c (init_fp_formats): Likewise.
* misc/tsearch.c (__tfind): Likewise.
* nptl/pthread_attr_destroy.c (__pthread_attr_destroy): Likewise.
* nptl/pthread_attr_getdetachstate.c
(__pthread_attr_getdetachstate): Likewise.
* nptl/pthread_attr_getguardsize.c (pthread_attr_getguardsize):
Likewise.
* nptl/pthread_attr_getinheritsched.c
(__pthread_attr_getinheritsched): Likewise.
* nptl/pthread_attr_getschedparam.c
(__pthread_attr_getschedparam): Likewise.
* nptl/pthread_attr_getschedpolicy.c
(__pthread_attr_getschedpolicy): Likewise.
* nptl/pthread_attr_getscope.c (__pthread_attr_getscope):
Likewise.
* nptl/pthread_attr_getstack.c (__pthread_attr_getstack):
Likewise.
* nptl/pthread_attr_getstackaddr.c (__pthread_attr_getstackaddr):
Likewise.
* nptl/pthread_attr_getstacksize.c (__pthread_attr_getstacksize):
Likewise.
* nptl/pthread_attr_init.c (__pthread_attr_init_2_1): Likewise.
(__pthread_attr_init_2_0): Likewise.
* nptl/pthread_attr_setdetachstate.c
(__pthread_attr_setdetachstate): Likewise.
* nptl/pthread_attr_setguardsize.c (pthread_attr_setguardsize):
Likewise.
* nptl/pthread_attr_setinheritsched.c
(__pthread_attr_setinheritsched): Likewise.
* nptl/pthread_attr_setschedparam.c
(__pthread_attr_setschedparam): Likewise.
* nptl/pthread_attr_setschedpolicy.c
(__pthread_attr_setschedpolicy): Likewise.
* nptl/pthread_attr_setscope.c (__pthread_attr_setscope):
Likewise.
* nptl/pthread_attr_setstack.c (__pthread_attr_setstack):
Likewise.
* nptl/pthread_attr_setstackaddr.c (__pthread_attr_setstackaddr):
Likewise.
* nptl/pthread_attr_setstacksize.c (__pthread_attr_setstacksize):
Likewise.
* nptl/pthread_condattr_setclock.c (pthread_condattr_setclock):
Likewise.
* nptl/pthread_create.c (__find_in_stack_list): Likewise.
* nptl/pthread_getattr_np.c (pthread_getattr_np): Likewise.
* nptl/pthread_mutex_cond_lock.c (__pthread_mutex_lock): Define to
use internal_function.
* nptl/pthread_mutex_init.c (__pthread_mutex_init): Convert to
prototype-style function definition.
* nptl/pthread_mutex_lock.c (__pthread_mutex_lock): Likewise.
(__pthread_mutex_cond_lock_adjust): Likewise. Use
internal_function.
* nptl/pthread_mutex_timedlock.c (pthread_mutex_timedlock):
Convert to prototype-style function definition.
* nptl/pthread_mutex_trylock.c (__pthread_mutex_trylock):
Likewise.
* nptl/pthread_mutex_unlock.c (__pthread_mutex_unlock_usercnt):
Likewise.
(__pthread_mutex_unlock): Likewise.
* nptl_db/td_ta_clear_event.c (td_ta_clear_event): Likewise.
* nptl_db/td_ta_set_event.c (td_ta_set_event): Likewise.
* nptl_db/td_thr_clear_event.c (td_thr_clear_event): Likewise.
* nptl_db/td_thr_event_enable.c (td_thr_event_enable): Likewise.
* nptl_db/td_thr_set_event.c (td_thr_set_event): Likewise.
* nss/makedb.c (process_input): Likewise.
* posix/fnmatch.c (__strchrnul): Likewise.
(__wcschrnul): Likewise.
(fnmatch): Likewise.
* posix/fnmatch_loop.c (FCT): Likewise.
* posix/glob.c (globfree): Likewise.
(__glob_pattern_type): Likewise.
(__glob_pattern_p): Likewise.
* posix/regcomp.c (re_compile_pattern): Likewise.
(re_set_syntax): Likewise.
(re_compile_fastmap): Likewise.
(regcomp): Likewise.
(regerror): Likewise.
(regfree): Likewise.
* posix/regexec.c (regexec): Likewise.
(re_match): Likewise.
(re_search): Likewise.
(re_match_2): Likewise.
(re_search_2): Likewise.
(re_search_stub): Likewise. Use internal_function
(re_copy_regs): Likewise.
(re_set_registers): Convert to prototype-style function
definition.
(prune_impossible_nodes): Likewise. Use internal_function.
* resolv/inet_net_pton.c (inet_net_pton): Convert to
prototype-style function definition.
(inet_net_pton_ipv4): Likewise.
* stdlib/strtod_l.c (____STRTOF_INTERNAL): Likewise.
* sysdeps/pthread/aio_cancel.c (aio_cancel): Likewise.
* sysdeps/pthread/aio_suspend.c (aio_suspend): Likewise.
* sysdeps/pthread/timer_delete.c (timer_delete): Likewise.
* sysdeps/unix/sysv/linux/dl-openat64.c (openat64): Likewise.
Make variadic.
* time/strptime_l.c (localtime_r): Convert to prototype-style
function definition.
* wcsmbs/mbsnrtowcs.c (__mbsnrtowcs): Likewise.
* wcsmbs/mbsrtowcs_l.c (__mbsrtowcs_l): Likewise.
* wcsmbs/wcsnrtombs.c (__wcsnrtombs): Likewise.
* wcsmbs/wcsrtombs.c (__wcsrtombs): Likewise.
The recent put*ent hardening changes broke the build for i386. i386
defines internal_function to __attribute__ ((regparm (3), stdcall)),
which affects type compatibility, so requiring internal_function to be
used consistently on declarations and definitions. This patch adds
internal_function to the definitions of the new functions using it on
their declarations.
Tested for i386 that this fixes the build.
* nss/rewrite_field.c (__nss_rewrite_field): Use
internal_function.
* nss/valid_field.c (__nss_valid_field): Likewise.
* nss/valid_list_field.c (__nss_valid_list_field): Likewise.
This prevents injection of ':' and '\n' into output functions which
use the NSS files database syntax. Critical fields (user/group names
and file system paths) are checked strictly. For backwards
compatibility, the GECOS field is rewritten instead.
The getent program is adjusted to use the put*ent functions in libc,
instead of local copies. This changes the behavior of getent if user
names start with '-' or '+'.
It used to be common practice to have a statically linked shell for an
alternative root account, as in:
root❌0:0:root:/root:/bin/bash
toor❌0:0:root recovery account:/root:/sbin/sash
This causes problems with passwd NSS tests because a UID-based lookup
will only retrieve one of those entries. The original version of
nss/bug17079.c detected this, but failed to use this information later
on.
These scripts use #!/bin/sh explicitly, so make sure they avoid echo -n
as different shells treat it differently. Use the portable printf func
instead.
Robin Hack discovered Samba would enter an infinite loop processing
certain quota-related requests. We eventually tracked this down to a
glibc issue.
Running a (simplified) test case under strace shows that /etc/passwd
is continuously opened and closed:
…
open("/etc/passwd", O_RDONLY|O_CLOEXEC) = 3
lseek(3, 0, SEEK_CUR) = 0
read(3, "root❌0:0:root:/root:/bin/bash\n"..., 4096) = 2717
lseek(3, 2717, SEEK_SET) = 2717
close(3) = 0
open("/etc/passwd", O_RDONLY|O_CLOEXEC) = 3
lseek(3, 0, SEEK_CUR) = 0
lseek(3, 0, SEEK_SET) = 0
read(3, "root❌0:0:root:/root:/bin/bash\n"..., 4096) = 2717
lseek(3, 2717, SEEK_SET) = 2717
close(3) = 0
open("/etc/passwd", O_RDONLY|O_CLOEXEC) = 3
lseek(3, 0, SEEK_CUR) = 0
…
The lookup function implementation in
nss/nss_files/files-XXX.c:DB_LOOKUP has code to prevent that. It is
supposed skip closing the input file if it was already open.
/* Reset file pointer to beginning or open file. */ \
status = internal_setent (keep_stream); \
\
if (status == NSS_STATUS_SUCCESS) \
{ \
/* Tell getent function that we have repositioned the file pointer. */ \
last_use = getby; \
\
while ((status = internal_getent (result, buffer, buflen, errnop \
H_ERRNO_ARG EXTRA_ARGS_VALUE)) \
== NSS_STATUS_SUCCESS) \
{ break_if_match } \
\
if (! keep_stream) \
internal_endent (); \
} \
keep_stream is initialized from the stayopen flag in internal_setent.
internal_setent is called from the set*ent implementation as:
status = internal_setent (stayopen);
However, for non-host database, this flag is always 0, per the
STAYOPEN magic in nss/getXXent_r.c.
Thus, the fix is this:
- status = internal_setent (stayopen);
+ status = internal_setent (1);
This is not a behavioral change even for the hosts database (where the
application can specify the stayopen flag) because with a call to
sethostent(0), the file handle is still not closed in the
implementation of gethostent.
In bug 14906 the user complains that the inotify support in nscd
is not sufficient when it comes to detecting changes in the
configurationfiles that should be watched for the various databases.
The current nscd implementation uses inotify to watch for changes in
the configuration files, but adds watches only for IN_DELETE_SELF and
IN_MODIFY. These watches are insufficient to cover even the most basic
uses by a system administrator. For example using emacs or vim to edit
a configuration file should trigger a reload but it might not if
the editors use move to atomically update the file. This atomic update
changes the inode and thus removes the notification on the file (as
inotify is based on inodes). Thus the inotify support in nscd for
configuration files is insufficient to account for the average use
cases of system administrators and users.
The inotify support is significantly enhanced and described here:
https://www.sourceware.org/ml/libc-alpha/2015-02/msg00504.html
Tested on x86_64 with and without inotify support.
Testing for 32-bit x86 shows up a warning "tst-nss-test1.c:25:3:
warning: format '%ju' expects argument of type 'uintmax_t', but
argument 2 has type 'int' [-Wformat=]". The argument is a difference
of two pointers, a signed quantity of type ptrdiff_t for which the
right format is %td; this patch makes this test use that format.
Tested for 32-bit x86.
* nss/tst-nss-test1.c (do_test): Use %td printf format for pointer
difference, not %ju.
Replace it with IS_IN (libc) and remove the one place that it
is defined in. The generated code remains unchanged on x86_64.
* include/shlib-compat.h [!NOT_IN_libc]: Remove.
* nss/nss_files/files-parse.c (IS_IN_libc): Replace with
IS_IN (libc).
nss_db uses nss_files code for services, but a continue on protocol
mismatch that doesn't affect nss_files skipped the code that advanced
to the next db entry. Any one of these changes would suffice to fix
it, but fixing both makes them both safer to reuse elsewhere.
for ChangeLog
[BZ #14498]
* NEWS: Fixed.
* nss/nss_db/db-XXX.c (_nss_db_get##name##_r): Update hidx
after parsing line but before break_if_match.
* nss/nss_files/files-service (DB_LOOKUP): Don't "continue;"
if there is a protocol mismatch.
The current scheme to identify which module a translation unit is
built in depends on defining multiple macros IS_IN_* and also defining
NOT_IN_libc if we're building a non-libc module. In addition, there
is an IN_LIB macro that does effectively the same thing, but for
different modules (notably the systemtap probes). This macro scheme
unifies both ideas to use just one macro IN_MODULE and assign it a
value depending on the module it is being built into. If the module
is not defined, it defaults to MODULE_libc.
Patches that follow will replace uses of IS_IN_* variables with the
IS_IN() macro. libc-symbols.h has been converted already to give an
example of how such a transition will look.
Verified that there are no relevant binary changes. One source change
that will crop up repeatedly is that of nscd_stat, since it uses the
build timestamp as a constant in its logic.
* Makeconfig (in-module): Get value of libof set for the
translation unit.
(CPPFLAGS): Use $(in-module).
* Makerules: Don't suffix routine names for nonlib.
* include/libc-modules.h: New file.
* include/libc-symbols.h: Include libc-modules.h
(IS_IN): New macro to replace IS_IN_* macros.
* elf/Makefile: Set libof-* for each routine.
* elf/rtld-Rules: Likewise.
* extra-modules.mk: Likewise.
* iconv/Makefile: Likewise.
* iconvdata/Makefile: Likewise.
* locale/Makefile: Likewise.
* malloc/Makefile: Likewise.
* nss/Makefile: Likewise.
* sysdeps/gnu/Makefile: Likewise.
* sysdeps/ieee754/ldbl-opt/Makefile: Likewise.
* sysdeps/unix/sysv/linux/Makefile: Likewise.
* sysdeps/s390/s390-64/Makefile: Likewise.
* nscd/Makefile: Set libof-* for each routine. Set CFLAGS and
CPPFLAGS for nscd instead of nonlib.
The netgroups nss modules in the glibc tree use NSS_STATUS_UNAVAIL
(with errno as ERANGE) when the supplied buffer does not have
sufficient space for the result. This is wrong, because the canonical
way to indicate insufficient buffer is to set the errno to ERANGE and
the status to NSS_STATUS_TRYAGAIN, as is used by all other modules.
This fixes nscd behaviour when the nss_ldap module returns
NSS_STATUS_TRYAGAIN to indicate that a netgroup entry is too long to
fit into the supplied buffer.
