Commit Graph

708 Commits

Author SHA1 Message Date
Wladimir J. van der Laan
622222846a Call the right helper function when setting mallopt M_ARENA_MAX (BZ #21338)
Fixes a typo introduced in commit
be7991c070. This caused
mallopt(M_ARENA_MAX) as well as the environment variable
MALLOC_ARENA_MAX to not work as intended because it set the
wrong internal parameter.

 	[BZ #21338]
	* malloc/malloc.c: Call do_set_arena_max for M_ARENA_MAX
	instead of incorrect do_set_arena_test
2017-04-01 12:39:10 +05:30
Stefan Liebler
e4e26210c3 Fix failing test malloc/tst-interpose-nothread with GCC 7.
The test malloc/tst-interpose-nothread fails on s390x if built
with GCC 7 and glibc commit "Remove the str(n)dup inlines
from string/bits/string2.h. Although inlining"
(ae65d4f3c3) with output:
error: free: 0x3fffdffa010: invalid allocation index: 0 (not less than 0)

The destructor check_for_allocations in malloc/tst-interpose-aux.c is
called twice.  One time after the test-child-process has finished successfully
and once after the test-parent-process finishes.
During the latter invocation, allocation_index == 0.  GCC 7 is now inlining the
free function and calls unconditionally fail in get_header as
header->allocation_index (type == size_t) is always >= allocation_index (= 0).
Before the mentioned commit above, strdup was replaced by strlen, malloc and
memcpy.  The malloc call was also inlined and allocation_index was set to one.

This patch moves the already existing compiler barrier before the invocation
of free.

ChangeLog:

	* malloc/tst-interpose-aux.c (check_for_allocations):
	Move compiler barrier before free.
2017-03-21 16:41:56 +01:00
DJ Delorie
17f487b7af Further harden glibc malloc metadata against 1-byte overflows.
Additional check for chunk_size == next->prev->chunk_size in unlink()

2017-03-17  Chris Evans  <scarybeasts@gmail.com>

	* malloc/malloc.c (unlink): Add consistency check between size and
	next->prev->size, to further harden against 1-byte overflows.
2017-03-17 15:31:38 -04:00
Zack Weinberg
9090848d06 Narrowing the visibility of libc-internal.h even further.
posix/wordexp-test.c used libc-internal.h for PTR_ALIGN_DOWN; similar
to what was done with libc-diag.h, I have split the definitions of
cast_to_integer, ALIGN_UP, ALIGN_DOWN, PTR_ALIGN_UP, and PTR_ALIGN_DOWN
to a new header, libc-pointer-arith.h.

It then occurred to me that the remaining declarations in libc-internal.h
are mostly to do with early initialization, and probably most of the
files including it, even in the core code, don't need it anymore.  Indeed,
only 19 files actually need what remains of libc-internal.h.  23 others
need libc-diag.h instead, and 12 need libc-pointer-arith.h instead.
No file needs more than one of them, and 16 don't need any of them!

So, with this patch, libc-internal.h stops including libc-diag.h as
well as losing the pointer arithmetic macros, and all including files
are adjusted.

        * include/libc-pointer-arith.h: New file.  Define
	cast_to_integer, ALIGN_UP, ALIGN_DOWN, PTR_ALIGN_UP, and
        PTR_ALIGN_DOWN here.
        * include/libc-internal.h: Definitions of above macros
	moved from here.  Don't include libc-diag.h anymore either.
	* posix/wordexp-test.c: Include stdint.h and libc-pointer-arith.h.
        Don't include libc-internal.h.

	* debug/pcprofile.c, elf/dl-tunables.c, elf/soinit.c, io/openat.c
	* io/openat64.c, misc/ptrace.c, nptl/pthread_clock_gettime.c
	* nptl/pthread_clock_settime.c, nptl/pthread_cond_common.c
	* string/strcoll_l.c, sysdeps/nacl/brk.c
	* sysdeps/unix/clock_settime.c
	* sysdeps/unix/sysv/linux/i386/get_clockfreq.c
	* sysdeps/unix/sysv/linux/ia64/get_clockfreq.c
	* sysdeps/unix/sysv/linux/powerpc/get_clockfreq.c
	* sysdeps/unix/sysv/linux/sparc/sparc64/get_clockfreq.c:
	Don't include libc-internal.h.

	* elf/get-dynamic-info.h, iconv/loop.c
	* iconvdata/iso-2022-cn-ext.c, locale/weight.h, locale/weightwc.h
	* misc/reboot.c, nis/nis_table.c, nptl_db/thread_dbP.h
	* nscd/connections.c, resolv/res_send.c, soft-fp/fmadf4.c
	* soft-fp/fmasf4.c, soft-fp/fmatf4.c, stdio-common/vfscanf.c
	* sysdeps/ieee754/dbl-64/e_lgamma_r.c
	* sysdeps/ieee754/dbl-64/k_rem_pio2.c
	* sysdeps/ieee754/flt-32/e_lgammaf_r.c
	* sysdeps/ieee754/flt-32/k_rem_pio2f.c
	* sysdeps/ieee754/ldbl-128/k_tanl.c
	* sysdeps/ieee754/ldbl-128ibm/k_tanl.c
	* sysdeps/ieee754/ldbl-96/e_lgammal_r.c
	* sysdeps/ieee754/ldbl-96/k_tanl.c, sysdeps/nptl/futex-internal.h:
	Include libc-diag.h instead of libc-internal.h.

        * elf/dl-load.c, elf/dl-reloc.c, locale/programs/locarchive.c
        * nptl/nptl-init.c, string/strcspn.c, string/strspn.c
	* malloc/malloc.c, sysdeps/i386/nptl/tls.h
	* sysdeps/nacl/dl-map-segments.h, sysdeps/x86_64/atomic-machine.h
	* sysdeps/unix/sysv/linux/spawni.c
        * sysdeps/x86_64/nptl/tls.h:
        Include libc-pointer-arith.h instead of libc-internal.h.

	* elf/get-dynamic-info.h, sysdeps/nacl/dl-map-segments.h
	* sysdeps/x86_64/atomic-machine.h:
        Add multiple include guard.
2017-03-01 20:33:46 -05:00
Zack Weinberg
e15f7de60c Split DIAG_* macros to new header libc-diag.h.
Quite a few tests include libc-internal.h just for the DIAG_* macros.
Split those macros to their own file, which can be included safely in
_ISOMAC mode.  I also moved ignore_value, since it seems logically
related, even though I didn't notice any tests needing it.

Also add -Wnonnull suppressions to two tests that _should_ have them,
but the error is masked when compiling against internal headers.

	* include/libc-diag.h: New file.  Define ignore_value,
	DIAG_PUSH_NEEDS_COMMENT, DIAG_POP_NEEDS_COMMENT,
	DIAG_IGNORE_NEEDS_COMMENT, and DIAG_IGNORE_Os_NEEDS_COMMENT here.

	* include/libc-internal.h: Definitions of above macros moved from
	here.  Include libc-diag.h.  Add copyright notice.

	* malloc/tst-malloc.c, malloc/tst-memcheck.c, malloc/tst-realloc.c
	* misc/tst-error1.c, posix/tst-dir.c, stdio-common/bug21.c
	* stdio-common/scanf14.c, stdio-common/scanf4.c, stdio-common/scanf7.c
	* stdio-common/test-vfprintf.c, stdio-common/tst-printf.c
	* stdio-common/tst-printfsz.c, stdio-common/tst-sprintf.c
	* stdio-common/tst-unlockedio.c, stdio-common/tstdiomisc.c
	* stdlib/bug-getcontext.c, string/tester.c, string/tst-endian.c
	* time/tst-strptime2.c, wcsmbs/tst-wcstof.c:
	Include libc-diag.h instead of libc-internal.h.

	* stdlib/tst-environ.c: Include libc-diag.h.  Suppress -Wnonnull for
	call to unsetenv (NULL).
	* nptl/tst-mutex1.c: Include libc-diag.h.  Suppress -Wnonnull for
	call to pthread_mutexattr_destroy (NULL).
2017-02-25 09:59:46 -05:00
Zack Weinberg
ceaa98897c Add missing header files throughout the testsuite.
* crypt/md5.h: Test _LIBC with #if defined, not #if.
	* dirent/opendir-tst1.c: Include sys/stat.h.
	* dirent/tst-fdopendir.c: Include sys/stat.h.
	* dirent/tst-fdopendir2.c: Include stdlib.h.
	* dirent/tst-scandir.c: Include stdbool.h.
	* elf/tst-auditmod1.c: Include link.h and stddef.h.
	* elf/tst-tls15.c: Include stdlib.h.
	* elf/tst-tls16.c: Include stdlib.h.
	* elf/tst-tls17.c: Include stdlib.h.
	* elf/tst-tls18.c: Include stdlib.h.
	* iconv/tst-iconv6.c: Include endian.h.
	* iconvdata/bug-iconv11.c: Include limits.h.
	* io/test-utime.c: Include stdint.h.
	* io/tst-faccessat.c: Include sys/stat.h.
	* io/tst-fchmodat.c: Include sys/stat.h.
	* io/tst-fchownat.c: Include sys/stat.h.
	* io/tst-fstatat.c: Include sys/stat.h.
	* io/tst-futimesat.c: Include sys/stat.h.
	* io/tst-linkat.c: Include sys/stat.h.
	* io/tst-mkdirat.c: Include sys/stat.h and stdbool.h.
	* io/tst-mkfifoat.c: Include sys/stat.h and stdbool.h.
	* io/tst-mknodat.c: Include sys/stat.h and stdbool.h.
	* io/tst-openat.c: Include stdbool.h.
	* io/tst-readlinkat.c: Include sys/stat.h.
	* io/tst-renameat.c: Include sys/stat.h.
	* io/tst-symlinkat.c: Include sys/stat.h.
	* io/tst-unlinkat.c: Include stdbool.h.
	* libio/bug-memstream1.c: Include stdlib.h.
	* libio/bug-wmemstream1.c: Include stdlib.h.
	* libio/tst-fwrite-error.c: Include stdlib.h.
	* libio/tst-memstream1.c: Include stdlib.h.
	* libio/tst-memstream2.c: Include stdlib.h.
	* libio/tst-memstream3.c: Include stdlib.h.
	* malloc/tst-interpose-aux.c: Include stdint.h.
	* misc/tst-preadvwritev-common.c: Include sys/stat.h.
	* nptl/tst-basic7.c: Include limits.h.
	* nptl/tst-cancel25.c: Include pthread.h, not pthreadP.h.
	* nptl/tst-cancel4.c: Include stddef.h, limits.h, and sys/stat.h.
	* nptl/tst-cancel4_1.c: Include stddef.h.
	* nptl/tst-cancel4_2.c: Include stddef.h.
	* nptl/tst-cond16.c: Include limits.h.
	Use sysconf(_SC_PAGESIZE) instead of __getpagesize.
	* nptl/tst-cond18.c: Include limits.h.
	Use sysconf(_SC_PAGESIZE) instead of __getpagesize.
	* nptl/tst-cond4.c: Include stdint.h.
	* nptl/tst-cond6.c: Include stdint.h.
	* nptl/tst-stack2.c: Include limits.h.
	* nptl/tst-stackguard1.c: Include stddef.h.
	* nptl/tst-tls4.c: Include stdint.h. Don't include tls.h.
	* nptl/tst-tls4moda.c: Include stddef.h.
	Don't include stdio.h, unistd.h, or tls.h.
	* nptl/tst-tls4modb.c: Include stddef.h.
	Don't include stdio.h, unistd.h, or tls.h.
	* nptl/tst-tls5.h: Include stddef.h. Don't include stdlib.h or tls.h.
	* posix/tst-getaddrinfo2.c: Include stdio.h.
	* posix/tst-getaddrinfo5.c: Include stdio.h.
	* posix/tst-pathconf.c: Include sys/stat.h.
	* posix/tst-posix_fadvise-common.c: Include stdint.h.
	* posix/tst-preadwrite-common.c: Include sys/stat.h.
	* posix/tst-regex.c: Include stdint.h.
	Don't include spawn.h or spawn_int.h.
	* posix/tst-regexloc.c: Don't include spawn.h or spawn_int.h.
	* posix/tst-vfork3.c: Include sys/stat.h.
	* resolv/tst-bug18665-tcp.c: Include stdlib.h.
	* resolv/tst-res_hconf_reorder.c: Include stdlib.h.
	* resolv/tst-resolv-search.c: Include stdlib.h.
	* stdio-common/tst-fmemopen2.c: Include stdint.h.
	* stdio-common/tst-vfprintf-width-prec.c: Include stdlib.h.
	* stdlib/test-canon.c: Include sys/stat.h.
	* stdlib/tst-tls-atexit.c: Include stdbool.h.
	* string/test-memchr.c: Include stdint.h.
	* string/tst-cmp.c: Include stdint.h.
	* sysdeps/pthread/tst-timer.c: Include stdint.h.
	* sysdeps/unix/sysv/linux/tst-sync_file_range.c: Include stdint.h.
	* sysdeps/wordsize-64/tst-writev.c: Include limits.h and stdint.h.
	* sysdeps/x86_64/fpu/math-tests-arch.h: Include cpu-features.h.
	Don't include init-arch.h.
	* sysdeps/x86_64/multiarch/test-multiarch.h: Include cpu-features.h.
	Don't include init-arch.h.
	* sysdeps/x86_64/tst-auditmod10b.c: Include link.h and stddef.h.
	* sysdeps/x86_64/tst-auditmod3b.c: Include link.h and stddef.h.
	* sysdeps/x86_64/tst-auditmod4b.c: Include link.h and stddef.h.
	* sysdeps/x86_64/tst-auditmod5b.c: Include link.h and stddef.h.
	* sysdeps/x86_64/tst-auditmod6b.c: Include link.h and stddef.h.
	* sysdeps/x86_64/tst-auditmod6c.c: Include link.h and stddef.h.
	* sysdeps/x86_64/tst-auditmod7b.c: Include link.h and stddef.h.
	* time/clocktest.c: Include stdint.h.
	* time/tst-posixtz.c: Include stdint.h.
	* timezone/tst-timezone.c: Include stdint.h.
2017-02-16 17:33:18 -05:00
Siddhesh Poyarekar
8cbc826c37 Fix getting tunable values on big-endian (BZ #21109)
The code to set value passed a tunable_val_t, which when cast to
int32_t on big-endian gives the wrong value.  Instead, use
tunable_val_t.numval instead, which can then be safely cast into
int32_t.
2017-02-08 14:17:17 +05:30
Joseph Myers
983a9637f7 Increase some test timeouts.
This patch increases timeouts on some tests I've observed timing out.

elf/tst-tls13 and iconvdata/tst-loading both dynamically load many
objects and so are slow when testing over NFS.  They had timeouts set
from before the default changed from 2 to 20 seconds; this patch
removes those old settings, so effectively increasing the timeout to
20 seconds (from 3 and 10 seconds respectively).

malloc/tst-malloc-thread-fail.c and malloc/tst-mallocfork2.c are slow
on slow systems and so I set a fairly arbitrary 100 second timeout,
which seems to suffice on the system where I saw them timing out.

nss/tst-cancel-getpwuid_r.c and nss/tst-nss-getpwent.c are slow on
systems with a large passwd file; I set timeouts that empirically
worked for me.  (It seems tst-cancel-getpwuid_r.c is hitting the
100000 getpwuid_r call limit in my testing, with each call taking a
bit over 0.007 seconds, so 700 seconds for the test.)

	* elf/tst-tls13.c (TIMEOUT): Remove.
	* iconvdata/tst-loading.c (TIMEOUT): Likewise.
	* malloc/tst-malloc-thread-fail.c (TIMEOUT): Increase to 100.
	* malloc/tst-mallocfork2.c (TIMEOUT): Define to 100.
	* nss/tst-cancel-getpwuid_r.c (TIMEOUT): Define to 900.
	* nss/tst-nss-getpwent.c (TIMEOUT): Define to 300.
2017-01-05 17:39:38 +00:00
Joseph Myers
3d7229c250 Fix malloc/ tests for GCC 7 -Walloc-size-larger-than=.
GCC 7 has a -Walloc-size-larger-than= warning for allocations of half
the address space or more.  This causes errors building glibc tests
that deliberately test failure of very large allocations.  This patch
arranges for this warning to be ignored around the problematic
function calls.

Tested compilation for aarch64 (GCC mainline) with
build-many-glibcs.py; did execution testing for x86_64 (GCC 5).

	* malloc/tst-malloc.c: Include <libc-internal.h>.
	(do_test): Disable -Walloc-size-larger-than= around tests of
	malloc with negative sizes.
	* malloc/tst-mcheck.c: Include <libc-internal.h>.
	(do_test): Disable -Walloc-size-larger-than= around tests of
	malloc and realloc with negative sizes.
	* malloc/tst-realloc.c: Include <libc-internal.h>.
	(do_test): Disable -Walloc-size-larger-than= around tests of
	realloc with negative sizes.
2017-01-04 23:32:14 +00:00
Florian Weimer
34a63b0973 malloc: Run tunables tests only if tunables are enabled
Otherwise, the environment variable will not have any effect and
the test will fail.
2017-01-01 09:27:03 +01:00
Joseph Myers
58b587c1f8 Update copyright dates not handled by scripts/update-copyrights.
I've updated copyright dates in glibc for 2017.  This is the patch for
the changes not generated by scripts/update-copyrights and subsequent
build / regeneration of generated files.

Please remember to include 2017 in the dates for any new files added
in future (which means updating any existing uncommitted patches you
have that add new files to use the new copyright dates in them).

	* NEWS: Update copyright dates.
	* catgets/gencat.c (print_version): Likewise.
	* csu/version.c (banner): Likewise.
	* debug/catchsegv.sh: Likewise.
	* debug/pcprofiledump.c (print_version): Likewise.
	* debug/xtrace.sh (do_version): Likewise.
	* elf/ldconfig.c (print_version): Likewise.
	* elf/ldd.bash.in: Likewise.
	* elf/pldd.c (print_version): Likewise.
	* elf/sotruss.sh: Likewise.
	* elf/sprof.c (print_version): Likewise.
	* iconv/iconv_prog.c (print_version): Likewise.
	* iconv/iconvconfig.c (print_version): Likewise.
	* locale/programs/locale.c (print_version): Likewise.
	* locale/programs/localedef.c (print_version): Likewise.
	* login/programs/pt_chown.c (print_version): Likewise.
	* malloc/memusage.sh (do_version): Likewise.
	* malloc/memusagestat.c (print_version): Likewise.
	* malloc/mtrace.pl: Likewise.
	* manual/libc.texinfo: Likewise.
	* nptl/version.c (banner): Likewise.
	* nscd/nscd.c (print_version): Likewise.
	* nss/getent.c (print_version): Likewise.
	* nss/makedb.c (print_version): Likewise.
	* posix/getconf.c (main): Likewise.
	* scripts/test-installation.pl: Likewise.
	* sysdeps/unix/sysv/linux/lddlibc4.c (main): Likewise.
2017-01-01 00:26:24 +00:00
Joseph Myers
bfff8b1bec Update copyright dates with scripts/update-copyrights. 2017-01-01 00:14:16 +00:00
Siddhesh Poyarekar
9dd409a5f4 Initialize tunable list with the GLIBC_TUNABLES environment variable
Read tunables values from the users using the GLIBC_TUNABLES
environment variable.  The value of this variable is a colon-separated
list of name=value pairs.  So a typical string would look like this:

GLIBC_TUNABLES=glibc.malloc.mmap_threshold=2048:glibc.malloc.trim_threshold=1024

	* config.make.in (have-loop-to-function): Define.
	* elf/Makefile (CFLAGS-dl-tunables.c): Add
	-fno-tree-loop-distribute-patterns.
	* elf/dl-tunables.c: Include libc-internals.h.
	(GLIBC_TUNABLES): New macro.
	(tunables_strdup): New function.
	(parse_tunables): New function.
	(min_strlen): New function.
	(__tunables_init): Use the new functions and macro.
	(disable_tunable): Disable tunable from GLIBC_TUNABLES.
	* malloc/tst-malloc-usable-tunables.c: New test case.
	* malloc/tst-malloc-usable-static-tunables.c: New test case.
	* malloc/Makefile (tests, tests-static): Add tests.
2016-12-31 23:49:24 +05:30
Siddhesh Poyarekar
67e58f3941 Add framework for tunables
The tunables framework allows us to uniformly manage and expose global
variables inside glibc as switches to users.  tunables/README has
instructions for glibc developers to add new tunables.

Tunables support can be enabled by passing the --enable-tunables
configure flag to the configure script.  This patch only adds a
framework and does not pose any limitations on how tunable values are
read from the user.  It also adds environment variables used in malloc
behaviour tweaking to the tunables framework as a PoC of the
compatibility interface.

	* manual/install.texi: Add --enable-tunables option.
	* INSTALL: Regenerate.
	* README.tunables: New file.
	* Makeconfig (CPPFLAGS): Define TOP_NAMESPACE.
	(before-compile): Generate dl-tunable-list.h early.
	* config.h.in: Add HAVE_TUNABLES.
	* config.make.in: Add have-tunables.
	* configure.ac: Add --enable-tunables option.
	* configure: Regenerate.
	* csu/init-first.c (__libc_init_first): Move
	__libc_init_secure earlier...
	* csu/init-first.c (LIBC_START_MAIN):... to here.
	Include dl-tunables.h, libc-internal.h.
	(LIBC_START_MAIN) [!SHARED]: Initialize tunables for static
	binaries.
	* elf/Makefile (dl-routines): Add dl-tunables.
	* elf/Versions (ld): Add __tunable_set_val to GLIBC_PRIVATE
	namespace.
	* elf/dl-support (_dl_nondynamic_init): Unset MALLOC_CHECK_
	only when !HAVE_TUNABLES.
	* elf/rtld.c (process_envvars): Likewise.
	* elf/dl-sysdep.c [HAVE_TUNABLES]: Include dl-tunables.h
	(_dl_sysdep_start): Call __tunables_init.
	* elf/dl-tunable-types.h: New file.
	* elf/dl-tunables.c: New file.
	* elf/dl-tunables.h: New file.
	* elf/dl-tunables.list: New file.
	* malloc/tst-malloc-usable-static.c: New test case.
	* malloc/Makefile (tests-static): Add it.
	* malloc/arena.c [HAVE_TUNABLES]: Include dl-tunables.h.
	Define TUNABLE_NAMESPACE.
	(DL_TUNABLE_CALLBACK (set_mallopt_check)): New function.
	(DL_TUNABLE_CALLBACK_FNDECL): New macro.  Use it to define
	callback functions.
	(ptmalloc_init): Set tunable values.
	* scripts/gen-tunables.awk: New file.
	* sysdeps/mach/hurd/dl-sysdep.c: Include dl-tunables.h.
	(_dl_sysdep_start): Call __tunables_init.
2016-12-31 23:49:24 +05:30
Florian Weimer
c23de0aacb support: Introduce new subdirectory for test infrastructure
The new test driver in <support/test-driver.c> has feature parity with
the old one.  The main difference is that its hooking mechanism is
based on functions and function pointers instead of macros.  This
commit also implements a new environment variable, TEST_COREDUMPS,
which disables the code which disables coredumps (that is, it enables
them if the invocation environment has not disabled them).

<test-skeleton.c> defines wrapper functions so that it is possible to
use existing macros with the new-style hook functionality.

This commit changes only a few test cases to the new test driver, to
make sure that it works as expected.
2016-12-09 08:18:27 +01:00
Florian Weimer
ae9166f2b8 malloc: Update comments about chunk layout 2016-10-28 22:36:58 +02:00
Florian Weimer
681421f3ca sysmalloc: Initialize previous size field of mmaped chunks
With different encodings of the header, the previous zero initialization
may be insufficient and produce an invalid encoding.
2016-10-28 16:49:04 +02:00
Florian Weimer
e9c4fe93b3 malloc: Use accessors for chunk metadata access
This change allows us to change the encoding of these struct members
in a centralized fashion.
2016-10-28 16:45:45 +02:00
Siddhesh Poyarekar
be7991c070 Static inline functions for mallopt helpers
Make mallopt helper functions for each mallopt parameter so that it
can be called consistently in other areas, like setting tunables.

	* malloc/malloc.c (do_set_mallopt_check): New function.
	(do_set_mmap_threshold): Likewise.
	(do_set_mmaps_max): Likewise.
	(do_set_top_pad): Likewise.
	(do_set_perturb_byte): Likewise.
	(do_set_trim_threshold): Likewise.
	(do_set_arena_max): Likewise.
	(do_set_arena_test): Likewise.
	(__libc_mallopt): Use them.
2016-10-27 08:34:55 +05:30
Florian Weimer
e863cce57b malloc: Remove malloc_get_state, malloc_set_state [BZ #19473]
After the removal of __malloc_initialize_hook, newly compiled
Emacs binaries are no longer able to use these interfaces.
malloc_get_state is only used during the Emacs build process,
so we provide a stub implementation only.  Existing Emacs binaries
will not call this stub function, but still reference the symbol.

The rewritten tst-mallocstate test constructs a dumped heap
which should approximates what existing Emacs binaries pass
to glibc malloc.
2016-10-26 13:28:28 +02:00
Siddhesh Poyarekar
68fc2ccc1a Remove redundant definitions of M_ARENA_* macros
The M_ARENA_MAX and M_ARENA_TEST macros are defined in malloc.c as
well as malloc.h, and the former is unnecessary.  This patch removes
the duplicate.  Tested on x86_64 to verify that the generated code
remains unchanged barring changed line numbers to __malloc_assert.

	* malloc/malloc.c (M_ARENA_TEST, M_ARENA_MAX): Remove.
2016-10-26 15:07:34 +05:30
Siddhesh Poyarekar
c1234e60f9 Document the M_ARENA_* mallopt parameters
The M_ARENA_* mallopt parameters are in wide use in production to
control the number of arenas that a long lived process creates and
hence there is no point in stating that this interface is non-public.
Document this interface and remove the obsolete comment.

	* manual/memory.texi (M_ARENA_TEST): Add documentation.
	(M_ARENA_MAX): Likewise.
	* malloc/malloc.c: Remove obsolete comment.
2016-10-26 15:06:21 +05:30
Florian Weimer
cbb47fa1c6 malloc: Manual part of conversion to __libc_lock
This removes the old mutex_t-related definitions from malloc-machine.h,
too.
2016-09-21 16:28:08 +02:00
Siddhesh Poyarekar
0f9317dcf5 Add tests-static to tests in malloc/Makefile
This is a trivial change to add the static tests only to tests-static
and then adding all of tests-static to the tests target to make it
look consistent with some other Makefiles.  This avoids having to
duplicate the test names across the two make targets.

	* malloc/Makefile (tests): Remove individual static test names
	and just add all of tests-static.
2016-09-10 16:10:51 +05:30
Florian Weimer
4bf5f2224b malloc: Automated part of conversion to __libc_lock 2016-09-06 12:49:54 +02:00
Florian Weimer
ef4f97648d malloc: Simplify static malloc interposition [BZ #20432]
Existing interposed mallocs do not define the glibc-internal
fork callbacks (and they should not), so statically interposed
mallocs lead to link failures because the strong reference from
fork pulls in glibc's malloc, resulting in multiple definitions
of malloc-related symbols.
2016-08-26 23:20:41 +02:00
Florian Weimer
5bc17330eb elf: dl-minimal malloc needs to respect fundamental alignment
The dynamic linker currently uses __libc_memalign for TLS-related
allocations.  The goal is to switch to malloc instead.  If the minimal
malloc follows the ABI fundamental alignment, we can assume that malloc
provides this alignment, and thus skip explicit alignment in a few
cases as an optimization.

It was requested on libc-alpha that MALLOC_ALIGNMENT should be used,
although this results in wasted space if MALLOC_ALIGNMENT is larger
than the fundamental alignment.  (The dynamic linker cannot assume
that the non-minimal malloc will provide an alignment of
MALLOC_ALIGNMENT; the ABI provides _Alignof (max_align_t) only.)
2016-08-03 16:11:01 +02:00
Florian Weimer
f690b56979 malloc: Run tests without calling mallopt [BZ #19469]
The compiled tests no longer refer to the mallopt symbol
from their main functions.  (Some tests still call mallopt
explicitly, which is fine.)
2016-08-02 17:06:11 +02:00
Florian Weimer
f88aab5d50 malloc: Preserve arena free list/thread count invariant [BZ #20370]
It is necessary to preserve the invariant that if an arena is
on the free list, it has thread attach count zero.  Otherwise,
when arena_thread_freeres sees the zero attach count, it will
add it, and without the invariant, an arena could get pushed
to the list twice, resulting in a cycle.

One possible execution trace looks like this:

Thread 1 examines free list and observes it as empty.
Thread 2 exits and adds its arena to the free list,
  with attached_threads == 0).
Thread 1 selects this arena in reused_arena (not from the free list).
Thread 1 increments attached_threads and attaches itself.
  (The arena remains on the free list.)
Thread 1 exits, decrements attached_threads,
  and adds the arena to the free list.

The final step creates a cycle in the usual way (by overwriting the
next_free member with the former list head, while there is another
list item pointing to the arena structure).

tst-malloc-thread-exit exhibits this issue, but it was only visible
with a debugger because the incorrect fix in bug 19243 removed
the assert from get_free_list.
2016-08-02 12:24:50 +02:00
Chris Metcalf
00068ce40c Bump up tst-malloc-thread-fail timeout from 20 to 30s
Right now tilegx is right on the verge of timeout when it runs,
so adding a bit of headroom seems like the right thing; we
see failures when running tests in parallel.
2016-07-05 17:05:28 -04:00
Florian Weimer
14699b6e37 test-skeleton.c: Add write_message function 2016-06-23 11:00:36 +02:00
Florian Weimer
a3b473373e malloc: Avoid premature fallback to mmap [BZ #20284]
Before this change, the while loop in reused_arena which avoids
returning a corrupt arena would never execute its body if the selected
arena were not corrupt.  As a result, result == begin after the loop,
and the function returns NULL, triggering fallback to mmap.
2016-06-21 21:29:21 +02:00
Florian Weimer
92e1ab0eb5 Revert __malloc_initialize_hook symbol poisoning
It turns out the Emacs-internal malloc implementation uses
__malloc_* symbols.  If glibc poisons them in <stdc-pre.h>,
Emacs will no longer compile.
2016-06-20 11:11:29 +02:00
Florian Weimer
073f82140c malloc_usable_size: Use correct size for dumped fake mapped chunks
The adjustment for the size computation in commit
1e8a8875d6 is needed in
malloc_usable_size, too.
2016-06-11 12:09:19 +02:00
Florian Weimer
2ba3cfa160 malloc: Remove __malloc_initialize_hook from the API [BZ #19564]
__malloc_initialize_hook is interposed by application code, so
the usual approach to define a compatibility symbol does not work.
This commit adds a new mechanism based on #pragma GCC poison in
<stdc-predef.h>.
2016-06-10 10:46:05 +02:00
Florian Weimer
1e8a8875d6 malloc: Correct size computation in realloc for dumped fake mmapped chunks
For regular mmapped chunks there are two size fields (hence a reduction
by 2 * SIZE_SZ bytes), but for fake chunks, we only have one size field,
so we need to subtract SIZE_SZ bytes.

This was initially reported as Emacs bug 23726.
2016-06-08 20:50:21 +02:00
Florian Weimer
dea39b13e2 malloc: Correct malloc alignment on 32-bit architectures [BZ #6527]
After the heap rewriting added in commit
4cf6c72fd2 (malloc: Rewrite dumped heap
for compatibility in __malloc_set_state), we can change malloc alignment
for new allocations because the alignment of old allocations no longer
matters.

We need to increase the malloc state version number, so that binaries
containing dumped heaps of the new layout will not try to run on
previous versions of glibc, resulting in obscure crashes.

This commit addresses a failure of tst-malloc-thread-fail on the
affected architectures (32-bit ppc and mips) because the test checks
pointer alignment.
2016-05-24 08:05:15 +02:00
Florian Weimer
e2cd73a2cc tst-mallocfork2: Fix race condition, use fewer resources
The first SIGUSR1 signal could arrive when sigusr1_sender_pid
was still 0.  As a result, kill would send SIGSTOP to the
entire process group.  This would cause the test to hang before
printing any output.

This commit also adds a sched_yield to the signal source, so that
it does not flood the parent process with signals it has never a
chance to handle.

Even with these changes, tst-mallocfork2 still fails reliably
after the fix in commit commit 56290d6e76
(Increase fork signal safety for single-threaded processes) is
backed out.
2016-05-13 20:43:14 +02:00
Florian Weimer
4cf6c72fd2 malloc: Rewrite dumped heap for compatibility in __malloc_set_state
This will allow us to change many aspects of the malloc implementation
while preserving compatibility with existing Emacs binaries.

As a result, existing Emacs binaries will have a larger RSS, and Emacs
needs a few more milliseconds to start.  This overhead is specific
to Emacs (and will go away once Emacs switches to its internal malloc).

The new checks to make free and realloc compatible with the dumped heap
are confined to the mmap paths, which are already quite slow due to the
munmap overhead.

This commit weakens some security checks, but only for heap pointers
in the dumped main arena.  By default, this area is empty, so those
checks are as effective as before.
2016-05-13 14:16:39 +02:00
Florian Weimer
56290d6e76 Increase fork signal safety for single-threaded processes [BZ #19703]
This provides a band-aid and addresses the scenario where fork is
called from a signal handler while the process is in the malloc
subsystem (or has acquired the libio list lock).  It does not
address the general issue of async-signal-safety of fork;
multi-threaded processes are not covered, and some glibc
subsystems have fork handlers which are not async-signal-safe.
2016-05-12 15:26:55 +02:00
Florian Weimer
66355680f8 malloc: Adjust header file guard in malloc-internal.h 2016-05-04 15:27:15 +02:00
Florian Weimer
186fe877f3 malloc: Add missing internal_function attributes on function definitions
Fixes build on i386 after commit 29d794863c.
2016-04-14 12:54:22 +02:00
Florian Weimer
8a727af925 malloc: Remove malloc hooks from fork handler
The fork handler now runs so late that there is no risk anymore that
other fork handlers in the same thread use malloc, so it is no
longer necessary to install malloc hooks which made a subset
of malloc functionality available to the thread that called fork.
2016-04-14 09:18:30 +02:00
Florian Weimer
29d794863c malloc: Run fork handler as late as possible [BZ #19431]
Previously, a thread M invoking fork would acquire locks in this order:

  (M1) malloc arena locks (in the registered fork handler)
  (M2) libio list lock

A thread F invoking flush (NULL) would acquire locks in this order:

  (F1) libio list lock
  (F2) individual _IO_FILE locks

A thread G running getdelim would use this order:

  (G1) _IO_FILE lock
  (G2) malloc arena lock

After executing (M1), (F1), (G1), none of the threads can make progress.

This commit changes the fork lock order to:

  (M'1) libio list lock
  (M'2) malloc arena locks

It explicitly encodes the lock order in the implementations of fork,
and does not rely on the registration order, thus avoiding the deadlock.
2016-04-14 09:17:02 +02:00
Florian Weimer
c04af6068b scratch_buffer_set_array_size: Include <limits.h>
It is needed for CHAR_BIT.
2016-04-07 13:46:28 +02:00
Samuel Thibault
b87e41378b Fix malloc threaded tests link on non-Linux
* malloc/Makefile ($(objpfx)tst-malloc-backtrace,
	$(objpfx)tst-malloc-thread-exit, $(objpfx)tst-malloc-thread-fail): Use
	$(shared-thread-library) instead of hardcoding the path to libpthread.
2016-03-22 09:58:48 +01:00
Tulio Magno Quites Machado Filho
b43f552a8a Fix type of parameter passed by malloc_consolidate
atomic_exchange_acq() expected a pointer, but was receiving an integer.
2016-03-11 18:09:40 -03:00
Florian Weimer
59eda029a8 malloc: Remove NO_THREADS
No functional change.  It was not possible to build without
threading support before.
2016-02-19 17:07:45 +01:00
Florian Weimer
ca135f824b malloc: Remove max_total_mem member form struct malloc_par
Also note that sumblks in struct mallinfo is always 0.
No functional change.
2016-02-19 17:07:04 +01:00
Florian Weimer
00d4e2ea35 malloc: Remove arena_mem variable
The computed value is never used.  The accesses were data races.
2016-02-19 17:06:33 +01:00