Go to file
Jakub Jelinek 1d47c0512a libatomic: Improve 16-byte atomics on Intel AVX [PR104688]
As mentioned in the PR, the latest Intel SDM has added:
"Processors that enumerate support for Intel® AVX (by setting the feature flag CPUID.01H:ECX.AVX[bit 28])
guarantee that the 16-byte memory operations performed by the following instructions will always be
carried out atomically:
• MOVAPD, MOVAPS, and MOVDQA.
• VMOVAPD, VMOVAPS, and VMOVDQA when encoded with VEX.128.
• VMOVAPD, VMOVAPS, VMOVDQA32, and VMOVDQA64 when encoded with EVEX.128 and k0 (masking disabled).
(Note that these instructions require the linear addresses of their memory operands to be 16-byte
aligned.)"

The following patch deals with it just on the libatomic library side so far,
currently (since ~ 2017) we emit all the __atomic_* 16-byte builtins as
library calls since and this is something that we can hopefully backport.

The patch simply introduces yet another ifunc variant that takes priority
over the pure CMPXCHG16B one, one that checks AVX and CMPXCHG16B bits and
on non-Intel clears the AVX bit during detection for now (if AMD comes
with the same guarantee, we could revert the config/x86/init.c hunk),
which implements 16-byte atomic load as vmovdqa and 16-byte atomic store
as vmovdqa followed by mfence.

2022-03-17  Jakub Jelinek  <jakub@redhat.com>

	PR target/104688
	* Makefile.am (IFUNC_OPTIONS): Change on x86_64 to -mcx16 -mcx16.
	(libatomic_la_LIBADD): Add $(addsuffix _16_2_.lo,$(SIZEOBJS)) for
	x86_64.
	* Makefile.in: Regenerated.
	* config/x86/host-config.h (IFUNC_COND_1): For x86_64 define to
	both AVX and CMPXCHG16B bits.
	(IFUNC_COND_2): Define.
	(IFUNC_NCOND): For x86_64 define to 2 * (N == 16).
	(MAYBE_HAVE_ATOMIC_CAS_16, MAYBE_HAVE_ATOMIC_EXCHANGE_16,
	MAYBE_HAVE_ATOMIC_LDST_16): Define to IFUNC_COND_2 rather than
	IFUNC_COND_1.
	(HAVE_ATOMIC_CAS_16): Redefine to 1 whenever IFUNC_ALT != 0.
	(HAVE_ATOMIC_LDST_16): Redefine to 1 whenever IFUNC_ALT == 1.
	(atomic_compare_exchange_n): Define whenever IFUNC_ALT != 0
	on x86_64 for N == 16.
	(__atomic_load_n, __atomic_store_n): Redefine whenever IFUNC_ALT == 1
	on x86_64 for N == 16.
	(atomic_load_n, atomic_store_n): New functions.
	* config/x86/init.c (__libat_feat1_init): On x86_64 clear bit_AVX
	if CPU vendor is not Intel.
2022-03-17 18:49:00 +01:00
c++tools Update copyright years. 2022-01-03 10:42:10 +01:00
config Daily bump. 2021-12-22 00:16:30 +00:00
contrib Daily bump. 2022-03-11 00:16:39 +00:00
fixincludes Daily bump. 2022-02-28 00:16:17 +00:00
gcc tree-optimization/104960 - unsplit edges after late sinking 2022-03-17 09:51:09 +01:00
gnattools Daily bump. 2021-10-23 00:16:26 +00:00
gotools Daily bump. 2022-02-14 00:16:23 +00:00
include Update copyright years. 2022-01-03 10:42:10 +01:00
INSTALL
intl Daily bump. 2021-11-30 00:16:44 +00:00
libada Update copyright years. 2022-01-03 10:42:10 +01:00
libatomic libatomic: Improve 16-byte atomics on Intel AVX [PR104688] 2022-03-17 18:49:00 +01:00
libbacktrace Daily bump. 2022-02-18 00:16:39 +00:00
libcc1 Update copyright years. 2022-01-03 10:42:10 +01:00
libcody Update Copyright in ChangeLog files 2022-01-03 10:31:39 +01:00
libcpp Daily bump. 2022-03-05 00:16:31 +00:00
libdecnumber Update copyright years. 2022-01-03 10:42:10 +01:00
libffi Daily bump. 2021-11-16 00:16:31 +00:00
libgcc Daily bump. 2022-03-10 00:16:28 +00:00
libgfortran Daily bump. 2022-01-27 00:16:29 +00:00
libgo libgo: update to final Go 1.18 release 2022-03-16 13:52:32 -07:00
libgomp Enhance further testcases to verify Openacc 'kernels' decomposition 2022-03-17 08:51:32 +01:00
libiberty Daily bump. 2022-02-23 00:16:24 +00:00
libitm Daily bump. 2022-02-04 00:16:24 +00:00
libobjc Update copyright years. 2022-01-03 10:42:10 +01:00
liboffloadmic Daily bump. 2021-10-20 00:16:43 +00:00
libphobos Daily bump. 2022-03-14 00:16:20 +00:00
libquadmath Daily bump. 2022-01-12 00:16:39 +00:00
libsanitizer Daily bump. 2022-03-15 00:16:49 +00:00
libssp Update copyright years. 2022-01-03 10:42:10 +01:00
libstdc++-v3 libstdc++: Fix comment in testsuite utility 2022-03-17 12:23:56 +00:00
libvtv Update copyright years. 2022-01-03 10:42:10 +01:00
lto-plugin Daily bump. 2022-03-12 00:16:27 +00:00
maintainer-scripts Daily bump. 2021-05-15 00:16:27 +00:00
zlib Daily bump. 2021-12-17 00:16:20 +00:00
.dir-locals.el dir-locals: Use https for bug references 2021-07-20 11:40:34 +01:00
.gitattributes
.gitignore Add cscope.out to git ignore. 2021-06-24 16:51:40 +05:30
ABOUT-NLS
ar-lib
ChangeLog Daily bump. 2022-03-17 00:17:00 +00:00
ChangeLog.jit
ChangeLog.tree-ssa
compile
config-ml.in
config.guess config.sub, config.guess : Import upstream 2021-01-25. 2021-02-23 17:21:10 +08:00
config.rpath
config.sub config.sub: change mode to 755. 2021-12-21 09:10:57 +01:00
configure top-level: Fix comment about --enable-libstdcxx in configure 2022-03-14 13:08:02 +00:00
configure.ac top-level: Fix comment about --enable-libstdcxx in configure 2022-03-14 13:08:02 +00:00
COPYING
COPYING3
COPYING3.LIB
COPYING.LIB
COPYING.RUNTIME
depcomp
install-sh
libtool-ldflags
libtool.m4 Revert "Sync with binutils: GCC: Pass --plugin to AR and RANLIB" 2021-12-15 20:45:58 -08:00
lt~obsolete.m4
ltgcc.m4
ltmain.sh
ltoptions.m4
ltsugar.m4
ltversion.m4
MAINTAINERS MAINTAINERS: Add myself to DCO section 2022-03-16 03:20:00 +00:00
Makefile.def toplevel: Makefile.def: Make configure-sim depend on all-readline 2022-03-09 20:54:37 +01:00
Makefile.in toplevel: Makefile.def: Make configure-sim depend on all-readline 2022-03-09 20:54:37 +01:00
Makefile.tpl Revert "Sync with binutils: GCC: Pass --plugin to AR and RANLIB" 2021-12-15 20:45:58 -08:00
missing
mkdep
mkinstalldirs
move-if-change
multilib.am
README
symlink-tree
test-driver
ylwrap

This directory contains the GNU Compiler Collection (GCC).

The GNU Compiler Collection is free software.  See the files whose
names start with COPYING for copying permission.  The manuals, and
some of the runtime libraries, are under different terms; see the
individual source files for details.

The directory INSTALL contains copies of the installation information
as HTML and plain text.  The source of this information is
gcc/doc/install.texi.  The installation information includes details
of what is included in the GCC sources and what files GCC installs.

See the file gcc/doc/gcc.texi (together with other files that it
includes) for usage and porting information.  An online readable
version of the manual is in the files gcc/doc/gcc.info*.

See http://gcc.gnu.org/bugs/ for how to report bugs usefully.

Copyright years on GCC source files may be listed using range
notation, e.g., 1987-2012, indicating that every year in the range,
inclusive, is a copyrightable year that could otherwise be listed
individually.