2
0
mirror of https://github.com/edk2-porting/linux-next.git synced 2024-12-25 21:54:06 +08:00
linux-next/include/asm-i386
Jakub Jelinek 4732efbeb9 [PATCH] FUTEX_WAKE_OP: pthread_cond_signal() speedup
ATM pthread_cond_signal is unnecessarily slow, because it wakes one waiter
(which at least on UP usually means an immediate context switch to one of
the waiter threads).  This waiter wakes up and after a few instructions it
attempts to acquire the cv internal lock, but that lock is still held by
the thread calling pthread_cond_signal.  So it goes to sleep and eventually
the signalling thread is scheduled in, unlocks the internal lock and wakes
the waiter again.

Now, before 2003-09-21 NPTL was using FUTEX_REQUEUE in pthread_cond_signal
to avoid this performance issue, but it was removed when locks were
redesigned to the 3 state scheme (unlocked, locked uncontended, locked
contended).

Following scenario shows why simply using FUTEX_REQUEUE in
pthread_cond_signal together with using lll_mutex_unlock_force in place of
lll_mutex_unlock is not enough and probably why it has been disabled at
that time:

The number is value in cv->__data.__lock.
        thr1            thr2            thr3
0       pthread_cond_wait
1       lll_mutex_lock (cv->__data.__lock)
0       lll_mutex_unlock (cv->__data.__lock)
0       lll_futex_wait (&cv->__data.__futex, futexval)
0                       pthread_cond_signal
1                       lll_mutex_lock (cv->__data.__lock)
1                                       pthread_cond_signal
2                                       lll_mutex_lock (cv->__data.__lock)
2                                         lll_futex_wait (&cv->__data.__lock, 2)
2                       lll_futex_requeue (&cv->__data.__futex, 0, 1, &cv->__data.__lock)
                          # FUTEX_REQUEUE, not FUTEX_CMP_REQUEUE
2                       lll_mutex_unlock_force (cv->__data.__lock)
0                         cv->__data.__lock = 0
0                         lll_futex_wake (&cv->__data.__lock, 1)
1       lll_mutex_lock (cv->__data.__lock)
0       lll_mutex_unlock (cv->__data.__lock)
          # Here, lll_mutex_unlock doesn't know there are threads waiting
          # on the internal cv's lock

Now, I believe it is possible to use FUTEX_REQUEUE in pthread_cond_signal,
but it will cost us not one, but 2 extra syscalls and, what's worse, one of
these extra syscalls will be done for every single waiting loop in
pthread_cond_*wait.

We would need to use lll_mutex_unlock_force in pthread_cond_signal after
requeue and lll_mutex_cond_lock in pthread_cond_*wait after lll_futex_wait.

Another alternative is to do the unlocking pthread_cond_signal needs to do
(the lock can't be unlocked before lll_futex_wake, as that is racy) in the
kernel.

I have implemented both variants, futex-requeue-glibc.patch is the first
one and futex-wake_op{,-glibc}.patch is the unlocking inside of the kernel.
 The kernel interface allows userland to specify how exactly an unlocking
operation should look like (some atomic arithmetic operation with optional
constant argument and comparison of the previous futex value with another
constant).

It has been implemented just for ppc*, x86_64 and i?86, for other
architectures I'm including just a stub header which can be used as a
starting point by maintainers to write support for their arches and ATM
will just return -ENOSYS for FUTEX_WAKE_OP.  The requeue patch has been
(lightly) tested just on x86_64, the wake_op patch on ppc64 kernel running
32-bit and 64-bit NPTL and x86_64 kernel running 32-bit and 64-bit NPTL.

With the following benchmark on UP x86-64 I get:

for i in nptl-orig nptl-requeue nptl-wake_op; do echo time elf/ld.so --library-path .:$i /tmp/bench; \
for j in 1 2; do echo ( time elf/ld.so --library-path .:$i /tmp/bench ) 2>&1; done; done
time elf/ld.so --library-path .:nptl-orig /tmp/bench
real 0m0.655s user 0m0.253s sys 0m0.403s
real 0m0.657s user 0m0.269s sys 0m0.388s
time elf/ld.so --library-path .:nptl-requeue /tmp/bench
real 0m0.496s user 0m0.225s sys 0m0.271s
real 0m0.531s user 0m0.242s sys 0m0.288s
time elf/ld.so --library-path .:nptl-wake_op /tmp/bench
real 0m0.380s user 0m0.176s sys 0m0.204s
real 0m0.382s user 0m0.175s sys 0m0.207s

The benchmark is at:
http://sourceware.org/ml/libc-alpha/2005-03/txt00001.txt
Older futex-requeue-glibc.patch version is at:
http://sourceware.org/ml/libc-alpha/2005-03/txt00002.txt
Older futex-wake_op-glibc.patch version is at:
http://sourceware.org/ml/libc-alpha/2005-03/txt00003.txt
Will post a new version (just x86-64 fixes so that the patch
applies against pthread_cond_signal.S) to libc-hacker ml soon.

Attached is the kernel FUTEX_WAKE_OP patch as well as a simple-minded
testcase that will not test the atomicity of the operation, but at least
check if the threads that should have been woken up are woken up and
whether the arithmetic operation in the kernel gave the expected results.

Acked-by: Ingo Molnar <mingo@redhat.com>
Cc: Ulrich Drepper <drepper@redhat.com>
Cc: Jamie Lokier <jamie@shareable.org>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Yoichi Yuasa <yuasa@hh.iij4u.or.jp>
Signed-off-by: Andrew Morton <akpm@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@osdl.org>
2005-09-07 16:57:17 -07:00
..
mach-bigsmp [PATCH] Do not enforce unique IO_APIC_ID check for xAPIC systems (i386) 2005-06-23 09:45:09 -07:00
mach-default [PATCH] x86: i8253/i8259A lock cleanup 2005-06-30 08:45:10 -07:00
mach-es7000 [PATCH] ES7000 platform update (i386) 2005-09-05 00:06:10 -07:00
mach-generic [PATCH] x86: sutomatically enable bigsmp when we have more than 8 CPUs 2005-09-05 00:06:10 -07:00
mach-numaq [PATCH] Do not enforce unique IO_APIC_ID check for xAPIC systems (i386) 2005-06-23 09:45:09 -07:00
mach-summit [PATCH] Do not enforce unique IO_APIC_ID check for xAPIC systems (i386) 2005-06-23 09:45:09 -07:00
mach-visws [PATCH] Make visws compile again 2005-08-07 10:00:38 -07:00
mach-voyager Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
8253pit.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
a.out.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
acpi.h [ACPI] enable C2 and C3 idle power states on SMP 2005-07-12 00:14:36 -04:00
agp.h [PATCH] i386: inline asm cleanup 2005-09-05 00:06:11 -07:00
apic.h [PATCH] kexec: x86: resture apic virtual wire mode on shutdown 2005-06-25 16:24:47 -07:00
apicdef.h [PATCH] x86: sutomatically enable bigsmp when we have more than 8 CPUs 2005-09-05 00:06:10 -07:00
arch_hooks.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
atomic.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
bitops.h x86: fix new find_first_bit() 2005-07-29 11:01:22 -04:00
boot.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
bug.h [PATCH] remove all kernel BUGs 2005-05-01 08:59:01 -07:00
bugs.h [PATCH] i386: inline asm cleanup 2005-09-05 00:06:11 -07:00
byteorder.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
cache.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
cacheflush.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
checksum.h [NET]: Fix ipl=>ihl typo in ip_fast_csum 2005-08-29 16:02:48 -07:00
cpu.h [PATCH] i386 CPU hotplug 2005-06-25 16:24:29 -07:00
cpufeature.h [PATCH] CPUID bug and inconsistency fix 2005-05-01 08:58:49 -07:00
cputime.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
current.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
debugreg.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
delay.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
desc.h [PATCH] x86: introduce a write acessor for updating the current LDT 2005-09-05 00:06:13 -07:00
div64.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
dma-mapping.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
dma.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
e820.h [PATCH] Increase number of e820 entries hard limit from 32 to 128 2005-05-01 08:58:51 -07:00
elf.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
emergency-restart.h [PATCH] i386: Implement machine_emergency_reboot 2005-07-26 14:35:42 -07:00
errno.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
fcntl.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
fixmap.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
floppy.h [PATCH] make some things static 2005-05-05 16:36:47 -07:00
futex.h [PATCH] FUTEX_WAKE_OP: pthread_cond_signal() speedup 2005-09-07 16:57:17 -07:00
genapic.h [PATCH] Do not enforce unique IO_APIC_ID check for xAPIC systems (i386) 2005-06-23 09:45:09 -07:00
hardirq.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
hdreg.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
highmem.h [PATCH] kdump: Routines for copying dump pages 2005-06-25 16:24:53 -07:00
hpet.h [PATCH] i386: fix hpet for systems that don't support legacy replacement 2005-05-01 08:58:50 -07:00
hw_irq.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
i387.h x86: use alternative instructions for fnsave/fxsave too 2005-07-22 18:19:20 -04:00
i8253.h [PATCH] x86: i8253/i8259A lock cleanup 2005-06-30 08:45:10 -07:00
i8259.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
ide.h [PATCH] ide: sensible probing for PCI systems 2005-06-27 17:36:41 -07:00
io_apic.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
io.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
ioctl.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
ioctls.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
ipc.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
ipcbuf.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
irq.h [PATCH] cpu state clean after hot remove 2005-06-25 16:24:30 -07:00
ist.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
kdebug.h [PATCH] x86 NMI: better support for debuggers 2005-09-05 00:06:13 -07:00
kexec.h [PATCH] Kdump: Export crash notes section address through sysfs 2005-06-25 16:24:51 -07:00
kmap_types.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
kprobes.h [PATCH] kprobes: function-return probes 2005-06-23 09:45:21 -07:00
ldt.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
linkage.h [PATCH] i386: fix prevent_tail_call 2005-05-26 16:16:16 -07:00
local.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
math_emu.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
mc146818rtc.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
mca_dma.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
mca.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
mman.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
mmu_context.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
mmu.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
mmx.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
mmzone.h [PATCH] Fix up non-NUMA breakage in mmzone.h 2005-07-07 18:23:42 -07:00
module.h [PATCH] x86: geode support fixes 2005-05-05 16:36:49 -07:00
mpspec_def.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
mpspec.h [PATCH] x86: sutomatically enable bigsmp when we have more than 8 CPUs 2005-09-05 00:06:10 -07:00
msgbuf.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
msi.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
msr.h [PATCH] x86: more asm cleanups 2005-09-05 00:06:12 -07:00
mtrr.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
namei.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
nmi.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
node.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
numaq.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
numnodes.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
page.h [PATCH] remove hugetlb_clean_stale_pgtable() and fix huge_pte_alloc() 2005-09-05 00:05:46 -07:00
param.h [PATCH] i386: Selectable Frequency of the Timer Interrupt 2005-06-23 09:45:10 -07:00
parport.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
pci-direct.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
pci.h Revert PCIBIOS_MIN_IO changes for 2.6.13 2005-08-14 18:21:30 -07:00
percpu.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
pgalloc.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
pgtable-2level-defs.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
pgtable-2level.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
pgtable-3level-defs.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
pgtable-3level.h [PATCH] i386: use set_pte macros in a couple places where they were missing 2005-09-05 00:06:12 -07:00
pgtable.h [PATCH] i386: encapsulate copying of pgd entries 2005-09-05 00:06:13 -07:00
poll.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
posix_types.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
processor.h [PATCH] x86: make IOPL explicit 2005-09-05 00:06:12 -07:00
ptrace.h [PATCH] x86: privilege cleanup 2005-09-05 00:06:12 -07:00
resource.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
rtc.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
rwlock.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
rwsem.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
scatterlist.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
seccomp.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
sections.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
segment.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
semaphore.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
sembuf.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
serial.h [PATCH] Serial: Split 8250 port table (part 2) 2005-06-29 18:45:19 +01:00
setup.h [PATCH] x86: fix EFI memory map parsing 2005-09-05 00:06:09 -07:00
shmbuf.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
shmparam.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
sigcontext.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
siginfo.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
signal.h [PATCH] asm/signal.h unification 2005-05-04 07:33:15 -07:00
smp.h [PATCH] i386 boottime for_each_cpu broken 2005-09-05 00:06:13 -07:00
socket.h [NET]: Introduce SO_{SND,RCV}BUFFORCE socket options 2005-08-29 15:31:35 -07:00
sockios.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
sparsemem.h [PATCH] sparsemem memory model for i386 2005-06-23 09:45:05 -07:00
spinlock.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
srat.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
stat.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
statfs.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
string.h Add "memory" clobbers to the x86 inline asm of strncmp and friends 2005-06-24 10:39:17 -07:00
suspend.h [PATCH] i386: Use loaddebug macro consistently 2005-04-16 15:24:46 -07:00
system.h [PATCH] x86: make IOPL explicit 2005-09-05 00:06:12 -07:00
termbits.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
termios.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
thread_info.h [PATCH] UML Support - Ptrace: adds the host SYSEMU support, for UML and general usage 2005-09-05 00:06:20 -07:00
timer.h [PATCH] add suspend/resume for timer 2005-09-05 00:06:18 -07:00
timex.h [PATCH] x86: cpu_khz type fix 2005-06-23 09:45:11 -07:00
tlb.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
tlbflush.h [PATCH] seccomp: tsc disable 2005-06-27 15:11:44 -07:00
topology.h [PATCH] sched: sched tuning 2005-06-25 16:24:42 -07:00
types.h [PATCH] sab: consolidate kmem_bufctl_t 2005-09-05 00:05:48 -07:00
uaccess.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
ucontext.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
unaligned.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
unistd.h [PATCH] inotify 2005-07-12 20:38:38 -07:00
user.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
vga.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
vic.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
vm86.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
voyager.h Linux-2.6.12-rc2 2005-04-16 15:20:36 -07:00
xor.h [PATCH] i386: inline asm cleanup 2005-09-05 00:06:11 -07:00