linux/kernel/locking
Waiman Long 0e06e5be70 locking/qrwlock: Better optimization for interrupt context readers
The qrwlock is fair in the process context, but becoming unfair when
in the interrupt context to support use cases like the tasklist_lock.

The current code isn't that well-documented on what happens when
in the interrupt context. The rspin_until_writer_unlock() will only
spin if the writer has gotten the lock. If the writer is still in the
waiting state, the increment in the reader count will cause the writer
to remain in the waiting state and the new interrupt context reader
will get the lock and return immediately. The current code, however,
does an additional read of the lock value which is not necessary as
the information has already been there in the fast path. This may
sometime cause an additional cacheline transfer when the lock is
highly contended.

This patch passes the lock value information gotten in the fast path
to the slow path to eliminate the additional read. It also documents
the action for the interrupt context readers more clearly.

Signed-off-by: Waiman Long <Waiman.Long@hp.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Will Deacon <will.deacon@arm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Douglas Hatch <doug.hatch@hp.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Scott J Norton <scott.norton@hp.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/1434729002-57724-3-git-send-email-Waiman.Long@hp.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2015-07-06 14:11:28 +02:00
..
lglock.c sched/stop_machine: Fix deadlock between multiple stop_two_cpus() 2015-06-19 10:03:12 +02:00
lockdep_internals.h lockdep: Increase static allocations 2014-04-18 14:20:50 +02:00
lockdep_proc.c lockdep: Fix a race between /proc/lock_stat and module unload 2015-06-07 15:46:30 +02:00
lockdep_states.h locking: Move the lockdep code to kernel/locking/ 2013-11-06 07:55:08 +01:00
lockdep.c Merge branch 'sched-hrtimers-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2015-06-24 15:09:40 -07:00
locktorture.c locktorture: Change longdelay_us to longdelay_ms 2015-05-27 12:59:57 -07:00
Makefile locking/qrwlock: Rename QUEUE_RWLOCK to QUEUED_RWLOCKS 2015-05-12 09:46:00 +02:00
mcs_spinlock.h locking/qspinlock: Introduce a simple generic 4-byte queued spinlock 2015-05-08 12:36:25 +02:00
mutex-debug.c mutex: Always clear owner field upon mutex_unlock() 2015-01-09 11:20:39 +01:00
mutex-debug.h
mutex.c locking/mutex: Further simplify mutex_spin_on_owner() 2015-04-09 08:10:23 +02:00
mutex.h locking/mutexes: Use MUTEX_SPIN_ON_OWNER when appropriate 2014-08-13 10:32:02 +02:00
osq_lock.c locking: Remove ACCESS_ONCE() usage 2015-02-24 08:44:16 +01:00
percpu-rwsem.c locking: Move the percpu-rwsem code to kernel/locking/ 2013-11-06 09:24:22 +01:00
qrwlock.c locking/qrwlock: Better optimization for interrupt context readers 2015-07-06 14:11:28 +02:00
qspinlock_paravirt.h locking/arch: Rename set_mb() to smp_store_mb() 2015-05-19 08:32:00 +02:00
qspinlock.c locking/pvqspinlock: Implement simple paravirt support for the qspinlock 2015-05-08 12:37:05 +02:00
rtmutex_common.h futex: Lower the lock contention on the HB lock during wake up 2015-06-19 21:26:38 +02:00
rtmutex-debug.c rtmutex: Cleanup deadlock detector debug logic 2014-06-21 22:05:30 +02:00
rtmutex-debug.h rtmutex: Cleanup deadlock detector debug logic 2014-06-21 22:05:30 +02:00
rtmutex-tester.c locking: Move the rtmutex code to kernel/locking/ 2013-11-06 09:23:59 +01:00
rtmutex.c Merge branch 'sched-locking-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2015-06-24 14:46:01 -07:00
rtmutex.h rtmutex: Cleanup deadlock detector debug logic 2014-06-21 22:05:30 +02:00
rwsem-spinlock.c locking/rwsem: Document barrier need when waking tasks 2015-02-18 16:57:10 +01:00
rwsem-xadd.c locking/rwsem: Reduce spinlock contention in wakeup after up_read()/up_write() 2015-05-08 12:27:59 +02:00
rwsem.c locking/rwsem: Set lock ownership ASAP 2015-02-18 16:57:13 +01:00
rwsem.h locking/rwsem: Set lock ownership ASAP 2015-02-18 16:57:13 +01:00
semaphore.c locking/semaphore: Resolve some shadow warnings 2014-09-04 07:17:24 +02:00
spinlock_debug.c locking: Move the spinlock code to kernel/locking/ 2013-11-06 07:55:21 +01:00
spinlock.c spinlock: Add spin_lock_bh_nested() 2015-01-03 14:32:57 -05:00