linux/kernel/rcu
Frederic Weisbecker e75bcd48e2 rcu/nocb: Unify timers
Now that ->nocb_timer and ->nocb_bypass_timer have become quite similar,
this commit merges them together.  A new RCU_NOCB_WAKE_BYPASS wake level
is introduced.  As a result, timers perform all kinds of deferred wake
ups but other deferred wakeup callsites only handle non-bypass wakeups
in order not to wake up rcuo too early.

The timer also unconditionally executes a full barrier so as to order
timer_pending() and callback enqueue although the path performing
RCU_NOCB_WAKE_FORCE that makes use of it is debatable. It should also
test against the rdp leader instead of the current rdp.

This unconditional full barrier shouldn't bring visible overhead since
these timers almost never fire.

Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Cc: Josh Triplett <josh@joshtriplett.org>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Joel Fernandes <joel@joelfernandes.org>
Cc: Neeraj Upadhyay <neeraju@codeaurora.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
2021-05-12 12:10:23 -07:00
..
Kconfig Merge branches 'doc.2021.01.06a', 'fixes.2021.01.04b', 'kfree_rcu.2021.01.04a', 'mmdumpobj.2021.01.22a', 'nocb.2021.01.06a', 'rt.2021.01.04a', 'stall.2021.01.06a', 'torture.2021.01.12a' and 'tortureall.2021.01.06a' into HEAD 2021-01-22 15:26:44 -08:00
Kconfig.debug Merge branch 'strictgp.2020.08.24a' into HEAD 2020-09-03 09:47:42 -07:00
Makefile rcuperf: Change rcuperf to rcuscale 2020-08-24 18:39:24 -07:00
rcu_segcblist.c rcu/nocb: Remove stale comment above rcu_segcblist_offload() 2021-03-15 13:54:54 -07:00
rcu_segcblist.h rcu/nocb: Code-style nits in callback-offloading toggling 2021-01-06 16:47:55 -08:00
rcu.h Merge branches 'doc.2021.01.06a', 'fixes.2021.01.04b', 'kfree_rcu.2021.01.04a', 'mmdumpobj.2021.01.22a', 'nocb.2021.01.06a', 'rt.2021.01.04a', 'stall.2021.01.06a', 'torture.2021.01.12a' and 'tortureall.2021.01.06a' into HEAD 2021-01-22 15:26:44 -08:00
rcuscale.c rcuscale: Add kfree_rcu() single-argument scale test 2021-03-08 14:18:07 -08:00
rcutorture.c Merge branches 'bitmaprange.2021.03.08a', 'fixes.2021.03.15a', 'kvfree_rcu.2021.03.08a', 'mmdumpobj.2021.03.08a', 'nocb.2021.03.15a', 'poll.2021.03.24a', 'rt.2021.03.08a', 'tasks.2021.03.08a', 'torture.2021.03.08a' and 'torturescript.2021.03.22a' into HEAD 2021-03-24 17:20:18 -07:00
refscale.c torture: Make refscale throttle high-rate printk()s 2021-01-06 17:17:20 -08:00
srcutiny.c srcu: Provide polling interfaces for Tiny SRCU grace periods 2021-01-04 13:53:38 -08:00
srcutree.c Merge branches 'doc.2021.01.06a', 'fixes.2021.01.04b', 'kfree_rcu.2021.01.04a', 'mmdumpobj.2021.01.22a', 'nocb.2021.01.06a', 'rt.2021.01.04a', 'stall.2021.01.06a', 'torture.2021.01.12a' and 'tortureall.2021.01.06a' into HEAD 2021-01-22 15:26:44 -08:00
sync.c rcu/sync: Simplify the state machine 2019-05-28 09:05:23 -07:00
tasks.h rcu-tasks: Add block comment laying out RCU Tasks Trace design 2021-03-08 14:22:02 -08:00
tiny.c rcu: Provide polling interfaces for Tiny RCU grace periods 2021-03-24 17:16:15 -07:00
tree_exp.h rcu/tree: Add a trace event for RCU CPU stall warnings 2021-03-15 13:53:24 -07:00
tree_plugin.h rcu/nocb: Unify timers 2021-05-12 12:10:23 -07:00
tree_stall.h rcu/tree: Add a trace event for RCU CPU stall warnings 2021-03-15 13:53:24 -07:00
tree.c rcu/nocb: Prepare for fine-grained deferred wakeup 2021-05-12 12:10:23 -07:00
tree.h rcu/nocb: Unify timers 2021-05-12 12:10:23 -07:00
update.c rcu: Enable rcu_normal_after_boot unconditionally for RT 2021-01-04 13:43:51 -08:00