linux/kernel/rcu
Paul E. McKenney b2b00ddf19 rcu: React to callback overload by aggressively seeking quiescent states
In default configutions, RCU currently waits at least 100 milliseconds
before asking cond_resched() and/or resched_rcu() for help seeking
quiescent states to end a grace period.  But 100 milliseconds can be
one good long time during an RCU callback flood, for example, as can
happen when user processes repeatedly open and close files in a tight
loop.  These 100-millisecond gaps in successive grace periods during a
callback flood can result in excessive numbers of callbacks piling up,
unnecessarily increasing memory footprint.

This commit therefore asks cond_resched() and/or resched_rcu() for help
as early as the first FQS scan when at least one of the CPUs has more
than 20,000 callbacks queued, a number that can be changed using the new
rcutree.qovld kernel boot parameter.  An auxiliary qovld_calc variable
is used to avoid acquisition of locks that have not yet been initialized.
Early tests indicate that this reduces the RCU-callback memory footprint
during rcutorture floods by from 50% to 4x, depending on configuration.

Reported-by: Joel Fernandes (Google) <joel@joelfernandes.org>
Reported-by: Tejun Heo <tj@kernel.org>
[ paulmck: Fix bug located by Qian Cai. ]
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Tested-by: Dexuan Cui <decui@microsoft.com>
Tested-by: Qian Cai <cai@lca.pw>
2020-02-20 16:00:20 -08:00
..
Kconfig rcu: Use CONFIG_PREEMPTION where appropriate 2019-12-09 12:37:51 -08:00
Kconfig.debug rcu: Add support for consolidated-RCU reader checking 2019-08-09 11:00:35 -07:00
Makefile rcu: Make PREEMPT_RCU be a modifier to TREE_RCU 2019-12-09 12:37:51 -08:00
rcu_segcblist.c rcu: Remove kfree_rcu() special casing and lazy-callback handling 2020-01-24 10:24:31 -08:00
rcu_segcblist.h rcu: Remove kfree_rcu() special casing and lazy-callback handling 2020-01-24 10:24:31 -08:00
rcu.h Merge branches 'doc.2019.12.10a', 'exp.2019.12.09a', 'fixes.2020.01.24a', 'kfree_rcu.2020.01.24a', 'list.2020.01.10a', 'preempt.2020.01.24a' and 'torture.2019.12.09a' into HEAD 2020-01-24 10:37:27 -08:00
rcuperf.c rcu: Remove kfree_call_rcu_nobatch() 2020-01-24 10:24:31 -08:00
rcutorture.c Merge branches 'doc.2019.12.10a', 'exp.2019.12.09a', 'fixes.2020.01.24a', 'kfree_rcu.2020.01.24a', 'list.2020.01.10a', 'preempt.2020.01.24a' and 'torture.2019.12.09a' into HEAD 2020-01-24 10:37:27 -08:00
srcutiny.c rcu: Use CONFIG_PREEMPTION where appropriate 2019-12-09 12:37:51 -08:00
srcutree.c Merge branches 'doc.2019.12.10a', 'exp.2019.12.09a', 'fixes.2020.01.24a', 'kfree_rcu.2020.01.24a', 'list.2020.01.10a', 'preempt.2020.01.24a' and 'torture.2019.12.09a' into HEAD 2020-01-24 10:37:27 -08:00
sync.c rcu/sync: Simplify the state machine 2019-05-28 09:05:23 -07:00
tiny.c rcu: Remove kfree_rcu() special casing and lazy-callback handling 2020-01-24 10:24:31 -08:00
tree_exp.h rcu: Forgive slow expedited grace periods at boot time 2020-01-25 12:00:40 -08:00
tree_plugin.h rcu: React to callback overload by aggressively seeking quiescent states 2020-02-20 16:00:20 -08:00
tree_stall.h Merge branches 'doc.2019.12.10a', 'exp.2019.12.09a', 'fixes.2020.01.24a', 'kfree_rcu.2020.01.24a', 'list.2020.01.10a', 'preempt.2020.01.24a' and 'torture.2019.12.09a' into HEAD 2020-01-24 10:37:27 -08:00
tree.c rcu: React to callback overload by aggressively seeking quiescent states 2020-02-20 16:00:20 -08:00
tree.h rcu: React to callback overload by aggressively seeking quiescent states 2020-02-20 16:00:20 -08:00
update.c Merge branches 'doc.2019.12.10a', 'exp.2019.12.09a', 'fixes.2020.01.24a', 'kfree_rcu.2020.01.24a', 'list.2020.01.10a', 'preempt.2020.01.24a' and 'torture.2019.12.09a' into HEAD 2020-01-24 10:37:27 -08:00