linux/init
Paul E. McKenney d5f177d35c rcu-tasks: Add an RCU Tasks Trace to simplify protection of tracing hooks
Because RCU does not watch exception early-entry/late-exit, idle-loop,
or CPU-hotplug execution, protection of tracing and BPF operations is
needlessly complicated.  This commit therefore adds a variant of
Tasks RCU that:

o	Has explicit read-side markers to allow finite grace periods in
	the face of in-kernel loops for PREEMPT=n builds.  These markers
	are rcu_read_lock_trace() and rcu_read_unlock_trace().

o	Protects code in the idle loop, exception entry/exit, and
	CPU-hotplug code paths.  In this respect, RCU-tasks trace is
	similar to SRCU, but with lighter-weight readers.

o	Avoids expensive read-side instruction, having overhead similar
	to that of Preemptible RCU.

There are of course downsides:

o	The grace-period code can send IPIs to CPUs, even when those
	CPUs are in the idle loop or in nohz_full userspace.  This is
	mitigated by later commits.

o	It is necessary to scan the full tasklist, much as for Tasks RCU.

o	There is a single callback queue guarded by a single lock,
	again, much as for Tasks RCU.  However, those early use cases
	that request multiple grace periods in quick succession are
	expected to do so from a single task, which makes the single
	lock almost irrelevant.  If needed, multiple callback queues
	can be provided using any number of schemes.

Perhaps most important, this variant of RCU does not affect the vanilla
flavors, rcu_preempt and rcu_sched.  The fact that RCU Tasks Trace
readers can operate from idle, offline, and exception entry/exit in no
way enables rcu_preempt and rcu_sched readers to do so.

The memory ordering was outlined here:
https://lore.kernel.org/lkml/20200319034030.GX3199@paulmck-ThinkPad-P72/

This effort benefited greatly from off-list discussions of BPF
requirements with Alexei Starovoitov and Andrii Nakryiko.  At least
some of the on-list discussions are captured in the Link: tags below.
In addition, KCSAN was quite helpful in finding some early bugs.

Link: https://lore.kernel.org/lkml/20200219150744.428764577@infradead.org/
Link: https://lore.kernel.org/lkml/87mu8p797b.fsf@nanos.tec.linutronix.de/
Link: https://lore.kernel.org/lkml/20200225221305.605144982@linutronix.de/
Cc: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Cc: Andrii Nakryiko <andriin@fb.com>
[ paulmck: Apply feedback from Steve Rostedt and Joel Fernandes. ]
[ paulmck: Decrement trc_n_readers_need_end upon IPI failure. ]
[ paulmck: Fix locking issue reported by rcutorture. ]
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
2020-04-27 11:03:51 -07:00
..
calibrate.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
do_mounts_initrd.c init: unify opening /dev/console as stdin/stdout/stderr 2019-12-12 18:58:24 +01:00
do_mounts_md.c init/: remove ineffective sparse disabling 2018-08-22 10:52:49 -07:00
do_mounts_rd.c init/: remove ineffective sparse disabling 2018-08-22 10:52:49 -07:00
do_mounts.c block: remove __bdevname 2020-03-24 07:57:07 -06:00
do_mounts.h fs: add do_mknodat() helper and ksys_mknod() wrapper; remove in-kernel calls to syscall 2018-04-02 20:15:56 +02:00
init_task.c rcu-tasks: Add an RCU Tasks Trace to simplify protection of tracing hooks 2020-04-27 11:03:51 -07:00
initramfs.c arm64: use generic free_initrd_mem() 2019-10-16 13:55:25 +01:00
Kconfig arm64 fixes: 2020-04-09 11:04:16 -07:00
main.c printk: queue wake_up_klogd irq_work only if per-CPU areas are ready 2020-04-10 13:18:57 -07:00
Makefile kbuild: do not pass $(KBUILD_CFLAGS) to scripts/mkcompile_h 2020-04-09 00:13:45 +09:00
noinitramfs.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 167 2019-05-30 11:26:39 -07:00
version.c treewide: Add SPDX license identifier for missed files 2019-05-21 10:50:45 +02:00