Tianchen Ding 28156108fe sched: Fix the check of nr_running at queue wakelist ... The commit 2ebb177175 ("sched/core: Offload wakee task activation if it the wakee is descheduling") checked rq->nr_running <= 1 to avoid task stacking when WF_ON_CPU. Per the ordering of writes to p->on_rq and p->on_cpu, observing p->on_cpu (WF_ON_CPU) in ttwu_queue_cond() implies !p->on_rq, IOW p has gone through the deactivate_task() in __schedule(), thus p has been accounted out of rq->nr_running. As such, the task being the only runnable task on the rq implies reading rq->nr_running == 0 at that point. The benchmark result is in [1]. [1] https://lore.kernel.org/all/e34de686-4e85-bde1-9f3c-9bbc86b38627@linux.alibaba.com/ Suggested-by: Valentin Schneider <vschneid@redhat.com> Signed-off-by: Tianchen Ding <dtcccc@linux.alibaba.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Reviewed-by: Valentin Schneider <vschneid@redhat.com> Link: https://lore.kernel.org/r/20220608233412.327341-2-dtcccc@linux.alibaba.com		2022-06-13 10:30:01 +02:00
..
autogroup.c	sched/autogroup: Fix sysctl move	2022-05-30 12:36:36 +02:00
autogroup.h	sched/headers: Add header guard to kernel/sched/stats.h and kernel/sched/autogroup.h	2022-02-23 08:22:00 +01:00
build_policy.c	sched: Fix missing prototype warnings	2022-05-01 10:03:43 +02:00
build_utility.c	sched: Fix missing prototype warnings	2022-05-01 10:03:43 +02:00
clock.c	sched/clock: Use try_cmpxchg64 in sched_clock_{local,remote}	2022-05-19 23:46:09 +02:00
completion.c	sched/headers: Introduce kernel/sched/build_utility.c and build multiple .c files there	2022-02-23 10:58:33 +01:00
core_sched.c	sched/headers: Introduce kernel/sched/build_utility.c and build multiple .c files there	2022-02-23 10:58:33 +01:00
core.c	sched: Fix the check of nr_running at queue wakelist	2022-06-13 10:30:01 +02:00
cpuacct.c	Merge branch 'sched/fast-headers' into sched/core	2022-03-15 09:05:05 +01:00
cpudeadline.c	sched/headers: Introduce kernel/sched/build_policy.c and build multiple .c files there	2022-02-23 10:58:33 +01:00
cpudeadline.h
cpufreq_schedutil.c	Changes in this cycle were:	2022-03-22 14:39:12 -07:00
cpufreq.c	sched/headers: Introduce kernel/sched/build_utility.c and build multiple .c files there	2022-02-23 10:58:33 +01:00
cpupri.c	sched/headers: Introduce kernel/sched/build_utility.c and build multiple .c files there	2022-02-23 10:58:33 +01:00
cpupri.h	sched/cpupri: Add CPUPRI_HIGHER	2020-10-29 11:00:30 +01:00
cputime.c	sched/headers: Introduce kernel/sched/build_policy.c and build multiple .c files there	2022-02-23 10:58:33 +01:00
deadline.c	sched/deadline: Use proc_douintvec_minmax() limit minimum value	2022-06-13 10:30:00 +02:00
debug.c	sched/headers: Introduce kernel/sched/build_utility.c and build multiple .c files there	2022-02-23 10:58:33 +01:00
fair.c	sched: Allow newidle balancing to bail out of load_balance	2022-06-13 10:30:01 +02:00
features.h	sched: Disable TTWU_QUEUE on RT	2021-10-05 15:52:12 +02:00
idle.c	Scheduler changes in this cycle were:	2022-05-24 11:11:13 -07:00
isolation.c	sched/headers: Introduce kernel/sched/build_utility.c and build multiple .c files there	2022-02-23 10:58:33 +01:00
loadavg.c	sched/headers: Introduce kernel/sched/build_utility.c and build multiple .c files there	2022-02-23 10:58:33 +01:00
Makefile	sched/headers: Introduce kernel/sched/build_policy.c and build multiple .c files there	2022-02-23 10:58:33 +01:00
membarrier.c	sched/headers: Introduce kernel/sched/build_utility.c and build multiple .c files there	2022-02-23 10:58:33 +01:00
pelt.c	sched/headers: Introduce kernel/sched/build_policy.c and build multiple .c files there	2022-02-23 10:58:33 +01:00
pelt.h	sched/fair: Fix cfs_rq_clock_pelt() for throttled cfs_rq	2022-04-22 12:14:07 +02:00
psi.c	sched/psi: report zeroes for CPU full at the system level	2022-04-22 12:14:08 +02:00
rt.c	sysctl changes for v5.19-rc1	2022-05-26 16:57:20 -07:00
sched-pelt.h	sched/fair: Fix "runnable_avg_yN_inv" not used warnings	2019-06-17 12:15:58 +02:00
sched.h	sysctl changes for v5.19-rc1	2022-05-26 16:57:20 -07:00
smp.h	smp: Rename flush_smp_call_function_from_idle()	2022-05-01 10:03:43 +02:00
stats.c	sched/headers: Introduce kernel/sched/build_utility.c and build multiple .c files there	2022-02-23 10:58:33 +01:00
stats.h	sched/headers: Reorganize, clean up and optimize kernel/sched/sched.h dependencies	2022-02-23 10:58:34 +01:00
stop_task.c	sched/headers: Introduce kernel/sched/build_utility.c and build multiple .c files there	2022-02-23 10:58:33 +01:00
swait.c	sched/headers: Introduce kernel/sched/build_utility.c and build multiple .c files there	2022-02-23 10:58:33 +01:00
topology.c	sched/numa: Adjust imb_numa_nr to a better approximation of memory channels	2022-06-13 10:30:00 +02:00
wait_bit.c	sched/headers: Introduce kernel/sched/build_utility.c and build multiple .c files there	2022-02-23 10:58:33 +01:00
wait.c	sched/headers: Introduce kernel/sched/build_utility.c and build multiple .c files there	2022-02-23 10:58:33 +01:00