Waiman Long 9d230c0996 blk-cgroup: Properly propagate the iostat update up the hierarchy ... During a cgroup_rstat_flush() call, the lowest level of nodes are flushed first before their parents. Since commit 3b8cc62987 ("blk-cgroup: Optimize blkcg_rstat_flush()"), iostat propagation was still done to the parent. Grandparent, however, may not get the iostat update if the parent has no blkg_iostat_set queued in its lhead lockless list. Fix this iostat propagation problem by queuing the parent's global blkg->iostat into one of its percpu lockless lists to make sure that the delta will always be propagated up to the grandparent and so on toward the root blkcg. Note that successive calls to __blkcg_rstat_flush() are serialized by the cgroup_rstat_lock. So no special barrier is used in the reading and writing of blkg->iostat.lqueued. Fixes: 3b8cc62987 ("blk-cgroup: Optimize blkcg_rstat_flush()") Reported-by: Dan Schatzberg <schatzberg.dan@gmail.com> Closes: https://lore.kernel.org/lkml/ZkO6l%2FODzadSgdhC@dschatzberg-fedora-PF3DHTBV/ Signed-off-by: Waiman Long <longman@redhat.com> Reviewed-by: Ming Lei <ming.lei@redhat.com> Acked-by: Tejun Heo <tj@kernel.org> Link: https://lore.kernel.org/r/20240515143059.276677-1-longman@redhat.com Signed-off-by: Jens Axboe <axboe@kernel.dk>		2024-05-15 20:15:54 -06:00
..
partitions	block: fix and simplify blkdevparts= cmdline parsing	2024-05-03 09:57:53 -06:00
badblocks.c	badblocks: avoid checking invalid range in badblocks_check()	2023-12-23 18:38:08 -07:00
bdev.c	vfs-6.10.misc	2024-05-13 11:40:06 -07:00
bfq-cgroup.c	block: add blk_time_get_ns() and blk_time_get() helpers	2024-02-05 10:07:22 -07:00
bfq-iosched.c	block: add blk_time_get_ns() and blk_time_get() helpers	2024-02-05 10:07:22 -07:00
bfq-iosched.h	block, bfq: remove BFQ_WEIGHT_LEGACY_DFL	2023-04-06 16:17:32 -06:00
bfq-wf2q.c	block, bfq: inject I/O to underutilized actuators	2023-01-29 15:18:33 -07:00
bio-integrity.c	block: support PI at non-zero offset within metadata	2024-02-12 08:49:31 -07:00
bio.c	blk-throttle: remove CONFIG_BLK_DEV_THROTTLING_LOW	2024-05-09 09:44:55 -06:00
blk-cgroup-fc-appid.c	block: Replace all non-returning strlcpy with strscpy	2023-06-01 09:13:31 -06:00
blk-cgroup-rwstat.c	blk-cgroup: use group allocation/free of per-cpu counters API	2024-04-03 09:10:17 -06:00
blk-cgroup-rwstat.h	block: Use the new blk_opf_t type	2022-07-14 12:14:30 -06:00
blk-cgroup.c	blk-cgroup: Properly propagate the iostat update up the hierarchy	2024-05-15 20:15:54 -06:00
blk-cgroup.h	block: fix q->blkg_list corruption during disk rebind	2024-04-07 15:50:13 -06:00
blk-core.c	for-6.10/block-20240511	2024-05-13 13:03:54 -07:00
blk-crypto-fallback.c	block, fs: Restore the per-bio/request data lifetime fields	2024-02-06 14:31:05 +01:00
blk-crypto-internal.h	blk-crypto: remove blk_crypto_insert_cloned_request()	2023-03-16 09:35:09 -06:00
blk-crypto-profile.c	blk-crypto: use dynamic lock class for blk_crypto_profile::lock	2023-07-05 16:36:12 -06:00
blk-crypto-sysfs.c	block: make kobj_type structures constant	2023-02-09 09:38:16 -07:00
blk-crypto.c	blk-crypto: make blk_crypto_evict_key() more robust	2023-03-16 09:35:09 -06:00
blk-flush.c	block: Fix flush request sector restore	2024-05-01 08:08:43 -06:00
blk-ia-ranges.c	block: make kobj_type structures constant	2023-02-09 09:38:16 -07:00
blk-integrity.c	block: support PI at non-zero offset within metadata	2024-02-12 08:49:31 -07:00
blk-ioc.c	blk-ioc: fix recursive spin_lock/unlock_irq() in ioc_clear_queue()	2023-06-07 07:51:00 -06:00
blk-iocost.c	blk-iocost: do not WARN if iocg was already offlined	2024-04-19 08:06:24 -06:00
blk-iolatency.c	block: add blk_time_get_ns() and blk_time_get() helpers	2024-02-05 10:07:22 -07:00
blk-ioprio.c	blk-ioprio: Introduce promote-to-rt policy	2023-06-06 22:26:26 -06:00
blk-ioprio.h	blk-ioprio: pass a gendisk to blk_ioprio_init and blk_ioprio_exit	2022-09-26 19:09:31 -06:00
blk-lib.c	block: add a blk_alloc_discard_bio helper	2024-05-07 07:29:42 -06:00
blk-map.c	block: Fix WARNING in _copy_from_iter	2024-01-23 08:56:55 -07:00
blk-merge.c	block: support to account io_ticks precisely	2024-05-09 07:59:44 -06:00
blk-mq-cpumap.c	blk-mq: include <linux/blk-mq.h> in block/blk-mq.h	2023-04-13 06:52:29 -06:00
blk-mq-debugfs.c	block: Remove zone write locking	2024-04-17 08:44:03 -06:00
blk-mq-debugfs.h	block: Replace zone_wlock debugfs entry with zone_wplugs entry	2024-04-17 08:44:03 -06:00
blk-mq-pci.c	blk-mq: include <linux/blk-mq.h> in block/blk-mq.h	2023-04-13 06:52:29 -06:00
blk-mq-sched.c	blk-mq: Remove the hctx 'run' debugfs attribute	2024-01-17 14:16:34 -07:00
blk-mq-sched.h	blk-mq: make sure elevator callbacks aren't called for passthrough request	2023-05-18 19:42:54 -06:00
blk-mq-sysfs.c	blk-mq: include <linux/blk-mq.h> in block/blk-mq.h	2023-04-13 06:52:29 -06:00
blk-mq-tag.c	for-6.5/block-2023-06-23	2023-06-26 12:47:20 -07:00
blk-mq-virtio.c	blk-mq: include <linux/blk-mq.h> in block/blk-mq.h	2023-04-13 06:52:29 -06:00
blk-mq.c	block: support to account io_ticks precisely	2024-05-09 07:59:44 -06:00
blk-mq.h	block: Do not special-case plugging of zone write operations	2024-04-17 08:44:03 -06:00
blk-pm.c	block: Remove blk_set_runtime_active()	2023-11-20 10:22:40 -07:00
blk-pm.h	block: Remove unused blk_pm_*() function definitions	2021-02-22 06:33:48 -07:00
blk-rq-qos.c	block: correct stale comment in rq_qos_wait	2023-09-18 14:15:28 -06:00
blk-rq-qos.h	block: skip QUEUE_FLAG_STATS and rq-qos for passthrough io	2023-12-01 18:29:18 -07:00
blk-settings.c	for-6.10/block-20240511	2024-05-13 13:03:54 -07:00
blk-stat.c	blk-throttle: remove CONFIG_BLK_DEV_THROTTLING_LOW	2024-05-09 09:44:55 -06:00
blk-stat.h	block: make queue stat accounting a reference	2021-12-14 17:23:05 -07:00
blk-sysfs.c	blk-throttle: delay initialization until configuration	2024-05-09 09:44:56 -06:00
blk-throttle.c	blk-throttle: delay initialization until configuration	2024-05-09 09:44:56 -06:00
blk-throttle.h	blk-throttle: delay initialization until configuration	2024-05-09 09:44:56 -06:00
blk-timeout.c	block: blk-timeout: delete duplicated word	2020-07-31 16:29:47 -06:00
blk-wbt.c	for-6.9/block-20240310	2024-03-11 11:43:44 -07:00
blk-wbt.h	blk-wbt: remove the separate write cache tracking	2023-12-26 09:28:10 -07:00
blk-zoned.c	block: Cleanup blk_revalidate_zone_cb()	2024-05-01 08:08:43 -06:00
blk.h	blk-throttle: remove CONFIG_BLK_DEV_THROTTLING_LOW	2024-05-09 09:44:55 -06:00
bounce.c	block, fs: Restore the per-bio/request data lifetime fields	2024-02-06 14:31:05 +01:00
bsg-lib.c	block: pass a queue_limits argument to blk_mq_init_queue	2024-02-13 08:56:59 -07:00
bsg.c	SCSI misc on 20230629	2023-06-30 11:57:07 -07:00
disk-events.c	block: move bdev_mark_dead out of disk_check_media_change	2023-10-28 13:29:23 +02:00
early-lookup.c	block: don't return -EINVAL for not found names in devt_from_devname	2023-06-22 09:09:33 -06:00
elevator.c	block: Remove elevator required features	2024-04-17 08:44:03 -06:00
elevator.h	block: Remove elevator required features	2024-04-17 08:44:03 -06:00
fops.c	for-6.10/block-20240511	2024-05-13 13:03:54 -07:00
genhd.c	block: fix that util can be greater than 100%	2024-05-09 07:59:44 -06:00
holder.c	block: fix deadlock between bd_link_disk_holder and partition scan	2024-02-23 07:44:19 -07:00
ioctl.c	for-6.10/block-20240511	2024-05-13 13:03:54 -07:00
ioprio.c	block: move __get_task_ioprio() into header file	2024-01-08 12:27:39 -07:00
Kconfig	blk-throttle: remove CONFIG_BLK_DEV_THROTTLING_LOW	2024-05-09 09:44:55 -06:00
Kconfig.iosched	block: Default to use cgroup support for BFQ	2023-01-30 09:42:42 -07:00
kyber-iosched.c	blk-mq: pass a flags argument to elevator_type->insert_requests	2023-04-13 06:52:30 -06:00
Makefile	block: Move zone related debugfs attribute to blk-zoned.c	2024-04-17 08:44:03 -06:00
mq-deadline.c	block/mq-deadline: Remove some unused functions	2024-04-19 08:10:36 -06:00
opal_proto.h	block: sed-opal: handle empty atoms when parsing response	2024-02-16 15:52:45 -07:00
sed-opal.c	for-6.9/block-20240310	2024-03-11 11:43:44 -07:00
t10-pi.c	block: support PI at non-zero offset within metadata	2024-02-12 08:49:31 -07:00