Waiman Long b1bee99312 blk-cgroup: Properly propagate the iostat update up the hierarchy ... [ Upstream commit 9d230c0996 ] During a cgroup_rstat_flush() call, the lowest level of nodes are flushed first before their parents. Since commit 3b8cc62987 ("blk-cgroup: Optimize blkcg_rstat_flush()"), iostat propagation was still done to the parent. Grandparent, however, may not get the iostat update if the parent has no blkg_iostat_set queued in its lhead lockless list. Fix this iostat propagation problem by queuing the parent's global blkg->iostat into one of its percpu lockless lists to make sure that the delta will always be propagated up to the grandparent and so on toward the root blkcg. Note that successive calls to __blkcg_rstat_flush() are serialized by the cgroup_rstat_lock. So no special barrier is used in the reading and writing of blkg->iostat.lqueued. Fixes: 3b8cc62987 ("blk-cgroup: Optimize blkcg_rstat_flush()") Reported-by: Dan Schatzberg <schatzberg.dan@gmail.com> Closes: https://lore.kernel.org/lkml/ZkO6l%2FODzadSgdhC@dschatzberg-fedora-PF3DHTBV/ Signed-off-by: Waiman Long <longman@redhat.com> Reviewed-by: Ming Lei <ming.lei@redhat.com> Acked-by: Tejun Heo <tj@kernel.org> Link: https://lore.kernel.org/r/20240515143059.276677-1-longman@redhat.com Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Sasha Levin <sashal@kernel.org>		2024-06-12 11:12:46 +02:00
..
partitions	block: fix and simplify blkdevparts= cmdline parsing	2024-06-12 11:11:35 +02:00
badblocks.c	block/badblocks: Remove redundant assignments	2022-04-23 07:15:26 -06:00
bdev.c	block: Provide bdev_open_* functions	2024-03-26 18:19:40 -04:00
bfq-cgroup.c	blkcg: Restructure blkg_conf_prep() and friends	2023-04-13 06:46:49 -06:00
bfq-iosched.c	SCSI misc on 20230629	2023-06-30 11:57:07 -07:00
bfq-iosched.h	block, bfq: remove BFQ_WEIGHT_LEGACY_DFL	2023-04-06 16:17:32 -06:00
bfq-wf2q.c	block, bfq: inject I/O to underutilized actuators	2023-01-29 15:18:33 -07:00
bio-integrity.c	bio-integrity: create multi-page bvecs in bio_integrity_add_page()	2023-08-09 16:05:35 -06:00
bio.c	block: Fix page refcounts for unaligned buffers in __bio_release_pages()	2024-04-03 15:28:27 +02:00
blk-cgroup-fc-appid.c	block: Replace all non-returning strlcpy with strscpy	2023-06-01 09:13:31 -06:00
blk-cgroup-rwstat.c	Revert "blk-cgroup: pin the gendisk in struct blkcg_gq"	2023-02-14 14:24:09 -07:00
blk-cgroup-rwstat.h	block: Use the new blk_opf_t type	2022-07-14 12:14:30 -06:00
blk-cgroup.c	blk-cgroup: Properly propagate the iostat update up the hierarchy	2024-06-12 11:12:46 +02:00
blk-cgroup.h	block: fix q->blkg_list corruption during disk rebind	2024-04-17 11:19:28 +02:00
blk-core.c	block: support to account io_ticks precisely	2024-06-12 11:11:35 +02:00
blk-crypto-fallback.c	blk-crypto: dynamically allocate fallback profile	2023-08-18 15:00:39 -06:00
blk-crypto-internal.h	blk-crypto: remove blk_crypto_insert_cloned_request()	2023-03-16 09:35:09 -06:00
blk-crypto-profile.c	blk-crypto: use dynamic lock class for blk_crypto_profile::lock	2023-07-05 16:36:12 -06:00
blk-crypto-sysfs.c	block: make kobj_type structures constant	2023-02-09 09:38:16 -07:00
blk-crypto.c	blk-crypto: make blk_crypto_evict_key() more robust	2023-03-16 09:35:09 -06:00
blk-flush.c	blk-flush: reuse rq queuelist in flush state machine	2023-07-17 08:18:21 -06:00
blk-ia-ranges.c	block: make kobj_type structures constant	2023-02-09 09:38:16 -07:00
blk-integrity.c	blk-integrity: register sysfs attributes on struct device	2023-04-26 18:22:50 -06:00
blk-ioc.c	blk-ioc: fix recursive spin_lock/unlock_irq() in ioc_clear_queue()	2023-06-07 07:51:00 -06:00
blk-iocost.c	blk-iocost: do not WARN if iocg was already offlined	2024-05-17 12:02:20 +02:00
blk-iolatency.c	block: fix bad lockdep annotation in blk-iolatency	2023-08-10 17:24:53 -06:00
blk-ioprio.c	blk-ioprio: Introduce promote-to-rt policy	2023-06-06 22:26:26 -06:00
blk-ioprio.h	blk-ioprio: pass a gendisk to blk_ioprio_init and blk_ioprio_exit	2022-09-26 19:09:31 -06:00
blk-lib.c	blk-lib: fix blkdev_issue_secure_erase	2022-09-15 00:25:17 -06:00
blk-map.c	block: Fix WARNING in _copy_from_iter	2024-03-01 13:34:49 +01:00
blk-merge.c	block: support to account io_ticks precisely	2024-06-12 11:11:35 +02:00
blk-mq-cpumap.c	blk-mq: include <linux/blk-mq.h> in block/blk-mq.h	2023-04-13 06:52:29 -06:00
blk-mq-debugfs-zoned.c	block: move zone related fields to struct gendisk	2022-07-06 06:46:26 -06:00
blk-mq-debugfs.c	blk-mq: fix potential io hang by wrong 'wake_batch'	2023-06-12 09:55:53 -06:00
blk-mq-debugfs.h	block: remove per-disk debugfs files in blk_unregister_queue	2022-06-17 07:31:05 -06:00
blk-mq-pci.c	blk-mq: include <linux/blk-mq.h> in block/blk-mq.h	2023-04-13 06:52:29 -06:00
blk-mq-sched.c	blk-mq: cleanup __blk_mq_sched_dispatch_requests	2023-04-13 06:57:18 -06:00
blk-mq-sched.h	blk-mq: make sure elevator callbacks aren't called for passthrough request	2023-05-18 19:42:54 -06:00
blk-mq-sysfs.c	blk-mq: include <linux/blk-mq.h> in block/blk-mq.h	2023-04-13 06:52:29 -06:00
blk-mq-tag.c	for-6.5/block-2023-06-23	2023-06-26 12:47:20 -07:00
blk-mq-virtio.c	blk-mq: include <linux/blk-mq.h> in block/blk-mq.h	2023-04-13 06:52:29 -06:00
blk-mq.c	block: support to account io_ticks precisely	2024-06-12 11:11:35 +02:00
blk-mq.h	blk-mq: fix potential io hang by wrong 'wake_batch'	2023-06-12 09:55:53 -06:00
blk-pm.c	blk-mq: include <linux/blk-mq.h> in block/blk-mq.h	2023-04-13 06:52:29 -06:00
blk-pm.h
blk-rq-qos.c	block: correct stale comment in rq_qos_wait	2023-09-18 14:15:28 -06:00
blk-rq-qos.h	blk-iolatency: s/blkcg_rq_qos/iolat_rq_qos/	2023-04-13 06:46:49 -06:00
blk-settings.c	block: Clear zone limits for a non-zoned stacked queue	2024-04-03 15:28:20 +02:00
blk-stat.c	block: prevent division by zero in blk_rq_stat_sum()	2024-04-13 13:07:37 +02:00
blk-stat.h	block: make queue stat accounting a reference	2021-12-14 17:23:05 -07:00
blk-sysfs.c	block: don't allow enabling a cache on devices that don't support it	2023-07-17 08:18:18 -06:00
blk-throttle.c	blk-throttle: fix lockdep warning of "cgroup_mutex or RCU read lock required!"	2023-12-20 17:01:55 +01:00
blk-throttle.h	blk-throttle: print signed value 'carryover_bytes/ios' for user	2023-08-30 10:15:01 -06:00
blk-timeout.c
blk-wbt.c	blk-wbt: Fix detection of dirty-throttled tasks	2024-02-23 09:25:16 +01:00
blk-wbt.h	blk-wbt: don't create wbt sysfs entry if CONFIG_BLK_WBT is disabled	2023-06-26 09:53:36 -06:00
blk-zoned.c	Merge branch '6.5/scsi-staging' into 6.5/scsi-fixes	2023-07-11 12:15:15 -04:00
blk.h	block: support to account io_ticks precisely	2024-06-12 11:11:35 +02:00
bounce.c	block: change the blk_queue_bounce calling convention	2022-08-02 17:22:54 -06:00
bsg-lib.c	scsi: replace the fmode_t argument to ->sg_io_fn with a simple bool	2023-06-12 08:04:04 -06:00
bsg.c	SCSI misc on 20230629	2023-06-30 11:57:07 -07:00
disk-events.c	block: fix kernel-doc for disk_force_media_change()	2023-09-26 00:43:34 -06:00
early-lookup.c	block: don't return -EINVAL for not found names in devt_from_devname	2023-06-22 09:09:33 -06:00
elevator.c	blk-mq: release scheduler resource when request completes	2023-08-19 07:47:17 -06:00
elevator.h	blk-mq: pass a flags argument to elevator_type->insert_requests	2023-04-13 06:52:30 -06:00
fops.c	block: refine the EOF check in blkdev_iomap_begin	2024-06-12 11:11:35 +02:00
genhd.c	block: support to account io_ticks precisely	2024-06-12 11:11:35 +02:00
holder.c	block: don't allow a disk link holder to itself	2022-11-16 15:19:56 -07:00
ioctl.c	block: fix overflow in blk_ioctl_discard()	2024-05-17 12:02:13 +02:00
ioprio.c	scsi: block: Improve ioprio value validity checks	2023-06-16 12:04:30 -04:00
Kconfig	block: sed-opal: keyring support for SED keys	2023-08-22 11:10:26 -06:00
Kconfig.iosched	block: Default to use cgroup support for BFQ	2023-01-30 09:42:42 -07:00
kyber-iosched.c	blk-mq: pass a flags argument to elevator_type->insert_requests	2023-04-13 06:52:30 -06:00
Makefile	block: move the code to do early boot lookup of block devices to block/	2023-06-05 10:57:40 -06:00
mq-deadline.c	Revert "block/mq-deadline: use correct way to throttling write requests"	2024-04-03 15:28:33 +02:00
opal_proto.h	block: sed-opal: handle empty atoms when parsing response	2024-03-26 18:19:12 -04:00
sed-opal.c	block: sed-opal: handle empty atoms when parsing response	2024-03-26 18:19:12 -04:00
t10-pi.c	block: add pi for extended integrity	2022-03-07 12:48:35 -07:00