John Garry e155b0c238 blk-mq: Use shared tags for shared sbitmap support ... Currently we use separate sbitmap pairs and active_queues atomic_t for shared sbitmap support. However a full sets of static requests are used per HW queue, which is quite wasteful, considering that the total number of requests usable at any given time across all HW queues is limited by the shared sbitmap depth. As such, it is considerably more memory efficient in the case of shared sbitmap to allocate a set of static rqs per tag set or request queue, and not per HW queue. So replace the sbitmap pairs and active_queues atomic_t with a shared tags per tagset and request queue, which will hold a set of shared static rqs. Since there is now no valid HW queue index to be passed to the blk_mq_ops .init and .exit_request callbacks, pass an invalid index token. This changes the semantics of the APIs, such that the callback would need to validate the HW queue index before using it. Currently no user of shared sbitmap actually uses the HW queue index (as would be expected). Signed-off-by: John Garry <john.garry@huawei.com> Reviewed-by: Ming Lei <ming.lei@redhat.com> Link: https://lore.kernel.org/r/1633429419-228500-13-git-send-email-john.garry@huawei.com Signed-off-by: Jens Axboe <axboe@kernel.dk>		2021-10-18 06:17:03 -06:00
..
partitions	block: drop unused includes in <linux/genhd.h>	2021-10-18 06:17:02 -06:00
badblocks.c	treewide: Use fallthrough pseudo-keyword	2020-08-23 17:36:59 -05:00
bdev.c	block: move integrity handling out of <linux/blkdev.h>	2021-10-18 06:17:02 -06:00
bfq-cgroup.c	block: move elevator.h to block/	2021-10-18 06:17:01 -06:00
bfq-iosched.c	block: move elevator.h to block/	2021-10-18 06:17:01 -06:00
bfq-iosched.h	block, bfq: cleanup the repeated declaration	2021-08-25 06:45:33 -06:00
bfq-wf2q.c	block: Introduce IOPRIO_NR_LEVELS	2021-08-18 07:21:12 -06:00
bio-integrity.c	block: move integrity handling out of <linux/blkdev.h>	2021-10-18 06:17:02 -06:00
bio.c	block: don't call rq_qos_ops->done_bio if the bio isn't tracked	2021-09-24 11:04:39 -06:00
blk-cgroup-rwstat.c	blk-cgroup: Fix the recursive blkg rwstat	2021-03-05 11:32:15 -07:00
blk-cgroup-rwstat.h	blk-cgroup: separate out blkg_rwstat under CONFIG_BLK_CGROUP_RWSTAT	2019-11-07 12:28:13 -07:00
blk-cgroup.c	blk-cgroup: blk_cgroup_bio_start() should use irq-safe operations on blkg->iostat_cpu	2021-10-18 06:17:01 -06:00
blk-core.c	blk-mq-sched: Rename blk_mq_sched_free_{requests -> rqs}()	2021-10-18 06:17:02 -06:00
blk-crypto-fallback.c	block: move struct request to blk-mq.h	2021-10-18 06:17:02 -06:00
blk-crypto-internal.h	block: move struct request to blk-mq.h	2021-10-18 06:17:02 -06:00
blk-crypto.c	blk-crypto: fix check for too-large dun_bytes	2021-08-25 06:45:00 -06:00
blk-exec.c	block: return errors from blk_execute_rq()	2021-06-30 15:35:45 -06:00
blk-flush.c	blk-mq: fix is_flush_rq	2021-08-17 20:17:34 -06:00
blk-integrity.c	block: move integrity handling out of <linux/blkdev.h>	2021-10-18 06:17:02 -06:00
blk-ioc.c	block: remove retry loop in ioc_release_fn()	2020-07-16 10:22:15 -06:00
blk-iocost.c	for-5.15/block-2021-08-30	2021-08-30 18:52:11 -07:00
blk-iolatency.c	mm: don't include <linux/blk-cgroup.h> in <linux/backing-dev.h>	2021-10-18 06:17:01 -06:00
blk-ioprio.c	block: Introduce the ioprio rq-qos policy	2021-06-21 15:03:40 -06:00
blk-ioprio.h	block: Introduce the ioprio rq-qos policy	2021-06-21 15:03:40 -06:00
blk-lib.c	block: export blk_next_bio()	2021-06-17 15:51:20 +02:00
blk-map.c	Merge branch 'akpm' (patches from Andrew)	2021-09-03 10:08:28 -07:00
blk-merge.c	block: move integrity handling out of <linux/blkdev.h>	2021-10-18 06:17:02 -06:00
blk-mq-cpumap.c	blk-mq: remove the calling of local_memory_node()	2020-10-20 07:08:17 -06:00
blk-mq-debugfs-zoned.c	block: Cleanup license notice	2019-01-17 21:21:40 -07:00
blk-mq-debugfs.c	block: decode QUEUE_FLAG_HCTX_ACTIVE in debugfs output	2021-10-04 06:58:39 -06:00
blk-mq-debugfs.h	blk-mq: no need to check return value of debugfs_create functions	2019-06-13 03:00:30 -06:00
blk-mq-pci.c	block: Fix blk_mq_*_map_queues() kernel-doc headers	2019-05-31 15:12:34 -06:00
blk-mq-rdma.c	block: Fix blk_mq_*_map_queues() kernel-doc headers	2019-05-31 15:12:34 -06:00
blk-mq-sched.c	blk-mq: Use shared tags for shared sbitmap support	2021-10-18 06:17:03 -06:00
blk-mq-sched.h	blk-mq-sched: Rename blk_mq_sched_free_{requests -> rqs}()	2021-10-18 06:17:02 -06:00
blk-mq-sysfs.c	block: remove blk-mq-sysfs dead code	2021-08-02 13:37:29 -06:00
blk-mq-tag.c	blk-mq: Use shared tags for shared sbitmap support	2021-10-18 06:17:03 -06:00
blk-mq-tag.h	blk-mq: Use shared tags for shared sbitmap support	2021-10-18 06:17:03 -06:00
blk-mq-virtio.c	blk-mq: Fix typo in comment	2020-03-17 20:55:21 +01:00
blk-mq.c	blk-mq: Use shared tags for shared sbitmap support	2021-10-18 06:17:03 -06:00
blk-mq.h	blk-mq: Use shared tags for shared sbitmap support	2021-10-18 06:17:03 -06:00
blk-pm.c	scsi: block: Fix a race in the runtime power management code	2020-12-09 11:41:41 -05:00
blk-pm.h	block: Remove unused blk_pm_*() function definitions	2021-02-22 06:33:48 -07:00
blk-rq-qos.c	rq-qos: fix missed wake-ups in rq_qos_throttle try two	2021-06-08 15:12:57 -06:00
blk-rq-qos.h	block: Introduce the ioprio rq-qos policy	2021-06-21 15:03:40 -06:00
blk-settings.c	block: add an explicit ->disk backpointer to the request_queue	2021-08-23 12:54:31 -06:00
blk-stat.c	blk-stat: make q->stats->lock irqsafe	2020-09-01 16:48:46 -06:00
blk-stat.h	block: deactivate blk_stat timer in wbt_disable_default()	2018-12-12 06:47:51 -07:00
blk-sysfs.c	block: call blk_register_queue earlier in device_add_disk	2021-08-23 12:55:45 -06:00
blk-throttle.c	blk-throttle: fix UAF by deleteing timer in blk_throtl_exit()	2021-09-07 08:36:56 -06:00
blk-timeout.c	block: blk-timeout: delete duplicated word	2020-07-31 16:29:47 -06:00
blk-wbt.c	block: add an explicit ->disk backpointer to the request_queue	2021-08-23 12:54:31 -06:00
blk-wbt.h	blk-wbt: introduce a new disable state to prevent false positive by rwb_enabled()	2021-06-21 15:03:41 -06:00
blk-zoned.c	blk-zoned: allow BLKREPORTZONE without CAP_SYS_ADMIN	2021-08-24 10:12:36 -06:00
blk.h	blk-mq-sched: Rename blk_mq_sched_free_{requests -> rqs}()	2021-10-18 06:17:02 -06:00
bounce.c	mm: don't include <linux/blk-cgroup.h> in <linux/backing-dev.h>	2021-10-18 06:17:01 -06:00
bsg-lib.c	scsi: bsg-lib: Fix commands without data transfer in bsg_transport_sg_io_fn()	2021-08-01 13:21:40 -04:00
bsg.c	scsi: bsg: Fix device unregistration	2021-09-14 00:22:15 -04:00
disk-events.c	block: return errors from disk_alloc_events	2021-08-23 12:55:45 -06:00
elevator.c	block: move elevator.h to block/	2021-10-18 06:17:01 -06:00
elevator.h	block: move elevator.h to block/	2021-10-18 06:17:01 -06:00
fops.c	block: hold ->invalidate_lock in blkdev_fallocate	2021-09-24 11:06:58 -06:00
genhd.c	block: drop unused includes in <linux/genhd.h>	2021-10-18 06:17:02 -06:00
holder.c	block: drop unused includes in <linux/genhd.h>	2021-10-18 06:17:02 -06:00
ioctl.c	block: pass a gendisk to bdev_resize_partition	2021-08-12 10:31:36 -06:00
ioprio.c	block: fix default IO priority handling	2021-08-18 07:23:15 -06:00
Kconfig	SCSI misc on 20210902	2021-09-02 15:09:46 -07:00
Kconfig.iosched	Revert "block/mq-deadline: Add cgroup support"	2021-08-11 13:47:26 -06:00
keyslot-manager.c	block: move integrity handling out of <linux/blkdev.h>	2021-10-18 06:17:02 -06:00
kyber-iosched.c	block: move elevator.h to block/	2021-10-18 06:17:01 -06:00
Makefile	block-5.15-2021-09-11	2021-09-11 10:19:51 -07:00
mq-deadline.c	block/mq-deadline: Prioritize high-priority requests	2021-10-18 06:17:02 -06:00
opal_proto.h	block: sed-opal: Change the check condition for regular session validity	2020-03-12 08:00:10 -06:00
sed-opal.c	block: sed-opal: Change the check condition for regular session validity	2020-03-12 08:00:10 -06:00
t10-pi.c	block: move integrity handling out of <linux/blkdev.h>	2021-10-18 06:17:02 -06:00