Pavel Begunkov 6f13860bba block: fix hctx checks for batch allocation ... [ Upstream commit 7746564793 ] When there are no read queues read requests will be assigned a default queue on allocation. However, blk_mq_get_cached_request() is not prepared for that and will fail all attempts to grab read requests from the cache. Worst case it doubles the number of requests allocated, roughly half of which will be returned by blk_mq_free_plug_rqs(). It only affects batched allocations and so is io_uring specific. For reference, QD8 t/io_uring benchmark improves by 20-35%. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/80d4511011d7d4751b4cf6375c4e38f237d935e3.1673955390.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Sasha Levin <sashal@kernel.org>		2023-02-06 08:06:33 +01:00
..
partitions	block: don't add partitions if GD_SUPPRESS_PART_SCAN is set	2022-09-03 11:29:03 -06:00
badblocks.c	block/badblocks: Remove redundant assignments	2022-04-23 07:15:26 -06:00
bdev.c	vfs: support STATX_DIOALIGN on block devices	2022-09-11 19:47:12 -05:00
bfq-cgroup.c	block, bfq: fix null pointer dereference in bfq_bio_bfqg()	2022-11-08 07:13:25 -07:00
bfq-iosched.c	block, bfq: fix uaf for bfqq in bfq_exit_icq_bfqq	2023-01-04 11:28:51 +01:00
bfq-iosched.h	block, bfq: remove unused variable for bfq_queue	2022-10-20 05:46:49 -07:00
bfq-wf2q.c	block, bfq: remove useless parameter for bfq_add/del_bfqq_busy()	2022-08-22 10:07:56 -06:00
bio-integrity.c	block: pass struct queue_limits to the bio splitting helpers	2022-08-02 21:08:53 -06:00
bio.c	block-6.1-2022-10-20	2022-10-21 15:14:14 -07:00
blk-cgroup-fc-appid.c	cgroup: Homogenize cgroup_get_from_id() return value	2022-08-26 10:57:41 -10:00
blk-cgroup-rwstat.c	blk-cgroup: Fix the recursive blkg rwstat	2021-03-05 11:32:15 -07:00
blk-cgroup-rwstat.h	block: Use the new blk_opf_t type	2022-07-14 12:14:30 -06:00
blk-cgroup.c	blk-cgroup: fix missing pd_online_fn() while activating policy	2023-02-06 08:06:32 +01:00
blk-cgroup.h	blk-cgroup: pass a gendisk to blkcg_init_queue and blkcg_exit_queue	2022-09-26 19:09:31 -06:00
blk-core.c	block: make dma_alignment a stacking queue_limit	2022-11-16 15:58:11 -07:00
blk-crypto-fallback.c	treewide: use get_random_bytes() when possible	2022-10-11 17:42:58 -06:00
blk-crypto-internal.h	blk-crypto: show crypto capabilities in sysfs	2022-02-28 06:40:23 -07:00
blk-crypto-profile.c	blk-crypto: remove blk_crypto_unregister()	2021-11-29 06:38:51 -07:00
blk-crypto-sysfs.c	blk-crypto: show crypto capabilities in sysfs	2022-02-28 06:40:23 -07:00
blk-crypto.c	blk-crypto: show crypto capabilities in sysfs	2022-02-28 06:40:23 -07:00
blk-flush.c	block: change request end_io handler to pass back a return value	2022-09-30 07:49:09 -06:00
blk-ia-ranges.c	block: simplify disk_set_independent_access_ranges	2022-06-29 08:36:46 -06:00
blk-integrity.c	blk-crypto: remove blk_crypto_unregister()	2021-11-29 06:38:51 -07:00
blk-ioc.c	block: fix default IO priority handling again	2022-06-27 06:29:12 -06:00
blk-iocost.c	blk-cgroup: pass a gendisk to blkcg_schedule_throttle	2022-09-26 19:17:28 -06:00
blk-iolatency.c	blk-cgroup: pass a gendisk to blkcg_schedule_throttle	2022-09-26 19:17:28 -06:00
blk-ioprio.c	blk-ioprio: pass a gendisk to blk_ioprio_init and blk_ioprio_exit	2022-09-26 19:09:31 -06:00
blk-ioprio.h	blk-ioprio: pass a gendisk to blk_ioprio_init and blk_ioprio_exit	2022-09-26 19:09:31 -06:00
blk-lib.c	blk-lib: fix blkdev_issue_secure_erase	2022-09-15 00:25:17 -06:00
blk-map.c	block: extend functionality to map bvec iterator	2022-09-30 07:51:13 -06:00
blk-merge.c	block: handle bio_split_to_limits() NULL return	2023-01-18 11:58:33 +01:00
blk-mq-cpumap.c	block: Change the return type of blk_mq_map_queues() into void	2022-08-22 10:07:53 -06:00
blk-mq-debugfs-zoned.c	block: move zone related fields to struct gendisk	2022-07-06 06:46:26 -06:00
blk-mq-debugfs.c	for-6.1/block-2022-10-03	2022-10-07 09:19:14 -07:00
blk-mq-debugfs.h	block: remove per-disk debugfs files in blk_unregister_queue	2022-06-17 07:31:05 -06:00
blk-mq-pci.c	block: Change the return type of blk_mq_map_queues() into void	2022-08-22 10:07:53 -06:00
blk-mq-rdma.c	block: Change the return type of blk_mq_map_queues() into void	2022-08-22 10:07:53 -06:00
blk-mq-sched.c	block: serialize all debugfs operations using q->debugfs_mutex	2022-06-17 07:31:05 -06:00
blk-mq-sched.h	block: move blk_mq_sched_assign_ioc to blk-ioc.c	2021-11-29 06:41:29 -07:00
blk-mq-sysfs.c	blk-mq: fix possible memleak when register 'hctx' failed	2022-12-31 13:33:03 +01:00
blk-mq-tag.c	sbitmap: fix batched wait_cnt accounting	2022-09-12 00:10:34 -06:00
blk-mq-tag.h	blk-mq: blk_mq_tag_busy is no need to return a value	2022-06-27 06:29:12 -06:00
blk-mq-virtio.c	block: Change the return type of blk_mq_map_queues() into void	2022-08-22 10:07:53 -06:00
blk-mq.c	block: fix hctx checks for batch allocation	2023-02-06 08:06:33 +01:00
blk-mq.h	block: adapt blk_mq_plug() to not plug for writes that require a zone lock	2022-09-29 07:45:47 -06:00
blk-pm.c	scsi: block: pm: Always set request queue runtime active in blk_post_runtime_resume()	2021-12-22 23:38:29 -05:00
blk-pm.h	block: Remove unused blk_pm_*() function definitions	2021-02-22 06:33:48 -07:00
blk-rq-qos.c	block/rq_qos: Use atomic_try_cmpxchg in atomic_inc_below	2022-07-12 14:38:52 -06:00
blk-rq-qos.h	block/blk-rq-qos: delete useless enmu RQ_QOS_IOPRIO	2022-09-21 19:50:53 -06:00
blk-settings.c	block: make blk_set_default_limits() private	2022-11-16 15:58:11 -07:00
blk-stat.c	block: make queue stat accounting a reference	2021-12-14 17:23:05 -07:00
blk-stat.h	block: make queue stat accounting a reference	2021-12-14 17:23:05 -07:00
blk-sysfs.c	blk-throttle: pass a gendisk to blk_throtl_register_queue	2022-09-26 19:17:27 -06:00
blk-throttle.c	blk-throttle: pass a gendisk to blk_throtl_cancel_bios	2022-09-26 19:17:28 -06:00
blk-throttle.h	blk-throttle: pass a gendisk to blk_throtl_cancel_bios	2022-09-26 19:17:28 -06:00
blk-timeout.c	block: blk-timeout: delete duplicated word	2020-07-31 16:29:47 -06:00
blk-wbt.c	blk-wbt: fix that 'rwb->wc' is always set to 1 in wbt_init()	2022-10-09 07:48:16 -06:00
blk-wbt.h	blk-wbt: remove wbt_track stub	2022-03-31 12:58:38 -06:00
blk-zoned.c	block: adapt blk_mq_plug() to not plug for writes that require a zone lock	2022-09-29 07:45:47 -06:00
blk.h	block: Do not reread partition table on exclusively open device	2023-01-04 11:29:01 +01:00
bounce.c	block: change the blk_queue_bounce calling convention	2022-08-02 17:22:54 -06:00
bsg-lib.c	blk-mq: Drop blk_mq_ops.timeout 'reserved' arg	2022-07-06 06:33:53 -06:00
bsg.c	scsi: core: bsg: Remove usage of the deprecated ida_simple_xxx() API	2022-06-21 21:22:51 -04:00
disk-events.c	block: remove genhd.h	2022-02-02 07:49:59 -07:00
elevator.c	blk-mq: use quiesced elevator switch when reinitializing queues	2022-09-27 09:58:56 -06:00
elevator.h	block: Use the new blk_opf_t type	2022-07-14 12:14:30 -06:00
fops.c	new iov_iter flavour - ITER_UBUF	2022-08-08 22:37:15 -04:00
genhd.c	block: Do not reread partition table on exclusively open device	2023-01-04 11:29:01 +01:00
holder.c	block: remove WARN_ON() from bd_link_disk_holder	2022-06-23 07:48:05 -06:00
ioctl.c	block: Do not reread partition table on exclusively open device	2023-01-04 11:29:01 +01:00
ioprio.c	block: Fix handling of tasks without ioprio in ioprio_get(2)	2022-06-27 06:29:12 -06:00
Kconfig	block: remove "select BLK_RQ_IO_DATA_LEN" from BLK_CGROUP_IOCOST dependency	2022-06-29 08:35:57 -06:00
Kconfig.iosched	block: only build the icq tracking code when needed	2021-12-16 10:59:02 -07:00
kyber-iosched.c	block/kyber: Use the new blk_opf_t type	2022-07-14 12:14:30 -06:00
Makefile	blk-cgroup: move blkcg_{get,set}_fc_appid out of line	2022-05-02 14:06:20 -06:00
mq-deadline.c	block: mq-deadline: Rename deadline_is_seq_writes()	2023-01-24 07:24:44 +01:00
opal_proto.h	block: sed-opal: Add ioctl to return device status	2022-08-22 07:52:51 -06:00
sed-opal.c	block: sed-opal: kmalloc the cmd/resp buffers	2022-11-08 07:14:35 -07:00
t10-pi.c	block: add pi for extended integrity	2022-03-07 12:48:35 -07:00