Jens Axboe 47c122e35d block: pre-allocate requests if plug is started and is a batch ... The caller typically has a good (or even exact) idea of how many requests it needs to submit. We can make the request/tag allocation a lot more efficient if we just allocate N requests/tags upfront when we queue the first bio from the batch. Provide a new plug start helper that allows the caller to specify how many IOs are expected. This sets plug->nr_ios, and we can use that for smarter request allocation. The plug provides a holding spot for requests, and request allocation will check it before calling into the normal request allocation path. The blk_finish_plug() is called, check if there are unused requests and free them. This should not happen in normal operations. The exception is if we get merging, then we may be left with requests that need freeing when done. This raises the per-core performance on my setup from ~5.8M to ~6.1M IOPS. Signed-off-by: Jens Axboe <axboe@kernel.dk>		2021-10-18 06:17:03 -06:00
..
partitions	block: drop unused includes in <linux/genhd.h>	2021-10-18 06:17:02 -06:00
badblocks.c	treewide: Use fallthrough pseudo-keyword	2020-08-23 17:36:59 -05:00
bdev.c	block: move integrity handling out of <linux/blkdev.h>	2021-10-18 06:17:02 -06:00
bfq-cgroup.c	block: move elevator.h to block/	2021-10-18 06:17:01 -06:00
bfq-iosched.c	blk-mq: Stop using pointers for blk_mq_tags bitmap tags	2021-10-18 06:17:03 -06:00
bfq-iosched.h	block, bfq: cleanup the repeated declaration	2021-08-25 06:45:33 -06:00
bfq-wf2q.c	block: Introduce IOPRIO_NR_LEVELS	2021-08-18 07:21:12 -06:00
bio-integrity.c	block: move integrity handling out of <linux/blkdev.h>	2021-10-18 06:17:02 -06:00
bio.c	block: don't call rq_qos_ops->done_bio if the bio isn't tracked	2021-09-24 11:04:39 -06:00
blk-cgroup-rwstat.c	blk-cgroup: Fix the recursive blkg rwstat	2021-03-05 11:32:15 -07:00
blk-cgroup-rwstat.h	blk-cgroup: separate out blkg_rwstat under CONFIG_BLK_CGROUP_RWSTAT	2019-11-07 12:28:13 -07:00
blk-cgroup.c	block: move blk-throtl fast path inline	2021-10-18 06:17:03 -06:00
blk-core.c	block: pre-allocate requests if plug is started and is a batch	2021-10-18 06:17:03 -06:00
blk-crypto-fallback.c	block: move struct request to blk-mq.h	2021-10-18 06:17:02 -06:00
blk-crypto-internal.h	block: move struct request to blk-mq.h	2021-10-18 06:17:02 -06:00
blk-crypto.c	blk-crypto: fix check for too-large dun_bytes	2021-08-25 06:45:00 -06:00
blk-exec.c	block: return errors from blk_execute_rq()	2021-06-30 15:35:45 -06:00
blk-flush.c	blk-mq: fix is_flush_rq	2021-08-17 20:17:34 -06:00
blk-integrity.c	block: move integrity handling out of <linux/blkdev.h>	2021-10-18 06:17:02 -06:00
blk-ioc.c	block: remove retry loop in ioc_release_fn()	2020-07-16 10:22:15 -06:00
blk-iocost.c	for-5.15/block-2021-08-30	2021-08-30 18:52:11 -07:00
blk-iolatency.c	mm: don't include <linux/blk-cgroup.h> in <linux/backing-dev.h>	2021-10-18 06:17:01 -06:00
blk-ioprio.c	block: Introduce the ioprio rq-qos policy	2021-06-21 15:03:40 -06:00
blk-ioprio.h	block: Introduce the ioprio rq-qos policy	2021-06-21 15:03:40 -06:00
blk-lib.c	block: export blk_next_bio()	2021-06-17 15:51:20 +02:00
blk-map.c	Merge branch 'akpm' (patches from Andrew)	2021-09-03 10:08:28 -07:00
blk-merge.c	block: move blk-throtl fast path inline	2021-10-18 06:17:03 -06:00
blk-mq-cpumap.c	blk-mq: remove the calling of local_memory_node()	2020-10-20 07:08:17 -06:00
blk-mq-debugfs-zoned.c	block: Cleanup license notice	2019-01-17 21:21:40 -07:00
blk-mq-debugfs.c	blk-mq: Stop using pointers for blk_mq_tags bitmap tags	2021-10-18 06:17:03 -06:00
blk-mq-debugfs.h	blk-mq: no need to check return value of debugfs_create functions	2019-06-13 03:00:30 -06:00
blk-mq-pci.c	block: Fix blk_mq_*_map_queues() kernel-doc headers	2019-05-31 15:12:34 -06:00
blk-mq-rdma.c	block: Fix blk_mq_*_map_queues() kernel-doc headers	2019-05-31 15:12:34 -06:00
blk-mq-sched.c	blk-mq: Change shared sbitmap naming to shared tags	2021-10-18 06:17:03 -06:00
blk-mq-sched.h	blk-mq-sched: Rename blk_mq_sched_free_{requests -> rqs}()	2021-10-18 06:17:02 -06:00
blk-mq-sysfs.c	block: remove blk-mq-sysfs dead code	2021-08-02 13:37:29 -06:00
blk-mq-tag.c	blk-mq: Change shared sbitmap naming to shared tags	2021-10-18 06:17:03 -06:00
blk-mq-tag.h	blk-mq: Change shared sbitmap naming to shared tags	2021-10-18 06:17:03 -06:00
blk-mq-virtio.c	blk-mq: Fix typo in comment	2020-03-17 20:55:21 +01:00
blk-mq.c	block: pre-allocate requests if plug is started and is a batch	2021-10-18 06:17:03 -06:00
blk-mq.h	block: pre-allocate requests if plug is started and is a batch	2021-10-18 06:17:03 -06:00
blk-pm.c	scsi: block: Fix a race in the runtime power management code	2020-12-09 11:41:41 -05:00
blk-pm.h	block: Remove unused blk_pm_*() function definitions	2021-02-22 06:33:48 -07:00
blk-rq-qos.c	rq-qos: fix missed wake-ups in rq_qos_throttle try two	2021-06-08 15:12:57 -06:00
blk-rq-qos.h	block: Introduce the ioprio rq-qos policy	2021-06-21 15:03:40 -06:00
blk-settings.c	block: add an explicit ->disk backpointer to the request_queue	2021-08-23 12:54:31 -06:00
blk-stat.c	blk-stat: make q->stats->lock irqsafe	2020-09-01 16:48:46 -06:00
blk-stat.h	block: deactivate blk_stat timer in wbt_disable_default()	2018-12-12 06:47:51 -07:00
blk-sysfs.c	block: move blk-throtl fast path inline	2021-10-18 06:17:03 -06:00
blk-throttle.c	block: move blk-throtl fast path inline	2021-10-18 06:17:03 -06:00
blk-throttle.h	block: move blk-throtl fast path inline	2021-10-18 06:17:03 -06:00
blk-timeout.c	block: blk-timeout: delete duplicated word	2020-07-31 16:29:47 -06:00
blk-wbt.c	block: add an explicit ->disk backpointer to the request_queue	2021-08-23 12:54:31 -06:00
blk-wbt.h	blk-wbt: introduce a new disable state to prevent false positive by rwb_enabled()	2021-06-21 15:03:41 -06:00
blk-zoned.c	blk-zoned: allow BLKREPORTZONE without CAP_SYS_ADMIN	2021-08-24 10:12:36 -06:00
blk.h	block: bump max plugged deferred size from 16 to 32	2021-10-18 06:17:03 -06:00
bounce.c	mm: don't include <linux/blk-cgroup.h> in <linux/backing-dev.h>	2021-10-18 06:17:01 -06:00
bsg-lib.c	scsi: bsg-lib: Fix commands without data transfer in bsg_transport_sg_io_fn()	2021-08-01 13:21:40 -04:00
bsg.c	scsi: bsg: Fix device unregistration	2021-09-14 00:22:15 -04:00
disk-events.c	block: return errors from disk_alloc_events	2021-08-23 12:55:45 -06:00
elevator.c	blk-mq: Change shared sbitmap naming to shared tags	2021-10-18 06:17:03 -06:00
elevator.h	block: move elevator.h to block/	2021-10-18 06:17:01 -06:00
fops.c	block: hold ->invalidate_lock in blkdev_fallocate	2021-09-24 11:06:58 -06:00
genhd.c	block: drop unused includes in <linux/genhd.h>	2021-10-18 06:17:02 -06:00
holder.c	block: drop unused includes in <linux/genhd.h>	2021-10-18 06:17:02 -06:00
ioctl.c	block: pass a gendisk to bdev_resize_partition	2021-08-12 10:31:36 -06:00
ioprio.c	block: fix default IO priority handling	2021-08-18 07:23:15 -06:00
Kconfig	SCSI misc on 20210902	2021-09-02 15:09:46 -07:00
Kconfig.iosched	Revert "block/mq-deadline: Add cgroup support"	2021-08-11 13:47:26 -06:00
keyslot-manager.c	block: move integrity handling out of <linux/blkdev.h>	2021-10-18 06:17:02 -06:00
kyber-iosched.c	blk-mq: Stop using pointers for blk_mq_tags bitmap tags	2021-10-18 06:17:03 -06:00
Makefile	block-5.15-2021-09-11	2021-09-11 10:19:51 -07:00
mq-deadline.c	blk-mq: Stop using pointers for blk_mq_tags bitmap tags	2021-10-18 06:17:03 -06:00
opal_proto.h	block: sed-opal: Change the check condition for regular session validity	2020-03-12 08:00:10 -06:00
sed-opal.c	block: sed-opal: Change the check condition for regular session validity	2020-03-12 08:00:10 -06:00
t10-pi.c	block: move integrity handling out of <linux/blkdev.h>	2021-10-18 06:17:02 -06:00