Omar Sandoval 6e25cb01ea kyber: implement improved heuristics ... Kyber's current heuristics have a few flaws: - It's based on the mean latency, but p99 latency tends to be more meaningful to anyone who cares about latency. The mean can also be skewed by rare outliers that the scheduler can't do anything about. - The statistics calculations are purely time-based with a short window. This works for steady, high load, but is more sensitive to outliers with bursty workloads. - It only considers the latency once an I/O has been submitted to the device, but the user cares about the time spent in the kernel, as well. These are shortcomings of the generic blk-stat code which doesn't quite fit the ideal use case for Kyber. So, this replaces the statistics with a histogram used to calculate percentiles of total latency and I/O latency, which we then use to adjust depths in a slightly more intelligent manner: - Sync and async writes are now the same domain. - Discards are a separate domain. - Domain queue depths are scaled by the ratio of the p99 total latency to the target latency (e.g., if the p99 latency is double the target latency, we will double the queue depth; if the p99 latency is half of the target latency, we can halve the queue depth). - We use the I/O latency to determine whether we should scale queue depths down: we will only scale down if any domain's I/O latency exceeds the target latency, which is an indicator of congestion in the device. These new heuristics are just as scalable as the heuristics they replace. Signed-off-by: Omar Sandoval <osandov@fb.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>		2018-09-27 17:34:57 -06:00
..
partitions	partitions/aix: append null character to print data from disk	2018-07-27 09:17:41 -06:00
badblocks.c	badblocks: fix wrong return value in badblocks_set if badblocks are disabled	2017-11-03 11:29:50 -07:00
bfq-cgroup.c	blkcg: fix ref count issue with bio_blkcg using task_css	2018-09-21 20:29:02 -06:00
bfq-iosched.c	blkcg: fix ref count issue with bio_blkcg using task_css	2018-09-21 20:29:02 -06:00
bfq-iosched.h	block, bfq: inject other-queue I/O into seeky idle queues on NCQ flash	2018-09-14 13:06:03 -06:00
bfq-wf2q.c	block, bfq: correctly charge and reset entity service in all cases	2018-09-14 13:06:02 -06:00
bio-integrity.c	block: remove bio_rewind_iter()	2018-09-06 15:12:24 -06:00
bio.c	block: merge BIOVEC_SEG_BOUNDARY into biovec_phys_mergeable	2018-09-24 12:33:57 -06:00
blk-cgroup.c	blkcg: rename blkg_try_get to blkg_tryget	2018-09-21 20:29:19 -06:00
blk-core.c	block: Make blk_get_request() block for non-PM requests while suspended	2018-09-26 15:11:29 -06:00
blk-exec.c	blk-mq-sched: remove unused 'can_block' arg from blk_mq_sched_insert_request	2018-01-17 09:49:21 -07:00
blk-flush.c	block: fix use-after-free in block flush handling	2018-06-09 06:37:14 -06:00
blk-integrity.c	block: merge BIOVEC_SEG_BOUNDARY into biovec_phys_mergeable	2018-09-24 12:33:57 -06:00
blk-ioc.c	block, mm: remove unnecessary __GFP_HIGH flag	2018-07-09 09:07:54 -06:00
blk-iolatency.c	blkcg: rename blkg_try_get to blkg_tryget	2018-09-21 20:29:19 -06:00
blk-lib.c	block: fix infinite loop if the device loses discard capability	2018-07-09 09:07:54 -06:00
blk-map.c	Merge branch 'for-4.16/block' of git://git.kernel.dk/linux-block	2018-01-29 11:51:49 -08:00
blk-merge.c	block: merge BIOVEC_SEG_BOUNDARY into biovec_phys_mergeable	2018-09-24 12:33:57 -06:00
blk-mq-cpumap.c	blk-mq: don't keep offline CPUs mapped to hctx 0	2018-04-10 08:38:46 -06:00
blk-mq-debugfs-zoned.c	block: Make struct request_queue smaller for CONFIG_BLK_DEV_ZONED=n	2018-07-09 09:07:52 -06:00
blk-mq-debugfs.c	block, scsi: Change the preempt-only flag into a counter	2018-09-26 15:11:28 -06:00
blk-mq-debugfs.h	block: Make struct request_queue smaller for CONFIG_BLK_DEV_ZONED=n	2018-07-09 09:07:52 -06:00
blk-mq-pci.c	blk-mq: code clean-up by adding an API to clear set->mq_map	2018-07-09 09:07:53 -06:00
blk-mq-rdma.c	block: Add rdma affinity based queue mapping helper	2017-08-08 14:58:03 -04:00
blk-mq-sched.c	blk-mq: init hctx sched after update ctx and hctx mapping	2018-08-21 09:02:55 -06:00
blk-mq-sched.h	block: move call of scheduler's ->completed_request() hook	2018-09-27 17:34:52 -06:00
blk-mq-sysfs.c	block drivers/block: Use octal not symbolic permissions	2018-05-24 13:38:59 -06:00
blk-mq-tag.c	blk-mq: Document the functions that iterate over requests	2018-09-21 20:30:22 -06:00
blk-mq-tag.h	Merge branch 'for-4.15/block' of git://git.kernel.dk/linux-block	2017-11-14 15:32:19 -08:00
blk-mq-virtio.c	blk-mq: provide a default queue mapping for virtio device	2017-02-27 20:54:05 +02:00
blk-mq.c	block: move call of scheduler's ->completed_request() hook	2018-09-27 17:34:52 -06:00
blk-mq.h	blk-mq: issue directly if hw queue isn't busy in case of 'none'	2018-07-17 16:04:00 -06:00
blk-pm.c	blk-mq: Enable support for runtime power management	2018-09-26 15:11:29 -06:00
blk-pm.h	block: Split blk_pm_add_request() and blk_pm_put_request()	2018-09-26 15:11:28 -06:00
blk-rq-qos.c	blk-rq-qos: make depth comparisons unsigned	2018-07-22 11:30:53 -06:00
blk-rq-qos.h	blk-rq-qos: make depth comparisons unsigned	2018-07-22 11:30:53 -06:00
blk-settings.c	block: allow max_discard_segments to be stacked	2018-07-24 14:46:39 -06:00
blk-softirq.c	block: fix timeout changes for legacy request drivers	2018-06-19 11:27:18 -06:00
blk-stat.c	block: export blk_stat_enable_accounting()	2018-09-27 17:34:54 -06:00
blk-stat.h	blk-stat: export helpers for modifying blk_rq_stat	2018-07-09 09:07:54 -06:00
blk-sysfs.c	blk-wbt: don't maintain inflight counts if disabled	2018-08-23 09:34:46 -06:00
blk-tag.c	for-linus-20180616	2018-06-17 05:37:55 +09:00
blk-throttle.c	blkcg: consolidate bio_issue_init to be a part of core	2018-09-21 20:29:08 -06:00
blk-timeout.c	blk-mq: Fix timeout handling in case the timeout handler returns BLK_EH_DONE	2018-06-23 10:25:45 -06:00
blk-wbt.c	blk-wbt: remove dead code	2018-08-27 13:32:12 -06:00
blk-wbt.h	block: remove external dependency on wbt_flags	2018-07-09 09:07:54 -06:00
blk-zoned.c	block: Remove a superfluous cast from blkdev_report_zones()	2018-07-09 09:07:52 -06:00
blk.h	block: remove ARCH_BIOVEC_PHYS_MERGEABLE	2018-09-26 08:45:11 -06:00
bounce.c	blkcg: remove bio->bi_css and instead use bio->bi_blkg	2018-09-21 20:29:13 -06:00
bsg-lib.c	block/bsg-lib: use PTR_ERR_OR_ZERO to simplify the flow path	2018-08-01 09:13:03 -06:00
bsg.c	block: bsg: move atomic_t ref_count variable to refcount API	2018-08-27 19:17:02 -06:00
cfq-iosched.c	blkcg: fix ref count issue with bio_blkcg using task_css	2018-09-21 20:29:02 -06:00
cmdline-parser.c	License cleanup: add SPDX GPL-2.0 license identifier to files with no license	2017-11-02 11:10:55 +01:00
compat_ioctl.c	License cleanup: add SPDX GPL-2.0 license identifier to files with no license	2017-11-02 11:10:55 +01:00
deadline-iosched.c	block drivers/block: Use octal not symbolic permissions	2018-05-24 13:38:59 -06:00
elevator.c	block: Schedule runtime resume earlier	2018-09-26 15:11:28 -06:00
genhd.c	block: Track DISCARD statistics and output them in stat and diskstat	2018-07-18 08:44:22 -06:00
ioctl.c	block: pass inclusive 'lend' parameter to truncate_inode_pages_range	2018-02-23 15:20:19 -07:00
ioprio.c	block: add ioprio_check_cap function	2018-05-31 10:50:54 -04:00
Kconfig	block: Move power management code into a new source file	2018-09-26 15:11:28 -06:00
Kconfig.iosched	License cleanup: add SPDX GPL-2.0 license identifier to files with no license	2017-11-02 11:10:55 +01:00
kyber-iosched.c	kyber: implement improved heuristics	2018-09-27 17:34:57 -06:00
Makefile	block: Move power management code into a new source file	2018-09-26 15:11:28 -06:00
mq-deadline.c	block drivers/block: Use octal not symbolic permissions	2018-05-24 13:38:59 -06:00
noop-iosched.c	block: move existing elevator ops to union	2017-01-17 10:03:33 -07:00
opal_proto.h	block: sed-opal: Set MBRDone on S3 resume path if TPER is MBREnabled	2017-09-11 09:45:52 -06:00
partition-generic.c	block: Track DISCARD statistics and output them in stat and diskstat	2018-07-18 08:44:22 -06:00
scsi_ioctl.c	block: consistently use GFP_NOIO instead of __GFP_NORECLAIM	2018-05-14 08:55:18 -06:00
sed-opal.c	block: sed-opal: Fix a couple off by one bugs	2018-06-20 12:04:06 -06:00
t10-pi.c	block: move dif_prepare/dif_complete functions to block layer	2018-07-30 08:27:02 -06:00