korg/linux
0
0
Fork 0
mirror of https://mirrors.bfsu.edu.cn/git/linux.git synced 2024-11-17 01:04:19 +08:00
Code Issues Packages Projects Releases Wiki Activity
bd11b3a391
linux/fs
History
Jens Axboe bd11b3a391 io_uring: don't use iov_iter_advance() for fixed buffers 
Hrvoje reports that when a large fixed buffer is registered and IO is
being done to the latter pages of said buffer, the IO submission time
is much worse:

reading to the start of the buffer: 11238 ns
reading to the end of the buffer:   1039879 ns

In fact, it's worse by two orders of magnitude. The reason for that is
how io_uring figures out how to setup the iov_iter. We point the iter
at the first bvec, and then use iov_iter_advance() to fast-forward to
the offset within that buffer we need.

However, that is abysmally slow, as it entails iterating the bvecs
that we setup as part of buffer registration. There's really no need
to use this generic helper, as we know it's a BVEC type iterator, and
we also know that each bvec is PAGE_SIZE in size, apart from possibly
the first and last. Hence we can just use a shift on the offset to
find the right index, and then adjust the iov_iter appropriately.
After this fix, the timings are:

reading to the start of the buffer: 10135 ns
reading to the end of the buffer:   1377 ns

Or about an 755x improvement for the tail page.

Reported-by: Hrvoje Zeba <zeba.hrvoje@gmail.com>
Tested-by: Hrvoje Zeba <zeba.hrvoje@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2019-07-21 21:46:36 -06:00
..
9p 9p: pass the correct prototype to read_cache_page 2019-07-12 11:05:43 -07:00
adfs treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
affs treewide: Add SPDX license identifier - Makefile/Kconfig 2019-05-21 10:50:46 +02:00
afs Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next 2019-07-11 10:55:49 -07:00
autofs treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 83 2019-05-24 17:37:52 +02:00
befs treewide: Add SPDX license identifier - Makefile/Kconfig 2019-05-21 10:50:46 +02:00
bfs treewide: Add SPDX license identifier - Makefile/Kconfig 2019-05-21 10:50:46 +02:00
btrfs for-linus-20190715 2019-07-15 21:20:52 -07:00
cachefiles treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 36 2019-05-24 17:27:11 +02:00
ceph Driver Core and debugfs changes for 5.3-rc1 2019-07-12 12:24:03 -07:00
cifs Changes to copy_file_range for 5.3 from Dave and Amir: 2019-07-10 20:32:37 -07:00
coda treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
configfs \n 2019-07-10 20:09:17 -07:00
cramfs treewide: Add SPDX license identifier - Makefile/Kconfig 2019-05-21 10:50:46 +02:00
crypto Revert "Merge tag 'keys-acl-20190703' of git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs" 2019-07-10 18:43:43 -07:00
debugfs Driver Core and debugfs changes for 5.3-rc1 2019-07-12 12:24:03 -07:00
devpts devpts: call fsnotify_unlink() hook 2019-06-20 14:46:34 +02:00
dlm dlm for 5.3 2019-07-12 17:37:53 -07:00
ecryptfs - Fix error handling when ecryptfs_read_lower() encounters an error 2019-07-14 19:29:04 -07:00
efivarfs New for 5.3: 2019-07-12 16:54:37 -07:00
efs treewide: Add SPDX license identifier - Makefile/Kconfig 2019-05-21 10:50:46 +02:00
exportfs treewide: Add SPDX license identifier - Makefile/Kconfig 2019-05-21 10:50:46 +02:00
ext2 New for 5.3: 2019-07-12 16:54:37 -07:00
ext4 for-linus-20190715 2019-07-15 21:20:52 -07:00
f2fs for-linus-20190715 2019-07-15 21:20:52 -07:00
fat treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 282 2019-06-05 17:36:37 +02:00
freevxfs treewide: Add SPDX license identifier - Makefile/Kconfig 2019-05-21 10:50:46 +02:00
fscache Revert "Merge tag 'keys-acl-20190703' of git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs" 2019-07-10 18:43:43 -07:00
fuse Changes to copy_file_range for 5.3 from Dave and Amir: 2019-07-10 20:32:37 -07:00
gfs2 New for 5.3: 2019-07-12 16:54:37 -07:00
hfs treewide: Add SPDX license identifier - Makefile/Kconfig 2019-05-21 10:50:46 +02:00
hfsplus vfs: create a generic checking and prep function for FS_IOC_SETFLAGS 2019-07-01 08:25:34 -07:00
hostfs This pull request contains the following changes for UML: 2019-05-12 17:52:13 -04:00
hpfs treewide: Add SPDX license identifier - Makefile/Kconfig 2019-05-21 10:50:46 +02:00
hugetlbfs treewide: Add SPDX license identifier - Makefile/Kconfig 2019-05-21 10:50:46 +02:00
isofs treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 142 2019-05-30 11:25:17 -07:00
jbd2 jbd2: drop declaration of journal_sync_buffer() 2019-06-20 17:32:21 -04:00
jffs2 jffs2: pass the correct prototype to read_cache_page 2019-07-12 11:05:43 -07:00
jfs vfs: create a generic checking and prep function for FS_IOC_SETFLAGS 2019-07-01 08:25:34 -07:00
kernfs treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 428 2019-06-05 17:37:16 +02:00
lockd lockd: Make two symbols static 2019-07-03 17:52:09 -04:00
minix treewide: Add SPDX license identifier - Makefile/Kconfig 2019-05-21 10:50:46 +02:00
nfs Changes to copy_file_range for 5.3 from Dave and Amir: 2019-07-10 20:32:37 -07:00
nfs_common treewide: Add SPDX license identifier - Makefile/Kconfig 2019-05-21 10:50:46 +02:00
nfsd Driver Core and debugfs changes for 5.3-rc1 2019-07-12 12:24:03 -07:00
nilfs2 vfs: create a generic checking and prep function for FS_IOC_SETFLAGS 2019-07-01 08:25:34 -07:00
nls treewide: Add SPDX license identifier - Makefile/Kconfig 2019-05-21 10:50:46 +02:00
notify memcg, fsnotify: no oom-kill for remote memcg charging 2019-07-12 11:05:43 -07:00
ntfs treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 97 2019-05-24 17:37:53 +02:00
ocfs2 New for 5.3: 2019-07-12 16:54:37 -07:00
omfs treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 209 2019-05-30 11:29:53 -07:00
openpromfs treewide: Add SPDX license identifier - Makefile/Kconfig 2019-05-21 10:50:46 +02:00
orangefs New for 5.3: 2019-07-12 16:54:37 -07:00
overlayfs SPDX update for 5.2-rc6 2019-06-21 09:58:42 -07:00
proc HMM patches for 5.3 2019-07-14 19:42:11 -07:00
pstore pstore: Fix double-free in pstore_mkfile() failure path 2019-07-08 21:04:42 -07:00
qnx4 treewide: Add SPDX license identifier - Makefile/Kconfig 2019-05-21 10:50:46 +02:00
qnx6 treewide: Add SPDX license identifier - Makefile/Kconfig 2019-05-21 10:50:46 +02:00
quota \n 2019-07-10 20:27:07 -07:00
ramfs treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
reiserfs vfs: create a generic checking and prep function for FS_IOC_SETFLAGS 2019-07-01 08:25:34 -07:00
romfs treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
squashfs treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 499 2019-06-19 17:09:53 +02:00
sysfs sysfs: Add sysfs_update_groups function 2019-06-03 11:58:20 +02:00
sysv treewide: Add SPDX license identifier - Makefile/Kconfig 2019-05-21 10:50:46 +02:00
tracefs \n 2019-07-10 20:09:17 -07:00
ubifs This pull request contains the following changes for UBIFS 2019-07-14 17:24:12 -07:00
udf \n 2019-07-10 20:27:07 -07:00
ufs treewide: Add SPDX license identifier - Makefile/Kconfig 2019-05-21 10:50:46 +02:00
unicode Many bug fixes and cleanups, and an optimization for case-insensitive 2019-07-10 21:06:01 -07:00
xfs for-linus-20190715 2019-07-15 21:20:52 -07:00
aio.c for-5.3/io_uring-20190711 2019-07-13 10:36:53 -07:00
anon_inodes.c treewide: Add SPDX license identifier for missed files 2019-05-21 10:50:45 +02:00
attr.c
bad_inode.c
binfmt_aout.c treewide: Add SPDX license identifier for more missed files 2019-05-21 10:50:45 +02:00
binfmt_elf_fdpic.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
binfmt_elf.c treewide: Add SPDX license identifier for more missed files 2019-05-21 10:50:45 +02:00
binfmt_em86.c treewide: Add SPDX license identifier for more missed files 2019-05-21 10:50:45 +02:00
binfmt_flat.c Merge branch 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/gerg/m68knommu 2019-07-10 21:42:03 -07:00
binfmt_misc.c treewide: Add SPDX license identifier for more missed files 2019-05-21 10:50:45 +02:00
binfmt_script.c treewide: Add SPDX license identifier for more missed files 2019-05-21 10:50:45 +02:00
block_dev.c block: properly handle IOCB_NOWAIT for async O_DIRECT IO 2019-07-21 21:46:29 -06:00
buffer.c for-linus-20190715 2019-07-15 21:20:52 -07:00
char_dev.c chardev: set variable ret to -EBUSY before checking minor range overlap 2019-05-24 20:50:36 +02:00
compat_binfmt_elf.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 193 2019-05-30 11:29:21 -07:00
compat_ioctl.c
compat.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
coredump.c
d_path.c
dax.c Merge branch 'locking-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2019-07-08 16:12:03 -07:00
dcache.c fsnotify: move fsnotify_nameremove() hook out of d_delete() 2019-06-20 14:47:44 +02:00
dcookies.c treewide: Add SPDX license identifier for missed files 2019-05-21 10:50:45 +02:00
direct-io.c direct-io: use bio_release_pages in dio_bio_complete 2019-06-29 09:47:31 -06:00
drop_caches.c fs/drop_caches.c: avoid softlockups in drop_pagecache_sb() 2019-02-01 15:46:24 -08:00
eventfd.c treewide: Add SPDX license identifier for missed files 2019-05-21 10:50:45 +02:00
eventpoll.c signal: remove the wrong signal_pending() check in restore_user_sigmask() 2019-06-29 16:43:45 +08:00
exec.c Merge branch 'siginfo-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace 2019-07-08 21:48:15 -07:00
fcntl.c fs: mark expected switch fall-throughs 2019-04-08 18:21:02 -05:00
fhandle.c
file_table.c treewide: Add SPDX license identifier for missed files 2019-05-21 10:50:45 +02:00
file.c io_uring-2019-03-06 2019-03-08 14:48:40 -08:00
filesystems.c vfs: Implement a filesystem superblock creation/configuration context 2019-02-28 03:29:26 -05:00
fs_context.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 36 2019-05-24 17:27:11 +02:00
fs_parser.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 36 2019-05-24 17:27:11 +02:00
fs_pin.c
fs_struct.c treewide: Add SPDX license identifier for missed files 2019-05-21 10:50:45 +02:00
fs_types.c fs: common implementation of file type 2019-01-21 17:48:13 +01:00
fs-writeback.c blkcg, writeback: Add wbc->no_cgroup_owner 2019-07-10 09:00:57 -06:00
fsopen.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 36 2019-05-24 17:27:11 +02:00
inode.c New for 5.3: 2019-07-12 16:54:37 -07:00
internal.h fs: fold __generic_write_end back into generic_write_end 2019-06-27 17:28:40 -07:00
io_uring.c io_uring: don't use iov_iter_advance() for fixed buffers 2019-07-21 21:46:36 -06:00
ioctl.c Remove 'type' argument from access_ok() function 2019-01-03 18:57:57 -08:00
iomap.c New for 5.3: 2019-07-10 20:29:45 -07:00
Kconfig fs: VALIDATE_FS_PARSER should default to n 2019-07-05 11:22:11 -04:00
Kconfig.binfmt binfmt_flat: make support for old format binaries optional 2019-06-24 09:16:47 +10:00
libfs.c treewide: Add SPDX license identifier for missed files 2019-05-21 10:50:45 +02:00
locks.c Highlights: 2019-07-10 21:22:43 -07:00
Makefile Add as a feature case-insensitive directories (the casefold feature) 2019-05-07 21:12:44 -07:00
mbcache.c treewide: Add SPDX license identifier for more missed files 2019-05-21 10:50:45 +02:00
mount.h saner handling of temporary namespaces 2019-01-30 17:44:07 -05:00
mpage.c blkcg, writeback: Rename wbc_account_io() to wbc_account_cgroup_owner() 2019-07-10 09:00:57 -06:00
namei.c fsnotify: add empty fsnotify_{unlink,rmdir}() hooks 2019-06-20 14:44:55 +02:00
namespace.c vfs: move_mount: reject moving kernel internal mounts 2019-07-01 10:46:36 -04:00
no-block.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 152 2019-05-30 11:26:32 -07:00
nsfs.c nsfs: unobfuscate 2019-04-09 19:20:57 -04:00
open.c treewide: Add SPDX license identifier for missed files 2019-05-21 10:50:45 +02:00
pipe.c Merge branch 'page-refs' (page ref overflow) 2019-04-14 15:09:40 -07:00
pnode.c fs/namespace: fix unprivileged mount propagation 2019-06-17 17:36:09 -04:00
pnode.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 209 2019-05-30 11:29:53 -07:00
posix_acl.c treewide: Add SPDX license identifier for missed files 2019-05-21 10:50:45 +02:00
proc_namespace.c
read_write.c vfs: allow copy_file_range to copy across devices 2019-06-09 10:06:20 -07:00
readdir.c Remove 'type' argument from access_ok() function 2019-01-03 18:57:57 -08:00
select.c signal: remove the wrong signal_pending() check in restore_user_sigmask() 2019-06-29 16:43:45 +08:00
seq_file.c nfsd: escape high characters in binary data 2019-07-03 17:52:50 -04:00
signalfd.c fs: mark expected switch fall-throughs 2019-04-08 18:21:02 -05:00
splice.c uio: make import_iovec()/compat_import_iovec() return bytes on success 2019-05-31 15:30:03 -06:00
stack.c treewide: Add SPDX license identifier for missed files 2019-05-21 10:50:45 +02:00
stat.c fs: move generic stat response attr handling to vfs_getattr_nosec 2019-02-01 01:55:45 -05:00
statfs.c vfs: add vfs_get_fsid() helper 2019-02-07 16:38:35 +01:00
super.c [fix] get rid of checking for absent device name in vfs_get_tree() 2019-04-28 21:34:21 -04:00
sync.c fs/sync.c: sync_file_range(2) may use WB_SYNC_ALL writeback 2019-05-14 09:47:50 -07:00
timerfd.c y2038: syscalls: rename y2038 compat syscalls 2019-02-07 00:13:27 +01:00
userfaultfd.c fs/userfaultfd.c: disable irqs for fault_pending and event locks 2019-07-05 11:12:07 +09:00
utimes.c y2038: syscalls: rename y2038 compat syscalls 2019-02-07 00:13:27 +01:00
xattr.c treewide: Add SPDX license identifier for missed files 2019-05-21 10:50:45 +02:00