linux/fs/xfs
Dave Chinner ed1128c2d0 xfs: reduce exclusive locking on unaligned dio
Attempt shared locking for unaligned DIO, but only if the the
underlying extent is already allocated and in written state. On
failure, retry with the existing exclusive locking.

Test case is fio randrw of 512 byte IOs using AIO and an iodepth of
32 IOs.

Vanilla:

  READ: bw=4560KiB/s (4670kB/s), 4560KiB/s-4560KiB/s (4670kB/s-4670kB/s), io=134MiB (140MB), run=30001-30001msec
  WRITE: bw=4567KiB/s (4676kB/s), 4567KiB/s-4567KiB/s (4676kB/s-4676kB/s), io=134MiB (140MB), run=30001-30001msec

Patched:
   READ: bw=37.6MiB/s (39.4MB/s), 37.6MiB/s-37.6MiB/s (39.4MB/s-39.4MB/s), io=1127MiB (1182MB), run=30002-30002msec
  WRITE: bw=37.6MiB/s (39.4MB/s), 37.6MiB/s-37.6MiB/s (39.4MB/s-39.4MB/s), io=1128MiB (1183MB), run=30002-30002msec

That's an improvement from ~18k IOPS to a ~150k IOPS, which is
about the IOPS limit of the VM block device setup I'm testing on.

4kB block IO comparison:

   READ: bw=296MiB/s (310MB/s), 296MiB/s-296MiB/s (310MB/s-310MB/s), io=8868MiB (9299MB), run=30002-30002msec
  WRITE: bw=296MiB/s (310MB/s), 296MiB/s-296MiB/s (310MB/s-310MB/s), io=8878MiB (9309MB), run=30002-30002msec

Which is ~150k IOPS, same as what the test gets for sub-block
AIO+DIO writes with this patch.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
[hch: rebased, split unaligned from nowait]
Signed-off-by: Christoph Hellwig <hch@lst.de>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
2021-02-01 09:47:19 -08:00
..
libxfs xfs: remove xfs_buf_t typedef 2020-12-16 16:07:34 -08:00
scrub xfs: don't catch dax+reflink inodes as corruption in verifier 2020-12-09 09:49:38 -08:00
Kconfig xfs: fix Kconfig asking about XFS_SUPPORT_V4 when XFS_FS=n 2020-10-16 15:34:28 -07:00
kmem.c xfs: remove kmem_realloc() 2020-09-06 18:05:51 -07:00
kmem.h xfs: Remove kmem_zalloc_large() 2020-09-15 20:52:41 -07:00
Makefile xfs: refactor log recovery item sorting into a generic dispatch structure 2020-05-08 08:49:58 -07:00
mrlock.h
xfs_acl.c xfs: open code updating i_mode in xfs_set_acl 2020-12-12 10:49:38 -08:00
xfs_acl.h xfs: improve xfs_forget_acl 2020-03-02 20:55:55 -08:00
xfs_aops.c xfs: fix missing CoW blocks writeback conversion retry 2020-11-04 08:52:47 -08:00
xfs_aops.h
xfs_attr_inactive.c xfs: cleanup xfs_idestroy_fork 2020-05-19 09:40:59 -07:00
xfs_attr_list.c xfs: Convert xfs_attr_sf macros to inline functions 2020-09-15 20:52:42 -07:00
xfs_bio_io.c
xfs_bmap_item.c xfs: refactor file range validation 2020-12-09 09:49:38 -08:00
xfs_bmap_item.h xfs: refactor intent item RECOVERED flag into the log item 2020-05-08 08:50:01 -07:00
xfs_bmap_util.c xfs: fix fallocate functions when rtextsize is larger than 1 2020-10-21 09:05:19 -07:00
xfs_bmap_util.h
xfs_buf_item_recover.c xfs: fix finobt btree block recovery ordering 2020-09-30 07:28:52 -07:00
xfs_buf_item.c xfs: remove xfs_buf_t typedef 2020-12-16 16:07:34 -08:00
xfs_buf_item.h xfs: move the buffer retry logic to xfs_buf.c 2020-09-15 20:52:38 -07:00
xfs_buf.c xfs: remove xfs_buf_t typedef 2020-12-16 16:07:34 -08:00
xfs_buf.h xfs: remove xfs_buf_t typedef 2020-12-16 16:07:34 -08:00
xfs_dir2_readdir.c xfs: move the fork format fields into struct xfs_ifork 2020-05-19 09:40:58 -07:00
xfs_discard.c xfs: remove XFS_BUF_TO_AGF 2020-03-11 09:11:39 -07:00
xfs_discard.h
xfs_dquot_item_recover.c xfs: rename the ondisk dquot d_flags to d_type 2020-07-28 20:24:14 -07:00
xfs_dquot_item.c xfs: stop using q_core.d_id in the quota code 2020-07-28 20:24:14 -07:00
xfs_dquot_item.h xfs: factor out quotaoff intent AIL removal and memory free 2020-03-18 08:12:23 -07:00
xfs_dquot.c xfs: fix some comments 2020-09-25 11:34:07 -07:00
xfs_dquot.h xfs: refactor default quota grace period setting code 2020-09-15 20:52:40 -07:00
xfs_error.c xfs: random buffer write failure errortag 2020-05-07 08:27:48 -07:00
xfs_error.h xfs: xfs_buf_corruption_error should take __this_address 2020-03-12 07:58:12 -07:00
xfs_export.c xfs: delete duplicated words + other fixes 2020-08-05 08:49:58 -07:00
xfs_export.h
xfs_extent_busy.c
xfs_extent_busy.h
xfs_extfree_item.c xfs: refactor data device extent validation 2020-12-09 09:49:38 -08:00
xfs_extfree_item.h xfs: refactor intent item RECOVERED flag into the log item 2020-05-08 08:50:01 -07:00
xfs_file.c xfs: reduce exclusive locking on unaligned dio 2021-02-01 09:47:19 -08:00
xfs_filestream.c xfs: drop the obsolete comment on filestream locking 2020-09-25 11:34:08 -07:00
xfs_filestream.h
xfs_fsmap.c xfs: fix deadlock and streamline xfs_getfsmap performance 2020-10-07 08:40:29 -07:00
xfs_fsmap.h xfs: fix deadlock and streamline xfs_getfsmap performance 2020-10-07 08:40:29 -07:00
xfs_fsops.c New code for 5.11: 2020-12-18 12:50:18 -08:00
xfs_fsops.h
xfs_globals.c
xfs_health.c
xfs_icache.c xfs: Remove unneeded semicolon 2020-09-15 20:52:42 -07:00
xfs_icache.h xfs: remove SYNC_WAIT and SYNC_TRYLOCK 2020-07-14 08:47:33 -07:00
xfs_icreate_item.c xfs: Remove kmem_zone_zalloc() usage 2020-07-28 20:24:14 -07:00
xfs_icreate_item.h
xfs_inode_item_recover.c xfs: widen ondisk inode timestamps to deal with y2038+ 2020-09-15 20:52:41 -07:00
xfs_inode_item.c xfs: widen ondisk inode timestamps to deal with y2038+ 2020-09-15 20:52:41 -07:00
xfs_inode_item.h xfs: move the buffer retry logic to xfs_buf.c 2020-09-15 20:52:38 -07:00
xfs_inode.c xfs: spilt xfs_dialloc() into 2 functions 2020-12-12 10:48:25 -08:00
xfs_inode.h xfs: move on-disk inode allocation out of xfs_ialloc() 2020-12-12 10:48:24 -08:00
xfs_ioctl32.c xfs: lift cursor copy in/out into xfs_ioc_attr_list 2020-03-02 20:55:54 -08:00
xfs_ioctl32.h xfs: rename compat_time_t to old_time32_t 2020-01-06 08:57:36 -08:00
xfs_ioctl.c xfs: fix deadlock and streamline xfs_getfsmap performance 2020-10-07 08:40:29 -07:00
xfs_ioctl.h xfs: embedded the attrlist cursor into struct xfs_attr_list_context 2020-03-02 20:55:55 -08:00
xfs_iomap.c xfs: reduce exclusive locking on unaligned dio 2021-02-01 09:47:19 -08:00
xfs_iomap.h
xfs_iops.c xfs: open code updating i_mode in xfs_set_acl 2020-12-12 10:49:38 -08:00
xfs_iops.h xfs: open code updating i_mode in xfs_set_acl 2020-12-12 10:49:38 -08:00
xfs_itable.c xfs: move the fork format fields into struct xfs_ifork 2020-05-19 09:40:58 -07:00
xfs_itable.h
xfs_iwalk.c xfs: fix the forward progress assertion in xfs_iwalk_run_callbacks 2020-12-09 09:49:38 -08:00
xfs_iwalk.h
xfs_linux.h xfs: fix fallocate functions when rtextsize is larger than 1 2020-10-21 09:05:19 -07:00
xfs_log_cil.c xfs: delete duplicated words + other fixes 2020-08-05 08:49:58 -07:00
xfs_log_priv.h xfs: Modify xlog_ticket_alloc() to use kernel's MM API 2020-07-28 20:24:14 -07:00
xfs_log_recover.c xfs: remove xfs_buf_t typedef 2020-12-16 16:07:34 -08:00
xfs_log.c xfs: expose the log push threshold 2020-10-07 08:40:29 -07:00
xfs_log.h xfs: expose the log push threshold 2020-10-07 08:40:29 -07:00
xfs_message.c xfs: refactor ratelimited buffer error messages into helper 2020-05-07 08:27:46 -07:00
xfs_message.h treewide: Convert macro and uses of __section(foo) to __section("foo") 2020-10-25 14:51:49 -07:00
xfs_mount.c xfs: return corresponding errcode if xfs_initialize_perag() fail 2020-11-18 09:23:51 -08:00
xfs_mount.h xfs: remove xfs_getsb 2020-09-15 20:52:39 -07:00
xfs_mru_cache.c
xfs_mru_cache.h
xfs_ondisk.h xfs: Remove typedef xfs_attr_shortform_t 2020-09-15 20:52:42 -07:00
xfs_pnfs.c xfs: fix a missing unlock on error in xfs_fs_map_blocks 2020-11-11 08:07:37 -08:00
xfs_pnfs.h
xfs_pwork.c block: remove the bd_queue field from struct block_device 2020-07-01 08:08:20 -06:00
xfs_pwork.h
xfs_qm_bhv.c xfs: rename XFS_DQ_{USER,GROUP,PROJ} to XFS_DQTYPE_* 2020-07-28 20:24:14 -07:00
xfs_qm_syscalls.c xfs: refactor default quota grace period setting code 2020-09-15 20:52:40 -07:00
xfs_qm.c xfs: move on-disk inode allocation out of xfs_ialloc() 2020-12-12 10:48:24 -08:00
xfs_qm.h xfs: refactor quota expiration timer modification 2020-09-15 20:52:40 -07:00
xfs_quota.h xfs: move the buffer retry logic to xfs_buf.c 2020-09-15 20:52:38 -07:00
xfs_quotaops.c xfs: create xfs_dqtype_t to represent quota types 2020-07-28 20:24:14 -07:00
xfs_refcount_item.c xfs: refactor data device extent validation 2020-12-09 09:49:38 -08:00
xfs_refcount_item.h xfs: refactor intent item RECOVERED flag into the log item 2020-05-08 08:50:01 -07:00
xfs_reflink.c xfs: only flush the unshared range in xfs_reflink_unshare 2020-11-04 17:41:56 -08:00
xfs_reflink.h xfs: move helpers that lock and unlock two inodes against userspace IO 2020-07-06 10:46:57 -07:00
xfs_rmap_item.c xfs: refactor file range validation 2020-12-09 09:49:38 -08:00
xfs_rmap_item.h xfs: refactor intent item RECOVERED flag into the log item 2020-05-08 08:50:01 -07:00
xfs_rtalloc.c xfs: remove xfs_buf_t typedef 2020-12-16 16:07:34 -08:00
xfs_rtalloc.h xfs: remove xfs_buf_t typedef 2020-12-16 16:07:34 -08:00
xfs_stats.c xfs: periodically relog deferred intent items 2020-10-07 08:40:28 -07:00
xfs_stats.h xfs: periodically relog deferred intent items 2020-10-07 08:40:28 -07:00
xfs_super.c xfs: show the proper user quota options 2020-12-09 09:49:38 -08:00
xfs_super.h
xfs_symlink.c xfs: remove xfs_buf_t typedef 2020-12-16 16:07:34 -08:00
xfs_symlink.h xfs: Correct comment tyops -> typos 2019-11-10 10:21:57 -08:00
xfs_sysctl.c xfs: remove deprecated sysctl options 2020-09-25 11:34:08 -07:00
xfs_sysctl.h
xfs_sysfs.c
xfs_sysfs.h xfs: Fix UBSAN null-ptr-deref in xfs_sysfs_init 2020-08-07 11:50:17 -07:00
xfs_trace.c xfs: support bulk loading of staged btrees 2020-03-18 08:12:23 -07:00
xfs_trace.h xfs: improve the reflink_bounce_dio_write tracepoint 2021-02-01 09:47:19 -08:00
xfs_trans_ail.c xfs: delete duplicated words + other fixes 2020-08-05 08:49:58 -07:00
xfs_trans_buf.c xfs: remove xfs_buf_t typedef 2020-12-16 16:07:34 -08:00
xfs_trans_dquot.c xfs: directly return if the delta equal to zero 2020-12-09 09:49:38 -08:00
xfs_trans_priv.h xfs: refactor adding recovered intent items to the log 2020-05-08 08:50:00 -07:00
xfs_trans.c xfs: remove xfs_buf_t typedef 2020-12-16 16:07:34 -08:00
xfs_trans.h xfs: periodically relog deferred intent items 2020-10-07 08:40:28 -07:00
xfs_xattr.c xfs: remove duplicate headers 2020-05-08 08:51:34 -07:00
xfs.h