linux/fs/xfs
Dave Chinner ec08c14ba2 xfs: type verification is expensive
From a concurrent rm -rf workload:

  41.04%  [kernel]  [k] xfs_dir3_leaf_check_int
   9.85%  [kernel]  [k] __xfs_dir3_data_check
   5.60%  [kernel]  [k] xfs_verify_ino
   5.32%  [kernel]  [k] xfs_agino_range
   4.21%  [kernel]  [k] memcpy
   3.06%  [kernel]  [k] xfs_errortag_test
   2.57%  [kernel]  [k] xfs_dir_ino_validate
   1.66%  [kernel]  [k] xfs_dir2_data_get_ftype
   1.17%  [kernel]  [k] do_raw_spin_lock
   1.11%  [kernel]  [k] xfs_verify_dir_ino
   0.84%  [kernel]  [k] __raw_callee_save___pv_queued_spin_unlock
   0.83%  [kernel]  [k] xfs_buf_find
   0.64%  [kernel]  [k] xfs_log_commit_cil

THere's an awful lot of overhead in just range checking inode
numbers in that, but each inode number check is not a lot of code.
The total is a bit over 14.5% of the CPU time is spent validating
inode numbers.

The problem is that they deeply nested global scope functions so the
overhead here is all in function call marshalling.

   text	   data	    bss	    dec	    hex	filename
   2077	      0	      0	   2077	    81d fs/xfs/libxfs/xfs_types.o.orig
   2197	      0	      0	   2197	    895	fs/xfs/libxfs/xfs_types.o

There's a small increase in binary size by inlining all the local
nested calls in the verifier functions, but the same workload now
profiles as:

  40.69%  [kernel]  [k] xfs_dir3_leaf_check_int
  10.52%  [kernel]  [k] __xfs_dir3_data_check
   6.68%  [kernel]  [k] xfs_verify_dir_ino
   4.22%  [kernel]  [k] xfs_errortag_test
   4.15%  [kernel]  [k] memcpy
   3.53%  [kernel]  [k] xfs_dir_ino_validate
   1.87%  [kernel]  [k] xfs_dir2_data_get_ftype
   1.37%  [kernel]  [k] do_raw_spin_lock
   0.98%  [kernel]  [k] xfs_buf_find
   0.94%  [kernel]  [k] __raw_callee_save___pv_queued_spin_unlock
   0.73%  [kernel]  [k] xfs_log_commit_cil

Now we only spend just over 10% of the time validing inode numbers
for the same workload. Hence a few "inline" keyworks is good enough
to reduce the validation overhead by 30%...

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Darrick J. Wong <djwong@kernel.org>
2021-03-25 16:47:51 -07:00
..
libxfs xfs: type verification is expensive 2021-03-25 16:47:51 -07:00
scrub xfs: remove return value from xchk_ag_btcur_init 2021-03-25 16:47:50 -07:00
Kconfig xfs: fix Kconfig asking about XFS_SUPPORT_V4 when XFS_FS=n 2020-10-16 15:34:28 -07:00
kmem.c xfs: remove kmem_realloc() 2020-09-06 18:05:51 -07:00
kmem.h xfs: Remove kmem_zalloc_large() 2020-09-15 20:52:41 -07:00
Makefile xfs: refactor log recovery item sorting into a generic dispatch structure 2020-05-08 08:49:58 -07:00
mrlock.h
xfs_acl.c xfs: support idmapped mounts 2021-01-24 14:43:46 +01:00
xfs_acl.h fs: make helpers idmap mount aware 2021-01-24 14:27:20 +01:00
xfs_aops.c xfs: use current->journal_info for detecting transaction recursion 2021-02-25 08:07:04 -08:00
xfs_aops.h
xfs_attr_inactive.c xfs: cleanup xfs_idestroy_fork 2020-05-19 09:40:59 -07:00
xfs_attr_list.c xfs: Convert xfs_attr_sf macros to inline functions 2020-09-15 20:52:42 -07:00
xfs_bio_io.c block: Add bio_max_segs 2021-02-26 15:49:51 -07:00
xfs_bmap_item.c xfs: Check for extent overflow when punching a hole 2021-01-22 16:54:47 -08:00
xfs_bmap_item.h xfs: refactor intent item RECOVERED flag into the log item 2020-05-08 08:50:01 -07:00
xfs_bmap_util.c xfs: allow reservation of rtblocks with xfs_trans_alloc_inode 2021-02-03 09:18:49 -08:00
xfs_bmap_util.h
xfs_buf_item_recover.c xfs: fix finobt btree block recovery ordering 2020-09-30 07:28:52 -07:00
xfs_buf_item.c xfs: optimise xfs_buf_item_size/format for contiguous regions 2021-03-25 16:47:51 -07:00
xfs_buf_item.h xfs: move the buffer retry logic to xfs_buf.c 2020-09-15 20:52:38 -07:00
xfs_buf.c block: Add bio_max_segs 2021-02-26 15:49:51 -07:00
xfs_buf.h xfs: don't drain buffer lru on freeze and read-only remount 2021-01-22 16:54:50 -08:00
xfs_dir2_readdir.c xfs: move the fork format fields into struct xfs_ifork 2020-05-19 09:40:58 -07:00
xfs_discard.c
xfs_discard.h
xfs_dquot_item_recover.c xfs: rename the ondisk dquot d_flags to d_type 2020-07-28 20:24:14 -07:00
xfs_dquot_item.c xfs: stop using q_core.d_id in the quota code 2020-07-28 20:24:14 -07:00
xfs_dquot_item.h
xfs_dquot.c xfs: fix incorrect root dquot corruption error when switching group/project quota types 2021-02-04 09:10:38 -08:00
xfs_dquot.h xfs: refactor default quota grace period setting code 2020-09-15 20:52:40 -07:00
xfs_error.c xfs: ensure xfs_errortag_random_default matches XFS_ERRTAG_MAX 2021-03-25 16:47:50 -07:00
xfs_error.h
xfs_export.c xfs: delete duplicated words + other fixes 2020-08-05 08:49:58 -07:00
xfs_export.h
xfs_extent_busy.c xfs: don't reuse busy extents on extent trim 2021-02-25 07:58:46 -08:00
xfs_extent_busy.h
xfs_extfree_item.c xfs: refactor data device extent validation 2020-12-09 09:49:38 -08:00
xfs_extfree_item.h xfs: refactor intent item RECOVERED flag into the log item 2020-05-08 08:50:01 -07:00
xfs_file.c idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
xfs_filestream.c xfs: drop the obsolete comment on filestream locking 2020-09-25 11:34:08 -07:00
xfs_filestream.h
xfs_fsmap.c xfs: drop freeze protection when running GETFSMAP 2021-03-24 10:36:05 -07:00
xfs_fsmap.h xfs: fix deadlock and streamline xfs_getfsmap performance 2020-10-07 08:40:29 -07:00
xfs_fsops.c xfs: get rid of xfs_growfs_{data,log}_t 2021-02-03 09:18:50 -08:00
xfs_fsops.h xfs: get rid of xfs_growfs_{data,log}_t 2021-02-03 09:18:50 -08:00
xfs_globals.c xfs: consolidate the eofblocks and cowblocks workers 2021-02-03 09:18:49 -08:00
xfs_health.c
xfs_icache.c xfs: rename the blockgc workqueue 2021-03-25 16:47:50 -07:00
xfs_icache.h xfs: rename block gc start and stop functions 2021-02-03 09:18:50 -08:00
xfs_icreate_item.c xfs: Remove kmem_zone_zalloc() usage 2020-07-28 20:24:14 -07:00
xfs_icreate_item.h
xfs_inode_item_recover.c xfs: widen ondisk inode timestamps to deal with y2038+ 2020-09-15 20:52:41 -07:00
xfs_inode_item.c xfs: widen ondisk inode timestamps to deal with y2038+ 2020-09-15 20:52:41 -07:00
xfs_inode_item.h xfs: move the buffer retry logic to xfs_buf.c 2020-09-15 20:52:38 -07:00
xfs_inode.c xfs: initialise attr fork on inode create 2021-03-25 16:47:51 -07:00
xfs_inode.h xfs: initialise attr fork on inode create 2021-03-25 16:47:51 -07:00
xfs_ioctl32.c xfs: remove the possibly unused mp variable in xfs_file_compat_ioctl 2021-02-03 20:31:14 +01:00
xfs_ioctl32.h
xfs_ioctl.c idmapped-mounts-v5.12 2021-02-23 13:39:45 -08:00
xfs_ioctl.h
xfs_iomap.c New code for 5.12: 2021-02-21 10:34:36 -08:00
xfs_iomap.h
xfs_iops.c xfs: initialise attr fork on inode create 2021-03-25 16:47:51 -07:00
xfs_iops.h xfs: support idmapped mounts 2021-01-24 14:43:46 +01:00
xfs_itable.c xfs: also reject BULKSTAT_SINGLE in a mount user namespace 2021-03-15 08:50:41 -07:00
xfs_itable.h xfs: support idmapped mounts 2021-01-24 14:43:46 +01:00
xfs_iwalk.c xfs: increase the default parallelism levels of pwork clients 2021-02-03 09:18:49 -08:00
xfs_iwalk.h
xfs_linux.h xfs: consolidate the eofblocks and cowblocks workers 2021-02-03 09:18:49 -08:00
xfs_log_cil.c xfs: delete duplicated words + other fixes 2020-08-05 08:49:58 -07:00
xfs_log_priv.h xfs: Modify xlog_ticket_alloc() to use kernel's MM API 2020-07-28 20:24:14 -07:00
xfs_log_recover.c xfs: remove xfs_buf_t typedef 2020-12-16 16:07:34 -08:00
xfs_log.c xfs: fix boolreturn.cocci warnings 2021-02-10 17:28:13 -08:00
xfs_log.h xfs: cover the log during log quiesce 2021-01-22 16:54:51 -08:00
xfs_message.c
xfs_message.h treewide: Convert macro and uses of __section(foo) to __section("foo") 2020-10-25 14:51:49 -07:00
xfs_mount.c xfs: force log and push AIL to clear pinned inodes when aborting mount 2021-03-15 08:50:40 -07:00
xfs_mount.h xfs: rename the blockgc workqueue 2021-03-25 16:47:50 -07:00
xfs_mru_cache.c xfs: set WQ_SYSFS on all workqueues in debug mode 2021-02-03 09:18:49 -08:00
xfs_mru_cache.h
xfs_ondisk.h xfs: Remove typedef xfs_attr_shortform_t 2020-09-15 20:52:42 -07:00
xfs_pnfs.c xfs: fix a missing unlock on error in xfs_fs_map_blocks 2020-11-11 08:07:37 -08:00
xfs_pnfs.h
xfs_pwork.c xfs: increase the default parallelism levels of pwork clients 2021-02-03 09:18:49 -08:00
xfs_pwork.h xfs: increase the default parallelism levels of pwork clients 2021-02-03 09:18:49 -08:00
xfs_qm_bhv.c xfs: rename XFS_DQ_{USER,GROUP,PROJ} to XFS_DQTYPE_* 2020-07-28 20:24:14 -07:00
xfs_qm_syscalls.c xfs: refactor default quota grace period setting code 2020-09-15 20:52:40 -07:00
xfs_qm.c xfs: initialise attr fork on inode create 2021-03-25 16:47:51 -07:00
xfs_qm.h xfs: refactor quota expiration timer modification 2020-09-15 20:52:40 -07:00
xfs_quota.h xfs: remove xfs_qm_vop_chown_reserve 2021-02-03 09:18:49 -08:00
xfs_quotaops.c xfs: create xfs_dqtype_t to represent quota types 2020-07-28 20:24:14 -07:00
xfs_refcount_item.c xfs: refactor data device extent validation 2020-12-09 09:49:38 -08:00
xfs_refcount_item.h xfs: refactor intent item RECOVERED flag into the log item 2020-05-08 08:50:01 -07:00
xfs_reflink.c xfs: flush eof/cowblocks if we can't reserve quota for file blocks 2021-02-03 09:18:49 -08:00
xfs_reflink.h xfs: move helpers that lock and unlock two inodes against userspace IO 2020-07-06 10:46:57 -07:00
xfs_rmap_item.c xfs: refactor file range validation 2020-12-09 09:49:38 -08:00
xfs_rmap_item.h xfs: refactor intent item RECOVERED flag into the log item 2020-05-08 08:50:01 -07:00
xfs_rtalloc.c xfs: Check for extent overflow when trivally adding a new extent 2021-01-22 16:54:47 -08:00
xfs_rtalloc.h xfs: remove xfs_buf_t typedef 2020-12-16 16:07:34 -08:00
xfs_stats.c xfs: periodically relog deferred intent items 2020-10-07 08:40:28 -07:00
xfs_stats.h xfs: periodically relog deferred intent items 2020-10-07 08:40:28 -07:00
xfs_super.c xfs: Skip repetitive warnings about mount options 2021-03-25 16:47:50 -07:00
xfs_super.h xfs: set WQ_SYSFS on all workqueues in debug mode 2021-02-03 09:18:49 -08:00
xfs_symlink.c xfs: initialise attr fork on inode create 2021-03-25 16:47:51 -07:00
xfs_symlink.h xfs: support idmapped mounts 2021-01-24 14:43:46 +01:00
xfs_sysctl.c xfs: restore speculative_cow_prealloc_lifetime sysctl 2021-02-24 10:16:08 -08:00
xfs_sysctl.h xfs: consolidate the eofblocks and cowblocks workers 2021-02-03 09:18:49 -08:00
xfs_sysfs.c
xfs_sysfs.h xfs: Fix UBSAN null-ptr-deref in xfs_sysfs_init 2020-08-07 11:50:17 -07:00
xfs_trace.c xfs: add a tracepoint for blockgc scans 2021-02-03 09:18:49 -08:00
xfs_trace.h New code for 5.12: 2021-02-21 10:34:36 -08:00
xfs_trans_ail.c xfs: delete duplicated words + other fixes 2020-08-05 08:49:58 -07:00
xfs_trans_buf.c xfs: remove xfs_buf_t typedef 2020-12-16 16:07:34 -08:00
xfs_trans_dquot.c xfs: shut down the filesystem if we screw up quota reservation 2021-02-03 09:18:49 -08:00
xfs_trans_priv.h xfs: refactor adding recovered intent items to the log 2020-05-08 08:50:00 -07:00
xfs_trans.c xfs: use current->journal_info for detecting transaction recursion 2021-02-25 08:07:04 -08:00
xfs_trans.h xfs: use current->journal_info for detecting transaction recursion 2021-02-25 08:07:04 -08:00
xfs_xattr.c xfs: prevent metadata files from being inactivated 2021-03-25 16:47:50 -07:00
xfs.h