linux/fs/xfs/libxfs
Brian Foster 1ca89fbc48 xfs: don't account extra agfl blocks as available
The block allocation AG selection code has parameters that allow a
caller to perform multiple allocations from a single AG and
transaction (under certain conditions). The parameters specify the
total block allocation count required by the transaction and the AG
selection code selects and locks an AG that will be able to satisfy
the overall requirement. If the available block accounting
calculation turns out to be inaccurate and a subsequent allocation
call fails with -ENOSPC, the resulting transaction cancel leads to
filesystem shutdown because the transaction is dirty.

This exact problem can be reproduced with a highly parallel space
consumer and fsstress workload running long enough to a large
filesystem against -ENOSPC conditions. A bmbt block allocation
request made for inode extent to bmap format conversion after an
extent allocation is expected to be satisfied by the same AG and the
same transaction as the extent allocation. The bmbt block allocation
fails, however, because the block availability of the AG has changed
since the AG was selected (outside of the blocks used for the extent
itself).

The inconsistent block availability calculation is caused by the
deferred block freeing behavior of the AGFL. This immediately
removes extra blocks from the AGFL to free up AGFL slots, but rather
than immediately freeing such blocks as was done in the past, the
block free is deferred such that said blocks are not available for
allocation until the current transaction commits. The AG selection
logic currently considers all AGFL blocks as available and executes
shortly before any extra AGFL blocks are freed. This means the block
availability of the current AG can change before the first
allocation even occurs, but in practice a failure is more likely to
manifest via a subsequent allocation because extent allocation
usually has a contiguity requirement larger than a single block that
can't be satisfied from the AGFL.

In general, XFS prefers operational robustness to absolute
allocation efficiency. In other words, we prefer to return -ENOSPC
slightly earlier at the expense of not being able to allocate every
last block in an AG to avoid this kind of problem. As such, update
the AG block availability calculation to consider extra AGFL blocks
as unavailable since they are immediately removed following the
calculation and will not become available until the current
transaction commits.

Signed-off-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
2019-04-14 18:15:57 -07:00
..
xfs_ag_resv.c xfs: rename m_inotbt_nores to m_finobt_nores 2019-02-14 22:42:57 -08:00
xfs_ag_resv.h xfs: pass transaction lock while setting up agresv on cyclic metadata 2018-07-29 22:37:08 -07:00
xfs_ag.c xfs: split up allocation btree verifier 2019-02-11 16:07:01 -08:00
xfs_ag.h xfs: factor the ag length extension code into libxfs 2018-05-15 18:12:51 -07:00
xfs_alloc_btree.c xfs: distinguish between bnobt and cntbt magic values 2019-02-11 16:07:01 -08:00
xfs_alloc_btree.h xfs: convert to SPDX license tags 2018-06-06 14:17:53 -07:00
xfs_alloc.c xfs: don't account extra agfl blocks as available 2019-04-14 18:15:57 -07:00
xfs_alloc.h xfs: const-ify xfs_owner_info arguments 2018-12-12 08:47:16 -08:00
xfs_attr_leaf.c xfs: fix xfs_buf magic number endian checks 2019-02-18 09:38:41 -08:00
xfs_attr_leaf.h xfs: convert to SPDX license tags 2018-06-06 14:17:53 -07:00
xfs_attr_remote.c xfs: miscellaneous verifier magic value fixups 2019-02-11 16:07:01 -08:00
xfs_attr_remote.h xfs: convert to SPDX license tags 2018-06-06 14:17:53 -07:00
xfs_attr_sf.h xfs: convert to SPDX license tags 2018-06-06 14:17:53 -07:00
xfs_attr.c xfs: check attribute name validity 2019-02-11 16:06:40 -08:00
xfs_attr.h xfs: check attribute name validity 2019-02-11 16:06:40 -08:00
xfs_bit.c xfs: convert to SPDX license tags 2018-06-06 14:17:53 -07:00
xfs_bit.h xfs: convert to SPDX license tags 2018-06-06 14:17:53 -07:00
xfs_bmap_btree.c xfs: miscellaneous verifier magic value fixups 2019-02-11 16:07:01 -08:00
xfs_bmap_btree.h xfs: convert to SPDX license tags 2018-06-06 14:17:53 -07:00
xfs_bmap.c xfs: always init bma in xfs_bmapi_write 2019-03-19 08:16:54 -07:00
xfs_bmap.h xfs: make COW fork unwritten extent conversions more robust 2019-02-21 07:55:07 -08:00
xfs_btree.c xfs: fix inverted return from xfs_btree_sblock_verify_crc 2018-12-04 08:50:49 -08:00
xfs_btree.h xfs: fold dfops into the transaction 2018-08-02 23:05:14 -07:00
xfs_cksum.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
xfs_da_btree.c xfs: fix xfs_buf magic number endian checks 2019-02-18 09:38:41 -08:00
xfs_da_btree.h xfs: fold dfops into the transaction 2018-08-02 23:05:14 -07:00
xfs_da_format.c xfs: convert to SPDX license tags 2018-06-06 14:17:53 -07:00
xfs_da_format.h xfs: factor xfs_da3_blkinfo verification into common helper 2019-02-11 16:07:01 -08:00
xfs_defer.c xfs: streamline defer op type handling 2018-12-12 08:47:16 -08:00
xfs_defer.h xfs: streamline defer op type handling 2018-12-12 08:47:16 -08:00
xfs_dir2_block.c xfs: miscellaneous verifier magic value fixups 2019-02-11 16:07:01 -08:00
xfs_dir2_data.c xfs: miscellaneous verifier magic value fixups 2019-02-11 16:07:01 -08:00
xfs_dir2_leaf.c xfs: clean up xfs_dir2_leaf_addname 2019-03-12 09:19:38 -07:00
xfs_dir2_node.c xfs: clean up xfs_dir2_leafn_add 2019-03-08 14:24:43 -08:00
xfs_dir2_priv.h xfs: convert to SPDX license tags 2018-06-06 14:17:53 -07:00
xfs_dir2_sf.c xfs: convert to SPDX license tags 2018-06-06 14:17:53 -07:00
xfs_dir2.c xfs: check directory name validity 2019-02-11 16:06:40 -08:00
xfs_dir2.h xfs: check directory name validity 2019-02-11 16:06:40 -08:00
xfs_dquot_buf.c xfs: fix xfs_buf magic number endian checks 2019-02-18 09:38:41 -08:00
xfs_errortag.h xfs: cache unlinked pointers in an rhashtable 2019-02-11 16:07:01 -08:00
xfs_format.h xfs: move XFS_INODE_FORMAT_STR mappings to libxfs 2018-12-19 14:02:01 -08:00
xfs_fs.h xfs: convert to SPDX license tags 2018-06-06 14:17:53 -07:00
xfs_ialloc_btree.c xfs: rename m_inotbt_nores to m_finobt_nores 2019-02-14 22:42:57 -08:00
xfs_ialloc_btree.h xfs: pass transaction lock while setting up agresv on cyclic metadata 2018-07-29 22:37:08 -07:00
xfs_ialloc.c xfs: miscellaneous verifier magic value fixups 2019-02-11 16:07:01 -08:00
xfs_ialloc.h xfs: remove dfops parameter from ifree call stack 2018-07-11 22:26:07 -07:00
xfs_iext_tree.c xfs: update fork seq counter on data fork changes 2019-02-11 16:07:00 -08:00
xfs_inode_buf.c xfs: fix xfs_buf magic number endian checks 2019-02-18 09:38:41 -08:00
xfs_inode_buf.h xfs: convert to SPDX license tags 2018-06-06 14:17:53 -07:00
xfs_inode_fork.c xfs: remove the xfs_ifork_t typedef 2018-07-30 07:57:48 -07:00
xfs_inode_fork.h xfs: update fork seq counter on data fork changes 2019-02-11 16:07:00 -08:00
xfs_log_format.h xfs: refactor unmount record write 2018-07-23 09:08:01 -07:00
xfs_log_recover.h xfs: convert to SPDX license tags 2018-06-06 14:17:53 -07:00
xfs_log_rlimit.c xfs: convert to SPDX license tags 2018-06-06 14:17:53 -07:00
xfs_quota_defs.h xfs: convert to SPDX license tags 2018-06-06 14:17:53 -07:00
xfs_refcount_btree.c xfs: miscellaneous verifier magic value fixups 2019-02-11 16:07:01 -08:00
xfs_refcount_btree.h xfs: pass transaction lock while setting up agresv on cyclic metadata 2018-07-29 22:37:08 -07:00
xfs_refcount.c xfs: pass transaction to xfs_defer_add() 2018-08-02 23:05:14 -07:00
xfs_refcount.h xfs: pass transaction to xfs_defer_add() 2018-08-02 23:05:14 -07:00
xfs_rmap_btree.c xfs: miscellaneous verifier magic value fixups 2019-02-11 16:07:01 -08:00
xfs_rmap_btree.h xfs: pass transaction lock while setting up agresv on cyclic metadata 2018-07-29 22:37:08 -07:00
xfs_rmap.c xfs: remove xfs_rmap_ag_owner and friends 2018-12-12 08:47:16 -08:00
xfs_rmap.h xfs: remove xfs_rmap_ag_owner and friends 2018-12-12 08:47:16 -08:00
xfs_rtbitmap.c xfs: cache minimum realtime summary level 2018-12-12 08:47:17 -08:00
xfs_sb.c xfs: fix uninitialized error variables 2019-02-25 10:16:41 -08:00
xfs_sb.h xfs: convert to SPDX license tags 2018-06-06 14:17:53 -07:00
xfs_shared.h xfs: split up allocation btree verifier 2019-02-11 16:07:01 -08:00
xfs_symlink_remote.c xfs: miscellaneous verifier magic value fixups 2019-02-11 16:07:01 -08:00
xfs_trans_resv.c xfs: clean up MIN/MAX 2018-06-08 10:07:52 -07:00
xfs_trans_resv.h xfs: convert to SPDX license tags 2018-06-06 14:17:53 -07:00
xfs_trans_space.h xfs: convert to SPDX license tags 2018-06-06 14:17:53 -07:00
xfs_types.c xfs: add xfs_verify_agino_or_null helper 2019-02-11 16:07:01 -08:00
xfs_types.h xfs: add xfs_verify_agino_or_null helper 2019-02-11 16:07:01 -08:00