linux/drivers/md
Ming Lei 8b211aaccb dm: add two stage requeue mechanism
Commit 61b6e2e532 ("dm: fix BLK_STS_DM_REQUEUE handling when dm_io
represents split bio") reverted DM core's bio splitting back to using
bio_split()+bio_chain() because it was found that otherwise DM's
BLK_STS_DM_REQUEUE would trigger a live-lock waiting for bio
completion that would never occur.

Restore using bio_trim()+bio_inc_remaining(), like was done in commit
7dd76d1fee ("dm: improve bio splitting and associated IO
accounting"), but this time with proper handling for the above
scenario that is covered in more detail in the commit header for
61b6e2e532.

Solve this issue by adding a two staged dm_io requeue mechanism that
uses the new dm_bio_rewind() via dm_io_rewind():

1) requeue the dm_io into the requeue_list added to struct
   mapped_device, and schedule it via new added requeue work. This
   workqueue just clones the dm_io->orig_bio (which DM saves and
   ensures its end sector isn't modified). dm_io_rewind() uses the
   sectors and sectors_offset members of the dm_io that are recorded
   relative to the end of orig_bio: dm_bio_rewind()+bio_trim() are
   then used to make that cloned bio reflect the subset of the
   original bio that is represented by the dm_io that is being
   requeued.

2) the 2nd stage requeue is same with original requeue, but
   io->orig_bio points to new cloned bio (which matches the requeued
   dm_io as described above).

This allows DM core to shift the need for bio cloning from bio-split
time (during IO submission) to the less likely BLK_STS_DM_REQUEUE
handling (after IO completes with that error).

Signed-off-by: Ming Lei <ming.lei@redhat.com>
Signed-off-by: Mike Snitzer <snitzer@kernel.org>
2022-07-07 11:49:32 -04:00
..
bcache block: remove blk_cleanup_disk 2022-06-28 06:33:15 -06:00
persistent-data dm space map common: add bounds check to sm_ll_lookup_bitmap() 2022-01-04 13:58:19 -05:00
dm-audit.c dm: introduce audit event module for device mapper 2021-10-27 16:53:47 -04:00
dm-audit.h dm: introduce audit event module for device mapper 2021-10-27 16:53:47 -04:00
dm-bio-prison-v1.c
dm-bio-prison-v1.h
dm-bio-prison-v2.c
dm-bio-prison-v2.h
dm-bio-record.h block: move integrity handling out of <linux/blkdev.h> 2021-10-18 06:17:02 -06:00
dm-bufio.c block: turn bio_kmalloc into a simple kmalloc wrapper 2022-04-17 19:30:41 -06:00
dm-builtin.c
dm-cache-background-tracker.c
dm-cache-background-tracker.h
dm-cache-block-types.h
dm-cache-metadata.c dm cache metadata: remove unnecessary variable in __dump_mapping 2022-05-09 15:40:10 -04:00
dm-cache-metadata.h
dm-cache-policy-internal.h
dm-cache-policy-smq.c dm cache policy smq: make static read-only array table const 2022-02-22 10:35:53 -05:00
dm-cache-policy.c
dm-cache-policy.h
dm-cache-target.c block: remove QUEUE_FLAG_DISCARD 2022-04-17 19:49:59 -06:00
dm-clone-metadata.c dm clone metadata: remove unused function 2021-04-19 13:20:31 -04:00
dm-clone-metadata.h
dm-clone-target.c block: remove QUEUE_FLAG_DISCARD 2022-04-17 19:49:59 -06:00
dm-core.h dm: add two stage requeue mechanism 2022-07-07 11:49:32 -04:00
dm-crypt.c dm crypt: make printing of the key constant-time 2022-05-09 12:34:03 -04:00
dm-delay.c dm: simplify basic targets 2022-05-05 17:31:35 -04:00
dm-dust.c dm: use bdev_nr_sectors and bdev_nr_bytes instead of open coding them 2021-10-18 14:43:22 -06:00
dm-ebs-target.c scsi: dm: Remove WRITE_SAME support 2022-02-22 21:11:08 -05:00
dm-era-target.c dm era: commit metadata in postsuspend after worker stops 2022-06-21 13:35:01 -04:00
dm-exception-store.c
dm-exception-store.h dm: use bdev_nr_sectors and bdev_nr_bytes instead of open coding them 2021-10-18 14:43:22 -06:00
dm-flakey.c dm: simplify basic targets 2022-05-05 17:31:35 -04:00
dm-ima.c dm ima: fix wrong length calculation for no_data string 2022-02-22 10:42:41 -05:00
dm-ima.h dm ima: add version info to dm related events in ima log 2021-08-20 15:59:47 -04:00
dm-init.c dm init: Set file local variable static 2020-08-04 15:51:28 -04:00
dm-integrity.c dm integrity: fix error code in dm_integrity_ctr() 2022-05-09 12:14:00 -04:00
dm-io-rewind.c dm: add two stage requeue mechanism 2022-07-07 11:49:32 -04:00
dm-io-tracker.h dm writecache: make writeback pause configurable 2021-06-28 16:30:13 -04:00
dm-io.c block: add a bdev_max_discard_sectors helper 2022-04-17 19:49:59 -06:00
dm-ioctl.c dm ioctl: log an error if the ioctl structure is corrupted 2022-04-01 10:29:43 -04:00
dm-kcopyd.c dm writecache: have ssd writeback wait if the kcopyd workqueue is busy 2021-06-15 15:42:03 -04:00
dm-linear.c libnvdimm for 5.19 2022-05-27 15:49:30 -07:00
dm-log-userspace-base.c dm: update target status functions to support IMA measurement 2021-08-10 13:34:23 -04:00
dm-log-userspace-transfer.c
dm-log-userspace-transfer.h
dm-log-writes.c libnvdimm for 5.19 2022-05-27 15:49:30 -07:00
dm-log.c dm mirror log: clear log bits up to BITS_PER_LONG boundary 2022-06-23 14:55:43 -04:00
dm-mpath.c dm mpath: provide high-resolution timer to HST for bio-based 2022-05-09 15:39:23 -04:00
dm-mpath.h
dm-path-selector.c
dm-path-selector.h dm mpath: provide high-resolution timer to HST for bio-based 2022-05-09 15:39:23 -04:00
dm-ps-historical-service-time.c dm mpath: provide high-resolution timer to HST for bio-based 2022-05-09 15:39:23 -04:00
dm-ps-io-affinity.c dm: update target status functions to support IMA measurement 2021-08-10 13:34:23 -04:00
dm-ps-queue-length.c dm: update target status functions to support IMA measurement 2021-08-10 13:34:23 -04:00
dm-ps-round-robin.c dm: update target status functions to support IMA measurement 2021-08-10 13:34:23 -04:00
dm-ps-service-time.c dm: update target status functions to support IMA measurement 2021-08-10 13:34:23 -04:00
dm-raid1.c dm: update target status functions to support IMA measurement 2021-08-10 13:34:23 -04:00
dm-raid.c Revert "md: don't unregister sync_thread with reconfig_mutex held" 2022-06-15 10:30:14 -07:00
dm-region-hash.c
dm-rq.c dm: unexport dm_get_reserved_rq_based_ios 2022-06-29 12:46:05 -04:00
dm-rq.h
dm-snap-persistent.c dm: update target status functions to support IMA measurement 2021-08-10 13:34:23 -04:00
dm-snap-transient.c dm: update target status functions to support IMA measurement 2021-08-10 13:34:23 -04:00
dm-snap.c dm-snap: use blkdev_issue_flush instead of open coding it 2022-02-02 07:49:59 -07:00
dm-stats.c dm stats: add cond_resched when looping over entries 2022-05-09 12:11:07 -04:00
dm-stats.h dm stats: fix too short end duration_ns when using precise_timestamps 2022-02-21 15:35:39 -05:00
dm-stripe.c dax: add .recovery_write dax_operation 2022-05-16 13:37:59 -07:00
dm-switch.c dm: use bdev_nr_sectors and bdev_nr_bytes instead of open coding them 2021-10-18 14:43:22 -06:00
dm-sysfs.c dm sysfs: use default_groups in kobj_type 2022-01-06 09:48:55 -05:00
dm-table.c dm: refactor dm_md_mempool allocation 2022-06-29 12:46:06 -04:00
dm-target.c dax: introduce DAX_RECOVERY_WRITE dax access mode 2022-05-16 13:35:56 -07:00
dm-thin-metadata.c dm thin metadata: remove unused dm_thin_remove_block and __remove 2022-02-22 13:55:50 -05:00
dm-thin-metadata.h dm thin metadata: remove unused dm_thin_remove_block and __remove 2022-02-22 13:55:50 -05:00
dm-thin.c block: decouple REQ_OP_SECURE_ERASE from REQ_OP_DISCARD 2022-04-17 19:49:59 -06:00
dm-uevent.c
dm-uevent.h
dm-unstripe.c dm: update target status functions to support IMA measurement 2021-08-10 13:34:23 -04:00
dm-verity-fec.c dm verity fec: fix misaligned RS roots IO 2021-04-14 14:28:29 -04:00
dm-verity-fec.h dm verity fec: fix misaligned RS roots IO 2021-04-14 14:28:29 -04:00
dm-verity-target.c dm verity: set DM_TARGET_IMMUTABLE feature flag 2022-05-31 16:22:30 -04:00
dm-verity-verify-sig.c dm verity: fix require_signatures module_param permissions 2021-05-25 16:14:05 -04:00
dm-verity-verify-sig.h dm verity: Fix compilation warning 2020-08-04 15:48:13 -04:00
dm-verity.h dm verity: add "panic_on_corruption" error handling mode 2020-07-13 11:47:33 -04:00
dm-writecache.c dax: introduce DAX_RECOVERY_WRITE dax access mode 2022-05-16 13:35:56 -07:00
dm-zero.c dm: add support for REQ_NOWAIT to various targets 2020-12-04 18:04:35 -05:00
dm-zone.c dm: don't grab target io reference in dm_zone_map_bio 2022-05-05 17:31:36 -04:00
dm-zoned-metadata.c dm-zoned: remove the ->name field in struct dmz_dev 2022-03-02 12:15:35 -05:00
dm-zoned-reclaim.c dm kcopyd: avoid useless atomic operations 2021-06-04 12:07:24 -04:00
dm-zoned-target.c dm-zoned: don't set the discard_alignment queue limit 2022-05-03 10:38:50 -06:00
dm-zoned.h dm-zoned: remove the ->name field in struct dmz_dev 2022-03-02 12:15:35 -05:00
dm.c dm: add two stage requeue mechanism 2022-07-07 11:49:32 -04:00
dm.h dm: refactor dm_md_mempool allocation 2022-06-29 12:46:06 -04:00
Kconfig blk-mq: make the blk-mq stacking code optional 2022-02-16 19:39:09 -07:00
Makefile dm: add dm_bio_rewind() API to DM core 2022-07-07 11:26:55 -04:00
md-autodetect.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
md-bitmap.c md: replace deprecated strlcpy & remove duplicated line 2022-04-25 14:00:36 -07:00
md-bitmap.h
md-cluster.c md: replace deprecated strlcpy & remove duplicated line 2022-04-25 14:00:36 -07:00
md-cluster.h
md-faulty.c block: pass a block_device to bio_clone_fast 2022-02-04 07:43:18 -07:00
md-linear.c md: remove most calls to bdevname 2022-05-22 23:07:21 -07:00
md-linear.h md/raid1: Replace zero-length array with flexible-array 2020-05-13 12:02:23 -07:00
md-multipath.c md: remove most calls to bdevname 2022-05-22 23:07:21 -07:00
md-multipath.h
md.c block: remove blk_cleanup_disk 2022-06-28 06:33:15 -06:00
md.h Revert "md: don't unregister sync_thread with reconfig_mutex held" 2022-06-15 10:30:14 -07:00
raid0.c md: Don't set mddev private to NULL in raid0 pers->free 2022-05-22 23:07:21 -07:00
raid0.h
raid1-10.c md: raid1/raid10: drop pending_cnt 2022-03-08 15:16:54 -08:00
raid1.c md: remove most calls to bdevname 2022-05-22 23:07:21 -07:00
raid1.h md: raid1/raid10: drop pending_cnt 2022-03-08 15:16:54 -08:00
raid5-cache.c md: remove most calls to bdevname 2022-05-22 23:07:21 -07:00
raid5-log.h
raid5-ppl.c md/raid5-ppl: Fix argument order in bio_alloc_bioset() 2022-06-15 10:32:48 -07:00
raid5.c md: remove most calls to bdevname 2022-05-22 23:07:21 -07:00
raid5.h md/raid5: Add __rcu annotation to struct disk_info 2022-04-25 14:00:36 -07:00
raid10.c md: remove most calls to bdevname 2022-05-22 23:07:21 -07:00
raid10.h md: raid1/raid10: drop pending_cnt 2022-03-08 15:16:54 -08:00