linux/drivers/iommu
Jason Gunthorpe f394576eb1 iommufd: PFN handling for iopt_pages
The top of the data structure provides an IO Address Space (IOAS) that is
similar to a VFIO container. The IOAS allows map/unmap of memory into
ranges of IOVA called iopt_areas. Multiple IOMMU domains (IO page tables)
and in-kernel accesses (like VFIO mdevs) can be attached to the IOAS to
access the PFNs that those IOVA areas cover.

The IO Address Space (IOAS) datastructure is composed of:
 - struct io_pagetable holding the IOVA map
 - struct iopt_areas representing populated portions of IOVA
 - struct iopt_pages representing the storage of PFNs
 - struct iommu_domain representing each IO page table in the system IOMMU
 - struct iopt_pages_access representing in-kernel accesses of PFNs (ie
   VFIO mdevs)
 - struct xarray pinned_pfns holding a list of pages pinned by in-kernel
   accesses

This patch introduces the lowest part of the datastructure - the movement
of PFNs in a tiered storage scheme:
 1) iopt_pages::pinned_pfns xarray
 2) Multiple iommu_domains
 3) The origin of the PFNs, i.e. the userspace pointer

PFN have to be copied between all combinations of tiers, depending on the
configuration.

The interface is an iterator called a 'pfn_reader' which determines which
tier each PFN is stored and loads it into a list of PFNs held in a struct
pfn_batch.

Each step of the iterator will fill up the pfn_batch, then the caller can
use the pfn_batch to send the PFNs to the required destination. Repeating
this loop will read all the PFNs in an IOVA range.

The pfn_reader and pfn_batch also keep track of the pinned page accounting.

While PFNs are always stored and accessed as full PAGE_SIZE units the
iommu_domain tier can store with a sub-page offset/length to support
IOMMUs with a smaller IOPTE size than PAGE_SIZE.

Link: https://lore.kernel.org/r/8-v6-a196d26f289e+11787-iommufd_jgg@nvidia.com
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Tested-by: Nicolin Chen <nicolinc@nvidia.com>
Tested-by: Yi Liu <yi.l.liu@intel.com>
Tested-by: Lixiao Yang <lixiao.yang@intel.com>
Tested-by: Matthew Rosato <mjrosato@linux.ibm.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2022-11-30 20:16:49 -04:00
..
amd iommu: Add IOMMU_CAP_ENFORCE_CACHE_COHERENCY 2022-11-29 16:34:15 -04:00
arm iommu: Define EINVAL as device/domain incompatibility 2022-11-03 15:51:48 +01:00
intel iommu: Add IOMMU_CAP_ENFORCE_CACHE_COHERENCY 2022-11-29 16:34:15 -04:00
iommufd iommufd: PFN handling for iopt_pages 2022-11-30 20:16:49 -04:00
apple-dart.c iommu: Add gfp parameter to iommu_alloc_resv_region 2022-10-21 10:49:32 +02:00
dma-iommu.c iommu/dma: Make header private 2022-09-09 09:26:22 +02:00
dma-iommu.h iommu/dma: Make header private 2022-09-09 09:26:22 +02:00
exynos-iommu.c iommu/exynos: Clean up bus_set_iommu() 2022-09-07 14:26:14 +02:00
fsl_pamu_domain.c iommu: Regulate EINVAL in ->attach_dev callback functions 2022-11-01 14:39:59 -03:00
fsl_pamu_domain.h iommu/fsl_pamu: remove the snoop_id field 2021-04-07 10:56:52 +02:00
fsl_pamu.c iommu: Regulate EINVAL in ->attach_dev callback functions 2022-11-01 14:39:59 -03:00
fsl_pamu.h iommu/fsl_pamu: hardcode the window address and size in pamu_config_ppaace 2021-04-07 10:56:52 +02:00
hyperv-iommu.c iommu/hyper-v: Use helper instead of directly accessing affinity 2022-08-04 10:02:09 +01:00
io-pgfault.c iommu: Rename iommu-sva-lib.{c,h} 2022-11-03 15:47:54 +01:00
io-pgtable-arm-v7s.c iommu/io-pgtable-arm-v7s: Add a quirk to allow pgtable PA up to 35bit 2022-07-07 09:42:59 +02:00
io-pgtable-arm.c Merge branches 'apple/dart', 'arm/mediatek', 'arm/omap', 'arm/smmu', 'virtio', 'x86/vt-d', 'x86/amd' and 'core' into next 2022-09-26 15:52:31 +02:00
io-pgtable-arm.h iommu/io-pgtable-arm: Move some definitions to a header 2020-09-28 23:48:06 +01:00
io-pgtable-dart.c iommu/io-pgtable-dart: Add DART PTE support for t6000 2022-09-26 13:49:40 +02:00
io-pgtable.c Merge branches 'apple/dart', 'arm/mediatek', 'arm/omap', 'arm/smmu', 'virtio', 'x86/vt-d', 'x86/amd' and 'core' into next 2022-09-26 15:52:31 +02:00
ioasid.c iommu/sva: Assign a PASID to mm on PASID allocation and free it on mm exit 2022-02-15 11:31:35 +01:00
iommu-debugfs.c iommu: Fix IOMMU debugfs fallout 2019-02-26 11:15:58 +01:00
iommu-sva.c iommu: Rename iommu-sva-lib.{c,h} 2022-11-03 15:47:54 +01:00
iommu-sva.h iommu: Rename iommu-sva-lib.{c,h} 2022-11-03 15:47:54 +01:00
iommu-sysfs.c drivers/iommu: Export core IOMMU API symbols to permit modular drivers 2019-12-23 14:06:05 +01:00
iommu-traces.c
iommu.c iommu: Add device-centric DMA ownership interfaces 2022-11-29 16:34:15 -04:00
iova.c iova: Remove iovad->rcaches check in iova_rcache_get() 2022-09-09 09:27:03 +02:00
ipmmu-vmsa.c iommu: Use EINVAL for incompatible device/domain in ->attach_dev 2022-11-01 14:39:59 -03:00
irq_remapping.c x86: Kill all traces of irq_remapping_get_irq_domain() 2020-10-28 20:26:28 +01:00
irq_remapping.h x86: Kill all traces of irq_remapping_get_irq_domain() 2020-10-28 20:26:28 +01:00
Kconfig iommufd: File descriptor, context, kconfig and makefiles 2022-11-30 20:16:49 -04:00
Makefile iommufd: File descriptor, context, kconfig and makefiles 2022-11-30 20:16:49 -04:00
msm_iommu_hw-8xxx.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 267 2019-06-05 17:30:29 +02:00
msm_iommu.c iommu: Clean up bus_set_iommu() 2022-09-07 14:26:17 +02:00
msm_iommu.h treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 267 2019-06-05 17:30:29 +02:00
mtk_iommu_v1.c iommu/mtk: Clean up bus_set_iommu() 2022-09-07 14:26:15 +02:00
mtk_iommu.c iommu: Propagate return value in ->attach_dev callback functions 2022-11-01 14:39:59 -03:00
of_iommu.c Revert "iommu/of: Delete usage of driver_deferred_probe_check_state()" 2022-08-23 13:14:02 +02:00
omap-iommu-debug.c iommu/omap: Fix buffer overflow in debugfs 2022-09-07 10:42:28 +02:00
omap-iommu.c iommu: Use EINVAL for incompatible device/domain in ->attach_dev 2022-11-01 14:39:59 -03:00
omap-iommu.h iommu/omap: add support for late attachment of iommu devices 2019-08-09 17:37:10 +02:00
omap-iopgtable.h iommu/omap: Fix -Woverflow warnings when compiling on 64-bit architectures 2020-03-04 16:24:46 +01:00
rockchip-iommu.c iommu: Clean up bus_set_iommu() 2022-09-07 14:26:17 +02:00
s390-iommu.c iommu: Clean up bus_set_iommu() 2022-09-07 14:26:17 +02:00
sprd-iommu.c iommu: Use EINVAL for incompatible device/domain in ->attach_dev 2022-11-01 14:39:59 -03:00
sun50i-iommu.c iommu: Clean up bus_set_iommu() 2022-09-07 14:26:17 +02:00
tegra-gart.c iommu: Use EINVAL for incompatible device/domain in ->attach_dev 2022-11-01 14:39:59 -03:00
tegra-smmu.c iommu/tegra-smmu: Clean up bus_set_iommu() 2022-09-07 14:26:16 +02:00
virtio-iommu.c iommu: Propagate return value in ->attach_dev callback functions 2022-11-01 14:39:59 -03:00