linux/drivers/vfio
Daniel Jordan 4d83de6da2 vfio/type1: Batch page pinning
Pinning one 4K page at a time is inefficient, so do it in batches of 512
instead.  This is just an optimization with no functional change
intended, and in particular the driver still calls iommu_map() with the
largest physically contiguous range possible.

Add two fields in vfio_batch to remember where to start between calls to
vfio_pin_pages_remote(), and use vfio_batch_unpin() to handle remaining
pages in the batch in case of error.

qemu pins pages for guests around 8% faster on my test system, a
two-node Broadwell server with 128G memory per node.  The qemu process
was bound to one node with its allocations constrained there as well.

                             base               test
          guest              ----------------   ----------------
       mem (GB)   speedup    avg sec    (std)   avg sec    (std)
              1      7.4%       0.61   (0.00)      0.56   (0.00)
              2      8.3%       0.93   (0.00)      0.85   (0.00)
              4      8.4%       1.46   (0.00)      1.34   (0.00)
              8      8.6%       2.54   (0.01)      2.32   (0.00)
             16      8.3%       4.66   (0.00)      4.27   (0.01)
             32      8.3%       8.94   (0.01)      8.20   (0.01)
             64      8.2%      17.47   (0.01)     16.04   (0.03)
            120      8.5%      32.45   (0.13)     29.69   (0.01)

perf diff confirms less time spent in pup.  Here are the top ten
functions:

             Baseline  Delta Abs  Symbol

               78.63%     +6.64%  clear_page_erms
                1.50%     -1.50%  __gup_longterm_locked
                1.27%     -0.78%  __get_user_pages
                          +0.76%  kvm_zap_rmapp.constprop.0
                0.54%     -0.53%  vmacache_find
                0.55%     -0.51%  get_pfnblock_flags_mask
                0.48%     -0.48%  __get_user_pages_remote
                          +0.39%  slot_rmap_walk_next
                          +0.32%  vfio_pin_map_dma
                          +0.26%  kvm_handle_hva_range
                ...

Suggested-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Signed-off-by: Daniel Jordan <daniel.m.jordan@oracle.com>
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
2021-02-22 16:30:47 -07:00
..
fsl-mc vfio/fsl-mc: Make vfio_fsl_mc_irqs_allocate static 2020-11-03 11:07:40 -07:00
mdev vfio-mdev: Wire in a request handler for mdev parent 2020-12-03 16:21:07 -07:00
pci vfio/pci: remove CONFIG_VFIO_PCI_ZDEV from Kconfig 2021-02-19 10:29:56 -07:00
platform vfio: platform: Switch to use platform_get_mem_or_io() 2020-12-10 16:31:46 +01:00
Kconfig vfio/fsl-mc: Add VFIO framework skeleton for fsl-mc devices 2020-10-07 14:17:33 -06:00
Makefile vfio/fsl-mc: Add VFIO framework skeleton for fsl-mc devices 2020-10-07 14:17:33 -06:00
vfio_iommu_spapr_tce.c vfio/spapr_tce: convert get_user_pages() --> pin_user_pages() 2020-07-27 13:43:38 -06:00
vfio_iommu_type1.c vfio/type1: Batch page pinning 2021-02-22 16:30:47 -07:00
vfio_spapr_eeh.c treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 500 2019-06-19 17:09:55 +02:00
vfio.c vfio: iommu driver notify callback 2021-02-01 13:20:07 -07:00
virqfd.c vfio/virqfd: Drain events from eventfd in virqfd_wakeup() 2020-11-15 09:49:10 -05:00