linux/drivers/pci
Logan Gunthorpe 9db8dc6d07 PCI: Don't disable bridge BARs when assigning bus resources
Some PCI bridges implement BARs in addition to bridge windows.  For
example, here's a PLX switch:

  04:00.0 PCI bridge: PLX Technology, Inc. PEX 8724 24-Lane, 6-Port PCI
            Express Gen 3 (8 GT/s) Switch, 19 x 19mm FCBGA (rev ca)
	    (prog-if 00 [Normal decode])
      Flags: bus master, fast devsel, latency 0, IRQ 30, NUMA node 0
      Memory at 90a00000 (32-bit, non-prefetchable) [size=256K]
      Bus: primary=04, secondary=05, subordinate=0a, sec-latency=0
      I/O behind bridge: 00002000-00003fff
      Memory behind bridge: 90000000-909fffff
      Prefetchable memory behind bridge: 0000380000800000-0000380000bfffff

Previously, when the kernel assigned resource addresses (with the
pci=realloc command line parameter, for example) it could clear the struct
resource corresponding to the BAR.  When this happened, lspci would report
this BAR as "ignored":

   Region 0: Memory at <ignored> (32-bit, non-prefetchable) [size=256K]

This is because the kernel reports a zero start address and zero flags
in the corresponding sysfs resource file and in /proc/bus/pci/devices.
Investigation with 'lspci -x', however, shows the BIOS-assigned address
will still be programmed in the device's BAR registers.

It's clearly a bug that the kernel lost track of the BAR value, but in most
cases, this still won't result in a visible issue because nothing uses the
memory, so nothing is affected.  However, when an IOMMU is in use, it will
not reserve this space in the IOVA because the kernel no longer thinks the
range is valid.  (See dmar_init_reserved_ranges() for the Intel
implementation of this.)

Without the proper reserved range, a DMA mapping may allocate an IOVA that
matches a bridge BAR, which results in DMA accesses going to the BAR
instead of the intended RAM.

The problem was in pci_assign_unassigned_root_bus_resources().  When any
resource from a bridge device fails to get assigned, the code set the
resource's flags to zero.  This makes sense for bridge windows, as they
will be re-enabled later, but for regular BARs, it makes the kernel
permanently lose track of the fact that they decode address space.

Change pci_assign_unassigned_root_bus_resources() and
pci_assign_unassigned_bridge_resources() so they only clear "res->flags"
for bridge *windows*, not bridge BARs.

Fixes: da7822e5ad ("PCI: update bridge resources to get more big ranges when allocating space (again)")
Link: https://lore.kernel.org/r/20200108213208.4612-1-logang@deltatee.com
[bhelgaas: commit log, check for pci_is_bridge()]
Reported-by: Kit Chow <kchow@gigaio.com>
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2020-01-13 13:23:02 -06:00
..
controller Merge branch 'pci/trivial' 2019-11-28 08:54:55 -06:00
endpoint Merge branch 'remotes/lorenzo/pci/endpoint' 2019-11-28 08:54:42 -06:00
hotplug pci-v5.5-changes 2019-12-03 13:58:22 -08:00
pcie Merge branch 'pci/misc' 2019-11-28 08:54:32 -06:00
switch pci-v5.5-changes 2019-12-03 13:58:22 -08:00
access.c PCI/AER: Save AER Capability for suspend/resume 2019-10-18 17:05:42 -05:00
ats.c PCI/ATS: Remove unnecessary EXPORT_SYMBOL_GPL() 2019-10-15 16:39:11 -05:00
bus.c PCI: Unexport pci_bus_get() and pci_bus_put() 2019-07-23 18:32:49 -05:00
ecam.c
host-bridge.c PCI: Tidy comments 2018-03-19 14:20:43 -05:00
iov.c Merge branch 'pci/trivial' 2019-11-28 08:54:55 -06:00
irq.c PCI: Use IRQF_ONESHOT if pci_request_irq() called with no handler 2018-07-31 10:43:43 -05:00
Kconfig Merge branch 'pci/trivial' 2019-11-28 08:54:55 -06:00
Makefile PCI: Allow building PCIe things without PCIEPORTBUS 2019-11-21 07:52:33 -06:00
mmap.c PCI: Fix typos and whitespace errors 2019-07-09 07:24:53 -05:00
msi.c PCI/MSI: Fix incorrect MSI-X masking on resume 2019-11-26 13:10:29 -06:00
of.c PCI: Make devm_of_pci_get_host_bridge_resources() static 2019-11-20 17:00:14 +00:00
p2pdma.c PCI/P2PDMA: Update pci_p2pdma_distance_many() documentation 2019-08-16 08:41:59 -05:00
pci-acpi.c Merge branch 'pci/enumeration' 2019-09-23 16:10:08 -05:00
pci-bridge-emul.c PCI: pci-bridge-emul: Fix big-endian support 2019-10-17 12:42:48 +01:00
pci-bridge-emul.h PCI: pci-bridge-emul: Fix big-endian support 2019-10-17 12:42:48 +01:00
pci-driver.c PCI/PM: Add missing link delays required by the PCIe spec 2019-11-20 17:37:24 -06:00
pci-label.c PCI: Tidy comments 2018-03-19 14:20:43 -05:00
pci-mid.c x86/cpu: Sanitize FAM6_ATOM naming 2018-10-02 10:14:32 +02:00
pci-pf-stub.c PCI: Fix typos and whitespace errors 2019-07-09 07:24:53 -05:00
pci-stub.c PCI: Replace printk(KERN_INFO) with pr_info(), etc 2019-05-09 07:49:54 -05:00
pci-sysfs.c Merge branch 'pci/resource' 2019-11-28 08:54:36 -06:00
pci.c pci-v5.5-changes 2019-12-03 13:58:22 -08:00
pci.h Merge branch 'remotes/lorenzo/pci/mmio-dma-ranges' 2019-11-28 08:54:53 -06:00
probe.c Merge branch 'pci/trivial' 2019-11-28 08:54:55 -06:00
proc.c PCI: Add PCI_STD_NUM_BARS for the number of standard BARs 2019-10-14 10:22:26 -05:00
quirks.c Merge branch 'pci/trivial' 2019-11-28 08:54:55 -06:00
remove.c PCI/ASPM: Fix link_state teardown on device removal 2018-09-17 16:32:23 -05:00
rom.c PCI: Make pci_get_rom_size() static 2018-06-29 21:17:26 -05:00
search.c PCI: Unexport pci_bus_sem 2019-07-23 18:32:50 -05:00
setup-bus.c PCI: Don't disable bridge BARs when assigning bus resources 2020-01-13 13:23:02 -06:00
setup-irq.c PCI: Tidy comments 2018-03-19 14:20:43 -05:00
setup-res.c PCI: Remove messages about reassigning resources 2018-04-11 08:46:50 -05:00
slot.c PCI: Decode PCIe 32 GT/s link speed 2019-06-13 16:49:45 -05:00
syscall.c PCI: Lock down BAR access when the kernel is locked down 2019-08-19 21:54:15 -07:00
vc.c Merge branch 'pci/trivial' 2019-09-23 16:10:31 -05:00
vpd.c PCI/VPD: Prevent VPD access for Amazon's Annapurna Labs Root Port 2019-09-16 14:10:09 +01:00
xen-pcifront.c Merge branch 'pci/printk' 2019-05-13 18:34:46 -05:00