Commit Graph

130 Commits

Author SHA1 Message Date
Xiongfeng Wang
6f7dc30767 PCI/IOV: Use pci_dev_id() to simplify the code
When we have a struct pci_dev *, use pci_dev_id() instead of manually
composing the ID with PCI_DEVID() from dev->bus->number and dev->devfn.

[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20230807134858.116051-4-wangxiongfeng2@huawei.com
Signed-off-by: Xiongfeng Wang <wangxiongfeng2@huawei.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2023-08-07 16:15:39 -05:00
Alexey V. Vissarionov
ea0b5aa5f1 PCI/IOV: Enlarge virtfn sysfs name buffer
The sysfs link name "virtfn%u" constructed by pci_iov_sysfs_link() requires
17 bytes to contain the longest possible string.  Increase VIRTFN_ID_LEN to
accommodate that.

Found by Linux Verification Center (linuxtesting.org) with SVACE.

[bhelgaas: commit log, comment at #define]
Fixes: dd7cc44d0b ("PCI: add SR-IOV API for Physical Function driver")
Link: https://lore.kernel.org/r/20221218033347.23743-1-gremlin@altlinux.org
Signed-off-by: Alexey V. Vissarionov <gremlin@altlinux.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2023-01-18 10:54:41 -06:00
Leon Romanovsky
8d26c4328b PCI/IOV: Fix wrong kernel-doc identifier
Replace "-" to be ":" in comment section to be aligned with
kernel-doc format.

drivers/pci/iov.c:67: warning: Function parameter or member 'dev' not described in 'pci_iov_get_pf_drvdata'
drivers/pci/iov.c:67: warning: Function parameter or member 'pf_driver' not described in 'pci_iov_get_pf_drvdata'

Fixes: a7e9f240c0 ("PCI/IOV: Add pci_iov_get_pf_drvdata() to allow VF reaching the drvdata of a PF")
Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
Acked-by: Randy Dunlap <rdunlap@infradead.org>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/8cecf7df45948a256dc56148cf9e87b2f2bb4198.1646652504.git.leonro@nvidia.com
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
2022-03-07 12:06:10 -07:00
Jason Gunthorpe
a7e9f240c0 PCI/IOV: Add pci_iov_get_pf_drvdata() to allow VF reaching the drvdata of a PF
There are some cases where a SR-IOV VF driver will need to reach into and
interact with the PF driver. This requires accessing the drvdata of the PF.

Provide a function pci_iov_get_pf_drvdata() to return this PF drvdata in a
safe way. Normally accessing a drvdata of a foreign struct device would be
done using the device_lock() to protect against device driver
probe()/remove() races.

However, due to the design of pci_enable_sriov() this will result in a
ABBA deadlock on the device_lock as the PF's device_lock is held during PF
sriov_configure() while calling pci_enable_sriov() which in turn holds the
VF's device_lock while calling VF probe(), and similarly for remove.

This means the VF driver can never obtain the PF's device_lock.

Instead use the implicit locking created by pci_enable/disable_sriov(). A
VF driver can access its PF drvdata only while its own driver is attached,
and the PF driver can control access to its own drvdata based on when it
calls pci_enable/disable_sriov().

To use this API the PF driver will setup the PF drvdata in the probe()
function. pci_enable_sriov() is only called from sriov_configure() which
cannot happen until probe() completes, ensuring no VF races with drvdata
setup.

For removal, the PF driver must call pci_disable_sriov() in its remove
function before destroying any of the drvdata. This ensures that all VF
drivers are unbound before returning, fencing concurrent access to the
drvdata.

The introduction of a new function to do this access makes clear the
special locking scheme and the documents the requirements on the PF/VF
drivers using this.

Link: https://lore.kernel.org/all/20220224142024.147653-5-yishaih@nvidia.com
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
2022-02-27 11:40:58 +02:00
Jason Gunthorpe
21ca9fb62d PCI/IOV: Add pci_iov_vf_id() to get VF index
The PCI core uses the VF index internally, often called the vf_id,
during the setup of the VF, eg pci_iov_add_virtfn().

This index is needed for device drivers that implement live migration
for their internal operations that configure/control their VFs.

Specifically, mlx5_vfio_pci driver that is introduced in coming patches
from this series needs it and not the bus/device/function which is
exposed today.

Add pci_iov_vf_id() which computes the vf_id by reversing the math that
was used to create the bus/device/function.

Link: https://lore.kernel.org/all/20220224142024.147653-2-yishaih@nvidia.com
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
2022-02-27 11:38:51 +02:00
Bjorn Helgaas
e0217c5ba1 Revert "PCI: Use to_pci_driver() instead of pci_dev->driver"
This reverts commit 2a4d9408c9.

Robert reported a NULL pointer dereference caused by the PCI core
(local_pci_probe()) calling the i2c_designware_pci driver's
.runtime_resume() method before the .probe() method.  i2c_dw_pci_resume()
depends on initialization done by i2c_dw_pci_probe().

Prior to 2a4d9408c9 ("PCI: Use to_pci_driver() instead of
pci_dev->driver"), pci_pm_runtime_resume() avoided calling the
.runtime_resume() method because pci_dev->driver had not been set yet.

2a4d9408c9 and b5f9c644eb ("PCI: Remove struct pci_dev->driver"),
removed pci_dev->driver, replacing it by device->driver, which *has* been
set by this time, so pci_pm_runtime_resume() called the .runtime_resume()
method when it previously had not.

Fixes: 2a4d9408c9 ("PCI: Use to_pci_driver() instead of pci_dev->driver")
Link: https://lore.kernel.org/linux-i2c/CAP145pgdrdiMAT7=-iB1DMgA7t_bMqTcJL4N0=6u8kNY3EU0dw@mail.gmail.com/
Reported-by: Robert Święcki <robert@swiecki.net>
Tested-by: Robert Święcki <robert@swiecki.net>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2021-11-11 13:36:22 -06:00
Bjorn Helgaas
ebf275b856 Merge branch 'pci/sysfs'
- Check for CAP_SYS_ADMIN before validating sysfs user input, not after
  (Krzysztof Wilczyński)

- Always return -EINVAL from sysfs "store" functions for invalid user input
  instead of -EINVAL sometimes and -ERANGE others (Krzysztof Wilczyński)

- Use kstrtobool() directly instead of the strtobool() wrapper (Krzysztof
  Wilczyński)

* pci/sysfs:
  PCI: Use kstrtobool() directly, sans strtobool() wrapper
  PCI/sysfs: Return -EINVAL consistently from "store" functions
  PCI/sysfs: Check CAP_SYS_ADMIN before parsing user input

# Conflicts:
#	drivers/pci/iov.c
2021-11-05 11:28:46 -05:00
Uwe Kleine-König
2a4d9408c9 PCI: Use to_pci_driver() instead of pci_dev->driver
Struct pci_driver contains a struct device_driver, so for PCI devices, it's
easy to convert a device_driver * to a pci_driver * with to_pci_driver().
The device_driver * is in struct device, so we don't need to also keep
track of the pci_driver * in struct pci_dev.

Replace pci_dev->driver with to_pci_driver().  This is a step toward
removing pci_dev->driver.

[bhelgaas: split to separate patch]
Link: https://lore.kernel.org/r/20211004125935.2300113-11-u.kleine-koenig@pengutronix.de
Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2021-10-18 09:20:15 -05:00
Krzysztof Wilczyński
36f354ec7b PCI/sysfs: Return -EINVAL consistently from "store" functions
Most of the "store" functions that handle userspace input via sysfs return
-EINVAL should the value fail validation and/or type conversion.  This
error code is a clear message to userspace that the value is not a valid
input.

However, some of the "show" functions return input parsing error codes
as-is, which may be either -EINVAL or -ERANGE.  The former would often be
from kstrtobool(), and the latter typically from other kstr*() functions
such as kstrtou8(), kstrtou32(), kstrtoint(), etc.

-EINVAL is commonly returned as the error code to indicate that the value
provided is invalid, but -ERANGE is not very useful in userspace.

Therefore, normalize the return error code to be -EINVAL for when the
validation and/or type conversion fails.

Link: https://lore.kernel.org/r/20210915230127.2495723-2-kw@linux.com
Signed-off-by: Krzysztof Wilczyński <kw@linux.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2021-09-28 17:47:04 -05:00
Bjorn Helgaas
524e6b0395 Merge branch 'pci/virtualization'
- Clarify error message for unbound devices (Moritz Fischer)

* pci/virtualization:
  PCI/IOV: Clarify error message for unbound devices
2021-07-06 10:56:26 -05:00
Moritz Fischer
e9c3bbd68e PCI/IOV: Clarify error message for unbound devices
Be more verbose to disambiguate the error case when trying to configure
SR-IOV with no driver bound vs. a driver that does not implement the
.sriov_configure() callback.

Link: https://lore.kernel.org/r/20210327175140.682708-1-mdf@kernel.org
Reported-by: Brian Foley <bpfoley@google.com>
Signed-off-by: Moritz Fischer <mdf@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Krzysztof Wilczyński <kw@linux.com>
2021-06-10 15:25:38 -05:00
Krzysztof Wilczyński
f8cf6e513e PCI/sysfs: Use sysfs_emit() and sysfs_emit_at() in "show" functions
The sysfs_emit() and sysfs_emit_at() functions were introduced to make
it less ambiguous which function is preferred when writing to the output
buffer in a device attribute's "show" callback [1].

Convert the PCI sysfs object "show" functions from sprintf(), snprintf()
and scnprintf() to sysfs_emit() and sysfs_emit_at() accordingly, as the
latter is aware of the PAGE_SIZE buffer and correctly returns the number
of bytes written into the buffer.

No functional change intended.

[1] Documentation/filesystems/sysfs.rst

Related commit: ad025f8e46 ("PCI/sysfs: Use sysfs_emit() and
sysfs_emit_at() in "show" functions").

Link: https://lore.kernel.org/r/20210603000112.703037-2-kw@linux.com
Signed-off-by: Krzysztof Wilczyński <kw@linux.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Logan Gunthorpe <logang@deltatee.com>
2021-06-03 22:14:47 -05:00
Leon Romanovsky
c3d5c2d96d PCI/IOV: Add sysfs MSI-X vector assignment interface
A typical cloud provider SR-IOV use case is to create many VFs for use by
guest VMs. The VFs may not be assigned to a VM until a customer requests a
VM of a certain size, e.g., number of CPUs. A VF may need MSI-X vectors
proportional to the number of CPUs in the VM, but there is no standard way
to change the number of MSI-X vectors supported by a VF.

Some Mellanox ConnectX devices support dynamic assignment of MSI-X vectors
to SR-IOV VFs. This can be done by the PF driver after VFs are enabled,
and it can be done without affecting VFs that are already in use. The
hardware supports a limited pool of MSI-X vectors that can be assigned to
the PF or to individual VFs.  This is device-specific behavior that
requires support in the PF driver.

Add a read-only "sriov_vf_total_msix" sysfs file for the PF and a writable
"sriov_vf_msix_count" file for each VF. Management software may use these
to learn how many MSI-X vectors are available and to dynamically assign
them to VFs before the VFs are passed through to a VM.

If the PF driver implements the ->sriov_get_vf_total_msix() callback,
"sriov_vf_total_msix" contains the total number of MSI-X vectors available
for distribution among VFs.

If no driver is bound to the VF, writing "N" to "sriov_vf_msix_count" uses
the PF driver ->sriov_set_msix_vec_count() callback to assign "N" MSI-X
vectors to the VF.  When a VF driver subsequently reads the MSI-X Message
Control register, it will see the new Table Size "N".

Link: https://lore.kernel.org/linux-pci/20210314124256.70253-2-leon@kernel.org
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
2021-04-04 10:26:30 +03:00
Matthew Rosato
12856e7acd PCI/IOV: Mark VFs as not implementing PCI_COMMAND_MEMORY
For VFs, the Memory Space Enable bit in the Command Register is
hard-wired to 0.

Add a new bit to signify devices where the Command Register Memory
Space Enable bit does not control the device's response to MMIO
accesses.

Fixes: abafbc551f ("vfio-pci: Invalidate mmaps and block MMIO access on disabled memory")
Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
2020-09-21 14:42:11 -06:00
Niklas Schnelle
e5794cf1a2 s390/pci: create links between PFs and VFs
On s390 PCI Virtual Functions (VFs) are scanned by firmware and are made
available to Linux via the hot-plug interface. As such the common code
path of doing the scan directly using the parent Physical Function (PF)
is not used and fenced off with the no_vf_scan attribute.

Even if the partition created the VFs itself e.g. using the sriov_numvfs
attribute of a PF, the PF/VF links thus need to be established after the
fact. To do this when a VF is plugged we scan through all functions on
the same zbus and test whether they are the parent PF in which case we
establish the necessary links.

With these links established there is now no more need to fence off
pci_iov_remove_virtfn() for pdev->no_vf_scan as the common code now
works fine.

Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Pierre Morel <pmorel@linux.ibm.com>
Link: https://lore.kernel.org/r/20200506154139.90609-3-schnelle@linux.ibm.com
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-05-20 10:22:51 +02:00
Niklas Schnelle
a1ceea67f2 PCI/IOV: Introduce pci_iov_sysfs_link() function
Currently pci_iov_add_virtfn() scans the SR-IOV BARs, adds the VF to the
bus and also creates the sysfs links between the newly added VF and its
parent PF.

With pdev->no_vf_scan fencing off the entire pci_iov_add_virtfn() call
s390 as the sole pdev->no_vf_scan user thus ends up missing these sysfs
links which are required for example by QEMU/libvirt.

Instead of duplicating the code refactor pci_iov_add_virtfn() to make
sysfs link creation callable separately.

Signed-off-by: Niklas Schnelle <schnelle@linux.ibm.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Pierre Morel <pmorel@linux.ibm.com>
Link: https://lore.kernel.org/r/20200506154139.90609-1-schnelle@linux.ibm.com
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
2020-05-20 10:22:50 +02:00
Navid Emamdoost
8c386cc817 PCI/IOV: Fix memory leak in pci_iov_add_virtfn()
In the implementation of pci_iov_add_virtfn() the allocated virtfn is
leaked if pci_setup_device() fails. The error handling is not calling
pci_stop_and_remove_bus_device(). Change the goto label to failed2.

Fixes: 156c55325d ("PCI: Check for pci_setup_device() failure in pci_iov_add_virtfn()")
Link: https://lore.kernel.org/r/20191125195255.23740-1-navid.emamdoost@gmail.com
Signed-off-by: Navid Emamdoost <navid.emamdoost@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-12-09 15:51:32 -06:00
Bjorn Helgaas
7e124c4051 Merge branch 'pci/trivial'
- Fix typos and comments (Bjorn Helgaas)

  - Fix Kconfig whitespace errors (Krzysztof Kozlowski)

* pci/trivial:
  PCI: Fix indentation
  PCI: Fix typos
  PCI: Remove useless comments and tidy others
  PCI: Remove unnecessary includes

# Conflicts:
#	drivers/pci/probe.c
2019-11-28 08:54:55 -06:00
Bjorn Helgaas
eceb86028d PCI: Remove unnecessary includes
Remove unnecessary includes.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-10-17 15:18:44 -05:00
Pierre Crégut
35ff867b76 PCI/IOV: Serialize sysfs sriov_numvfs reads vs writes
When sriov_numvfs is being updated, we call the driver->sriov_configure()
function, which may enable VFs and call probe functions, which may make new
devices visible.  This all happens before before sriov_numvfs_store()
updates sriov->num_VFs, so previously, concurrent sysfs reads of
sriov_numvfs returned stale values.

Serialize the sysfs read vs the write so the read returns the correct
num_VFs value.

[bhelgaas: hold device_lock instead of checking mutex_is_locked()]
Link: https://bugzilla.kernel.org/show_bug.cgi?id=202991
Link: https://lore.kernel.org/r/20190911072736.32091-1-pierre.cregut@orange.com
Signed-off-by: Pierre Crégut <pierre.cregut@orange.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2019-10-15 16:39:12 -05:00
Bjorn Helgaas
0ca0ef1042 Merge branch 'pci/resource'
- Convert pci_resource_to_user() to a weak function to remove
    HAVE_ARCH_PCI_RESOURCE_TO_USER #defines (Denis Efremov)

  - Use PCI_SRIOV_NUM_BARS for idiomatic loop structure (Denis Efremov)

  - Fix Resizable BAR size suspend/restore for 1MB BARs (Sumit Saxena)

  - Correct "pci=resource_alignment" example in documentation (Alexey
    Kardashevskiy)

* pci/resource:
  PCI: Correct pci=resource_alignment parameter example
  PCI: Restore Resizable BAR size bits correctly for 1MB BARs
  PCI: Use PCI_SRIOV_NUM_BARS in loops instead of PCI_IOV_RESOURCE_END
  PCI: Convert pci_resource_to_user() to a weak function

# Conflicts:
#	drivers/pci/pci.c
2019-09-23 16:10:15 -05:00
Kelsey Skunberg
244c06c3b6 PCI/IOV: Remove group write permission from sriov_numvfs, sriov_drivers_autoprobe
Previously the sriov_numvfs and sriov_drivers_autoprobe sysfs files had
0664 permissions, which allowed group write.  libvirt runs as root when
dealing with PCI, and it chowns files needed by qemu, so group write
permission should not be needed.

Change these permissions from 0664 to 0644, which is what DEVICE_ATTR_RW()
does by default.

[bhelgaas: commit log]
Link: https://lore.kernel.org/r/20190905063226.43269-1-skunberg.kelsey@gmail.com
Link: https://lore.kernel.org/r/850cf536-0b72-d78c-efaf-855dcb391087@redhat.com
Signed-off-by: Kelsey Skunberg <skunberg.kelsey@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Acked-by: Donald Dutile <ddutile@redhat.com>
2019-09-05 13:26:23 -05:00
Kelsey Skunberg
aaee0c1ffd PCI/IOV: Move sysfs SR-IOV functions to iov.c
The sysfs SR-IOV functions are only needed when the kernel is built with
SR-IOV support.  Rather than put them in pci-sysfs.c under #ifdef
CONFIG_PCI_IOV, move them to iov.c, which is only compiled when
CONFIG_PCI_IOV=y.

Link: https://lore.kernel.org/r/20190813204513.4790-4-skunberg.kelsey@gmail.com
Signed-off-by: Kelsey Skunberg <skunberg.kelsey@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Reviewed-by: Donald Dutile <ddutile@redhat.com>
Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
2019-08-20 14:05:05 -05:00
Denis Efremov
39098edbd7 PCI: Use PCI_SRIOV_NUM_BARS in loops instead of PCI_IOV_RESOURCE_END
Writing loop conditions as "i < NUM" is a common C idiom; using "i <= END"
is unusual and thus prone to errors.  Change loops to use the former.

Link: https://lore.kernel.org/r/20190806140715.19847-1-efremov@linux.com
Signed-off-by: Denis Efremov <efremov@linux.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
2019-08-08 15:12:12 -05:00
Alex Williamson
76bf6a8634 Revert "PCI/IOV: Use VF0 cached config space size for other VFs"
Revert 975bb8b4dc ("PCI/IOV: Use VF0 cached config space size for other
VFs"), which attempted to cache the config space size from the first VF to
re-use for subsequent VFs.

The cached value was determined prior to discovering the PCIe capability on
the VF, which resulted in the first VF reporting the correct config space
size (4K), as it has a special case through pci_cfg_space_size(), while all
the other VFs only reported 256 bytes.  As this was only a performance
optimization, we're better off without it.

Fixes: 975bb8b4dc ("PCI/IOV: Use VF0 cached config space size for other VFs")
Link: https://lore.kernel.org/r/156046663197.29869.3633634445109057665.stgit@gimli.home
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: KarimAllah Ahmed <karahmed@amazon.de>
Cc: Kuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
Cc: Hao Zheng <yinhe@linux.alibaba.com>
2019-07-03 08:58:19 -05:00
Sebastian Ott
aff68a5a62 PCI/IOV: Add flag so platforms can skip VF scanning
Provide a flag to skip scanning for new VFs after SR-IOV enablement.  This
can be set by implementations for which the VFs are already reported by
other means.

Signed-off-by: Sebastian Ott <sebott@linux.ibm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2019-01-01 19:04:37 -06:00
Sebastian Ott
18f9e9d150 PCI/IOV: Factor out sriov_add_vfs()
Provide sriov_add_vfs() as a wrapper to scan for VFs that cleans up after
itself.  This is just a code simplification.  No functional change.

Signed-off-by: Sebastian Ott <sebott@linux.ibm.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2019-01-01 19:03:51 -06:00
Bjorn Helgaas
4c243716e7 PCI/IOV: Remove unnecessary include of <linux/pci-ats.h>
iov.c uses nothing declared in <linux/pci-ats.h>, so remove the include of
it.  No functional change intended.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2018-10-11 14:35:12 -05:00
KarimAllah Ahmed
975bb8b4dc PCI/IOV: Use VF0 cached config space size for other VFs
Cache the config space size from VF0 and use it for all other VFs instead
of reading it from the config space of each VF.  We assume that it will be
the same across all associated VFs.

This is an optimization when enabling SR-IOV on a device with many VFs.

Signed-off-by: KarimAllah Ahmed <karahmed@amazon.de>
[bhelgaas: use CONFIG_PCI_IOV (not CONFIG_PCI_ATS)]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2018-10-11 11:49:58 -05:00
Bjorn Helgaas
3a48dc6fc2 Merge branch 'pci/virtualization'
- To avoid bus errors, enable PASID only if entire path supports End-End
    TLP prefixes (Sinan Kaya)

  - Unify slot and bus reset functions and remove hotplug knowledge from
    callers (Sinan Kaya)

  - Add Function-Level Reset quirks for Intel and Samsung NVMe devices to
    fix guest reboot issues (Alex Williamson)

  - Add function 1 DMA alias quirk for Marvell 88SS9183 PCIe SSD Controller
    (Bjorn Helgaas)

* pci/virtualization:
  PCI: Add function 1 DMA alias quirk for Marvell 88SS9183
  PCI: Delay after FLR of Intel DC P3700 NVMe
  PCI: Disable Samsung SM961/PM961 NVMe before FLR
  PCI: Export pcie_has_flr()
  PCI: Rename pci_try_reset_bus() to pci_reset_bus()
  PCI: Deprecate pci_reset_bus() and pci_reset_slot() functions
  PCI: Unify try slot and bus reset API
  PCI: Hide pci_reset_bridge_secondary_bus() from drivers
  IB/hfi1: Use pci_try_reset_bus() for initiating PCI Secondary Bus Reset
  PCI: Handle error return from pci_reset_bridge_secondary_bus()
  PCI/IOV: Tidy pci_sriov_set_totalvfs()
  PCI: Enable PASID only if entire path supports End-End TLP prefixes

# Conflicts:
#	drivers/pci/hotplug/pciehp_hpc.c
2018-08-15 14:59:06 -05:00
Bjorn Helgaas
51259d0022 PCI/IOV: Tidy pci_sriov_set_totalvfs()
Fix minor style issues in pci_sriov_set_totalvfs().  No functional change
intended.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2018-07-19 16:42:21 -05:00
Jakub Kicinski
38972375ef PCI/IOV: Reset total_VFs limit after detaching PF driver
The TotalVFs register in the SR-IOV capability is the hardware limit on the
number of VFs.  A PF driver can limit the number of VFs further with
pci_sriov_set_totalvfs().  When the PF driver is removed, reset any VF
limit that was imposed by the driver because that limit may not apply to
other drivers.

Before 8d85a7a4f2 ("PCI/IOV: Allow PF drivers to limit total_VFs to 0"),
pci_sriov_set_totalvfs(pdev, 0) meant "we can enable TotalVFs virtual
functions", and the nfp driver used that to remove the VF limit when the
driver unloads.

8d85a7a4f2 broke that because instead of removing the VF limit,
pci_sriov_set_totalvfs(pdev, 0) actually sets the limit to zero, and that
limit persists even if another driver is loaded.

We could fix that by making the nfp driver reset the limit when it unloads,
but it seems more robust to do it in the PCI core instead of relying on the
driver.

The regression scenario is:

  nfp_pci_probe (driver 1)
  ...
  nfp_pci_remove
    pci_sriov_set_totalvfs(pf->pdev, 0)   # limits VFs to 0

  ...
  nfp_pci_probe (driver 2)
    nfp_rtsym_read_le("nfd_vf_cfg_max_vfs")
    # no VF limit from firmware

Now driver 2 is broken because the VF limit is still 0 from driver 1.

Fixes: 8d85a7a4f2 ("PCI/IOV: Allow PF drivers to limit total_VFs to 0")
Signed-off-by: Jakub Kicinski <jakub.kicinski@netronome.com>
[bhelgaas: changelog, rename functions]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2018-06-29 15:08:52 -05:00
Jakub Kicinski
8d85a7a4f2 PCI/IOV: Allow PF drivers to limit total_VFs to 0
Some SR-IOV PF drivers implement .sriov_configure(), which allows
user-space to enable VFs by writing the desired number of VFs to the sysfs
"sriov_numvfs" file (see sriov_numvfs_store()).

The PCI core limits the number of VFs to the TotalVFs advertised by the
device in its SR-IOV capability.  The PF driver can limit the number of VFs
to even fewer (it may have pre-allocated data structures or knowledge of
device limitations) by calling pci_sriov_set_totalvfs(), but previously it
could not limit the VFs to 0.

Change pci_sriov_get_totalvfs() so it always respects the VF limit imposed
by the PF driver, even if the limit is 0.

This sequence:

  pci_sriov_set_totalvfs(dev, 0);
  x = pci_sriov_get_totalvfs(dev);

previously set "x" to TotalVFs from the SR-IOV capability.  Now it will set
"x" to 0.

Signed-off-by: Jakub Kicinski <jakub.kicinski@netronome.com>
[bhelgaas: split to separate patch]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2018-05-25 08:18:34 -05:00
Alexander Duyck
8effc395c2 PCI/IOV: Add pci_sriov_configure_simple()
SR-IOV (Single Root I/O Virtualization) is an optional PCIe capability (see
PCIe r4.0, sec 9).  A PCIe Function with the SR-IOV capability is referred
to as a PF (Physical Function).  If SR-IOV is enabled on the PF, several
VFs (Virtual Functions) may be created.  The VFs can be individually
assigned to virtual machines, which allows them to share a single hardware
device while being isolated from each other.

Some SR-IOV devices have resources such as queues and interrupts that must
be set up in the PF before enabling the VFs, so they require a PF driver to
do that.

Other SR-IOV devices don't require any PF setup before enabling VFs.  Add a
pci_sriov_configure_simple() interface so PF drivers for such devices can
use it without repeating the VF-enabling code.

Tested-by: Mark Rustad <mark.d.rustad@intel.com>
Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
[bhelgaas: changelog, comment]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Greg Rose <gvrose8192@gmail.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>:wq
2018-04-24 16:46:56 -05:00
Bjorn Helgaas
a4b88505ac Merge branch 'pci/virtualization'
- probe for device reset support during enumeration instead of runtime
    (Bjorn Helgaas)

  - add ACS quirk for Ampere (née APM) root ports (Feng Kan)

  - add function 1 DMA alias quirk for Marvell 88SE9220 (Thomas
    Vincent-Cross)

  - protect device restore with device lock (Sinan Kaya)

  - handle failure of FLR gracefully (Sinan Kaya)

  - handle CRS (config retry status) after device resets (Sinan Kaya)

  - skip various config reads for SR-IOV VFs as an optimization (KarimAllah
    Ahmed)

* pci/virtualization:
  PCI/IOV: Add missing prototypes for powerpc pcibios interfaces
  PCI/IOV: Use VF0 cached config registers for other VFs
  PCI/IOV: Skip BAR sizing for VFs
  PCI/IOV: Skip INTx config reads for VFs
  PCI: Wait for device to become ready after secondary bus reset
  PCI: Add a return type for pci_reset_bridge_secondary_bus()
  PCI: Wait for device to become ready after a power management reset
  PCI: Rename pci_flr_wait() to pci_dev_wait() and make it generic
  PCI: Handle FLR failure and allow other reset types
  PCI: Protect restore with device lock to be consistent
  PCI: Add function 1 DMA alias quirk for Marvell 88SE9220
  PCI: Add ACS quirk for Ampere root ports
  PCI: Remove redundant probes for device reset support
  PCI: Probe for device reset support during enumeration

Conflicts:
	include/linux/pci.h
2018-04-04 13:28:26 -05:00
KarimAllah Ahmed
cf0921bea6 PCI/IOV: Use VF0 cached config registers for other VFs
Cache some config data from VF0 and use it for all other VFs instead of
reading it from the config space of each VF.  We assume these items are the
same across all associated VFs:

   Revision ID
   Class Code
   Subsystem Vendor ID
   Subsystem ID

This is an optimization when enabling SR-IOV on a device with many VFs.

Signed-off-by: KarimAllah Ahmed <karahmed@amazon.de>
[bhelgaas: changelog, simplify comments, remove unused "device", test
CONFIG_PCI_IOV instead of CONFIG_PCI_ATS, rename functions]
Signed-off-by: Bjorn Helgaas <helgaas@kernel.org>
2018-03-31 15:32:43 -05:00
Bjorn Helgaas
df62ab5e0f PCI: Tidy comments
Remove pointless comments that tell us the file name, remove blank line
comments, follow multi-line comment conventions.  No functional change
intended.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2018-03-19 14:20:43 -05:00
Linus Torvalds
105cf3c8c6 pci-v4.16-changes
-----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQIcBAABAgAGBQJad5lgAAoJEFmIoMA60/r8s2kQAI3PztawDpaCP9Z12pkbBHSt
 Ho0xTyk9rCZi9kQJbNjc+a+QrlA3QmTHXIXerB3LSWoh7M+XhsECjem92eHpgLNS
 JvYPhTfOrCr0vdiAmOz6hD0AqN/psrbfzgiJhSwomsGEFS77k7kERSJckRv81sxb
 Aj5F/WjucAgLorwm4auveAJEQ7atE7/6pkXzoqYm4G6NLOb46jUcRGndrnvXZBlz
 fws8fBM4BHyi7i25CYQl24tFq1CGax1rIPgLg+4KnH76bQk/N6Ju0sGVSzfh+hG8
 SIerK9bJbzGRAuNKoxB3aO1dyzsK3x9WztE2mG98w5trOISPIR1FqnvC/225FWAU
 d6eIXiC7wKnEx+DElNTzCjzfHc7SAJoupO32H7CoiTe5zPUlWlxJ1zLYkK1gt50q
 m8PRBiYTglxyznzrO0drtcdjEzvbdZNRrsYnul4wi1vSHzjk6F6XLtzT10XWM1M1
 1pXLB8384FTj0Hu4bq6Y3Aivkmz0Sf+eQM2NaOwe+Zj7/1VV0d3lvi4LUXkqzLCA
 FoXPJSMxG2Qu+iflCeYRQBJjExaZH3eNLZ3dT6QpcJrjaFVedd9u5DeeFqNL27zV
 bhr8TdqrR4p4rc8EBAGoCapw96IxLZROKB3gxbrZVOpfIZpzthwHbElHX6aqUgF4
 w/EV1JWs36WXWaxFk8wd
 =ttq9
 -----END PGP SIGNATURE-----

Merge tag 'pci-v4.16-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci

Pull PCI updates from Bjorn Helgaas:

 - skip AER driver error recovery callbacks for correctable errors
   reported via ACPI APEI, as we already do for errors reported via the
   native path (Tyler Baicar)

 - fix DPC shared interrupt handling (Alex Williamson)

 - print full DPC interrupt number (Keith Busch)

 - enable DPC only if AER is available (Keith Busch)

 - simplify DPC code (Bjorn Helgaas)

 - calculate ASPM L1 substate parameter instead of hardcoding it (Bjorn
   Helgaas)

 - enable Latency Tolerance Reporting for ASPM L1 substates (Bjorn
   Helgaas)

 - move ASPM internal interfaces out of public header (Bjorn Helgaas)

 - allow hot-removal of VGA devices (Mika Westerberg)

 - speed up unplug and shutdown by assuming Thunderbolt controllers
   don't support Command Completed events (Lukas Wunner)

 - add AtomicOps support for GPU and Infiniband drivers (Felix Kuehling,
   Jay Cornwall)

 - expose "ari_enabled" in sysfs to help NIC naming (Stuart Hayes)

 - clean up PCI DMA interface usage (Christoph Hellwig)

 - remove PCI pool API (replaced with DMA pool) (Romain Perier)

 - deprecate pci_get_bus_and_slot(), which assumed PCI domain 0 (Sinan
   Kaya)

 - move DT PCI code from drivers/of/ to drivers/pci/ (Rob Herring)

 - add PCI-specific wrappers for dev_info(), etc (Frederick Lawler)

 - remove warnings on sysfs mmap failure (Bjorn Helgaas)

 - quiet ROM validation messages (Alex Deucher)

 - remove redundant memory alloc failure messages (Markus Elfring)

 - fill in types for compile-time VGA and other I/O port resources
   (Bjorn Helgaas)

 - make "pci=pcie_scan_all" work for Root Ports as well as Downstream
   Ports to help AmigaOne X1000 (Bjorn Helgaas)

 - add SPDX tags to all PCI files (Bjorn Helgaas)

 - quirk Marvell 9128 DMA aliases (Alex Williamson)

 - quirk broken INTx disable on Ceton InfiniTV4 (Bjorn Helgaas)

 - fix CONFIG_PCI=n build by adding dummy pci_irqd_intx_xlate() (Niklas
   Cassel)

 - use DMA API to get MSI address for DesignWare IP (Niklas Cassel)

 - fix endpoint-mode DMA mask configuration (Kishon Vijay Abraham I)

 - fix ARTPEC-6 incorrect IS_ERR() usage (Wei Yongjun)

 - add support for ARTPEC-7 SoC (Niklas Cassel)

 - add endpoint-mode support for ARTPEC (Niklas Cassel)

 - add Cadence PCIe host and endpoint controller driver (Cyrille
   Pitchen)

 - handle multiple INTx status bits being set in dra7xx (Vignesh R)

 - translate dra7xx hwirq range to fix INTD handling (Vignesh R)

 - remove deprecated Exynos PHY initialization code (Jaehoon Chung)

 - fix MSI erratum workaround for HiSilicon Hip06/Hip07 (Dongdong Liu)

 - fix NULL pointer dereference in iProc BCMA driver (Ray Jui)

 - fix Keystone interrupt-controller-node lookup (Johan Hovold)

 - constify qcom driver structures (Julia Lawall)

 - rework Tegra config space mapping to increase space available for
   endpoints (Vidya Sagar)

 - simplify Tegra driver by using bus->sysdata (Manikanta Maddireddy)

 - remove PCI_REASSIGN_ALL_BUS usage on Tegra (Manikanta Maddireddy)

 - add support for Global Fabric Manager Server (GFMS) event to
   Microsemi Switchtec switch driver (Logan Gunthorpe)

 - add IDs for Switchtec PSX 24xG3 and PSX 48xG3 (Kelvin Cao)

* tag 'pci-v4.16-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci: (140 commits)
  PCI: cadence: Add EndPoint Controller driver for Cadence PCIe controller
  dt-bindings: PCI: cadence: Add DT bindings for Cadence PCIe endpoint controller
  PCI: endpoint: Fix EPF device name to support multi-function devices
  PCI: endpoint: Add the function number as argument to EPC ops
  PCI: cadence: Add host driver for Cadence PCIe controller
  dt-bindings: PCI: cadence: Add DT bindings for Cadence PCIe host controller
  PCI: Add vendor ID for Cadence
  PCI: Add generic function to probe PCI host controllers
  PCI: generic: fix missing call of pci_free_resource_list()
  PCI: OF: Add generic function to parse and allocate PCI resources
  PCI: Regroup all PCI related entries into drivers/pci/Makefile
  PCI/DPC: Reformat DPC register definitions
  PCI/DPC: Add and use DPC Status register field definitions
  PCI/DPC: Squash dpc_rp_pio_get_info() into dpc_process_rp_pio_error()
  PCI/DPC: Remove unnecessary RP PIO register structs
  PCI/DPC: Push dpc->rp_pio_status assignment into dpc_rp_pio_get_info()
  PCI/DPC: Squash dpc_rp_pio_print_error() into dpc_rp_pio_get_info()
  PCI/DPC: Make RP PIO log size check more generic
  PCI/DPC: Rename local "status" to "dpc_status"
  PCI/DPC: Squash dpc_rp_pio_print_tlp_header() into dpc_rp_pio_print_error()
  ...
2018-02-06 09:59:40 -08:00
Bjorn Helgaas
ab8c609356 Merge branch 'pci/spdx' into next
* pci/spdx:
  PCI: Add SPDX GPL-2.0+ to replace implicit GPL v2 or later statement
  PCI: Add SPDX GPL-2.0+ to replace GPL v2 or later boilerplate
  PCI: Add SPDX GPL-2.0 to replace COPYING boilerplate
  PCI: Add SPDX GPL-2.0 to replace GPL v2 boilerplate
  PCI: Add SPDX GPL-2.0 when no license was specified
2018-02-01 11:40:07 -06:00
Bjorn Helgaas
7328c8f48d PCI: Add SPDX GPL-2.0 when no license was specified
b24413180f ("License cleanup: add SPDX GPL-2.0 license identifier to
files with no license") added SPDX GPL-2.0 to several PCI files that
previously contained no license information.

Add SPDX GPL-2.0 to all other PCI files that did not contain any license
information and hence were under the default GPL version 2 license of the
kernel.

Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2018-01-26 11:45:16 -06:00
Frederick Lawler
7506dc7989 PCI: Add wrappers for dev_printk()
Add PCI-specific dev_printk() wrappers and use them to simplify the code
slightly.  No functional change intended.

Signed-off-by: Frederick Lawler <fred@fredlawl.com>
[bhelgaas: squash into one patch]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2018-01-18 12:55:24 -06:00
Bryant G. Ly
608c0d8804 PCI/IOV: Add pci_vf_drivers_autoprobe() interface
Add a pci_vf_drivers_autoprobe() interface. Setting autoprobe to false
on the PF prevents drivers from binding to VFs when they are enabled.

Signed-off-by: Bryant G. Ly <bryantly@linux.vnet.ibm.com>
Signed-off-by: Juan J. Alvarez <jjalvare@linux.vnet.ibm.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Russell Currey <ruscur@russell.cc>
Reviewed-by: Alexey Kardashevskiy <aik@ozlabs.ru>
Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2017-12-11 13:03:36 +11:00
Tony Nguyen
ff26449e41 PCI: Restore ARI Capable Hierarchy before setting numVFs
In the restore path, we previously read PCI_SRIOV_VF_OFFSET and
PCI_SRIOV_VF_STRIDE before restoring PCI_SRIOV_CTRL_ARI:

  pci_restore_state
    pci_restore_iov_state
      sriov_restore_state
        pci_iov_set_numvfs
          pci_read_config_word(... PCI_SRIOV_VF_OFFSET, &iov->offset)
          pci_read_config_word(... PCI_SRIOV_VF_STRIDE, &iov->stride)
        pci_write_config_word(... PCI_SRIOV_CTRL, iov->ctrl)

But per SR-IOV r1.1, sec 3.3.3.5, the device can use PCI_SRIOV_CTRL_ARI to
determine PCI_SRIOV_VF_OFFSET and PCI_SRIOV_VF_STRIDE.  Therefore, this
path, which is used for suspend/resume and AER recovery, can corrupt
iov->offset and iov->stride.

Since the iov state is associated with the device, not the driver, if we
reload the driver, it will use the the corrupted data, which may cause
crashes like this:

  kernel BUG at drivers/pci/iov.c:157!
  RIP: 0010:pci_iov_add_virtfn+0x2eb/0x350
  Call Trace:
   pci_enable_sriov+0x353/0x440
   ixgbe_pci_sriov_configure+0xd5/0x1f0 [ixgbe]
   sriov_numvfs_store+0xf7/0x170
   dev_attr_store+0x18/0x30
   sysfs_kf_write+0x37/0x40
   kernfs_fop_write+0x120/0x1b0
   vfs_write+0xb5/0x1a0
   SyS_write+0x55/0xc0

Restore PCI_SRIOV_CTRL_ARI before calling pci_iov_set_numvfs(), then
restore the rest of PCI_SRIOV_CTRL (which may set PCI_SRIOV_CTRL_VFE)
afterwards.

Signed-off-by: Tony Nguyen <anthony.l.nguyen@intel.com>
[bhelgaas: changelog, add comment, also clear ARI if necessary]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Alexander Duyck <alexander.h.duyck@intel.com>
CC: Emil Tantilov <emil.s.tantilov@intel.com>
2017-10-10 19:15:29 -05:00
Stuart Hayes
27d6162944 PCI: Create SR-IOV virtfn/physfn links before attaching driver
When creating virtual functions, create the "virtfn%u" and "physfn" links
in sysfs *before* attaching the driver instead of after.  When we attach
the driver to the new virtual network interface first, there is a race when
the driver attaches to the new sends out an "add" udev event, and the
network interface naming software (biosdevname or systemd, for example)
tries to look at these links.

Signed-off-by: Stuart Hayes <stuart.w.hayes@gmail.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2017-10-10 19:13:38 -05:00
Filippo Sironi
3142d832af PCI: Cache the VF device ID in the SR-IOV structure
Cache the VF device ID in the SR-IOV structure and use it instead of
reading it over and over from the PF config space capability.

Signed-off-by: Filippo Sironi <sironi@amazon.de>
[bhelgaas: rename to "vf_device" to match pci_dev->device]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2017-10-05 15:54:58 -05:00
Jan H. Schönherr
753f612471 PCI: Remove reset argument from pci_iov_{add,remove}_virtfn()
The "reset" argument passed to pci_iov_add_virtfn() and
pci_iov_remove_virtfn() is always zero since 46cb7b1bd8 ("PCI: Remove
unused SR-IOV VF Migration support")

Remove the argument together with the associated code.

Signed-off-by: Jan H. Schönherr <jschoenh@amazon.de>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Russell Currey <ruscur@russell.cc>
2017-10-05 15:54:56 -05:00
Gavin Shan
0fc690a7c3 PCI: Disable VF decoding before pcibios_sriov_disable() updates resources
A struct resource represents the address space consumed by a device.  We
should not modify that resource while the device is actively using the
address space.  For VFs, pci_iov_update_resource() enforces this by
printing a warning and doing nothing if the VFE (VF Enable) and MSE (VF
Memory Space Enable) bits are set.

Previously, both sriov_enable() and sriov_disable() called the
pcibios_sriov_disable() arch hook, which may update the struct resource,
while VFE and MSE were enabled.  This effectively dropped the resource
update pcibios_sriov_disable() intended to do.

Disable VF memory decoding before calling pcibios_sriov_disable().

Reported-by: Carol L Soto <clsoto@us.ibm.com>
Tested-by: Carol L Soto <clsoto@us.ibm.com>
Signed-off-by: Gavin Shan <gwshan@linux.vnet.ibm.com>
Signed-off-by: Alexey Kardashevskiy <aik@ozlabs.ru>
[bhelgaas: changelog]
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: shan.gavin@gmail.com
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
2017-08-29 17:24:02 -05:00
Jakub Kicinski
17530e71e0 PCI: Protect pci_driver->sriov_configure() usage with device_lock()
Every method in struct device_driver or structures derived from it like
struct pci_driver MUST provide exclusion vs the driver's ->remove() method,
usually by using device_lock().

Protect use of pci_driver->sriov_configure() by holding the device lock
while calling it.

The PCI core sets the pci_dev->driver pointer in local_pci_probe() before
calling ->probe() and only clears it after ->remove().  This means driver's
->sriov_configure() callback will happily race with probe() and remove(),
most likely leading to BUGs, since drivers don't expect this.

Remove the iov lock completely, since we remove the last user.

[bhelgaas: changelog, thanks to Christoph for locking rule]
Link: http://lkml.kernel.org/r/20170522225023.14010-1-jakub.kicinski@netronome.com
Signed-off-by: Jakub Kicinski <jakub.kicinski@netronome.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
2017-06-14 17:41:19 -05:00
Bodong Wang
0e7df22401 PCI: Add sysfs sriov_drivers_autoprobe to control VF driver binding
Sometimes it is not desirable to bind SR-IOV VFs to drivers.  This can save
host side resource usage by VF instances that will be assigned to VMs.

Add a new PCI sysfs interface "sriov_drivers_autoprobe" to control that
from the PF.  To modify it, echo 0/n/N (disable probe) or 1/y/Y (enable
probe) to:

  /sys/bus/pci/devices/<DOMAIN:BUS:DEVICE.FUNCTION>/sriov_drivers_autoprobe

Note that this must be done before enabling VFs.  The change will not take
effect if VFs are already enabled.  Simply, one can disable VFs by setting
sriov_numvfs to 0, choose whether to probe or not, and then re-enable the
VFs by restoring sriov_numvfs.

[bhelgaas: changelog, ABI doc]
Signed-off-by: Bodong Wang <bodong@mellanox.com>
Signed-off-by: Eli Cohen <eli@mellanox.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Gavin Shan <gwshan@linux.vnet.ibm.com>
Reviewed-by: Alex Williamson <alex.williamson@redhat.com>
2017-04-20 08:53:51 -05:00
Emil Tantilov
5b0948dfe1 PCI: Lock each enable/disable num_vfs operation in sysfs
Enabling/disabling SRIOV via sysfs by echo-ing multiple values
simultaneously:

  # echo 63 > /sys/class/net/ethX/device/sriov_numvfs&
  # echo 63 > /sys/class/net/ethX/device/sriov_numvfs

  # sleep 5

  # echo 0 > /sys/class/net/ethX/device/sriov_numvfs&
  # echo 0 > /sys/class/net/ethX/device/sriov_numvfs

results in the following bug:

  kernel BUG at drivers/pci/iov.c:495!
  invalid opcode: 0000 [#1] SMP
  CPU: 1 PID: 8050 Comm: bash Tainted: G   W   4.9.0-rc7-net-next #2092
  RIP: 0010:[<ffffffff813b1647>]
	    [<ffffffff813b1647>] pci_iov_release+0x57/0x60

  Call Trace:
   [<ffffffff81391726>] pci_release_dev+0x26/0x70
   [<ffffffff8155be6e>] device_release+0x3e/0xb0
   [<ffffffff81365ee7>] kobject_cleanup+0x67/0x180
   [<ffffffff81365d9d>] kobject_put+0x2d/0x60
   [<ffffffff8155bc27>] put_device+0x17/0x20
   [<ffffffff8139c08a>] pci_dev_put+0x1a/0x20
   [<ffffffff8139cb6b>] pci_get_dev_by_id+0x5b/0x90
   [<ffffffff8139cca5>] pci_get_subsys+0x35/0x40
   [<ffffffff8139ccc8>] pci_get_device+0x18/0x20
   [<ffffffff8139ccfb>] pci_get_domain_bus_and_slot+0x2b/0x60
   [<ffffffff813b09e7>] pci_iov_remove_virtfn+0x57/0x180
   [<ffffffff813b0b95>] pci_disable_sriov+0x65/0x140
   [<ffffffffa00a1af7>] ixgbe_disable_sriov+0xc7/0x1d0 [ixgbe]
   [<ffffffffa00a1e9d>] ixgbe_pci_sriov_configure+0x3d/0x170 [ixgbe]
   [<ffffffff8139d28c>] sriov_numvfs_store+0xdc/0x130
  ...
  RIP  [<ffffffff813b1647>] pci_iov_release+0x57/0x60

Use the existing mutex lock to protect each enable/disable operation.

Signed-off-by: Emil Tantilov <emil.s.tantilov@intel.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Gavin Shan <gwshan@linux.vnet.ibm.com>
CC: Alexander Duyck <alexander.h.duyck@intel.com>
2017-02-03 13:42:38 -06:00