Commit Graph

70 Commits

Author SHA1 Message Date
Serge Semin
acf994151d dmaengine: dw-edma: Depend on DW_EDMA instead of selecting it
Kconfig "select" is discouraged for visible symbols like DW_EDMA because it
makes it possible to set DW_EDMA even if DW_EDMA depends on things that are
not set (see Documentation/kbuild/kconfig-language.txt).

Convert DW_EDMA_PCIE so it depends on DW_EDMA instead of selecting it.

There will likely be several future drivers that depend on DW_EDMA, so this
uses "if DW_EDMA" to enclose them all rather than repeating "depends on
DW_EDMA" for each.

[bhelgaas: split to separate patch, commit log]
Link: https://lore.kernel.org/r/20230113171409.30470-25-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Vinod Koul <vkoul@kernel.org>
2023-02-22 13:46:04 -06:00
Serge Semin
16f8a08643 dmaengine: dw-edma: Add mem-mapped LL-entries support
Currently the DW eDMA driver only supports the linked lists memory
allocated locally with respect to the remote eDMA engine setup. It means
the linked lists will be accessible by the CPU via the MMIO space only. If
eDMA is embedded into the DW PCIe Root Ports or local Endpoints (which
support will be added in subsequent commits) the linked lists are supposed
to be allocated in the CPU memory. In that case the LL-entries can be
directly accessed, while the former case implies using the MMIO accessors
for that.

In order to have both cases supported by the driver, the dw_edma_region
descriptor should be fixed to contain the MMIO-backed and just memory-based
virtual addresses. The linked lists initialization procedure will use one
of them depending on the eDMA device nature. If the eDMA engine is embedded
into the local DW PCIe Root Port/Endpoint controllers, the list entries
will be directly accessed by referencing the corresponding structure
fields.  Otherwise the MMIO accessors usage will be preserved.

Link: https://lore.kernel.org/r/20230113171409.30470-24-Sergey.Semin@baikalelectronics.ru
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-by: Vinod Koul <vkoul@kernel.org>
2023-02-22 13:45:53 -06:00
Serge Semin
4ac1662105 dmaengine: dw-edma: Skip cleanup procedure if no private data found
DW eDMA driver private data is preserved in the passed DW eDMA chip info
structure. If the probe fails or for some reason the passed info object
doesn't have the private data pointer initialized, halt the DMA device
cleanup procedure to prevent system crashes.

Link: https://lore.kernel.org/r/20230113171409.30470-23-Sergey.Semin@baikalelectronics.ru
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-by: Vinod Koul <vkoul@kernel.org>
2023-02-10 17:15:23 -06:00
Serge Semin
53c0e2f9b8 dmaengine: dw-edma: Replace chip ID number with device name
Using an abstract number as the DW eDMA chip identifier isn't practical
because there can be more than one DW eDMA controller on the platform. Some
may be detected as the PCIe Endpoints, and others may be embedded in DW
PCIe Root Port/Endpoint controllers.  An abstract number in, for instance,
the IRQ handlers list, doesn't give a notion regarding their reference to
the particular DMA controller.

To preserve the code simplicity and support multi-eDMA platforms, use the
parental device name to create the DW eDMA controller name.

Link: https://lore.kernel.org/r/20230113171409.30470-22-Sergey.Semin@baikalelectronics.ru
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-by: Vinod Koul <vkoul@kernel.org>
2023-02-10 17:15:23 -06:00
Serge Semin
7119685cf4 dmaengine: dw-edma: Drop DT-region allocation
There is no point in allocating additional memory for the data target
regions passed to the client drivers. Use the already available structures
defined in the dw_edma_chip instance.

Note: these regions are unused in normal circumstances since they are
specific to the case of eDMA being embedded into the DW PCIe Endpoint and
having its CSRs accessible via an Endpoint BAR. This case is only known to
be implemented as a part of the Synopsys PCIe Endpoint IP prototype kit.

Link: https://lore.kernel.org/r/20230113171409.30470-21-Sergey.Semin@baikalelectronics.ru
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-by: Vinod Koul <vkoul@kernel.org>
2023-02-10 17:15:23 -06:00
Serge Semin
b73bdc5054 dmaengine: dw-edma: Use non-atomic io-64 methods
Instead of splitting 64-bits IOs up into two 32-bits ones, use the existing
non-atomic readq()/writeq() functions. By doing so we can discard
CONFIG_64BIT #ifdefs from the code.

Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-by: Vinod Koul <vkoul@kernel.org>
2023-02-10 17:15:23 -06:00
Serge Semin
5fdca4a995 dmaengine: dw-edma: Fix readq_ch() return value truncation
Previously, readq_ch() did a 64-bit readq(), but truncated the result by
storing it in the u32 "value".  Change "value" to u64 to avoid the
truncation.

Note: the method is currently unused, so the bug hasn't caused any problem
so far.

Fixes: 04e0a39fc1 ("dmaengine: dw-edma: Add writeq() and readq() for 64 bits architectures")
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
2023-02-10 17:15:23 -06:00
Serge Semin
6cb6e9c605 dmaengine: dw-edma: Use DMA engine device debugfs subdirectory
Since all DW eDMA read and write channels are now installed in a framework
of a single DMA engine device, move all the DW eDMA-specific debugfs nodes
into a ready-to-use DMA-engine debugfs subdirectory. It's created during
the DMA-device registration and can be found in the dma_device.dbg_dev_root
field.

Link: https://lore.kernel.org/r/20230113171409.30470-19-Sergey.Semin@baikalelectronics.ru
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-by: Vinod Koul <vkoul@kernel.org>
2023-02-10 17:15:23 -06:00
Serge Semin
3883d64449 dmaengine: dw-edma: Join read/write channels into a single device
There is no point in splitting read/write channels.  First of all, eDMA
read and write channels belong to one physical controller. Secondly,
channel differentiation can be done by filtering and dma_get_slave_caps().
Finally, having these channels handled separately needlessly complicates
the code and causes this debugfs warning:

  debugfs: Directory '1f052000.pcie' with parent 'dmaengine' already present!

Join the read/write channels into a single DMA device.  Client drivers can
choose the correct channel via the DMA slave direction setting. The default
value is overridden by the dw_edma_device_caps() callback in accordance
with the channel type.

Link: https://lore.kernel.org/r/20230113171409.30470-18-Sergey.Semin@baikalelectronics.ru
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-by: Vinod Koul <vkoul@kernel.org>
2023-02-10 17:15:23 -06:00
Serge Semin
d015216853 dmaengine: dw-edma: Move eDMA data pointer to debugfs node descriptor
The last thing that stops the debugfs part of the eDMA driver from
supporting multi-eDMA platforms is keeping the eDMA private data pointer in
the static area of the debugfs module. Since the debugfs node descriptors
are now heap-allocated, we can freely move that pointer to being preserved
in the descriptors. After the debugfs initialization procedure, that
pointer will be used in the debugfs files getter to access the common CSRs
space and the context CSRs spinlock. So the main part of this change is
connected with the debugfs nodes descriptors initialization macros, which
aside with already defined prototypes now require to have the DW eDMA
private data pointer passed.

[bhelgaas: squash in https://lore.kernel.org/r/20230130185101.2883245-1-arnd@kernel.org]
Link: https://lore.kernel.org/r/20230113171409.30470-17-Sergey.Semin@baikalelectronics.ru
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-by: Vinod Koul <vkoul@kernel.org>
2023-02-10 17:14:42 -06:00
Serge Semin
0049816765 dmaengine: dw-edma: Simplify debugfs context CSRs init procedure
DW eDMA v4.70a and older have the read and write channels context CSRs
indirectly accessible, which means CSRs like Channel Control, Xfer size,
SAR, DAR and LLP address are accessed at a fixed MMIO address, with their
reference to the corresponding channel determined by the Viewport CSR. To
have a coherent access to these registers the CSR IOs are supposed to be
protected with a spinlock. DW eDMA v4.80a and newer normally have unrolled
Read/Write channel context registers, with these CSRs directly mapped in
the controller MMIO space.

Both normal and viewport-based registers are exposed via debugfs nodes, and
the original algorithm was based on the unrolled CSRs mapping and
recalculated the viewport addresses when required. This is unscalable (it
only supports a platform with a single eDMA since a base address is
statically preserved) and also needlessly overcomplicated (it loops over
all Rd/Wr context addresses and recalculates the viewport base address on
each debugfs node access).

Simplify the algorithm by adding the channel ID and its direction fields in
the eDMA debugfs node descriptor. These new fields can be used to find a
CSR offset in the channel register space. The DW eDMA debugfs node getter
will also use them to activate the respective context CSRs viewport before
reading data from the specified register. For the unrolled CSR mapping, no
spinlock or viewport activation is needed.

Note: this replaces some REGISTER() uses with CTX_REGISTER(), which avoids
an implicit dependency on a local variable name.  The same problem with the
rest of the macro will be fixed in the next commit.

Link: https://lore.kernel.org/r/20230113171409.30470-16-Sergey.Semin@baikalelectronics.ru
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-by: Vinod Koul <vkoul@kernel.org>
2023-01-27 17:15:33 +01:00
Serge Semin
95c55b7836 dmaengine: dw-edma: Rename debugfs dentry variables to 'dent'
Since we are about to add the eDMA channels direction support to the
debugfs module it will be confusing to have both the debugfs directory and
the channels direction short names used in the same code.

Rename the debugfs dentry 'dir' variables to 'dent' to prevent confusion.

Suggested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Link: https://lore.kernel.org/r/20230113171409.30470-15-Sergey.Semin@baikalelectronics.ru
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-by: Vinod Koul <vkoul@kernel.org>
2023-01-27 17:15:33 +01:00
Serge Semin
782536aac1 dmaengine: dw-edma: Convert debugfs descs to being heap-allocated
Currently DW eDMA debugfs node descriptors are allocated on the stack,
which won't work for multi-eDMA platforms. As a preparation to supporting
multi-eDMA systems, allocate each debugfs node separately.  Afterwards
we'll add info like Read/Write channel flag, channel ID, DW eDMA private
data reference.

Note: this conversion is mainly required due to having the legacy DW eDMA
controllers with indirect Read/Write channels context CSRs access. If we
didn't need to synchronize access to these registers, the debugfs code of
the driver would have been much simpler.

Link: https://lore.kernel.org/r/20230113171409.30470-14-Sergey.Semin@baikalelectronics.ru
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-by: Vinod Koul <vkoul@kernel.org>
2023-01-27 17:15:33 +01:00
Serge Semin
345e3a95b2 dmaengine: dw-edma: Add dw_edma prefix to debugfs nodes descriptor
Other local names include a "dw_edma" prefix.

Add a "dw_edma" prefix to the debugfs_entries structure, too, so it won't
be confused with global debugfs things.

Link: https://lore.kernel.org/r/20230113171409.30470-13-Sergey.Semin@baikalelectronics.ru
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-by: Vinod Koul <vkoul@kernel.org>
2023-01-27 17:15:33 +01:00
Serge Semin
37d058aae7 dmaengine: dw-edma: Stop checking debugfs_create_*() return value
The debugfs_create_*() functions never return NULL, so checking their
return value for NULL is pointless. Secondly, the debugfs subsystem is
designed to be as simple as possible, so if one of the debugfs_create_*()
method in a hierarchy fails, the following methods should silently return
the passed erroneous parental dentry. Finally, the code should work no
matter whether anything debugfs-related fails.

To make code simpler and debugfs-independent, stop checking the
debugfs_create_*() return values.

If the debugfs file system is unavailable, skip the debugfs node
initialization altogether to preserve some memory space.

Link: https://lore.kernel.org/r/20230113171409.30470-12-Sergey.Semin@baikalelectronics.ru
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-by: Vinod Koul <vkoul@kernel.org>
2023-01-27 17:15:33 +01:00
Serge Semin
7ca9f025a7 dmaengine: dw-edma: Drop unnecessary debugfs reg casts
The debugfs_entries structure declared in dw-edma-v0-debugfs.c contains the
debugfs node register address. The address is declared as dma_addr_t type,
but is cast to "void *".

Change the type to "void __iomem *" and drop the unnecessary casts.

Link: https://lore.kernel.org/r/20230113171409.30470-11-Sergey.Semin@baikalelectronics.ru
Fixes: 305aebeff8 ("dmaengine: Add Synopsys eDMA IP version 0 debugfs support")
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-by: Vinod Koul <vkoul@kernel.org>
2023-01-27 17:15:33 +01:00
Serge Semin
2271216e0a dmaengine: dw-edma: Drop chancnt initialization
The DMA engine core manages dma_device.chancnt itself, e.g., in
dma_async_device_register().  DMA device drivers should not initialize
chancnt because it causes the wrong number of channels printed in the
device summary.

Drop the dw-edma chancnt initialization.

Link: https://lore.kernel.org/r/20230113171409.30470-10-Sergey.Semin@baikalelectronics.ru
Fixes: e63d79d1ff ("dmaengine: Add Synopsys eDMA IP core driver")
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-by: Vinod Koul <vkoul@kernel.org>
2023-01-27 17:15:33 +01:00
Serge Semin
aa92fa1e53 dmaengine: dw-edma: Add PCI bus address getter to the remote EP glue driver
The Synopsys PCIe Endpoint IP prototype kit can be attached via any PCI
host controller, including one where the PCI bus address space is different
from the CPU address space.  Therefore, we need to make sure the source and
destination addresses of the DMA slave devices are converted to the PCI bus
address space; otherwise DMA transactions may cause memory corruption.

Add a new dw_edma_pcie_address() interface to perform this translation by
using pcibios_resource_to_bus().

Link: https://lore.kernel.org/r/20230113171409.30470-9-Sergey.Semin@baikalelectronics.ru
Fixes: 41aaff2a2a ("dmaengine: Add Synopsys eDMA IP PCIe glue-logic")
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-by: Vinod Koul <vkoul@kernel.org>
2023-01-27 17:15:33 +01:00
Serge Semin
993d57bbaa dmaengine: dw-edma: Add CPU to PCI bus address translation
Since 9575632052 ("dmaengine: make slave address physical"), the source
and destination addresses of the DMA slave device have been converted to
physical addresses in the CPU address space. It's the DMA device driver's
responsibility to convert them to the DMA bus address space. In case of the
DW eDMA device, the source or destination peripheral (slave) devices reside
in PCI bus space. Thus we need to perform the PCI Host/Endpoint windows-
based (i.e. DT "ranges" property) address translation; otherwise the eDMA
transactions won't work as expected (or can be even harmful) if the CPU and
PCI address spaces don't match.

Note 1: Even though the DMA interleaved template has both source and
destination addresses declared as dma_addr_t, only the CPU memory range
should be mapped to be seen by the DMA device since it's a subject of the
DMA getting towards the system side. The device part must not be mapped
since the slave device resides in the PCI bus space, which isn't affected
by IOMMUs or iATU translations. DW PCIe eDMA generates corresponding
MWr/MRd TLPs on its own.

Note 2: This functionality is mainly required for the remote eDMA setup
since the CPU address must be manually translated into the PCI bus space
before being written to LLI.{SAR,DAR}. If eDMA is embedded in the locally
accessible DW PCIe Root Port/Endpoint, software-based translation isn't
required since hardware will translate it via the Outbound iATU as long as
the DMA_BYPASS flag is cleared. If DMA_BYPASS is set or there is no
Outbound iATU entry that contains the SAR or DAR (for Read and Write
channel respectively), there won't be any translation performed but DMA
will proceed with the corresponding source/destination address as-is.

Link: https://lore.kernel.org/r/20230113171409.30470-8-Sergey.Semin@baikalelectronics.ru
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-by: Vinod Koul <vkoul@kernel.org>
2023-01-27 17:15:33 +01:00
Serge Semin
7ad06f2184 dmaengine: dw-edma: Fix invalid interleaved xfers semantics
The interleaved DMA transfer support added by 85e7518f42 ("dmaengine:
dw-edma: Add device_prep_interleave_dma() support") seems contradictory to
what the DMA engine defines. The next conditional statements:

  if (!xfer->xfer.il->numf)
    return NULL;
  if (xfer->xfer.il->numf > 0 && xfer->xfer.il->frame_size > 0)
    return NULL;

mean that numf can't be zero and frame_size must always be zero, otherwise
the transfer won't be executed. Furthermore, the transfer execution method
takes the frame size from the dma_interleaved_template.sgl[] array for each
frame. That array in accordance with [1] is supposed to be of
dma_interleaved_template.frame_size size, which as we discovered before the
code expects to be zero. So judging by the dw_edma_device_transfer()
implementation, the method implies the dma_interleaved_template.sgl[] array
being of dma_interleaved_template.numf size, which is wrong. Since the
dw_edma_device_transfer() method doesn't permit
dma_interleaved_template.frame_size being non-zero, the multi-chunk
interleaved transfer turns to be unsupported even though the code implies
having it supported.

Add fully functioning support of interleaved DMA transfers.

First of all, dma_interleaved_template.frame_size is supposed to be greater
or equal to one thus having at least simple linear chunked frames.
Secondly, we can create a walk-through over all the chunks and frames by
initializing the number of the eDMA burst transactions as a multiple of
dma_interleaved_template.numf and dma_interleaved_template.frame_size and
getting the frame_size-modulo of the iteration step as an index of the
dma_interleaved_template.sgl[] array.

[1] include/linux/dmaengine.h: doc struct dma_interleaved_template

Link: https://lore.kernel.org/r/20230113171409.30470-7-Sergey.Semin@baikalelectronics.ru
Fixes: 85e7518f42 ("dmaengine: dw-edma: Add device_prep_interleave_dma() support")
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-by: Vinod Koul <vkoul@kernel.org>
2023-01-27 17:15:33 +01:00
Serge Semin
c8ed491822 dmaengine: dw-edma: Don't permit non-inc interleaved xfers
The DW eDMA controller always increments both source and destination
addresses. Permitting DMA interleaved transfers with no src_inc/dst_inc
flags set may lead to unexpected behaviour for the device users.

Terminate interleaved transfers if at least one of the
dma_interleaved_template.{src_inc,dst_inc} flag is initialized to "false".

Note that in addition, we need to increase the source and destination
addresses after each iteration.

Link: https://lore.kernel.org/r/20230113171409.30470-6-Sergey.Semin@baikalelectronics.ru
Fixes: 85e7518f42 ("dmaengine: dw-edma: Add device_prep_interleave_dma() support")
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-by: Vinod Koul <vkoul@kernel.org>
2023-01-27 17:15:33 +01:00
Serge Semin
13b6299cf6 dmaengine: dw-edma: Fix missing src/dst address of interleaved xfers
Interleaved DMA transfer support was added by 85e7518f42 ("dmaengine:
dw-edma: Add device_prep_interleave_dma() support"), but depending on the
selected channel, either source or destination address are left
uninitialized which was obviously wrong.

Initialize the destination address of the eDMA burst descriptors for
DEV_TO_MEM interleaved operations and the source address for MEM_TO_DEV
operations.

Link: https://lore.kernel.org/r/20230113171409.30470-5-Sergey.Semin@baikalelectronics.ru
Fixes: 85e7518f42 ("dmaengine: dw-edma: Add device_prep_interleave_dma() support")
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-by: Vinod Koul <vkoul@kernel.org>
2023-01-27 17:15:33 +01:00
Serge Semin
002bbaa2f6 dmaengine: dw-edma: Convert ll/dt phys address to PCI bus/DMA address
The dw_edma_region.paddr field should be a memory base address visible by
the DW eDMA controller. If the DMA engine is embedded in the DW PCIe
Host/Endpoint controller, the address should belong to the Local CPU/
Application memory.  If eDMA is remotely accessible across the PCI bus via
PCI memory IOs, the address should be part of the PCI bus memory space.
The latter case hasn't been well covered in the corresponding glue-driver.

Since pci_dev.resource[] contains resources defined in the CPU memory
space, they need to be converted to the PCI bus address space.  Convert the
LL, DT and CSRs PCI memory ranges with pci_bus_address().

In addition, extend the dw_edma_region.paddr field size. The field normally
contains a memory range base address to be set in the DW eDMA Linked-List
pointer register or as a base address of the Linked-List data buffer. In
accordance with [1] the LL range is supposed to be created in the Local
CPU/Application memory, but depending on the DW eDMA utilization the memory
can be created as a part of the PCI bus address space (as in the case of
the DW PCIe Endpoint prototype kit).

In the former case dw_edma_region.paddr should be a dma_addr_t, while in
the latter one it should be a pci_bus_addr_t. Since the corresponding CSRs
are always 64 bits wide, convert dw_edma_region.paddr to be u64, and let
the client make sure it has a valid address visible by the DW eDMA
controller. For instance, the DW eDMA PCIe glue-driver initializes the
field with addresses from the PCI bus memory space.

[1] DesignWare Cores PCI Express Controller Databook - DWC PCIe Root Port,
    v.5.40a, March 2019, p.1103

Link: https://lore.kernel.org/r/20230113171409.30470-4-Sergey.Semin@baikalelectronics.ru
Fixes: 41aaff2a2a ("dmaengine: Add Synopsys eDMA IP PCIe glue-logic")
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-by: Vinod Koul <vkoul@kernel.org>
2023-01-27 17:15:33 +01:00
Serge Semin
0278067445 dmaengine: dw-edma: Release requested IRQs on failure
If dw_edma_irq_request() fails to initialize an IRQ handler, any previously
requested IRQs will be left initialized.

Release the previously requested IRQs in the cleanup-on-error path of
dw_edma_irq_request().

Link: https://lore.kernel.org/r/20230113171409.30470-3-Sergey.Semin@baikalelectronics.ru
Fixes: e63d79d1ff ("dmaengine: Add Synopsys eDMA IP core driver")
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-by: Vinod Koul <vkoul@kernel.org>
2023-01-27 17:15:33 +01:00
Manivannan Sadhasivam
a0188eb6e7 dmaengine: dw-edma: Remove runtime PM support
Currently, the dw-edma driver enables the runtime_pm for parent device
(chip->dev) and increments/decrements the refcount during alloc/free
chan resources callbacks.

This leads to a problem when the eDMA driver has been probed, but the
channels were not used. This scenario can happen when the DW PCIe driver
probes eDMA driver successfully, but the PCI EPF driver decides not to
use eDMA channels and use iATU instead for PCI transfers.

In this case, the underlying device would be runtime suspended due to
pm_runtime_enable() in dw_edma_probe() and the PCI EPF driver would have
no knowledge of it.

Ideally, the eDMA driver should not be the one doing the runtime PM of
the parent device. The responsibility should instead belong to the client
drivers like PCI EPF.

So let's remove the runtime PM support from eDMA driver.

Cc: Serge Semin <fancer.lancer@gmail.com>
Cc: Frank Li <Frank.Li@nxp.com>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Link: https://lore.kernel.org/r/20220910054700.12205-1-manivannan.sadhasivam@linaro.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2022-09-29 22:46:08 +05:30
Linus Torvalds
eff0cb3d91 pci-v5.20-changes
-----BEGIN PGP SIGNATURE-----
 
 iQJIBAABCgAyFiEEgMe7l+5h9hnxdsnuWYigwDrT+vwFAmLr+2wUHGJoZWxnYWFz
 QGdvb2dsZS5jb20ACgkQWYigwDrT+vxfZg//eChkC2EUdT6K3zuQDbJJhsGcuOQF
 lnZuUyDn4xw7BkEoZf8V6YdAnp7VvgKhLOq1/q3Geu/LBbCaczoEogOCaR/WcVOs
 C+MsN0RWZQtgfuZKncQoqp25NeLPK9PFToeiIX/xViAYZF7NVjDY7XQiZHQ6JkEA
 /7cUqv/4nS3KCMsKjfmiOxGnqohMWtICiw9qjFvJ40PEDnNB1b53rkiVTxBFePpI
 ePfsRfi/C7klE3xNfoiEgrPp+Jfw+oShsCwXUsId7bEL2oLBc7ClqP05ZYZD3bTK
 QQYyZ12Cq8TysciYpUGBjBnywUHS5DIO5YaV3wxyVAR2Z+6GY2/QVjOa2kKvoK0o
 Hba6TJf8bL58AhSI8Q62pBM0sS7dqJSff+9c2BGpZvII5spP/rQQLlJO56TJjwkw
 Dlf0d3thhZOc9vSKjKw+0v0FdAyc4L11EOwUsw95jZeT5WWgqJYGFnWPZwqBI1KM
 DI1E5wVO5tA2H3NEn+BTTHbLWL+UppqyXPXBHiW52b2q5Bt8fJWMsFvnEEjclxmG
 pYCI7VgF8jqbYKxjobxPFY2x6PH9hfaGMxwzZSdOX6e/Eh+1esgyyaC5APpCO+Pp
 e4OkJaOzCmggrD0jYeLWu+yDm5KRrYo5cdfKHrKgAof0Am41lAa1OhJ2iH4ckNqP
 1qmHereDOe0zNVw=
 =9TAR
 -----END PGP SIGNATURE-----

Merge tag 'pci-v5.20-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci

Pull pci updates from Bjorn Helgaas:
 "Enumeration:

   - Consolidate duplicated 'next function' scanning and extend to allow
     'isolated functions' on s390, similar to existing hypervisors
     (Niklas Schnelle)

  Resource management:
   - Implement pci_iobar_pfn() for sparc, which allows us to remove the
     sparc-specific pci_mmap_page_range() and pci_mmap_resource_range().

     This removes the ability to map the entire PCI I/O space using
     /proc/bus/pci, but we believe that's already been broken since
     v2.6.28 (Arnd Bergmann)

   - Move common PCI definitions to asm-generic/pci.h and rework others
     to be be more specific and more encapsulated in arches that need
     them (Stafford Horne)

  Power management:

   - Convert drivers to new *_PM_OPS macros to avoid need for '#ifdef
     CONFIG_PM_SLEEP' or '__maybe_unused' (Bjorn Helgaas)

  Virtualization:

   - Add ACS quirk for Broadcom BCM5750x multifunction NICs that isolate
     the functions but don't advertise an ACS capability (Pavan Chebbi)

  Error handling:

   - Clear PCI Status register during enumeration in case firmware left
     errors logged (Kai-Heng Feng)

   - When we have native control of AER, enable error reporting for all
     devices that support AER. Previously only a few drivers enabled
     this (Stefan Roese)

   - Keep AER error reporting enabled for switches. Previously we
     enabled this during enumeration but immediately disabled it (Stefan
     Roese)

   - Iterate over error counters instead of error strings to avoid
     printing junk in AER sysfs counters (Mohamed Khalfella)

  ASPM:

   - Remove pcie_aspm_pm_state_change() so ASPM config changes, e.g.,
     via sysfs, are not lost across power state changes (Kai-Heng Feng)

  Endpoint framework:

   - Don't stop an EPC when unbinding an EPF from it (Shunsuke Mie)

  Endpoint embedded DMA controller driver:

   - Simplify and clean up support for the DesignWare embedded DMA
     (eDMA) controller (Frank Li, Serge Semin)

  Broadcom STB PCIe controller driver:

   - Avoid config space accesses when link is down because we can't
     recover from the CPU aborts these cause (Jim Quinlan)

   - Look for power regulators described under Root Ports in DT and
     enable them before scanning the secondary bus (Jim Quinlan)

   - Disable/enable regulators in suspend/resume (Jim Quinlan)

  Freescale i.MX6 PCIe controller driver:

   - Simplify and clean up clock and PHY management (Richard Zhu)

   - Disable/enable regulators in suspend/resume (Richard Zhu)

   - Set PCIE_DBI_RO_WR_EN before writing DBI registers (Richard Zhu)

   - Allow speeds faster than Gen2 (Richard Zhu)

   - Make link being down a non-fatal error so controller probe doesn't
     fail if there are no Endpoints connected (Richard Zhu)

  Loongson PCIe controller driver:

   - Add ACPI and MCFG support for Loongson LS7A (Huacai Chen)

   - Avoid config reads to non-existent LS2K/LS7A devices because a
     hardware defect causes machine hangs (Huacai Chen)

   - Work around LS7A integrated devices that report incorrect Interrupt
     Pin values (Jianmin Lv)

  Marvell Aardvark PCIe controller driver:

   - Add support for AER and Slot capability on emulated bridge (Pali
     Rohár)

  MediaTek PCIe controller driver:

   - Add Airoha EN7532 to DT binding (John Crispin)

   - Allow building of driver for ARCH_AIROHA (Felix Fietkau)

  MediaTek PCIe Gen3 controller driver:

   - Print decoded LTSSM state when the link doesn't come up (Jianjun
     Wang)

  NVIDIA Tegra194 PCIe controller driver:

   - Convert DT binding to json-schema (Vidya Sagar)

   - Add DT bindings and driver support for Tegra234 Root Port and
     Endpoint mode (Vidya Sagar)

   - Fix some Root Port interrupt handling issues (Vidya Sagar)

   - Set default Max Payload Size to 256 bytes (Vidya Sagar)

   - Fix Data Link Feature capability programming (Vidya Sagar)

   - Extend Endpoint mode support to devices beyond Controller-5 (Vidya
     Sagar)

  Qualcomm PCIe controller driver:

   - Rework clock, reset, PHY power-on ordering to avoid hangs and
     improve consistency (Robert Marko, Christian Marangi)

   - Move pipe_clk handling to PHY drivers (Dmitry Baryshkov)

   - Add IPQ60xx support (Selvam Sathappan Periakaruppan)

   - Allow ASPM L1 and substates for 2.7.0 (Krishna chaitanya chundru)

   - Add support for more than 32 MSI interrupts (Dmitry Baryshkov)

  Renesas R-Car PCIe controller driver:

   - Convert DT binding to json-schema (Herve Codina)

   - Add Renesas RZ/N1D (R9A06G032) to rcar-gen2 DT binding and driver
     (Herve Codina)

  Samsung Exynos PCIe controller driver:

   - Fix phy-exynos-pcie driver so it follows the 'phy_init() before
     phy_power_on()' PHY programming model (Marek Szyprowski)

  Synopsys DesignWare PCIe controller driver:

   - Simplify and clean up the DWC core extensively (Serge Semin)

   - Fix an issue with programming the ATU for regions that cross a 4GB
     boundary (Serge Semin)

   - Enable the CDM check if 'snps,enable-cdm-check' exists; previously
     we skipped it if 'num-lanes' was absent (Serge Semin)

   - Allocate a 32-bit DMA-able page to be MSI target instead of using a
     driver data structure that may not be addressable with 32-bit
     address (Will McVicker)

   - Add DWC core support for more than 32 MSI interrupts (Dmitry
     Baryshkov)

  Xilinx Versal CPM PCIe controller driver:

   - Add DT binding and driver support for Versal CPM5 Gen5 Root Port
     (Bharat Kumar Gogada)"

* tag 'pci-v5.20-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci: (150 commits)
  PCI: imx6: Support more than Gen2 speed link mode
  PCI: imx6: Set PCIE_DBI_RO_WR_EN before writing DBI registers
  PCI: imx6: Reformat suspend callback to keep symmetric with resume
  PCI: imx6: Move the imx6_pcie_ltssm_disable() earlier
  PCI: imx6: Disable clocks in reverse order of enable
  PCI: imx6: Do not hide PHY driver callbacks and refine the error handling
  PCI: imx6: Reduce resume time by only starting link if it was up before suspend
  PCI: imx6: Mark the link down as non-fatal error
  PCI: imx6: Move regulator enable out of imx6_pcie_deassert_core_reset()
  PCI: imx6: Turn off regulator when system is in suspend mode
  PCI: imx6: Call host init function directly in resume
  PCI: imx6: Disable i.MX6QDL clock when disabling ref clocks
  PCI: imx6: Propagate .host_init() errors to caller
  PCI: imx6: Collect clock enables in imx6_pcie_clk_enable()
  PCI: imx6: Factor out ref clock disable to match enable
  PCI: imx6: Move imx6_pcie_clk_disable() earlier
  PCI: imx6: Move imx6_pcie_enable_ref_clk() earlier
  PCI: imx6: Move PHY management functions together
  PCI: imx6: Move imx6_pcie_grp_offset(), imx6_pcie_configure_type() earlier
  PCI: imx6: Convert to NOIRQ_SYSTEM_SLEEP_PM_OPS()
  ...
2022-08-04 19:30:35 -07:00
Frank Li
d6b03171f9 dmaengine: dw-edma: Add support for chip-specific flags
Add a "flags" field to the "struct dw_edma_chip" so that the controller
drivers can pass flags that are relevant to the platform.

DW_EDMA_CHIP_LOCAL - Used by the controller drivers accessing eDMA
locally. Local eDMA access doesn't require generating MSIs to the remote.

Link: https://lore.kernel.org/r/20220524152159.2370739-8-Frank.Li@nxp.com
Tested-by: Serge Semin <fancer.lancer@gmail.com>
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-By: Vinod Koul <vkoul@kernel.org>
2022-06-23 14:56:34 -05:00
Serge Semin
c1e3397917 dmaengine: dw-edma: Fix eDMA Rd/Wr-channels and DMA-direction semantics
In accordance with [1, 2] the DW eDMA controller has been created to be
part of the DW PCIe Root Port and DW PCIe End-point controllers and to
offload the transferring of large blocks of data between application and
remote PCIe domains leaving the system CPU free for other tasks. In the
first case (eDMA being part of DW PCIe Root Port) the eDMA controller is
always accessible via the CPU DBI interface and never over the PCIe wire.

The latter case is more complex. Depending on the DW PCIe End-Point IP-core
synthesize parameters it's possible to have the eDMA registers accessible
not only from the application CPU side, but also via mapping the eDMA CSRs
over a dedicated endpoint BAR. So based on the specifics denoted above the
eDMA driver is supposed to support two types of the DMA controller setups:

  1) eDMA embedded into the DW PCIe Root Port/End-point and accessible over
     the local CPU from the application side.

  2) eDMA embedded into the DW PCIe End-point and accessible via the PCIe
     wire with MWr/MRd TLPs generated by the CPU PCIe host controller.

Since the CPU memory resides different sides in these cases the semantics
of the MEM_TO_DEV and DEV_TO_MEM operations is flipped with respect to the
Tx and Rx DMA channels. So MEM_TO_DEV/DEV_TO_MEM corresponds to the Tx/Rx
channels in setup 1) and to the Rx/Tx channels in case of setup 2).

The DW eDMA driver has supported the case 2) since e63d79d1ff
("dmaengine: Add Synopsys eDMA IP core driver") in the framework of the
drivers/dma/dw-edma/dw-edma-pcie.c driver.

The case 1) support was added later by bd96f1b2f4 ("dmaengine: dw-edma:
support local dma device transfer semantics").  Afterwards the driver was
supposed to cover the both possible eDMA setups, but the latter commit
turned out to be not fully correct.

The problem was that the commit together with the new functionality support
also changed the channel direction semantics so the eDMA Read-channel
(corresponding to the DMA_DEV_TO_MEM direction for case 1) now uses the
sgl/cyclic base addresses as the Source addresses of the DMA transfers and
dma_slave_config.dst_addr as the Destination address of the DMA transfers.

Similarly the eDMA Write-channel (corresponding to the DMA_MEM_TO_DEV
direction for case 1) now uses dma_slave_config.src_addr as a source
address of the DMA transfers and sgl/cyclic base address as the Destination
address of the DMA transfers. This contradicts the logic of the
DMA-interface, which implies that DEV side is supposed to belong to the
PCIe device memory and MEM - to the CPU/Application memory. Indeed it seems
irrational to have the SG-list defined in the PCIe bus space, while
expecting a contiguous buffer allocated in the CPU memory. Moreover the
passed SG-list and cyclic DMA buffers are supposed to be mapped in a way so
to be seen by the DW eDMA Application (CPU) interface.

So in order to have the correct DW eDMA interface we need to invert the
eDMA Rd/Wr-channels and DMA-slave directions semantics by selecting the
src/dst addresses based on the DMA transfer direction instead of using the
channel direction capability.

[1] DesignWare Cores PCI Express Controller Databook - DWC PCIe Root Port,
    v.5.40a, March 2019, p.1092
[2] DesignWare Cores PCI Express Controller Databook - DWC PCIe Endpoint,
    v.5.40a, March 2019, p.1189

Co-developed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Fixes: bd96f1b2f4 ("dmaengine: dw-edma: support local dma device transfer semantics")
Link: https://lore.kernel.org/r/20220524152159.2370739-7-Frank.Li@nxp.com
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-By: Vinod Koul <vkoul@kernel.org>
2022-06-23 14:56:34 -05:00
Serge Semin
794a0f7b6f dmaengine: dw-edma: Drop dma_slave_config.direction field usage
The dma_slave_config.direction field usage in the DW eDMA driver was
introduced by bd96f1b2f4 ("dmaengine: dw-edma: support local dma device
transfer semantics"). Mainly the change introduced there was correct
(indeed DEV_TO_MEM means using RD-channel and MEM_TO_DEV - WR-channel for
the case of having eDMA accessed locally from CPU/Application side), but
providing an additional MEM_TO_MEM/DEV_TO_DEV-based semantics was quite
redundant if not to say potentially harmful (when it comes to removing the
denoted field). First of all since the dma_slave_config.direction field has
been marked as obsolete (see [1] and the struct dma_slave_config [2]) and
will be discarded in future, using it especially in a non-standard way is
discouraged. Secondly in accordance with the commit denoted above the
default dw_edma_device_transfer() semantics has been changed despite what
its message said. So claiming that the method was left backward compatible
was wrong.

Fix the problems denoted above and simplify the dw_edma_device_transfer()
method by dropping the parsing of the DMA-channel direction field. Instead
of having that implicit dma_slave_config.direction field semantic, use the
recently added DW_EDMA_CHIP_LOCAL flag to distinguish between the local and
remote DW eDMA setups thus preserving support for both cases. Add an ASCII
figure to clarify the situation.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/driver-api/dmaengine/provider.rst?id=v5.18#n478
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/linux/dmaengine.h?id=v5.18#n389

[bhelgaas: convert references to specific URLs]
Co-developed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Link: https://lore.kernel.org/r/20220524152159.2370739-6-Frank.Li@nxp.com
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Acked-By: Vinod Koul <vkoul@kernel.org>
2022-06-23 14:56:34 -05:00
Frank Li
6951ee96c6 dmaengine: dw-edma: Rename wr(rd)_ch_cnt to ll_wr(rd)_cnt in struct dw_edma_chip
The struct dw_edma contains wr(rd)_ch_cnt fields. The EDMA driver gets
write(read) channel number from register, then saves these into dw_edma.
The wr(rd)_ch_cnt in dw_edma_chip actually means how many link list memory
are available in ll_region_wr(rd)[EDMA_MAX_WR_CH]. Rename it to
ll_wr(rd)_cnt to indicate actual usage.

Link: https://lore.kernel.org/r/20220524152159.2370739-5-Frank.Li@nxp.com
Tested-by: Serge Semin <fancer.lancer@gmail.com>
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-By: Vinod Koul <vkoul@kernel.org>
2022-06-23 14:56:34 -05:00
Frank Li
e51b304811 dmaengine: dw-edma: Change rg_region to reg_base in struct dw_edma_chip
struct dw_edma_region rg_region included virtual address, physical address
and size information. But only the virtual address is used by EDMA driver.
Change it to void __iomem *reg_base to clean up code.

Link: https://lore.kernel.org/r/20220524152159.2370739-4-Frank.Li@nxp.com
Tested-by: Serge Semin <fancer.lancer@gmail.com>
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-By: Vinod Koul <vkoul@kernel.org>
2022-06-23 14:56:34 -05:00
Frank Li
2031845713 dmaengine: dw-edma: Detach the private data and chip info structures
"struct dw_edma_chip" contains an internal structure "struct dw_edma" that
is used by the eDMA core internally and should not be touched by the eDMA
controller drivers themselves. But currently, the eDMA controller drivers
like "dw-edma-pci" allocate and populate this internal structure before
passing it on to the eDMA core. The eDMA core further populates the
structure and uses it. This is wrong!

Hence, move all the "struct dw_edma" specifics from controller drivers to
the eDMA core.

Link: https://lore.kernel.org/r/20220524152159.2370739-3-Frank.Li@nxp.com
Tested-by: Serge Semin <fancer.lancer@gmail.com>
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-By: Vinod Koul <vkoul@kernel.org>
2022-06-23 14:49:30 -05:00
Frank Li
5a0e4529d9 dmaengine: dw-edma: Remove unused irq field in struct dw_edma_chip
The "irq" field of struct dw_edma_chip was never used. Remove it.

Link: https://lore.kernel.org/r/20220524152159.2370739-2-Frank.Li@nxp.com
Tested-by: Serge Semin <fancer.lancer@gmail.com>
Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Signed-off-by: Frank Li <Frank.Li@nxp.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Acked-By: Vinod Koul <vkoul@kernel.org>
2022-06-16 11:31:05 -05:00
Vladimir Zapolskiy
c9357195c5 dmaengine: dw-edma: remove a macro conditional with similar branches
After adding commit 8fc5133d6d ("dmaengine: dw-edma: Fix unaligned
64bit access") two branches under macro conditional become identical,
thus the code can be simplified without any functional change.

Signed-off-by: Vladimir Zapolskiy <vladimir.zapolskiy@linaro.org>
Acked-by: Herve Codina <herve.codina@bootlin.com>
Link: https://lore.kernel.org/r/20220610100700.2295522-1-vladimir.zapolskiy@linaro.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2022-06-10 19:19:24 +05:30
Jiapeng Chong
d4860224e6 dmaengine: dw-edma: Fix inconsistent indenting
Eliminate the follow smatch warning:

drivers/dma/dw-edma/dw-edma-v0-core.c:419 dw_edma_v0_core_start() warn:
inconsistent indenting.

Reported-by: Abaci Robot <abaci@linux.alibaba.com>
Signed-off-by: Jiapeng Chong <jiapeng.chong@linux.alibaba.com>
Link: https://lore.kernel.org/r/20220413023442.18856-1-jiapeng.chong@linux.alibaba.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2022-04-20 17:24:42 +05:30
Herve Codina
8fc5133d6d dmaengine: dw-edma: Fix unaligned 64bit access
On some arch (ie aarch64 iMX8MM) unaligned PCIe accesses are
not allowed and lead to a kernel Oops.
  [ 1911.668835] Unable to handle kernel paging request at virtual address ffff80001bc00a8c
  [ 1911.668841] Mem abort info:
  [ 1911.668844]   ESR = 0x96000061
  [ 1911.668847]   EC = 0x25: DABT (current EL), IL = 32 bits
  [ 1911.668850]   SET = 0, FnV = 0
  [ 1911.668852]   EA = 0, S1PTW = 0
  [ 1911.668853] Data abort info:
  [ 1911.668855]   ISV = 0, ISS = 0x00000061
  [ 1911.668857]   CM = 0, WnR = 1
  [ 1911.668861] swapper pgtable: 4k pages, 48-bit VAs, pgdp=0000000040ff4000
  [ 1911.668864] [ffff80001bc00a8c] pgd=00000000bffff003, pud=00000000bfffe003, pmd=0068000018400705
  [ 1911.668872] Internal error: Oops: 96000061 [#1] PREEMPT SMP
  ...

The llp register present in the channel group registers is not
aligned on 64bit.

Fix unaligned 64bit access using two 32bit accesses

Fixes: 04e0a39fc1 ("dmaengine: dw-edma: Add writeq() and readq() for 64 bits architectures")
Signed-off-by: Herve Codina <herve.codina@bootlin.com>
Link: https://lore.kernel.org/r/20220225120252.309404-1-herve.codina@bootlin.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2022-04-12 16:45:42 +05:30
Christophe JAILLET
1ffc6f359f dmaengine: dw-edma: Fix return value check for dma_set_mask_and_coherent()
The commit in the Fixes: tag has changed the logic of the code and now it
is likely that the probe will return an early success (0), even if not
completely executed.

This should lead to a crash or similar issue later on when the code
accesses to some never allocated resources.

Change the '!err' into a 'err' when checking if
'dma_set_mask_and_coherent()' has failed or not.

While at it, simplify the code and remove the "can't success code" related
to 32 DMA mask.
As stated in [1], 'dma_set_mask_and_coherent(DMA_BIT_MASK(64))' can't fail
if 'dev->dma_mask' is non-NULL. And if it is NULL, it would fail for the
same reason when tried with DMA_BIT_MASK(32).

[1]: https://lkml.org/lkml/2021/6/7/398

Fixes: ecb8c88bd3 ("dmaengine: dw-edma-pcie: switch from 'pci_' to 'dma_' API")
Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
Link: https://lore.kernel.org/r/935fbb40ae930c5fe87482a41dcb73abf2257973.1636492127.git.christophe.jaillet@wanadoo.fr
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2021-11-22 12:14:12 +05:30
Wang Qing
ecb8c88bd3 dmaengine: dw-edma-pcie: switch from 'pci_' to 'dma_' API
The wrappers in include/linux/pci-dma-compat.h should go away.

pci_set_dma_mask()/pci_set_consistent_dma_mask() should be
replaced with dma_set_mask()/dma_set_coherent_mask(),
and use dma_set_mask_and_coherent() for both.

Signed-off-by: Wang Qing <wangqing@vivo.com>
Link: https://lore.kernel.org/r/1633663733-47199-2-git-send-email-wangqing@vivo.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2021-10-26 10:54:23 +05:30
Christophe JAILLET
981703aae3 dmaengine: dw-edma: Remove an unused variable
'head' is unused, remove it.

Signed-off-by: Christophe JAILLET <christophe.jaillet@wanadoo.fr>
Link: https://lore.kernel.org/r/46e071be21fbc5ac5c35d4796a7e4249e94c3a77.1633847306.git.christophe.jaillet@wanadoo.fr
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2021-10-18 12:11:08 +05:30
Gustavo Pimentel
84b0aa2e0d dmaengine: dw-edma: Add pcim_iomap_table return check
Currently, is missing a null check on a pcim_iomap_table() return value
and this can lead to a null pointer dereference if the desired BAR
wasn't mapped previously.
Fix this by adding a null check and returning -ENOMEM.

Addresses-Coverity: ("Dereference null return")
Signed-off-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Link: https://lore.kernel.org/r/bc5e6b8632c84660bb6dae454980e9419992ed14.1613674948.git.gustavo.pimentel@synopsys.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2021-03-16 22:58:54 +05:30
Gustavo Pimentel
b671d098a9 dmaengine: dw-edma: Revert fix scatter-gather address calculation
Reverting the applied patch because it caused a regression on
ARC700 platform (32 bits).

Fixes: 05655541c9 ("dmaengine: dw-edma: Fix scatter-gather address calculation")
Signed-off-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Link: https://lore.kernel.org/r/1778422e389fe40032e216b59b1b992c61ec9887.1613674948.git.gustavo.pimentel@synopsys.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2021-03-16 22:58:54 +05:30
Gustavo Pimentel
cb498d7f3b dmaengine: dw-edma: Change DMA abbreviation from lower into upper case
To keep code consistent, some comments with dma keyword written in lower
case are now in upper case.

Signed-off-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Link: https://lore.kernel.org/r/8c4b3db90767972a2b4cbb6fa818cf0e9c3d6fe3.1613674948.git.gustavo.pimentel@synopsys.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2021-03-16 22:58:53 +05:30
Gustavo Pimentel
e970dcc4bd dmaengine: dw-edma: Fix crash on loading/unloading driver
When the driver is compiled as a module and loaded if we try to unload
it, the Kernel shows a crash log. This Kernel crash is due to the
dma_async_device_unregister() call done after deleting the channels,
this patch fixes this issue.

Signed-off-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Link: https://lore.kernel.org/r/4aa850c035cf7ee488f1d3fb6dee0e37be0dce0a.1613674948.git.gustavo.pimentel@synopsys.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2021-03-16 22:58:53 +05:30
Gustavo Pimentel
5244ac2e2e dmaengine: dw-edma: Move struct dentry variable from static definition into dw_edma struct
Move struct dentry variable from static definition (dw-edma-v0-debugfs.c)
into dw_edma struct (dw-edma-core.h)

Also the variable was renamed from base_dir to debugfs.

Signed-off-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Link: https://lore.kernel.org/r/07c1167b671e7b175700e2e7061cf0b3dd8c6adb.1613674948.git.gustavo.pimentel@synopsys.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2021-03-16 22:58:53 +05:30
Gustavo Pimentel
da6e0dd541 dmaengine: dw-edma: Change linked list and data blocks offset and sizes
Changes the linked list and data blocks offset and sizes to follow the
recommendation given by the hardware team for the IPK solution.

Although the previous data blocks offset and sizes are still valid and
functional, using them that might present some issues related to the IPK
solution, since this solution is based on FPGA and might be subjected to
timmings constrains.

Signed-off-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Link: https://lore.kernel.org/r/f682e7f7f06dc6b2efdd431481d6fb4d762c2c05.1613674948.git.gustavo.pimentel@synopsys.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2021-03-16 22:58:53 +05:30
Gustavo Pimentel
31fb8c1ff9 dmaengine: dw-edma: Improve the linked list and data blocks definition
In the previous implementation, the driver assumed that there existed
only two memory spaces that would equally distribute the amount of
read/write channels.

This might not be the case on some other implementations, therefore this
patch change this requirement so that each write/read channel has
its own linked list and data space well defined, which allows
different sizes and locations.

Signed-off-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Link: https://lore.kernel.org/r/2e316cb983f8a1e09ce929029f87619dc92a52de.1613674948.git.gustavo.pimentel@synopsys.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2021-03-16 22:58:53 +05:30
Gustavo Pimentel
f3167dc163 dmaengine: dw-edma: Reorder variables to keep consistency
In the driver code structure, I tried to keep the code style consistency
by writing the write channels instructions first, and then follow by the
read channels instructions, mimicking the hardware implementation.

However, this code style failed in some cases. This patch fixes that and
no functional changes are expected.

Signed-off-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Link: https://lore.kernel.org/r/9bd1f86f19df8bb5de502fb85a0c5dc07978a9ba.1613674948.git.gustavo.pimentel@synopsys.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2021-03-16 22:58:53 +05:30
Gustavo Pimentel
16b90dd94d dmaengine: dw-edma: Improve number of channels check
It was added some extra checks to ensure that the driver doesn't try to
use more DMA channels than actually are available in hardware.

Signed-off-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Link: https://lore.kernel.org/r/cfb2b0a4f97ae9dc83ebe5ea59d6a51d69ea3654.1613674948.git.gustavo.pimentel@synopsys.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2021-03-16 22:58:53 +05:30
Gustavo Pimentel
85e7518f42 dmaengine: dw-edma: Add device_prep_interleave_dma() support
Add device_prep_interleave_dma() support to Synopsys DMA driver.

This feature implements a similar data transfer mechanism to the
scatter-gather implementation.

Signed-off-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Link: https://lore.kernel.org/r/73dc36264910654e266ae25814d892a0476e4427.1613674948.git.gustavo.pimentel@synopsys.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2021-03-16 22:58:53 +05:30
Gustavo Pimentel
1aef6ffe99 dmaengine: dw-edma: Add PCIe VSEC data retrieval support
The latest eDMA IP development implements a Vendor-Specific Extended
Capability that contains the eDMA BAR, offset, map format, and the
number of read/write channels available.

Signed-off-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Link: https://lore.kernel.org/r/0b880b8893ff457ffc1b5071a1c7f47e61ceea1c.1613674948.git.gustavo.pimentel@synopsys.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
2021-03-16 22:58:53 +05:30