Pull dmaengine updates from Vinod Koul:
"A new driver, couple of device support and binding conversion along
with bunch of driver updates are the main features of this.
New hardware support:
- TI AM62Ax controller support
- Xilinx xdma driver
- Qualcomm SM6125, SM8550, QDU1000/QRU1000 GPI controller
Updates:
- Runtime pm support for at_xdmac driver
- IMX sdma binding conversion to yaml and HDMI audio support
- IMX mxs binding conversion to yaml"
* tag 'dmaengine-6.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine: (35 commits)
dmaengine: idma64: Update bytes_transferred field
dmaengine: imx-sdma: Set DMA channel to be private
dmaengine: dw: Move check for paused channel to dwc_get_residue()
dmaengine: ptdma: check for null desc before calling pt_cmd_callback
dmaengine: dw-axi-dmac: Do not dereference NULL structure
dmaengine: idxd: Fix default allowed read buffers value in group
dmaengine: sf-pdma: pdma_desc memory leak fix
dmaengine: Simplify dmaenginem_async_device_register() function
dmaengine: use sysfs_emit() to instead of scnprintf()
dmaengine: Make an order in struct dma_device definition
dt-bindings: dma: cleanup examples - indentation, lowercase hex
dt-bindings: dma: drop unneeded quotes
dmaengine: xilinx: xdma: Add user logic interrupt support
dmaengine: xilinx: xdma: Add xilinx xdma driver
dmaengine: drivers: Use devm_platform_ioremap_resource()
dmaengine: at_xdmac: remove empty line
dmaengine: at_xdmac: add runtime pm support
dmaengine: at_xdmac: align properly function members
dmaengine: ppc4xx: Convert to use sysfs_emit()/sysfs_emit_at() APIs
dmaengine: sun6i: Set the maximum segment size
...
Add driver to enable PCIe board which uses XDMA (the DMA/Bridge Subsystem
for PCI Express). For example, Xilinx Alveo PCIe devices.
https://www.xilinx.com/products/boards-and-kits/alveo.html
The XDMA engine support up to 4 Host to Card (H2C) and 4 Card to Host (C2H)
channels. Memory transfers are specified on a per-channel basis in
descriptor linked lists, which the DMA fetches from host memory and
processes. Events such as descriptor completion and errors are signaled
using interrupts. The hardware detail is provided by
https://docs.xilinx.com/r/en-US/pg195-pcie-dma/Introduction
This driver implements dmaengine APIs.
- probe the available DMA channels
- use dma_slave_map for channel lookup
- use virtual channel to manage dmaengine tx descriptors
- implement device_prep_slave_sg callback to handle host scatter gather
list
- implement device_config to config device address for DMA transfer
Signed-off-by: Lizhi Hou <lizhi.hou@amd.com>
Signed-off-by: Sonal Santan <sonal.santan@amd.com>
Signed-off-by: Max Zhen <max.zhen@amd.com>
Signed-off-by: Brian Xu <brian.xu@amd.com>
Tested-by: Martin Tuma <tumic@gpxsee.org>
Link: https://lore.kernel.org/r/1674145926-29449-2-git-send-email-lizhi.hou@amd.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
dmaengine_synchronize implementation is required to synchronize proper
termination of current transfers so that any memory resources are not freed
while still in use.
Implement this callback in the driver so that framework can use the same
(in dmaengine_terminate_sync/ dmaengine_synchronize).
Signed-off-by: Swati Agarwal <swati.agarwal@amd.com>
Link: https://lore.kernel.org/r/20220915090516.5812-1-swati.agarwal@amd.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
This was removed before due to the complete lack of users, but
3218910fd5 ("dmaengine: Add core function and capability check for
DMA_MEMCPY_SG") and 29cf37fa6d ("dmaengine: Add consumer for the new
DMA_MEMCPY_SG API function.") added it back despite still not having
any users whatsoever.
Fixes: 3218910fd5 ("dmaengine: Add core function and capability check for DMA_MEMCPY_SG")
Fixes: 29cf37fa6d ("dmaengine: Add consumer for the new DMA_MEMCPY_SG API function.")
Signed-off-by: Christoph Hellwig <hch@lst.de>
Acked-by: Michal Simek <michal.simek@amd.com>
Link: https://lore.kernel.org/r/20220606074733.622616-1-hch@lst.de
Signed-off-by: Vinod Koul <vkoul@kernel.org>
In zynqmp_dma_alloc/free_chan_resources functions there is a
potential overflow in the below expressions.
dma_alloc_coherent(chan->dev, (2 * chan->desc_size *
ZYNQMP_DMA_NUM_DESCS),
&chan->desc_pool_p, GFP_KERNEL);
dma_free_coherent(chan->dev,(2 * ZYNQMP_DMA_DESC_SIZE(chan) *
ZYNQMP_DMA_NUM_DESCS),
chan->desc_pool_v, chan->desc_pool_p);
The arguments desc_size and ZYNQMP_DMA_NUM_DESCS were 32 bit. Though
this overflow condition is not observed but it is a potential problem
in the case of 32-bit multiplication. Hence fix it by changing the
desc_size data type to size_t.
In addition to coverity fix it also reuse ZYNQMP_DMA_DESC_SIZE macro in
dma_alloc_coherent API argument.
Addresses-Coverity: Event overflow_before_widen.
Signed-off-by: Radhey Shyam Pandey <radhey.shyam.pandey@xilinx.com>
Link: https://lore.kernel.org/r/1652166762-18317-2-git-send-email-radhey.shyam.pandey@xilinx.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Handle errors when trying to map the IRQ for the DMA channels.
The main motivation here is to be able to handle probe deferral. E.g. when
using DT overlays it is possible that the DMA controller is probed before
interrupt controller, depending on the order in the DT.
In order to support this switch from irq_of_parse_and_map() to
of_irq_get(), which internally does the same, but it will return
EPROBE_DEFER when the interrupt controller is not yet available.
As a result other errors, such as an invalid IRQ specification, or missing
IRQ are also properly handled.
Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Reviewed-by: Radhey Shyam Pandey <radhey.shyam.pandey@xilinx.com>
Link: https://lore.kernel.org/r/20211208114212.234130-1-lars@metafoo.de
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The display driver wants to pass a custom flag to the DMA engine driver,
which it started doing by using the slave_id field that was traditionally
used for a different purpose.
As there is no longer a correct use for the slave_id field, it should
really be removed, and the remaining users changed over to something
different.
The new mechanism for passing nonstandard settings is using the
.peripheral_config field, so use that to pass a newly defined structure
here, making it clear that this will not work in portable drivers.
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Mark Brown <broonie@kernel.org>
Link: https://lore.kernel.org/r/20211122222203.4103644-10-arnd@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
DMA clients can provide one of two types of callbacks. For this reason
dmaengine drivers should not directly invoke `callback`, but always use
`dmaengine_desc_callback_invoke()`. This makes sure that both types of
callbacks are handled correctly.
The zynqmp_dma driver currently doesn't do this and only handles the
`callback` type callback. If the client used the `callback_result` type
callback it will not be called.
Fix this by switching to `dmaengine_desc_callback_valid()` and
`dmaengine_desc_callback_invoke()`.
Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Link: https://lore.kernel.org/r/20211025075428.2094-3-lars@metafoo.de
Signed-off-by: Vinod Koul <vkoul@kernel.org>
DMA clients can provide one of two types of callbacks. For this reason
dmaengine drivers should not directly invoke `callback`, but always use
`dmaengine_desc_callback_invoke()`. This makes sure that both types of
callbacks are handled correctly.
The xilinx_dma driver currently doesn't do this for cyclic descriptors and
only handles the `callback` type callback. If the client used the
`callback_result` type callback it will not be called.
Fix this by switching to `dmaengine_desc_callback_valid()` and
`dmaengine_desc_callback_invoke()`.
Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Link: https://lore.kernel.org/r/20211025075428.2094-2-lars@metafoo.de
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Modify the prototype from xilinx_dma_tx_descriptor to
xilinx_dma_alloc_tx_descriptor and xilinx_dma_channel_set_config
to xilinx_vdma_channel_set_config in API description to
fix below linux kernel-doc warnings.
drivers/dma/xilinx/xilinx_dma.c:800: warning: expecting
prototype for xilinx_dma_tx_descriptor(). Prototype was
for xilinx_dma_alloc_tx_descriptor() instead.
drivers/dma/xilinx/xilinx_dma.c:2471: warning: expecting
prototype for xilinx_dma_channel_set_config(). Prototype
was for xilinx_vdma_channel_set_config() instead.
Signed-off-by: Shravya Kumbham <shravya.kumbham@xilinx.com>
Signed-off-by: Radhey Shyam Pandey <radhey.shyam.pandey@xilinx.com>
Link: https://lore.kernel.org/r/1631525316-2323-1-git-send-email-radhey.shyam.pandey@xilinx.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The ptdma driver has added debugfs support, but this fails to build
when debugfs is disabled:
drivers/dma/ptdma/ptdma-debugfs.c: In function 'ptdma_debugfs_setup':
drivers/dma/ptdma/ptdma-debugfs.c:93:54: error: 'struct dma_device' has no member named 'dbg_dev_root'
93 | debugfs_create_file("info", 0400, pt->dma_dev.dbg_dev_root, pt,
| ^
drivers/dma/ptdma/ptdma-debugfs.c:96:55: error: 'struct dma_device' has no member named 'dbg_dev_root'
96 | debugfs_create_file("stats", 0400, pt->dma_dev.dbg_dev_root, pt,
| ^
drivers/dma/ptdma/ptdma-debugfs.c:102:52: error: 'struct dma_device' has no member named 'dbg_dev_root'
102 | debugfs_create_dir("q", pt->dma_dev.dbg_dev_root);
| ^
Remove the #ifdef in the header, as this only saves a few bytes,
but would require ugly #ifdefs in each driver using it.
Simplify the other user while we're at it.
Fixes: e2fb2e2a33 ("dmaengine: ptdma: Add debugfs entries for PTDMA")
Fixes: 26cf132de6 ("dmaengine: Create debug directories for DMA devices")
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Link: https://lore.kernel.org/r/20210920122017.205975-1-arnd@kernel.org
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The tasklet that handles the completed dma transfers uses spin_unlock
for unlocking a spin lock that was previously locked with
spin_lock_irqsave.
This caused the following lockdep warning about an inconsistent lock
state:
inconsistent {HARDIRQ-ON-W} -> {IN-HARDIRQ-W} usage.
We must use spin_lock_irqsave, because it is possible to queue DMA
transfers from an irq handler.
Replace the spin_unlock and spin_lock by spin_unlock_irqrestore and
spin_lock_irqsave.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Link: https://lore.kernel.org/r/20210826094742.1302009-8-m.tretter@pengutronix.de
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The descriptor lists are locked for the entire tasklet that completes
the descriptors. This is not necessary, because the lock actually only
protects the descriptor lists.
Make the spin lock more fine-grained and only protect functions that
actually operate on the descriptor lists. This decreases the time when
the lock is held.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Link: https://lore.kernel.org/r/20210826094742.1302009-7-m.tretter@pengutronix.de
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The current implementation iterates the entire done list for each
completed dma descriptor even if there are multiple completed
descriptors.
Avoid this by first moving all completed descriptors to the done list
and afterwards iterating the done list and finishing the descriptors.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Link: https://lore.kernel.org/r/20210826094742.1302009-6-m.tretter@pengutronix.de
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The clocks are provided by the ZynqMP firmware driver and are deferred
until the firmware driver has probed. This leads to misleading error
messages during probe of the zynqmp_dma driver.
Use dev_err_probe for printing errors during probe to avoid error
messages for -EPROBE_DEFER.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Link: https://lore.kernel.org/r/20210826094742.1302009-2-m.tretter@pengutronix.de
Signed-off-by: Vinod Koul <vkoul@kernel.org>
The xilinx dma driver uses the consistent allocations, so for correct
operation also set the DMA mask for coherent APIs. It fixes the below
kernel crash with dmatest client when DMA IP is configured with 64-bit
address width and linux is booted from high (>4GB) memory.
Call trace:
[ 489.531257] dma_alloc_from_pool+0x8c/0x1c0
[ 489.535431] dma_direct_alloc+0x284/0x330
[ 489.539432] dma_alloc_attrs+0x80/0xf0
[ 489.543174] dma_pool_alloc+0x160/0x2c0
[ 489.547003] xilinx_cdma_prep_memcpy+0xa4/0x180
[ 489.551524] dmatest_func+0x3cc/0x114c
[ 489.555266] kthread+0x124/0x130
[ 489.558486] ret_from_fork+0x10/0x3c
[ 489.562051] ---[ end trace 248625b2d596a90a ]---
Signed-off-by: Radhey Shyam Pandey <radhey.shyam.pandey@xilinx.com>
Reviewed-by: Harini Katakam <harini.katakam@xilinx.com>
Link: https://lore.kernel.org/r/1629363528-30347-1-git-send-email-radhey.shyam.pandey@xilinx.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
When user calls dmaengine_terminate_sync, the driver will clean up any
remaining descriptors for all the pending or active transfers that had
previously been submitted. However, this might happen whilst the tasklet is
invoking the DMA callback for the last finished transfer, so by the time it
returns and takes over the channel's spinlock, the list of completed
descriptors it was traversing is no longer valid. This leads to a
read-after-free situation.
Fix it by signalling whether a user-triggered termination has happened by
means of a boolean variable.
Signed-off-by: Adrian Larumbe <adrian.martinezlarumbe@imgtec.com>
Link: https://lore.kernel.org/r/20210706234338.7696-3-adrian.martinezlarumbe@imgtec.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Pull dmaengine updates from Vinod Koul:
"This time around we have a smaller pull request than usual and this
includes code removal, so should be good!
New drivers/devices
- Support for QCOM SM8250 GPI DMA
- removal of shdma-of driver and binding
Updates:
- arm-pl08x yaml binding move
- altera-msgdma gained DT support
- removal of imx-sdma platform data support
- idxd and xilinx driver updates"
* tag 'dmaengine-5.14-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine: (22 commits)
dmaengine: imx-sdma: Remove platform data header
dmaengine: xilinx: dpdma: Fix spacing around addr[i-1]
dmaengine: xilinx: dpdma: Use kernel type u32 over uint32_t
dmaengine: altera-msgdma: add OF support
MAINTAINERS: add entry for Altera mSGDMA
dt-bindings: dma: add schema for altera-msgdma
dmaengine: xilinx: dpdma: fix kernel-doc
dmaengine: sf-pdma: apply proper spinlock flags in sf_pdma_prep_dma_memcpy()
dmaengine: sh: Remove unused shdma-of driver
dt-bindings: dmaengine: Remove SHDMA Device Tree bindings
dmaengine: qcom: gpi: Add SM8250 compatible
dt-bindings: dmaengine: qcom: gpi: add compatible for sm8250
dmaengine: sun4i: Use list_move_tail instead of list_del/list_add_tail
dmaengine: ti: omap-dma: Skip pointless cpu_pm context restore on errors
dmaengine: hsu: Account transferred bytes
dmaengine: Move kdoc description of struct dma_chan_percpu closer to it
dmaengine: xilinx: dpdma: Print debug message when losing vsync race
dmaengine: xilinx: dpdma: Print channel number in kernel log messages
dt-bindings: dma: convert arm-pl08x to yaml
dmaengine: idxd: remove devm allocation for idxd->int_handles
...
The Xilinx dmaengine driver uses a tasklet to process completed
descriptors and execute their callbacks.
Currently consumers of the DMA channel have to no method of synchronization
against this tasklet when using the Xilinx dmaengine drivers. This can lead
to race conditions when the consumer frees resources that are accessed in
the callback before the tasklet has finished running.
It is not enough to just call dmaengine_terminal_all() since on a
multi-processor system the tasklet can run concurrently to it and might
call the callback after dmaengine_terminate_all() has already finished.
To mitigate this issue implement the synchronize() callback for the driver,
which will wait until the tasklet has finished.
Signed-off-by: Lars-Peter Clausen <lars@metafoo.de>
Link: https://lore.kernel.org/r/20210313125311.4823-1-lars@metafoo.de
Signed-off-by: Vinod Koul <vkoul@kernel.org>
