Do DMA mappings get cleared on suspend?

Do DMA mappings get cleared on suspend?

[Paul Pawlowski]

Hello,
Do DMA mappings get cleared when the device is suspended to RAM? A
device I'm writing a driver for requires the DMA addresses not to
change after a resume and trying to use DMA memory allocated before
the suspend causes a device error. Is there a way to persist the
mappings through a suspend?

Thank you,
Paul Pawlowski

Re: Do DMA mappings get cleared on suspend?

[James Bottomley]

On Mon, 2019-08-19 at 21:49 +0200, Paul Pawlowski wrote:
> Hello,
> Do DMA mappings get cleared when the device is suspended to RAM? A
> device I'm writing a driver for requires the DMA addresses not to
> change after a resume and trying to use DMA memory allocated before
> the suspend causes a device error. Is there a way to persist the
> mappings through a suspend?

What are you actually asking?  The state of the IOMMU mappings should
be saved and restored on suspend/resume.  However, whether mappings
that are inside actual PCI devices are saved and restored depends on
the actual device.  In general we don't expect them to remember in-
flight I/O which is why I/O is quiesced before devices are suspended,
so the device should be inactive and any I/O in the upper layers will
be mapped on resume.  The DMA addresses of the mailboxes are usually
saved and restored, but how is up to the driver.

James

Re: Do DMA mappings get cleared on suspend?

[Paul Pawlowski]

Hello,
Thank you for your reply and sorry for the confusion.

I mean the IOMMU mapping state, not anything regarding the device state.

If I create a DMA mapping before the suspend (as in,
dma_alloc_coherent was called before suspend), and then try to pass
the DMA mapping address to the device after the system is resumed, the
device gives me an error back.
However, if the DMA mapping is created after the suspend is completed,
the device accepts it and processes the data.

In my particular case, the device stores state into a DMA buffer for
suspend. I attempted to persist the DMA buffer mapping through the
suspend however this caused a device error when I tried to tell the
device to restore the state on resume.
If I copied the data from the old buffer into a newly allocated one
(using dma_alloc_coherent), and then tried to use that address, the
device successfully resumed.
The buffer is obviously not freed, at least not in my driver code.

I assume this is not expected and I had to end up with a bug somewhere
in my driver code?

Thank you,
Paul Pawlowski

On Tue, Aug 20, 2019 at 12:37 PM James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
>
> On Mon, 2019-08-19 at 21:49 +0200, Paul Pawlowski wrote:
> > Hello,
> > Do DMA mappings get cleared when the device is suspended to RAM? A
> > device I'm writing a driver for requires the DMA addresses not to
> > change after a resume and trying to use DMA memory allocated before
> > the suspend causes a device error. Is there a way to persist the
> > mappings through a suspend?
>
> What are you actually asking?  The state of the IOMMU mappings should
> be saved and restored on suspend/resume.  However, whether mappings
> that are inside actual PCI devices are saved and restored depends on
> the actual device.  In general we don't expect them to remember in-
> flight I/O which is why I/O is quiesced before devices are suspended,
> so the device should be inactive and any I/O in the upper layers will
> be mapped on resume.  The DMA addresses of the mailboxes are usually
> saved and restored, but how is up to the driver.
>
> James
>

Re: Do DMA mappings get cleared on suspend?

[Paul Pawlowski]

Hello,

I have done some more debugging now and it turned out that the bug was
indeed in my driver, and that the dma_alloc_coherent()+memcpy()
somehow magically caused the issue not to happen, however it had
nothing to do with the issue.

Sorry for the inconvenience.

Thank you,
Paul Pawlowski


On Tue, Aug 20, 2019 at 1:58 PM Paul Pawlowski <mrarmdev@gmail.com> wrote:
>
> Hello,
> Thank you for your reply and sorry for the confusion.
>
> I mean the IOMMU mapping state, not anything regarding the device state.
>
> If I create a DMA mapping before the suspend (as in,
> dma_alloc_coherent was called before suspend), and then try to pass
> the DMA mapping address to the device after the system is resumed, the
> device gives me an error back.
> However, if the DMA mapping is created after the suspend is completed,
> the device accepts it and processes the data.
>
> In my particular case, the device stores state into a DMA buffer for
> suspend. I attempted to persist the DMA buffer mapping through the
> suspend however this caused a device error when I tried to tell the
> device to restore the state on resume.
> If I copied the data from the old buffer into a newly allocated one
> (using dma_alloc_coherent), and then tried to use that address, the
> device successfully resumed.
> The buffer is obviously not freed, at least not in my driver code.
>
> I assume this is not expected and I had to end up with a bug somewhere
> in my driver code?
>
> Thank you,
> Paul Pawlowski
>
> On Tue, Aug 20, 2019 at 12:37 PM James Bottomley
> <James.Bottomley@hansenpartnership.com> wrote:
> >
> > On Mon, 2019-08-19 at 21:49 +0200, Paul Pawlowski wrote:
> > > Hello,
> > > Do DMA mappings get cleared when the device is suspended to RAM? A
> > > device I'm writing a driver for requires the DMA addresses not to
> > > change after a resume and trying to use DMA memory allocated before
> > > the suspend causes a device error. Is there a way to persist the
> > > mappings through a suspend?
> >
> > What are you actually asking?  The state of the IOMMU mappings should
> > be saved and restored on suspend/resume.  However, whether mappings
> > that are inside actual PCI devices are saved and restored depends on
> > the actual device.  In general we don't expect them to remember in-
> > flight I/O which is why I/O is quiesced before devices are suspended,
> > so the device should be inactive and any I/O in the upper layers will
> > be mapped on resume.  The DMA addresses of the mailboxes are usually
> > saved and restored, but how is up to the driver.
> >
> > James
> >