Re: [LSF/MM/BPF TOPIC] The future of ZONE_DEVICE pages

[Jason Gunthorpe <jgg@nvidia.com>]

On Fri, Jan 31, 2025 at 01:59:09PM +1100, Alistair Popple wrote:
> Combining P2PDMA and DEVICE_PRIVATE pages
> =========================================
> 
> Currently device memory that cannot be directly accessed via the CPU can be
> represented by DEVICE_PRIVATE pages allowing it to be mapped and treated like
> a normal virtual page by userpsace. Many devices also support accessing device
> memory directly from the CPU via a PCIe BAR.
> 
> This access requires a P2PDMA page, meaning there are potentially two pages
> tracking the same piece of physical memory. This not only seems wasteful but
> fraught - for example device drivers need to keep page lifetimes in sync. I
> would like to discuss ways of solving this.

My general plan for this has been to teach the DMA API how to do P2P
without struct page. Leon's topic is the frist step on this journey.
  https://lore.kernel.org/linux-mm/97f385db-42c9-4c04-8fba-9b1ba8ffc525@nvidia.com/

When we can DMA map P2P memory without struct page then we can talk
about what API changes would be needed to take advantage of that.

However, merging P2P and DEVICE_PRIVATE seems very tricky to me, you'd
have to make the type of page dependent on how it was read out of the
PTE - a swap entry is PRIVATE, a normal PTE is P2P.

Further, a struct page is necessary if any P2P pages are being placed
into VMAs..

Jason