Re: Why memory lending is needed for GPU acceleration

["Teddy Astie" <teddy.astie@vates.tech>]

Le 30/03/2026 à 22:13, Val Packett a écrit :
> Hi,
>
> On 3/29/26 2:32 PM, Demi Marie Obenour wrote:
>> On 3/24/26 10:17, Demi Marie Obenour wrote:
>>> Here is a proposed design document for supporting mapping GPU VRAM
>>> and/or file-backed memory into other domains.  It's not in the form of
>>> a patch because the leading + characters would just make it harder to
>>> read for no particular gain, and because this is still RFC right now.
>>> Once it is ready to merge, I'll send a proper patch.  Nevertheless,
>>> you can consider this to be
>>>
>>> Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com>
>>>
>>> This approach is very different from the "frontend-allocates"
>>> approach used elsewhere in Xen.  It is very much Linux-centric,
>>> rather than Xen-centric.  In fact, MMU notifiers were invented for
>>> KVM, and this approach is exactly the same as the one KVM implements.
>>> However, to the best of my understanding, the design described here is
>>> the only viable one.  Linux MM and GPU drivers require it, and changes
>>> to either to relax this requirement will not be accepted upstream.
>> Teddy Astie (CCd) proposed a couple of alternatives on Matrix:
>>
>> 1. Create dma-bufs for guest pages and import them into the host.
>>
>>     This is a win not only for Xen, but also for KVM.  Right now, shared
>>     (CPU) memory buffers must be copied from the guest to the host,
>>     which is pointless.  So fixing that is a good thing!  That said,
>>     I'm still concerned about triggering GPU driver code-paths that
>>     are not tested on bare metal.
>
> To expand on this: the reason cross-domain Wayland proxies have been
> doing this SHM copy dance was a deficiency in Linux UAPI. Basically,
> applications allocate shared memory using local mechanisms like memfd
> (and good old unlink-of-regular-file, ugh) which weren't compatible with
> cross-VM sharing. However udmabuf should basically solve it, at least
> for memfds. (I haven't yet investigated what happens with "unlinked
> regular files" yet but I don't expect anything good there, welp.)
>
> But I have landed a patch in Linux that removes a silly restriction that
> tied dmabuf import into virtgpu to KMS-only mode:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/
> commit/?id=df4dc947c46bb9f80038f52c6e38cb2d40c10e50
>
> And I have experimented with it and got a KVM-based VMM to successfully
> access and print guest memfd contents that were passed to the host via
> this mechanism. (Time to actually properly implement it into the full
> system..)
>
>> 2. Use PASID and 2-stage translation so that the GPU can operate in
>>     guest physical memory.
>>     This is also a win.  AMD XDNA absolutely requires PASID support,
>>     and apparently AMD GPUs can also use PASID.  So being able to use
>>     PASID is certainly helpful.
>>
>> However, I don't think either approach is sufficient for two reasons.
>>
>> First, discrete GPUs have dedicated VRAM, which Xen knows nothing about.
>> Only dom0's GPU drivers can manage VRAM, and they will insist on being
>> able to migrate it between the CPU and the GPU.  Furthermore, VRAM
>> can only be allocated using GPU driver ioctls, which will allocate
>> it from dom0-owned memory.
>>
>> Second, Certain Wayland protocols, such as screencapture, require
>> programs
>> to be able to import dmabufs.  Both of the above solutions would
>> require that the pages be pinned.  I don't think this is an option,
>> as IIUC pin_user_pages() fails on mappings of these dmabufs.  It's why
>> direct I/O to dmabufs doesn't work.
>>
>> To the best of my knowledge, these problems mean that lending memory
>> is the only way to get robust GPU acceleration for both graphics and
>> compute workloads under Xen.  Simpler approaches might work for pure
>> compute workloads, for iGPUs, or for drivers that have Xen-specific
>> changes.  None of them, however, support graphics workloads on dGPUs
>> while using the GPU driver the same way bare metal workloads do.
>> […]
> To recap, how virtio-gpu Host3d memory currently works with KVM is:
>
> - the VMM/virtgpu receives a dmabuf over a socket (Wayland/D-Bus/
> whatever) and registers it internally with some resource ID that's
> passed to the guest;
> - When the guest imports that resource, it calls
> VIRTIO_GPU_CMD_RESOURCE_MAP_BLOB to get a PRIME buffer that can be
> turned into a dmabuf fd;
> - the VMM's handler for VIRTIO_GPU_CMD_RESOURCE_MAP_BLOB (referencing
> libkrun here) literally just calls mmap() on the host dmabuf, using the
> MAP_FIXED flag to place it correctly inside of the VMM process's guest-
> exposed VA region (configured via KVM_SET_USER_MEMORY_REGION);
> - so any resource imported by the guest, even before guest userspace
> does mmap(), is mapped (as VM_PFNMAP|VM_IO) until the guest releases it.
>
> So the generic kernel MM is out of the way, these mappings can't be
> paged out to swap etc. But accessing them may fault, as the comment for
> drm_gem_mmap_obj says:
>
>   * Depending on their requirements, GEM objects can either
>   * provide a fault handler in their vm_ops (in which case any accesses to
>   * the object will be trapped, to perform migration, GTT binding, surface
>   * register allocation, or performance monitoring), or mmap the buffer
> memory
>   * synchronously after calling drm_gem_mmap_obj
>
> It all "just works" in KVM because KVM's resolution of the guest's
> memory accesses tries to be literally equivalent to what's mapped into
> the userspace VMM process: hva_to_pfn_remapped explicitly calls
> fixup_user_fault and eventually gets to the GPU driver's fault handler.
>
> Now for Xen this would be… painful,
>

indeed

> but,
>
> we have no need to replicate what KVM does. That's far from the only
> thing that can be done with a dmabuf.
>
> The import-export machinery on the other hand actually does pin the
> buffers on the driver level, importers are not obligated to support
> movable buffers (move_notify in dma_buf_attach_ops is entirely optional).
>

dma-buf is by concept non-movable if actively used (otherwise, it would
break DMA). It's just a foreign buffer, and from device standpoint, just
plain RAM that needs to be mapped.

> Interestingly, there is already XEN_GNTDEV_DMABUF…
>
> Wait, do we even have any reason at all to suspect
> that XEN_GNTDEV_DMABUF doesn't already satisfy all of our buffer-sharing
> requirements?
>

XEN_GNTDEV_DMABUF has been designed for GPU use-cases, and more
precisely for paravirtualizing a display. The only issue I would have
with it is that grants are not scalable for GPU 3D use cases (with
hundreds of MB to share).
But we can still keep the concept of a structured guest-owned memory
that is shared with Dom0 (but for larger quantities), I have some ideas
regarding improving that area in Xen.

The only issue with changing the memory sharing model is that you would
need to adjust the virtio-gpu aspect, but the rest can stay the same.

The biggest concern regarding driver compatibility is more about :
- can dma-buf be used as general buffers : probably yes (even with
OpenGL/Vulkan); exception may be proprietary Nvidia drivers that lacks
the feature; maybe very old hardware may struggle more with it
- can guest UMD work without access to vram : yes (apparently), AMDGPU
has a special case where VRAM is not visible (e.g too small PCI BAR),
there is vram size vs "vram visible size" (which could be 0); you could
fallback vram-guest-visible with ram mapped on device
- can it be defined in Vulkan terms (from driver) : You can have
device_local memory without having it host-visible (i.e memory exists,
but can't be added in the guest). You would probably just lose some
zero-copy paths with VRAM. Though you still have RAM shared with GPU
(GTT in AMDGPU) if that matters.

Worth noting that if you're on integration graphics, you don't have VRAM
and everything is RAM anyway.

>
> Thanks,
> ~val
>
> P.S. while I have everyone's attention, can I get some eyes on:
> https://lore.kernel.org/all/20251126062124.117425-1-
> val@invisiblethingslab.com/ ?
>
>



--
Teddy Astie | Vates XCP-ng Developer

XCP-ng & Xen Orchestra - Vates solutions

web: https://vates.tech