Re: [RFC PATCH] mm: Introduce new MADV_NOMOVABLE behavior

[David Hildenbrand <david@redhat.com>]

On 17.10.22 09:32, Baolin Wang wrote:
> When creating a virtual machine, we will use memfd_create() to get
> a file descriptor which can be used to create share memory mappings
> using the mmap function, meanwhile the mmap() will set the MAP_POPULATE
> flag to allocate physical pages for the virtual machine.
> 
> When allocating physical pages for the guest, the host can fallback to
> allocate some CMA pages for the guest when over half of the zone's free
> memory is in the CMA area.
> 
> In guest os, when the application wants to do some data transaction with
> DMA, our QEMU will call VFIO_IOMMU_MAP_DMA ioctl to do longterm-pin and
> create IOMMU mappings for the DMA pages. However, when calling
> VFIO_IOMMU_MAP_DMA ioctl to pin the physical pages, we found it will be
> failed to longterm-pin sometimes.
> 
> After some invetigation, we found the pages used to do DMA mapping can
> contain some CMA pages, and these CMA pages will cause a possible
> failure of the longterm-pin, due to failed to migrate the CMA pages.
> The reason of migration failure may be temporary reference count or
> memory allocation failure. So that will cause the VFIO_IOMMU_MAP_DMA
> ioctl returns error, which makes the application failed to start.
> 
> To fix this issue, this patch introduces a new madvise behavior, named
> as MADV_NOMOVABLE, to avoid allocating CMA pages and movable pages if
> the users want to do longterm-pin, which can remove the possible failure
> of movable or CMA pages migration.

Sorry to say, but that sounds like a hack to work around a kernel 
implementation detail (how often we retry to migrate pages).

If there are CMA/ZONE_MOVABLE issue, please fix them instead, and avoid 
leaking these details to user space.

ALSO, with MAP_POPULATE as described by you this madvise flag doesn't 
make too much sense, because it will gets et after all memory already 
was allocated ...

NAK

-- 
Thanks,

David / dhildenb