Re: [LSF/MM/BPF TOPIC][RFC PATCH v4 00/27] Private Memory Nodes (w/ Compressed RAM)

["David Hildenbrand (Arm)" <david@kernel.org>]

On 4/13/26 19:05, Gregory Price wrote:
> On Mon, Apr 13, 2026 at 03:11:12PM +0200, David Hildenbrand (Arm) wrote:
>>> Normally cloud-hypervisor VMs with virtio-net can't be subject to KSM
>>> because the entire boot region gets marked shared.  
>>
>> What exactly do you mean with "mark shared". Do you mean, that "shared
>> memory" is used in the hypervisor for all boot memory?
>>
> 
> Sorry, meant MAP_SHARED.  But yes, in some setups the hypervisor simply
> makes a memfd with the entire main memory region MAP_SHARED.
> 
> This is because the virtio-net device / network stack does GFP_KERNEL
> allocations and then pins them on the host to allow zero-copy - so all
> of ZONE_NORMAL is a valid target.
> 
> (At least that's my best understanding of the entire setup).

I think with vhost-kernel virtio-net just supports MAP_PRIVATE, KSM and
all of that.

The problem is vhost-user, where the other process needs to access all
of VM's memory. That's not only a problem for virtio-net, but also
virtio-fs and all the other stuff that uses vhost-user.

One idea discussed in the past was to let vhost-user access selected
guest memory through QEMU, so there would be no need to even map all of
guest memory into the other processes.

That in turn would stop requiring MAP_SHARED for most guest RAM, only
focusing it on some key parts. Not sure what happened with that idea.

A related series proposed some  MEM_READ/WRITE backend requests [1]

[1] https://lists.nongnu.org/archive/html/qemu-devel/2024-09/msg02693.html


Something else people were discussing in the past was to physically
limit the area where virtio queues could be placed.

> 
>>
>> You mean, in the VM, memory usable by virtio-net can only be consumed
>> from a dedicated physical memory region, and that region would be a
>> separate node?
>>
> 
> Correct - it does requires teaching the network stack numa awareness.
> 
> I was surprised by how little code this required, though I can't be
> 100% sure of its correctness since networking isn't my normal space.

One problem might be that VMs with NUMA disabled or reconfigured would
just break. So you cannot run arbitrary guests in there. That was also
one of the problems of "physically limit the area where virtio queues
could be placed", if you have to be prepared to run arbitrary OSes in
your VM (Windows says hi).

> 
> Alternatively you could imagine this as a real device bringing its own
> dedicated networking memory for network buffers, and then telling the
> network start "Hey, prefer this node over normal kernel allocations".
> 
> What I'd been hacking on was cobbled together with memfd + SRAT bits to
> bring up a private node statically and then have the device claim it -
> but this is just a proof of concept.  A proper implementation would be
> extending virtio-net to report a dedicated EFI_RESERVED region.
> 
>>>
>>> I see you saw below that one of the extensions is removing the nodes
>>> from the fallback list.  That is part one, but it's insufficient to
>>> prevent complete leakage (someone might iterate over the nodes-possible
>>> list and try migrating memory).
>>
>> Which code would do that?
>>
> 
> There are many callers of for_each_node() throughout the system.
> 
> but one discrete example:
> 
> int alloc_shrinker_info(struct mem_cgroup *memcg)
> {
> ... snip ...
>   for_each_node(nid) {
>     struct shrinker_info *info = kvzalloc_node(sizeof(*info) + array_size,
>                                                GFP_KERNEL, nid);
> ... snip ..
> }
> 
> If you disallow fallbacks in this scenario, this allocation always fails.
> 
> This partially answers your question about slub fallback allocations,
> there are slab allocations like this that depend on fallbacks (more
> below on this explicitly).

But that's a different "fallback" problem, no?

You want allocations that target the "special node" to fallback to
*other* nodes, but not other allocations to fallback to *this special* node.

> 
>>> Basically the only isolation mechanism we have today is ZONE_DEVICE.
>>>
>>> Either via mbind and friends, or even just the driver itself managing it
>>> directly via alloc_pages_node() and exposing some userland interface.
>>
>> Would mbind() work here? I thought mbind() would not suddenly give
>> access to some ZONE_DEVICE memory.
>>
> 
> Sorry these were orthogonal thoughts.
> 
> 1) We don't have such a mechanism. ZONE_DEVICE's preferred mechanism is
>    setting up explicit migrations via migrate_device.c

Makes sense.

> 
> 2) mbind / alloc_pages_node would only work for private nodes.
> 
>    Extending ZONE_DEVICE to enable mbind() would be an extreme lift,
>    as the kernel makes a lot of assumptions about folio->lru.
> 
>    This is why i went the node route in the first place.


Agreed.

> 
>>>
>>> in the NP_OPS_MIGRATION patch, this gets covered.
>>
>> Right, but I am not sure if NP_OPS_MIGRATION is really the right
>> approach for that. Have to think about that.
>>
> 
> So, OPS is a bit misleading, but it's the closest i came to some
> existing pattern.  OPS does not necessarily need to imply callbacks.
> 
> I've been trying to minimize the patch set and I'm starting to think
> the MVP may actually be able to do away with the private_ops structure
> for a basic migration+mempolicy example by simply teaching some services
> (migrate.c, mempolicy.c) how/when to inject __GFP_PRIVATE.
> 
> the mempolicy.c patch already does this, but not migrate.c - i haven't
> figured out the right pattern for that yet.

I assume you will be as LSF/MM? Would be good to discuss some of that in
person.

> 
>>> 1) as you note, removing it from the default bitmaps, which is actually
>>>    hard.  You can't remove it from the possible-node bitmap, so that
>>>    just seemed non-tractable.
>>
>> What about making people use a different set of bitmaps here? Quite some
>> work, but maybe that's the right direction given that we'll now treat
>> some nodes differently.
>>
> 
> It's an option, although it is fragile.  That means having to police all
> future users of possible-nodes and for_each_node and etc.
> 
> I've been err'ing on the side of "not fragile", but i'm open to rework.
> 
>>>
>>> 2) __GFP_THISNODE actually means (among other things) "don't fallback".
>>>    And, in fact, there are some hotplug-time allocations that occur in
>>>    SLAB (pglist_data) that target the private node that *must* fallback
>>>    to successfully allocate for successful kernel operation.
>>
>>
>> Can you point me at the code?
>>
> 
> There is actually a comment in slub.c that addresses this directly:
> 
> static int slab_mem_going_online_callback(int nid)
> {
> ... snip ...
> 	/*
> 	 * XXX: kmem_cache_alloc_node will fallback to other nodes
> 	 *      since memory is not yet available from the node that
> 	 *      is brought up.
> 	 */
> 	n = kmem_cache_alloc(kmem_cache_node, GFP_KERNEL);
> ... snip ...
> }
> 
> Slab basically acknowledges the behavior is required on existing nodes
> and just falls back immediately for the "going online" path.
> 
> Other specific calls in the hotplug path:
> 
>   mm/sparse.c:           kzalloc_node(size, GFP_KERNEL, nid)
>   mm/sparse-vmemmap.c:   alloc_pages_node(nid, GFP_KERNEL|...)
>   mm/slub.c:             kmalloc_node(sizeof(*barn), GFP_KERNEL, nid)
> 
> There are quite a number of callers to kmem_cache_alloc_node() that
> would have to be individually audited.
> 
> And some non-slab interfaces examples as well:
> 	alloc_shrinker_info
> 	alloc_node_nr_active
> 
> I've been looking at this for a while, but I'm starting to think trying
> to touch all this surface area is simply too fragile compared to just
> letting normal memory be a fallback for private nodes and adding:
> 
>       __GFP_PRIVATE   - unlock's private node, but allow fallback
> #define GFP_PRIVATE   (__GFP_PRIVATE | __GFP_THISNODE) - only this node
> 
> __GFP_PRIVATE vs GFP_PRIVATE then is just a matter of use case.
> 
> For mbind() it probably makes sense we'd use GFP_PRIVATE - either it
> succeeds or it OOMs.

Needs a second thought regarding fallback logic I raised above.

What I think would have to be audited is the usage of __GFP_THISNODE by
kernel allocations, where we would not actually want to allocate from
this private node.

Maybe we could just outright refuse *any* non-user (movable) allocations
that target the node, even with __GFP_THISNODE.

Because, why would we want kernel allocations to even end up on a
private node that is supposed to only be consumed by user space? Or
which use cases are there where we would want to place kernel
allocations on there?

Needs a second thought, hoping we can discuss that in person.

> 
>>> The flexibility is kind of the point :]
>>
>> Yeah, but it would be interesting which minimal support we would need to
>> just let some special memory be managed by the kernel, allowing mbind()
>> users to use it, but not have any other fallback allocations end up on it.
>>
>> Something very basic, on which we could build additional functionality.
>>
> 
> I actually have a simplistic CXL driver that does exactly this:
> https://github.com/gourryinverse/linux/blob/072ecf7cbebd9871e76c0b52fd99aa1321405a59/drivers/cxl/type3_drivers/cxl_mempolicy/mempolicy.c#L65
> 

Great.

> We have to support migration because mbind can migrate on bind if the
> VMA already has memory - but all this means is the migrate interfaces
> are live - not that the kernel actually uses them.
> 
> so mbind requires (OPS_MIGRATE | OPS_MEMPOLICY)
> 
> All these flags say is:
>    - move_pages() syscalls can accept these nodes
>    - migrate_pages() function calls can accept these nodes
>    - mempolicy.c nodemasks allow the nodes (should restrict to mbind)
>    - vma's with these nodes now inject __GFP_PRIVATE on fault
> 
> All other services (reclaim, compaction, khugepaged, etc) do not scan
> these nodes and do not know about __GFP_PRIVATE, so they never see
> private node folios and can't allocate from the node.
> 
> In this example, all migrate_to() really does is inject __GFP_THISNODE,
> but I've been thinking about whether we can just do this in migrate.c
> and leave implementing the .ops to a user that requires is.
> 
> But otherwise "it just works".
> 
> One note here though - OOM conditions and allocation failures are not
> intuitive, especially when THP/non-order-0 allocations are involved.
> 
> But that might just mean this minimal setup should only allow order-0
> allocations - which is fiiiiiiiiiiiiiine :P.


Again, I am not sure about compaction and khugepaged. All we want to
guarantee is that our memory does not leave the private node.

That doesn't require any __GFP_PRIVATE magic, just en-lighting these
subsuystems that private nodes must use __GFP_THISNODE and must not leak
to other nodes.

> 
> -----------------
> 
> For basic examples
> 
> I've implemented 4 examples to consider building on:
> 
>   1) CXL mempolicy driver:
>      https://github.com/gourryinverse/linux/blob/072ecf7cbebd9871e76c0b52fd99aa1321405a59/drivers/cxl/type3_drivers/cxl_mempolicy/mempolicy.c#L65
> 
>      As described above
> 
>   2) Virtio-net / CXL.mem Network Card
>      (Not published yet)
> 
>      This doesn't require any ops at all - the plumbing happens entirely
>      inside the kernel.  I onlined the node with an SRAT hack and no ops
>      structure at all associated with the device (just set node affinity
>      to the pcie_dev and plumbed it through the network stack).
> 
>      A proper implementation would have virtio-net register is own
>      reserved memory region and online it during probe.
>   
>   3) Accelerator
>      (Not published yet)
> 
>      I have converted an open source but out of tree GPU driver which
>      uses NUMA nodes to use private nodes.  This required:
>             NP_OPS_MIGRATION
>             NP_OPS_MEMPOLICY
> 
>      The pattern is very similar to the CXL mempolicy driver, except
>      that the driver had alloc_pages_node() calls that needed to have
>      __GFP_PRIVATE added to ensure allocations landed on the device.
> 
> 
>   4) CXL Compressed RAM driver:
>      https://github.com/gourryinverse/linux/blob/55c06eb6bced58132d9001e318f2958e8ac80614/mm/cram.c#L340
>      needs pretty much everything - it's "normal memory" with access
>      rules, so the driver isn't really in the management lifecycle.
> 
>      In this example - the only way to allocate memory on the node is
>      via demotion.  This allows us to close off the device to new
>      allocations if the hardware reports low memory but the OS percieves
>      the device to still have free memory.
> 
>      Which is a cool example:  The driver just sets up the node with
>      certain attributes and then lets the kernel deal with it.
> 
> 
> I have started compacting the _OPS_* flags related to reclaim into a
> single NP_OPS_RECLAIM flag while testing with this.  Really i've come
> around to thinking many mm/ services need to be taken as a package,
> not fully piecemeal.
> 
> The tl;dr: Once you cede some control over to the kernel, you're
> very close to ceding ALL control, but you still get some control
> over how/when allocations on the node can be made.
> 
> 
> It is important to note that even if we don't expose callbacks, we do
> still need a modicum of node filtering in some places that still use
> for_each_node() (vmscan.c, compaction.c, oom_kill.c, etc).
> 
> These are basically all the places ZONE_DEVICE *implicitly* opts itself
> out of by having managed_pages=0.  We have to make those situations
> explicit - but that doesn't mean we need callbacks.
> 
>>>
>>> I would simply state: "That depends on the memory device"
>>
>> Let's keep it very simple: just some memory that you mbind(), and you
>> only want the mbind() user to make use of that memory.
>>
>> What would be the minimal set of hooks to guarantee that.
>>
> 
> If you want the mbind contract to stay intact:
> 
>    NP_OPS_MIGRATION (mbind can generate migrations)
>    NP_OPS_MEMPOLICY (this just tells mempolicy.c to allow the node)

I'm missing why these are even opt-in. What's the problem with allowing
mbind and mempolicy to use these nodes in some of your drivers?

I also have some questions about longterm pinnings, but that's better
discussed in person :)

> 
> The set of callbacks required should be exactly 0 (assuming we teach
> migrate.c to inject __GFP_PRIVATE like we have mempolicy.c).
> 
> If your device requires some special notification on allocation, free
> or migration to/from you need:
> 
>    ops.free_folio(folio)
>    ops.migrate_to(folios, nid, mode, reason, nr_success)
>    ops.migrate_folio(src_folio, dst_folio)
> 
> The free path is the tricky one to get right.  You can imagine:
> 
>    buf = malloc(...);
>    mbind(buf, private_node);
>    memset(buf, 0x42, ...);
>    ioctl(driver, CHECK_OUT_THIS_DATA, buf); 
>    exit(0);
> 
> The task dies and frees the pages back to the buddy - the question is
> whether the 4-5 free_folio paths (put_folio, put_unref_folios, etc) can
> all eat an ops.free_folio() callback to inform the driver the memory has
> been freed.

Right, that's rather invasive.

-- 
Cheers,

David