[LSF/MM/BPF TOPIC] userspace control of memory management

[LSF/MM/BPF TOPIC] userspace control of memory management

[Frank van der Linden]

I propose this discussion topic for LSF/MM/BPF.

In a world where memory topologies are becoming more complicated, is
it still possible to have an approach where the kernel deals with
memory management to everyone's satisfaction?

The answer seemingly has been "not quite", since madvise and mempolicy
exist. With things like cxl.mem coming into existence, a heterogeneous
memory setup will become more common.

The number of madvise options keeps growing. There is now a
process_madvise, and there are proposed extensions for the mempolicy
systemcalls, allowing one process to control the policy of another, as
well. There are exported cgroup interfaces to control reclaim, and
discussions have taken place on explicit control reclaim-as-demotion
to other nodes.

Is this the right approach? If so, would it be a good idea to
optionally provide BPF hooks to control certain behavior, and let
userspace direct things even more? Is that even possible,
performance-wise? Would it make sense to be able to influence the
MGLRU generation process in a more direct way if needed?

I think a discussion about these points would be interesting. Or, I
should say, further discussion.

What do you think?

Thanks,

- Frank

Re: [LSF/MM/BPF TOPIC] userspace control of memory management

[David Rientjes]

On Tue, 28 Feb 2023, Frank van der Linden wrote:

> I propose this discussion topic for LSF/MM/BPF.
> 
> In a world where memory topologies are becoming more complicated, is
> it still possible to have an approach where the kernel deals with
> memory management to everyone's satisfaction?
> 
> The answer seemingly has been "not quite", since madvise and mempolicy
> exist. With things like cxl.mem coming into existence, a heterogeneous
> memory setup will become more common.
> 
> The number of madvise options keeps growing. There is now a
> process_madvise, and there are proposed extensions for the mempolicy
> systemcalls, allowing one process to control the policy of another, as
> well. There are exported cgroup interfaces to control reclaim, and
> discussions have taken place on explicit control reclaim-as-demotion
> to other nodes.
> 
> Is this the right approach? If so, would it be a good idea to
> optionally provide BPF hooks to control certain behavior, and let
> userspace direct things even more? Is that even possible,
> performance-wise? Would it make sense to be able to influence the
> MGLRU generation process in a more direct way if needed?
> 
> I think a discussion about these points would be interesting. Or, I
> should say, further discussion.
> 
> What do you think?
> 

I think this is definitely a topic worth discussing and would like to 
participate in it.  Cc'ing others that I think would likewise be 
interested.

I think the APIs that you mention that already provide placement 
preferences to varying degrees of strictness, like mempolicies and 
madvise, won't be going away but what can be *supplemented* by other 
mechanisms (perhaps with BPF) is an interesting topic especially for users 
who need finer-grained control for optimal behavior.

It's a topic that is deserving of broad discussion.

Re: [Lsf-pc] Fwd: [LSF/MM/BPF TOPIC] userspace control of memory management

[Michal Hocko]

For some reason I cannot find this email in my linux-mm inbox and I
cannot find it in any archives so let me add linux-mm and lkml again for
future reference.

On Tue 28-02-23 21:20:57, Frank van der Linden via Lsf-pc wrote:
> ---------- Forwarded message ---------
> From: Frank van der Linden <fvdl@google.com>
> Date: Tue, Feb 28, 2023 at 4:15 PM
> Subject: [LSF/MM/BPF TOPIC] userspace control of memory management
> To: <linux-mm@kvack.org>
> 
> 
> I propose this discussion topic for LSF/MM/BPF.
> 
> In a world where memory topologies are becoming more complicated, is
> it still possible to have an approach where the kernel deals with
> memory management to everyone's satisfaction?
> 
> The answer seemingly has been "not quite", since madvise and mempolicy
> exist. With things like cxl.mem coming into existence, a heterogeneous
> memory setup will become more common.
> 
> The number of madvise options keeps growing. There is now a
> process_madvise, and there are proposed extensions for the mempolicy
> systemcalls, allowing one process to control the policy of another, as
> well. There are exported cgroup interfaces to control reclaim, and
> discussions have taken place on explicit control reclaim-as-demotion
> to other nodes.
> 
> Is this the right approach? If so, would it be a good idea to
> optionally provide BPF hooks to control certain behavior, and let
> userspace direct things even more? Is that even possible,
> performance-wise? Would it make sense to be able to influence the
> MGLRU generation process in a more direct way if needed?
> 
> I think a discussion about these points would be interesting. Or, I
> should say, further discussion.
> 
> What do you think?
> 
> Thanks,
> 
> - Frank
> _______________________________________________
> Lsf-pc mailing list
> Lsf-pc@lists.linux-foundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lsf-pc

-- 
Michal Hocko
SUSE Labs

Re: [Lsf-pc] Fwd: [LSF/MM/BPF TOPIC] userspace control of memory management

[Matthew Wilcox]

On Fri, Apr 28, 2023 at 04:18:16PM +0200, Michal Hocko wrote:
> For some reason I cannot find this email in my linux-mm inbox and I
> cannot find it in any archives so let me add linux-mm and lkml again for
> future reference.

Hm, I found it by searching for 'lsf' on lore.kernel.org in the linux-mm
archive.

https://lore.kernel.org/linux-mm/CAPTztWYAiroY3E8pwB+rnPGA1K9HLhkpQp1Gy9C1dEuS1FhWGg@mail.gmail.com/

Here are the other topics I found:

Eliminate vmap/vmalloc lock contention
Reducing direct map fragmentation
Stable process
Phyrs discussion
Live migration over CXL
SMDK MM changes for CXL
Sunsetting buffer_heads
HGM for hugetlbfs
Reducing zombie memcgs
Swap abstraction / native zswap
SLOB/SLAB allocator removal + future SLUB improvements
Session for CXL memory
Memory profiling using code tagging
Cloud storage optimizations
Flexible orders for anonymous folios
VM Memory Overcommit
State of The Page
Userspace control of memory management
IOMAP conversion status update
DAMON Updates and Future Plans
Make BPF memory allocator more robust
Virtual Machine Memory Passthrough
Scalable Pagefaults
Single Owner Memory
CXL Fabric Manager Architecture
Using hardware counters to determine hot/cold pages
Sframe: An orc like stack unwinder
Mm docs
Tracing mapped pages for quicker boot performance

(not all of these are the exact titles used by the authors)

Re: [LSF/MM/BPF TOPIC] userspace control of memory management

[Lorenzo Stoakes]

On Tue, Feb 28, 2023 at 04:15:21PM -0800, Frank van der Linden wrote:
> I propose this discussion topic for LSF/MM/BPF.
>
> In a world where memory topologies are becoming more complicated, is
> it still possible to have an approach where the kernel deals with
> memory management to everyone's satisfaction?
>
> The answer seemingly has been "not quite", since madvise and mempolicy
> exist. With things like cxl.mem coming into existence, a heterogeneous
> memory setup will become more common.
>
> The number of madvise options keeps growing. There is now a
> process_madvise, and there are proposed extensions for the mempolicy
> systemcalls, allowing one process to control the policy of another, as
> well. There are exported cgroup interfaces to control reclaim, and
> discussions have taken place on explicit control reclaim-as-demotion
> to other nodes.
>
> Is this the right approach? If so, would it be a good idea to
> optionally provide BPF hooks to control certain behavior, and let
> userspace direct things even more? Is that even possible,
> performance-wise? Would it make sense to be able to influence the
> MGLRU generation process in a more direct way if needed?
>
> I think a discussion about these points would be interesting. Or, I
> should say, further discussion.
>
> What do you think?
>
> Thanks,
>
> - Frank
>

Surely userfaultfd is part of this equation too? I definitely think this is a
useful converastion to have, in any case!

Re: [Lsf-pc] Fwd: [LSF/MM/BPF TOPIC] userspace control of memory management

[Michal Hocko]

On Fri 28-04-23 15:54:31, Matthew Wilcox wrote:
> On Fri, Apr 28, 2023 at 04:18:16PM +0200, Michal Hocko wrote:
> > For some reason I cannot find this email in my linux-mm inbox and I
> > cannot find it in any archives so let me add linux-mm and lkml again for
> > future reference.
> 
> Hm, I found it by searching for 'lsf' on lore.kernel.org in the linux-mm
> archive.
> 
> https://lore.kernel.org/linux-mm/CAPTztWYAiroY3E8pwB+rnPGA1K9HLhkpQp1Gy9C1dEuS1FhWGg@mail.gmail.com/

I have tried to search for the msg-id via lore.kernel.org
(https://lore.kernel.org/all/?q=CAPTztWY49XP-7GDHuvV2fNDCeJzd0vAac6n%2BrJ9KfWr6cyZ5ww%40mail.gmail.com)
as well as the subject
(https://lore.kernel.org/?q=userspace+control+of+memory+management).

I've had the email in my lsf-pc mailbox but that is not archived at lore
and we are trying to prepare schedule with links to the archive so that
people can find associated disucssions easier.

> Here are the other topics I found:

This matches my list as well

> Memory profiling using code tagging

Except for this one which somehow escaped my attention. Now added and
scheduled. Thanks Matthew!

-- 
Michal Hocko
SUSE Labs

Re: [Lsf-pc] Fwd: [LSF/MM/BPF TOPIC] userspace control of memory management

[Hao Luo]

On Fri, Apr 28, 2023 at 7:18 AM Michal Hocko <mhocko@suse.com> wrote:
>
> For some reason I cannot find this email in my linux-mm inbox and I
> cannot find it in any archives so let me add linux-mm and lkml again for
> future reference.
>
> On Tue 28-02-23 21:20:57, Frank van der Linden via Lsf-pc wrote:
> > ---------- Forwarded message ---------
> > From: Frank van der Linden <fvdl@google.com>
> > Date: Tue, Feb 28, 2023 at 4:15 PM
> > Subject: [LSF/MM/BPF TOPIC] userspace control of memory management
> > To: <linux-mm@kvack.org>
> >
> >
> > I propose this discussion topic for LSF/MM/BPF.
> >
> > In a world where memory topologies are becoming more complicated, is
> > it still possible to have an approach where the kernel deals with
> > memory management to everyone's satisfaction?
> >
> > The answer seemingly has been "not quite", since madvise and mempolicy
> > exist. With things like cxl.mem coming into existence, a heterogeneous
> > memory setup will become more common.
> >
> > The number of madvise options keeps growing. There is now a
> > process_madvise, and there are proposed extensions for the mempolicy
> > systemcalls, allowing one process to control the policy of another, as
> > well. There are exported cgroup interfaces to control reclaim, and
> > discussions have taken place on explicit control reclaim-as-demotion
> > to other nodes.
> >
> > Is this the right approach? If so, would it be a good idea to
> > optionally provide BPF hooks to control certain behavior, and let
> > userspace direct things even more? Is that even possible,
> > performance-wise? Would it make sense to be able to influence the
> > MGLRU generation process in a more direct way if needed?
> >
> > I think a discussion about these points would be interesting. Or, I
> > should say, further discussion.
> >
> > What do you think?
> >
> > Thanks,
> >
> > - Frank
> > _______________________________________________

Please allow me to cc bpf mailing list for visibility. The idea seems
interesting.

Hao