Re: [RFC PATCH v2 0/5] mm/swap, memcg: Introduce swap tiers for cgroup based swap control

[Chris Li <chrisl@kernel.org>]

Hi YoungJun,

On Mon, Mar 9, 2026 at 7:14 PM YoungJun Park <youngjun.park@lge.com> wrote:
>
> On Mon, Mar 02, 2026 at 01:27:31PM -0800, Shakeel Butt wrote:
> >
> > Hi YoungJun,
> >
> > Sorry for the late response.
> >
> > On Sun, Feb 22, 2026 at 10:16:04AM +0900, YoungJun Park wrote:
> > [...]
> >
> > Let me summarize our discussion first:
> >
> > You have a use-case where they have systems running multiple workloads and
> > have multiple swap devices. Those swap devices have different performance
> > capabilities and they want to restrict/assign swap devices to the workloads. For
> > example assigning a low latency SSD swap device to latency sensitive workload
> > and slow disk swap to latency tolerant workload. (please correct me if
> > I misunderstood something).
> >
> > The use-case seems reasonable to me but I have concerns related to adding an
> > interface to memory cgroups. Mainly I am not clear how hierarchical semantics on
> > such interface would look like. In addition, I think it would be too rigid and
> > will be very hard to evolve for future features. To me enabling this
> > functionality through BPF would give much more flexibility and will be more
> > future proof.
> >
> > >
> > > After reading the reply and re-think more of it.
> > >
> > > I have a few questions regarding the BPF-first approach you
> > > suggested, if you don't mind. Some of them I am re-asking
> > > because I feel they have not been clearly addressed yet.
> > >
> > > - We are in an embedded environment where enabling additional
> > >   kernel compile options is costly. BPF is disabled by
> > >   default in some of our production configurations. From a
> > >   trade-off perspective, does it make sense to enable BPF
> > >   just for swap device control?
> >
> > To me, it is reasonable to enable BPF for environment running multiple
> > workloads and having multiple swap devices.
> >
> > >
> > > - You suggest starting with BPF and discussing a stable
> > >   interface later. I am genuinely curious, are there actual
> > >   precedents where a BPF prototype graduated into a stable
> > >   kernel interface?
> >
> > After giving some thought, I think once we have BPF working, adding another
> > interface for the same feature would not be an option. So, we have decide
> > upfront which route to take.
> >
> > >
> > > - You raised that stable interfaces are hard to remove. Would
> > >   gating it behind a CONFIG option or marking it experimental
> > >   be an acceptable compromise?
> >
> > I think hiding behind CONFIG options do not really protect against the usage and
> > the rule of no API breakage usually apply.
> >
> > >
> > > - You already acknowledged the use-case for assigning
> > >   different swap devices to different workloads. Your
> > >   objection is specifically about hierarchical parent-child
> > >   partitioning. If the interface enforced uniform policy
> > >   within a subtree, would that be acceptable?
> >
> > Let's start with that or maybe comeup with concrete examples on how that would
> > look like.
> >
> > Beside, give a bit more thought on potential future features e.g. demotion and
> > reason about how you would incorporate those features.
> Hello Shakeel, Chris Li,
>
> Just sending a gentle ping on my previous reply. :D

Sorry for the late reply, busy days.

>
> To quickly summarize the main points:
> (I might wrongly undestand your intentaion, then correct me please :) )
>
> * Regarding Shakeel's BPF approach, stable interface movement would be difficult,
>   so we need to choose a direction. I prefer adding it to memcg for immediate
>   usage, and if it proves highly effective, we can consider transitioning
>   entirely to BPF later.

I am very concerned about locking down the kernel user interface just
because things might change in the future. If we need to use BPF to
get the stable user space API, I am fine with that. Completely
blocking the new cgroup interface because of a worry about future
change is not justifiable IMHO. There ought to be some intermediate
staging we can do e.g. debugfs interface to test and play with the new
API. We should focus on designing the interface as well as possible
right now.

> * Shakeel seemed somewhat positive about matching all child tiers from the
>   parent if tiers are applied to a specific cgroup use case, and I would like
>   to start the discussion from here. Chris, I would appreciate your thoughts
>   on whether you agree with this direction of unifying all swap tiers within
>   the hierarchy as a first step.

Does that mean all children will only use the parent cgroup setting?
Wouldn't that be more restrictive and counteract the goal of making
the API more future-proof?
For the record, the current Google deployment can uses a different
swap device for the child cgroup, in the current delpyment.
The typical setup is that the top-level cgroup is a job running on a
VM. Then there is a second cgroup level for the VMM guest memory
allocation; swap device selection occurs at this second level.
There is also zswap vs SSD, the SSD is something new starting the
deployment. So it's not just about enabling zswap or not. We also need
to select the swap device.
If you get the current cgroup, it will need to walk the parent cgroup
chain to find the toplevel cgroup any way. I just think having the
hierarchy makes more sense.

> Here are some additional thoughts I had after my last reply:
> (Thanks for the insight and discussion. Hyungjun Cho)
>
> * Cgroup distribution:
>   A direct use case where cgroup A distributes a portion to A' is hard to
>   imagine, but the following scenario is possible:
>
>   swap: +SSD +HDD +NET
>   cgroup hierarchy:
>   /
>   A : +HDD +NET
>   A'(app 1) +HDD, A''(app 2) +NET
>
>   Cgroup A has two interdependent apps, and +SSD is excluded for more critical
>   services. App1 (A') avoids reclaim with a large hot working set using fast
>   +HDD, while App2 (A'') has a cold working set using slow/large +NET.

The app interface is a huge departure from the cgroup. The cgroup is a
well defined interface.

>
> * Promotion / Demotion:
>   Unlike memory tiers, swap tiers are directly assigned by the user, providing
>   flexibility beyond just speed. Since swap priority is already a user choice,
>   this design makes perfect sense.

We need to find customers willing to use this promotion/demotion. I
hesitate to build something while hoping to find someone to use it
later.
It would be good to identify someone who can immediately use and test
this promotion/demotion feature.
We should focus the discussion on achieving a more flexible swap
device selection approach and reach a conclusion on the API discussion
before discussing promotion/demotion. If we can't even have a usable
swap tier interface, there is nothing to promote.

>   With this arbitrary assignment, we can support higher-to-slower tier
>   allocation, similar to current memory tiers, if user properly bind the tier.
>   (more flexible as I think)
>
>   Within the same tier (meaning we define it as equal speed(tier)), we could apply round-robin or other
>   distribution policies via an additional tier layer interface. The current
>   equal-priority round-robin policy could also be elevated to the tier layer.


Chris