Re: [PATCH RFC 00/15] mm, swap: swap table phase IV with dynamic ghost swapfile

[Kairui Song <ryncsn@gmail.com>]

On Tue, Feb 24, 2026 at 2:22 AM Nhat Pham <nphamcs@gmail.com> wrote:
>
> On Thu, Feb 19, 2026 at 3:42 PM Kairui Song via B4 Relay
> <devnull+kasong.tencent.com@kernel.org> wrote:
> > Huge thanks to Chris Li for the layered swap table and ghost swapfile
> > idea, without whom the work here can't be archived. Also, thanks to Nhat
> > for pushing and suggesting using an Xarray for the swapfile [11] for
> > dynamic size. I was originally planning to use a dynamic cluster
> > array, which requires a bit more adaptation, cleanup, and convention
> > changes. But during the discussion there, I got the inspiration that
> > Xarray can be used as the intermediate step, making this approach
> > doable with minimal changes. Just keep using it in the future, it
> > might not hurt too, as Xarray is only limited to ghost / virtual
> > files, so plain swaps won't have any extra overhead for lookup or high
> > risk of swapout allocation failure.
>
> Thanks for your effort. Dynamic swap space is a very important
> consideration anyone deploying compressed swapping backend on large
> memory systems in general. And yeah, I think using a radix tree/xarray
> is easiest out-of-the-box solution for this - thanks for citing me :P

Thanks for the discussion :)

>
> I still have some confusion and concerns though. Johannes already made
> some good points - I'll just add some thoughts from my point of view,
> having gone back and forth with virtual swap designs:
>
> 1. At which layer should the metadata (swap count, swap cgroup, etc.) live?
>
> I remember that in your LSFMMBPF presentation (time flies), your
> proposal was to store a redirection entry in the top layer, and keep
> all the metadata at the bottom (i.e backend) layer? This has problems
> - for once, you might not know suitable backend at swap allocation
> time, but only at writeout time. For e.g, in certain zswap setups, we
> reject the incompressible page and cycle it back to the active LRU, so
> we have no space in zswap layer to store swap entry metadata (note
> that at this point the swap entry cannot be freed, because we have
> already unmapped the page from the PTEs (and would require a page
> table walk to undo this a la swapoff). Similarly, when we
> exclusive-load a page from zswap, we invalidate the zswap metadata
> struct, so we will no longer have a space for the swap entry metadata.
>
> The zero-filled (or same-filled) swap entry case is an even more
> egregious example :) It really shouldn't be a state in any backend -
> it should be a completely independent backend.
>
> The only design that makes sense is to store metadata in the top layer
> as well. It's what I'm doing for my virtual swap patch series, but if
> we're pursuing this opt-in swapfile direction we are going to
> duplicate metadata :)

It's already doing that, storing metadata at the top layer, only a
reverse mapping in the lower layer.

So none of these issues are still there. Don't worry, I do remember
that conversation and kept that in mind :)

> > And if you consider these ops are too complex to set up and maintain, we
> > can then only allow one ghost / virtual file, make it infinitely large,
> > and be the default one and top tier, then it achieves the identical thing
> > to virtual swap space, but with much fewer LOC changed and being runtime
> > optional.
>
> 2. I think the "fewer LOC changed" claim here is misleading ;)
>
> A lot of the behaviors that is required in a virtual swap setup is
> missing from this patch series. You are essentially just implementing
> a swapfile with a dynamic allocator. You still need a bunch more logic
> to support a proper multi-tier virtual swap setup - just on top of my
> mind:

I left that part undone kind of on purpose, since it's only RFC, and
in hope that there could be collaboration.

And the dynamic allocator is only ~200 LOC now. Other parts of this
series are not only for virtual swap. For example the unified folio
alloc for swapin, which gives us 15% performance gain in real
workloads, can still get merged and benifit all of us without
involving the virtual swap or memcg part.

And meanwhile, with the later patches, we don't have to re-implement
the whole infrastructure to have a virtual table. And future plans
like data compaction should benifit every layer naturally (same
infra).

> a. Charging: virtual swap usage not be charged the same as the
> physical swap usage, especially when you have a zswap + disk swap
> setup, powered by virtual swap. For once, I don't believe in sizing
> virtual swap, but also a latency-sensitive cgroup allowe to use only
> zswap (backed by virtual swap) is using and competing for resources
> very differently from a cgroup whose memory is incompressible and only
> allowed to use disk swap.

Ah, now as you mention it, I see in the beginning of this series I
added: "Swap table P4 is stable and good to merge if we are OK with a
few memcg reparent behavior (there is also a solution if we don't)".
The "other solution" also fits your different charge idea here. Just
have a ci->memcg_table, then each layer can have their own charge
design, and the shadow is still only used for refault check. That
gives us 10 bytes per slot overhead though, but still lower than
before and stays completely dynamic.

Also, no duplicated memcg, since the upper layer and lower layer
should be charged differently. If they don't, then just let
ci->memcg_table stay NULL.

>
> b. Backend decision making and efficient backend transfer - as you
> said, "folio_realloc_swap" is yet to be implemented :) And as I
> mention earlier, we CANNOT determine swap backend before PTE unmap

And we are not doing that at all. folio_alloc_swap happens before
unmap, but realloc happens after that. VSS does the same thing.

> time, because backend suitability is content-dependent. You will have
> to add extra logic to handle this nuanced swap allocation behavior.
>
> c. Virtual swap freeing - it requires more work, as you have to free
> both the virtual swap entry itself, as well as digging into the
> physical backend layer.
>
> d. Swapoff - now you have to both page tables and virtual swap table.

Swapoff is actually easy here... If it sees a reverse map slot, read
into the upper layer. Else goto the old logic. Then it's done. If
ghost swap is the layer with highest priority, then every slot is a
reverse map slot.

>
> By the time you implement all of this, I think it will be MORE
> complex, especially since you want to maintain BOTH the new setup and
> the old non-virtual swap setup. You'll have to litter the codes with a
> bunch of ifs (or ifdefs) to check - hey do we have a virtual swapfile?
> Hey is this a virtual swap slot? Etc. Etc. everywhere, from the PTE
> infra (zapping, page fault, etc.), to cgroup infra, to physical swap
> architecture.

It is using the same infrastructure, which means a lot of things are
reused and unified. Isn't that a good sign? And again we don't need to
re-implement the whole infra.

And if you need multiple layers then there will be more "if"s and
overhead however you implement it. But with unified infra, each layer
can stay optional. And checking "si->flags & GHOST / VIRTUAL" really
shouldn't be costly or trouble some at all, compared to a mandatory
layer with layers of Xarray walk.

And we can move, maintain the virt part in a separate place.

> Comparing this line of work by itself with the vswap series, which
> already comes with all of these included, is a bit apples-to-oranges
> (and especially with the fact that vswap simplifies logic and removes
> LoCs in a lot of places too, such as in swapoff. The delta LoC is only
> 300-400 IIRC?).

One thing I want to highlight here is that the old swapoff really
shouldn't just die. That gives us no chance to clear up the swap cache
at all (vss holding swap data in RAM is also just swap cache). Pages
still in swap cache means minor page faults will still trigger. If the
workload is opaque but we knows a high load of traffic is coming and
wants to get rid of any performance bottleneck by reading all folios
into the right place, swapoff gives the guarantee that no anon fault
will be ever triggered, that happens a lot in multiple tenant cloud
environments, and these workload are opaque so madvise doesn't apply.

> > The size of the swapfile (si->max) is now just a number, which could be
> > changeable at runtime if we have a proper idea how to expose that and
> > might need some audit of a few remaining users. But right now, we can
> > already easily have a huge swap device with no overhead, for example:
> >
> > free -m
> >                total        used        free      shared  buff/cache   available
> > Mem:            1465         250         927           1         356        1215
> > Swap:       15269887           0    15269887
> >
>
> 3. I don't think we should expose virtual swap state to users (in this
> case, in the swapfile summary view i.e in free). It is just confusing,
> as it poorly reflects the physical state (be it compressed memory
> footprint, or actual disk usage). We obviously should expose a bunch
> of sysfs debug counters for troubleshootings, but for average users,
> it should be all transparent.

Using sysfs can also be a choice, that's really just a demonstration
interface. But I do think it's worse if the user has no idea what is
actually going on.