Re: [PATCH v2 03/19] mm, swap: never bypass the swap cache even for SWP_SYNCHRONOUS_IO

[Kairui Song <ryncsn@gmail.com>]

On Fri, Nov 21, 2025 at 12:56 PM Barry Song <21cnbao@gmail.com> wrote:
>
> On Fri, Nov 21, 2025 at 10:42 AM Kairui Song <ryncsn@gmail.com> wrote:
> >
> > On Fri, Nov 21, 2025 at 8:55 AM Barry Song <21cnbao@gmail.com> wrote:
> > >
> > > Hi Kairui,
> > >
> > > >
> > > > +       /*
> > > > +        * If a large folio already belongs to anon mapping, then we
> > > > +        * can just go on and map it partially.
> > >
> > > this is right.
> > >
> >
> > Hi Barry,
> >
> > Thanks for the review.
> >
> > > > +        * If not, with the large swapin check above failing, the page table
> > > > +        * have changed, so sub pages might got charged to the wrong cgroup,
> > > > +        * or even should be shmem. So we have to free it and fallback.
> > > > +        * Nothing should have touched it, both anon and shmem checks if a
> > > > +        * large folio is fully appliable before use.
> > >
> > > I'm curious about one case:
> > >
> > > - Process 1: nr_pages are in swap.
> > > - Process 2: "nr_pages - m" pages are in swap (with m slots already
> > >   unmapped).
> > >
> > > Sequence:
> > >
> > > 1. Process 1 swap-ins the page, allocates it, and adds it to the
> > >    swapcache — but the rmap hasn’t been added yet.
> >
> > Yes, whoever wants to use the folio will have to lock it first.
> >
> > > 2. Process 2 swap-ins the same folio and finds it in the swapcache, but
> > >    it’s not associated with anon_mapping yet.
> >
> > If P2 found it in the swap cache, it will try to acquire the folio lock.
> >
> > > What will process 2 do in this situation? Does it go to out_nomap? If so,
> > > what happens on the second swapin attempt? Will it keep retrying
> > > indefinitely until Process 1 completes the rmap installation?
> >
> > P2 will wait on the folio lock, which I think is the right thing to
> > do. After P1 finishes the rmap installation, P2 wakes up and tries to
> > map the folio, no busy loop or repeated fault.
>
> Right. The folio lock might help. But consider this case:
> p1 runs on a small core,
> p2(partially unmapped) runs on a big core,
> p2 grabs the folio lock before p1 and therefore goes to out_nomap.
> After p2 unlocks the folio and wakes p1, p1 may not preempt the
> current task on its CPU in time. p2 may repeatedly fault and take the
> lock again. But yes, the race window is small though mTHP might
> increase the race address range.

P2 would remove the folio from the swap cache if I understand you
correctly. Because from P2's perspective, the page table supporting
the folio is gone, so it would remove the folio from swap cache and
try to fallback to order 0. This may lead to repeated IO indeed
though.

>
> For small folios, this problem does not exist — whoever gets the lock
> first can map it first.
> Ideally, we would enhance the rmap API so it can partially add new
> anon mappings.

Yes, a very insightful idea. That's also one of the reasons why I
added a WARN_ON and this comment below:
"This will be removed once we unify folio allocation in the swap cache".

By then we can either split the folio or map it partially. We just
can't do that right now because the folio may not belong to a single
VMA / Cgroup.

> I guess we may defer handling this problem till someone reports it?

Yeah, that's what I have in mind too. In later phases this will be
improved, and so far the worst thing could happen is repeated fault /
IO, and the time window is really tiny and only happens for the
particular case as you described.


> > If P1 somehow failed to install the rmap (eg. a concurrent partial
> > unmap made part of the page table invalidated), it will remove the
> > folio from swap cache then unlock it (the code right below). P2 also
> > wakes up by then, and sees the invalid folio will then fallback to
> > order 0.
>
> Yes, I understand this is the case for p1. The concern is that p2 is
> partially unmapped.
>
> >
> > >
> > > > +        *
> > > > +        * This will be removed once we unify folio allocation in the swap cache
> > > > +        * layer, where allocation of a folio stabilizes the swap entries.
> > > > +        */
> > > > +       if (!folio_test_anon(folio) && folio_test_large(folio) &&
> > > > +           nr_pages != folio_nr_pages(folio)) {
> > > > +               if (!WARN_ON_ONCE(folio_test_dirty(folio)))
> > > > +                       swap_cache_del_folio(folio);
> > > > +               goto out_nomap;
> > > > +       }
> > > > +