Re: [PATCH v10 05/14] mm: multi-gen LRU: groundwork

[Yu Zhao <yuzhao@google.com>]

On Mon, Apr 11, 2022 at 8:16 PM Andrew Morton <akpm@linux-foundation.org> wrote:
>
> On Wed,  6 Apr 2022 21:15:17 -0600 Yu Zhao <yuzhao@google.com> wrote:
>
> > Evictable pages are divided into multiple generations for each lruvec.
> > The youngest generation number is stored in lrugen->max_seq for both
> > anon and file types as they are aged on an equal footing. The oldest
> > generation numbers are stored in lrugen->min_seq[] separately for anon
> > and file types as clean file pages can be evicted regardless of swap
> > constraints. These three variables are monotonically increasing.
> >
> > ...
> >
> > +static inline bool lru_gen_del_folio(struct lruvec *lruvec, struct folio *folio, bool reclaiming)
>
> There's a lot of function inlining here.  Fortunately the compiler will
> ignore it all, because some of it looks wrong.  Please review (and
> remeasure!).  If inlining is reqlly justified, use __always_inline, and
> document the reasons for doing so.

I totally expect modern compilers to make better decisions than I do.
And personally, I'd never use __always_inline; instead, I'd strongly
recommend FDO/LTO.

> > +{
> > +     int gen;
> > +     unsigned long old_flags, new_flags;
> > +
> > +     do {
> > +             new_flags = old_flags = READ_ONCE(folio->flags);
> > +             if (!(new_flags & LRU_GEN_MASK))
> > +                     return false;
> > +
> > +             VM_BUG_ON_FOLIO(folio_test_active(folio), folio);
> > +             VM_BUG_ON_FOLIO(folio_test_unevictable(folio), folio);
> > +
> > +             gen = ((new_flags & LRU_GEN_MASK) >> LRU_GEN_PGOFF) - 1;
> > +
> > +             new_flags &= ~LRU_GEN_MASK;
> > +             /* for shrink_page_list() */
> > +             if (reclaiming)
> > +                     new_flags &= ~(BIT(PG_referenced) | BIT(PG_reclaim));
> > +             else if (lru_gen_is_active(lruvec, gen))
> > +                     new_flags |= BIT(PG_active);
> > +     } while (cmpxchg(&folio->flags, old_flags, new_flags) != old_flags);
>
> Clearly the cmpxchg loop is handling races against a concurrent
> updater.  But it's unclear who that updater is, what are the dynamics
> here and why the designer didn't use, say, spin_lock().  The way to
> clarify such thigs is with code comments!

Right. set_mask_bits() should suffice here.

> > +#endif /* !__GENERATING_BOUNDS_H */
> > +
> > +/*
> > + * Evictable pages are divided into multiple generations. The youngest and the
> > + * oldest generation numbers, max_seq and min_seq, are monotonically increasing.
> > + * They form a sliding window of a variable size [MIN_NR_GENS, MAX_NR_GENS]. An
> > + * offset within MAX_NR_GENS, gen, indexes the LRU list of the corresponding
>
> The "within MAX_NR_GENS, gen," text here is unclear?

Will update: "i.e., gen".

> > + * generation. The gen counter in folio->flags stores gen+1 while a page is on
> > + * one of lrugen->lists[]. Otherwise it stores 0.
> > + *
> > + * A page is added to the youngest generation on faulting. The aging needs to
> > + * check the accessed bit at least twice before handing this page over to the
> > + * eviction. The first check takes care of the accessed bit set on the initial
> > + * fault; the second check makes sure this page hasn't been used since then.
> > + * This process, AKA second chance, requires a minimum of two generations,
> > + * hence MIN_NR_GENS. And to maintain ABI compatibility with the active/inactive
>
> Where is the ABI compatibility issue?  Is it in some way in which the
> legacy LRU is presented to userspace?

Will update: yes, active/inactive LRU sizes in /proc/vmstat.

> > + * LRU, these two generations are considered active; the rest of generations, if
> > + * they exist, are considered inactive. See lru_gen_is_active(). PG_active is
> > + * always cleared while a page is on one of lrugen->lists[] so that the aging
> > + * needs not to worry about it. And it's set again when a page considered active
> > + * is isolated for non-reclaiming purposes, e.g., migration. See
> > + * lru_gen_add_folio() and lru_gen_del_folio().
> > + *
> > + * MAX_NR_GENS is set to 4 so that the multi-gen LRU can support twice of the
>
> "twice the number of"?

Will update.

> > + * categories of the active/inactive LRU when keeping track of accesses through
> > + * page tables. It requires order_base_2(MAX_NR_GENS+1) bits in folio->flags.
> > + */
>
> Helpful comment, overall.
>
> >
> > ...
> >
> > --- a/mm/Kconfig
> > +++ b/mm/Kconfig
> > @@ -909,6 +909,14 @@ config ANON_VMA_NAME
> >         area from being merged with adjacent virtual memory areas due to the
> >         difference in their name.
> >
> > +config LRU_GEN
> > +     bool "Multi-Gen LRU"
> > +     depends on MMU
> > +     # the following options can use up the spare bits in page flags
> > +     depends on !MAXSMP && (64BIT || !SPARSEMEM || SPARSEMEM_VMEMMAP)
> > +     help
> > +       A high performance LRU implementation to overcommit memory.
> > +
> >  source "mm/damon/Kconfig"
>
> This is a problem.  I had to jump through hoops just to be able to
> compile-test this.  Turns out I had to figure out how to disable
> MAXSMP.
>
> Can we please figure out a way to ensure that more testers are at least
> compile testing this?  Allnoconfig, defconfig, allyesconfig, allmodconfig.
>
> Also, I suggest that we actually make MGLRU the default while in linux-next.

The !MAXSMP is to work around [1], which I haven't had the time to
fix. That BUILD_BUG_ON() shouldn't assert sizeof(struct page) == 64
since the true size depends on WANT_PAGE_VIRTUAL as well as
LAST_CPUPID_NOT_IN_PAGE_FLAGS. My plan is here [2].

[1] https://lore.kernel.org/r/20190905154603.10349-4-aneesh.kumar@linux.ibm.com/
[2] https://lore.kernel.org/r/Ygl1Gf+ATBuI%2Fm2q@google.com/