Re: [PATCH] mm: test page->flags directly in page_lru()

[Yu Zhao <yuzhao@google.com>]

On Thu, Feb 25, 2021 at 03:55:53AM +0000, Matthew Wilcox wrote:
> On Wed, Feb 24, 2021 at 04:50:39PM -0700, Yu Zhao wrote:
> > On Wed, Feb 24, 2021 at 10:48:46PM +0000, Matthew Wilcox wrote:
> > > On Wed, Feb 24, 2021 at 03:34:16PM -0700, Yu Zhao wrote:
> > > > > If only somebody were working on a patch series to get rid of
> > > > > all those calls to compound_head()!  Some reviews on
> > > > > https://lore.kernel.org/linux-mm/20210128070404.1922318-2-willy@infradead.org/
> > > > > would be nice.
> > > > 
> > > > I'm on board with the idea and have done some research in this
> > > > direction. We've found that the ideal *anon* page size for Chrome OS
> > > > is not 4KB or 2MB, but 32KB. I hope we could leverage the folio to
> > > > support flexible anon page size to reduce the number of page faults
> > > > (vs 4KB) or internal fragmentation (vs 2MB).
> > > > 
> > > > That being said, it seems to me this is a long term plan and right
> > > > now we need something smaller. So if you don't mind, I'll just go
> > > > ahead and remove compound_head() from Page{LRU,Active,Unevictable,
> > > > SwapBacked} first?
> > > 
> > > It's really not a big change I'm suggesting here.  You need
> > > https://lore.kernel.org/linux-mm/20210128070404.1922318-2-willy@infradead.org/
> > > https://lore.kernel.org/linux-mm/20210128070404.1922318-5-willy@infradead.org/
> > > https://lore.kernel.org/linux-mm/20210128070404.1922318-8-willy@infradead.org/
> > > and then the patch I sent above to create folio_lru().
> > > 
> > > Then any changes you want to make to use folios more broadly will
> > > incrementally move us towards your goal of 32kB anon pages.
> > 
> > Well, these patches introduce a new concept which I'm on board with.
> 
> It's not really a new concept ... it's a new type for an existing concept
> (a head page).
> 
> > Assume everybody else is too, it still seems to me it's an overkill
> > to employee folio to just get rid of unnecessary compound_head()
> > in page_lru() -- this is not a criticism but a compliment.
> 
> It's not overkill, that really is the point of a folio!  If you
> think about it, only head pages can be on the LRU list (because the
> compound_head is in the union with the lru list_head).  So it
> always makes sense to talk about folios on the LRU list.
> 
> > Let me work out something *conceptually* smaller first, and if you
> > think folio is absolutely more suitable even for this specific issue,
> > I'll go review and test the four patches you listed. Sounds good?
> 
> Umm.  It seems to me that no matter what you do, it'll be equivalent to
> this, only without the type-safety?

I'm thinking about something trivial but still very effective. So far
I've only tested it with PG_{active,unevictable}, and I'm already
seeing a 4KB gain less the 2KB loss from page_lru().

I didn't go with this at the beginning because it's also time-
consuming. I need to go over every single use of
PG_{active,unevictable,swapbacked,lru}.

add/remove: 0/0 grow/shrink: 1/37 up/down: 4/-4129 (-4125)

diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
index 3cec6fbef725..c866c363bb41 100644
--- a/fs/proc/task_mmu.c
+++ b/fs/proc/task_mmu.c
@@ -1712,6 +1712,7 @@ static void gather_stats(struct page *page, struct numa_maps *md, int pte_dirty,
                        unsigned long nr_pages)
 {
        int count = page_mapcount(page);
+       struct page *head = compound_head(page);
 
        md->pages += nr_pages;
        if (pte_dirty || PageDirty(page))
@@ -1720,7 +1721,7 @@ static void gather_stats(struct page *page, struct numa_maps *md, int pte_dirty,
        if (PageSwapCache(page))
                md->swapcache += nr_pages;
 
-       if (PageActive(page) || PageUnevictable(page))
+       if (PageActive(head) || PageUnevictable(head))
                md->active += nr_pages;
 
        if (PageWriteback(page))
diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
index db914477057b..35b3d272ab4c 100644
--- a/include/linux/page-flags.h
+++ b/include/linux/page-flags.h
@@ -335,8 +335,8 @@ PAGEFLAG(Dirty, dirty, PF_HEAD) TESTSCFLAG(Dirty, dirty, PF_HEAD)
        __CLEARPAGEFLAG(Dirty, dirty, PF_HEAD)
 PAGEFLAG(LRU, lru, PF_HEAD) __CLEARPAGEFLAG(LRU, lru, PF_HEAD)
        TESTCLEARFLAG(LRU, lru, PF_HEAD)
-PAGEFLAG(Active, active, PF_HEAD) __CLEARPAGEFLAG(Active, active, PF_HEAD)
-       TESTCLEARFLAG(Active, active, PF_HEAD)
+PAGEFLAG(Active, active, PF_ONLY_HEAD) __CLEARPAGEFLAG(Active, active, PF_ONLY_HEAD)
+       TESTCLEARFLAG(Active, active, PF_ONLY_HEAD)
 PAGEFLAG(Workingset, workingset, PF_HEAD)
        TESTCLEARFLAG(Workingset, workingset, PF_HEAD)
 __PAGEFLAG(Slab, slab, PF_NO_TAIL)
@@ -407,9 +407,9 @@ CLEARPAGEFLAG(SwapCache, swapcache, PF_NO_TAIL)
 PAGEFLAG_FALSE(SwapCache)
 #endif
 
-PAGEFLAG(Unevictable, unevictable, PF_HEAD)
-       __CLEARPAGEFLAG(Unevictable, unevictable, PF_HEAD)
-       TESTCLEARFLAG(Unevictable, unevictable, PF_HEAD)
+PAGEFLAG(Unevictable, unevictable, PF_ONLY_HEAD)
+       __CLEARPAGEFLAG(Unevictable, unevictable, PF_ONLY_HEAD)
+       TESTCLEARFLAG(Unevictable, unevictable, PF_ONLY_HEAD)
 
 #ifdef CONFIG_MMU
 PAGEFLAG(Mlocked, mlocked, PF_NO_TAIL)