Re: [PATCH] mm: free idle swap cache page after COW

[Johannes Weiner <hannes@cmpxchg.org>]

On Tue, Jun 01, 2021 at 12:48:15PM +0100, Matthew Wilcox wrote:
> On Tue, Jun 01, 2021 at 01:31:43PM +0800, Huang Ying wrote:
> > With commit 09854ba94c6a ("mm: do_wp_page() simplification"), after
> > COW, the idle swap cache page (neither the page nor the corresponding
> > swap entry is mapped by any process) will be left in the LRU list,
> > even if it's in the active list or the head of the inactive list.  So,
> > the page reclaimer may take quite some overhead to reclaim these
> > actually unused pages.
> > 
> > To help the page reclaiming, in this patch, after COW, the idle swap
> > cache page will be tried to be freed.  To avoid to introduce much
> > overhead to the hot COW code path,
> > 
> > a) there's almost zero overhead for non-swap case via checking
> >    PageSwapCache() firstly.
> > 
> > b) the page lock is acquired via trylock only.
> > 
> > To test the patch, we used pmbench memory accessing benchmark with
> > working-set larger than available memory on a 2-socket Intel server
> > with a NVMe SSD as swap device.  Test results shows that the pmbench
> > score increases up to 23.8% with the decreased size of swap cache and
> > swapin throughput.
> 
> So 2 percentage points better than my original idea?  Sweet.
> 
> > diff --git a/mm/memory.c b/mm/memory.c
> > index 2b7ffcbca175..d44425820240 100644
> > --- a/mm/memory.c
> > +++ b/mm/memory.c
> > @@ -3104,6 +3104,8 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf)
> >  				munlock_vma_page(old_page);
> >  			unlock_page(old_page);
> >  		}
> > +		if (page_copied)
> > +			free_swap_cache(old_page);
> >  		put_page(old_page);
> >  	}
> >  	return page_copied ? VM_FAULT_WRITE : 0;
> 
> Why not ...
> 
> 		if (page_copied)
> 			free_page_and_swap_cache(old_page);
> 		else
> 			put_page(old_page);
> 
> then you don't need to expose free_swap_cache().  Or does the test for
> huge_zero_page mess this up?

It's free_page[s]_and_swap_cache() we should reconsider, IMO.

free_swap_cache() makes for a clean API function that does one thing,
and does it right. free_page_and_swap_cache() combines two independent
operations, which has the habit of accumulating special case-handling
for some callers that is unncessary overhead for others (Abstraction
Inversion anti-pattern).

For example, free_page_and_swap_cache() adds an is_huge_zero_page()
check around the put_page() for the tlb batching code. This isn't
needed here. AFAICS it is also unnecessary for the other callsite,
__collapse_huge_page_copy(), where context rules out zero pages.

The common put_page() in Huang's version also makes it slighly easier
to follow the lifetime of old_page.

So I'd say exposing free_swap_cache() is a good move, for this patch
and in general.