Re: Are THPs the right model for the pagecache?

[John Hubbard <jhubbard@nvidia.com>]

On 11/12/20 8:46 PM, Matthew Wilcox wrote:
> When I started working on using larger pages in the page cache, I was
> thinking about calling them large pages or lpages.  As I worked my way
> through the code, I switched to simply adopting the transparent huge
> page terminology that is used by anonymous and shmem.  I just changed
> the definition so that a thp is a page of arbitrary order.
> 
> But now I'm wondering if that expediency has brought me to the right
> place.  To enable THP, you have to select CONFIG_TRANSPARENT_HUGEPAGE,
> which is only available on architectures which support using larger TLB
> entries to map PMD-sized pages.  Fair enough, since that was the original
> definition, but the point of suppoting larger page sizes in the page
> cache is to reduce software overhead.  Why shouldn't Alpha or m68k use
> large pages in the page cache, even if they can't use them in their TLBs?
> 
> I'm also thinking about the number of asserts about
> PageHead/PageTail/PageCompound and the repeated invocations of
> compound_head().  If we had a different type for large pages, we could use
> the compiler to assert these things instead of putting in runtime asserts.

This seems like a really good idea to me, anyway. Even in the fairly
small area of gup.c, some type safety (normal pages vs. large pages)
would have helped keep things straight when I was fooling around with
pinning pages.


> 
> IOWs, something like this:
> 
> struct lpage {
> 	struct page subpages[4];
> };
> 
> static inline struct lpage *page_lpage(struct page *page)
> {
> 	unsigned long head = READ_ONCE(page->compound_head);
> 
> 	if (unlikely(head & 1))
> 		return (struct lpage *)(head - 1);
> 	return (struct lpage *)page;
> }

This is really a "get_head_page()" function, not a "get_large_page()"
function. But even renaming it doesn't seem quite right, because
wouldn't it be better to avoid discarding that tail bit information? In
other words, you might be looking at 3 cases, one of which is *not*
involving large pages at all:

     The page is a single, non-compound page.
     The page is a head page of a compound page
     The page is a tail page of a compound page

...but this function returns a type of "large page", even for the first
case. That's misleading, isn't it?

Given that you've said we could get compile time asserts, I guess you're
not envisioning writing any code that could get the first case at
runtime?

thanks,
-- 
John Hubbard
NVIDIA