Re: [PATCHv2 06/14] mm: Rework compound_head() for power-of-2 sizeof(struct page)

[Kiryl Shutsemau <kas@kernel.org>]

On Mon, Dec 22, 2025 at 11:20:48AM +0800, Muchun Song wrote:
> 
> 
> On 2025/12/18 23:09, Kiryl Shutsemau wrote:
> > For tail pages, the kernel uses the 'compound_info' field to get to the
> > head page. The bit 0 of the field indicates whether the page is a
> > tail page, and if set, the remaining bits represent a pointer to the
> > head page.
> > 
> > For cases when size of struct page is power-of-2, change the encoding of
> > compound_info to store a mask that can be applied to the virtual address
> > of the tail page in order to access the head page. It is possible
> > because struct page of the head page is naturally aligned with regards
> > to order of the page.
> > 
> > The significant impact of this modification is that all tail pages of
> > the same order will now have identical 'compound_info', regardless of
> > the compound page they are associated with. This paves the way for
> > eliminating fake heads.
> > 
> > The HugeTLB Vmemmap Optimization (HVO) creates fake heads and it is only
> > applied when the sizeof(struct page) is power-of-2. Having identical
> > tail pages allows the same page to be mapped into the vmemmap of all
> > pages, maintaining memory savings without fake heads.
> > 
> > If sizeof(struct page) is not power-of-2, there is no functional
> > changes.
> > 
> > Signed-off-by: Kiryl Shutsemau <kas@kernel.org>
> 
> Reviewed-by: Muchun Song <muchun.song@linux.dev>
> 
> One nit bellow.
> 
> > ---
> >   include/linux/page-flags.h | 62 +++++++++++++++++++++++++++++++++-----
> >   mm/util.c                  | 16 +++++++---
> >   2 files changed, 66 insertions(+), 12 deletions(-)
> > 
> > diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
> > index 0de7db7efb00..fac5f41b3b27 100644
> > --- a/include/linux/page-flags.h
> > +++ b/include/linux/page-flags.h
> > @@ -210,6 +210,13 @@ static __always_inline const struct page *page_fixed_fake_head(const struct page
> >   	if (!static_branch_unlikely(&hugetlb_optimize_vmemmap_key))
> >   		return page;
> > +	/*
> > +	 * Fake heads only exists if size of struct page is power-of-2.
> > +	 * See hugetlb_vmemmap_optimizable_size().
> > +	 */
> > +	if (!is_power_of_2(sizeof(struct page)))
> > +		return page;
> > +
> >   	/*
> >   	 * Only addresses aligned with PAGE_SIZE of struct page may be fake head
> >   	 * struct page. The alignment check aims to avoid access the fields (
> > @@ -223,10 +230,14 @@ static __always_inline const struct page *page_fixed_fake_head(const struct page
> >   		 * because the @page is a compound page composed with at least
> >   		 * two contiguous pages.
> >   		 */
> > -		unsigned long head = READ_ONCE(page[1].compound_info);
> > +		unsigned long info = READ_ONCE(page[1].compound_info);
> > -		if (likely(head & 1))
> > -			return (const struct page *)(head - 1);
> > +		/* See set_compound_head() */
> > +		if (likely(info & 1)) {
> > +			unsigned long p = (unsigned long)page;
> > +
> > +			return (const struct page *)(p & info);
> > +		}
> >   	}
> >   	return page;
> >   }
> > @@ -281,11 +292,27 @@ static __always_inline int page_is_fake_head(const struct page *page)
> >   static __always_inline unsigned long _compound_head(const struct page *page)
> >   {
> > -	unsigned long head = READ_ONCE(page->compound_info);
> > +	unsigned long info = READ_ONCE(page->compound_info);
> > -	if (unlikely(head & 1))
> > -		return head - 1;
> > -	return (unsigned long)page_fixed_fake_head(page);
> > +	/* Bit 0 encodes PageTail() */
> > +	if (!(info & 1))
> > +		return (unsigned long)page_fixed_fake_head(page);
> > +
> > +	/*
> > +	 * If the size of struct page is not power-of-2, the rest of
> > +	 * compound_info is the pointer to the head page.
> > +	 */
> > +	if (!is_power_of_2(sizeof(struct page)))
> > +		return info - 1;
> > +
> > +	/*
> > +	 * If the size of struct page is power-of-2 the rest of the info
> > +	 * encodes the mask that converts the address of the tail page to
> > +	 * the head page.
> > +	 *
> > +	 * No need to clear bit 0 in the mask as 'page' always has it clear.
> > +	 */
> > +	return (unsigned long)page & info;
> >   }
> >   #define compound_head(page)	((typeof(page))_compound_head(page))
> > @@ -294,7 +321,26 @@ static __always_inline void set_compound_head(struct page *page,
> >   					      const struct page *head,
> >   					      unsigned int order)
> >   {
> > -	WRITE_ONCE(page->compound_info, (unsigned long)head + 1);
> > +	unsigned int shift;
> > +	unsigned long mask;
> > +
> > +	if (!is_power_of_2(sizeof(struct page))) {
> > +		WRITE_ONCE(page->compound_info, (unsigned long)head | 1);
> > +		return;
> > +	}
> > +
> > +	/*
> > +	 * If the size of struct page is power-of-2, bits [shift:0] of the
> > +	 * virtual address of compound head are zero.
> > +	 *
> > +	 * Calculate mask that can be applied to the virtual address of
> > +	 * the tail page to get address of the head page.
> > +	 */
> > +	shift = order + order_base_2(sizeof(struct page));
> 
> We already have a macro for order_base_2(sizeof(struct page)),
> that is STRUCT_PAGE_MAX_SHIFT.

I used it before, but the name is obscure and opencoded version is
easier to follow in my view.

-- 
  Kiryl Shutsemau / Kirill A. Shutemov