Re: [PATCHv2 06/14] mm: Rework compound_head() for power-of-2 sizeof(struct page)

[Muchun Song <muchun.song@linux.dev>]

> On Dec 22, 2025, at 22:03, Kiryl Shutsemau <kas@kernel.org> wrote:
> 
> On Mon, Dec 22, 2025 at 11:20:48AM +0800, Muchun Song wrote:
>> 
>> 
>> On 2025/12/18 23:09, Kiryl Shutsemau wrote:
>>> For tail pages, the kernel uses the 'compound_info' field to get to the
>>> head page. The bit 0 of the field indicates whether the page is a
>>> tail page, and if set, the remaining bits represent a pointer to the
>>> head page.
>>> 
>>> For cases when size of struct page is power-of-2, change the encoding of
>>> compound_info to store a mask that can be applied to the virtual address
>>> of the tail page in order to access the head page. It is possible
>>> because struct page of the head page is naturally aligned with regards
>>> to order of the page.
>>> 
>>> The significant impact of this modification is that all tail pages of
>>> the same order will now have identical 'compound_info', regardless of
>>> the compound page they are associated with. This paves the way for
>>> eliminating fake heads.
>>> 
>>> The HugeTLB Vmemmap Optimization (HVO) creates fake heads and it is only
>>> applied when the sizeof(struct page) is power-of-2. Having identical
>>> tail pages allows the same page to be mapped into the vmemmap of all
>>> pages, maintaining memory savings without fake heads.
>>> 
>>> If sizeof(struct page) is not power-of-2, there is no functional
>>> changes.
>>> 
>>> Signed-off-by: Kiryl Shutsemau <kas@kernel.org>
>> 
>> Reviewed-by: Muchun Song <muchun.song@linux.dev>
>> 
>> One nit bellow.
>> 
>>> ---
>>>  include/linux/page-flags.h | 62 +++++++++++++++++++++++++++++++++-----
>>>  mm/util.c                  | 16 +++++++---
>>>  2 files changed, 66 insertions(+), 12 deletions(-)
>>> 
>>> diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
>>> index 0de7db7efb00..fac5f41b3b27 100644
>>> --- a/include/linux/page-flags.h
>>> +++ b/include/linux/page-flags.h
>>> @@ -210,6 +210,13 @@ static __always_inline const struct page *page_fixed_fake_head(const struct page
>>>   if (!static_branch_unlikely(&hugetlb_optimize_vmemmap_key))
>>>   return page;
>>> + /*
>>> +  * Fake heads only exists if size of struct page is power-of-2.
>>> +  * See hugetlb_vmemmap_optimizable_size().
>>> +  */
>>> + if (!is_power_of_2(sizeof(struct page)))
>>> + return page;
>>> +
>>>   /*
>>>    * Only addresses aligned with PAGE_SIZE of struct page may be fake head
>>>    * struct page. The alignment check aims to avoid access the fields (
>>> @@ -223,10 +230,14 @@ static __always_inline const struct page *page_fixed_fake_head(const struct page
>>>    * because the @page is a compound page composed with at least
>>>    * two contiguous pages.
>>>    */
>>> - unsigned long head = READ_ONCE(page[1].compound_info);
>>> + unsigned long info = READ_ONCE(page[1].compound_info);
>>> - if (likely(head & 1))
>>> - return (const struct page *)(head - 1);
>>> + /* See set_compound_head() */
>>> + if (likely(info & 1)) {
>>> + unsigned long p = (unsigned long)page;
>>> +
>>> + return (const struct page *)(p & info);
>>> + }
>>>   }
>>>   return page;
>>>  }
>>> @@ -281,11 +292,27 @@ static __always_inline int page_is_fake_head(const struct page *page)
>>>  static __always_inline unsigned long _compound_head(const struct page *page)
>>>  {
>>> - unsigned long head = READ_ONCE(page->compound_info);
>>> + unsigned long info = READ_ONCE(page->compound_info);
>>> - if (unlikely(head & 1))
>>> - return head - 1;
>>> - return (unsigned long)page_fixed_fake_head(page);
>>> + /* Bit 0 encodes PageTail() */
>>> + if (!(info & 1))
>>> + return (unsigned long)page_fixed_fake_head(page);
>>> +
>>> + /*
>>> +  * If the size of struct page is not power-of-2, the rest of
>>> +  * compound_info is the pointer to the head page.
>>> +  */
>>> + if (!is_power_of_2(sizeof(struct page)))
>>> + return info - 1;
>>> +
>>> + /*
>>> +  * If the size of struct page is power-of-2 the rest of the info
>>> +  * encodes the mask that converts the address of the tail page to
>>> +  * the head page.
>>> +  *
>>> +  * No need to clear bit 0 in the mask as 'page' always has it clear.
>>> +  */
>>> + return (unsigned long)page & info;
>>>  }
>>>  #define compound_head(page) ((typeof(page))_compound_head(page))
>>> @@ -294,7 +321,26 @@ static __always_inline void set_compound_head(struct page *page,
>>>         const struct page *head,
>>>         unsigned int order)
>>>  {
>>> - WRITE_ONCE(page->compound_info, (unsigned long)head + 1);
>>> + unsigned int shift;
>>> + unsigned long mask;
>>> +
>>> + if (!is_power_of_2(sizeof(struct page))) {
>>> + WRITE_ONCE(page->compound_info, (unsigned long)head | 1);
>>> + return;
>>> + }
>>> +
>>> + /*
>>> +  * If the size of struct page is power-of-2, bits [shift:0] of the
>>> +  * virtual address of compound head are zero.
>>> +  *
>>> +  * Calculate mask that can be applied to the virtual address of
>>> +  * the tail page to get address of the head page.
>>> +  */
>>> + shift = order + order_base_2(sizeof(struct page));
>> 
>> We already have a macro for order_base_2(sizeof(struct page)),
>> that is STRUCT_PAGE_MAX_SHIFT.
> 
> I used it before, but the name is obscure and opencoded version is
> easier to follow in my view.

OK. I'm fine with opencoded version as well.

> 
> -- 
>  Kiryl Shutsemau / Kirill A. Shutemov