[PATCH v2] mm: prohibit the last subpage from reusing the entire large folio

[PATCH v2] mm: prohibit the last subpage from reusing the entire large folio

[Barry Song]

From: Barry Song <v-songbaohua@oppo.com>

In a Copy-on-Write (CoW) scenario, the last subpage will reuse the entire
large folio, resulting in the waste of (nr_pages - 1) pages. This wasted
memory remains allocated until it is either unmapped or memory
reclamation occurs.

The following small program can serve as evidence of this behavior

 main()
 {
 #define SIZE 1024 * 1024 * 1024UL
         void *p = malloc(SIZE);
         memset(p, 0x11, SIZE);
         if (fork() == 0)
                 _exit(0);
         memset(p, 0x12, SIZE);
         printf("done\n");
         while(1);
 }

For example, using a 1024KiB mTHP by:
 echo always > /sys/kernel/mm/transparent_hugepage/hugepages-1024kB/enabled

(1) w/o the patch, it takes 2GiB,

Before running the test program,
 / # free -m
                total        used        free      shared  buff/cache   available
 Mem:            5754          84        5692           0          17        5669
 Swap:              0           0           0

 / # /a.out &
 / # done

After running the test program,
 / # free -m
                 total        used        free      shared  buff/cache   available
 Mem:            5754        2149        3627           0          19        3605
 Swap:              0           0           0

(2) w/ the patch, it takes 1GiB only,

Before running the test program,
 / # free -m
                 total        used        free      shared  buff/cache   available
 Mem:            5754          89        5687           0          17        5664
 Swap:              0           0           0

 / # /a.out &
 / # done

After running the test program,
 / # free -m
                total        used        free      shared  buff/cache   available
 Mem:            5754        1122        4655           0          17        4632
 Swap:              0           0           0

This patch migrates the last subpage to a small folio and immediately
returns the large folio to the system. It benefits both memory availability
and anti-fragmentation.

Cc: David Hildenbrand <david@redhat.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Lance Yang <ioworker0@gmail.com>
Signed-off-by: Barry Song <v-songbaohua@oppo.com>
---
 -v2:
  * return at the 1st beginning for a large folio according to David's comment,
    thanks!
 -v1:
 https://lore.kernel.org/linux-mm/20240308085653.124180-1-21cnbao@gmail.com/

 mm/memory.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/mm/memory.c b/mm/memory.c
index e17669d4f72f..f2bc6dd15eb8 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -3498,6 +3498,16 @@ static vm_fault_t wp_page_shared(struct vm_fault *vmf, struct folio *folio)
 static bool wp_can_reuse_anon_folio(struct folio *folio,
 				    struct vm_area_struct *vma)
 {
+	/*
+	 * We could currently only reuse a subpage of a large folio if no
+	 * other subpages of the large folios are still mapped. However,
+	 * let's just consistently not reuse subpages even if we could
+	 * reuse in that scenario, and give back a large folio a bit
+	 * sooner.
+	 */
+	if (folio_test_large(folio))
+		return false;
+
 	/*
 	 * We have to verify under folio lock: these early checks are
 	 * just an optimization to avoid locking the folio and freeing
-- 
2.34.1

Re: [PATCH v2] mm: prohibit the last subpage from reusing the entire large folio

[David Hildenbrand]

>>> This patch migrates the last subpage to a small folio and immediately
>>> returns the large folio to the system. It benefits both memory availability
>>> and anti-fragmentation.
>>
>> It might be controversial optimization, and as Ryan said, there, are likely
>> other cases where we'd want to migrate off-of a thp if possible earlier.
> 
> Personally, I think there might also be cases where you want to copy/reuse the
> entire large folio. If you're application is using 16K THPs perhaps it's a
> bigger win to just treat it like a base page? I expect the cost/benefit will
> change as the THP size increases?

Yes, I think for small folios (i.e., 16KiB) it will be rather easy to 
make a decision. The larger the folio, the larger the page fault latency 
due to scanning, copying, modifying, which can easily turn undesirable.

At least when it comes to page reuse, I have some simple backup plans 
for small folios if I won't be able to make progress with my other 
approach. For larger folios, it won't really work/be desirable, though.

-- 
Cheers,

David / dhildenb

Re: [PATCH v2] mm: prohibit the last subpage from reusing the entire large folio

[Ryan Roberts]

On 08/03/2024 13:24, David Hildenbrand wrote:
>>>> This patch migrates the last subpage to a small folio and immediately
>>>> returns the large folio to the system. It benefits both memory availability
>>>> and anti-fragmentation.
>>>
>>> It might be controversial optimization, and as Ryan said, there, are likely
>>> other cases where we'd want to migrate off-of a thp if possible earlier.
>>
>> Personally, I think there might also be cases where you want to copy/reuse the
>> entire large folio. If you're application is using 16K THPs perhaps it's a
>> bigger win to just treat it like a base page? I expect the cost/benefit will
>> change as the THP size increases?
> 
> Yes, I think for small folios (i.e., 16KiB) it will be rather easy to make a
> decision. The larger the folio, the larger the page fault latency due to
> scanning, copying, modifying, which can easily turn undesirable.
> 
> At least when it comes to page reuse, I have some simple backup plans for small
> folios if I won't be able to make progress with my other approach. 

Do you mean "small large folios" here? i.e. order >= 1? If so, great!


For larger
> folios, it won't really work/be desirable, though.
> 

Re: [PATCH v2] mm: prohibit the last subpage from reusing the entire large folio

[David Hildenbrand]

On 08.03.24 14:45, Ryan Roberts wrote:
> On 08/03/2024 13:24, David Hildenbrand wrote:
>>>>> This patch migrates the last subpage to a small folio and immediately
>>>>> returns the large folio to the system. It benefits both memory availability
>>>>> and anti-fragmentation.
>>>>
>>>> It might be controversial optimization, and as Ryan said, there, are likely
>>>> other cases where we'd want to migrate off-of a thp if possible earlier.
>>>
>>> Personally, I think there might also be cases where you want to copy/reuse the
>>> entire large folio. If you're application is using 16K THPs perhaps it's a
>>> bigger win to just treat it like a base page? I expect the cost/benefit will
>>> change as the THP size increases?
>>
>> Yes, I think for small folios (i.e., 16KiB) it will be rather easy to make a
>> decision. The larger the folio, the larger the page fault latency due to
>> scanning, copying, modifying, which can easily turn undesirable.
>>
>> At least when it comes to page reuse, I have some simple backup plans for small
>> folios if I won't be able to make progress with my other approach.
> 
> Do you mean "small large folios" here? i.e. order >= 1? If so, great!

*smaller*, yes :)

-- 
Cheers,

David / dhildenb