[PATCH] mm/gup: fix GUP-fast fallback for NULL-mapping order-0 folios

[PATCH] mm/gup: fix GUP-fast fallback for NULL-mapping order-0 folios

[John Hubbard]

Since commit f002882ca369 ("mm: merge folio_is_secretmem() and
folio_fast_pin_allowed() into gup_fast_folio_allowed()"),
gup_fast_folio_allowed() falls back to the slow path for any order-0
folio with a NULL mapping when CONFIG_SECRETMEM=y. This causes a
performance regression for drivers that allocate pages with alloc_page()
and insert them into VMAs via vm_insert_page(). These pages legitimately
have a NULL folio->mapping, but they cannot be secretmem pages.

Secretmem pages are always added to the secretmem inode's page cache via
filemap_add_folio(), which sets folio->mapping to the inode's i_mapping.
A folio with a NULL mapping can never be a secretmem folio. The
NULL-mapping check was intended to handle truncated file-backed pages (a
reject_file_backed concern), not secretmem detection.

When only check_secretmem is true (and reject_file_backed is false), a
NULL mapping is sufficient to prove the folio is not secretmem, so the
fast path can proceed.

Tested-by: Sourab Gupta <sougupta@nvidia.com>
Fixes: f002882ca369 ("mm: merge folio_is_secretmem() and folio_fast_pin_allowed() into gup_fast_folio_allowed()")
Signed-off-by: John Hubbard <jhubbard@nvidia.com>
---
 mm/gup.c | 13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/mm/gup.c b/mm/gup.c
index 8e7dc2c6ee73..3ea661e67eea 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -2784,12 +2784,17 @@ static bool gup_fast_folio_allowed(struct folio *folio, unsigned int flags)
 	mapping = READ_ONCE(folio->mapping);
 
 	/*
-	 * The mapping may have been truncated, in any case we cannot determine
-	 * if this mapping is safe - fall back to slow path to determine how to
-	 * proceed.
+	 * If the mapping is NULL (truncated, or never set), we cannot
+	 * determine whether the folio is file-backed, so a long-term writable
+	 * pin must fall back to the slow path.
+	 *
+	 * Otherwise, a NULL mapping proves this is not a secretmem folio
+	 * (secretmem folios always have a valid mapping to the secretmem
+	 * inode's address_space), so in that case, we can continue with the
+	 * fast path.
 	 */
 	if (!mapping)
-		return false;
+		return !reject_file_backed;
 
 	/* Anonymous folios pose no problem. */
 	mapping_flags = (unsigned long)mapping & FOLIO_MAPPING_FLAGS;

base-commit: 7f87a5ea75f011d2c9bc8ac0167e5e2d1adb1594
-- 
2.53.0

Re: [PATCH] mm/gup: fix GUP-fast fallback for NULL-mapping order-0 folios

[Andrew Morton]

On Wed,  8 Apr 2026 18:46:47 -0700 John Hubbard <jhubbard@nvidia.com> wrote:

> Since commit f002882ca369 ("mm: merge folio_is_secretmem() and
> folio_fast_pin_allowed() into gup_fast_folio_allowed()"),
> gup_fast_folio_allowed() falls back to the slow path for any order-0
> folio with a NULL mapping when CONFIG_SECRETMEM=y. This causes a
> performance regression for drivers that allocate pages with alloc_page()
> and insert them into VMAs via vm_insert_page(). These pages legitimately
> have a NULL folio->mapping, but they cannot be secretmem pages.

How significant is the slowdown?

> Secretmem pages are always added to the secretmem inode's page cache via
> filemap_add_folio(), which sets folio->mapping to the inode's i_mapping.
> A folio with a NULL mapping can never be a secretmem folio. The
> NULL-mapping check was intended to handle truncated file-backed pages (a
> reject_file_backed concern), not secretmem detection.
> 
> When only check_secretmem is true (and reject_file_backed is false), a
> NULL mapping is sufficient to prove the folio is not secretmem, so the
> fast path can proceed.

Re: [PATCH] mm/gup: fix GUP-fast fallback for NULL-mapping order-0 folios

[John Hubbard]

On 4/8/26 7:07 PM, Andrew Morton wrote:
> On Wed,  8 Apr 2026 18:46:47 -0700 John Hubbard <jhubbard@nvidia.com> wrote:
> 
>> Since commit f002882ca369 ("mm: merge folio_is_secretmem() and
>> folio_fast_pin_allowed() into gup_fast_folio_allowed()"),
>> gup_fast_folio_allowed() falls back to the slow path for any order-0
>> folio with a NULL mapping when CONFIG_SECRETMEM=y. This causes a
>> performance regression for drivers that allocate pages with alloc_page()
>> and insert them into VMAs via vm_insert_page(). These pages legitimately
>> have a NULL folio->mapping, but they cannot be secretmem pages.
> 
> How significant is the slowdown?

About 2x!

Sourab has a bunch of peer to peer IO cases that use 4K pages,
that he ran.


thanks,
-- 
John Hubbard

Re: [PATCH] mm/gup: fix GUP-fast fallback for NULL-mapping order-0 folios

[David Hildenbrand (Arm)]

On 4/9/26 03:46, John Hubbard wrote:
> Since commit f002882ca369 ("mm: merge folio_is_secretmem() and
> folio_fast_pin_allowed() into gup_fast_folio_allowed()"),
> gup_fast_folio_allowed() falls back to the slow path for any order-0
> folio with a NULL mapping when CONFIG_SECRETMEM=y. This causes a
> performance regression for drivers that allocate pages with alloc_page()
> and insert them into VMAs via vm_insert_page(). These pages legitimately
> have a NULL folio->mapping, but they cannot be secretmem pages.
> 
> Secretmem pages are always added to the secretmem inode's page cache via
> filemap_add_folio(), which sets folio->mapping to the inode's i_mapping.
> A folio with a NULL mapping can never be a secretmem folio. The
> NULL-mapping check was intended to handle truncated file-backed pages (a
> reject_file_backed concern), not secretmem detection.
> 
> When only check_secretmem is true (and reject_file_backed is false), a
> NULL mapping is sufficient to prove the folio is not secretmem, so the
> fast path can proceed.

Hm, what if secretmem folio just got truncated? I hate to rely on some
handling in the caller to detect truncation differently during GUP-fast,
but this function returning "true".

Zi is working on a way to distinguish folios from non-folio things: that
we can identify whatever was added through vm_insert_page().

Because that's really the key problem here: vm_insert_page() pages are
not actually folios, they just look like a folio today, but looking at
fields like ->mapping does not make any sense.

> 
> Tested-by: Sourab Gupta <sougupta@nvidia.com>
> Fixes: f002882ca369 ("mm: merge folio_is_secretmem() and folio_fast_pin_allowed() into gup_fast_folio_allowed()")
> Signed-off-by: John Hubbard <jhubbard@nvidia.com>
> ---
>  mm/gup.c | 13 +++++++++----
>  1 file changed, 9 insertions(+), 4 deletions(-)
> 
> diff --git a/mm/gup.c b/mm/gup.c
> index 8e7dc2c6ee73..3ea661e67eea 100644
> --- a/mm/gup.c
> +++ b/mm/gup.c
> @@ -2784,12 +2784,17 @@ static bool gup_fast_folio_allowed(struct folio *folio, unsigned int flags)
>  	mapping = READ_ONCE(folio->mapping);
>  
>  	/*
> -	 * The mapping may have been truncated, in any case we cannot determine
> -	 * if this mapping is safe - fall back to slow path to determine how to
> -	 * proceed.
> +	 * If the mapping is NULL (truncated, or never set), we cannot
> +	 * determine whether the folio is file-backed, so a long-term writable
> +	 * pin must fall back to the slow path.
> +	 *
> +	 * Otherwise, a NULL mapping proves this is not a secretmem folio
> +	 * (secretmem folios always have a valid mapping to the secretmem
> +	 * inode's address_space), so in that case, we can continue with the
> +	 * fast path.
>  	 */
>  	if (!mapping)
> -		return false;
> +		return !reject_file_backed;
>  
>  	/* Anonymous folios pose no problem. */
>  	mapping_flags = (unsigned long)mapping & FOLIO_MAPPING_FLAGS;
> 
> base-commit: 7f87a5ea75f011d2c9bc8ac0167e5e2d1adb1594


-- 
Cheers,

David

Re: [PATCH] mm/gup: fix GUP-fast fallback for NULL-mapping order-0 folios

[Zi Yan]

On 9 Apr 2026, at 3:52, David Hildenbrand (Arm) wrote:

> On 4/9/26 03:46, John Hubbard wrote:
>> Since commit f002882ca369 ("mm: merge folio_is_secretmem() and
>> folio_fast_pin_allowed() into gup_fast_folio_allowed()"),
>> gup_fast_folio_allowed() falls back to the slow path for any order-0
>> folio with a NULL mapping when CONFIG_SECRETMEM=y. This causes a
>> performance regression for drivers that allocate pages with alloc_page()
>> and insert them into VMAs via vm_insert_page(). These pages legitimately
>> have a NULL folio->mapping, but they cannot be secretmem pages.
>>
>> Secretmem pages are always added to the secretmem inode's page cache via
>> filemap_add_folio(), which sets folio->mapping to the inode's i_mapping.
>> A folio with a NULL mapping can never be a secretmem folio. The
>> NULL-mapping check was intended to handle truncated file-backed pages (a
>> reject_file_backed concern), not secretmem detection.
>>
>> When only check_secretmem is true (and reject_file_backed is false), a
>> NULL mapping is sufficient to prove the folio is not secretmem, so the
>> fast path can proceed.
>
> Hm, what if secretmem folio just got truncated? I hate to rely on some
> handling in the caller to detect truncation differently during GUP-fast,
> but this function returning "true".
>
> Zi is working on a way to distinguish folios from non-folio things: that
> we can identify whatever was added through vm_insert_page().

The idea is to give every allocated page a NotRmappable page_type and
1) let page_rmappable_folio() remove the page_type by setting mapcount
to 0, since page_type overlaps with mapcount;
2) allow callers of alloc_page*() to change page_type from NotRmappable
to other types.

After these, rmappable folios with any order is a folio without any
page_type. HugeTLB is an exception, but rmap code has special handling
for them, so it should be fine.

I have a draft[1], but have not had time to refresh it yet. I am planning
to work on this after my two other jobs (make READ_ONLY_THP_FOR_FS enabled
for all FS with large folio support and ensure ->private is 0 for tail pages).

[1] https://github.com/x-y-z/linux-dev/tree/remove_mapcount_for_non_folio-2026-02-16-23-27

>
> Because that's really the key problem here: vm_insert_page() pages are
> not actually folios, they just look like a folio today, but looking at
> fields like ->mapping does not make any sense.

Based on my understanding of some callers of alloc_page()/get_free_pages(),
vm_insert_page(), they use GFP_COMP to get a compound page and vm_insert_page()
individual pages of the compound page. These pages might be pinned.
During past THP cabal meeting, we discussed about it if
1. we should disallow such uses and only allow order-0 allocations, or
2. we need a new type like new_type_folio for them and the type does not
have folio fields except ->pincount.

Willy (cc’d) said he coded some experiment on it some time ago. I do not know if
he is able to recover the code.

>
>>
>> Tested-by: Sourab Gupta <sougupta@nvidia.com>
>> Fixes: f002882ca369 ("mm: merge folio_is_secretmem() and folio_fast_pin_allowed() into gup_fast_folio_allowed()")
>> Signed-off-by: John Hubbard <jhubbard@nvidia.com>
>> ---
>>  mm/gup.c | 13 +++++++++----
>>  1 file changed, 9 insertions(+), 4 deletions(-)
>>
>> diff --git a/mm/gup.c b/mm/gup.c
>> index 8e7dc2c6ee73..3ea661e67eea 100644
>> --- a/mm/gup.c
>> +++ b/mm/gup.c
>> @@ -2784,12 +2784,17 @@ static bool gup_fast_folio_allowed(struct folio *folio, unsigned int flags)
>>  	mapping = READ_ONCE(folio->mapping);
>>
>>  	/*
>> -	 * The mapping may have been truncated, in any case we cannot determine
>> -	 * if this mapping is safe - fall back to slow path to determine how to
>> -	 * proceed.
>> +	 * If the mapping is NULL (truncated, or never set), we cannot
>> +	 * determine whether the folio is file-backed, so a long-term writable
>> +	 * pin must fall back to the slow path.
>> +	 *
>> +	 * Otherwise, a NULL mapping proves this is not a secretmem folio
>> +	 * (secretmem folios always have a valid mapping to the secretmem
>> +	 * inode's address_space), so in that case, we can continue with the
>> +	 * fast path.
>>  	 */
>>  	if (!mapping)
>> -		return false;
>> +		return !reject_file_backed;
>>
>>  	/* Anonymous folios pose no problem. */
>>  	mapping_flags = (unsigned long)mapping & FOLIO_MAPPING_FLAGS;
>>
>> base-commit: 7f87a5ea75f011d2c9bc8ac0167e5e2d1adb1594
>
>
> -- 
> Cheers,
>
> David


Best Regards,
Yan, Zi

Re: [PATCH] mm/gup: fix GUP-fast fallback for NULL-mapping order-0 folios

[David Hildenbrand (Arm)]

On 4/9/26 17:05, Zi Yan wrote:
> On 9 Apr 2026, at 3:52, David Hildenbrand (Arm) wrote:
> 
>> On 4/9/26 03:46, John Hubbard wrote:
>>> Since commit f002882ca369 ("mm: merge folio_is_secretmem() and
>>> folio_fast_pin_allowed() into gup_fast_folio_allowed()"),
>>> gup_fast_folio_allowed() falls back to the slow path for any order-0
>>> folio with a NULL mapping when CONFIG_SECRETMEM=y. This causes a
>>> performance regression for drivers that allocate pages with alloc_page()
>>> and insert them into VMAs via vm_insert_page(). These pages legitimately
>>> have a NULL folio->mapping, but they cannot be secretmem pages.
>>>
>>> Secretmem pages are always added to the secretmem inode's page cache via
>>> filemap_add_folio(), which sets folio->mapping to the inode's i_mapping.
>>> A folio with a NULL mapping can never be a secretmem folio. The
>>> NULL-mapping check was intended to handle truncated file-backed pages (a
>>> reject_file_backed concern), not secretmem detection.
>>>
>>> When only check_secretmem is true (and reject_file_backed is false), a
>>> NULL mapping is sufficient to prove the folio is not secretmem, so the
>>> fast path can proceed.
>>
>> Hm, what if secretmem folio just got truncated? I hate to rely on some
>> handling in the caller to detect truncation differently during GUP-fast,
>> but this function returning "true".
>>
>> Zi is working on a way to distinguish folios from non-folio things: that
>> we can identify whatever was added through vm_insert_page().
> 
> The idea is to give every allocated page a NotRmappable page_type and
> 1) let page_rmappable_folio() remove the page_type by setting mapcount
> to 0, since page_type overlaps with mapcount;
> 2) allow callers of alloc_page*() to change page_type from NotRmappable
> to other types.

Could we instead just set the special type in vm_insert_page(), after
verifying that it does not have any other type?

> 
> After these, rmappable folios with any order is a folio without any
> page_type. HugeTLB is an exception, but rmap code has special handling
> for them, so it should be fine.
> 
> I have a draft[1], but have not had time to refresh it yet. I am planning
> to work on this after my two other jobs (make READ_ONLY_THP_FOR_FS enabled
> for all FS with large folio support and ensure ->private is 0 for tail pages).
> 
> [1] https://github.com/x-y-z/linux-dev/tree/remove_mapcount_for_non_folio-2026-02-16-23-27
> 
>>
>> Because that's really the key problem here: vm_insert_page() pages are
>> not actually folios, they just look like a folio today, but looking at
>> fields like ->mapping does not make any sense.
> 
> Based on my understanding of some callers of alloc_page()/get_free_pages(),
> vm_insert_page(), they use GFP_COMP to get a compound page and vm_insert_page()
> individual pages of the compound page. These pages might be pinned.
> During past THP cabal meeting, we discussed about it if
> 1. we should disallow such uses and only allow order-0 allocations, or

Right.

> 2. we need a new type like new_type_folio for them and the type does not
> have folio fields except ->pincount.

Right.

-- 
Cheers,

David

Re: [PATCH] mm/gup: fix GUP-fast fallback for NULL-mapping order-0 folios

[Zi Yan]

On 9 Apr 2026, at 11:11, David Hildenbrand (Arm) wrote:

> On 4/9/26 17:05, Zi Yan wrote:
>> On 9 Apr 2026, at 3:52, David Hildenbrand (Arm) wrote:
>>
>>> On 4/9/26 03:46, John Hubbard wrote:
>>>> Since commit f002882ca369 ("mm: merge folio_is_secretmem() and
>>>> folio_fast_pin_allowed() into gup_fast_folio_allowed()"),
>>>> gup_fast_folio_allowed() falls back to the slow path for any order-0
>>>> folio with a NULL mapping when CONFIG_SECRETMEM=y. This causes a
>>>> performance regression for drivers that allocate pages with alloc_page()
>>>> and insert them into VMAs via vm_insert_page(). These pages legitimately
>>>> have a NULL folio->mapping, but they cannot be secretmem pages.
>>>>
>>>> Secretmem pages are always added to the secretmem inode's page cache via
>>>> filemap_add_folio(), which sets folio->mapping to the inode's i_mapping.
>>>> A folio with a NULL mapping can never be a secretmem folio. The
>>>> NULL-mapping check was intended to handle truncated file-backed pages (a
>>>> reject_file_backed concern), not secretmem detection.
>>>>
>>>> When only check_secretmem is true (and reject_file_backed is false), a
>>>> NULL mapping is sufficient to prove the folio is not secretmem, so the
>>>> fast path can proceed.
>>>
>>> Hm, what if secretmem folio just got truncated? I hate to rely on some
>>> handling in the caller to detect truncation differently during GUP-fast,
>>> but this function returning "true".
>>>
>>> Zi is working on a way to distinguish folios from non-folio things: that
>>> we can identify whatever was added through vm_insert_page().
>>
>> The idea is to give every allocated page a NotRmappable page_type and
>> 1) let page_rmappable_folio() remove the page_type by setting mapcount
>> to 0, since page_type overlaps with mapcount;
>> 2) allow callers of alloc_page*() to change page_type from NotRmappable
>> to other types.
>
> Could we instead just set the special type in vm_insert_page(), after
> verifying that it does not have any other type?

You mean in validate_page_before_insert() we set the page to NotRmappable
if it can be inserted, then in the rmap code we reject NotRmappable
page/folios? And NotRmappable will remain until the page/folio is freed.
That should work. So the purpose is to avoid setting the page_type if
a page/folio is never going through vm_insert_page()?

>
>>
>> After these, rmappable folios with any order is a folio without any
>> page_type. HugeTLB is an exception, but rmap code has special handling
>> for them, so it should be fine.
>>
>> I have a draft[1], but have not had time to refresh it yet. I am planning
>> to work on this after my two other jobs (make READ_ONLY_THP_FOR_FS enabled
>> for all FS with large folio support and ensure ->private is 0 for tail pages).
>>
>> [1] https://github.com/x-y-z/linux-dev/tree/remove_mapcount_for_non_folio-2026-02-16-23-27
>>
>>>
>>> Because that's really the key problem here: vm_insert_page() pages are
>>> not actually folios, they just look like a folio today, but looking at
>>> fields like ->mapping does not make any sense.
>>
>> Based on my understanding of some callers of alloc_page()/get_free_pages(),
>> vm_insert_page(), they use GFP_COMP to get a compound page and vm_insert_page()
>> individual pages of the compound page. These pages might be pinned.
>> During past THP cabal meeting, we discussed about it if
>> 1. we should disallow such uses and only allow order-0 allocations, or
>
> Right.
>
>> 2. we need a new type like new_type_folio for them and the type does not
>> have folio fields except ->pincount.
>
> Right.
>
> -- 
> Cheers,
>
> David


Best Regards,
Yan, Zi

Re: [PATCH] mm/gup: fix GUP-fast fallback for NULL-mapping order-0 folios

[David Hildenbrand (Arm)]

On 4/9/26 18:55, Zi Yan wrote:
> On 9 Apr 2026, at 11:11, David Hildenbrand (Arm) wrote:
> 
>> On 4/9/26 17:05, Zi Yan wrote:
>>>
>>>
>>> The idea is to give every allocated page a NotRmappable page_type and
>>> 1) let page_rmappable_folio() remove the page_type by setting mapcount
>>> to 0, since page_type overlaps with mapcount;
>>> 2) allow callers of alloc_page*() to change page_type from NotRmappable
>>> to other types.
>>
>> Could we instead just set the special type in vm_insert_page(), after
>> verifying that it does not have any other type?
> 
> You mean in validate_page_before_insert() we set the page to NotRmappable
> if it can be inserted, then in the rmap code we reject NotRmappable
> page/folios? And NotRmappable will remain until the page/folio is freed.
> That should work. So the purpose is to avoid setting the page_type if
> a page/folio is never going through vm_insert_page()?

Yes, only these non-folio pages will show up process page tables.

We would add a safety check that the mapcount really is 0 (-1) or
already the proper type (NotRmappable).

Rmap code would then simply skip any such thing.

-- 
Cheers,

David

Re: [PATCH] mm/gup: fix GUP-fast fallback for NULL-mapping order-0 folios

[Zi Yan]

On 9 Apr 2026, at 13:31, David Hildenbrand (Arm) wrote:

> On 4/9/26 18:55, Zi Yan wrote:
>> On 9 Apr 2026, at 11:11, David Hildenbrand (Arm) wrote:
>>
>>> On 4/9/26 17:05, Zi Yan wrote:
>>>>
>>>>
>>>> The idea is to give every allocated page a NotRmappable page_type and
>>>> 1) let page_rmappable_folio() remove the page_type by setting mapcount
>>>> to 0, since page_type overlaps with mapcount;
>>>> 2) allow callers of alloc_page*() to change page_type from NotRmappable
>>>> to other types.
>>>
>>> Could we instead just set the special type in vm_insert_page(), after
>>> verifying that it does not have any other type?
>>
>> You mean in validate_page_before_insert() we set the page to NotRmappable
>> if it can be inserted, then in the rmap code we reject NotRmappable
>> page/folios? And NotRmappable will remain until the page/folio is freed.
>> That should work. So the purpose is to avoid setting the page_type if
>> a page/folio is never going through vm_insert_page()?
>
> Yes, only these non-folio pages will show up process page tables.
>
> We would add a safety check that the mapcount really is 0 (-1) or
> already the proper type (NotRmappable).
>
> Rmap code would then simply skip any such thing.

OK, I can give this a try in my new version.

Best Regards,
Yan, Zi