[PATCH] mm/mmu_gather: define RCU version tlb_remove_table_one() in CONFIG_MMU_GATHER_RCU_TABLE_FREE

[PATCH] mm/mmu_gather: define RCU version tlb_remove_table_one() in CONFIG_MMU_GATHER_RCU_TABLE_FREE

[Wei Yang]

Before commit 718b13861d22 ("x86: mm: free page table pages by RCU
instead of semi RCU"), the page table pages will be freed by semi RCU,
that is:

  - batch table freeing: asynchronous free by RCU
  - single table freeing: IPI + synchronous free

This commit introduce a RCU version for single table freeing to support
PT_RECLAIM. While it is not necessary to limit the RCU version only in
CONFIG_PT_RECLAIM. It is reasonable to asynchronous free single table by
RCU if CONFIG_MMU_GATHER_RCU_TABLE_FREE.

This patch moves the definition of single table freeing in
CONFIG_MMU_GATHER_RCU_TABLE_FREE, so we have both RCU version
batch/single table freeing if CONFIG_MMU_GATHER_RCU_TABLE_FREE.

Signed-off-by: Wei Yang <richard.weiyang@gmail.com>
Cc: Qi Zheng <qi.zheng@linux.dev>
Cc: "David Hildenbrand (Arm)" <david@kernel.org>
---
 mm/mmu_gather.c | 51 +++++++++++++++++++++----------------------------
 1 file changed, 22 insertions(+), 29 deletions(-)

diff --git a/mm/mmu_gather.c b/mm/mmu_gather.c
index fe5b6a031717..6c1cbd9c717b 100644
--- a/mm/mmu_gather.c
+++ b/mm/mmu_gather.c
@@ -296,6 +296,22 @@ static void tlb_remove_table_free(struct mmu_table_batch *batch)
 	call_rcu(&batch->rcu, tlb_remove_table_rcu);
 }
 
+static inline void __tlb_remove_table_one_rcu(struct rcu_head *head)
+{
+	struct ptdesc *ptdesc;
+
+	ptdesc = container_of(head, struct ptdesc, pt_rcu_head);
+	__tlb_remove_table(ptdesc);
+}
+
+static inline void tlb_remove_table_one(void *table)
+{
+	struct ptdesc *ptdesc;
+
+	ptdesc = table;
+	call_rcu(&ptdesc->pt_rcu_head, __tlb_remove_table_one_rcu);
+}
+
 #else /* !CONFIG_MMU_GATHER_RCU_TABLE_FREE */
 
 static void tlb_remove_table_free(struct mmu_table_batch *batch)
@@ -303,6 +319,12 @@ static void tlb_remove_table_free(struct mmu_table_batch *batch)
 	__tlb_remove_table_free(batch);
 }
 
+static inline void tlb_remove_table_one(void *table)
+{
+	tlb_remove_table_sync_one();
+	__tlb_remove_table(table);
+}
+
 #endif /* CONFIG_MMU_GATHER_RCU_TABLE_FREE */
 
 /*
@@ -320,35 +342,6 @@ static inline void tlb_table_invalidate(struct mmu_gather *tlb)
 	}
 }
 
-#ifdef CONFIG_PT_RECLAIM
-static inline void __tlb_remove_table_one_rcu(struct rcu_head *head)
-{
-	struct ptdesc *ptdesc;
-
-	ptdesc = container_of(head, struct ptdesc, pt_rcu_head);
-	__tlb_remove_table(ptdesc);
-}
-
-static inline void __tlb_remove_table_one(void *table)
-{
-	struct ptdesc *ptdesc;
-
-	ptdesc = table;
-	call_rcu(&ptdesc->pt_rcu_head, __tlb_remove_table_one_rcu);
-}
-#else
-static inline void __tlb_remove_table_one(void *table)
-{
-	tlb_remove_table_sync_one();
-	__tlb_remove_table(table);
-}
-#endif /* CONFIG_PT_RECLAIM */
-
-static void tlb_remove_table_one(void *table)
-{
-	__tlb_remove_table_one(table);
-}
-
 static void tlb_table_flush(struct mmu_gather *tlb)
 {
 	struct mmu_table_batch **batch = &tlb->batch;
-- 
2.34.1

Re: [PATCH] mm/mmu_gather: define RCU version tlb_remove_table_one() in CONFIG_MMU_GATHER_RCU_TABLE_FREE

[Andrew Morton]

On Sun, 22 Feb 2026 00:45:12 +0000 Wei Yang <richard.weiyang@gmail.com> wrote:

> Before commit 718b13861d22 ("x86: mm: free page table pages by RCU
> instead of semi RCU"), the page table pages will be freed by semi RCU,
> that is:
> 
>   - batch table freeing: asynchronous free by RCU
>   - single table freeing: IPI + synchronous free
> 
> This commit introduce a RCU version for single table freeing to support
> PT_RECLAIM. While it is not necessary to limit the RCU version only in
> CONFIG_PT_RECLAIM. It is reasonable to asynchronous free single table by
> RCU if CONFIG_MMU_GATHER_RCU_TABLE_FREE.
> 
> This patch moves the definition of single table freeing in
> CONFIG_MMU_GATHER_RCU_TABLE_FREE, so we have both RCU version
> batch/single table freeing if CONFIG_MMU_GATHER_RCU_TABLE_FREE.

What are the expected (and measured!) benefits of this?

Re: [PATCH] mm/mmu_gather: define RCU version tlb_remove_table_one() in CONFIG_MMU_GATHER_RCU_TABLE_FREE

[Wei Yang]

On Sun, Feb 22, 2026 at 11:42:01AM -0800, Andrew Morton wrote:
>On Sun, 22 Feb 2026 00:45:12 +0000 Wei Yang <richard.weiyang@gmail.com> wrote:
>
>> Before commit 718b13861d22 ("x86: mm: free page table pages by RCU
>> instead of semi RCU"), the page table pages will be freed by semi RCU,
>> that is:
>> 
>>   - batch table freeing: asynchronous free by RCU
>>   - single table freeing: IPI + synchronous free
>> 
>> This commit introduce a RCU version for single table freeing to support
>> PT_RECLAIM. While it is not necessary to limit the RCU version only in
>> CONFIG_PT_RECLAIM. It is reasonable to asynchronous free single table by
>> RCU if CONFIG_MMU_GATHER_RCU_TABLE_FREE.
>> 
>> This patch moves the definition of single table freeing in
>> CONFIG_MMU_GATHER_RCU_TABLE_FREE, so we have both RCU version
>> batch/single table freeing if CONFIG_MMU_GATHER_RCU_TABLE_FREE.
>
>What are the expected (and measured!) benefits of this?
>

No, this is just a cleanup.

Last time checked with Qi and thought it worth a formal patch. [1]

[1]: http://lkml.kernel.org/r/fd90c84b-9829-4699-b7df-c43020519ec9@linux.dev

-- 
Wei Yang
Help you, Help me

Re: [PATCH] mm/mmu_gather: define RCU version tlb_remove_table_one() in CONFIG_MMU_GATHER_RCU_TABLE_FREE

[Qi Zheng]

Hi Wei,

On 2/22/26 8:45 AM, Wei Yang wrote:
> Before commit 718b13861d22 ("x86: mm: free page table pages by RCU
> instead of semi RCU"), the page table pages will be freed by semi RCU,
> that is:
> 
>    - batch table freeing: asynchronous free by RCU
>    - single table freeing: IPI + synchronous free
> 
> This commit introduce a RCU version for single table freeing to support
> PT_RECLAIM. While it is not necessary to limit the RCU version only in
> CONFIG_PT_RECLAIM. It is reasonable to asynchronous free single table by
> RCU if CONFIG_MMU_GATHER_RCU_TABLE_FREE.
> 
> This patch moves the definition of single table freeing in
> CONFIG_MMU_GATHER_RCU_TABLE_FREE, so we have both RCU version
> batch/single table freeing if CONFIG_MMU_GATHER_RCU_TABLE_FREE.

Thank you for doing this, but we can't do it right now.

See the latest version: 
https://github.com/torvalds/linux/commit/9c8c02df3f8742f6db927e787ab971fd0b5ac08a, 
when the
HAVE_ARCH_TLB_REMOVE_TABLE is selected, the pt_rcu_head member
cannot be used.

Thanks,
Qi

> 
> Signed-off-by: Wei Yang <richard.weiyang@gmail.com>
> Cc: Qi Zheng <qi.zheng@linux.dev>
> Cc: "David Hildenbrand (Arm)" <david@kernel.org>
> ---
>   mm/mmu_gather.c | 51 +++++++++++++++++++++----------------------------
>   1 file changed, 22 insertions(+), 29 deletions(-)
> 
> diff --git a/mm/mmu_gather.c b/mm/mmu_gather.c
> index fe5b6a031717..6c1cbd9c717b 100644
> --- a/mm/mmu_gather.c
> +++ b/mm/mmu_gather.c
> @@ -296,6 +296,22 @@ static void tlb_remove_table_free(struct mmu_table_batch *batch)
>   	call_rcu(&batch->rcu, tlb_remove_table_rcu);
>   }
>   
> +static inline void __tlb_remove_table_one_rcu(struct rcu_head *head)
> +{
> +	struct ptdesc *ptdesc;
> +
> +	ptdesc = container_of(head, struct ptdesc, pt_rcu_head);
> +	__tlb_remove_table(ptdesc);
> +}
> +
> +static inline void tlb_remove_table_one(void *table)
> +{
> +	struct ptdesc *ptdesc;
> +
> +	ptdesc = table;
> +	call_rcu(&ptdesc->pt_rcu_head, __tlb_remove_table_one_rcu);
> +}
> +
>   #else /* !CONFIG_MMU_GATHER_RCU_TABLE_FREE */
>   
>   static void tlb_remove_table_free(struct mmu_table_batch *batch)
> @@ -303,6 +319,12 @@ static void tlb_remove_table_free(struct mmu_table_batch *batch)
>   	__tlb_remove_table_free(batch);
>   }
>   
> +static inline void tlb_remove_table_one(void *table)
> +{
> +	tlb_remove_table_sync_one();
> +	__tlb_remove_table(table);
> +}
> +
>   #endif /* CONFIG_MMU_GATHER_RCU_TABLE_FREE */
>   
>   /*
> @@ -320,35 +342,6 @@ static inline void tlb_table_invalidate(struct mmu_gather *tlb)
>   	}
>   }
>   
> -#ifdef CONFIG_PT_RECLAIM
> -static inline void __tlb_remove_table_one_rcu(struct rcu_head *head)
> -{
> -	struct ptdesc *ptdesc;
> -
> -	ptdesc = container_of(head, struct ptdesc, pt_rcu_head);
> -	__tlb_remove_table(ptdesc);
> -}
> -
> -static inline void __tlb_remove_table_one(void *table)
> -{
> -	struct ptdesc *ptdesc;
> -
> -	ptdesc = table;
> -	call_rcu(&ptdesc->pt_rcu_head, __tlb_remove_table_one_rcu);
> -}
> -#else
> -static inline void __tlb_remove_table_one(void *table)
> -{
> -	tlb_remove_table_sync_one();
> -	__tlb_remove_table(table);
> -}
> -#endif /* CONFIG_PT_RECLAIM */
> -
> -static void tlb_remove_table_one(void *table)
> -{
> -	__tlb_remove_table_one(table);
> -}
> -
>   static void tlb_table_flush(struct mmu_gather *tlb)
>   {
>   	struct mmu_table_batch **batch = &tlb->batch;

Re: [PATCH] mm/mmu_gather: define RCU version tlb_remove_table_one() in CONFIG_MMU_GATHER_RCU_TABLE_FREE

[Wei Yang]

On Tue, Feb 24, 2026 at 10:09:18AM +0800, Qi Zheng wrote:
>Hi Wei,
>
>On 2/22/26 8:45 AM, Wei Yang wrote:
>> Before commit 718b13861d22 ("x86: mm: free page table pages by RCU
>> instead of semi RCU"), the page table pages will be freed by semi RCU,
>> that is:
>> 
>>    - batch table freeing: asynchronous free by RCU
>>    - single table freeing: IPI + synchronous free
>> 
>> This commit introduce a RCU version for single table freeing to support
>> PT_RECLAIM. While it is not necessary to limit the RCU version only in
>> CONFIG_PT_RECLAIM. It is reasonable to asynchronous free single table by
>> RCU if CONFIG_MMU_GATHER_RCU_TABLE_FREE.
>> 
>> This patch moves the definition of single table freeing in
>> CONFIG_MMU_GATHER_RCU_TABLE_FREE, so we have both RCU version
>> batch/single table freeing if CONFIG_MMU_GATHER_RCU_TABLE_FREE.
>
>Thank you for doing this, but we can't do it right now.
>
>See the latest version: https://github.com/torvalds/linux/commit/9c8c02df3f8742f6db927e787ab971fd0b5ac08a,
>when the
>HAVE_ARCH_TLB_REMOVE_TABLE is selected, the pt_rcu_head member
>cannot be used.
>

Thanks for point out. Don't notice the latest change.

Andrew,

Sorry for the bothering. Please drop this one.

-- 
Wei Yang
Help you, Help me