From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9DE13C19F32 for ; Mon, 3 Mar 2025 01:45:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EDE7D6B007B; Sun, 2 Mar 2025 20:45:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E675B6B0083; Sun, 2 Mar 2025 20:45:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CE0F36B0085; Sun, 2 Mar 2025 20:45:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id AD5856B007B for ; Sun, 2 Mar 2025 20:45:53 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 37B1DA2F62 for ; Mon, 3 Mar 2025 01:45:53 +0000 (UTC) X-FDA: 83178548586.03.988A31F Received: from mail-oa1-f46.google.com (mail-oa1-f46.google.com [209.85.160.46]) by imf08.hostedemail.com (Postfix) with ESMTP id B9C0A16000A for ; Mon, 3 Mar 2025 01:45:50 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=T6nho3Ya; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf08.hostedemail.com: domain of cuiyunhui@bytedance.com designates 209.85.160.46 as permitted sender) smtp.mailfrom=cuiyunhui@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740966351; a=rsa-sha256; cv=none; b=1H6VSTnEVLMywvfzCCmffhnZcv6pz+mjrX7HkAc9WxyQHTUIiBZ3Zt88i57jrNJ3JQCWRS 960rkEtesy/3Splg7woF2vC3NfSzsaikTvwJvK9xO/JXChZ/pgljtjCB4IwknSFEGgryq5 XwAzMPZFSD65regS5hhthT1iDM5C0lo= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=T6nho3Ya; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf08.hostedemail.com: domain of cuiyunhui@bytedance.com designates 209.85.160.46 as permitted sender) smtp.mailfrom=cuiyunhui@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740966351; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=W8SnZDZRbvLSaoNIXAx3tJOrDKr8WC27+4Kk8dOe4BE=; b=kyOeWw9UATs5MP1siEpcg3spGgeMTSxAStQSxpKpcC286B+wSbLrFYE4AX2aHgDxKmuLvl o2WuNzRfjjj8Ii6ZRnueKrLQN6hjA5fW2Q9htk20/b82WfAfvZDkMb4H+0btgl0MSVG49S ttppchaJd4cAWhQBHSiPocFk6JJANic= Received: by mail-oa1-f46.google.com with SMTP id 586e51a60fabf-2b1a9cbfc8dso989605fac.2 for ; Sun, 02 Mar 2025 17:45:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1740966349; x=1741571149; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=W8SnZDZRbvLSaoNIXAx3tJOrDKr8WC27+4Kk8dOe4BE=; b=T6nho3YatPRjPiltmMhErEACB4g7S/CENStSCrxOQ7JdJ8u/X2oD6DDAZAz5TxkS4k iwqo3dXvyEFfqwYVH2vjNt7XRmIRkzdV0DZDjWZTF7+LJXZOMRfBhAUL/dDASrMix3GK 53KnwO5uf6apTQLYtscg5m0OnlxkCmpy/9oJxma93+1nFZMTWCPbvTOot383gfeV+5yg FL4Q+OWb6aDSMwttEvFAruRj8jch9bvV+DtM2SdZOvovlh2EATQdRtQMQ5G5O2Kk+Nc7 4OmcWPA5ScvmI4pN231Jp47JuE7snVn/xJh6v5VYLTyt4IpYyu5my7t6TFmzAk8ebAmi wJvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740966349; x=1741571149; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=W8SnZDZRbvLSaoNIXAx3tJOrDKr8WC27+4Kk8dOe4BE=; b=dlDQ6YllVeHdjEQ8yJOEjiG61n4MqNtmOh+0aeiKn/gFJVvXwB1xVshrLoQSzM0jfc /DiyY39iuGjHgaWBOlRRID33CePhdjR4eFkBYZ9EZmu+0XgrP6VvJvpM90WCeHzM1EHM pqMpa7GfVY27lG25wxKxmBJ7ql60TYy3vEYZR8LA28xcAe/rtgGsR98QfGaCeHfl6V5b yXRk2pDCdI7/3t4cvyK3oJ1+3fIwzGlybUlHy6HtEij4VqVAuZeOE6NgjroLy5qUIXEg xpe2owbNv5NG3o+xhtqY+I6hGZzttd9GFz9BVUagUhzfGYWvdEzgkOwnn37hZPF25SXh oceA== X-Forwarded-Encrypted: i=1; AJvYcCUcvJpeXp4afT6Oy+zVHQTlpzDMZep/C6kxexnH99eaNGiP51UpKiLw1EhkvD2WwWYNFpwy3+N1Pw==@kvack.org X-Gm-Message-State: AOJu0Yy6ZqJRjAQnxnz9gKQaZr2gGcb/jhU2ybMbw/K+3XJayyWdssAw AaoP3+8xXNuFuYQZnijRNtrke+sPmSTSL5KQbGSq7noTBDKpmHsKY+o8RAj87sMTBgGWVG71EsG +guz7Jr7eFUZupEM5TYYagSQi365X/gycOpwxmA== X-Gm-Gg: ASbGncuyn4DFNDu5INPZ2yy+moK8u3qg6Wf+TwtWOrkg+n6bTwdosifgfCRQzG23O+R QxXS+Qz3R21tftBmsc1TQCwFMt1IdHxvc668Gw4jX+ERalRLR4gXkrLCY3YS6AWLlFfnFolNpgz 2AvGISrSRiYD2aaujRkIdFnKknunCV X-Google-Smtp-Source: AGHT+IGOWIRMroQZxbRNJcNH5tnmnGD/bIDamuPKwa3GDm5B/gWkRsBelrb1kdvpy/ijW+hukYwC4esnzOdnOPSB3MY= X-Received: by 2002:a05:6871:aa16:b0:2b8:3d1c:772 with SMTP id 586e51a60fabf-2c178335be6mr6945276fac.5.1740966349475; Sun, 02 Mar 2025 17:45:49 -0800 (PST) MIME-Version: 1.0 References: <9025595e895515515c95e48db54b29afa489c41d.1740454179.git.zhengqi.arch@bytedance.com> In-Reply-To: <9025595e895515515c95e48db54b29afa489c41d.1740454179.git.zhengqi.arch@bytedance.com> From: yunhui cui Date: Mon, 3 Mar 2025 09:45:37 +0800 X-Gm-Features: AQ5f1Jrb5nkY3jTSkwu3MIgtit28ISDfNBjj4Qwj5SXBUXKwFnVTECUIEIzLwQo Message-ID: Subject: Re: [External] [PATCH v2 4/6] riscv: pgtable: unconditionally use tlb_remove_ptdesc() To: Qi Zheng Cc: peterz@infradead.org, kevin.brodsky@arm.com, riel@surriel.com, vishal.moola@gmail.com, david@redhat.com, jannh@google.com, hughd@google.com, willy@infradead.org, yuzhao@google.com, muchun.song@linux.dev, akpm@linux-foundation.org, will@kernel.org, aneesh.kumar@kernel.org, npiggin@gmail.com, arnd@arndb.de, dave.hansen@linux.intel.com, rppt@kernel.org, alexghiti@rivosinc.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, loongarch@lists.linux.dev, linux-m68k@lists.linux-m68k.org, linux-mips@vger.kernel.org, linux-openrisc@vger.kernel.org, linux-sh@vger.kernel.org, linux-um@lists.infradead.org, x86@kernel.org, linux-riscv@lists.infradead.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: B9C0A16000A X-Stat-Signature: 3cdhru7jj9bmcwirhyrxkyhh14u9uags X-Rspam-User: X-HE-Tag: 1740966350-658046 X-HE-Meta: U2FsdGVkX18wTw90IEQvjIZQoqWxHapQW0CRfeCnuFjKL8wDY4IC1BA2jiavxL7yujuQxmup2jLzUzdkfv63U8z6cfCnEIj1lvIQMFHCfpK7zIJHQkwYguKaBeKz6E4ExSAac24gDwwymHLiP1hh2xLuKB+ynDA9JzpKVMo8VEPkNc3E5GSdEgwTMd5gjNqrcF6CfYeOK6u8TyDwN2TAD6MbH32SyyGZDxVWo53ce1g9QxLV2CjD3WTN0WGbx/90mcMznKnWGbmO7j7aCZvQh1AMX9DqGa745J4a0wGq7YtglB0xuu5VKKlZiE+MvUR6TmawiprtrnSKIDM2GEG3GoiRikgynCseoXh67YIXEv0JEkXPg5uKL4ITxAsnoFwclLd3Dyk4AyqPF+w/e0C6qqOWXr/onlPekSz048L8wOcFOXbx874hMvRiQ+W/cc69VPu6Yw8BDO5Aqx2S9QNl9RAIo2RCEnIS8CNByx7rxq0MaHubqUHUrLxp7m7XwFMLqLg4gCcsiyJZjFXry/NplE2eWN298Ffa8xDRU0WsBrKHN8KWaTm18MMm8ZaiOEaMEBOFFWbjffKdpZ9rDJK0aIGfRbmma6moB6gTaNN3iHZsy/z6rYlILuXE8V2YjKZS0u4RLZ7HGHKppk6Sr+XJ1AQhFlwnhzkiiumb7eeZyZisBqle/MHgFuxjksRrJvy+popqFwsxGML6B9/B7n6ykFTeDzNxj5exAgsabHdGwyQTVYlQfam1ST8Jruj7/A8pMcMD2ZPMvv4vdg9lzHRTdRHUhpqm4VXaL3FMPqPKAlkDmz7uJsmcqYcM7QgQkpm7wbP6oQy1F/A40X218pf6ja9bXmnJ5sTW4ovWGRPGSOFxkpLi1BHccEJSOJiZg2jQSKagNDBQ69UMM6iq+T1W7thHhD39ITaqZaUdWE8uZvxhstPG6S28lSSdf204+0cJHoFAMGrDuP/ZWmGDLjh DhQbzBPw h1QAnrVyrAQ7n50O7himTjVc3TcfuNtnBYnuc7eeWc8m/ULv9NJca7QIQxvZIvItYG6r8XFo32ssxH7b2Da5HO6DKButgIbpFVHrxtx2zhLC0sQPSWbcTy54G2wLCfveSrnGlAiavMmOGZquuqpGEAdi7qfxOiq60kzB3obwDugoRPNq13fPKAKMNg72wjvp2s2W7dmh7QeV1mB+jYWvC5efx/hReoIbD6nKEAqNztA6PmcoikHN34gtcQULdmYdgFhPukxkBmmBXVu8yvSumRdgLaleREbeCGDrWYOx1Ya0dul0PMrKGskuzYlrTwkuufjG/LAqzzvfaojtUZBUZS9gJ+xVXSDGZQAfWf4XT1rKnrFtov7fvan3IxMQS/dFrmGQKKUCaCr6l0f62JykvfIU96BLFmXHAF1l+ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi Qi, On Tue, Feb 25, 2025 at 11:48=E2=80=AFAM Qi Zheng wrote: > > To support fast gup, the commit 69be3fb111e7 ("riscv: enable > MMU_GATHER_RCU_TABLE_FREE for SMP && MMU") did the following: > > 1) use tlb_remove_page_ptdesc() for those platforms which use IPI to > perform TLB shootdown > > 2) use tlb_remove_ptdesc() for those platforms which use SBI to perform > TLB shootdown > > The tlb_remove_page_ptdesc() is the wrapper of the tlb_remove_page(). By > design, the tlb_remove_page() should be used to remove a normal page from > a page table entry, and should not be used for page table pages. > > The tlb_remove_ptdesc() is the wrapper of the tlb_remove_table(), which i= s > designed specifically for freeing page table pages. If the > CONFIG_MMU_GATHER_TABLE_FREE is enabled, the tlb_remove_table() will use > semi RCU to free page table pages, that is: > > - batch table freeing: asynchronous free by RCU > - single table freeing: IPI + synchronous free > > If the CONFIG_MMU_GATHER_TABLE_FREE is disabled, the tlb_remove_table() > will fall back to pagetable_dtor() + tlb_remove_page(). > > For case 1), since we need to perform TLB shootdown before freeing the > page table page, the local_irq_save() in fast gup can block the freeing > and protect the fast gup page walker. Therefore we can ensure safety by > just using tlb_remove_page_ptdesc(). In addition, we can also the > tlb_remove_ptdesc()/tlb_remove_table() to achieve it, and it doesn't > matter whether CONFIG_MMU_GATHER_RCU_TABLE_FREE is selected. And in > theory, the performance of freeing pages asynchronously via RCU will not > be lower than synchronous free. > > For case 2), since local_irq_save() only disable S-privilege IPI irq but > not M-privilege's, which is used by the SBI implementation to perform TLB > shootdown, so we must select CONFIG_MMU_GATHER_RCU_TABLE_FREE and use > tlb_remove_ptdesc() to ensure safety. The riscv selects this config for > SMP && MMU, the CONFIG_RISCV_SBI is dependent on MMU. Therefore, only the > UP system may have the situation where CONFIG_MMU_GATHER_RCU_TABLE_FREE i= s > disabled but CONFIG_RISCV_SBI is enabled. But there is no freeing vs fast > gup race in the UP system. > > So, in summary, we can use tlb_remove_ptdesc() to support fast gup in all > cases, and this interface is specifically designed for page table pages. > So let's use it unconditionally. > > Signed-off-by: Qi Zheng > Suggested-by: Peter Zijlstra (Intel) > --- > arch/riscv/include/asm/pgalloc.h | 26 ++++---------------------- > 1 file changed, 4 insertions(+), 22 deletions(-) > > diff --git a/arch/riscv/include/asm/pgalloc.h b/arch/riscv/include/asm/pg= alloc.h > index 3e2aebea6312d..770ce18a7328b 100644 > --- a/arch/riscv/include/asm/pgalloc.h > +++ b/arch/riscv/include/asm/pgalloc.h > @@ -15,24 +15,6 @@ > #define __HAVE_ARCH_PUD_FREE > #include > > -/* > - * While riscv platforms with riscv_ipi_for_rfence as true require an IP= I to > - * perform TLB shootdown, some platforms with riscv_ipi_for_rfence as fa= lse use > - * SBI to perform TLB shootdown. To keep software pagetable walkers safe= in this > - * case we switch to RCU based table free (MMU_GATHER_RCU_TABLE_FREE). S= ee the > - * comment below 'ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE' in include/asm= -generic/tlb.h > - * for more details. > - */ > -static inline void riscv_tlb_remove_ptdesc(struct mmu_gather *tlb, void = *pt) > -{ > - if (riscv_use_sbi_for_rfence()) { > - tlb_remove_ptdesc(tlb, pt); > - } else { > - pagetable_dtor(pt); > - tlb_remove_page_ptdesc(tlb, pt); > - } > -} > - > static inline void pmd_populate_kernel(struct mm_struct *mm, > pmd_t *pmd, pte_t *pte) > { > @@ -108,14 +90,14 @@ static inline void __pud_free_tlb(struct mmu_gather = *tlb, pud_t *pud, > unsigned long addr) > { > if (pgtable_l4_enabled) > - riscv_tlb_remove_ptdesc(tlb, virt_to_ptdesc(pud)); > + tlb_remove_ptdesc(tlb, virt_to_ptdesc(pud)); > } > > static inline void __p4d_free_tlb(struct mmu_gather *tlb, p4d_t *p4d, > unsigned long addr) > { > if (pgtable_l5_enabled) > - riscv_tlb_remove_ptdesc(tlb, virt_to_ptdesc(p4d)); > + tlb_remove_ptdesc(tlb, virt_to_ptdesc(p4d)); > } > #endif /* __PAGETABLE_PMD_FOLDED */ > > @@ -143,7 +125,7 @@ static inline pgd_t *pgd_alloc(struct mm_struct *mm) > static inline void __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd, > unsigned long addr) > { > - riscv_tlb_remove_ptdesc(tlb, virt_to_ptdesc(pmd)); > + tlb_remove_ptdesc(tlb, virt_to_ptdesc(pmd)); > } > > #endif /* __PAGETABLE_PMD_FOLDED */ > @@ -151,7 +133,7 @@ static inline void __pmd_free_tlb(struct mmu_gather *= tlb, pmd_t *pmd, > static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte, > unsigned long addr) > { > - riscv_tlb_remove_ptdesc(tlb, page_ptdesc(pte)); > + tlb_remove_ptdesc(tlb, page_ptdesc(pte)); > } > #endif /* CONFIG_MMU */ > > -- > 2.20.1 > > This set of patches mainly refactors the remove page table function interfaces. The comment "riscv_ipi_for_rfence" has been removed, as it was no longer needed, and this patch handles that. Additionally, whether riscv_use_sbi_for_rfence is true or false, page tables can now be released using RCU. This patch changes the previous synchronous release logic (for !riscv_use_sbi_for_rfence) to an RCU-based release. So, Reviewed-by: Yunhui Cui Based on qemu-system-riscv64, I tested this patch. The log is as follows: ./gup_test TAP version 13 1..1 # GUP_FAST_BENCHMARK: Time: get:663365 put:117 us # ok 1 ioctl status 0 # Totals: pass:1 fail:0 xfail:0 xpass:0 skip:0 error:0 So, Tested-by: Yunhui Cui Thanks, Yunhui