From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0582AC3DA6E for ; Sun, 31 Dec 2023 06:32:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 52F146B00A6; Sun, 31 Dec 2023 01:32:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4DEA96B00D4; Sun, 31 Dec 2023 01:32:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 37F996B01F6; Sun, 31 Dec 2023 01:32:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 21EB16B00A6 for ; Sun, 31 Dec 2023 01:32:54 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id E44A9A0245 for ; Sun, 31 Dec 2023 06:32:53 +0000 (UTC) X-FDA: 81626145426.19.9AF91B7 Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) by imf06.hostedemail.com (Postfix) with ESMTP id 0355118000F for ; Sun, 31 Dec 2023 06:32:51 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf06.hostedemail.com: domain of alex@ghiti.fr designates 217.70.183.199 as permitted sender) smtp.mailfrom=alex@ghiti.fr ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704004372; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mS8ljBrQGM0Dw8oUwaUJP8Qo8uJpK7K71lcWKtoY9Ls=; b=Ju0EGwy5yuHPLm+C0ikXRyR2m7bflvDK5Coz7tAD7MyNUfiRb/4rSvgSKyc5ZM3i3YBiY5 8G+HyV65xc6s/1aVcyLLv6td7CuzSczEAdsyJmD6KFCtwUHf/d3tdmfeqplaL/Rw7VqMKj 6AIjVf5AO4TA4xuGzBc5LoltXix8BDY= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf06.hostedemail.com: domain of alex@ghiti.fr designates 217.70.183.199 as permitted sender) smtp.mailfrom=alex@ghiti.fr ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704004372; a=rsa-sha256; cv=none; b=c1GkA42PvkMV10q5vUK6148OFqqQyDGknTa7w6M4XvuwpPebIn8hHAawT6iSO+p8Kphn3g HA2CSVVFnpGTglcsFb7L8KGy9JLfmw7JEyefIoGtNPoYQKXVm5Pdb2/ep6QMAExUU+f46L 8pN3mZ0+31e4RUokSzpjTVyBXgG64CA= Received: by mail.gandi.net (Postfix) with ESMTPSA id 6ED39FF805; Sun, 31 Dec 2023 06:32:47 +0000 (UTC) Message-ID: Date: Sun, 31 Dec 2023 07:32:47 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 3/4] riscv: enable MMU_GATHER_RCU_TABLE_FREE for SMP && MMU Content-Language: en-US To: Jisheng Zhang , Paul Walmsley , Palmer Dabbelt , Albert Ou , Will Deacon , "Aneesh Kumar K . V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org References: <20231219175046.2496-1-jszhang@kernel.org> <20231219175046.2496-4-jszhang@kernel.org> From: Alexandre Ghiti In-Reply-To: <20231219175046.2496-4-jszhang@kernel.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-GND-Sasl: alex@ghiti.fr X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 0355118000F X-Stat-Signature: u1sbyirq51qyynhet3qyt6rue9k8i3u6 X-Rspam-User: X-HE-Tag: 1704004371-369107 X-HE-Meta: U2FsdGVkX1+oWWRc4qAb1ul4EOiXTubCpqC+SP/siiTWQtpH1Tv9xkiL2/LXLGqtkx070flF3MzxCC4M56bk1ndKkXRMiFYISFM6HuAu67KzH5Z/Aa0PKuHSM7hvKN6BcKlxYwgtHa70qm5dBcT1eEY4gaRE3ryyqcEINwaerB7a4XijZ+flseDdX6pkSxt7wKTP9D1siniZKZByxNUt7kevrrx1x/ItVBFeSCf8AXLSTqrk9NzIW8iq8Zo0oImTFMOZ/+8K/1dblOMy0y1M2D2C8DhDz1MQ9eTnKy863Ek3WXHNFJhL1FImjXwiAcVORn8YeaX02D7byN2RqPXqGp6eAhMJF5ZrFV86DS0GyKj2TS8zU7LJvwfj/6HBkVVNTMQex+cD9/5Iv+zcPMyGJWIqPuU5PUVdlk4TiUtdWclZRxVYomWFsGQhfwEC9rNELPVEBwROKi0fblGrmK2qJXAWGj9sg7+9KiBtDUjdj1VICpAY9l6Nu9iEWc6ob0nYNoXUhmkMuedLn+D4nQjxM83Jb2H9F/wsAseaAANVp860d2kwsiqWpJZ6h49DV7PR4kS1/O9vqXSA3H/r4O7fMoXeoPxtLtDwgoKmEQ8BaPmnSzQk7+B16RJoxGBhpIIRJxxE4V+EXOTWbuJayLICM9qqTpa+PZlMyyJc6mH99fk70MQ2F3QmKdn9cvA7iKIIYRZbM7vin44r0mTf5ojWSZkf1FIWSduhBCCvSpEZ42oOktsyo8Xnyu3H68l/xsx1YleUjrPIn7OyqxjuGr9v7DZ49o/2RYk9LVjdMEZqCcKhAIMXLjz0g7Pi3kNeIJOWTfEXJmNQXzvDbeeRKaRVGYdQa2wwsHe+zjkuT5mZvZ60XmF3S7gdq2tFmlG8SfxNzhnUP2rdPb58zhu+C9N/rrQECZ9sNkE83Vl4YUTRsu319W3X3SWVP9LqTlMYThFXDpftc4E1TiSVFHVNt6+ UgnElhPX CY1Dq2zGuzWAwYh8E2yBM3eURIkimu25pcLl0vcjBoYXq5BlYyUSjnMRKPB5ByFtu19l1rKSuIz8832kI1H4rSx9Yu8BM5e/LEFcss1FJJIj5bOpqVz+DIlTp8QpktnXm/9rUd1+wLkrCE6bgxAsRxneSEAH4nC6noriaqW6xKRCdZiv7VYLKXvdqnsbEn2Vi0lbY X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 19/12/2023 18:50, Jisheng Zhang wrote: > In order to implement fast gup we need to ensure that the page > table walker is protected from page table pages being freed from > under it. > > riscv situation is more complicated than other architectures: some > riscv platforms may use IPI to perform TLB shootdown, for example, > those platforms which support AIA, usually the riscv_ipi_for_rfence is > true on these platforms; some riscv platforms may rely on the SBI to > perform TLB shootdown, usually the riscv_ipi_for_rfence is false on > these platforms. To keep software pagetable walkers safe in this case > we switch to RCU based table free (MMU_GATHER_RCU_TABLE_FREE). See the > comment below 'ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE' in > include/asm-generic/tlb.h for more details. > > This patch enables MMU_GATHER_RCU_TABLE_FREE, then use > > *tlb_remove_page_ptdesc() for those platforms which use IPI to perform > TLB shootdown; > > *tlb_remove_ptdesc() for those platforms which use SBI to perform TLB > shootdown; Can you elaborate a bit more on what those functions do differently and why we need to differentiate IPI vs SBI TLB shootdown? I don't understand this. Thanks, Alex > Both case mean that disabling interrupts will block the free and > protect the fast gup page walker. > > Signed-off-by: Jisheng Zhang > --- > arch/riscv/Kconfig | 1 + > arch/riscv/include/asm/pgalloc.h | 23 ++++++++++++++++++----- > arch/riscv/include/asm/tlb.h | 18 ++++++++++++++++++ > 3 files changed, 37 insertions(+), 5 deletions(-) > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig > index 24c1799e2ec4..d3555173d9f4 100644 > --- a/arch/riscv/Kconfig > +++ b/arch/riscv/Kconfig > @@ -147,6 +147,7 @@ config RISCV > select IRQ_FORCED_THREADING > select KASAN_VMALLOC if KASAN > select LOCK_MM_AND_FIND_VMA > + select MMU_GATHER_RCU_TABLE_FREE if SMP && MMU > select MODULES_USE_ELF_RELA if MODULES > select MODULE_SECTIONS if MODULES > select OF > diff --git a/arch/riscv/include/asm/pgalloc.h b/arch/riscv/include/asm/pgalloc.h > index 3c5e3bd15f46..deaf971253a2 100644 > --- a/arch/riscv/include/asm/pgalloc.h > +++ b/arch/riscv/include/asm/pgalloc.h > @@ -102,7 +102,10 @@ static inline void __pud_free_tlb(struct mmu_gather *tlb, pud_t *pud, > struct ptdesc *ptdesc = virt_to_ptdesc(pud); > > pagetable_pud_dtor(ptdesc); > - tlb_remove_page_ptdesc(tlb, ptdesc); > + if (riscv_use_ipi_for_rfence()) > + tlb_remove_page_ptdesc(tlb, ptdesc); > + else > + tlb_remove_ptdesc(tlb, ptdesc); > } > } > > @@ -136,8 +139,12 @@ static inline void p4d_free(struct mm_struct *mm, p4d_t *p4d) > static inline void __p4d_free_tlb(struct mmu_gather *tlb, p4d_t *p4d, > unsigned long addr) > { > - if (pgtable_l5_enabled) > - tlb_remove_page_ptdesc(tlb, virt_to_ptdesc(p4d)); > + if (pgtable_l5_enabled) { > + if (riscv_use_ipi_for_rfence()) > + tlb_remove_page_ptdesc(tlb, virt_to_ptdesc(p4d)); > + else > + tlb_remove_ptdesc(tlb, virt_to_ptdesc(p4d)); > + } > } > #endif /* __PAGETABLE_PMD_FOLDED */ > > @@ -169,7 +176,10 @@ static inline void __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd, > struct ptdesc *ptdesc = virt_to_ptdesc(pmd); > > pagetable_pmd_dtor(ptdesc); > - tlb_remove_page_ptdesc(tlb, ptdesc); > + if (riscv_use_ipi_for_rfence()) > + tlb_remove_page_ptdesc(tlb, ptdesc); > + else > + tlb_remove_ptdesc(tlb, ptdesc); > } > > #endif /* __PAGETABLE_PMD_FOLDED */ > @@ -180,7 +190,10 @@ static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte, > struct ptdesc *ptdesc = page_ptdesc(pte); > > pagetable_pte_dtor(ptdesc); > - tlb_remove_page_ptdesc(tlb, ptdesc); > + if (riscv_use_ipi_for_rfence()) > + tlb_remove_page_ptdesc(tlb, ptdesc); > + else > + tlb_remove_ptdesc(tlb, ptdesc); > } > #endif /* CONFIG_MMU */ > > diff --git a/arch/riscv/include/asm/tlb.h b/arch/riscv/include/asm/tlb.h > index 1eb5682b2af6..a0b8b853503f 100644 > --- a/arch/riscv/include/asm/tlb.h > +++ b/arch/riscv/include/asm/tlb.h > @@ -10,6 +10,24 @@ struct mmu_gather; > > static void tlb_flush(struct mmu_gather *tlb); > > +#ifdef CONFIG_MMU > +#include > + > +/* > + * While riscv platforms with riscv_ipi_for_rfence as true require an IPI to > + * perform TLB shootdown, some platforms with riscv_ipi_for_rfence as false use > + * SBI to perform TLB shootdown. To keep software pagetable walkers safe in this > + * case we switch to RCU based table free (MMU_GATHER_RCU_TABLE_FREE). See the > + * comment below 'ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE' in include/asm-generic/tlb.h > + * for more details. > + */ > +static inline void __tlb_remove_table(void *table) > +{ > + free_page_and_swap_cache(table); > +} > + > +#endif /* CONFIG_MMU */ > + > #define tlb_flush tlb_flush > #include >