From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F365CEE7FF4 for ; Mon, 11 Sep 2023 08:33:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6B5656B0184; Mon, 11 Sep 2023 04:33:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 665746B0187; Mon, 11 Sep 2023 04:33:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5535B6B0188; Mon, 11 Sep 2023 04:33:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 477436B0184 for ; Mon, 11 Sep 2023 04:33:50 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 01A5C1A0821 for ; Mon, 11 Sep 2023 08:33:49 +0000 (UTC) X-FDA: 81223653420.23.247DF79 Received: from relay2-d.mail.gandi.net (relay2-d.mail.gandi.net [217.70.183.194]) by imf21.hostedemail.com (Postfix) with ESMTP id D71C41C0019 for ; Mon, 11 Sep 2023 08:33:47 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=none; spf=pass (imf21.hostedemail.com: domain of alex@ghiti.fr designates 217.70.183.194 as permitted sender) smtp.mailfrom=alex@ghiti.fr; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694421228; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mtplgGlZNhLv0RZVKcZ8QzBLm8FlL6Lubb/ONJuGzq8=; b=2YT5BgEFPwggMhuoUVyiUtXTGGYRN3zFHGQYCAGbhdoycAJ7q7TEtb/lKo3eT5xOf/dFz6 svy15FL2MRy9WkcxBJo9lv5NakKha09eF4W4ekcDALdxjPLz6SpgU6wFtsPKprl9r6VS8F PB4wUoXvhy9mh28c8aUxY5g8tfJUJFw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694421228; a=rsa-sha256; cv=none; b=qgukHfoyhyK9V3IxhFmkkcJDpoiyc+eWlyQYjKlz6UP6TrNszMwvW8EQfzB2IPnhFYgsGt e5J2cDv+ZA266NW+AcqleLu3KWuaf7+XQ+qCLbI9mfI5ErCOVOhdlUYLEa1awgFl7Euyjo da82nyF77+JJQGumSM2oEI5q71m5BCI= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=none; spf=pass (imf21.hostedemail.com: domain of alex@ghiti.fr designates 217.70.183.194 as permitted sender) smtp.mailfrom=alex@ghiti.fr; dmarc=none Received: by mail.gandi.net (Postfix) with ESMTPSA id A6B7E40003; Mon, 11 Sep 2023 08:33:39 +0000 (UTC) Message-ID: Date: Mon, 11 Sep 2023 10:33:38 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.15.0 Subject: Re: [PATCH v3 4/4] riscv: Improve flush_tlb_kernel_range() Content-Language: en-US To: Samuel Holland , Alexandre Ghiti Cc: Andrew Jones , Will Deacon , "Aneesh Kumar K . V" , Andrew Morton , Nick Piggin , Peter Zijlstra , Mayuresh Chitale , Vincent Chen , Paul Walmsley , Palmer Dabbelt , Albert Ou , linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org References: <20230801085402.1168351-1-alexghiti@rivosinc.com> <20230801085402.1168351-5-alexghiti@rivosinc.com> <0e101df0-397a-0d1a-0080-2e60c68c79b6@sholland.org> From: Alexandre Ghiti In-Reply-To: <0e101df0-397a-0d1a-0080-2e60c68c79b6@sholland.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-GND-Sasl: alex@ghiti.fr X-Rspamd-Queue-Id: D71C41C0019 X-Rspam-User: X-Stat-Signature: 489gzzp5wajouqgqu1yjjkbmi9dumubb X-Rspamd-Server: rspam03 X-HE-Tag: 1694421227-215737 X-HE-Meta: U2FsdGVkX181qX106r/cQbbemwNpVJxlZRkDBLiWbtXQw0WVM3tfpx9bEHkTYcknvrwYexlDAptKR3TTVc2vQawYwPRW3OgdDpsMMHF+N5S6UTeFEaiau/5lO9qywqNtbQt8Qk7KiLJV9++BFwT6y7rt5GatYirn84V9dy0nfFGMVp3V6r6TAr1o3mj8wC8/vESNtb1QIKnNITm+tBKnUK9bATOTl4lzOPvAiOWjNMIHmVh118L5sU1JZB18KSB5FaOehFdy5qQxSxidPe0SbTYga1rvlTd/GOLuycrd3msj1D+BBNHupD7dgLM5KcgjXheFPGs6n345NE5VYJkG4AI81gMF10kGNUtopS9bJlYuavUPEy7tLs3alM+QIAsfl/vxdl6EPsP3jNqacVi+dQiNaZaoTQChw4AOsnL6k/9zWMSTCfOZxrc0dj2lFSQxGfHLuVDoV7GwsWVBdv4fsKkkNZ+Z5dd9wTZrwtz9H39eIB74f+AkdJm4kbhoJ4eGGjXUQXRgL/EAnw5eP57mGPsuNNdhrh7kJmRlyjrxmDfstSnCEXL4d/T2jxlJot4gw/Xf0NbzaYqr2PQ50EulxD8kRwXnEOtRrAyNZOTUDCuB7JUrevJbM0poxgjf/oRRTpq45WSURRgPFigAkGON8247qp9FDCi5Xd24cCfzWKSlXV164Xgwid0sBtkD+jZlg2w+9sgdlJNq837bSGjGznlaU2jEXNKrIlAMSL8/x4EMGuVNuflZwgMRYtqU823+ieo4cwf/QbibVz2ge2xSbWBni1jqjGUBHT1ByNlskcnQr4Ihm+QaF9Unj78x+Y9v3ACTXl48XbV/qjAXKkQXp7SWWlu4Xb3Xh+WyJ6lXm8qyuFdyB4AwOkhhHYp3y6adZQL1G2nDaUc6b5gYsdDfXCVLVftiyE5jln32+gUZfDotReaDQgNyS0xbe3Mp6m+R8+nChdohyFdHh1TmMA1 sDTb6FQA //V1IL4WO75ovrS5F80TymIWHOrnVl13KRJ3WZ+AZIrfb4fdXQIj8Koseu2Kxpv/3kLcHy6+EUvanON+kUpFsqVEkbFkG1FQodkFYr13ln0qR6mSOCIR28O1NkDgSf4+iRRO9REVqGvWwYJ/vecTeqcVfbBmsCLR5Ga/gexHmIWHENvqB/w7wHew5nO17gC8vtojgrROiok7+Hx0xMTBy23q4xj7TO4FxlUTlFHGyag+UCVt8cpNIKpKyrK+UnRDE7oAmwoyAX79M9ozOdQ/fxXPfDw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 09/09/2023 21:00, Samuel Holland wrote: > Hi Alex, > > On 8/1/23 03:54, Alexandre Ghiti wrote: >> This function used to simply flush the whole tlb of all harts, be more >> subtile and try to only flush the range. >> >> The problem is that we can only use PAGE_SIZE as stride since we don't know >> the size of the underlying mapping and then this function will be improved >> only if the size of the region to flush is < threshold * PAGE_SIZE. >> >> Signed-off-by: Alexandre Ghiti >> Reviewed-by: Andrew Jones >> --- >> arch/riscv/include/asm/tlbflush.h | 11 +++++----- >> arch/riscv/mm/tlbflush.c | 34 +++++++++++++++++++++++-------- >> 2 files changed, 31 insertions(+), 14 deletions(-) >> >> diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h >> index f5c4fb0ae642..7426fdcd8ec5 100644 >> --- a/arch/riscv/include/asm/tlbflush.h >> +++ b/arch/riscv/include/asm/tlbflush.h >> @@ -37,6 +37,7 @@ void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start, >> void flush_tlb_page(struct vm_area_struct *vma, unsigned long addr); >> void flush_tlb_range(struct vm_area_struct *vma, unsigned long start, >> unsigned long end); >> +void flush_tlb_kernel_range(unsigned long start, unsigned long end); >> #ifdef CONFIG_TRANSPARENT_HUGEPAGE >> #define __HAVE_ARCH_FLUSH_PMD_TLB_RANGE >> void flush_pmd_tlb_range(struct vm_area_struct *vma, unsigned long start, >> @@ -53,15 +54,15 @@ static inline void flush_tlb_range(struct vm_area_struct *vma, >> local_flush_tlb_all(); >> } >> >> -#define flush_tlb_mm(mm) flush_tlb_all() >> -#define flush_tlb_mm_range(mm, start, end, page_size) flush_tlb_all() >> -#endif /* !CONFIG_SMP || !CONFIG_MMU */ >> - >> /* Flush a range of kernel pages */ >> static inline void flush_tlb_kernel_range(unsigned long start, >> unsigned long end) >> { >> - flush_tlb_all(); >> + local_flush_tlb_all(); >> } >> >> +#define flush_tlb_mm(mm) flush_tlb_all() >> +#define flush_tlb_mm_range(mm, start, end, page_size) flush_tlb_all() >> +#endif /* !CONFIG_SMP || !CONFIG_MMU */ >> + >> #endif /* _ASM_RISCV_TLBFLUSH_H */ >> diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c >> index 0c955c474f3a..687808013758 100644 >> --- a/arch/riscv/mm/tlbflush.c >> +++ b/arch/riscv/mm/tlbflush.c >> @@ -120,18 +120,27 @@ static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, >> unsigned long size, unsigned long stride) >> { >> struct flush_tlb_range_data ftd; >> - struct cpumask *cmask = mm_cpumask(mm); >> - unsigned int cpuid; >> + struct cpumask *cmask, full_cmask; >> bool broadcast; >> >> - if (cpumask_empty(cmask)) >> - return; >> + if (mm) { >> + unsigned int cpuid; >> + >> + cmask = mm_cpumask(mm); >> + if (cpumask_empty(cmask)) >> + return; >> + >> + cpuid = get_cpu(); >> + /* check if the tlbflush needs to be sent to other CPUs */ >> + broadcast = cpumask_any_but(cmask, cpuid) < nr_cpu_ids; >> + } else { >> + cpumask_setall(&full_cmask); >> + cmask = &full_cmask; >> + broadcast = true; >> + } >> >> - cpuid = get_cpu(); >> - /* check if the tlbflush needs to be sent to other CPUs */ >> - broadcast = cpumask_any_but(cmask, cpuid) < nr_cpu_ids; >> if (static_branch_unlikely(&use_asid_allocator)) { >> - unsigned long asid = atomic_long_read(&mm->context.id) & asid_mask; >> + unsigned long asid = mm ? atomic_long_read(&mm->context.id) & asid_mask : 0; > I think the bug is here. Passing a value of 0 for the ASID is not the > same as passing the ASID in register x0. Only in the latter case does > the TLB flush apply to global mappings, which is what you need for > flush_tlb_kernel_range(). Fantastic, thank you, I was miles away from finding this! Really nice catch, thanks again. I'm fixing this and while doing so, I may be stepping a bit on your patchset (some code removal), sorry about that. I'll provide a new version quickly for Prabhakar to test, and we'll see how we'll rebase each other series. Thanks again Samuel, well done! Alex > Regards, > Samuel > >> >> if (broadcast) { >> if (riscv_use_ipi_for_rfence()) { >> @@ -165,7 +174,8 @@ static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, >> } >> } >> >> - put_cpu(); >> + if (mm) >> + put_cpu(); >> } >> >> void flush_tlb_mm(struct mm_struct *mm) >> @@ -196,6 +206,12 @@ void flush_tlb_range(struct vm_area_struct *vma, unsigned long start, >> >> __flush_tlb_range(vma->vm_mm, start, end - start, stride_size); >> } >> + >> +void flush_tlb_kernel_range(unsigned long start, unsigned long end) >> +{ >> + __flush_tlb_range(NULL, start, end, PAGE_SIZE); >> +} >> + >> #ifdef CONFIG_TRANSPARENT_HUGEPAGE >> void flush_pmd_tlb_range(struct vm_area_struct *vma, unsigned long start, >> unsigned long end) > > _______________________________________________ > linux-riscv mailing list > linux-riscv@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-riscv