From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ABB15C001DE for ; Fri, 28 Jul 2023 13:48:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 47E628D0002; Fri, 28 Jul 2023 09:48:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 42F218D0001; Fri, 28 Jul 2023 09:48:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2F6FE8D0002; Fri, 28 Jul 2023 09:48:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 1F5748D0001 for ; Fri, 28 Jul 2023 09:48:46 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id DCA2441216 for ; Fri, 28 Jul 2023 13:48:45 +0000 (UTC) X-FDA: 81061151010.10.77E1837 Received: from mail-ej1-f43.google.com (mail-ej1-f43.google.com [209.85.218.43]) by imf22.hostedemail.com (Postfix) with ESMTP id DD10CC000D for ; Fri, 28 Jul 2023 13:48:43 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=ventanamicro.com header.s=google header.b=AB6+Mgpe; dmarc=none; spf=pass (imf22.hostedemail.com: domain of ajones@ventanamicro.com designates 209.85.218.43 as permitted sender) smtp.mailfrom=ajones@ventanamicro.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690552124; a=rsa-sha256; cv=none; b=Edrte5AYMXeslXBMGXpaJKqWP9Wj5Gw6nyESZXSyRcvrPybxRHJIkwUyEH2K7WcPD0v61b 8LkLE8YZ1I+Ln0hWpFr75AqIk2hk9LLNhFFjgXzX6EK/Ul/mD4jy5/uimxstplJv0+mvus 5aFA93inBuby90Qcnb1fee7L5Ty03YA= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=ventanamicro.com header.s=google header.b=AB6+Mgpe; dmarc=none; spf=pass (imf22.hostedemail.com: domain of ajones@ventanamicro.com designates 209.85.218.43 as permitted sender) smtp.mailfrom=ajones@ventanamicro.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690552124; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xglqSrB2iPFW2I/fuXfPFH2AjeG9G8DHiCGCdZ7AftA=; b=cXBSyvoIOS5aNB+IczGSEOBGh8DJRmfD5m83fj0fGm4K0JkPYKM9iIGPKfQUjKhNJ3Gjqs u4mSZ1npX1R1D5Tr5+MM0PkEvG5s4GJy8Bo3uVv1663AkmZ9HRmH7nK4OqZ8Lidmq2PoP0 hTx7JYlKowSOiLbsWkwEClqWwTxyg4s= Received: by mail-ej1-f43.google.com with SMTP id a640c23a62f3a-991da766865so302813366b.0 for ; Fri, 28 Jul 2023 06:48:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ventanamicro.com; s=google; t=1690552122; x=1691156922; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=xglqSrB2iPFW2I/fuXfPFH2AjeG9G8DHiCGCdZ7AftA=; b=AB6+MgpeqsfDfT0Gtsokz5BQci7npK3FQ2KziJGiv0UyZ8bcgvhqc+jcNFZrgvptI9 AIrPkUC39/8FTuu/2fdqNt4nPY7RpvzNBpE7u/1AXmaIiKrDozV1gP+5xgOZOIjJVg14 v4JrgVF+IQYC8AW77Co4dn69YatbvqQJ1SQm5mYkHE6NZyD60qjFEj4H0s0FvdQ5X9/s ujEuMICe9jOyJ0EPcuouBIXvftZ9Tthnvhpr+wf+Bx5uPEZrkR50ZxRGeActqvJAADh7 bvgMp3mmy12GUl0qjKexge/lNeaeMf3tZsrb+qxNw4/y3M9VQ2w6+z84jeC0dMO0xJRS 5fFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690552122; x=1691156922; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=xglqSrB2iPFW2I/fuXfPFH2AjeG9G8DHiCGCdZ7AftA=; b=AM2R3RnwrwAdP1B/ALJFgFKY7DN2JzXViVON2JEY5BVF0rzBy3Mw0jgP8Q9TQe7QCc l5ZyiSDx/7zJlOoMMU4RaOgboHoO156XvMFGMEg0MW8QzMXhxSk1cS65FAcZ/kTs5u8S 6x2SjUVacmV0WSzIdJs8Q1d0xnPC3CslkeKDM6oeEuprIc19Cbf8ulS2KkRUofkYtqUP fSJOCOpLP9wcvQWSJf+zsLPiSKYne5mEAeq4vWKa08AuWXrUNVvsRXvGskrmql1H1tkS 8cQe0iuRBBaqu7Pa+o3NvR870EZAP4oXJp6Du1hHNHjUSVl/G2VgE6nhSrUgDnTq/FZn 0lVA== X-Gm-Message-State: ABy/qLaAtEFS2UNddD+t5UGt0DOZEBs0STqw1WsOYbxmWn6RhWkODnV7 eRVERaJJBAQ/L540rQOk/q1pbQ== X-Google-Smtp-Source: APBJJlGnEXAlsMCADN+Nq6BK+wfKYf1MY6AOSQRz7CvibJgwy8Y7+jhKeE+hbVuLfVBpD8mGaYmFsQ== X-Received: by 2002:a17:907:7710:b0:988:6491:98e1 with SMTP id kw16-20020a170907771000b00988649198e1mr2173678ejc.42.1690552122403; Fri, 28 Jul 2023 06:48:42 -0700 (PDT) Received: from localhost (2001-1ae9-1c2-4c00-20f-c6b4-1e57-7965.ip6.tmcz.cz. [2001:1ae9:1c2:4c00:20f:c6b4:1e57:7965]) by smtp.gmail.com with ESMTPSA id t10-20020a17090616ca00b00982a352f078sm2066763ejd.124.2023.07.28.06.48.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Jul 2023 06:48:41 -0700 (PDT) Date: Fri, 28 Jul 2023 15:48:41 +0200 From: Andrew Jones To: Alexandre Ghiti Cc: Will Deacon , "Aneesh Kumar K . V" , Andrew Morton , Nick Piggin , Peter Zijlstra , Mayuresh Chitale , Vincent Chen , Paul Walmsley , Palmer Dabbelt , Albert Ou , linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH v2 4/4] riscv: Improve flush_tlb_kernel_range() Message-ID: <20230728-f5c389ac7f2a9aadf93939f5@orel> References: <20230727185553.980262-1-alexghiti@rivosinc.com> <20230727185553.980262-5-alexghiti@rivosinc.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230727185553.980262-5-alexghiti@rivosinc.com> X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: DD10CC000D X-Stat-Signature: zinixkuprwb36pj7merdxeinx8rdinhd X-HE-Tag: 1690552123-90093 X-HE-Meta: U2FsdGVkX1/ZDzNpnuR3rnCFYcMn1kV39q7ENqbHO1Adl1p2uz3UCdItODvMADvOk7njap7ln+QTDTmu4+j8vaWLkswuFea090/bGm+gcl2352e62Ey4yex2vA0+yx7XeLiC6fsWMJwZvah9aKmfy3bs9xmfTf+0CaElmOX1VegGQ+BfnZMjouhW36FfS07FzF54VtgQ0VFDxlNiqJpNJDotYVAwc91DcSoSH2EkbAowFU7hmRaSWnF+Wsep2FeZ14MOZyR/e/icUqqL8HQDt65KVBYM2dc8g5AOVjNemX6B62wbLR/Kbr6v7oO2cEmhcDW0Q3mUg6ARmnCSOeWslogJCxo8G3rzrliLDrHN9vqKSnRF+UJdu+jCg4gW08yhXxV/8TxoD4Iqt7f4CkNLc4dNVuZjX0irn0rqI9lcv6Svd0K1fr755T0VmbdLwnQaKlfGZCmT1dhREq+Z+qF38RO4rfckNxX9XLIdWYxrIPx7UtND558fmBOt9Wll4tcB3M7R8n5b9Myn0711PX2n6Q6/gX6RP4/x0sEDRV7ZPNNSlsdiaKCNbPRRTexuVTuE1HaIc/1BkFw45oKjcJjaDxtNN3rKNw8bUTfeQ8zGGZqDCmcHTI7OFSbpxXNUPniR4AZbZUObWtxEJH83H9w8+2HwelCsxzsyn6SciloJXcUomQFBjmk1Rg42Ivm2OzfbEl3DuApV2e2KGyhFJW1xyWACDI4ApP+MWT/pd2Z9yRfbLK/GqPKPRLgIVfaavSwM+h6kXeSj8ymgoxBG45ITOz5O/MGhiEn/ZXIgA71ZD7jEteAV7kzGm8jbIS3Xin3g6cyYu+P4Pf+3Pf2LYCmhz8LgRFfldTJUQckxoVjScRMhPyOCKkAd10Bsq7IrPqGK/n//lLT0dek1AEHvo7dBHzBDBchu9K97T4uJm/i/pviFfaFAvoCMdU/okrgIpJl5UnG1ETuVFlbe4cJltr7 Qxyc7mbH zTPdGsc+1OZxegWvlsvzgq2mL1XWTm2xxBJ/s3g3utYhYlNGmUrOHIDuD//9oXKkkivyfwP1D0+GAs1UNspBdAsrW3/r+rX7mwFKa4Yj+i/E+Lu7edTYQYOzIqiRaNrDXFronHKx9D8ZtF0PsZAZjYb21AOt3+smiNJdtdfmS1hFULmCOlMzuIUXaFwbyWYxV5Zgaop+sng8Cr1+s5jtJ+fOkuXswDWSCNsMOvoFyi92xEqxMYYN77QxVvExkzoHVvwofNLGJTPXgzmEPq67HPsR/ZFdg+bIPBsCqWNg1bjB37gC5A102XSoOJwgBClqKAWiHqCNWQMUnY4u7g66TM8I/gzI3jSSH88irzeC+VbNgqFD7b5ZtRnbMcz1dK7OfuJBHAdCjgWdn8WBCmLtzeWuNm/G1Ey1EWQ4ZpHoXPOiIGa0/1jbPVbY9SOMJPITm65jRLJ3b5t9apdxQ9hVFa71cc5hBF8sNmJmwGPyLrPHIdReQu7xW7FLUwg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Jul 27, 2023 at 08:55:53PM +0200, Alexandre Ghiti wrote: > This function used to simply flush the whole tlb of all harts, be more > subtile and try to only flush the range. > > The problem is that we can only use PAGE_SIZE as stride since we don't know > the size of the underlying mapping and then this function will be improved > only if the size of the region to flush is < threshold * PAGE_SIZE. > > Signed-off-by: Alexandre Ghiti > --- > arch/riscv/include/asm/tlbflush.h | 11 +++++----- > arch/riscv/mm/tlbflush.c | 35 +++++++++++++++++++++++-------- > 2 files changed, 32 insertions(+), 14 deletions(-) > > diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h > index f5c4fb0ae642..7426fdcd8ec5 100644 > --- a/arch/riscv/include/asm/tlbflush.h > +++ b/arch/riscv/include/asm/tlbflush.h > @@ -37,6 +37,7 @@ void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start, > void flush_tlb_page(struct vm_area_struct *vma, unsigned long addr); > void flush_tlb_range(struct vm_area_struct *vma, unsigned long start, > unsigned long end); > +void flush_tlb_kernel_range(unsigned long start, unsigned long end); > #ifdef CONFIG_TRANSPARENT_HUGEPAGE > #define __HAVE_ARCH_FLUSH_PMD_TLB_RANGE > void flush_pmd_tlb_range(struct vm_area_struct *vma, unsigned long start, > @@ -53,15 +54,15 @@ static inline void flush_tlb_range(struct vm_area_struct *vma, > local_flush_tlb_all(); > } > > -#define flush_tlb_mm(mm) flush_tlb_all() > -#define flush_tlb_mm_range(mm, start, end, page_size) flush_tlb_all() > -#endif /* !CONFIG_SMP || !CONFIG_MMU */ > - > /* Flush a range of kernel pages */ > static inline void flush_tlb_kernel_range(unsigned long start, > unsigned long end) > { > - flush_tlb_all(); > + local_flush_tlb_all(); > } > > +#define flush_tlb_mm(mm) flush_tlb_all() > +#define flush_tlb_mm_range(mm, start, end, page_size) flush_tlb_all() > +#endif /* !CONFIG_SMP || !CONFIG_MMU */ > + > #endif /* _ASM_RISCV_TLBFLUSH_H */ > diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c > index 8017d2130e27..96aeacb269d5 100644 > --- a/arch/riscv/mm/tlbflush.c > +++ b/arch/riscv/mm/tlbflush.c > @@ -117,18 +117,27 @@ static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, > unsigned long size, unsigned long stride) > { > struct flush_tlb_range_data ftd; > - struct cpumask *cmask = mm_cpumask(mm); > - unsigned int cpuid; > + struct cpumask *cmask, full_cmask; > bool broadcast; > > - if (cpumask_empty(cmask)) > - return; > + if (mm) { > + unsigned int cpuid; > + > + cmask = mm_cpumask(mm); > + if (cpumask_empty(cmask)) > + return; > + > + cpuid = get_cpu(); > + /* check if the tlbflush needs to be sent to other CPUs */ > + broadcast = cpumask_any_but(cmask, cpuid) < nr_cpu_ids; > + } else { > + cpumask_setall(&full_cmask); > + cmask = &full_cmask; > + broadcast = true; > + } > > - cpuid = get_cpu(); > - /* check if the tlbflush needs to be sent to other CPUs */ > - broadcast = cpumask_any_but(cmask, cpuid) < nr_cpu_ids; > if (static_branch_unlikely(&use_asid_allocator)) { > - unsigned long asid = atomic_long_read(&mm->context.id) & asid_mask; > + unsigned long asid = mm ? atomic_long_read(&mm->context.id) & asid_mask : 0; > > if (broadcast) { > if (riscv_use_ipi_for_rfence()) { > @@ -162,7 +171,8 @@ static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, > } > } > > - put_cpu(); > + if (mm) > + put_cpu(); > } > > void flush_tlb_mm(struct mm_struct *mm) > @@ -194,6 +204,13 @@ void flush_tlb_range(struct vm_area_struct *vma, unsigned long start, > __flush_tlb_range(vma->vm_mm, > start, end - start, 1 << stride_shift); > } > + > +void flush_tlb_kernel_range(unsigned long start, > + unsigned long end) No need to wrap this line. > +{ > + __flush_tlb_range(NULL, start, end, PAGE_SIZE); > +} > + > #ifdef CONFIG_TRANSPARENT_HUGEPAGE > void flush_pmd_tlb_range(struct vm_area_struct *vma, unsigned long start, > unsigned long end) > -- > 2.39.2 > Otherwise, Reviewed-by: Andrew Jones Thanks, drew