From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A686C5475B for ; Mon, 11 Mar 2024 03:07:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A2A656B006E; Sun, 10 Mar 2024 23:07:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9B3936B0072; Sun, 10 Mar 2024 23:07:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 82CC36B0074; Sun, 10 Mar 2024 23:07:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 6D3046B006E for ; Sun, 10 Mar 2024 23:07:05 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 284A340CD1 for ; Mon, 11 Mar 2024 03:07:05 +0000 (UTC) X-FDA: 81883271610.21.EF1569D Received: from fhigh8-smtp.messagingengine.com (fhigh8-smtp.messagingengine.com [103.168.172.159]) by imf27.hostedemail.com (Postfix) with ESMTP id 3AE4340011 for ; Mon, 11 Mar 2024 03:07:03 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=fastmail.com header.s=fm1 header.b=PIybsOHk; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=OYgk8EXR; dmarc=pass (policy=none) header.from=fastmail.com; spf=pass (imf27.hostedemail.com: domain of sorear@fastmail.com designates 103.168.172.159 as permitted sender) smtp.mailfrom=sorear@fastmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710126423; a=rsa-sha256; cv=none; b=mSaYfMSUhICiD9+6GA0prgy16lKCD8Jl5NtGwXlH8vKrQYJ80K58X62ItpLxtacLnLz854 ExH2F3UGyLIPoA3ULTQN+sYfUIbogmeJL/p/kOXy3Vk52LNAdVYhMQ7d8G/A8qgpH5f+0C gcyCevCp2hLP5qmIpXGBPZklLwZcl4I= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=fastmail.com header.s=fm1 header.b=PIybsOHk; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=OYgk8EXR; dmarc=pass (policy=none) header.from=fastmail.com; spf=pass (imf27.hostedemail.com: domain of sorear@fastmail.com designates 103.168.172.159 as permitted sender) smtp.mailfrom=sorear@fastmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710126423; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=O5M6nvFrUXdurJlufzvKykPOM0SFgbJMl6kJtFomcZE=; b=CaESdSevsE7wueGF+nxy9mIjo6gcg7r8wr6mbI69SOdAbdnTgJWae8hVu1Q7CLvxSyojBh zQMSl3k3OCsCUfjBcClxcn8HfHDnMj7f+dhWzzWUaMSDWn6ufHvQiscGU5CfYRwcarqxZS j4XQEwZLZURi7OCHnPtKWUxByrdEnv8= Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailfhigh.nyi.internal (Postfix) with ESMTP id 726AC114008E; Sun, 10 Mar 2024 23:07:02 -0400 (EDT) Received: from imap50 ([10.202.2.100]) by compute3.internal (MEProxy); Sun, 10 Mar 2024 23:07:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastmail.com; h= cc:cc:content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm1; t=1710126422; x=1710212822; bh=O5M6nvFrUX durJlufzvKykPOM0SFgbJMl6kJtFomcZE=; b=PIybsOHk1FfZdRNhQFfvvtxphA LL/FidYvlR8T0OKcrjy5OL5KL+ZgG971AlMX4ifDh1FQduuiI0bVu8kbWTwVORjr vOkrWCNrrzf21YLWggFdbV/dW1IrUEW3yB1ZoyAxCppVWBdfMjKrw0WnYYhj8DUr dQnnz0oDYpYCfKo6jKMgC6hKcE4LvLNtnkscDcWGlIbF9z0NiThIG/gRk6kX0o8t SMTYNFTFWUrUEFcjluL9EGCSTNpLHrIWHRFfjygFj01u7RNbt1obis3Z2OwfuUtk VCESDcGLhOQKyKC2EZH5eWCrhP/Ip4sMi8YKzdQhibnSIRT/XEAORwrjOQpg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1710126422; x=1710212822; bh=O5M6nvFrUXdurJlufzvKykPOM0SF gbJMl6kJtFomcZE=; b=OYgk8EXRjqsazyh6FTvqjPYsBnForTuXapNTLnI1scOv YCKhmTqRVAwaE09LB2keJoMqV2bj6/fuZOPkQ+tNqTSRgzXVnY1GIWNm+zoXePPw l9ZZZwI3S8Wej19IIjIWUmAsrd7f/g3pmPXdiLXSqU/8a6/sm8WAl6MRgYJOiCSa L9DfyjA2owZxGrbL5Q8pwnAQLh4Vr/afuCPVGK1T5gui1Ob8KAIyNnDrRpMRPiDM KflsYmXPCVoYnAn2x9gWkeZENC19mowPCpxK4q7SxGnML7v1orPMor2UhXelyuHO 3HVaJW9F1Y1qNwPr23pmMXGiXS9yFhwbVRGGqkQGgQ== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrjedtgdehiecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefofgggkfgjfhffhffvvefutgesthdtredtreertdenucfhrhhomhepfdfuthgv fhgrnhcuqfdktfgvrghrfdcuoehsohhrvggrrhesfhgrshhtmhgrihhlrdgtohhmqeenuc ggtffrrghtthgvrhhnpeejueehgedtueetgefhheejjeeigffhieefjeehuddvueegtdfh heevgfeggfektdenucffohhmrghinhepihhnfhhrrgguvggrugdrohhrghenucevlhhush htvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehsohhrvggrrhesfhgr shhtmhgrihhlrdgtohhm X-ME-Proxy: Feedback-ID: i84414492:Fastmail Received: by mailuser.nyi.internal (Postfix, from userid 501) id D8AA81700093; Sun, 10 Mar 2024 23:07:01 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.11.0-alpha0-251-g8332da0bf6-fm-20240305.001-g8332da0b MIME-Version: 1.0 Message-Id: <10c4851d-a8d8-4289-a9f9-509c70a0ca22@app.fastmail.com> In-Reply-To: <20240229232211.161961-4-samuel.holland@sifive.com> References: <20240229232211.161961-1-samuel.holland@sifive.com> <20240229232211.161961-4-samuel.holland@sifive.com> Date: Sun, 10 Mar 2024 23:06:41 -0400 From: "Stefan O'Rear" To: "Samuel Holland" , "Palmer Dabbelt" , linux-riscv@lists.infradead.org Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, "Alexandre Ghiti" , "Jisheng Zhang" , "yunhui cui" Subject: Re: [PATCH v5 03/13] riscv: Use IPIs for remote cache/TLB flushes by default Content-Type: text/plain X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 3AE4340011 X-Stat-Signature: qqo8x6cuuhysjnd7htb5bmf7bq5g84xk X-HE-Tag: 1710126423-699751 X-HE-Meta: U2FsdGVkX19QCZ2X06wDzDoH6LFe6J33zjhjY6CqkicB59KtXsIUJme4rJvtz5of9C6iENeCw8Pox9BcHw9Kg4315W10chd4z6L72aD9Id8WYwckXjCBvT8ei4o0ovoE5ISoWV8dFvSaWwe94a6SM64Mc2lyxr+xnFIxQ0Z7r5QLx65RiEFTc9Eb7yCMOo8I/9nvbs0k9RG5mocBDVA6SSJz9FSizu9yUhywnSojB+S2XVENey+FZSugwiyYwRgWtMGJ9fLAWnIgA08XoiruKvYzYo9covunb5K7po4g5zmk4Z2T3uI2MijqzhI/d31272WETRh4K1yUjOXqIwqmCu8i59Qj6pOZVjR7GPRddBbrOpnIxatxI1EO+Tr4rCYbU+qgtn0SputRWv1lAs4/kGpxpnu2PHW6eHcNJ+Euc0mxJy09aGMrgLpKaoGP/1IIWyiaqij2XpL+3/1rdzrLxjAAmYiCdvNUpPTZvl3W/CVnMv0GvWb7rDQtyERMPMc8v82nhhWuOSeuO+pm5IuOuucmAVayMpwSWA6cXAb/Q1kmsEHzhFX/eNnHZYb3VUw6vmcVxqLk0Srz92vx2pCsTCjodehVcaUplBSwwnbaf/E0jx4SrqiecRcC8+vKIFP7Us98I7n9344g8BvNdlZGMIY49NagrtMtvMbM5Y0hdIL7JXgRs/A5mwpJCBFNLuZlrLUPopAtsSqjOsyjRIvTB7XyODN5MwmruBLldEVfilnwlIwaBHM61m9mkTMaiCfXkuIGjFGqBkID66vwsyuvTZZbOzoSk086IpR4b+mFWxYuUElk8yfpd04U14xLZP5CoTUzesGJDvGpcSp2RRnoB1mGyxlBqPNajkArg4qeeGdym20KJEaZTLXKSwJ8BdHjp7CFEej1HiW42Hqss+1/s7jDlIrnvhK+LrC7FE97qiqxOeJGUwd3FzFvaeD1lEAxOySbuleWj7Lmvwg5hg8 RDw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Feb 29, 2024, at 6:21 PM, Samuel Holland wrote: > An IPI backend is always required in an SMP configuration, but an SBI > implementation is not. For example, SBI will be unavailable when the > kernel runs in M mode. > > Generally, IPIs are assumed to be faster than SBI calls due to the SBI > context switch overhead. However, when SBI is used as the IPI backend, > then the context switch cost must be paid anyway, and performing the > cache/TLB flush directly in the SBI implementation is more efficient > than inserting an interrupt to the kernel. This is the only scenario > where riscv_ipi_set_virq_range()'s use_for_rfence parameter is false. > > Thus, it makes sense for remote fences to use IPIs by default, and make > the SBI remote fence extension the special case. The historical intention of providing SBI calls for remote fences is that it abstracts over hardware that allows for performing remote fences _without an IPI at all_. The TH1520 is an example of such hardware, since it can (at least according to the documentation) perform broadcast fence, fence.i, and sfence.vma operations using bits in the mhint register. T-Head's public opensbi repository doesn't actually use this feature, and in general SBI remote fences come from a much more optimistic time about how much we can successfully hide from supervisor software. But I don't think we can generalize that an IPI will always do less work than a SBI remote fence. -s > sbi_ipi_init() already checks riscv_ipi_have_virq_range(), so it only > calls riscv_ipi_set_virq_range() when no other IPI device is available. > So we can move the static key and drop the use_for_rfence parameter. > > Furthermore, the static branch only makes sense when CONFIG_RISCV_SBI is > enabled. Optherwise, IPIs must be used. Add a fallback definition of > riscv_use_sbi_for_rfence() which handles this case and removes the need > to check CONFIG_RISCV_SBI elsewhere, such as in cacheflush.c. > > Signed-off-by: Samuel Holland > --- > > Changes in v5: > - Also switch to riscv_use_sbi_for_rfence() in asm/pgalloc.h > > Changes in v4: > - New patch for v4 > > arch/riscv/include/asm/pgalloc.h | 7 ++++--- > arch/riscv/include/asm/sbi.h | 4 ++++ > arch/riscv/include/asm/smp.h | 15 ++------------- > arch/riscv/kernel/sbi-ipi.c | 11 ++++++++++- > arch/riscv/kernel/smp.c | 11 +---------- > arch/riscv/mm/cacheflush.c | 5 ++--- > arch/riscv/mm/tlbflush.c | 31 ++++++++++++++----------------- > drivers/clocksource/timer-clint.c | 2 +- > 8 files changed, 38 insertions(+), 48 deletions(-) > > diff --git a/arch/riscv/include/asm/pgalloc.h b/arch/riscv/include/asm/pgalloc.h > index 87468f67951a..6578054977ef 100644 > --- a/arch/riscv/include/asm/pgalloc.h > +++ b/arch/riscv/include/asm/pgalloc.h > @@ -8,6 +8,7 @@ > #define _ASM_RISCV_PGALLOC_H > > #include > +#include > #include > > #ifdef CONFIG_MMU > @@ -90,10 +91,10 @@ static inline pud_t *pud_alloc_one(struct mm_struct > *mm, unsigned long addr) > > static inline void riscv_tlb_remove_ptdesc(struct mmu_gather *tlb, void *pt) > { > - if (riscv_use_ipi_for_rfence()) > - tlb_remove_page_ptdesc(tlb, pt); > - else > + if (riscv_use_sbi_for_rfence()) > tlb_remove_ptdesc(tlb, pt); > + else > + tlb_remove_page_ptdesc(tlb, pt); > } > > #define pud_free pud_free > diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h > index 6e68f8dff76b..ea84392ca9d7 100644 > --- a/arch/riscv/include/asm/sbi.h > +++ b/arch/riscv/include/asm/sbi.h > @@ -375,8 +375,12 @@ unsigned long riscv_cached_marchid(unsigned int cpu_id); > unsigned long riscv_cached_mimpid(unsigned int cpu_id); > > #if IS_ENABLED(CONFIG_SMP) && IS_ENABLED(CONFIG_RISCV_SBI) > +DECLARE_STATIC_KEY_FALSE(riscv_sbi_for_rfence); > +#define riscv_use_sbi_for_rfence() \ > + static_branch_unlikely(&riscv_sbi_for_rfence) > void sbi_ipi_init(void); > #else > +static inline bool riscv_use_sbi_for_rfence(void) { return false; } > static inline void sbi_ipi_init(void) { } > #endif > > diff --git a/arch/riscv/include/asm/smp.h b/arch/riscv/include/asm/smp.h > index 0d555847cde6..7ac80e9f2288 100644 > --- a/arch/riscv/include/asm/smp.h > +++ b/arch/riscv/include/asm/smp.h > @@ -49,12 +49,7 @@ void riscv_ipi_disable(void); > bool riscv_ipi_have_virq_range(void); > > /* Set the IPI interrupt numbers for arch (called by irqchip drivers) */ > -void riscv_ipi_set_virq_range(int virq, int nr, bool use_for_rfence); > - > -/* Check if we can use IPIs for remote FENCEs */ > -DECLARE_STATIC_KEY_FALSE(riscv_ipi_for_rfence); > -#define riscv_use_ipi_for_rfence() \ > - static_branch_unlikely(&riscv_ipi_for_rfence) > +void riscv_ipi_set_virq_range(int virq, int nr); > > /* Check other CPUs stop or not */ > bool smp_crash_stop_failed(void); > @@ -104,16 +99,10 @@ static inline bool riscv_ipi_have_virq_range(void) > return false; > } > > -static inline void riscv_ipi_set_virq_range(int virq, int nr, > - bool use_for_rfence) > +static inline void riscv_ipi_set_virq_range(int virq, int nr) > { > } > > -static inline bool riscv_use_ipi_for_rfence(void) > -{ > - return false; > -} > - > #endif /* CONFIG_SMP */ > > #if defined(CONFIG_HOTPLUG_CPU) && (CONFIG_SMP) > diff --git a/arch/riscv/kernel/sbi-ipi.c b/arch/riscv/kernel/sbi-ipi.c > index a4559695ce62..1026e22955cc 100644 > --- a/arch/riscv/kernel/sbi-ipi.c > +++ b/arch/riscv/kernel/sbi-ipi.c > @@ -13,6 +13,9 @@ > #include > #include > > +DEFINE_STATIC_KEY_FALSE(riscv_sbi_for_rfence); > +EXPORT_SYMBOL_GPL(riscv_sbi_for_rfence); > + > static int sbi_ipi_virq; > > static void sbi_ipi_handle(struct irq_desc *desc) > @@ -72,6 +75,12 @@ void __init sbi_ipi_init(void) > "irqchip/sbi-ipi:starting", > sbi_ipi_starting_cpu, NULL); > > - riscv_ipi_set_virq_range(virq, BITS_PER_BYTE, false); > + riscv_ipi_set_virq_range(virq, BITS_PER_BYTE); > pr_info("providing IPIs using SBI IPI extension\n"); > + > + /* > + * Use the SBI remote fence extension to avoid > + * the extra context switch needed to handle IPIs. > + */ > + static_branch_enable(&riscv_sbi_for_rfence); > } > diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c > index 45dd4035416e..8e6eb64459af 100644 > --- a/arch/riscv/kernel/smp.c > +++ b/arch/riscv/kernel/smp.c > @@ -171,10 +171,7 @@ bool riscv_ipi_have_virq_range(void) > return (ipi_virq_base) ? true : false; > } > > -DEFINE_STATIC_KEY_FALSE(riscv_ipi_for_rfence); > -EXPORT_SYMBOL_GPL(riscv_ipi_for_rfence); > - > -void riscv_ipi_set_virq_range(int virq, int nr, bool use_for_rfence) > +void riscv_ipi_set_virq_range(int virq, int nr) > { > int i, err; > > @@ -197,12 +194,6 @@ void riscv_ipi_set_virq_range(int virq, int nr, > bool use_for_rfence) > > /* Enabled IPIs for boot CPU immediately */ > riscv_ipi_enable(); > - > - /* Update RFENCE static key */ > - if (use_for_rfence) > - static_branch_enable(&riscv_ipi_for_rfence); > - else > - static_branch_disable(&riscv_ipi_for_rfence); > } > > static const char * const ipi_names[] = { > diff --git a/arch/riscv/mm/cacheflush.c b/arch/riscv/mm/cacheflush.c > index 55a34f2020a8..47c485bc7df0 100644 > --- a/arch/riscv/mm/cacheflush.c > +++ b/arch/riscv/mm/cacheflush.c > @@ -21,7 +21,7 @@ void flush_icache_all(void) > { > local_flush_icache_all(); > > - if (IS_ENABLED(CONFIG_RISCV_SBI) && !riscv_use_ipi_for_rfence()) > + if (riscv_use_sbi_for_rfence()) > sbi_remote_fence_i(NULL); > else > on_each_cpu(ipi_remote_fence_i, NULL, 1); > @@ -69,8 +69,7 @@ void flush_icache_mm(struct mm_struct *mm, bool local) > * with flush_icache_deferred(). > */ > smp_mb(); > - } else if (IS_ENABLED(CONFIG_RISCV_SBI) && > - !riscv_use_ipi_for_rfence()) { > + } else if (riscv_use_sbi_for_rfence()) { > sbi_remote_fence_i(&others); > } else { > on_each_cpu_mask(&others, ipi_remote_fence_i, NULL, 1); > diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c > index 8d12b26f5ac3..0373661bd1c4 100644 > --- a/arch/riscv/mm/tlbflush.c > +++ b/arch/riscv/mm/tlbflush.c > @@ -78,10 +78,10 @@ static void __ipi_flush_tlb_all(void *info) > > void flush_tlb_all(void) > { > - if (riscv_use_ipi_for_rfence()) > - on_each_cpu(__ipi_flush_tlb_all, NULL, 1); > - else > + if (riscv_use_sbi_for_rfence()) > sbi_remote_sfence_vma_asid(NULL, 0, FLUSH_TLB_MAX_SIZE, FLUSH_TLB_NO_ASID); > + else > + on_each_cpu(__ipi_flush_tlb_all, NULL, 1); > } > > struct flush_tlb_range_data { > @@ -102,7 +102,6 @@ static void __flush_tlb_range(struct cpumask > *cmask, unsigned long asid, > unsigned long start, unsigned long size, > unsigned long stride) > { > - struct flush_tlb_range_data ftd; > bool broadcast; > > if (cpumask_empty(cmask)) > @@ -118,20 +117,18 @@ static void __flush_tlb_range(struct cpumask > *cmask, unsigned long asid, > broadcast = true; > } > > - if (broadcast) { > - if (riscv_use_ipi_for_rfence()) { > - ftd.asid = asid; > - ftd.start = start; > - ftd.size = size; > - ftd.stride = stride; > - on_each_cpu_mask(cmask, > - __ipi_flush_tlb_range_asid, > - &ftd, 1); > - } else > - sbi_remote_sfence_vma_asid(cmask, > - start, size, asid); > - } else { > + if (!broadcast) { > local_flush_tlb_range_asid(start, size, stride, asid); > + } else if (riscv_use_sbi_for_rfence()) { > + sbi_remote_sfence_vma_asid(cmask, start, size, asid); > + } else { > + struct flush_tlb_range_data ftd; > + > + ftd.asid = asid; > + ftd.start = start; > + ftd.size = size; > + ftd.stride = stride; > + on_each_cpu_mask(cmask, __ipi_flush_tlb_range_asid, &ftd, 1); > } > > if (cmask != cpu_online_mask) > diff --git a/drivers/clocksource/timer-clint.c > b/drivers/clocksource/timer-clint.c > index 09fd292eb83d..0bdd9d7ec545 100644 > --- a/drivers/clocksource/timer-clint.c > +++ b/drivers/clocksource/timer-clint.c > @@ -251,7 +251,7 @@ static int __init clint_timer_init_dt(struct > device_node *np) > } > > irq_set_chained_handler(clint_ipi_irq, clint_ipi_interrupt); > - riscv_ipi_set_virq_range(rc, BITS_PER_BYTE, true); > + riscv_ipi_set_virq_range(rc, BITS_PER_BYTE); > clint_clear_ipi(); > #endif > > -- > 2.43.1 > > > _______________________________________________ > linux-riscv mailing list > linux-riscv@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-riscv