From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17EB3C021B8 for ; Tue, 4 Mar 2025 11:52:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A2C4628000C; Tue, 4 Mar 2025 06:52:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9B45C280005; Tue, 4 Mar 2025 06:52:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 82DA028000C; Tue, 4 Mar 2025 06:52:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 5FDD9280005 for ; Tue, 4 Mar 2025 06:52:43 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 18BB7161BBD for ; Tue, 4 Mar 2025 11:52:43 +0000 (UTC) X-FDA: 83183706606.03.D984137 Received: from mail.alien8.de (mail.alien8.de [65.109.113.108]) by imf09.hostedemail.com (Postfix) with ESMTP id 17D6D14000D for ; Tue, 4 Mar 2025 11:52:40 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=alien8.de header.s=alien8 header.b=ja9i780D; spf=pass (imf09.hostedemail.com: domain of bp@alien8.de designates 65.109.113.108 as permitted sender) smtp.mailfrom=bp@alien8.de; dmarc=pass (policy=none) header.from=alien8.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741089161; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=t86EropyYdY0BNiy/V2dzJxb6W+n+s3AAWsyjnhsK7k=; b=DGFrF988dvtIHqoBiH6h6c6Br95WJ0wFDDwDeghnF/KB76+ormcGs8iZ3nrAvZNvtBP5VN GlF+HeULCMvWQfxXsardU1kJec50TY45ONvL1rLKlATY+qlMk3dGDCaQUwlHM+4Nx8O1S1 rUO6YzUqXy5v6H5WAT/lgUGrxWncJfc= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=alien8.de header.s=alien8 header.b=ja9i780D; spf=pass (imf09.hostedemail.com: domain of bp@alien8.de designates 65.109.113.108 as permitted sender) smtp.mailfrom=bp@alien8.de; dmarc=pass (policy=none) header.from=alien8.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741089161; a=rsa-sha256; cv=none; b=HoxL+Kg+96TvVPEjSsb4uyZPH6AgHCfEDgqVQEUXay9i2hcm9cW7OCU+GuC0ekwhPGLNoq fAVSU8hYf2KLq6bIUKZE5Mvac78wIYUWuLMJTUoY+eFo7BuMnt6c6SlTc7op0NvnAB8UaO SAZyUQq1xseg7szid/BZzvNL+QmK6qI= Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.alien8.de (SuperMail on ZX Spectrum 128k) with ESMTP id 3B45840E0176; Tue, 4 Mar 2025 11:52:38 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at mail.alien8.de Received: from mail.alien8.de ([127.0.0.1]) by localhost (mail.alien8.de [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id KJ37BVY-cF9M; Tue, 4 Mar 2025 11:52:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alien8.de; s=alien8; t=1741089154; bh=t86EropyYdY0BNiy/V2dzJxb6W+n+s3AAWsyjnhsK7k=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=ja9i780DKJQi1kl/pDe+GFUTej5jzVmzpl1RjJwT7+bSwGzH5ATB2Z0VVZEPUtpy/ H8V4Jiv/+K65D7XJiuQv33J+GRod7iZpddqUpdD7bd+1rRzdS7G9InFPepHsbYYvj6 TVZ7PgZ1a2XsgKHAckyqHT+LPK1WImDvBjaGn1HVG0xuVdQwaO6hX//g6Ky8Ru8F9D vOCBgUwS3yo4kmc2hd3WQcu8yauXIpQgAUNoSH2nKl1RYSV1c+9UFpz+93vvlJRVhc 5xmjp4Oqs43GUD7KiWp0PvaZCRQ0EVffBx2dmO3Ldz1jVkRbsvUhchS4oe4JTVYTA9 CHfHNTLY8aglPtwdcW6ViqSgu747Ooz+hi1Y7X3UjIGJyQ1kjT3zBHxkKKwae3iMs6 B2QUGJBMEWmQtgoMJpWAdV5IA+EwkPTi/9OGTkC/h3j/dsGNZ4BcRlf5F/NA1kKeN6 7NAqCpNDoJeehFddrHg9+i3Q9A9I0bu5sUj0WsBLLTqvHoOHeVxuUlR4kHpBs7Taix TBPLwLViAKOS9o5rX+dFw2HRMSuaPq+U4bpwG8SO/gBru7GUdUhzrXSIxFciYfjpM1 Eo2FWT2XArWR1AJ858XwmA9LpJ08RB6AfNQ++KMntg+DV+7fjBDZ+Z+K6VJO7/RA5b H3jIJlnIg4ACfHNTcl0DhCVM= Received: from zn.tnic (pd95303ce.dip0.t-ipconnect.de [217.83.3.206]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature ECDSA (P-256) server-digest SHA256) (No client certificate requested) by mail.alien8.de (SuperMail on ZX Spectrum 128k) with ESMTPSA id AB5D840E020E; Tue, 4 Mar 2025 11:52:15 +0000 (UTC) Date: Tue, 4 Mar 2025 12:52:07 +0100 From: Borislav Petkov To: Dave Hansen Cc: Rik van Riel , x86@kernel.org, linux-kernel@vger.kernel.org, peterz@infradead.org, dave.hansen@linux.intel.com, zhengqi.arch@bytedance.com, nadav.amit@gmail.com, thomas.lendacky@amd.com, kernel-team@meta.com, linux-mm@kvack.org, akpm@linux-foundation.org, jackmanb@google.com, jannh@google.com, mhklinux@outlook.com, andrew.cooper3@citrix.com, Manali.Shukla@amd.com, mingo@kernel.org Subject: Re: [PATCH v14 11/13] x86/mm: do targeted broadcast flushing from tlbbatch code Message-ID: <20250304115207.GFZ8bpZ83u7L9x43Rq@fat_crate.local> References: <20250226030129.530345-1-riel@surriel.com> <20250226030129.530345-12-riel@surriel.com> <20250303114618.GBZ8WWihMDjf-oy8P0@fat_crate.local> <7e1ca8c7-6f3e-44dc-9dd8-bd273a03a83e@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <7e1ca8c7-6f3e-44dc-9dd8-bd273a03a83e@intel.com> X-Rspam-User: X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 17D6D14000D X-Stat-Signature: 8kw4piqb5doba7tur4rhur7e7wyhgugk X-HE-Tag: 1741089160-509021 X-HE-Meta: U2FsdGVkX18FHHMNNMbI5zLIKRVgWFa32WxpYZm7miU20a4hKiA8sN4KJAhRnzTNscH+KQL6BWMgtqEQC0sEJPXasz1H0gWYU5wMS9obMEmCDTuQhCHcq9NyVaxtYaSeZdxLn1IXq+2Qd1uiKffIUAdliL4cbyGkmehSrzFcJpynxWEDKrqlCfEuiaj5S1seV5Xh62pXoDY0aZfhalQA4a9mGqHfAe6qO3Bg2i8+fsRpaGyxs/NZqTB+qEwC3Qa2DugLZZsXloUBwcFSffmZIYlKvZNgkfL7v1gQhiBj9/w1Bat3lXcnAifFEFHruF3ptYyYyeiqUqCoKcN+9YDvh8QPZFqCbRk08wnBxw9Yz7xP88VcpBoD+oaxeJGX7aSIS/QEus7rRc3vrEOWeNW9uo8howHmVfbCLWJMDbIX5Y7CFWmAh9bQTQyojlqODF6RB++kJlLvGLG7L76TLgAqUcMPuuXChrCRi19q10GoNz9X+OQqSIwSDHD8AOrK/jNM1AVJJpvL0taO70QmpKt/QQ2koXLZSwhaEkYMa05t//ILdg12USE7ZjtU9k+pAZCHW8yi5I3IRSQUvE5wRSiTxyRqjHS2UgwKo7eLaJkNXP50qxQcL0Av8YB/kOitSMxccO5OTVb2W/D/gpds5cBd4AexZJWYkG8YnNunFwdBL7kKsEz4Y8D7OtQANfYRCbIhD10sgacpChCniLHeH5Nv03hNJGjGt6lsbgATw5tXl0Fti4uDY65u1/oiB3ELJHLIqt+EvzaQOvIcoHDrwwUakrLX3Cq1QOPvAyQvNMi1Y2HwMNH/ckVytvSmPtSJl96X0Tmw6+LVOzWZa9Hx1cRNuE9xDgEIK620YaRbWjEbwvTHKJIzzW0VHAbm+Mrv3hJSLXWM4Wr3S1C4TqJaJvNk5JaSb/I//OwkqrUWduGwRLWN5Y19wmDA1dKfz5LAA5D+jLrUym+yZ8ZRuWbEzN5 uR6/WxFp EgP7rTxZpwtVrccGBO+oqYjii/C1TfUhaAnLg0kFKJV0jvq8KQXMEZMLkyBf0/H/538zOQkN7A1EamycKWZes5IAuJ8q01bBOSWi0k6ZN8xG/3a1T/2RLuEP2vnTe55F1g3IfrO7s3FDFwrgfu7CRwdcRcQKLYbyQXYdl7jQCRt/Vgm2YKixT8HxStK2MaMknUaFz/0SlVluolkkJVsDdJhmzTeRvwt/XkVSP4oaDojolNzwbraOiTg4d3eEZL00tJ/MDnmIKGIZVkLTdrXZ6ab+cEN/AWcrcWvi4o304M3IbNFQ9pgySzh9Qx+hwhy/ewakrZdfXd6PC8NI3oTPZPm76og== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Mar 03, 2025 at 01:47:42PM -0800, Dave Hansen wrote: > > +static inline void invlpgb_flush_addr_nosync(unsigned long addr, u16 nr) > > +{ > > + __invlpgb_flush_addr_nosync(addr, nr); > > + if (!cpu_need_tlbsync()) > > + cpu_set_tlbsync(true); > > +} > > One thought on these: > > Instead of having three functions: > > 1. A raw __invlpgb_*_nosync() > 2. A wrapper invlpgb_*_nosync() that flips cpu_set_tlbsync() > 3. A wrapper invlpgb_*() > > Could we get away with just two? For instance, what if we had *ALL* > __invlpgb()'s do cpu_set_tlbsync()? Then we'd universally call tlbsync(). > > static inline void invlpgb_flush_all_nonglobals(void) > { > guard(preempt)(); > __invlpgb(0, 0, 0, 1, NO_STRIDE, INVLPGB_MODE_ALL_NONGLOBALS); > tlbsync(); > } > > Then we wouldn't need any of those three new wrappers. The only downside > is that we'd end up with paths that logically do: > > __invlpgb() > cpu_set_tlbsync(true); > if (cpu_need_tlbsync()) { // always true > __tlbsync(); > cpu_set_tlbsync(true); > } > > In other words, a possibly superfluous set/check/clear of the > "need_tlbsync" state. But I'd expect that to be a pittance compared to > the actual cost of INVLPGB/TLBSYNC. Lemme see whether I can grasp properly what you mean: What you really want is for the _nosync() variants to set need_tlbsync, right? Because looking at all the call sites which do set tlbsync, the flow is this: __invlpgb_flush_user_nr_nosync(pcid, addr, nr, pmd_stride, freed_tables); if (!cpu_need_tlbsync()) cpu_set_tlbsync(true); So we should move that cpu_set_tlbsync(true); into the __invlpgb_*_nosync() variant. And in order to make it even simpler, we should drop the testing too: IOW, this: /* Flush all mappings for a given PCID, not including globals. */ static inline void __invlpgb_flush_single_pcid_nosync(unsigned long pcid) { __invlpgb(0, pcid, 0, 1, 0, INVLPGB_PCID); cpu_set_tlbsync(true); } Right? > > static void broadcast_tlb_flush(struct flush_tlb_info *info) > > { > > bool pmd = info->stride_shift == PMD_SHIFT; > > @@ -790,6 +821,8 @@ void switch_mm_irqs_off(struct mm_struct *unused, struct mm_struct *next, > > if (IS_ENABLED(CONFIG_PROVE_LOCKING)) > > WARN_ON_ONCE(!irqs_disabled()); > > > > + tlbsync(); > > This one is in dire need of comments. Maybe this: diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 08672350536f..b97249ffff1f 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -822,6 +822,9 @@ void switch_mm_irqs_off(struct mm_struct *unused, struct mm_struct *next, if (IS_ENABLED(CONFIG_PROVE_LOCKING)) WARN_ON_ONCE(!irqs_disabled()); + /* + * Finish any remote TLB flushes pending from this CPU: + */ tlbsync(); /* > Ditto, *especially* if this hits the init_mm state. There really > shouldn't be any deferred flushes for the init_mm. Basically what you said but as a comment. :-P Merged in the rest. Thx. -- Regards/Gruss, Boris. https://people.kernel.org/tglx/notes-about-netiquette