From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 54405E77188 for ; Thu, 2 Jan 2025 12:43:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 927F76B00C1; Thu, 2 Jan 2025 07:43:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8D7A76B00C2; Thu, 2 Jan 2025 07:43:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 79FE86B00C3; Thu, 2 Jan 2025 07:43:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 5D3CA6B00C1 for ; Thu, 2 Jan 2025 07:43:16 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 0F51F12078E for ; Thu, 2 Jan 2025 12:43:16 +0000 (UTC) X-FDA: 82962476184.04.75781F1 Received: from mail.alien8.de (mail.alien8.de [65.109.113.108]) by imf22.hostedemail.com (Postfix) with ESMTP id DD814C000D for ; Thu, 2 Jan 2025 12:42:21 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=alien8.de header.s=alien8 header.b=EHEnAy7O; spf=pass (imf22.hostedemail.com: domain of bp@alien8.de designates 65.109.113.108 as permitted sender) smtp.mailfrom=bp@alien8.de; dmarc=pass (policy=none) header.from=alien8.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735821759; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cDzjNvcWPtVAenookZgfV5fkvpgx4xlHoGyoKP+XST8=; b=MSw2pG/gSN/h6MH8rWVKn47wOPTLEkc/bca/YQlBOZGAi4mMoWkTgMfSWWhlajqnjnI+Lk qVxSWInNYdWR4sNguxRdAvYxZbeasRJDH9R1XCD32s5X3y7EFBr9hv+8x1TyyOUz1bYiLK pmiRpQLHfUcKVzSyVr41DdxaN1HLaVA= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=alien8.de header.s=alien8 header.b=EHEnAy7O; spf=pass (imf22.hostedemail.com: domain of bp@alien8.de designates 65.109.113.108 as permitted sender) smtp.mailfrom=bp@alien8.de; dmarc=pass (policy=none) header.from=alien8.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735821760; a=rsa-sha256; cv=none; b=Hjzysd33KtecUCLSoM/hx11O3XiUYhCh48st/bHU80/DRbun69ZwYn20BHe4lNwINEUxiS 54CnES3Po+xqPffgRxcszdkNX7UJwfLUmYGZWBJ5QYJVbD2cg36dNsJwmGF/FtmRqtg7hK 3XOSwze6GvCTYt61rhw4VMIHolOKzJQ= Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.alien8.de (SuperMail on ZX Spectrum 128k) with ESMTP id 0395840E02C0; Thu, 2 Jan 2025 12:43:12 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at mail.alien8.de Received: from mail.alien8.de ([127.0.0.1]) by localhost (mail.alien8.de [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id Rc5_MselygjB; Thu, 2 Jan 2025 12:43:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alien8.de; s=alien8; t=1735821788; bh=cDzjNvcWPtVAenookZgfV5fkvpgx4xlHoGyoKP+XST8=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=EHEnAy7O9KXI2DIvcszCTwzy90AZ5ZY+V6OWocoYtK/x9hDptqI8dfhWZijjELDzf ZsEd7xXAvSfncY3+B4T66RTY6ZymrEIJBeQwE83E64e9tFB1FduKJL5pZfCxuzTM/l 8U0pEJuuW7nPkadrWuReJbgIYrEFp1Sizzy0COoauNExHYUXe/VU1EtSuDXgMHvJk0 RfT5BRfLozouUIP0x1XORoQS1DFPICk93g3H5Jder9MPG9N9MTqw7XVHTiB5UmTlol LAr6PD9a4C/VHpMtweRbUmgJ6PP8TxrJwjNz5YQdMYog4XS19xmZoj2tgYHnPLDd9Y tlWPozpYi1+kq8bS5gTEfMLzUEw5KnJ/DRriTaCeNFxWH4xLAkIwkPB88CZ5/hqCIV 9qx12nsJY6b2nQtGa8Kq74GElfMS+bJlqUiVOaGLRvvicPjRME7aVEytjxp7lMr22A RHj6FQGhv23MsZZlDEPpnA1LvN6P067JPHaQb82v9mGH2/nB5DOZsKn5/SvhMRR9mv Gi1XkmoeNlmMxIMJW1b1lQr58xH9tAaR5FE2bWnusr1ZEpL2TWU44k9BA90LT6SRrg qGKkZ933zaaxPCrwgBQa69cBxS2BUQmKB5Z+cc00syhMcDTdakM0oVsPLNeHT0szqB DSoUzOFOWSVigAZSL1dLuNsg= Received: from zn.tnic (p200300ea971F9374329c23FFFEa6A903.dip0.t-ipconnect.de [IPv6:2003:ea:971f:9374:329c:23ff:fea6:a903]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature ECDSA (P-256) server-digest SHA256) (No client certificate requested) by mail.alien8.de (SuperMail on ZX Spectrum 128k) with ESMTPSA id B1CB640E0289; Thu, 2 Jan 2025 12:42:53 +0000 (UTC) Date: Thu, 2 Jan 2025 13:42:47 +0100 From: Borislav Petkov To: Rik van Riel Cc: x86@kernel.org, linux-kernel@vger.kernel.org, kernel-team@meta.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, tglx@linutronix.de, mingo@redhat.com, hpa@zytor.com, akpm@linux-foundation.org, nadav.amit@gmail.com, zhengqi.arch@bytedance.com, linux-mm@kvack.org Subject: Re: [PATCH 05/12] x86/mm: add INVLPGB support code Message-ID: <20250102124247.GPZ3aJx8JTJa6PcaOW@fat_crate.local> References: <20241230175550.4046587-1-riel@surriel.com> <20241230175550.4046587-6-riel@surriel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20241230175550.4046587-6-riel@surriel.com> X-Rspamd-Server: rspam05 X-Stat-Signature: 55izpaacfy4qom3jz3yg45rrkfqtqstd X-Rspamd-Queue-Id: DD814C000D X-Rspam-User: X-HE-Tag: 1735821741-89130 X-HE-Meta: U2FsdGVkX19dDLmv0ebcWhYmYIcSKRB074jZG5I4lcg+URsA2SBIQQkLDMWNyTuKo4WHM04ksXyIpTxWgP1pBgdSCtfz/kVpPx+vNUhslrb1SFJ04LShe2IxEGaQ56skzh02wS5Y8i5YWIpg2li1IVWyb/ovj1dbO828lp6dpl5ppe0B7WS7uGa0PQD6ESyrfQ5TUvO7gOIKHiW65ih6N6StQ5Mh5rFglnd1q6QeAEXDyZm8QqzG7stilZgvSX8SHKO12fr9Xo3XNPD0DQPivtTKb+IsTmFSkvrx7Yv5NrYn3yNEgIG2tD770Q977xz2gFUi73lN9RhBmv8wiy9OZrUdg08hQAaSdvULgxfI3eFtpNlQYGX1nWHHSXjFhcBt0FEFRcQgaU4SRcZrLz/8pIUlAnXbQqGIGO4aTmntsANrW+8jhH9wUCkCWNvgZU90/lr9SfUWqX+VfDYLTdTDzl3wTqtZTNea0Q//L06XVXqarNyiXbY4A2B2vZ5o3GmufakpnZ6MkYywdEgHsnXU6mfSGrVVceft7mJYQyn7vKNLPKiafExe3OK7Z1OVHSSYudSu6L3QAyqwDzr9NiMzG4wpR8iJmDcgIdjUtpiPHwwK6TRqhsS+iXOPl4xptVDkwJ+BHEHxXDFIxJ3w6jvFwjy6S1HjB9tWs7BWDrqX2B26ud41pZOqGEuenyX+KgEO8I8fuWoC+u2h9db4jswfVuGWEO4vkrj34dSh5c+HxB5A1RUrv5QAX3F0p66dtfCozCUwtSngm+Ju2WMA8Fo1DFekfYYBONAog4+S8B9mSmmDV+qhp+GTBUY+Yfkl+mCzEQc1xkoF5uPVrq280N+qcE6Kcg/sGWoV+Xt4T/vKzqEOadiP7jkb/XSnzcq4it7pJkb+bOilY8UbGi39IrFTk/gkU9xhYyAPIdrqZHrc3el4C8v9ZcXYCR5CH3wqY73RQFVaqPHOLOJOT9zmvx6 W36KsEKn jLuH40EdrQs5MUT3Cpmh4NSnq/TDwdIkbzyKBs0cmL7sPMGsw9PhYRI7Q6wJXzUjhO7CjzH9Xwy5XmdCMwRD8oLkQJNImP1t4C9pAQSE95rRQTJn6VlJk2o6yRk0zqlH74ir4GodQC/jGwQbMKgXpZNJ1gjlCVody4IBrxcNOmcjljeh6sgZaq/PyocT41lUNMGAutMj1nxZTUA/OQeVgVHpY/Q7ITq5tZrYRm2yaCoARyx2vGPEHTzf779/hvVX2sYyx4/IBeKrAlML+FP0oRU/r5crzWNF/+mAn6UWEtlFyHa5optYhn5Y6NZWwKRsJzUDOqwfb8qvnzMQzsMUMd2qINOjV2AAR2ufCqhhy89HFzKfmJie5WRG5NJ/aZZTeV9Xz X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Dec 30, 2024 at 12:53:06PM -0500, Rik van Riel wrote: > Add invlpgb.h with the helper functions and definitions needed to use > broadcast TLB invalidation on AMD EPYC 3 and newer CPUs. > > Signed-off-by: Rik van Riel > --- > arch/x86/include/asm/invlpgb.h | 93 +++++++++++++++++++++++++++++++++ > arch/x86/include/asm/tlbflush.h | 1 + > 2 files changed, 94 insertions(+) > create mode 100644 arch/x86/include/asm/invlpgb.h > > diff --git a/arch/x86/include/asm/invlpgb.h b/arch/x86/include/asm/invlpgb.h > new file mode 100644 > index 000000000000..862775897a54 > --- /dev/null > +++ b/arch/x86/include/asm/invlpgb.h I don't see the point for a separate header just for that. We have arch/x86/include/asm/tlb.h. > @@ -0,0 +1,93 @@ > +/* SPDX-License-Identifier: GPL-2.0 */ > +#ifndef _ASM_X86_INVLPGB > +#define _ASM_X86_INVLPGB > + > +#include > + > +/* > + * INVLPGB does broadcast TLB invalidation across all the CPUs in the system. > + * > + * The INVLPGB instruction is weakly ordered, and a batch of invalidations can > + * be done in a parallel fashion. > + * > + * TLBSYNC is used to ensure that pending INVLPGB invalidations initiated from > + * this CPU have completed. > + */ > +static inline void __invlpgb(unsigned long asid, unsigned long pcid, unsigned long addr, > + int extra_count, bool pmd_stride, unsigned long flags) See below. Once you prune the functions you're not using in your patchset, this argument list will drop too. We can always extend it later, if really needed, so let's keep it simple here. I had slimmed it down to this internally: static inline void invlpgb(unsigned long va, unsigned long count, unsigned long id) { /* INVLPGB; supported in binutils >= 2.36. */ asm volatile(".byte 0x0f, 0x01, 0xfe" : : "a" (va), "c" (count), "d" (id) : "memory"); } I had the memory clobber too but now that I think of it, it probably isn't needed because even if the compiler reorders INVLPGB, it is weakly-ordered anyway. TLBSYNC should probably have a memory clobber tho, to prevent the compiler from doing funky stuff... > +{ > + u64 rax = addr | flags; > + u32 ecx = (pmd_stride << 31) | extra_count; > + u32 edx = (pcid << 16) | asid; > + > + asm volatile("invlpgb" : : "a" (rax), "c" (ecx), "d" (edx)); No, you do: /* INVLPGB; supported in binutils >= 2.36. */ asm volatile(".byte 0x0f, 0x01, 0xfe" ... > +/* > + * INVLPGB can be targeted by virtual address, PCID, ASID, or any combination > + * of the three. For example: > + * - INVLPGB_VA | INVLPGB_INCLUDE_GLOBAL: invalidate all TLB entries at the address > + * - INVLPGB_PCID: invalidate all TLB entries matching the PCID ^^^^^^^^^^^^^^^^^^^^^^ Whitespace damage here. Needs tabs. > + * > + * The first can be used to invalidate (kernel) mappings at a particular > + * address across all processes. > + * > + * The latter invalidates all TLB entries matching a PCID. > + */ > +#define INVLPGB_VA BIT(0) > +#define INVLPGB_PCID BIT(1) > +#define INVLPGB_ASID BIT(2) > +#define INVLPGB_INCLUDE_GLOBAL BIT(3) > +#define INVLPGB_FINAL_ONLY BIT(4) > +#define INVLPGB_INCLUDE_NESTED BIT(5) Please add only the defines which are actually being used. Ditto for the functions. > +/* Wait for INVLPGB originated by this CPU to complete. */ > +static inline void tlbsync(void) > +{ > + asm volatile("tlbsync"); > +} /* TLBSYNC; supported in binutils >= 2.36. */ asm volatile(".byte 0x0f, 0x01, 0xff" ::: "memory"); > + > +#endif /* _ASM_X86_INVLPGB */ > diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h > index 7d1468a3967b..20074f17fbcd 100644 > --- a/arch/x86/include/asm/tlbflush.h > +++ b/arch/x86/include/asm/tlbflush.h > @@ -10,6 +10,7 @@ > #include > #include > #include > +#include > #include > #include > #include > -- > 2.47.1 > -- Regards/Gruss, Boris. https://people.kernel.org/tglx/notes-about-netiquette