From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02455C021B8 for ; Tue, 4 Mar 2025 13:59:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C17A828000A; Tue, 4 Mar 2025 08:59:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BC4EF280004; Tue, 4 Mar 2025 08:59:03 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A3EE128000A; Tue, 4 Mar 2025 08:59:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 85CDC280004 for ; Tue, 4 Mar 2025 08:59:03 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 3CC7B141CD9 for ; Tue, 4 Mar 2025 13:59:03 +0000 (UTC) X-FDA: 83184024966.02.9B5B7EE Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf09.hostedemail.com (Postfix) with ESMTP id 9895A140007 for ; Tue, 4 Mar 2025 13:59:01 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=li36mmNf; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf09.hostedemail.com: domain of bp@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=bp@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741096741; a=rsa-sha256; cv=none; b=XjKKfLwyfcos0/SDsFimQZodlsft0rqJywJ7R9Nfwcm2/CcYzVwPmbgK22mhyknPZg6LHH dguuHzyqUoYErMifxwRrzNPTtqTPcplkQ/yg5BM7UBtqIz1ISDx0gzpil0kqXghkRVgV4E 0/dyj97KeMdq38XiMrAIMgOF1mGau/Y= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=li36mmNf; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf09.hostedemail.com: domain of bp@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=bp@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741096741; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hOF/oNewM9EWYwLL2Lq6iBUPtoytGFygnrlu4K7b4n0=; b=uo+G+8Ub7JyWhx1PWxcKV8pRl9HJ0MCxDUVYiCyIgbq31M3/3lyeHNBsd8Gh9mwOBpFBDc 2KiAC7T5ZYsYhB1g1tezG64K+dQKl8e4vacq3w+gD0ScveAnbXsxqx9O09EVFlMnOjB/Vj ffuXqRZiW3Ahja/nzaX4dAhLEGIwA0E= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 1A7F35C5D33; Tue, 4 Mar 2025 13:56:44 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 95CCAC4CEE5; Tue, 4 Mar 2025 13:58:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1741096740; bh=9c2Sjy45ucPykbiLx09DjVwbvVa+jkeWfLgJg4HPpMw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=li36mmNfukBpxBkysoR3bxmyUpKSu6bxn6vCrX/WOwfkw6/ZMkyU2UBjUbzp9EIdO 0RKSgZ1sUYMKzoBN7WAh8ToBmY7S0k/5vfOnWOlJg6eXNl76M3spdD/h6G0WzMK4EL iuWOdPCneAD/eG7wnFtqJj8/DaY0dRcjc9NnfsT73A0Fj/t5WLW8blOsynLGlVKQNs q3aQELgkEn3cO2RBmwos/wAVpTSMjZhsQdVtP6n+25lF/6MA865p14lSRUdhus28BL DJcPDIszXJBAyDtNlg8Gd1H7/DkNUGyDnOVXtwiU1HhwLy7S6vbh8eMthWNqmqePmC uSSJ1Ogw0mwDw== From: Borislav Petkov To: riel@surriel.com Cc: Manali.Shukla@amd.com, akpm@linux-foundation.org, andrew.cooper3@citrix.com, jackmanb@google.com, jannh@google.com, kernel-team@meta.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, mhklinux@outlook.com, nadav.amit@gmail.com, thomas.lendacky@amd.com, x86@kernel.org, zhengqi.arch@bytedance.com, Borislav Petkov Subject: [PATCH v15 11/11] x86/mm: Enable AMD translation cache extensions Date: Tue, 4 Mar 2025 14:58:16 +0100 Message-ID: <20250304135816.12356-12-bp@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250304135816.12356-1-bp@kernel.org> References: <20250304135816.12356-1-bp@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 9895A140007 X-Stat-Signature: msbhbjfopicgsr7dg3kt5f4966uxxh3p X-Rspam-User: X-HE-Tag: 1741096741-534384 X-HE-Meta: U2FsdGVkX1/gNxbvYojBopIHaWow/e4FDi+1HsCQtXuCPw1+3z+VpxajCNZtZiZZoTHGEJpawVM6xxP3WgcwG3mcT8zrB0g/3z9d46GWd9ZgI/9HKH339Ndps4EqYJClj4eFZno5OCQ9BOjh12uQAQ/SU6MHBgZj5BuJbrD0zOl6MxxZDIumF3A/Bl+MNMPu+pSzY5tKC/nr9ojjHhxhTg3Q8BFoSvFyQhsw/EKQ/OGBGALyhL0f5ewhwG4AHOqf3myqAWQV/MhzLpWX+hOXnmQCwJ9pHMmd06uTrEcz+kKzbvaFlvMdJO6XOtodprhVgbAd9KdMJ/obhp9L8QMXGaDQkoqyEVSjmbs2LHgFy5GDWeRTobh5SeIiynR9wqIzyZV6cp0KHb0uJZnxIs15q0/gZgjEBBimga1xrX2aoEJPr6P3e0z1wPkTGj9FUz6sDiDzbwU/MOriFU5/7FxpIunIXsuLXIHsQocO+uTBcHFFRSHN93n49aXvt/xkfRoi8MEzBY/t8nXS4d96lMrAF5pJ97mGd8q+WNdhCeD9/H3eQwTlSbNYvzQJSx3x8gynHyCxWQjE7jtSahiMSo11lY8/ic3+kc9fILsHDFGAAO6HYz5/B+dX7dphjwEK07whclA4yxtqX+KMl8xAAP3raUW492FzuCPagP3mVnfwLXTWYdRx5CmpKRaGuvelAv2EP2iWnSpeNxnGucq4MA9qCO4haAHpX6+wqOkg1yiiBgQegIzN55VTgMpuGh5OaZnGjKGoheOoOtKUs//eJEEUl9hK4CjWyeXuaIeFfwQXlxTCTTQ/Gmqtgw8CQ81Cu1ygs5tuf3mtKfhn4h66l5eUvU18eSCZ90gP+/193Dt4GfLCJBxYRm9PggszwpMd3niTN6SM/bXTzt9F9jhlnKn9PSYKKivTxRAZRq/SHMa+X/NEMrqcHyVTC8WlivBICHHNCJMLCNQhH8rN2VXpqMo w6xUvaPv a3mT2rfTfRVTKJEtHLZtvi1I9Z5ArjAYzMInyUh1bNX9GhvmGe9cFJ2X7U7YviYF61DbqdAnbnV7Jx3sSj2EunHzkaYDu0Vbr8CDLpO5qOjGku86ZzrfS3oXF9Qp1/7gTqFq20a5A2h5JgfRW4d/gs0IB9n5x7KWFl1N1F/RmQGgsK/N6xJvBgqZTKpAN2U9ycaYejh1ZbnrLiwYwJk+vfCR0J5ofgbuTSPgIu/NkGXFzwAgF01Ud9BzIumB9gp/GqIJcjuwufEC5NKkSR471Zrp6Aml0b73rsytsOXGT1MbJ/dXWAsVRMzc1xEh4Eof+KlOKfchmNKHyS785cP7aGmnYCMHNlKUwo2bbkZ/fOHH3RHw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Rik van Riel With AMD TCE (translation cache extensions) only the intermediate mappings that cover the address range zapped by INVLPG / INVLPGB get invalidated, rather than all intermediate mappings getting zapped at every TLB invalidation. This can help reduce the TLB miss rate, by keeping more intermediate mappings in the cache. >From the AMD manual: Translation Cache Extension (TCE) Bit. Bit 15, read/write. Setting this bit to 1 changes how the INVLPG, INVLPGB, and INVPCID instructions operate on TLB entries. When this bit is 0, these instructions remove the target PTE from the TLB as well as all upper-level table entries that are cached in the TLB, whether or not they are associated with the target PTE. When this bit is set, these instructions will remove the target PTE and only those upper-level entries that lead to the target PTE in the page table hierarchy, leaving unrelated upper-level entries intact. [ bp: use cpu_has()... I know, it is a mess. ] Signed-off-by: Rik van Riel Signed-off-by: Borislav Petkov (AMD) Link: https://lore.kernel.org/r/20250226030129.530345-13-riel@surriel.com --- arch/x86/include/asm/msr-index.h | 2 ++ arch/x86/kernel/cpu/amd.c | 4 ++++ tools/arch/x86/include/asm/msr-index.h | 2 ++ 3 files changed, 8 insertions(+) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 72765b2fe0d8..1aacd6b68fab 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -25,6 +25,7 @@ #define _EFER_SVME 12 /* Enable virtualization */ #define _EFER_LMSLE 13 /* Long Mode Segment Limit Enable */ #define _EFER_FFXSR 14 /* Enable Fast FXSAVE/FXRSTOR */ +#define _EFER_TCE 15 /* Enable Translation Cache Extensions */ #define _EFER_AUTOIBRS 21 /* Enable Automatic IBRS */ #define EFER_SCE (1<<_EFER_SCE) @@ -34,6 +35,7 @@ #define EFER_SVME (1<<_EFER_SVME) #define EFER_LMSLE (1<<_EFER_LMSLE) #define EFER_FFXSR (1<<_EFER_FFXSR) +#define EFER_TCE (1<<_EFER_TCE) #define EFER_AUTOIBRS (1<<_EFER_AUTOIBRS) /* diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c index 7a72ef47a983..705853315c0d 100644 --- a/arch/x86/kernel/cpu/amd.c +++ b/arch/x86/kernel/cpu/amd.c @@ -1075,6 +1075,10 @@ static void init_amd(struct cpuinfo_x86 *c) /* AMD CPUs don't need fencing after x2APIC/TSC_DEADLINE MSR writes. */ clear_cpu_cap(c, X86_FEATURE_APIC_MSRS_FENCE); + + /* Enable Translation Cache Extension */ + if (cpu_has(c, X86_FEATURE_TCE)) + msr_set_bit(MSR_EFER, _EFER_TCE); } #ifdef CONFIG_X86_32 diff --git a/tools/arch/x86/include/asm/msr-index.h b/tools/arch/x86/include/asm/msr-index.h index 3ae84c3b8e6d..dc1c1057f26e 100644 --- a/tools/arch/x86/include/asm/msr-index.h +++ b/tools/arch/x86/include/asm/msr-index.h @@ -25,6 +25,7 @@ #define _EFER_SVME 12 /* Enable virtualization */ #define _EFER_LMSLE 13 /* Long Mode Segment Limit Enable */ #define _EFER_FFXSR 14 /* Enable Fast FXSAVE/FXRSTOR */ +#define _EFER_TCE 15 /* Enable Translation Cache Extensions */ #define _EFER_AUTOIBRS 21 /* Enable Automatic IBRS */ #define EFER_SCE (1<<_EFER_SCE) @@ -34,6 +35,7 @@ #define EFER_SVME (1<<_EFER_SVME) #define EFER_LMSLE (1<<_EFER_LMSLE) #define EFER_FFXSR (1<<_EFER_FFXSR) +#define EFER_TCE (1<<_EFER_TCE) #define EFER_AUTOIBRS (1<<_EFER_AUTOIBRS) /* -- 2.43.0