From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67E91C3ABBF for ; Tue, 6 May 2025 00:38:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2235C6B0088; Mon, 5 May 2025 20:38:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 047ED6B0096; Mon, 5 May 2025 20:38:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CA35A6B0098; Mon, 5 May 2025 20:38:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 31EB06B008A for ; Mon, 5 May 2025 20:38:28 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 3824DBCCA3 for ; Tue, 6 May 2025 00:38:29 +0000 (UTC) X-FDA: 83410621938.20.1415BDD Received: from shelob.surriel.com (shelob.surriel.com [96.67.55.147]) by imf11.hostedemail.com (Postfix) with ESMTP id A264940005 for ; Tue, 6 May 2025 00:38:27 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf11.hostedemail.com: domain of riel@shelob.surriel.com designates 96.67.55.147 as permitted sender) smtp.mailfrom=riel@shelob.surriel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1746491907; a=rsa-sha256; cv=none; b=F6XZmKdokHO3788/o8UnxVGUs5ljYClJt+G7gRCiQmfICUcoYT5ahGwusDVRhTPgbnKrn8 6qzaZOmFMjW3cBjZLSZLxBGUjIcTEbkDgy6IOGHRoHxjHoqr40rQbhlAOyKbc3wm0cclxW HnhhNZ2LG7hM0JZa+uJGhcNxKLt+KD4= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf11.hostedemail.com: domain of riel@shelob.surriel.com designates 96.67.55.147 as permitted sender) smtp.mailfrom=riel@shelob.surriel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1746491907; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yWi7KDM75+vEOYMwK0ydceCizm66VnTGxr4GaurP2BA=; b=qeXw82ui1VPiYlWj7diuyjRW7M0zvWujdpb5oLMVh8NH1lnBEw4RxAGuLiYNlC4pRTMHT8 slif3nIGmZlJ6Jsv9vFcncjiaQE2AdsX3AEoi+BwSR2VbsEH9QXbTz8cv3ZqvtTcsUOpM6 6FtMSioa1Q751GNv8z2zeQUElDpaY6I= Received: from fangorn.home.surriel.com ([10.0.13.7]) by shelob.surriel.com with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.97.1) (envelope-from ) id 1uC6K6-000000000IF-3aS7; Mon, 05 May 2025 20:38:14 -0400 From: Rik van Riel To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, x86@kernel.org, kernel-team@meta.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, Rik van Riel , Yu-cheng Yu , Rik van Riel Subject: [RFC PATCH 5/9] x86/mm: Change cpa_flush() to call flush_kernel_range() directly Date: Mon, 5 May 2025 20:37:43 -0400 Message-ID: <20250506003811.92405-6-riel@surriel.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250506003811.92405-1-riel@surriel.com> References: <20250506003811.92405-1-riel@surriel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: A264940005 X-Stat-Signature: py9hott96aax76z7jsmrc9pag1j1zo4n X-HE-Tag: 1746491907-844026 X-HE-Meta: U2FsdGVkX1/DDAoGGMWta6FWI1FhyVo3iONN5oEGGEYJzujxvK31wYwGOfb2f1Ieqsmma94P4Anknz2eTyNLwl440EW4BUtFqrjhDmpb1lwJ1FYK6flMhRiNzo26JS2Rl6e3/oOBrqnSKPu5Kgd7v+/6ycn2YvTpsQWw/MJMetNc1tVprzeqfsTo+V2yzaghgJGVhcVxXxmpWRbxnjIp5DzcSpc4CC2OP29VGeNTQYYZyMeQrCF8NgUdR0xBMbWXJ8qsNW6GJThKqiSCe47cKsok4sStPTV2bXbwW2a2rWsD4XomWQFVCauHmnuwyUE/69qoPhdIgVWUwYbE8o+AbkEkIYhWzSVYc5KJsBJ4t3y2KjyUYkYPz4FGyNExi0SuFjBcA/0VG+4voRR6ixgbRMzk/bzlduXexbMlQcPF4STU/UmQZSQqiNmAnrdzymK7ZPDxomCN4g0LJY+9N8Hiq2Y3/BLawSy7uGLR/2asS4USNai44cJmEt0rDQRDAjyJej12PZbyjjvLUemRmGu0QMCdXCrn4A0j6s1ecEPNN85JSPW+MW3vR13Vjlh+8Y/a/lU9RH1jkiWcveUDfhn6q3256tGKI7iu3jP2AY3m40+dc0GEo7kE6s4KhVY3r/taF/YhIwvD5NCUle7yUaj3fjBfT3BirUy1clGBEPSnyVbJ0+vDcHAtmQ6nwlPj0EisRnyWDXwAGllUIwTcce6IQJZimrmyHzrIOCXJ50B8qjq0S3OwY0xaHdORJ9EwPqUA14HyaRwRdXRZT0XNDumCmJx2NOAEWm+d/QZLTpNIaKAhhxzVsY2mO9FN+ZOtR6sJ3MG6OnM4BMhf1GGrSrTAza+sF+omoubn+mYwEhike6fAL6Ez2zGFPRusEQAp4pVxjg6RBI+VzP9ryihnCzhXFhOSEaSlN6j0A+6mIxfs4jadY9kSDbhpEXThjpaJfI4Gnbmetu5/RU1UP888pRn VaXv9Mz3 1J/vnNsuY7JT6VEzdE7J5bYbSQY6ZBBgG6JRsI97WKG3JxdewmVAH+thD64o7/Q6DGAYUk1mUibj41ajySYoVenVixRyEsgfK8hIp0wacOsPdqB61b/sRH2IQF08J7pyrJlp376ucvJ2QECmxUgiXwUdirpid3aCwk0MsTp1rP2ppBUT8YcWE+qNSFJvgfUs8S/XdMes9tBwgdKwbovQDFaalRwOFxik5AknBCKlHbw1alk34JZNLOk2X6sBoUocZ5IQ54BPrBb1+upRRRV3r3gNepaASv59t0P1xyEocpW9A+j9v+ABz+Ozxak2xAV7Fc8Kvtp3zhfTtF6KcrS/PscJWsHyu9ygsslPY2ep4azlKayarHmk6IS4cScjgmdZwgMioZo93X/UGL/hwDsQTbJQlMg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Rik van Riel The function cpa_flush() calls __flush_tlb_one_kernel() and flush_tlb_all(). Replacing that with a call to flush_tlb_kernel_range() allows cpa_flush() to make use of INVLPGB or RAR without any additional changes. Initialize invlpgb_count_max to 1, since flush_tlb_kernel_range() can now be called before invlpgb_count_max has been initialized to the value read from CPUID. [riel: remove now unused __cpa_flush_tlb] Signed-off-by: Yu-cheng Yu Signed-off-by: Rik van Riel --- arch/x86/kernel/cpu/amd.c | 2 +- arch/x86/mm/pat/set_memory.c | 20 +++++++------------- 2 files changed, 8 insertions(+), 14 deletions(-) diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c index 13a48ec28f32..c85ecde786f3 100644 --- a/arch/x86/kernel/cpu/amd.c +++ b/arch/x86/kernel/cpu/amd.c @@ -31,7 +31,7 @@ #include "cpu.h" -u16 invlpgb_count_max __ro_after_init; +u16 invlpgb_count_max __ro_after_init = 1; static inline int rdmsrq_amd_safe(unsigned msr, u64 *p) { diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c index 30ab4aced761..2454f5249329 100644 --- a/arch/x86/mm/pat/set_memory.c +++ b/arch/x86/mm/pat/set_memory.c @@ -399,15 +399,6 @@ static void cpa_flush_all(unsigned long cache) on_each_cpu(__cpa_flush_all, (void *) cache, 1); } -static void __cpa_flush_tlb(void *data) -{ - struct cpa_data *cpa = data; - unsigned int i; - - for (i = 0; i < cpa->numpages; i++) - flush_tlb_one_kernel(fix_addr(__cpa_addr(cpa, i))); -} - static int collapse_large_pages(unsigned long addr, struct list_head *pgtables); static void cpa_collapse_large_pages(struct cpa_data *cpa) @@ -444,6 +435,7 @@ static void cpa_collapse_large_pages(struct cpa_data *cpa) static void cpa_flush(struct cpa_data *cpa, int cache) { + unsigned long start, end; unsigned int i; BUG_ON(irqs_disabled() && !early_boot_irqs_disabled); @@ -453,10 +445,12 @@ static void cpa_flush(struct cpa_data *cpa, int cache) goto collapse_large_pages; } - if (cpa->force_flush_all || cpa->numpages > tlb_single_page_flush_ceiling) - flush_tlb_all(); - else - on_each_cpu(__cpa_flush_tlb, cpa, 1); + start = fix_addr(__cpa_addr(cpa, 0)); + end = fix_addr(__cpa_addr(cpa, cpa->numpages)); + if (cpa->force_flush_all) + end = TLB_FLUSH_ALL; + + flush_tlb_kernel_range(start, end); if (!cache) goto collapse_large_pages; -- 2.49.0