From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 794C7E77188 for ; Mon, 30 Dec 2024 17:57:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ECE846B009C; Mon, 30 Dec 2024 12:57:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CFE546B00A1; Mon, 30 Dec 2024 12:57:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7CB866B00A5; Mon, 30 Dec 2024 12:57:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 2E2ED6B00A7 for ; Mon, 30 Dec 2024 12:57:35 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id DC95CA1451 for ; Mon, 30 Dec 2024 17:57:34 +0000 (UTC) X-FDA: 82952380518.03.1AD95A3 Received: from shelob.surriel.com (shelob.surriel.com [96.67.55.147]) by imf18.hostedemail.com (Postfix) with ESMTP id 15A341C0007 for ; Mon, 30 Dec 2024 17:57:11 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=none; spf=pass (imf18.hostedemail.com: domain of riel@shelob.surriel.com designates 96.67.55.147 as permitted sender) smtp.mailfrom=riel@shelob.surriel.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735581409; a=rsa-sha256; cv=none; b=aDh7zS9W5+fIg/qnM3rxggJzMjCmUUPUkH57Px0wGIlJURsKiREBHkHSyqIBWMlWje7bbi SooOEFlqYpP0bTWw9m+5WaV6ZTIEy7G9Und9aa7I7hse9ObMkiH1KLo/nilQNH/Bomlew2 BCBS1z3+690h2SUjtMdGXpvAH3Jfw98= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=none; spf=pass (imf18.hostedemail.com: domain of riel@shelob.surriel.com designates 96.67.55.147 as permitted sender) smtp.mailfrom=riel@shelob.surriel.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735581409; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gubB01sncxFro+ZwW5/gOVnSXZsx7A6xtmmdEMjV+d4=; b=PvU5eLic5qnTFmFuK5Bv3GSk6Wu4g/VA0/JoWucsYqayAYhg9vJxrI7EB5dxN8feGBjbRs wgfRSUJZWslVC6+nGCz5aDyXzcJYqBkwtkfRQLjY+0Y7aepCR3Aeix89kFGNp+dwVpLFbO 9EbsQv4nD9shaee3O8WUv6VzTTtD/ac= Received: from fangorn.home.surriel.com ([10.0.13.7]) by shelob.surriel.com with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.97.1) (envelope-from ) id 1tSJzc-000000008Lf-2Fys; Mon, 30 Dec 2024 12:55:52 -0500 From: Rik van Riel To: x86@kernel.org Cc: linux-kernel@vger.kernel.org, kernel-team@meta.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, akpm@linux-foundation.org, nadav.amit@gmail.com, zhengqi.arch@bytedance.com, linux-mm@kvack.org, Rik van Riel Subject: [PATCH 01/12] x86/mm: make MMU_GATHER_RCU_TABLE_FREE unconditional Date: Mon, 30 Dec 2024 12:53:02 -0500 Message-ID: <20241230175550.4046587-2-riel@surriel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241230175550.4046587-1-riel@surriel.com> References: <20241230175550.4046587-1-riel@surriel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 15A341C0007 X-Stat-Signature: 4sfkbgiqwwpxnkyryd377tjganiymcy7 X-Rspam-User: X-Rspamd-Server: rspam09 X-HE-Tag: 1735581431-796970 X-HE-Meta: U2FsdGVkX18x+oUKM2z3mLPnC9uGNpqIm4nh0rOdjrK3HhLdMc3gNr5/yBmoaHVuuncZjIOd65cWadjy9X5YHaqwrFoObILlNTYiC4Esnwpn1teYgULrxY6j8beQbbUR8pU7lrhb72rNpB9NM5L85NgUmtVNIKbNx4436I8BsMuWR4U/BGIgyZ840Fs+5kmxyCsyawwmOhq840rJ1vaSecC6YELpEig9ZhC8R7xChAR4Os2yyT4U2DyqqqO0dLqzw1g+hGGZkWtMvzFxeG/GOmxb0U0FchnxB7gwKTCBfKuZGW8aM4jQly30hYEhVAHGbkys6KiiazG7tkcghRf0u5KKKIdlc3X/kCk2twdkMYFAPg8LFHgbwqeU5rybPNQpR40fIQUYKvbL//NqyRcDKjkobljthjFUIyAmrAnTQvPC6HZrm9z4wArtbBxlV6Ql5Qjc0pIz4k1u6IgZZBN4rRShLXpGmHjb0v/7SeGBZ9xIuHDaovXW5czAQi9lRpwK3B9IauV81lc8XOpWVtbCfazWEGykDBsa1MHjM6/APFLRv51RzknrsboAUxCT1TQ28WexOTtezw95X29tVhDSpVfj+XClX7LqScHFgtfmv5LEzC6Q6nNg4NO9+PgNntvVQWFEPPacfjGH2IABG1Mf8Fxck1A+4yMI7LTer3mWPbslj5BP87FWtJBns90fmHQG/Mtjsw/BLoADA4sX0NiDL1ppVaU0z6PJ893FDCLV/s1fpnFqkFKCmrhHO+ks23a+t2Zf0sUozX/yOK9KorenM79ME5H8+Kalzhn6R7q1QI8YWfqunsmd10kW7YW6Wd8XKhyDXiNuWQBReazHuLzF23X7Ihu4BctX2Eoh9j90SS8QgZ2ILOAr5snz51dN+N5KemIg5K2mL6PeFppt331WHI2vzqhQ3mW8fJi8LOqbNY6sgfKgCuAmj3v2JoOLCEe7x1ORQ+nA0EUubo3zirf DF0jPvMU KAPg+6BlgnA2P6YGsGzc3FM0FBWk5eDtJsVEOsdbx1RkqLTCvslJXgIxqapw3uU303efuECoqJ7FTkYtzyPWQGR0pFBGsrsihmg7GOE8h3LwRDJo2Of5UitNWn+d1UHSIXQvDnTUGzbVoE+KC/AYIkVpmb/8rgjfI79mx+pA674LddHK1DY/U4MBju+Kcar7li+7+P/ES5Dl96QGQiK2GeBIE49dXyIDUH2q1v416mTZ9xjGnXM4amSgoFjPTotyX3bwbQvapyRQxxm4pUoA3FJ6v/Mf5MDRGCMYSTqqeMD13JwGvpxIg7uEHL/iXZNLUnoQ0eWJW3oTpRSY5c4qHioU8JFRnLlrvzpWbWknwMDHi4XBf07ZdPcBYz9uECmX1ff4kKMjhHL5g1RAUGiwqEtTeooB13bXAi1g7xNm5utInbUcJQBKnW+u1aASAlMeKUqusPeb8YYEXxwFG937yK6nk9/pFbka89BBasY3Qgj2IUEL/vjqx1/FHGA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently x86 uses CONFIG_MMU_GATHER_TABLE_FREE when using paravirt, and not when running on bare metal. There is no real good reason to do things differently for each setup. Make them all the same. After this change, the synchronization between get_user_pages_fast and page table freeing is handled by RCU, which prevents page tables from being reused for other data while get_user_pages_fast is walking them. This allows us to invalidate page tables while other CPUs have interrupts disabled. Signed-off-by: Rik van Riel Suggested-by: Peter Zijlstra --- arch/x86/Kconfig | 2 +- arch/x86/kernel/paravirt.c | 7 +------ 2 files changed, 2 insertions(+), 7 deletions(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 9d7bd0ae48c4..e8743f8c9fd0 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -274,7 +274,7 @@ config X86 select HAVE_PCI select HAVE_PERF_REGS select HAVE_PERF_USER_STACK_DUMP - select MMU_GATHER_RCU_TABLE_FREE if PARAVIRT + select MMU_GATHER_RCU_TABLE_FREE select MMU_GATHER_MERGE_VMAS select HAVE_POSIX_CPU_TIMERS_TASK_WORK select HAVE_REGS_AND_STACK_ACCESS_API diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c index fec381533555..2b78a6b466ed 100644 --- a/arch/x86/kernel/paravirt.c +++ b/arch/x86/kernel/paravirt.c @@ -59,11 +59,6 @@ void __init native_pv_lock_init(void) static_branch_enable(&virt_spin_lock_key); } -static void native_tlb_remove_table(struct mmu_gather *tlb, void *table) -{ - tlb_remove_page(tlb, table); -} - struct static_key paravirt_steal_enabled; struct static_key paravirt_steal_rq_enabled; @@ -191,7 +186,7 @@ struct paravirt_patch_template pv_ops = { .mmu.flush_tlb_kernel = native_flush_tlb_global, .mmu.flush_tlb_one_user = native_flush_tlb_one_user, .mmu.flush_tlb_multi = native_flush_tlb_multi, - .mmu.tlb_remove_table = native_tlb_remove_table, + .mmu.tlb_remove_table = tlb_remove_table, .mmu.exit_mmap = paravirt_nop, .mmu.notify_page_enc_status_changed = paravirt_nop, -- 2.47.1