From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B96CEB64DA for ; Wed, 5 Jul 2023 20:42:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D4DA48D0002; Wed, 5 Jul 2023 16:42:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CFD7B8D0001; Wed, 5 Jul 2023 16:42:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B9DDF8D0002; Wed, 5 Jul 2023 16:42:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id A96B78D0001 for ; Wed, 5 Jul 2023 16:42:56 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 893A51602F7 for ; Wed, 5 Jul 2023 20:42:55 +0000 (UTC) X-FDA: 80978732310.02.26E8AEF Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) by imf26.hostedemail.com (Postfix) with ESMTP id DC933140009 for ; Wed, 5 Jul 2023 20:42:50 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=djk8cOqX; spf=none (imf26.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688589773; a=rsa-sha256; cv=none; b=uF9ElUmc2dkSWxSSVWQq4WyBmJ+Ugaczcka8Ur20kOavw6wKIqHGambn7/lgdeesU4aZoI 2Xl2HR9Ln8hrO9VenRrneZxOPVcXcpR/SlOJcP8Lwr1UleZBRztaaPsBJ3XyuRa3yT1pxt NcPvz7DKeMMGRbe1hRMoZwkDB3ay2h8= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=djk8cOqX; spf=none (imf26.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688589773; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2uvD9mYwUyQ82PXrNwiJR0+WRJM0PQCF8y2F5N/cONE=; b=55tySZPfn3r8Yk9qu5GP/HcUyrur2HsihmswEZwx9RqXjL4V98RtDRDTE+c0vUY3uBc9uF ZLUzsE/b/Z++P0UE6M35MwVyFm4D3UOCE6p9fzR7E98mcvYxpKsI+hscGVLvkkoaoh3AGy v2LvNDy3Wk9aUWlMi0/3fGhvHRHvUr0= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=2uvD9mYwUyQ82PXrNwiJR0+WRJM0PQCF8y2F5N/cONE=; b=djk8cOqXqggfU6Ml1I4iR48Y7x K1cf8UXkaxjEDlwcadptEwEIaddtNIkZ0QgDE05z3bHQsFK/T5BZk/A8EptDgFhNmtZxPrwNXlvJV XqR9O+Eu8jaR3tznarXclGqLXhttQm8PKIkMB2RlU+j6j7xs3SRhCNvr1dqBVjy/o/qjV7pqiuzx2 dq1/AwZbWVkovVvjc8c9ypAE/fOd7iZIGqbxgefCqqqOg7/4r8r9OHqD5LuscwVdfRwKLVQhf60Zi Es6WF26h4jXaLqFM+2HbZbyO0rrMU+w7jsQZkzWgNUV3/3f7udrqYKcc3yQc5ERx5e65xwVlGsfrz w+G8Gp6w==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1qH9Jr-00CFZa-24; Wed, 05 Jul 2023 20:41:50 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 32F36300023; Wed, 5 Jul 2023 22:41:43 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 1000) id 0F151200E5E98; Wed, 5 Jul 2023 22:41:43 +0200 (CEST) Date: Wed, 5 Jul 2023 22:41:42 +0200 From: Peter Zijlstra To: Valentin Schneider Cc: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-doc@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org, x86@kernel.org, Steven Rostedt , Masami Hiramatsu , Jonathan Corbet , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Paolo Bonzini , Wanpeng Li , Vitaly Kuznetsov , Andy Lutomirski , Frederic Weisbecker , "Paul E. McKenney" , Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Lorenzo Stoakes , Josh Poimboeuf , Kees Cook , Sami Tolvanen , Ard Biesheuvel , Nicholas Piggin , Juerg Haefliger , Nicolas Saenz Julienne , "Kirill A. Shutemov" , Nadav Amit , Dan Carpenter , Chuang Wang , Yang Jihong , Petr Mladek , "Jason A. Donenfeld" , Song Liu , Julian Pidancet , Tom Lendacky , Dionna Glaze , Thomas =?iso-8859-1?Q?Wei=DFschuh?= , Juri Lelli , Daniel Bristot de Oliveira , Marcelo Tosatti , Yair Podemsky Subject: Re: [RFC PATCH 08/14] BROKEN: context_tracking: Make context_tracking_key __ro_after_init Message-ID: <20230705204142.GB2813335@hirez.programming.kicks-ass.net> References: <20230705181256.3539027-1-vschneid@redhat.com> <20230705181256.3539027-9-vschneid@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230705181256.3539027-9-vschneid@redhat.com> X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: DC933140009 X-Stat-Signature: phuu96huk5bkm8o146t77m4sf87fy96w X-Rspam-User: X-HE-Tag: 1688589770-784798 X-HE-Meta: U2FsdGVkX18lz5P9B4seoHArArLYbF6X2kSEqFP+u4qOgzur50OVCCiph3K5NOV0N5/fATlpGXKzzO8/YaR/SGak9fOd7+O1qaYibAdj9DjNQhDpS3+JazN8EyDml4yjtyKjpdjtQj1D6NWRzAXaneOiorvvQhk437/m1kpczeYOLt9188tlI+kcrDqeTPG9btF6hIcx76J2NdtJxmbh+cSGIp/eKRoXVmzYaCIinkPcce3U3NjuFCSV8ZppxzorhOTPuxyop5MpLpdHyKmqKaDaz+DGPf3dNC4RseUGa6XLDlxIkinMsnUkr4lilPuRb+oEI2suFz0pGjmmsVDsKH3Ac0NvtqbI8FVMsOQoUAD4RcYlLxeDU4M0lrhRyoFaFktbnX4jlCoj8g5wLQZGYCyfjN78MjXMPx7JvZq+Gcs/2BWhPjv+wqCCICot8RHlUQOWH1Tj5e+c8nCwfgmVMBpLgLb2gRrhmUPHR9Pqw9nrdPfm0FjdT9k12jUIndU1AhsJDdQAb7hEif/HmIrgDZ09uvGmLFi/F4qfVpJ0j0Ynf1qoXgilPyF389R3kgBmSYcZunJDBxf2ASY0fcSXBZVi5vK3r5K1P2SOyi67RvyvVO0E7Q2oGWBb4OKPe1RqSzQmt7rXvDGCXwS+M61HhPVIvqo2en8HA8IJ/Iccv8YTBFO7BbMs3RuHvr3/0idb3D89+JqUTPBDqPvE6menL28GG7k93+EpAIj/XxOV41B5Ud1ORkPxjfuAaZij1iYJHmgOYW//aZgt/A65jkyAKYsreOAOF4WlFu/FK/n2egGuLWM35U72+oX5n4jW/NFleuw7fwAr5k8oAh2sMM2YrXknIDMq5an43LC1SFXmYPuC6PWzrRO8eTXFhD/xqkZhaGQ2LihsYZtmCNQe4hPHz0bL3V3dXrHfp9kBYe2/AgMOdbFECBBD8uthPQwm35hr346YlHizQ0K2fJ+GJ+H 2odIO7xp SXUSo/LbKRVaYsnYfS55/bHh9KFNNAf6KMeY9PiPx9NIRdmDNMOWvkyaPjm55dppkQqTpg2tM6aO0L7yP7pb+fTLaSrXWVIt7m2uVpJ3fnjevItYfuhdXOMM+rYJ4hQ63wAbeMSa+VuSNvvYl3JrRDIOFlYQM+tB+5dGq4UfHf4kx9i8SIOJY+Q8RNAJSoBCv2V4rywGCR2tyRmChxW9URO4UBaUjNzMgOuuPPkvgGbLFL8awe9HV+6GfZA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Jul 05, 2023 at 07:12:50PM +0100, Valentin Schneider wrote: > BROKEN: the struct static_key lives in a read-only mapping after > mark_rodata_ro(), which falls apart when the KVM module is loaded after > init and a write to the struct happens due to e.g. guest_state_exit_irqoff() > relying on the static key: Right.. so whoever added the whole ro_after_init jump_label support did a very poor job of it. That said; I think it is fixable. Since the key cannot be changed, we don't actually need to track the entries list and can thus avoid the key update. Something like the completely untested below... --- Subject: jump_label: Seal __ro_after_init keys When a static_key is marked ro_after_init, its state will never change (after init), therefore jump_label_update() will never need to iterate the entries, and thus module load won't actually need to track this -- avoiding the static_key::next write. Therefore, mark these keys such that jump_label_add_module() might recognise them and avoid the modification. Use the special state: 'static_key_linked(key) && !static_key_mod(key)' to denote such keys. *UNTESTED* NOT-Signed-off-by: Peter Zijlstra (Intel) --- include/asm-generic/sections.h | 5 +++++ include/linux/jump_label.h | 1 + init/main.c | 1 + kernel/jump_label.c | 44 ++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 51 insertions(+) diff --git a/include/asm-generic/sections.h b/include/asm-generic/sections.h index db13bb620f52..c768de6f19a9 100644 --- a/include/asm-generic/sections.h +++ b/include/asm-generic/sections.h @@ -180,6 +180,11 @@ static inline bool is_kernel_rodata(unsigned long addr) addr < (unsigned long)__end_rodata; } +static inline bool is_kernel_ro_after_init(unsigned long addr) +{ + return addr >= (unsigned long)__start_ro_after_init && + addr < (unsigned long)__end_ro_after_init; +} /** * is_kernel_inittext - checks if the pointer address is located in the * .init.text section diff --git a/include/linux/jump_label.h b/include/linux/jump_label.h index f0a949b7c973..88ef9e776af8 100644 --- a/include/linux/jump_label.h +++ b/include/linux/jump_label.h @@ -216,6 +216,7 @@ extern struct jump_entry __start___jump_table[]; extern struct jump_entry __stop___jump_table[]; extern void jump_label_init(void); +extern void jump_label_ro(void); extern void jump_label_lock(void); extern void jump_label_unlock(void); extern void arch_jump_label_transform(struct jump_entry *entry, diff --git a/init/main.c b/init/main.c index ad920fac325c..cb5304ca18f4 100644 --- a/init/main.c +++ b/init/main.c @@ -1403,6 +1403,7 @@ static void mark_readonly(void) * insecure pages which are W+X. */ rcu_barrier(); + jump_label_ro(); mark_rodata_ro(); rodata_test(); } else diff --git a/kernel/jump_label.c b/kernel/jump_label.c index d9c822bbffb8..40fb72d79d7a 100644 --- a/kernel/jump_label.c +++ b/kernel/jump_label.c @@ -530,6 +530,46 @@ void __init jump_label_init(void) cpus_read_unlock(); } +static inline bool static_key_sealed(struct static_key *key) +{ + return (key->type & JUMP_TYPE_LINKED) && !(key->type & ~JUMP_TYPE_MASK); +} + +static inline void static_key_seal(struct static_key *key) +{ + unsigned long type = key->type & JUMP_TYPE_TRUE; + key->type = JUMP_TYPE_LINKED | type; +} + +void jump_label_ro(void) +{ + struct jump_entry *iter_start = __start___jump_table; + struct jump_entry *iter_stop = __stop___jump_table; + struct static_key *key = NULL; + struct jump_entry *iter; + + if (WARN_ON_ONCE(!static_key_initialized)) + return; + + cpus_read_lock(); + jump_label_lock(); + + for (iter = iter_start; iter < iter_stop; iter++) { + struct static_key *iterk = jump_entry_key(iter); + + if (!is_kernel_ro_after_init(iterk)) + continue; + + if (static_key_sealed(iterk)) + continue; + + static_key_seal(iterk); + } + + jump_label_unlock(); + cpus_read_unlock(); +} + #ifdef CONFIG_MODULES enum jump_label_type jump_label_init_type(struct jump_entry *entry) @@ -650,6 +690,9 @@ static int jump_label_add_module(struct module *mod) static_key_set_entries(key, iter); continue; } + if (static_key_sealed(key)) + goto do_poke; + jlm = kzalloc(sizeof(struct static_key_mod), GFP_KERNEL); if (!jlm) return -ENOMEM; @@ -675,6 +718,7 @@ static int jump_label_add_module(struct module *mod) static_key_set_linked(key); /* Only update if we've changed from our initial state */ +do_poke: if (jump_label_type(iter) != jump_label_init_type(iter)) __jump_label_update(key, iter, iter_stop, true); }