From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73A7CC83F22 for ; Fri, 18 Jul 2025 02:17:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 021936B00B7; Thu, 17 Jul 2025 22:17:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EEC426B00B8; Thu, 17 Jul 2025 22:17:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DDA086B00B9; Thu, 17 Jul 2025 22:17:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id CBE346B00B7 for ; Thu, 17 Jul 2025 22:17:07 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 9B15810F6F8 for ; Fri, 18 Jul 2025 02:17:07 +0000 (UTC) X-FDA: 83675772894.01.078856F Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) by imf14.hostedemail.com (Postfix) with ESMTP id BC5AD100009 for ; Fri, 18 Jul 2025 02:17:04 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=YPAq1Ewb; spf=pass (imf14.hostedemail.com: domain of alexei.starovoitov@gmail.com designates 209.85.210.179 as permitted sender) smtp.mailfrom=alexei.starovoitov@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1752805024; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=uWETJlOKGazWkISOD3+6VC2F2WWqxCGLlibMASETCes=; b=Bk15Lf9ZMrZp/9Z06nigpi2L5yjra2lHpvvYUjKHvoHuClG7O8pG0ky96oHbeHqZtFct+E uQbjEilYaczlU9lbUoJgb9Q/RFYStRCIGziRUJ5aWafVoiBeLIuFeR25oLL/eSXP7zdUjH yOKnQ7PG3fc80zFuc2ynCBY3qgtrz6U= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1752805024; a=rsa-sha256; cv=none; b=VBfaGyINUhlFVULFeIX+8E9zAUSDCzdAa3robW6J+6hRzy/CRPpdfE1TSNdDjIqmfL/wKA EY3S3UqAb2z9ZMyWoBCVs5YzVOtE1H46hFGQ68o17vR3iKD22UgDFnvEz76rvsSF2rrm/O YTEx0jJNHipS4XzWIuu45cs4FCr7UVw= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=YPAq1Ewb; spf=pass (imf14.hostedemail.com: domain of alexei.starovoitov@gmail.com designates 209.85.210.179 as permitted sender) smtp.mailfrom=alexei.starovoitov@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pf1-f179.google.com with SMTP id d2e1a72fcca58-749248d06faso1453647b3a.2 for ; Thu, 17 Jul 2025 19:17:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1752805024; x=1753409824; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uWETJlOKGazWkISOD3+6VC2F2WWqxCGLlibMASETCes=; b=YPAq1Ewbm9yWWsymtxgHjp9tApKtApg7QMEduMRTHlp7XdZe+3RDNTSh4NI1vu81Yg 3CSQvIYpBatRHHbDtzYOFoTZfNSkXIfHZwUIWI5gCtQ9IDYsrwfb1NREBNR5VCPQQsmn tjluwOi2zQ3Y7Sc7rFc714/qmPSXPFtdeseAOYpGmdTyiIXpOJ3mqkoQeK7jj8tRsr3M XCSgBMm8CiMjN7oE3bXu2sZIuT8ZZCu0OpW60Rg7gxKZxAjFMiiK6kVHApWsIBNIIfqb GOpctjAgFS/zfp0CfpeF+/vI2i6HIbypolup/uDRPTRCo/AXlhA/FMGaX5ecpKYYy8VW sJhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752805024; x=1753409824; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uWETJlOKGazWkISOD3+6VC2F2WWqxCGLlibMASETCes=; b=fik4NRDsSmN6d4go2J1D19Te0InME7TgexTHjP9H9g9Ie91pE6Ts3ihKwAOju/gxO+ niwVYEk5DIXLrXtqtn73pWTrdj7f8ZILmFJ/bBv68JLaZRkpX4WiSb/gRMNWavOVyCUK Eg2y+HQi/UA7eQZKXvu0i9IIMRkHzvpgSk/bsMfr5+WbmCcTv2hIbaxyLNZaly9lPy9Q 37T9py3xbQfR9e6etZfcEyRtgemQGv2qB2TmPHRqaMV6dYJf6JXy4Ufic9FZ9JGRVI4J AbepfKarGkk1U0Uwf2aVbKrLsqv+jnosL2GuUexbVbJdxnndWRm7P0bedPJMplPv3Kca Bh2A== X-Forwarded-Encrypted: i=1; AJvYcCVSunwA5wzU/ZGtdZ7D+5w1ZcoK7OUY/QM9HdRRhiqje/Pizv5uZurCnuAPmqX1ib4cJ41jLVtKvw==@kvack.org X-Gm-Message-State: AOJu0YwRa2aseHhZaWGdNrXcvOvD9DDIbCPstQHd5+INR3EU5/bJbwCt k3APVtOjHFf5NgtsASwTLBhg6xbNhkqWPODkL4nMqQl0xZoWlxP6ILyZ X-Gm-Gg: ASbGncuRmEVpl+P1glIZonadPfYCQSuG4NtpvvYAvwh3WMQohNgvnb/pSEDDdiZEpHU 3pyNA4AquTHlmXL6ofTjv5YKgPEaM1XHZGeFmq672VXSXRkt/X6k1/NbSzhIiHv40NcF9Tbkfkd bpI4gnON3J1XQv5ttU+GnTs4cji3GshF65Seq0LviG7XB2nmwKy35M6LsbraeB1X0fwAW43OLra tfxPKGOEkoczgm7/hjsbFZu3r4wq3vuw7FKh1aN0b6WBssm1wTCvbE7oPYXxY9ChXK1ViADO9f6 jhL5LHWw3Hz6igDPs4PAVJI8lkFBtRMaOQVB/+XmCaBsZqbIi+bSO2GQtmsYhviXOcdMuZCcH1T mOUa2dj2hQQjGtchlVYQCfJqlMLcg7KcTOWP8zJLbbmPOfQOIXmp2n2gnEpEGPTI= X-Google-Smtp-Source: AGHT+IHWyxaj7icsLAL7jj7PUwL0yEJU4J9Ecb5MsKbovfcnY7bdDm2SOePe+iLUf+n5utsLBghogw== X-Received: by 2002:a05:6a21:7109:b0:238:351a:6438 with SMTP id adf61e73a8af0-2390dc71a6fmr7468515637.44.1752805023593; Thu, 17 Jul 2025 19:17:03 -0700 (PDT) Received: from localhost.localdomain ([2001:558:600a:7:a83d:600f:32cc:235a]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-759cb154755sm224084b3a.69.2025.07.17.19.17.02 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Thu, 17 Jul 2025 19:17:03 -0700 (PDT) From: Alexei Starovoitov To: bpf@vger.kernel.org, linux-mm@kvack.org Cc: vbabka@suse.cz, harry.yoo@oracle.com, shakeel.butt@linux.dev, mhocko@suse.com, bigeasy@linutronix.de, andrii@kernel.org, memxor@gmail.com, akpm@linux-foundation.org, peterz@infradead.org, rostedt@goodmis.org, hannes@cmpxchg.org Subject: [PATCH v4 5/6] slab: Make slub local_(try)lock more precise for LOCKDEP Date: Thu, 17 Jul 2025 19:16:45 -0700 Message-Id: <20250718021646.73353-6-alexei.starovoitov@gmail.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20250718021646.73353-1-alexei.starovoitov@gmail.com> References: <20250718021646.73353-1-alexei.starovoitov@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: BC5AD100009 X-Rspam-User: X-Rspamd-Server: rspam09 X-Stat-Signature: s76dghzbki44mnzocxkkzmpsgouweuar X-HE-Tag: 1752805024-787420 X-HE-Meta: U2FsdGVkX1+Rf8zWbkPSMY/Oq70XZiZ7ba78j7ZIMzCXpt2IoCQ3fAG1e4LCjwU+sEYrxcDt99iMpCmIrohe5o4r9GHs+YgMLml65MZIkv6DDtyI6tKHmCNMP/lUQ1vlikyBkbdD7pzlhnMOnyOJTtMLg6VlVghH8TYwfeFGOHiRfeSHkP6+TFq9/BztteioUeykygBlC7lmV33mTLKgUj7h0qht0O24I8byzpVLh0kvnKYzQSwPXGdE/u60jSZO6v75S0+jHHWAR7Hul4SxSxHi8h3NAdE0JgHF1ZaRlANlfUPiCejn3aeHsMA95ku9DDzB0MurbmRwu0OzJf+e93+eQQqMRxZikbALbYKsuwvhfKhx4YXdtkDS0PyCnh6E4FNeR2gULdhaWVTJN5jei3Yzn6GeHfxVqAbL5fPb8uKTyNGJUlbizwxdIZ2T4L2xdMbjESPKffoq1YQ8KnyHFV0w84g/uVzlZ5qzai3wTd6CtTIyrBGziLFSVYgb7Is5sCV/V5uNeaElXumNcKsUZDrBBCtczJGDX2wtE7eRoaBdv64lq/iDxWxWAE2saCkIR3J0Cmxowis/3OLoaRaeJ5IXIo4FAT2DhllK3aOH/UmUQOjsa0OT0XxCe8al1G9mXX1EPcmIAw69Km1SSc396rzP/9Rq+tcHWjtN8VGKFX5M+KJXN7Q+iFGWveAUOrxAlMopSNWd2tZABsjY3im2TFqG0WJ8byjhZXbF2uQdkl8qnrXnTT8kamC2MmPT7LXR4Puxfmp781XrcWYaMP5ixYBLaFdZXV7xcfehcOBxXqQNhssqHoAzjQ5dZNcEOfEgpkzq4xLIwjpdgHfJdo5J3z3j6ugEr3cLPytgW44+xeFDeQzagaszyihlFFwVTXGM2UANiJkMTiwsc8ltuPXmXu7KSyjHZMoE37nyZIxkpHzLaMGC+/2UqchxEAsCv98bAThwm84WLecJZhSDc6i bUrTAaED nbzhJt+w1F0PLNPv372hQUBm8ZcWKG/au8uTAWfSHb3adDLbcobdjFaUnPyr7cRklvQK9AhDwgGZ/NeVhmB3ZMKu+5D9IbTtU1ViC5q0QfvilDMOLxCBfHjtug9x9KFDNLDBnhjLXYCdZsK0EDiypmUBdHIiMJ9f+CebdTcf8+DqFDiQ0GVEE9UHUmAHhv8mGpR2G/hQl4tNwNl0Dx6R5M/eEOk+QeGxBKAS1GJOcBaqDo7+5HvPAz7F6WSnMNyA5DUZ2B1yd4WfstAKEWHmygXrbOdW9dYeL4zAlWiLR5FyYhQ/I4TheA/NqOXSuYxaQXzvwD740VVdBi95S3ZnRRZiE6DCkBda5FRQ2LvFZhBWmoaePBlJZATGWDSzL/HwPpaDlMrxIrnJrNHcn0WW+9OjPwmkpb7eR4NB1fh+A4L8Eyucp/rjO7L2UaS4OvUjs0nSA7vC76eN0hjTljomRcc9KskG78JsIPrAhWGg8piNV4JefY6nAhrI0uQTK3ZeolWuQmrjY3qr++9hOFaJg+hxifD3lGcahN3fA X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alexei Starovoitov kmalloc_nolock() can be called from any context the ___slab_alloc() can acquire local_trylock_t (which is rt_spin_lock in PREEMPT_RT) and attempt to acquire a different local_trylock_t while in the same task context. The calling sequence might look like: kmalloc() -> tracepoint -> bpf -> kmalloc_nolock() or more precisely: __lock_acquire+0x12ad/0x2590 lock_acquire+0x133/0x2d0 rt_spin_lock+0x6f/0x250 ___slab_alloc+0xb7/0xec0 kmalloc_nolock_noprof+0x15a/0x430 my_debug_callback+0x20e/0x390 [testmod] ___slab_alloc+0x256/0xec0 __kmalloc_cache_noprof+0xd6/0x3b0 Make LOCKDEP understand that local_trylock_t-s protect different kmem_caches. In order to do that add lock_class_key for each kmem_cache and use that key in local_trylock_t. This stack trace is possible on both PREEMPT_RT and !PREEMPT_RT, but teach lockdep about it only for PREEMPT_RT, since in !PREEMPT_RT the ___slab_alloc() code is using local_trylock_irqsave() when lockdep is on. Note, this patch applies this logic to local_lock_t while the next one converts it to local_trylock_t. Both are mapped to rt_spin_lock in PREEMPT_RT. Signed-off-by: Alexei Starovoitov --- mm/slab.h | 1 + mm/slub.c | 20 ++++++++++++++++++++ 2 files changed, 21 insertions(+) diff --git a/mm/slab.h b/mm/slab.h index 05a21dc796e0..4f4dfc3d239c 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -258,6 +258,7 @@ struct kmem_cache_order_objects { struct kmem_cache { #ifndef CONFIG_SLUB_TINY struct kmem_cache_cpu __percpu *cpu_slab; + struct lock_class_key lock_key; #endif /* Used for retrieving partial slabs, etc. */ slab_flags_t flags; diff --git a/mm/slub.c b/mm/slub.c index c4b64821e680..54444bce218e 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -3051,12 +3051,29 @@ static inline void note_cmpxchg_failure(const char *n, static void init_kmem_cache_cpus(struct kmem_cache *s) { +#ifdef CONFIG_PREEMPT_RT + /* + * Register lockdep key for non-boot kmem caches to avoid + * WARN_ON_ONCE(static_obj(key))) in lockdep_register_key() + */ + bool finegrain_lockdep = !init_section_contains(s, 1); +#else + /* + * Don't bother with different lockdep classes for each + * kmem_cache, since we only use local_trylock_irqsave(). + */ + bool finegrain_lockdep = false; +#endif int cpu; struct kmem_cache_cpu *c; + if (finegrain_lockdep) + lockdep_register_key(&s->lock_key); for_each_possible_cpu(cpu) { c = per_cpu_ptr(s->cpu_slab, cpu); local_lock_init(&c->lock); + if (finegrain_lockdep) + lockdep_set_class(&c->lock, &s->lock_key); c->tid = init_tid(cpu); } } @@ -5614,6 +5631,9 @@ void __kmem_cache_release(struct kmem_cache *s) { cache_random_seq_destroy(s); #ifndef CONFIG_SLUB_TINY +#ifdef CONFIG_PREEMPT_RT + lockdep_unregister_key(&s->lock_key); +#endif free_percpu(s->cpu_slab); #endif free_kmem_cache_nodes(s); -- 2.47.1