From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3ADF3C8303C for ; Wed, 9 Jul 2025 01:53:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CE5196B0099; Tue, 8 Jul 2025 21:53:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C9CA76B00A5; Tue, 8 Jul 2025 21:53:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B5CAA6B00A6; Tue, 8 Jul 2025 21:53:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id A00916B0099 for ; Tue, 8 Jul 2025 21:53:17 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 6919B1D8C4E for ; Wed, 9 Jul 2025 01:53:17 +0000 (UTC) X-FDA: 83643053634.26.9AC1BDD Received: from mail-pg1-f182.google.com (mail-pg1-f182.google.com [209.85.215.182]) by imf10.hostedemail.com (Postfix) with ESMTP id 81F51C0005 for ; Wed, 9 Jul 2025 01:53:15 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="c2Dt/8IS"; spf=pass (imf10.hostedemail.com: domain of alexei.starovoitov@gmail.com designates 209.85.215.182 as permitted sender) smtp.mailfrom=alexei.starovoitov@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1752025995; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=U/Ua5sKnsrIyRqKFLekLDDfmQREWYUJcYc6+mMsIips=; b=iJmwAt1emp2cC3Tb8HOuW0DwBxvGyirfp/NtNeEeQyT/fZiiHnE2++oF2WML5voHTgPj9R JhJNzVRLGmltqlOfMon+0KKGIs3quu4TMie6/q6ER/e1fnmeqeuXtwGIJv49V1PuXbMUZS KIQBabz/7Z1WQTy/TqvdiqcrfydhPvo= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="c2Dt/8IS"; spf=pass (imf10.hostedemail.com: domain of alexei.starovoitov@gmail.com designates 209.85.215.182 as permitted sender) smtp.mailfrom=alexei.starovoitov@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1752025995; a=rsa-sha256; cv=none; b=y0eLws9tjh1UQBgY9sCs3RPtVVKucKmBzFw/o/Dx447ChF56hvWcuyBgLDXUlsUVKV6FIl 2BlFw7tL4H0nEQoIyK6irTQtHmfNqPu5nMq+SlS0LIEQm/dzOCiTTBrTF+3dWHU04SV/jt TNOowLnp/tb4XI9WFo9afOlkqn07/yM= Received: by mail-pg1-f182.google.com with SMTP id 41be03b00d2f7-7fd35b301bdso5312312a12.2 for ; Tue, 08 Jul 2025 18:53:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1752025994; x=1752630794; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=U/Ua5sKnsrIyRqKFLekLDDfmQREWYUJcYc6+mMsIips=; b=c2Dt/8IS5oEdKkrrRSf2jUDheSXNkeznEE3gwdfagt7XQfGK6HfagcsxzoPDwcOHsZ /LWeXDGXcKAVwN5oHtbJXZBvMNgWf+mnFsNdINnxnCSzwFRt88lol3r3KUTNcZ7xjPEJ WmzIzQlQFHDRDAt95J8y1cJLl10T+1wJOEbi4quMDZq6X7IWcZ+jQmM0wp1p5KcKVR+u 7etfb1vyT5oVwtn8Co4IHPeAW/o5bRJUB5b9qpa2XEP8qvBsPJ4D5LK6nALqebDx8CWA Ync7EFjcGr9db1k469ZD640Zcb8WTn2YVkGELDJwSqbEYzJ7GiPQ6LavceRHGFBdpSUw vqxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752025994; x=1752630794; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=U/Ua5sKnsrIyRqKFLekLDDfmQREWYUJcYc6+mMsIips=; b=TjElM+axvLyPGWhdO4/IzERDD3+Rjy3Sybk7bLN8eZBQVy6he3MuJh0VeZZ0LQIbhn ZIIR5tA1rNeTmFhOjWpnKNe+McQXP1tcMxQ1V0/CVpf5y5Cw9/4HKQoClzERcFgf59DA qHgzJKR/f38giAXHEEP4BPAM1ZTasZWrIScdaHb3AmE8em5ZK0TdoxA4FkBiY6GxPbiz iLh6m52GtofxaeQsIyhwSTIaJmy5rdozMsr/DvwPmlxgn7tCZNAN4fDMLQE3AhKCtR4d Ow/W2dG2rc2R6aluse4uudMFssi56jx0Vh8LW3ZD7DB9HZOUVKNJDKW01pSNnpae3yO4 rbmw== X-Forwarded-Encrypted: i=1; AJvYcCVauXQ24AEyuRrvw6usBp0P32w8MLnQKmcuohgYUwnryoP5oTyB5YRG6h7cblgk/n4kR2mM3KSwTA==@kvack.org X-Gm-Message-State: AOJu0YwkG/LCqU2czm434LmFXvnBxOK2+o9/XTj4MvjEbjku5GBlWRc7 7Vhj/ZH7UPiidVAJn5/aPkYflpMSmBpr1+Wm7vnLqHRksx1tO7uSsooO X-Gm-Gg: ASbGncsPIAKUomR2/Ocapc88B64ZT6Pm1sOFcdn5KKB9BaXs+JsRW0ljMbpWP/4cZxz PHuLMN4NigBNlt1XhubHCcgciWg1loWRDkC6Xp7a7bBu8mxCMMOkCqmDd+/b+TpEI7fjYJ+lf3M wfidxk+IcGUZvSeewTB95r2DNxHTIE3K+hZ6vHqCDfxCayGVVFIe2UoM1ikJvOhZgtQeu+Mo5UG MQ8GRf+KWhqwsYY8VRFzyYWNAzAk9Mp6/ZD+vxoAlDCIOCRd48vNfyypARGn6ronbPA7/IVsUMI VJxFeYftRTvfYywStpGEFprzKvOba0uqRLiPPg7mzYjDJ60ndK0MPV0saHmuggUOOaWJUbuxlHk rqRJdfiWt36gjxI576Psa2hhjg/M= X-Google-Smtp-Source: AGHT+IEn9bm8Gu9IwAgDOMIlYpQ4UCQB5S6UKhK7vqUV6G2PUzb1SmtZ/06k2i9CXCRL3vcfqpDstQ== X-Received: by 2002:a17:90b:1b0f:b0:310:cea4:e3b9 with SMTP id 98e67ed59e1d1-31c2fdfb963mr886368a91.34.1752025994358; Tue, 08 Jul 2025 18:53:14 -0700 (PDT) Received: from localhost.localdomain ([2001:558:600a:7:a83d:600f:32cc:235a]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-31c3017cc30sm440282a91.32.2025.07.08.18.53.13 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 08 Jul 2025 18:53:14 -0700 (PDT) From: Alexei Starovoitov To: bpf@vger.kernel.org, linux-mm@kvack.org Cc: vbabka@suse.cz, harry.yoo@oracle.com, shakeel.butt@linux.dev, mhocko@suse.com, bigeasy@linutronix.de, andrii@kernel.org, memxor@gmail.com, akpm@linux-foundation.org, peterz@infradead.org, rostedt@goodmis.org, hannes@cmpxchg.org Subject: [PATCH v2 3/6] locking/local_lock: Introduce local_lock_lockdep_start/end() Date: Tue, 8 Jul 2025 18:53:00 -0700 Message-Id: <20250709015303.8107-4-alexei.starovoitov@gmail.com> X-Mailer: git-send-email 2.39.5 (Apple Git-154) In-Reply-To: <20250709015303.8107-1-alexei.starovoitov@gmail.com> References: <20250709015303.8107-1-alexei.starovoitov@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 81F51C0005 X-Stat-Signature: doxmyt43tkgwmow9mfp3rufw5nig95hh X-Rspam-User: X-HE-Tag: 1752025995-387682 X-HE-Meta: U2FsdGVkX1/LNedvH7ZhiImDjHOptdkjveO9gMmCheCiH7K9azCJ/1vQAz53wB6sqyMBZhQtHw4KWt84hi2vNEiOWEqjxcFuRfon1C1n4G71GyKx298xTATXLU68ysVhwezOcWxNnk+HXxXn7FJxeVeQPu4gO+w4z5hngXSv/VBWp3Et1qUjhNuP+waO0MzzWm3h3ZQta7ZYgRcn0n9tPR5fTlkzPQ/qU9/GeVAi2qdMxQ1iAOYp47E9Cbq9m8/YsAIjnAVfQzU48JJJ9rlUdMkkTYUs5cIkxX6Ak+BXpHmwbqNUHkiGyVK7cJFl7XWoGtPfxtETIm4ZZETtKp/n3NjEFK3hF+IUSNvk8piubNfe3Y2S7z2uoycgCNc8rsjx9SuFLBmP0UXOH7M2Egg8SlnrlZ8YrhihK1JTjdIjGlMoVVqtbk5erS9PtFXwys/jZKwCj8znZ9jxfDrttLgU81C27u1GRr9phaCXJvr32ur+XG//VvhtzfG73deuiavcsQfPsJcRiJH8ojbHt+aek7l9JQDqxQDZhIMdk7PoD9GayBzrIIHo5eTQznIPqZU2X0WFSWyUxKRwGX/93pf0csgnZGpspENUIyqphuKEQXkxfoxdkcwkFiC6SK9420SiLhSF0sIz1BVQtA7r1FY04p8ycapoHrjyIxCCfzDsXbmgrdag3E1inxZvdLIUIm1ApDJ9F9nuiVWOsmB1C0r6zVrjDMnJgpPnyIpCkiZdA6AJdmBmr6c9z0b61QUOFXwvJ0U/nakCaj2a2dqMjU8Qcf1YWNa/ByB+rwzm+ds39POYkjiOxqzJKQiNeLTosH6JmG8hGGUG9aqh18MxZVjo+Vw4RunSgrJs8vD8yOUIDBtnVbtiESt4PnsWxfHyKZqOXww1dJsx3ukOyejNrEcIfEz/MjQs91Pislzad8G8FMWVuO+QCFeuKOpoYdCsfYXW90RrFpONo612HkfwV5a h03hNmZz /Dv1SWbq38LeTYK4THn/foyvu/T7P7ie4/L9Il81ceAYVnYc75e10cJSbdoFTFqROdnkYjJrs1bh0XLiV9tUxF8NRwi9q2QnIgb8jDHotvd1soIZzbLQ+xdfDI8ZQx0C7EKVgQaKCSYeWZ6ozJL+QxORAV6DzBo0Ds8my7vVpLA6acztcnwLYcITZr5bwtPpurX+vQ2bZT0Y9PIie+yAqrFi1K4X5pEosXBfW6OwUbGSc+S30iIb/7sHStvyuLg42OP63k1SPghrP4E/qPnppG2PqFT3fxSfcDA4Yk6aZCOL2SkX8tnZtQJDtAZpeNVaroiKOBarCB4emZI0fFcrIndGatbe3DeAaQKDiPlX2D9nxWYo9hnCbk1pUidB2ob6lJG2v8ZzR+Oo7uowopHL/7mWpwJtNctbnpgswyCc0Jo0EomEVAdLS/VY4Yhh13oSf/PYhkWZpN/vCeukJ0xCwL6lESNwLFP/Md8uQw68e3cykMn6Ksvr4qnwWfO5wlPfJtHdx8w1K26D55agUCt4l1MpceRUzBTVBs7byMwuWKO4ti05CTXDBoFlN/RRbgW+mXdk/ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Alexei Starovoitov Introduce local_lock_lockdep_start/end() pair to teach lockdep about a region of execution where per-cpu local_lock is not taken and lockdep should consider such local_lock() as "trylock" to avoid multiple false-positives: - lockdep doesn't like when the same lock is taken in normal and in NMI context - lockdep cannot recognize that local_locks that protect kmalloc buckets are different local_locks and not taken together This pair of lockdep aid is used by slab in the following way: if (local_lock_is_locked(&s->cpu_slab->lock)) goto out; local_lock_lockdep_start(&s->cpu_slab->lock); p = ___slab_alloc(s, gfpflags, node, addr, c, orig_size); local_lock_lockdep_end(&s->cpu_slab->lock); Where ___slab_alloc() is calling local_lock_irqsave(&s->cpu_slab->lock, ...) many times, and all of them will not deadlock since this lock is not taken. In other words this lockdep-aid avoids the following false positives: page_alloc_kthr/1965 is trying to acquire lock: ffff8881f6ebe0f0 ((local_lock_t *)&c->lock){-.-.}-{3:3}, at: ___slab_alloc+0x9a9/0x1ab0 but task is already holding lock: ffff8881f6ebd490 ((local_lock_t *)&c->lock){-.-.}-{3:3}, at: ___slab_alloc+0xc7/0x1ab0 other info that might help us debug this: Possible unsafe locking scenario: CPU0 ---- lock((local_lock_t *)&c->lock); lock((local_lock_t *)&c->lock); and inconsistent {INITIAL USE} -> {IN-NMI} usage. CPU0 ---- lock(per_cpu_ptr(&lock)); lock(per_cpu_ptr(&lock)); Note that this lockdep-aid does _not_ reduce lockdep coverage for local_locks. If the code guarded by local_lock_lockdep_start/end() will attempt to deadlock via local_lock(&lockA); local_lock(&lockA); __local_lock_acquire() logic will yell with lockdep_assert(). Signed-off-by: Alexei Starovoitov --- include/linux/local_lock.h | 15 +++++++++++++++ include/linux/lockdep_types.h | 4 +++- kernel/locking/lockdep.c | 4 ++++ 3 files changed, 22 insertions(+), 1 deletion(-) diff --git a/include/linux/local_lock.h b/include/linux/local_lock.h index 092ce89b162a..04de6ae9e5f0 100644 --- a/include/linux/local_lock.h +++ b/include/linux/local_lock.h @@ -81,6 +81,21 @@ #define local_trylock_irqsave(lock, flags) \ __local_trylock_irqsave(lock, flags) +#ifdef CONFIG_DEBUG_LOCK_ALLOC +#define local_lock_lockdep_start(lock) \ + do { \ + lockdep_assert(!__local_lock_is_locked(lock)); \ + this_cpu_ptr(lock)->dep_map.flags = LOCAL_LOCK_UNLOCKED;\ + } while (0) + +#define local_lock_lockdep_end(lock) \ + do { this_cpu_ptr(lock)->dep_map.flags = 0; } while (0) + +#else +#define local_lock_lockdep_start(lock) /**/ +#define local_lock_lockdep_end(lock) /**/ +#endif + DEFINE_GUARD(local_lock, local_lock_t __percpu*, local_lock(_T), local_unlock(_T)) diff --git a/include/linux/lockdep_types.h b/include/linux/lockdep_types.h index 9f361d3ab9d9..6c580081ace3 100644 --- a/include/linux/lockdep_types.h +++ b/include/linux/lockdep_types.h @@ -190,13 +190,15 @@ struct lockdep_map { u8 wait_type_outer; /* can be taken in this context */ u8 wait_type_inner; /* presents this context */ u8 lock_type; - /* u8 hole; */ + u8 flags; #ifdef CONFIG_LOCK_STAT int cpu; unsigned long ip; #endif }; +#define LOCAL_LOCK_UNLOCKED 1 + struct pin_cookie { unsigned int val; }; #define MAX_LOCKDEP_KEYS_BITS 13 diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c index dd2bbf73718b..461f70f4ca28 100644 --- a/kernel/locking/lockdep.c +++ b/kernel/locking/lockdep.c @@ -4963,6 +4963,7 @@ void lockdep_init_map_type(struct lockdep_map *lock, const char *name, lock->wait_type_outer = outer; lock->wait_type_inner = inner; lock->lock_type = lock_type; + lock->flags = 0; /* * No key, no joy, we need to hash something. @@ -5844,6 +5845,9 @@ void lock_acquire(struct lockdep_map *lock, unsigned int subclass, */ kasan_check_byte(lock); + if (unlikely(lock->flags == LOCAL_LOCK_UNLOCKED)) + trylock = 1; + if (unlikely(!lockdep_enabled())) { /* XXX allow trylock from NMI ?!? */ if (lockdep_nmi() && !trylock) { -- 2.47.1