From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB202C7619A for ; Wed, 12 Apr 2023 16:44:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 832E2900005; Wed, 12 Apr 2023 12:44:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7E180900003; Wed, 12 Apr 2023 12:44:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6A8F8900005; Wed, 12 Apr 2023 12:44:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 5B71D900003 for ; Wed, 12 Apr 2023 12:44:59 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 356A61201A3 for ; Wed, 12 Apr 2023 16:44:59 +0000 (UTC) X-FDA: 80673313518.08.744465A Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) by imf13.hostedemail.com (Postfix) with ESMTP id 1CB0920007 for ; Wed, 12 Apr 2023 16:44:53 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=erPiXlRd; spf=pass (imf13.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681317895; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=mblYIOQp15DPpNJLZonRj1Q4Mutz9sHweMpSOIOysuo=; b=s8qxnH+DIT9xgnRqrP1UhzaPWx+JyoPrVyFtDX5Wvyi+kw0DXvmIN/LknyFBF7W1UEdNNl L22obNjS78ttXqCJYBy++bUcHGSN+7y0ZU6R1z4TAeF3OMfsV5kJ/AYwtXtIJ52Jy0PGQL JvI5LWJdnhrXl48rB4bbRo/XEKPAp0k= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=erPiXlRd; spf=pass (imf13.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.214.179 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1681317895; a=rsa-sha256; cv=none; b=BK41hnR18FtZZUCo3IbwdvPMS4Tf6BYgck4Qcqf9UnyW5CXDaYSTFPaR7E11qB0crgWq8K holvy+6lbj/N9CLYFAsD/9R1/zarR2WEYSqac7+GSvvgzLcVqTkYpyj6dVeM4D6uTIqs6P c8/qWzzsJSIkFgSivNXWd/0aCZenmQA= Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-1a5266d204dso3776085ad.1 for ; Wed, 12 Apr 2023 09:44:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1681317892; x=1683909892; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=mblYIOQp15DPpNJLZonRj1Q4Mutz9sHweMpSOIOysuo=; b=erPiXlRdUqCIakRu1oMia3Dog0aIQXlkglwS6vTrMxlss7fwcC+h+CTYbqa1cPR5H/ dKdLJa+uCjEG41vNTzeplvpf8rTd8HxLjoxXX/G3XvC3g9HeHbRcUYbJvnf2h0tpBRZN 4wrAirJiciiw/Jc6xh89K4G6K4i4bAUANPAFlj7Q4YDNWcTnZpR0t2t/U4/ZSZ/GcH1C pMGIuEsx+S5VTpO0f0GnX4bFUKdrDUBS7243d7a3TVGZ269Rj34Ojq1Xbfb37bBOA6th dUHli54N2TWld3+dGHW1ieY9Bkh4I9wkeirVd5Y23BH8rlRKJmvAp8jZajpJ1mUFgX19 WmBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681317892; x=1683909892; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=mblYIOQp15DPpNJLZonRj1Q4Mutz9sHweMpSOIOysuo=; b=AMa2dh8FNyS5mqsSgRMERmUyjGdVyNSvDP4TQWT9TURxILBo9m3duB2eJb0HculHjj G+Lu69VWXx6mAbpS5EzHThrUEPRSEL4ovGaVrK2kN6MkDgtQE5FvIeWHfF8V9KguAIp1 W+xigJvQamxaKNp671DkrW4KWqJLmC5Nf41WxsV4mh2RVNQYh8an11s8Wp9PK2VU+Bl/ GjihMd0Mmop7j6y4ARVS0UJXRPiz24kEp2mZXDO7rIHm51O2zeyECmVOOrdLrCYtX/nI 6yUhod3CPzYL9IRYvPiqCscTuZY5I9oxBc3VU6q//l/rfV7vnoGGuMMCtTsp+CyAapb+ PHTA== X-Gm-Message-State: AAQBX9djmJaxfrG1l2AkVoToNQzPucw3XZzKQHW8zFqEv6m/oFygmd5H qbCBlRdXgwSo5pbnpZhpHT61mA== X-Google-Smtp-Source: AKy350bJt65jHqdmpGK2wF697D8XC8+bpLyMk+/sJfcfIg118HnQ/s1c7ViNh3qzSLz6Ou7D2topVA== X-Received: by 2002:a17:902:8f82:b0:19d:1e21:7f59 with SMTP id z2-20020a1709028f8200b0019d1e217f59mr19185705plo.0.1681317892609; Wed, 12 Apr 2023 09:44:52 -0700 (PDT) Received: from [10.200.10.123] ([139.177.225.225]) by smtp.gmail.com with ESMTPSA id iz14-20020a170902ef8e00b001a2104d706fsm1873858plb.225.2023.04.12.09.44.46 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 12 Apr 2023 09:44:52 -0700 (PDT) Message-ID: <76e15f10-d3f1-2cab-63cb-25aa3b4f2cd4@bytedance.com> Date: Thu, 13 Apr 2023 00:44:42 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.9.1 Subject: Re: [PATCH] mm: slub: annotate kmem_cache_node->list_lock as raw_spinlock Content-Language: en-US To: Peter Zijlstra Cc: Vlastimil Babka , "Zhang, Qiang1" , Boqun Feng , Qi Zheng , "42.hyeyoo@gmail.com" <42.hyeyoo@gmail.com>, "akpm@linux-foundation.org" , "roman.gushchin@linux.dev" , "iamjoonsoo.kim@lge.com" , "rientjes@google.com" , "penberg@kernel.org" , "cl@linux.com" , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" , Zhao Gongyi , Sebastian Andrzej Siewior , Thomas Gleixner , RCU , "Paul E . McKenney" References: <20230411130854.46795-1-zhengqi.arch@bytedance.com> <932bf921-a076-e166-4f95-1adb24d544cf@bytedance.com> <20230412124735.GE628377@hirez.programming.kicks-ass.net> From: Qi Zheng In-Reply-To: <20230412124735.GE628377@hirez.programming.kicks-ass.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 1CB0920007 X-Stat-Signature: d4n9cx1udmtt8gpwq1zyqgwtsij51s3e X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1681317893-262451 X-HE-Meta: U2FsdGVkX1/X/alBoqVJraANXBvFh3YKbkbK5jYdCtSyJKsx+AHugvWtyYwtAcS5eAyvudAqIAamIC1VZXi24qcK4153GNyWsT5wS+FG1lha0M3CMP3N3QrUxuwU07mYMctplYvcSSYZYWpuVVjHRgMbQ/WPVxFdAqyTf6WivXAQo/hFAbylJgPqqmr3AzraV2N1+rhGEEh2fEIfclw8bn6yjZQKHj9eQiNfbUioBUYjJ1WLWe5yQa2m0ubk7H2tVfyeAreY0EFufzDzmTWG2IDjuoT+KkBBb4NTqfZA4RZiW1q+gBYaDbKX/U77YKg7+6m/ZO0EfpuOw5IFMsfsns+7WyrlOXmhjD3k3G5zRWaa1Gn/MKpsAXb/87HK6EkIEm+/18ND1q6D23ig4AR2tQWemCILWyczfrfVDxGul1kS+9K05YKgT+SZg/5cG4w4rk6WhC3MmKHRi5x/DuV5xyiTewj/GCryHZ4FwmGZXzWWrx25xv0mZn2HcKMoIpAB79EQ5NjiI5aHU8X/WV+AtWiuIDWjG5EZO2GTJHh54ikcJr7cSQEKTz4xTps7rIyT/y+hbdJKfWmcIzJquVb6Tg7m2RF61DUU5PnX1PkBacmpEdh5Qip98zZcaOrdZyZ/jS2MRLB2qrnfebX3dEmPVXe/d8k5p1GdFGx/H1NkoSM36t1yaEarVDuDIMN5+GEPjK+O05nPsygKB3g9vPWxyyaQOn/7NRFfK6nnX5ST52Ny9jLe+BA6+WSStDEWk0vHaQNHQ5ZHCnmZhREBAIQi5abX8PJk+3FXcY7y2roS2iP0uWwg1cWWPKROxXXjAdB7LEjt9ZesqIjikQ+/f3Y+djiNxDeMaFAwaszmiDTL47p1IlWumhaykA6Wgtk03GVeyP/rnsLK86E7qQ2LaUoRP76JzNAhbQp6ga42Ixl6E0+TfNiT4Kz5EaC1rFd4Ik99iY/gURbhAPtDp6YyQE6 EgQr6xwy tisc2Kij4TawZsnHRtDvM+Z6hlnbmtA4Z5QMIywuBBRqxQn8RAm4BYo4juncnIXZPOMHLf+wTY9k29B6zxGTBrIH5WYMz0qBB8r7jsVHURz8DnNINRk1cJk+ay5kwePvm0G3EyQISS7cEobR72ZGWDsdERkaWxkodxkRpsRmOs5G9k0wQD+AWjMruVC8QAxMz9By8e+b40oL1hyqHJ9/zKy0fV/OrSdzxk6sHs0x/ymOYC4de9oJWZYMkrKZbLK3aqhn+lYgLraORLQwcMbDXCfp0iHyAJ/SXlzLd/HS3dJ8V0NINVoKE0Lhb2d1IJlud9p90AYMh8LRB5snbk+28QTHMi9Dz3a6BBIzLAOe0O6CvIAw+X5xkkjJXe2GvE1zj1omtAGt1Kaki+/gqKI8ArIEXXdVRWkhBNToMXluFpXx+CmwElOz/dl/tB7QlLOrshsaQhFsKESGZy/WbVC44G2Pzl+ObmIAk+vM+XymstlIUgXj3vX8XVgF6XY4hqlUTVrt+iV6UPjgxUuDD2Ozv/ADRw1doajz+H3BLJWmuuey1T08= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 2023/4/12 20:47, Peter Zijlstra wrote: > On Wed, Apr 12, 2023 at 08:50:29AM +0200, Vlastimil Babka wrote: > >>> --- a/lib/debugobjects.c >>> +++ b/lib/debugobjects.c >>> @@ -562,10 +562,10 @@ __debug_object_init(void *addr, const struct debug_obj_descr *descr, int onstack >>> unsigned long flags; >>> >>> /* >>> - * On RT enabled kernels the pool refill must happen in preemptible >>> + * The pool refill must happen in preemptible >>> * context: >>> */ >>> - if (!IS_ENABLED(CONFIG_PREEMPT_RT) || preemptible()) >>> + if (preemptible()) >>> fill_pool(); >> >> +CC Peterz >> >> Aha so this is in fact another case where the code is written with >> actual differences between PREEMPT_RT and !PREEMPT_RT in mind, but >> CONFIG_PROVE_RAW_LOCK_NESTING always assumes PREEMPT_RT? > > Ooh, tricky, yes. PROVE_RAW_LOCK_NESTING always follows the PREEMP_RT > rules and does not expect trickery like the above. > > Something like the completely untested below might be of help.. > > --- > diff --git a/include/linux/lockdep_types.h b/include/linux/lockdep_types.h > index d22430840b53..f3120d6a7d9e 100644 > --- a/include/linux/lockdep_types.h > +++ b/include/linux/lockdep_types.h > @@ -33,6 +33,7 @@ enum lockdep_wait_type { > enum lockdep_lock_type { > LD_LOCK_NORMAL = 0, /* normal, catch all */ > LD_LOCK_PERCPU, /* percpu */ > + LD_LOCK_WAIT, /* annotation */ > LD_LOCK_MAX, > }; > > diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c > index 50d4863974e7..a4077f5bb75b 100644 > --- a/kernel/locking/lockdep.c > +++ b/kernel/locking/lockdep.c > @@ -2279,8 +2279,9 @@ static inline bool usage_skip(struct lock_list *entry, void *mask) > * As a result, we will skip local_lock(), when we search for irq > * inversion bugs. > */ > - if (entry->class->lock_type == LD_LOCK_PERCPU) { > - if (DEBUG_LOCKS_WARN_ON(entry->class->wait_type_inner < LD_WAIT_CONFIG)) > + if (entry->class->lock_type != LD_LOCK_NORMAL) { > + if (entry->class->lock_type == LD_LOCK_PERCPU && > + DEBUG_LOCKS_WARN_ON(entry->class->wait_type_inner < LD_WAIT_CONFIG)) > return false; > > return true; > @@ -4752,7 +4753,8 @@ static int check_wait_context(struct task_struct *curr, struct held_lock *next) > > for (; depth < curr->lockdep_depth; depth++) { > struct held_lock *prev = curr->held_locks + depth; > - u8 prev_inner = hlock_class(prev)->wait_type_inner; > + struct lock_class *class = hlock_class(prev); > + u8 prev_inner = class->wait_type_inner; > > if (prev_inner) { > /* > @@ -4762,6 +4764,12 @@ static int check_wait_context(struct task_struct *curr, struct held_lock *next) > * Also due to trylocks. > */ > curr_inner = min(curr_inner, prev_inner); > + > + /* > + * Allow override for annotations. > + */ > + if (unlikely(class->lock_type == LD_LOCK_WAIT)) > + curr_inner = prev_inner; > } > } > > diff --git a/lib/debugobjects.c b/lib/debugobjects.c > index df86e649d8be..fae71ef72a16 100644 > --- a/lib/debugobjects.c > +++ b/lib/debugobjects.c > @@ -565,8 +565,16 @@ __debug_object_init(void *addr, const struct debug_obj_descr *descr, int onstack > * On RT enabled kernels the pool refill must happen in preemptible > * context: > */ > - if (!IS_ENABLED(CONFIG_PREEMPT_RT) || preemptible()) > + if (!IS_ENABLED(CONFIG_PREEMPT_RT) || preemptible()) { > + static lockdep_map dep_map = { static struct lockdep_map dep_map = { > + .name = "wait-type-override", > + .wait_type_inner = LD_WAIT_SLEEP, > + .lock_type = LD_LOCK_WAIT, > + }; > + lock_map_acquire(&dep_map); > fill_pool(); > + lock_map_release(&dep_map); > + } > > db = get_bucket((unsigned long) addr); > I just tested the above code, and then got the following warning: [ 0.001000][ T0] ============================= [ 0.001000][ T0] [ BUG: Invalid wait context ] [ 0.001000][ T0] 6.3.0-rc6-next-20230412+ #21 Not tainted [ 0.001000][ T0] ----------------------------- [ 0.001000][ T0] swapper/0/0 is trying to lock: [ 0.001000][ T0] ffffffff825bcb80 (wait-type-override){....}-{4:4}, at: __debug_object_init+0x0/0x590 [ 0.001000][ T0] other info that might help us debug this: [ 0.001000][ T0] context-{5:5} [ 0.001000][ T0] 2 locks held by swapper/0/0: [ 0.001000][ T0] #0: ffffffff824f5178 (timekeeper_lock){....}-{2:2}, at: timekeeping_init+0xf1/0x270 [ 0.001000][ T0] #1: ffffffff824f5008 (tk_core.seq.seqcount){....}-{0:0}, at: start_kernel+0x31a/0x800 [ 0.001000][ T0] stack backtrace: [ 0.001000][ T0] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.3.0-rc6-next-20230412+ #21 [ 0.001000][ T0] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.14.0-2 04/01/2014 [ 0.001000][ T0] Call Trace: [ 0.001000][ T0] [ 0.001000][ T0] dump_stack_lvl+0x77/0xc0 [ 0.001000][ T0] __lock_acquire+0xa74/0x2960 [ 0.001000][ T0] ? save_trace+0x3f/0x320 [ 0.001000][ T0] ? add_lock_to_list+0x97/0x130 [ 0.001000][ T0] lock_acquire+0xe0/0x300 [ 0.001000][ T0] ? debug_object_active_state+0x180/0x180 [ 0.001000][ T0] __debug_object_init+0x47/0x590 [ 0.001000][ T0] ? debug_object_active_state+0x180/0x180 [ 0.001000][ T0] ? lock_acquire+0x100/0x300 [ 0.001000][ T0] hrtimer_init+0x23/0xc0 [ 0.001000][ T0] ntp_init+0x70/0x80 [ 0.001000][ T0] timekeeping_init+0x12c/0x270 [ 0.001000][ T0] ? start_kernel+0x31a/0x800 [ 0.001000][ T0] ? _printk+0x5c/0x80 [ 0.001000][ T0] start_kernel+0x31a/0x800 [ 0.001000][ T0] secondary_startup_64_no_verify+0xf4/0xfb [ 0.001000][ T0] It seems that the LD_WAIT_SLEEP we set is already greater than the LD_WAIT_SPIN of the current context. -- Thanks, Qi