From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93BFBCEACF3 for ; Wed, 2 Oct 2024 18:58:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E7F0F6B04E8; Wed, 2 Oct 2024 14:58:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E2DC56B04E9; Wed, 2 Oct 2024 14:58:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CA6C76B04EA; Wed, 2 Oct 2024 14:58:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 9D7686B04E8 for ; Wed, 2 Oct 2024 14:58:39 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 3EFEF1C6C07 for ; Wed, 2 Oct 2024 18:58:39 +0000 (UTC) X-FDA: 82629573558.01.ED077ED Received: from mail-lj1-f170.google.com (mail-lj1-f170.google.com [209.85.208.170]) by imf02.hostedemail.com (Postfix) with ESMTP id 6756980011 for ; Wed, 2 Oct 2024 18:58:37 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=LHBgxkgB; spf=pass (imf02.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.208.170 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1727895477; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pJihvFLY+/PB1Gfvqw4s7qDkzZW2UtDYa8qi2yTs9+c=; b=s0oxbsuVgUgHMzmUWRb2xXXx/RmEceMhx71fQUSK2iXCKfxVIIoIzScAztJIUPtJpxoFRr a7kY1CDqdYTsAZEsXQn2Jl/eEotQ9QZ8ZpTPzVkA7twMMmjx+Fp7dz9BacEE55W+qhIR2f 9SZtj6CdHLreblKYQeuXvH1iBt89pyE= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=LHBgxkgB; spf=pass (imf02.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.208.170 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1727895477; a=rsa-sha256; cv=none; b=4sAnNh2bHak4qPhc3ArFoe8Q4zGvXxoRnIlC2l+9VvhAlkVOiFk9xvldDTegSvz7UX9yOI hsDeEubDyU3MmbTpM0D9BBicuaNK//SJivrz0dtz5AnmmWBCiUKK4ovcSJ1thSJFNF8sKr ef4ndC/YzxdJj27KYlwtUMo/UR/0/ew= Received: by mail-lj1-f170.google.com with SMTP id 38308e7fff4ca-2facf48157dso1484711fa.2 for ; Wed, 02 Oct 2024 11:58:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727895515; x=1728500315; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=pJihvFLY+/PB1Gfvqw4s7qDkzZW2UtDYa8qi2yTs9+c=; b=LHBgxkgBKeVrpDBbX3haQrJm8gduGM2qBrDVQSp//e9kh/pog4htLXLPXFz9OFcjmg Aq1u48gYrxyZFUPCCiGV5n5QHxGZqzenR+FQb8IV59NN7ppvDM5k9zZ2E+7d6rojTXzV fEeBcY/u7LesVVMGUig7mXtA+N9XZwlf404aei1/eaQBIf2UKW3jpQ6a/sPY4RvJNQFd UcPwGiXc80mCJn2Xkk4Yb1Y7hXmjk+OYmf9U/YwC3U8BcarbAfdZaoKkEOOeP6M6Z/13 EGAayKJrD5jftl6uv76KmrNRbWsK73BmwzYsUD6/QtybIk/9HOJ7LZMdl+e0lIIugf3a WIfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727895515; x=1728500315; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pJihvFLY+/PB1Gfvqw4s7qDkzZW2UtDYa8qi2yTs9+c=; b=cAj42PR48JIRPSts9P5kpYDLoZFD08U9yqjhnXPdcymDFP1JYE78tBMIJ4hGBLoxFH o+24iSh+POgaLxxey5vX0df09k98yqKuF/v9RXszXDmQQeALTiuv4aKsCa+sOJ8K5zKE io23qrMTk558vX1abzOVbwFgKjeybpmJq3O9kWDsT6YFDwbHO1N/wbdrMV7rpvqWb9w+ 7o3vY3x0CUq56hSGKJOfG3Ous9CFM/ODmwnTA0rlTav4lGrIZMHO6dHFVkjTI8gY20Ky RtLxqnlqVpb1vuU+2as6lJVYQqIsy8VmeGdLPXvsU6wmtS36NEiNTmPEk314vHKztR3S LveQ== X-Forwarded-Encrypted: i=1; AJvYcCUuQiwrJy3Vq8glZTOCAEt4gYJTS6FM9NYTu3Bpx1zXWJ2EqHvgWKZWTfgSUmpLUPX5I91G6LYoUw==@kvack.org X-Gm-Message-State: AOJu0YxzqNKR6jzCqXmS8bhr7NFiQcp1ZSWMpsvrYSlcw5ZgUl1lqOaX hwNlXWwqHAice/2EZW7978YxSF593XdmWrD49WpodSqMyAXZ+td/HFEdfqUivlQwvlz1dr5o+YF 92oECkvro4EFk30UdCkC84TP5j7hMNehLR+jo1Q== X-Google-Smtp-Source: AGHT+IH8MpcRdjd9DhqsAwLkZKoQVw1e6Q6fMrgqRcM0MtqUumsbTF1Cw4qA/fQw5PWFfIakZctyS6QFiOvCg1HPfV8= X-Received: by 2002:a2e:e09:0:b0:2fa:ded3:f6aa with SMTP id 38308e7fff4ca-2fae1023cccmr21302301fa.20.1727895515241; Wed, 02 Oct 2024 11:58:35 -0700 (PDT) MIME-Version: 1.0 References: <62a65418-2393-40ec-b462-151605a5efcf@stanley.mountain> In-Reply-To: <62a65418-2393-40ec-b462-151605a5efcf@stanley.mountain> From: Kairui Song Date: Thu, 3 Oct 2024 02:58:19 +0800 Message-ID: Subject: Re: next-20241001: WARNING: at mm/list_lru.c:77 list_lru_del (mm/list_lru.c:212 mm/list_lru.c:200) To: Dan Carpenter Cc: Naresh Kamboju , open list , lkft-triage@lists.linaro.org, Linux Regressions , linux-mm , Andrew Morton , Arnd Bergmann , Anders Roxell Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Stat-Signature: 5fgty3chh8wqmocr64s65f3txnx5s4wx X-Rspamd-Queue-Id: 6756980011 X-Rspamd-Server: rspam11 X-HE-Tag: 1727895517-869955 X-HE-Meta: U2FsdGVkX1+NOHh9ZM4pBz4MO2Phpe7TMHZ7dqfk1k9DkWyEPK6oHgnX6gMgRFw24xubx18ngg6yayMUc3OfOa9LOYsZuPkdxtjVCtt7AkBfC4aksxkAs08ikLTwNU9W/qHmA/TXvVtWOW2+mRkhwLfOGFEgdTxQzX2ZgDxO9r+3MFjV4FyLCZnT8p3s7era8V9fclCvTN2XgzL0MPVl5O2cCE2RbeSDPxGIkrjrU3Uj/XSiiA/EXHJsZyRTegP9EufFdOOuO3YJs3bkRwbpPZqgNvinJGg954ZcNWOO09gXKg74SQbUMDSFZhZVNvMFNahzswJaQKOjQT5EqAKRTZZHFgBiopx3TrcTerS23i817kSuwHMTYBlf+7mpv+TIymlRpWcgkDI7x6XiZIHGWFG3NOD6gP91ayhm6pSKTDnu3hO0KVUnu28pkM0AwqhVO3QPMe3Z9LhWgbY3+ZEfFMOmmvG/gm61c58aHWXnuBOOHu7BfvEm4gWwahP2+nF3j6xVaTNkhmd9Co0+4qznO5km0JRfwkomGKF5Uw0eURQehoaDZq6+ory/iS3mIYBik6CxFPh7D2g3sPsvEaqbkkUsn11mOqqZVs5gwu+jflEY7qafbR9p3mQm4pL5zEDYqzwLmBKqz2pX/Ghi0aTll5aa3eQXTOT6uBNyi5k5cs8We1I76UwKR8WLbSaTe+Nn5JE9+xilvenTplrf/ajtPqMWJCcpiieTKrEPXI/TDrGmeqBYXZYM7nBMYLNCpDQahiIR9XQlElsfvv4tPAkgelproSJewGEySJ5JPltnxSLVS3zaUI83RB8QgLLz9CA3p0a+zKjCRLFle/KtLSJ0qGy2IvKMI1/R+w1VIvyc56Dm2Hq7qdQVjS1N4KG88vhMHA0fsS0pZ7PuqcYP5yKvx+gWUVwjdAD7qxHbuXJBUY/35uU1mo/qz5ooISMaLuetWEFVAYxDPdweoCrfgmP QrrLwjaa y/rHH2n9r8lw0uJerDLemw6VX3DOnjUxGH8fQpeC5LOw5FHcFNI73ABE7QNYK2i/1nKsEptycEQmKU3GN4SM4jND1E6VL2TAb4m0UORdCAhRR+kJztS/iBPGQ7p7o0Um0A2xRz05O1Fey3Vwutvk20y92FdsHXpWbXt97Jnjm+8i1hRxlULlnn1nCdY6Lv+RQUrfgxXamL7eamesU56qYWAdc1Mbn5j/W/ZTNk8QnWCf3zDQs1MwJAiqlY2Y5QBuDxIIHDyrTz42erJEo3MI0xGfZL2rSLmD+Lnb1EsQT4wrl/Esr7RBokw4JF3UJTF84otxlX5GDAnVzHRaSoh1S4cdIi03Y6JKryaYCHvRcTxjuLBHsWzk2QPlnLmfQrwAE6umMx3aPIscWm1Tm7QSCZF8HDdTmRdnC+Dt+9nV1IcWrRNU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Oct 2, 2024 at 7:28=E2=80=AFPM Dan Carpenter wrote: > > On Wed, Oct 02, 2024 at 02:25:34PM +0300, Dan Carpenter wrote: > > On Wed, Oct 02, 2024 at 02:24:20PM +0300, Dan Carpenter wrote: > > > Let's add Kairui Song to the CC list. > > > > > > One simple thing is that we should add a READ_ONCE() to the compariso= n. Naresh, > > > could you test the attached diff? I don't know that it will fix it b= ut it's > > > worth checking the easy stuff first. > > > > > > > Actually that's not right. Let me write a different patch. > > Try this one. > > regards, > dan carpenter > > diff --git a/mm/list_lru.c b/mm/list_lru.c > index 79c2d21504a2..2c429578ed31 100644 > --- a/mm/list_lru.c > +++ b/mm/list_lru.c > @@ -65,6 +65,7 @@ lock_list_lru_of_memcg(struct list_lru *lru, int nid, s= truct mem_cgroup *memcg, > bool irq, bool skip_empty) > { > struct list_lru_one *l; > + long nr_items; > rcu_read_lock(); > again: > l =3D list_lru_from_memcg_idx(lru, nid, memcg_kmem_id(memcg)); > @@ -73,8 +74,9 @@ lock_list_lru_of_memcg(struct list_lru *lru, int nid, s= truct mem_cgroup *memcg, > spin_lock_irq(&l->lock); > else > spin_lock(&l->lock); > - if (likely(READ_ONCE(l->nr_items) !=3D LONG_MIN)) { > - WARN_ON(l->nr_items < 0); > + nr_items =3D READ_ONCE(l->nr_items); > + if (likely(nr_items !=3D LONG_MIN)) { > + WARN_ON(nr_items < 0); > rcu_read_unlock(); > return l; > } > Thanks. The warning is a new added sanity check, I'm not sure if this WARN_ON triggered by an existing list_lru leak or if it's a new issue. And unfortunately so far I can't reproduce it locally on my ARM machine, it should be easily reproducible according to the description. And if the WARN only triggered once, and only during boot, mayce some static data wasn't initialized correctly? Or the enablement of memcg caused some list_lru leak (mem_cgroup_from_slab_obj changed from returning NULL to returning actual memcg, so a item added to rootcg before will be attempt removed from actual memcg, seems a real race). If it's the latter case, then it's an existing issue caught by the new sanity check. The READ_ONCE patch may be worth trying, I'll also try to do more debugging on this and try to send a fix later.