From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF505CEE32D for ; Wed, 9 Oct 2024 16:51:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 51F906B00B6; Wed, 9 Oct 2024 12:51:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4CE226B00C5; Wed, 9 Oct 2024 12:51:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 351006B00C8; Wed, 9 Oct 2024 12:51:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 12F046B00B6 for ; Wed, 9 Oct 2024 12:51:40 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 05A0FC180D for ; Wed, 9 Oct 2024 16:51:37 +0000 (UTC) X-FDA: 82654655118.28.E4DFB13 Received: from mail-wm1-f48.google.com (mail-wm1-f48.google.com [209.85.128.48]) by imf08.hostedemail.com (Postfix) with ESMTP id B45ED160017 for ; Wed, 9 Oct 2024 16:51:37 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=linaro.org header.s=google header.b=DKil+6hF; spf=pass (imf08.hostedemail.com: domain of dan.carpenter@linaro.org designates 209.85.128.48 as permitted sender) smtp.mailfrom=dan.carpenter@linaro.org; dmarc=pass (policy=none) header.from=linaro.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728492516; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JcvhzK/M2OB+8jA9EG3fqKF6d49AM3B/50Z3kYSBu9Q=; b=J3OC5yZvs8rTtySjTyC3yx09Ew+2ZRmcpKnTg+FlupxKnmJZrVsMhXRFCDmdspKYLMoI68 aqqPkNHRug0A0sGbiFPFNW+cRwyFlBA2pBqI4RhOPNzw+WyycBYcm31+vURFMte77vWpgw irLcZXfRMDtdiR+o4FMxaXp51y3gQu4= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=linaro.org header.s=google header.b=DKil+6hF; spf=pass (imf08.hostedemail.com: domain of dan.carpenter@linaro.org designates 209.85.128.48 as permitted sender) smtp.mailfrom=dan.carpenter@linaro.org; dmarc=pass (policy=none) header.from=linaro.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728492516; a=rsa-sha256; cv=none; b=levNLLVSQB3Q/dTHNVFtHm4eQCW9QuMHtPx6G3Vvc3UUZGONBRI1I5iPl8TOiX5NaTsAmo Fpki0+2LpRreTg5xDF27KhwnZrX39Qfi1nrZVXGuSsNAgwFd8cg+0MLXjsXa+exxpv3Xt2 n2iRjzTXpKVumymN9G4UgW1RzmaYWTk= Received: by mail-wm1-f48.google.com with SMTP id 5b1f17b1804b1-42cbb08a1a5so71665255e9.3 for ; Wed, 09 Oct 2024 09:51:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1728492696; x=1729097496; darn=kvack.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=JcvhzK/M2OB+8jA9EG3fqKF6d49AM3B/50Z3kYSBu9Q=; b=DKil+6hFKydCyCzZpwEpykbcuqJB2vSZPYiWVidaMniVP4eIsbK+d9xM+s5QvUnlnu UzSEvxe9hc99mELMbYtjzOtkNKppMjyyvvB3+JsfUDp/WfqOxaK8dejcvtG1pabv7TJe chlAY2NnOrCt23afDWudIy5F8TMSq/LtElz+9V5G0nM3OoZ7auYOgBljySlt+e6nIxgL v+XBcLW0FfKbQ6cANPm760NG7UiFykrPs9TrRuBF2y8Q7q0PlGCrw0uRDwxyK42yTZ/0 lbc+Qwoh2GX+xwCVX9l6KVGDW4rYgGeDF7Ny6GqpmcDCmMszRhlnKvtACV9DSZxMCzK5 BLRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728492696; x=1729097496; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=JcvhzK/M2OB+8jA9EG3fqKF6d49AM3B/50Z3kYSBu9Q=; b=XZYe/KP4+yeKSpL0EA18LIWRtlPLa02aomQbQAwWd92JOUvE29DI28dTDHxIqJ+vG/ ImSljBk0pJysFt6mzy3H/JmaKJONwlkJ53SQvzsxMUtRcML/AHJTw2vy74uFjWwhRnVC GrCfNS0oY9Fmdf6mXiTajDa+H9Gr4zVso89Gsfh3KLsdhNa8gCbF7/ULNsDY6YJAwcl0 IkpQWrwPFnGeitFKRFLUjjxQGQU9oX6W+bUfUxQGGCCTe96sP6oQOyjgbEYZHm9LgZ9x KlMHCFzJ2LdhcmmEVKbiRIDzUOusYCvH4oFfxKdGhtcdPaQSGwuQsp+z8Sz1TejERGfo qJfA== X-Forwarded-Encrypted: i=1; AJvYcCXESr19gr9y2jVgqHyDyLZqceEJKq6KPegTwHOvJ1PBamgsngXMQXGX8Iu6YDD21fHc5wwxO9qfnA==@kvack.org X-Gm-Message-State: AOJu0YyUKcg3bI6TdA4XOKR2ArYCY829AbPE6n9/g4wdeXOWycEKblrk foygO2YIvLKuiZepdsQo6/f9Jz3piqARkEs/98Xj0spyUbf4RZHFfwvS69w0TxX2BfMqT4QUaf5 2 X-Google-Smtp-Source: AGHT+IF0T4FAqQj9qW87fHcjoIgPXQ2XJcxemN8K/sFZeHug+nBirxN9MNXyz33vKxSVnAXSj6ipOQ== X-Received: by 2002:a05:600c:3b1e:b0:42f:8515:e490 with SMTP id 5b1f17b1804b1-430ccf0ce1emr27481625e9.5.1728492696288; Wed, 09 Oct 2024 09:51:36 -0700 (PDT) Received: from localhost ([196.207.164.177]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-430ccf516f7sm25063995e9.21.2024.10.09.09.51.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Oct 2024 09:51:35 -0700 (PDT) Date: Wed, 9 Oct 2024 19:51:31 +0300 From: Dan Carpenter To: Kairui Song Cc: Naresh Kamboju , open list , lkft-triage@lists.linaro.org, Linux Regressions , linux-mm , Andrew Morton , Arnd Bergmann , Anders Roxell Subject: Re: next-20241001: WARNING: at mm/list_lru.c:77 list_lru_del (mm/list_lru.c:212 mm/list_lru.c:200) Message-ID: <892332fa-e1d0-4581-9c42-045660d7dc80@stanley.mountain> References: <62a65418-2393-40ec-b462-151605a5efcf@stanley.mountain> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: B45ED160017 X-Stat-Signature: dwregaqgi7imbprjg4nnypnkemnrkkti X-Rspam-User: X-HE-Tag: 1728492697-625915 X-HE-Meta: U2FsdGVkX1+Z24ZDJUboOK2UPbTsYrAJaRmeu9/R0oRG8tjpZLj53RGmuwAZ3y9FIMZUIeB/WrDltUTtc4AiTGmQikzFm44tbfrz2aldBQF7eKJo1RmyB6DQjdPoFN+GYH687pLObxsqV6RpcmEHowp5vYMJsQR904tWAyiLA4VAgaJXyL9P1QfIE8FXWDlqy3jsWprbpNBy5lGP69vkKx5huj+w3ywC7mlnRySR26HlbRj/CPQuLA+7J+8x4sJYgyw1/kdJwQ+KxVZas06k7kok3woEPitJMk/ORoalQCgi8ZayjPvYImpRKxNGvzelHm6HlTEuO+KNRl7pEdd0kLbHS6m/oP0XkJE1lhEoD8akXyDp1z8eYntTZ8qks9kP/it81i9lKN9VahSwMIwqmalHlOUsLryBZfdUsimFe2UL2QYh0GI9RXgPCereQ4nhIaxlooR/HrvrvDPwGXZwfpBpspEswLUMLzj/X54qiW6oBeVHbLZQkTksXqPMiUkosWlClvMdGVre5dcn5v/9u+j/MyjCnd9mFOAGF6IEJRoaRpPd7u9E2ffn31/JcYcAZ2IFfPyZSwUPo6JGcFJUJpP2SKpErtNhp/JfP6HbxswnhIv8nnBn7ZBl0X52hhC4dPa+UN/KpOVVfgDnweLequNpEJdvtjJe76OvcXPlszGvrVDF+H/i3R/wjscUHlpTkH/DRRwWgRxwWLDCvxoB0Z8C4lapL2d7oN/bA8AsMILrgPBDNv24xe60TO56ioxCoQIe+pb1Ghg9QceMMPc9XOt3U5F7tzBhBobyDOdNVqtnM8CjD+fFwad72cvxQdldhHg+AjyPjRKcUKOflyE/buO/yfoWv98lcLoXqp7Sn1gggOj6moGPhJ3mPlL5qNMwQjxi8kDjsZ8Dm1/QPjNwYlQsB6YIpVD1cC7hg+zSw5zm7REAL58tEKfnhKcA0C8BRPHDp9Hh0z7CrwrnYJV S/pscTUW Y6AbmF78QGbf0CKx+B7MgBubcjwzpyxinBMecB//1YEdkOkXrrgh3zyOybEKByTbTAgx3oeh+XS1J9ObBAyVEteLvOxCE0dhMoicL0mbIVAv3mnkgy4T0hTrWUYoLExHDblGTuzYzuLogYFXiNy6Tzr+URiHT+M9QO7B7sqgrUDwg9KLPIeN+kJybGLMCHOt38XvLKLj3JyTr5O/nuio6r9aecLPhVlcu4CUdMvRwPG5SBVn2mzpDDUFrSab25Ma4Ntki2HdCpFL2AuNVQZ6Bbf5lVu/a4MtOnttd4RbxwHJtrIed1tGAK8knOM8TCeRB1meWsVKb80Yadj2M2oMPJYGy7mt9rT5t9JT4dgwZV+Nzv4t4NvqsX80jbMmtwSCBJNqLysEgBJbMuZwq9/XVXyBjaA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Oct 03, 2024 at 02:58:19AM +0800, Kairui Song wrote: > On Wed, Oct 2, 2024 at 7:28 PM Dan Carpenter wrote: > > > > On Wed, Oct 02, 2024 at 02:25:34PM +0300, Dan Carpenter wrote: > > > On Wed, Oct 02, 2024 at 02:24:20PM +0300, Dan Carpenter wrote: > > > > Let's add Kairui Song to the CC list. > > > > > > > > One simple thing is that we should add a READ_ONCE() to the comparison. Naresh, > > > > could you test the attached diff? I don't know that it will fix it but it's > > > > worth checking the easy stuff first. > > > > > > > > > > Actually that's not right. Let me write a different patch. > > > > Try this one. > > > > regards, > > dan carpenter > > > > diff --git a/mm/list_lru.c b/mm/list_lru.c > > index 79c2d21504a2..2c429578ed31 100644 > > --- a/mm/list_lru.c > > +++ b/mm/list_lru.c > > @@ -65,6 +65,7 @@ lock_list_lru_of_memcg(struct list_lru *lru, int nid, struct mem_cgroup *memcg, > > bool irq, bool skip_empty) > > { > > struct list_lru_one *l; > > + long nr_items; > > rcu_read_lock(); > > again: > > l = list_lru_from_memcg_idx(lru, nid, memcg_kmem_id(memcg)); > > @@ -73,8 +74,9 @@ lock_list_lru_of_memcg(struct list_lru *lru, int nid, struct mem_cgroup *memcg, > > spin_lock_irq(&l->lock); > > else > > spin_lock(&l->lock); > > - if (likely(READ_ONCE(l->nr_items) != LONG_MIN)) { > > - WARN_ON(l->nr_items < 0); > > + nr_items = READ_ONCE(l->nr_items); > > + if (likely(nr_items != LONG_MIN)) { > > + WARN_ON(nr_items < 0); > > rcu_read_unlock(); > > return l; > > } > > > > Thanks. The warning is a new added sanity check, I'm not sure if this > WARN_ON triggered by an existing list_lru leak or if it's a new issue. > > And unfortunately so far I can't reproduce it locally on my ARM > machine, it should be easily reproducible according to the > description. And if the WARN only triggered once, and only during > boot, mayce some static data wasn't initialized correctly? I have a config where it printed twice and the second time wasn't during boot. https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20241009/testrun/25363339/suite/boot/test/gcc-13-lkftconfig-rcutorture/log > Or the enablement of memcg caused some list_lru leak > (mem_cgroup_from_slab_obj changed from returning NULL to returning > actual memcg, so a item added to rootcg before will be attempt removed > from actual memcg, seems a real race). If it's the latter case, then > it's an existing issue caught by the new sanity check. > > The READ_ONCE patch may be worth trying, I'll also try to do more > debugging on this and try to send a fix later. The READ_ONCE() patch *seemed* to work, but the bug is intermittent so maybe it just changed the timing or something. Still, I feel from a correctness perspective the READ_ONCE() thing is probably correct, right? regards, dan carpenter