From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85769E77180 for ; Sat, 14 Dec 2024 19:43:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C98B96B007B; Sat, 14 Dec 2024 14:43:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C46A56B0083; Sat, 14 Dec 2024 14:43:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B0E886B0085; Sat, 14 Dec 2024 14:43:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 96F4E6B007B for ; Sat, 14 Dec 2024 14:43:42 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id E555EA276C for ; Sat, 14 Dec 2024 19:43:41 +0000 (UTC) X-FDA: 82894588350.27.64B799C Received: from mail-lj1-f177.google.com (mail-lj1-f177.google.com [209.85.208.177]) by imf16.hostedemail.com (Postfix) with ESMTP id 3DC6C180006 for ; Sat, 14 Dec 2024 19:43:11 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=VnXsK6hM; spf=pass (imf16.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.208.177 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734205391; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ja0iHQq4x8YYrPwua2vK6bgvK8Ftnd8EXGgl2zucixM=; b=DXjKWm/GkO7B6wMcvXXdP5IBbdAGWcbaJkSFVt+j7uFtOYmD/HgoNVV2vJ1oMdPNG+if5I aTL8BRhuDDHZ8FwOGUog856bx5PgjDFg2nxv3qpHIYkt/grxmMmC7BR69c/t7vThQkVgVx j0w3ITppXD4xJK9tGtjiiY4vVqVTLk4= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=VnXsK6hM; spf=pass (imf16.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.208.177 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734205391; a=rsa-sha256; cv=none; b=vdyHL7+wsGlPqTYw0M88REgt7z06DSGnCuB7rG4JI7LCrWPk3oR1V/CGcUVRRf0WeOaMSI 2B/cIiClM1QXxKQD85uPgVLhEGV8USrunm2i9raPhZI7Gr9UfqjBNyN6p9+BTDdd6P7c7e CJjU6yqoF4gD85ji9/SmVadh7THCtZ0= Received: by mail-lj1-f177.google.com with SMTP id 38308e7fff4ca-2ffd6af012eso32129191fa.2 for ; Sat, 14 Dec 2024 11:43:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734205418; x=1734810218; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=ja0iHQq4x8YYrPwua2vK6bgvK8Ftnd8EXGgl2zucixM=; b=VnXsK6hMZL/QAJuMJoUcLM21uXlahM+tEylLlRFX8i7agRAG7hJBgQqpjdkqg9ChSr f+YJfw2G7EhYEPgCxZPcXyaaAjFQFVNmBtaHbNa1nV2HR1qZLeXGiQbkoepOp7ZPN5am sOIx/37CzYndfof/S4AqlZE7SlT45LWhD/n27er3W7j1yRwZ1BHMkH3QqIrvVPE+npLU G0FuY4z9aQYI6ryaYGAHNjEfEHyFjCWBONBl1kv4+wbPpuGUZ6XnhatoVsG7B+jJM5Ai ooI//adneee9T2kADdiqcsp06paO1TUFCLhMXiW4dWlhc3nnqHrxyEPG2UBl6pJCUvtq 48dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734205418; x=1734810218; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ja0iHQq4x8YYrPwua2vK6bgvK8Ftnd8EXGgl2zucixM=; b=CP7t91sh81cIKEnaDtvwqWzvz/iOu3gENLMEEIu8OGSSRt3Xp+YhcAf+nSp9pP4sHX 2jsXbZV9rvuhw+OmRyokCqkgrxu/SuvE2cPHP2CkkBYDzg9/BMolbRnByBeDDqVKBr7j 6n6iSBBSHUpdqiCdPGP9wNdQn8VDJse72BTFU3mYjxpxUixCU5rcHs82SrM6KMLsZ1bG 38g1xfwTAWcn3WtbFft/zqo854Nbjhyp16/ffSV6Zcu8MF6SbiL21Cvy1dRD+JdFQIma xVIiq0AdlE8s2b3ld55i2zRSpRe5sfnRGLgtXBLmPX90sqhNdz09Xx7b/fsZY/ahZqaU NSnw== X-Forwarded-Encrypted: i=1; AJvYcCXgE0uzHQmrUugp/wO1fOMWoziMKUsClRubVEaG/Fi3m0darvXXr1dzuckuOWRSkbMqyXiihTmkFA==@kvack.org X-Gm-Message-State: AOJu0YylMeQ0ft0MzBYhDH9e5f28mnxz9qnJl681KwR69/LoH7pDaIdt kpM+fH9634mdECzvNsszSsW35QRo2W8lXQ4kiIIfh2IGhR/2S33mwAKr2UElB1JcK/yCiL/Vvdb hKR2pPSWbxodJuk1lMsK3He5UFr4= X-Gm-Gg: ASbGncvGhGj86aulj50kCHEsEIwyLHpwlIWcRDDcX/KF6cbgMsi63knhBDVLl3WAMnP HBzJT7T2xqv2swGctB9pdX6arzuq+b0w5/4N2qA== X-Google-Smtp-Source: AGHT+IHvkdVjNXVm2If40s6Ce3AhrSKz3rZaHoNknyHsTGxXZFzuIka28+JyOehwjs0+jgrJ44TJCiytBItrbC91+AM= X-Received: by 2002:a05:651c:2116:b0:2ff:a89b:4210 with SMTP id 38308e7fff4ca-302544264d6mr25787961fa.8.1734205417785; Sat, 14 Dec 2024 11:43:37 -0800 (PST) MIME-Version: 1.0 References: <675d01e9.050a0220.37aaf.00be.GAE@google.com> In-Reply-To: From: Kairui Song Date: Sun, 15 Dec 2024 03:43:21 +0800 Message-ID: Subject: Re: [syzbot] [mm?] WARNING in lock_list_lru_of_memcg To: Yu Zhao Cc: syzbot , akpm@linux-foundation.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, syzkaller-bugs@googlegroups.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 3DC6C180006 X-Rspam-User: X-Stat-Signature: hx3nbi9h7fnnz15igt4d7thfff8qay5x X-HE-Tag: 1734205391-169861 X-HE-Meta: U2FsdGVkX1+33/wfGyRseaEnNaVpIanI7rGUXyeOjab1DuJVpcvhkCsRr7qX/hvroJedv4apfbbKnmi58pcYhJ2I/Hus/dgtelzCO7rPfb8GtklE1bKONpdSOKTz5iDva8rUAILrSLvhWIw4mVSifNiaF2Rz3s7Xi7PRqQqLJTSt09mC7NWPslp+5GvIjC2tybvkjrnZUiyLvdJNrzXNoMhhF+QX4jVMxoM3YFWB8QFyPtr1F+EzFPpY+uQwnPdzCW8h0/pXaqPPQspisQf6L9jDJHp6iH0+0oer5ef3Zlc99YMGgZ5Fu0k5JHn+6Mj4Z90nX0OzhlchbztjVyYcC+9lqjsWFY4PDdhuT0wAGHJ3WUGKbqekQIIk8kpt0Q5JG2F3RoLc1FXvUbMCJ9exZGuHeFGkh42JsKtCtgEo45Cg6Xs6KXjjUqcUYJvJEgHGq5UCDAPnrpTOT/AZjESBFjwefKDu9XL4Rvsu0vRIdQGcn3Rh9PlFNwGjYXhHGyN2lQI/24AWfJMktMAX4lptFQh00YAwq4pZKWhgSjrPWnG2eElE3LHI5Xt9zlGY+XRIsu9qtPWkrh0w5UiCA4b6+lV9U9CZLBKuf0R4Meb209TbpyMjvqjXW9pxTJ3OpUVi8emO94zkeeeUSXF8JANkkLGobfMhDOeNoR3twbS0D1UsiswFlwuwyKEtTpuV/Y9zzSzB7PIdjoW8Iuw3N9zPywU45nrPuBhiazMFrKLB8h8fXMp5QhHqH9rysqRzUyyvBJiHDJ+48h6Lb6SaUrvJF237hDLo86X6zSuNA4MYFT47JjHIG9JI6rnBG1Fg/yCmJO5ao5LCuT8OyVB4Nh3tl7xJaGyP0+cp2O0K9UWF+Skm1JNoG6Sp8PjdGLJaH6HdcEztp40fte8PG++tpWoPquNCHd0WpO+wXfHl7KiIlKMd1oDm5Yuv/HjGkgwHYpW4Ps76SpxURQKUTuZLpuy pSp6WYMm AkLJCL32Qqr8F2dT2bPrQeBQefcUk84qZfnJhGEueaCqlLJIjiZCtwCrRYwlRX6F5Zj/SLnV0FAl2SjiqoUmCss1mp4+p8FbOQAyzHgpztGRPbcq3woEBOqw8PpqBaCxQIF4inful7nivmcMmEVCjDjfUONQHjKfhe65pnqNDYaRKV28K/QnDSnUDsCJTCl3Dsf6HPjO/wHvkUOEE3/y9jQPbXrDTEXaOgK94nyjJa4IbzoErPcJ+H6VMCbmBwEW/xSbvYuQAR3cD3PqIeP05VOJao6A0zWPKkoeVU5a7n96CMEwp2KSqOqIEllQ3wbmFXPN1V6fQBl/cPcdAKbqnpJEpBCrxOML1bycWE4ZWpwcOuZrvwJwfY4SG3nElnAZkAX9nG+H+poyrYGjPcdq98Tls2WzU1JJF9KOgsGRPKrlkv1tmtUpcDItN8Li8Uk9zrnaJ6GvFHS+euESyNiB/4Ewa1bpptEiBusIkt8+1sTpDJLFOrNQg7lvzu7ZwshBFAEuQczluR6V0LHCW4ndmkmCirzSwm2oFG+CiOXHGa8onJgWfBEoOJYeYkRG9qwPu7m7em6EmaFOGAnZ9PveRB8+h6mphhhgcvvdkbEJEa3CvFxtxhtYxBNpxNStDKxf1fkbocEcurS2pRIeDzQuHGndWeeJdgrSzUSQWo1hvOy8rRS2ri4pwjnct00jdhwJAF5D9ChQK8Z2LSEb669U+svDjIuQup9uorpVkPqMJkpF3HwOImGyI3PXMC6+ItyCzzzHf06mnTKpFglFxSj9JK4CgikjSIUDa+CNanSa1PWcJqj4raOjML/QQONwq97wTW9PzYlidDUVhMClclGYahyqDsmZRG+aKx0+PI+/A7qknVdHxlT9KRRuq5PNI/7qZbw7lqv7bU3r4Cx53yfDRWbMz84ptzrS6UO87q3mNhYC8m1hCo0daKfsF2w5S8OzhXGaC2UrI1mWmQ4fMlbh2sTc2TDnD wty8qBHG ZvR5BFgZ7wXm9ESG9CFbug== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Sat, Dec 14, 2024 at 2:06=E2=80=AFPM Yu Zhao wrote: > > On Fri, Dec 13, 2024 at 8:56=E2=80=AFPM syzbot > wrote: > > > > Hello, > > > > syzbot found the following issue on: > > > > HEAD commit: 7cb1b4663150 Merge tag 'locking_urgent_for_v6.13_rc3' o= f g.. > > git tree: upstream > > console output: https://syzkaller.appspot.com/x/log.txt?x=3D16e96b30580= 000 > > kernel config: https://syzkaller.appspot.com/x/.config?x=3Dfee25f93665= c89ac > > dashboard link: https://syzkaller.appspot.com/bug?extid=3D38a0cbd267eff= 2d286ff > > compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for D= ebian) 2.40 > > > > Unfortunately, I don't have any reproducer for this issue yet. > > > > Downloadable assets: > > disk image (non-bootable): https://storage.googleapis.com/syzbot-assets= /7feb34a89c2a/non_bootable_disk-7cb1b466.raw.xz > > vmlinux: https://storage.googleapis.com/syzbot-assets/13e083329dab/vmli= nux-7cb1b466.xz > > kernel image: https://storage.googleapis.com/syzbot-assets/fe3847d08513= /bzImage-7cb1b466.xz > > > > IMPORTANT: if you fix the issue, please add the following tag to the co= mmit: > > Reported-by: syzbot+38a0cbd267eff2d286ff@syzkaller.appspotmail.com > > > > ------------[ cut here ]------------ > > WARNING: CPU: 0 PID: 80 at mm/list_lru.c:97 lock_list_lru_of_memcg+0x39= 5/0x4e0 mm/list_lru.c:97 > > Modules linked in: > > CPU: 0 UID: 0 PID: 80 Comm: kswapd0 Not tainted 6.13.0-rc2-syzkaller-00= 018-g7cb1b4663150 #0 > > Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.16.3-debian-= 1.16.3-2~bpo12+1 04/01/2014 > > RIP: 0010:lock_list_lru_of_memcg+0x395/0x4e0 mm/list_lru.c:97 > > Code: e9 22 fe ff ff e8 9b cc b6 ff 4c 8b 7c 24 10 45 84 f6 0f 84 40 ff= ff ff e9 37 01 00 00 e8 83 cc b6 ff eb 05 e8 7c cc b6 ff 90 <0f> 0b 90 eb = 97 89 e9 80 e1 07 80 c1 03 38 c1 0f 8c 7a fd ff ff 48 > > RSP: 0018:ffffc9000105e798 EFLAGS: 00010093 > > RAX: ffffffff81e891c4 RBX: 0000000000000000 RCX: ffff88801f53a440 > > RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 > > RBP: ffff888042e70054 R08: ffffffff81e89156 R09: 1ffffffff2032cae > > R10: dffffc0000000000 R11: fffffbfff2032caf R12: ffffffff81e88e5e > > R13: ffffffff9a3feb20 R14: 0000000000000000 R15: ffff888042e70000 > > FS: 0000000000000000(0000) GS:ffff88801fc00000(0000) knlGS:00000000000= 00000 > > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > > CR2: 0000000020161000 CR3: 0000000032d12000 CR4: 0000000000352ef0 > > DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 > > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 > > Call Trace: > > > > list_lru_add+0x59/0x270 mm/list_lru.c:164 > > list_lru_add_obj+0x17b/0x250 mm/list_lru.c:187 > > workingset_update_node+0x1af/0x230 mm/workingset.c:634 > > xas_update lib/xarray.c:355 [inline] > > update_node lib/xarray.c:758 [inline] > > xas_store+0xb8f/0x1890 lib/xarray.c:845 > > page_cache_delete mm/filemap.c:149 [inline] > > __filemap_remove_folio+0x4e9/0x670 mm/filemap.c:232 > > __remove_mapping+0x86f/0xad0 mm/vmscan.c:791 > > shrink_folio_list+0x30a6/0x5ca0 mm/vmscan.c:1467 > > evict_folios+0x3c86/0x5800 mm/vmscan.c:4593 > > try_to_shrink_lruvec+0x9a6/0xc70 mm/vmscan.c:4789 > > shrink_one+0x3b9/0x850 mm/vmscan.c:4834 > > shrink_many mm/vmscan.c:4897 [inline] > > lru_gen_shrink_node mm/vmscan.c:4975 [inline] > > shrink_node+0x37c5/0x3e50 mm/vmscan.c:5956 > > kswapd_shrink_node mm/vmscan.c:6785 [inline] > > balance_pgdat mm/vmscan.c:6977 [inline] > > kswapd+0x1ca9/0x36f0 mm/vmscan.c:7246 > > kthread+0x2f0/0x390 kernel/kthread.c:389 > > ret_from_fork+0x4b/0x80 arch/x86/kernel/process.c:147 > > ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:244 > > > > This one seems to be related to "mm/list_lru: split the lock to > per-cgroup scope". > > Kairui, can you please take a look? Thanks. Thanks for pinging, yes that's a new sanity check added by me. Which is supposed to mean, a list_lru is being reparented while the memcg it belongs to isn't dying. More concretely, list_lru is marked dead by memcg_offline_kmem -> memcg_reparent_list_lrus, if the function is called for one memcg, but now the memcg is not dying, this WARN triggers. I'm not sure how this is caused. One possibility is if alloc_shrinker_info() in mem_cgroup_css_online failed, then memcg_offline_kmem is called early? Doesn't seem to fit this case though.. Or maybe just sync issues with the memcg dying flag so the user saw the list_lru dying before seeing memcg dying? The object might be leaked to the parent cgroup, seems not too terrible though. I'm not sure how to reproduce this. I will keep looking.