From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D78A6C4345F for ; Wed, 17 Apr 2024 00:22:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0CF106B0083; Tue, 16 Apr 2024 20:22:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 07F916B0085; Tue, 16 Apr 2024 20:22:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E88F66B0087; Tue, 16 Apr 2024 20:22:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id C9B506B0083 for ; Tue, 16 Apr 2024 20:22:57 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 8F31B80C71 for ; Wed, 17 Apr 2024 00:22:57 +0000 (UTC) X-FDA: 82017123594.22.DB953BF Received: from mail-oo1-f46.google.com (mail-oo1-f46.google.com [209.85.161.46]) by imf30.hostedemail.com (Postfix) with ESMTP id C633E8000A for ; Wed, 17 Apr 2024 00:22:55 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=P1jTMwyz; spf=pass (imf30.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.161.46 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1713313375; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=aP26FrywztgVkfzE2tfyTzU9hcB6QnE7fDvry2nRPDE=; b=vopawyG8dZZWwMuFEtFjjpeq+pX6ebIbiJQH5pstBP6vjwyAb1xQnz9NSl3C7LUzcDhZoS IfuRxybI28v9CCqcq4GB/JB0eLmFw6gxijHcUKuMs2jWWbQmdL+cjeQsa7Uvc9I1Mi8tWr L0PvpFBRQtdbZ6xfNFQBTeb5QRgqP80= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1713313375; a=rsa-sha256; cv=none; b=yNED9IBGF88qSXne5McmncKYJiJ2+ntHNLuHN5OqWPsU6+OPs7C5rPGpa6Oe1crRkMkOCa O9CPQ3bzQKWUNNJD2MxdRHJ9A64g6XSKnpKtACQJiEmQIKbFI32c4xvq2eiuhS0q9JFFBH dDHd6GwddY4R0E94junC0w76wZOB8jU= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=P1jTMwyz; spf=pass (imf30.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.161.46 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-oo1-f46.google.com with SMTP id 006d021491bc7-5aa28cde736so2973929eaf.1 for ; Tue, 16 Apr 2024 17:22:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1713313375; x=1713918175; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=aP26FrywztgVkfzE2tfyTzU9hcB6QnE7fDvry2nRPDE=; b=P1jTMwyzXuf12fFZun+yqq5yY8JdJz6YewPTgxWIfB0ot5I1XfUGc9CeZfA/8Dsl5P Qy36KtomePKdbMf7atdkjtHz61mMLOn0rNSzKLzsXb1va3/wMViDkyfX+dllju8ljXHw OaIxP4w24TE+vIjErW3NMtOs7tHwsOeT60Asm2PngoS8/p6N+inzu5yGzDiSwzY6xhFB 7yRdJFCKkD1zcL+pcYkDjgK+ThSvEmuFLxdwxMSob2nFMKIY+zrIr2oXh+T1/vUUcsa5 rMo6dmgIY8ilo78rQomN1T/j2KI1d/Q7HiTis/30Y3JiDMFMxKXPiTfDg6or+wb8irsN Daow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713313375; x=1713918175; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aP26FrywztgVkfzE2tfyTzU9hcB6QnE7fDvry2nRPDE=; b=YWd+X27U04l21OIG4CP/QUcLnMhjpFucYvEAK7D8QOd6AYf+qDH34jCyCOF3a6jjOC GZ64Mu71toNSTc1sFRejL0WFFGUg2G5u2c0oXn0fsZIVTmh1TgDdHjy9ZVgXn3YFo9M8 z0bWGNVteyuDZazGrRprXtM8wuaVE4V1P7jSZM5Gs71rXBSMDuu7K/T4gjSSs/Rs41HX p0b1tNKu1wl5YK0qzMhxLoCmZ4mIAEOCoRVMwyPIBWQEZJSe3URwLN/gmER9Aqia53Qf AmNE2R9xAfUO72bj0MycVVAc7QH6GUIlBEEf5D1CmWVa4i7P8dGoyf8VYnM7ziY7fLvq LjoA== X-Forwarded-Encrypted: i=1; AJvYcCXuM3wZ667C7SbAwzWV85WM2Pe77DLXxN/kIkH9Q0EAkUF6/HPaMAjd/XF66UUtJUYxqcTXrf/GZa25lwnohWkR/YQ= X-Gm-Message-State: AOJu0Ywt4MUGS01PuukojB8s8w+080rzUEb3zy8/+FFxGo2PujRvfbqZ mw5YRGgEy/Mau6IzRiFGOvkH4tWLqh9gwPG3ElkH22xv/5VVXT617KQBw7vfKUnD/8XC+1S+wRs rHCAg9pJEKhpepsVX+p3NrWVhAEk= X-Google-Smtp-Source: AGHT+IFn4iXE20yPNxhNmTGeEwmEqivQnTCqqF+lbczSroqVrG41U3nbTCfc9BmBjSQtefmnE+r4+EIVmpDdOA47A9k= X-Received: by 2002:a05:6358:690d:b0:17f:5797:b0ee with SMTP id d13-20020a056358690d00b0017f5797b0eemr18856500rwh.10.1713313374643; Tue, 16 Apr 2024 17:22:54 -0700 (PDT) MIME-Version: 1.0 References: <3iccc6vjl5gminut3lvpl4va2lbnsgku5ei2d7ylftoofy3n2v@gcfdvtsq6dx2> In-Reply-To: From: Nhat Pham Date: Tue, 16 Apr 2024 17:22:43 -0700 Message-ID: Subject: Re: [REGRESSION] Null pointer dereference while shrinking zswap To: Christian Heusel Cc: Seth Jennings , Dan Streetman , Vitaly Wool , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, David Runge , "Richard W.M. Jones" , Mark W , regressions@lists.linux.dev, Johannes Weiner , Yosry Ahmed , Chengming Zhou Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: wh13rimadso81catzcxiadkoit5e5wcf X-Rspamd-Queue-Id: C633E8000A X-Rspamd-Server: rspam06 X-Rspam-User: X-HE-Tag: 1713313375-993784 X-HE-Meta: U2FsdGVkX188GL9nSBfYjRx6Uhcfn5nQwdeC+dYDGJMTI2evDIypEl+6XOP39W71Y7NZ81Lo2Wrco8PuNuQV9aUGtGFiZ/cBcEAeeHtgnnKStEGZewlaIVyAu3kKgOeLsHmOp5KAdBjt/7peN9iUQQSWlDpcJBqcekntVekmwrfZ7xDS9pgEhxwF3KC9uoTIK36eoAri/eta9bZV46wHG4nujU8lWBOSdInalanN9/WHPg6SoFIjb3+fcN+KD1AghiFtgdc0XYpMcMeeZZ72Pxr4L2NWzFpwCxiv30I/pRoY3xZaXocVxKcEWOz8tPf3hdO9/fWF+edRQtAVuoBtF4oJv7kmWHtXFTD5na9JU+AYYF+z7Og5jTZB3KbjRalC2J7P8n7drWK0SIJuDwMkhOq1iaUO8FafvJ1JfbyXTMRxp5+AUuTDyCHPwhF9vvrIZB48LEDaa/nbUPA5oC2PpGg+mcrcGdbqV+YHOG26MPLqCAGOk1wQdWDNsl9gMDY0f7U5Bj1X6PPqBvpTkkvR7+MZ4e/wB3NfPCUDwsWhds3DrMt44BUDyWZMdTKuS0INJoKKpgGNPI4o3+h4MpvD7pNe8D6FiDBXzC8nYjaTFCLYPFX4m8LX2jVGO2a0+WyJR7dVLGv4p+/iCiYAFOKiy+k3INyVefFQQopFZRv5RqHttC+Ot/jCfbVci4vQ6G05IjY/QyriLE2NPfLFalvRLCLdtIaRo2gymhfNoR9ZQ6p2kG+CdREqB+8JsPQv4yGsDgBKk9qb3RlWFHAabzuEOL6DpAI/XF+utCh0bNic75TjlqgH0apEaEaGtInOYQxosZ75dLox/rTv4ssTVUhifqyWwGSuuxhzD8I40FpL2vMLpN3C1PNkxHeyMpHzkmGB9l1Lf6sLKf1Krlh4gA8ZfEjkMXo+wOr3HF13ZnVTXU3MvwM2YOOS/6jzNEAouwpFX8f8Xf/Z0GVfmPh5Ks4 Xn9eSirB gsz+5Lrag7U5k+mAle3Et73sGeD6TMA/WWJQawZ9YxiJwljAS9nAilpLt0COzqTiJUgvNMCSAJsv7gvCUjuogF2Z8IL0CE2A3KuMICDdf2Ki02qZIVavBeL74XC6L0t93+9HS5SyVd8YM5ttxWtMgB8cQoOhdUXOFzp6EySGXVhSEd4YOMTn5LkgIesgAHFKeWZvXSSQScBAE2cfBG3cmG4CEvu5QA7Llz49iLYQzJkteJBvY61WCSx+IcVUyAOATsc0TvxaRrghT5dEsw4FbBWbbqnXBgddMRI/U5iiqLbnXk0Rv59MOrF6GCYcV3rLpI+5Ip1IXLTwQPsElJ0pLcb313+6Iep+pQO2X+/bppL6Hn+SSb9lM0h8To4Nco2Pix2MblIkBmEpa52HSsZMHFlHo3bol+H9F/Nn2Lmk9HrKeMgTL/VFh/04Ylg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Apr 16, 2024 at 4:29=E2=80=AFPM Nhat Pham wrote= : > > On Tue, Apr 16, 2024 at 3:14=E2=80=AFPM Nhat Pham wro= te: > > > > On Tue, Apr 16, 2024 at 5:19=E2=80=AFAM Christian Heusel wrote: > > > > > > Hello everyone, > > > > Thanks for the report, Christian! Looking at it now. > > > > > > > > while rebuilding a few packages in Arch Linux we have recently come > > > across a regression in the linux kernel which was made visible by a t= est > > > failure in libguestfs[0], where the booted kernel showed a Call Trace > > > like the following one: > > > > > > [ 218.738568] CPU: 0 PID: 167 Comm: guestfsd Not tainted 6.7.0-rc4-1= -mainline-00158-gb5ba474f3f51 #1 bf39861cf50acae7a79c534e25532f28afe4e593^M > > > > Is this one of the kernel versions that was broken? That looks a bit > > odd, as zswap shrinker landed on 6.8... > > Ah ignore this - I understand the versioning now... > > > > > > [ 218.739007] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BI= OS Arch Linux 1.16.3-1-1 04/01/2014^M > > > [ 218.739787] RIP: 0010:memcg_page_state+0x9/0x30^M > > > [ 218.740299] Code: 0d b8 ff ff 66 66 2e 0f 1f 84 00 00 00 00 00 66 = 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 66 0f 1f 00 0f 1f 44 00 = 00 <48> 8b 87 00 06 00 00 48 63 f6 31 d2 48 8b 04 f0 48 85 c0 48 0f 48^M > > > [ 218.740727] RSP: 0018:ffffb5fa808dfc10 EFLAGS: 00000202^M > > > [ 218.740862] RAX: 0000000000000000 RBX: ffffb5fa808dfce0 RCX: 00000= 00000000002^M > > > [ 218.741016] RDX: 0000000000000001 RSI: 0000000000000033 RDI: 00000= 00000000000^M > > > [ 218.741168] RBP: 0000000000000000 R08: ffff976681ff8000 R09: 00000= 00000000000^M > > > [ 218.741322] R10: 0000000000000001 R11: ffff9766833f9d00 R12: ffff9= 766ffffe780^M > > > [ 218.742167] R13: 0000000000000000 R14: ffff976680cc1800 R15: ffff9= 76682204d80^M > > > [ 218.742376] FS: 00007f1479d9f540(0000) GS:ffff9766fbc00000(0000) = knlGS:0000000000000000^M > > > [ 218.742569] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033^M > > > [ 218.743256] CR2: 0000000000000600 CR3: 0000000103606000 CR4: 00000= 00000750ef0^M > > > [ 218.743494] PKRU: 55555554^M > > > [ 218.743593] Call Trace:^M > > > [ 218.743733] ^M > > > [ 218.743847] ? __die+0x23/0x70^M > > > [ 218.743957] ? page_fault_oops+0x171/0x4e0^M > > > [ 218.744056] ? free_unref_page+0xf6/0x180^M > > > [ 218.744458] ? exc_page_fault+0x7f/0x180^M > > > [ 218.744551] ? asm_exc_page_fault+0x26/0x30^M > > > [ 218.744684] ? memcg_page_state+0x9/0x30^M > > > [ 218.744779] zswap_shrinker_count+0x9d/0x110^M > > > [ 218.744896] do_shrink_slab+0x3a/0x360^M > > > [ 218.744990] shrink_slab+0xc7/0x3c0^M > > > [ 218.745609] drop_slab+0x85/0x140^M > > > [ 218.745691] drop_caches_sysctl_handler+0x7e/0xd0^M > > > [ 218.745799] proc_sys_call_handler+0x1c0/0x2e0^M > > > [ 218.745912] vfs_write+0x23d/0x400^M > > > [ 218.745998] ksys_write+0x6f/0xf0^M > > > [ 218.746080] do_syscall_64+0x64/0xe0^M > > > [ 218.746169] ? exit_to_user_mode_prepare+0x132/0x1f0^M > > > [ 218.746873] entry_SYSCALL_64_after_hwframe+0x6e/0x76^M > > > Actually, inspecting the code a bit more - can memcg be null here? Specifically, if mem_cgroup_disabled() is true, can we see null memcg here? Looks like in this case, mem_cgroup_iter() can return null, and the first iteration of drop_slab_node() can have null memcg if it's returned by mem_cgroup_iter(). shrink_slab() will still proceed and call do_shrink_slab() if the memcg is null - provided that mem_cgroup_disabled() holds: if (!mem_cgroup_disabled() && !mem_cgroup_is_root(memcg)) return shrink_slab_memcg(gfp_mask, nid, memcg, priority); Inside zswap_shrink_count(), all the memcg accessors in this area seem to always check for null memcg (mem_cgroup_lruvec, mem_cgroup_zswap_writeback_enabled), *except* memcg_page_state, which is the one line that fail. If this is all to it, we can simply add a null check or mem_cgroup_disabled() check here, and use pool stats instead?