From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93BD0C4345F for ; Tue, 16 Apr 2024 19:18:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EE3E16B0085; Tue, 16 Apr 2024 15:18:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E95496B0093; Tue, 16 Apr 2024 15:18:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D5C836B0096; Tue, 16 Apr 2024 15:18:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id B7CDC6B0093 for ; Tue, 16 Apr 2024 15:18:08 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 44149A10D3 for ; Tue, 16 Apr 2024 19:18:08 +0000 (UTC) X-FDA: 82016355456.26.3A1A251 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf07.hostedemail.com (Postfix) with ESMTP id AD12840008 for ; Tue, 16 Apr 2024 19:18:06 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=pVdheGOo; dmarc=none; spf=pass (imf07.hostedemail.com: domain of akpm@linux-foundation.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1713295086; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=r2yqJs96/D/EMCla1VROK1JGhtXnZWloqrIC5NeYkdc=; b=vazgAzieQskfMQGxsYvLXwYskYo6bxBqzNiX21lSacnUcYNbJ/oHiOFN3pdAaJb9pX3I/8 WpDZuXDoUmPrg4kPF9XDJrYwF0GjMuABu7mdN2UCXZgN00qNBc2VzRjDWUBMiMki4e9ah9 1MgCC6mgP5V3dSZtRLdm3KXt5V7msJM= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=pVdheGOo; dmarc=none; spf=pass (imf07.hostedemail.com: domain of akpm@linux-foundation.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1713295086; a=rsa-sha256; cv=none; b=Na1okbVDLS2hjNgOukvzdtFon6zwRMY5K5kQE4AcRV4Ui9N2zKHMy1vcu8q2rpwTI3zCWD ws0GKNZytW8vXS4fh1V+enoNPxI4tr/EGrKPG/THcapd22kR5KQZXBN2EMw22n7xrRB+b4 MoZ01fz3ywbRaYcMG9WkXxzLEVwHKwo= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 89F436124B; Tue, 16 Apr 2024 19:18:05 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B7D88C113CE; Tue, 16 Apr 2024 19:18:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1713295085; bh=qe83P9vvTKzKa5UvpEdn6fNXNpGNSmqGdNylsccrOPg=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=pVdheGOoICvekvJn1ImFKQUiCCrvaWXTHv0NGii/S5H8qa4b/gMw7RUsRncLQGheH QoQw8hNH+BtYx0X0PSDPk3yfINo8dnrUVtBOdlff6gODI3AzNz/FgYJ1jN9hI5cLh5 oR3RfafPTtgJoGL6I1VHhoa15sMYx824E9SFr3yk= Date: Tue, 16 Apr 2024 12:18:04 -0700 From: Andrew Morton To: Christian Heusel Cc: Seth Jennings , Dan Streetman , Vitaly Wool , linux-mm@kvack.org, linux-kernel@vger.kernel.org, David Runge , "Richard W.M. Jones" , Mark W , regressions@lists.linux.dev, Johannes Weiner , Yosry Ahmed , Nhat Pham , Chengming Zhou Subject: Re: [REGRESSION] Null pointer dereference while shrinking zswap Message-Id: <20240416121804.a46c4174f6cec30ab51f18e5@linux-foundation.org> In-Reply-To: <3iccc6vjl5gminut3lvpl4va2lbnsgku5ei2d7ylftoofy3n2v@gcfdvtsq6dx2> References: <3iccc6vjl5gminut3lvpl4va2lbnsgku5ei2d7ylftoofy3n2v@gcfdvtsq6dx2> X-Mailer: Sylpheed 3.8.0beta1 (GTK+ 2.24.33; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: AD12840008 X-Stat-Signature: 4mawqmcb8kqri4sefu4uqesu48bspc6r X-HE-Tag: 1713295086-880854 X-HE-Meta: U2FsdGVkX1+ZqOwcV63O3b3XfdVA62qiLbCC0Rnsgf2TI7oyr1LTNAzdBi9GJyCsDe1lZylUPoxQi0zSDyyGyf4T7qXldDb2biriMyZ2qSPwSrSdnuFexhxAyZ3RZNVrzESpRSld1p05bfVud+t5OENiIgFaacqW9q0AAWbiWFkUS2QrSHzX/gb2rapnnd4nRKugCvVDxtX6wRplwF+uhKg+emQzlkGFWURqabsgkfTB5gz7hkUGhY09zVvTK9/lyoo5b+v5VA3ud+gV4I2DDo+DrHETugsymj5pMudSTwo8MPf1MeQe9akeutCE00TCfDCElYOwHLNONxKQBmknep6VG3k+NOs1rQnK47Iyb8DNBi2fMecMwEkCUCby7Me3KfEyNNNM2wljofPU7H4KSuq6H8eG8gRxuSAp/amw4ZwvIrbAwckPjlmxHyo3k9RtBzi3MJdnDQrhD90N2K4Q9xC3BKrjAt5ksiTInt8OxBY9E0n2p/USYFzWdJOBRpaoP9Ok+5my6yfOgNBwNx7whohZW32g2qZT2j4/eLixSt+jZxTZwWR4QQU7MoplOcbByXWAgQbpcvMPNOLv14ZrhNrebcs+L37XHGBTXd12sUcGGZ/KYFTpZn3kSmfzKV9doxLcvTyO4L/Ms8D+fidVcUj2xdWhQsMN3FFePOpmurwsYcgsDRl9bYYLfAOpGm/x1rVsWEHMh+lfJcPvczL/C/I+gVdz9Ng+C2ymqbFSSISCkbAJNpD3jSR+3bW7EYaJGrj96/F8E8CZMMLgMSOJqoHP551jBWQ4XYXyneWETe8RoCbdFj+cCcImRhAPqXkOlRGGQ3BC8T0gpiFcKbl6+HHwtewpK3jCMtbDjT6FQ7WbEeZ7Yp9X+qFUIls2pgaLJeNC0an6bHjxvZeLMWwGOIgLgaTNpOGgqLb1wDMaeAysHHZO0HJWKw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, 16 Apr 2024 14:19:24 +0200 Christian Heusel wrote: > Hello everyone, > > while rebuilding a few packages in Arch Linux we have recently come > across a regression in the linux kernel which was made visible by a test > failure in libguestfs[0], where the booted kernel showed a Call Trace > like the following one: > > [ 218.738568] CPU: 0 PID: 167 Comm: guestfsd Not tainted 6.7.0-rc4-1-mainline-00158-gb5ba474f3f51 #1 bf39861cf50acae7a79c534e25532f28afe4e593^M (cc more zswap people) Fairly old kernel. We might have fixed it since then, but it would be good to know what fixed this, for backporting reasons. > [ 218.739007] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS Arch Linux 1.16.3-1-1 04/01/2014^M > [ 218.739787] RIP: 0010:memcg_page_state+0x9/0x30^M > [ 218.740299] Code: 0d b8 ff ff 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 66 0f 1f 00 0f 1f 44 00 00 <48> 8b 87 00 06 00 00 48 63 f6 31 d2 48 8b 04 f0 48 85 c0 48 0f 48^M > [ 218.740727] RSP: 0018:ffffb5fa808dfc10 EFLAGS: 00000202^M > [ 218.740862] RAX: 0000000000000000 RBX: ffffb5fa808dfce0 RCX: 0000000000000002^M > [ 218.741016] RDX: 0000000000000001 RSI: 0000000000000033 RDI: 0000000000000000^M > [ 218.741168] RBP: 0000000000000000 R08: ffff976681ff8000 R09: 0000000000000000^M > [ 218.741322] R10: 0000000000000001 R11: ffff9766833f9d00 R12: ffff9766ffffe780^M > [ 218.742167] R13: 0000000000000000 R14: ffff976680cc1800 R15: ffff976682204d80^M > [ 218.742376] FS: 00007f1479d9f540(0000) GS:ffff9766fbc00000(0000) knlGS:0000000000000000^M > [ 218.742569] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033^M > [ 218.743256] CR2: 0000000000000600 CR3: 0000000103606000 CR4: 0000000000750ef0^M > [ 218.743494] PKRU: 55555554^M > [ 218.743593] Call Trace:^M > [ 218.743733] ^M > [ 218.743847] ? __die+0x23/0x70^M > [ 218.743957] ? page_fault_oops+0x171/0x4e0^M > [ 218.744056] ? free_unref_page+0xf6/0x180^M > [ 218.744458] ? exc_page_fault+0x7f/0x180^M > [ 218.744551] ? asm_exc_page_fault+0x26/0x30^M > [ 218.744684] ? memcg_page_state+0x9/0x30^M > [ 218.744779] zswap_shrinker_count+0x9d/0x110^M > [ 218.744896] do_shrink_slab+0x3a/0x360^M > [ 218.744990] shrink_slab+0xc7/0x3c0^M > [ 218.745609] drop_slab+0x85/0x140^M > [ 218.745691] drop_caches_sysctl_handler+0x7e/0xd0^M > [ 218.745799] proc_sys_call_handler+0x1c0/0x2e0^M > [ 218.745912] vfs_write+0x23d/0x400^M > [ 218.745998] ksys_write+0x6f/0xf0^M > [ 218.746080] do_syscall_64+0x64/0xe0^M > [ 218.746169] ? exit_to_user_mode_prepare+0x132/0x1f0^M > [ 218.746873] entry_SYSCALL_64_after_hwframe+0x6e/0x76^M > > The regression is present in the mainline kernel and also was > independently reported to the redhat bugtracker[1]. > > I have bisected (see log[2]) the regression between v6.9-rc4 and v6.6 > and have landed on the following results (removed unrelated test commit) > as remainders since some of the commits were not buildable for me: > - 7108cc3f765c ("mm: memcg: add per-memcg zswap writeback stat") > - a65b0e7607cc ("zswap: make shrinking memcg-aware") > - b5ba474f3f51 ("zswap: shrink zswap pool based on memory pressure") > > I have decided on good/bad commits with the relevant libguestfs tests, > but I think the reproducer in the redhat bugzilla is simpler (although I > only became aware of it during the bisection and therefore didn't test > it myself): > > LIBGUESTFS_MEMSIZE=4096 LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1 make -C /build/libguestfs/src/libguestfs-1.52.0/tests -k check TESTS=c-api/tests > > I hope I have included everything needed to debug this further, if there > is more to add I'm happy to provide more details! > > Cheers, > Christian > > [0]: https://github.com/libguestfs/libguestfs/issues/139 > [1]: https://bugzilla.redhat.com/show_bug.cgi?id=2275252 > [2]: https://gist.github.com/christian-heusel/d5095c36b72ae90871e27dfed32ddc46