From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8C47C4345F for ; Tue, 16 Apr 2024 12:19:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 348176B0085; Tue, 16 Apr 2024 08:19:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2F8176B0088; Tue, 16 Apr 2024 08:19:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1C0936B0092; Tue, 16 Apr 2024 08:19:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id EB8186B0085 for ; Tue, 16 Apr 2024 08:19:32 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id AD955120AEC for ; Tue, 16 Apr 2024 12:19:32 +0000 (UTC) X-FDA: 82015300584.09.DE90EE6 Received: from mout.kundenserver.de (mout.kundenserver.de [217.72.192.74]) by imf28.hostedemail.com (Postfix) with ESMTP id B5F60C0015 for ; Tue, 16 Apr 2024 12:19:30 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=heusel.eu header.s=s1-ionos header.b="HDZJ/mpv"; dmarc=none; spf=pass (imf28.hostedemail.com: domain of christian@heusel.eu designates 217.72.192.74 as permitted sender) smtp.mailfrom=christian@heusel.eu ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1713269971; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=IUD8Jjs8JYpUGxZJgs43b1N3UGc4tTFtGJp9VuOo2cI=; b=VP5NQgYGMz8C0ESEJGc21A1tBsbCOAS+Mqb1eDMvm8rteC09bQdx97TWecoSJUdKWL6rI/ YU5mVq6JATsLEPe3TbsAYOLWV37rbNKwf2bNRlwu8U/k3hlsBD7LDOhdwYF/ka2t4hyBUn 78lCZgiA/JKhZh4uWhFXkcEC7XhLQSc= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=heusel.eu header.s=s1-ionos header.b="HDZJ/mpv"; dmarc=none; spf=pass (imf28.hostedemail.com: domain of christian@heusel.eu designates 217.72.192.74 as permitted sender) smtp.mailfrom=christian@heusel.eu ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1713269971; a=rsa-sha256; cv=none; b=kcZi15Ves89inQ1/0REhrCJfXQOl8hBpJeOFxrXgScb286Pm0oNnkGwC9rUn3fIzocpteW 2E8YrMZ1haxbxJPC4yPZfsr4ItD2SLBDpAdNTwYBteTMHUNdIo/4OYvKowlUGl4pCpZrsJ yXpmWv7vabBwk9xSkKCTjfJpjGAioRs= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=heusel.eu; s=s1-ionos; t=1713269966; x=1713874766; i=christian@heusel.eu; bh=IUD8Jjs8JYpUGxZJgs43b1N3UGc4tTFtGJp9VuOo2cI=; h=X-UI-Sender-Class:Date:From:To:Cc:Subject:Message-ID: MIME-Version:Content-Type:cc:content-transfer-encoding: content-type:date:from:message-id:mime-version:reply-to:subject: to; b=HDZJ/mpvJkHBerXqVuZR1gj2p1uKhz+2JJxzOw3yYBYH0y2py0YOlV3lvUk0nf4c ewfpawl46kMSu9fggpWVmfdB/7nT5u16Fn18aPjPPXBhsnk0rsfCYksjtJlCzxZpC SZ3F7Ns9tAkJ8lIHCMDGRqZRJyYscTD4f8spewTsNf6c7YRMW3wcTElxbC63X1LMU HTpt2GhHTBke9yCTWB7pSjDGNGLMAtzuoiDAxfrKPhtHhIEqMkYZoJQ1w0KUtjaOW 57Pqj1oUb/CSFilxtHv7oUgYz45yKV+4wL7cDxiS0eXk+brwN6Q4VD13pAcE6GSUq 2xwHFM6srcwJCMkRtw== X-UI-Sender-Class: 55c96926-9e95-11ee-ae09-1f7a4046a0f6 Received: from localhost ([141.70.80.5]) by mrelayeu.kundenserver.de (mreue106 [212.227.15.183]) with ESMTPSA (Nemesis) id 1Msqpq-1slMSU1Xom-00tACu; Tue, 16 Apr 2024 14:19:26 +0200 Date: Tue, 16 Apr 2024 14:19:24 +0200 From: Christian Heusel To: Seth Jennings , Dan Streetman , Vitaly Wool , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: David Runge , "Richard W.M. Jones" , Mark W , regressions@lists.linux.dev Subject: [REGRESSION] Null pointer dereference while shrinking zswap Message-ID: <3iccc6vjl5gminut3lvpl4va2lbnsgku5ei2d7ylftoofy3n2v@gcfdvtsq6dx2> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="c77hs6q24bk2juqy" Content-Disposition: inline X-Provags-ID: V03:K1:4da/o3EIRrGjeM25Qy59Ppz83BR5JlO/HHAHd8xhis3qK90zcWs DSTJOGMlAsJvkY628cw9q1WdiGkIAxiQw41M2e5nRhpew3NpzBsGDfLsg5/+0oUUJwkBeET /BzSDfKdPcjynQfy46dg78gEURXDd35HYszT2dqzd1ip+KNkePY/Y+SiVIl2I26LQ+fyQS4 39NcQWctB+HRDXIfIFs0A== UI-OutboundReport: notjunk:1;M01:P0:g+NtGFPogMI=;H4DZ6BtNbvsAVYMzxB1oSodCbCt lmJTuLtRZ9CAmojrrf0e/YoaRCPQ7VmalBYmQ6jurMexzQTQZ0ImKLfJANDUu5fVpKeH8YsIJ 0olaTfcMDrZfeJLNYwNZIYfxLDsn+ZcjQRYWKCxcawOubfqFmt+f2Fr5am6gyhc2fuElYAp4F QqF85FY5KRo2rv+ecjbGYcTwuOa2Z5+29NAPZ+HDcP3Icsb4KB/EBgPf64l+LakKp9S5JIXwD Z2uIYC0RqW6zkWf3ZldYuJnXtg0qottlw2yCbmKCOnDFezewTV+B4JpuDsCG06HuT3KT8GLE2 +4n38ijyhtRghPG3j/LFOZovGFC5v1XOXsNKf6dlGQm5qETjVd0Un+2FIXNe4Hj+yKcKAhcbO MkU+i27R13lHqDX701Jp1vSVOsD5FSbo9zWnfg8VxwNs15024cExRPkitpKvD5vpasgDjP82t nanMLKFGIMRwkSZJNOftmThTSjPxisNilzgG/Tu4PlfZ6OFtDjP6M1AGM5mmo2CArSd20GdRU RWHLVyalLn1fbck04vcMza5dxaXndpZS59KobB1JajwUbC2sNjH9GZuLRgrOQdKSxIqATnYls UHYtl7lxpw2FUorGfge834DjIlEx+jUld9C2JsWuViXR1A/KDra6ALApiSrdNMDWdNzHNgEKk /F1hqTjxg/Kdk0fWmRjLOI/u5S4ApqFj2ZqWoDYwGbXbK2k+rSMW4WrdMTh8WWmE+FOBPl1qk Io+HjmLmW469jGTL81rgPQfLkOswsXaFlIieSdBjLlR1O2tM00qqwM= X-Rspamd-Queue-Id: B5F60C0015 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: tj5repmeo78x6s94xpnhzsfnfze4nruj X-HE-Tag: 1713269970-24883 X-HE-Meta: U2FsdGVkX19YQXsp3XaUm3auGvugwRxcDMhYEP14Q6vGNSJrKB0SUR+hGL+qRfmRtQUKjSE21z+1jKhjEmq9zkzpUPg0oTzsDNrynUK4Egj3+8DxRCQJlTbfCaosdIpuQ570LxKANx9oX4g7Hj9Fan6COBwFOiUQGTeARmEJR3umttZCaLsCEWK0TQMg2C1eUpBPAOTe3F01UfoYE6K55eDRgh6OTjlNrVN2vJKCGOuTetMV+W6n0/AiqLmPta7zrXfe5JJfTgKBJAW/Ta0yoMwKp39GLUmutbDNe3+nBDKEbDvxCo1is4qiu1LqcPMkb2op8XzNI5wqcfnWrixCR0z5iwpdahsUgKf2pQHLcCf8Ke3ywyxANmpCZ5mqzD2ymccK+PKArTgkOTaSBPV6vgDZqBzBE+OBtkDtSIXP/fDyeFvvMxmZFdz+4NgnCVWolHAuZJOD0cfBaBf6dh2jvEQsYjEd61w+A6oeqZ2ZF4To8UHoPkc19X4xnjFjFBBBmfuGrDdBzq3rUnpX06MB34bH8ut7khwMC00R7BunphrffYMSRFiN+2PWKNlN7vLE3P+784Ss/f2KFrHf9ETklsNHB8ghxPpI6i2kzk59JN2HSRZlKcJ9FngTwsf4kYu8ZhSmrCIX/aV0Unme+PmQTLdOCh/ns34+Z7qdW3eYT/IJyonSModz/VgmOGBK+rLklJTXjYuhqu1crr7i1D1x/scfKwz5RBvmY5mIXXEyyLlzJ9N9iGFPa1psAR2JAyBB5DnT/8IGrPPM8W0mW7OSMzs7u1vxIzZSGgIplfzHRJzGRE3ygz3XYW7N6voHyIdDjHd7kqI5CgStqPYXQqrzPamfozVzrAuCPAItEr0atEx00quHeGpdLhLBt539lCNgd8FOKfbQHC3aPTRpdTKlhv3610iLM/b8KuS/DjUf1dUXMrwDZWke5Q0lgqMStrGbggd8yKDkPy/RDw1KY1l 0URbGoj1 Yls3MEkV7xytnmGh+Av17OQhY8dIFScBNVDuxQcKU+SOXqflAnNXj3MR0TOSyu3roxK9Fm37riDfDWdPNkFYGZnq+PhTPZfEoWpMcLgKMyf6GDBv1ZgwCycprJSKaGDt3Goj78D3x6NymMkdL9Hk36PYQutnzFMOJq3VImcphZTrfYOZWG32NE4CaoDSX9hqx+dbV1yT+7N+zCzS7XJ5IKtyiYxqcy20FK2PWxB8ErDyeBZIO8za9z5tO/yVSy1T/bCTh8U16OzWhjJmhjfbR3DzPXhyLIhBWFjYcVTLT4cUzg4/+ECGCRd8I//AmN8yzAeCTf0otlFr/FIebIrmmGmGbegsSdlX5ZdbnAfvGcwfCk+1o84KtcJZLxja4XTf6AgU6MsIlRcVx79woGsYSWASmXw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: --c77hs6q24bk2juqy Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hello everyone, while rebuilding a few packages in Arch Linux we have recently come across a regression in the linux kernel which was made visible by a test failure in libguestfs[0], where the booted kernel showed a Call Trace like the following one: [ 218.738568] CPU: 0 PID: 167 Comm: guestfsd Not tainted 6.7.0-rc4-1-mainline-00158-gb5ba474f3f51 #1 bf39861cf50acae7a79c534e25532f28afe4e593^M [ 218.739007] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS Arch Linux 1.16.3-1-1 04/01/2014^M [ 218.739787] RIP: 0010:memcg_page_state+0x9/0x30^M [ 218.740299] Code: 0d b8 ff ff 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 66 0f 1f 00 0f 1f 44 00 00 <48> 8b 87 00 06 00 00 48 63 f6 31 d2 48 8b 04 f0 48 85 c0 48 0f 48^M [ 218.740727] RSP: 0018:ffffb5fa808dfc10 EFLAGS: 00000202^M [ 218.740862] RAX: 0000000000000000 RBX: ffffb5fa808dfce0 RCX: 0000000000000002^M [ 218.741016] RDX: 0000000000000001 RSI: 0000000000000033 RDI: 0000000000000000^M [ 218.741168] RBP: 0000000000000000 R08: ffff976681ff8000 R09: 0000000000000000^M [ 218.741322] R10: 0000000000000001 R11: ffff9766833f9d00 R12: ffff9766ffffe780^M [ 218.742167] R13: 0000000000000000 R14: ffff976680cc1800 R15: ffff976682204d80^M [ 218.742376] FS: 00007f1479d9f540(0000) GS:ffff9766fbc00000(0000) knlGS:0000000000000000^M [ 218.742569] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033^M [ 218.743256] CR2: 0000000000000600 CR3: 0000000103606000 CR4: 0000000000750ef0^M [ 218.743494] PKRU: 55555554^M [ 218.743593] Call Trace:^M [ 218.743733] ^M [ 218.743847] ? __die+0x23/0x70^M [ 218.743957] ? page_fault_oops+0x171/0x4e0^M [ 218.744056] ? free_unref_page+0xf6/0x180^M [ 218.744458] ? exc_page_fault+0x7f/0x180^M [ 218.744551] ? asm_exc_page_fault+0x26/0x30^M [ 218.744684] ? memcg_page_state+0x9/0x30^M [ 218.744779] zswap_shrinker_count+0x9d/0x110^M [ 218.744896] do_shrink_slab+0x3a/0x360^M [ 218.744990] shrink_slab+0xc7/0x3c0^M [ 218.745609] drop_slab+0x85/0x140^M [ 218.745691] drop_caches_sysctl_handler+0x7e/0xd0^M [ 218.745799] proc_sys_call_handler+0x1c0/0x2e0^M [ 218.745912] vfs_write+0x23d/0x400^M [ 218.745998] ksys_write+0x6f/0xf0^M [ 218.746080] do_syscall_64+0x64/0xe0^M [ 218.746169] ? exit_to_user_mode_prepare+0x132/0x1f0^M [ 218.746873] entry_SYSCALL_64_after_hwframe+0x6e/0x76^M The regression is present in the mainline kernel and also was independently reported to the redhat bugtracker[1]. I have bisected (see log[2]) the regression between v6.9-rc4 and v6.6 and have landed on the following results (removed unrelated test commit) as remainders since some of the commits were not buildable for me: - 7108cc3f765c ("mm: memcg: add per-memcg zswap writeback stat") - a65b0e7607cc ("zswap: make shrinking memcg-aware") - b5ba474f3f51 ("zswap: shrink zswap pool based on memory pressure") I have decided on good/bad commits with the relevant libguestfs tests, but I think the reproducer in the redhat bugzilla is simpler (although I only became aware of it during the bisection and therefore didn't test it myself): LIBGUESTFS_MEMSIZE=4096 LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1 make -C /build/libguestfs/src/libguestfs-1.52.0/tests -k check TESTS=c-api/tests I hope I have included everything needed to debug this further, if there is more to add I'm happy to provide more details! Cheers, Christian [0]: https://github.com/libguestfs/libguestfs/issues/139 [1]: https://bugzilla.redhat.com/show_bug.cgi?id=2275252 [2]: https://gist.github.com/christian-heusel/d5095c36b72ae90871e27dfed32ddc46 --c77hs6q24bk2juqy Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEb3ea3iR6a4oPcswTwEfU8yi1JYUFAmYebMwACgkQwEfU8yi1 JYW51g//crfDQssE4eRaxPjjjlY05pwC+XKztfeyE6M6H/LuPGJnUhQZ/SLwvHgX 4uTOdh+6kWr1ykVxIpdSMuPMEdWA78GAkDwapwhXu1Hy3PD7WruSlFrplzWW/uSb Rfthb76x/jSAl8GTXfO/EeotVfw0FwvzfdeNxgOMTsSG3tEzAyfWs0WXUrfNQvnJ 5AvpfnrsJ5sv2BfSpiwZDTFX+awBrMBgrKD71hEgd9ln79BhZWm3768rLeeWSlOG 51u/vmw5Gia42i7r/ekdeH/P7Tw2Gftj1Ke7BMOmjiPiqQ05MvarN7KVUcmfVMny PaouhwwEoYtlCgdd//zhmXRKYJL6I5lxfyOv32USYVK25E/jeAQi1aJVSXAnNgsU lmHj4QirJ3jNgIWtwQyRjKLJPbV+Bw0IBXNXbExSCSLTYJb3UB91WUgeo2nrlULL Y7aUcFlN0KkM7BuIggSUms16sbMMqMpTx7UELbyPZozA1ALEkq/H8vP3ljcFL9RP v2YAKkihBfEh7UBnEZ9fHYp/s9kF0Nria8bxVCtgKPSWbqaaO01ow6M8JMfsObIV QpDMWvcWGROftffBztc+BAFglkDPHKe4bIgV0D0V9nnx+AGktZ2MTBJETbKi1RqX WKuZDDLTrJBCJxLF1bmlqn0ZuP1a81JgG71CE+8FC599RBGF64Q= =Djhd -----END PGP SIGNATURE----- --c77hs6q24bk2juqy--