From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E783C3064D for ; Tue, 2 Jul 2024 15:28:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AB8776B009F; Tue, 2 Jul 2024 11:28:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A66FF6B00A0; Tue, 2 Jul 2024 11:28:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 930B86B00A1; Tue, 2 Jul 2024 11:28:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 730346B009F for ; Tue, 2 Jul 2024 11:28:14 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 362BB802FB for ; Tue, 2 Jul 2024 15:28:14 +0000 (UTC) X-FDA: 82295193708.30.E51CBA5 Received: from mail-qk1-f182.google.com (mail-qk1-f182.google.com [209.85.222.182]) by imf12.hostedemail.com (Postfix) with ESMTP id 70DEE40008 for ; Tue, 2 Jul 2024 15:28:12 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=iaGSwNRt; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf12.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.222.182 as permitted sender) smtp.mailfrom=nphamcs@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719934067; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=k0eh3p6qPcP7dC2h7ien1wFzUSgMua9EcLtjou/ga4M=; b=aCFRQkjaQmDjUUIv+5WsI3fHwPNM0xlYS1Suri9FuU470Sv6PnIRPSqEScH8TNp3xPKvdC +fBes//NyFdOkC6vu4wTg0NdrgCeg8CMUxe+hxG3pEvGc4BguVXDQX6toUlXPUQ10WS6tt rhG+RmGod9ZdOhNQa0waiFS1UUXx8tY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1719934067; a=rsa-sha256; cv=none; b=XeTWxsH4I25Og48q7BRlaI4uf+wVGtG9FknCAI5z9Aa9B8UJlDjSDpZnLSs6G2dDENakwK ZhFuwQchz+wXzugdOs4ti/PfAUEMxsjszXJgOwWFL+vqXxqqXFnop6i6umR6t1Z91vUos8 jpAZ0ONtjbdQ+8gWrB1jwGZzc0+/VWY= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=iaGSwNRt; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf12.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.222.182 as permitted sender) smtp.mailfrom=nphamcs@gmail.com Received: by mail-qk1-f182.google.com with SMTP id af79cd13be357-79d90e01b30so134381885a.2 for ; Tue, 02 Jul 2024 08:28:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1719934091; x=1720538891; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=k0eh3p6qPcP7dC2h7ien1wFzUSgMua9EcLtjou/ga4M=; b=iaGSwNRtEb43ZNdIJhNCKwPq8fCfjWpX+VD3lPtpfoF4VxrZN53VvaOvbWyNumPPFd s42Nwps5dZXBKS+wz/y2cxGKLufdxYKpy1bgbT2yqfNwKX/IIq8x4zW/xDVN5L/5IiL/ ugGdJwFx1jv8sfeTwePDXHsY340cgXP0UZ0HgKtQONfbhXfeS+9znw/SUXs5dmWL051e 4w8PoV+OpCKm+Ds/xI7CMQlwL4pGONXvG0HMMdiJ+6AyJYB0vDViHhHx2wsaeYfWgi1z 1UMG27DCbm3R3C+2vidUCyd0DlI6KLzpIPWRrAAv7hwQ+Q93Q0ktcAlOW9D2XRmY8biW IMlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719934091; x=1720538891; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=k0eh3p6qPcP7dC2h7ien1wFzUSgMua9EcLtjou/ga4M=; b=UMTQ8VfkcyIHF/qVIonlFrhaj0secIwBcg9UcXQb19jvL4EkInyOtOVzzE9lONtAwv I+BbclXw2t0jKw8rxuCVmDzrYreKoQyHfRqyz137i9x6cmuwrJAWSUn0mo1tXxZn06L/ ppC0mak8wT1bfBlaP4tea4D6mfjSpdP+7ZmqtlVEDBiu8zMuA+WPUgjfKsetfAOoJYM3 LazqQj2wllmkjBmTZJDVJKbd4kxu7Qf+nOHe6m3LGULG1XJgTeopNL8XfJ61/9xWBGqv NPJqYtp5wBXCJUmFloRCyolcAGVqHGzYdfLmKaP+CZUBu8FTsi5D4k96aTH0Dj/qM+qh Df6A== X-Forwarded-Encrypted: i=1; AJvYcCW03jWp5a9QVNsoHkR5Di4ZoI6Tyr9naNT7639sQ00ucjgemi/5AOfW1c1RhWQ+9lIppP9jST1I7iCc7u+YfPa6/WQ= X-Gm-Message-State: AOJu0YwQeHPjU+LxQudeWZqJ/wW++9b83JG5MckY4AakRLuoQKeZb1Q6 gB/q9tcDAafhMJum6toSmx2/ni72gCEEO+BOqiwLyrhfxNP6APLRqfLpwAEGBNRLgaCDRokpwIi retOVo7OBnUtO52vHcxgpCDbO04M= X-Google-Smtp-Source: AGHT+IHvWk7453VuLpTof4HZQwUnMyUU2esYkvshRC3QIEhYFnMdUpYq7QJ+drdrhCZq+1pjXAE3cpY0ynuvrhfX/ds= X-Received: by 2002:a05:6214:194a:b0:6b5:9c9c:a736 with SMTP id 6a1803df08f44-6b5b704f0f6mr124222236d6.6.1719934091338; Tue, 02 Jul 2024 08:28:11 -0700 (PDT) MIME-Version: 1.0 References: <20240702003343.2756828-1-builder@example.org> In-Reply-To: <20240702003343.2756828-1-builder@example.org> From: Nhat Pham Date: Tue, 2 Jul 2024 08:28:00 -0700 Message-ID: Subject: Re: zswap_writeback_entry crashes in 6.9.5 To: Builder Cc: pedro.falcato@gmail.com, chengming.zhou@linux.dev, christian@heusel.eu, hannes@cmpxchg.org, linux-mm@kvack.org, regressions@lists.linux.dev, yosryahmed@google.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 70DEE40008 X-Stat-Signature: ersjpzwb5s8ii4f13g1py1oe7e731tws X-Rspam-User: X-HE-Tag: 1719934092-993876 X-HE-Meta: U2FsdGVkX18gsGLKf3V1cKFxOj2Wos9krk0appHjEMk1IRcvNgsi+i1EBMECP+FhjOTUMJ29sEVZ9/XAYmb79NU0ts3mPFhp4w9FNNCxvqXMc5vmNv1CiFHVn/kpTCOSrIBiYiUpu9ksdHYLQUycbD3fskiHwTzLyoGvjBv5ixLXMUuVJWhPHWYBvbFJ8DEDUU1RhfNiwKadnI35jAUua+IPQL/gY9zo/9fEj75mNoXN9Y7J08MXseSxdIJS5m+nh6+hV7sk+tpqp83B6f4+O1uOi8M+oZz5mu9GkATbpaUYARI41bJgVvKCsOwZctuA//J57tWW9p5nnB0aQ8RfiGVQM1UctEX5G3p7Tpr3rl1Xm5oMQEH6IKrlIaOZp/ZZRzqM8OOJX6vRcyuT4JuDOz9Vck9oRUrDmrlhLRblCvPkxr9cbpsqgT7qYY+N3etAbmq2D5wME2HvSMacFe9093G+yhuE/g9dPc2ByoNnfEPIa88q4O9uCBGkCy5MuLveXckU1CVwDf2Iv+p7XXqvZv2XxKvCwZxg8fcD4dINnHb6Q8E7VNBp01vaOHL8U9FxnB8xoY+2Sob9CXer99HF/R8Vrvp5qTzHOJcsFCBAKj5SeG9djgGArDCRlfiqPV2v2Yv3Baeqj6Iy/YPObkLpfqhjcxrga/rhwQOcw/BkdYebNvDjI/1KqY5RCMzauPejxtTOVRv9XQFvmxqG4udOwN6dKAbiX4sF442bAwZ4S8hEl4LUcnY9ooyfxbVcikNPpeVydYDEsiDkE0ZTnXiom4EMuUOxeOVLdVK7jEXbFi3a4lzO0Q8qYi3VO55sIEb9MHCk4uk4pMjF5KVWenYu4pbGsYSpGyL4n23DrMzjr9j3VyOmpCxyDK/38NwR5iKKIS1iBz3nHhgjh2LWbs5htlvnAzdZaM3G/BH9IuGqnxRWG3VOXFz2VcLaBFHcfTljgjt67z9crX1NYXZ7933 sj8qfnvp l2prajvLRnq4J6jWgcPYN72Cm2EJSs298rn1EoKNu8t4ckhJw6DFf/cytSB6xlD6RqtI3Qq3F/7Cwq7tAOxIFN98OC5QcGtgpINQSxvA4FmXw9RkQv7Aigw1nO6Fx/ldXuG+mGlnihrLGG2NBOBZNS4lOo4ARZ8gtN6nvv2NEAVwzK8LZWa/tYEKERWjGFn/teZwYQecRK9U3Ofl2Bi+HV6pxrZVKan1C7F8kXRCLkjjvRyK6JnURMAhW0Xn6vgb2GSsHkGfKkl2bNVn3vFaUgw+96RwaLAgqmszChz56DJ1WrIgR34J9/6eB1H7CfhpMoydeLLK0Gvz6I51CuGC+XqYk2+pyz1MZXhX9+xbAAueyl3jL9FdT+tbLlUm2ukr/fq09YaQkaq0eozMKBJf3eOX6DNU+tTOe4DujpyCJvXeCMk4KaCGgYyKb4W71yoL8JCgV7LrnqbBwz/c= X-Bogosity: Ham, tests=bogofilter, spamicity=0.164498, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Jul 1, 2024 at 5:33=E2=80=AFPM Builder wrote: > > On Sun, Jun 30, 2024 at 10:58=E2=80=AFAM Pedro Falcato wrote: > > > > Hi everyone, > > Hi, > > I think I have hit this problem a well. I actually reported this on RedHa= t's > bug tracker a while back, along with a couple of stack traces: > > https://bugzilla.redhat.com/show_bug.cgi?id=3D2275252 > > Reverting the commit I mentioned there seems to make this problem go away= for > me. This is a long shot, but I am curious if it will also fix the problem= for > you. > > (Also inserting myself into this thread so I will get updates.) > > Regards, > Yuxuan Shui This looks like a different issue. The hang-up is one task waiting for the mutex lock (&acomp_ctx->mutex), whose holder is the other task that crashes. Looking at that trace in particular, the line that triggers the BUG_ON call (mm/zswap.c:1395): BUG_ON(crypto_wait_req(crypto_acomp_decompress(acomp_ctx->req), &acomp_ctx->wait)); is the compressor failing to decompress the data. This looks like some sort of memory corruption, and could happen for a lot of reasons - a zswap bug, a backend allocator bug, a compression library bug, or a hardware issue that corrupts memory. If it only happens on 6.8.9 (and not 6.8.5), then it's likely some changes in between, but I'd be very surprised if the bug somehow comes from the patch you reverted. If you look at the patch's content, all it does is essentially handling the case where the shrinker receives a NULL memcg, by using an alternative source of stats. It could potentially reveal the problems previously hidden, but definitely not the cause of those problems itself. I'd recommend that you send a separate bug report with the build config, steps to reproduce, and more information about your setup overall (what backend allocator are you using for zswap - it should be zsmalloc btw, what compression algorithm you are using, etc.)