From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2530C5321D for ; Tue, 20 Aug 2024 09:30:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2DC106B007B; Tue, 20 Aug 2024 05:30:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 28C946B0082; Tue, 20 Aug 2024 05:30:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 17BBC6B0083; Tue, 20 Aug 2024 05:30:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id EE2A36B007B for ; Tue, 20 Aug 2024 05:30:01 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 99E191A1959 for ; Tue, 20 Aug 2024 09:30:01 +0000 (UTC) X-FDA: 82472102202.10.09B00F9 Received: from mail-lj1-f172.google.com (mail-lj1-f172.google.com [209.85.208.172]) by imf03.hostedemail.com (Postfix) with ESMTP id A32F22001F for ; Tue, 20 Aug 2024 09:29:59 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=d9QkFbzw; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf03.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.208.172 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724146112; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xPTzMFAm15Rb2HBvAR6AdEa5g9tMtaOT9x7A3ClYX70=; b=cvg5K1vez426b2oEZuVMapNdm8fPrJcqO2D4oui/XYBliRKmhSzVWmFeQT6K28KA6rrz48 GhfjRj3VOjMcKo9jhI20o6vyLnGbbzBtBNdexKl3cKnz6dz7rXoG0NrRNs4dwBwPhbRCEi 8uLxLmE85OWnn/YheDCkC1mKOOsJd5c= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724146112; a=rsa-sha256; cv=none; b=NkHmB/hx5f4d0YoJXqQ/Tqmu2hrlp6a7K6nzIxLsWOz+kPAhL2+KFRbsjL7JHd0R4dXOjA OqE48wLHlux0C6SQQA/m+VRKcsAH/091GthELU7QaKXvmV+wtbPVe9t4Z2ZTOUy8vEY0Rn IY+kn08eiyKn4Yv6CKymCxbEs8mSTB8= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=d9QkFbzw; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf03.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.208.172 as permitted sender) smtp.mailfrom=ryncsn@gmail.com Received: by mail-lj1-f172.google.com with SMTP id 38308e7fff4ca-2f15790b472so64574261fa.0 for ; Tue, 20 Aug 2024 02:29:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724146198; x=1724750998; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=xPTzMFAm15Rb2HBvAR6AdEa5g9tMtaOT9x7A3ClYX70=; b=d9QkFbzwt1UFYHtVEDun7n+LKISBlGRlf0q+70Nr5OvV/igaFoWx/SNTiChgCtxtFe j/elNiBlocqe/E3SHk/AjrjlKA5eN320rRnOR/dSKhcstLAcuPejGga1taMtzokrDl+e 5mQ0cemzNzWh+Z7hemraNESpCvYpX+TXQr+KXaIZodgWxFaQDDh2u4ukRHAeecpSXTur oHArMc6kcgLlAwGNt8sVsNjiqt3CnzF47KNfqwzBQvgTeQ2SJxzJGie6inddvzpjORd1 l64mX2KOXcqAo+umGEs5xDDbuZqJRSAA8E/hAWj7hBAd7STcv6s0IGqP+ITcuDvCPrev U0IA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724146198; x=1724750998; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xPTzMFAm15Rb2HBvAR6AdEa5g9tMtaOT9x7A3ClYX70=; b=ZCFleg9JBrPBxDLKaN7ofhekSdCjLA+03Vj9W5btObqeb4uxfZhTflG3ccWmwBmwIp zCz+TT+3pohNQ/f0zFAcoq0/fZFGqmuoAsyRnAIvXnyYrk9qBcuyBjDrM5PTYO+EA/Mh JKK1dODQambP72Ydp+wKrZgqm0b4TKmmUJak30A0ISM7sD0JjgMCBF7+sYRHGI9VeNL6 SliZ4Xj3wyTqraud0yEhOofcd3KKfJCK35LUo/7Umwu74tSvlGaDVDdbfDyQoHz9ZLb0 ir71FxeH5KqWP6sODfFk6XZRnFWV+7MOlxckSWhxaX9oa2DG9SNcvnFYG794CpLvlWiA OwZg== X-Forwarded-Encrypted: i=1; AJvYcCVQYJ6oUzkrtBn6OXOtryMuNbMXF90LrLAgjEsI4rHLs7Hf5IC18TjuP7MsMnYKG2jfx216es3Fgg==@kvack.org X-Gm-Message-State: AOJu0YwP5iyd4NbBQso/MfTWSeLDO7/IN8BIXiXP1i78536/tOM73wFm YAyYkIewhjd+GeF6rE014uB5g2iUtEpq5Cj9tsHKcO8MJzSTWkSs/czyfzQotJYEdq0h/nc2I5M yUFCg1bCMY2v0+ggwfUNxWjXtpyI= X-Google-Smtp-Source: AGHT+IF0JZYQi6B+P/K31GPutVfUW1JnSSBX9EyqQMXvDKYuQQcwdDvnpJDiZ4dxq59aga+uDtN7Vsn2jU0+e3SxIwM= X-Received: by 2002:a2e:bea5:0:b0:2f3:f1cf:5313 with SMTP id 38308e7fff4ca-2f3f1cf544dmr3675231fa.24.1724146197233; Tue, 20 Aug 2024 02:29:57 -0700 (PDT) MIME-Version: 1.0 References: <00000000000060cf79061fd24ca8@google.com> In-Reply-To: From: Kairui Song Date: Tue, 20 Aug 2024 17:29:40 +0800 Message-ID: Subject: Re: [syzbot] [mm?] WARNING in zswap_swapoff To: Barry Song <21cnbao@gmail.com> Cc: Yosry Ahmed , syzbot , akpm@linux-foundation.org, chengming.zhou@linux.dev, hannes@cmpxchg.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, nphamcs@gmail.com, syzkaller-bugs@googlegroups.com, Chris Li , Ying , Ryan Roberts Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: A32F22001F X-Stat-Signature: rzwuxx6aeckmqt68bjcu17uu3bwire3j X-Rspam-User: X-HE-Tag: 1724146199-961738 X-HE-Meta: U2FsdGVkX19X0mAoabuz0yB8BzUqyaaVW7h96OhPzSknEgESd3lKOZcMOX8KrbHIOCgras5Nqtr1UNowS8mfJg8NnljMVNmC8Pt75DwoRtBBqGzGZ3zMs9lm5eWqONmwr6/F0YCcbL2Wu9s22y5iQ1Jcas1zo69BHqCz8WLCCW2amQFViZmPkqGTmPcXF4ECJQzWYS9/EavTGPPgGgYguSNpXo5T9cSEPrO0tIDGUoVrNmrhLoZKZNr51akFk07zAEbRfnudPraiCEjrCkbUr6M5Cehv67wK/SOIw0iwNBnjcZrOod6yWiesUjIGj5Q0kW4SWWfRb67+PyyA4jVvJhpI9ukRG7Xodey+4WyCU2DQk9Su1zhromlEnNMhRgcmUjIWfWwAhINrH5QocjwOKdYomxJUXZOmDc40egTX7NxXAEnAGCRgFGy77OYXiZpB0MiO5+RVhcDdMgIi2yqsW3FOinoStoQltlSUlo4td4Nx+8eo/f8EMrmJI+Q7VWaaENyG6CyhJijOzRMkre8VjFMRqd5mac5fTp2hqw+6gYDw9JRp8Skp0NBN61BOr83SS+GAX8+DiUA9rjrV2inx2UmuETan525kWEBwJtYK7+S6UvE4VzJtz7e5ShE2HSWABN8GxROPHRPpTwN2lS1T6mipL7wcEy2FFJA6ZBRZMW0oAw/g4oleq43240pG5LSZy0M1EO8LJQeq9ds4/+HBo2IcIZTW7mEhEHo1VMxryxUPVKKy2KtDXtnRQQwXitKI/yWJFgpk5ZrnLDAtnWbSkWuE8woVTQVgonsz1FNWASWGZLauBCe6E9Ca6Qs7ihJno+tdEvvdU3hajz6o8fdmbl1btilpbq4CTNnuLEIKndYxwN3wDxKKej9B17/wULqhvGXetMxr7eF/RpHJ0lejkrrDP4Kta4L8kRRffVv23yw5EIqIwlKLQBls7yEUZQA4nPRXFFp5v5cW1zDb7C9 rvxiTlDW pQ2tTK6YPOOTxQnEm+3dCEwczATxod0HrRalma/OKULENjZMbIdAaUijvC69zRuGxkt5za+/qZ4oZfoI1k+Jz8LdpZtI305MEmzXjXzhnwcvSsTzasD1X4u7w33vuUOljD4k8p/rxtHko6vBkVEQVXs/flJENOVWVu2/SCvVs+1O/4Np4BkyMFGZZakIFJaKxQRuKkzxhPdK0Ni+6DML6Xftm5D7dxR1AFWlLBifVGbXYyHcU1BXAWGbSixrMBbTZE9Kk3BWMLJ5vriOUqZEt6c0aVGro+8RPLDaGdjRCGmwT7l+0hLHwFOspHsK/WKjENIALLQdek6LR3CsfjixtFvnCEeCu8chtbsuhe6lCYr4EIeK0CGqLKZDeoepS3KS+pe6JTl5Nl4FYifthMD1nA1MKHVY4bexMPUNUT+FEsc8dwWLY/dp9OVEh6EyiERnne+l6nzTrg5XJaSkj/lwjzhuhZ0NFAsFlf0tUwr2kDlpXmnWXDep6PZ5MXcOejfhdRg9wDp/VFaSJZAqKfrSZp2to/gJo6Tcu4P7nL35Dphk9bbSfZHq6EheDMfWpiTdebCU1oBBtmX5V50dLH2/oeH0FmEkpGdp5qJ/liq6X57HKolpINedQ1npq5mznAF34a4xYXRb/tez3PSESTAkSu+eR1OdBE5iLPWCYN1FHCx5aUka6A66BMmeT2lOTOBa/mm3ic1zV27ae23MHep2ibUffvJwZWWlBstCLRNWMwRes0txRG9GILVpq44BMl1RShf/b/79WoxxWAarrRKTtoTWDbT1r2jI8pwlruHvXwW9Oo6wHn901bOlQsI5VxjQUo6DnMScWUisRnvG86E3d8RZWMYPE8yvAV+ACD9kViMbwNccaEVhx9RKgV28WZC9XxWd5zX+9vjZ3Gbe3zq17YEsCWwGWnazpPsjrA380rOVLmn6gCfXTneZmUN9k9KiFgsgCDHv1GphnKNE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Aug 20, 2024 at 5:22=E2=80=AFPM Barry Song <21cnbao@gmail.com> wrot= e: > > On Tue, Aug 20, 2024 at 8:47=E2=80=AFPM Kairui Song wr= ote: > > > > On Tue, Aug 20, 2024 at 4:13=E2=80=AFAM Yosry Ahmed wrote: > > > On Fri, Aug 16, 2024 at 12:52=E2=80=AFPM syzbot > > > wrote: > > > > > > > > Hello, > > > > > > > > syzbot found the following issue on: > > > > > > > > HEAD commit: 367b5c3d53e5 Add linux-next specific files for 2024= 0816 > > > > I can't find this commit, seems this commit is not in linux-next any mo= re? > > > > > > git tree: linux-next > > > > console output: https://syzkaller.appspot.com/x/log.txt?x=3D1248910= 5980000 > > > > kernel config: https://syzkaller.appspot.com/x/.config?x=3D61ba6f3= b22ee5467 > > > > dashboard link: https://syzkaller.appspot.com/bug?extid=3Dce6029250= d7fd4d0476d > > > > compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils f= or Debian) 2.40 > > > > > > > > Unfortunately, I don't have any reproducer for this issue yet. > > > > > > > > Downloadable assets: > > > > disk image: https://storage.googleapis.com/syzbot-assets/0b1b4e3cad= 3c/disk-367b5c3d.raw.xz > > > > vmlinux: https://storage.googleapis.com/syzbot-assets/5bb090f7813c/= vmlinux-367b5c3d.xz > > > > kernel image: https://storage.googleapis.com/syzbot-assets/6674cb07= 09b1/bzImage-367b5c3d.xz > > > > > > > > IMPORTANT: if you fix the issue, please add the following tag to th= e commit: > > > > Reported-by: syzbot+ce6029250d7fd4d0476d@syzkaller.appspotmail.com > > > > > > > > ------------[ cut here ]------------ > > > > WARNING: CPU: 0 PID: 11298 at mm/zswap.c:1700 zswap_swapoff+0x11b/0= x2b0 mm/zswap.c:1700 > > > > Modules linked in: > > > > CPU: 0 UID: 0 PID: 11298 Comm: swapoff Not tainted 6.11.0-rc3-next-= 20240816-syzkaller #0 > > > > Hardware name: Google Google Compute Engine/Google Compute Engine, = BIOS Google 06/27/2024 > > > > RIP: 0010:zswap_swapoff+0x11b/0x2b0 mm/zswap.c:1700 > > > > Code: 74 05 e8 78 73 07 00 4b 83 7c 35 00 00 75 15 e8 1b bd 9e ff 4= 8 ff c5 49 83 c6 50 83 7c 24 0c 17 76 9b eb 24 e8 06 bd 9e ff 90 <0f> 0b 90= eb e5 48 8b 0c 24 80 e1 07 80 c1 03 38 c1 7c 90 48 8b 3c > > > > RSP: 0018:ffffc9000302fa38 EFLAGS: 00010293 > > > > RAX: ffffffff81f4d66a RBX: dffffc0000000000 RCX: ffff88802c19bc00 > > > > RDX: 0000000000000000 RSI: 0000000000000002 RDI: ffff888015986248 > > > > RBP: 0000000000000000 R08: ffffffff81f4d620 R09: 1ffffffff1d476ac > > > > R10: dffffc0000000000 R11: fffffbfff1d476ad R12: dffffc0000000000 > > > > R13: ffff888015986200 R14: 0000000000000048 R15: 0000000000000002 > > > > FS: 00007f9e628a5380(0000) GS:ffff8880b9000000(0000) knlGS:0000000= 000000000 > > > > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > > > > CR2: 0000001b30f15ff8 CR3: 000000006c5f0000 CR4: 00000000003506f0 > > > > DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 > > > > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 > > > > Call Trace: > > > > > > > > __do_sys_swapoff mm/swapfile.c:2837 [inline] > > > > __se_sys_swapoff+0x4653/0x4cf0 mm/swapfile.c:2706 > > > > do_syscall_x64 arch/x86/entry/common.c:52 [inline] > > > > do_syscall_64+0xf3/0x230 arch/x86/entry/common.c:83 > > > > entry_SYSCALL_64_after_hwframe+0x77/0x7f > > > > RIP: 0033:0x7f9e629feb37 > > > > Code: 73 01 c3 48 8b 0d f1 52 0d 00 f7 d8 64 89 01 48 83 c8 ff c3 6= 6 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 b8 a8 00 00 00 0f 05 <48> 3d 01= f0 ff ff 73 01 c3 48 8b 0d c1 52 0d 00 f7 d8 64 89 01 48 > > > > RSP: 002b:00007fff17734f68 EFLAGS: 00000246 ORIG_RAX: 0000000000000= 0a8 > > > > RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f9e629feb37 > > > > RDX: 00007f9e62a9e7e8 RSI: 00007f9e62b9beed RDI: 0000563090942a20 > > > > RBP: 0000563090942a20 R08: 0000000000000000 R09: 77872e07ed164f94 > > > > R10: 000000000000001f R11: 0000000000000246 R12: 00007fff17735188 > > > > R13: 00005630909422a0 R14: 0000563073724169 R15: 00007f9e62bdda80 > > > > > > > > > > I am hoping syzbot would find a reproducer and bisect this for us. > > > Meanwhile, from a high-level it looks to me like we are missing a > > > zswap_invalidate() call in some paths. > > > > > > If I have to guess, I would say it's related to the latest mTHP swap > > > changes, but I am not following closely. Perhaps one of the following > > > things happened: > > > > > > (1) We are not calling zswap_invalidate() in some invalidation paths. > > > It used to not be called for the cluster freeing path, so maybe we en= d > > > up with some order-0 swap entries in a cluster? or maybe there is an > > > entirely new invalidation path that does not go through > > > free_swap_slot() for order-0 entries? > > > > > > (2) Some higher order swap entries (i.e. a cluster) end up in zswap > > > somehow. zswap_store() has a warning to cover that though. Maybe > > > somehow some swap entries are allocated as a cluster, but then pages > > > are swapped out one-by-one as order-0 (which can go to zswap), but > > > then we still free the swap entries as a cluster? > > > > Hi Yosry, thanks for the report. > > > > There are many mTHP related optimizations recently, for this problem I > > can reproduce this locally. Can confirm the problem is gone for me > > after reverting: > > > > "mm: attempt to batch free swap entries for zap_pte_range()" > > > > Hi Barry, > > > > If a set of continuous slots are having the same value, they are > > considered a mTHP and freed, bypassing the slot cache, and causing > > zswap leak. > > This didn't happen in put_swap_folio because that function is > > expecting an actual mTHP folio behind the slots but > > free_swap_and_cache_nr is simply walking the slots. > > Hi Kairui, > > I don't understand, if anyone has a folio backend, the code will > go fallback to __try_to_reclaim_swap(), it won't call > swap_entry_range_free(). > > ci =3D lock_cluster_or_swap_info(si, offset); > if (!swap_is_last_map(si, offset, nr, &has_cache)) { > unlock_cluster_or_swap_info(si, ci); > goto fallback; > } > for (i =3D 0; i < nr; i++) > WRITE_ONCE(si->swap_map[offset + i], SWAP_HAS_CACHE); > unlock_cluster_or_swap_info(si, ci); > > if (!has_cache) { > spin_lock(&si->lock); > swap_entry_range_free(si, entry, nr); > spin_unlock(&si->lock); > } > return has_cache; > > Am i missing something? Hi Barry, Per my understanding, ZSWAP invalidation could happen after the folio is gone from the swap cache, especially in free_swap_and_cache_nr, it will iterate and zap the swap slots without swapping them in. So a slot doesn't have a folio backed doesn't mean it doesn't have ZSWAP da= ta.