From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D2ABC27C6E for ; Mon, 17 Jun 2024 08:03:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A81A36B0152; Mon, 17 Jun 2024 04:03:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A31C06B0153; Mon, 17 Jun 2024 04:03:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 923196B0154; Mon, 17 Jun 2024 04:03:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 74C2C6B0152 for ; Mon, 17 Jun 2024 04:03:03 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id DCB4C1C2975 for ; Mon, 17 Jun 2024 08:03:02 +0000 (UTC) X-FDA: 82239639804.07.93E941F Received: from mail-ua1-f54.google.com (mail-ua1-f54.google.com [209.85.222.54]) by imf10.hostedemail.com (Postfix) with ESMTP id EDD44C000E for ; Mon, 17 Jun 2024 08:02:59 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=MwPAF7He; spf=pass (imf10.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.222.54 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718611377; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=E/uPRBGoZoLGCBkeZjiswy/3hxNMDxtntxyuCTrUh0g=; b=qfn87IrknEg6AX0cL/bEo3ufWExO2Yu9OwIahzN90kw9GRR59kGy2cz7n5Jpzsboqay65R NOvm8pHBsOpm9q5WGTjJasA5uKBf+/BzyXAG9oNYfqiE4SUsiUzIRr2QB9ljz2lDS2lJBY K0x8cXWfHhzHG46LgQWnuBHx9y3B7k4= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=MwPAF7He; spf=pass (imf10.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.222.54 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718611377; a=rsa-sha256; cv=none; b=voxAB1gD3s+9tewu4n5I3PROBGe6WulewMLd6UceIFIDjjWnTGDN+Fh5i/f6HtZ9H/BMwt xGP6rLDeViWdb51NO6UhHsaDlA5V/rf9fXRI2SNM9pV42eT9HYAQ+RtkVhKwf1CqDkZ1BE hZG52ibe8FOFxpZaXrWoxHugJ2lQUVg= Received: by mail-ua1-f54.google.com with SMTP id a1e0cc1a2514c-80b8e5cab7bso1084812241.0 for ; Mon, 17 Jun 2024 01:02:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718611379; x=1719216179; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=E/uPRBGoZoLGCBkeZjiswy/3hxNMDxtntxyuCTrUh0g=; b=MwPAF7He+0/Jx/4Sr9iAVZ4/9LQMDmao/2jzBcdziHdeN5uMHGGDhuYzNWCY37mwiz NydDYDTxceFI0b+3hfsL/W3qt2JEUaWzWT4RJKe2hzU5PXQx/Bk5Jtm9IosWfGVn1vQ0 1Q6TefnPSdszCPkzIfYZ6EP70BR0fhYw8LAxpciQg9XRts+AmOITJRP0jHwHh8UgkxmG eslDyu3lRhLVwWo8dmflozErkN4M/5eYBYjVSliIGWCy0DPFJzDbjhYrvhbfC9bwfi9X YvC15d/Xo95xREBpx6g3/DKWXEC+gHfyCb3ltalpTcSU20Fukq0YT+vn3gHnJm/bVKSO Bhcg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718611379; x=1719216179; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=E/uPRBGoZoLGCBkeZjiswy/3hxNMDxtntxyuCTrUh0g=; b=dGS+mR6MgRHGC7ETMR6/+YGsXmPrjWakVBjih6xPMIiPHdfK2iIk5kKJcnZLLd5fzW VNiXnbVmshlJ8kwg4U/q8RBb5Hknj/V1zIqBCuJdFZODa15INDZpHaLIJyiQGt8rW+TA pCj1vANLlpTmkmf8QiemJu0soEB/dc/8fnffFUfGmHXWGewMW6x3bvyLMGRbMR+F4fAH DkRpL08ENhuVQi/PypH8vEJMEMKrzFHF1dhWIDV2UnvfKK9uahE8vlVGPLY6xlqkFHqV 7DGCB1q6jN9edEhWuCAHlD2qc8dJU6gXvKQ79xWZgwyj7K751eEaSEU1ZRF8fAq+9SfM WQIA== X-Forwarded-Encrypted: i=1; AJvYcCUTeCpDZz2qZrqKzjxRAAI//vQYebN/vKtG1Fqtz10zq7ciOh+mta5hIYyQuGQmYc4SHzLpC9oCqVVWI9RSEX4upjM= X-Gm-Message-State: AOJu0YzUyTUHSQJvCc8BO8mUSX2NRy34Oe6mfqfVQHVlmJGh5QqwRr1Z YpuDpkchmtN6ln2XahHu6Ax47zO8GEfz33leHuB1Ey1wKNfjduHOxuUqojqIDWTzgXqW5ttmByu cTMPKK0iuHq/cVsnMCznWkQlGrhQ= X-Google-Smtp-Source: AGHT+IGYSxUGXIMfY2cXE9b3X/EvwBDCMKoy45X4SyeBAvYyPm1woNYbiq0ynH0J2BI1QSk6xkRTrg8LkNeVCHvlx/Y= X-Received: by 2002:a05:6102:c15:b0:48d:920a:bd40 with SMTP id ada2fe7eead31-48dae303badmr10876712137.15.1718611378877; Mon, 17 Jun 2024 01:02:58 -0700 (PDT) MIME-Version: 1.0 References: <20240614100329.1203579-1-hch@lst.de> <20240614100329.1203579-2-hch@lst.de> <20240614112148.cd1961e84b736060c54bdf26@linux-foundation.org> <20240616085436.GA28058@lst.de> <20240617053201.GA16852@lst.de> In-Reply-To: <20240617053201.GA16852@lst.de> From: Barry Song <21cnbao@gmail.com> Date: Mon, 17 Jun 2024 20:02:47 +1200 Message-ID: Subject: Re: [PATCH] nfs: fix nfs_swap_rw for large-folio swap To: Christoph Hellwig Cc: Andrew Morton , Trond Myklebust , Anna Schumaker , Steve French , linux-nfs@vger.kernel.org, linux-cifs@vger.kernel.org, linux-mm@kvack.org, Barry Song , Ryan Roberts Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: EDD44C000E X-Stat-Signature: tahkw556xx3kn96m6id8i8mzamjxra7t X-HE-Tag: 1718611379-511906 X-HE-Meta: U2FsdGVkX1/VKUxulbPyEhCraho8XxUh88PcWupF6TtmCG+1dMdBGpZNBwEstetuOxiLb612Ox/zdb/IlcOKPUOEKxDVEtyezsS5sUN2ZG7NXhYjl2D11nwt51fD9jyZN8+YA8ab8z6xYCcRNMlRo0xPfq6Jb+OQM+kKDvFi3TJSlR7a/6snG47hLYQWmbZdQ1h1IfYzbkwmSPZX8Vf5MPEmXDasx6uTDmVSttbpXaEVGCe5l3Z6bMXyuxOtbfdnjPUICVwPtX8BJVXtcDHJHyDwsVrC4tn92wCnBW2cJbjmCQTeXHVGHYGxmlxkucb3xB5HtrzPsq1aWAll/lzfCNf6qfWOczBkNp1il7nPeqjcEkxGF1538AwiZa72DnjiYVmC2S79khcFshjDYpV5a61Vf6eu/LnffLK9iWcDL9q8ltQFAlWDigglM1nhkSHWxVzDxI6uYUkes6lGC9S1Z7uUjhwMNWXJnc1yL9q5L2PHcH9hulpviLV1/OkVwlBGUMevc4tvpSO5fo1k0vEq3al6PKJqTHeKYbt3a6sCZisKr8mlgFiELnuTWx/b1qbPzjlBLh1F9tmnzEGCvJFhCwbGIHJ4MWDEZr1jmtwfYspyogBcLxZD7olvGb8wYy7La0kbQJmv01jdWCuYDEemRYptf+kjxl7fcZ5miYizXT/xLTFug1f65PuiDryhvxlYQyfT5b0RnqQw2kwOLzwxGooKf4Rm3dTa6wjga2NiMTcK/HYQaR0TvYoA6cDAc34UFf01YfE8dJHgrpUZQr6/LAmJd+rK1PphWF1/iKWQVrpsa0oSTM55Hfj9Mwc8pQzsniHVcn8RVZLEV4GoENrum4H3Xvdzu40ejQ9S/RtpI7uiZJ6h+Zmg3+2WgFViqDsxJ112Z9Zq37HjFmiDvipItL1swe+u6TCkC4jpLeku4nOWYyBKfB0gIYlGFVwtKATOPR3Ulvg162K/aTMjZW5 Gw7JoIcY yTn9MwekwYNE6g6vMgU7TkgisKA9wW5EAl0s1SJrD6JtulQNyzqc9OFXba4h1E5gmULOERv9SSzNzssea+1CIsOS7zjNn2X5Ow+9zpE8mZLvKa4t3FH0lhOoPNlMCS67x7bONLxmkcs33KVOEe1Qq82loYn68t3VtAAfWhWebO1NicvOJ+pVVxYnMACQI5++JdYw/aaJMANZzexkqrsdsokWGq7T9qXTQVacTpnunUxKYD0bUYvu2OTLSJPYU66A/pgu8+7zHRCxvS3lI6dtu3+8euUrJG1qVNGJAltEX5s7o0yw0ISuxI3uyerc0uOtu5eGTuQlBcMB/+oL3j45wPQexdg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Jun 17, 2024 at 5:32=E2=80=AFPM Christoph Hellwig wrot= e: > > On Sun, Jun 16, 2024 at 06:23:16PM +0800, Barry Song wrote: > > This is quite unusual. Could you share your setup and backtrace? > > Do this in an x86 kvm vm with 1G of memory: > > mkfs.xfs -f /dev/vdb > > mount /dev/vdb /mnt/nfs1 > > # this requires the following line in /etc/exports to actually export the > # file system: "/mnt/nfs2 localhost(rw,no_root_squash)" > systemctl restart nfs-kernel-server > > mount -t nfs 127.0.0.1:/mnt/nfs1/ /mnt/test/ > > cd /mnt/test > dd if=3D/dev/zero of=3Dswapfile bs=3D1024 count=3D524288 > mkswap swapfile > chmod 0600 swapfile > swapon swapfile I did everything the same as above, but always got failure at the last step to swapon: /mnt/test # swapon swapfile swapon: /mnt/test/swapfile: swapon failed: Invalid argument the mount status: /dev/vdb on /mnt/nfs1 type xfs (rw,relatime,attr2,inode64,logbufs=3D8,logbsize=3D32k,noquota) 127.0.0.1:/mnt/nfs1 on /mnt/test type nfs4 (rw,relatime,vers=3D4.2,rsize=3D1048576,wsize=3D1048576,namlen=3D255,hard,p= roto=3Dtcp,timeo=3D600,retrans=3D2,sec=3Dsys,clientaddr=3D127.0.0.1,local_l= ock=3Dnone,addr=3D127.0.0.1) As long as the swapfile is not on nfs, I am ok to swapon: /mnt # swapon swapfile [ 1150.302641] Adding 65532k swap on /mnt/swapfile. Priority:-2 extents:1 across:65532k Anything I am missing? > > # xfstests-dev is a checked out and built xfstests repo > ~/xfstests-dev/src/usemem 1024 > > The backtrace is at the end of the mail. > > > I'd > > like to reproduce the issue, as the mm code only supports mTHP > > swapout on block devices. What is your swap device or swap file? > > The above swap f=D1=96le on NFS is the only swap in the system > > > Additionally, on what kind of filesystem is the executable file built > > from usemem.c located? > > Doesn't matter. I've tried xfs, ext4 and nfs itself. > > root@testvm:~# ./xfstests-dev/src/usemem 1024 > [ 65.065292] ------------[ cut here ]------------ > [ 65.065528] kernel BUG at fs/nfs/direct.c:144! > [ 65.065724] Oops: invalid opcode: 0000 [#1] PREEMPT SMP NOPTI > [ 65.065988] CPU: 0 PID: 71 Comm: kswapd0 Not tainted 6.10.0-rc3+ #2587 > [ 65.066270] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIO= S 1.16.3-debian-1.16.3-2 04/01/2014 > [ 65.066683] RIP: 0010:nfs_swap_rw+0x3f/0x50 > [ 65.066893] Code: 00 00 74 13 e8 e2 fb ff ff 31 d2 48 85 c0 48 0f 4f c= 2 c3 cc cc cc cc e8 9f f8 ff ff 31 0 > [ 65.067698] RSP: 0018:ffffc900001b79c8 EFLAGS: 00010206 > [ 65.067949] RAX: ffffffff8162afc0 RBX: ffff88800ca24c00 RCX: 000000000= 0000004 > [ 65.068274] RDX: ffff88800ca24c30 RSI: ffffc900001b79d0 RDI: ffff88800= ca24c00 > [ 65.068588] RBP: ffff88800dd710e8 R08: 0000000000004000 R09: 000000000= 0000010 > [ 65.068911] R10: 0000000000035a40 R11: 000000000000001d R12: ffffc9000= 01b7a78 > [ 65.069225] R13: ffffc900001b7db0 R14: ffffc900001b7a68 R15: ffffea000= 03f1c40 > [ 65.069542] FS: 0000000000000000(0000) GS:ffff88803ec00000(0000) knlG= S:0000000000000000 > [ 65.069904] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ 65.070157] CR2: 00005606ddec4dd0 CR3: 000000000365a003 CR4: 000000000= 0770ef0 > [ 65.070472] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 000000000= 0000000 > [ 65.070784] DR3: 0000000000000000 DR6: 00000000ffff07f0 DR7: 000000000= 0000400 > [ 65.071282] PKRU: 55555554 > [ 65.071407] Call Trace: > [ 65.071519] > [ 65.071619] ? die+0x31/0x80 > [ 65.071754] ? do_trap+0xd7/0x100 > [ 65.071915] ? do_error_trap+0x65/0x90 > [ 65.072084] ? nfs_swap_rw+0x3f/0x50 > [ 65.072247] ? exc_invalid_op+0x50/0x70 > [ 65.072420] ? nfs_swap_rw+0x3f/0x50 > [ 65.072581] ? asm_exc_invalid_op+0x1a/0x20 > [ 65.072770] ? __pfx_nfs_swap_rw+0x10/0x10 > [ 65.072966] ? nfs_swap_rw+0x3f/0x50 > [ 65.073125] swap_write_unplug+0x57/0x90 > [ 65.073307] shrink_folio_list+0x3a1/0xed0 > [ 65.073500] ? __mod_zone_page_state+0x61/0xa0 > [ 65.073700] ? isolate_lru_folios+0x2f3/0x400 > [ 65.073902] shrink_lruvec+0x587/0xb60 > [ 65.074071] ? shrink_slab+0x3ad/0x3c0 > [ 65.074239] ? shrink_slab+0x3ad/0x3c0 > [ 65.074408] shrink_node+0x2b7/0x7e0 > [ 65.074569] balance_pgdat+0x291/0x6f0