From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2E24C64ED6 for ; Mon, 27 Feb 2023 10:07:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 59B036B0073; Mon, 27 Feb 2023 05:07:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 54B246B0074; Mon, 27 Feb 2023 05:07:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 413136B0075; Mon, 27 Feb 2023 05:07:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 306C26B0073 for ; Mon, 27 Feb 2023 05:07:01 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id EFCEEA09D1 for ; Mon, 27 Feb 2023 10:07:00 +0000 (UTC) X-FDA: 80512643400.10.2A4FD51 Received: from mail1.bemta37.messagelabs.com (mail1.bemta37.messagelabs.com [85.158.142.1]) by imf09.hostedemail.com (Postfix) with ESMTP id 5E6D6140005 for ; Mon, 27 Feb 2023 10:06:58 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=fujitsu.com header.s=170520fj header.b=koraLTrY; spf=pass (imf09.hostedemail.com: domain of ruansy.fnst@fujitsu.com designates 85.158.142.1 as permitted sender) smtp.mailfrom=ruansy.fnst@fujitsu.com; dmarc=pass (policy=none) header.from=fujitsu.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1677492418; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kn2Vp6WJgOrQzbMBhvCHz2oSn3RJLwTNnYDWT0jl7Cc=; b=Bmcn5KiaAdJpY1jToKD+RtEXP4WKTnlWSCc6KnrUTh24hONQNibgpr+ec+pzOw2Hl23cc/ mANAvoMPq2fB5iO6/2BREyPR8DdO3KPPh14DQdJf8sRtFVqeAbGrqUigOfqDcyaSPJNHwy LwtLk1njsQNTE3R7Voxu5eZSxS5yn3Q= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=fujitsu.com header.s=170520fj header.b=koraLTrY; spf=pass (imf09.hostedemail.com: domain of ruansy.fnst@fujitsu.com designates 85.158.142.1 as permitted sender) smtp.mailfrom=ruansy.fnst@fujitsu.com; dmarc=pass (policy=none) header.from=fujitsu.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1677492418; a=rsa-sha256; cv=none; b=BVCPcP9boc/Faa8lG+XNFWQGOIqT9nTY7c4GnrfLrscUNIFvfjpx1VYO3WGc9gBbeeR2EA 5fqW5VY0yLvYkHuloLYsFCp3LUSfOSkhnlFcVs8ji0WA4k8UiEymmkn8zyP5l/8hXHryKh 1M9roAJ8JKzS5O1xaqVAjTREq+RsEfU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fujitsu.com; s=170520fj; t=1677492416; i=@fujitsu.com; bh=kn2Vp6WJgOrQzbMBhvCHz2oSn3RJLwTNnYDWT0jl7Cc=; h=Message-ID:Date:MIME-Version:Subject:To:CC:References:From: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=koraLTrYlbR95you/2452Wtpd9PYBDutiyMgt9grAnNfYK1/oekpnio+WRVSkFPDu Mv0mToXi9or9cvmKLqXdY3tf9CJxivML3iYq1Bc27xuqMeyjfkC1v/cjvNYU09nAq3 raDN0v3XwZIto+YIg+g6Dj+TXjokJtZWm3C1VcUa4PXhym8MRhLhs5C+bkT28rzZgm jB4mDG0mOtTgvf3lqa0Eaap5+4Fiuax/IpCy+hCk8dO8kKcw9GVQIeuEhOS9lomHDj HXpRlpxft83DnJTf6hn2RaYMK9TJ0HSQ8L6UdE3kjFE2A03L5GVRObwPCN3r8i94/u +8WJDDCzcFLKA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrCKsWRWlGSWpSXmKPExsViZ8ORpLu14U+ ywbQuAYs569ewWUyfeoHRYsuxe4wWl5/wWZyesIjJYvfrm2wWe/aeZLG4t+Y/q8WuPzvYLVb+ +MNq8fvHHDYHbo9TiyQ8Nq/Q8li85yWTx6ZVnWwemz5NYvc4MeM3i8eLzTMZPT4+vcXi8XmTX ABnFGtmXlJ+RQJrxrqTm1kLLolXLGo9wtbAuE64i5GLQ0hgI6PE3M+rGSGcpUwS1xd1sUM42x kl5mzdB+RwcvAK2Encmj6XFcRmEVCVuL7hNFRcUOLkzCcsILaoQLLEsfOtbF2MHBzCAn4SF9a Yg4RFBNQkJk3awQxiMwu0MEls6a6AmL+WUWLfojawBJuAjsSFBX9ZQXo5Bawkbm9mhai3kFj8 5iA7hC0v0bx1NjNIiYSAksTM7niQsIRApUTrh18sELaaxNVzm5gnMArNQnLcLCSTZiGZtICRe RWjeXFqUVlqka6huV5SUWZ6RkluYmaOXmKVbqJeaqluXn5RSYauoV5iebFeanGxXnFlbnJOil 5easkmRmBkphQn3dnBuLPvr94hRkkOJiVR3kVOf5KF+JLyUyozEosz4otKc1KLDzHKcHAoSfA 2lADlBItS01Mr0jJzgEkCJi3BwaMkwqteBJTmLS5IzC3OTIdInWJUlBLn9aoFSgiAJDJK8+Da YInpEqOslDAvIwMDgxBPQWpRbmYJqvwrRnEORiVh3qIqoCk8mXklcNNfAS1mAlq8/eEPkMUli QgpqQamVO9H95ecqvbN/MMgH//96FuBku3lp27qd0TuuHO3Uzp3+1r1Gel/uE77lKy9YypQe0 2zQVtnx9G4MzU5hqHL0h/P64vcf3VOZ05vmlbsNTMV93zX+uwVT0UijrSJZMV58p1Rlr3xfO0 MJY0qY7O43t0epiv+cjBrV87/JcPpcy3ndN+qX+3/2XMTT5xccvAE76aln/a9OResxRcdUTv9 77HINNX5bBGX//i0dHVtidrHp38sa8OOP8wLhTNqpu35N9+OQz9gzd9sVo7fHt8jzgT8/G8jO olJ9LuC5IVIg3dszwI0T3Qtk9DZOXtrgJ2JRN1blc36ZWnFDidPa9js3LpT++vW7ddVvqUkGN xQYinOSDTUYi4qTgQAnarrQscDAAA= X-Env-Sender: ruansy.fnst@fujitsu.com X-Msg-Ref: server-19.tower-728.messagelabs.com!1677492404!204198!1 X-Originating-IP: [62.60.8.98] X-SYMC-ESS-Client-Auth: outbound-route-from=pass X-StarScan-Received: X-StarScan-Version: 9.103.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 30840 invoked from network); 27 Feb 2023 10:06:45 -0000 Received: from unknown (HELO n03ukasimr03.n03.fujitsu.local) (62.60.8.98) by server-19.tower-728.messagelabs.com with ECDHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 27 Feb 2023 10:06:45 -0000 Received: from n03ukasimr03.n03.fujitsu.local (localhost [127.0.0.1]) by n03ukasimr03.n03.fujitsu.local (Postfix) with ESMTP id 83DF11AD; Mon, 27 Feb 2023 10:06:44 +0000 (GMT) Received: from R01UKEXCASM223.r01.fujitsu.local (R01UKEXCASM223 [10.182.185.121]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by n03ukasimr03.n03.fujitsu.local (Postfix) with ESMTPS id 77AB51AC; Mon, 27 Feb 2023 10:06:44 +0000 (GMT) Received: from [192.168.50.5] (10.167.225.141) by R01UKEXCASM223.r01.fujitsu.local (10.182.185.121) with Microsoft SMTP Server (TLS) id 15.0.1497.42; Mon, 27 Feb 2023 10:06:40 +0000 Message-ID: <56e0a5e8-74db-95eb-d6fb-5d4a3b5cb156@fujitsu.com> Date: Mon, 27 Feb 2023 18:06:28 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.7.1 Subject: Re: [PATCH v10 3/3] mm, pmem, xfs: Introduce MF_MEM_REMOVE for unbind To: Dave Chinner CC: , , , , , , , , , References: <1676645312-13-1-git-send-email-ruansy.fnst@fujitsu.com> <1676645312-13-4-git-send-email-ruansy.fnst@fujitsu.com> <20230227000759.GZ360264@dread.disaster.area> From: Shiyang Ruan In-Reply-To: <20230227000759.GZ360264@dread.disaster.area> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-Originating-IP: [10.167.225.141] X-ClientProxiedBy: G08CNEXCHPEKD07.g08.fujitsu.local (10.167.33.80) To R01UKEXCASM223.r01.fujitsu.local (10.182.185.121) X-Virus-Scanned: ClamAV using ClamSMTP X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 5E6D6140005 X-Stat-Signature: j75p1ukmtfrh6qso4ej55pt4bqpo8qpe X-HE-Tag: 1677492418-994947 X-HE-Meta: U2FsdGVkX1+XD7hmvPKkpj/cRT7jnJe206SscAlyh5tQqku494zeFsm2f6IldIwD73ajRq0Gxb7yDMi/1m7K4O+2u+CIXgB4GwWztbFOHQ91VQvbJXoRQXvVjCVmd2dDb6ThwVCPdwIlfh/qaQII2pWlr1wWbCBNA4iioMYn47nPy46tk6uAP0j+wVyAAKYJeRLrfxN6WcaEpePXpc+2qQOjFNJKoa3vJPB4bLChs+4I93cOrF3YM8TZdTWiGVZSYu7juhgfvqh1vB9jFGd32iPq4i61HeXsKvrYK44KezRLemfeeRxe4VB80Gz3KLwIzod4uokCjT/qnwwvLsqQ/Erw7No9g2n4UOwH07dOVJay84IGc/1vJvYK9QSB93jnEI3lFwDH6wm8OdmK65RSqrjql/VWubpdJJNTpL/y/z3sm9xAgftvx5MwVES7JArhkYayvMJU893PDhL01fBLcwNgjSgHnGzlxVrnqo1Oi0IcJ3USSwWIoIxTbBDtj2tusjPjOI57+9QSP6h3TzQ19Aa+mYpfv2krcl1FjB248zIY6AbwPEzI+WWDhk7bWwe5Vv6Qv29OfQsODXIOywhqgATtuC3+6XuQKy6XgxlOFwgUhJKGXzHE2Mp9bFS0Zx49a/RM0Ebr7H3dgAVCNAklIJe+gEqIInj8PNDJvNg2/uT0NuXkft+8RghnI0fde/uCU17SE6bO1tatWdEOkCVxD94+ui2BhaT3JDIOUUgBKNpKhEfVrrY0TxarJIZl53CUOqJKb485n4dTkfmQJB1Y+pfJqGEx3qtbL2J+yhNU3J62YIz5jGFlOTa1Q5TrDTQ7Tiby5oKvZ8BJl8vwW58pFqjkQ+C1PCqMErz6tJv3nsI2lPO6OYGyZRE0D7cbfE27J2jzkPf1RsmQ/t9wLR8LwiieXACX1laWr0KVrD4hRdcwqcJqv8+2YiNdQB0GDxchO9Y8WTVCAhNMf3zONra UTzDZTQz JSFsPDjUYtaRT2wTqTC2uC7mak2zC6ngDnwMtPRb3KSKx+R9KsBD+5RcyHVDfO83aKaTshQIEvDEA6DJgRXeltCJP1MpS22LfIC2f0slKbLf0iGkQ59w2Rg74YpL0HQjq4C/vifWuJMtSbMUt3Sm2noGkIcvQ/hJSzjJFc++PNrpIrNlg1jGE4maeiRLnQCRqLp4XW+c/X8EPl/p0k0SO+tqT6aTf3IybzdRofQZon4ol8z2brGDCpclf2AvLI1RF7nhd+1NqhckLepk9M62CK82VPsTOwTqzyyLk5nfmPN5lS4Mjfj26B7hnH92hgf8AQjjFyMSPBfLSqwW613IOcz2wz/emE5SyPXNWUlG/u996I0OhROqSUxNvHQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: 在 2023/2/27 8:07, Dave Chinner 写道: > On Fri, Feb 17, 2023 at 02:48:32PM +0000, Shiyang Ruan wrote: >> This patch is inspired by Dan's "mm, dax, pmem: Introduce >> dev_pagemap_failure()"[1]. With the help of dax_holder and >> ->notify_failure() mechanism, the pmem driver is able to ask filesystem >> (or mapped device) on it to unmap all files in use and notify processes >> who are using those files. >> >> Call trace: >> trigger unbind >> -> unbind_store() >> -> ... (skip) >> -> devres_release_all() # was pmem driver ->remove() in v1 >> -> kill_dax() >> -> dax_holder_notify_failure(dax_dev, 0, U64_MAX, MF_MEM_PRE_REMOVE) >> -> xfs_dax_notify_failure() >> >> Introduce MF_MEM_PRE_REMOVE to let filesystem know this is a remove >> event. So do not shutdown filesystem directly if something not >> supported, or if failure range includes metadata area. Make sure all >> files and processes are handled correctly. >> >> [1]: https://lore.kernel.org/linux-mm/161604050314.1463742.14151665140035795571.stgit@dwillia2-desk3.amr.corp.intel.com/ >> >> Signed-off-by: Shiyang Ruan > > ..... > >> --- >> @@ -225,6 +242,15 @@ xfs_dax_notify_failure( >> if (offset + len - 1 > ddev_end) >> len = ddev_end - offset + 1; >> >> + if (mf_flags & MF_MEM_PRE_REMOVE) { >> + xfs_info(mp, "device is about to be removed!"); >> + error = freeze_super(mp->m_super); >> + if (error) >> + return error; >> + /* invalidate_inode_pages2() invalidates dax mapping */ >> + super_drop_pagecache(mp->m_super, invalidate_inode_pages2); >> + } > > Why do you still need to drop the pagecache here? My suggestion was > to replace it with freezing the filesystem at this point is to stop > it being dirtied further before the device remove actually occurs. > The userspace processes will be killed, their DAX mappings reclaimed > and the filesystem shut down before device removal occurs, so > super_drop_pagecache() is largely superfluous as it doesn't actually > provide any protection against racing with new mappings or dirtying > of existing/newly created mappings. > > Freezing doesn't stop the creation of new mappings, either, it just > cleans all the dirty mappings and halts anything that is trying to This is the point I wasn't aware of. > dirty existing clean mappings. It's not until we kill the userspace > processes that new mappings will be stopped, and it's not until we > shut the filesystem down that the filesystem itself will stop > accessing the storage. > > Hence I don't see why you retained super_drop_pagecache() here at > all. Can you explain why it is still needed? So I was just afraid that it's not enough for rmap & processes killer to invalidate the dax mappings. If something error happened during the rmap walker, the fs will shutdown and there is no chance to invalidate the rest mappings whose user didn't be killed yet. Now that freezing the fs is enough, I will remove the drop cache code. -- Thanks, Ruan. > > -Dave.