From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69839CDB474 for ; Fri, 20 Oct 2023 10:01:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A838C6B0110; Fri, 20 Oct 2023 06:01:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A33216B0111; Fri, 20 Oct 2023 06:01:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 922506B0112; Fri, 20 Oct 2023 06:01:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 83FA56B0110 for ; Fri, 20 Oct 2023 06:01:39 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 397B2B5CD0 for ; Fri, 20 Oct 2023 10:01:39 +0000 (UTC) X-FDA: 81365397918.02.BD53268 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf20.hostedemail.com (Postfix) with ESMTP id 845F21C0029 for ; Fri, 20 Oct 2023 10:01:37 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Ukw+27nf; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf20.hostedemail.com: domain of chandanbabu@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=chandanbabu@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697796097; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=FiePfnRLuR0IIWg+4a4viDTUWgod1xJF7bFbKx12Qzs=; b=h2zHrLyhFgfPrcJTiSaa110YhPmgpL66H1cvbzFNaBZHP2veTIAccTgg+toCuLJ2UV7MCI 9c0l3gH/OYRAtFR9AUZ30CHKAiNDhdvWx3RIv2wzVdnE96ypck4HTb8aJ30jP3X/yGmqcg hYkw6JCjdhSiXh9RfwXju/YCApvNEt8= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Ukw+27nf; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf20.hostedemail.com: domain of chandanbabu@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=chandanbabu@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697796097; a=rsa-sha256; cv=none; b=VVKR846ql5C9tDpJjLPzW2pT6UY7ZOoZZWr+5AH4pGa2atdXh9lkQd2iq6d3NvqV7DgXtC Du7vHX5GA1Hl452wI098VZkHOmkNqsybAP6yb7XMeevHlQKqXYQbJETcrA826UAfEgQEZU 2B++gMrrosTasUL+vMwFEbZSVhW8X+8= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 6AE0361E1F; Fri, 20 Oct 2023 10:01:36 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7B358C433C7; Fri, 20 Oct 2023 10:01:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1697796096; bh=lzHAk3dExR6FwsdPVTRy5vF1LS3VUKmX2W/bDr/QLoI=; h=References:From:To:Cc:Subject:Date:In-reply-to:From; b=Ukw+27nf/Tijp3Zq0ul5lRnbqUtAFqAtg7gINa2ZueIF8VfPgtXoJRXPOMGlA+FoF b1zPl60AUrzHMljNOukoj6fjRA1Q1hKWRwEJR/6N4LSDkZOD5KRmHFVWxsHvPYilr0 VZQPeycZUgdsk7rPYT+uVXKbhUfj4/UzgtpIF5h9xTVSno4/SBUPgd7q11H8gyw+ju Zs8tUWS92TYsG1zWu/YCb8fmPn1SHheBlow6SR0TjmqazyLKZqiFzvGiNUFqwrq9Cb w9cPlWTpuBYioFlAzXz47nDOilJi/Ilz1a9wMmjnpBOQj9k1idf4TGMdlw6uDGPUZf WN0YDjjnQp7YQ== References: <20230828065744.1446462-1-ruansy.fnst@fujitsu.com> <20230928103227.250550-1-ruansy.fnst@fujitsu.com> User-agent: mu4e 1.8.10; emacs 27.1 From: Chandan Babu R To: akpm@linux-foundation.org Cc: Shiyang Ruan , linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev, linux-xfs@vger.kernel.org, linux-mm@kvack.org, dan.j.williams@intel.com, willy@infradead.org, jack@suse.cz, djwong@kernel.org, mcgrof@kernel.org Subject: Re: [PATCH v15] mm, pmem, xfs: Introduce MF_MEM_PRE_REMOVE for unbind Date: Fri, 20 Oct 2023 15:26:32 +0530 In-reply-to: <20230928103227.250550-1-ruansy.fnst@fujitsu.com> Message-ID: <875y31wr2d.fsf@debian-BULLSEYE-live-builder-AMD64> MIME-Version: 1.0 Content-Type: text/plain X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 845F21C0029 X-Stat-Signature: ot6jfztwibf6hy15a7jwbf4j1jqqmos8 X-HE-Tag: 1697796097-941327 X-HE-Meta: U2FsdGVkX181VqNhUY93K7BPYDkdmo8v+Wiid3HpXpW0NPriwxgphv3Lyy42yVbWg+Z8NON58AJZYkrHm1nlyhzszElmvHf1IBYSrEU+Y/JLngk/U8mc8Bwe5Td4CQ7rZAeY2wMoGhEodCMkw/j+VvDbPS2Lh3B4BtJpWqCbJr5JIe8QJ1Q+sSD5YUISKeJT5nVAXQxWU+joBe9MI3vuHspwI6XQMc8eLuAxn/PGW5/aPQyh4PZO3lKyBxiYfGd4PyXJ7EXz7RhK0Ia5pV8YgqMr9Ufb/1wJhlFQMaVVO53l42acJrcB3tYqn9YFhLKVMc/5utcSft278hzGiGVZ+XkeJVbvdeKXVwTM0QfCB4ivd3zgHfytwtgB0aTC/Fx60G/KXm9+hCueEEI1jYU0zgSnMHPIFkmkj9yyyXcn6QFd7/Vpmcr9qyJJps3Fs9K06UW3wFqCVqkabGBktc1m+43OT1p1YlW1SbidKixPGB2fwfyLX6GZTpsuiJ01pep8qMC6hhkfWS3wlUzooRpIgSHQN2rjQ5tnGfNnp5IWyczQXUg/KFiY04YmnsrvoOThEljAWK39K+HcrFHxElO/6wgiTAa+7/Q0ySr0eKyNIslPkbO0s0Psld6O7zl13GDsNDswHiyvpn8B2Xm6ll+/LHv3LwMrVIWIOzayubxPC+TO5gE4/mZYy5bilfQTApaJbnkJYzL8OgaXxIeGe28qGejnxy5KdwEGkzKEmbokgjnHbnTz6V5zEmzMgMOxSBHqj90jKTjCNSD/nzxttu4midmJcOkLmIU8TueGDPTdkgF/5uCdg9I7SSTFn30oI+/5s92NDsvSl8sDT7INnloz8FmraQ6ZsEMWX8fjbzzVgRJCnb17y4R2DA75LU3P6+ZXtdDm/6l6buYLMevnHzmxayr/ebLXne8M0xbh5kvDepG+Soj+5KNJtukvsHDFOgJM+8fHP8JkX22SyU/4X8i IGYLzkwl He+rBFUe82x/KwERlPhH9qRtAPVsrawb+xVQGlIe8GvuiWVcLCTE3uK9+QmoHussYBSuE0DqH3ffCjNyMrky0aJDxK88RTW3RKwVy6A04X/L8u0fecujyI0kOemhqeiI96GecKv34P0s+5ax5b53aZDMQ6uOCmm4ubQsVCCIY1ubIrn2E6BFMGCeTYkCWZMz2Rm4EFtlxPbvFwiuToXzbgZacrGFp9lZ3OI311j2CO+jXDPe6shuZvKOtTv6r5XapRwwhVthAI7zhXi8/mQbktOWDSjSNxlZ6gP3SJOIz9m98QR4e7399sn9qwRWP9uAWtKjstGmKx2FyMOo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Sep 28, 2023 at 06:32:27 PM +0800, Shiyang Ruan wrote: > ==== > Changes since v14: > 1. added/fixed code comments per Dan's comments > ==== > > Now, if we suddenly remove a PMEM device(by calling unbind) which > contains FSDAX while programs are still accessing data in this device, > e.g.: > ``` > $FSSTRESS_PROG -d $SCRATCH_MNT -n 99999 -p 4 & > # $FSX_PROG -N 1000000 -o 8192 -l 500000 $SCRATCH_MNT/t001 & > echo "pfn1.1" > /sys/bus/nd/drivers/nd_pmem/unbind > ``` > it could come into an unacceptable state: > 1. device has gone but mount point still exists, and umount will fail > with "target is busy" > 2. programs will hang and cannot be killed > 3. may crash with NULL pointer dereference > > To fix this, we introduce a MF_MEM_PRE_REMOVE flag to let it know that we > are going to remove the whole device, and make sure all related processes > could be notified so that they could end up gracefully. > > This patch is inspired by Dan's "mm, dax, pmem: Introduce > dev_pagemap_failure()"[1]. With the help of dax_holder and > ->notify_failure() mechanism, the pmem driver is able to ask filesystem > on it to unmap all files in use, and notify processes who are using > those files. > > Call trace: > trigger unbind > -> unbind_store() > -> ... (skip) > -> devres_release_all() > -> kill_dax() > -> dax_holder_notify_failure(dax_dev, 0, U64_MAX, MF_MEM_PRE_REMOVE) > -> xfs_dax_notify_failure() > `-> freeze_super() // freeze (kernel call) > `-> do xfs rmap > ` -> mf_dax_kill_procs() > ` -> collect_procs_fsdax() // all associated processes > ` -> unmap_and_kill() > ` -> invalidate_inode_pages2_range() // drop file's cache > `-> thaw_super() // thaw (both kernel & user call) > > Introduce MF_MEM_PRE_REMOVE to let filesystem know this is a remove > event. Use the exclusive freeze/thaw[2] to lock the filesystem to prevent > new dax mapping from being created. Do not shutdown filesystem directly > if configuration is not supported, or if failure range includes metadata > area. Make sure all files and processes(not only the current progress) > are handled correctly. Also drop the cache of associated files before > pmem is removed. > > [1]: https://lore.kernel.org/linux-mm/161604050314.1463742.14151665140035795571.stgit@dwillia2-desk3.amr.corp.intel.com/ > [2]: https://lore.kernel.org/linux-xfs/169116275623.3187159.16862410128731457358.stg-ugh@frogsfrogsfrogs/ > > Signed-off-by: Shiyang Ruan > Reviewed-by: Darrick J. Wong > Acked-by: Dan Williams Hi Andrew, Shiyang had indicated that this patch has been added to akpm/mm-hotfixes-unstable branch. However, I don't see the patch listed in that branch. I am about to start collecting XFS patches for v6.7 cycle. Please let me know if you have any objections with me taking this patch via the XFS tree. -- Chandan