From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BF94CDB474 for ; Mon, 23 Oct 2023 06:42:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A611A6B00A9; Mon, 23 Oct 2023 02:42:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A11116B00AA; Mon, 23 Oct 2023 02:42:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8FFA16B00AB; Mon, 23 Oct 2023 02:42:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 80F556B00A9 for ; Mon, 23 Oct 2023 02:42:06 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 51BB812013A for ; Mon, 23 Oct 2023 06:42:06 +0000 (UTC) X-FDA: 81375781452.03.7F222D8 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf27.hostedemail.com (Postfix) with ESMTP id ADED940015 for ; Mon, 23 Oct 2023 06:42:04 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=ZHI0yAH1; spf=pass (imf27.hostedemail.com: domain of chandanbabu@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=chandanbabu@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1698043324; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VBEgoOQscCwLjvXyY1u3xgPAfNCBgDzRi07hqrXw9R8=; b=qqGIQGKx48FVlqsP2hMFr2uCpLYjfGIHIMrC/1glg7QNzOx4MIeYWwms25Mq/rooxeo1E3 Dk6+ee81rrBax6S8wsBfyaiW4VaC4OMBa7/ECyRyPuZXmciaIeIZwpIKA14ecAWJ5SlWby sNkWFqocb+Fy6thJzqn1cEMxSXxCbl0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1698043324; a=rsa-sha256; cv=none; b=UR9wA4M5B8fGqMyXB20l/KEM9rMx0kBru2Dr5D/0rF031pi20T/R/pqC0fKXmsysAw2f/5 JmG8gSyt2MF2TeQvrxwgpvHqEzjHrxyuqzDPutYem18hRswJcc5aOhSVyOkmYOxK4mI7L7 6REFRFTORI6Qm0dLBYJPulzjxv5t+E8= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=ZHI0yAH1; spf=pass (imf27.hostedemail.com: domain of chandanbabu@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=chandanbabu@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id CC16760AF0; Mon, 23 Oct 2023 06:42:03 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 07E34C433C7; Mon, 23 Oct 2023 06:42:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1698043323; bh=TtUtqC54sB4zN71mLaaG1Nx+Kiivp6Rce9RehCA/AiY=; h=References:From:To:Cc:Subject:Date:In-reply-to:From; b=ZHI0yAH149Uc0iS7zncfy2evBfd0OJyTNLRX7L+HVXKbcq+jW/8+vmSIgUFk9MWTz fimCJY75BT19RMwVDKUPn2e701QXOdv+VWPBBM+kPNo3y1ab/KRasFbaXomIBplmyY yt9Xoaww0tDfzDXkP+77SlV1L/WGPluB3ABKq2XoGPXZuzOdIBpD/+CMXCKz69N8cv w3rWZ6BkmAsK6bwVzZaySGtR1nXt+UfoIA6KNutPJB4+7nb5GvHvzzvmSmSYXdSGUA VzNavw9kH4nff/ZpoC80SSlBgxSBXzdnaumFaQJQEjdBSn2tMimlbyHzhl18alflx8 1SBy10GM61NQQ== References: <20230828065744.1446462-1-ruansy.fnst@fujitsu.com> <20230928103227.250550-1-ruansy.fnst@fujitsu.com> <875y31wr2d.fsf@debian-BULLSEYE-live-builder-AMD64> <20231020154009.GS3195650@frogsfrogsfrogs> User-agent: mu4e 1.8.10; emacs 27.1 From: Chandan Babu R To: Shiyang Ruan Cc: akpm@linux-foundation.org, "Darrick J. Wong" , linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev, linux-xfs@vger.kernel.org, linux-mm@kvack.org, dan.j.williams@intel.com, willy@infradead.org, jack@suse.cz, mcgrof@kernel.org Subject: Re: [PATCH v15] mm, pmem, xfs: Introduce MF_MEM_PRE_REMOVE for unbind Date: Mon, 23 Oct 2023 12:10:10 +0530 In-reply-to: <20231020154009.GS3195650@frogsfrogsfrogs> Message-ID: <87msw9zvpk.fsf@debian-BULLSEYE-live-builder-AMD64> MIME-Version: 1.0 Content-Type: text/plain X-Rspamd-Queue-Id: ADED940015 X-Rspam-User: X-Stat-Signature: 98syj68rpiyoukf4pa9emrhnpwpts9ge X-Rspamd-Server: rspam03 X-HE-Tag: 1698043324-521879 X-HE-Meta: U2FsdGVkX1/uDDbWgqYB0MGN1zeG60Emslnkq8N9VqzAmG8sdie8IrzE52E1dlYxutfA+9xKS5d3xkMhUrxh/du/8NqijEIF0Ld3Fd31aDfnX78cP3+Hi/Y/vHcoYjBQFWhdag7M7I7O4I8mQWhStFs0hSQLHB/7i/gmHSTWIJBMAXeoD/FJf39AVe9pVgp06of4XZSrkUNzqdXfKTCkOQ1gr6GBYnoHhMfSBtwY+ipZ04o4QupPcnuhmcBh+9gzkZ/Vt7l0PwNWDZL3yK8saI5/A+yr14mRWRhOnR7ZB+hUrMwo00rNErVAhzbzYv5eYhdaaf269QuBwIT7vX4gKN5zn3VQqeohsmO6Tzgcgjs1A8k2jmO8FcS5K96MIdkQ38FhNrLwn6oii/VloODo26A3nAOwS8GFQX2uwq/LkZh9WoP69sx4bOKsA4BRw01wn16c3RVeGYTbUsC5fBbOWqH9vVD1IQGK/pxdK5NoXvLZFvXkaA5giihXcR+dOXtbNfS7tvapXD2sCoKIyVRIRO+527KA+tK9Nnvm3vknfB8TNvWX+TNRtWPE7KfrxkLEMATpwZDQ+RFNliA3IfiC7SwLQ4v1c5DP041uv5XF+I+nA9GtRfGeW4N1kBEPFDlCEUA3KvqnbrtL2P6uhsPkglLH/1KejOdFkHB/chrMYfSnwJcp8b4QLhfU4pMUS0iJFct6omaTqagXVaDlBrSRfLxF4uuFitXUV9r24sY058Gp7uUKca9Ped/nMTgQFnBs3yPnP2awXarNBml1u6onjw6uxHpRoAigjrAwUZMhhuYPLB2fa5vCt1irCf8MumLMmVTf0pQIQufCWn0LxDLJrRSSIjVtKyKUV3QlRpkna8EES9APhj1jQ/vDzweg7ox9m5Ag4w5ak1pjr9qKzdLXg+iynkn4vQu8j3M/5xD/s1z7Taz0w5GPfrE0NWADfZ/+YbU3F3JB9o1qm8qTD5T j5HRnuHF 65dQOTXrt7ftPw+DOXf3JVD2RfKkowgYbZaqia61kB6+FFWDsruBX5bGqZIyddSC6Ylq9xd6uE8ITONmWusHzmdYwZKtZKH9kb4QIDr96rusVQwqG9feq/noIWtcwdjYwL84+znsrsKSSjVzZyB1S9349yHZYyC4yf4vrHzTJbDf5TdqJiPBsOna9hD8QenaQCbgCuZVdLiiA+9oJA7IuVlTAGvUODoyG6gQ9Wc9pz6605Z42yj0Q+fpQoVlJUeYPh264wh2fyKiddIqC3v5ScKUZJ5il7TuXxvAx+sauyphPI9AlQMHS1LSWg02GueiVIQKBRMu46YqRAcc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Oct 20, 2023 at 08:40:09 AM -0700, Darrick J. Wong wrote: > On Fri, Oct 20, 2023 at 03:26:32PM +0530, Chandan Babu R wrote: >> On Thu, Sep 28, 2023 at 06:32:27 PM +0800, Shiyang Ruan wrote: >> > ==== >> > Changes since v14: >> > 1. added/fixed code comments per Dan's comments >> > ==== >> > >> > Now, if we suddenly remove a PMEM device(by calling unbind) which >> > contains FSDAX while programs are still accessing data in this device, >> > e.g.: >> > ``` >> > $FSSTRESS_PROG -d $SCRATCH_MNT -n 99999 -p 4 & >> > # $FSX_PROG -N 1000000 -o 8192 -l 500000 $SCRATCH_MNT/t001 & >> > echo "pfn1.1" > /sys/bus/nd/drivers/nd_pmem/unbind >> > ``` >> > it could come into an unacceptable state: >> > 1. device has gone but mount point still exists, and umount will fail >> > with "target is busy" >> > 2. programs will hang and cannot be killed >> > 3. may crash with NULL pointer dereference >> > >> > To fix this, we introduce a MF_MEM_PRE_REMOVE flag to let it know that we >> > are going to remove the whole device, and make sure all related processes >> > could be notified so that they could end up gracefully. >> > >> > This patch is inspired by Dan's "mm, dax, pmem: Introduce >> > dev_pagemap_failure()"[1]. With the help of dax_holder and >> > ->notify_failure() mechanism, the pmem driver is able to ask filesystem >> > on it to unmap all files in use, and notify processes who are using >> > those files. >> > >> > Call trace: >> > trigger unbind >> > -> unbind_store() >> > -> ... (skip) >> > -> devres_release_all() >> > -> kill_dax() >> > -> dax_holder_notify_failure(dax_dev, 0, U64_MAX, MF_MEM_PRE_REMOVE) >> > -> xfs_dax_notify_failure() >> > `-> freeze_super() // freeze (kernel call) >> > `-> do xfs rmap >> > ` -> mf_dax_kill_procs() >> > ` -> collect_procs_fsdax() // all associated processes >> > ` -> unmap_and_kill() >> > ` -> invalidate_inode_pages2_range() // drop file's cache >> > `-> thaw_super() // thaw (both kernel & user call) >> > >> > Introduce MF_MEM_PRE_REMOVE to let filesystem know this is a remove >> > event. Use the exclusive freeze/thaw[2] to lock the filesystem to prevent >> > new dax mapping from being created. Do not shutdown filesystem directly >> > if configuration is not supported, or if failure range includes metadata >> > area. Make sure all files and processes(not only the current progress) >> > are handled correctly. Also drop the cache of associated files before >> > pmem is removed. >> > >> > [1]: https://lore.kernel.org/linux-mm/161604050314.1463742.14151665140035795571.stgit@dwillia2-desk3.amr.corp.intel.com/ >> > [2]: https://lore.kernel.org/linux-xfs/169116275623.3187159.16862410128731457358.stg-ugh@frogsfrogsfrogs/ >> > >> > Signed-off-by: Shiyang Ruan >> > Reviewed-by: Darrick J. Wong >> > Acked-by: Dan Williams >> >> Hi Andrew, >> >> Shiyang had indicated that this patch has been added to >> akpm/mm-hotfixes-unstable branch. However, I don't see the patch listed in >> that branch. >> >> I am about to start collecting XFS patches for v6.7 cycle. Please let me know >> if you have any objections with me taking this patch via the XFS tree. > > V15 was dropped from his tree on 28 Sept., you might as well pull it > into your own tree for 6.7. It's been testing fine on my trees for the > past 3 weeks. > > https://lore.kernel.org/mm-commits/20230928172815.EE6AFC433C8@smtp.kernel.org/ Shiyang, this patch does not apply cleanly on v6.6-rc7. Can you please rebase the patch on v6.6-rc7 and send it to the mailing list? -- Chandan