From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6284FC32771 for ; Wed, 21 Sep 2022 14:03:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9418A940007; Wed, 21 Sep 2022 10:03:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8F0B96B0073; Wed, 21 Sep 2022 10:03:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7B8F7940007; Wed, 21 Sep 2022 10:03:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 6D5DC6B0071 for ; Wed, 21 Sep 2022 10:03:31 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 43EB61C521C for ; Wed, 21 Sep 2022 14:03:31 +0000 (UTC) X-FDA: 79936260222.08.0836FF4 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2047.outbound.protection.outlook.com [40.107.220.47]) by imf22.hostedemail.com (Postfix) with ESMTP id 3B6C9C000D for ; Wed, 21 Sep 2022 14:03:29 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NRNUtRSsHE3adKUax27AF0zEJjC73VaivZ/vqaqlP1FnN47iNCnR7phjdcjtxkHr81KBtFbUuBYgmhPaw/npCRsJsbTWIZaftm0o2nU8nGbtSjIppSnJmTVGQMto/1ISbQySApOtLv3JZxqRaS62XJuoZT9Was0//ap+pWLO6VpOmBit0OkZlnHfNW5w5XbsrFlm/yCLPOo8qrkVyEaPwBwxi9iOelSgX2cZiAst7c/yIUFKD7/avWcyc2RmJrN1rGVWcxGriazQuy6vJuAMvldMZaltubzqMo/ijBBMY0/tFTk8/hqqnXoy7DsZLwy6tYw6S0T57vlWzS7XjkYBoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Yvqs4A/mqsRrXyiHP3KDP4QYwI5/VBZJY9qmgn6n8C0=; b=Qfynx3SujWj0qaqThfiFnm2P91I7rYiK+3wPaSnbp7JiZOszPDkozqNT/riCWfnECu1/RKh88UdJZySCCBi3/nzaTfvTmWwfczjeSyVC2itnC+0ZOH3qHEcWRYEw7PFx7yMNZkgt3/189qGh3KbrI79bM2W9mXOGadNnqTCi4leqMWOz72oDWf3qkUp1l49kG0r4HtGh6VQwfahf9v74fgpmfgdDKPwv7nc5up+o08yMkqBEPApuFKFQHlZl9MrTZiVq+lOgXQhdm64z9HDnC5w6blgiNW36soJndNx9t6yt0j+WinW4zDEsHi542CsZ+e9Q9noVrA5B3waAJJV9/w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Yvqs4A/mqsRrXyiHP3KDP4QYwI5/VBZJY9qmgn6n8C0=; b=CO2YhnQFL4/oIwDEgBk2Q4/gfn3uk4sZY6OnzbJpHj/984P1gvOM8quLOyuil9njksaoCL5Zsx2ncC2muyK0q+tUmnghprwzdK7AqlmlUEKS807YIA19e3ciX8whPuEng7Mxs0Q0/sSUGZL+6FHoirip7vRhyZl4XMvuHujC9E9SJUtKQ3FG6iyzp+Hr7WETRyZEro2Y/titEkbKRjMTpf900PczBK7fy0qwXJ2jIpDOdTDcvAr5K9JINDz8/yCj4wqWSpC5CXm3+OdfoSb+0pdOpIEwl5BPSKP6s4IZCLVDbK4xXG4mmJSZaG7K7fr+khRHAHJw4E8RKxV5+V43fQ== Received: from MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) by SA1PR12MB7443.namprd12.prod.outlook.com (2603:10b6:806:2b7::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5632.18; Wed, 21 Sep 2022 14:03:26 +0000 Received: from MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::462:7fe:f04f:d0d5]) by MN2PR12MB4192.namprd12.prod.outlook.com ([fe80::462:7fe:f04f:d0d5%7]) with mapi id 15.20.5654.016; Wed, 21 Sep 2022 14:03:26 +0000 Date: Wed, 21 Sep 2022 11:03:25 -0300 From: Jason Gunthorpe To: Dan Williams Cc: akpm@linux-foundation.org, Matthew Wilcox , Jan Kara , "Darrick J. Wong" , Christoph Hellwig , John Hubbard , linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev, linux-xfs@vger.kernel.org, linux-mm@kvack.org, linux-ext4@vger.kernel.org Subject: Re: [PATCH v2 10/18] fsdax: Manage pgmap references at entry insertion and deletion Message-ID: References: <166329930818.2786261.6086109734008025807.stgit@dwillia2-xfh.jf.intel.com> <166329936739.2786261.14035402420254589047.stgit@dwillia2-xfh.jf.intel.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <166329936739.2786261.14035402420254589047.stgit@dwillia2-xfh.jf.intel.com> X-ClientProxiedBy: MN2PR16CA0063.namprd16.prod.outlook.com (2603:10b6:208:234::32) To MN2PR12MB4192.namprd12.prod.outlook.com (2603:10b6:208:1d5::15) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN2PR12MB4192:EE_|SA1PR12MB7443:EE_ X-MS-Office365-Filtering-Correlation-Id: ef07b360-34d7-466a-997b-08da9bda0dd9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: IepXeZcPeBxe0vN344WDYmIlOhi4+OrkVOGWyhTKo2Wb8sQV7ZLa9VqGERzISS9NJTYYz9qjrmanjoUlF0eWsuPww0uraLyzZN5ciCOcvmTASPzK7AuiKQyiBAdldyLXifZVM+7M1/wGKBgERxqHfggBsCQFo9FRSzht9WJpujoROU3dLgkH4XGLROXH/UQ1tcLSN3XzY8un7IJYnsETLGmYiv8bTgFTle4K7WDHM4Y0x+l//Ztzq4GMvm06YX0QyedKrBU4oughAaywKRqLo6oB0cUBsV1n21TyRfWwVxhbfs0Cnw0iO4YrCM5z3JKwd417f6fMFdXGCG5zw94giuxJCKKhShu3FL6Zmfoh0NwV5xe9QNQGDubsZQqal23PDxN5KVCkam0c2rK147CxKTDwmXpcEGWFGfoZdyYquzdZJLIX74qL5/l2RbA3eNEfUhzC/zBhJa2mlDY6xH41/9z8orgz23Z4oykJDNSBKdhtwmftKUuiVZWeB7oABE3qf3Hm2IDdNJq5bS8A1gNX/nAGEP+y2lBrQ3XtRVbRoJZ12r1uM+ISDkQnhmiefJSAHcmmEWdaAbHP78PDD2mlfUinMEcZnALMdw3w+FWhRvfBaRazLSrWjFT4YTKYNbx6rRcq/yUTNsHw+CrSUTrzA8i44DNtfhv1KzpD5GemRHkfG5yOq94BNqNqJqpXwsJD6wqaibVmHhJ83jgVQEzmeA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR12MB4192.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230022)(4636009)(396003)(136003)(366004)(39860400002)(376002)(346002)(451199015)(41300700001)(186003)(83380400001)(5660300002)(38100700002)(2906002)(54906003)(66556008)(6916009)(478600001)(316002)(8936002)(7416002)(6486002)(36756003)(66946007)(2616005)(6506007)(66476007)(4326008)(8676002)(86362001)(6512007)(26005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?k9NNNFW/EAQBgUM+DK1oZp/HQjNtZEdxNvhTSwp60nPL0OShlIftoM16BFFE?= =?us-ascii?Q?TxnxyThJ36JOTekmALC4ppTqvR5oEDEKHFXCT6jQO1eOpyq9iSW/d4ouw8Fe?= =?us-ascii?Q?0PAmhJ9H7vR5mS6WTzueijsBJmt0t8t+VH6Lbhx1oMXNp+RTZWVq0BPhSZhc?= =?us-ascii?Q?dGmZGlYYnJ3oxyh0fETembdM2v1acxKrI+a2jP74fre+AjH2nWc63uWDyMzt?= =?us-ascii?Q?Vr+mvRnuWECH29dwG5buGeL2N11ra6GgkKL3VhyVyPLm1ahr+QUuklLGVaz5?= =?us-ascii?Q?DhBE9GyZC5d/kAJjW+fI/kBzZwbux/GOL/+rwtuwTmWyh24Qroyz9OFkyKEK?= =?us-ascii?Q?V5iP7xz9H6cU6ed1n5Yi2HPKGrFT53ufOiv+EoEmFQr2HYcQgID3xTHQKQg+?= =?us-ascii?Q?7BP07zl0DMDl2JG2AFfU7MUrHdCkGEDc0hEQfDGQhzHJ6Tq3X7LMPCRkJ4BS?= =?us-ascii?Q?0lhOQ/0cyYBEE17TOXdq1NuP4v4QCS2suzc6oo6zG28GWkh2VlhwVgtJmXZf?= =?us-ascii?Q?VsaCUFKRva+CMj/QFI0pb78yLb7myLfdsGEykpGh98MFFqCwHfBOr2Xo8z8H?= =?us-ascii?Q?d/F4Zzw+CAanTuriFCGh0TgVGZwuNU84qPgtE2drXp6VhxMl22KWyCLOS75t?= =?us-ascii?Q?R3ngl1cJUW4XZv39f8uZdu4o6eb/6B1C/weAdUx7rfv6nnIqiY/eUUsFyYLd?= =?us-ascii?Q?FKxgLLQpLJSPgKe9TJQZgvyV8o6ig1DbwKbu5iPIQKYCJD5eJ6ziJainoQUH?= =?us-ascii?Q?E8aO0YtDfhpOq8+MGGW2xGVCtLI2iUZ2jKULPcUobhlvLIfHNA/jQMbEkIKl?= =?us-ascii?Q?gLDTy8EC7t2jwE/oTY5oETy9mvTmBGKh4HpjTtcB+3CRzSb9BUbunixBXt0w?= =?us-ascii?Q?lQh8xciOn8SPFm59VVPYXW7Mf4ZJBN0JdGr24q/88+uiR1gHlvtTYy63RkH3?= =?us-ascii?Q?1SsFIlgZK9lcOsTmsiskbbJF5AmerKd1c7MFFeilscSYMGARLtiKAw5N9b7l?= =?us-ascii?Q?j9swqAF7+AMuIcFZU9mV8mZoTIOkg6GO7hnHjMXq/+BvASIW2iiIF5BOEMHH?= =?us-ascii?Q?m+55PcOf1U1YTI5zBs7vLcDqFUkNpKYcSSsI9Gyt9bMGhVL1zbvNf5g/riWJ?= =?us-ascii?Q?BBLNrtCO9AzZJFJXr4n/krBYxx9EQEqky5igL1xi2AkqlgofaTLkWdR7wRSf?= =?us-ascii?Q?xEeNZcVy9tPAOWOI3wXEuaA/I4VdrbkgJg0jZvfzCqAhM+/wCJHzl0J4vodE?= =?us-ascii?Q?bAbjSDVOZuGJiR8BCi4SA6L8DX2pMA5Qvi59AJt6kgWcH+AfKm7WnRYtuwbB?= =?us-ascii?Q?wUmx1H+K0XqJ4J+twHUjCVYsoWsIlKsDC8KWwdI6CJTBMCfkeHi0Z3qC2UaU?= =?us-ascii?Q?KziZWfY0H+XlGiLnKyFCculZToSoNuwb5GRpPT0yOD/8XOedwX5KgM5f3xy0?= =?us-ascii?Q?5LsiJBB/K9rqbS0Dq7CHC4++i7w1rRTfBvrBQzrejdm6SgwqhnmaOf4pT6cw?= =?us-ascii?Q?y0F95elXbGSdZh45plpHGDm4q89adjNA9PgnZ4LsxwStHbsi1wCOB5hZYIOA?= =?us-ascii?Q?iVasPDqmkXygYMG7r+dzNQAtmf5lv1NR9rBg3o7j?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: ef07b360-34d7-466a-997b-08da9bda0dd9 X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2022 14:03:26.5618 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 5o2QvWsQVMxBoGhUx4ypqUUMg+akEqEACasop4laEQ1pKWsE2nf1wb98da+kE7LC X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB7443 ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1663769011; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Yvqs4A/mqsRrXyiHP3KDP4QYwI5/VBZJY9qmgn6n8C0=; b=4u3sgBx9JjCSYjrB0X60AptYXVTZiEqkczSnDCOvG6NJNL7yftLYVm3TkpQmRhKwpHgJbW wz5ClHv1aGIvp4ATYpd9eUmYNBICRh3HNd+Myof/eSpYvkJp+bbwxLmwGqA9SGz/2ipMnK EI86oQL5S5/TOZpKkrYNyYaLUMTD2As= ARC-Authentication-Results: i=2; imf22.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=CO2YhnQF; spf=pass (imf22.hostedemail.com: domain of jgg@nvidia.com designates 40.107.220.47 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1663769011; a=rsa-sha256; cv=pass; b=XY1dipU4ywcUX46ioRJwqzax+J9YMx2oZlOAZV265sfn/72js8m7K3AU8ctBqdTwT/DQN2 9lMU7Le5yC9wbUbLjbMa6CgoYNiDjNto+SjBW2lZTlwIJBggcXfq9DN+QduaYRStMObeA9 meZQGhyc0fRxMlEQC5KJCnPXRf/J7mE= X-Stat-Signature: 8n5bd3aikuf6zzra7xqk81q9iate5cqp X-Rspamd-Queue-Id: 3B6C9C000D X-Rspam-User: Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=CO2YhnQF; spf=pass (imf22.hostedemail.com: domain of jgg@nvidia.com designates 40.107.220.47 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com X-Rspamd-Server: rspam09 X-HE-Tag: 1663769009-235922 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Sep 15, 2022 at 08:36:07PM -0700, Dan Williams wrote: > The percpu_ref in 'struct dev_pagemap' is used to coordinate active > mappings of device-memory with the device-removal / unbind path. It > enables the semantic that initiating device-removal (or > device-driver-unbind) blocks new mapping and DMA attempts, and waits for > mapping revocation or inflight DMA to complete. This seems strange to me The pagemap should be ref'd as long as the filesystem is mounted over the dax. The ref should be incrd when the filesystem is mounted and decrd when it is unmounted. When the filesystem unmounts it should zap all the mappings (actually I don't think you can even unmount a filesystem while mappings are open) and wait for all page references to go to zero, then put the final pagemap back. The rule is nothing can touch page->pgmap while page->refcount == 0, and if page->refcount != 0 then page->pgmap must be valid, without any refcounting on the page map itself. So, why do we need pgmap refcounting all over the place? It seems like it only existed before because of the abuse of the page->refcount? Jason