From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC9F5C04A95 for ; Fri, 23 Sep 2022 19:08:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4750680009; Fri, 23 Sep 2022 15:08:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3FDDE80007; Fri, 23 Sep 2022 15:08:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2783580009; Fri, 23 Sep 2022 15:08:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 128B480007 for ; Fri, 23 Sep 2022 15:08:46 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id B3355160EAF for ; Fri, 23 Sep 2022 19:08:45 +0000 (UTC) X-FDA: 79944287010.26.8DFDF71 Received: from ale.deltatee.com (ale.deltatee.com [204.191.154.188]) by imf04.hostedemail.com (Postfix) with ESMTP id A16894002D for ; Fri, 23 Sep 2022 19:08:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=deltatee.com; s=20200525; h=Subject:In-Reply-To:From:References:Cc:To: MIME-Version:Date:Message-ID:content-disposition; bh=ZQmZb+lXFhX7pnMcsk23AOR4qV5DlMOOudYTRaQuI38=; b=iyj5atjf9mY5DRHI6pWs1X4Bi6 i7VWXXFHJq6mQF3UhKDCt1azxiYmU/3In5EAKRjGznKha0IhkHuKd/7hnByYfIMa0A10xryy3cNSI 9/HmYFmTyE4XR1neliZtOrRXXnkaJ9R21hyg32c9fhQFArWvSxFVueJNleIU1j2erseIiSoUWDXbG tOhWjlXnaPU8ESbjgzqPj71YG3maPzlUvgOVWPqFkAVKrS7NJZv9TfQKrhcACp5dMGStbB6KGjxO+ eKaMDd7tJjeaipjxYkgs+FEV1joq4uNiT2sAhEOHHWDyXHWLYIghjVmN8Fujqsk6/dLFSMh9Fta85 HjoSW77g==; Received: from s0106a84e3fe8c3f3.cg.shawcable.net ([24.64.144.200] helo=[192.168.0.10]) by ale.deltatee.com with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1obo2O-008xXP-NI; Fri, 23 Sep 2022 13:08:37 -0600 Message-ID: <64f8da81-7803-4db4-73da-a158295cbc9c@deltatee.com> Date: Fri, 23 Sep 2022 13:08:31 -0600 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.13.0 Content-Language: en-CA To: Jason Gunthorpe Cc: linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, linux-pci@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , Greg Kroah-Hartman , Dan Williams , =?UTF-8?Q?Christian_K=c3=b6nig?= , John Hubbard , Don Dutile , Matthew Wilcox , Daniel Vetter , Minturn Dave B , Jason Ekstrand , Dave Hansen , Xiong Jianxin , Bjorn Helgaas , Ira Weiny , Robin Murphy , Martin Oliveira , Chaitanya Kulkarni , Ralph Campbell , Stephen Bates References: <20220922163926.7077-1-logang@deltatee.com> <20220922163926.7077-2-logang@deltatee.com> From: Logan Gunthorpe In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-SA-Exim-Connect-IP: 24.64.144.200 X-SA-Exim-Rcpt-To: jgg@ziepe.ca, linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, linux-pci@vger.kernel.org, linux-mm@kvack.org, hch@lst.de, gregkh@linuxfoundation.org, dan.j.williams@intel.com, christian.koenig@amd.com, jhubbard@nvidia.com, ddutile@redhat.com, willy@infradead.org, daniel.vetter@ffwll.ch, dave.b.minturn@intel.com, jason@jlekstrand.net, dave.hansen@linux.intel.com, jianxin.xiong@intel.com, helgaas@kernel.org, ira.weiny@intel.com, robin.murphy@arm.com, martin.oliveira@eideticom.com, ckulkarnilinux@gmail.com, rcampbell@nvidia.com, sbates@raithlin.com X-SA-Exim-Mail-From: logang@deltatee.com Subject: Re: [PATCH v10 1/8] mm: introduce FOLL_PCI_P2PDMA to gate getting PCI P2PDMA pages X-SA-Exim-Version: 4.2.1 (built Sat, 13 Feb 2021 17:57:42 +0000) X-SA-Exim-Scanned: Yes (on ale.deltatee.com) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1663960125; a=rsa-sha256; cv=none; b=6XcQe1RX4S0Pni+dQU1qXKrSx857NF+F/GZHTJGS6N42MM9j79mG8FlJrl9kcZ4ZoXXXlm N9bW+ujwg1oQSKTDxPditHk51aktXHZI3mOT+oyBULRHTg3+QPOewnCzLB/jDroGjD7CoA LlSjbyPZ/GFVdQ3ibckUdPFhTzKGG8E= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=deltatee.com header.s=20200525 header.b=iyj5atjf; dmarc=pass (policy=none) header.from=deltatee.com; spf=pass (imf04.hostedemail.com: domain of logang@deltatee.com designates 204.191.154.188 as permitted sender) smtp.mailfrom=logang@deltatee.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1663960125; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZQmZb+lXFhX7pnMcsk23AOR4qV5DlMOOudYTRaQuI38=; b=hhC9RnABGOnljMIgL4YY7iWZ9cUtDThs6aCKoKfp8sQhn7Hrzy2UtxaGCIigBHFpDtVnN2 WgDgHeQWFa4Z2x2QQXCGm3MdWNBdGgFkqTuY6LLHVOaaudaWj11UhoLd4s0RnSFaKvG7g6 QKel7RyCoK3ubqZzkV5KE0JFLCeebNQ= X-Rspamd-Server: rspam11 X-Rspam-User: X-Rspamd-Queue-Id: A16894002D Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=deltatee.com header.s=20200525 header.b=iyj5atjf; dmarc=pass (policy=none) header.from=deltatee.com; spf=pass (imf04.hostedemail.com: domain of logang@deltatee.com designates 204.191.154.188 as permitted sender) smtp.mailfrom=logang@deltatee.com X-Stat-Signature: rhfhkifsfwbryydo6jcuj4wn1npd9kth X-HE-Tag: 1663960123-55373 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 2022-09-23 12:13, Jason Gunthorpe wrote: > On Thu, Sep 22, 2022 at 10:39:19AM -0600, Logan Gunthorpe wrote: >> GUP Callers that expect PCI P2PDMA pages can now set FOLL_PCI_P2PDMA to >> allow obtaining P2PDMA pages. If GUP is called without the flag and a >> P2PDMA page is found, it will return an error. >> >> FOLL_PCI_P2PDMA cannot be set if FOLL_LONGTERM is set. > > What is causing this? It is really troublesome, I would like to fix > it. eg I would like to have P2PDMA pages in VFIO iommu page tables and > in RDMA MR's - both require longterm. You had said it was required if we were relying on unmap_mapping_range()... https://lore.kernel.org/all/20210928200506.GX3544071@ziepe.ca/T/#u > Is it just because ZONE_DEVICE was created for DAX and carried that > revocable assumption over? Does anything in your series require > revocable? We still rely on unmap_mapping_range() indirectly in the unbind path. So I expect if something takes a LONGERM mapping that would block until whatever process holds the pin releases it. That's less than ideal and I'm not sure what can be done about it. >> @@ -2383,6 +2392,10 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, >> VM_BUG_ON(!pfn_valid(pte_pfn(pte))); >> page = pte_page(pte); >> >> + if (unlikely(!(flags & FOLL_PCI_P2PDMA) && >> + is_pci_p2pdma_page(page))) >> + goto pte_unmap; >> + >> folio = try_grab_folio(page, 1, flags); >> if (!folio) >> goto pte_unmap; > > On closer look this is not in the right place, we cannot touch the > content of *page without holding a ref, and that doesn't happen until > until try_grab_folio() completes. > > It would be simpler to put this check in try_grab_folio/try_grab_page > after the ref has been obtained. That will naturally cover all the > places that need it. Ok, I can make that change. Logan