From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E12FC04A95 for ; Fri, 23 Sep 2022 23:21:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8567880030; Fri, 23 Sep 2022 19:21:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 805A980016; Fri, 23 Sep 2022 19:21:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 67F7F80030; Fri, 23 Sep 2022 19:21:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 556AC80016 for ; Fri, 23 Sep 2022 19:21:38 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 39280160A1E for ; Fri, 23 Sep 2022 23:21:38 +0000 (UTC) X-FDA: 79944924276.03.EA12144 Received: from mail-qt1-f176.google.com (mail-qt1-f176.google.com [209.85.160.176]) by imf09.hostedemail.com (Postfix) with ESMTP id E2548140005 for ; Fri, 23 Sep 2022 23:21:37 +0000 (UTC) Received: by mail-qt1-f176.google.com with SMTP id w2so990943qtv.9 for ; Fri, 23 Sep 2022 16:21:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date; bh=5ASiwm1avkHZc/hoLIIFI/eZ/ZpBbfiIv79qS4VlyJQ=; b=S5exWhlgK2IwN/41jFZWVgtUOIPgOVHZzklRjzN/LhjEdicvHTUHfUAsCYVTnPVnJ2 pLEkXf8ppCaivB8zKkiV5xHWOxHPCLi3Oh+ndtGqO5/3PUUide2X+tNk9f/o4UzdzyYs l723v+ePlu64jTV7J6j32h1l2vV58qu0PGnwnjWq1iAlo8weVGw4X7p2xu80tzvCf8Bc ooZJQ9qz1n896baC9VOTDnZDGs1Udgq290N5MKBgwJb9UGx/0yIR9beq7q1pXKyuOrOG qyywJClQqtur0GI4MaezATtC6BZH247PjXoZSk168tI57X6i1340gKDYtNSceLs7zRnN PqMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date; bh=5ASiwm1avkHZc/hoLIIFI/eZ/ZpBbfiIv79qS4VlyJQ=; b=XkJOSsuW1miem/7q8+JpuDNLOiRHpIvc3Aum/6HMcrPFQEqzmzaD1P22xKKMyMVEeA iT5LLh8bTKD+WW7r9Nzt8sxx0GmXtCr1/nGtHXglbbbB/xT4RP+4icXUoiqMj5kns076 rmvmg6M/J+Qn7myYTVx2bqRIaWpKZRXp/h3fXVHc54f5A7+WVUh/OVH40PrD8MbkDeUB RBKHI/a4Sksq04lASLwJTx/NSU5ePSboTqxl8Ieejsj1gD7npgWVhCDoCU9F27A8TgFu 3fd3s8zi3sxnduSKbV1E52WQ1ay2QFjSMoqqM4Xq1M9uFXtPAsKwyn3rAP1RIbkY2zM4 XD6Q== X-Gm-Message-State: ACrzQf30uhA9Oew1e7KfQs87Kc87lcY6qirSh1LQfx/Rq4UZqL/rJlZ2 0ibxpcyjkFUsVdIyHhpFzNmJxQ== X-Google-Smtp-Source: AMsMyM6UQR7CtJ4cZm41QVD6wT/F9o4m06KKQo4p97qx0szYOwFcvEevzn63HXF/BrGZl/ZOmXXdrA== X-Received: by 2002:a05:622a:174b:b0:35d:1be5:c65e with SMTP id l11-20020a05622a174b00b0035d1be5c65emr6445841qtk.422.1663975297167; Fri, 23 Sep 2022 16:21:37 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-162-113-129.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.162.113.129]) by smtp.gmail.com with ESMTPSA id t83-20020a374656000000b006ce9d6e51f4sm6650434qka.67.2022.09.23.16.21.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 23 Sep 2022 16:21:36 -0700 (PDT) Received: from jgg by wakko with local (Exim 4.95) (envelope-from ) id 1obrzD-0035zj-Lo; Fri, 23 Sep 2022 20:21:35 -0300 Date: Fri, 23 Sep 2022 20:21:35 -0300 From: Jason Gunthorpe To: Logan Gunthorpe Cc: linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, linux-pci@vger.kernel.org, linux-mm@kvack.org, Christoph Hellwig , Greg Kroah-Hartman , Dan Williams , Christian =?utf-8?B?S8O2bmln?= , John Hubbard , Don Dutile , Matthew Wilcox , Daniel Vetter , Minturn Dave B , Jason Ekstrand , Dave Hansen , Xiong Jianxin , Bjorn Helgaas , Ira Weiny , Robin Murphy , Martin Oliveira , Chaitanya Kulkarni , Ralph Campbell , Stephen Bates Subject: Re: [PATCH v10 1/8] mm: introduce FOLL_PCI_P2PDMA to gate getting PCI P2PDMA pages Message-ID: References: <20220922163926.7077-1-logang@deltatee.com> <20220922163926.7077-2-logang@deltatee.com> <64f8da81-7803-4db4-73da-a158295cbc9c@deltatee.com> <2327d393-af5c-3f4c-b9b9-6852b9d72f90@deltatee.com> <3840c1c6-3a5c-2286-e577-949f0d4ea7a6@deltatee.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=ziepe.ca header.s=google header.b=S5exWhlg; spf=pass (imf09.hostedemail.com: domain of jgg@ziepe.ca designates 209.85.160.176 as permitted sender) smtp.mailfrom=jgg@ziepe.ca; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1663975297; a=rsa-sha256; cv=none; b=czvc7YXk779INNL12N2OP7wMHy8Lbb4YbUCjIjIlv5OrV78PRYGMkZZy8MBKqPnIzvw6Dq Fw/MSk4eKRcir9HCEmjz9WUp67+DC/1eKoQhLaAlAKrz1x0PBJ9SE2wgFvdLd3ipnGfNm6 i9ul0GSjKkjC2OQv/mRNeHjwqszYChM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1663975297; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5ASiwm1avkHZc/hoLIIFI/eZ/ZpBbfiIv79qS4VlyJQ=; b=zk/tyefjiTkQUjehrNiR+4OW7C3eVAKbYGA5kVpT9hDjm5elSo/35YSjFt+sD8GsoYRMSZ maXyqZfuHnrCf7Vd/pYZbcckX+WpFACXiPV53jyS2onRc064vFgCnFGQ8gagqOg9c6Zq3G 7YVu+QZoqfTDNU5RUqk49ah62TibloM= X-Stat-Signature: 7u55yoto4gk317dwh6yfx9freubfhs5c X-Rspamd-Queue-Id: E2548140005 Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=ziepe.ca header.s=google header.b=S5exWhlg; spf=pass (imf09.hostedemail.com: domain of jgg@ziepe.ca designates 209.85.160.176 as permitted sender) smtp.mailfrom=jgg@ziepe.ca; dmarc=none X-Rspam-User: X-Rspamd-Server: rspam03 X-HE-Tag: 1663975297-657931 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Sep 23, 2022 at 05:14:11PM -0600, Logan Gunthorpe wrote: > > > On 2022-09-23 17:07, Jason Gunthorpe wrote: > > On Fri, Sep 23, 2022 at 05:01:26PM -0600, Logan Gunthorpe wrote: > >> > >> > >> > >> On 2022-09-23 16:58, Jason Gunthorpe wrote: > >>> On Fri, Sep 23, 2022 at 02:11:03PM -0600, Logan Gunthorpe wrote: > >>>> > >>>> > >>>> On 2022-09-23 13:53, Jason Gunthorpe wrote: > >>>>> On Fri, Sep 23, 2022 at 01:08:31PM -0600, Logan Gunthorpe wrote: > >>>>> I'm encouraging Dan to work on better infrastructure in pgmap core > >>>>> because every pgmap implementation has this issue currently. > >>>>> > >>>>> For that reason it is probably not so relavent to this series. > >>>>> > >>>>> Perhaps just clarify in the commit message that the FOLL_LONGTERM > >>>>> restriction is to copy DAX until the pgmap page refcounts are fixed. > >>>> > >>>> Ok, I'll add that note. > >>>> > >>>> Per the fix for the try_grab_page(), to me it doesn't fit well in > >>>> try_grab_page() without doing a bunch of cleanup to change the > >>>> error handling, and the same would have to be added to try_grab_folio(). > >>>> So I think it's better to leave it where it was, but move it below the > >>>> respective grab calls. Does the incremental patch below look correct? > >>> > >>> Oh? I was thinking of just a very simple thing: > >> > >> Really would like it to return -EREMOTEIO instead of -ENOMEM as that's the > >> error used for bad P2PDMA page everywhere. > > > > I'd rather not see GUP made more fragile just for that.. > > Not sure how that's more fragile... You're way seems more dangerous given > the large number of call sites we are adding it to when it might not > apply. No, that is the point, it *always* applies. A devmap struct page of the wrong type should never exit gup, from any path, no matter what. We have two central functions that validate a page is OK to return, that *everyone* must call. If you don't put it there then we will probably miss copying it into a call site eventually. > > try_grab_page() calls folio_ref_inc(), that is only legal if it knows > > the page is already a valid pointer under the PTLs, so it is safe to > > check the pgmap as well. > > My point is it doesn't get a reference or a pin unless FOLL_PIN or FOLL_GET is > set and the documentation states that neither might be set, in which case > folio_ref_inc() will not be called... That isn't how GUP is structured, all the calls to try_grab_page() are in places where PIN/GET might be set and are safe for that usage. If we know PIN/GET is not set then we don't even need to call the function because it is a NOP. Jason