From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EFDA5C433DB for ; Wed, 3 Feb 2021 23:37:31 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7A0C464F4E for ; Wed, 3 Feb 2021 23:37:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7A0C464F4E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=nvidia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D8ECC8D0002; Wed, 3 Feb 2021 18:37:30 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D3ECE8D0001; Wed, 3 Feb 2021 18:37:30 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C2E8C8D0002; Wed, 3 Feb 2021 18:37:30 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0050.hostedemail.com [216.40.44.50]) by kanga.kvack.org (Postfix) with ESMTP id AEC988D0001 for ; Wed, 3 Feb 2021 18:37:30 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 72119180AD817 for ; Wed, 3 Feb 2021 23:37:30 +0000 (UTC) X-FDA: 77778570660.24.owl27_0b02c27275d7 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin24.hostedemail.com (Postfix) with ESMTP id 5A2511A4A5 for ; Wed, 3 Feb 2021 23:37:30 +0000 (UTC) X-HE-Tag: owl27_0b02c27275d7 X-Filterd-Recvd-Size: 7543 Received: from hqnvemgate25.nvidia.com (hqnvemgate25.nvidia.com [216.228.121.64]) by imf10.hostedemail.com (Postfix) with ESMTP for ; Wed, 3 Feb 2021 23:37:29 +0000 (UTC) Received: from hqmail.nvidia.com (Not Verified[216.228.121.13]) by hqnvemgate25.nvidia.com (using TLS: TLSv1.2, AES256-SHA) id ; Wed, 03 Feb 2021 15:37:28 -0800 Received: from [10.2.50.90] (172.20.145.6) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Wed, 3 Feb 2021 23:37:27 +0000 Subject: Re: [PATCH 3/4] mm/gup: add a range variant of unpin_user_pages_dirty_lock() To: Joao Martins , CC: , , Andrew Morton , Jason Gunthorpe , Doug Ledford , Matthew Wilcox References: <20210203220025.8568-1-joao.m.martins@oracle.com> <20210203220025.8568-4-joao.m.martins@oracle.com> From: John Hubbard Message-ID: <5e372e25-7202-e0b6-0763-d267698db5b6@nvidia.com> Date: Wed, 3 Feb 2021 15:37:26 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:85.0) Gecko/20100101 Thunderbird/85.0 MIME-Version: 1.0 In-Reply-To: <20210203220025.8568-4-joao.m.martins@oracle.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Originating-IP: [172.20.145.6] X-ClientProxiedBy: HQMAIL107.nvidia.com (172.20.187.13) To HQMAIL107.nvidia.com (172.20.187.13) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1612395448; bh=qmULijZhnU2V5sy1R2oqyUqss+1cKK98EBpQ9ju3bbA=; h=Subject:To:CC:References:From:Message-ID:Date:User-Agent: MIME-Version:In-Reply-To:Content-Type:Content-Language: Content-Transfer-Encoding:X-Originating-IP:X-ClientProxiedBy; b=sRmiPr89EHkBrP1GmMCoZXLqZ8VNSxITNjBQAusN3GvzxUWFBhrL2lcJWkF2P+kIy 9l/L96pp283mqBsKXSdXMP0B7cEfRLetUuITXjeswSnr8XLx0P4/UIvKie54tKU1mf RpmbXU9qWq5buJRRakkHwQhFKSy9loIcIm/yezE7DT0hw6LsMEcHUL4OVD68nONe+4 5yc2+k/QQrp9pk3u0lIFjE8/A8jeBAMHka7l5ndPFJtTbRlCzg3Nzc/W6CJBNCIGM6 vnm3YQcA8HDupkLPIGStYq25sRtz4qCPmfPCh2wom3KbLqf+xlQhZr8JM0Iq56otyK dKoU440SkQ1xA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 2/3/21 2:00 PM, Joao Martins wrote: > Add a unpin_user_page_range() API which takes a starting page > and how many consecutive pages we want to dirty. > > Given that we won't be iterating on a list of changes, change > compound_next() to receive a bool, whether to calculate from the starting > page, or walk the page array. Finally add a separate iterator, A bool arg is sometimes, but not always, a hint that you really just want a separate set of routines. Below... > for_each_compound_range() that just operate in page ranges as opposed > to page array. > > For users (like RDMA mr_dereg) where each sg represents a > contiguous set of pages, we're able to more efficiently unpin > pages without having to supply an array of pages much of what > happens today with unpin_user_pages(). > > Suggested-by: Jason Gunthorpe > Signed-off-by: Joao Martins > --- > include/linux/mm.h | 2 ++ > mm/gup.c | 48 ++++++++++++++++++++++++++++++++++++++-------- > 2 files changed, 42 insertions(+), 8 deletions(-) > > diff --git a/include/linux/mm.h b/include/linux/mm.h > index a608feb0d42e..b76063f7f18a 100644 > --- a/include/linux/mm.h > +++ b/include/linux/mm.h > @@ -1265,6 +1265,8 @@ static inline void put_page(struct page *page) > void unpin_user_page(struct page *page); > void unpin_user_pages_dirty_lock(struct page **pages, unsigned long npages, > bool make_dirty); > +void unpin_user_page_range_dirty_lock(struct page *page, unsigned long npages, > + bool make_dirty); > void unpin_user_pages(struct page **pages, unsigned long npages); > > /** > diff --git a/mm/gup.c b/mm/gup.c > index 971a24b4b73f..1b57355d5033 100644 > --- a/mm/gup.c > +++ b/mm/gup.c > @@ -215,11 +215,16 @@ void unpin_user_page(struct page *page) > } > EXPORT_SYMBOL(unpin_user_page); > > -static inline unsigned int count_ntails(struct page **pages, unsigned long npages) > +static inline unsigned int count_ntails(struct page **pages, > + unsigned long npages, bool range) > { > - struct page *head = compound_head(pages[0]); > + struct page *page = pages[0], *head = compound_head(page); > unsigned int ntails; > > + if (range) > + return (!PageCompound(head) || compound_order(head) <= 1) ? 1 : > + min_t(unsigned int, (head + compound_nr(head) - page), npages); Here, you clearly should use a separate set of _range routines. Because you're basically creating two different routines here! Keep it simple. Once you're in a separate routine, you might feel more comfortable expanding that to a more readable form, too: if (!PageCompound(head) || compound_order(head) <= 1) return 1; return min_t(unsigned int, (head + compound_nr(head) - page), npages); thanks, -- John Hubbard NVIDIA > + > for (ntails = 1; ntails < npages; ntails++) { > if (compound_head(pages[ntails]) != head) > break; > @@ -229,20 +234,32 @@ static inline unsigned int count_ntails(struct page **pages, unsigned long npage > } > > static inline void compound_next(unsigned long i, unsigned long npages, > - struct page **list, struct page **head, > - unsigned int *ntails) > + struct page **list, bool range, > + struct page **head, unsigned int *ntails) > { > + struct page *p, **next = &p; > + > if (i >= npages) > return; > > - *ntails = count_ntails(list + i, npages - i); > - *head = compound_head(list[i]); > + if (range) > + *next = *list + i; > + else > + next = list + i; > + > + *ntails = count_ntails(next, npages - i, range); > + *head = compound_head(*next); > } > > +#define for_each_compound_range(i, list, npages, head, ntails) \ > + for (i = 0, compound_next(i, npages, list, true, &head, &ntails); \ > + i < npages; i += ntails, \ > + compound_next(i, npages, list, true, &head, &ntails)) > + > #define for_each_compound_head(i, list, npages, head, ntails) \ > - for (i = 0, compound_next(i, npages, list, &head, &ntails); \ > + for (i = 0, compound_next(i, npages, list, false, &head, &ntails); \ > i < npages; i += ntails, \ > - compound_next(i, npages, list, &head, &ntails)) > + compound_next(i, npages, list, false, &head, &ntails)) > > /** > * unpin_user_pages_dirty_lock() - release and optionally dirty gup-pinned pages > @@ -306,6 +323,21 @@ void unpin_user_pages_dirty_lock(struct page **pages, unsigned long npages, > } > EXPORT_SYMBOL(unpin_user_pages_dirty_lock); > > +void unpin_user_page_range_dirty_lock(struct page *page, unsigned long npages, > + bool make_dirty) > +{ > + unsigned long index; > + struct page *head; > + unsigned int ntails; > + > + for_each_compound_range(index, &page, npages, head, ntails) { > + if (make_dirty && !PageDirty(head)) > + set_page_dirty_lock(head); > + put_compound_head(head, ntails, FOLL_PIN); > + } > +} > +EXPORT_SYMBOL(unpin_user_page_range_dirty_lock); > + > /** > * unpin_user_pages() - release an array of gup-pinned pages. > * @pages: array of pages to be marked dirty and released. >