From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03343C43461 for ; Wed, 16 Sep 2020 06:09:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id ED23621D7D for ; Wed, 16 Sep 2020 06:09:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ED23621D7D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=lst.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 683C66B0003; Wed, 16 Sep 2020 02:09:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 659FA6B0037; Wed, 16 Sep 2020 02:09:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5713D6B0055; Wed, 16 Sep 2020 02:09:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0152.hostedemail.com [216.40.44.152]) by kanga.kvack.org (Postfix) with ESMTP id 3FE516B0003 for ; Wed, 16 Sep 2020 02:09:26 -0400 (EDT) Received: from smtpin10.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 05705180AD807 for ; Wed, 16 Sep 2020 06:09:26 +0000 (UTC) X-FDA: 77267897532.10.scale14_5b0f4fe27117 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin10.hostedemail.com (Postfix) with ESMTP id CBFE416A4A4 for ; Wed, 16 Sep 2020 06:09:25 +0000 (UTC) X-HE-Tag: scale14_5b0f4fe27117 X-Filterd-Recvd-Size: 8895 Received: from verein.lst.de (verein.lst.de [213.95.11.211]) by imf32.hostedemail.com (Postfix) with ESMTP for ; Wed, 16 Sep 2020 06:09:24 +0000 (UTC) Received: by verein.lst.de (Postfix, from userid 2407) id BDE1368B05; Wed, 16 Sep 2020 08:09:21 +0200 (CEST) Date: Wed, 16 Sep 2020 08:09:21 +0200 From: Christoph Hellwig To: Ralph Campbell Cc: linux-mm@kvack.org, kvm-ppc@vger.kernel.org, nouveau@lists.freedesktop.org, linux-kernel@vger.kernel.org, Dan Williams , Ira Weiny , Matthew Wilcox , Jerome Glisse , John Hubbard , Alistair Popple , Christoph Hellwig , Jason Gunthorpe , Bharata B Rao , Zi Yan , "Kirill A . Shutemov" , Yang Shi , Paul Mackerras , Ben Skeggs , Andrew Morton Subject: Re: [PATCH] mm: remove extra ZONE_DEVICE struct page refcount Message-ID: <20200916060921.GB7321@lst.de> References: <20200914224509.17699-1-rcampbell@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200914224509.17699-1-rcampbell@nvidia.com> User-Agent: Mutt/1.5.17 (2007-11-01) X-Rspamd-Queue-Id: CBFE416A4A4 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam01 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: > diff --git a/include/linux/mm.h b/include/linux/mm.h > index 517751310dd2..5a82037a4b26 100644 > --- a/include/linux/mm.h > +++ b/include/linux/mm.h > @@ -1093,34 +1093,6 @@ static inline bool is_zone_device_page(const struct page *page) > #ifdef CONFIG_DEV_PAGEMAP_OPS > void free_devmap_managed_page(struct page *page); > DECLARE_STATIC_KEY_FALSE(devmap_managed_key); The export for devmap_managed_key can be dropped now. In fact I think we can remove devmap_managed_key entirely now - it is only checked in the actual page free path instead of for each refcount manipulation, so a good old unlikely is probably enough. Also free_devmap_managed_page can move to mm/internal.h. > +#ifdef CONFIG_DEV_PAGEMAP_OPS > +static void __put_devmap_managed_page(struct page *page) > +{ > + if (!static_branch_unlikely(&devmap_managed_key)) > + return; > + > + switch (page->pgmap->type) { > + case MEMORY_DEVICE_PRIVATE: > + case MEMORY_DEVICE_FS_DAX: > + free_devmap_managed_page(page); > + break; > + default: > + break; > + } > +} > +#else > +static inline void __put_devmap_managed_page(struct page *page) > +{ > +} > +#endif I think this should be moved to mm/memremap.c or even better actually be folded into free_devmap_managed_page, which would need a new name like free_zone_device_page(). Something like this incremental patch: diff --git a/include/linux/mm.h b/include/linux/mm.h index 7bb9e93cf86cde..29350dc27cd0cd 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1090,11 +1090,6 @@ static inline bool is_zone_device_page(const struct page *page) } #endif -#ifdef CONFIG_DEV_PAGEMAP_OPS -void free_devmap_managed_page(struct page *page); -DECLARE_STATIC_KEY_FALSE(devmap_managed_key); -#endif /* CONFIG_DEV_PAGEMAP_OPS */ - static inline bool is_device_private_page(const struct page *page) { return IS_ENABLED(CONFIG_DEV_PAGEMAP_OPS) && diff --git a/mm/internal.h b/mm/internal.h index 6345b08ce86ccf..629959a6f26d7c 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -618,4 +618,12 @@ struct migration_target_control { gfp_t gfp_mask; }; +#ifdef CONFIG_DEV_PAGEMAP_OPS +void free_zone_device_page(struct page *page); +#else +static inline void free_zone_device_page(struct page *page) +{ +} +#endif + #endif /* __MM_INTERNAL_H */ diff --git a/mm/memremap.c b/mm/memremap.c index d549e3733f4098..b15ad2264a4f1c 100644 --- a/mm/memremap.c +++ b/mm/memremap.c @@ -12,6 +12,7 @@ #include #include #include +#include "internal.h" static DEFINE_XARRAY(pgmap_array); @@ -37,36 +38,6 @@ unsigned long memremap_compat_align(void) EXPORT_SYMBOL_GPL(memremap_compat_align); #endif -#ifdef CONFIG_DEV_PAGEMAP_OPS -DEFINE_STATIC_KEY_FALSE(devmap_managed_key); -EXPORT_SYMBOL(devmap_managed_key); - -static void devmap_managed_enable_put(void) -{ - static_branch_dec(&devmap_managed_key); -} - -static int devmap_managed_enable_get(struct dev_pagemap *pgmap) -{ - if (pgmap->type == MEMORY_DEVICE_PRIVATE && - (!pgmap->ops || !pgmap->ops->page_free)) { - WARN(1, "Missing page_free method\n"); - return -EINVAL; - } - - static_branch_inc(&devmap_managed_key); - return 0; -} -#else -static int devmap_managed_enable_get(struct dev_pagemap *pgmap) -{ - return -EINVAL; -} -static void devmap_managed_enable_put(void) -{ -} -#endif /* CONFIG_DEV_PAGEMAP_OPS */ - static void pgmap_array_delete(struct range *range) { xa_store_range(&pgmap_array, PHYS_PFN(range->start), PHYS_PFN(range->end), @@ -181,7 +152,6 @@ void memunmap_pages(struct dev_pagemap *pgmap) pageunmap_range(pgmap, i); WARN_ONCE(pgmap->altmap.alloc, "failed to free all reserved pages\n"); - devmap_managed_enable_put(); } EXPORT_SYMBOL_GPL(memunmap_pages); @@ -319,7 +289,6 @@ void *memremap_pages(struct dev_pagemap *pgmap, int nid) .pgprot = PAGE_KERNEL, }; const int nr_range = pgmap->nr_range; - bool need_devmap_managed = true; int error, i; if (WARN_ONCE(!nr_range, "nr_range must be specified\n")) @@ -331,8 +300,9 @@ void *memremap_pages(struct dev_pagemap *pgmap, int nid) WARN(1, "Device private memory not supported\n"); return ERR_PTR(-EINVAL); } - if (!pgmap->ops || !pgmap->ops->migrate_to_ram) { - WARN(1, "Missing migrate_to_ram method\n"); + if (!pgmap->ops || + !pgmap->ops->migrate_to_ram || !pgmap->ops->page_free) { + WARN(1, "Missing ops\n"); return ERR_PTR(-EINVAL); } if (!pgmap->owner) { @@ -348,11 +318,9 @@ void *memremap_pages(struct dev_pagemap *pgmap, int nid) } break; case MEMORY_DEVICE_GENERIC: - need_devmap_managed = false; break; case MEMORY_DEVICE_PCI_P2PDMA: params.pgprot = pgprot_noncached(params.pgprot); - need_devmap_managed = false; break; default: WARN(1, "Invalid pgmap type %d\n", pgmap->type); @@ -376,12 +344,6 @@ void *memremap_pages(struct dev_pagemap *pgmap, int nid) } } - if (need_devmap_managed) { - error = devmap_managed_enable_get(pgmap); - if (error) - return ERR_PTR(error); - } - /* * Clear the pgmap nr_range as it will be incremented for each * successfully processed range. This communicates how many @@ -496,16 +458,9 @@ struct dev_pagemap *get_dev_pagemap(unsigned long pfn, EXPORT_SYMBOL_GPL(get_dev_pagemap); #ifdef CONFIG_DEV_PAGEMAP_OPS -void free_devmap_managed_page(struct page *page) +static void free_device_private_page(struct page *page) { - /* notify page idle for dax */ - if (!is_device_private_page(page)) { - wake_up_var(&page->_refcount); - return; - } - __ClearPageWaiters(page); - mem_cgroup_uncharge(page); /* @@ -540,4 +495,19 @@ void free_devmap_managed_page(struct page *page) page->mapping = NULL; page->pgmap->ops->page_free(page); } + +void free_zone_device_page(struct page *page) +{ + switch (page->pgmap->type) { + case MEMORY_DEVICE_FS_DAX: + /* notify page idle */ + wake_up_var(&page->_refcount); + return; + case MEMORY_DEVICE_PRIVATE: + free_device_private_page(page); + return; + default: + return; + } +} #endif /* CONFIG_DEV_PAGEMAP_OPS */ diff --git a/mm/swap.c b/mm/swap.c index bcab5db351184a..83451ac70d0f05 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -113,36 +113,14 @@ static void __put_compound_page(struct page *page) destroy_compound_page(page); } -#ifdef CONFIG_DEV_PAGEMAP_OPS -static void __put_devmap_managed_page(struct page *page) -{ - if (!static_branch_unlikely(&devmap_managed_key)) - return; - - switch (page->pgmap->type) { - case MEMORY_DEVICE_PRIVATE: - case MEMORY_DEVICE_FS_DAX: - free_devmap_managed_page(page); - break; - default: - break; - } -} -#else -static inline void __put_devmap_managed_page(struct page *page) -{ -} -#endif - void __put_page(struct page *page) { if (is_zone_device_page(page)) { - __put_devmap_managed_page(page); - /* * The page belongs to the device that created pgmap. Do * not return it to page allocator. */ + free_zone_device_page(page); return; } @@ -923,7 +901,7 @@ void release_pages(struct page **pages, int nr) flags); locked_pgdat = NULL; } - __put_devmap_managed_page(page); + free_zone_device_page(page); return; }