From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9A5BEC433DB for ; Wed, 6 Jan 2021 10:04:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F105E2310C for ; Wed, 6 Jan 2021 10:04:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F105E2310C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 114488D00FF; Wed, 6 Jan 2021 05:04:51 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0C5258D0090; Wed, 6 Jan 2021 05:04:51 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F1C958D00FF; Wed, 6 Jan 2021 05:04:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0022.hostedemail.com [216.40.44.22]) by kanga.kvack.org (Postfix) with ESMTP id DD7C38D0090 for ; Wed, 6 Jan 2021 05:04:50 -0500 (EST) Received: from smtpin28.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id A15A1180AD80F for ; Wed, 6 Jan 2021 10:04:50 +0000 (UTC) X-FDA: 77674916340.28.grape28_100b9ac274e0 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin28.hostedemail.com (Postfix) with ESMTP id 835916C06 for ; Wed, 6 Jan 2021 10:04:50 +0000 (UTC) X-HE-Tag: grape28_100b9ac274e0 X-Filterd-Recvd-Size: 11075 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf19.hostedemail.com (Postfix) with ESMTP for ; Wed, 6 Jan 2021 10:04:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1609927489; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gc4t22jO8xiv+9zgWMNeKDXcVuB45LJHhyAzQf4b9PU=; b=djAtWwEepA+r+8UEimuCunKDbg/nHm6gfxd2VkCFFc0M+o2Vqh9wc0OJXEztaVBG8KYub3 rI07t8TiSdVfp98ecdI8xlKdXU2W9Se8fDE2AT95LnAUzJHf4j9Cho+RksP35e6mIfCKxg 3LEbGoAOPx9p8Zqoep87UPpJM+O1eMY= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-497-gdE9anQWNkaMZO5bDWCy7A-1; Wed, 06 Jan 2021 05:04:47 -0500 X-MC-Unique: gdE9anQWNkaMZO5bDWCy7A-1 Received: by mail-wr1-f72.google.com with SMTP id q18so1135898wrc.20 for ; Wed, 06 Jan 2021 02:04:46 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:content-transfer-encoding:from:mime-version :subject:date:message-id:references:cc:in-reply-to:to; bh=IzSNP7abDMwkiH6rCYGRfFP65hgLj/ZbfnRGb+2zvM4=; b=HW4biVr/MsgJJtMXuz2+ojVriecc/4HfMQ+O1CwH5FMDu/pU37SBT4TecGQImREo1Q KMYEUOUMY598WkP45lzXPjj2d51rmsTKUHHPCCTPMUHjeRzxjst0kAMM8j3Rge4+vARC AsIBXVvnMJ3tAeaGYxGZ/q9iEYYChCluNrOQWN7HfuainUhmtt8hOWs23d/Dr/xpXUhb SoSvgaZrBHj8APqv+wAT5uoCbU/KoUJzsb1oWxMUMr900tW2/Tl96XxW+dVrWbTgT2rJ pBYpBgYpp2X04npdrbQ+oK+n573ZiKf/KS5ZzAgcxDmgewgDZIfGN7pdjvCkilAvDkcl OZHw== X-Gm-Message-State: AOAM531hIAQT/vYPVstRNUY5Xx/PCfRkIvYgJjmdZ52nfjzpNw9fO9UI caqgS1d6F6LG+G4ULpSmqydTKKi1XsRRBdxTbkmLI4QdrvhJat93V2tUGJNktGtMLEBV9Z20a8D 0hQ349sCbRm9fqU99Z9z2XkZHNW1MW+vUJ4OOlp2rkDYzFHdHFqQ38Cdv4Xk= X-Received: by 2002:a7b:cc94:: with SMTP id p20mr2995415wma.22.1609927485802; Wed, 06 Jan 2021 02:04:45 -0800 (PST) X-Google-Smtp-Source: ABdhPJzUY8j1WVx0H0nlOn9VNhfP+fbN+lGegGlXjtXbOA8T3VPGFUIs4yKOZiZfoZn/zqUBn64qyQ== X-Received: by 2002:a7b:cc94:: with SMTP id p20mr2995385wma.22.1609927485516; Wed, 06 Jan 2021 02:04:45 -0800 (PST) Received: from [192.168.3.108] (p5b0c68cf.dip0.t-ipconnect.de. [91.12.104.207]) by smtp.gmail.com with ESMTPSA id g191sm2602635wmg.39.2021.01.06.02.04.44 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 06 Jan 2021 02:04:44 -0800 (PST) From: David Hildenbrand Mime-Version: 1.0 (1.0) Subject: Re: [PATCH] mm: Teach pfn_to_online_page() about ZONE_DEVICE section collisions Date: Wed, 6 Jan 2021 11:04:44 +0100 Message-Id: <5F9798F4-1FA3-450C-AC9F-6A2ABCAA4781@redhat.com> References: <160990599013.2430134.11556277600719835946.stgit@dwillia2-desk3.amr.corp.intel.com> Cc: linux-mm@kvack.org, Andrew Morton , Michal Hocko , David Hildenbrand , linux-kernel@vger.kernel.org In-Reply-To: <160990599013.2430134.11556277600719835946.stgit@dwillia2-desk3.amr.corp.intel.com> To: Dan Williams X-Mailer: iPhone Mail (18C66) Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=david@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: > Am 06.01.2021 um 05:08 schrieb Dan Williams : >=20 > =EF=BB=BFWhile pfn_to_online_page() is able to determine pfn_valid() at > subsection granularity it is not able to reliably determine if a given > pfn is also online if the section is mixed with ZONE_DEVICE pfns. >=20 > Update move_pfn_range_to_zone() to flag (SECTION_TAINT_ZONE_DEVICE) a > section that mixes ZONE_DEVICE pfns with other online pfns. With > SECTION_TAINT_ZONE_DEVICE to delineate, pfn_to_online_page() can fall > back to a slow-path check for ZONE_DEVICE pfns in an online section. >=20 > With this implementation of pfn_to_online_page() pfn-walkers mostly only > need to check section metadata to determine pfn validity. In the > rare case of mixed-zone sections the pfn-walker will skip offline > ZONE_DEVICE pfns as expected. >=20 > Other notes: >=20 > Because the collision case is rare, and for simplicity, the > SECTION_TAINT_ZONE_DEVICE flag is never cleared once set. >=20 > pfn_to_online_page() was already borderline too large to be a macro / > inline function, but the additional logic certainly pushed it over that > threshold, and so it is moved to an out-of-line helper. >=20 > Fixes: ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug") > Cc: Andrew Morton > Reported-by: Michal Hocko > Reported-by: David Hildenbrand > Signed-off-by: Dan Williams > --- >=20 > This compiles and passes the nvdimm unit tests, but I have not tested > with pfn walkers in the presence of ZONE_DEVICE collisions. >=20 >=20 > include/linux/memory_hotplug.h | 17 +--------- > include/linux/mmzone.h | 22 ++++++++---- > mm/memory_hotplug.c | 71 +++++++++++++++++++++++++++++++++++= +++++ > 3 files changed, 87 insertions(+), 23 deletions(-) >=20 > diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplu= g.h > index 15acce5ab106..3d99de0db2dd 100644 > --- a/include/linux/memory_hotplug.h > +++ b/include/linux/memory_hotplug.h > @@ -16,22 +16,7 @@ struct resource; > struct vmem_altmap; >=20 > #ifdef CONFIG_MEMORY_HOTPLUG > -/* > - * Return page for the valid pfn only if the page is online. All pfn > - * walkers which rely on the fully initialized page->flags and others > - * should use this rather than pfn_valid && pfn_to_page > - */ > -#define pfn_to_online_page(pfn) \ > -({ \ > - struct page *___page =3D NULL; \ > - unsigned long ___pfn =3D pfn; \ > - unsigned long ___nr =3D pfn_to_section_nr(___pfn); \ > - \ > - if (___nr < NR_MEM_SECTIONS && online_section_nr(___nr) && \ > - pfn_valid_within(___pfn)) \ > - ___page =3D pfn_to_page(___pfn); \ > - ___page; \ > -}) > +struct page *pfn_to_online_page(unsigned long pfn); >=20 > /* > * Types for free bootmem stored in page->lru.next. These have to be in > diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h > index b593316bff3d..0b5c44f730b4 100644 > --- a/include/linux/mmzone.h > +++ b/include/linux/mmzone.h > @@ -1273,13 +1273,14 @@ extern size_t mem_section_usage_size(void); > * which results in PFN_SECTION_SHIFT equal 6. > * To sum it up, at least 6 bits are available. > */ > -#define SECTION_MARKED_PRESENT (1UL<<0) > -#define SECTION_HAS_MEM_MAP (1UL<<1) > -#define SECTION_IS_ONLINE (1UL<<2) > -#define SECTION_IS_EARLY (1UL<<3) > -#define SECTION_MAP_LAST_BIT (1UL<<4) > -#define SECTION_MAP_MASK (~(SECTION_MAP_LAST_BIT-1)) > -#define SECTION_NID_SHIFT 3 > +#define SECTION_MARKED_PRESENT (1UL<<0) > +#define SECTION_HAS_MEM_MAP (1UL<<1) > +#define SECTION_IS_ONLINE (1UL<<2) > +#define SECTION_IS_EARLY (1UL<<3) > +#define SECTION_TAINT_ZONE_DEVICE (1UL<<4) > +#define SECTION_MAP_LAST_BIT (1UL<<5) > +#define SECTION_MAP_MASK (~(SECTION_MAP_LAST_BIT-1)) > +#define SECTION_NID_SHIFT 3 >=20 > static inline struct page *__section_mem_map_addr(struct mem_section *sec= tion) > { > @@ -1318,6 +1319,13 @@ static inline int online_section(struct mem_sectio= n *section) > return (section && (section->section_mem_map & SECTION_IS_ONLINE)); > } >=20 > +static inline int online_device_section(struct mem_section *section) > +{ > + unsigned long flags =3D SECTION_IS_ONLINE | SECTION_TAINT_ZONE_DEVIC= E; > + > + return section && ((section->section_mem_map & flags) =3D=3D flags); > +} > + > static inline int online_section_nr(unsigned long nr) > { > return online_section(__nr_to_section(nr)); > diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c > index f9d57b9be8c7..9f36968e6188 100644 > --- a/mm/memory_hotplug.c > +++ b/mm/memory_hotplug.c > @@ -300,6 +300,47 @@ static int check_hotplug_memory_addressable(unsigned= long pfn, > return 0; > } >=20 > +/* > + * Return page for the valid pfn only if the page is online. All pfn > + * walkers which rely on the fully initialized page->flags and others > + * should use this rather than pfn_valid && pfn_to_page > + */ > +struct page *pfn_to_online_page(unsigned long pfn) > +{ > + unsigned long nr =3D pfn_to_section_nr(pfn); > + struct dev_pagemap *pgmap; > + struct mem_section *ms; > + > + if (nr >=3D NR_MEM_SECTIONS) > + return NULL; > + > + ms =3D __nr_to_section(nr); > + > + if (!online_section(ms)) > + return NULL; > + > + if (!pfn_valid_within(pfn)) > + return NULL; > + > + if (!online_device_section(ms)) > + return pfn_to_page(pfn); > + > + /* > + * Slowpath: when ZONE_DEVICE collides with > + * ZONE_{NORMAL,MOVABLE} within the same section some pfns in > + * the section may be 'offline' but 'valid'. Only > + * get_dev_pagemap() can determine sub-section online status. > + */ > + pgmap =3D get_dev_pagemap(pfn, NULL); > + put_dev_pagemap(pgmap); > + > + /* The presence of a pgmap indicates ZONE_DEVICE offline pfn */ > + if (pgmap) > + return NULL; > + return pfn_to_page(pfn); > +} > +EXPORT_SYMBOL_GPL(pfn_to_online_page); > + > /* > * Reasonably generic function for adding memory. It is > * expected that archs that support memory hotplug will > @@ -678,6 +719,27 @@ static void __meminit resize_pgdat_range(struct pgli= st_data *pgdat, unsigned lon > pgdat->node_spanned_pages =3D max(start_pfn + nr_pages, old_end_pfn) -= pgdat->node_start_pfn; >=20 > } > + > +static int zone_id(const struct zone *zone) > +{ > + struct pglist_data *pgdat =3D zone->zone_pgdat; > + > + return zone - pgdat->node_zones; > +} > + > +static void section_taint_zone_device(struct zone *zone, unsigned long p= fn) > +{ > + struct mem_section *ms =3D __nr_to_section(pfn_to_section_nr(pfn)); > + > + if (zone_id(zone) !=3D ZONE_DEVICE) > + return; > + > + if (IS_ALIGNED(pfn, PAGES_PER_SECTION)) > + return; I think you can simplify this. Just taint any early section here, independe= nt of the pfn.