From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 12E7FC433EF for ; Thu, 16 Jun 2022 02:45:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 250478D0001; Wed, 15 Jun 2022 22:45:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 200196B0073; Wed, 15 Jun 2022 22:45:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 02BA48D0001; Wed, 15 Jun 2022 22:45:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id E1E246B0072 for ; Wed, 15 Jun 2022 22:45:30 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 6C93434358 for ; Thu, 16 Jun 2022 02:45:30 +0000 (UTC) X-FDA: 79582558020.06.56130B7 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) by imf30.hostedemail.com (Postfix) with ESMTP id 1383380052 for ; Thu, 16 Jun 2022 02:45:21 +0000 (UTC) Received: by mail-pl1-f174.google.com with SMTP id o17so184998pla.6 for ; Wed, 15 Jun 2022 19:45:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=TlLQ7tosji2YkhAul6rILt0FgOyJFX5PDRocyDPTg+Y=; b=RI32rOnGwOPA5gsdV4n1uZ8GnPXTgWdA+QXDkeyb0Mp+N7AxQJ5cANlDDv66PpbCNx X53He4UMduIQt8iou2s/guBWDRDUHxKF8lNL9GcRrfi9bkRMf5I7tUyOYpW3hIBgkje8 TlpYwGFECnzsuSBSEWdQth1qLfn+WyYmrL9GdVqyE3Eis93x0tTHwPePBQCTn/VL/Gzb z8c7ayKOD8mGj80driftnCEMDYQQNWgQUeCoIIC1ryRf2rOn5RjfQwM0pDycAg/Of8ng B1N4qWXr8thkCmsl8KsocD9FKe56EVKbZFr5FcEgvbWWkbRRiRaXQV/VvSgcoBeltqjM 4WjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=TlLQ7tosji2YkhAul6rILt0FgOyJFX5PDRocyDPTg+Y=; b=V5zgEHmA3LZAUlxtud4RxJfYweJgsJ90Ek4hlFRb7NPRMiykEFqqDJvhIo4J+T7WKt 59D0FeONyJc0F5oMngJ3XcOUhl0Kfwe2Fv3H25mcQpFrY6EOJpxnxTTo1ouMRiWGM/Eh XlJFTsQZ2HvcU+OH0Nk0wGAlbadPRZfRyMoYJY/Y/HxyZBPmHMBJ1Zhr7sSsWSE9aKTp wIOHmrd9oXGgkJXhqZhHzDvggFr3yGv5k+Jzz+cQR83efhLza1XAoOR1Brr9UB8dXclF 5kJ7df20XVOnySVBH2L0AUXlSat58fOq5bHNo04j8CT1Z5mVcDATdZ9qH4QicvNU2GBO QMjg== X-Gm-Message-State: AJIora/ou3DOocBH7VVG8N9hk+5cSBTotEV1NYXmwK/Voh59TAdrRhSB Byb3A+9e2k3q2jL1L+aS7b6V4A== X-Google-Smtp-Source: AGRyM1sCcm3nUs0MXMRNeFgPdlkO8phixsKPfk6BM/du3+VL+6b8V8UFzHEPRaCM2/QPj94/XqqBdg== X-Received: by 2002:a17:90a:e7d2:b0:1e8:97ac:da0b with SMTP id kb18-20020a17090ae7d200b001e897acda0bmr13451995pjb.242.1655347520681; Wed, 15 Jun 2022 19:45:20 -0700 (PDT) Received: from localhost ([139.177.225.255]) by smtp.gmail.com with ESMTPSA id k16-20020a17090aaa1000b001e3351cb7fbsm2514708pjq.28.2022.06.15.19.45.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Jun 2022 19:45:20 -0700 (PDT) Date: Thu, 16 Jun 2022 10:45:14 +0800 From: Muchun Song To: David Hildenbrand Cc: corbet@lwn.net, akpm@linux-foundation.org, paulmck@kernel.org, mike.kravetz@oracle.com, osalvador@suse.de, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, smuchun@gmail.com Subject: Re: [PATCH v2 2/2] mm: memory_hotplug: introduce SECTION_CANNOT_OPTIMIZE_VMEMMAP Message-ID: References: <20220520025538.21144-1-songmuchun@bytedance.com> <20220520025538.21144-3-songmuchun@bytedance.com> <53024884-0182-df5f-9ca2-00652c64ce36@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <53024884-0182-df5f-9ca2-00652c64ce36@redhat.com> ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655347522; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TlLQ7tosji2YkhAul6rILt0FgOyJFX5PDRocyDPTg+Y=; b=HiQJX+02k9debNfyGKSWJ8mGuBBL7s7ESVgEDBAdRYncEiGKcbGzWWNJ27hayc1G/zc0+c Wk8i2DhYxY6MyEl5uB+aYxTDa+GRlySndgczSuix5IqX5O77JlliLrldBnGdmngRElLXae 6qn6GFAbTQstjzMkz1LTK5F7jJoaMFo= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=RI32rOnG; dmarc=pass (policy=none) header.from=bytedance.com; spf=pass (imf30.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.214.174 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1655347522; a=rsa-sha256; cv=none; b=5iXAkF1+z7q9RZCXUl//stiWAkxXSiS3pkEbq3rRM8XBWGSH0H1gQD7ay1j1xu33BBDa3b FEOsjjiD71lJrHKq6cny4FyaB3godejhKDu6trqjKBYFLDML+7gljzekPNjfH2RfDPagJG RXidC751PJwm4+RXdcEk0aKAHkSo//Y= X-Stat-Signature: 4nwddfs7aymjgjarc8sjfxfgnnh9u1yh X-Rspamd-Queue-Id: 1383380052 X-Rspam-User: X-Rspamd-Server: rspam05 Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=RI32rOnG; dmarc=pass (policy=none) header.from=bytedance.com; spf=pass (imf30.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.214.174 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com X-HE-Tag: 1655347521-157042 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Jun 15, 2022 at 11:51:49AM +0200, David Hildenbrand wrote: > On 20.05.22 04:55, Muchun Song wrote: > > For now, the feature of hugetlb_free_vmemmap is not compatible with the > > feature of memory_hotplug.memmap_on_memory, and hugetlb_free_vmemmap > > takes precedence over memory_hotplug.memmap_on_memory. However, someone > > wants to make memory_hotplug.memmap_on_memory takes precedence over > > hugetlb_free_vmemmap since memmap_on_memory makes it more likely to > > succeed memory hotplug in close-to-OOM situations. So the decision > > of making hugetlb_free_vmemmap take precedence is not wise and elegant. > > The proper approach is to have hugetlb_vmemmap.c do the check whether > > the section which the HugeTLB pages belong to can be optimized. If > > the section's vmemmap pages are allocated from the added memory block > > itself, hugetlb_free_vmemmap should refuse to optimize the vmemmap, > > otherwise, do the optimization. Then both kernel parameters are > > compatible. So this patch introduces SECTION_CANNOT_OPTIMIZE_VMEMMAP > > to indicate whether the section could be optimized. > > > > In theory, we have that information stored in the relevant memory block, > but I assume that lookup in the xarray + locking is impractical. > > I wonder if we can derive that information simply from the vmemmap pages > themselves, because *drumroll* > > For one vmemmap page (the first one), the vmemmap corresponds to itself > -- what?! > > > [ hotplugged memory ] > [ memmap ][ usable memory ] > | | | > ^--- | | > ^------- | > ^---------------------- > > The memmap of the first page of hotplugged memory falls onto itself. > We'd have to derive from actual "usable memory" that condition. > > > We currently support memmap_on_memory memory only within fixed-size > memory blocks. So "hotplugged memory" is guaranteed to be aligned to > memory_block_size_bytes() and the size is memory_block_size_bytes(). > > If we'd have a page falling into usbale memory, we'd simply lookup the > first page and test if the vmemmap maps to itself. > I think this can work. Should we use this approach in next version? > > Of course, once we'd support variable-sized memory blocks, it would be > different. > > > An easier/future-proof approach might simply be flagging the vmemmap > pages as being special. We reuse page flags for that, which don't have > semantics yet (i.e., PG_reserved indicates a boot-time allocation via > memblock). > I think you mean flag vmemmap pages' struct page as PG_reserved if it can be optimized, right? When the vmemmap pages are allocated in hugetlb_vmemmap_alloc(), is it valid to flag them as PG_reserved (they are allocated from buddy allocator not memblock)? Thanks. > You'd walk the applicable vmemmap pages you want to optimize and check > if they are marked as special. You don't have to walk all but can > optimize: memmap_on_memory uses a vmemmap size that's at least PMD_SIZE. > So it's sufficient to check a single vmemmap page inside a PMD_SIZE > vmemmap range. > > > > Signed-off-by: Muchun Song > > --- > > Documentation/admin-guide/kernel-parameters.txt | 22 +++++++++---------- > > Documentation/admin-guide/sysctl/vm.rst | 5 ++--- > > include/linux/memory_hotplug.h | 9 -------- > > include/linux/mmzone.h | 17 +++++++++++++++ > > mm/hugetlb_vmemmap.c | 28 ++++++++++++++++++------- > > mm/memory_hotplug.c | 22 +++++++------------ > > mm/sparse.c | 8 +++++++ > > 7 files changed, 66 insertions(+), 45 deletions(-) > > > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt > > index c087f578d9d8..5359ffb04a84 100644 > > --- a/Documentation/admin-guide/kernel-parameters.txt > > +++ b/Documentation/admin-guide/kernel-parameters.txt > > @@ -1730,9 +1730,11 @@ > > Built with CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP_DEFAULT_ON=y, > > the default is on. > > > > - This is not compatible with memory_hotplug.memmap_on_memory. > > - If both parameters are enabled, hugetlb_free_vmemmap takes > > - precedence over memory_hotplug.memmap_on_memory. > > + Note that the vmemmap pages may be allocated from the added > > + memory block itself when memory_hotplug.memmap_on_memory is > > + enabled, those vmemmap pages cannot be optimized even if this > > + feature is enabled. Other vmemmap pages not allocated from > > + the added memory block itself do not be affected. > > > > hung_task_panic= > > [KNL] Should the hung task detector generate panics. > > @@ -3077,10 +3079,12 @@ > > [KNL,X86,ARM] Boolean flag to enable this feature. > > Format: {on | off (default)} > > When enabled, runtime hotplugged memory will > > - allocate its internal metadata (struct pages) > > - from the hotadded memory which will allow to > > - hotadd a lot of memory without requiring > > - additional memory to do so. > > + allocate its internal metadata (struct pages, > > + those vmemmap pages cannot be optimized even > > + if hugetlb_free_vmemmap is enabled) from the > > + hotadded memory which will allow to hotadd a > > + lot of memory without requiring additional > > + memory to do so. > > This feature is disabled by default because it > > has some implication on large (e.g. GB) > > allocations in some configurations (e.g. small > > @@ -3090,10 +3094,6 @@ > > Note that even when enabled, there are a few cases where > > the feature is not effective. > > > > - This is not compatible with hugetlb_free_vmemmap. If > > - both parameters are enabled, hugetlb_free_vmemmap takes > > - precedence over memory_hotplug.memmap_on_memory. > > - > > memtest= [KNL,X86,ARM,M68K,PPC,RISCV] Enable memtest > > Format: > > default : 0 > > diff --git a/Documentation/admin-guide/sysctl/vm.rst b/Documentation/admin-guide/sysctl/vm.rst > > index 5c9aa171a0d3..d7374a1e8ac9 100644 > > --- a/Documentation/admin-guide/sysctl/vm.rst > > +++ b/Documentation/admin-guide/sysctl/vm.rst > > @@ -565,9 +565,8 @@ See Documentation/admin-guide/mm/hugetlbpage.rst > > hugetlb_optimize_vmemmap > > ======================== > > > > -This knob is not available when memory_hotplug.memmap_on_memory (kernel parameter) > > -is configured or the size of 'struct page' (a structure defined in > > -include/linux/mm_types.h) is not power of two (an unusual system config could > > +This knob is not available when the size of 'struct page' (a structure defined > > +in include/linux/mm_types.h) is not power of two (an unusual system config could > > result in this). > > > > Enable (set to 1) or disable (set to 0) the feature of optimizing vmemmap pages > > diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h > > index 20d7edf62a6a..e0b2209ab71c 100644 > > --- a/include/linux/memory_hotplug.h > > +++ b/include/linux/memory_hotplug.h > > @@ -351,13 +351,4 @@ void arch_remove_linear_mapping(u64 start, u64 size); > > extern bool mhp_supports_memmap_on_memory(unsigned long size); > > #endif /* CONFIG_MEMORY_HOTPLUG */ > > > > -#ifdef CONFIG_MHP_MEMMAP_ON_MEMORY > > -bool mhp_memmap_on_memory(void); > > -#else > > -static inline bool mhp_memmap_on_memory(void) > > -{ > > - return false; > > -} > > -#endif > > - > > #endif /* __LINUX_MEMORY_HOTPLUG_H */ > > diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h > > index 2cf2a76535ab..607a4fcabbd4 100644 > > --- a/include/linux/mmzone.h > > +++ b/include/linux/mmzone.h > > @@ -1434,6 +1434,7 @@ extern size_t mem_section_usage_size(void); > > MAPPER(IS_ONLINE) \ > > MAPPER(IS_EARLY) \ > > MAPPER(TAINT_ZONE_DEVICE, CONFIG_ZONE_DEVICE) \ > > + MAPPER(CANNOT_OPTIMIZE_VMEMMAP, CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP) \ > > MAPPER(MAP_LAST_BIT) > > > > #define __SECTION_SHIFT_FLAG_MAPPER_0(x) > > @@ -1471,6 +1472,22 @@ static inline struct page *__section_mem_map_addr(struct mem_section *section) > > return (struct page *)map; > > } > > > > +#ifdef CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP > > +static inline void section_mark_cannot_optimize_vmemmap(struct mem_section *ms) > > +{ > > + ms->section_mem_map |= SECTION_CANNOT_OPTIMIZE_VMEMMAP; > > +} > > + > > +static inline int section_cannot_optimize_vmemmap(struct mem_section *ms) > > +{ > > + return (ms && (ms->section_mem_map & SECTION_CANNOT_OPTIMIZE_VMEMMAP)); > > +} > > +#else > > +static inline void section_mark_cannot_optimize_vmemmap(struct mem_section *ms) > > +{ > > +} > > +#endif > > + > > static inline int present_section(struct mem_section *section) > > { > > return (section && (section->section_mem_map & SECTION_MARKED_PRESENT)); > > diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c > > index fcd9f7872064..f12170520337 100644 > > --- a/mm/hugetlb_vmemmap.c > > +++ b/mm/hugetlb_vmemmap.c > > @@ -97,18 +97,32 @@ int hugetlb_vmemmap_alloc(struct hstate *h, struct page *head) > > return ret; > > } > > > > +static unsigned int optimizable_vmemmap_pages(struct hstate *h, > > + struct page *head) > > +{ > > + unsigned long pfn = page_to_pfn(head); > > + unsigned long end = pfn + pages_per_huge_page(h); > > + > > + if (READ_ONCE(vmemmap_optimize_mode) == VMEMMAP_OPTIMIZE_OFF) > > + return 0; > > + > > + for (; pfn < end; pfn += PAGES_PER_SECTION) { > > + if (section_cannot_optimize_vmemmap(__pfn_to_section(pfn))) > > + return 0; > > + } > > + > > + return hugetlb_optimize_vmemmap_pages(h); > > +} > > + > > void hugetlb_vmemmap_free(struct hstate *h, struct page *head) > > { > > unsigned long vmemmap_addr = (unsigned long)head; > > unsigned long vmemmap_end, vmemmap_reuse, vmemmap_pages; > > > > - vmemmap_pages = hugetlb_optimize_vmemmap_pages(h); > > + vmemmap_pages = optimizable_vmemmap_pages(h, head); > > if (!vmemmap_pages) > > return; > > > > - if (READ_ONCE(vmemmap_optimize_mode) == VMEMMAP_OPTIMIZE_OFF) > > - return; > > - > > static_branch_inc(&hugetlb_optimize_vmemmap_key); > > > > vmemmap_addr += RESERVE_VMEMMAP_SIZE; > > @@ -199,10 +213,10 @@ static struct ctl_table hugetlb_vmemmap_sysctls[] = { > > static __init int hugetlb_vmemmap_sysctls_init(void) > > { > > /* > > - * If "memory_hotplug.memmap_on_memory" is enabled or "struct page" > > - * crosses page boundaries, the vmemmap pages cannot be optimized. > > + * If "struct page" crosses page boundaries, the vmemmap pages cannot > > + * be optimized. > > */ > > - if (!mhp_memmap_on_memory() && is_power_of_2(sizeof(struct page))) > > + if (is_power_of_2(sizeof(struct page))) > > register_sysctl_init("vm", hugetlb_vmemmap_sysctls); > > > > return 0; > > diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c > > index 3b360eda933f..7309694c4dee 100644 > > --- a/mm/memory_hotplug.c > > +++ b/mm/memory_hotplug.c > > @@ -43,30 +43,22 @@ > > #include "shuffle.h" > > > > #ifdef CONFIG_MHP_MEMMAP_ON_MEMORY > > -static int memmap_on_memory_set(const char *val, const struct kernel_param *kp) > > -{ > > - if (hugetlb_optimize_vmemmap_enabled()) > > - return 0; > > - return param_set_bool(val, kp); > > -} > > - > > -static const struct kernel_param_ops memmap_on_memory_ops = { > > - .flags = KERNEL_PARAM_OPS_FL_NOARG, > > - .set = memmap_on_memory_set, > > - .get = param_get_bool, > > -}; > > - > > /* > > * memory_hotplug.memmap_on_memory parameter > > */ > > static bool memmap_on_memory __ro_after_init; > > -module_param_cb(memmap_on_memory, &memmap_on_memory_ops, &memmap_on_memory, 0444); > > +module_param(memmap_on_memory, bool, 0444); > > MODULE_PARM_DESC(memmap_on_memory, "Enable memmap on memory for memory hotplug"); > > > > -bool mhp_memmap_on_memory(void) > > +static inline bool mhp_memmap_on_memory(void) > > { > > return memmap_on_memory; > > } > > +#else > > +static inline bool mhp_memmap_on_memory(void) > > +{ > > + return false; > > +} > > #endif > > > > enum { > > diff --git a/mm/sparse.c b/mm/sparse.c > > index cb3bfae64036..1f353bf9ea6b 100644 > > --- a/mm/sparse.c > > +++ b/mm/sparse.c > > @@ -913,6 +913,14 @@ int __meminit sparse_add_section(int nid, unsigned long start_pfn, > > ms = __nr_to_section(section_nr); > > set_section_nid(section_nr, nid); > > __section_mark_present(ms, section_nr); > > + /* > > + * Mark whole section as non-optimizable once there is a subsection > > + * whose vmemmap pages are allocated from alternative allocator. The > > + * early section is always optimizable since the early section's > > + * vmemmap pages do not consider partially being populated. > > + */ > > + if (!early_section(ms) && altmap) > > + section_mark_cannot_optimize_vmemmap(ms); > > > > /* Align memmap to section boundary in the subsection case */ > > if (section_nr_to_pfn(section_nr) != start_pfn) > > > -- > Thanks, > > David / dhildenb > >