From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49320C43334 for ; Tue, 21 Jun 2022 20:53:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D3C328E004F; Tue, 21 Jun 2022 16:53:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CECCC8E004B; Tue, 21 Jun 2022 16:53:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BB4938E004F; Tue, 21 Jun 2022 16:53:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id A90788E004B for ; Tue, 21 Jun 2022 16:53:33 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 80333605F6 for ; Tue, 21 Jun 2022 20:53:33 +0000 (UTC) X-FDA: 79603443906.02.CF3A4CF Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf21.hostedemail.com (Postfix) with ESMTP id ECFC61C00B5 for ; Tue, 21 Jun 2022 20:53:32 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 261EB61856; Tue, 21 Jun 2022 20:53:32 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5E70BC3411C; Tue, 21 Jun 2022 20:53:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1655844811; bh=20N4ZigNt3ywDZXkpxWcD1TBryesu5HI23JwFB7LT6Y=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=Ki9Ly2I17JqvfHtT9RHEyv92O0rFsOjs8eDP3o2/BGTxMObJIPpQZgeqVnU/Vn8VT +qrGnD6sUvLGN1MjXJtGByEvIB0XWyBMMCbt0IuJN1hocwlnH54YS62IhyT+LYf+is 01mlP/9fWSPpl+j6bDoLXjDXLRZzJ8Df5WNTKDDE= Date: Tue, 21 Jun 2022 13:53:13 -0700 From: Andrew Morton To: Muchun Song Cc: corbet@lwn.net, david@redhat.com, mike.kravetz@oracle.com, osalvador@suse.de, paulmck@kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, smuchun@gmail.com Subject: Re: [PATCH v5 0/2] make hugetlb_optimize_vmemmap compatible with memmap_on_memory Message-Id: <20220621135313.ae6fbc28338f1220328694f7@linux-foundation.org> In-Reply-To: <20220620110616.12056-1-songmuchun@bytedance.com> References: <20220620110616.12056-1-songmuchun@bytedance.com> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1655844813; a=rsa-sha256; cv=none; b=dV08gMJX9yblC8uXqz9E2B7ffKoxe3zrO05em0CNYOa9T3R6DjG3pM7ls7nPui40dl7Knp Z0CM7kXGN/BoZ5nwz4tRYMgmbD72mleY5v9in7uQ+RTjKyO19rpR5VIQ36Xz5Z1oITCTFn 2FE+coyxPrXZpSL5W9joLutGyfplBzE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655844813; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9FXTHFop3kkjFqLBfGk6/F+TDpGux0CjvepXrPYDEOQ=; b=u1qzCZnM1/0CkvPnjDqYfv13RjbGYOl43qlz4fMfOZcHMaLB3xzJMSN4FgjVt6bU0H790s rnpzF9asrbqyJQoVPW0Zd1tBk9umFQNfjmkofFOidbPa+QZGJgUgcB0cs8cqZ3zRGwkxtO kLmIupmdMcoU17Oz8qMkxjLpaSBR6No= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=Ki9Ly2I1; dmarc=none; spf=pass (imf21.hostedemail.com: domain of akpm@linux-foundation.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=Ki9Ly2I1; dmarc=none; spf=pass (imf21.hostedemail.com: domain of akpm@linux-foundation.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: ECFC61C00B5 X-Stat-Signature: 3ucrakdeytp1oeqdrrpun5czztc5ripb X-HE-Tag: 1655844812-977145 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, 20 Jun 2022 19:06:14 +0800 Muchun Song wrote: > This series makes hugetlb_optimize_vmemmap compatible with memmap_on_memory > and is based on mm-stable. The reason refers to the patch 2's commit log. > > v5: > - Replace enum to defines per David. > - Walk vmemmap page tables to avoid false-positive. I can't see this second change in the v3->v5 deltas? From: Muchun Song Subject: mm-memory_hotplug-enumerate-all-supported-section-flags-v5 Date: Mon, 20 Jun 2022 19:06:15 +0800 replace enum with defines per David Link: https://lkml.kernel.org/r/20220620110616.12056-2-songmuchun@bytedance.com Signed-off-by: Muchun Song Signed-off-by: Andrew Morton --- include/linux/mmzone.h | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) --- a/include/linux/mmzone.h~mm-memory_hotplug-enumerate-all-supported-section-flags-v5 +++ a/include/linux/mmzone.h @@ -1439,16 +1439,13 @@ enum { SECTION_MAP_LAST_BIT, }; -enum { - SECTION_MARKED_PRESENT = BIT(SECTION_MARKED_PRESENT_BIT), - SECTION_HAS_MEM_MAP = BIT(SECTION_HAS_MEM_MAP_BIT), - SECTION_IS_ONLINE = BIT(SECTION_IS_ONLINE_BIT), - SECTION_IS_EARLY = BIT(SECTION_IS_EARLY_BIT), +#define SECTION_MARKED_PRESENT BIT(SECTION_MARKED_PRESENT_BIT) +#define SECTION_HAS_MEM_MAP BIT(SECTION_HAS_MEM_MAP_BIT) +#define SECTION_IS_ONLINE BIT(SECTION_IS_ONLINE_BIT) +#define SECTION_IS_EARLY BIT(SECTION_IS_EARLY_BIT) #ifdef CONFIG_ZONE_DEVICE - SECTION_TAINT_ZONE_DEVICE = BIT(SECTION_TAINT_ZONE_DEVICE_BIT), +#define SECTION_TAINT_ZONE_DEVICE BIT(SECTION_TAINT_ZONE_DEVICE_BIT) #endif -}; - #define SECTION_MAP_MASK (~(BIT(SECTION_MAP_LAST_BIT) - 1)) #define SECTION_NID_SHIFT SECTION_MAP_LAST_BIT _ From: Muchun Song Subject: mm-memory_hotplug-make-hugetlb_optimize_vmemmap-compatible-with-memmap_on_memory-v5 Date: Mon, 20 Jun 2022 19:06:16 +0800 walk vmemmap page tables to avoid false-positive Link: https://lkml.kernel.org/r/20220620110616.12056-3-songmuchun@bytedance.com Signed-off-by: Muchun Song Co-developed-by: Oscar Salvador Signed-off-by: Oscar Salvador Signed-off-by: Andrew Morton --- mm/hugetlb_vmemmap.c | 69 ++++++++++++++++++++++++++--------------- 1 file changed, 44 insertions(+), 25 deletions(-) --- a/mm/hugetlb_vmemmap.c~mm-memory_hotplug-make-hugetlb_optimize_vmemmap-compatible-with-memmap_on_memory-v5 +++ a/mm/hugetlb_vmemmap.c @@ -10,6 +10,7 @@ */ #define pr_fmt(fmt) "HugeTLB: " fmt +#include #include "hugetlb_vmemmap.h" /* @@ -99,34 +100,52 @@ int hugetlb_vmemmap_alloc(struct hstate static unsigned int vmemmap_optimizable_pages(struct hstate *h, struct page *head) { - struct mem_section *ms; - struct page *vmemmap_page; - unsigned long pfn = page_to_pfn(head); - if (READ_ONCE(vmemmap_optimize_mode) == VMEMMAP_OPTIMIZE_OFF) return 0; - ms = __pfn_to_section(pfn); - vmemmap_page = sparse_decode_mem_map(ms->section_mem_map, - pfn_to_section_nr(pfn)); - /* - * Only the vmemmap pages' vmemmap may be marked as VmemmapSelfHosted. - * - * Due to HugeTLB alignment requirements, and the vmemmap pages being - * at the start of the hotplugged memory region. Checking any vmemmap - * page's vmemmap is fine. - * - * [ hotplugged memory ] - * [ vmemmap ][ usable memory ] - * ^ | | | - * +---+ | | - * ^ | | - * +--------+ | - * ^ | - * +-----------------+ - */ - if (PageVmemmapSelfHosted(vmemmap_page)) - return 0; + if (IS_ENABLED(CONFIG_MEMORY_HOTPLUG)) { + pmd_t *pmdp, pmd; + struct page *vmemmap_page; + unsigned long vaddr = (unsigned long)head; + + /* + * Only the vmemmap page's vmemmap page can be self-hosted. + * Walking the page tables to find the backing page of the + * vmemmap page. + */ + pmdp = pmd_off_k(vaddr); + /* + * The READ_ONCE() is used to stabilize *pmdp in a register or + * on the stack so that it will stop changing under the code. + * The only concurrent operation where it can be changed is + * split_vmemmap_huge_pmd() (*pmdp will be stable after this + * operation). + */ + pmd = READ_ONCE(*pmdp); + if (pmd_leaf(pmd)) + vmemmap_page = pmd_page(pmd) + pte_index(vaddr); + else + vmemmap_page = pte_page(*pte_offset_kernel(pmdp, vaddr)); + /* + * Due to HugeTLB alignment requirements and the vmemmap pages + * being at the start of the hotplugged memory region in + * memory_hotplug.memmap_on_memory case. Checking any vmemmap + * page's vmemmap page if it is marked as VmemmapSelfHosted is + * sufficient. + * + * [ hotplugged memory ] + * [ section ][...][ section ] + * [ vmemmap ][ usable memory ] + * ^ | | | + * +---+ | | + * ^ | | + * +-------+ | + * ^ | + * +-------------------------------------------+ + */ + if (PageVmemmapSelfHosted(vmemmap_page)) + return 0; + } return hugetlb_optimize_vmemmap_pages(h); } _