From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7AF4C433EF for ; Tue, 28 Jun 2022 03:22:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 58A378E0002; Mon, 27 Jun 2022 23:22:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 512F88E0001; Mon, 27 Jun 2022 23:22:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 38D818E0002; Mon, 27 Jun 2022 23:22:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 223C98E0001 for ; Mon, 27 Jun 2022 23:22:30 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id ED26D120FC1 for ; Tue, 28 Jun 2022 03:22:29 +0000 (UTC) X-FDA: 79626196818.26.DF6CD31 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf05.hostedemail.com (Postfix) with ESMTP id 49850100027 for ; Tue, 28 Jun 2022 03:22:29 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id B9ABEB81C0E for ; Tue, 28 Jun 2022 03:22:27 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 76271C341CD for ; Tue, 28 Jun 2022 03:22:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1656386546; bh=IX9GpJCf5zFk83qLHuc5OtFoLqfpiJ8Lh1FdnhW9AN8=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=VQrIYAO6vwg+696izoP5SrdL119ptxuLxDTzlMy5952K7SVAji6NzqdfgeYKpO57Q Be2+pA3wgSkglkl1si+HjfhLb2sx8Z89pnuJsoe6YdX0xyqIoeIp4jZ8fXvLgoQ4rw PtBN3woDHU2blgSxAFrIjEjaL9+U9zHkmXqbxo1dul56S+5hzgZfuJ3m0y6BPcEk+p 3mA5xVn4OYmK3xC92hv3mcftMjiqmsjALEGEwFGKs8XjGvstBkv/E9m/nkXNB7eOWJ NS169tFOgOZRQ5G+8UrwZIlb8LGuYbc3zFyEGH0VgCB44a5+7zKmopaKI/y2gStc0c ddrPGp7EbHZPw== Received: by mail-ua1-f42.google.com with SMTP id s4so990261uad.0 for ; Mon, 27 Jun 2022 20:22:26 -0700 (PDT) X-Gm-Message-State: AJIora9Hur8eoRkwC6UA0mxVnCQmJc69bBN7Br17b7zQMjdo3K5DBpTJ j8LfnVXVG7PbJDK3WyXA2qlXT9babDZT3RPaan8= X-Google-Smtp-Source: AGRyM1sjidVZYwRvP3tNHlyivh9KqOslGHyvzIkw0RPJ6Lxv4Z7nL1Y3kfPSPQa8e7yoJah9FVuf86MK62jV+u504Gw= X-Received: by 2002:a9f:2c9a:0:b0:381:c1c7:82a6 with SMTP id w26-20020a9f2c9a000000b00381c1c782a6mr4752539uaj.23.1656386545380; Mon, 27 Jun 2022 20:22:25 -0700 (PDT) MIME-Version: 1.0 References: <20220625095459.3786827-1-chenhuacai@loongson.cn> <20220625095459.3786827-2-chenhuacai@loongson.cn> In-Reply-To: From: Huacai Chen Date: Tue, 28 Jun 2022 11:22:14 +0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH 2/3] LoongArch: Add sparse memory vmemmap support To: Muchun Song Cc: Huacai Chen , Arnd Bergmann , Thomas Bogendoerfer , Dave Hansen , Andy Lutomirski , Peter Zijlstra , Catalin Marinas , Will Deacon , loongarch@lists.linux.dev, linux-arch , Xuefeng Li , Guo Ren , Xuerui Wang , Jiaxun Yang , Andrew Morton , Linux Memory Management List , "open list:MIPS" , LKML , LAK , Feiyang Chen , Min Zhou Content-Type: text/plain; charset="UTF-8" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656386549; a=rsa-sha256; cv=none; b=La2sWzkrxhXbdqwLl4moVh49Es9OTK84724zYF8dJNRxLzHv4Xggz11iMx2WcWRfmEStMU Bt43XLqzPAB4UFEeMJimKNl/i844y2aa79imRcYUCALJjtLzH3NQhkCE2IOIo8jyg8FS2+ VJr60Mp2oPqsPlb46WK0G6phkSPltvA= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=VQrIYAO6; spf=pass (imf05.hostedemail.com: domain of chenhuacai@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=chenhuacai@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656386549; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=DRQZS+wO2+PjOUJoLYoaHvlWGefOcXK2lUi53DdAk/I=; b=GtQJ1s/1poc/AkC/EPw1mgSyLdz6H/SiSuNXJ/cdBDN4pbafGwQFNcZpR7ObMX9MLqJOnV SghXwaA/NbsiwyO/mbW+ChuXZkWPH4q+LdGTxFR/nSRPRxevvTCV6bPrbo4AbXQFVuL/yb 6tO4THMk0EyyMOxHLY9E1vAA6KLWHGk= X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 49850100027 X-Rspam-User: X-Stat-Signature: wo7syp34gf3ziz33tqsemk8iyfzywg1y Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=VQrIYAO6; spf=pass (imf05.hostedemail.com: domain of chenhuacai@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=chenhuacai@kernel.org; dmarc=pass (policy=none) header.from=kernel.org X-HE-Tag: 1656386549-997975 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi, Muchun, On Mon, Jun 27, 2022 at 6:33 PM Muchun Song wrote: > > On Sat, Jun 25, 2022 at 5:54 PM Huacai Chen wrote: > > > > From: Feiyang Chen > > > > Add sparse memory vmemmap support for LoongArch. SPARSEMEM_VMEMMAP > > uses a virtually mapped memmap to optimise pfn_to_page and page_to_pfn > > operations. This is the most efficient option when sufficient kernel > > resources are available. > > > > Signed-off-by: Min Zhou > > Signed-off-by: Huacai Chen > > Signed-off-by: Feiyang Chen > > --- > > arch/loongarch/Kconfig | 2 + > > arch/loongarch/include/asm/pgtable.h | 5 +- > > arch/loongarch/include/asm/sparsemem.h | 8 +++ > > arch/loongarch/mm/init.c | 71 +++++++++++++++++++++++++- > > include/linux/mm.h | 2 + > > mm/sparse-vmemmap.c | 10 ++++ > > 6 files changed, 96 insertions(+), 2 deletions(-) > > > > diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig > > index dc19cf3071ea..8e56ca28165e 100644 > > --- a/arch/loongarch/Kconfig > > +++ b/arch/loongarch/Kconfig > > @@ -49,6 +49,7 @@ config LOONGARCH > > select ARCH_USE_QUEUED_RWLOCKS > > select ARCH_USE_QUEUED_SPINLOCKS > > select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT > > + select ARCH_WANT_HUGETLB_PAGE_OPTIMIZE_VMEMMAP > > I think this should be a separate patch to enable HVO (HugeTLB Vmemmap > Optimization) since it is irrelevant to this patch. It seems I have misunderstood HVO, then I will remove HVO parts from this patch. Thank you. Huacai > > Thanks. > > > select ARCH_WANTS_NO_INSTR > > select BUILDTIME_TABLE_SORT > > select COMMON_CLK > > @@ -422,6 +423,7 @@ config ARCH_FLATMEM_ENABLE > > > > config ARCH_SPARSEMEM_ENABLE > > def_bool y > > + select SPARSEMEM_VMEMMAP_ENABLE > > help > > Say Y to support efficient handling of sparse physical memory, > > for architectures which are either NUMA (Non-Uniform Memory Access) > > diff --git a/arch/loongarch/include/asm/pgtable.h b/arch/loongarch/include/asm/pgtable.h > > index 9c811c3f7572..b701ec7a0309 100644 > > --- a/arch/loongarch/include/asm/pgtable.h > > +++ b/arch/loongarch/include/asm/pgtable.h > > @@ -92,7 +92,10 @@ extern unsigned long zero_page_mask; > > #define VMALLOC_START MODULES_END > > #define VMALLOC_END \ > > (vm_map_base + \ > > - min(PTRS_PER_PGD * PTRS_PER_PUD * PTRS_PER_PMD * PTRS_PER_PTE * PAGE_SIZE, (1UL << cpu_vabits)) - PMD_SIZE) > > + min(PTRS_PER_PGD * PTRS_PER_PUD * PTRS_PER_PMD * PTRS_PER_PTE * PAGE_SIZE, (1UL << cpu_vabits)) - PMD_SIZE - VMEMMAP_SIZE) > > + > > +#define vmemmap ((struct page *)((VMALLOC_END + PMD_SIZE) & PMD_MASK)) > > +#define VMEMMAP_END ((unsigned long)vmemmap + VMEMMAP_SIZE - 1) > > > > #define pte_ERROR(e) \ > > pr_err("%s:%d: bad pte %016lx.\n", __FILE__, __LINE__, pte_val(e)) > > diff --git a/arch/loongarch/include/asm/sparsemem.h b/arch/loongarch/include/asm/sparsemem.h > > index 3d18cdf1b069..a1e440f6bec7 100644 > > --- a/arch/loongarch/include/asm/sparsemem.h > > +++ b/arch/loongarch/include/asm/sparsemem.h > > @@ -11,6 +11,14 @@ > > #define SECTION_SIZE_BITS 29 /* 2^29 = Largest Huge Page Size */ > > #define MAX_PHYSMEM_BITS 48 > > > > +#ifndef CONFIG_SPARSEMEM_VMEMMAP > > +#define VMEMMAP_SIZE 0 > > +#else > > +#define VMEMMAP_SIZE (sizeof(struct page) * (1UL << (cpu_pabits + 1 - PAGE_SHIFT))) > > +#endif > > + > > +#include > > + > > #endif /* CONFIG_SPARSEMEM */ > > > > #ifdef CONFIG_MEMORY_HOTPLUG > > diff --git a/arch/loongarch/mm/init.c b/arch/loongarch/mm/init.c > > index 7094a68c9b83..35128229fe46 100644 > > --- a/arch/loongarch/mm/init.c > > +++ b/arch/loongarch/mm/init.c > > @@ -22,7 +22,7 @@ > > #include > > #include > > #include > > -#include > > +#include > > #include > > > > #include > > @@ -157,6 +157,75 @@ void arch_remove_memory(u64 start, u64 size, struct vmem_altmap *altmap) > > #endif > > #endif > > > > +#ifdef CONFIG_SPARSEMEM_VMEMMAP > > +int __meminit vmemmap_populate_hugepages(unsigned long start, unsigned long end, > > + int node, struct vmem_altmap *altmap) > > +{ > > + unsigned long addr = start; > > + unsigned long next; > > + pgd_t *pgd; > > + p4d_t *p4d; > > + pud_t *pud; > > + pmd_t *pmd; > > + > > + for (addr = start; addr < end; addr = next) { > > + next = pmd_addr_end(addr, end); > > + > > + pgd = vmemmap_pgd_populate(addr, node); > > + if (!pgd) > > + return -ENOMEM; > > + p4d = vmemmap_p4d_populate(pgd, addr, node); > > + if (!p4d) > > + return -ENOMEM; > > + pud = vmemmap_pud_populate(p4d, addr, node); > > + if (!pud) > > + return -ENOMEM; > > + > > + pmd = pmd_offset(pud, addr); > > + if (pmd_none(*pmd)) { > > + void *p = NULL; > > + > > + p = vmemmap_alloc_block_buf(PMD_SIZE, node, NULL); > > + if (p) { > > + pmd_t entry; > > + > > + entry = pfn_pmd(virt_to_pfn(p), PAGE_KERNEL); > > + pmd_val(entry) |= _PAGE_HUGE | _PAGE_HGLOBAL; > > + set_pmd_at(&init_mm, addr, pmd, entry); > > + > > + continue; > > + } > > + } else if (pmd_val(*pmd) & _PAGE_HUGE) { > > + vmemmap_verify((pte_t *)pmd, node, addr, next); > > + continue; > > + } > > + if (vmemmap_populate_basepages(addr, next, node, NULL)) > > + return -ENOMEM; > > + } > > + > > + return 0; > > +} > > + > > +#if CONFIG_PGTABLE_LEVELS == 2 > > +int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, > > + struct vmem_altmap *altmap) > > +{ > > + return vmemmap_populate_basepages(start, end, node, NULL); > > +} > > +#else > > +int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, > > + struct vmem_altmap *altmap) > > +{ > > + return vmemmap_populate_hugepages(start, end, node, NULL); > > +} > > +#endif > > + > > +void vmemmap_free(unsigned long start, unsigned long end, > > + struct vmem_altmap *altmap) > > +{ > > +} > > +#endif > > + > > /* > > * Align swapper_pg_dir in to 64K, allows its address to be loaded > > * with a single LUI instruction in the TLB handlers. If we used > > diff --git a/include/linux/mm.h b/include/linux/mm.h > > index bc8f326be0ce..3472b924a1ea 100644 > > --- a/include/linux/mm.h > > +++ b/include/linux/mm.h > > @@ -3203,6 +3203,8 @@ void *sparse_buffer_alloc(unsigned long size); > > struct page * __populate_section_memmap(unsigned long pfn, > > unsigned long nr_pages, int nid, struct vmem_altmap *altmap, > > struct dev_pagemap *pgmap); > > +void pmd_init(void *addr); > > +void pud_init(void *addr); > > pgd_t *vmemmap_pgd_populate(unsigned long addr, int node); > > p4d_t *vmemmap_p4d_populate(pgd_t *pgd, unsigned long addr, int node); > > pud_t *vmemmap_pud_populate(p4d_t *p4d, unsigned long addr, int node); > > diff --git a/mm/sparse-vmemmap.c b/mm/sparse-vmemmap.c > > index f4fa61dbbee3..33e2a1ceee72 100644 > > --- a/mm/sparse-vmemmap.c > > +++ b/mm/sparse-vmemmap.c > > @@ -587,6 +587,10 @@ pmd_t * __meminit vmemmap_pmd_populate(pud_t *pud, unsigned long addr, int node) > > return pmd; > > } > > > > +void __weak __meminit pmd_init(void *addr) > > +{ > > +} > > + > > pud_t * __meminit vmemmap_pud_populate(p4d_t *p4d, unsigned long addr, int node) > > { > > pud_t *pud = pud_offset(p4d, addr); > > @@ -594,11 +598,16 @@ pud_t * __meminit vmemmap_pud_populate(p4d_t *p4d, unsigned long addr, int node) > > void *p = vmemmap_alloc_block_zero(PAGE_SIZE, node); > > if (!p) > > return NULL; > > + pmd_init(p); > > pud_populate(&init_mm, pud, p); > > } > > return pud; > > } > > > > +void __weak __meminit pud_init(void *addr) > > +{ > > +} > > + > > p4d_t * __meminit vmemmap_p4d_populate(pgd_t *pgd, unsigned long addr, int node) > > { > > p4d_t *p4d = p4d_offset(pgd, addr); > > @@ -606,6 +615,7 @@ p4d_t * __meminit vmemmap_p4d_populate(pgd_t *pgd, unsigned long addr, int node) > > void *p = vmemmap_alloc_block_zero(PAGE_SIZE, node); > > if (!p) > > return NULL; > > + pud_init(p); > > p4d_populate(&init_mm, p4d, p); > > } > > return p4d; > > -- > > 2.27.0 > >