From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93FE0C04FFE for ; Tue, 14 May 2024 17:49:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 141A66B0317; Tue, 14 May 2024 13:49:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0CADB8D000D; Tue, 14 May 2024 13:49:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EABDC6B031F; Tue, 14 May 2024 13:49:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id CCD796B0317 for ; Tue, 14 May 2024 13:49:48 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 958EFC12F2 for ; Tue, 14 May 2024 17:49:48 +0000 (UTC) X-FDA: 82117739256.07.E9A3625 Received: from mail-ed1-f44.google.com (mail-ed1-f44.google.com [209.85.208.44]) by imf05.hostedemail.com (Postfix) with ESMTP id AD312100010 for ; Tue, 14 May 2024 17:49:46 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=QwtJTNji; spf=pass (imf05.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.208.44 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1715708986; a=rsa-sha256; cv=none; b=oaIAwSlAclChI6P7ABE4I8lHMbrQeoxb3D7JwtIhUxcOz+vuE89FahsT3rdyf4MVeadnV3 54pNuvZB+E6mUVzj03iyXl36tdGWDdcYoZRZ5gZUP3R70D3AmQHjbBuBXwJGTxSF4TRSrT Izlr7zQdQN1avPUBmGGyWGmMBqkdTLA= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=rivosinc-com.20230601.gappssmtp.com header.s=20230601 header.b=QwtJTNji; spf=pass (imf05.hostedemail.com: domain of alexghiti@rivosinc.com designates 209.85.208.44 as permitted sender) smtp.mailfrom=alexghiti@rivosinc.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1715708986; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=TY0tfx5oaYLrx0tYYs3sB08jBexY0Rsr4FzYN9NEWiA=; b=pLA4shiThoZ+Pc+6fbDIcBq/AixDi0ZHTarX+1R/YOJFocPcNvhHii/iKwuZCZ86NN+qQA 4YbhwBM1y2CJkyFVXWRkqE+1NFNV+uTUlJZReQEVuodlFN0JcTQ8/vlINazfg/UIk5zWQm wtq7nHWx8kx2ZBW2K21AMiI/IalNQNs= Received: by mail-ed1-f44.google.com with SMTP id 4fb4d7f45d1cf-574ea5184abso609587a12.3 for ; Tue, 14 May 2024 10:49:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1715708985; x=1716313785; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=TY0tfx5oaYLrx0tYYs3sB08jBexY0Rsr4FzYN9NEWiA=; b=QwtJTNjincKLvuEvgc4bsHR1B+0lT/bZFNajaSr07nZ79PncK7pdax/VHFRn4Q8JE+ 0i7Y9YiY3n6E8ywy7GwKunQE7wBX6VyvjDLPC63yHlAi+1isYWtOwCiOTnbvntZPfoz6 nhU6elZNaSbfyU5kb45qJzOlLGYaBOs4ubpFmtoVoSM0ZHCEx77FqZiyAeZyVRnaVXUZ 5c6YEe9a3XQ9FOTGmvapataR1++epd0PfVvrMMKqnaAsZ7SPYiNr5JkaFTKIOPQDuV2N SMuvxzLVrMUf2YrYfUv+DrsW2lPt5p9SoqHzlxgMTysi1xcC8ZCX68yhX5eDqoEpslsj rn8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715708985; x=1716313785; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TY0tfx5oaYLrx0tYYs3sB08jBexY0Rsr4FzYN9NEWiA=; b=cqvIL1wRxczVzSHlxwjbr79KKUqa1/W11fcTLBBXVhsiK2MD+igp3oct8PTVo3uW86 RslIy+bviRK6B33EpHEdeoy4MSMa33AP7QLBKmxzA4EwIDJLqxfjFjuwdUVhaycVIlUY AWmvkVT7bxlLDN/NKoktD0dWxKD/6a5p0KxxnB5pCf0wGDkUZHtbXybLQ3Zi/Nb3SkiY 25uqLGxG0MRaQrHntuJyWK2JAqLcb0YbHSL6b23Ch3Mp6Y758ymj/y6WEV6V7De1MXbn AyxebjiOk1GC0zDl0I7z8Fx+sH5RsmDrC3m8qt6+7VcOVg+k2+CHMb+iw9Fkuj7uLpPk wOHA== X-Forwarded-Encrypted: i=1; AJvYcCVRlhMepQP/AEyGZ5XE4/fSts8tP9hgZdsQLb/pVOeVZYwQk75+pQiLnSzQongkA8siA/26IgSoPTzzXpCGScjRxGE= X-Gm-Message-State: AOJu0Ywy4MUvD1Vv50gsJfPD12azO/f2GbwkYRiaSzmW+YLUNGGeQNTA eL6s4yb/HjkuuF6ZIMru8iI5lYzUibt60pZ5RbNv9otBLiCMQt/E37oo3ZxU8oe9DLHKR49R9fS gPD8/WDh8a6EY6ssELv+FSLzt6TDzxbQU4OpbbQ== X-Google-Smtp-Source: AGHT+IFg6npd6iD2j23oMq1V5Cn9b9652L/fy/5SUbmfC2mk/UjYsz6+j/RmnTO4+bQNIrS303JyyPb7kJb9Kle1ovg= X-Received: by 2002:a17:906:3197:b0:a59:cf38:5339 with SMTP id a640c23a62f3a-a5a2d55a6e6mr962266966b.6.1715708985105; Tue, 14 May 2024 10:49:45 -0700 (PDT) MIME-Version: 1.0 References: <20240514140446.538622-1-bjorn@kernel.org> <20240514140446.538622-5-bjorn@kernel.org> In-Reply-To: <20240514140446.538622-5-bjorn@kernel.org> From: Alexandre Ghiti Date: Tue, 14 May 2024 19:49:33 +0200 Message-ID: Subject: Re: [PATCH v2 4/8] riscv: mm: Add memory hotplugging support To: =?UTF-8?B?QmrDtnJuIFTDtnBlbA==?= Cc: Albert Ou , David Hildenbrand , Palmer Dabbelt , Paul Walmsley , linux-riscv@lists.infradead.org, =?UTF-8?B?QmrDtnJuIFTDtnBlbA==?= , Andrew Bresticker , Chethan Seshadri , Lorenzo Stoakes , Oscar Salvador , Santosh Mamila , Sivakumar Munnangi , Sunil V L , linux-kernel@vger.kernel.org, linux-mm@kvack.org, virtualization@lists.linux-foundation.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: AD312100010 X-Rspam-User: X-Rspamd-Server: rspam12 X-Stat-Signature: xe8djgom16z37f9z4f44dfaqczof5zx9 X-HE-Tag: 1715708986-965073 X-HE-Meta: U2FsdGVkX180VMf5geQvo56GeIGGEjzFlfb33alCE7paYiNTRonffHMwmM9Ha0/cZxQShC5vv7bdtkDRvQw1AXztT5YZbMKssrU8P2LNXENM90alPdq8zC+0zq9bnop+RciLVNNTYHBCiEH1tloOffTdKKhmI6X4R9f9ponrrx5oJkzrbEexbZMqusE2j0XX442Mu0dCxz7ghqD/A+TWSg99apt0uAso4amB9IJLzmejEY8y+XqV3NnNUKmplKckA9o9P7CUAPeFoTXkA81hTa+rf9gpiqr+rbowhL7OQNTIRzQjvQQqqHgYWtBX2e4ZmapKz+fc6pt/tr0oMcniJv3y17yndEabqZd6e+WTvM7JNzNebj1COFbVib/ROUEQy9yC/cpc5UicV5dgkDnmyHtwiwTv8RBlEHhpkTB1u6xJ5Cbpn6F3dhflwaIT+O3gTOF2TD/GkUjDzawXl494WnVqHGDKdKYcDfy0nSjU6EDmpp4EbLL6DuMiKww5OUXO2C0sLomkY86eZDA6EHjIT/h1KpCAn2AjOIf2RObM2Bx+IE4vrHyitsCBlOHD5rUZzyMTenufZl+lozmnvNN/D0NzeL+D4PaPUVvaNa3dCedmcxTD/cEOJpqi80I4Y8xe4IaebXAl41cq9weVviMmTRkeQhsovbUkwzzS6Ob3KZhzpcTSPChP3MnAvVqm+jX1ceyVFAX8X3oilicwMwOUHNjIdBKmJRqim0MPMUwhYS0MThdXBdzOI+bMoeNPBcYjbHijSE5/KHmPfVUU0MlzsFDCXpI7m6BswDXWfyHAdCyWBnmwCAr9ONXtujabOtUyDxfYcTiNJnytHbCNSzTpPifzw8ScC7Z5vRNIWSgj+7ilw4dcIwnvtvUmxr/vrjD7taCPfSUOxo8EWh8mWXGalF8XTRq5AoHwVlMHKxCSNssq8Gwr/B+SPPXSQIAFS22cZkq81jclG7WKVyAlD+6 QWAmOyG3 voqoQqlLJfuMUYwr7wTXPUIum0B3Yz4laY8rk8U1cPCrqa944MziySw1LIQlEHgWJ4wMnYcHqgSoDpEp7oGD5faZu8Z2dkl1cEEE13ke9WUPu3CI75AEGyFcdqQNDHYlVRjDvs7miDmHgn0S03+51HLCWKP8mMrcLVsxdQPE77SU3dUMhyyxkit1jLfZot1/mcpgXgV7sGSeQ3PNz6OSUwBMsunGfWVoiOTWAhg3lVPp3KS37nNEXxaSUgdYO9rYYPz0GMrYl8dMs8g717utH1U9sLPTbn7vLwuQWbwpZVGm1HxQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, May 14, 2024 at 4:05=E2=80=AFPM Bj=C3=B6rn T=C3=B6pel wrote: > > From: Bj=C3=B6rn T=C3=B6pel > > For an architecture to support memory hotplugging, a couple of > callbacks needs to be implemented: > > arch_add_memory() > This callback is responsible for adding the physical memory into the > direct map, and call into the memory hotplugging generic code via > __add_pages() that adds the corresponding struct page entries, and > updates the vmemmap mapping. > > arch_remove_memory() > This is the inverse of the callback above. > > vmemmap_free() > This function tears down the vmemmap mappings (if > CONFIG_SPARSEMEM_VMEMMAP is enabled), and also deallocates the > backing vmemmap pages. Note that for persistent memory, an > alternative allocator for the backing pages can be used; The > vmem_altmap. This means that when the backing pages are cleared, > extra care is needed so that the correct deallocation method is > used. > > arch_get_mappable_range() > This functions returns the PA range that the direct map can map. > Used by the MHP internals for sanity checks. > > The page table unmap/teardown functions are heavily based on code from > the x86 tree. The same remove_pgd_mapping() function is used in both > vmemmap_free() and arch_remove_memory(), but in the latter function > the backing pages are not removed. > > Signed-off-by: Bj=C3=B6rn T=C3=B6pel > --- > arch/riscv/mm/init.c | 242 +++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 242 insertions(+) > > diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c > index 6f72b0b2b854..7f0b921a3d3a 100644 > --- a/arch/riscv/mm/init.c > +++ b/arch/riscv/mm/init.c > @@ -1493,3 +1493,245 @@ void __init pgtable_cache_init(void) > } > } > #endif > + > +#ifdef CONFIG_MEMORY_HOTPLUG > +static void __meminit free_pte_table(pte_t *pte_start, pmd_t *pmd) > +{ > + pte_t *pte; > + int i; > + > + for (i =3D 0; i < PTRS_PER_PTE; i++) { > + pte =3D pte_start + i; > + if (!pte_none(*pte)) > + return; > + } > + > + free_pages((unsigned long)page_address(pmd_page(*pmd)), 0); > + pmd_clear(pmd); > +} > + > +static void __meminit free_pmd_table(pmd_t *pmd_start, pud_t *pud) > +{ > + pmd_t *pmd; > + int i; > + > + for (i =3D 0; i < PTRS_PER_PMD; i++) { > + pmd =3D pmd_start + i; > + if (!pmd_none(*pmd)) > + return; > + } > + > + free_pages((unsigned long)page_address(pud_page(*pud)), 0); > + pud_clear(pud); > +} > + > +static void __meminit free_pud_table(pud_t *pud_start, p4d_t *p4d) > +{ > + pud_t *pud; > + int i; > + > + for (i =3D 0; i < PTRS_PER_PUD; i++) { > + pud =3D pud_start + i; > + if (!pud_none(*pud)) > + return; > + } > + > + free_pages((unsigned long)page_address(p4d_page(*p4d)), 0); > + p4d_clear(p4d); > +} > + > +static void __meminit free_vmemmap_storage(struct page *page, size_t siz= e, > + struct vmem_altmap *altmap) > +{ > + if (altmap) > + vmem_altmap_free(altmap, size >> PAGE_SHIFT); > + else > + free_pages((unsigned long)page_address(page), get_order(s= ize)); > +} > + > +static void __meminit remove_pte_mapping(pte_t *pte_base, unsigned long = addr, unsigned long end, > + bool is_vmemmap, struct vmem_alt= map *altmap) > +{ > + unsigned long next; > + pte_t *ptep, pte; > + > + for (; addr < end; addr =3D next) { > + next =3D (addr + PAGE_SIZE) & PAGE_MASK; > + if (next > end) > + next =3D end; > + > + ptep =3D pte_base + pte_index(addr); > + pte =3D READ_ONCE(*ptep); > + > + if (!pte_present(*ptep)) > + continue; > + > + pte_clear(&init_mm, addr, ptep); > + if (is_vmemmap) > + free_vmemmap_storage(pte_page(pte), PAGE_SIZE, al= tmap); > + } > +} > + > +static void __meminit remove_pmd_mapping(pmd_t *pmd_base, unsigned long = addr, unsigned long end, > + bool is_vmemmap, struct vmem_alt= map *altmap) > +{ > + unsigned long next; > + pte_t *pte_base; > + pmd_t *pmdp, pmd; > + > + for (; addr < end; addr =3D next) { > + next =3D pmd_addr_end(addr, end); > + pmdp =3D pmd_base + pmd_index(addr); > + pmd =3D READ_ONCE(*pmdp); > + > + if (!pmd_present(pmd)) > + continue; > + > + if (pmd_leaf(pmd)) { > + pmd_clear(pmdp); > + if (is_vmemmap) > + free_vmemmap_storage(pmd_page(pmd), PMD_S= IZE, altmap); > + continue; > + } > + > + pte_base =3D (pte_t *)pmd_page_vaddr(*pmdp); > + remove_pte_mapping(pte_base, addr, next, is_vmemmap, altm= ap); > + free_pte_table(pte_base, pmdp); > + } > +} > + > +static void __meminit remove_pud_mapping(pud_t *pud_base, unsigned long = addr, unsigned long end, > + bool is_vmemmap, struct vmem_alt= map *altmap) > +{ > + unsigned long next; > + pud_t *pudp, pud; > + pmd_t *pmd_base; > + > + for (; addr < end; addr =3D next) { > + next =3D pud_addr_end(addr, end); > + pudp =3D pud_base + pud_index(addr); > + pud =3D READ_ONCE(*pudp); > + > + if (!pud_present(pud)) > + continue; > + > + if (pud_leaf(pud)) { > + if (pgtable_l4_enabled) { > + pud_clear(pudp); > + if (is_vmemmap) > + free_vmemmap_storage(pud_page(pud= ), PUD_SIZE, altmap); > + } > + continue; > + } > + > + pmd_base =3D pmd_offset(pudp, 0); > + remove_pmd_mapping(pmd_base, addr, next, is_vmemmap, altm= ap); > + > + if (pgtable_l4_enabled) > + free_pmd_table(pmd_base, pudp); > + } > +} > + > +static void __meminit remove_p4d_mapping(p4d_t *p4d_base, unsigned long = addr, unsigned long end, > + bool is_vmemmap, struct vmem_alt= map *altmap) > +{ > + unsigned long next; > + p4d_t *p4dp, p4d; > + pud_t *pud_base; > + > + for (; addr < end; addr =3D next) { > + next =3D p4d_addr_end(addr, end); > + p4dp =3D p4d_base + p4d_index(addr); > + p4d =3D READ_ONCE(*p4dp); > + > + if (!p4d_present(p4d)) > + continue; > + > + if (p4d_leaf(p4d)) { > + if (pgtable_l5_enabled) { > + p4d_clear(p4dp); > + if (is_vmemmap) > + free_vmemmap_storage(p4d_page(p4d= ), P4D_SIZE, altmap); > + } > + continue; > + } > + > + pud_base =3D pud_offset(p4dp, 0); > + remove_pud_mapping(pud_base, addr, next, is_vmemmap, altm= ap); > + > + if (pgtable_l5_enabled) > + free_pud_table(pud_base, p4dp); > + } > +} > + > +static void __meminit remove_pgd_mapping(unsigned long va, unsigned long= end, bool is_vmemmap, > + struct vmem_altmap *altmap) > +{ > + unsigned long addr, next; > + p4d_t *p4d_base; > + pgd_t *pgd; > + > + for (addr =3D va; addr < end; addr =3D next) { > + next =3D pgd_addr_end(addr, end); > + pgd =3D pgd_offset_k(addr); > + > + if (!pgd_present(*pgd)) > + continue; > + > + if (pgd_leaf(*pgd)) > + continue; > + > + p4d_base =3D p4d_offset(pgd, 0); > + remove_p4d_mapping(p4d_base, addr, next, is_vmemmap, altm= ap); > + } > + > + flush_tlb_all(); > +} > + > +static void __meminit remove_linear_mapping(phys_addr_t start, u64 size) > +{ > + unsigned long va =3D (unsigned long)__va(start); > + unsigned long end =3D (unsigned long)__va(start + size); > + > + remove_pgd_mapping(va, end, false, NULL); > +} > + > +struct range arch_get_mappable_range(void) > +{ > + struct range mhp_range; > + > + mhp_range.start =3D __pa(PAGE_OFFSET); > + mhp_range.end =3D __pa(PAGE_END - 1); > + return mhp_range; > +} > + > +int __ref arch_add_memory(int nid, u64 start, u64 size, struct mhp_param= s *params) > +{ > + int ret; > + > + create_linear_mapping_range(start, start + size, 0, ¶ms->pgpr= ot); > + flush_tlb_all(); > + ret =3D __add_pages(nid, start >> PAGE_SHIFT, size >> PAGE_SHIFT,= params); > + if (ret) { > + remove_linear_mapping(start, size); > + return ret; > + } > + You need to flush the TLB here too since __add_pages() populates the page table with the new vmemmap mapping (only because riscv allows to cache invalid entries, I'll adapt this in my next version of Svvptc support). > + max_pfn =3D PFN_UP(start + size); > + max_low_pfn =3D max_pfn; > + return 0; > +} > + > +void __ref arch_remove_memory(u64 start, u64 size, struct vmem_altmap *a= ltmap) > +{ > + __remove_pages(start >> PAGE_SHIFT, size >> PAGE_SHIFT, altmap); > + remove_linear_mapping(start, size); You need to flush the TLB here too. > +} > + > +#ifdef CONFIG_SPARSEMEM_VMEMMAP > +void __ref vmemmap_free(unsigned long start, unsigned long end, struct v= mem_altmap *altmap) > +{ > + remove_pgd_mapping(start, end, true, altmap); > +} > +#endif /* CONFIG_SPARSEMEM_VMEMMAP */ > +#endif /* CONFIG_MEMORY_HOTPLUG */ > -- > 2.40.1 >