From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2431E77188 for ; Wed, 8 Jan 2025 10:11:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 491BD6B00A2; Wed, 8 Jan 2025 05:11:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 441716B00A4; Wed, 8 Jan 2025 05:11:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 309B16B00A5; Wed, 8 Jan 2025 05:11:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 1132E6B00A2 for ; Wed, 8 Jan 2025 05:11:46 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id AC2B443219 for ; Wed, 8 Jan 2025 10:11:45 +0000 (UTC) X-FDA: 82983868170.25.B1633C1 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf25.hostedemail.com (Postfix) with ESMTP id 83971A000B for ; Wed, 8 Jan 2025 10:11:43 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=none; spf=pass (imf25.hostedemail.com: domain of anshuman.khandual@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736331104; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2Mr6Dl3c90TC0ugCj5cSJ87+hbG6R2W9QdyHaN19dbQ=; b=PNaNJwkgApKyautKkoxtZoLpdgtEPc8/YAXS5TFsqCC8sBtDGL4QqSNLyKltxTCdBUT7G4 3zm3xLi1g8H5A3tnoA0/7Nr9pySIwcUJqAw0+Jm8f2QuLOHEZh9i3KqToJIL4l/9Jk5/DL PfzoqERQlhawi/gZQRpNjQ8IWQ6+8/g= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=none; spf=pass (imf25.hostedemail.com: domain of anshuman.khandual@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736331104; a=rsa-sha256; cv=none; b=s+q1LarwacbVRmbeNqYzEaAkNSoaDd5aX0XMeTfZCmA9NtrJC5x8cyXDo0sTSAzhd6ogV6 4Yezjka1Xvgp9VStXrmb/SYJG7nELvM4GwkjBWDJDtkfXJtckoYSosLvmMuzS5Z3NXcm53 ARruAL0V/Ma9ryyLter/ooqJ2zpvw1w= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id CB47213D5; Wed, 8 Jan 2025 02:12:10 -0800 (PST) Received: from [10.162.16.95] (a077893.blr.arm.com [10.162.16.95]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2D1833F59E; Wed, 8 Jan 2025 02:11:37 -0800 (PST) Message-ID: Date: Wed, 8 Jan 2025 15:41:35 +0530 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v4] arm64: mm: Populate vmemmap/linear at the page level for hotplugged sections To: Catalin Marinas , Zhenhua Huang Cc: will@kernel.org, ardb@kernel.org, ryan.roberts@arm.com, mark.rutland@arm.com, joey.gouly@arm.com, dave.hansen@linux.intel.com, akpm@linux-foundation.org, chenfeiyang@loongson.cn, chenhuacai@kernel.org, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, quic_tingweiz@quicinc.com, stable@vger.kernel.org References: <20250107074252.1062127-1-quic_zhenhuah@quicinc.com> Content-Language: en-US From: Anshuman Khandual In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 83971A000B X-Rspam-User: X-Stat-Signature: 6oome6ohqh7pm6hkwwr6owib85z17uz4 X-HE-Tag: 1736331103-521778 X-HE-Meta: U2FsdGVkX18I+vY9sWsFeLZdnnn7d6m1DS5dMP2X6+FQ4UPgsdhCMQ1NiXqUKROVNmUz17GmLgpxcic31fahGE8MmR/iCPzGYkQZ6GRaXd5N8aItd730EdG4tHijtLIjymgQVBxh0bEhHsPcEsdKXTEGzjR1gv+4y5gg+sqe4GTDgi9EqQVDPfD/7x3SGuU/8fepSrrMwD4xlXo5gPZi1xML0/CB8zyPT+NH8hJ+PCuTI4A6HJZ/bDYWE/2/Rnp+rCfF4Vhz/aLOpGDPOTxN+sVYd27XJnJi14fWg8q1w7optrom0eiuoBAmwMsRsbq2SQTMRW6sOPlp8aGbGU1zr2gVIPQMGySwp/cZLI5JqOYaybOmqlgQ1CBB0th7zeSp/C22P/Nr8dP7iUaYC0RwYpvm1cURL8XtHRYQAS4pTLtsDF1V3Cj83ybVIEXv0h0MUP/nVckn33qgCmY/wdQKtSnPXn7iU9co/1uc5RTH8mIjDiA9c+BCWYOKNk2e8X5Ymbux23USTY5K70KVMf/5adfWiO5DikT3az67Ku8T/LhD8htlaRD3xu5iRLRsN+2iWZvpdIXea2b9OzTDssfYjfEb0unGUOxmYCgvkQ5CEfk+VRDC/SN/HwOwdDHpRzuwznaBQpqEcOgqQlZULl+nfoTA426s+/Mo9T8R4NYfgd2j+xzQpCNufcG4XXuGhg+qcTH8qdksf9wfLNlcRRe3zEvB7abgRrcpSnFyu9NBycuPCFeVSGDLlQGDNzNv+kcnV+VHhPuJJfrJ4Oh9us64G00mZLIOBRm3rLXxD0aSS9rsGyy3yU4SErW+jSZT3/kOULci4HXOsecrJF1049lZUpDxszMfPvSZVwnfV/DOt9QdIKR7n9umuw1fJWH2NLa2rInz9UOq68nsUf+4DODfKHg+bVb6veJyts5AIwkFFbXNo/FvTaz4V655FUwyxxaNYGjxwA4UwL8YYgDyGxm 6CzBkgNf HTtyIhPHRAWBS+C2IemLX1X2p2T5U6ywbWc572vhXOXt8KZAQRFR+2+gclNWxr0rVMwTiDfyeKyguTfU9SlMVu89wS8EUcS0u1iNC3HmjlGGg/QWsujDfYqEs80bSYKTHjfC02cUmTBSFnUx2I05SjIZfQg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 1/8/25 00:52, Catalin Marinas wrote: > On Tue, Jan 07, 2025 at 03:42:52PM +0800, Zhenhua Huang wrote: >> diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c >> index e2739b69e11b..5e0f514de870 100644 >> --- a/arch/arm64/mm/mmu.c >> +++ b/arch/arm64/mm/mmu.c >> @@ -42,9 +42,11 @@ >> #include >> #include >> >> -#define NO_BLOCK_MAPPINGS BIT(0) >> -#define NO_CONT_MAPPINGS BIT(1) >> -#define NO_EXEC_MAPPINGS BIT(2) /* assumes FEAT_HPDS is not used */ >> +#define NO_PMD_BLOCK_MAPPINGS BIT(0) >> +#define NO_PUD_BLOCK_MAPPINGS BIT(1) /* Hotplug case: do not want block mapping for PUD */ >> +#define NO_BLOCK_MAPPINGS (NO_PMD_BLOCK_MAPPINGS | NO_PUD_BLOCK_MAPPINGS) > > Nit: please use a tab instead of space before (NO_PMD_...) > >> +#define NO_CONT_MAPPINGS BIT(2) >> +#define NO_EXEC_MAPPINGS BIT(3) /* assumes FEAT_HPDS is not used */ >> >> u64 kimage_voffset __ro_after_init; >> EXPORT_SYMBOL(kimage_voffset); >> @@ -254,7 +256,7 @@ static void init_pmd(pmd_t *pmdp, unsigned long addr, unsigned long end, >> >> /* try section mapping first */ >> if (((addr | next | phys) & ~PMD_MASK) == 0 && >> - (flags & NO_BLOCK_MAPPINGS) == 0) { >> + (flags & NO_PMD_BLOCK_MAPPINGS) == 0) { >> pmd_set_huge(pmdp, phys, prot); >> >> /* >> @@ -356,10 +358,11 @@ static void alloc_init_pud(p4d_t *p4dp, unsigned long addr, unsigned long end, >> >> /* >> * For 4K granule only, attempt to put down a 1GB block >> + * Hotplug case: do not attempt 1GB block >> */ > > I don't think we need this comment added here. The hotplug case is a > decision of the caller, so better to have the comment there. Agreed. > >> if (pud_sect_supported() && >> ((addr | next | phys) & ~PUD_MASK) == 0 && >> - (flags & NO_BLOCK_MAPPINGS) == 0) { >> + (flags & NO_PUD_BLOCK_MAPPINGS) == 0) { >> pud_set_huge(pudp, phys, prot); > > Nit: something wrong with the alignment here. I think the unmodified > line after the 'if' one above was misaligned before your patch. > >> >> /* >> @@ -1175,9 +1178,21 @@ int __meminit vmemmap_check_pmd(pmd_t *pmdp, int node, >> int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, >> struct vmem_altmap *altmap) >> { >> + unsigned long start_pfn; >> + struct mem_section *ms; >> + >> WARN_ON((start < VMEMMAP_START) || (end > VMEMMAP_END)); >> >> - if (!IS_ENABLED(CONFIG_ARM64_4K_PAGES)) >> + start_pfn = page_to_pfn((struct page *)start); >> + ms = __pfn_to_section(start_pfn); > > Hmm, it would have been better if the core code provided the start pfn > as it does for vmemmap_populate_compound_pages() but I'm fine with > deducting it from 'start'. Right, that will require changing arguments in generic vmemmap_populate(). > >> + /* >> + * Hotplugged section does not support hugepages as >> + * PMD_SIZE (hence PUD_SIZE) section mapping covers >> + * struct page range that exceeds a SUBSECTION_SIZE >> + * i.e 2MB - for all available base page sizes. >> + */ >> + if (!IS_ENABLED(CONFIG_ARM64_4K_PAGES) || !early_section(ms)) >> return vmemmap_populate_basepages(start, end, node, altmap); >> else >> return vmemmap_populate_hugepages(start, end, node, altmap); >> @@ -1339,9 +1354,25 @@ int arch_add_memory(int nid, u64 start, u64 size, >> struct mhp_params *params) >> { >> int ret, flags = NO_EXEC_MAPPINGS; >> + unsigned long start_pfn = page_to_pfn((struct page *)start); >> + struct mem_section *ms = __pfn_to_section(start_pfn); > > This looks wrong. 'start' here is a physical address, you want > PFN_DOWN() instead. Agreed. > >> >> VM_BUG_ON(!mhp_range_allowed(start, size, true)); >> >> + /* should not be invoked by early section */ >> + WARN_ON(early_section(ms)); >> + >> + /* >> + * 4K base page's PMD_SIZE matches SUBSECTION_SIZE i.e 2MB. Hence >> + * PMD section mapping can be allowed, but only for 4K base pages. >> + * Where as PMD_SIZE (hence PUD_SIZE) for other page sizes exceed >> + * SUBSECTION_SIZE. >> + */ >> + if (IS_ENABLED(CONFIG_ARM64_4K_PAGES)) >> + flags |= NO_PUD_BLOCK_MAPPINGS | NO_CONT_MAPPINGS; > > In theory we can allow contiguous PTE mappings but not PMD. You could > probably do the same as a NO_BLOCK_MAPPINGS and split it into multiple > components - NO_PTE_CONT_MAPPINGS and so on. That's a good idea. > >> + else >> + flags |= NO_BLOCK_MAPPINGS | NO_CONT_MAPPINGS; > > Similarly with 16K/64K pages we can allow contiguous PTEs as they all go > up to 2MB blocks. > > I think we should write the flags setup in a more readable way than > trying to do mental maths on the possible combinations, something like: > > flags = NO_PUD_BLOCK_MAPPINGS | NO_PMD_CONT_MAPPINGS; > if (SUBSECTION_SHIFT < PMD_SHIFT) > flags |= NO_PMD_BLOCK_MAPPINGS; > if (SUBSECTION_SHIFT < CONT_PTE_SHIFT) > flags |= NO_PTE_CONT_MAPPINGS; Just wondering why not start with PUD level itself ? Although SUBSECTION_SHIFT might never reach the PUD level but this will help keep the flags calculations bit simple and ready for all future changes. flags = 0; if (SUBSECTION_SHIFT < PUD_SHIFT) flags |= NO_PUD_BLOCK_MAPPINGS; if (SUBSECTION_SHIFT < CONT_PMD_SHIFT) flags |= NO_PMD_CONT_MAPPINGS; > > This way we don't care about the page size and should cover any changes > to SUBSECTION_SHIFT making it smaller than 2MB. > Agreed.