From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E6CAC83F1B for ; Fri, 11 Jul 2025 16:24:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AC8686B00A4; Fri, 11 Jul 2025 12:24:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A78F06B00A5; Fri, 11 Jul 2025 12:24:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 967D56B00A6; Fri, 11 Jul 2025 12:24:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 814446B00A4 for ; Fri, 11 Jul 2025 12:24:38 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 0FB8C1126A7 for ; Fri, 11 Jul 2025 16:24:38 +0000 (UTC) X-FDA: 83652507036.17.5C1151B Received: from mail-wm1-f44.google.com (mail-wm1-f44.google.com [209.85.128.44]) by imf26.hostedemail.com (Postfix) with ESMTP id 0F290140011 for ; Fri, 11 Jul 2025 16:24:35 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=neon.tech header.s=google header.b=SBXXXSBg; spf=pass (imf26.hostedemail.com: domain of sharnoff@neon.tech designates 209.85.128.44 as permitted sender) smtp.mailfrom=sharnoff@neon.tech; dmarc=pass (policy=reject) header.from=neon.tech ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1752251076; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VmqWzINlR7ueURt8ykpnYBNe7vdszeCG5zNyAg94yWQ=; b=K/g9zjtAGCslt1Eh2ZmCHkbE+Qnlk9/xyc2iiBqJTx4jeWcqmgucXdvA/NVUdGwYOqgmNj rP1ccSFoDAwB/C+HnmGjJKDdntW9u+JWXE58jp48YN14YQshThigIQpsZ4cWklwSOySYJk xPNZDNKecdQzue9rsc4izwPEwxd+MyY= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=neon.tech header.s=google header.b=SBXXXSBg; spf=pass (imf26.hostedemail.com: domain of sharnoff@neon.tech designates 209.85.128.44 as permitted sender) smtp.mailfrom=sharnoff@neon.tech; dmarc=pass (policy=reject) header.from=neon.tech ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1752251076; a=rsa-sha256; cv=none; b=NE8tdkOnSym4UobY2hgiTuys3e8fsrO+tdP97Za/PT6PY4wjOkwuxqbVyHMyB8PSp98Wxh uvLIlqE5OvM4d/QHTHfaNKL0JWItl/FSqhEIqiGxHAB/TcqqE4xkDxeSJZSlCqt4Po7qeh 8iym0mhpDg9EwpuEEz7KcTYkxIe3Us0= Received: by mail-wm1-f44.google.com with SMTP id 5b1f17b1804b1-455b002833bso3420725e9.0 for ; Fri, 11 Jul 2025 09:24:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=neon.tech; s=google; t=1752251075; x=1752855875; darn=kvack.org; h=content-transfer-encoding:in-reply-to:content-language:references :cc:to:from:subject:user-agent:mime-version:date:message-id:from:to :cc:subject:date:message-id:reply-to; bh=VmqWzINlR7ueURt8ykpnYBNe7vdszeCG5zNyAg94yWQ=; b=SBXXXSBg5Ia4acQbJENNIyKjToyBRR2Fd6uJwU0qEBAIgFJ8eP9Mqfc+5bmihRN41O 1LlDB7dYgy9fBkZdTnuWHcVAlhWNGzSznN50QtGWkd75hcCCNtzXRrkfZLHFZfxNJdUV qpDYTleedlLGFqSxem0HAuTx8YMyGfExgTaBY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1752251075; x=1752855875; h=content-transfer-encoding:in-reply-to:content-language:references :cc:to:from:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=VmqWzINlR7ueURt8ykpnYBNe7vdszeCG5zNyAg94yWQ=; b=WaogJcIXAhq9pqNOIkdZ+uAAk/H4A4yA5WfBr3uUdpzKruJPBWxTS8SpdNH6yh3t/M iUx5MHoMHAqO4V9MVDPF1Aw8rogSQ4yRm+X25X3HZCogLoS/7N0naiIq+JaMXt4Ej3Nf CKPJnZuLsbcnuGqz9Bi2vVMN6CLAI7TZGV2AyyN1mEeVg3Y3X5rVLBRpepADydNpOvGH bbUxm2Oeuzq77flwKe0eG9HuHTCB+HaD6A+c/rnfkGr3JKq5fltSjqN3nu1CQDvxqNth kj3TcGMI4Jn6hOkySaI8C38H6AX1Dr2OANwMsdCrExaQE6qsV/L2yz1x7knV+/txploV OhLw== X-Forwarded-Encrypted: i=1; AJvYcCUe7GBjJYH70pKkqd00nsaqWBzO8Mo0nZZlMZI2eQqBDN99nYPZ55Al8VnWuLS5nb9lnlHl1NJvIw==@kvack.org X-Gm-Message-State: AOJu0YzWW3ipEEUNyV3Pcy4BJbadtGmCb5Fd7hFD9t79bHh6IpVi7GR0 785cIOAFOpJOi43ICBd+QjAL5KIdMotvisgGxbs7JQj+s/El1+cC4ox0OEFKYtjaODc= X-Gm-Gg: ASbGncs0y7QJXICiN8TRJXXNk5r4F+e2Ag/QJaNTDQco57fYfWuexOu9xl+gl3PQS2B EwexGG2MOdz9P9k/6Up6OVyaIBkqwbVUg6E43e7Lbahd0G4lrZtskR9aJxSK+UHvlxokMsJUIPi AJZ0qKtvZcPYBnZLtH4Zu0SyluwwWLMcRCDlO0vhJ82HAHFxM1+v6fbFpVaE03cC9DWUtO6vUpX 6B7UKYJSMCOfMyE56a4aG+4ienR+yW7OtIhCisGg3BCBiTL56M9lKhEAGtMhS+Akly8ChGOJXLf LpPaPsBEaVfefY68VBbCmH66NtKYqVS6IrtkCnMiL+7WZIX2A8KbAfEX0+R9PWbgl1ONzIWWIhk zCN/Uqj6xR1d9jbXBPzaHIzMbpU9JddjErOg= X-Google-Smtp-Source: AGHT+IHU9h0oitDhYyJ5Jd3VSokKs4uRGWzwtm/i43M875ekjrtkNtDt0ymoLk+jcdr65MBB1vNhbw== X-Received: by 2002:a05:600c:3b03:b0:454:b97b:cabb with SMTP id 5b1f17b1804b1-454ec283b2cmr34823395e9.28.1752251074495; Fri, 11 Jul 2025 09:24:34 -0700 (PDT) Received: from [192.168.86.142] ([90.253.47.31]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3b5e8dc3a54sm4964637f8f.39.2025.07.11.09.24.33 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 11 Jul 2025 09:24:34 -0700 (PDT) Message-ID: <05d016e1-2530-4cab-b5b2-41e8aa547a4b@neon.tech> Date: Fri, 11 Jul 2025 17:24:33 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: [PATCH v5 1/4] x86/mm: Update mapped addresses in phys_{pmd,pud}_init() From: Em Sharnoff To: linux-kernel@vger.kernel.org, x86@kernel.org, linux-mm@kvack.org Cc: Dave Hansen , Andy Lutomirski , Peter Zijlstra , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , "Edgecombe, Rick P" , Oleg Vasilev , Arthur Petukhovsky , Stefan Radig , Misha Sakhnov References: <4fe0984f-74dc-45fe-b2b6-bdd81ec15bac@neon.tech> Content-Language: en-US In-Reply-To: <4fe0984f-74dc-45fe-b2b6-bdd81ec15bac@neon.tech> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 0F290140011 X-Stat-Signature: 34s4u7r9xhdz3s4ttt51twhf6udme99t X-Rspam-User: X-HE-Tag: 1752251075-8182 X-HE-Meta: U2FsdGVkX1/HpFj62pgYaS34qf0zveIe+m4HjvD3pkemFCP4/YnjZT5BFnsXqzDaj+smMDs7okixodlpPyq0MYWgJ3ctFz/FqS0WNPfQ8qV9WoMQPKRbo9LR8vO1wWrgVK6CitKh8g4dw/SInGjCCsk1OwtoS78aB+P93b13kplSJSr/AuSvZLaDQ39GDpjW1beqm37kxmLwNltf53IYDGErTQa3TZNCIYBfEUN+25Uyrfq6/GEnswS8YS2ph86ul67tLvnD+0sk3PTiQUzfZ1XK5DmeDsb/K0SG5r/apMGq9pY3QN1dPyZ6Ss85C/pxxwf4hMJObl9THI70S824Tn9aABWF14JY/1mVxhtzdQMBC5kTmbefeqCQydEUNfGqVkgDOenJKuJC9AY8A8w8wS7yDcmnGbLDLu6cdzHngJbJgV4ND/ng2aJamQg9JsNxSkONw+SoQxhq9EvkY5of6bAX3JL1mqXmda+toV9nEF7kyMNufXsoMrwu2xMNOH6k289KOfutWRo63EMLDApq65taWDTtz1V8DrnfHbXeF1aJRgce6EcwlS/g2McftwGr29l76qLwuunz0jYN5PcgANJy+24rwyBG5yF+5g90I5MYy/Z4Zu0K6AGFDs77rk/Fui+6ogC9XyvQQrYEI/COyqhWbkDekrrx2Zigve09WtPq3XOmMM9dzMWN7J23dzkVZwKf7kvA0+bdAoIYDYYBUhoKIC3fcaYbK4dm6c00/p/UZvxpIvqhYG4tI8zkxZWwBmTo+D+xqOXMMQYR7WITJO8Bk2fvJFvNxIaUzNXt5aaTqEjwqN0b1VlktvKbV8n4kQJMteu5RCUr1gGHlz4CBhLmisvQ3VoyYLQ63hU44tJpM0IQBqkqcKZmfOh/92v/+Mp0c4EwIvDybrgKKmqetxu55I/ikgr8Snl7tDSHFRwA0IQEEKHPNPkwFQJVok4Q2ENg1gw+54CVKaikuag q+FPsk2w xGHD6GIAYMVLhUBMcoq7MNuU4uIojSGIVw1z4xeyAk4fbdq0QnRcJ8j+xL2m8nyDRVe2Q2FFqe8eEBFnoC7xsltACI+pe0jDd5eL6uE1xLipsmvWzgYvoGsPTvLcAvqeaJCrRYhK0MNVg7ztFCJN7yX1EMBuogeG+Ie+opAzcWOhYDFLJ1kn9NOQHo/vh4CUsKAiImhfTtPhqxdl+5fGRuaqJwC8Hnhb752kW9vjE2plU5ixjeuIX3rhpZg56bCUwwRuwek7VJgl+ZsSV8B9r0JYRK0rGcSputKodoSEdRCqsmIcWlvqznKtqC4MmGtp+bbdPXmu+Y32DSgjzBcgaPcFw1yXR0kQgVl3BpTcdXLJzhXpN8iCQwV8Sh7+xz17ZEt1A/kNhUCiIBOc0btMdoENYsYF/IfQxTMoYFCZ6q5cFd2pYTO3HM95zCklgwDfkkV5Z X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently kernel_physical_mapping_init() and its dependents return the last physical address mapped ('paddr_last'). This makes it harder to cleanly handle allocation errors in those functions. 'paddr_last' is used to update 'pfn_mapped'/'max_pfn_mapped', so: 1. Introduce add_paddr_range_mapped() to do the update, translating from physical addresses to pfns 2. Call add_paddr_range_mapped() in phys_pud_init() where 'paddr_last' would otherwise be updated due to 1Gi pages. - Note: this includes places where we set 'paddr_last = paddr_next', as was added in 20167d3421a0 ("x86-64: Fix accounting in kernel_physical_mapping_init()") add_paddr_range_mapped() is probably too expensive to be called every time a page is updated, so instead, phys_pte_init() continues to return 'paddr_last', and phys_pmd_init() calls add_paddr_range_mapped() only at the end of the loop (should mean it's called every 1Gi). Signed-off-by: Em Sharnoff --- Changelog: - v4: Add this patch --- arch/x86/include/asm/pgtable.h | 3 +- arch/x86/mm/init.c | 23 +++++---- arch/x86/mm/init_32.c | 6 ++- arch/x86/mm/init_64.c | 88 +++++++++++++++++----------------- arch/x86/mm/mm_internal.h | 13 +++-- 5 files changed, 69 insertions(+), 64 deletions(-) diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h index 97954c936c54..5d71cb192c57 100644 --- a/arch/x86/include/asm/pgtable.h +++ b/arch/x86/include/asm/pgtable.h @@ -1224,8 +1224,7 @@ extern int direct_gbpages; void init_mem_mapping(void); void early_alloc_pgt_buf(void); void __init poking_init(void); -unsigned long init_memory_mapping(unsigned long start, - unsigned long end, pgprot_t prot); +void init_memory_mapping(unsigned long start, unsigned long end, pgprot_t prot); #ifdef CONFIG_X86_64 extern pgd_t trampoline_pgd_entry; diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c index 7456df985d96..e87466489c66 100644 --- a/arch/x86/mm/init.c +++ b/arch/x86/mm/init.c @@ -526,16 +526,24 @@ bool pfn_range_is_mapped(unsigned long start_pfn, unsigned long end_pfn) return false; } +/* + * Update max_pfn_mapped and range_pfn_mapped with the range of physical + * addresses mapped. The range may overlap with previous calls to this function. + */ +void add_paddr_range_mapped(unsigned long start_paddr, unsigned long end_paddr) +{ + add_pfn_range_mapped(start_paddr >> PAGE_SHIFT, end_paddr >> PAGE_SHIFT); +} + /* * Setup the direct mapping of the physical memory at PAGE_OFFSET. * This runs before bootmem is initialized and gets pages directly from * the physical memory. To access them they are temporarily mapped. */ -unsigned long __ref init_memory_mapping(unsigned long start, - unsigned long end, pgprot_t prot) +void __ref init_memory_mapping(unsigned long start, + unsigned long end, pgprot_t prot) { struct map_range mr[NR_RANGE_MR]; - unsigned long ret = 0; int nr_range, i; pr_debug("init_memory_mapping: [mem %#010lx-%#010lx]\n", @@ -545,13 +553,10 @@ unsigned long __ref init_memory_mapping(unsigned long start, nr_range = split_mem_range(mr, 0, start, end); for (i = 0; i < nr_range; i++) - ret = kernel_physical_mapping_init(mr[i].start, mr[i].end, - mr[i].page_size_mask, - prot); + kernel_physical_mapping_init(mr[i].start, mr[i].end, + mr[i].page_size_mask, prot); - add_pfn_range_mapped(start >> PAGE_SHIFT, ret >> PAGE_SHIFT); - - return ret >> PAGE_SHIFT; + return; } /* diff --git a/arch/x86/mm/init_32.c b/arch/x86/mm/init_32.c index 607d6a2e66e2..a9a16d3d0eb2 100644 --- a/arch/x86/mm/init_32.c +++ b/arch/x86/mm/init_32.c @@ -246,7 +246,7 @@ static inline int is_x86_32_kernel_text(unsigned long addr) * of max_low_pfn pages, by creating page tables starting from address * PAGE_OFFSET: */ -unsigned long __init +void __init kernel_physical_mapping_init(unsigned long start, unsigned long end, unsigned long page_size_mask, @@ -383,7 +383,9 @@ kernel_physical_mapping_init(unsigned long start, mapping_iter = 2; goto repeat; } - return last_map_addr; + + add_paddr_range_mapped(start, last_map_addr); + return; } #ifdef CONFIG_HIGHMEM diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index ee66fae9ebcc..f0dc4a0e8cde 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -503,13 +503,13 @@ phys_pte_init(pte_t *pte_page, unsigned long paddr, unsigned long paddr_end, /* * Create PMD level page table mapping for physical addresses. The virtual * and physical address have to be aligned at this level. - * It returns the last physical address mapped. */ -static unsigned long __meminit +static void __meminit phys_pmd_init(pmd_t *pmd_page, unsigned long paddr, unsigned long paddr_end, unsigned long page_size_mask, pgprot_t prot, bool init) { unsigned long pages = 0, paddr_next; + unsigned long paddr_first = paddr; unsigned long paddr_last = paddr_end; int i = pmd_index(paddr); @@ -580,21 +580,25 @@ phys_pmd_init(pmd_t *pmd_page, unsigned long paddr, unsigned long paddr_end, spin_unlock(&init_mm.page_table_lock); } update_page_count(PG_LEVEL_2M, pages); - return paddr_last; + /* + * In case of recovery from previous state, add_paddr_range_mapped() may + * be called with an overlapping range from previous operations. + * It is idempotent, so this is ok. + */ + add_paddr_range_mapped(paddr_first, paddr_last); + return; } /* * Create PUD level page table mapping for physical addresses. The virtual * and physical address do not have to be aligned at this level. KASLR can * randomize virtual addresses up to this level. - * It returns the last physical address mapped. */ -static unsigned long __meminit +static void __meminit phys_pud_init(pud_t *pud_page, unsigned long paddr, unsigned long paddr_end, unsigned long page_size_mask, pgprot_t _prot, bool init) { unsigned long pages = 0, paddr_next; - unsigned long paddr_last = paddr_end; unsigned long vaddr = (unsigned long)__va(paddr); int i = pud_index(vaddr); @@ -620,10 +624,8 @@ phys_pud_init(pud_t *pud_page, unsigned long paddr, unsigned long paddr_end, if (!pud_none(*pud)) { if (!pud_leaf(*pud)) { pmd = pmd_offset(pud, 0); - paddr_last = phys_pmd_init(pmd, paddr, - paddr_end, - page_size_mask, - prot, init); + phys_pmd_init(pmd, paddr, paddr_end, + page_size_mask, prot, init); continue; } /* @@ -641,7 +643,7 @@ phys_pud_init(pud_t *pud_page, unsigned long paddr, unsigned long paddr_end, if (page_size_mask & (1 << PG_LEVEL_1G)) { if (!after_bootmem) pages++; - paddr_last = paddr_next; + add_paddr_range_mapped(paddr, paddr_next); continue; } prot = pte_pgprot(pte_clrhuge(*(pte_t *)pud)); @@ -654,13 +656,13 @@ phys_pud_init(pud_t *pud_page, unsigned long paddr, unsigned long paddr_end, pfn_pud(paddr >> PAGE_SHIFT, prot_sethuge(prot)), init); spin_unlock(&init_mm.page_table_lock); - paddr_last = paddr_next; + add_paddr_range_mapped(paddr, paddr_next); continue; } pmd = alloc_low_page(); - paddr_last = phys_pmd_init(pmd, paddr, paddr_end, - page_size_mask, prot, init); + phys_pmd_init(pmd, paddr, paddr_end, + page_size_mask, prot, init); spin_lock(&init_mm.page_table_lock); pud_populate_init(&init_mm, pud, pmd, init); @@ -669,22 +671,23 @@ phys_pud_init(pud_t *pud_page, unsigned long paddr, unsigned long paddr_end, update_page_count(PG_LEVEL_1G, pages); - return paddr_last; + return; } -static unsigned long __meminit +static void __meminit phys_p4d_init(p4d_t *p4d_page, unsigned long paddr, unsigned long paddr_end, unsigned long page_size_mask, pgprot_t prot, bool init) { - unsigned long vaddr, vaddr_end, vaddr_next, paddr_next, paddr_last; + unsigned long vaddr, vaddr_end, vaddr_next, paddr_next; - paddr_last = paddr_end; vaddr = (unsigned long)__va(paddr); vaddr_end = (unsigned long)__va(paddr_end); - if (!pgtable_l5_enabled()) - return phys_pud_init((pud_t *) p4d_page, paddr, paddr_end, - page_size_mask, prot, init); + if (!pgtable_l5_enabled()) { + phys_pud_init((pud_t *) p4d_page, paddr, paddr_end, + page_size_mask, prot, init); + return; + } for (; vaddr < vaddr_end; vaddr = vaddr_next) { p4d_t *p4d = p4d_page + p4d_index(vaddr); @@ -706,33 +709,32 @@ phys_p4d_init(p4d_t *p4d_page, unsigned long paddr, unsigned long paddr_end, if (!p4d_none(*p4d)) { pud = pud_offset(p4d, 0); - paddr_last = phys_pud_init(pud, paddr, __pa(vaddr_end), - page_size_mask, prot, init); + phys_pud_init(pud, paddr, __pa(vaddr_end), + page_size_mask, prot, init); continue; } pud = alloc_low_page(); - paddr_last = phys_pud_init(pud, paddr, __pa(vaddr_end), - page_size_mask, prot, init); + phys_pud_init(pud, paddr, __pa(vaddr_end), + page_size_mask, prot, init); spin_lock(&init_mm.page_table_lock); p4d_populate_init(&init_mm, p4d, pud, init); spin_unlock(&init_mm.page_table_lock); } - return paddr_last; + return; } -static unsigned long __meminit +static void __meminit __kernel_physical_mapping_init(unsigned long paddr_start, unsigned long paddr_end, unsigned long page_size_mask, pgprot_t prot, bool init) { bool pgd_changed = false; - unsigned long vaddr, vaddr_start, vaddr_end, vaddr_next, paddr_last; + unsigned long vaddr, vaddr_start, vaddr_end, vaddr_next; - paddr_last = paddr_end; vaddr = (unsigned long)__va(paddr_start); vaddr_end = (unsigned long)__va(paddr_end); vaddr_start = vaddr; @@ -745,16 +747,14 @@ __kernel_physical_mapping_init(unsigned long paddr_start, if (pgd_val(*pgd)) { p4d = (p4d_t *)pgd_page_vaddr(*pgd); - paddr_last = phys_p4d_init(p4d, __pa(vaddr), - __pa(vaddr_end), - page_size_mask, - prot, init); + phys_p4d_init(p4d, __pa(vaddr), __pa(vaddr_end), + page_size_mask, prot, init); continue; } p4d = alloc_low_page(); - paddr_last = phys_p4d_init(p4d, __pa(vaddr), __pa(vaddr_end), - page_size_mask, prot, init); + phys_p4d_init(p4d, __pa(vaddr), __pa(vaddr_end), + page_size_mask, prot, init); spin_lock(&init_mm.page_table_lock); if (pgtable_l5_enabled()) @@ -770,7 +770,7 @@ __kernel_physical_mapping_init(unsigned long paddr_start, if (pgd_changed) sync_global_pgds(vaddr_start, vaddr_end - 1); - return paddr_last; + return; } @@ -778,15 +778,15 @@ __kernel_physical_mapping_init(unsigned long paddr_start, * Create page table mapping for the physical memory for specific physical * addresses. Note that it can only be used to populate non-present entries. * The virtual and physical addresses have to be aligned on PMD level - * down. It returns the last physical address mapped. + * down. */ -unsigned long __meminit +void __meminit kernel_physical_mapping_init(unsigned long paddr_start, unsigned long paddr_end, unsigned long page_size_mask, pgprot_t prot) { - return __kernel_physical_mapping_init(paddr_start, paddr_end, - page_size_mask, prot, true); + __kernel_physical_mapping_init(paddr_start, paddr_end, + page_size_mask, prot, true); } /* @@ -795,14 +795,14 @@ kernel_physical_mapping_init(unsigned long paddr_start, * when updating the mapping. The caller is responsible to flush the TLBs after * the function returns. */ -unsigned long __meminit +void __meminit kernel_physical_mapping_change(unsigned long paddr_start, unsigned long paddr_end, unsigned long page_size_mask) { - return __kernel_physical_mapping_init(paddr_start, paddr_end, - page_size_mask, PAGE_KERNEL, - false); + __kernel_physical_mapping_init(paddr_start, paddr_end, + page_size_mask, PAGE_KERNEL, + false); } #ifndef CONFIG_NUMA diff --git a/arch/x86/mm/mm_internal.h b/arch/x86/mm/mm_internal.h index 097aadc250f7..5b873191c3c9 100644 --- a/arch/x86/mm/mm_internal.h +++ b/arch/x86/mm/mm_internal.h @@ -10,13 +10,12 @@ static inline void *alloc_low_page(void) void early_ioremap_page_table_range_init(void); -unsigned long kernel_physical_mapping_init(unsigned long start, - unsigned long end, - unsigned long page_size_mask, - pgprot_t prot); -unsigned long kernel_physical_mapping_change(unsigned long start, - unsigned long end, - unsigned long page_size_mask); +void add_paddr_range_mapped(unsigned long start_paddr, unsigned long end_paddr); + +void kernel_physical_mapping_init(unsigned long start, unsigned long end, + unsigned long page_size_mask, pgprot_t prot); +void kernel_physical_mapping_change(unsigned long start, unsigned long end, + unsigned long page_size_mask); void zone_sizes_init(void); extern int after_bootmem; -- 2.39.5