From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 959FBC369CB for ; Wed, 23 Apr 2025 13:38:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A4CA36B00A5; Wed, 23 Apr 2025 09:38:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9F60A6B00AA; Wed, 23 Apr 2025 09:38:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7FAAF6B00AB; Wed, 23 Apr 2025 09:38:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 5354C6B00A5 for ; Wed, 23 Apr 2025 09:38:33 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id D94C75AE25 for ; Wed, 23 Apr 2025 13:38:34 +0000 (UTC) X-FDA: 83365413348.04.FA5A772 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf18.hostedemail.com (Postfix) with ESMTP id 104431C0002 for ; Wed, 23 Apr 2025 13:38:32 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=VDj7ttIR; dmarc=none; spf=none (imf18.hostedemail.com: domain of BATV+cfb108382ec1185fc08d+7913+infradead.org+dwmw2@casper.srs.infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=BATV+cfb108382ec1185fc08d+7913+infradead.org+dwmw2@casper.srs.infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1745415513; a=rsa-sha256; cv=none; b=vdR3ECub0XUIGCaP8cHgmkm/MHiJy2/XxVbsb8kC4vcEuT620ypdQ9M6wgZ9rUiOCGeLNb 2FTjaapeR8JLGbGsfvJTC69o4Syt6nmHrs3YmHZBT7P3SHvjuGATTZGnjSkCc0fIgul6a9 CkizG11W465Mhp4vtUUVOKtRXg0OLdc= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=VDj7ttIR; dmarc=none; spf=none (imf18.hostedemail.com: domain of BATV+cfb108382ec1185fc08d+7913+infradead.org+dwmw2@casper.srs.infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=BATV+cfb108382ec1185fc08d+7913+infradead.org+dwmw2@casper.srs.infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1745415513; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NJbaNH4W3yPbLyC+ZTcgmLXA2i0jKLuqPtxQoIRJ/KA=; b=laWVSRCsAnQET0VvKJq3NwaWRfDwsI33i7S/BvV3pQPTXj14D7Td6L5oMe0HFbxrvDbkFh uSVSk/3CFMDKayI7lEHgwkdAqXObHZpakmgstS1WV97eRuo/MOqVpJK6kMlLQYzN6bsnc5 IbzjJOTeksVozfD1lf3S+MDOn2wpJwY= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc: To:From:Reply-To:Content-ID:Content-Description; bh=NJbaNH4W3yPbLyC+ZTcgmLXA2i0jKLuqPtxQoIRJ/KA=; b=VDj7ttIRbdw7PsgFRyTX1xiiWi w0xC3leSIn0vHLjJKQLj4OyZlSOdSaIUWdQMZEOmLAU780t0pu/KRmbSqXaxp5Z54S/1SCzcFyrbM j+INOOcLjBqxPKW4ePYcz97fcAx+2swpdj2EGLIsaPLLZgDKvaTtw+BbgSlT6fBCNHFNB3mjZI9Re +C6GoJcXXJeoLlv5DPnEHomPjaeY8vziT3hPOEwY64gOW6gjiaZJlTdBA5wdjRtwoQ4vJatrlVCOV COKwsJUoVmurtunT8I1K/VFK+Q0txm3EKttwIRPjb7U00J0nZRaMEfPMBQzydkuSc4f40+r9uiziQ b9l9Itkw==; Received: from [2001:8b0:10b:1::ebe] (helo=i7.infradead.org) by casper.infradead.org with esmtpsa (Exim 4.98.2 #2 (Red Hat Linux)) id 1u7aIx-000000097Jd-1LOr; Wed, 23 Apr 2025 13:38:23 +0000 Received: from dwoodhou by i7.infradead.org with local (Exim 4.98.1 #2 (Red Hat Linux)) id 1u7aIx-00000003JOc-0I7B; Wed, 23 Apr 2025 14:38:23 +0100 From: David Woodhouse To: Mike Rapoport Cc: Andrew Morton , "Sauerwein, David" , Anshuman Khandual , Ard Biesheuvel , Catalin Marinas , David Hildenbrand , Marc Zyngier , Mark Rutland , Mike Rapoport , Will Deacon , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Ruihan Li Subject: [PATCH v4 1/7] mm: Introduce for_each_valid_pfn() and use it from reserve_bootmem_region() Date: Wed, 23 Apr 2025 14:33:37 +0100 Message-ID: <20250423133821.789413-2-dwmw2@infradead.org> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250423133821.789413-1-dwmw2@infradead.org> References: <20250423133821.789413-1-dwmw2@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-SRS-Rewrite: SMTP reverse-path rewritten from by casper.infradead.org. See http://www.infradead.org/rpr.html X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 104431C0002 X-Stat-Signature: 48ozd7f9ihffgdsamdzamq1hqensn5q8 X-Rspam-User: X-HE-Tag: 1745415512-619334 X-HE-Meta: U2FsdGVkX19/Ff3eIJDtM3hVpTSBj+De3xk8CJTorgJX11N4wH267CLm4R5rfZRGGHbDfzq6Bx/pcZJ7P24v6gZiT4he/oJt1d5EIwTaW3GYCcAdI8peg3AC+L/5qvCoAodQiZcPX9/YVrJCD7rUbTVsXMVvrVOoyCzp2bwDpKaS0Tk77Lt4Eu/XkE++Qy+d1jGjAqyuybhxMGbTWdgMNOyBTIaqjmyJMA3ttvjKxu8EHotyGBsewe3XnbEYXmLWqGS1/cBq69+VnBm2KAcmhkfQ/aBigrZH1YaG22DItZzEM0YuZX54Lo28MK6cMSodXszYwugwxBCpu10OvDYNqPZEgyf87gW34bYXHxq1Npquxs2egu+q6634TiQxYeNyoZ+pHgaad2OTA9fr46xNuZ11a/63KdeEQxQRWCdCOYk3MkumDr/8i4O8USdrCot5z1Wc4B916PLch+z8IgjRJXTT+mWh6fqqOdyQIo3PPENCTMnA3PW1os6/cYSa/eEl9oAUSw6mAI9xdhJ5UaaxOEhwwr/JK8gGFqbfa7cS8fEcd8D27Ey2vhT4m8r8XJV7Lez0L1rZtjprRgPQiIOu5DsJnTULkW5mGnAIebap/kH4e6glI2F203/o0mQ+WMW1QyNceqjctj+zJ7F+qBJBhjtMLEh0IAW4yXXEkckwvJXhgYHQUM5KIFWKNL2EhCOgMpwo9scyaF5IGQssjRDnF0hro8LrvX5ZaJ5g5tb7z3oBq6QfQIM8jjYN300M46vUp8Xl2KQsyS7o1utPmgrHmswCmoN5tanUFh95RuqR5C4guJO2EJqFqkhzn/Gv29kfi2/v7y/1W3Q1j5ebnQo7ZSB+7lxZQTycA9Jb849B4Xg3kNIDK7dmXhHVPjWv6DOS2QVI3cyphgBW30NXFFXadJbm0l2ZpVFD8n/4AJ98iQbLzYq1ktRwiMm4gZJk5erpWtQa5qvTsaaACHyldNJ jLAmGb4p 4AZhSdI9JtQSJG+e31q6BCrOkZJi2WlQkJqI5kzpld/iwseTtIbVekoaOUbXbtL5+azCoCXJIGJybButGFuqcSw2koaxa7Eo+KjQZq51RxlvAD/+uCb99VVvGokeBIfGAvAOAWC0Us40cgiCgqa8epV08qgXjF7FghRmdwD/2VRZbI9UFYvprAR/dsbQfwpxGv518EypOiUsB0BxTuRY9eLxhNsDigEHlXJOstYiOzEG0+Uo/9DoxwiWXyYPzWLZZjc4UkFiuk0sjogu7ANdVMRztMAqdCZFzQWsVTpKc2GQ/AZ4VB7DqTn/KsK9Xvkr+as3M4cgQI8Z1aaKnNFJJ0T53EBdAeE4LxGPSgM4pD0vmKyElObet4pkCtTcrdZSR86yYeqC3vkHeWXSbaV2Zz8j5/BBzWlaPaM7CZ08BzKuwEpKYBPMPUlKCGVCDqo7QJCmrEPrEirkjWYoP0/J5FXTew+R/ZotXxqV/qUql/AaI3fylVRfm39auseBzBtrs2SCM1lATNHXpsTkFb8mCZS7jv5G8C9tMGkXoAUlOTzlTecYBSb+j0E5W2VXcb3D1ZkC9urVelY8fwCMYIImBdRq36Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: David Woodhouse Especially since commit 9092d4f7a1f8 ("memblock: update initialization of reserved pages"), the reserve_bootmem_region() function can spend a significant amount of time iterating over every 4KiB PFN in a range, calling pfn_valid() on each one, and ultimately doing absolutely nothing. On a platform used for virtualization, with large NOMAP regions that eventually get used for guest RAM, this leads to a significant increase in steal time experienced during kexec for a live update. Introduce for_each_valid_pfn() and use it from reserve_bootmem_region(). This implementation is precisely the same naïve loop that the function used to have, but subsequent commits will provide optimised versions for FLATMEM and SPARSEMEM, and this version will remain for those architectures which provide their own pfn_valid() implementation, until/unless they also provide a matching for_each_valid_pfn(). Signed-off-by: David Woodhouse Reviewed-by: Mike Rapoport (Microsoft) --- include/linux/mmzone.h | 10 ++++++++++ mm/mm_init.c | 23 ++++++++++------------- 2 files changed, 20 insertions(+), 13 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 6ccec1bf2896..230a29c2ed1a 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -2177,6 +2177,16 @@ void sparse_init(void); #define subsection_map_init(_pfn, _nr_pages) do {} while (0) #endif /* CONFIG_SPARSEMEM */ +/* + * Fallback case for when the architecture provides its own pfn_valid() but + * not a corresponding for_each_valid_pfn(). + */ +#ifndef for_each_valid_pfn +#define for_each_valid_pfn(_pfn, _start_pfn, _end_pfn) \ + for ((_pfn) = (_start_pfn); (_pfn) < (_end_pfn); (_pfn)++) \ + if (pfn_valid(_pfn)) +#endif + #endif /* !__GENERATING_BOUNDS.H */ #endif /* !__ASSEMBLY__ */ #endif /* _LINUX_MMZONE_H */ diff --git a/mm/mm_init.c b/mm/mm_init.c index 9659689b8ace..41884f2155c4 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -777,22 +777,19 @@ static inline void init_deferred_page(unsigned long pfn, int nid) void __meminit reserve_bootmem_region(phys_addr_t start, phys_addr_t end, int nid) { - unsigned long start_pfn = PFN_DOWN(start); - unsigned long end_pfn = PFN_UP(end); + unsigned long pfn; - for (; start_pfn < end_pfn; start_pfn++) { - if (pfn_valid(start_pfn)) { - struct page *page = pfn_to_page(start_pfn); + for_each_valid_pfn (pfn, PFN_DOWN(start), PFN_UP(end)) { + struct page *page = pfn_to_page(pfn); - init_deferred_page(start_pfn, nid); + init_deferred_page(pfn, nid); - /* - * no need for atomic set_bit because the struct - * page is not visible yet so nobody should - * access it yet. - */ - __SetPageReserved(page); - } + /* + * no need for atomic set_bit because the struct + * page is not visible yet so nobody should + * access it yet. + */ + __SetPageReserved(page); } } -- 2.49.0