From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1CD0BC021B8 for ; Tue, 25 Feb 2025 07:46:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 89F9C6B007B; Tue, 25 Feb 2025 02:46:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8508E6B0082; Tue, 25 Feb 2025 02:46:52 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6F03A6B0085; Tue, 25 Feb 2025 02:46:52 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 508A16B007B for ; Tue, 25 Feb 2025 02:46:52 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id E7E9B1C7272 for ; Tue, 25 Feb 2025 07:46:51 +0000 (UTC) X-FDA: 83157685422.15.CD62BD1 Received: from tor.source.kernel.org (tor.source.kernel.org [172.105.4.254]) by imf16.hostedemail.com (Postfix) with ESMTP id 58CF518000A for ; Tue, 25 Feb 2025 07:46:50 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=mNF5vIXE; spf=pass (imf16.hostedemail.com: domain of rppt@kernel.org designates 172.105.4.254 as permitted sender) smtp.mailfrom=rppt@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740469610; a=rsa-sha256; cv=none; b=QCroyNV4xyWt0jtpVY+A3ZUesVqh+IXeW+HSzUpBsqn5HmsElQS+m1z7e0xEv5AD2fbAkK XaHieRWFUOduGOWPlWJItWSLCapdLimsKUWOfqZY9ngfQEi/V9/Ohk5b7ZeskVt7ZZADd2 KrQ4PHjQoTvF5gPvdZ2oMZfaj+ZqNbY= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=mNF5vIXE; spf=pass (imf16.hostedemail.com: domain of rppt@kernel.org designates 172.105.4.254 as permitted sender) smtp.mailfrom=rppt@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740469610; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Uwv0a4dzm+5HpJk54FntQugqsgzOGHWa5a/atNDvfEI=; b=o/ABdHj+t9YxBng5XHftG7EXA5l6Ldl69u9E29hY16E94TTgTXoVx6WjkOQJOELHb2zbtZ IWEznjM0thWUVJOenvEYwWPWmFX3JCAVCfoeJEFC73/7rnIbRWzfJnf4Ws0dbm/QWqRh8Z e40kVhss6Pj+oLxs0ll4YBbWzTs1gx8= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by tor.source.kernel.org (Postfix) with ESMTP id ECB006123A; Tue, 25 Feb 2025 07:46:42 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 94BF0C4CEDD; Tue, 25 Feb 2025 07:46:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1740469609; bh=ufWHWQ1xjvwW0hl8NZb6j0p51Ljh1vhupp5zrhn05g0=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=mNF5vIXE8AI1M72/t0vxMc2XxKToFenm/PAwm6NZlyzcXaF3GjKp3p8wjKOQxZsS7 U5SdmrZ6BYaIYN/FwLyRn6F6fE1le5OiAG1N1ERsiHU12nBZ0Mpu+uoTJxsNcrzgL5 FecXJjFgXjXiQvXjfk0dv540gXmPw1NfbWIvf34KHRA1eYhDFFirjnTSuwgDS6kEOd 1qDcx0OgryYeV466p9C+RQAkPDKOvmVtK2XQyZFdYwXGOngf8RClrW5TVTs70NL83+ yvakFILGXUbl3fYUBTPeGqmiA4fnXW4OaTfniPVNR1nBcO4RYgvzzBPR4DQ+sz2u+j fzz+pKVE/XpLg== Date: Tue, 25 Feb 2025 09:46:28 +0200 From: Mike Rapoport To: Wei Yang Cc: linux-kernel@vger.kernel.org, Alexander Graf , Andrew Morton , Andy Lutomirski , Anthony Yznaga , Arnd Bergmann , Ashish Kalra , Benjamin Herrenschmidt , Borislav Petkov , Catalin Marinas , Dave Hansen , David Woodhouse , Eric Biederman , Ingo Molnar , James Gowans , Jonathan Corbet , Krzysztof Kozlowski , Mark Rutland , Paolo Bonzini , Pasha Tatashin , "H. Peter Anvin" , Peter Zijlstra , Pratyush Yadav , Rob Herring , Rob Herring , Saravana Kannan , Stanislav Kinsburskii , Steven Rostedt , Thomas Gleixner , Tom Lendacky , Usama Arif , Will Deacon , devicetree@vger.kernel.org, kexec@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org Subject: Re: [PATCH v4 02/14] memblock: add MEMBLOCK_RSRV_KERN flag Message-ID: References: <20250206132754.2596694-1-rppt@kernel.org> <20250206132754.2596694-3-rppt@kernel.org> <20250218155004.n53fcuj2lrl5rxll@master> <20250224013131.fzz552bn7fs64umq@master> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20250224013131.fzz552bn7fs64umq@master> X-Rspamd-Queue-Id: 58CF518000A X-Stat-Signature: f7okc9o6gcy7emaom5riwk3jbxdg4z49 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1740469610-430988 X-HE-Meta: U2FsdGVkX18hKR9Edjxcz+Qz6oUvHOxu4oG5eDVcj222QcMeaq+kQeuWd0fUblF0DmVWYRenUx1gorA1o04Rk9y7fVY6DmH4rTaffn9El3m1E47EttWd1SZtRBv0xqFYfM5wPCqfNtH5Eom5x5orwFPySU6Rd/owm3z+NGqzgDVPXvDQOKRe75p10iwmxEmZcczTq3SCQMAdAkyvotzZLT2mhqG9ZnVheRXixpceqGqJ8nmpz2BQdUeQMR8ttrk3JIB/kxRJWepLlxAS2p7hCHgP1haIwJkGyGPSXXoug1Yb9c72QkoGTFblurUWqCvIfyNC5rKiAU/TI4WtDFYVX4PJdIugIVXHLTf5yHkaDlbJIpaYjrdkNKIMgIDoKor7YosmcXdsbkH3CnKa1q56CTaCGDhP0dxhYakg8ZTQbZPVu1jC/orbd2OxsJrVT3/Gm2nHdXVrD+sRSpQ6DfygZ8LBtf2YS1ACI5JUnTNbdDV7IAf+ExU51JE76JN/ND5ers4AA9nsLKhxx5T0LRxKZI8QZUHncfnwPxXskcqzisgD9YxEhLkdsLiH5G/H/KIxh7BRHj0Erf79kn14CLjsjDC5PUj3VZBAfjUqoQCaJlZfHOTm/IyyNmJuV09STgpBmb2fKi82NnWldMGbqIpM5HSTIEEa3eacPBOvqgWYDj3gZO67bzJagfSnb9lTSPNFHJD5kv7BMVJ8bIBpNKnvqHM8+gKE7fDFdDg7DzQwwJFIlgp+6BHdXypgznRB74pO7YEx8/A7s+aTE7mVVI3GZfqdip0a/Sw66/PGDai9yxHE65VwKq7OSIRIlcN39YFW+v3BOjCc+swq8U/uSVHqGvi2ivb5U+kcqlHok+bx431KTTY+1RkHPIVqPArOaJmSeg9UW25ZPea4bOnU7pbFt52Ob3tc8o5c3Omp8yMgaeXO5pAhJhaOM4DHBaTwBkN8SXfaqCKecB9tWHw/A8c Buq6OF/q VLvXwsNEI6gRlprr2rSdnsX7cpZysrpV2QLe0coV/MTcUVr9eHVF7vdtixhed4+Mb2uSl+HZbjXrKU4VXu6M5L0laprtitiDjnBQyARviK6gQ5mAwz2L7+dPbfP2JHZ3cMMO3cFosce7a/IeKKJjzSTog9tEjRAV7HMO6/C5Nqyoa093vO4yrq/E21gCQhEdkYUVsRm2fK6bE+cTAAsyzFKiKxHojeh2agkvtavc9LbrCap/3XGsKTFjxoE54AoKzr6lEyh2CnbMaanGSZ1AfOjm8rhoccpHXBI8WZny879qatMqpCRDMfT95zsVfCZwWh56KkNDDxxtG8sxEyGykXe/VGrpv3ac2861CK6Tgvn8rUbs2KawR6xmc3VQhOzrLqbCt2DfQgAm/UMaATfjSu/j/PQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Feb 24, 2025 at 01:31:31AM +0000, Wei Yang wrote: > On Wed, Feb 19, 2025 at 09:24:31AM +0200, Mike Rapoport wrote: > >Hi, > > > >On Tue, Feb 18, 2025 at 03:50:04PM +0000, Wei Yang wrote: > >> On Thu, Feb 06, 2025 at 03:27:42PM +0200, Mike Rapoport wrote: > >> >From: "Mike Rapoport (Microsoft)" > >> > > >> >to denote areas that were reserved for kernel use either directly with > >> >memblock_reserve_kern() or via memblock allocations. > >> > > >> >Signed-off-by: Mike Rapoport (Microsoft) > >> >--- > >> > include/linux/memblock.h | 16 +++++++++++++++- > >> > mm/memblock.c | 32 ++++++++++++++++++++++++-------- > >> > 2 files changed, 39 insertions(+), 9 deletions(-) > >> > > >> >diff --git a/include/linux/memblock.h b/include/linux/memblock.h > >> >index e79eb6ac516f..65e274550f5d 100644 > >> >--- a/include/linux/memblock.h > >> >+++ b/include/linux/memblock.h > >> >@@ -50,6 +50,7 @@ enum memblock_flags { > >> > MEMBLOCK_NOMAP = 0x4, /* don't add to kernel direct mapping */ > >> > MEMBLOCK_DRIVER_MANAGED = 0x8, /* always detected via a driver */ > >> > MEMBLOCK_RSRV_NOINIT = 0x10, /* don't initialize struct pages */ > >> >+ MEMBLOCK_RSRV_KERN = 0x20, /* memory reserved for kernel use */ > >> > >> Above memblock_flags, there are comments on explaining those flags. > >> > >> Seems we miss it for MEMBLOCK_RSRV_KERN. > > > >Right, thanks! > > > >> > > >> > #ifdef CONFIG_HAVE_MEMBLOCK_PHYS_MAP > >> >@@ -1459,14 +1460,14 @@ phys_addr_t __init memblock_alloc_range_nid(phys_addr_t size, > >> > again: > >> > found = memblock_find_in_range_node(size, align, start, end, nid, > >> > flags); > >> >- if (found && !memblock_reserve(found, size)) > >> >+ if (found && !__memblock_reserve(found, size, nid, MEMBLOCK_RSRV_KERN)) > >> > >> Maybe we could use memblock_reserve_kern() directly. If my understanding is > >> correct, the reserved region's nid is not used. > > > >We use nid of reserved regions in reserve_bootmem_region() (commit > >61167ad5fecd ("mm: pass nid to reserve_bootmem_region()")) but KHO needs to > >know the distribution of reserved memory among the nodes before > >memmap_init_reserved_pages(). > > > >> BTW, one question here. How we handle concurrent memblock allocation? If two > >> threads find the same available range and do the reservation, it seems to be a > >> problem to me. Or I missed something? > > > >memblock allocations end before smp_init(), there is no possible concurrency. > > > > Thanks, I still have one question here. > > Below is a simplified call flow. > > mm_core_init() > mem_init() > memblock_free_all() > free_low_memory_core_early() > memmap_init_reserved_pages() > memblock_set_node(..., memblock.reserved, ) --- (1) > __free_memory_core() > kmem_cache_init() > slab_state = UP; --- (2) > > And memblock_allloc_range_nid() is not supposed to be called after > slab_is_available(). Even someone do dose it, it will get memory from slab > instead of reserve region in memblock. > > From the above call flow and background, there are three cases when > memblock_alloc_range_nid() would be called: > > * If it is called before (1), memblock.reserved's nid would be adjusted correctly. > * If it is called after (2), we don't touch memblock.reserved. > * If it happens between (1) and (2), it looks would break the consistency of > nid information in memblock.reserved. Because when we use > memblock_reserve_kern(), NUMA_NO_NODE would be stored in region. > > So my question is if the third case happens, would it introduce a bug? If it > won't happen, seems we don't need to specify the nid here? We don't really care about proper assignment of nodes between (1) and (2) from one side and the third case does not happen on the other side. Nothing should call membloc_alloc() after memblock_free_all(). But it's easy to make the window between (1) and (2) disappear by replacing checks for slab_is_available() in memblock with a variable local to memblock. > -- > Wei Yang > Help you, Help me -- Sincerely yours, Mike.