From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF0ADC47077 for ; Wed, 17 Jan 2024 02:20:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F3C936B0080; Tue, 16 Jan 2024 21:20:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EECD76B00B2; Tue, 16 Jan 2024 21:20:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DB4BF6B00B6; Tue, 16 Jan 2024 21:20:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id C80226B0080 for ; Tue, 16 Jan 2024 21:20:18 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 99D67120987 for ; Wed, 17 Jan 2024 02:20:18 +0000 (UTC) X-FDA: 81687198516.07.DC25C8C Received: from out-173.mta0.migadu.com (out-173.mta0.migadu.com [91.218.175.173]) by imf09.hostedemail.com (Postfix) with ESMTP id 308A1140013 for ; Wed, 17 Jan 2024 02:20:14 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=hAz+cVqR; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf09.hostedemail.com: domain of yajun.deng@linux.dev designates 91.218.175.173 as permitted sender) smtp.mailfrom=yajun.deng@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705458017; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bXDzAov/eT5uorwlp2lCOxq4VcwswM9XM5Rnv6sFF88=; b=0BqVzmhAe7ps+FuHWM+rcEPg3uyv7fegVkNhhVEXz2rKSl08cDiH+tX1hnfyoanqOBCdMy woD2YH5cRIMaRyzJ9YORjg9b3S9XhsqKaqrVUC/p2wS44ZIrE48f66dipSRTKjDq5ERjyA VCDXusCKyW9idOW0FohYe8BWLshhwb0= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=hAz+cVqR; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf09.hostedemail.com: domain of yajun.deng@linux.dev designates 91.218.175.173 as permitted sender) smtp.mailfrom=yajun.deng@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705458017; a=rsa-sha256; cv=none; b=NN8yLw+kdbXVhDHlFBdMc6og013qeTOmvlxsGFLz0OuTJsMqwEEaO9vIjdFyY0RC+iuUUi SMR0d5WyY0PskwuNMEewI4OWasxsGEwljDaxEBBugL/nZ1Vwiz57Lqvwr/fMwGvgVSKWkY oISnAqXAOCnQ7Fo1/j8S4z6GzCq4K3A= Message-ID: <3fc2f75e-d163-1ad1-009a-0e4538011885@linux.dev> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1705458010; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bXDzAov/eT5uorwlp2lCOxq4VcwswM9XM5Rnv6sFF88=; b=hAz+cVqRmHwjVsJ/044guG9PwLrawXcGWx4WxojuaBwP0Tfz9VTCZp6qu6ooEiaWJ4jqQQ KdEhDH/FfOxpKRXw6nGVxzxpota9+fd3XSKU4dpFY/DAYHoYo5uVs9ebR2kyP2zvB7Mvwo 71s8A/lnZM8S4j2k7ofwbzNZj7T6uEg= Date: Wed, 17 Jan 2024 10:20:00 +0800 MIME-Version: 1.0 Subject: Re: memblock_reserve for unadded region (was: [PATCH] MIPS: loongson64: fix boot failure) Content-Language: en-US To: Huang Pei , Mike Rapoport Cc: Jiaxun Yang , linux-mm@kvack.org, Bibo Mao , linux-mips@vger.kernel.org, Paul Burton , Li Xuefeng , Yang Tiezhu , Gao Juxin , Huacai Chen , Thomas Bogendoerfer , linux-kernel@vger.kernel.org References: <20231225093025.23215-1-huangpei@loongson.cn> <731134fd-4b3d-418c-84ee-80646bffcc01@flygoat.com> <20240116122304.qwzy7san2vgspt2x@Board-3A3000> X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Yajun Deng In-Reply-To: <20240116122304.qwzy7san2vgspt2x@Board-3A3000> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: 308A1140013 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: mtmjirnahs8ikefmt41j8erwi9z68q9y X-HE-Tag: 1705458014-176760 X-HE-Meta: U2FsdGVkX1+v3qhlx/PEvb+cRs1lIa2Wf3blfRvY9G0ndxhNomnWxQRrhJtRIGHHcigSwWECcYD68B1SC3+IInaRXRprxWfPt/WGDKsM4uC/+XXCXMqb0g+sXzG0ZWcnt111+c9pN6sRYp4ngm+s8PwDGecVeVw/fRtW8r0X3taQWLpUyRJOvtOUjY9MDRpyxuftzTQr0mJaMyisE5Cu34uEX8Ui385mSHlaQOAw+rcZqVCcmH7ktSTBKjY86x3lPo+P7LbIZqoa/dGkvNcjOEOWBwlo9RRBbRcJAZaXkEfsOY/BmKcZZoSv1ijZPybxgfgJDBPpfAYB8BuRWXUgckh+Bxk6tP+DrNM0LSIYu2eoBjDd8EuCYhprU/YeJG5hE1WSiOPIp5sCzxnw7Knbye0lub2nZd37WMuLb4vgrgtLfwEQ+wp1RccNLD+8f74hjEPeLb2EgrthcFWEmhau/7Fra2Y9YnJb+O/peazgeUBRBgB0hBpIOPGyEPvD8AZNQ95+owwOPcz+ei3VUZQrtV2osUZcjiFTcrKwhUxEq0slR6mnr7bEby8PN+alE6nRN7w0WTQcfUhUl/rG7SiFXBV0ZNQsGWAGAHoYeDgtd85K5GHn8bZTIoYgA4xkSIMsni3XMwjKZrIY41BEC4HkvsEBCpp/+RO4D/KoMWh1GSGJ7fLgAsJq9/s6Zpcbq5rzYeb5EoXwbSP6u3jLJEpDSGHZeCoLB/g/uHqZcUbxdOGJB+vfo3SKyzCV3HN4xFYNytk0qUgbCx/a2rfK+cQsmtuav+ymdadsTDJvixoZ8aM2rO/oaktaOMfdIinoyEBGLVKwRy0qFafA1AWK+u9iVJe4531ECGZyAl1tw1SlOtUAT5ZAsAWHDdhojPSsYlIQ94l5vtniZcSHPv2czhOe5/z4ko9zTgC/uqhiyMdcoy2F5fqlc3BtV+GKeLIiGdQ6+QukdPtqUDaR9zNRFP1 OjJqqBFP YPi8NBTFMvNj+kWqf5KtMlHikihHKBsVDdwnvolFq4apaPhwmjYfy1XUktNUMr4+fBwUO6EezHIQNXVxONh5irt7NU9z2b6IiooGwqc3x0+eGsqJHzTADcbIZ6DbDQ1mmJeLg X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024/1/16 20:23, Huang Pei wrote: > On Tue, Jan 16, 2024 at 10:39:04AM +0200, Mike Rapoport wrote: >> On Mon, Jan 15, 2024 at 02:08:21PM +0000, Jiaxun Yang wrote: >>> Hi mm folks, >>> >>> Just a quick question, what is the expected behavior of memblock_reserve >>> a region that is not added to memblock with memblock_add? >>> >>> I'm unable to find any documentation about memblock_reserve in comments and >>> boot-time-mm, but as per my understanding to the code, this should be a >>> legit usage? >> Yes, memblock allows reserving memory that was not added to memblock with >> memblock_add(). > I think arch/platform specific code should fix this bug, like, > -------------------------------------------------------------------------- > //for loongson64 > memblock_set_node(0, 1ULL << 44, &memblock.reserved, 0); > > -------------------------------------------------------------------------- > > or maybe memblock provide something like memblock_reserve_node Hi pei, Can you test the following patch to see if it fixes this bug? diff --git a/mm/mm_init.c b/mm/mm_init.c index 2c19f5515e36..97721d99fdce 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -708,6 +708,9 @@ static void __meminit init_reserved_page(unsigned long pfn, int nid)         pg_data_t *pgdat;         int zid; +       if (unlikely(nid == NUMA_NO_NODE || nid >= MAX_NUMNODES)) +               nid = early_pfn_to_nid(pfn); +         if (early_page_initialised(pfn, nid))                 return; >> >>> In practical we run into uninitialized nid of reserved block problem, should >>> we fix it >>> in our usage, or on memblock side? >> Apparently it's a bug in memblock :( >> >> If you revert 61167ad5fecd ("mm: pass nid to reserve_bootmem_region()") >> does the issue disappear? > Yes, I git bisect this commit. > > But I don't think it is a bug in memblock. IMO, memblock_reserve under > NUMA set nid of reserved region to MAX_NUMNODES, which is the point > that cause the "memblock_get_region_node from memmap_init_reserved_pages " > passing a invalid node id(aka MAX_NUMNODES) to "reserver_bootmem_region > -> init_reserved_page -> early_pfn_to_nid". If arch-specific code DOES NOT > initialize the nid of reserved region(only it know that), or the reserved > region NOT within a memblock added by memblock_add, memblock can not > give a valid node id to the reserved region. Commit 61167ad5fecd ("mm: pass nid to > reserve_bootmem_region()") just reveals the embarrassment case by an > out of bound memory access. > >> >>> Thanks >>> >>> 在 2023/12/25 09:30, Huang Pei 写道: >>>> Since commit 61167ad5fecd("mm: pass nid to reserve_bootmem_region()), >>>> loongson64 booting failed with CONFIG_DEFERRED_STRUCT_PAGE_INIT like >>>> this: >>>> ---------------------------------------------------------------------- >>>> Call Trace: >>>> [] reserve_bootmem_region+0xa8/0x184 >>>> [] memblock_free_all+0x104/0x2a8 >>>> [] mem_init+0x84/0x94 >>>> [] mm_core_init+0xf8/0x308 >>>> [] start_kernel+0x43c/0x86c >>>> >>>> Code: 10400028 2402fff0 de420000 0203182b 14600022 >>>> 64420070 00003025 24040003 >>>> >>>> ---[ end trace 0000000000000000 ]--- >>>> Kernel panic - not syncing: Attempted to kill the idle task! >>>> ---[ end Kernel panic - not syncing: Attempted to kill the idle task! ]--- >>>> ---------------------------------------------------------------------- >>>> >>>> The root cause is no memory region "0x0-0x1fffff" paired with >>>> memory-reserved region "0x0-0x1fffff" and "0x0-0xfff", with "memblock >>>> =debug": >>>> >>>> ---------------------------------------------------------------------- >>>> memory[0x0] [0x0000000000200000-0x000000000effffff], >>>> 0x000000000ee00000 bytes on node 0 flags: 0x0 !!!!here >>>> memory[0x1] [0x0000000090000000-0x00000000fdffffff], >>>> 0x000000006e000000 bytes on node 0 flags: 0x0 >>>> memory[0x2] [0x0000000100000000-0x000000027fffffff], >>>> 0x0000000180000000 bytes on node 0 flags: 0x0 >>>> memory[0x3] [0x0000100000000000-0x000010000fffffff], >>>> 0x0000000010000000 bytes on node 1 flags: 0x0 >>>> memory[0x4] [0x0000100090000000-0x000010027fffffff], >>>> 0x00000001f0000000 bytes on node 1 flags: 0x0 >>>> reserved.cnt = 0x1f >>>> reserved[0x0] [0x0000000000000000-0x000000000190c80a], >>>> 0x000000000190c80b bytes flags: 0x0 !!!!oops 0x0-0x1fffff not in memory[0] >>>> reserved[0x1] [0x000000000190c810-0x000000000190eea3], >>>> 0x0000000000002694 bytes flags: 0x0 >>>> ---------------------------------------------------------------------- >>>> >>>> It caused memory-reserved region "0x0-0x1fffff" without valid node id >>>> in "memblock_get_region_node" from "memmap_init_reserved_pages", lead to >>>> "reserve_bootmem_region-> init_reserved_page -> early_pfn_to_nid()" >>>> accessing "node_data" out of bound. >>>> >>>> To fix this bug, we should remove unnecessary memory block reservation. >>>> >>>> +. no need to reserve 0x0-0x1fffff below kernel loading address, since >>>> it is not registered by "memblock_add_node" >>>> >>>> +. no need to reserve 0x0-0xfff for exception handling if it is not >>>> registered by "memblock_add" either. >>>> >>>> Fixes: commit 61167ad5fecd("mm: pass nid to reserve_bootmem_region()) >>>> Signed-off-by: Huang Pei >>>> --- >>>> arch/mips/kernel/traps.c | 3 ++- >>>> arch/mips/loongson64/numa.c | 2 -- >>>> 2 files changed, 2 insertions(+), 3 deletions(-) >>>> >>>> diff --git a/arch/mips/kernel/traps.c b/arch/mips/kernel/traps.c >>>> index 246c6a6b0261..9b632b4c10c3 100644 >>>> --- a/arch/mips/kernel/traps.c >>>> +++ b/arch/mips/kernel/traps.c >>>> @@ -2007,7 +2007,8 @@ unsigned long vi_handlers[64]; >>>> void reserve_exception_space(phys_addr_t addr, unsigned long size) >>>> { >>>> - memblock_reserve(addr, size); >>>> + if(memblock_is_region_memory(addr, size)) >>>> + memblock_reserve(addr, size); >>>> } >>>> void __init *set_except_vector(int n, void *addr) >>>> diff --git a/arch/mips/loongson64/numa.c b/arch/mips/loongson64/numa.c >>>> index 8f61e93c0c5b..0f516dde81da 100644 >>>> --- a/arch/mips/loongson64/numa.c >>>> +++ b/arch/mips/loongson64/numa.c >>>> @@ -130,8 +130,6 @@ static void __init node_mem_init(unsigned int node) >>>> memblock_reserve((node_addrspace_offset | 0xfe000000), >>>> 32 << 20); >>>> - /* Reserve pfn range 0~node[0]->node_start_pfn */ >>>> - memblock_reserve(0, PAGE_SIZE * start_pfn); >>>> } >>>> } >>> -- >>> --- >>> Jiaxun Yang >>> >> -- >> Sincerely yours, >> Mike.