From: Zi Yan <ziy@nvidia.com>
To: Mike Rapoport <rppt@kernel.org>
Cc: linux-kernel@vger.kernel.org,
Alexander Gordeev <agordeev@linux.ibm.com>,
Andreas Larsson <andreas@gaisler.com>,
Andrew Morton <akpm@linux-foundation.org>,
Arnd Bergmann <arnd@arndb.de>, Borislav Petkov <bp@alien8.de>,
Catalin Marinas <catalin.marinas@arm.com>,
Christophe Leroy <christophe.leroy@csgroup.eu>,
Dan Williams <dan.j.williams@intel.com>,
Dave Hansen <dave.hansen@linux.intel.com>,
David Hildenbrand <david@redhat.com>,
"David S. Miller" <davem@davemloft.net>,
Davidlohr Bueso <dave@stgolabs.net>,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
Heiko Carstens <hca@linux.ibm.com>,
Huacai Chen <chenhuacai@kernel.org>,
Ingo Molnar <mingo@redhat.com>,
Jiaxun Yang <jiaxun.yang@flygoat.com>,
John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de>,
Jonathan Cameron <jonathan.cameron@huawei.com>,
Jonathan Corbet <corbet@lwn.net>,
Michael Ellerman <mpe@ellerman.id.au>,
Palmer Dabbelt <palmer@dabbelt.com>,
"Rafael J. Wysocki" <rafael@kernel.org>,
Rob Herring <robh@kernel.org>,
Samuel Holland <samuel.holland@sifive.com>,
Thomas Bogendoerfer <tsbogend@alpha.franken.de>,
Thomas Gleixner <tglx@linutronix.de>,
Vasily Gorbik <gor@linux.ibm.com>, Will Deacon <will@kernel.org>,
devicetree@vger.kernel.org, linux-acpi@vger.kernel.org,
linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org,
linux-cxl@vger.kernel.org, linux-doc@vger.kernel.org,
linux-mips@vger.kernel.org, linux-mm@kvack.org,
linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org,
linux-sh@vger.kernel.org, linuxppc-dev@lists.ozlabs.org,
loongarch@lists.linux.dev, nvdimm@lists.linux.dev,
sparclinux@vger.kernel.org, x86@kernel.org
Subject: Re: [PATCH v2 00/25] mm: introduce numa_memblks
Date: Wed, 24 Jul 2024 20:35:53 -0400 [thread overview]
Message-ID: <6336C276-113E-4D93-A09E-13420A6438D8@nvidia.com> (raw)
In-Reply-To: <1D474894-F8AC-427B-8F90-5A6808E77CC5@nvidia.com>
[-- Attachment #1: Type: text/plain, Size: 8458 bytes --]
On 24 Jul 2024, at 18:44, Zi Yan wrote:
> On 23 Jul 2024, at 2:41, Mike Rapoport wrote:
>
>> From: "Mike Rapoport (Microsoft)" <rppt@kernel.org>
>>
>> Hi,
>>
>> Following the discussion about handling of CXL fixed memory windows on
>> arm64 [1] I decided to bite the bullet and move numa_memblks from x86 to
>> the generic code so they will be available on arm64/riscv and maybe on
>> loongarch sometime later.
>>
>> While it could be possible to use memblock to describe CXL memory windows,
>> it currently lacks notion of unpopulated memory ranges and numa_memblks
>> does implement this.
>>
>> Another reason to make numa_memblks generic is that both arch_numa (arm64
>> and riscv) and loongarch use trimmed copy of x86 code although there is no
>> fundamental reason why the same code cannot be used on all these platforms.
>> Having numa_memblks in mm/ will make it's interaction with ACPI and FDT
>> more consistent and I believe will reduce maintenance burden.
>>
>> And with generic numa_memblks it is (almost) straightforward to enable NUMA
>> emulation on arm64 and riscv.
>>
>> The first 9 commits in this series are cleanups that are not strictly
>> related to numa_memblks.
>> Commits 10-16 slightly reorder code in x86 to allow extracting numa_memblks
>> and NUMA emulation to the generic code.
>> Commits 17-19 actually move the code from arch/x86/ to mm/ and commits 20-22
>> does some aftermath cleanups.
>> Commit 23 switches arch_numa to numa_memblks.
>> Commit 24 enables usage of phys_to_target_node() and
>> memory_add_physaddr_to_nid() with numa_memblks.
>> Commit 25 moves the description for numa=fake from x86 to admin-guide
>>
>> [1] https://lore.kernel.org/all/20240529171236.32002-1-Jonathan.Cameron@huawei.com/
>>
>> v1: https://lore.kernel.org/all/20240716111346.3676969-1-rppt@kernel.org
>> * add cleanup for arch_alloc_nodedata and HAVE_ARCH_NODEDATA_EXTENSION
>> * add patch that moves description of numa=fake kernel parameter from
>> x86 to admin-guide
>> * reduce rounding up of node_data allocations from PAGE_SIZE to
>> SMP_CACHE_BYTES
>> * restore single allocation attempt of numa_distance
>> * fix several comments
>> * added review tags
>>
>> Mike Rapoport (Microsoft) (25):
>> mm: move kernel/numa.c to mm/
>> MIPS: sgi-ip27: make NODE_DATA() the same as on all other architectures
>> MIPS: sgi-ip27: ensure node_possible_map only contains valid nodes
>> MIPS: sgi-ip27: drop HAVE_ARCH_NODEDATA_EXTENSION
>> MIPS: loongson64: rename __node_data to node_data
>> MIPS: loongson64: drop HAVE_ARCH_NODEDATA_EXTENSION
>> mm: drop CONFIG_HAVE_ARCH_NODEDATA_EXTENSION
>> arch, mm: move definition of node_data to generic code
>> arch, mm: pull out allocation of NODE_DATA to generic code
>> x86/numa: simplify numa_distance allocation
>> x86/numa: use get_pfn_range_for_nid to verify that node spans memory
>> x86/numa: move FAKE_NODE_* defines to numa_emu
>> x86/numa_emu: simplify allocation of phys_dist
>> x86/numa_emu: split __apicid_to_node update to a helper function
>> x86/numa_emu: use a helper function to get MAX_DMA32_PFN
>> x86/numa: numa_{add,remove}_cpu: make cpu parameter unsigned
>> mm: introduce numa_memblks
>> mm: move numa_distance and related code from x86 to numa_memblks
>> mm: introduce numa_emulation
>> mm: numa_memblks: introduce numa_memblks_init
>> mm: numa_memblks: make several functions and variables static
>> mm: numa_memblks: use memblock_{start,end}_of_DRAM() when sanitizing
>> meminfo
>> arch_numa: switch over to numa_memblks
>> mm: make range-to-target_node lookup facility a part of numa_memblks
>> docs: move numa=fake description to kernel-parameters.txt
>>
> Hi,
>
> I have tested this series on both x86_64 and arm64. It works fine on x86_64.
> All numa=fake= options work as they did before the series.
>
> But I am not able to boot the kernel (no printout at all) on arm64 VM
> (Mac mini M1 VMWare). By git bisecting, arch_numa: switch over to numa_memblks
> is the first patch causing the boot failure. I see the warning:
>
> WARNING: modpost: vmlinux: section mismatch in reference: numa_add_cpu+0x1c (section: .text) -> early_cpu_to_node (section: .init.text)
>
> I am not sure if it is red herring or not, since changing early_cpu_to_node
> to cpu_to_node in numa_add_cpu() from mm/numa_emulation.c did get rid of the
> warning, but the system still failed to boot.
>
> Please note that you need binutils 2.40 to build the arm64 kernel, since there
> is a bug(https://sourceware.org/bugzilla/show_bug.cgi?id=31924) in 2.42 preventing
> arm64 kernel from booting as well.
>
> My config is attached.
I get more info after adding earlycon to the boot option.
pgdat is NULL, causing issues when free_area_init_node() is dereferencing
it at first WARN_ON.
FYI, my build is this series on top of v6.10 instead of the base commit,
where the series applies cleanly on top v6.10.
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000080000000-0x00000000ffd82fff]
[ 0.000000] node 0: [mem 0x00000000ffd83000-0x00000000fffb5fff]
[ 0.000000] node 0: [mem 0x00000000fffb6000-0x000000017befffff]
[ 0.000000] node 0: [mem 0x000000017bf00000-0x000000017bfbffff]
[ 0.000000] node 0: [mem 0x000000017bfc0000-0x000000017c02ffff]
[ 0.000000] node 0: [mem 0x000000017c030000-0x000000017c03ffff]
[ 0.000000] node 0: [mem 0x000000017c040000-0x000000017c09ffff]
[ 0.000000] node 0: [mem 0x000000017c0a0000-0x000000017c13ffff]
[ 0.000000] node 0: [mem 0x000000017c140000-0x000000017f41ffff]
[ 0.000000] node 0: [mem 0x000000017f420000-0x000000017f4affff]
[ 0.000000] node 0: [mem 0x000000017f4b0000-0x000000017f5bffff]
[ 0.000000] node 0: [mem 0x000000017f5c0000-0x000000017f5dffff]
[ 0.000000] node 0: [mem 0x000000017f5e0000-0x000000017fffffff]
[ 0.000000] pgdat: 0000000000000000, nid: 0
[ 0.000000] Unable to handle kernel paging request at virtual address 0000000000002220
[ 0.000000] Mem abort info:
[ 0.000000] ESR = 0x0000000096000004
[ 0.000000] EC = 0x25: DABT (current EL), IL = 32 bits
[ 0.000000] SET = 0, FnV = 0
[ 0.000000] EA = 0, S1PTW = 0
[ 0.000000] FSC = 0x04: level 0 translation fault
[ 0.000000] Data abort info:
[ 0.000000] ISV = 0, ISS = 0x00000004, ISS2 = 0x00000000
[ 0.000000] CM = 0, WnR = 0, TnD = 0, TagAccess = 0
[ 0.000000] GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
[ 0.000000] [0000000000002220] user address but active_mm is swapper
[ 0.000000] Internal error: Oops: 0000000096000004 [#1] SMP
[ 0.000000] Modules linked in:
[ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 6.10.0+ #17
[ 0.000000] pstate: 600000c5 (nZCv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 0.000000] pc : free_area_init+0x720/0xf90
[ 0.000000] lr : free_area_init+0x714/0xf90
[ 0.000000] sp : ffff800081eb3c20
[ 0.000000] x29: ffff800081eb3c20 x28: 000000017b5e710c x27: ffff800082158000
[ 0.000000] x26: 000000017ffff168 x25: ffff800081ecc000 x24: 0000000000000000
[ 0.000000] x23: 0000000000000000 x22: 0000000000000000 x21: ffff8000821f0480
[ 0.000000] x20: 0000000000000000 x19: ffff8000818863f0 x18: 0000000000000006
[ 0.000000] x17: 00000000007fb000 x16: 000000017f805000 x15: ffff800081eb36b0
[ 0.000000] x14: 0000000000000000 x13: 30203a64696e202c x12: ffff800081f3ef10
[ 0.000000] x11: 0000000000000001 x10: 0000000000000001 x9 : 0000000000017fe8
[ 0.000000] x8 : c0000000ffffefff x7 : ffff800081ee6d40 x6 : 0000000000057fa8
[ 0.000000] x5 : ffff800081f3eeb8 x4 : 0000000000000000 x3 : 0000000000000000
[ 0.000000] x2 : 0000000000000000 x1 : ffff800081ec8c40 x0 : 000000000000001f
[ 0.000000] Call trace:
[ 0.000000] free_area_init+0x720/0xf90
[ 0.000000] bootmem_init+0x158/0x218
[ 0.000000] setup_arch+0x220/0x650
[ 0.000000] start_kernel+0x74/0x7e0
[ 0.000000] __primary_switched+0x80/0x90
[ 0.000000] Code: 97a64606 b940b7f6 a90dffff f876dab4 (b9622280)
[ 0.000000] ---[ end trace 0000000000000000 ]---
[ 0.000000] Kernel panic - not syncing: Attempted to kill the idle task!
[ 0.000000] ---[ end Kernel panic - not syncing: Attempted to kill the idle task! ]---
--
Best Regards,
Yan, Zi
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 854 bytes --]
next prev parent reply other threads:[~2024-07-25 0:36 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-07-23 6:41 Mike Rapoport
2024-07-23 6:41 ` [PATCH v2 01/25] mm: move kernel/numa.c to mm/ Mike Rapoport
2024-07-23 6:41 ` [PATCH v2 02/25] MIPS: sgi-ip27: make NODE_DATA() the same as on all other architectures Mike Rapoport
2024-07-23 6:41 ` [PATCH v2 03/25] MIPS: sgi-ip27: ensure node_possible_map only contains valid nodes Mike Rapoport
2024-07-23 6:41 ` [PATCH v2 04/25] MIPS: sgi-ip27: drop HAVE_ARCH_NODEDATA_EXTENSION Mike Rapoport
2024-07-23 6:41 ` [PATCH v2 05/25] MIPS: loongson64: rename __node_data to node_data Mike Rapoport
2024-07-23 6:41 ` [PATCH v2 06/25] MIPS: loongson64: drop HAVE_ARCH_NODEDATA_EXTENSION Mike Rapoport
2024-07-23 6:41 ` [PATCH v2 07/25] mm: drop CONFIG_HAVE_ARCH_NODEDATA_EXTENSION Mike Rapoport
2024-07-23 6:41 ` [PATCH v2 08/25] arch, mm: move definition of node_data to generic code Mike Rapoport
2024-07-23 6:41 ` [PATCH v2 09/25] arch, mm: pull out allocation of NODE_DATA " Mike Rapoport
2024-07-23 6:41 ` [PATCH v2 10/25] x86/numa: simplify numa_distance allocation Mike Rapoport
2024-07-23 6:41 ` [PATCH v2 11/25] x86/numa: use get_pfn_range_for_nid to verify that node spans memory Mike Rapoport
2024-07-23 6:41 ` [PATCH v2 12/25] x86/numa: move FAKE_NODE_* defines to numa_emu Mike Rapoport
2024-07-23 6:41 ` [PATCH v2 13/25] x86/numa_emu: simplify allocation of phys_dist Mike Rapoport
2024-07-23 6:41 ` [PATCH v2 14/25] x86/numa_emu: split __apicid_to_node update to a helper function Mike Rapoport
2024-07-23 6:41 ` [PATCH v2 15/25] x86/numa_emu: use a helper function to get MAX_DMA32_PFN Mike Rapoport
2024-07-23 6:41 ` [PATCH v2 16/25] x86/numa: numa_{add,remove}_cpu: make cpu parameter unsigned Mike Rapoport
2024-07-23 6:41 ` [PATCH v2 17/25] mm: introduce numa_memblks Mike Rapoport
2024-07-23 6:41 ` [PATCH v2 18/25] mm: move numa_distance and related code from x86 to numa_memblks Mike Rapoport
2024-07-23 6:41 ` [PATCH v2 19/25] mm: introduce numa_emulation Mike Rapoport
2024-07-23 6:41 ` [PATCH v2 20/25] mm: numa_memblks: introduce numa_memblks_init Mike Rapoport
2024-07-23 6:41 ` [PATCH v2 21/25] mm: numa_memblks: make several functions and variables static Mike Rapoport
2024-07-23 6:41 ` [PATCH v2 22/25] mm: numa_memblks: use memblock_{start,end}_of_DRAM() when sanitizing meminfo Mike Rapoport
2024-07-23 6:41 ` [PATCH v2 23/25] arch_numa: switch over to numa_memblks Mike Rapoport
2024-07-23 6:41 ` [PATCH v2 24/25] mm: make range-to-target_node lookup facility a part of numa_memblks Mike Rapoport
2024-07-23 6:41 ` [PATCH v2 25/25] docs: move numa=fake description to kernel-parameters.txt Mike Rapoport
2024-07-24 22:44 ` [PATCH v2 00/25] mm: introduce numa_memblks Zi Yan
2024-07-25 0:35 ` Zi Yan [this message]
2024-07-25 2:48 ` Zi Yan
2024-07-26 9:40 ` Mike Rapoport
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=6336C276-113E-4D93-A09E-13420A6438D8@nvidia.com \
--to=ziy@nvidia.com \
--cc=agordeev@linux.ibm.com \
--cc=akpm@linux-foundation.org \
--cc=andreas@gaisler.com \
--cc=arnd@arndb.de \
--cc=bp@alien8.de \
--cc=catalin.marinas@arm.com \
--cc=chenhuacai@kernel.org \
--cc=christophe.leroy@csgroup.eu \
--cc=corbet@lwn.net \
--cc=dan.j.williams@intel.com \
--cc=dave.hansen@linux.intel.com \
--cc=dave@stgolabs.net \
--cc=davem@davemloft.net \
--cc=david@redhat.com \
--cc=devicetree@vger.kernel.org \
--cc=glaubitz@physik.fu-berlin.de \
--cc=gor@linux.ibm.com \
--cc=gregkh@linuxfoundation.org \
--cc=hca@linux.ibm.com \
--cc=jiaxun.yang@flygoat.com \
--cc=jonathan.cameron@huawei.com \
--cc=linux-acpi@vger.kernel.org \
--cc=linux-arch@vger.kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-cxl@vger.kernel.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mips@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-riscv@lists.infradead.org \
--cc=linux-s390@vger.kernel.org \
--cc=linux-sh@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=loongarch@lists.linux.dev \
--cc=mingo@redhat.com \
--cc=mpe@ellerman.id.au \
--cc=nvdimm@lists.linux.dev \
--cc=palmer@dabbelt.com \
--cc=rafael@kernel.org \
--cc=robh@kernel.org \
--cc=rppt@kernel.org \
--cc=samuel.holland@sifive.com \
--cc=sparclinux@vger.kernel.org \
--cc=tglx@linutronix.de \
--cc=tsbogend@alpha.franken.de \
--cc=will@kernel.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox