From: Bruno Faccini <bfaccini@nvidia.com>
To: David Hildenbrand <david@redhat.com>, linux-kernel@vger.kernel.org
Cc: linux-mm@kvack.org, akpm@linux-foundation.org, rppt@kernel.org,
ziy@nvidia.com, jhubbard@nvidia.com, mrusiniak@nvidia.com
Subject: Re: [PATCH 0/1] mm/fake-numa: allow later numa node hotplug
Date: Fri, 10 Jan 2025 15:59:38 +0100 [thread overview]
Message-ID: <0b673652-8a84-4769-a193-d090a50e91cd@nvidia.com> (raw)
In-Reply-To: <2f19432c-cc3a-4860-9aa3-e6bc1f60de3d@redhat.com>
Hello David,
Le 07/01/2025 à 11:08, David Hildenbrand a écrit :
>
> Hi,
>
>>
>> With recent M.Rapoport set of fake-numa patches in mm-everything
>> and using numa=fake=4 boot parameter :
>> $ numactl --hardware
>> available: 4 nodes (0-3)
>> node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
>> 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42
>> 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64
>> 65 66 67 68 69 70 71
>> node 0 size: 122518 MB
>> node 0 free: 117141 MB
>> node 1 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
>> 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42
>> 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64
>> 65 66 67 68 69 70 71
>> node 1 size: 219911 MB
>> node 1 free: 219751 MB
>> node 2 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
>> 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42
>> 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64
>> 65 66 67 68 69 70 71
>> node 2 size: 122599 MB
>> node 2 free: 122541 MB
>> node 3 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
>> 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42
>> 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64
>> 65 66 67 68 69 70 71
>> node 3 size: 122479 MB
>> node 3 free: 122408 MB
>
> Why are all CPUs indicated as belonging to all nodes? Is that expected
> or a BUG?
This behaviour comes from original fake-numa implementation, and has
been left as is by M.Rapoport recent fake-numa changes.
>
> I would have thought, just like memory, that one resource only belongs
> to one NUMA node.
"All fake-NUMA nodes that belong to a physical NUMA node share the same
CPU cores", this was already the case in original/x86-only
implementation so that fake-NUMA does not affect application launch
commands.
>
>
>>
>> With recent M.Rapoport set of fake-numa patches in mm-everything,
>> this patch on top, using numa=fake=4 boot parameter :
>> # numactl —hardware
>> available: 12 nodes (0-11)
>> node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
>> 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42
>> 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64
>> 65 66 67 68 69 70 71
>> node 0 size: 122518 MB
>> node 0 free: 116429 MB
>> node 1 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
>> 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42
>> 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64
>> 65 66 67 68 69 70 71
>> node 1 size: 122631 MB
>> node 1 free: 122576 MB
>> node 2 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
>> 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42
>> 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64
>> 65 66 67 68 69 70 71
>> node 2 size: 122599 MB
>> node 2 free: 122544 MB
>> node 3 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
>> 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42
>> 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64
>> 65 66 67 68 69 70 71
>> node 3 size: 122479 MB
>> node 3 free: 122419 MB
>> node 4 cpus:
>> node 4 size: 97280 MB
>> node 4 free: 97279 MB
>
>
> ^ Is this where your driver hotplugged a single node and hotplugged memory?
Yes, Node 4 is a GPU node and its memory has been hotplugged by the driver.
>
>
> --
> Cheers,
>
> David / dhildenb
>
Thanks for your review and comments/questions, bye,
Bruno
next prev parent reply other threads:[~2025-01-10 15:00 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-01-06 12:06 Bruno Faccini
2025-01-06 12:06 ` [PATCH 1/1] " Bruno Faccini
2025-01-14 16:05 ` David Hildenbrand
2025-01-17 5:00 ` [fake numa not working][PATCH " Hyeonggon Yoo
2025-01-17 19:09 ` [fake numa not working] [PATCH " Zi Yan
2025-01-20 0:37 ` Hyeonggon Yoo
2025-01-20 2:16 ` Zi Yan
2025-01-20 2:37 ` Hyeonggon Yoo
2025-01-20 3:27 ` Zi Yan
2025-01-20 4:35 ` Hyeonggon Yoo
2025-01-20 8:11 ` Bruno Faccini
2025-01-27 17:38 ` Bruno Faccini
2025-01-07 10:08 ` [PATCH 0/1] " David Hildenbrand
2025-01-10 14:59 ` Bruno Faccini [this message]
2025-01-14 14:15 ` David Hildenbrand
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0b673652-8a84-4769-a193-d090a50e91cd@nvidia.com \
--to=bfaccini@nvidia.com \
--cc=akpm@linux-foundation.org \
--cc=david@redhat.com \
--cc=jhubbard@nvidia.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mrusiniak@nvidia.com \
--cc=rppt@kernel.org \
--cc=ziy@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox