From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A216E7719A for ; Mon, 13 Jan 2025 04:47:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 69C1A6B007B; Sun, 12 Jan 2025 23:47:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 64B576B0083; Sun, 12 Jan 2025 23:47:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4EB7C6B0085; Sun, 12 Jan 2025 23:47:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 31F176B007B for ; Sun, 12 Jan 2025 23:47:57 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id BEBFBAE0F6 for ; Mon, 13 Jan 2025 04:47:56 +0000 (UTC) X-FDA: 83001196152.22.43F1505 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by imf14.hostedemail.com (Postfix) with ESMTP id 77168100005 for ; Mon, 13 Jan 2025 04:47:54 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=KYrNjnRZ; spf=pass (imf14.hostedemail.com: domain of sourabhjain@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=sourabhjain@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736743674; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2PRWxMkp9y5tv7cPQW7nLIdAwCNP4nh2N1dC4lMGuuc=; b=0vW0Svtahtufme7NgJIrZWyeFgbhqy2eI3A5dazAKM8Vuq0zo83HIVP5fCPVb/B6xFcp8V x94XN/uHbDlaYIngTqAVX/omIMwz5QAACojVMKMcgzjlkK1rEtnbaptlJrm7DxSzz03mQ5 8yfT/z7qi1p7HJXyjb7BG6Yvg2pC7UQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736743674; a=rsa-sha256; cv=none; b=zHjCbFPi56zTf6ZKwUZ9ydF6cAlmOzormLq5BKkBwGFD5SwsXq51VY7WW3zEB47DTeNbY7 ai5J92iKOtYI+3icfYJnjuk5J2AKziqmh8Y3x5TDoREvyJPXNyPFmOp2AAq8IYowLAQs00 pKTU/qsepX3Ip9w2CerMlgwij0KUrA8= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=KYrNjnRZ; spf=pass (imf14.hostedemail.com: domain of sourabhjain@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=sourabhjain@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 50CLs3nF023956; Mon, 13 Jan 2025 04:47:53 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pp1; bh=2PRWxM kp9y5tv7cPQW7nLIdAwCNP4nh2N1dC4lMGuuc=; b=KYrNjnRZ0ettRL7gHpwheo UKkg0t18JMeZQ0GIKBf9aNL1gPUBHhfcNht368wmcZ/dGQbejnCQoyTRUIvFKaq3 tkP9JysX32BmrZjs0pesqBwQgHJGr6ClPYQmpSizMIS2ns66q6N1O8XC3ykTTShf 0gD4oLTQfSH3sGfQ0Pineke2r4ymj+OOjxfJMlmuKScWQVVBQoBSfJt4db9p9dDW yoH7hcRhxyGRZsPFWpi8EKAzSYFl4q3Er3/U4C6rOvchXBuKlB++gHjeDa5ZmvKn 7BGdfYmWi6tio/VITjtAVG27pUv0cfCUFNXb+8eHk0z1F5sIrSTGghEtzgh8r8bw == Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4449x2aphw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 13 Jan 2025 04:47:53 +0000 (GMT) Received: from m0360072.ppops.net (m0360072.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 50D4lqkP025646; Mon, 13 Jan 2025 04:47:52 GMT Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4449x2aphu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 13 Jan 2025 04:47:52 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 50D4680S017014; Mon, 13 Jan 2025 04:47:52 GMT Received: from smtprelay05.fra02v.mail.ibm.com ([9.218.2.225]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4444fjvev0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 13 Jan 2025 04:47:52 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay05.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 50D4lmt352887890 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 13 Jan 2025 04:47:48 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4972320043; Mon, 13 Jan 2025 04:47:48 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CD20620040; Mon, 13 Jan 2025 04:47:46 +0000 (GMT) Received: from [9.109.204.94] (unknown [9.109.204.94]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Mon, 13 Jan 2025 04:47:46 +0000 (GMT) Message-ID: Date: Mon, 13 Jan 2025 10:17:45 +0530 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2] mm/hugetlb: Fix hugepage allocation for interleaved memory nodes To: "Ritesh Harjani (IBM)" , linux-mm@kvack.org Cc: Donet Tom , Gang Li , Daniel Jordan , Muchun Song , David Rientjes , Pavithra Prakash References: Content-Language: en-US From: Sourabh Jain In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-TM-AS-GCONF: 00 X-Proofpoint-GUID: 3ObbU1dCrrrAyGrF4k3Ig5Nh6jwarN-f X-Proofpoint-ORIG-GUID: 4-viO2wBnVt2CPYxwOsJrEyG78lXQjnz X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-15_01,2024-10-11_01,2024-09-30_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 malwarescore=0 bulkscore=0 suspectscore=0 mlxscore=0 lowpriorityscore=0 clxscore=1011 mlxlogscore=999 spamscore=0 priorityscore=1501 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2411120000 definitions=main-2501130036 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 77168100005 X-Stat-Signature: 8o7tbczsmzczugyoz6s1zhz38ug4wgka X-Rspam-User: X-HE-Tag: 1736743674-292011 X-HE-Meta: U2FsdGVkX18eVH3vMV8bFlmbXXZQ0R9L/tedKpG5auY3zCsSWXGl4uLKmJb0W151iiJ4Mueb8nZryXFvzeNnLnw4NLeuZGLdLbfoYvUx/tIsLnqAJ/EodvEEvTlMblhMWrQUJ0FOr3IyBJAoKco5Kwb2sak8x2XYECd5D4pRG35lPVNo86TLgcIoRu5o9+7T9/00UCgu0hc4q+mGEiB7RzklMK2Bo+oqaJh5Yl1dSZpdfGIpquCFQUp6Uy/PMTVNG5mpUbABoTfapdGRz98wQMsmVP7jE+GPOeDAtL9Mj0vSL4Xtg9Efqm1isanTdjk+iFzfkFSz+2nWiKg35O0UIZwG/QdQr1jZc44wHujVzwbdsVmWUiWaFd1jr6DauUQUdV4jVM8ZvJwKlVpl3IPsunk6ux7lphpj9luPuANcCmcBJ/gH6YY1hR6jd0pfPDU9J6FN67Mfd2N+PaBO7iz5qysEuAXy/7Gqss19t2ei6BB0DwaqP+LxZYS0gAs51jWpAioLvs1/Yl/TERAIsh2UzXmDE5eCaO3jqI0/0WjJolIOgjemVJ2TiTcrItpHUDnhciUMM7X/CjX/eZucJv2psLGOnCObDd7dxEHRZKDW2rOD2vhTRVcxNHwgoBqvgzfaT90VOL5uRUr39EyiRc9Vo64dFvBgFZaA8ldNPXqTLkjtXh41aqV0bc1EMrpNYpfilYGY5UOF5/iJpldoz1IqLhZelyTuKurUYSvTQemCOJmq/9l45os2otkVHIqd9N/Si0wXuC7/QGSuEm4A69G/oI4+wTNmFfkjrXMZn1xs3IAZ/02KzAPSU5Gpvs6XYKFM0R6p6oBXaHH+loyiJCzhDX4IjRJ/YhEXUpEErJxoag1z7dbi1T/ZC3HPVuDHbY3un7zna2x186FvAaEYzUdm77HlW9imigYqmnt6bQMjJnBjt/pv7oFJFr/U4R6zddBQ6mQ7/Bftt0PgO1Yh9w7 Zsqw+5yf ozTVoaqnDkoNLw7VUxLKpLeGAOYqeJ7dZ1ajAN5PHBYncZzfZpxrq78LihuUxNgmidDc/69NyJb2SeE/iCCJElavbXFILsZD95q70XdmC6QxH1oFQX4cus//7pxb/X14EqW35Ow5EA+Y26JspUcgMpTYxZsCLOnfAGawbbDG22xAxzf19iKXCnDy7dDMGQFWomr3JC+nkY21I5IcxojnCrytku1Z0IPxdcfca1pzyXtKOtJbeiMPtXmcVpJUBzYD5lPkq75XIzUw8NBWX79DiLCTJUhf3ackLBKE9M+lo3J81L07OgMP7bU0tQ5Rau8nT4YvlA1j0fJZ//YPpHzHUxqLHjU31MvN0/UxT+h8tNsDPN5Iff4CACYRmz0kbf0K9mtTn1zlThwKIlit6XH8bJLIy4ZYH4yYcs2pKeUtn0lZT3tduRo5X4MQzShuIWOcOp5xQgwpiiu/tGs7+namBdQvp2Z9GwgbuzxfBagSy/56SUySMXYwTKhQYTbvvh7lTQjVjUdGIE421PJ+nHbAPB6HbtPJtm5fkWeeR X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hello Ritesh, On 11/01/25 16:36, Ritesh Harjani (IBM) wrote: > gather_bootmem_prealloc() function assumes the start nid as 0 and size as > num_node_state(N_MEMORY). That means in case if memory attached numa nodes > are interleaved, then gather_bootmem_prealloc_parallel() will fail to scan > few of these nodes. Yes, I observed this issue on PowerVM platform where: [root ~]# dmesg | grep -i numa [    0.000000] numa: Partition configured for 32 NUMA nodes. [    0.631132] numa: Node 1 CPUs: 0-15 With the above numa configuration passing `hugepagesz=1G hugepages=2` the Hugetlb states in /proc/meminfo was 0. #cat /proc/meminfo snip... HugePages_Total:       0 HugePages_Free:        0 HugePages_Rsvd:        0 HugePages_Surp:        0 Hugepagesize:       2048 kB Hugetlb:               0 kB Where as with this FIX Included passing `hugepagesz=1G hugepages=2` on the same machine showing correct Hugetlb states in /proc/meminfo: # cat /proc/meminfo snip... HugePages_Total:       0 HugePages_Free:        0 HugePages_Rsvd:        0 HugePages_Surp:        0 Hugepagesize:       2048 kB Hugetlb:         2097152 kB Thanks for the fix: Tested-by: Sourabh Jain - Sourabh Jain > > Since memory attached numa nodes can be interleaved in any fashion, hence > ensure that the current code checks for all numa node ids > (.size = nr_node_ids). Let's still keep max_threads as N_MEMORY, so that > it can distributes all nr_node_ids among the these many no. threads. > > e.g. qemu cmdline > ======================== > numa_cmd="-numa node,nodeid=1,memdev=mem1,cpus=2-3 -numa node,nodeid=0,cpus=0-1 -numa dist,src=0,dst=1,val=20" > mem_cmd="-object memory-backend-ram,id=mem1,size=16G" > > w/o this patch for cmdline (default_hugepagesz=1GB hugepagesz=1GB hugepages=2): > ========================== > ~ # cat /proc/meminfo |grep -i huge > AnonHugePages: 0 kB > ShmemHugePages: 0 kB > FileHugePages: 0 kB > HugePages_Total: 0 > HugePages_Free: 0 > HugePages_Rsvd: 0 > HugePages_Surp: 0 > Hugepagesize: 1048576 kB > Hugetlb: 0 kB > > with this patch for cmdline (default_hugepagesz=1GB hugepagesz=1GB hugepages=2): > =========================== > ~ # cat /proc/meminfo |grep -i huge > AnonHugePages: 0 kB > ShmemHugePages: 0 kB > FileHugePages: 0 kB > HugePages_Total: 2 > HugePages_Free: 2 > HugePages_Rsvd: 0 > HugePages_Surp: 0 > Hugepagesize: 1048576 kB > Hugetlb: 2097152 kB > > Fixes: b78b27d02930 ("hugetlb: parallelize 1G hugetlb initialization") > Cc: Donet Tom > Cc: Gang Li > Cc: Daniel Jordan > Cc: Muchun Song > Cc: David Rientjes > Cc: Sourabh Jain > Cc: linux-mm@kvack.org > Suggested-by: Muchun Song > Reported-by: Pavithra Prakash > Signed-off-by: Ritesh Harjani (IBM) > --- > v1 -> v2: > 1. Made .size = nr_node_ids instead of only online nodes as suggested by Muchun. > > [v1]: https://lore.kernel.org/linux-mm/7e0ca1e8acd7dd5c1fe7cbb252de4eb55a8e851b.1727984881.git.ritesh.list@gmail.com > > mm/hugetlb.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/mm/hugetlb.c b/mm/hugetlb.c > index c498874a7170..4e2a1e907ec5 100644 > --- a/mm/hugetlb.c > +++ b/mm/hugetlb.c > @@ -3289,7 +3289,7 @@ static void __init gather_bootmem_prealloc(void) > .thread_fn = gather_bootmem_prealloc_parallel, > .fn_arg = NULL, > .start = 0, > - .size = num_node_state(N_MEMORY), > + .size = nr_node_ids, > .align = 1, > .min_chunk = 1, > .max_threads = num_node_state(N_MEMORY), > -- > 2.39.5 >