From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0044CC02181 for ; Mon, 20 Jan 2025 21:39:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5D2EE6B007B; Mon, 20 Jan 2025 16:39:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 582EA6B0082; Mon, 20 Jan 2025 16:39:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 449F76B0083; Mon, 20 Jan 2025 16:39:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 26F8D6B007B for ; Mon, 20 Jan 2025 16:39:40 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id CC261440E7 for ; Mon, 20 Jan 2025 21:39:39 +0000 (UTC) X-FDA: 83029147278.24.219A910 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf12.hostedemail.com (Postfix) with ESMTP id 597C440004 for ; Mon, 20 Jan 2025 21:39:37 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=dVJJJV1q; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf12.hostedemail.com: domain of luizcap@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=luizcap@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737409177; a=rsa-sha256; cv=none; b=y0lD8B/mzTf9xa/RR9fVlnEQv1P+nt0Bdf4aq7sVxyBu5ACeD6L37wdRlaXDXTsd02J7Ib hh/X53rUQFqQlzWarZqOcwsm9nKvbIu0unKuBH+vPbMrbqRE33S1mP4uaQnNarii5kQ1Nw p8TFnjHCFHYD3x17tm5bEyyitZYRDlY= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=dVJJJV1q; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf12.hostedemail.com: domain of luizcap@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=luizcap@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737409177; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cFnlT3hz8FPF5SnuPu1wyK4sRvNjSp2auh2ea/YzRsc=; b=kHRRxFnt3OahGitvLWcqwrCtOlxs7W3rv2itqHaqSs0UbfDOKsQ1H4e9GdrO19MkSmITdS +fgtsvqgEFzI3PRb5fwu/Q7SnzyF4DmCpBx+QTddMHm3XpOYYBkQxiw8BK40IvFMKLZq7o Ii3EnCCLBrdexU5PK4skasPKxw71DMI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1737409176; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cFnlT3hz8FPF5SnuPu1wyK4sRvNjSp2auh2ea/YzRsc=; b=dVJJJV1q33B/OHHADBarPMWBVgDo33DnDxFYyVtIJ/6h0tTjjIe04aAWbnEp3A0GPau1aj /tM+OPLa0h6diyK16un/5iFh/zoL5JbZuCIm36vXWkqK0LxNqnP7xmMoFeobYSCom+2gPc w7tqfU387TUTedKJm0Dwdb9nQl+kEUM= Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-36-mD9kyw0gO3OIP2hYrZi9bw-1; Mon, 20 Jan 2025 16:39:35 -0500 X-MC-Unique: mD9kyw0gO3OIP2hYrZi9bw-1 X-Mimecast-MFC-AGG-ID: mD9kyw0gO3OIP2hYrZi9bw Received: by mail-qk1-f198.google.com with SMTP id af79cd13be357-7b6f482362aso720235385a.1 for ; Mon, 20 Jan 2025 13:39:35 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737409175; x=1738013975; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=cFnlT3hz8FPF5SnuPu1wyK4sRvNjSp2auh2ea/YzRsc=; b=PH4S2Bo3DTEeTUq2RBE2gg7tDVeKVTTke2tIrjrHo+9nPWqCp7CxXEosyXImrQWTsH ldMaGzouRyBfz8Yhhs1QzAFpuVHMySJK9mZqQ15O0JHfnfyyMeejbDcPauYHw6xgtaZp I8kE63cpLLgymOpeiVvLCH8pE2z2zw9c0IvMy6fMLzfbIUfgTBs8u/00dmNCRm0aU7lA Hn+vxUhoCQgvFcMc0uW8iL130wPqQeD2gBkOQdN/AszaTzrdjF4oD4ZAkHUTzWQvTj4g tpNNYIrlyw94Ucf9ZneXP7rgGXnqkjXOBgaeTrI+Z8ggx2FgGhNwm9jI7zHeaXZtpOeB 1TlA== X-Forwarded-Encrypted: i=1; AJvYcCXR/g2DaV6P3ctwc6ocO0I/s1OlD84KUybei4DA4PsTKeNfQZlIWuDBQvdQ2osKUZL0QjeXjt95tA==@kvack.org X-Gm-Message-State: AOJu0Yx8Oyaer01dsot0vNw4xZz/CQ38KgXNP9bUf0/m0IwE+NVXmz6+ tZTwvCNWN9txy0WDbTmGRSrFF6JUZFUTe1+s0WCK7Z/MQGFTBnBvZIiKr6aBRQ/eDOOd7VshOQ4 OtoFA/IF47S+w2grHrVyuuBDc22sREIkirJsq0ndgQFr8b657 X-Gm-Gg: ASbGncviv5cdaMpISL6w1e840+YFm1RhiN9uj7r6/E5pW+JcCCyD5hbBJMLPHC0D0nF rOi93aZzqUnS1Y4az+Bc6UKNVby0rGB67fevA7X4RPSNDNLzHESaeUZJeOpOb5YtYioGtTZTsp1 z3Pxo/yjTAUZqoVpK14w2I0SMQl0xHj8L+JLbtOMPXCGDdpBuQ6ZSj+KPg4eCHuOZnxV4MTfqs/ xCV9Guu/CMvY4TbmlKqTMbxMBTB/kB2bp0WVQwHVrLoMZzoTts1+K9SdaAhfVxis+eT8Q== X-Received: by 2002:a05:620a:2b44:b0:7b6:eab3:cdd6 with SMTP id af79cd13be357-7be6327cf2bmr2419146085a.46.1737409174907; Mon, 20 Jan 2025 13:39:34 -0800 (PST) X-Google-Smtp-Source: AGHT+IGFCSqeRNwqhwPO9dhroWFAN+T91Rq9FdCQxBxnnal1RRKOT6DiuV6uXOm34uaQwjcJfdJDwg== X-Received: by 2002:a05:620a:2b44:b0:7b6:eab3:cdd6 with SMTP id af79cd13be357-7be6327cf2bmr2419143085a.46.1737409174594; Mon, 20 Jan 2025 13:39:34 -0800 (PST) Received: from [192.168.2.110] ([70.52.22.87]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7be61480e63sm485223985a.46.2025.01.20.13.39.33 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 20 Jan 2025 13:39:34 -0800 (PST) Message-ID: <99f2ab0c-1e1f-43ab-82ab-20438dab34a6@redhat.com> Date: Mon, 20 Jan 2025 16:39:23 -0500 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2] mm/hugetlb: Fix hugepage allocation for interleaved memory nodes To: "Ritesh Harjani (IBM)" , linux-mm@kvack.org Cc: Donet Tom , Gang Li , Daniel Jordan , Muchun Song , David Rientjes , Sourabh Jain , Pavithra Prakash References: From: Luiz Capitulino In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: GMhyzfJTx-I1a64ah8P9sm_ZQHVBmDy9F2V8gbwPSXU_1737409175 X-Mimecast-Originator: redhat.com Content-Language: en-US, en-CA Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 597C440004 X-Stat-Signature: 79erqfyhq9dkzjxkxxkiusb4fgs3mhma X-HE-Tag: 1737409177-282671 X-HE-Meta: U2FsdGVkX1/mG/1inMTEIH9Imfqv/6bJYI9d2aSLnNtxaG+ts4a7aoJBHlsO/wFrZ3qSUtImTIO1UD0Aco4oXsZqJtY+7x1mdIaFEH4h/7RtafGf79w/MGkIcZSzHQQPp3AvWZ3mwkYx8vQWYYA1AF2jtEoACjBwr2GS5oNhBrEtl1gBhoHnXY+9fsEcct8GWwOrjEFFzosRf3qSZU2CorlOGq7/YXMSP3HGdxSvom5DTFjUpQ8ObQ9fVnmJSd0NRGhUM7sfxggk/5EoljaMrSIopnF00XUtTiW0Fg1YI80MQjs/fB+fwHi2sTRTbRyif+a7hkqN/4FNEr+6rSeLMMnKEJAnNf436cf6BeoLCfxFWmG3uwFDajCuzpHuPhQVxJFwJJlg3E4n6A2hsq4T5UesmegBjg/coYIw6fzl0jZlFWBDbM9/eoAccEAVe9rpG0DHDFFeJVK18Iqr7/K9ZJgMpuNuR46uVdXaoFHsP4k969eNper/cY5eiRdjiun7zpRpggoi3zRkXs6V/0YPTdiKCFXiMSgzSn7dWoEabkZJ7WdFQVsc+oSVxhXTPGB38rtlI3Yrw2tM+NQDjiDD0Ri28saoKSA/vwBxXsHz37ESDZWS3AI4L4pC5tdIq01d3IEnd5tjgKoWuJewu0UzXssxSiGxCUZZOS90pqDvE1fp4U4ZqrFpl6rk22zPf8fmtvuAbvYWiCL0THmbVcLi+8DUhQnO8YqbVTOb8nhhcCQDTYD8LcxJ6ZGJlO70x7DZqeEtX1V1E9BJ89AzhjU5k2iFB7ogVhjPEezSbNp/8jjtpf5plFfgCvd8TepQK5yg9nt0oBP5z59VCBRJFohU4Uu69V17iDjuQWkGawKXyUQqN8Uo8GztvEvMAPeJUkBoL91q4DvL5PMBkn/dG6Y52scbmJCjib7Fl8fSSFP/gksPC+62R5AM2jTR/oUWyMSb96fWCCe5WLrPMCIzaMN zd7aO+H5 9dw8+Hk5jJR/Sk0PAsmH/HlY4kx4Lc60CG5162/GWei2JItTSUTZjPg8O4LnXtG6ADCoOoE0ZvtESgtYcE5lkR6Addx697aysJGFEMzBoHr1UTQcIOjc/EJKdE6Epj8nhRZdTwkprdTvQZoNgC/5UYZu23peHDm7asVrXuyEQ/zLhHkUIskcB3sQGQTtIkqipwBVY7CVVlyEOaBlHgWEzzATfs+ZTwRNmG5MO2xJdqnG0UHbJFtK6jmrGpvYjxD+I/N0aBRKNTgcK7bZtPTnDvodMvvPPUSk1AN83TqjPoz5NeT41soY5FdAXV2kvfSkEI0MvtftgucS4Nr3DGOlAsd94ikNmy1xBkwkBYHClIHCBG/LbrGPh/r7nyRXQnSYL4TNUIvfdlYl6pQlovpEsBxV3TVjByo9B9iYuy71vUEAlzfJy1IWBJdDxvnSVG2M6Tp93hm5dclWPzAVqcd0Uh9HIanDNXU0H5Tbogii6YVcwMO+uAOladxIQOFRiLzWYdM+gZoktLVrvPLVxLFqkjPGFH719O9rAl1lzEFaybhBK4BLSZmoISdGiUqhg0KupJEZddw9bjD4y9BoHNuyuGgi4+c80RH0Wedkj213Ayi92S1y4dKckeNv6u9ycs9KkVl/E2vFcOPUQLdY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2025-01-11 06:06, Ritesh Harjani (IBM) wrote: > gather_bootmem_prealloc() function assumes the start nid as 0 and size as > num_node_state(N_MEMORY). That means in case if memory attached numa nodes > are interleaved, then gather_bootmem_prealloc_parallel() will fail to scan > few of these nodes. > > Since memory attached numa nodes can be interleaved in any fashion, hence > ensure that the current code checks for all numa node ids > (.size = nr_node_ids). Let's still keep max_threads as N_MEMORY, so that > it can distributes all nr_node_ids among the these many no. threads. With this change we guarantee that we check huge_boot_pages[] for all available nodes (vs. only for num_node_state(N_MEMORY)) which looks correct to me so: Reviewed-by: Luiz Capitulino Now, although not really related to this patch, there's one detail: in gather_bootmem_prealloc_node() we call prep_and_add_bootmem_folios() even when folio_list is empty. This may cause a few calls to flush_tlb_all() down the code path when CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP=y even when huge_boot_pages[] is empty... > > e.g. qemu cmdline > ======================== > numa_cmd="-numa node,nodeid=1,memdev=mem1,cpus=2-3 -numa node,nodeid=0,cpus=0-1 -numa dist,src=0,dst=1,val=20" > mem_cmd="-object memory-backend-ram,id=mem1,size=16G" > > w/o this patch for cmdline (default_hugepagesz=1GB hugepagesz=1GB hugepages=2): > ========================== > ~ # cat /proc/meminfo |grep -i huge > AnonHugePages: 0 kB > ShmemHugePages: 0 kB > FileHugePages: 0 kB > HugePages_Total: 0 > HugePages_Free: 0 > HugePages_Rsvd: 0 > HugePages_Surp: 0 > Hugepagesize: 1048576 kB > Hugetlb: 0 kB > > with this patch for cmdline (default_hugepagesz=1GB hugepagesz=1GB hugepages=2): > =========================== > ~ # cat /proc/meminfo |grep -i huge > AnonHugePages: 0 kB > ShmemHugePages: 0 kB > FileHugePages: 0 kB > HugePages_Total: 2 > HugePages_Free: 2 > HugePages_Rsvd: 0 > HugePages_Surp: 0 > Hugepagesize: 1048576 kB > Hugetlb: 2097152 kB > > Fixes: b78b27d02930 ("hugetlb: parallelize 1G hugetlb initialization") > Cc: Donet Tom > Cc: Gang Li > Cc: Daniel Jordan > Cc: Muchun Song > Cc: David Rientjes > Cc: Sourabh Jain > Cc: linux-mm@kvack.org > Suggested-by: Muchun Song > Reported-by: Pavithra Prakash > Signed-off-by: Ritesh Harjani (IBM) > --- > v1 -> v2: > 1. Made .size = nr_node_ids instead of only online nodes as suggested by Muchun. > > [v1]: https://lore.kernel.org/linux-mm/7e0ca1e8acd7dd5c1fe7cbb252de4eb55a8e851b.1727984881.git.ritesh.list@gmail.com > > mm/hugetlb.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/mm/hugetlb.c b/mm/hugetlb.c > index c498874a7170..4e2a1e907ec5 100644 > --- a/mm/hugetlb.c > +++ b/mm/hugetlb.c > @@ -3289,7 +3289,7 @@ static void __init gather_bootmem_prealloc(void) > .thread_fn = gather_bootmem_prealloc_parallel, > .fn_arg = NULL, > .start = 0, > - .size = num_node_state(N_MEMORY), > + .size = nr_node_ids, > .align = 1, > .min_chunk = 1, > .max_threads = num_node_state(N_MEMORY), > -- > 2.39.5 > >