From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6592C3DA6E for ; Mon, 25 Dec 2023 05:21:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DC86A6B006E; Mon, 25 Dec 2023 00:21:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D774E6B0071; Mon, 25 Dec 2023 00:21:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C3EBC6B0072; Mon, 25 Dec 2023 00:21:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id B31F96B006E for ; Mon, 25 Dec 2023 00:21:07 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 779C180308 for ; Mon, 25 Dec 2023 05:21:07 +0000 (UTC) X-FDA: 81604191774.06.B9A30CC Received: from mail-il1-f182.google.com (mail-il1-f182.google.com [209.85.166.182]) by imf17.hostedemail.com (Postfix) with ESMTP id B3BAF40002 for ; Mon, 25 Dec 2023 05:21:05 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=DbrCgZ5+; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf17.hostedemail.com: domain of rientjes@google.com designates 209.85.166.182 as permitted sender) smtp.mailfrom=rientjes@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1703481665; a=rsa-sha256; cv=none; b=dhwgL2yza8MRpLQxbD8ltAbibQNJ+HSspQeEHGF+3/kezJr4yu53en8+Ix8P3lf2PuM7Fz IYgV9OT8+1OEUBoYjTa3swf2ONwFCxUvsZ6CKYLSEqwEk5CtHs2yQ3H/pVE3vMWy0MQrHI wf4f69CUCK8BELqLVMLutaB+WXM4d38= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=DbrCgZ5+; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf17.hostedemail.com: domain of rientjes@google.com designates 209.85.166.182 as permitted sender) smtp.mailfrom=rientjes@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1703481665; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JTMbzj7ozG/NFL0qtSoENzSfcq7X8aUmLe01ANOMZhU=; b=VJENgJlw22nwY7/o+5q65xSsadZyIGidxZ6+o1rVzZF/RBrTn/EeMELfqdO0GgYyi+8WUc gUJu9V/G4RNy294LgqTlIzRfEy+EYwsZb8AThzkHKFfpjQBvxU371UHns2YGLgSRpbu68+ 47zlv3u8sX6er5ICPliTxCORW5ZJ6p8= Received: by mail-il1-f182.google.com with SMTP id e9e14a558f8ab-35ffc781b3bso106225ab.1 for ; Sun, 24 Dec 2023 21:21:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1703481665; x=1704086465; darn=kvack.org; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=JTMbzj7ozG/NFL0qtSoENzSfcq7X8aUmLe01ANOMZhU=; b=DbrCgZ5+o2nwrxq/s73wzq0np83jDU3wfZwrGWbMRotnktBZdp3alGRZlRd6h9bCSP 4feVab3BaN+vg+OSnxLz+bZ7w/+Wqh6SusZsKKzr6ZA5c8GfZTmOCbwvUqRWLP0q1Nbf sg4/U2oCsZZq1OrlNV58rkFfKHgUY/dhsMMeZ42ibSC58JvRDKAdqFHQfuNLVI+v7K1V hcDgrGNfmyn9S5kbi3YnAAPcfMMEBtLv+iCk06zbm5cNHBlxCEkgTusH/8QWKYWsySJ4 8xu1ibdfGZ/T15dhLWMNg2Nuc7eNCk65nTqBP8+Z9Nb5AycLaV2q/wQBXn2QB0dfxhou z/0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703481665; x=1704086465; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=JTMbzj7ozG/NFL0qtSoENzSfcq7X8aUmLe01ANOMZhU=; b=Nr3yG8UDDkXLM2y4ugKxqBLvNMeMZayF9fk8M6jcuk+BogaGJkw+xjUaTmYzh7DRFf qVuu5LtH68BIr/CS7J1+WDUAkjL/SRWYQ5cGTLePhUn4pUPV+o3YpvqQ36XZJ/ckkImg 1OjlnaHJyvrpBMjSv0t0YMztltZTpM6AdMfna7o9/7u3utFCK+pEE/DD++xkngjykRJX 53jyht8AwNDna5t1M3S+zbqSRCwv/eBBPPhZHXq+bULxC/no3YLG5GJM3ZRf10PIxRxA I7ckSmGPKIsdTI+6d9ndvdm47ILHrQ6AhufFKFbnhvo0nXciPNO8ytUQWqd9VxoqPIIQ MsUw== X-Gm-Message-State: AOJu0YydRMSyb3iK1k8zhowuAkYha8hG2G8soCrQiN1sq1yazc5UerpN cCvhUgYf8PnLC1BNakP2pGPvYtvxg7Fa X-Google-Smtp-Source: AGHT+IGSKPMU74vf1sgsIw8ea8H5OTZUG/+a6Dc87BWf2D/h73XmwrsQjqU62nU/eoayj69tQUV2DA== X-Received: by 2002:a05:6e02:d0c:b0:35f:79b5:b980 with SMTP id g12-20020a056e020d0c00b0035f79b5b980mr370882ilj.27.1703481664705; Sun, 24 Dec 2023 21:21:04 -0800 (PST) Received: from [2620:0:1008:15:1457:193a:aad7:2606] ([2620:0:1008:15:1457:193a:aad7:2606]) by smtp.gmail.com with ESMTPSA id n12-20020a17090ac68c00b0028ae54d988esm7634768pjt.48.2023.12.24.21.21.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 24 Dec 2023 21:21:03 -0800 (PST) Date: Sun, 24 Dec 2023 21:21:03 -0800 (PST) From: David Rientjes To: Gang Li cc: Mike Kravetz , Gang Li , David Hildenbrand , Muchun Song , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [RFC PATCH v2 0/5] hugetlb: parallelize hugetlb page init on boot In-Reply-To: <4c6de257-ebb4-e9ad-4092-b81a8039aff4@google.com> Message-ID: <76becfc1-e609-e3e8-2966-4053143170b6@google.com> References: <20231208025240.4744-1-gang.li@linux.dev> <996ba32c-78f0-1807-5e64-af5841a820e7@google.com> <20231212230813.GB7043@monkey> <55c6c1f6-0792-61c3-86ed-4729d4a3fdf5@google.com> <46bc7aa3-4b08-4e5f-9563-485ee17e2785@bytedance.com> <4c6de257-ebb4-e9ad-4092-b81a8039aff4@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: B3BAF40002 X-Stat-Signature: x7caq3xf4u57phmn9cfoe6mnhubm4do3 X-HE-Tag: 1703481665-86765 X-HE-Meta: U2FsdGVkX1+BvOKdLsmy2I66giAQyz1V0LXSq39tvE7Wsvv6Y0IR8/TmttZXa0gqs9CiSgSVwqAo5a1s95SOAsdTdxyXd95eKhXvDkD4X16Q8SURxmhv7/BhG2IE55w1n0/kBCBgd74zTD5lHB4cPqp1bO8IMoeA46wnKO8UjkpjUQdFTtL9PhcXWgr0QmA5EmvDgDgBNIOW6se4sdZPEo4LJadmI9WqDJaPMdf8d+PZFfZUm8yQtXoaaHlN8JdvnMvS1L/JMCYiRI/9IcL2DsXI+O5LinR+lsa12aJ0cjPJ/ifRsu2BQR4kDAm/GLOwZ7C1WeOagEsRSKxKsCAjRax8gopOoV/iIgmpxvwXSi9cFLHXZy3L4L+XzIrCYSr0dsEtnzJf48mvP2qgJimpxSiElyMNrDyWK9TpaJJ/WzEOT6PXKlQy7yKIFrSlUDru0Duv/xSxzavjRQ4AXdRKnI3V4wXu7cml+N7Ax6hVHK8aW/rXsemDLdBXja6IlvcFyHf6TuYQud+BrDB8sInFTDf9Jr76Yv312fX8ihqMO/7UWYTROrV5N0O1bmnCasEalEjJONyNomaO9JqzAh/yd81BuOQ0K4yVWzFWIaZxVq1RgbczHjUjrDV7WypBPGcEfyxBxoeDdGV3j1RiKs6L5fsTQoDtJ4L+lqQtfv8f+4bE0MmBoPVdpFHUprJSRE1UZHX/lL2pCgBSS0pTt9WFtyshR3KWQR54zAiggiBYwxBL4/R+R8mu61hqaggfg7MrvXi89ryuWMfUZwmdNQKrsdMgPcjEpjMsna/c0EPXsb1p1aRPFT3Dmza7SVSqf8uyD1JdojZ398f5Iz41piJhiFhyJ+XktnYZ3hguNtksNF+i4fpVX+QKvQABG+jAGG3CbfJ++PX/h6kTM4xbdYgw2ZMb/xxnTgPkV8+OzfowTRtzLCKlMkcg9FAZPk0NxaeBIjcYSnHgE89cfYPcJpl sAHYLSkQ K6mD57wibFRXjkySOCHlkRpOYBw392YU84Hzk0jJACS4hObfdaCprBzIBjiMb+DNV+rG9mkMx88KQF9p/R2yJ559gV93gUDWE40+7LzpaXd4qWqS7RZvpbwONR6dX/g2povzGOW6f7ddXvJNsBxZr1uOdEXk8rSkhC0q1Y4lN3sZl9drneyMJtwiJMmtiSxPDlkGoXWxna2HCcg4gSJpxvHFr7YpKuEU68D+nN8mmtaFrJ9NW/W4bE0dVcaDl1B1kjZQdeJlILI1Iupwrd8v8EKvfczmj7B7r7lObxFKPDeMyErRoauNslgRPwzGjYw9B1NKoA3UX4zLP6jW1HG63OByxvcPIDGU4oy0pUa1jk+hOPBCjHQ7kvCoEEEmLzC2+mqGyyucLN7CxqKS9i4IsHX0uwA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, 21 Dec 2023, David Rientjes wrote: > > Hi, > > > > On 2023/12/13 08:10, David Rientjes wrote: > > > On 6.6 I measured "hugepagesz=1G hugepages=11776" on as 12TB host to be > > > 77s this time around. > > > > Thanks for your test! Is this the total kernel boot time, or just the > > hugetlb initialization time? > > > > Ah, sorry for not being specific. It's just the hugetlb preallocation of > 11776 1GB hugetlb pages, total boot takes a few more minutes. > I had to apply this to get the patch series to compile on 6.7-rc7: diff --git a/kernel/padata.c b/kernel/padata.c --- a/kernel/padata.c +++ b/kernel/padata.c @@ -485,7 +485,7 @@ void __init padata_do_multithreaded(struct padata_mt_job *job) struct padata_work my_work, *pw; struct padata_mt_job_state ps; LIST_HEAD(works); - int nworks, nid; + int nworks, nid = 0; if (job->size == 0) return; diff --git a/mm/hugetlb.c b/mm/hugetlb.c --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3300,7 +3300,7 @@ int alloc_bootmem_huge_page(struct hstate *h, int nid) int __alloc_bootmem_huge_page(struct hstate *h, int nid) { struct huge_bootmem_page *m = NULL; /* initialize for clang */ - int nr_nodes, node; + int nr_nodes, node = NUMA_NO_NODE; /* do node specific alloc */ if (nid != NUMA_NO_NODE) { With that, I compared "hugepagesz=1G hugepages=11776" before and after on a 12TB host with eight NUMA nodes. Compared to 77s of total initialization time before, with this series I measured 18.3s. Feel free to add this into the changelog once the initialization issues are fixed up and I'm happy to ack it. Thanks!