From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 091FBC46CA3 for ; Wed, 29 Nov 2023 19:41:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9C2986B0251; Wed, 29 Nov 2023 14:41:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 973446B03AF; Wed, 29 Nov 2023 14:41:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 83B236B03B1; Wed, 29 Nov 2023 14:41:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 6F7E86B0251 for ; Wed, 29 Nov 2023 14:41:57 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 4125BA05FB for ; Wed, 29 Nov 2023 19:41:57 +0000 (UTC) X-FDA: 81512012274.28.64527CD Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) by imf28.hostedemail.com (Postfix) with ESMTP id 8166FC0009 for ; Wed, 29 Nov 2023 19:41:55 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=4CbuPmlH; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf28.hostedemail.com: domain of rientjes@google.com designates 209.85.214.181 as permitted sender) smtp.mailfrom=rientjes@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701286915; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=De6xxx8/OcpsczWD4sU3Fi2NLfEqw+uaoC66p0XYL50=; b=jT9ydLNmbZKQVCln1gCdTqsqATuD11oWIwNF3Ja0ViRQfA8Q/fa7kaOtvVaerFpUldEuz9 eHq9KNisIsC0gMwDntf9LPnfJs5f/DSYiU/1HszOgylHEYjCt6UiYJPzUMOcERyynAQRCC d9opydhjebSzlVsxDI45OaKvzs5f0yk= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=4CbuPmlH; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf28.hostedemail.com: domain of rientjes@google.com designates 209.85.214.181 as permitted sender) smtp.mailfrom=rientjes@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701286915; a=rsa-sha256; cv=none; b=ZW+CW3OxxIjBeA5ind6fkOP810nY6gHjN37NUOv6qGwvbzhEtyd3MI5SzFyjHSNVHj56mO MIo2FqtwLPr4umVAGKDqDHPkDGIt87bxAGeNuY4QpbVdLcEAjHbISVNdkxpUpanXnuyxpg Ysdtd7q5DDcuWBsbwyAtVhUF1aAPtY0= Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-1cfaf05db73so25605ad.0 for ; Wed, 29 Nov 2023 11:41:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1701286914; x=1701891714; darn=kvack.org; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=De6xxx8/OcpsczWD4sU3Fi2NLfEqw+uaoC66p0XYL50=; b=4CbuPmlHV9OChs99TM1RK/2ZSPmzcfbmOlDTctQhGeLLFUf2gH/xFsm1lqq30hN8jJ vj1CesGWnkgNpgu63Tez2vkUXLkK8HSojSdErRrsIdMAwKALB0h6voU3DyV6GuS34TDZ eMCi5jD3Napb/zUXOOkMwXA++60RXjIhyWjOVU9bc/WkCDdYaKmuYLwdrojXG+1d7FFp 2o//+NzRGRznzOkfA7MSjZWH2x9wW8Tmc2tMqjbzGxYeh92RsuGnz3lp0h62v8OS1176 zq0RrGWtLtwM/PgeFOTrR0GYPjJspKydnXIymh8M8XmgSoYO7m61GXwtJtz7IN7Mw66u 0v/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701286914; x=1701891714; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=De6xxx8/OcpsczWD4sU3Fi2NLfEqw+uaoC66p0XYL50=; b=WrCUpISU5+tGVUsVL4n5RLQE1q9NqFPQ7YEvqScHZBO2aYibTzZewFhwIe4BTlCYQk M/6CcIAiz8hOTjfz1BrqvlH2MJDwj9nVE1YqDB7W8oCG6BinEMoXnbRRwJbgUH01sj+8 UpFzARKz+5Ep42ef+Jaw4EgSLk9Fu5iUuym8idEHXInwjqUtZWbJ0vMatMi9gY7TPT0P Z0G0CwduqWpYSAGGDrV9Ru7Qzt63gfKc849USE7wzg/k9RpLs0yE5eoGQUSDquoCyuNK 1Jp+Iy5m3f/wBOQI9x2YTN32p5aUFv3y0oRzfy+MU+RokKoyPhjrKq3e5z1n2jwugHds krCw== X-Gm-Message-State: AOJu0YzhjYEfkVQVKLA09AvqhUkaRsINqfTFBu2yOiGzb3l7pp8Srizk mzFWgMkrK3AqsbIAHB+v5iQ3/A== X-Google-Smtp-Source: AGHT+IE/rLfR934Pc/7oeHzX02TL1UhWj/9mOZYOmi6cOCUWiWcKKKIu0Qv2b8dkem4+uVwTQVDufQ== X-Received: by 2002:a17:903:32c1:b0:1cf:d2c5:af12 with SMTP id i1-20020a17090332c100b001cfd2c5af12mr24857plr.12.1701286914105; Wed, 29 Nov 2023 11:41:54 -0800 (PST) Received: from [2620:0:1008:15:9ddf:fd79:28e8:6290] ([2620:0:1008:15:9ddf:fd79:28e8:6290]) by smtp.gmail.com with ESMTPSA id w8-20020aa78588000000b006c4d47a7668sm11081724pfn.127.2023.11.29.11.41.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 Nov 2023 11:41:53 -0800 (PST) Date: Wed, 29 Nov 2023 11:41:52 -0800 (PST) From: David Rientjes To: Gang Li cc: David Hildenbrand , Gang Li , Mike Kravetz , Muchun Song , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [RFC PATCH v1 0/4] hugetlb: parallelize hugetlb page allocation on boot In-Reply-To: Message-ID: <965d3ea8-4dd9-8f66-7ac0-0d6aa7fcadc7@google.com> References: <20231123133036.68540-1-gang.li@linux.dev> <5c0e27f2-5826-4537-a1ab-1debfab65b9a@redhat.com> <28e28c2a-e72d-a181-e87a-39cecc8c3c76@google.com> <9d5de524-28ee-4d71-9493-f77967ea213c@redhat.com> <783f8bac-55b8-5b95-eb6a-11a583675000@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 8166FC0009 X-Stat-Signature: effocy6bp6uobo4wzrhsgzgfgoz43cax X-Rspam-User: X-HE-Tag: 1701286915-449173 X-HE-Meta: U2FsdGVkX19O636Yhni9n8CIdaMD5wAOmnj4F2XoRXhM7BMKw/BHQ36g9YlxKggrEFXy2i1UALaa7rNOM7qmInLboOMdS575eU9tFgjj8yqmLYeGmAUbGBH7s6bzuLXLWdjTnUyVv3YsspHO+fR0cFdM8l1LN0sbE44ALt4G0YrXmWN0PLcZgP38RjLmHaL9WB07DmiSj3ExeJNEqE8J32fYNqBh6mN0RPl3B4Ok1jXndHqo1zJQLmMG02ejB22B/E8rkTlPaox4B/+WRlde8l0udGuTdX7Lkevjez/w8x6IGZkHEx/OJ7L8jh9Xg1gF3OwYOPb45Ub3AbKWLITZ2JHma7TnaNqUpsqqIyTE3b0fZWsxlk2QkGpQbBtOs+CkF006ut6h0nC7HRXp7YrPOcKLYQbM3fRM0Y2euGDzeFJlCqeDX8/UdXRjiohkgeRGYj2yGIRJcCXsSAWx84Yx/ALs7CQYpNPRR3Ujo3oCE8rlkAEbuMDWHnyjEeoY0kn/LHSDCFp1+XLGnHEDZpReouryveqg4Z9dwNiICVkWVgcqlbq+2oia3P1I6slhRccehpXYvwjsqmVZgX4j5K4iu0bhap9JGjGVQ4Xbyqffvd+anYrMErRNiXSKkiXOqJc1t7x9tfPHHmR1SiM5qE83MTCu0ZkHRljGYzHPOx4LvGmccuxjBVPKsv6btZZqHGaNYooMUeMK8EjxwocHiW1Ec/1k0MBUwuSy4z9zBULkyTfeisNYKDUDPFUhI3OtD8h8g8h8ti0XvTyriRpxgGdesFTojBav5uClqeU2yWczRJwd5L77HwiAY7Oh0mXk6rzEbio4Z1AonMo10O13gG+2r4gti/+iu1FnfD9cdZRK1MM3EbevVgBsAV3kPCM2PJj/6avWeQFak19pvD1z69SEy7ddExdIphxmt5zP8o19sMpaF9SyR++4iSFwS4zhQ70rSHLGmNyd8905lc3GAoJ aaZyYMlF ucQFESM+DuqRugJH7OMHoSl0GmUCjrOKm+0C0NO5mMK63l3l/y8DKvYHEk+BK3Zjwsy253BEQ+7lblZSyEx5LbCbHK09YyHiFWxUaDfFkaI+m8L8N16zg7PtAODrTOa+owIIyvv1hk08Xx8GvKVPgCWyA8jCl3sYNd245GVIheYEljZqV2yQvQzEDm0z12O6V6Qj2cRkYWkbjOGyEMYa6K/V5j53XGVsWbI1nwwLY0h+uAfU5ZTLSQt9uRMfa4Usk8KPkzS7gwoIcrKVbhDex39x+rOLkvyozyiJ7pY21ZYNnRHvCcJwVgDWdAZo7AeitIUXlrCgkl/4tj1m7PrJi4gbo+EOMfUcAvjUUHf7i4TDKsHc0ha39kYfUMnizjhhv6t+vPsdZTneMdbpzR2P5NoFP+VAavKg8KAryNKCsp1sdjDM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000412, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, 28 Nov 2023, Gang Li wrote: > > > > > And there, the 65.2s won't be noise because that 12TB system is up by a > > > snap > > > of a finger? :) > > > > > > > In this single boot test, total boot time was 373.78s, so 1GB hugetlb > > allocation is 17.4% of that. > > Thank you for sharing these data. Currently, I don't have access to a machine > of such large capacity, so the benefits in my tests are not as pronounced. > > I believe testing on a system of this scale would yield significant benefits. > > > > > Would love to see what the numbers would look like if 1GB pages were > > supported. > > > > Support for 1GB hugetlb is not yet perfect, so it wasn't included in v1. But > I'm happy to refine and introduce 1GB hugetlb support in future versions. > That would be very appreciated, thank you! I'm happy to test and collect data for any proposed patch series on 12TB systems booted with a lot of 1GB hugetlb pages on the kernel command line.