From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9019BC19F32 for ; Thu, 27 Feb 2025 18:03:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F0234280002; Thu, 27 Feb 2025 13:03:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EB3D5280001; Thu, 27 Feb 2025 13:03:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DADF9280002; Thu, 27 Feb 2025 13:03:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id C0737280001 for ; Thu, 27 Feb 2025 13:03:19 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 7D97F121A96 for ; Thu, 27 Feb 2025 18:03:19 +0000 (UTC) X-FDA: 83166496518.25.1D00CA8 Received: from mail-qt1-f180.google.com (mail-qt1-f180.google.com [209.85.160.180]) by imf30.hostedemail.com (Postfix) with ESMTP id 75EC880008 for ; Thu, 27 Feb 2025 18:03:17 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="KLvg/OGp"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf30.hostedemail.com: domain of fvdl@google.com designates 209.85.160.180 as permitted sender) smtp.mailfrom=fvdl@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740679397; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=r36xY43iw45fgW7xbcfzK7x9eAnBCMHKjbDFGDY1hmA=; b=lTCm+hI7rcIEJO8HXpvU+IEyT7zqRjkI5EkAl5DHE3iR/ikPyc9e4LXo6250/EF4l8KC0C 1XL2zQGPKNjq9ekAU/fTTotHohx/BPwuHSAc+yEElq5LEaPEVfFU630tbX+sc/0+v5tjbG OJ+iwpJFgoY+23ky7nkRS2BhNw/o4Kw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740679397; a=rsa-sha256; cv=none; b=Qpkv2LuCpMeYWu62Bq4t39FMuaMXiozl6gyCPMgy3ARqw2u2zQhgvR0KI2CtCDNuXmwtTW GWmC0VECo/2RIU6xa/XwGei9NCOSR7QZR++iUjEHsoUCbcCdXFhBy5L6zTFpT9pi58CO5h ylQCstCKX5Th/vkoEh1DFxzPdNVapH0= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="KLvg/OGp"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf30.hostedemail.com: domain of fvdl@google.com designates 209.85.160.180 as permitted sender) smtp.mailfrom=fvdl@google.com Received: by mail-qt1-f180.google.com with SMTP id d75a77b69052e-471f1dd5b80so5451cf.1 for ; Thu, 27 Feb 2025 10:03:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1740679396; x=1741284196; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=r36xY43iw45fgW7xbcfzK7x9eAnBCMHKjbDFGDY1hmA=; b=KLvg/OGpfpGY1Q/llCJWC01mJsjdVL2yZ7fYlklT7AKdYs+IUv3sk9wtO91yZlOWES KIKB4dMRzOQDGk9zp7UOxTkDEP5zyJpi0IMygi/9dJume+YgQKwV/Sdhw2pM0dANU1pz oVoF2T8Q+5g1nWKqfEAS+Fm05dbszyFpXawgFK9mImRCeEhOFkY9Bo9D8+edGdO1Hfhl dED2QtK+xDtQKT+cS5bdWIFNXbgP6eAYO2eGmkl8LImcMMy8/8r5YYXvV1cXepjI25sm zInmOnirik/nQ0jnBlWzAftVY/2XvrfoYn4LOkzxiNAR6hN4U8itMchuBPERsimFIQnk E6vQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740679396; x=1741284196; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=r36xY43iw45fgW7xbcfzK7x9eAnBCMHKjbDFGDY1hmA=; b=ItjHrTU2Sex2TI+6SZ3zcEFJHlWqJkt9OPnE8utnVsUA0uVzAFZejFI5PwwbnWck8q 7s6uK2rHCSTparttojkQnNzi9eeGlHxG0t6YhmMsWny02gf4NKWJBuGN4mjzunN9s5Oz hdgiOVT4saEAIBY4CvQ68ufMZYId4HuMRQzDv3p8WZLS291vGDgp94U0bmGGUi1HNy0X 8NLBfb7FzPoxpnKNLcfMGHEShNUUYNVJCpmv98c2KiS4UvD3fTJgP+vrKsUoJS6jfHO1 6OmPlBiyHIRKcCqs5xuq8w4yQsJQe9dbNIzZxQDHRoaATzyqxPu2iEJHY6fM/iYsVw4D 2Z5A== X-Forwarded-Encrypted: i=1; AJvYcCUNslrMr9XVTWCCvPPqvC9vdoy5Kn9r/TwzqGaqzFbM+z0nGajIsz7PyQ87e6xZlSIiV1KPb3zzZw==@kvack.org X-Gm-Message-State: AOJu0Yz5MV7FzuV+5+xBKg1jfNR1pO07YFKT2lhpRJARKarBTmi5yDA0 E3Lj7n8vFVkaauIZTT+qq7tViGZlvPQ1Hn2YF3gjl8kbGl1GG3e/fX5Rz+ts/30ern60GyfseRf U8iV/JzzzAhY82OpIxsFTngEPt+dUpIs3aEbx X-Gm-Gg: ASbGncsrYVjGXyUuBnaNplZfgY8vk/FS/wdVMMBIW2FYUEwbEMEEBeTAtFacxHMOHvx V9X9eFkifRkFvy8Zh3v1wQt5DlXzMAz4+yXrbBlUTZYSMT8XO2hYq6XFeEh2jRJ4q2M11Ly07mL 7o29Qp X-Google-Smtp-Source: AGHT+IH1WMMA/3uqqJEGl4Gx6/wQ5va9s9OLY0x0xxcXVM4AeIevuwH7yEXb+VEN8j9a5QKiDZanXiTCGVNjwIjTvhQ= X-Received: by 2002:ac8:7f02:0:b0:471:f8af:3231 with SMTP id d75a77b69052e-474bbe36183mr70881cf.19.1740679396201; Thu, 27 Feb 2025 10:03:16 -0800 (PST) MIME-Version: 1.0 References: <20250218181656.207178-1-fvdl@google.com> <20250218181656.207178-11-fvdl@google.com> <20250226180900.GA1042@cmpxchg.org> <20250227172014.GB115948@cmpxchg.org> In-Reply-To: From: Frank van der Linden Date: Thu, 27 Feb 2025 10:03:04 -0800 X-Gm-Features: AQ5f1JrPBa0uNQITihglYJKMGmza9l9OemfEMOfYjelco8ii8Fc1hRBhjks-3Ck Message-ID: Subject: Re: [PATCH v4 10/27] mm/sparse: allow for alternate vmemmap section init at boot To: Johannes Weiner Cc: akpm@linux-foundation.org, muchun.song@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, yuzhao@google.com, usamaarif642@gmail.com, joao.m.martins@oracle.com, roman.gushchin@linux.dev Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 75EC880008 X-Stat-Signature: kwanwerfnuy4wnz6e8cayffkjp9p5q74 X-HE-Tag: 1740679397-167223 X-HE-Meta: U2FsdGVkX19jt4E9BJYIdnOSCISRSf4BvqQLs0o/rltdy4447YaOh2cu/p6n13v3Gq0662QlLn/Oq2oS6bXJPkEzny0TnfJU0zUe1mxFha2EulgfvIeuge1zWgeAKepCgzkXQJkVHdf1I2JqdWojmtvObuCl8sYE07OlgtTOLlsaKSCPr6ucLkKh5wtd0Ae9Y7nInVrNgEJ+BWGqyQeSn+5+kAdsYfBZU1U32w0e/Rt951I3LNP6zyMXaDUvWclI9wEVoWV9yjIywWoVudhD/hSz5HN9PV0yVnqZtfU0oroUQFuuCh/n+1NnKbDs+NDB94TV7/HN0ftg1MvIjQzxC0Zy+Zaav7iDt2iGvUxF1m7hAzZvGPHhN7UxW1tt6H7HF6r6GSAv2YQEObUh4kvT4iJs40Ay/2vNayjU8runL3acT27nDQ7m/iPzmiEfBUk3bJD/DS5D6ZcEKaKKr6vOo4ATA0dfecXkEmFn6UPpoKLeHLqa1d7VHXtm41LWxfiqpvIrX3PfFXtPYMOJAhvuVL9c7+yIiD8T/WVU6CNB9990NeftvEI56a92Wu++erLw9vHY/C1cf3069EYGd00Hld19YprPg+4St6M6x6n8sJqcBOwsZ3H2f12ywLdzf/wOYpC0mK9DuiCcGE+FnDSsWh9hw/YHelKvEZPWWtpsQYSG8Gj/AYTyRnX3VmZKQuy6is3/JYoLKjYJHItGQzetsEO7CEZEoXgoSJ579NZzf4J0e/y+4TiaJKNdnX4DL/VwE2K+uZ++8SC/JkANBEV2u80BI2zHbkpUXs2UkKGiQ1Lh8iRaebP0L98d14Bx8C8MvHCigrhDWToSr+rUUan/MJJuY/bCN+EQEPPZ4tenu5/i+YUDvxDM3qk0U+TytxWfdCCt5PyltU4wOLF9Bw06xw8HyQvIaW9qo+VrlTD2w0/MJ8J/FGkGac4CEtqIA7u16x3y/1C8rSuPIPLOX3c Hi/Puuw9 tYdzZBeER8CThU6Hth/iGp9O+XChc6JfiNfHeDXMH6kES93RUXEQiEmr0fKnZAbRabTIrtyEGO/melU7iOB5bO9yLvK8/4kyfOGPOOU6L2XbXaHCLgtPcgI2DWp04ShPnnuvGehmb9GO3shuPKd1Ou+ATiN/Sh7eBi2CjsYJ8LPs6jtkl/7tJIO2ziGR1Igc6Cz1nfAJ5iZuow2++D5N2UU/z2pHtlGCb1kaJQOm5kfGjV0c5+0fitpIXqw96Ljgg7hXoMWmusfNoVDVw94kdny5BRw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Feb 27, 2025 at 9:56=E2=80=AFAM Frank van der Linden wrote: > > On Thu, Feb 27, 2025 at 9:32=E2=80=AFAM Frank van der Linden wrote: > > > > On Thu, Feb 27, 2025 at 9:20=E2=80=AFAM Johannes Weiner wrote: > > > > > > On Thu, Feb 27, 2025 at 08:47:18AM -0800, Frank van der Linden wrote: > > > > On Wed, Feb 26, 2025 at 10:09=E2=80=AFAM Johannes Weiner wrote: > > > > > > > > > > On Tue, Feb 18, 2025 at 06:16:38PM +0000, Frank van der Linden wr= ote: > > > > > > @@ -489,6 +489,14 @@ config SPARSEMEM_VMEMMAP > > > > > > SPARSEMEM_VMEMMAP uses a virtually mapped memmap to opt= imise > > > > > > pfn_to_page and page_to_pfn operations. This is the mo= st > > > > > > efficient option when sufficient kernel resources are a= vailable. > > > > > > + > > > > > > +config ARCH_WANT_SPARSEMEM_VMEMMAP_PREINIT > > > > > > + bool > > > > > > + > > > > > > +config SPARSEMEM_VMEMMAP_PREINIT > > > > > > + bool "Early init of sparse memory virtual memmap" > > > > > > + depends on SPARSEMEM_VMEMMAP && ARCH_WANT_SPARSEMEM_VMEMM= AP_PREINIT > > > > > > + default y > > > > > > > > > > oldconfig just prompted me on this, but it's not clear to me what= it > > > > > does. Not even after skimming the changelog of the patch to be ho= nest. > > > > > > > > > > Can you please add a help text that explains the user-visible eff= ects > > > > > of the toggle, as well as guidance as to who might care to change= it? > > > > > > > > Hi Johannes, > > > > > > > > Thanks for your comment. How's this: > > > > > > Thanks for the quick reply! > > > > > > > Enables subsystems to pre-initialize memmap in their own way, > > > > allowing for memory savings during boot. The HugeTLB code uses > > > > this to initialize memmap for bootmem allocated gigantic hugepages > > > > in a way that is done by HUGETLB_PAGE_OPTIMIZE_VMEMMAP. This > > > > means saving this memory right away, instead of allocating it > > > > first and then freeing it later. Not allocating these pages > > > > at all during boot allows for specifying a bigger number of > > > > hugepages on the kernel commandline on larger systems. > > > > > > That makes sense. > > > > > > But if it's infra code for a hugetlb feature, it should either be > > > something that HUGETLB_PAGE_OPTIMIZE_VMEMMAP pulls in automatically, > > > or at least be a hugetlb-specific option that pulls it in. > > > > > > Keep in mind that not everybody enables HUGETLBFS. In fact, hugetlb i= s > > > default N. It's moot to ask users whether they want to enable infra > > > code for a feature they aren't using, and default to Y no less. You'r= e > > > regressing innocent bystanders doing this. > > > > The main reason that I added a separate config was: > > > > 1) I could see other subsystems use this. > > 2) The number of section flags is limited, so I wanted to put the one > > I added inside an option instead of always using it. > > > > If especially 2) is not a concern or can be solved differently, I'll > > be happy to remove the option. I don't particularly like having it, > > but I didn't see a better way. > > > > Let me think of a way to clean this up a little, and suggestions are > > welcome, of course. > > > > - Frank > > I'll just do: > > diff --git a/fs/Kconfig b/fs/Kconfig > index 64d420e3c475..fb9831927a08 100644 > --- a/fs/Kconfig > +++ b/fs/Kconfig > @@ -286,6 +286,7 @@ config HUGETLB_PAGE_OPTIMIZE_VMEMMAP > def_bool HUGETLB_PAGE > depends on ARCH_WANT_OPTIMIZE_HUGETLB_VMEMMAP > depends on SPARSEMEM_VMEMMAP > + select SPARSEMEM_VMEMMAP_PREINIT > > config HUGETLB_PMD_PAGE_TABLE_SHARING > def_bool HUGETLB_PAGE > diff --git a/mm/Kconfig b/mm/Kconfig > index f984dd928ce7..44b52f8e5296 100644 > --- a/mm/Kconfig > +++ b/mm/Kconfig > @@ -496,7 +496,6 @@ config ARCH_WANT_SPARSEMEM_VMEMMAP_PREINIT > config SPARSEMEM_VMEMMAP_PREINIT > bool "Early init of sparse memory virtual memmap" > depends on SPARSEMEM_VMEMMAP && ARCH_WANT_SPARSEMEM_VMEMMAP_PREIN= IT > - default y > > Does that seem ok? I'll send an mm-unstable follow-up patch. > Wait, that's actually not correct. Anyway, I'll stop spamming - I'll do it along these lines but properly, and will send a follow-up patch. - Frank