From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E99C4C19F32 for ; Thu, 27 Feb 2025 19:36:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7813F6B0083; Thu, 27 Feb 2025 14:36:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 730F56B0085; Thu, 27 Feb 2025 14:36:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5F7AD280001; Thu, 27 Feb 2025 14:36:36 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 423E26B0083 for ; Thu, 27 Feb 2025 14:36:36 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id D5A2FA40F3 for ; Thu, 27 Feb 2025 19:36:35 +0000 (UTC) X-FDA: 83166731550.27.014514A Received: from mail-qk1-f179.google.com (mail-qk1-f179.google.com [209.85.222.179]) by imf07.hostedemail.com (Postfix) with ESMTP id 5E7D940006 for ; Thu, 27 Feb 2025 19:36:33 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=HweOa4Wd; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf07.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.179 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740684993; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9qQGmvDxGhjhQGrQCPpUIPeeAlhy/ylNJXmNHALKC5Q=; b=rsxlf3WujXKjcW1G9C3iCZ/t+7bxyeoT8Af7wyuJ8ssh1MmC1ZYxhgjpYHMnp6jVLTkjnC YFNsNbI6o8YtwnAiJUSr8BSPNY33jMWAXMyCqJiFKKFP6H/EG2l4pX+VcFXXqHU000iDDv hHjKZx6s3E6iMY5rDFzDH1MGHU2LDiA= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=HweOa4Wd; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf07.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.179 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740684993; a=rsa-sha256; cv=none; b=mQHndXXI4jGHj+konpsJyoWkiRa/H1PCqJJfxPYF1u4D79aQfdNkxjGtnFwRaBjsJZvBEp 2DpSQC0Z8MEzm5Nf1hNXci7FP8X3cEGnknBDG4FutO5hAtD3gFOyB4gDYHKzVqZ2mPauSo 5jbM1CVqrkkRa6OvOdd5yIpG6JuYRN8= Received: by mail-qk1-f179.google.com with SMTP id af79cd13be357-7c0ba89dda9so139114885a.0 for ; Thu, 27 Feb 2025 11:36:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1740684992; x=1741289792; darn=kvack.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=9qQGmvDxGhjhQGrQCPpUIPeeAlhy/ylNJXmNHALKC5Q=; b=HweOa4WdINkWxi4msLJ92JJ+010yt9Vz2IyZzOHenmlvANBSUJLx+q2SPo+K+SePw0 R9NyuJn4EWaBPDFVyTa3CHXb6/jg+XDkK6fmmQ9aUQPwFuGwM1kjRAOt0Is+GMRcJ5yG cjKD+F26xqR91v8Hm+3pvurwUBakhP75BNsFycZSyeXL6MfXyadgqe7WctzUAF8io61h 30wU47EJJOROAjoDnKwZc1nNEs3ls4nAapMkaBOWZdVs2bmXxa9A98l2yxnwdO9OkKSI qvqDQnnvyDtGmqN8FfnsMpOSwGLQLH0VQH2kAVyGhyA9TXqnCf2lZ3mE4LYWQeAX1kN4 elYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740684992; x=1741289792; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=9qQGmvDxGhjhQGrQCPpUIPeeAlhy/ylNJXmNHALKC5Q=; b=Uvcm+ajNaD524QzQhAED8x/I5TLMcK+37SefhdaTCiTXfz1OjbnnD/odXSR0FAtom+ o0DrM66/Q3xJrWOwkU8HE0mdUz2aWvS8PqynVNysVueEtRtWzmW7Oi62Cd2dGVN/w9Es kr0p/lgQZ7zZ58GJtA2fhVDpSY3SlHOXhjDePuxBZfxr2xm5SF55/8X4Cj3EKdbxFatE L4+LgEuLXCWRwi7zE2mK4uS7R/QKmGYXqcicVBRJ7sg8+LR7Y7VAWhgRmblDxA1nBWeV JF6uf40kFNhghHE1XGIAFuJRPL8c7sPJ1hV6xmO+qi5CwVWoInvxpMGPcAw8JxX9vyaI JadQ== X-Forwarded-Encrypted: i=1; AJvYcCU5Je9urWtva9/zYTRYaZ8Z8eO32UfhKqtWZfZNEvbm4dfoTxoejlQ4SsA4mpsPNMNtKfmI7LYI6Q==@kvack.org X-Gm-Message-State: AOJu0YxA6M7Xb+LgYtmG/YU0QTH7C5G9fvnF/GB/RadQghbjk5NkZ6vR 2BU/qGUPLJHqro2EZplZYuhTfPb7SSYt38Z8hcqh/2Dkx+914N+u0tnxdeD2TSU= X-Gm-Gg: ASbGncs+fXPI/f5CpVUivIxuDwa4eDb/Z0ZVyh9OxUFKJb4r8ic3du1ZhiaUsNE4UTK 9uBU8CRPTJj7mAima+VFyKFR0RbJvoiMlUSaP87ANm6bqdNcXIOfsJwSJ43B/xqfo06LgJ/qhnV Q/D8knHtBRJqQc87dmRc9AU+efOh7eEPmfx/sL+gxFn8mjDkgPf/nVLgYhqiJGNS+/1EmebAL21 xiGGl8vSLAP3sbzkEQxBsH2pJ+Rq2FjJf2pehzV3X8CIxv6+5ZH7oS9Rb34B/W6REH4Re/9/JFv nS31RAw10GDecLEAimA3OQhl X-Google-Smtp-Source: AGHT+IEJhvb4x+uuyGVxu9aNvL9XmFgdQgjMcKRWIm6G4rB2pS6X0O2WlZ7omF+icm6hJ3XTVEd+xg== X-Received: by 2002:a05:620a:178e:b0:7c0:a0fd:7b2c with SMTP id af79cd13be357-7c39c4bea6bmr74185785a.22.1740684992142; Thu, 27 Feb 2025 11:36:32 -0800 (PST) Received: from localhost ([2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with UTF8SMTPSA id af79cd13be357-7c378dadf9dsm142215285a.111.2025.02.27.11.36.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Feb 2025 11:36:31 -0800 (PST) Date: Thu, 27 Feb 2025 14:36:27 -0500 From: Johannes Weiner To: Frank van der Linden Cc: akpm@linux-foundation.org, muchun.song@linux.dev, linux-mm@kvack.org, linux-kernel@vger.kernel.org, yuzhao@google.com, usamaarif642@gmail.com, joao.m.martins@oracle.com, roman.gushchin@linux.dev Subject: Re: [PATCH v4 10/27] mm/sparse: allow for alternate vmemmap section init at boot Message-ID: <20250227193627.GC115948@cmpxchg.org> References: <20250218181656.207178-1-fvdl@google.com> <20250218181656.207178-11-fvdl@google.com> <20250226180900.GA1042@cmpxchg.org> <20250227172014.GB115948@cmpxchg.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 5E7D940006 X-Rspam-User: X-Stat-Signature: wjktzbzopzuxur6s84dqqdoqaoucwn8y X-HE-Tag: 1740684993-989214 X-HE-Meta: U2FsdGVkX190MHrbO4jye5IvC5qx/XfqFe70MkLTvN8mWzQskvmQVb5/Tt7m5YA3M5NjJvj7FpNVV/0wDkbgNNT70E7k/TJqVqOkvx0zzkE6uQ+rZ3+NvG6FKrLYTvo3+JW6v1RE3QeUkqlzj0cd0uehSTu5DyFVZMzeO03m0yYEepWDp/cjU3CabUX4VcWHAtdosYryJ1LRq7D40B8kHFWuVS/D7r1Nd/0BHNsdM8fRD9XkfTUIqLd0ySvu21DXSVZStPAu8f1NDzQC2NumJpLIXKLHnBsMsLULV00rBUzpDs/V73IHuP+cFxu2qNzWJsVQZ/i4txAMsf0Zp1SPcogk+DtU1n5VNzG8WhRhvmvtq3KZ8p2uL5Ui5Xws31x+HEYRDLxA+1kexczK6p9EYKuy8nOdI1TPhwDyAV9YQA5NcPEfZ1MYe8eJV0qR/n3fWVEL7sMPMHxoA97tm8vwLXYwK7Ig3FrfCz9rU8jOMMsEIT/5GwFCLrQqlvg55sjkoFsNu43gyGZtQIFIzOCcYcZ1UM9xUr5zV1D68W9wsYpfFZxbx1RhI72+g8O+m9QT5Wh2j/YbueqpKkUXKBngqn1hM2NU0afzomAr7j/oUD+9kw28rPOboEiYOTU2PfEZ3uYtT8pKKhXCvHAhTWpv8beAO9jmXbr3K5OSlu8T5Gl1x/pEyqg/YmJqv/l1SpDgx2G32UtlcexZfBWYtXhh2F+qRtxu4VrGPB3gHgHgyzi63ZA3pNMnaPMxiSO7JqL2KHKVN/fMuqCezCXJ9mpXPquMuz5rnWEH3llBmAMgJMDWpJ0TvP61UfOHjYk4XpzZvFgifWyzRrixYalzFQiLyFvRKSwmxaFjBm8c7/PL/wOejNbUNySyxCcmfoIc7qEbBfkFj3TQd/XuSBKhF2IzCfDhc9ukHz6PzF+FjbE/TvOLmV8RKc1nw64WFcoHt52hqzVEZFlX3lFomJkUgIo av/yf1vc 6sNYZU0UWsJpf7t6lznMi5N1G6lDqxogStmWREdfhfcq3rFf0DkkXiN1fVEx8jh3m3fTPN7+/lBU91mazfeFb123oSpzaE2VaSGFnAGghVpwyx/601Rz9wmW3s9Xfq1YwCrNqty5CqcBGyVfJOhUEnGnjb0F8Zg1Hlw0vkpubbvWKAYjYnXuoTqxx155194XxbcktaHnIK6xESFRzNZLu7B7TIkGtvQOhpn2ee3N8Ype7bWI5PmzW6dcFSuAOlemU12iJOQjIB0d4F3n3sh2Z2RGcx6DeoiRt6GbigINllHsYLzIbrkHgwQVqNsWM8N4ymehxWHI6NaXH2wI7FRbJ78iQ1Hc6ELmzQtAlIe7f6HMXVNZdgqz6yLLfc79FvmGt7WHfaI/HApilElOEry2BOGRNxG/wDogm3ePZ+u/xjg2cKCvAxibX3xXBamlWs7ECdrvnzFNbgdmFrYW3POQnOWblFimRnR2c/FQUE560EB8Z5bXMW9jlcYqj2yxXJntOdwf69CRjVsm6px6sDZ2so/LWCA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Feb 27, 2025 at 10:03:04AM -0800, Frank van der Linden wrote: > On Thu, Feb 27, 2025 at 9:56 AM Frank van der Linden wrote: > > > > On Thu, Feb 27, 2025 at 9:32 AM Frank van der Linden wrote: > > > > > > On Thu, Feb 27, 2025 at 9:20 AM Johannes Weiner wrote: > > > > > > > > On Thu, Feb 27, 2025 at 08:47:18AM -0800, Frank van der Linden wrote: > > > > > On Wed, Feb 26, 2025 at 10:09 AM Johannes Weiner wrote: > > > > > > > > > > > > On Tue, Feb 18, 2025 at 06:16:38PM +0000, Frank van der Linden wrote: > > > > > > > @@ -489,6 +489,14 @@ config SPARSEMEM_VMEMMAP > > > > > > > SPARSEMEM_VMEMMAP uses a virtually mapped memmap to optimise > > > > > > > pfn_to_page and page_to_pfn operations. This is the most > > > > > > > efficient option when sufficient kernel resources are available. > > > > > > > + > > > > > > > +config ARCH_WANT_SPARSEMEM_VMEMMAP_PREINIT > > > > > > > + bool > > > > > > > + > > > > > > > +config SPARSEMEM_VMEMMAP_PREINIT > > > > > > > + bool "Early init of sparse memory virtual memmap" > > > > > > > + depends on SPARSEMEM_VMEMMAP && ARCH_WANT_SPARSEMEM_VMEMMAP_PREINIT > > > > > > > + default y > > > > > > > > > > > > oldconfig just prompted me on this, but it's not clear to me what it > > > > > > does. Not even after skimming the changelog of the patch to be honest. > > > > > > > > > > > > Can you please add a help text that explains the user-visible effects > > > > > > of the toggle, as well as guidance as to who might care to change it? > > > > > > > > > > Hi Johannes, > > > > > > > > > > Thanks for your comment. How's this: > > > > > > > > Thanks for the quick reply! > > > > > > > > > Enables subsystems to pre-initialize memmap in their own way, > > > > > allowing for memory savings during boot. The HugeTLB code uses > > > > > this to initialize memmap for bootmem allocated gigantic hugepages > > > > > in a way that is done by HUGETLB_PAGE_OPTIMIZE_VMEMMAP. This > > > > > means saving this memory right away, instead of allocating it > > > > > first and then freeing it later. Not allocating these pages > > > > > at all during boot allows for specifying a bigger number of > > > > > hugepages on the kernel commandline on larger systems. > > > > > > > > That makes sense. > > > > > > > > But if it's infra code for a hugetlb feature, it should either be > > > > something that HUGETLB_PAGE_OPTIMIZE_VMEMMAP pulls in automatically, > > > > or at least be a hugetlb-specific option that pulls it in. > > > > > > > > Keep in mind that not everybody enables HUGETLBFS. In fact, hugetlb is > > > > default N. It's moot to ask users whether they want to enable infra > > > > code for a feature they aren't using, and default to Y no less. You're > > > > regressing innocent bystanders doing this. > > > > > > The main reason that I added a separate config was: > > > > > > 1) I could see other subsystems use this. > > > 2) The number of section flags is limited, so I wanted to put the one > > > I added inside an option instead of always using it. Yeah, an *internal* config symbol make sense, so that the sparse flag and the code generation are gated on whether there is an actual user. I'm just proposing to make it invisible and let HUGETLB_PAGE_OPTIMIZE_VMEMMAP (and future users) select/depend on it. > > > If especially 2) is not a concern or can be solved differently, I'll > > > be happy to remove the option. I don't particularly like having it, > > > but I didn't see a better way. > > > > > > Let me think of a way to clean this up a little, and suggestions are > > > welcome, of course. > > > > > > - Frank > > > > I'll just do: > > > > diff --git a/fs/Kconfig b/fs/Kconfig > > index 64d420e3c475..fb9831927a08 100644 > > --- a/fs/Kconfig > > +++ b/fs/Kconfig > > @@ -286,6 +286,7 @@ config HUGETLB_PAGE_OPTIMIZE_VMEMMAP > > def_bool HUGETLB_PAGE > > depends on ARCH_WANT_OPTIMIZE_HUGETLB_VMEMMAP > > depends on SPARSEMEM_VMEMMAP > > + select SPARSEMEM_VMEMMAP_PREINIT > > > > config HUGETLB_PMD_PAGE_TABLE_SHARING > > def_bool HUGETLB_PAGE > > diff --git a/mm/Kconfig b/mm/Kconfig > > index f984dd928ce7..44b52f8e5296 100644 > > --- a/mm/Kconfig > > +++ b/mm/Kconfig > > @@ -496,7 +496,6 @@ config ARCH_WANT_SPARSEMEM_VMEMMAP_PREINIT > > config SPARSEMEM_VMEMMAP_PREINIT > > bool "Early init of sparse memory virtual memmap" > > depends on SPARSEMEM_VMEMMAP && ARCH_WANT_SPARSEMEM_VMEMMAP_PREINIT > > - default y > > > > Does that seem ok? I'll send an mm-unstable follow-up patch. > > > > Wait, that's actually not correct. Anyway, I'll stop spamming - I'll > do it along these lines but properly, and will send a follow-up patch. If you remove the prompt after "bool" it becomes an internal symbol that you can then pull in as needed. I agree that unconditionally consuming the sparse flag would be unfortunate, but consuming it when HUGETLB_PAGE_OPTIMIZE_VMEMMAP is enabled is fine, right? Seems like a specialized enough config. Thanks!