From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 239F2C04AA5 for ; Thu, 25 Aug 2022 09:31:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A30CB6B0074; Thu, 25 Aug 2022 05:31:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9E0846B0075; Thu, 25 Aug 2022 05:31:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8A745940007; Thu, 25 Aug 2022 05:31:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 7A70C6B0074 for ; Thu, 25 Aug 2022 05:31:46 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 4BEBB1C62D0 for ; Thu, 25 Aug 2022 09:31:46 +0000 (UTC) X-FDA: 79837597812.10.3C8D678 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) by imf13.hostedemail.com (Postfix) with ESMTP id 0AE6220004 for ; Thu, 25 Aug 2022 09:31:44 +0000 (UTC) Received: by mail-pl1-f171.google.com with SMTP id jm11so18003155plb.13 for ; Thu, 25 Aug 2022 02:31:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=FkF/ieW9DrmFPlYn4kxhbKZWUjDQxs4hOhq1nOWUbV8=; b=v6DiykumEFWXgdqMcMhIspfDE1oja9wFVIrnV8rqLwVwlnY39s+EM5gzT9vhmoBeLo v6VwF77QU0XGq5IBHzHQZKNHBfq/yAcsBgDuE1tGHxpG8CCU0YwxK/4ljpdJb6+5YF82 VXR5kUA7A0IHQG+kJc8soAaPclxYNsbm98MqP6J03BSWjz1Jd0EedEHmyjQn1vf6WSMz SZhGOyJ61vUsjs2XzLlvTfNpQQIePS1f/2ofHVhTxICx1JkcOAmgbCD9IfwDNze4WNon FJ9Zsx23eV0KGVkqm+Sz1DhRhI9a/jeETIL+KNiQffsbqBsbWJwDOm1FwkQtWlP9Td7V 7keg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=FkF/ieW9DrmFPlYn4kxhbKZWUjDQxs4hOhq1nOWUbV8=; b=KQOgPq43mV/X2F487iEjaVHchi4hilSxRdmcJ8QpyzVTE+ZGkAyaK0JGOy6V3mA7Lv fyakl50ofwrBOvXaNffxZqLR3ND2zBU84UIaZYYaeGShQezXgBotZYX8NBjXs/juMCn1 sSJMt6Yp0e8TvOIV9DTBvLPZph8vEqUa4z2/j/QA4EtmUnOo2vpvAciRbI0XJAV+F+86 FJP5EYS5Ui88XbsrycGfd3Jj3FU6ceTOUHd239+OOnznTqNBpgWQB+SnvAAmwsDMUidV 9c9Q3V6HISUn2Q94CH2Wg/BknQ3+4igidZQxZYjq75Mz7emVuByyPbCwSwNvwp1s6hLG vtJA== X-Gm-Message-State: ACgBeo1EP5Yhgs9/4765R+ncHQjPN4U+Hm9u/tkF3nTZKwpwoLYJkP8A jL/jt5i4GG3uE5gvlRaY8AQxZQ== X-Google-Smtp-Source: AA6agR64FNDVCcY/q1Ls1c6co+VRCiun2r+tsha4TZ/3lAU+XvQ+Fla9eV7HgK3GzB4+tLAwjARlyw== X-Received: by 2002:a17:902:9894:b0:172:ca00:f305 with SMTP id s20-20020a170902989400b00172ca00f305mr3136942plp.107.1661419903738; Thu, 25 Aug 2022 02:31:43 -0700 (PDT) Received: from MacBook-Pro.local.bytedance.net ([139.177.225.241]) by smtp.gmail.com with ESMTPSA id n9-20020a170902e54900b00172f4835f53sm7205389plf.192.2022.08.25.02.31.38 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 25 Aug 2022 02:31:43 -0700 (PDT) From: lizhe.67@bytedance.com To: mhocko@suse.com Cc: Jason@zx2c4.com, akpm@linux-foundation.org, corbet@lwn.net, keescook@chromium.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, lizefan.x@bytedance.com, lizhe.67@bytedance.com, mark-pk.tsai@mediatek.com, mhiramat@kernel.org, rostedt@goodmis.org, vbabka@suse.cz, yuanzhu@bytedance.com Subject: Re: [PATCH v3] page_ext: introduce boot parameter early_page_ext Date: Thu, 25 Aug 2022 17:31:30 +0800 Message-Id: <20220825093130.98332-1-lizhe.67@bytedance.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1661419906; a=rsa-sha256; cv=none; b=n4VuKzMy+mucxTb/+ZeiJO32yFrEtPQ299l0stV7vSjmAXlZlYiceJTTlLn/S4Cft/FxMc NnmXerL+Ds8UM/qzIhdQ6yhJQW0cwEe7tfzLZBkAEvzmPHjYkh5LO/LcgpbLvi1nGYfErr cP9ZP+VpYLklsUi72XURRtfp7iJZBQc= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=v6Diykum; dmarc=pass (policy=none) header.from=bytedance.com; spf=pass (imf13.hostedemail.com: domain of lizhe.67@bytedance.com designates 209.85.214.171 as permitted sender) smtp.mailfrom=lizhe.67@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1661419906; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=FkF/ieW9DrmFPlYn4kxhbKZWUjDQxs4hOhq1nOWUbV8=; b=euGuuJ82eOFUBxAy8aj9hRKGaETHt0i141lrTkZJrgHJehWKiMTju2RgtSGA0awCvp9CQt MHtqwWOdSCYXVLmXaz4G4uiyULZEXpNF3GLw5LL6i2gm4bR042GV0X1x3NzSp+VqbRFeKU ib5csQ21vwg0Iu2z/PTLxmGzKH65TiA= Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=v6Diykum; dmarc=pass (policy=none) header.from=bytedance.com; spf=pass (imf13.hostedemail.com: domain of lizhe.67@bytedance.com designates 209.85.214.171 as permitted sender) smtp.mailfrom=lizhe.67@bytedance.com X-Rspam-User: X-Rspamd-Queue-Id: 0AE6220004 X-Rspamd-Server: rspam08 X-Stat-Signature: ip5oyz9bk6k1ii8hjerunq86a58az88p X-HE-Tag: 1661419904-396618 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 25 Aug 2022 09:11:29 +0200, mhocko@suse.com wrote: >On Thu 25-08-22 14:31:02, lizhe.67@bytedance.com wrote: >> From: Li Zhe >> >> In 'commit 2f1ee0913ce5 ("Revert "mm: use early_pfn_to_nid in page_ext_init"")', >> we call page_ext_init() after page_alloc_init_late() to avoid some panic >> problem. It seems that we cannot track early page allocations in current >> kernel even if page structure has been initialized early. >> >> This patch introduce a new boot parameter 'early_page_ext' to resolve this >> problem. If we pass it to kernel, function page_ext_init() will be moved >> up and feature 'deferred initialization of struct pages' will be disabled. > >will be disabled to initialize the page allocator early and prevent from >the OOM mentioned above. > >> It can help us to catch early page allocations. This is useful especially >> when we find that the free memory value is not the same right after >> different kernel booting. >> >> Changelogs: >> >> v1->v2: >> - use a cmd line parameter to move up function page_ext_init() instead of >> using CONFIG_DEFERRED_STRUCT_PAGE_INIT >> - fix oom problem[1] >> >> v2->v3: >> - make adjustments suggested by Michal Hocko >> >> v1 patch: https://lore.kernel.org/lkml/Yv3r6Y1vh+6AbY4+@dhcp22.suse.cz/T/ >> v2 patch: https://lore.kernel.org/lkml/20220824065058.81051-1-lizhe.67@bytedance.com/T/ >> >> [1]: https://lore.kernel.org/linux-mm/YwHmXLu5txij+p35@xsang-OptiPlex-9020/ > >the changelog is usually not part of the changelog and goes under --- Thanks for correcting my mistake! >> >> Suggested-by: Michal Hocko >> Signed-off-by: Li Zhe > >I still have few comments below before I am going to ack. But this looks >much better already. > >> --- >> Documentation/admin-guide/kernel-parameters.txt | 6 ++++++ >> include/linux/page_ext.h | 11 +++++++++++ >> init/main.c | 6 +++++- >> mm/page_alloc.c | 2 ++ >> mm/page_ext.c | 12 ++++++++++++ >> 5 files changed, 36 insertions(+), 1 deletion(-) >> >> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt >> index d7f30902fda0..7b5726828ac0 100644 >> --- a/Documentation/admin-guide/kernel-parameters.txt >> +++ b/Documentation/admin-guide/kernel-parameters.txt >> @@ -1471,6 +1471,12 @@ >> Permit 'security.evm' to be updated regardless of >> current integrity status. >> >> + early_page_ext [KNL] Boot-time early page_ext initializing option. >> + This boot parameter disables the deferred initialization >> + of struct page and move up function page_ext_init() in >> + order to catch early page allocations. Available with >> + CONFIG_PAGE_EXTENSION=y. > >For admins it would likely be more easier to understand something like >following > early_page_ext [KNL] Enforces page_ext initialization to earlier > stages so cover more early boot allocations. > Please note that as side effect some > optimizations might be disabled to achieve that > (e.g. parallelized memory initialization is > disabled) so the boot process might take longer, > especially on systems with a lot of memory. > Available with CONFIG_PAGE_EXTENSION=y Great, I will use this description in my v4 patch. It is much more easier for us to understand. Thanks! >[...] >> diff --git a/mm/page_ext.c b/mm/page_ext.c >> index 3dc715d7ac29..bf4f2a12d7dc 100644 >> --- a/mm/page_ext.c >> +++ b/mm/page_ext.c >> @@ -85,6 +85,18 @@ unsigned long page_ext_size = sizeof(struct page_ext); >> >> static unsigned long total_usage; >> >> +#ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT >> +bool early_page_ext __meminitdata; >> +#else >> +bool early_page_ext __meminitdata = true; >> +#endif > >Why should default depend on DEFERRED_STRUCT_PAGE_INIT at all. This is >just confusing and I do not see how it serves a purpose. We might grow >more optimizations which would prefent early page_ext init. > >Let's just have default false and only enforce with the parameter. This >is more predictable and easier to understand. Yes, this is confusing. Without depending on DEFERRED_STRUCT_PAGE_INIT, the logic here will be more clear. I will remove it from my v4 patch. Thanks!