From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4BF1C32774 for ; Wed, 24 Aug 2022 03:18:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 47C2C940007; Tue, 23 Aug 2022 23:18:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 403746B0074; Tue, 23 Aug 2022 23:18:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 27CD7940007; Tue, 23 Aug 2022 23:18:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 113156B0073 for ; Tue, 23 Aug 2022 23:18:15 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id DFCA0120B72 for ; Wed, 24 Aug 2022 03:18:14 +0000 (UTC) X-FDA: 79833027708.02.347DFCC Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) by imf17.hostedemail.com (Postfix) with ESMTP id C29A140014 for ; Wed, 24 Aug 2022 03:18:12 +0000 (UTC) Received: by mail-pf1-f175.google.com with SMTP id 1so8069126pfu.0 for ; Tue, 23 Aug 2022 20:18:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=r/Iq1H6m/C2yL0fNrluazrrCDTlyKaFFMtLoSCNrNGg=; b=vzL3rr0kGodNUcoOfC/mX7XDjB+/DvHaVBxMBIsUVpWthu/fTy4ws2dq9PEPZsuNcV 7jQI8ROW64nU7Aub1jlNRo+B2681onaLy63o32OSqebQgZAMn/UjHqiV0c97FWhGvayP pgROkiJ2hFI9fnALDipXBPg8xZrCU7iZwcyfMMy2EhF3aBViv9VIHDggkXef1jccxD4Y 9M6rTggjJxExm3ed2wmbQz4WFXkoUoQOCTxofDCeZHiM78jizmqPu01+//XRTK8EHGDv 3s7IP4kGl6sbyUe1Am2kT4PKLL1UcBVfKIvnbfUBExaeJoaFoRztu4Vf9omrIz8Fv1DO Orhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=r/Iq1H6m/C2yL0fNrluazrrCDTlyKaFFMtLoSCNrNGg=; b=D3k69BYEGsEH9QPapDLr/BuijXJq+BzA8n+50IDIdBjU8i3ZCttr5wcZTNva6E0ADn KfZ5LxmiMen8srUqb3T6QEZAeoTgiIM3ZbCICcgLLODZXslPjFcdJrOMbJU3jnfZBM2d mK9o+gHHL8kLfBgNTYoKvJ1MxWYCN9naFN5aDLOPK8nXy09CF/RF1jYLcIIb3nlojFNQ Aa3Xg/Fq3gUZBEfdZ2XTN5faEaUjE+EyYJiJHMg2/Brxt69Mz6UbOeIBfre7YDwkDgfL aZfmq+hjGTBYZL0bCvn0qqPw+jGkm9Qe9t7ZYhityJgd01NX7sUH9RzooSJM0Y+feWnv v+eg== X-Gm-Message-State: ACgBeo3sx6WJYLhatRZDi08gtixSrOY11T2lqur4IbwrnxqS0MZ1t3cC 5tU2i5BvNM7EFWIdQ7gYHZKLzw== X-Google-Smtp-Source: AA6agR4AOBIsvTAu2hTmlJsAKVgdHZ8nmL42wKkElJYuOfxm/RVcE21M/j2CHdmtSDgjv7JCs2eVng== X-Received: by 2002:a63:b246:0:b0:42a:3d57:7d5d with SMTP id t6-20020a63b246000000b0042a3d577d5dmr19549228pgo.294.1661311091737; Tue, 23 Aug 2022 20:18:11 -0700 (PDT) Received: from MacBook-Pro.local.bytedance.net ([139.177.225.241]) by smtp.gmail.com with ESMTPSA id x24-20020aa79418000000b0052d50e14f1dsm11500868pfo.78.2022.08.23.20.18.07 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 23 Aug 2022 20:18:11 -0700 (PDT) From: lizhe.67@bytedance.com To: mhocko@suse.com Cc: Jason@zx2c4.com, akpm@linux-foundation.org, keescook@chromium.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, lizefan.x@bytedance.com, lizhe.67@bytedance.com, mark-pk.tsai@mediatek.com, mhiramat@kernel.org, rostedt@goodmis.org, vbabka@suse.cz, yuanzhu@bytedance.com Subject: Re: [PATCH] page_ext: move up page_ext_init() to catch early page allocation if DEFERRED_STRUCT_PAGE_INIT is n Date: Wed, 24 Aug 2022 11:17:59 +0800 Message-Id: <20220824031759.75875-1-lizhe.67@bytedance.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1661311094; a=rsa-sha256; cv=none; b=wJ0DzMyKOwswWwUOnk3Kp2JuMBVBgAd7AeHB3X/ZBMB5lEpR2KaXkVjGxsadN7Ps0DcHML 84qO48NGXU0SshGwT33bhjSEmw+KaEcWd4pAtlRRIm0ZQFTcsFUm08IzpqhJUdKIWs92zO czQ6SSgDzkMgEppRlKYyG8Du74t0A2s= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=vzL3rr0k; dmarc=pass (policy=none) header.from=bytedance.com; spf=pass (imf17.hostedemail.com: domain of lizhe.67@bytedance.com designates 209.85.210.175 as permitted sender) smtp.mailfrom=lizhe.67@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1661311094; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=r/Iq1H6m/C2yL0fNrluazrrCDTlyKaFFMtLoSCNrNGg=; b=H9uAKCSAxrEQKpU+U11ikCbks+uPhYOiqR5B0kxzuaNxxHuxBpBEHTm8jL6blvKW/Ll/E+ halBAeBwdMMf2ZtMlhTC7Xfr6AIAd0v5oGw4UzSwjIKUWGZl+wrxe7y9ovKcVR634JxVQ+ U5IS1z04+zquLZWDdfR4LZfCYwOm1uw= X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: C29A140014 X-Rspam-User: Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=vzL3rr0k; dmarc=pass (policy=none) header.from=bytedance.com; spf=pass (imf17.hostedemail.com: domain of lizhe.67@bytedance.com designates 209.85.210.175 as permitted sender) smtp.mailfrom=lizhe.67@bytedance.com X-Stat-Signature: 1ynzmgw5doqj9xa8mkxmddfay6jf5ggm X-HE-Tag: 1661311092-331847 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 2022-08-22 7:08 UTC, mhocko@suse.com wrote: >On Sat 20-08-22 09:02:57, lizhe.67@bytedance.com wrote: >> On 2022-08-18 7:36 UTC, mhocko@suse.com wrote: >> >> From: Li Zhe >> >> >> >> In 'commit 2f1ee0913ce5 ("Revert "mm: use early_pfn_to_nid in page_ext_init"")', >> >> we call page_ext_init() after page_alloc_init_late() to avoid some panic >> >> problem. It seems that we cannot track early page allocations in current >> >> kernel even if page structure has been initialized early. >> >> >> >> This patch move up page_ext_init() to catch early page allocations when >> >> DEFERRED_STRUCT_PAGE_INIT is n. After this patch, we only need to turn >> >> DEFERRED_STRUCT_PAGE_INIT to n then we are able to analyze the early page >> >> allocations. This is useful especially when we find that the free memory >> >> value is not the same right after different kernel booting. >> > >> >is this actually useful in practice? I mean who is going to disable >> >DEFERRED_STRUCT_PAGE_INIT and recompile the kernel for debugging early >> >allocations? >> >> Yes it is useful. We use this method to catch the difference of early >> page allocations between two kernel. > >I was not questioning the functionality itself but the way how it is >achieved. Recompiling the kernel to achieve debuggability has proven to >be really a bad approach historically. Most people are using >pre-compiled kernels these days. > >> > I do see how debugging those early allocations might be useful but that >> > would require a boot time option to be practical IMHO. Would it make >> > sense to add a early_page_ext parameter which would essentially disable >> > the deferred ipage initialization. That should be quite trivial to >> > achieve (just hook into defer_init AFAICS). >> >> It is a good idea. A cmdline parameter is a flexible and dynamic method for >> us to decide whether to defer page's and page_ext's initilization. For >> comparison, this patch provides a static method to decide whether to defer >> page's and page_ext's initilization. They are not conflicting. My next >> work is trying to achieve your idea. > >They are not conflicting but this patch adds ifdefs and additional code >that needs compile time testing with different options set. I.e. it adds >maintenance burden for something that can be achieved by better means. >So if you are ok to work on the runtime knob then I would propose to >drop this patch from the mm tree and replace it by a trivial patch to >allow early boot debugging by a cmd line parameter. Yes you are right. Recompiling the kernel is not a clever method. I will send another patch with a cmd line parameter 'early_page_ext' to achieve this idea. Thanks for your advice.