From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AAE8AC71153 for ; Tue, 29 Aug 2023 20:04:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DFFEB8E003A; Tue, 29 Aug 2023 16:04:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D88F28E0029; Tue, 29 Aug 2023 16:04:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C03768E003A; Tue, 29 Aug 2023 16:04:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id AD0828E0029 for ; Tue, 29 Aug 2023 16:04:51 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 7642E802C7 for ; Tue, 29 Aug 2023 20:04:51 +0000 (UTC) X-FDA: 81178220382.18.A868E86 Received: from mail-pf1-f182.google.com (mail-pf1-f182.google.com [209.85.210.182]) by imf27.hostedemail.com (Postfix) with ESMTP id B383B4001B for ; Tue, 29 Aug 2023 20:04:49 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=DgUK7XEb; spf=pass (imf27.hostedemail.com: domain of shy828301@gmail.com designates 209.85.210.182 as permitted sender) smtp.mailfrom=shy828301@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1693339489; a=rsa-sha256; cv=none; b=fDbaASzHgdI88w0jJIr15Jdg6S0De1E8yeYdpJjyhm3I2jvXay7J3IA6pzCp4rDNrDZF4l zK8KscIV0EigpslpZ4q7A0zRTHVz0WIqwFALgSAB7FZ8HtH6AseviYBWjF1+p9IH4ds3wd OBWiowz25yVWI92V18I8joYWo6ACJ2w= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=DgUK7XEb; spf=pass (imf27.hostedemail.com: domain of shy828301@gmail.com designates 209.85.210.182 as permitted sender) smtp.mailfrom=shy828301@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1693339489; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YShQRRIoUyeyVMKKWVcVmBlF4Nh0zIEcHUDD0S50i3Y=; b=gv2pzNlUiUwQZWgI1SnMgPArLGrTTexvOnmTNPmGz5Ot+tuc6t8+NF66JiZFsYuF/gvnnU q3Ecg3aATKJ0ynCCIuuWun5BsT5xCGKoLFmtlTNqsVFF50WhPI4aUCNKhN+HzjsOsovIJC NDWvVyFsPkwHq05eqhwXlrKqDP+HdBo= Received: by mail-pf1-f182.google.com with SMTP id d2e1a72fcca58-68c3ec0578bso2238570b3a.2 for ; Tue, 29 Aug 2023 13:04:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1693339488; x=1693944288; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=YShQRRIoUyeyVMKKWVcVmBlF4Nh0zIEcHUDD0S50i3Y=; b=DgUK7XEbPheKLGOwpv+log+NQf+ERc5c7b7l+CFjSU8K54T2YgtNRi7bcnfp+NvJSO nHbQiXVnsHkeBKuxs89EGa7fzLBmRp0kjOvn/IsZkrn/7Njg3UR/oc3azhKlMIQiSn/l VEzUHt/PGtHMEJ6hnWxaiSD3O6fFdOtQjYBBAq5UqeT2rcSuLZW930j1yZvlAl57tAvI LUhTZ+PeRmJfy4WM3k52OIqS5InCdIQLENJU3+FgFzpEEdELFRdpiAjl1YjSvqDYbLM0 K8uvDO4EqlnFOHlHHh8/qeZy3soH39YNx0JR75YG4aONlfkRMVH1680Zb5nJ7X4iSUlu JMaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693339488; x=1693944288; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YShQRRIoUyeyVMKKWVcVmBlF4Nh0zIEcHUDD0S50i3Y=; b=QXx9dMiCYyUCmSoTplH7H07QOWk/J/ODnwVpkj/CEmz7c+K0rT0dkVbNKdsJ9dBkWO Ocz/inPNgBmulHLWGWVjLK4tVMvBNIgUXVNDs8mItLrTzO0OdWqBS5nkKZU6PIwROFpw gSEl5ugBhQhjxe3xU4mlvDEiL46tUtG0EFoG8fo3WKYiLtrK7ySkkwqFc+iBfPk3zcHL C3/lnnN4tcgmnvmfmkMJ0d17DYt74jwQhzX/GNZIiel3B6OPteemJv0I6Q9F/MelcFdw Z0T1kAtl3M24ouuu81GwVDtJ6g0dclgLZQSQcKwPLttAvG49heBpiuf6zUFjOoheZ3fN TPjQ== X-Gm-Message-State: AOJu0Yw4YIGDLimUIfXAYV4EbfZO8oyZqgD69tfD+p1uik7WSplYqEaT f8vXp8wu9Nccyzjyj8JTYUrjW3QMMCWeAYhoR6c= X-Google-Smtp-Source: AGHT+IF6MhqDzAOisq0kHguEIaPPsIYjONL9aE/QSQ+8Gt4dzCwb8f5PtvxGbko/WhJSAI9ZUBceCJ/2/Jyov7xHLBg= X-Received: by 2002:a17:90a:d41:b0:263:931b:bb5f with SMTP id 1-20020a17090a0d4100b00263931bbb5fmr262103pju.14.1693339488394; Tue, 29 Aug 2023 13:04:48 -0700 (PDT) MIME-Version: 1.0 References: <20230817035155.84230-1-liusong@linux.alibaba.com> In-Reply-To: <20230817035155.84230-1-liusong@linux.alibaba.com> From: Yang Shi Date: Tue, 29 Aug 2023 13:04:36 -0700 Message-ID: Subject: Re: [PATCH] mm/khugepaged: increase transparent_hugepage_recommend_disable parameter to disable active modification of min_free_kbytes To: Liu Song Cc: corbet@lwn.net, akpm@linux-foundation.org, paulmck@kernel.org, rdunlap@infradead.org, catalin.marinas@arm.com, dave.hansen@linux.intel.com, rostedt@goodmis.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: B383B4001B X-Stat-Signature: j1pbbnxd36nkr1g8w9h1t5kyynfa5nx1 X-Rspam-User: X-HE-Tag: 1693339489-823937 X-HE-Meta: U2FsdGVkX19mtGShJdVvYzAVmsl92VIXQYMcrcamW2G0pv0DQtTj4pAbREyQrYLZey8/P16VJbYieWzPepvMhr//7IfAgv1Yft+oAurU1J6TvfWh2FvoZsVC3pQHQW9VCPQnOKS7TPvgQwhe2jAiTAODeJzA9VK1b8TTYvNa/6XdlSHoyjNs1u2o8wUegfpkYrvNZLvqeES8fiV/HR0B54aCtNK8mRF/rRs065juNIdVCGOsj13qKNqp3lv5nSUeSYDzcvIlCGVfXuVrNz2Z6pfrAoeuzFC72g7TPX4Gc2yV7wjtugE+zMmcrJlsWGADO4I94u/O3leHiyj2nnDUdS4je34YN0zklkqgliOyhwVRSC08XVuwXDDwdTuLMi9+IUV+3MaKSMoQzI+WACdKjqJOperhX9SbkgyrNKGHmRGOaow7u2tXxRz5IMzZhkn4sKz2zLpSRDZvB/YCfqoW08hTM0yihU5D0kyAcZ4Jpo/DkxtA1LeoUFCH/6FK09r9utG0ae+bAECaFwDC8/SE8Hcpzq9UtC8zB6dsm7ht31bQUDlfMREvdDEM5DngPg2SQC7lwaC8pbBs8tEepRpcQwn7GpJcRVpO2Omai9vWvlahJeRM/rl0EMCNq1sm+rgRZ66b2uHNsa+8ddu4U7G2bbA63LTgXY1SctfiqbUZxK9cUuzU2MEq9zlDakUQ1yawWnoZ2LMprmlQOC9MvvxLzbxN65Rpowd1+SrqWPM6MXjFv1nKA3fHHKoP+52kFzRRjEK/YIHLqvI9S8HMd8XLnKO7x4lCuJKqQbM1TBz2a9+TE5QB6D5533w1q98OYPh+BYbRxS9VD9launIVSac/w0XSPUKDX1LVthJH4/vn67kwiXDoLMQiBLP4oHEhHW0YgB3Tky+DZd1IHwz5xmSJcf3WmdDzRxpT3m9r9EjtMJS9qkVj3W6MxUbpVGH2bmUzA26yWP8JeYRyEKzsloU A7GXHCM9 xCmSrCXxiKAiQqB6B3Muvn4mq8e/cIlF5G+sxJmbOFRsKTTic8p5pQPSE/UFdpvN0MXyMC3hHx0PQ14cXodaW8mIGTGxKaPX+uccm0Nn8iDijZiO+2bTthZE3GBNAlmfrnioB3aW7tJzObt6F4IdPcDdq5iqUA4qUpECw4xN+veYwzHAdERCXpdOdj3Si54W/6UqIZP4DUDh6lsZJ4C3DF0dNmLoxCSo81Wl63Bnpv7eV+ijZ/apq+9sRwQh54xG4U2OHjJamIU7oWbI6nvC7j9gxKXK0l1NvErg93r5e6y+r/QQf35+Jc8jlH/zclXCPTNcrWcxtr1mW8uZky+o6NY4El6yeuj9CFjRhdxPd8vUBi0Ruh88Gzp5DGqA/3k9HpRhQmkO7rfR2cRG8bZhkVcXxLmcqZw1a6bXDTLS7QbZWkgU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Aug 16, 2023 at 8:52=E2=80=AFPM Liu Song wrote: > > In the arm64 environment, when PAGESIZE is 4K, the "pageblock_nr_pages" > value is 512, and the recommended min_free_kbytes in > "set_recommended_min_free_kbytes" usually does not exceed 44MB. > > However, when PAGESIZE is 64K, the "pageblock_nr_pages" value is 8192, > and the recommended min_free_kbytes in "set_recommended_min_free_kbytes" > is 8192 * 2 * (2 + 9) * 64K, which directly increases to 11GB. > > According to this calculation method, due to the modification of min_free= _kbytes, > the reserved memory in my 128GB memory environment reaches 10GB, and MemA= vailable > is correspondingly reduced by 10GB. > > In the case of PAGESIZE 64K, transparent hugepages are 512MB, and we only > need them to be used on demand. If transparent hugepages cannot be alloca= ted, > falling back to regular 64K pages is completely acceptable. > > Therefore, we added the transparent_hugepage_recommend_disable parameter > to disable active modification of min_free_kbytes, thereby meeting our > requirements for transparent hugepages in the 64K scenario, and it will > not excessively reduce the available memory. Thanks for debugging this. I agree 11GB for min_free_kbytes is too much. But a kernel parameter sounds overkilling to me either. IMHO we just need to have a better scaling for bigger base page size. For example, we just keep one or two pageblock for min_free_kbytes when the base page size is bigger than 4K. > > Signed-off-by: Liu Song > --- > .../admin-guide/kernel-parameters.txt | 5 +++++ > mm/khugepaged.c | 20 ++++++++++++++++++- > 2 files changed, 24 insertions(+), 1 deletion(-) > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentat= ion/admin-guide/kernel-parameters.txt > index 654d0d921101..612bdf601cce 100644 > --- a/Documentation/admin-guide/kernel-parameters.txt > +++ b/Documentation/admin-guide/kernel-parameters.txt > @@ -6553,6 +6553,11 @@ > See Documentation/admin-guide/mm/transhuge.rst > for more details. > > + transparent_hugepage_recommend_disable > + [KNL,THP] > + Can be used to disable transparent hugepage to ac= tively modify > + /proc/sys/vm/min_free_kbytes during enablement pr= ocess. > + > trusted.source=3D [KEYS] > Format: > This parameter identifies the trust source as a b= ackend > diff --git a/mm/khugepaged.c b/mm/khugepaged.c > index 78fc1a24a1cc..ac40c618f4f6 100644 > --- a/mm/khugepaged.c > +++ b/mm/khugepaged.c > @@ -88,6 +88,9 @@ static unsigned int khugepaged_max_ptes_none __read_mos= tly; > static unsigned int khugepaged_max_ptes_swap __read_mostly; > static unsigned int khugepaged_max_ptes_shared __read_mostly; > > +/* default enable recommended */ > +static unsigned int transparent_hugepage_recommend __read_mostly =3D 1; > + > #define MM_SLOTS_HASH_BITS 10 > static DEFINE_READ_MOSTLY_HASHTABLE(mm_slots_hash, MM_SLOTS_HASH_BITS); > > @@ -2561,6 +2564,11 @@ static void set_recommended_min_free_kbytes(void) > goto update_wmarks; > } > > + if (!transparent_hugepage_recommend) { > + pr_info("do not allow to recommend modify min_free_kbytes= \n"); > + return; > + } > + > for_each_populated_zone(zone) { > /* > * We don't need to worry about fragmentation of > @@ -2591,7 +2599,10 @@ static void set_recommended_min_free_kbytes(void) > > if (recommended_min > min_free_kbytes) { > if (user_min_free_kbytes >=3D 0) > - pr_info("raising min_free_kbytes from %d to %lu t= o help transparent hugepage allocations\n", > + pr_info("raising user specified min_free_kbytes f= rom %d to %lu to help transparent hugepage allocations\n", > + min_free_kbytes, recommended_min); > + else > + pr_info("raising default min_free_kbytes from %d = to %lu to help transparent hugepage allocations\n", > min_free_kbytes, recommended_min); > > min_free_kbytes =3D recommended_min; > @@ -2601,6 +2612,13 @@ static void set_recommended_min_free_kbytes(void) > setup_per_zone_wmarks(); > } > > +static int __init setup_transparent_hugepage_recommend_disable(char *str= ) > +{ > + transparent_hugepage_recommend =3D 0; > + return 1; > +} > +__setup("transparent_hugepage_recommend_disable", setup_transparent_huge= page_recommend_disable); > + > int start_stop_khugepaged(void) > { > int err =3D 0; > -- > 2.19.1.6.gb485710b > >