From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08954C021A9 for ; Mon, 17 Feb 2025 15:14:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 63102280024; Mon, 17 Feb 2025 10:14:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5E1486B013B; Mon, 17 Feb 2025 10:14:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4A89E280024; Mon, 17 Feb 2025 10:14:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 25B9E6B013A for ; Mon, 17 Feb 2025 10:14:00 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id C51BD16094E for ; Mon, 17 Feb 2025 15:13:59 +0000 (UTC) X-FDA: 83129781798.28.F2C65C8 Received: from mail-ej1-f53.google.com (mail-ej1-f53.google.com [209.85.218.53]) by imf01.hostedemail.com (Postfix) with ESMTP id B4CDF40002 for ; Mon, 17 Feb 2025 15:13:57 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=WpYzjyBd; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf01.hostedemail.com: domain of usamaarif642@gmail.com designates 209.85.218.53 as permitted sender) smtp.mailfrom=usamaarif642@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739805237; a=rsa-sha256; cv=none; b=JHul3O2v4XsLvXuNYAt0iSAg/PLMIXh5Ivk+029zZAJpA4b+YomuqppvX8yRRm7H1+z87u cVxUozUd8RMuxUPiehKv3GwJ3btd1E0y6xZhg3402vFc9vXxL7qB06bOU1V9lsWtIkySgB 7BRd/N/XU592yc+gSjuhCPZO2Ti/MwA= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=WpYzjyBd; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf01.hostedemail.com: domain of usamaarif642@gmail.com designates 209.85.218.53 as permitted sender) smtp.mailfrom=usamaarif642@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739805237; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=DoNbjmdbvjY/XVJJlaAAFeUh14svEzqtVgPXSMwNfBw=; b=ZJlSIM6QWi+K5P8MrpX1H4xkdd8+YotRsw3xrdI+urcX2LJPZuI2au6035wnkqmrtPo14O PySavkA4FRF/b6TEeE8St8qCVRZ4PI9O/rALzkFRVhRdHzp6ww7pReetTsfYQqnWdjYrTO gcEDczuCt+VNNiR6EDgpGCVCiMixBck= Received: by mail-ej1-f53.google.com with SMTP id a640c23a62f3a-abbac134a19so86302166b.0 for ; Mon, 17 Feb 2025 07:13:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739805236; x=1740410036; darn=kvack.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=DoNbjmdbvjY/XVJJlaAAFeUh14svEzqtVgPXSMwNfBw=; b=WpYzjyBddPzNZnx/xszZTqbQ5Wh54ghAv9HBvB2nrnEA4kScXTUDQxisdI6FicQOrR 27Le72GtRE1FinYSiY2cq2FImxvDAkio49lVvWipbOdmPM6pDO1r4X955IawPvmFUD9N HUlfMcm/x8CIriCTAOAhCRToeXPHqMjC9DKrPkWohC0f6Z8bBuHv3KstQxfvQMRhvQ09 SU6HYEB4jUTHCl7Yr5+6858JN5q9UO0yzGruv2EZLWvC/EEvG8a+7yXGxI2QgXQOJvgw NEFcKoVgSS3/VqP6pE4wr2rzzM8evcNrOp+zkg2Dbooa8X/qCNHKEaKWS/ggvgpt3JkM us7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739805236; x=1740410036; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=DoNbjmdbvjY/XVJJlaAAFeUh14svEzqtVgPXSMwNfBw=; b=gujJgR+z0/E0wQuR2st0i3emcGONGU67lygVV/w55hJSTlR67JF3e3aG2kGf0+32h1 32W5ZoL+pm03H0mlCSS8802QP8Qdx7wKAzcDq0lmzr4xcIQ5W1CiqGeg40ZjVmlDZbv/ 7v9liOm5A2NHxDl8ZdzcZVVt1YILEF3xN13dagbJhizOL+WQl0XJzBexfQS/caAFshIV w5cX2RqSVlu+3gH8REfoOo0r+qkRipEkfMRCaHStFy0hx6DfZllcfhO66NPh6xCfUsVn FG7gKnrr+wAGc3+wd9OcBbFkui7HdccGVYQes9mhtnVrN2ia0IcYcbeFC/BnoPWbJwQf PIjQ== X-Forwarded-Encrypted: i=1; AJvYcCU9W1MO5lNWP5kUHTufBWVXyTgvc9uHWT/8qRXatm0ucVuRDM/zYQUzzaA1zkBgsPO15Eo6ujpA6A==@kvack.org X-Gm-Message-State: AOJu0YzWUN2jSSrWltqkL/Vq9dQAwkOlRtwp0mRAmQ/sYV4N7Uroy38X 186XuHXkp6Jaei3q+szJm0a88srnBu0xose1/h2RciuqQgGr/GPl X-Gm-Gg: ASbGncsfswN2tMQUK+gDXWP9Tyg5tvLiq69eD7s9EmPaeeOaTorccFHfud3kDlGCpdJ wFZKoBB3/O8ZcP/xVKSRfJW2lzcLTpLa392vkegdyymhvDuWwPwHbBZDLdLrpMSyMfPQDA/1aco phGbJBhxTPBHV7WkCh1PZ4ewR69pEyJpJlFkc9cg2FrEZKJVfvAaSqhbSYTjstaJEGKWjH5mIkb fF0k/qMJeJdIE/dmxHzgTmZDEhR8/8oGm8UeR6AlX5JB19S/ToKQRIRE62wHMmw+GaHtLMDhuwB luJ6s7vYt4SeLHNU1MXOmSk4niP+HVGRFmPFWQ6LCUuImhqAUrJcLVH+W2fIGQ== X-Google-Smtp-Source: AGHT+IEP10eXUY8i3oDDDg3lF4ilHQydvyQYUQww7NrtLLL+sTgv656S34fPOBxn2U+5a6BaYbbw3A== X-Received: by 2002:a17:907:72cf:b0:aa6:96ad:f903 with SMTP id a640c23a62f3a-abb70db0230mr684578266b.31.1739805235904; Mon, 17 Feb 2025 07:13:55 -0800 (PST) Received: from ?IPV6:2a03:83e0:1126:4:fb:39c9:9a24:d181? ([2620:10d:c092:500::7:6466]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-abb904a4ae5sm303112766b.29.2025.02.17.07.13.54 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 17 Feb 2025 07:13:55 -0800 (PST) Message-ID: <3ef9a5f3-2d63-46db-b0b5-d6f7e78c7888@gmail.com> Date: Mon, 17 Feb 2025 15:13:54 +0000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC v2 5/5] mm: document mTHP defer setting To: Nico Pache , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org Cc: ryan.roberts@arm.com, anshuman.khandual@arm.com, catalin.marinas@arm.com, cl@gentwo.org, vbabka@suse.cz, mhocko@suse.com, apopple@nvidia.com, dave.hansen@linux.intel.com, will@kernel.org, baohua@kernel.org, jack@suse.cz, srivatsa@csail.mit.edu, haowenchao22@gmail.com, hughd@google.com, aneesh.kumar@kernel.org, yang@os.amperecomputing.com, peterx@redhat.com, ioworker0@gmail.com, wangkefeng.wang@huawei.com, ziy@nvidia.com, jglisse@google.com, surenb@google.com, vishal.moola@gmail.com, zokeefe@google.com, zhengqi.arch@bytedance.com, jhubbard@nvidia.com, 21cnbao@gmail.com, willy@infradead.org, kirill.shutemov@linux.intel.com, david@redhat.com, aarcange@redhat.com, raquini@redhat.com, dev.jain@arm.com, sunnanyong@huawei.com, audra@redhat.com, akpm@linux-foundation.org, rostedt@goodmis.org, mathieu.desnoyers@efficios.com, tiwai@suse.de, baolin.wang@linux.alibaba.com, corbet@lwn.net, shuah@kernel.org References: <20250211004054.222931-1-npache@redhat.com> <20250211004054.222931-6-npache@redhat.com> Content-Language: en-US From: Usama Arif In-Reply-To: <20250211004054.222931-6-npache@redhat.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspam-User: X-Rspamd-Queue-Id: B4CDF40002 X-Rspamd-Server: rspam12 X-Stat-Signature: 1x1d6yxstp1m6p4w3fyp57itrxbew6im X-HE-Tag: 1739805237-689300 X-HE-Meta: U2FsdGVkX1/cDnzrEInoEPUnAYkK9OR80h/ukKvwx7bDo2hWVZr0gtTwGavoRC4LL4180kwobJRvbpn2HM18kQ8s/Tl5uLz07S7ql2t4CaCFzQsnFsAMbYDxBCEgsQjMRVUsuBze0tdlkSsqqA2V2M4oUbVe9l5QhM4gq3P1CAAfU7qPEdLrJ/aAz/A6aZjMhnSROtLVafuUzZRcFXGvvz/5EdNdvFQ2O7e98kjEmSAlUfubeNHWB1z9fZBPjlL+r14h5lnDou8hgQlDnjWCcH0zg1CT5276nWTzr6+PSJra6XUa2aI/QWlpGeNrwpuv/afgShwkn7O74rmRhNp+VSPlhG+IIJVZG+32o8BdZYbDeRQGZmaCEYW/ejzAKf2rddG3EVHUK/bsQhftpqbpy7fos3N6twChyFbTy9rtvsQ+cZQIYwmO6Q5816B3+0Ha06LbA6jYrqhp/THuCT3JX1PESnQ3rjeHMu2EJfkY4oS30lTAQ2u1rQ+W6v4HKT7YqM+/YJkUF9gRTQzO+eOfNZJV0heXHZWEAiXnuzbshM5N8ZMlrjhi5qjemXZvvvRLZ0iwJo5NVimlAPOBLu/bU0PMmf3d0KNYLlK6fM8MDieqWRl4M7H4AB4fSgLEI7DiRUOJV+KAPAM342TkmCM1p640fyiedJVrDzYyaF5mglUbU/LBNm7Bz5VgXXlHtMhT9ELOAEnKXo7hnx+wqGYLD/Ljv3qN8K2qh6fFy7YRxoLjXzKhYq1FQjkUqWo21/7phJ9w1mxfth/uZ1snGg0n1RiXDpk71f/5wtDHIBpOSSIc5TcywdG/jZ9liLuo86kDzBlQOPxlWxlXwqy91yn74tGCWIwsvQ2tAklWhuqRSEHBa8+pGoCmIPDEzEaXPaSPkRASIToBpiwNiQRWi1RnTtb9RxH5/1GC3txPSn6kA02b4LRIZtNIZBlJPomOtrEzlN8D2MPLbTlw9HDzFsl 8iWH3zlz 0xzrSpvocn6Jlf5BlsCC9/Lem+14eJYUUSAM7J1k75hBp+j7DmkP3T+VwY4zN4yrfW2/FeW4HKnHYDQGFCSnxwMFOd0QdrXEdIFOZ1qwFAAkkf+1hb/i6yCkgixjdLfo4iahUiX31KbrpalVW3WaeeVy53xroGJtKfSM87XaF27kcdoTt0kLRdoSXkds994XZHyBedI+JN2MMoIvFNogQ63CvMCDCC0j863fYWO1lkNukke8YOx/jNf7UGmeBu4bBnlJsfoXjEAHch87lz4OgMlBJ0T9bq/UGhGEiBmwWXeuHnH25Wm1XaxXXRaSWIcHRNdmR8LZU6/gtm5zFabTpyDOFx/3eoVj1hXu8wDAXw+Db0AxxtfdLQg29F66B8Zq7n90+O8liP9Lt4aMAz7nAMcw1wj+JoWY43NoIBLdM1lZGNrEHLfhNnzOWsYLz+7Dy0xK+tdtxlQTuTKGRlyaQZAT6QLbxkWZWrtGEd6vSWkPURvTSxXpw3Fs1wwvMXJ3+BFR+61bS8X/kImsyCzVMpyity/upcrmkoaEP00X7H6kvLUfm6nQt4mzoAtMDTvj285DI/cDdubjpJf8xF6c3/53HXN0RENr+ZPEA X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 11/02/2025 00:40, Nico Pache wrote: > Now that we have mTHP support in khugepaged, lets add it to the > transhuge admin guide to provide proper guidance. > I think you should move this patch to the mTHP khugepaged series, and just send THP=defer separately from mTHP khguepaged. > Signed-off-by: Nico Pache > --- > Documentation/admin-guide/mm/transhuge.rst | 22 ++++++++++++++++------ > 1 file changed, 16 insertions(+), 6 deletions(-) > > diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst > index b3b18573bbb4..99ba3763c1c4 100644 > --- a/Documentation/admin-guide/mm/transhuge.rst > +++ b/Documentation/admin-guide/mm/transhuge.rst > @@ -63,7 +63,7 @@ often. > THP can be enabled system wide or restricted to certain tasks or even > memory ranges inside task's address space. Unless THP is completely > disabled, there is ``khugepaged`` daemon that scans memory and > -collapses sequences of basic pages into PMD-sized huge pages. > +collapses sequences of basic pages into huge pages. > > The THP behaviour is controlled via :ref:`sysfs ` > interface and using madvise(2) and prctl(2) system calls. > @@ -103,8 +103,8 @@ madvise(MADV_HUGEPAGE) on their critical mmapped regions. > Applications that would like to benefit from THPs but would still like a > more memory conservative approach can choose 'defer'. This avoids > inserting THPs at the page fault handler unless they are MADV_HUGEPAGE. > -Khugepaged will then scan the mappings for potential collapses into PMD > -sized pages. Admins using this the 'defer' setting should consider > +Khugepaged will then scan the mappings for potential collapses into (m)THP > +pages. Admins using this the 'defer' setting should consider > tweaking khugepaged/max_ptes_none. The current default of 511 may > aggressively collapse your PTEs into PMDs. Lower this value to conserve > more memory (ie. max_ptes_none=64). > @@ -119,11 +119,14 @@ Global THP controls > > Transparent Hugepage Support for anonymous memory can be entirely disabled > (mostly for debugging purposes) or only enabled inside MADV_HUGEPAGE > -regions (to avoid the risk of consuming more memory resources) or enabled > -system wide. This can be achieved per-supported-THP-size with one of:: > +regions (to avoid the risk of consuming more memory resources), defered to > +khugepaged, or enabled system wide. > + > +This can be achieved per-supported-THP-size with one of:: > > echo always >/sys/kernel/mm/transparent_hugepage/hugepages-kB/enabled > echo madvise >/sys/kernel/mm/transparent_hugepage/hugepages-kB/enabled > + echo defer >/sys/kernel/mm/transparent_hugepage/hugepages-kB/enabled > echo never >/sys/kernel/mm/transparent_hugepage/hugepages-kB/enabled > > where is the hugepage size being addressed, the available sizes > @@ -155,6 +158,13 @@ hugepage sizes have enabled="never". If enabling multiple hugepage > sizes, the kernel will select the most appropriate enabled size for a > given allocation. > > +khugepaged use max_ptes_none scaled to the order of the enabled mTHP size to > +determine collapses. When using mTHPs its recommended to set max_ptes_none low. > +Ideally less than HPAGE_PMD_NR / 2 (255 on 4k page size). This will prevent > +undesired "creep" behavior that leads to continously collapsing to a larger > +mTHP size. max_ptes_shared and max_ptes_swap have no effect when collapsing to a > +mTHP, and mTHP collapse will fail on shared or swapped out pages. > + This paragraph definitely belongs in the khugepaged series, as it doesn't have anything to do with THP=defer. re "Ideally less than HPAGE_PMD_NR / 2", what if you are running on amd, and using 16K and 2M THP=always only as, thats where the most TLB benefit is. Than this recommendation doesnt make sense? Also even if you have all mTHP sizes as always, shouldnt you start by collapsing to the largest THP size first? (I haven't reviewed the khugepaged series yet, so might be have been discussed there, I will try and review it). Did you see the creep behavior you mentioned in your experiments? > It's also possible to limit defrag efforts in the VM to generate > anonymous hugepages in case they're not immediately free to madvise > regions or to never try to defrag memory and simply fallback to regular > @@ -318,7 +328,7 @@ Alternatively, each supported anonymous THP size can be controlled by > passing ``thp_anon=[KMG],[KMG]:;[KMG]-[KMG]:``, > where ```` is the THP size (must be a power of 2 of PAGE_SIZE and > supported anonymous THP) and ```` is one of ``always``, ``madvise``, > -``never`` or ``inherit``. > +``defer``, ``never`` or ``inherit``. > > For example, the following will set 16K, 32K, 64K THP to ``always``, > set 128K, 512K to ``inherit``, set 256K to ``madvise`` and 1M, 2M