From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD1BDE77184 for ; Thu, 19 Dec 2024 08:07:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F13936B0095; Thu, 19 Dec 2024 03:07:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EC3A46B0098; Thu, 19 Dec 2024 03:07:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D64366B009A; Thu, 19 Dec 2024 03:07:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id B95BD6B0095 for ; Thu, 19 Dec 2024 03:07:38 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 5B1FB45E31 for ; Thu, 19 Dec 2024 08:07:38 +0000 (UTC) X-FDA: 82910978346.23.B1D82E3 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf20.hostedemail.com (Postfix) with ESMTP id 677B31C000C for ; Thu, 19 Dec 2024 08:07:02 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf20.hostedemail.com: domain of dev.jain@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=dev.jain@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734595641; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OzR24eBA1DMnFc74a7GsBoPByg+zZI0onV+dfsh3MxA=; b=EDlFCtAEGyrA8nJRsCIpGXp65ydDEC1pDvSCCupgD3qQunJPaitJvziDLPOg/UF0fCvbIP NcSiUNOZkO6NAIfaklxB71FDSpvdPOOEYzknQ9pmRl+cKHWkHcrT2zaB1mQKsiCRuf2b4I 0eklYZLGgTpHkqArsyIR0dY0w/K44zM= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf20.hostedemail.com: domain of dev.jain@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=dev.jain@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734595641; a=rsa-sha256; cv=none; b=ICobcBOpXrNNE9/0+2LqJKx3FparZBvKdfXes6nX4lHYW7PPITo5zFa1qTEAAtRqjvB//x yycy6wAzVI24puZq4a+tdjWSe0Y22j7VthcQRRvfuHdCgb1S+Egl4bsyuzLOzWoZhJNf3m YaM8ou6PQtlqMC61PZoKw7NOux8eWXg= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C48DF1477; Thu, 19 Dec 2024 00:08:03 -0800 (PST) Received: from [10.163.79.180] (unknown [10.163.79.180]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 606033F720; Thu, 19 Dec 2024 00:07:25 -0800 (PST) Message-ID: <33488d3c-a176-4779-a9c6-e9cbdd1afc62@arm.com> Date: Thu, 19 Dec 2024 13:37:21 +0530 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [RFC PATCH 10/12] khugepaged: Skip PTE range if a larger mTHP is already mapped From: Dev Jain To: John Hubbard , Ryan Roberts , akpm@linux-foundation.org, david@redhat.com, willy@infradead.org, kirill.shutemov@linux.intel.com Cc: anshuman.khandual@arm.com, catalin.marinas@arm.com, cl@gentwo.org, vbabka@suse.cz, mhocko@suse.com, apopple@nvidia.com, dave.hansen@linux.intel.com, will@kernel.org, baohua@kernel.org, jack@suse.cz, srivatsa@csail.mit.edu, haowenchao22@gmail.com, hughd@google.com, aneesh.kumar@kernel.org, yang@os.amperecomputing.com, peterx@redhat.com, ioworker0@gmail.com, wangkefeng.wang@huawei.com, ziy@nvidia.com, jglisse@google.com, surenb@google.com, vishal.moola@gmail.com, zokeefe@google.com, zhengqi.arch@bytedance.com, 21cnbao@gmail.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <20241216165105.56185-1-dev.jain@arm.com> <20241216165105.56185-11-dev.jain@arm.com> <7cc1840b-6f6c-4f82-86b8-41bb6fbc1b81@arm.com> <00d429c9-6ade-42c9-a1f3-a7519375324f@nvidia.com> Content-Language: en-US In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 677B31C000C X-Stat-Signature: 1k5kggukqth3irnth9xkarsypt4rd5if X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1734595622-608730 X-HE-Meta: U2FsdGVkX1+BARi5lucJCGhPG8gMUImFSx5s+tTomNgADPIaODY3+yIbDsp7s8EDrhatGOt4EfaLwb6evvXfIyycf+P84RayCjeQhj6jsSu1+qdpKxw70cJ5wRthdOneuD4cm4tpg5WpfMV8C7nzJ1G2C5/HooDf2lszZQ+kzXTTvHgmptWTSpO+4AbqnIGV0BEAkjH5+bjHqF2H0uAC/jRFlH4DNmUBjwvVo3fMOOzo/iJ3nUkVBzIw5jgKc5wrzYwvvzbkHrQlv3Ig3xSAwwoEEI4kbEkqlnr456hJx+DKAf2eA/VlPYGvaP0Pzmn3Zb5HXI9rNqO8EmY3mComncHELuve3oCmCmRUZClDlQ7E+lnq0FZYokpIy/PJY7sNzlUCCP/2gOVhp9WRHDSOV5hlvbGvXQbs5CjOENN8/uxW2jRZXSe8II3/VkwnKg6vBfgv2IW1R1a1/dx1NOcyZrUJ8QH8BW229ReV/hpC8pdv0cMRu0GvnnMP8/bceY7lg2FC0GxqNlmmusIX9z4wANqppUYEjMP6b5AVcwiBz1cBxKDGwD2ZtY5/296Lx06dh97+8MBxW92PpfUT+fguR3ZAgaqyOSzcIxBSMvA5z0r/bOePhJxjdSvgcTA9+5x2rTBjqPjSh5+h1/TXBOW0Vk7yOn23pVDZBn+VkSzKUpuZkbl2gXR5CGw+8LeVPfZUYwBILN3iqUEzYze/75bPn97hUOLOiAPZEaMLFelwwx8F0M7ijPcPoektR3jDD0v5HuAoFvAURXEOSyynwaGHJZZRAMk60GxYzhlGsarFG9An2rMGXDMUynVN2DYwHTlZIyFaI2D33dbRhby/ur/vmI6B07hD8vz/49a6fjxahw9rz/bqGrCOh2MHte0NloZD13B0edHKnkZZt9XweWyU8kV11kDS+Pimt1fg2LDOT1h3DSepBadujg3umQqiQEHLMjXu4DhJPiQvtA1xD8g gw5YF/NH Tf0DNSSMqmnnL/ctmmv4CwhhUpbif8IGygQrCa+1Op2eSP1bHnpHS10IsC0KTyE+kvMHUA66Os/OuquoxkeM7/Kh8VB6Z3KG5LfzzBBjQrH5z0hVsZCTu/2Q/3evlUFgzal3dKIVI/Kp0lm29/0xX56j4tVWkXix/uroHJRuwDnoLcSspdE7sqYiWqcPRgpfIs68XTOBlb5Fu3U29yMIXKLTV88EWuzrY1nH6HMxuN2yMN8Pgm2B2JBPEH4viS4ofLndCYAoASl9nhG2xA/8e8hFlovTYvpe4w9KJ7Tt6C/TeO1xIAgc7TxxaXg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 19/12/24 1:29 pm, Dev Jain wrote: > > On 19/12/24 9:10 am, John Hubbard wrote: >> On 12/18/24 1:34 AM, Dev Jain wrote: >>> On 18/12/24 1:06 pm, Ryan Roberts wrote: >>>> On 16/12/2024 16:51, Dev Jain wrote: >>>>> We may hit a situation wherein we have a larger folio mapped. It >>>>> is incorrect >>>>> to go ahead with the collapse since some pages will be unmapped, >>>>> leading to >>>>> the entire folio getting unmapped. Therefore, skip the >>>>> corresponding range. >> ... >>>> It would be good if you can spell out the desired policy when >>>> khugepaged hits >>>> partially unmapped large folios and unaligned large folios. I think >>>> the simple >>>> approach is to always collapse them to fully mapped, aligned folios >>>> even if the >>>> resulting order is smaller than the original. But I'm not sure >>>> that's definitely >>>> going to always be the best thing. >>>> >>>> Regardless, I'm struggling to understand the logic in this patch. >>>> Taking the >>>> order of a folio based on having hit one of it's pages says >>>> anything about >>>> whether the whole of that folio is mapped or not or it's alignment. >>>> And it's not >>>> clear to me how we would get to a situation where we are scanning >>>> for a lower >>>> order and find a (fully mapped, aligned) folio of higher order in >>>> the first place. >>>> >>>> Let's assume the desired policy is that khugepaged should always >>>> collapse to >>>> naturally aligned large folios. If there happens to be an existing >>>> aligned >>>> order-4 folio that is fully mapped, we will identify that for >>>> collapse as part >>>> of the scan for order-4. At that point, we should just notice that >>>> it is already >>>> an aligned order-4 folio and bypass collapse. Of course we may have >>>> already >>>> chosen to collapse it into a higher order, but we should definitely >>>> not get to a >>>> lower order before we notice it. >>>> >>>> Hmm... I guess if the sysfs thp settings have been changed then >>>> things could get >>>> spicy... if order-8 was previously enabled and we have an order-8 >>>> folio, then it >>>> get's disabled and khugepaged is scanning for order-4 (which is >>>> still enabled) >>>> then hits the order-8; what's the expected policy? rework into 2 >>>> order-4 folios >>>> or leave it as as single order-8? >>> >>> Exactly, sorry, I should have made it clear in the patch description >>> that I am >>> handling the following scenario: there is a long running system on >>> which we are >>> using order-8 folios, and now we decide to downgrade to order-4. >>> Will it be a >>> good idea to take the pain of splitting order-8 to 16 order-4 >>> folios? This should >>> be a rare situation in the first place, so I have currently decided >>> to ignore the >>> folios set up by the previous sysfs setting and only focus on >>> collapsing fresh memory. >>> >>> Thinking again, a sys-admin deciding to downgrade order of folios, >>> should do that in >>> the hopes of reducing internal fragmentation or increasing swap >>> speed etc, so it makes >>> sense to shatter large folios....maybe we can have a sysfs tunable >>> for this? >> >> Maybe we should not support it (at runtime) at all. We are trying to >> build >> systems that don't require incredibly detailed sysadmin involvement, and >> this level of tweaking qualifies, thoroughly, as "incredibly detailed >> sysadmin micromanagement", imho. > > Ryan pointed out one thing: what about unaligned, or partially mapped > large > folios? For the previous sysfs settings, it may happen that we have an > unaligned > order-8 folio, let us say it got unaligned due to mremap(). Then it is > a good > idea to start from the order-4 aligned page and start collapsing > memory so > that we can take advantage of the contig bit. Otherwise if it is a > fully-mapped > aligned order-8 folio, then we anyways are abusing the contig bit > advantage > so collapsing is pointless. In fact, in the current code, we are collapsing an unaligned PMD-size folio to an aligned PMD-mapped folio; we will not see a block mapping in the PMD, and go ahead with the scan...so the logic should be, skip the scan if the VAs and PAs are aligned. >> >> Apologies for not having gone through the series in detail yet, but this >> point jumped out at me. >> >> thanks, >