From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0EB70C433F5 for ; Tue, 31 May 2022 02:40:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9486C6B0074; Mon, 30 May 2022 22:40:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8F1706B0078; Mon, 30 May 2022 22:40:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7DEEE6B007D; Mon, 30 May 2022 22:40:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 6EE466B0074 for ; Mon, 30 May 2022 22:40:14 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay12.hostedemail.com (Postfix) with ESMTP id 40029120294 for ; Tue, 31 May 2022 02:40:14 +0000 (UTC) X-FDA: 79524483948.26.BD6C1B3 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf14.hostedemail.com (Postfix) with ESMTP id 1E64410003A for ; Tue, 31 May 2022 02:40:09 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 0F5F323A; Mon, 30 May 2022 19:40:13 -0700 (PDT) Received: from [10.162.41.9] (unknown [10.162.41.9]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 663353F66F; Mon, 30 May 2022 19:40:07 -0700 (PDT) Message-ID: <84d81a4b-84fb-a1e9-9c9f-d0239f2b3841@arm.com> Date: Tue, 31 May 2022 08:10:04 +0530 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.5.0 Subject: Re: [PATCH v2] arm64: enable THP_SWAP for arm64 Content-Language: en-US To: Steven Price , Barry Song <21cnbao@gmail.com> Cc: Andrew Morton , Catalin Marinas , Will Deacon , Linux-MM , LAK , LKML , =?UTF-8?B?5byg6K+X5piOKFNpbW9uIFpoYW5n?= =?UTF-8?Q?=29?= , =?UTF-8?B?6YOt5YGl?= , hanchuanhua , Barry Song , "Huang, Ying" , Minchan Kim , Johannes Weiner , Hugh Dickins , Andrea Arcangeli , Yang Shi References: <20220527100644.293717-1-21cnbao@gmail.com> From: Anshuman Khandual In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Authentication-Results: imf14.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf14.hostedemail.com: domain of anshuman.khandual@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 1E64410003A X-Stat-Signature: 3oihujmr4ih4jjdd9i5hik5mca6p7ti3 X-Rspam-User: X-HE-Tag: 1653964809-70332 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 5/30/22 16:39, Steven Price wrote: > On 30/05/2022 10:53, Barry Song wrote: >> On Mon, May 30, 2022 at 7:07 PM Anshuman Khandual >> wrote: >>> >>> Hello Barry, >> >> Hi Anshuman, >> thanks! >> >>> >>> On 5/27/22 15:36, Barry Song wrote: >>>> From: Barry Song >>>> >>>> THP_SWAP has been proved to improve the swap throughput significantly >>>> on x86_64 according to commit bd4c82c22c367e ("mm, THP, swap: delay >>>> splitting THP after swapped out"). >>> It will be useful to run similar experiments on arm64 platform to >>> demonstrate tangible benefit, else we might be just enabling this >>> feature just because x86 has it. Do you have some data points ? >>> >>>> As long as arm64 uses 4K page size, it is quite similar with x86_64 >>>> by having 2MB PMD THP. So we are going to get similar improvement. >>> >>> This is an assumption without any data points (until now). Please >>> do provide some results. >> >> Fair enough though I believe THP_SWP is arch-independent. Our testing >> will post data. Plus, we do need it for real use cases with some possible >> out-of-tree code for this moment. so this patch does not originate only >> because x86 has it :-) >> >>> >>>> For other page sizes such as 16KB and 64KB, PMD might be too large. >>>> Negative side effects such as IO latency might be a problem. Thus, >>>> we can only safely enable the counterpart of X86_64. >>> >>> Incorrect reasoning. Although sometimes it might be okay to enable >>> a feature on platforms with possible assumptions about its benefits, >>> but to claim 'similar improvement, safely, .. etc' while comparing >>> against x86 4K page config without data points, is not very helpful. >>> >>>> A corner case is that MTE has an assumption that only base pages >>>> can be swapped. We won't enable THP_SWP for ARM64 hardware with >>>> MTE support until MTE is re-arched. >>> >>> re-arched ?? Did you imply that MTE is reworked to support THP ? >> >> I think at least MTE should be able to coexist with THP_SWP though >> I am not quite sure if MTE can be re-worked to fully support THP. > > There's no fundamental reason it cannot coexist, but there are many open > areas around MTE support in general. For example at the moment swap > support keeps the tags in memory because there's no easy way to plumb > the extra tag data into the swap infrastructure. > > The lazy zeroing of MTE tag storage has introduced a lot of complexity > and THP is another case where this complexity would show. It's possible > that it might make sense to take the hit of clearing tags in all pages > (i.e. make clear_page() clear the tags like mte_zero_clear_page_tags()). > >>> >>>> >>>> Cc: "Huang, Ying" >>>> Cc: Minchan Kim >>>> Cc: Johannes Weiner >>>> Cc: Hugh Dickins >>>> Cc: Andrea Arcangeli >>>> Cc: Anshuman Khandual >>>> Cc: Steven Price >>>> Cc: Yang Shi >>>> Signed-off-by: Barry Song >>>> --- >>>> arch/arm64/Kconfig | 1 + >>>> arch/arm64/include/asm/pgtable.h | 2 ++ >>>> include/linux/huge_mm.h | 12 ++++++++++++ >>>> mm/swap_slots.c | 2 +- >>>> 4 files changed, 16 insertions(+), 1 deletion(-) >>>> >>>> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig >>>> index a4968845e67f..5306009df2dc 100644 >>>> --- a/arch/arm64/Kconfig >>>> +++ b/arch/arm64/Kconfig >>>> @@ -101,6 +101,7 @@ config ARM64 >>>> select ARCH_WANT_HUGETLB_PAGE_OPTIMIZE_VMEMMAP >>>> select ARCH_WANT_LD_ORPHAN_WARN >>>> select ARCH_WANTS_NO_INSTR >>>> + select ARCH_WANTS_THP_SWAP if ARM64_4K_PAGES >>>> select ARCH_HAS_UBSAN_SANITIZE_ALL >>>> select ARM_AMBA >>>> select ARM_ARCH_TIMER >>>> diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h >>>> index 0b6632f18364..06076139c72c 100644 >>>> --- a/arch/arm64/include/asm/pgtable.h >>>> +++ b/arch/arm64/include/asm/pgtable.h >>>> @@ -45,6 +45,8 @@ >>>> __flush_tlb_range(vma, addr, end, PUD_SIZE, false, 1) >>>> #endif /* CONFIG_TRANSPARENT_HUGEPAGE */ >>>> >>>> +#define arch_thp_swp_supported !system_supports_mte >>> >>> Does it check for 'system_supports_mte' as a symbol or call system_supports_mte() >>> to ascertain runtime MTE support ? It might well be correct, but just does not >>> look much intuitive. >> >> yep. looks a bit weird. but considering we only need this for arm64 >> and arch_thp_swp_supported >> is a macro, I can't find a better way to make code modification >> smaller than this in mm core, arm64 >> and x86. and probably we will totally remove it once we make MTE >> co-exist with THP_SWP. >> >> Do you have any suggestions for a better solution? > > It would be better to write it as a function macro: > > #define arch_thp_swp_supported() (!system_supports_mte()) > > or you could go the whole way and introduce a static inline function > (overkill in this case IMHO): > > #define arch_thp_swp_supported > static inline bool arch_thp_swp_supported(void) > { > return !system_supports_mte(); > } I guess this approach is slightly better.