From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B003FC001DF for ; Fri, 4 Aug 2023 08:27:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 254AC2802D8; Fri, 4 Aug 2023 04:27:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 205502802D4; Fri, 4 Aug 2023 04:27:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0F4432802D8; Fri, 4 Aug 2023 04:27:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id F37FA2802D4 for ; Fri, 4 Aug 2023 04:27:56 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id BC0118134D for ; Fri, 4 Aug 2023 08:27:56 +0000 (UTC) X-FDA: 81085744152.09.5F3ACA2 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf14.hostedemail.com (Postfix) with ESMTP id 9F594100011 for ; Fri, 4 Aug 2023 08:27:54 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1691137675; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4Fr+cHLhIFzyEar9Y1zm5Nwc/S7ot5qIf9vMcPoV5S8=; b=t1UBtfFrDGv0ZR9r+YGsYt8XVRYh397alsU32WZBh0WuKnTDe1C9DwjdSKVnJAm8HHFnTW 9ripht92XaH0SRbdjkoazw9Xqd7+j9yY6FUiALqePPCccFBbsKLnVMeB1iTd2x5uDFwQzJ dpUIRv8S3Qz2/bP8h6mJqhHhQQcT0+c= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1691137675; a=rsa-sha256; cv=none; b=tQTsvUy4N3pQSfgVpasQvrGkDNWp/6dXVdQcQc2GVEuNest956lze6z8GLGbfyygiJt1Bj rYIB7m1eUuFPLHewYMSlOFIl8x6bYrcjQz+bIo60oFv7z7CEJ0ypi6SbLuTIuRAikbIow/ bnJYh8y9ARstjrSvBpEtCy//T60P9J4= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4AF072F4; Fri, 4 Aug 2023 01:28:36 -0700 (PDT) Received: from [10.57.77.247] (unknown [10.57.77.247]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id DF1E63F6C4; Fri, 4 Aug 2023 01:27:50 -0700 (PDT) Message-ID: <5e595904-3dca-0e15-0769-7ed10975fd0d@arm.com> Date: Fri, 4 Aug 2023 09:27:49 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: [PATCH v4 2/5] mm: LARGE_ANON_FOLIO for improved performance To: Yu Zhao Cc: Andrew Morton , Matthew Wilcox , Yin Fengwei , David Hildenbrand , Catalin Marinas , Will Deacon , Anshuman Khandual , Yang Shi , "Huang, Ying" , Zi Yan , Luis Chamberlain , Itaru Kitayama , "Kirill A. Shutemov" , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org References: <20230726095146.2826796-1-ryan.roberts@arm.com> <20230726095146.2826796-3-ryan.roberts@arm.com> From: Ryan Roberts In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 9F594100011 X-Rspam-User: X-Stat-Signature: ekpie9e1a9cs6z1sacogq75hieskr59p X-Rspamd-Server: rspam03 X-HE-Tag: 1691137674-877429 X-HE-Meta: U2FsdGVkX18jLnkG+0h/PXwuOvG2Xoska/pTLG0KNV0n+9srmMMX/tMsYQ9lD4aAranF2eS09CpqwgaPyPyRKzjTcj0YXsm3hi8ViEcn0BENHesCszqType4FHB4bVL9wAdF6dRZO0VQNS794NJejFBPZCoUcd3QvOKAfNQ1A3m1JZwepPm34F+mdQdvFx7pUmymKCR+HoQg+WxrY6jZKtYWUP8+5QiVVGXb1eoWFiKVLRfGGJyM16WqB7xlzbbQ39gcTwi2DMD/80rRjVwIuGR3i76The4Oh9PGUq23Dc3STO2xFBmxtwqQ9Q4VWbMj8Yvdle27uep9u0fRxrsZZocAlnrqGAinTkFr2wXB0Q3f+h4y5FyofpJ6Bp4ov/ZsS8PHoHc+JxxQB77Z52yjxcFWnY7qJW9SopVHCyK0Hzpo8x/PhuA4/HGbAePTH1Ttt8w7RnE46A8Tgi0WJJStoxCZVoIr81BSUjCmD1UV1lnWQZLPpZ7m5jeRWOlny/COhP1dHGLd8NAY/yMPwoymUBj4dK8J8mzgrbl39Z6NLV5bxpz/G7SIEtF3N9rZ5MpbXRSptgoMRjBLGuUT5uaMVSpB1XtPdsTCaSMpcIvC8uyhRRGHyUzSoKWrO/5eEh5yblX5R/hpe2SKH/vcfXOxz2X6ZJPEzM7S1ncVXh0W/H+CDLn1HDKjnpVRN0Qq2amJOuzZIdC5by5So7V1Vj1EzqEIYjeqOMEhIfJe1hcXSl/fjJClVoBPoiD46JU5gLhKi/j8TUasgqjCXusZ9z1FMNpiERzwmHeW6VrsoiAmcGVj3Ync1pmLxV3N1+V+CC0kS4SFz7CiiPHiWKn6c4yf7ExnFqV9ewNjfZORJVUn0Ee9RxImesGf8Ct2+Q8gDwIzCm4MsP2MfjKmSkXzyfYpPGTnyKxpykTGlhrInL5NXJpfDxjEiSdnMVDhijO0KPa1BjJPS6MjpWh52zmuTEi 46MfnE6a QuISsqcimJFb2ATjOnJFij/0PMVWz17nWR8XqvPTjM4K0+OItIWoZh795DQCxaht7h+MlzFHbhME5joAA2z8q2yRzHYSyE2nI+67CL61+qoE8Rtvz3SbPmBxLi3HssCzeiONRTYqjN0RQEz/rBXsWmow5XxhgevEN8OwwH22SgSXpvrW/dTntano3QaTDzDlFwpZ1xbSH4vyfAb3Tvd7jzUuIBf2O4ToaXNZJ8BJ/AOKAqhj2Mlp7T5vQ2srouyS0EtTYekg5kppk4tUYGknG1F9n4ZFnrgAPPcgw X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 04/08/2023 00:50, Yu Zhao wrote: > On Thu, Aug 3, 2023 at 6:43 AM Ryan Roberts wrote: >> >> + Kirill >> >> On 26/07/2023 10:51, Ryan Roberts wrote: >>> Introduce LARGE_ANON_FOLIO feature, which allows anonymous memory to be >>> allocated in large folios of a determined order. All pages of the large >>> folio are pte-mapped during the same page fault, significantly reducing >>> the number of page faults. The number of per-page operations (e.g. ref >>> counting, rmap management lru list management) are also significantly >>> reduced since those ops now become per-folio. >>> >>> The new behaviour is hidden behind the new LARGE_ANON_FOLIO Kconfig, >>> which defaults to disabled for now; The long term aim is for this to >>> defaut to enabled, but there are some risks around internal >>> fragmentation that need to be better understood first. >>> >>> When enabled, the folio order is determined as such: For a vma, process >>> or system that has explicitly disabled THP, we continue to allocate >>> order-0. THP is most likely disabled to avoid any possible internal >>> fragmentation so we honour that request. >>> >>> Otherwise, the return value of arch_wants_pte_order() is used. For vmas >>> that have not explicitly opted-in to use transparent hugepages (e.g. >>> where thp=madvise and the vma does not have MADV_HUGEPAGE), then >>> arch_wants_pte_order() is limited to 64K (or PAGE_SIZE, whichever is >>> bigger). This allows for a performance boost without requiring any >>> explicit opt-in from the workload while limitting internal >>> fragmentation. >>> >>> If the preferred order can't be used (e.g. because the folio would >>> breach the bounds of the vma, or because ptes in the region are already >>> mapped) then we fall back to a suitable lower order; first >>> PAGE_ALLOC_COSTLY_ORDER, then order-0. >>> >> >> ... >> >>> +#define ANON_FOLIO_MAX_ORDER_UNHINTED \ >>> + (ilog2(max_t(unsigned long, SZ_64K, PAGE_SIZE)) - PAGE_SHIFT) >>> + >>> +static int anon_folio_order(struct vm_area_struct *vma) >>> +{ >>> + int order; >>> + >>> + /* >>> + * If THP is explicitly disabled for either the vma, the process or the >>> + * system, then this is very likely intended to limit internal >>> + * fragmentation; in this case, don't attempt to allocate a large >>> + * anonymous folio. >>> + * >>> + * Else, if the vma is eligible for thp, allocate a large folio of the >>> + * size preferred by the arch. Or if the arch requested a very small >>> + * size or didn't request a size, then use PAGE_ALLOC_COSTLY_ORDER, >>> + * which still meets the arch's requirements but means we still take >>> + * advantage of SW optimizations (e.g. fewer page faults). >>> + * >>> + * Finally if thp is enabled but the vma isn't eligible, take the >>> + * arch-preferred size and limit it to ANON_FOLIO_MAX_ORDER_UNHINTED. >>> + * This ensures workloads that have not explicitly opted-in take benefit >>> + * while capping the potential for internal fragmentation. >>> + */ >>> + >>> + if ((vma->vm_flags & VM_NOHUGEPAGE) || >>> + test_bit(MMF_DISABLE_THP, &vma->vm_mm->flags) || >>> + !hugepage_flags_enabled()) >>> + order = 0; >>> + else { >>> + order = max(arch_wants_pte_order(), PAGE_ALLOC_COSTLY_ORDER); >>> + >>> + if (!hugepage_vma_check(vma, vma->vm_flags, false, true, true)) >>> + order = min(order, ANON_FOLIO_MAX_ORDER_UNHINTED); >>> + } >>> + >>> + return order; >>> +} >> >> >> Hi All, >> >> I'm writing up the conclusions that we arrived at during discussion in the THP >> meeting yesterday, regarding linkage with exiting THP ABIs. It would be great if >> I can get explicit "agree" or disagree + rationale from at least David, Yu and >> Kirill. >> >> In summary; I think we are converging on the approach that is already coded, but >> I'd like confirmation. >> >> >> >> The THP situation today >> ----------------------- >> >> - At system level: THP can be set to "never", "madvise" or "always" >> - At process level: THP can be "never" or "defer to system setting" >> - At VMA level: no-hint, MADV_HUGEPAGE, MADV_NOHUGEPAGE >> >> That gives us this table to describe how a page fault is handled, according to >> process state (columns) and vma flags (rows): >> >> | never | madvise | always >> ----------------|-----------|-----------|----------- >> no hint | S | S | THP>S >> MADV_HUGEPAGE | S | THP>S | THP>S >> MADV_NOHUGEPAGE | S | S | S >> >> Legend: >> S allocate single page (PTE-mapped) >> LAF allocate lage anon folio (PTE-mapped) >> THP allocate THP-sized folio (PMD-mapped) >>> fallback (usually because vma size/alignment insufficient for folio) >> >> >> >> Principles for Large Anon Folios (LAF) >> -------------------------------------- >> >> David tells us there are use cases today (e.g. qemu live migration) which use >> MADV_NOHUGEPAGE to mean "don't fill any PTEs that are not explicitly faulted" >> and these use cases will break (i.e. functionally incorrect) if this request is >> not honoured. > > I don't remember David saying this. I think he was referring to UFFD, > not MADV_NOHUGEPAGE, when discussing what we need to absolutely > respect. My understanding was that MADV_NOHUGEPAGE was being applied to regions *before* UFFD was being registered, and the app relied on MADV_NOHUGEPAGE to not back any unfaulted pages. It's not completely clear to me how not honouring MADV_NOHUGEPAGE would break things though. David?