From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2057EB64DC for ; Thu, 6 Jul 2023 08:02:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 609D98D0003; Thu, 6 Jul 2023 04:02:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5BA208D0001; Thu, 6 Jul 2023 04:02:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4A8408D0003; Thu, 6 Jul 2023 04:02:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 3A9188D0001 for ; Thu, 6 Jul 2023 04:02:38 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id E98CF1A0C2A for ; Thu, 6 Jul 2023 08:02:37 +0000 (UTC) X-FDA: 80980445154.15.1AC105C Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf15.hostedemail.com (Postfix) with ESMTP id 8AAACA0010 for ; Thu, 6 Jul 2023 08:02:35 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf15.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688630556; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=j+SbKMH6OI6PELpyAxJWocTdrTEeSjPnyCuyOz3G0lE=; b=fbt8gP0ED02Q9bVVs2rGtDkujmtsDQJGidGL8Z7Ucs4T0TLjD3lHNtWRx8/15lXysaDKRg NVwEPG64Q/FbtuD2QHIDwx2DLGjGLW/tPKUQcyrbg2XwgEb1GDsUJt+ij5m2u1QWjBfVed HxtBKWsxjH6iefIxlGYh4zV/IV1qVX4= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf15.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688630556; a=rsa-sha256; cv=none; b=1nkyqzpviJJZyec+qzBD/Uz2cQLTvnLlupw1c3iI6F2m5B9nimOCG3wNgYbodffQMIc2Ue 7Od5S1vKjFZB4Y0TnwUFybwm/OxXrny7DIypirHpad7vE3kWLIw2+QX9IVY5SFd+l08VGI YBFjzQAagNCFdgRrecwLYrbTlELDCYU= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 34F832F4; Thu, 6 Jul 2023 01:03:16 -0700 (PDT) Received: from [10.57.76.116] (unknown [10.57.76.116]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 6F5A53F73F; Thu, 6 Jul 2023 01:02:32 -0700 (PDT) Message-ID: <4d4c45a2-0037-71de-b182-f516fee07e67@arm.com> Date: Thu, 6 Jul 2023 09:02:30 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 Subject: Re: [PATCH v2 0/5] variable-order, large folios for anonymous memory To: David Hildenbrand , Andrew Morton , Matthew Wilcox , "Kirill A. Shutemov" , Yin Fengwei , Yu Zhao , Catalin Marinas , Will Deacon , Anshuman Khandual , Yang Shi Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org References: <20230703135330.1865927-1-ryan.roberts@arm.com> <78159ed0-a233-9afb-712f-2df1a4858b22@redhat.com> From: Ryan Roberts In-Reply-To: <78159ed0-a233-9afb-712f-2df1a4858b22@redhat.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 8AAACA0010 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: qbrfhsh1f3jk7q9am1ejt9bzzmjhq5s7 X-HE-Tag: 1688630555-675097 X-HE-Meta: U2FsdGVkX18MAJANwCPzdUg2TzWbURvBwTUKbwCVnILRlKuAsdhykvgjz4JgFazYNB/QU9LGExm/ooZTxDl7lob4RD3PA7VsvO/HCNROf2pz6XFE5j0PDnIFapHh1dOBciqXK2RneZSXkwm/pKdLEAKKQ2fIdrpX8U2u4diP4VVR5zIhU2+jbmKuLe1pJROhrLkzm05AJ5qVZKVu/O/gfd6PbyxvKzN2b2RqwY5sKWMP+PtDwiKbY1udRJCPtsdabilA70ZR6euCUOnX/WiDENHtkhR+AiygGD8mHEhxD/XkNG9Tub8WiNC9lWvCMeN2cPLbURsnWN6fnygTKoyqOkOReNRCSxZAmV/TwcmxhwY/pLSln59VErQKEjwPHFy5jKoNGo8QLZPE+B9X+kHpg0VTYN2TgKCHT+owW4z1y/0abzRfE2NVyANnl8lOd3fGpdZddIPhyENOfm7Mo9wm3AdTwFZZ89lPjsuzq9yLMNzxn2ku/x7LhaM1hfTfdUUMe0nIDDxavH6CbBIy0jy2oQ/zWZXTJENsFLO28CLzQdLGALAkb04ov6Jd/HCT76shrEBDWplQjzQg/WOYMqosAXk1CNCPE0UA/wOYw9zOmcJ7UTF0K3pyl0HMYm5CTK0OsnHYC0nlziWHCvDnJMg2loYSIP3Hc5AgGl0gO2vl7kYuwcBsFZ9xqc0J4JOvH/STI0oIbBMkIMR1gXb30XNN2T4+kV3g29Xp4FcStNK97bu22mtoS1oR3XJsynOy767DMA+Qu/uJ9AipWYf//7vmSqwVhjVd1XrpKSk+grEoC97noUqQlVbt2gfTFEfyZJyXDYKPnuvS8LChQEmdQeB0Q2cxgsMZ3Up/aJQ23V8mhwYc3eE7j+mS0Enj9wYSUcVz7GA8n7uGyflczIOhoZlxsWqhrmURnWi+ZHF2JwmOc/Gs85hL8RereN1nLK0RSGNIlXfqtjUNSp7p9ay7Rkd 61URLGEC l8Ewbwz2KCjeY1cxd9GFba7+Agmm972IBbZi9Qe4WPmr2lXVVo0Z2b83JpeV02lhc8jhrh/+ZGjnbrxWfrVqFq2VXysGj2nIHOerXZ6MniLnyMRzeHOTI9OWTuuyJPTjrntP62/IdyNrtsERzvbO71hhZ0RkwVnaTDUUmQFaJOWzoLQHMcAGAXfdnvoqjzhuoHAc0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 05/07/2023 20:38, David Hildenbrand wrote: > On 03.07.23 15:53, Ryan Roberts wrote: >> Hi All, >> >> This is v2 of a series to implement variable order, large folios for anonymous >> memory. The objective of this is to improve performance by allocating larger >> chunks of memory during anonymous page faults. See [1] for background. >> [...] >> Thanks, >> Ryan > > Hi Ryan, > > is page migration already working as expected (what about page compaction?), and > do we handle migration -ENOMEM when allocating a target page: do we split an > fallback to 4k page migration? > Hi David, All, This series aims to be the bare minimum to demonstrate allocation of large anon folios. As such, there is a laundry list of things that need to be done for this feature to play nicely with other features. My preferred route is to merge this with it's Kconfig defaulted to disabled, and its Kconfig description clearly shouting that it's EXPERIMENTAL with an explanation of why (similar to READ_ONLY_THP_FOR_FS). That said, I've put together a table of the items that I'm aware of that need attention. It would be great if people can review and add any missing items. Then we can hopefully parallelize the implementation work. David, I don't think the items you raised are covered - would you mind providing a bit more detail so I can add them to the list? (or just add them to the list yourself, if you prefer). --- - item: mlock description: >- Large, pte-mapped folios are ignored when mlock is requested. Code comment for mlock_vma_folio() says "...filter out pte mappings of THPs, which cannot be consistently counted: a pte mapping of the THP head cannot be distinguished by the page alone." location: - mlock_pte_range() - mlock_vma_folio() assignee: Yin, Fengwei - item: numa balancing description: >- Large, pte-mapped folios are ignored by numa-balancing code. Commit comment (e81c480): "We're going to have THP mapped with PTEs. It will confuse numabalancing. Let's skip them for now." location: - do_numa_page() assignee: - item: madvise description: >- MADV_COLD, MADV_PAGEOUT, MADV_FREE: For large folios, code assumes exclusive only if mapcount==1, else skips remainder of operation. For large, pte-mapped folios, exclusive folios can have mapcount upto nr_pages and still be exclusive. Even better; don't split the folio if it fits entirely within the range? Discussion at https://lore.kernel.org/linux-mm/6cec6f68-248e-63b4-5615-9e0f3f819a0a@redhat.com/ talks about changing folio mapcounting - may help determine if exclusive without pgtable scan? location: - madvise_cold_or_pageout_pte_range() - madvise_free_pte_range() assignee: - item: shrink_folio_list description: >- Raised by Yu Zhao; I can't see the problem in the code - need clarification location: - shrink_folio_list() assignee: - item: compaction description: >- Raised at LSFMM: Compaction skips non-order-0 pages. Already problem for page-cache pages today. Is my understand correct? location: - assignee: --- Thanks, Ryan