From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B436FEB64D9 for ; Fri, 7 Jul 2023 11:41:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 106278D0002; Fri, 7 Jul 2023 07:41:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 08EBB8D0001; Fri, 7 Jul 2023 07:41:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E49C98D0002; Fri, 7 Jul 2023 07:41:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id CE7D88D0001 for ; Fri, 7 Jul 2023 07:41:00 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 97E1916040A for ; Fri, 7 Jul 2023 11:41:00 +0000 (UTC) X-FDA: 80984624280.15.969AF75 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf24.hostedemail.com (Postfix) with ESMTP id 4C78D18001E for ; Fri, 7 Jul 2023 11:40:58 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=E5S7Dvcd; spf=pass (imf24.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688730058; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=B42AZOQjOcVF/Y0j7fMumdgD1JfYw1Jno81BLnsFXtE=; b=wY8zvem1qxllUtrlBNY3vwmHhywpIs9tgESZSVwAx9DiJunJbTKktK3Mx6ZoT37btuz5oG LvxP+xIvCd4UrYDapt0xs+WOBZkoPjMCbw0fb9NmLgULqc9ttpwDFm6bGDHNkmuQkBcSnm cxvI0C+1VEv3H7HriLDXrnKEFDgqYiw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688730058; a=rsa-sha256; cv=none; b=Dqy1/5KzLT5W/ykpxvY5IKAwOHwzkCoshh1VwoFI//iOMnWoXnaE4Tx+8Btd3tk0wKUwz5 Lx0NbYJQUlFS3DGZMhPnAuNICXCwsir1t6xsFhQssIksfNCVYGbYMLpZ7LOG89DY8L5out kIUw9La6i+94rIqmf+TGM6evc0JPH9k= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=E5S7Dvcd; spf=pass (imf24.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1688730057; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=B42AZOQjOcVF/Y0j7fMumdgD1JfYw1Jno81BLnsFXtE=; b=E5S7DvcdtJXwUCsX5ztKp6kcht8lnHs3AiRtY/GAoc7w1haYg0ByzOkMtJpv8vzhp7CR4h ndljfaVxPY7m9kN8xNSmv1JeeAztIyJZt9zqa372/P7kRFUhekIdJ4PSJFbP8n5r1Gx4M1 TNCGXCoKyi+SWT4DAkFYkW9RFRWmwpU= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-401-i_xfCJa5NFSgWSPdPrLwzA-1; Fri, 07 Jul 2023 07:40:56 -0400 X-MC-Unique: i_xfCJa5NFSgWSPdPrLwzA-1 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-3fa8db49267so10701625e9.3 for ; Fri, 07 Jul 2023 04:40:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688730055; x=1691322055; h=content-transfer-encoding:in-reply-to:subject:organization:from :references:cc:to:content-language:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=B42AZOQjOcVF/Y0j7fMumdgD1JfYw1Jno81BLnsFXtE=; b=IuwSG3VZvHXIRI6HbR9+uXUaFiw9wfq7PwUJQXWssTcE54H3dG0sNymeTpJfrx0o+T EXAEK5zaQ73SktNAWOcQXv3O/IArQg+bATqAfGHY6VuFReTEpGP8t+Ymq1NYs2poYftW UfLJS9GMkTsp+9X/uBfBrQgNh+v72uXzMfVQJNZ5339L48IQcVqaRJr9IL21XQqLAeDI e7FyzDt3QYFk8n8Rh3+jDRTTIc0FvI3JW0pWSf+LSILgF2t6EzsFrGuAZYCK0K/DjZ2d +IRmGvEkZxLY8KHoCS+PQH9aCZ1AcANZT1ovVAIByjLs8+QwhlTj3Usn3Xn043QQ/WEA yEmw== X-Gm-Message-State: ABy/qLbR52pNBJgp/VlOtXmTdzwlFWvAGRdoZ3vJh1L8eAknNKRGlFlw vNTtxDhdXltwOOQ8D7vqQpGYErxWif0PvcLBmNGY3nc5AY2SevyBv76eCf+EvAWWBv9XPEOLjwq lGWbpU2ioeWU= X-Received: by 2002:a05:600c:2901:b0:3fb:e189:3532 with SMTP id i1-20020a05600c290100b003fbe1893532mr3613102wmd.20.1688730055294; Fri, 07 Jul 2023 04:40:55 -0700 (PDT) X-Google-Smtp-Source: APBJJlE2u5GKvVIRg71UDKdc/YaTynxzfOeVJPhDExv67pRdggkX9UXBnpHZW0fkzXhIX/EU8tjPNA== X-Received: by 2002:a05:600c:2901:b0:3fb:e189:3532 with SMTP id i1-20020a05600c290100b003fbe1893532mr3613084wmd.20.1688730054926; Fri, 07 Jul 2023 04:40:54 -0700 (PDT) Received: from ?IPV6:2003:d8:2f04:3c00:248f:bf5b:b03e:aac7? (p200300d82f043c00248fbf5bb03eaac7.dip0.t-ipconnect.de. [2003:d8:2f04:3c00:248f:bf5b:b03e:aac7]) by smtp.gmail.com with ESMTPSA id q15-20020a7bce8f000000b003fbe791a0e8sm2299013wmj.0.2023.07.07.04.40.53 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 07 Jul 2023 04:40:54 -0700 (PDT) Message-ID: Date: Fri, 7 Jul 2023 13:40:53 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 To: Ryan Roberts , Andrew Morton , Matthew Wilcox , "Kirill A. Shutemov" , Yin Fengwei , Yu Zhao , Catalin Marinas , Will Deacon , Anshuman Khandual , Yang Shi Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org References: <20230703135330.1865927-1-ryan.roberts@arm.com> <78159ed0-a233-9afb-712f-2df1a4858b22@redhat.com> <4d4c45a2-0037-71de-b182-f516fee07e67@arm.com> From: David Hildenbrand Organization: Red Hat Subject: Re: [PATCH v2 0/5] variable-order, large folios for anonymous memory In-Reply-To: <4d4c45a2-0037-71de-b182-f516fee07e67@arm.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Stat-Signature: eafauxk391jub8e4pi34mor4jrn63k7t X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 4C78D18001E X-Rspam-User: X-HE-Tag: 1688730058-19093 X-HE-Meta: U2FsdGVkX18uJJYP7HBMJwmFb1Uy3gz+easeV6OW0sBVJNwJZ4agDsceRIMqGnFe2S/A3tfB/BQzSDhatxVViiUjsGZWdU2BU/QWCxz4N3bUfPDJBijw2K4Ez8QqkI87NMJ+0MeJPJZreC9dO+ZMSsbxM3x1BhuobbgcsIEnKU++zBU+Tu93kXNpxXBq/CszvpfuB6i5EvFuGMgzFhGVqSThokJrwxsqGu5TlgHTVKLxUqp3YnTeDHBwLyTnDVz8ttqP4Y8kbx6oMZg5aIpB/bjDSyIGf9WSRCUwLGVdUaJpS2LFoYoMtKw42efKaeXsf0drq+PUYHsNb8wuHZfII1lmxKG2jEOVMzTudzghg4ZzsaIrxBXQWZ7Ja5BOx90ndUYgiEwI100obGeqZude/WDW7SAqiEXVa4NIkv4+mvyOCdXvTUq9p5CAHnj2M6hXtbdIPIKim5nEW7wa3R9gvPate07Nitujc6hIsspMrn0Jx3jDzjdHT/FAy795+AjWoqSHmNOERroQqBfwWVCF883BRFRHrdmc9ZAfgq/Z72/0yORCcm7S5CPhrnpE0iLUTG+QbxNKJBC3mqvIo3DnSiuWuXB/ikuAvqnNE5S6AW9lVSy6iXzgeeAZmh+v62yqxKY2CfiD6F2VNiqPvYLEIu3wve4sr+rT8eNdhhdGjD0qpgprHNd89Axi5bXlQyFuNtqJ4NkNa8/2bbY3W8sHxxCf1Rl2ZvSjP/8t8SLEvW8aGo7zCHoPwIPDPXenoInlJ36mqu8AvsntCErOfWVxK/ACgLGOWeEeXjuEPWiu8mX5GXP/mC3prgrZvyhicbacMxE1irhu885/Wwp6iG+VoKs/3KNLr+9SI2VB6Hokc9anabkk5eQ/0UxyjNDbPRg2a59BNjTEsaUFkA8XHAIBEnATareoIE9EbkSioQTVqgy+Ria72GS47ABhzjjeYqzpnoNl1JIv8LwS8JtsMx6 k2bBLaxK 3I6Y7KkTf8V2ijN8u0Ts/3E5viYR1wV4bWva66ExHLPRViJbum8M59rgbwDIJSM4lCDtHV9ZV0mdJPdXU3lOOI0pObaxF31fjHtS5tMoBTRtoVfDK5LJmrhnVfNJ6//BNA5aTXWEhVKIyALPeOkketATcdlFW0gP75ME1R8SpeRrRYu0TEgmCwaLUbtmw0tbEBVG2De9kQA9qBPcosvcbPJGrNrbF5rW+g47Y3mmD89ar9eRwYgDwhlaaxspCPn7rgZtD6cdn0QofHS9b7UOEENK8+xO3M7Bk1P5adKIp2GTQx4+t2NspNmDTFCNZb7MWB/ITpC+vNUW5zMPA11cBJnNQD1m7UVvotM7c/BMOQktiikPJr11FIQ0QsuaMI9ZxByQnh+TvPOOVKOovBE51Ws3evM7dKDiNjRumceYNHHhtfMywtQpB337dpNSF2k1KBd/YQ6RJd1iN3DDnwRoOfvWBNg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 06.07.23 10:02, Ryan Roberts wrote: > On 05/07/2023 20:38, David Hildenbrand wrote: >> On 03.07.23 15:53, Ryan Roberts wrote: >>> Hi All, >>> >>> This is v2 of a series to implement variable order, large folios for anonymous >>> memory. The objective of this is to improve performance by allocating larger >>> chunks of memory during anonymous page faults. See [1] for background. >>> > > [...] > >>> Thanks, >>> Ryan >> >> Hi Ryan, >> >> is page migration already working as expected (what about page compaction?), and >> do we handle migration -ENOMEM when allocating a target page: do we split an >> fallback to 4k page migration? >> > > Hi David, All, Hi Ryan, thanks a lot for the list. But can you comment on the page migration part (IOW did you try it already)? For example, memory hotunplug, CMA, MCE handling, compaction all rely on page migration of something that was allocated using GFP_MOVABLE to actually work. Compaction seems to skip any higher-order folios, but the question is if the udnerlying migration itself works. If it already works: great! If not, this really has to be tackled early, because otherwise we'll be breaking the GFP_MOVABLE semantics. > > This series aims to be the bare minimum to demonstrate allocation of large anon > folios. As such, there is a laundry list of things that need to be done for this > feature to play nicely with other features. My preferred route is to merge this > with it's Kconfig defaulted to disabled, and its Kconfig description clearly > shouting that it's EXPERIMENTAL with an explanation of why (similar to > READ_ONLY_THP_FOR_FS). As long as we are not sure about the user space control and as long as basic functionality is not working (example, page migration), I would tend to not merge this early just for the sake of it. But yes, something like mlock can eventually be tackled later: as long as there is a runtime interface to disable it ;) > > That said, I've put together a table of the items that I'm aware of that need > attention. It would be great if people can review and add any missing items. > Then we can hopefully parallelize the implementation work. David, I don't think > the items you raised are covered - would you mind providing a bit more detail so > I can add them to the list? (or just add them to the list yourself, if you prefer). > > --- > > - item: > mlock > > description: >- > Large, pte-mapped folios are ignored when mlock is requested. Code comment > for mlock_vma_folio() says "...filter out pte mappings of THPs, which > cannot be consistently counted: a pte mapping of the THP head cannot be > distinguished by the page alone." > > location: > - mlock_pte_range() > - mlock_vma_folio() > > assignee: > Yin, Fengwei > > > - item: > numa balancing > > description: >- > Large, pte-mapped folios are ignored by numa-balancing code. Commit > comment (e81c480): "We're going to have THP mapped with PTEs. It will > confuse numabalancing. Let's skip them for now." > > location: > - do_numa_page() > > assignee: > > > > - item: > madvise > > description: >- > MADV_COLD, MADV_PAGEOUT, MADV_FREE: For large folios, code assumes > exclusive only if mapcount==1, else skips remainder of operation. For > large, pte-mapped folios, exclusive folios can have mapcount upto nr_pages > and still be exclusive. Even better; don't split the folio if it fits > entirely within the range? Discussion at > > https://lore.kernel.org/linux-mm/6cec6f68-248e-63b4-5615-9e0f3f819a0a@redhat.com/ > talks about changing folio mapcounting - may help determine if exclusive > without pgtable scan? > > location: > - madvise_cold_or_pageout_pte_range() > - madvise_free_pte_range() > > assignee: > > > > - item: > shrink_folio_list > > description: >- > Raised by Yu Zhao; I can't see the problem in the code - need > clarification > > location: > - shrink_folio_list() > > assignee: > > > > - item: > compaction > > description: >- > Raised at LSFMM: Compaction skips non-order-0 pages. Already problem for > page-cache pages today. Is my understand correct? > > location: > - > > assignee: > I'm still thinking about the whole mapcount thingy (and I burned way too much time on that yesterday), which is a big item for such a list and affects some of these items. A pagetable scan is pretty much irrelevant for order-2 pages. But once we're talking about higher orders we really don't want to do that. I'm preparing a writeup with users and challenges. Is swapping working as expected? zswap? -- Cheers, David / dhildenb