From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ECFD6C02185 for ; Tue, 14 Jan 2025 16:07:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 81399280006; Tue, 14 Jan 2025 11:07:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7E980280005; Tue, 14 Jan 2025 11:07:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6D9A0280006; Tue, 14 Jan 2025 11:07:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 511DD280005 for ; Tue, 14 Jan 2025 11:07:59 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id A43EF80762 for ; Tue, 14 Jan 2025 16:07:58 +0000 (UTC) X-FDA: 83006538636.16.32A65D8 Received: from smtp-fw-9106.amazon.com (smtp-fw-9106.amazon.com [207.171.188.206]) by imf13.hostedemail.com (Postfix) with ESMTP id 7FB6220013 for ; Tue, 14 Jan 2025 16:07:56 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=amazon.com header.s=amazon201209 header.b=r46jbrii; spf=pass (imf13.hostedemail.com: domain of "prvs=102fcd225=kalyazin@amazon.co.uk" designates 207.171.188.206 as permitted sender) smtp.mailfrom="prvs=102fcd225=kalyazin@amazon.co.uk"; dmarc=pass (policy=quarantine) header.from=amazon.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736870876; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=X7ZylsU2EhTKBUzFmMhtT8tsRSiHQINi1OiMWtiHxJI=; b=HkLAY4B2+WwOHaj30Oxk+IYekN/p4gnM9ddclEfFihdmjEiKPWAEO4QTmTatCiDJl2CnX9 YH1pPdkFRUxZghMbTfbSB/G86YCWW2obDdQ/0gXyBoeo76Ddg5HYk6VTsVM/MgnT5Z4VCV n63VTc1WYyrHwEWmcwYCoji2g4UqC7k= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736870876; a=rsa-sha256; cv=none; b=zHABL47LxD/t/y/5tmIkLfG3nOXuMGK7G9e9Eu9rm3gcHmklk3gDp1f3I1B9rbDk4ZQydd 4j3S76enS3lLUGLIGaeGAHx1zbhL3lPjOcgJ6rJrb1fYZbb6TDGwMmiDREb/YC8ohP+Rki QulpoXhvCIK4ldxz2DaqZDMveX8/v3Q= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=amazon.com header.s=amazon201209 header.b=r46jbrii; spf=pass (imf13.hostedemail.com: domain of "prvs=102fcd225=kalyazin@amazon.co.uk" designates 207.171.188.206 as permitted sender) smtp.mailfrom="prvs=102fcd225=kalyazin@amazon.co.uk"; dmarc=pass (policy=quarantine) header.from=amazon.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1736870877; x=1768406877; h=message-id:date:mime-version:reply-to:subject:to:cc: references:from:in-reply-to:content-transfer-encoding; bh=X7ZylsU2EhTKBUzFmMhtT8tsRSiHQINi1OiMWtiHxJI=; b=r46jbriiyDhwA8Mqg/ef0RByITErSHK9CwW0BLPB0JHjQ+0fv1zhtXId js+IGo9yWPfZWyRmu3TJwiTLspPIZnUw5QDb7vuDEwGppd8ZzaBZkmBJp KUOFHkXonLfpwpxqv7KwE6uvOkWx6dGmcBefjxKUX/P/5/jKYkPJaE0QM k=; X-IronPort-AV: E=Sophos;i="6.12,314,1728950400"; d="scan'208";a="790990836" Received: from pdx4-co-svc-p1-lb2-vlan2.amazon.com (HELO smtpout.prod.us-west-2.prod.farcaster.email.amazon.dev) ([10.25.36.210]) by smtp-border-fw-9106.sea19.amazon.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jan 2025 16:07:50 +0000 Received: from EX19MTAEUC002.ant.amazon.com [10.0.17.79:1598] by smtpin.naws.eu-west-1.prod.farcaster.email.amazon.dev [10.0.13.19:2525] with esmtp (Farcaster) id 13a91ab9-83c4-4235-b79e-b137bc62dd5b; Tue, 14 Jan 2025 16:07:48 +0000 (UTC) X-Farcaster-Flow-ID: 13a91ab9-83c4-4235-b79e-b137bc62dd5b Received: from EX19D022EUC002.ant.amazon.com (10.252.51.137) by EX19MTAEUC002.ant.amazon.com (10.252.51.245) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.39; Tue, 14 Jan 2025 16:07:47 +0000 Received: from [192.168.8.40] (10.106.82.12) by EX19D022EUC002.ant.amazon.com (10.252.51.137) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA) id 15.2.1258.39; Tue, 14 Jan 2025 16:07:46 +0000 Message-ID: <9a188baa-034b-4dd5-b90e-7182f1fbaec6@amazon.com> Date: Tue, 14 Jan 2025 16:07:45 +0000 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Reply-To: Subject: Re: [RFC PATCH 0/2] mm: filemap: add filemap_grab_folios To: David Hildenbrand , , , , , , CC: , , , , , , , , , References: <20250110154659.95464-1-kalyazin@amazon.com> <5608af05-0b7a-4e11-b381-8b57b701e316@redhat.com> <5c62bdbb-7a4e-4178-8c03-e84491d8d150@redhat.com> Content-Language: en-US From: Nikita Kalyazin Autocrypt: addr=kalyazin@amazon.com; keydata= xjMEY+ZIvRYJKwYBBAHaRw8BAQdA9FwYskD/5BFmiiTgktstviS9svHeszG2JfIkUqjxf+/N JU5pa2l0YSBLYWx5YXppbiA8a2FseWF6aW5AYW1hem9uLmNvbT7CjwQTFggANxYhBGhhGDEy BjLQwD9FsK+SyiCpmmTzBQJj5ki9BQkDwmcAAhsDBAsJCAcFFQgJCgsFFgIDAQAACgkQr5LK IKmaZPOR1wD/UTcn4GbLC39QIwJuWXW0DeLoikxFBYkbhYyZ5CbtrtAA/2/rnR/zKZmyXqJ6 ULlSE8eWA3ywAIOH8jIETF2fCaUCzjgEY+ZIvRIKKwYBBAGXVQEFAQEHQCqd7/nb2tb36vZt ubg1iBLCSDctMlKHsQTp7wCnEc4RAwEIB8J+BBgWCAAmFiEEaGEYMTIGMtDAP0Wwr5LKIKma ZPMFAmPmSL0FCQPCZwACGwwACgkQr5LKIKmaZPNCxAEAxwnrmyqSC63nf6hoCFCfJYQapghC abLV0+PWemntlwEA/RYx8qCWD6zOEn4eYhQAucEwtg6h1PBbeGK94khVMooF In-Reply-To: <5c62bdbb-7a4e-4178-8c03-e84491d8d150@redhat.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-Originating-IP: [10.106.82.12] X-ClientProxiedBy: EX19D012EUC002.ant.amazon.com (10.252.51.162) To EX19D022EUC002.ant.amazon.com (10.252.51.137) X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 7FB6220013 X-Stat-Signature: fko9gnr84nrgy5ds7gje7cqfet1fsju7 X-Rspam-User: X-HE-Tag: 1736870876-402259 X-HE-Meta: U2FsdGVkX18Tu8nLVH6xLyIBfqV/ei6n3uyrBSHcvDq9Ygk3n+apo1IRB+4gP7+kq8mYhldI5cH+aSG33dfTX/hdKy2enDzomM0jdWhOicjvugSS5QvYzKsMjZNTyktQFi7X5sAY/x0Pua3/XQRnqycACcQ88BRRbdFPeHla2mDML1aU35GBZpqVhzSCHtiMRF1PRB4cnYkybsQv8jnUfo3Xkylub/oemiBQ//zvYZrahpMBigx7g4A9hV+wttAiyFbGDez7lga2QRJTjn/yeHjcIgE0kbJCEOpiV1d/wzzV9szz83NTBqAokB9ibHibTa2OTnEZEZMfVDIsGg3Kc+kcfcmiFge21cG4nWwOMmhEKIxSvxrGncrsHF8qWVeWqHfeEmAd8sWoRtJ0j0euQYHkC1aRX0mvYqVz2GKfvFhHFx5VjO3//sU3f2lgf0tgr9kR4qdLnS/S8dl35f5wmlUmLXwEgauO5OA6hQ/0iv/zb+tBtofrQyC7P2H0vAGUi4bi6tjxxSXxxfYfnaY4iZzWpfZCU5g3NRKPf8O7IY1acTj4EC8VGueeuVBnXaAR48SKHiO2tNu4sAqlGtfnobPx1VwMOJadThViIobCvCcwq0eLHf2GrYme7QUXcmd24iJs7n/QmUmQFnL6PRNGsSclCMgPTuA56Xt1XSK9J6vXGG8rYkYaJhFVey7WHPKkfR1UB0K1mlRw0Ne0dJ1fvOlVoQv+8qZipkQ8nGqhR+hlH9jtnUSnbEvwJbrEoDWt0wgMjn5cIv6AZMNasS+FxBUnMGdNSwI2UA1o/k1tfz/9tseuY+8xj3wwx46g8mbjIsYFGvhLJHBOr3rdyJVAy4uwoBOMoqFvU+8WisKMecAhDfERVUhRKCS4FdlWb+xtl3LG8nIkfMLgaCSMreOjA/7bywpWuVHQ6tAOS01mWJfsHJy5A+/gYUMhTBuJ3eoBTdBqVfIDKwLE0b+qEOg 4WL/Y3/i GWRewa37RFVDRCqnGqjSvK07TAUXTwYpu0W+o8ARUMopMVO+nAbl1T+85ZQtlUhJrPn5dSYk113CI5Lg1TqbxYR0UMdjpMwqSrwuTd4M1FqIGF1qdx6v7YoDflOmTH+yjrje8S8GjTJHPuNXr3pl2U3HPGHcv/ie/V62To2eo0AfKAHwuy8X2qlpYR6P+k3o4ESzK3HpbIrxCVvmT28ZavnpAhDlp9CvFte4J3jqeTIvWcZk7xW/lnVhwGgGM9kvyiFTz4Qa04e7Sj5ZIIqHRUKQ4w7k/oh2gtS8gYW4GwFg77wMvDvkUvIOE+JkgBeh1/BM2aD3fxZOGq41JBFICYu9rxgWrUNo0itBBQ6ZGH1BGKJRUUx+5ybs8pfKOLYnBW0vn3JPoA9wQvyizO3M6qMifawSUrpjzIee4gcLP0tifxIWd097A7AyUYjnDK8rzQkUZ8gkbMtrg5/SxJCoJtfSSMZFT/tmiY/Gc4bbjDkL0pHHuVtzk58imm8EUxv2JlvXIcvzT22rSpdXiJ7xvjtIvDqAICvej3DoTqpLhVPDdmGn369+DsWLq1jlBDyqPoES7WtVkM9JpuVR2VekdPeKgBAryhYHaco8LF9U1q/ZXtW7a3WcNOWqCfBWyLoyq3Gnf+jgE6Cut2pIFIKuhZ8P9W91UFNSWHnjuAnLak7pwm938Z7VAt+yKkQSIllpJfg9nWLQC715MD/2dN4pC60/vDg0RdZE0HQhPfGo06ilj8r3Lq6Q+F5ynUSUenK3P8gvezoGgvGAt32X3FZ1fZzQ5Z0eXpgFx2OQB X-Bogosity: Ham, tests=bogofilter, spamicity=0.005522, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 13/01/2025 12:20, David Hildenbrand wrote: > On 10.01.25 19:54, Nikita Kalyazin wrote: >> On 10/01/2025 17:01, David Hildenbrand wrote: >>> On 10.01.25 16:46, Nikita Kalyazin wrote: >>>> Based on David's suggestion for speeding up guest_memfd memory >>>> population [1] made at the guest_memfd upstream call on 5 Dec 2024 [2], >>>> this adds `filemap_grab_folios` that grabs multiple folios at a time. >>>> >>> >>> Hi, >> >> Hi :) >> >>> >>>> Motivation >>>> >>>> When profiling guest_memfd population and comparing the results with >>>> population of anonymous memory via UFFDIO_COPY, I observed that the >>>> former was up to 20% slower, mainly due to adding newly allocated pages >>>> to the pagecache.  As far as I can see, the two main contributors to it >>>> are pagecache locking and tree traversals needed for every folio.  The >>>> RFC attempts to partially mitigate those by adding multiple folios at a >>>> time to the pagecache. >>>> >>>> Testing >>>> >>>> With the change applied, I was able to observe a 10.3% (708 to 635 ms) >>>> speedup in a selftest that populated 3GiB guest_memfd and a 9.5% >>>> (990 to >>>> 904 ms) speedup when restoring a 3GiB guest_memfd VM snapshot using a >>>> custom Firecracker version, both on Intel Ice Lake. >>> >>> Does that mean that it's still 10% slower (based on the 20% above), or >>> were the 20% from a different micro-benchmark? >> >> Yes, it is still slower: >>    - isolated/selftest: 2.3% >>    - Firecracker setup: 8.9% >> >> Not sure why the values are so different though.  I'll try to find an >> explanation. > > The 2.3% looks very promising. It does. I sorted out my Firecracker setup and saw a similar figure there, which made me more confident. >> >>>> >>>> Limitations >>>> >>>> While `filemap_grab_folios` handles THP/large folios internally and >>>> deals with reclaim artifacts in the pagecache (shadows), for simplicity >>>> reasons, the RFC does not support those as it demonstrates the >>>> optimisation applied to guest_memfd, which only uses small folios and >>>> does not support reclaim at the moment. >>> >>> It might be worth pointing out that, while support for larger folios is >>> in the works, there will be scenarios where small folios are unavoidable >>> in the future (mixture of shared and private memory). >>> >>> How hard would it be to just naturally support large folios as well? >> >> I don't think it's going to be impossible.  It's just one more dimension >> that needs to be handled.  `__filemap_add_folio` logic is already rather >> complex, and processing multiple folios while also splitting when >> necessary correctly looks substantially convoluted to me.  So my idea >> was to discuss/validate the multi-folio approach first before rolling >> the sleeves up. > > We should likely try making this as generic as possible, meaning we'll > support roughly what filemap_grab_folio() would have supported (e.g., > also large folios). > > Now I find filemap_get_folios_contig() [thas is already used in memfd > code], > and wonder if that could be reused/extended fairly easily. Fair, I will see into how it could be made generic. > -- > Cheers, > > David / dhildenb >