From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81B36C30653 for ; Sun, 7 Jul 2024 16:39:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 639BD6B0082; Sun, 7 Jul 2024 12:39:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5E9356B0083; Sun, 7 Jul 2024 12:39:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 48A246B0088; Sun, 7 Jul 2024 12:39:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 2A8156B0082 for ; Sun, 7 Jul 2024 12:39:35 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 83C98A0CEA for ; Sun, 7 Jul 2024 16:39:34 +0000 (UTC) X-FDA: 82313517468.30.F609678 Received: from mailout1.w1.samsung.com (mailout1.w1.samsung.com [210.118.77.11]) by imf24.hostedemail.com (Postfix) with ESMTP id 923D0180017 for ; Sun, 7 Jul 2024 16:39:30 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=samsung.com header.s=mail20170921 header.b=qYyWUGoF; spf=pass (imf24.hostedemail.com: domain of da.gomez@samsung.com designates 210.118.77.11 as permitted sender) smtp.mailfrom=da.gomez@samsung.com; dmarc=pass (policy=none) header.from=samsung.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720370358; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ck7kOg6cm3eaDL4aJHN1z8N3YaVIAz2E2fFFluJKZ84=; b=oWeyvxC/toCtdOmBczJL2gZoNBtUsRXl5egIs4XHJjWnoady0HxJIaOZ9S9a/2B0MLA9u0 F1AOE0cwxnbc17NdTqpY6wssLMddVJHCBvm/Cxa2VJs30kW2zZgWmqgT038G1SYiJOEUtx DsOExZp77QzAyNlNApem1LW+aCYhLlI= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=samsung.com header.s=mail20170921 header.b=qYyWUGoF; spf=pass (imf24.hostedemail.com: domain of da.gomez@samsung.com designates 210.118.77.11 as permitted sender) smtp.mailfrom=da.gomez@samsung.com; dmarc=pass (policy=none) header.from=samsung.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720370358; a=rsa-sha256; cv=none; b=78CHR+PF6aOLveo7DL0HK/eqx4dwvW+bwRUVP3eN+/OxSlCkfx1hl5jXf72nDk8i0HKxB0 lhfYBBXgxWhjmLkF22sZC3xcHnf4ehYd+Sty67eX7YWg3Sl85UTSRiaSTz60F0kykFGxoc iWvLNjynipnCMh1oilhr2v+YrGYao5c= Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout1.w1.samsung.com (KnoxPortal) with ESMTP id 20240707163928euoutp017759536d58c28f292a996812f95ae041~f_7wndb5x1373213732euoutp01N for ; Sun, 7 Jul 2024 16:39:28 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.w1.samsung.com 20240707163928euoutp017759536d58c28f292a996812f95ae041~f_7wndb5x1373213732euoutp01N DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1720370368; bh=Ck7kOg6cm3eaDL4aJHN1z8N3YaVIAz2E2fFFluJKZ84=; h=From:To:CC:Subject:Date:In-Reply-To:References:From; b=qYyWUGoFEpOl/wj6vm7gHLsbgwDcobUY19EitEYKRqCl5hXIt1/KAqoOz6FcopSco 8NWNaGF8NKcifNJdcVzQ7Hvst6yLoVgsdQl7v7rKsS0fenSPhHQh4T7680aSC3n4Kx b/5VNRvFqKn900DGwkk0j6BvRf4DYH5l1Iety7BQ= Received: from eusmges2new.samsung.com (unknown [203.254.199.244]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20240707163927eucas1p22b0ff8a5973ee97e0e6a4e0db68c4bc8~f_7wZo77n1566215662eucas1p2o; Sun, 7 Jul 2024 16:39:27 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges2new.samsung.com (EUCPMTA) with SMTP id E5.11.09875.FB4CA866; Sun, 7 Jul 2024 17:39:27 +0100 (BST) Received: from eusmtrp2.samsung.com (unknown [182.198.249.139]) by eucas1p1.samsung.com (KnoxPortal) with ESMTPA id 20240707163926eucas1p10cba583480dcbb91b37796353d378473~f_7vRW6ms2145221452eucas1p1Q; Sun, 7 Jul 2024 16:39:26 +0000 (GMT) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eusmtrp2.samsung.com (KnoxPortal) with ESMTP id 20240707163926eusmtrp2875b553fc9767ce801f2b8f5c166cd6c~f_7vQuvTk2193921939eusmtrp2e; Sun, 7 Jul 2024 16:39:26 +0000 (GMT) X-AuditID: cbfec7f4-131ff70000002693-0b-668ac4bf4a95 Received: from eusmtip2.samsung.com ( [203.254.199.222]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id 2C.77.09010.EB4CA866; Sun, 7 Jul 2024 17:39:26 +0100 (BST) Received: from CAMSVWEXC01.scsc.local (unknown [106.1.227.71]) by eusmtip2.samsung.com (KnoxPortal) with ESMTPA id 20240707163926eusmtip2a69e87812e4ab0662a3f9cf8d2b3cda9~f_7vDBRZ93267732677eusmtip2t; Sun, 7 Jul 2024 16:39:26 +0000 (GMT) Received: from CAMSVWEXC01.scsc.local (2002:6a01:e347::6a01:e347) by CAMSVWEXC01.scsc.local (2002:6a01:e347::6a01:e347) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Sun, 7 Jul 2024 17:39:26 +0100 Received: from CAMSVWEXC01.scsc.local ([::1]) by CAMSVWEXC01.scsc.local ([fe80::7d73:5123:34e0:4f73%13]) with mapi id 15.00.1497.012; Sun, 7 Jul 2024 17:39:26 +0100 From: Daniel Gomez To: David Hildenbrand CC: Ryan Roberts , Baolin Wang , Matthew Wilcox , "akpm@linux-foundation.org" , "hughd@google.com" , "wangkefeng.wang@huawei.com" , "ying.huang@intel.com" , "21cnbao@gmail.com" <21cnbao@gmail.com>, "shy828301@gmail.com" , "ziy@nvidia.com" , "ioworker0@gmail.com" , Pankaj Raghav , "linux-mm@kvack.org" , "linux-kernel@vger.kernel.org" Subject: Re: [PATCH v5 0/6] add mTHP support for anonymous shmem Thread-Topic: [PATCH v5 0/6] add mTHP support for anonymous shmem Thread-Index: AQHau+f0u3UalAdLwEiKkul1osqzdLHm+4gAgAAFkYCAAARrAIAACHIAgACnJICAADGWgIAAA+UAgAOlSYA= Date: Sun, 7 Jul 2024 16:39:25 +0000 Message-ID: In-Reply-To: <32f04739-0cd0-4a9e-9419-c5a13c333c28@redhat.com> Accept-Language: en-US, en-GB Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-exchange-messagesentrepresentingtype: 1 x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [106.210.248.234] Content-Type: text/plain; charset="iso-8859-1" Content-ID: <922FEAA4C15BB442800D27E7BD284862@scsc.local> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrPKsWRmVeSWpSXmKPExsWy7djP87r7j3SlGSxbYWDx+a6QxZz1a9gs /u89xmjxdf0vZounn/pYLBb9Nra4vGsOm8W9Nf9ZLXp2T2W0WHBiMaNF4+f7jBa/fwAlTs6a zGIx++g9dgc+jzXz1jB67Jx1l91jwaZSj5Yjb1k9Nq/Q8li85yWTx6ZPk9g9Tsz4zeKx86Gl R2/zOzaP9/uusnl83iQXwBPFZZOSmpNZllqkb5fAldG3YAZrwVf5irenbzA1MH6X6GLk5JAQ MJH403+YqYuRi0NIYAWjxNdDP1khnC+MEvNaZ7BAOJ8ZJRpPTWaCafl77gNUYjmjxIW1S9ng qtb+3cgM4ZxmlPh87SYj3OR3R3+xgPSzCWhK7Du5iR3EFhHQkNjUtgGsg1mgh1XibsNrRpCE sICDxPV5/VBFjhIL55xkg7CTJO78eQ52CIuAisTd+0fA4rwCvhKrz70Hsjk4OAXsJM6eCwQJ MwrISjxa+QtsDLOAuMStJ/OhfhCUWDR7DzOELSbxb9dDNghbR+Ls9SeMELaBxNal+1hARkoI KEv8XyIOMUZP4sbUKWwQtqXE0b5JzBC2tsSyha+ZIa4RlDg58wk4iCQEtnJJrOv6BbXXReLV rltQu4QlXh3fwj6BUWcWkvNmIdkxC8mOWUh2zEKyYwEj6ypG8dTS4tz01GKjvNRyveLE3OLS vHS95PzcTYzANHn63/EvOxiXv/qod4iRiYPxEKMEB7OSCO/px+1pQrwpiZVVqUX58UWlOanF hxilOViUxHlVU+RThQTSE0tSs1NTC1KLYLJMHJxSDUzOPqf+1Uu29d6O+KJ050YH2633i11N 7wc8ljjfKXVnjf//ZxbvfsYfv6NUpCuy6qDADs5XSqkr8yMPKpzdI8CQrnN6s9KcXw/uc0Z2 Pb894T/LvUDWIuftTwx9Vyleil+hsUxhz5uXm7mPb99qI6XTfV2NSbQ3oONwvffzf65hm0yE PDzvMO7YatEVmMRosETw5GbWw2Zdat9fyebpxZ0+4Re+belqgROXH5fOjdCec+jow6+8zWIG Ja4nTv3eVR0hem3ybPelcSeO/Q890MKWrDj1eo+iQGiXbaW6RKUJ04G079obChTLnkfl9TSy XLcI4b608M0Suw8hrle275nw+nNnr3hO54RrEtqct84qsRRnJBpqMRcVJwIAqUp1KQIEAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrOKsWRmVeSWpSXmKPExsVy+t/xe7r7jnSlGaw4qm7x+a6QxZz1a9gs /u89xmjxdf0vZounn/pYLBb9Nra4vGsOm8W9Nf9ZLXp2T2W0WHBiMaNF4+f7jBa/fwAlTs6a zGIx++g9dgc+jzXz1jB67Jx1l91jwaZSj5Yjb1k9Nq/Q8li85yWTx6ZPk9g9Tsz4zeKx86Gl R2/zOzaP9/uusnl83iQXwBOlZ1OUX1qSqpCRX1xiqxRtaGGkZ2hpoWdkYqlnaGwea2VkqqRv Z5OSmpNZllqkb5egl9G3YAZrwVf5irenbzA1MH6X6GLk5JAQMJH4e+4DSxcjF4eQwFJGiflX njFDJGQkNn65ygphC0v8udbFBlH0kVFi54HrzBDOaUaJew83s0M4Kxglbj59zQjSwiagKbHv 5CZ2EFtEQENiU9sGsA5mgR5WibsNEEXCAg4S1+f1QxU5Siycc5INwk6SuPPnOROIzSKgInH3 /hGwOK+Ar8Tqc++h7vjALLHo6Fugyzk4OAXsJM6eCwSpYRSQlXi08hfYTGYBcYlbT+YzQfwg ILFkz3mo30QlXj7+B/WbjsTZ608YIWwDia1L94GNlBBQlvi/RBxijJ7EjalT2CBsS4mjfZOY IWxtiWULXzNDnCYocXLmE5YJjDKzkGyehaR9FpL2WUjaZyFpX8DIuopRJLW0ODc9t9hIrzgx t7g0L10vOT93EyMwBW479nPLDsaVrz7qHWJk4mA8xCjBwawkwnv6cXuaEG9KYmVValF+fFFp TmrxIUZTYNBNZJYSTc4HJuG8knhDMwNTQxMzSwNTSzNjJXFez4KORCGB9MSS1OzU1ILUIpg+ Jg5OqQYm1qP/kz6umeqaXmm8OvkC93ydnu1xQs2mJjJhnqu31CkrPVy/Iz6ufC1DHBvvAZ3P 1v+Wvnt3eknKfxWTkvn5k6/3dHv6J93MWTPDzeb9iTf/Q5ke/Tp4qGadnLLZ7fy3xqujPlX4 btNlvejvwXp90pQ24esN5ktPyh5uZWK4sGbH3ObDP1U9mrQvaCrWnd4vkHXAWPPpudPX3n84 cvCkaKlb41Wlc8IHnkeJHV56QYYnzVpQ+Jz7vIhSZ7/5bxc4mUTzCshXGR4vldTccpqxc9LL rSnvjt/9YlaV8/9ViaPrxKDov/EdNzfcsbyVYNgU0qmR++xtnc2yq5bTHnl/uXvuzJnq15yZ pTPnXWGyVmIpzkg01GIuKk4EABR0DqEKBAAA X-CMS-MailID: 20240707163926eucas1p10cba583480dcbb91b37796353d378473 X-Msg-Generator: CA X-RootMTR: 20240705085911eucas1p17f1e79c871c6290b426737ca1738e529 X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20240705085911eucas1p17f1e79c871c6290b426737ca1738e529 References: <27beaa0e-697e-4e30-9ac6-5de22228aec1@redhat.com> <6d4c0191-18a9-4c8f-8814-d4775557383e@redhat.com> <32f04739-0cd0-4a9e-9419-c5a13c333c28@redhat.com> X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 923D0180017 X-Stat-Signature: kxa7ye685sads9ck3o3q6cgbfidk78pw X-HE-Tag: 1720370370-761065 X-HE-Meta: U2FsdGVkX1+ZYgfHS9n7XbXuCmQ7Zpxh4KIaf+kRHkg9CdQYR71IqdWIwIwmoN5acfvyFTaxLnWlgzK8oZOXWdB/o+BFqqNjgWmG2uoPzgtP0kTOQ32HR01MWp7TS3+EwASAjNHKJQ7ctQnGmWoF2jqphXbXMyLlGYFbKlE++U4QZWSaWHOw5vGdt0cMQM76srZWbjUpigJ28wZytJluB7unkNJ3MD3uVjk+mRCfb40P8FPZquZ8Md9gf2vKLr29eXX4ucAm+6lwcdn2MXwpL16DHawsYxpv1bUuMXPU1Ad7gnXEGZlox+42nm06MjJqhN2RHkWdOgB5Zr/70CegjM62hHZ1KxxqmzPzgkPjgmV7LeyNhMMZW6gfvsEffzELff810gwT1JrB4yJDmFYgS8l/bNGZvyQQuT5J2wfixS9bsdm1RnS1OfCfhfiB7jT6SqN2BFfcOD04zBa9Hu3XZ6x6vJu6W/1CWTHzYNUWwcNtNxOWPXC4Zl4YpQV7KcvS+TFM6kytLr8wqi+T4jpp3rKMhSO4LP+qjqi69IdWV7aTrnq9LwaB+NSVXEuQACn9LMOlOfnaFJbO6GEQjpmeQlqEslQzIkMRjdD3aXjUR4/Fqj0yt+JqCRRPu1s0EPNRKod6ZwgxHNj3tCNyZCKyTzPe69Zqq9QM/6UjSOIjMXR7+5tieqXF1+yN0rMVOZ6jVv9+mxIuk2jjzD+gb6iXyc8E3NsNHYEml3JZ3qKSh++QjSGaoV4yP1ZF7yEW/IhPREaSSWuSGXAGAtoASvjHBpvlqM7LwgbdpJiAEJA346KSKpGQ0ZcaDE2EkT9iXUnQkgMq1Ot6TeTuYcp+TpA0wM1lhOStJ9fo6DpDIruLpw+Ctn6kSxLak5//hna+kq/e3UlcL5mmQ5y3jVG0wRuepW3V+3JBqVQCqFi4HQTVtxl2OJjEY5o9EFzelL1wKl5tzMlSdKfQpybVmrHonvS 7LxnuoxD RSVDzSLzFkHp6kSoY0Nb3p3EtPu+h/VoM2LL118nlQ1s4jDPhuzSB5Cc/EJgkMGXeFtXLBWdULE+UrEk1bxk5QkSZtbwqL4Pdd3uxPM+spNBCbJz4PtD2s3mEWJS49j4N/oPLFCGf/sUwkV5LQONKOt7hT9mwQ7401uakIv0QGsRPkuuJ/0R97xwUrZFjj4tjN+8x X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Jul 05, 2024 at 10:59:02AM GMT, David Hildenbrand wrote: > On 05.07.24 10:45, Ryan Roberts wrote: > > On 05/07/2024 06:47, Baolin Wang wrote: > > >=20 > > >=20 > > > On 2024/7/5 03:49, Matthew Wilcox wrote: > > > > On Thu, Jul 04, 2024 at 09:19:10PM +0200, David Hildenbrand wrote: > > > > > On 04.07.24 21:03, David Hildenbrand wrote: > > > > > > > shmem has two uses: > > > > > > >=20 > > > > > > > =A0=A0=A0 - MAP_ANONYMOUS | MAP_SHARED (this patch set) > > > > > > > =A0=A0=A0 - tmpfs > > > > > > >=20 > > > > > > > For the second use case we don't want controls *at all*, we w= ant the > > > > > > > same heiristics used for all other filesystems to apply to tm= pfs. > > > > > >=20 > > > > > > As discussed in the MM meeting, Hugh had a different opinion on= that. > > > > >=20 > > > > > FWIW, I just recalled that I wrote a quick summary: > > > > >=20 > > > > > https://lkml.kernel.org/r/f1783ff0-65bd-4b2b-8952-52b6822a0835@re= dhat.com > > > > >=20 > > > > > I believe the meetings are recorded as well, but never looked at = recordings. > > > >=20 > > > > That's not what I understood Hugh to mean.=A0 To me, it seemed that= Hugh > > > > was expressing an opinion on using shmem as shmem, not as using it = as > > > > tmpfs. > > > >=20 > > > > If I misunderstood Hugh, well, I still disagree.=A0 We should not h= ave > > > > separate controls for this.=A0 tmpfs is just not that special. > >=20 > > I wasn't at the meeting that's being referred to, but I thought we prev= iously > > agreed that tmpfs *is* special because in some configurations its not b= acked by > > swap so is locked in ram? >=20 > There are multiple things to that, like: >=20 > * Machines only having limited/no swap configured > * tmpfs can be configured to never go to swap > * memfd/tmpfs files getting used purely for mmap(): there is no real > difference to MAP_ANON|MAP_SHARE besides the processes we share that > memory with. >=20 > Especially when it comes to memory waste concerns and access behavior in > some cases, tmpfs behaved much more like anonymous memory. But there are = for > sure other use cases where tmpfs is not that special. Having controls to select the allowable folio order allocations for tmpfs does not address any of these issues. The suggested filesystem approach [1] involves allocating orders in larger chunks, but always the same size you would allocate when using order-0 folios. So, it's a conservative approach. Using mTHP knobs in tmpfs would cause: * Over allocation when using mTHP and/ord THP under the 'always' flag. * Allocate in bigger chunks in a non optimal way, when not all mTHP and THP orders are enabled. * Operate in a similar manner as in [1] when all mTHP and THP orders are enabled and 'within_size' flag is used (assuming we use patch 11 from [1]). [1] Last 3 patches of these series: https://lore.kernel.org/all/20240515055719.32577-1-da.gomez@samsung.com/ My understanding of why mTHP was preferred is to raise awareness in user space and allow tmpfs mounts used at boot time to operate in 'safe' mode (no large folios). Does it make more sense to have a large folios enable flag to control order allocation as in [1], instead of every single order possible? >=20 > My opinion is that we need to let people configure orders (if you feel li= ke > it, configure all), but *select* the order to allocate based on readahead > information -- in contrast to anonymous memory where we start at the high= est > order and don't have readahead information available. >=20 > Maybe we need different "order allcoation" logic for read/write vs. fault= , > not sure. I would suggest [1] the file size of the write for the write and fallocate paths. But when does make sense to use readahead information? Maybe when swap is involved? >=20 > But I don't maintain that code, so I can only give stupid suggestions and > repeat what I understood from the meeting with Hugh and Kirill :) >=20 > --=20 > Cheers, >=20 > David / dhildenb > =