From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3E5BCE7A88 for ; Sat, 23 Sep 2023 00:33:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 11EF56B0276; Fri, 22 Sep 2023 20:33:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0A8A16B0280; Fri, 22 Sep 2023 20:33:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E639C6B0284; Fri, 22 Sep 2023 20:33:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id D16706B0276 for ; Fri, 22 Sep 2023 20:33:50 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 906A3B4686 for ; Sat, 23 Sep 2023 00:33:50 +0000 (UTC) X-FDA: 81265989420.13.31B820F Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2047.outbound.protection.outlook.com [40.107.243.47]) by imf24.hostedemail.com (Postfix) with ESMTP id 51398180008 for ; Sat, 23 Sep 2023 00:33:47 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=lnbGRWSp; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf24.hostedemail.com: domain of jhubbard@nvidia.com designates 40.107.243.47 as permitted sender) smtp.mailfrom=jhubbard@nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695429227; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WXKmSZX6ZNeGk/aS6wxcxB/ExCea6Rrip+lZo3W9F2g=; b=LNhetKI+53icCqQPnz/wq+QkRasZPpNXga/ib7wDAFJwZ/6qjlK6minbnrnluTBBGcAaBh 9f7nxpMuGY+t6uY/zntaI56ITnfZCzMxR/0YKcZ/2+Z4uDkWcYd2OCMYhBNbR/XLyVx0m7 RlSv6sfLzvV2CP/xUoFyoLKSFKWAY6U= ARC-Authentication-Results: i=2; imf24.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=lnbGRWSp; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf24.hostedemail.com: domain of jhubbard@nvidia.com designates 40.107.243.47 as permitted sender) smtp.mailfrom=jhubbard@nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1695429227; a=rsa-sha256; cv=pass; b=o86M6sAykx/1bjIbqr6pKpcxzhkU+/XO6hLdfYpgLRMnFdN6Np6Vv0EEsPRcfxAD7JHA// qEBk7HbCF9sBQfwlA8cbBy3xxM/oi/7ZMorzSjoFD1CujKDUqSvUdpsoJJ0+gE/qFfvIpA yPEOIv3GbwfjVwMIEDwhBY9lt5n3Luc= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lMlQ5+9mUICV+XC1qCh4R79O/ezJj0o6vZxDE35Dujw4I09Cct4kH8XnvFZs4d43ZOKfaVsvRWAzR+KOvcsXffHVtTsVsSMSR25NDAnsZphEXEKkPsL3JHj2TMm3XfJfmg20stCQs2SOXvT1uMp9h9sYSjPN7tNec5osYr9AwfVw+0+toren6NNvp8OmsRKSP3aztGC1OqcjqxIDhZWV1KI2S0V1CR/Pe9Zl+n1Ifc7QXrACi159qPEt7QyyZAwH0Yt9LttHp19pdx/0b6odIEbwUkLPZeUbHlfyx2G43K5w8P/bh7qj8cc8G6xfEJ7aJqPPKw/mJ2Vj7lXd7Y81yQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=WXKmSZX6ZNeGk/aS6wxcxB/ExCea6Rrip+lZo3W9F2g=; b=bkQafg0o/j/k2rdQ9S6h8jRTFiCCJKfgw2QBQZK4FqLa0uVxPK1TfhRlt+6ZCeBuA7N42AOIDcOTTA1nfcpwV1aRk6scMgp+Geop15hnsbEalPTVhm8tsdHbVi6yxcPQnAQYeEIRU2v0RKrQx+nK/HgCR7W7yT9jzyGSo4XaCPuoZcxLOlyigH+qpCO1meR1L18xgOH4mlSO6Zbai3XTJWOoB7HlIi8QWo4csWQn+1VvV+Id69CL5XDTk9mv7HbDxbz+pCkWeq9EMIo3dlUf6/QNPgOR+S9KIzYqGi5KzGTNu1XE0spyuhwdMoUpNqKETLOg3H3b6qPp9Ku6tAb4Yg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=arm.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WXKmSZX6ZNeGk/aS6wxcxB/ExCea6Rrip+lZo3W9F2g=; b=lnbGRWSpmgwAbFM6DKaA3PCa1FhgtCOj2jNavQnBMKEGCZu7DlpGOr1AFNgkVGaWuD3rfxA1JjYNZmjF0G49THap9M0Pc/RP0O0r4VaWt8RPRS0wHL/+DyreEpgbt0+AsgeDvnSRwhk7VY/HswFcrCdIYayU8Mtp5Wd52Cn4drzs1qyOwkjUC4TCdwEvW6XCE7fYuTOiGsgCTjN/lOwm1lClB7J4cNRDw02u/8UU28ciUHvaoV2JE9tVYhrdLh08PmI+X9tsnuCkz2DiL5ExUBQS1MKW8nF+U5kieIOe273zkNoXW+Lk8DeGoIibtIPPBgEqO3KXzF/jpv8Pab7yIQ== Received: from BY5PR20CA0009.namprd20.prod.outlook.com (2603:10b6:a03:1f4::22) by PH7PR12MB6695.namprd12.prod.outlook.com (2603:10b6:510:1b2::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6813.23; Sat, 23 Sep 2023 00:33:44 +0000 Received: from MWH0EPF000971E3.namprd02.prod.outlook.com (2603:10b6:a03:1f4:cafe::95) by BY5PR20CA0009.outlook.office365.com (2603:10b6:a03:1f4::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.34 via Frontend Transport; Sat, 23 Sep 2023 00:33:44 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by MWH0EPF000971E3.mail.protection.outlook.com (10.167.243.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6838.14 via Frontend Transport; Sat, 23 Sep 2023 00:33:44 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Fri, 22 Sep 2023 17:33:29 -0700 Received: from [10.110.48.28] (10.126.230.35) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Fri, 22 Sep 2023 17:33:28 -0700 Message-ID: <7301771f-d654-4e5a-a197-3a3d8750440c@nvidia.com> Date: Fri, 22 Sep 2023 17:33:28 -0700 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: ANON_LARGE_FOLIOS meeting follow-up & refined proposal Content-Language: en-US To: Ryan Roberts , Matthew Wilcox , Yang Shi , "Yin, Fengwei" , "Yu Zhao" , Zi Yan , David Hildenbrand , David Rientjes , Andrew Morton , Vlastimil Babka , "Kirill A. Shutemov" CC: Linux-MM References: <4966f496-9f71-460c-b2ab-8661384ce626@arm.com> <4830fb3e-4a35-4842-98f4-9e7baa0e692a@arm.com> From: John Hubbard In-Reply-To: <4830fb3e-4a35-4842-98f4-9e7baa0e692a@arm.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [10.126.230.35] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail202.nvidia.com (10.129.68.7) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000971E3:EE_|PH7PR12MB6695:EE_ X-MS-Office365-Filtering-Correlation-Id: e7b77374-6694-498a-eb9d-08dbbbccbe5f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 90rLXc1pc592Jx4liPMeodiXWz5V5mnPAhqRYi5+kiIO3uHJ3c7N+oVO4+p1L4AIN+EZxyaOcqMYaL4ycOD/tiSc3yhzdcUp9ZiaLLm80MaV5SoRIwpotAA7MWdZdO1wC1qgXhYhDi0RQsHVSpeVPVOLBwSQTkZzNx0/mxIhJD1VPTnKTJrRVhAozXUHoRqvVAh3+1+yaDx1jnHoSo8DsYtbsYay6nrNXXJbDLbpnrQ+c6XzxvhAptfWmGk9roTpVEzrtJFaR0UrLZSyCaWJG7mugIrye9m1XTRhK3NZhbc7faUC/9lxISsGavBNVVvyTjdvB4QezHTj6fGGh/qRc8M3hn/akifA3qm3GB0A7PZVDQrS48jh+ZcqaQTcGuwtINwug8U2XdAymPrn+n31vaxKeIMPF1yjh6vH9VB+y//5xMFdi8B2LO3QyP4XVfH8Z9jcouz2B01m7PpdUiGn6J/AThkm0Ab5NRzab8gE1DvDztD/oHy/PgljEFY7U8eBOCkN4efput0ofy2PAkpA/s7pTRftsbLSncO889kRVfjix10aExRuZ2TViTQ2gVEaWFBeQaaRPRssTXxZpgdXLoeVmy6UjpdO0TJkvCIyFXRMyu3MlWDMl4HB721sEtuJM2H29b6b71cDowtXzrMQNgwy/Um7mUro6SagcScj8zhK6Wz2BLvjKJFonCfpYomoUcJiN/lxLi0wom/Onb9NkbBotVi5Bn5b85Q+5lMFtVN86UgGL0Pkg9Jua04yRr/igr1HVvmMNvl1S+kyOQcBg3gkrGpj4S/Xh+FpWg6mrZlFEZ+QtVMvEeKS9ZcGKjoH X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(136003)(376002)(396003)(39860400002)(346002)(230922051799002)(82310400011)(1800799009)(186009)(451199024)(36840700001)(46966006)(40470700004)(7416002)(40460700003)(336012)(41300700001)(356005)(82740400003)(7636003)(921005)(36756003)(478600001)(2906002)(2616005)(31696002)(86362001)(47076005)(426003)(26005)(53546011)(83380400001)(316002)(16526019)(36860700001)(16576012)(70586007)(70206006)(40480700001)(3613699003)(4326008)(110136005)(31686004)(8676002)(8936002)(5660300002)(43740500002);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Sep 2023 00:33:44.3955 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e7b77374-6694-498a-eb9d-08dbbbccbe5f X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000971E3.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6695 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 51398180008 X-Stat-Signature: 7oz9zr1xmgtnj1u91u88jemngk9kd7ao X-Rspam-User: X-HE-Tag: 1695429227-605261 X-HE-Meta: U2FsdGVkX18MDUDKC4Bx5SGzWCPjn4LtGP0PvpivRhgQWJxatKXgLDqwKS37HwTDzL9QvC55U+AXJ039aQ1atYa7r5UMk/GrPeU3BIJXO0uy7nMyCfaYC5q4VJJuZb424B4dGvr1py3tgZoLNdyBiZJepiiBq2wR1e1+SRGiLmtQxMw2Qw0WgjUaRSBsooPpmxOgWDJ+J7AuYCqY3rKI7NJtjIeFsEhTYXIuGEbCkb9dLZ8A/g9E77dDgkG7uagfxMBuyLjRKOKvjU1rsr3bQN59hHXZ3MJoNSRGK4cmUx2xt9Dkz9nlfvay155gnB1/yr6YARVU4tPQX0i0fKhzENvxWDc3qT3jwUS7oYSe7ycDjayVz++YxUSnqnr0fR2mfDowxNFHEDZqAEIIKAIQCC+CqmqxqyEyYHFNZqwWe3cEeWRtwLioYbVoCTsZ8hOg6yKmdhSEeoIXdjOX/v+kstTpLKuekCn0GlG8B4ijqpItNyGUg1GNc/+aQ/mzUb0cTJEAU6gDT70h2mh1OyTYEtMF0H13GAqhbA911gYAzODq7vbyws/xyBn5A2TY93L/yAXqYwEaXBmzG5qIRdonuxEdMV2comGNldLnOFk9x6J5goM5P+lVKv+/7tt1/7roX9JbQuZ9OoLttRUo63tz2vtBb3vdOdjvDbSPjprBOt/WczP5NI+pwU/XAdilryJ/8/VaM/s578tqzNMxbyd/BeivWb1t1NRx8SCzs6LzPBt1sVxPzQnKdE2u22lUpfRDq6eJ0lKOijp/0lyqnKspZgXUImUE9/rBAKinqESASS0bF7jM2AxvpdjcrKpjp+aSoACIWhmYsUFtFrSBapYNmko0148IMWEg7I9DIKedrnYFug7LLMcHE12M9bIKkR6n7cgUqYHUfxGSoQlA6gzqDv6SNjMhR3aYcrWnEOaPLJ0E4KMhznpA8Ioa1rvzv0N39UVYibXBoexFJDWmNuZ Q15TgNyy QkvNIc0VbG/qps39DSuRBKhDIPsFBnprpDKT7WScaandEI3n0NSylia1Q+pPstD7j89PnbviFvSRbLwIfnDaf19+CuM/dNM1EFRnac2PCJAJMwQc5cPWZOKrtK4bfAu9FBTXOcxWHbtzLNT+8wFixMNOTxl0X0v4GxqcZSsIWaA5z7xKvT6LnPU+DgBwrgrPVd5y9cSplhe6RX0a/LH64HbHV8RZe/kFWUsCVmU3Dou1NQtb85l8zK0JtxzpzrTi/6vM4/4d9idcA3dwzTQ6FCR2ldmZkGBCzUMSXVJChSs2P2Q7V9ijS0yiM+yXat1JgVv2q6z8G4vZEGbmBs5G3sKwFVwgyiojWngX/w3dADGiZyI35AV5XS1CeONohq5tIhM8XLfRBWx7z3vY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 9/22/23 08:48, Ryan Roberts wrote: ... > I never had any feedback on the below; I'm not sure if that means everyone is > happy or that nobody read it?? One can never really know: zero or more people read it, and of those, no one hated it enough to send out a quick NAK. So that's a *possible*, lukewarm endorsement of sorts. Success! :) ... > BUT I've had yet another idea on the controls front, which would enable exposing > this to user space as an extension to transparent_hugepage, while continuing to > support THP as is and also be able to control THP and ALF (anon large folio) The new ALF / ANON_LARGE_FOLIO naming looks good to me. The grep aspect is a nice touch. ... > Add 2 controls to sysfs: > > /sys/kernel/mm/transparent_hugepage/anon_orders > - bitfield where set bits are orders that will be tried during allocation > - defaults to 1< - For now, 1< - To enable ALF, set the appropriate lower bits > - To disable THP, clear 1< - (In future we could add an "auto" option too) > > /sys/kernel/mm/transparent_hugepage/anon_always_mask > - orders in (anon_orders & anon_always_mask) are not subject to madvise > - so when enabled=madvise, still try (anon_orders & anon_always_mask) orders > as if enabled=always > - defaults to 0 (all subject to madvise) > I *think* I like this a lot, although I have some clarifying question below. It seems to address the key things that have been complicating the discussions: the API is now looking more flexible, and yet still easy to understand and reason about. Nice. A couple of questions about how this works: > > The defaults for those controls give you "legacy THP". But you can modify the > controls to generate policies like this: > > For these tables, a small key or legend would help. I've forgotten already what "S" means, and am also vague about exactly what "THP>ALF>S" behavior means, too. > THP only - existing behaviour (default): > ---------------------------------------- > > anon_orders = 1< anon_always_mask = 0 > > thp prctl: | dis | ena | ena | ena All I see in the prctl(2) man page is PR_SET_THP_DISABLE, I don't see any _ENABLE. What does the above refer to? > thp sysfs: | X | never | madvise | always > ----------------------|-----------|-----------|-----------|------------- > no hint | S | S | S | THP>S > MADV_HUGEPAGE | S | S | THP>S | THP>S > MADV_NOHUGEPAGE | S | S | S | S > > ... > > It does have the disadvantage that ALF is tied to MADV_HUGEPAGE, whereas the Right, that is a little awkward. But maybe less so now, with this new proposal, which leaves THP a little closer to ALF. thanks, -- John Hubbard NVIDIA