From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F757C77B6F for ; Tue, 11 Apr 2023 15:36:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AB5B66B0072; Tue, 11 Apr 2023 11:36:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A65996B0075; Tue, 11 Apr 2023 11:36:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9064F900002; Tue, 11 Apr 2023 11:36:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 813C96B0072 for ; Tue, 11 Apr 2023 11:36:46 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id D2F27C0E48 for ; Tue, 11 Apr 2023 15:36:45 +0000 (UTC) X-FDA: 80669512770.27.FDADECF Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2055.outbound.protection.outlook.com [40.107.93.55]) by imf05.hostedemail.com (Postfix) with ESMTP id 088FE100018 for ; Tue, 11 Apr 2023 15:36:41 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=memverge.com header.s=selector2 header.b="h/ZI1P0x"; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=none (imf05.hostedemail.com: domain of gregory.price@memverge.com has no SPF policy when checking 40.107.93.55) smtp.mailfrom=gregory.price@memverge.com; dmarc=pass (policy=none) header.from=memverge.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681227402; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GHzH2aHTThc6v/+yQWnbJKWfDBGQ2HUc47bOVknAe0c=; b=4nAyxAXJwUEaIdaAVxRFfmF2hA0z2E5AqcDdHbhtVR0rSmKWMmXzAJimqF7OmyKwART48G wvahzRJZv4qxcyaBRxeHxz6bym9QR+M7LNmzTTdhX6ie0w6vpSfhX/fSRyNq8XuzP5kR1I b9zPyCsgJssWEjt/Ds/+qpcsY3LS8sE= ARC-Authentication-Results: i=2; imf05.hostedemail.com; dkim=pass header.d=memverge.com header.s=selector2 header.b="h/ZI1P0x"; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=none (imf05.hostedemail.com: domain of gregory.price@memverge.com has no SPF policy when checking 40.107.93.55) smtp.mailfrom=gregory.price@memverge.com; dmarc=pass (policy=none) header.from=memverge.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1681227402; a=rsa-sha256; cv=pass; b=UyrClRHhEmFM5F7wdcuVz7OCZFFw+00wcSIpvkyCANTlWbto1nIJ4Kmi2OJX41IuSKs44E rKOEbZ9BTU/xQOVDjdTRwK54ZVvMIOrWZFt3mYVelcIa+QK2IUv7mGPVdFvmK9yXwH7M8S m8YJcNPx61HdHHVF7a5ZXue7rApzWgU= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mq1vpSAd0rPKGpdbQbD7dtxfgwEGPm92l4CgPUqFV//Z7B7ANgjo7odywUfH3n0slWhPKipXqAwE9zq3mIFr0m5QqkSYKumTXCEAu3+yMvTohxB9N5so9tX75pTDsjQ3udLjGKw/97rpxxD9qfRzCswGcHKWwKAypKckliEBvJRRZx7rReuSN1W88Ofn6DsOwTiJ1m0z21CkLCuS6517btH6R3qbddB6M1HgTo7ONbnP5SCiwpmgjg22bMB1yLszYcHXEtOga+SWY2MlZS2z6Y5YpANVqnZkgBaa3I02MJLMpeo+Z2MegiOJjf7ClEp487UHJC437BkJRt+PXfhw0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=GHzH2aHTThc6v/+yQWnbJKWfDBGQ2HUc47bOVknAe0c=; b=F53s/jmH1M0ho8B6CYL65VWvB+vTK2K7L5tsx12CBzDdPMKNisOlUlqE2bnaa+C8HZavd/D3hZNoyB5xlzIJcVf/4kQbQJj7sJV1llthUVSXHX3kyeyxtqLChE2R63Tw38l9qzwH9u1M90lQevB+KxcUJjX0KoPcAVo1VWpHoWjbXUnkZatM7z3Q0BcOgtWzCHA2qbSpiKFoE0D45fz6yEfNtG8wabje8TNY/te7o9fMcbi9BbZt4VZqttDrn1+CyQR3lZ9xeec/XraDp9a50S1UnNB+FphBFiMuyrusvfi+vUj821UwVrIMkF3/s62pGlM6v/0/hlAABSi39QIAug== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=memverge.com; dmarc=pass action=none header.from=memverge.com; dkim=pass header.d=memverge.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=memverge.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GHzH2aHTThc6v/+yQWnbJKWfDBGQ2HUc47bOVknAe0c=; b=h/ZI1P0xq1RUksin5juSyDnr0Z+VTc3EOK38KL+bcZXY5dgfGX+iZnxxy+d2SsAAqwQNQzRmyKxI3reWkzoYGxR04Q/kF1T6nzZiggFINItGoQjeZhsHQ2fYokWbEmVsUy51yQ+JEvHcEhmUkgjukz/RT2OhjCqSmcy8IBpeIlA= Received: from SJ0PR17MB5512.namprd17.prod.outlook.com (2603:10b6:a03:394::19) by PH0PR17MB4640.namprd17.prod.outlook.com (2603:10b6:510:84::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6277.36; Tue, 11 Apr 2023 15:36:38 +0000 Received: from SJ0PR17MB5512.namprd17.prod.outlook.com ([fe80::7b97:62c3:4602:b47a]) by SJ0PR17MB5512.namprd17.prod.outlook.com ([fe80::7b97:62c3:4602:b47a%6]) with mapi id 15.20.6277.036; Tue, 11 Apr 2023 15:36:38 +0000 Date: Tue, 11 Apr 2023 11:36:31 -0400 From: Gregory Price To: "Huang, Ying" Cc: Dragan Stancevic , lsf-pc@lists.linux-foundation.org, nil-migration@lists.linux.dev, linux-cxl@vger.kernel.org, linux-mm@kvack.org Subject: Re: [LSF/MM/BPF TOPIC] BoF VM =?utf-8?Q?li?= =?utf-8?Q?ve_migration_over_CXL_memory=E2=80=8B?= Message-ID: References: <5d1156eb-02ae-a6cc-54bb-db3df3ca0597@stancevic.com> <87v8i22abl.fsf@yhuang6-desk2.ccr.corp.intel.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87v8i22abl.fsf@yhuang6-desk2.ccr.corp.intel.com> X-ClientProxiedBy: SJ0PR05CA0072.namprd05.prod.outlook.com (2603:10b6:a03:332::17) To SJ0PR17MB5512.namprd17.prod.outlook.com (2603:10b6:a03:394::19) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR17MB5512:EE_|PH0PR17MB4640:EE_ X-MS-Office365-Filtering-Correlation-Id: 3c0f68b9-034e-4df2-ec37-08db3aa28a4e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: p1Ao6ZEpiTcTN0Yyu5m0H1SnGwjLT5GPkcIiXMNCYaacRu5Rj2y5RfsRL1kWgOoM1v3f0fAos0c6x6I21ePV3OfI9szI6AAtSVuc1C2IQnfB2oL5Z1Ji4lcZw5wQqMkI12F+J/KtVLo3eFMrxwF3WruCrGar04BQ+OohRfI9430fIGO5KTEzy9PvPkl42eqSUdu2CesvSSPmgtsOx+n2bcDpIzLCVy7RV551F67AQZj8mun6q7lF1scNi+MCFwfL9/L/w6KQrnwa9jPTcIA0b/2CQirL26yHPZuTc6joV8K/rbs5mQaurEWKDP2pN4UkPU5oGmWp3wQPn+Wd9rIydcb1tvHZfAORcFTBrLM2dCvOSSJUEwcne0irj383cLbD5E1L4VY5CVQ4t7nF7g/Lh5uvQNi7GgC+A+UDG3SVwyXXN+L5RIpXZOCzPDcKh1iR6lXkpV7AkfVber5dpYoPep/TjCyyt2jTWufgcimgkaUOKI8gZSUNyZPVzDM4J+/YiCQhnSKRGavXzxGrRvJjYUdNsVDja0MHr0j/k2AfT3G08nEdv64ecYIOqtHerozI X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ0PR17MB5512.namprd17.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(376002)(39840400004)(366004)(136003)(396003)(346002)(451199021)(478600001)(36756003)(86362001)(38100700002)(2616005)(6486002)(2906002)(316002)(6506007)(44832011)(6512007)(26005)(186003)(66476007)(66556008)(8936002)(6666004)(6916009)(41300700001)(5660300002)(66946007)(4326008);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?iPIeO+nJbiuj4mqM6HADFn3D94GstXvw58lxYrngUOZ79wdHedqg2np5mBzp?= =?us-ascii?Q?bA2IgfUWWCjbAAhiRDDtzi5vAeIj1RGc5VUqYmINBaPJ3ZmhGZrARaheAbM8?= =?us-ascii?Q?+XqMpMqpdPg/ra/y6UUNm9janFhWWy2AO2eiT6emOLIwgTyOW+QvyTo9BSzV?= =?us-ascii?Q?jCLn0nVkSmbq5ah4bg8xSxa4nMSfl7eAc1zvEq0h/It8rPZr2RvDC8IpmqNT?= =?us-ascii?Q?PvTxr6jN7DrKhMsWK/clo1a0MUn+a6wTDGOoyj3QMKBH87knNF2PqCtik1pg?= =?us-ascii?Q?pgtAOK4IFF1ZkOwfrv32CZXrtMpAVewfm8n+GpAn23YWWQfka7jWU+Od2eyb?= =?us-ascii?Q?N7LBWEq3JEcPdUhA8msnLaYc2PhhLEJeY60IgyZxsx0BxOO9pM0ByYvwcAnM?= =?us-ascii?Q?0mT8fAf0eXYEiWxTyDT1/phvH1bTrFejLmTUmeO7k/6LPP/3O2hxLW/G3bL5?= =?us-ascii?Q?fc7KYTSVJKH4uVXggelnWA0QhxgIDgQyumPjFD0YLLSHlGYebxH0pAZk2m5h?= =?us-ascii?Q?SRE9aAOxBJBqg/G0ADZVTbCat797WcWhMV5XMG9zT4Dg1IMawIOmTQ8hY2uQ?= =?us-ascii?Q?CXgExBUP1Rm+Raku4HU2JDxHXBvMyGy3oRtS2sYQPtYITS5tieZOV13iSdai?= =?us-ascii?Q?GqpG3s+vHeShRiJfhsGLGmb+yByPWP8MOeHkUqmlY6yGqy7sS+KLDMeI0Ca9?= =?us-ascii?Q?uGHnkkJbmKjTIbDfhtvZQFJFT1rAZTTMsj6DpgH4JiqKYABFvET74vHbV6+N?= =?us-ascii?Q?XWuS3VMGwXnCjMxA+pNZNLMJkLX3sfiWmbiMvK4toxmYiGVQGSNFk3sVRdvX?= =?us-ascii?Q?bktDl1W78kG3uo/yIxsSeurEgvZoNEzjCwMJzgj7f22T22e8DNnfE97oLPuR?= =?us-ascii?Q?Evb0FKxNQuc2+hGqEeUdwlGLueolgo7i8peDxr21S4s8YyEGWvVYWCBnDWFY?= =?us-ascii?Q?Giwr0rCPv8KG12PzWRp+nkfmblrc+3snFYEH2UMIY6/HLgAyriTDSkHF98n4?= =?us-ascii?Q?i9cjcNJ11fMnVRH/PoBDwEF0mKT1y+Tl6oSK66HepPWt6SWrUqbtlxcYj4KY?= =?us-ascii?Q?ZnxAFha+HOoSYR20x3KBMWdNmVLcyCwrVSM6v8vMDZWNpW6CWllSOZgbbXy3?= =?us-ascii?Q?CoPEqw9L51I8j1uQajBmr0YN/s+WiUG/N8djs55fAbdBCf22rU1VR2vPOxs5?= =?us-ascii?Q?csJI+HjaE0SbgPRpNoBhL84HOpvIKs4jmcXAkRV2m0BqTw4i0wJ4mvQWX8vX?= =?us-ascii?Q?J+2qiu4ZXeEcOPnctQrVj8mop3E7f6fpvrnoVbit3GTkuwo1YC2K8ZadplJe?= =?us-ascii?Q?lcj5ey3nx4SkKtSjwdRccdcOVdmJRSigeDRQuJ28sBBpn+k19SLmcLeW7cFO?= =?us-ascii?Q?gbCmzJuApZ9ef0Z2b81/9U5gjWTOmBRwDFRzfagwhWevigY9BPrK+sZdnAS2?= =?us-ascii?Q?hDVpSQNrTNAgDhRT0jVuvfp9AEd07FSgi4EnllnWiBjYNFmNieidzET3X4jE?= =?us-ascii?Q?Mzd6nYad7QxFfsGuhg8l0DqnUfDCEwdGJadX7YOoSsUY/D2Xk2e2uJkoZCx3?= =?us-ascii?Q?Tl6Qidi0uT5DInQtC0zq4I/18nmADLI5WdOAaJvT+nfJ9fXoPGpD8Avq+klg?= =?us-ascii?Q?Jw=3D=3D?= X-OriginatorOrg: memverge.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3c0f68b9-034e-4df2-ec37-08db3aa28a4e X-MS-Exchange-CrossTenant-AuthSource: SJ0PR17MB5512.namprd17.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Apr 2023 15:36:38.4968 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 5c90cb59-37e7-4c81-9c07-00473d5fb682 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: lEa70hyIq7TlKlhdB3idHOUM4p4nVv5PCrH3VYKdA0QHZPa4AR0KR6GxhcWiNbKZo6rJmQdJBU7BJSdaq/MdLZ00m/vjicWChjC7F3eErbg= X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR17MB4640 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 088FE100018 X-Rspam-User: X-Stat-Signature: 9mwe4rm1tnocxt5h4cyzctpripecjg6t X-HE-Tag: 1681227401-253521 X-HE-Meta: U2FsdGVkX1/Yv2VdR+8jibMXO8TiQqkJrmatDPZZ3LgfSReEYvBLPjf/mBd3wNNiKfW/+l6Kl7lt/YmoCThNWVpVxqkC3xFhHnxfLgQEE8k1yK/CrdVFMI6r9BBfOYWtu8hlp8y1EI1skN137Fuv8ieChFWvRpwGRjZbbecGTLJfwnJp64ISKxwglwzTX4wuiZ/Y41ml6/K1SMD3uJNZYchF1U2PrZSoDDi5Y4q48SnW2aD0icmqpfDbXmZg98FXdg4R2RqOlVN0cnSdXvipY6llXrvOy9z19O18XZ4aYinGAlJzQLh5i9E7Pv2kTUgjPh6RzynoKvryTXoAqHLkL6X9sYFY4B8oO2SdhPyWUIjC7qJUme39kCf9+hIjlm36CXMJzJ6O1Iz3dtcTaA8MlkpkOnvhzmQWBoSfWHQ1qtVc5KioyhO8+s0f8eIO/vueXh2tDSrWzzJB3QQuFs8zIHiy8BF57uNTjzxOK3YPL1L6m4gzbPUQ8OvD7ecO3qQY/2m6Ils0Evvq7pEpYyOCvswbEaBhkS5b5gFTwlRIeIq19HR2hd2Pxj7OlaPml1uy189MkklOx2+A5GMU2oPh9Lye1j5AeOGOYLlFtlw9rr8tRGYFObJu7U04a1qKLXfqYYfDNvwra9mZDN1byRTQ9AwFd2xXetU5Vwi4HhJKcjO/oCqfF7CFCR4msxHs2jdyp73zE2T4iWuZJxd2m6CRiZsQdIQWZSlPxXxFoIqOKJUPSH7wUkIDl05EOqhcVyNE4Pev7wc08Xh4DUS/UDFvy9CbjcA72b1ffp+nqUVucfol0qWmu3h0SNF3tVkTkKyKqkzwKXf+4GzpvXFDNXjOiQOKwT+BNHnsY69Ulpdtqo9if4/GLgkAZHzX5CthTeNgUCUHZBxk1+Q6uQmGX5UM2zhPLhqSk0k1e4vzlaggLePIhZcjX4GNyUA344ajuXp6FPWsHW7sgImgKeLAgl8 5yD3/vFq zdM2h7wmshO+SIm0PICANRIPtabO1aU958+1c7IKwuiaueCq9/XlOmR/qGbBWJnlEJP5x4O39r99TSG0durlCkP4Kx5Y63H9JNziZV80IRav1UGeMRCGK7688BWFB6CCX5ygmz4F5x5g6RZsGDFwH+XF9leKCMa1GGb44w3VNjVeSIejB8LpkMxhadI4eg3pgEL98114DQ+xvw8zLAfEvVS8cACAj6Bv3hPWcY6MTiPbcaVJqKyNVCeCYmwppfF4xLWYD1UNO1ELyDgX40fcV4x5HrCiGW2XAH7BaRJHyfNteKaSUCY6YyL7BU8ks6TM6qYsjpbD6t2EHuac= X-Bogosity: Ham, tests=bogofilter, spamicity=0.222501, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Apr 11, 2023 at 02:37:50PM +0800, Huang, Ying wrote: > Gregory Price writes: > > [snip] > > > 2. During the migration process, the memory needs to be forced not to be > > migrated to another node by other means (tiering software, swap, > > etc). The obvious way of doing this would be to migrate and > > temporarily pin the page... but going back to problem #1 we see that > > ZONE_MOVABLE and Pinning are mutually exclusive. So that's > > troublesome. > > Can we use memory policy (cpusets, mbind(), set_mempolicy(), etc.) to > avoid move pages out of CXL.mem node? Now, there are gaps in tiering, > but I think it is fixable. > > Best Regards, > Huang, Ying > > [snip] That feels like a hack/bodge rather than a proper solution to me. Maybe this is an affirmative argument for the creation of an EXMEM zone. Specifically to allow page pinning, but with far more stringent controls - i.e. the zone is excluded from use via general allocations. The point of ZONE_MOVABLE is to allow general allocation of userland data into hotpluggable memory regions. This memory region is not for general use, and wants to allow pinning and be hotpluggable under very controlled circumstances. That seems like a reasonable argument for the creation of EXMEM. ~Gregory