From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56D43C77B70 for ; Wed, 12 Apr 2023 02:55:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C29846B0075; Tue, 11 Apr 2023 22:55:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BD9126B0078; Tue, 11 Apr 2023 22:55:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ACA5D900003; Tue, 11 Apr 2023 22:55:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 9E64E6B0075 for ; Tue, 11 Apr 2023 22:55:25 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 5D9D41A0E59 for ; Wed, 12 Apr 2023 02:55:25 +0000 (UTC) X-FDA: 80671223010.07.11C3105 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by imf05.hostedemail.com (Postfix) with ESMTP id 2697B10000A for ; Wed, 12 Apr 2023 02:55:21 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="RFa6F/C6"; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf05.hostedemail.com: domain of ying.huang@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681268123; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WgG9YSokoGyZvV7KHwbZagvhEVLdy5zIofVcftya7jw=; b=kiAJkFshXcaQVPnFywWuKxIYF8eq7CKp5IOuQrmp7kzT/AI6/3lKFaPzLkq7tgJFdmglVn 20j/w2yNWnAxsfm9sBlgpx/XT//eppsodLb3KbZBjY8KGNeXYSjuEQJ5kjv96XVxmopCbU tkyuQNsePxbywFuDloKW+nepSfXE64U= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b="RFa6F/C6"; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf05.hostedemail.com: domain of ying.huang@intel.com designates 192.55.52.120 as permitted sender) smtp.mailfrom=ying.huang@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1681268123; a=rsa-sha256; cv=none; b=FuofLQQPcdoyt++cVTv0qbwmW57rOipDowMoP3twm1wGzDdm/WK12qY1pXdqi/5779qA+f h8olqYu7rtyKnD+xfMcmEMlWMr0Je3CwCKFHStElb7VXdT79C8Uq04X/PgtaWgB66rG7aj RWnAUf63xSsHyvuPmvuWZUvz9QEixmI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681268121; x=1712804121; h=from:to:cc:subject:references:date:in-reply-to: message-id:mime-version; bh=bmmVPN1E0noe7kMQzpmF/q1kXcSL0ht9eiRXgfNIaRI=; b=RFa6F/C61WBODhbC4wrzf/GyV6k2g7ontrtZlLQrqhukVgJeAu2FqIGO NXnNHG3ZXih3/1c6P9fuCipAMueXhrMRLFxM6aDJT7Lw29wGK1tHwaoX4 klGBP9bRMYUMjtHW7fKu1CzHyWzUIDlIdxC2uH0z71/zyZRhZkYhh0DQN KSbAb62kwiq1m4IHWBmpYzx3NSt44gmM474RmvGAmE1S/Zda2s4b4+jKI 40F8iUoQ3JZA7rRJnvYvWT+sNpansyxWTj6Sxlr4qSYtPJfnkNGn2OGWu ImbYzIRRlWiTYBE0nglXHt0UTFN4BaxbRRE8x1ksFAyflgPXPbLoOFhfP Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10677"; a="342543961" X-IronPort-AV: E=Sophos;i="5.98,338,1673942400"; d="scan'208";a="342543961" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Apr 2023 19:55:19 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10677"; a="691375821" X-IronPort-AV: E=Sophos;i="5.98,338,1673942400"; d="scan'208";a="691375821" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by fmsmga007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Apr 2023 19:55:18 -0700 From: "Huang, Ying" To: Gregory Price Cc: Dragan Stancevic , , , , Subject: Re: [LSF/MM/BPF TOPIC] BoF VM live migration over CXL =?utf-8?Q?m?= =?utf-8?Q?emory=E2=80=8B?= References: <5d1156eb-02ae-a6cc-54bb-db3df3ca0597@stancevic.com> <87v8i22abl.fsf@yhuang6-desk2.ccr.corp.intel.com> Date: Wed, 12 Apr 2023 10:54:13 +0800 In-Reply-To: (Gregory Price's message of "Tue, 11 Apr 2023 11:36:31 -0400") Message-ID: <87bkjtzu7e.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 2697B10000A X-Stat-Signature: suo1y6bh8f6rnex6a9n7wh1139nd8gr3 X-HE-Tag: 1681268121-28165 X-HE-Meta: U2FsdGVkX1+GMY4rc1+vuD21XcuWwxBVZFjL4mBO06Ljncm7GdZXBRlzumn55+zVPybWoDaVeQGFqstPgEP25PSpiTOS346tH4c5B22SndBrwwtnyXfSjx57lcyClyJeNInWr6mlokKn1PXMevhbGp6w/3S5hhO9G3eXRrYWGQ0sjYjhvCewpDtYC3J60NtZyfjoCwFE7+8pv0WQA1sYO81fjnVPFWYgf69CtK4S1AR2zeZIWjkHnh90/OLOjy/pgQDRDgW/ZYDbnAO6CUcuM0hFHiK8suiYi+wXAhRTotF7PGDkjKAX1JfwkxGk1M97K13Lnmb2PB1sXNa8gLsTsY6OCsqXTIJVP22lF7vsO53pZjXgD95X1HDcpCU3Uj+KTf5ZgG9K6hh8CoywoO8j+nuPYqGfgxrxy/1vAdUkQ4ij4dZcwz+JMIiFZNwKj5ZHAL43oUPcN9luJnM2e3ogmtQ4Ds3D6FVJNcoy/EIYNv+DICcBX+OaiRRoU56HP2FgqKzuTdDY67MDtv3ANJ2tAOiEdPgddOTPqS/D5ktO9+RSBhbi0PWToRF4MM36w3KwxPaBIceQQETzvDyOR2dkcicdy9eT9Js+Eo9MDPRp2Tqp6n8LFkAoqHtHAL9d4lOCR933q2cHpMKjca1faBI2SO47HcrH4F9pgBObcv84XssYMY8Sv4NBgWSaLolJgV+K3MeQ42V1puXgCjriRYyMDKl+L6Gf3+UnEyS5bbvvT4HGYuZC2gE0yY8xuheJxNAhtifxtyWj1DHHpYHA4TE/8+s+l+j+EDh3IRMkiXzg/F8kH3grhvKIn05wbisgFsBG7PZpVCsd1JTil+PGEi2k2tSN0SG2xmwno/kOTepkdbbIPix+twM2mg4v4tKQIAAhOhD1l3l/MBHFUd2wsj6TNYv04eDsLmGP+ArvzleVHrXnKpOhxPgl6TQKMRkP05wgKM7CYNknqSGE9C4VwpQ 5YNxsgnx wsT9YEOQPjU4TKuKtFmgSAOAAIJrF4J/4AN5RL3WsFkOLxGs5EdxMcYHq9zWUVfSH7QI5dGWpuc0nYdsOUR4XmBjTtavVNK519ykgQzg87BA/Z/kG1e5BwhqtbZdmEDb7VjTzkXo3GTeF/Y6+KLgpxbgslCXJWrb7DSJ5vDM+pdhIB5Eusy28V0dLOhdPUH8G1DE64oCHSzxsK3N3Ohb0ZVTB46N/NlYQTNh5TIMTftIA5R5dPpWAfrQyPk/o1/RsSYJO7RIfTS73o4A= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000912, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Gregory Price writes: > On Tue, Apr 11, 2023 at 02:37:50PM +0800, Huang, Ying wrote: >> Gregory Price writes: >> >> [snip] >> >> > 2. During the migration process, the memory needs to be forced not to be >> > migrated to another node by other means (tiering software, swap, >> > etc). The obvious way of doing this would be to migrate and >> > temporarily pin the page... but going back to problem #1 we see that >> > ZONE_MOVABLE and Pinning are mutually exclusive. So that's >> > troublesome. >> >> Can we use memory policy (cpusets, mbind(), set_mempolicy(), etc.) to >> avoid move pages out of CXL.mem node? Now, there are gaps in tiering, >> but I think it is fixable. >> >> Best Regards, >> Huang, Ying >> >> [snip] > > That feels like a hack/bodge rather than a proper solution to me. > > Maybe this is an affirmative argument for the creation of an EXMEM > zone. Let's start with requirements. What is the requirements for a new zone type? > Specifically to allow page pinning, but with far more stringent > controls - > i.e. the zone is excluded from use via general allocations. This can also be controlled via memory policy. The alternative solution is to add a per node attribute. > The point of ZONE_MOVABLE is to allow general allocation of userland > data into hotpluggable memory regions. IIUC, one typical requirement of CXL.mem is hotpluggable, right? Best Regards, Huang, Ying > This memory region is not for general use, and wants to allow pinning > and be hotpluggable under very controlled circumstances. That seems > like a reasonable argument for the creation of EXMEM. > > ~Gregory