From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9EED5C76196 for ; Tue, 11 Apr 2023 18:00:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 020706B0074; Tue, 11 Apr 2023 14:00:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F12666B0075; Tue, 11 Apr 2023 14:00:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E015F900002; Tue, 11 Apr 2023 14:00:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D0CAE6B0074 for ; Tue, 11 Apr 2023 14:00:09 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id A2084120C32 for ; Tue, 11 Apr 2023 18:00:09 +0000 (UTC) X-FDA: 80669874138.13.4A42CCB Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by imf13.hostedemail.com (Postfix) with ESMTP id DFE3020024 for ; Tue, 11 Apr 2023 18:00:06 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=aX5SR9J+; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf13.hostedemail.com: domain of dave.hansen@intel.com designates 192.55.52.115 as permitted sender) smtp.mailfrom=dave.hansen@intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681236007; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GNoYKoON81fkayjxTTGuxAhYNL27gZfa4tpxzIT2yYo=; b=Tjl4BMRFYC2tnIpUR0/Hhdh5THH8PGjsd3cC1vsC5QwbMoUoA5U9xLiwxpwh0ItgSflkQh oDaqCf/lSRGVsKcU4Znq7oOmM+hCHyS6bOGzvxKaZqM9n6DtNyY91SgZufaZAMpPad/o5b szx5xJyoK2CZ+sFlCc6repBXf7aBT+Y= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=aX5SR9J+; dmarc=pass (policy=none) header.from=intel.com; spf=pass (imf13.hostedemail.com: domain of dave.hansen@intel.com designates 192.55.52.115 as permitted sender) smtp.mailfrom=dave.hansen@intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1681236007; a=rsa-sha256; cv=none; b=1v8OxDDH0TkcWe6A9HG+KFMzJMJxmCgbCf2iBn0Ylmr5GlNaPrPo4o/MSo9cVxSdssD9lJ xGab+WCHvBjYedS7cOYyOLumksPN+wIYbYz0r2JZOg3K5fP3RASmRtRaSYlgptrgFnhiP9 zFimF/1o7M6OvSCC5XhPaR7pEhVqi8o= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1681236007; x=1712772007; h=message-id:date:mime-version:subject:to:cc:references: from:in-reply-to:content-transfer-encoding; bh=vmA3xk4DjZmDJu8SYkGhKpH80tDfxKa0abTU4/7ZjRU=; b=aX5SR9J+Ab7XDh2EBdpdojjW/ZsYp3JJ70XUHdAY9kc3JKxbE/8tlPZz Ca1O3mdKk0XEBCqarIk6tczwoVYiQGlHQ8hWwO5SGE1cVFtGF6YMZI0u8 ogBGNORqYsBR4trA/MOf84Ru+eIa9YiS4LvydTMDQ0Fym70vT9oKAOaO+ VrZD9VY/fXoIzj4gDVd2aEGOmQkiDPhOUceyXSZuxg/OpYQRhhCHBnH0q a2MrJxrhqR0s5qRm7IDyiDI7az5kHPS14AWRIF8GMo9jmqX0jU/mrNy7W phJ4gUWxL3yXSPPomyessL5PKtuIaWJmopkL36wK5NNSfeeH4Sf/k53L7 Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10677"; a="343699267" X-IronPort-AV: E=Sophos;i="5.98,336,1673942400"; d="scan'208";a="343699267" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Apr 2023 11:00:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10677"; a="757942293" X-IronPort-AV: E=Sophos;i="5.98,336,1673942400"; d="scan'208";a="757942293" Received: from gtryonx-mobl.amr.corp.intel.com (HELO [10.209.72.81]) ([10.209.72.81]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Apr 2023 11:00:03 -0700 Message-ID: <14a601ea-8cf8-bb9c-a87a-63567c5aba5b@intel.com> Date: Tue, 11 Apr 2023 11:00:02 -0700 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.8.0 Subject: =?UTF-8?Q?Re=3a_=5bLSF/MM/BPF_TOPIC=5d_BoF_VM_live_migration_over_C?= =?UTF-8?B?WEwgbWVtb3J54oCL?= Content-Language: en-US To: Dragan Stancevic , lsf-pc@lists.linux-foundation.org Cc: nil-migration@lists.linux.dev, linux-cxl@vger.kernel.org, linux-mm@kvack.org References: <5d1156eb-02ae-a6cc-54bb-db3df3ca0597@stancevic.com> From: Dave Hansen In-Reply-To: <5d1156eb-02ae-a6cc-54bb-db3df3ca0597@stancevic.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: DFE3020024 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: jdhx6gpbx8xfimix5mtjse4do1i36ewa X-HE-Tag: 1681236006-138393 X-HE-Meta: U2FsdGVkX18YYgXRaiAXqri+WoGQqUVACWHFMG0C0PQjWGASjFLOvJeVXxCuVDdC3LiIAR/Fm7BzyDIG548g1nXrZGGaY1a8eE/DI7jdGIJBw957dUDnvY+3BzIJG617kuI/VRI66EWHi0q4XpPxO9AmIdoFx7nHrQJBwhf3/qbQR8HRWPZxsE4YWFF7RvhohbPhViQNmGZ6wRR34tDBC3RkgZ1dYCSL7t2shczFyXaJ960pO8Jtoe79vWMqJXP76FZRLqsuxwRU4WUBNXfYdzi9TishsQHRC66ImiBftF4Jt50O9yD+LBbIy7+D4tWbXi/c1uzrB+gi+VyR1YDJO0CdRJ7Ty8sBogzNCLmPaTM74xQ8SZNbCRV4cgKWeTMyGYL4MSglEOfX1NHaRl/QzoKoEEAY/vgOQybsn0mPsnOs5Ar1c9D+mYQi+80jC1ax4vYK/5dooneaX+lDjrmIAiycycQhgo1KQyD+COVipmNEJ8hvwZ2k3r0J48begjJPuqQ0aMYkBmolbxkNKPPxTQMSQfKr1otkr7C3EINccOAjHjvi0ltl6Qa8VGnLzWvZpnYKQfrlPtcw61EhP25QeUYVPtR+RPWtDCKtZUMaO2fCUNYP6tBJ0H0kUF1cMRZtXCdaybfcVj/3JGct38Y/ZUHiVMZfRjhVBcQ0Vn9jrab06Is/Dk+nMfNW7c2vADRxsFxauE2XhtSWJgKM3V6wCoB/n0FUW22SV5iHEQnhZzQmiKcHCLggUvj5OnTvS8RJKkRhwpXGP/sSMHhnS9m+lf/ysDVS4dK6ZZ+sXsECZ7njT1QaO+nUWImPIy0La9h8GOR+X86vHmtIk3Q3w7f2LXHMr1kzeFo13G+zDUjmJce0qU+Lg+nGdAW9uUp63QyIkdTS+8PvFgSu8fQEeIgEEbpnwGwAAiDbnfgH31gTyIW7t/N7AHGdV8vPt3fWorwzePgRV8fsAF7KLLFxyrR w+kv++Zb hF43YzHpdznisw3aIPbx5u/S/QCSo2N77uh6m4z8DXF1VUIwaUz73T0PuxiKmh1hhVWJOjRhlkN1KyuufmTe/7Ef2/ZClWEIU+Z60Rimbd/4GEfhOpeAl8dkMm4kNX3Mq+Es4BhHTBfNWUzdGCidiAhR5uqguCRa2YJ19ID59XLZnnmha2aOuTbTotRaHRKuUYY4ImoSagLlyB28= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 4/7/23 14:05, Dragan Stancevic wrote: > I'd be interested in doing a small BoF session with some slides and get > into a discussion/brainstorming with other people that deal with VM/LM > cloud loads. Among other things to discuss would be page migrations over > switched CXL memory, shared in-memory ABI to allow VM hand-off between > hypervisors, etc... How would 'struct page' or other kernel metadata be handled? I assume you'd want a really big CXL memory device with as many hosts connected to it as is feasible. But, in order to hand the memory off from one host to another, both would need to have metadata for it at _some_ point. So, do all hosts have metadata for the whole CXL memory device all the time? Or, would they create the metadata (hotplug) when a VM is migrated in and destroy it (hot unplug) when a VM is migrated out? That gets back to the granularity question discussed elsewhere in the thread. How would the metadata allocation granularity interact with the page allocation granularity? How would fragmentation be avoided so that hosts don't eat up all their RAM with unused metadata?