From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17782C6FD1D for ; Tue, 4 Apr 2023 19:43:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 90A106B0071; Tue, 4 Apr 2023 15:43:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8B9E26B0074; Tue, 4 Apr 2023 15:43:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 70C9C6B0075; Tue, 4 Apr 2023 15:43:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 5BCA16B0071 for ; Tue, 4 Apr 2023 15:43:26 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 34237409DC for ; Tue, 4 Apr 2023 19:43:26 +0000 (UTC) X-FDA: 80644732812.12.E2F049C Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2080.outbound.protection.outlook.com [40.107.220.80]) by imf11.hostedemail.com (Postfix) with ESMTP id 2EDC240002 for ; Tue, 4 Apr 2023 19:43:22 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=memverge.com header.s=selector2 header.b=sZOvrYLa; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=none (imf11.hostedemail.com: domain of gregory.price@memverge.com has no SPF policy when checking 40.107.220.80) smtp.mailfrom=gregory.price@memverge.com; dmarc=pass (policy=none) header.from=memverge.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680637403; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Qapz4nlM4pl+FdBQf7W7o22a92AmaV6teirFDT0ekas=; b=6gbruBsOHtlotsZhxn8qnaY2O3BnLFQTToTiOMAes/Jy8zbk9uMsPL8svQg02/apA6QTo3 +nf/3x5Akxp/ZyPNbPwQoi/mGUeANDv/qIK+Og5WAR3sXN5h2eXrol3OkokYKS7Saa3AvG UrKR6nGzM7ir+EvXczGPVssmukRxMO4= ARC-Authentication-Results: i=2; imf11.hostedemail.com; dkim=pass header.d=memverge.com header.s=selector2 header.b=sZOvrYLa; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=none (imf11.hostedemail.com: domain of gregory.price@memverge.com has no SPF policy when checking 40.107.220.80) smtp.mailfrom=gregory.price@memverge.com; dmarc=pass (policy=none) header.from=memverge.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1680637403; a=rsa-sha256; cv=pass; b=ezKlgjBD1TKhft+UmEhPqD9rSvQxI53TqRJNSx3+c/JvG6+iQ/sZtV1WpdFShiSu1IsCCi VH06GgpWns2Ymq/aMAz667/bFtMhcffEWrnXGLRTSiJlJ5ddFVFsGS1IpUzYn79DH8hYgC vyIUXytN5daLGKP7gMs8JGxLblc1b4Q= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YHuLWPB3XV7IRqgMFGJLqLmNFL00uglj+8/xTre2OTGe9cilP4ZAHHIOwOPdKv+gklOuPQ2VyuJ5sInWhVHi3PlAqXl+825zhUKbDE7AFdB9p57IPGlSRx57LNJW1du2644lBaO63s+PPjV0lokV6C4kwkKiD0+ZB7Y3RUmSJCEmvrI4dymfvClniGXT2SR7U+DNFTYKIEYaLynEMVLabjE8FCMUujdlq+ZpGqgA89hpIBdUJYqf3ZLEWkzcEDx1czCjJbPwVe6C0vaCYoFehK/Ftn0ErublC0npuLV9ENrF2EDywWHoY4/wmOzZbYyoPo8Amqg9W5qdcA8Ry7QCMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Qapz4nlM4pl+FdBQf7W7o22a92AmaV6teirFDT0ekas=; b=oCahE46IbI1TmupW7b/ge02VFuBe2R31wRjRDx+RG2OLd5pMWfE5JFpnZUV5dW47Ix+Ugq0j83aO24X5uPZxMwuno7RI6dh5+ZWw7jGafcLCIsPc9uOs8lVRRDTxLFZMoseg64kQaXkalZEm50oPDxeaCHa11lY5+QvcmbGqP4jkm/UDKonQjop4M3ScLhHJhzIp7KyC/Q+eWazRRL/Ji57Q/jukSc5yOPK2Pdk7Beny9+dB7qbPkA8cnOC8Gll26gRhUF7FzDbqEYpZhjpVr4XgYJMkac/2KqLcVNmC2LCIi/x5mldTkC+ZvocEaxr9sVIgPoI7tz9cuEAbHA7+Kw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=memverge.com; dmarc=pass action=none header.from=memverge.com; dkim=pass header.d=memverge.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=memverge.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Qapz4nlM4pl+FdBQf7W7o22a92AmaV6teirFDT0ekas=; b=sZOvrYLaSUbP2ugRbT83rpdK1O831Gcr1m31R8Q2ZUnsziK3BPdKMqb2lcuqLYWl7L7yEvuH5w5LoIN4pagcfUGE5cJ8a/chFwSQuKTNp7WI0P8ipax0QTS7X1KSJQ8xegt4OShoj4uCki7rgpfKZIjoPfRHfmJQyxuCNcXltWA= Received: from SJ0PR17MB5512.namprd17.prod.outlook.com (2603:10b6:a03:394::19) by SA0PR17MB4223.namprd17.prod.outlook.com (2603:10b6:806:80::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6254.22; Tue, 4 Apr 2023 19:43:19 +0000 Received: from SJ0PR17MB5512.namprd17.prod.outlook.com ([fe80::7b97:62c3:4602:b47a]) by SJ0PR17MB5512.namprd17.prod.outlook.com ([fe80::7b97:62c3:4602:b47a%5]) with mapi id 15.20.6254.035; Tue, 4 Apr 2023 19:43:19 +0000 Date: Sat, 1 Apr 2023 07:51:12 -0400 From: Gregory Price To: "Viacheslav A.Dubeyko" Cc: Adam Manzanares , Mike Rapoport , Kyungsan Kim , "lsf-pc@lists.linux-foundation.org" , "linux-mm@kvack.org" , "linux-fsdevel@vger.kernel.org" , "linux-cxl@vger.kernel.org" , "dan.j.williams@intel.com" , "seungjun.ha@samsung.com" , "wj28.lee@samsung.com" Subject: Re: [External] RE(2): FW: [LSF/MM/BPF TOPIC] SMDK inspired MM changes for CXL Message-ID: References: <20230331114525.400375-1-ks0204.kim@samsung.com> <20230404175754.GA633356@bgt-140510-bm01> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: SJ0PR03CA0209.namprd03.prod.outlook.com (2603:10b6:a03:2ef::34) To SJ0PR17MB5512.namprd17.prod.outlook.com (2603:10b6:a03:394::19) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ0PR17MB5512:EE_|SA0PR17MB4223:EE_ X-MS-Office365-Filtering-Correlation-Id: 5ad2f421-64d8-4c88-62b1-08db3544d74c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: q2ixuFaThYJKTE1BHtBsUp3oWpIdMS4HQj8MlAuxRG9uHmzkvLIVaWovd1OJ6H3bdY7gwhmD2CHIUX6pG3AW3jpRve/u548V/IZKeFRbj+3aep6yAwEqy+1SblJbDURzy5u2DDsILh0ZG1LxAOELOKqG8xOL3KgN8/5OCnEDhq3b4bfsY3qBgyCQ2IhvgiULuz1FaWWRBMWnuVQmoo38LQgN1yFlJhLcSpPPqUQMhrdF5h4k+AhdDc3C9gNEkSUkXlNORu91TfMFzUpGD63lJiz3oWmPuL1eWxeqvRszZvs5na2VjDUIyk2hwD/V/OLHN9E2hi0aQKtayd5QLx6GVXIB6An0LyvvPjDlByR/1s3Ourj6D0gkXA0T1jqZgw8/a0a/1+uAkvaoAg+Pwyl4iC2dt+cRZD7y8J5IS2ljVGuHhffMQRSm827bSRdRAlZLvrZFnPcoMuO4m1Exm46R5kXUr2CoTjL3K068yooGaIIR3JZ4sEOGDzYC2Hq60qJ2ZwLXZiWPfGDKvIzLVKJ85ALe1OIdhFliEKHaYL1xqW2lFSD6Lf0Mzpo6Rn+Gdq1P X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SJ0PR17MB5512.namprd17.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(136003)(39850400004)(366004)(396003)(346002)(376002)(451199021)(4326008)(8676002)(6916009)(66476007)(66556008)(41300700001)(8936002)(86362001)(2906002)(5660300002)(7416002)(44832011)(478600001)(36756003)(66946007)(6486002)(6512007)(53546011)(6666004)(26005)(6506007)(83380400001)(186003)(316002)(54906003)(2616005)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?kIVB/T8Dy6ZKtEZvbEHJETZW+9xsWA3yY5m7va10oSM7ftF//lvQMvIBmR+O?= =?us-ascii?Q?DXgPPCk0bVvW1qJi+l7ZgTjFMu8FPMnSgMCwg1Ck++Iaxf0Z88vS8CO5Ve61?= =?us-ascii?Q?UR/bi3mY84NcKcWOiXUKjPJVHR9Tl6W97/HDe/8KpOu8uljPPhhi70TgXe+n?= =?us-ascii?Q?8wipxcHIvBsivo2XXVX27BQLdyOnKbIdRcvSpYFhoxu2b0pDNp2Go0zRzefn?= =?us-ascii?Q?iguIsWierERWJftNbJeA/dWRZFs3OdqEn3LuO6maVJnhnpBrQN8gdFsy6upC?= =?us-ascii?Q?KHNMXv9zdeRF38no8cjJMS6T/wJwJNSOghaQwXO0tHI4HaO+csOJxwfHQdiU?= =?us-ascii?Q?saG+2EbBxPTN518YT/ik3RuEIMSvh2952r0rt4s6UufaR5VtBjoNaCE7mLCd?= =?us-ascii?Q?5mPHDFXY8sNXOpOZF5CYmfecY+Av+tnkZzSY3K2mt9YVJcD4T5gYO4yBsNBV?= =?us-ascii?Q?BHumAQ+iFnHBjVjASwqNR8Nr5QDE9KM0gmXwtPAkpcCdbhxqPgQyCtFscWuG?= =?us-ascii?Q?OHCUNbMpQ1V50Obe6MTXNcw+SUo0NeileuErBVaNzQxHgN1P+pElzZOtdF2D?= =?us-ascii?Q?qOqULrSXkntqpuAVfFecXf26XETW4JoSjDDs4cT/60RvENi9bL1ojNp6NLwW?= =?us-ascii?Q?2qTRRn0o3scPRWNaYVsMq0rI3dm698CHyN148MnKFb7YZn6q1rS0VrmhO0Vy?= =?us-ascii?Q?aeoGFEMS1C8irMZFEzzDX4vJr1Vhu5KTcbWEz1cAhhSl6mufjK7dLADjB2qC?= =?us-ascii?Q?m8/lIf/96G75G+k437AJRDlcSz9uQ4U8T3t7+J2fa99iO9i59TlgkXFa++fj?= =?us-ascii?Q?e629kpscbagYpkwkeZse12f2svjHC2DbbVxsRbUF+X2FK8P/8dXwPy0TjnyK?= =?us-ascii?Q?C/UNnjDIH0bcREhUPWRnyUbMbX2FP2Ezm7nnDt/r9eXaxRwQGuyaTuyGuqS0?= =?us-ascii?Q?F6VILSU5mdT1jqn0xibXTl3Vc3kD/vqlwZuCoPPoOLk9fYvL+JnR3wcadg6r?= =?us-ascii?Q?mutR7LgSIV1cQCPIkgOeGqTToqOLPhtLBxPIyCjmh1MhcYHkTrBU+iQ7Mvpm?= =?us-ascii?Q?OaAxwMNmKeOeHmd9V+yBoIdcCdkahICe0v+iwcqhfMb832W2TOUgxM4jLyHQ?= =?us-ascii?Q?3au4IvJ14E4cNiKbUfAAwuwVlhgVxwwkMxc7TcfwC5tWrF9UTOttMzwp7Kip?= =?us-ascii?Q?7obgsDqng++54MM4y+brvBaBa/Dn6wiGdTyurDWBqi3UY2LU1kmnxRROVU4t?= =?us-ascii?Q?2vrZyiONeJ+tZRfBX1yqQVO3j03EjG0FiiJC2wsaodUVoZfwUJXC7gJTEW9Q?= =?us-ascii?Q?heMd+yC3bE6cKKTSgVQU2btWgvGuzlgkDU49VbdL4LAVONSCSR/djFwE7MUA?= =?us-ascii?Q?NsW+YtfFF6+xLbodTkuab6x5qvye+pm8EfPBNZnUsqDvwKATB3y+85o9QLhF?= =?us-ascii?Q?DYClPUdItrKB9NIbF6JECwPoJrOpKnNTOCUrhT2U3MTHdv1XIPrgFq11qW5+?= =?us-ascii?Q?5I0ha1UYNJxjhtj/SHz+H/wY2MRb8mpwe//x40yw12FrpFSNfUbwxV4ZQZXZ?= =?us-ascii?Q?7FdK1OfPrQhIVW3VOS/7J2olfvJQiIUXRslpjjwrnk2cZFoi6Asaff1hTKrQ?= =?us-ascii?Q?FQ=3D=3D?= X-OriginatorOrg: memverge.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5ad2f421-64d8-4c88-62b1-08db3544d74c X-MS-Exchange-CrossTenant-AuthSource: SJ0PR17MB5512.namprd17.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Apr 2023 19:43:19.1919 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 5c90cb59-37e7-4c81-9c07-00473d5fb682 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 0aH3clZFPWanMivNkWJzyUJvOBJiygMYfTkIBynzmu7y4XibK0hRohes6EC2EZLJOjKdI8Qh1zHDqi6mpAp3dFZuS2CF8fCJFIVDMu3TY0U= X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR17MB4223 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 2EDC240002 X-Stat-Signature: sbsb5iuds3wrpax187ptee5oprk7ghgh X-Rspam-User: X-HE-Tag: 1680637402-308529 X-HE-Meta: U2FsdGVkX18WhwQ9yuaNV6VqlGrs5rsRd5Y85YZfnrafVKXQ7SGjwj733Yg/2VOy19wswJkI4RI6zTrJPRc/4qw4WGS+IEGHJZjmiMs5+vTbHlxpFqkF5E+K2kOIuT62jjSP5Kd6VSXjgIl5IFzuSsCLsHJaslpvr99l3K6qJ9X5mHiwCxIP2ImypCK786XNyl0O8zWI3H02n5mjpeW1hC9LTreZJ1BUTggMLvfJq4KWr8JdztHstdpaGfFVhC5T//W6n7EDHbnhxSS6IzSejAyfR4/cbWi7QZ7ysIJZgRuuoyZJvN0GTLvO/syfxpuuALO43UIt00Hkvhy35Ov8isJJY6gCBiwaOQWRwC+0Pw3lDQc57d9g9rSm455EkNZ/LeUUqbvNNZyZVBY8AnMAOnDDFsV3eG27AaIROLeP5lVLcZF+0mgyJvplwnDaDT5Fivaj5yX9muunIrtdQcRDja6BawhLmbywIV2sox9LNRyKFtj6ZpcMm8T98pRJmikL6GF/JIqE+GYji23ZMohh+JGugiaUte9c1/HaFzK4MV2W/oL/omXf1T/pE6pPV7+NYK1ILfJxmE4I9BDXV1FcXL4cvRQvJ4pSNy7vgKX1t3zz+WCOW11jpiVKTx1f8COcxup2z+c+GWk7bQorfsQCN6E93TvGNumT0/HDVW4fcK0oXLmIbiOofHH6l7b6qU4vvNJELXZGrtVB8C/xjH7WXO1Pw7ifMCgneCYN9r+L2Qa34HEX1ZYTOHErTniwI0pRGQGMcikMafEcZ6NjbGLY7XeLpdk/MIC7Q3QBz2TltCtjzTli9H2/29JtBAp+2G34/rsZ2HHvRqAJ9IQ2VhRd1mgG6hWXDXp2SoXBXdcBqa4SWOLsCmS616UEfBBdM2ky4EXF5ejw9qRX1TYLu7NEt9CM1iH0SOfXvkcs8E0sMC2i6fue9Y+i/iwSS0W3okogWMUE5ENvMxBC19vajJp T5sxf0c8 X4CpLdxdPhBO+0Kaab7LnPVGAfhObfurpMP7NyXHV1sp91NTgZSN12gq5z3AsPDO1fbuw6mKBo935sz+k+ZqpDPXLOqF8ykj81ls8xRThfqZ9WT/Aa/xYod3iNaNB/MmSN1NOQQtm7lvZDyGgEWjoDVszEfJkUDi/+im0V3pPDSn8YVaGAyGeTe27sk2ulxwQQNw4gOcb/5tHvKOSSBm7t7a1k5ReEXsR6GzR1OEB4q5HL+UMMq4NqnTpitSQEWOx3I29iclnZ0SVVhnxGTyrwPi0w3ARR8k6nsqhyAoUZBBcHrNFTtEzxJ+Osse80XqzafTLTzER/AZPK/J6orhxgXhAsZAt7ZsjYrDZoojEDuCODjG7qtEEgYPbY3b9eSLxRRrUXJYTtG8k/0XEcU6qPOmnPSnSDOF6f85NLzoa+9voP5U= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000402, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Apr 04, 2023 at 11:59:22AM -0700, Viacheslav A.Dubeyko wrote: > > > > On Apr 1, 2023, at 3:51 AM, Gregory Price wrote: > > > > On Tue, Apr 04, 2023 at 05:58:05PM +0000, Adam Manzanares wrote: > >> On Tue, Apr 04, 2023 at 11:31:08AM +0300, Mike Rapoport wrote: > >>> > >>> The point of zswap IIUC is to have small and fast swap device and > >>> compression is required to better utilize DRAM capacity at expense of CPU > >>> time. > >>> > >>> Presuming CXL memory will have larger capacity than DRAM, why not skip the > >>> compression and use CXL as a swap device directly? > >> > >> I like to shy away from saying CXL memory should be used for swap. I see a > >> swap device as storing pages in a manner that is no longer directly addressable > >> by the cpu. > >> > >> Migrating pages to a CXL device is a reasonable approach and I believe we > >> have the ability to do this in the page reclaim code. > >> > > > > The argument is "why do you need swap if memory itself is elastic", and > > I think there are open questions about how performant using large > > amounts of high-latency memory is. > > > > Think 1us-1.5us+ cross-rack attached memory. > > > > Does it make sense to use that as CPU-addressible and migrate it on > > first use? Isn't that just swap with more steps? What happens if we > > just use it as swap, is the performance all that different? > > > > I think there's a reasonable argument for exploring the idea at the > > higher ends of the latency spectrum. And the simplicity of using an > > existing system (swap) to implement a form of proto-tiering is rather > > attractive in my opinion. > > > > I think the problem with swap that we need to take into account the additional > latency of swap-in/swap-out logic. I assume that this logic is expensive enough. > And if we considering the huge graph, for example, I am afraid the swap-in/swap-out > logic could be expensive. So, the question here is about use-case. Which use-case could > have benefits to employ the swap as a big space of high-latency memory? I see your point > that such swap could be faster than persistent storage. But which use-case can be happy > user of this space of high-latency memory? > > Thanks, > Slava. > Just spitballing here - to me this problem is two fold: I think the tiering use case and the swap use case are exactly the same. If tiering is sufficiently valuable, there exists a spectrum of compute density (cpu:dram:cxl:far-cxl) where simply using far-cxl as fast-swap becomes easier and less expensive than a complex tiering system. So rather than a single use-case question, it reads like a tiering question to me: 1) Where on the 1us-20us (far cxl : nvme) spectrum does it make sense to switch from a swap mechanism to simply byte-addressable memory? There's a point, somewhere, where promote on first access (effectively swap) is the same performance as active tiering (for a given workload). If that point is under 2us, there's a good chance that a high-latency CXL swap-system would be a major win for any workload on any cloud-based system. It's simple, clean, and reclaim doesn't have to worry about the complexities of hotpluggable memory zones. Beyond that, to your point, what use-case is happy with this class of memory, and in what form? 2) This is likely obscurred by the fact that many large-memory applications avoid swap like the plague by sharding data and creating clusters. So it's hard to answer this until it's tested, and you can't test it unless you make it... woo! Bit of a chicken/egg in here. I don't know that anyone can say definitively what workload can make use of it, but that doesn't mean there isn't one. So in the spectrum of risk/reward, at least enabling some simple mechanism for the sake of exploration feels exciting to say the least. More generally, I think a cxl-swap (cswap? ;V) would be useful exactly to help identify when watch-and-wait tiering becomes more performant than promote-on-first-use. If you can't beat a simple fast-swap, why bother? Again, I think this is narrowly applicable to high-latency CXL. My gut tells me that anything under 1us is better used in a byte-addressable manner, but once you start hitting 1us "It makes me go hmmm..." I concede this is largely conjecture until someone tests it out, but certainly a fun thing to discess. ~Gregory