From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF4DFC28B28 for ; Tue, 18 Mar 2025 17:10:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1ABEA280003; Tue, 18 Mar 2025 13:10:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 15B8F280001; Tue, 18 Mar 2025 13:10:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 02194280003; Tue, 18 Mar 2025 13:10:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id D709C280001 for ; Tue, 18 Mar 2025 13:10:02 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id B6F7B161576 for ; Tue, 18 Mar 2025 17:10:03 +0000 (UTC) X-FDA: 83235309486.05.4C707A5 Received: from mail-qv1-f44.google.com (mail-qv1-f44.google.com [209.85.219.44]) by imf18.hostedemail.com (Postfix) with ESMTP id C8E7A1C001F for ; Tue, 18 Mar 2025 17:10:01 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b=MNo89NBv; spf=pass (imf18.hostedemail.com: domain of gourry@gourry.net designates 209.85.219.44 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742317801; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=EAoBcA4JtXxpzIjbx779EoaKbfBPS1llR5iDCjOnQKA=; b=dInJRSHFmXpc11B3rM6Y5fiK7TCfskWymiNepTmGLO0DOqXkuBHiIRttmE16mJDDSf4qSk 9S7xl2EWn9OZOGk3uTbeyK96G0iX42i1v1GbZ5RLsJn5NDXc1QjkQ0sg5uZo08wvsHrK6d EkCi+eCItWKgkNZd5MqiEDm9xZVwFZ4= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b=MNo89NBv; spf=pass (imf18.hostedemail.com: domain of gourry@gourry.net designates 209.85.219.44 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1742317801; a=rsa-sha256; cv=none; b=aUWWQpiFiY0bHnLAWORod2YaHx8y4Gxi8HLjikvsj2OxPioLy0XhLmx6oi+ZnlDgFUtoI3 lt4QBvVnZ3L1AsWGR+uAgJK4gSsAmmHb49e1n6MCSM2LX6gWo7fow0qshN7tzZRfWbNV51 FBtt9IPQ8o8MJzsH61AkLg5L5ctYjc4= Received: by mail-qv1-f44.google.com with SMTP id 6a1803df08f44-6e8f916e75fso76527626d6.1 for ; Tue, 18 Mar 2025 10:10:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1742317801; x=1742922601; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=EAoBcA4JtXxpzIjbx779EoaKbfBPS1llR5iDCjOnQKA=; b=MNo89NBvNyWRTYIq88wZstaWeyODlzdeLjUNo2tIHLrcoDDj0h7U8Q7AHbgYYnNj4r 6kXnjlPkSOfU36+nSHBg+jtLlad8+fEdXgCxuJa9Y0grZ8d2+QLrTZ6VoblweJBCxj73 pLGMuhsy/P2aBdNYt8/FziqeZXIktZCkjRkJFwFpX7br8HdN30KDGDPJfULi/JiWeJSq pxRy7MFqw6Zc4uFEygwmOiCIhBqF52bbKh2AxLU179oKiLl2QMad8RSzyVvrnFSvn+C9 SiTVbYSNZKH39ieFQpZ07lSaHQTqKFfLHyONeleyc2gUHLoh6UfDWlbXdeyfzaJc1Lsj qJ9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742317801; x=1742922601; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=EAoBcA4JtXxpzIjbx779EoaKbfBPS1llR5iDCjOnQKA=; b=o9K2hlgkqigrmptZSUQ31Ju6Tnk7Z8jHRAYKka+UHKlIjxQjV3lwbx1RZRatC+svlj rKMU+bRuL+vIrMZmv8C51WFCZuWoUqamlcbj3cu43hDxIFbs3wqr4vKTpstWliaiFlRJ x9R+VeIyhbAZPZ/5bk2PmY6xTmaJwZmFAdFUawNB2ToPKLDEZ99813kA6IJ5KobVFLQ/ W2YKxo/tEK5QW6k+cA5xOr6WGh5MF+RGrP1lw+8HzBcqJ2MyggTlgmvT/0ad3Ebbagvq MFy69wG9399AIE4tnIDBZFYO/lMS4Iqv3fWD2/4WS2qW0PA+AY7ZRFcWeYzHOmf5G1+2 QdXg== X-Gm-Message-State: AOJu0Yzg9v9SCb/GY66uF1rTwpEyjIgMUsuLJDnQ0Ppv+ZDY9Sx0POO6 szYIyrvmwHYbjlqI3/np+5SuVLnnIDHwdt4OmauMvCUMgUtjwpe1CU3izyf5Do8= X-Gm-Gg: ASbGncuv30ByNngWPI90crdR2hmPIQUGVktwocRwE/igZcNYS4QAp9Z6zWMttz8b9Pu xrrVs7LrKb5Y1wE+a/Y1ASOFrbWcC2TqvP+D4qEC2yFeLKk6qzQ7BUS2mpmEX69C4XVB6Jt4Yaf PZvemuMorikQ+/FVkYocLXNvyfO33PsDMtBr7ypNoFhjeItrGA9KBK/IzvBE6HwWtuXIc/WY3JR BY/Uwsrj4hyPFQWJWKP/Civgmbe1QzCI3d8eVwK+J0IlZoXRMyFgOMFbTHJCSo9G4dHoOwWJeTp iSZE86mvhyu74oqZo1bI5ZJUOS1C7GbPMU9BlC9xYBU677lAtEEcnbJvAYsnHhN50HKxlyByTp0 40K4kuTmz0YrWrSFEeJMQSp85txQ= X-Google-Smtp-Source: AGHT+IGDKMLKrW98rtnIQnYwYyplJuQKNFT9i7Ymkc7/amGmSQWmmu9JY7piNnTLzFpuWgwVF1lrSQ== X-Received: by 2002:a05:6214:2d49:b0:6e6:5a25:e118 with SMTP id 6a1803df08f44-6eb1bbc7675mr60892246d6.6.1742317800825; Tue, 18 Mar 2025 10:10:00 -0700 (PDT) Received: from gourry-fedora-PF4VCD3F (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6eade231bb5sm69811136d6.34.2025.03.18.10.10.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Mar 2025 10:10:00 -0700 (PDT) Date: Tue, 18 Mar 2025 13:09:58 -0400 From: Gregory Price To: lsf-pc@lists.linux-foundation.org Cc: linux-mm@kvack.org, linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, mhocko@suse.com Subject: [LSFMM] Updated: Linux Management of Volatile CXL Memory Devices Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Stat-Signature: ca5cfg1t1gdoh495exois1syhypekpcx X-Rspamd-Queue-Id: C8E7A1C001F X-Rspam-User: X-Rspamd-Server: rspam09 X-HE-Tag: 1742317801-221751 X-HE-Meta: U2FsdGVkX19hspMsN6oEaMWWFVyISfO4+iapfky53QJUuCdv3UtjBIgHM1fTWK7JNNrQzS24NOwjqxWnM27nF5XH/KwtbPPkOWqUxF2iJhkrIohEVjufFK/4ALg7YiBfGTzN3ERNT8jkxsq3zYTDBXcXNFKHMmRzJZkaWA9BwMN0Gv195DhyEf4Rz2eap/+8n/VkvtfIAdCl1igPKEBa7AdpRn/rEw+tK3BTI2u+/ewqfRjp/Yd+RYVdawjtxKgeKF2GIXqWkj4jQK3L9Wx15yqonwvTI0QT0+z5uwl4EHCyv8XI/Yml+J0MF5odkPPTM5LiavrrA8t4f4PEThH8W+bxY/IzNBmJZiYdA8WAJXADjSwOtFUZ0O7+fCLzhPaUxJ0BC8RL5jPbzcd843ZXAnAzrPUdV23PzfudsJ5nQAEg5qw7t6Z/Xp8JUbaEAgSsPHtJGggen/CKP852hXNlpsbtPRkfnpq/W8YIsTNyq+Q6IivuzkqpQTJ8PxYaP4PYfUqfyc4dkan3cfsv8gkEAyDL46q+MkVe3djYnZCdXm3zwTiyJeWkR8a8XyA94O1KejQGOY5Orj/rPZe1xJatLD67UNrTMqwvec8yQssl0vUPK/cxk7joB6c2Paz0cEHDO+vHLLZpRxLEWa14q+BBbyO0sQc/VOcimen6dTNeqffQSl6kcx8WdllzYiOaccWzATYXCvzTN60h8wWVvmAQte8klixNxNmx+MQ9bAyFJ8dfSOa7ISM0ATOuuHiWb0Sv3+5rGifoHyi8ZxGRBngEildtLGQlais8VFwi7CEkGQsBLTu/j0YK27KUNlMxy2DSELcY+aWEsKxyp8OeybcPN5DUZbjyOFeuPgUlooXIu+6VMd+jxhQAtNIbOqPH+9tyXUvz4Cd0o/vD81Czyya1bJOTEpgNHJUFnvc1Ev5Om3526Ilx2mo3FdK3Pdqh8dYy6kO6TBpR5ysDAxYqOoR HvZDwemd cl5pd2vAQG5zdtNJAnHfmmMab1aFw3e7sDUWVlLg6Lay5mkYb2KiVtbtPO3PKNsoc0HaqNfWl7i8ULahPoNKQqViMQqllUyH74dRi1QNmrGEzfdJYRMgbg7ek2+EY2s6DI9fLouRHiiYJx8gBaYit9xYfTaCSWCMHl5sLIjU5LAikZur4v0bZnEJMyq0zPIO+VU9OtxOZkYidUZOJydxc0tmiNJ2YLmAjkYZ5ekGU4yg5Cdk8RNTtpCiDc07jHDwhEghTSrgxxpe/7UDwGjoTQi7v2NhhhthYEDzaxOtgn5Mlk/ZoM5Ul3BE/SiOypzHKXFYYFwwUemsfkNz4V+I53ChGSyff5z3C/53X X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Updating the summary to this because the old one no longer describes the goal of the session well enough. --- I'd like to propose a discussion of how CXL volatile memory device configuration is creating a complex memory initialization/allocation process. This is making it difficult for users to get "what they want". Example issues for discussion: - Linux-arch defined memory block size and alignment requirements - CXL as ZONE_NORMAL vs ZONE_MOVABLE impacts on memmap and performance - NUMA node and memory tier allocations, and new evolutions a) multiple NUMA-nodes per device b) confusing state of memory-tier component - cma and 1GB hugepage allocation from ZONE_MOVABLE on CXL Most of this complexity is being generated by how BIOS, ACPI, build, and boot config options interactions with each other. The best documented description of these interactions is: "Complicated, Subtle, and Scary". Having better documentation on "Linux Expectations for CXL Platforms" will help guide innovations in this space - but requires some agreement on limitations to keep existing components functional. This is becoming increasingly important as more CXL hardware is placed into production. The live session will not cover the full initialization process, which I've attempted to describe and discuss in [1][2][3][4][5][6][7][8]. The intent is to evolve this series into more explicit documentation for platform developers and help bridge the gap between maintainers that need background on "Why the heck does CXL need this?". [1] Section 0: ACPI Tables https://lore.kernel.org/linux-mm/Z8jORKIWC3ZwtzI4@gourry-fedora-PF4VCD3F/ [2] Section 0a: CFMWS and NUMA Flexibility https://lore.kernel.org/linux-mm/Z8u4GTrr-UytqXCB@gourry-fedora-PF4VCD3F/ [3] Section 1: BIOS, EFI, and Early Boot https://lore.kernel.org/linux-mm/Z6LKJZkcdjuit2Ck@gourry-fedora-PF4VCD3F/ [4] Section 2: The Drivers https://lore.kernel.org/linux-mm/Z6OMcLt3SrsZjgvw@gourry-fedora-PF4VCD3F/ [5] Section 2a: CXL Decoder Programming https://lore.kernel.org/linux-mm/Z8o2HfVd0P_tMhV2@gourry-fedora-PF4VCD3F/ [6] Section 3: Memory (block) Hotplug https://lore.kernel.org/linux-mm/Z7OWmDXEYhT0BB0X@gourry-fedora-PF4VCD3F/ [7] Section 4: Interleave https://lore.kernel.org/linux-mm/Z9DQnjPWbkjqrI9n@gourry-fedora-PF4VCD3F/ [8] Section 5: Page Allocation https://lore.kernel.org/linux-mm/Z9Ogp9fCEPoORfnh@gourry-fedora-PF4VCD3F/ ~Gregory