From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 952B3E77188 for ; Thu, 26 Dec 2024 20:19:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 239FF6B0082; Thu, 26 Dec 2024 15:19:27 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1EA216B0085; Thu, 26 Dec 2024 15:19:27 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 08A786B0089; Thu, 26 Dec 2024 15:19:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id DE4BF6B0085 for ; Thu, 26 Dec 2024 15:19:26 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 8DD95C0959 for ; Thu, 26 Dec 2024 20:19:26 +0000 (UTC) X-FDA: 82938223158.08.E11359F Received: from mail-qk1-f182.google.com (mail-qk1-f182.google.com [209.85.222.182]) by imf13.hostedemail.com (Postfix) with ESMTP id 540B620003 for ; Thu, 26 Dec 2024 20:18:44 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b=CTG2L9i2; spf=pass (imf13.hostedemail.com: domain of gourry@gourry.net designates 209.85.222.182 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735244332; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=qz5ytK1GRdvKgO8e6oKZGmSzgE/hU2itL0CRU+M0PxQ=; b=5gkiIYVqifBUS0uh5UG/aDX22NPDM4BmnBVAgY6Wsdho6G1tLN1/N7U2r8FDWNkUw13zeV C71htfMMAyaL7SP0nnGyw8RRNvobhTT/7yTTs3YYv68D+zkGU/NvOW5aX9bkswRDbzHD6w PQ0NWo2/CH3sJ25PYdqRmO3WqBsQ53M= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b=CTG2L9i2; spf=pass (imf13.hostedemail.com: domain of gourry@gourry.net designates 209.85.222.182 as permitted sender) smtp.mailfrom=gourry@gourry.net; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735244332; a=rsa-sha256; cv=none; b=zkDTKBunb8Sf49zkvEzQZPhNCqfrL6gc/+2lqjwd2Iop7s8YUR/AwdR4R3N+IDX70h7FQz RkfdsMQ2aLV6n7oECut4TondXxsv9wQBFDeQsWVnc2BXy2AlPvqIkqNjqFbn2WC91IORlx I4sP7j01nDhXU3ciXaOo9sLU4TmYwsk= Received: by mail-qk1-f182.google.com with SMTP id af79cd13be357-7b98a2e3b3eso315558985a.2 for ; Thu, 26 Dec 2024 12:19:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1735244364; x=1735849164; darn=kvack.org; h=content-disposition:mime-version:message-id:subject:cc:to:from:date :from:to:cc:subject:date:message-id:reply-to; bh=qz5ytK1GRdvKgO8e6oKZGmSzgE/hU2itL0CRU+M0PxQ=; b=CTG2L9i2xAdZlpZvQTNneTs9lMqRNEJ3FtJKDoPmnNGiX1+rUSkiUtHnl7M8grUti0 TK+09bSQo8iBiH1/UwVmA/Xf+Va/bISZYiuR5IJx9HOk4JP/tY5T5FB47mFwEkiVkvzm VhwD5wN5Rnyr/Ws1Aj+IyKpKH/cGY9IR6lkQezbfWDBEiVr6PVG7rjdTzB+gjvH0zHxv 2UAhXz5rKBRE8+LMuNYQGdbKXWpOS/gbai1JER/jnrnUbh9MkKi1Yl4tD7VmFg2kwNI2 Xt4efsMnRZsT3kHHlWdcZCDMTqad7hL3t6k8cDquZCMgdWEwHEM+Nt1bQ6codAVZeuo/ Ka1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735244364; x=1735849164; h=content-disposition:mime-version:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=qz5ytK1GRdvKgO8e6oKZGmSzgE/hU2itL0CRU+M0PxQ=; b=coNfrLUbCv8IuGLZGKYs7w38AGNhOEsn2jKhF4R3CNz/VIPgVPOPKU3TMTUObf1Bbn 3nt3edc0Z2WeIVenyv93Wgfb0RTIfxWFZkZK0Buda3i8OiMlqc2imTVG+tqSywQB44Fx cIxucAwx+rusAS73roJ1LQmivw5uRNCAFGcPeMC4iqGim5d4cdBEFGl5jDBgRUiSxLoh BkecOEr/R1NRGO0cOnp72sEOELQbPjfBczwPhMpb4J5ri2wcQQschc3zRf9m8rP42TZJ TqJY1TE5ihODif11cvlMqlHHvp/hPlrj3htBHwsc6xlCzV5fmoWF7QXCW+HrROJ8JOwA jq8w== X-Gm-Message-State: AOJu0YzaAyUb3L/qHWSoFJZdxGRWHW6upUxZT2pRTfzAAWdeEIhJBOBI 5646vUQQ1xQeK5unRRUpx+fm6rhPxuCsk54D1JeB9i443t43vV9KXrhARxgxdPY= X-Gm-Gg: ASbGncudacEIzbZ5vwZ0VeDvr9EOxVzY2mXWQ5vU1qYmmKD3SeWj1+4DyZwvG3BtxgJ 2ikaSqUMYgT4xMTaWto/Mz5kAGG1Q1IAMbYd4RiYoT6mLuOBmnUkG4h9AJYEQbnHcou3SyS+/U/ frOENp8F062ysE2zVN5TcQI8yYJJDcwroj4sQsPnK61dl7RpZr7Yk9rzC4/Q23/wgUYcWK8D+xI 9OBMYsarB6on58SvsHRk6zkaV3rdM8ilfKMOjq31Lb6ZoMRPZ6h9zJTFpjThI7+ X-Google-Smtp-Source: AGHT+IHwbSBUELCWH88to3eKbIbKFQlIGmiEO20KuDJPlMSg8m7xY1XIjb3s2ATL2I5p1b8aOzoeHQ== X-Received: by 2002:a05:620a:4551:b0:7b6:6c46:55b with SMTP id af79cd13be357-7b9ba6efbbamr4034804785a.7.1735244363986; Thu, 26 Dec 2024 12:19:23 -0800 (PST) Received: from gourry-fedora-PF4VCD3F ([184.169.45.4]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7b9ac2d1054sm648499485a.32.2024.12.26.12.19.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 26 Dec 2024 12:19:23 -0800 (PST) Date: Thu, 26 Dec 2024 13:19:08 -0700 From: Gregory Price To: lsf-pc@lists.linux-foundation.org Cc: linux-mm@kvack.org, linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [LSF/MM] Linux management of volatile CXL memory devices - boot to bash. Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Rspamd-Queue-Id: 540B620003 X-Rspamd-Server: rspam12 X-Stat-Signature: 458uumqbb5mwshz77i8uwrtxmnnfec3j X-Rspam-User: X-HE-Tag: 1735244324-6099 X-HE-Meta: U2FsdGVkX1+9xRlBpUaDGigem3VkBxE9rXtEwGAuwQgDsLdnKklrDV1bjJScTPOEDlAI78BjYPD7+yf76BEOdLLN84mDLSRHl8zHfzGKJjiwcuTkiO6AN1eSONhnRAPulJd9iWsABKifDloCjKkrwjgFQ9ocKsOSS8CiNOWVQx2vsrHm/aQ+jqsV4cJObt4Twar1bqsJWT3C/UF6uV+m4Zhj5Ca2fAFVYSZrC1A/7o+UmmjZnr64FpFAtyT4mf6yUH88YEZFHzxHOkl1Q8NLFS0yEiHm1pA9zOVqFQS1ps7HNbbaQlXyW5a0nNwqT30ZgEO5DmeKlR4qfGYxHZvR5pd8Ol7iKGGDZaobTe03AsUBV2DUMcP8WLE6XgONjS8GxhZR+FzworTLshwpQb2vgsgYEjPsFZSJxR54LtXMzDWTQAFfIxcYsVtFxQHf+eJ9DDApEFi2wmDDIVFu5D5UgcAk6Sl06RxFsf8O1IcMVqNhf26EX3vEmfFfGjUDe3v/Q2/1I2f8Ai2PnGzo0mN4y7SHJQVp6deHD/LitePSpuA6NbFpzbYYuEWy36eC6d+rtxsFycAFFvhUIcGiI1ROYh5FyeozqghkoiI/t3HoVPuyi/L4Vt9oEZsuQhI/zcsbIvMnCwqA9SUVLnzAJrKH7VcJyRf+qb/kRw7o8sA4iuUFlz40YGoe/xdNOi56xWdRsAaKZpSyAuBmQZc+4QzBE6w6Y8IrhN1N+ifLIbFDMWM/cs1uEaNvwbE1KdKblcJ82yHoBasphQXw0GYk2D4vxTmt3EeP8CZPrAwMw4fPTA8LlT+jzdTjxXUtwg6GphkzY4BFgRJrHv8lJaTAf+dBTO2F4H+fLIPZ11IKtOqpNoGJVHX24uMYawu8U3lixJUwRhdVEtJWGvr2R16nzDe772j95xL86d7t7Di/40uqK/ssYGZWuvAKymJ39Y8NKcCGRFBH7Yo7tADtCjeHoLl +sCX8ahY wlZet9kksC2ZQsurN58fzBj2JXsRORICY6Rw1jRlTTVOu357Uf7VQ82TZKDUru1z+qcwUcF3/oVGj7p4P0Uy4ymy1UPu2Cj20bLN7m0KeL+n+Qv0+uOABPrjZknK0ac0QWSf3Gczk+QcKFRB4F9EPl/ua2THhfmXQVDUmvlpS7ad/VODpfeBBdglJFrrbnYAdZPb3mawuUTuj8bwQuEk41CXLGsvKCLXycWcVqPFCdJh96lXmeQFkJwlFmL2PUjDd7gaGflw5oyLMsuOhBeNh4L2Lg4IjxSt0jUxfGmnLdYf9DMUrl7W4JrlzrgoWr8AH/v22aeVRoERODQM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.343625, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: I'd like to propose a discussion about the variety of CXL configuration decisions made by platforms, BIOS, EFI, and linux that have created a complex, and sometimes subtly confusing administration environment. In particular, when and how memory configuration occurs have major implications for major feature support (interleave, ras, hotplug, etc). For example, treating CXL memory as "conventional" without marking is "special purpose" may limit the applicability of certain RAS features - but even marking it "special purpose" may be insufficient for (coordinated) device hotplug compatibility. Another example, RAS features like POISON have different end-state implications (full system crash vs userland crash) that depend on whether the memory was being used by kernel or userland (which is actually somewhat controllable, and therefore useful for administrators!) Some of this complexity stems from interleave settings and how CXL memory is distributed among NUMA nodes (1 per device, 1 per homogeneous set, or a single heterogeneous numa node). Specifically we'll talk about - iomem resource allocation - EFI_CONVENTIONAL_MEMORY, MEMORY_SP, and CONFIG_EFI_SOFT_RESERVE - e820 & EFI mmemory map inclusion - driver-time allocation - hotplug implications - Addressing - SPA == HPA vs SPA != HPA - Boot-time configuration vs Driver Configuration - Interleave configuration - Platform configuration vs Driver configuration - PRMT-provided translation - RAS feature implications - Management implications (hotplug, teardown, etc) - Linux Memory (Block) Hotplug - auto-online vs user-policy - systemd / typical user story - Zone-assignment and Poison I'd like to lay out (as best I can, with help!) the current environment in linux kernel, the "maintenance implications" of certain configurations decisions, and discuss where ambiguities are present / challenging. I'll add some additional follow-on emails that break down some of these scenarios more in-depth over the next few months for some background reading. ~Gregory