From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99CDBC282DE for ; Thu, 13 Mar 2025 18:18:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7F26528000F; Thu, 13 Mar 2025 14:18:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 77ECA280007; Thu, 13 Mar 2025 14:18:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5F85A28000F; Thu, 13 Mar 2025 14:18:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 3FC47280007 for ; Thu, 13 Mar 2025 14:18:02 -0400 (EDT) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id A749D4546D for ; Thu, 13 Mar 2025 18:18:02 +0000 (UTC) X-FDA: 83217336804.08.1B8A5B5 Received: from mail-qk1-f181.google.com (mail-qk1-f181.google.com [209.85.222.181]) by imf22.hostedemail.com (Postfix) with ESMTP id B92CCC000B for ; Thu, 13 Mar 2025 18:18:00 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b=roSJonlu; dmarc=none; spf=pass (imf22.hostedemail.com: domain of gourry@gourry.net designates 209.85.222.181 as permitted sender) smtp.mailfrom=gourry@gourry.net ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741889880; a=rsa-sha256; cv=none; b=PTuI6sVWLnn1i/nkCmwfWvRK+/cKCRSwJWbZp4Yc2riTeWrefiH/EeEBS3D44IDuX7L07n Nr5qGv5Vr7eKeJ2qUZRrkRJ2w44RO9E+ptOyBLddbIhUiUe86W6Na90324rk77XRxZBTVi V14GOzlOxMe084KfaR094OuwMrY9+C8= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=gourry.net header.s=google header.b=roSJonlu; dmarc=none; spf=pass (imf22.hostedemail.com: domain of gourry@gourry.net designates 209.85.222.181 as permitted sender) smtp.mailfrom=gourry@gourry.net ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741889880; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UpgcW7UU29tzc2HSUMpC8m7j2QavoELN53utjERwIQo=; b=4eeAm6pFqVLdXsQp5Gb3muRB1Z8Otz6IEeGMHut6cKd3BggnYHjEVFvEh+01vtnsRjXxXe TIBiGI5CPkBqDiH1VdDta6AZpVybrBR1G4Kf/i3HSYA7ceSuO6fc6S7BH4Z4u2M0mS9Kr5 Z+lR2eggt5wzeK6oNGiCTpzTdUWcfbw= Received: by mail-qk1-f181.google.com with SMTP id af79cd13be357-7c0155af484so166576785a.0 for ; Thu, 13 Mar 2025 11:18:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1741889880; x=1742494680; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=UpgcW7UU29tzc2HSUMpC8m7j2QavoELN53utjERwIQo=; b=roSJonluod/peUV5NkWXFKHiOX1YSx0X//0J657xpqgfdOBvOPikM/YHPELqppTlvh jKPbbx+ZXqYcaSgogIKEUdNKpdj5+xGuYfYa8XJN192xfgSV3jaszLLYf1eVIh5Jqa9s te8bQB0dBgLOTorCBVKI18zy62r03Yf6Fcc7t4MrIVttSacdrBqtNTXZTyqbijMDDWQG DfOHL3idINqQz9vIV9+NNA+jfSrT0V86Ec5QBeu3o7KrEpTv/TYb0Ag3o8ka/UEegJyK bnI5Cv7CXewLHOM7pHv7FTLXvSbcIBXrxGT16k2XJACvO3yaZVeCe/19p5VLqSa2Ao7U btHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741889880; x=1742494680; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=UpgcW7UU29tzc2HSUMpC8m7j2QavoELN53utjERwIQo=; b=PVHbRwUX4MxSJHCO4eunS5kNrUjQ8uRhJJnMU4OB2wJbqEOaJC+qdZ5QgcJtAXGiq8 8B98LbdhfPXGxskHoLJvj2khIQ1pOPsf8T73Arj93okA0yXTBGurUh2PAPRvF+8Xzi6w DgT/XR/iv9iCcLHptyogNBgq3cQGNx0mjolaJRUpmJq5jOWC+bxmEIyBSGDakuue/rmF qwkEycqQecnFnALgAo41V/e40n2+FnTt8VXc1bBSmp6D9al71o1p8snbZEhg9iZoW4zq hyYG9q4jp00K/30gAJyE35n14MCaGpgprGeYGth3LZCxS21c4zd7Ue1kPo4iohJd5REK qTDg== X-Forwarded-Encrypted: i=1; AJvYcCV/ZzG0/FUVANzaqMRH5EaxB/plUIt25HwOMb3qUFwxPjS8CiSK+UlLdeBIRPiPtJA9pKVVkZ1VBg==@kvack.org X-Gm-Message-State: AOJu0YzKsbxmQIn6LDxo6OGjIrBRmnvFIXyrae5Rj1IfmMC1qXzExkph 3af+J5HZvvnT4jTKibavLVKu5n1d4JRtx8plHdJ0QBrObB1C5QuSnNg91gfbmT4= X-Gm-Gg: ASbGncsfbE0lARViMJwVipyjIFnDNWYI8EXN5JBDOrMdEu+o7qhkTwcrkibhmMhFxma Tw3OPMu5keYUEUVvN8rK6JI7HMpKm2/d+4MqnJxm+HsgLSfY/4PYUfmLdesMXMtW7fzNs0altY6 K6dbodDn0JH4VXK4d6Jc0nxfqa/lJhYTLSZkWvk9DDGS2e0qIQ2srtI8CkodlJTkLDRgbxXEM7/ D+JVQnkIZ3y77JqVZtYzCnwM2toQqy30aBHYZdKvHB8j9+lk93N1QvyEhi6ynh8qq0rp+RrScJK V4qiLI3Ec1X+JsrEEsYu6aDf9cqOYXVN1X+PejgxlIY2o/bO/yk/tFoLaSKFS/2buH9otcZ7x6r IeIF2Y0vSJzzY3Hnb0wr1I42P9bE= X-Google-Smtp-Source: AGHT+IFU07FAOug463Uwa+DOe07sChN0wwFnYBffXB/Iobm2IJ99vzyHde0TeD4Rsxvo8fielsuN7Q== X-Received: by 2002:a05:620a:2606:b0:7c5:59a6:bad7 with SMTP id af79cd13be357-7c579ebc8afmr95372085a.17.1741889879813; Thu, 13 Mar 2025 11:17:59 -0700 (PDT) Received: from gourry-fedora-PF4VCD3F (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7c573c9be48sm127266785a.60.2025.03.13.11.17.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Mar 2025 11:17:59 -0700 (PDT) Date: Thu, 13 Mar 2025 14:17:57 -0400 From: Gregory Price To: Jonathan Cameron Cc: lsf-pc@lists.linux-foundation.org, linux-mm@kvack.org, linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: [LSF/MM] CXL Boot to Bash - Section 0a: CFMWS and NUMA Flexiblity Message-ID: References: <20250313172004.00002236@huawei.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20250313172004.00002236@huawei.com> X-Stat-Signature: 58658f3q4mmu6tcwxo4kxt3s1cqqtbz4 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: B92CCC000B X-Rspam-User: X-HE-Tag: 1741889880-667815 X-HE-Meta: U2FsdGVkX18wMFNuVVFRcPn+2LPbjQjKfj63LDKhlarXgwYGT7JqINgLs4SNRvssQIUsebkvVDGjgHqFAEv1gl2RzcMPaVqOk2zXx1K20q77V1VNuw17LYUuvhJowSkD1oL3NfXb21mYGA7zWqPwQN16C7Y9k8DPnDULYBOPWL6jc8KVtHupr4+z3ejbwlRKa+FBo30Dqo/DAzYyuAt9EaGjhKFKVeeg069QNjAiHOc4HpS9kHkvBXrz4u1QMM8GoJy3Zgoc1dkUFKRkvB7bsW07MwvFrL6lel78ogledCszOQlfNYdr6n0c+AYhX9cI6PrQpzXOHRdGBeKX2oPC2MC+4tsW75vCchu3iDZ49AAAi8M4iIiEr5Xz8AyLq5fuL7hkGUwycd0JMZ4em51S4TWRrmzY3W9blwh2lE0ZhF0Dt0uKtoN+8a/J8vIO8cBcUpDFzoA7z0XcVk0eh18jzJIAspEf6/cNKNcLYJw7WBCd3UACwL8GEwG0YHHBOb7P5+db+SXcnKkkGJd4A60ro2BaF97vWtF22GA7EmdnKG9Dk8Kt41GOOFx9Vat9bAKGjNC2KVCpjjK9R/r329h2X7dTHL2xhbJKftRZIFOruBxZdhBIpfxzu23OByIi3H0NS9Mv5IJOjy9if89wDQR+0D9MOxW+FL+avoq0gnC8XSN81A1MlSWNRh9UfancufVrEsQpJNdaWLE8Otw25FqDbcxsPRA9PUp3g2Y+fFYhOArx+px8egDziirCyfCElzbev/jRdrGQ+pbEk4B8N4a2LEtrpscjVJUDDvvbOvw4FZcUhh/aZgouaHoX0DMIeKn1RMRQIPLS7SdBiDIXIHuiZbR/79TvVQUSw4/FrlDxU7mCMFD/16IZMCYjBg9kbulh6ydWUc8BnvuXZD1xj4eVdGomyn12miOg3gDymXRUcgBDZdTOfNeoZ4T4nUFfVFyDSL+guQRox4JAU2OXfNR VwIcCMuq seG2knetraQZrDeVV6Hn3MFLIFK2wg5EtnrzCAgK5B029fegvVjkQoZcJVkOXC9lDqCGpA37uWb6MfwFxsO2Cg270gNum6nDULXliOB0ry8MsdyYpLMC4ynQU3bdoJ9qIawzi9Fan8VO8Mn+FmL6fIohikpJS3XM/QulWJz7JpeR8Mi54s0X83iZwce4w98asCn7e7wTRGX/ZIYjm9QL+7KZsEp/asaoyL3WyXMfQpRh235a2PH7sfgGhGO3H/xKUSV4B3VRTOm3iBF9Vq2o4y/zrYW3W5HsakYm2UGT6jmgezEYCPYmb6EX2lEBoNBx3HB+luCIwRd+k2Us/wasYrZouv0d1CJaPAwaKE/7KK3LFZsvl02U1k91FENaKZSEyotZv X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Mar 13, 2025 at 05:20:04PM +0000, Jonathan Cameron wrote: > Gregory Price wrote: > > > ------------------------------- > > One 2GB Device, Multiple CFMWS. > > ------------------------------- > > Lets imagine we have one 2GB device attached to a host bridge. > > > > In this example, the device hosts 2GB of persistent memory - but we > > might want the flexibility to map capacity as volatile or persistent. > > Fairly sure we block persistent in a volatile CFMWS in the kernel. > Any bios actually does this? > > You might have a variable partition device but I thought in kernel at > least we decided that no one was building that crazy? > This was an example I pulled from Dan's notes elsewhere (i think). I was unaware that we blocked mapping persistent as volatile. I was working off the assumption that could be flexible mapped similar to... er... older, non-cxl hardware... cough. > Maybe a QoS split is a better example to motivate one range, two places? > That probably makes sense? > > ------------------------------------------------------------- > > Two Devices On One Host Bridge - With and Without Interleave. > > ------------------------------------------------------------- > > What if we wanted some capacity on each endpoint hosted on its own NUMA > > node, and wanted to interleave a portion of each device capacity? > > If anyone hits the lock on commit (i.e. annoying BIOS) the ordering > checks on HPA kick in here and restrict flexibility a lot > (assuming I understand them correctly that is) > > This is a good illustration of why we should at some point revisit > multiple NUMA nodes per CFMWS. We have to burn SPA space just > to get nodes. From a spec point of view all that is needed here > is a single CFMWS. > Along with the above note, and as mentioned on discord, I think this whole section naturally evolves into a library of "Sane configurations" and "We promise nothing for `reasons`" configurations. Maybe that turns into a kernel doc section that requires updating if a platform disagrees / comes up with new sane configurations. This is certainly the most difficult area to lock down because we have no idea who is going to `innovate` and how. ~Gregory