Re: [LSF/MM] CXL Boot to Bash - Section 1: BIOS, EFI, and Early Boot

[Gregory Price <gourry@gourry.net>]

On Tue, Feb 18, 2025 at 06:12:47PM +0800, Yuquan Wang wrote:
> On Tue, Feb 04, 2025 at 09:17:09PM -0500, Gregory Price wrote:
> > 
> >    1) This memory may be placed in any zone (ZONE_NORMAL, typically)
> >    2) The kernel may use this memory for arbitrary allocations
> >    4) The driver still enumerates CXL devices and memory regions, but
> >    3) The CXL driver CANNOT manage this memory (as of today)
> >       (Caveat: *some* RAS features may still work, possibly)
> > 
> 
> Hi, Gregory
> 
> Thanks for the in-depth introduction and analysis.  
> 
> Here I have some confusion:
> 
> 1) In this scenario, does it mean users could not create a CXL region
> dynamically after OS boot? 
> 

It helps to be a bit more precise here.

"A CXL Region" is a device managed by the CXL driver:

  /sys/bus/cxl/devices/regionN

In this setup, a "CXL region" is not required, because the memory has
already been associated with memory blocks in ZONE_NORMAL by the kernel.

The blocks themselves are not managed by the driver, they are created
during early boot - and are not related to driver operation at all.

The driver can still enumerate the fabric and the devices that back this
memory, but presently it does not manage the memory blocks themselves.

more explicitly:  There is no link between a memdev and memory blocks,
which would normally be created via a region+dax_region+dax device.


> 2) A CXL region (interleave set) would influence the real used memory
> in this memory range. Therefore, apart from devices, does platforms
> have to configure CXL regions in this stage?
> 

Again, you need to be more explicit about "CXL region".  A "CXL region
device" is a construct created by the driver.  In this scenario, the
platform configures the CXL memory for use as normal system RAM by
marking it EFI_CONVENTIONAL_MEMORY without EFI_MEMORY_SP.

Some platforms configure interleave in BIOS - how this is done is
platform specific but ultimately constrained by the CXL specification on
programming decoders throughout the fabric.

> 3) How bios/EFI to describe a CXL region?
> 

You would have to discuss this with the individual platform folks.

The main mechanism to communicate CXL configuration from BIOS/EFI to
kernel is the CEDT/CFMW and HMAT.

~Gregory.