Re: [LSF/MM TOPIC] CAPI/CCIX cache coherent device memory (NUMA too ?)

["Figo.zhang" <figo1802@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 2376 bytes --]

2018-01-17 5:03 GMT+08:00 Jerome Glisse <jglisse@redhat.com>:

> CAPI (on IBM Power8 and 9) and CCIX are two new standard that
> build on top of existing interconnect (like PCIE) and add the
> possibility for cache coherent access both way (from CPU to
> device memory and from device to main memory). This extend
> what we are use to with PCIE (where only device to main memory
> can be cache coherent but not CPU to device memory).
>

the UPI bus also support cache coherency for Intel platform, right?
it seem the specification of CCIX/CAPI protocol is not public, we cannot
know the details
about them, your topic will cover the details?


>
> How is this memory gonna be expose to the kernel and how the
> kernel gonna expose this to user space is the topic i want to
> discuss. I believe this is highly device specific for instance
> for GPU you want the device memory allocation and usage to be
> under the control of the GPU device driver. Maybe other type
> of device want different strategy.
>
i see it lack of some simple example for how to use the
HMM, because GPU driver is more
complicate for linux driver developer  except the ATI/NVIDIA developers.

>
> The HMAT patchset is partialy related to all this as it is about
> exposing different type of memory available in a system for CPU
> (HBM, main memory, ...) and some of their properties (bandwidth,
> latency, ...).
>
>
> We can start by looking at how CAPI and CCIX plan to expose this
> to the kernel and try to list some of the type of devices we
> expect to see. Discussion can then happen on how to represent this
> internaly to the kernel and how to expose this to userspace.
>
> Note this might also trigger discussion on a NUMA like model or
> on extending/replacing it by something more generic.
>
>
> Peoples (alphabetical order on first name) sorry if i missed
> anyone:
>     "Anshuman Khandual" <khandual@linux.vnet.ibm.com>
>     "Balbir Singh" <bsingharora@gmail.com>
>     "Dan Williams" <dan.j.williams@intel.com>
>     "John Hubbard" <jhubbard@nvidia.com>
>     "Jonathan Masters" <jcm@redhat.com>
>     "Ross Zwisler" <ross.zwisler@linux.intel.com>
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>

[-- Attachment #2: Type: text/html, Size: 3725 bytes --]