Re: [Ksummit-discuss] [CORE TOPIC] Core Kernel support for Compute-Offload Devices

[Joerg Roedel <joro@8bytes.org>]

Hi Ben,

thanks for your thoughts.

On Fri, Jul 31, 2015 at 08:32:21AM +1000, Benjamin Herrenschmidt wrote:
> > Across architectures and vendors there are new devices coming up for
> > offloading tasks from the CPUs. Most of these devices are capable to
> > operate on user address spaces.
> 
> There is cross-overs with the proposed FPGA topic as well, for example
> CAPI is typically FPGA's that can operate on user address spaces ;-)

True, I was not sure how to put this into the proposal, as FPGAs are a
bit different from other compute-offload devices. GPUs take a kernel to
execute that is basically a piece of software while FPGAs take a
hardware description which in the end might be able to execute its own
software. But there is overlap between the topics, thats right.

> So I'd think that such an off-core scheduler, while a useful thing for
> some of these devices, should be an optional component, ie, the other
> functionalities shouldn't necessarily depend on it.

Yes, of course. The scheduler(s) could be implemented as a library and
optionally be used by the device drivers.

> Right. Some of this (GPUs, MLX) use the proposed HMM infrastructure that
> Jerome Glisse have been developing, so he would be an interested party
> here, which hooks into the existing MM. Some of these like CAPI (or more
> stuff I can't quite talk about just yet) will just share the MMU data
> structures (direct access to the host page tables).

Everything (what I am aware of), besides of the hardware HMM targets,
reuses the CPU MMU structures :) For example all three hardware
implementations of ATS/PRI/PASID I am aware of can share them, and as
you said, CAPI on Power too.

But they also need to attach some state to mm_struct. As David already
said, there will be a need to a global PASID allocation, for example.



	Joerg