Re: [HMM-v25 00/19] HMM (Heterogeneous Memory Management) v25

["Figo.zhang" <figo1802@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 6335 bytes --]

2017-12-14 11:16 GMT+08:00 Jerome Glisse <jglisse@redhat.com>:

> On Thu, Dec 14, 2017 at 10:48:36AM +0800, Figo.zhang wrote:
> > 2017-12-14 0:12 GMT+08:00 Jerome Glisse <jglisse@redhat.com>:
> >
> > > On Wed, Dec 13, 2017 at 08:10:42PM +0800, Figo.zhang wrote:
>
> [...]
>
> > > Basic example is without HMM:
> > >     mul_mat_on_gpu(float *r, float *a, float *b, unsigned m)
> > >     {
> > >         gpu_buffer_t gpu_r, gpu_a, gpu_b;
> > >
> > >         gpu_r = gpu_alloc(m*m*sizeof(float));
> > >         gpu_a = gpu_alloc(m*m*sizeof(float));
> > >         gpu_b = gpu_alloc(m*m*sizeof(float));
> > >         gpu_copy_to(gpu_a, a, m*m*sizeof(float));
> > >         gpu_copy_to(gpu_b, b, m*m*sizeof(float));
> > >
> > >         gpu_mul_mat(gpu_r, gpu_a, gpu_b, m);
> > >
> > >         gpu_copy_from(gpu_r, r, m*m*sizeof(float));
> > >     }
> > >
> >
> > The traditional workflow is:
> > 1. the pointer a, b and r are total point to the CPU memory
> > 2. create/alloc three GPU buffers: gpu_a, gpu_b, gpu_r
> > 3. copy CPU memory a and b to GPU memory gpu_b and gpu_b
> > 4. let the GPU to do the calculation
> > 5.  copy the result from GPU buffer (gpu_r) to CPU buffer (r)
> >
> > is it right?
>
> Right.
>
>
> > > With HMM:
> > >     mul_mat_on_gpu(float *r, float *a, float *b, unsigned m)
> > >     {
> > >         gpu_mul_mat(r, a, b, m);
> > >     }
> > >
> >
> > with HMM workflow:
> > 1. CPU has three buffer: a, b, r, and it is physical addr is : pa, pb, pr
> >      and GPU has tree physical buffer: gpu_a, gpu_b, gpu_r
> > 2. GPU want to access buffer a and b, cause a GPU page fault
> > 3. GPU report a page fault to CPU
> > 4. CPU got a GPU page fault:
> >                 * unmap the buffer a,b,r (who do it? GPU driver?)
> >                 * copy the buffer a ,b's content to GPU physical buffers:
> > gpu_a, gpu_b
> >                 * fill the GPU page table entry with these pages (gpu_a,
> > gpu_b, gpu_r) of the CPU virtual address: a,b,r;
> >
> > 5. GPU do the calculation
> > 6. CPU want to get result from buffer r and will cause a CPU page fault:
> > 7. in CPU page fault:
> >              * unmap the GPU page table entry for virtual address a,b,r.
> > (who do the unmap? GPU driver?)
> >              * copy the GPU's buffer content (gpu_a, gpu_b, gpu_r) to
> > CPU buffer (abr)
> >              * fill the CPU page table entry: virtual_addr -> buffer
> > (pa,pb,pr)
> > 8. so the CPU can get the result form buffer r.
> >
> > my guess workflow is right?
> > it seems need two copy, from CPU to GPU, and then GPU to CPU for result.
> > * is it CPU and GPU have the  page table concurrently, so
> > no page fault occur?
> > * how about the performance? it sounds will create lots of page fault.
>
> This is not what happen. Here is the workflow with HMM mirror (note that
> physical address do not matter here so i do not even reference them it is
> all about virtual address):
>  1 They are 3 buffers a, b and r at given virtual address both CPU and
>    GPU can access them (concurently or not this does not matter).
>  2 GPU can fault so if any virtual address do not have a page table
>    entry inside the GPU page table this trigger a page fault that will
>    call HMM mirror helper to snapshot CPU page table into the GPU page
>    table. If there is no physical memory backing the virtual address
>    (ie CPU page table is also empty for the given virtual address) then
>    the regular page fault handler of the kernel is invoked.
>

so when HMM mirror done, the content of GPU page table entry and
CPU page table entry
are same, right? so the GPU and CPU can access the same physical address,
this physical
address is allocated by CPU malloc systemcall. is it conflict and race
condition? CPU and GPU
write to this physical address concurrently.

i see this slides said:
http://on-demand.gputechconf.com/gtc/2017/presentation/s7764_john-hubbardgpus-using-hmm-blur-the-lines-between-cpu-and-gpu.pdf

in page 22~23：
When CPU page fault occurs:
* UM (unified memory driver) copies page data to CPU, umaps from GPU
*HMM maps page to CPU

when GPU page fault occurs:
*HMM has a malloc record buffer, so UM copy page data to GPU
*HMM unmaps page from CPU

so in this slides, it said it will has two copies, from CPU to GPU, and
from GPU to CPU. so in this case (mul_mat_on_gpu()), is it really need two
copies in kernel space?


>
> Without HMM mirror but ATS/PASI (CCIX or CAPI):
>  1 They are 3 buffers a, b and r at given virtual address both CPU and
>    GPU can access them (concurently or not this does not matter).
>  2 GPU use the exact same page table as the CPU and fault exactly like
>    CPU on empty page table entry
>
> So in the end with HMM mirror or ATS/PASID you get the same behavior.
> There is no complexity like you seem to assume. This all about virtual
> address. At any point in time any given valid virtual address of a process
> point to a given physical memory address and that physical memory address
> is the same on both the CPU and the GPU at any point in time they are
> never out of sync (both in HMM mirror and in ATS/PASID case).
>
> The exception is for platform that do not have CAPI or CCIX property ie
> cache coherency for CPU access to device memory. On such platform when
> you migrate a virtual address to use device physical memory you update
> the CPU page table with a special entry. If the CPU try to access the
> virtual address with special entry it trigger fault and HMM will migrate
> the virtual address back to regular memory. But this does not apply for
> CAPI or CCIX platform.
>

the example of the virtual address using device physical memory is : gpu_r
= gpu_alloc(m*m*sizeof(float)),
so CPU want to access gpu_r will trigger migrate back to CPU memory,
it will allocate CPU page and copy
to gpu_r's content to CPU pages, right?



>
>
> Too minimize page fault the device driver is encourage to pre-fault and
> prepopulate its page table (the HMM mirror case). Often device driver has
> enough context information to guess what range of virtual address is
> about to be access by the device and thus pre-fault thing.
>
>
> Hope this clarify thing for you.
>
> Cheers,
> Jérôme
>

[-- Attachment #2: Type: text/html, Size: 8203 bytes --]