Re: [External] Re: [RFC 0/4] Introduce VDUSE - vDPA Device in Userspace

[Jason Wang <jasowang@redhat.com>]

On 2020/10/20 下午4:35, Yongji Xie wrote:
>
>
> On Tue, Oct 20, 2020 at 4:01 PM Jason Wang <jasowang@redhat.com 
> <mailto:jasowang@redhat.com>> wrote:
>
>
>     On 2020/10/20 下午3:39, Yongji Xie wrote:
>     >
>     >
>     > On Tue, Oct 20, 2020 at 11:20 AM Jason Wang <jasowang@redhat.com
>     <mailto:jasowang@redhat.com>
>     > <mailto:jasowang@redhat.com <mailto:jasowang@redhat.com>>> wrote:
>     >
>     >
>     >     On 2020/10/19 下午10:56, Xie Yongji wrote:
>     >     > This series introduces a framework, which can be used to
>     implement
>     >     > vDPA Devices in a userspace program. To implement it, the work
>     >     > consist of two parts: control path emulating and data path
>     >     offloading.
>     >     >
>     >     > In the control path, the VDUSE driver will make use of message
>     >     > mechnism to forward the actions (get/set features, get/st
>     status,
>     >     > get/set config space and set virtqueue states) from
>     virtio-vdpa
>     >     > driver to userspace. Userspace can use read()/write() to
>     >     > receive/reply to those control messages.
>     >     >
>     >     > In the data path, the VDUSE driver implements a MMU-based
>     >     > on-chip IOMMU driver which supports both direct mapping and
>     >     > indirect mapping with bounce buffer. Then userspace can access
>     >     > those iova space via mmap(). Besides, eventfd mechnism is
>     used to
>     >     > trigger interrupts and forward virtqueue kicks.
>     >
>     >
>     >     This is pretty interesting!
>     >
>     >     For vhost-vdpa, it should work, but for virtio-vdpa, I think we
>     >     should
>     >     carefully deal with the IOMMU/DMA ops stuffs.
>     >
>     >
>     >     I notice that neither dma_map nor set_map is implemented in
>     >     vduse_vdpa_config_ops, this means you want to let vhost-vDPA
>     to deal
>     >     with IOMMU domains stuffs.  Any reason for doing that?
>     >
>     > Actually, this series only focus on virtio-vdpa case now. To
>     support
>     > vhost-vdpa,  as you said, we need to implement
>     dma_map/dma_unmap. But
>     > there is a limit that vm's memory can't be anonymous pages which
>     are
>     > forbidden in vm_insert_page(). Maybe we need to add some limits on
>     > vhost-vdpa?
>
>
>     I'm not sure I get this, any reason that you want to use
>     vm_insert_page() to VM's memory. Or do you mean you want to implement
>     some kind of zero-copy? 
>
>
>
> If my understanding is right, we will have a QEMU (VM) process and a 
> device emulation process in the vhost-vdpa case, right? When I/O 
> happens, the virtio driver in VM will put the IOVA to vring and device 
> emulation process will get the IOVA from vring. Then the device 
> emulation process will translate the IOVA to its VA to access the dma 
> buffer which resides in VM's memory. That means the device emulation 
> process needs to access VM's memory, so we should use vm_insert_page() 
> to build the page table of the device emulation process.


Ok, I get you now. So it looks to me the that the real issue is not the 
limitation to anonymous page but see the comments above vm_insert_page():

"

  * The page has to be a nice clean _individual_ kernel allocation.
"

So I suspect that using vm_insert_page() to share pages between 
processes is legal. We need inputs from MM experts.

Thanks


>
>     I guess from the software device implemention in user space it
>     only need
>     to receive IOVA ranges and map them in its own address space.
>
>
> How to map them in its own address space if we don't use vm_insert_page()?