On Tue, Oct 20, 2020 at 5:13 PM Jason Wang wrote: > > On 2020/10/20 下午4:35, Yongji Xie wrote: > > > > > > On Tue, Oct 20, 2020 at 4:01 PM Jason Wang > > wrote: > > > > > > On 2020/10/20 下午3:39, Yongji Xie wrote: > > > > > > > > > On Tue, Oct 20, 2020 at 11:20 AM Jason Wang > > > > >> wrote: > > > > > > > > > On 2020/10/19 下午10:56, Xie Yongji wrote: > > > > This series introduces a framework, which can be used to > > implement > > > > vDPA Devices in a userspace program. To implement it, the > work > > > > consist of two parts: control path emulating and data path > > > offloading. > > > > > > > > In the control path, the VDUSE driver will make use of > message > > > > mechnism to forward the actions (get/set features, get/st > > status, > > > > get/set config space and set virtqueue states) from > > virtio-vdpa > > > > driver to userspace. Userspace can use read()/write() to > > > > receive/reply to those control messages. > > > > > > > > In the data path, the VDUSE driver implements a MMU-based > > > > on-chip IOMMU driver which supports both direct mapping and > > > > indirect mapping with bounce buffer. Then userspace can > access > > > > those iova space via mmap(). Besides, eventfd mechnism is > > used to > > > > trigger interrupts and forward virtqueue kicks. > > > > > > > > > This is pretty interesting! > > > > > > For vhost-vdpa, it should work, but for virtio-vdpa, I think we > > > should > > > carefully deal with the IOMMU/DMA ops stuffs. > > > > > > > > > I notice that neither dma_map nor set_map is implemented in > > > vduse_vdpa_config_ops, this means you want to let vhost-vDPA > > to deal > > > with IOMMU domains stuffs. Any reason for doing that? > > > > > > Actually, this series only focus on virtio-vdpa case now. To > > support > > > vhost-vdpa, as you said, we need to implement > > dma_map/dma_unmap. But > > > there is a limit that vm's memory can't be anonymous pages which > > are > > > forbidden in vm_insert_page(). Maybe we need to add some limits on > > > vhost-vdpa? > > > > > > I'm not sure I get this, any reason that you want to use > > vm_insert_page() to VM's memory. Or do you mean you want to implement > > some kind of zero-copy? > > > > > > > > If my understanding is right, we will have a QEMU (VM) process and a > > device emulation process in the vhost-vdpa case, right? When I/O > > happens, the virtio driver in VM will put the IOVA to vring and device > > emulation process will get the IOVA from vring. Then the device > > emulation process will translate the IOVA to its VA to access the dma > > buffer which resides in VM's memory. That means the device emulation > > process needs to access VM's memory, so we should use vm_insert_page() > > to build the page table of the device emulation process. > > > Ok, I get you now. So it looks to me the that the real issue is not the > limitation to anonymous page but see the comments above vm_insert_page(): > > " > > * The page has to be a nice clean _individual_ kernel allocation. > " > > So I suspect that using vm_insert_page() to share pages between > processes is legal. We need inputs from MM experts. > > Yes, vm_insert_page() can't be used in this case. So could we add the shmfd into the vhost iotlb msg and pass it to the device emulation process as a new iova_domain, just like vhost-user does. Thanks, Yongji > > > > > > I guess from the software device implemention in user space it > > only need > > to receive IOVA ranges and map them in its own address space. > > > > > > How to map them in its own address space if we don't use > vm_insert_page()? > >