Re: Re: [RFC v2 08/13] vdpa: Introduce process_iotlb_msg() in vdpa_config_ops

[Yongji Xie <xieyongji@bytedance.com>]

On Thu, Dec 24, 2020 at 10:37 AM Jason Wang <jasowang@redhat.com> wrote:
>
>
> On 2020/12/23 下午7:06, Yongji Xie wrote:
> > On Wed, Dec 23, 2020 at 4:37 PM Jason Wang <jasowang@redhat.com> wrote:
> >>
> >> On 2020/12/22 下午10:52, Xie Yongji wrote:
> >>> This patch introduces a new method in the vdpa_config_ops to
> >>> support processing the raw vhost memory mapping message in the
> >>> vDPA device driver.
> >>>
> >>> Signed-off-by: Xie Yongji <xieyongji@bytedance.com>
> >>> ---
> >>>    drivers/vhost/vdpa.c | 5 ++++-
> >>>    include/linux/vdpa.h | 7 +++++++
> >>>    2 files changed, 11 insertions(+), 1 deletion(-)
> >>>
> >>> diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c
> >>> index 448be7875b6d..ccbb391e38be 100644
> >>> --- a/drivers/vhost/vdpa.c
> >>> +++ b/drivers/vhost/vdpa.c
> >>> @@ -728,6 +728,9 @@ static int vhost_vdpa_process_iotlb_msg(struct vhost_dev *dev,
> >>>        if (r)
> >>>                return r;
> >>>
> >>> +     if (ops->process_iotlb_msg)
> >>> +             return ops->process_iotlb_msg(vdpa, msg);
> >>> +
> >>>        switch (msg->type) {
> >>>        case VHOST_IOTLB_UPDATE:
> >>>                r = vhost_vdpa_process_iotlb_update(v, msg);
> >>> @@ -770,7 +773,7 @@ static int vhost_vdpa_alloc_domain(struct vhost_vdpa *v)
> >>>        int ret;
> >>>
> >>>        /* Device want to do DMA by itself */
> >>> -     if (ops->set_map || ops->dma_map)
> >>> +     if (ops->set_map || ops->dma_map || ops->process_iotlb_msg)
> >>>                return 0;
> >>>
> >>>        bus = dma_dev->bus;
> >>> diff --git a/include/linux/vdpa.h b/include/linux/vdpa.h
> >>> index 656fe264234e..7bccedf22f4b 100644
> >>> --- a/include/linux/vdpa.h
> >>> +++ b/include/linux/vdpa.h
> >>> @@ -5,6 +5,7 @@
> >>>    #include <linux/kernel.h>
> >>>    #include <linux/device.h>
> >>>    #include <linux/interrupt.h>
> >>> +#include <linux/vhost_types.h>
> >>>    #include <linux/vhost_iotlb.h>
> >>>    #include <net/genetlink.h>
> >>>
> >>> @@ -172,6 +173,10 @@ struct vdpa_iova_range {
> >>>     *                          @vdev: vdpa device
> >>>     *                          Returns the iova range supported by
> >>>     *                          the device.
> >>> + * @process_iotlb_msg:               Process vhost memory mapping message (optional)
> >>> + *                           Only used for VDUSE device now
> >>> + *                           @vdev: vdpa device
> >>> + *                           @msg: vhost memory mapping message
> >>>     * @set_map:                        Set device memory mapping (optional)
> >>>     *                          Needed for device that using device
> >>>     *                          specific DMA translation (on-chip IOMMU)
> >>> @@ -240,6 +245,8 @@ struct vdpa_config_ops {
> >>>        struct vdpa_iova_range (*get_iova_range)(struct vdpa_device *vdev);
> >>>
> >>>        /* DMA ops */
> >>> +     int (*process_iotlb_msg)(struct vdpa_device *vdev,
> >>> +                              struct vhost_iotlb_msg *msg);
> >>>        int (*set_map)(struct vdpa_device *vdev, struct vhost_iotlb *iotlb);
> >>>        int (*dma_map)(struct vdpa_device *vdev, u64 iova, u64 size,
> >>>                       u64 pa, u32 perm);
> >>
> >> Is there any reason that it can't be done via dma_map/dma_unmap or set_map?
> >>
> > To get the shmfd, we need the vma rather than physical address. And
> > it's not necessary to pin the user pages in VDUSE case.
>
>
> Right, actually, vhost-vDPA is planning to support shared virtual
> address space.
>
> So let's try to reuse the existing config ops. How about just introduce
> an attribute to vdpa device that tells the bus tells the bus it can do
> shared virtual memory. Then when the device is probed by vhost-vDPA, use
> pages won't be pinned and we will do VA->VA mapping as IOVA->PA mapping
> in the vhost IOTLB and the config ops. vhost IOTLB needs to be extended
> to accept opaque pointer to store the file. And the file was pass via
> the config ops as well.
>

OK, I see. Will try it in v3.

Thanks,
Yongji