From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3980BC433E0 for ; Wed, 30 Dec 2020 07:09:18 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 867B9207C8 for ; Wed, 30 Dec 2020 07:09:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 867B9207C8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C7C6C8D00A1; Wed, 30 Dec 2020 02:09:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C4F468D00A0; Wed, 30 Dec 2020 02:09:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B189C8D00A1; Wed, 30 Dec 2020 02:09:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0163.hostedemail.com [216.40.44.163]) by kanga.kvack.org (Postfix) with ESMTP id 998F88D007F for ; Wed, 30 Dec 2020 02:09:16 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 51A0F8249980 for ; Wed, 30 Dec 2020 07:09:16 +0000 (UTC) X-FDA: 77649072312.05.soap31_1406d2c274a3 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin05.hostedemail.com (Postfix) with ESMTP id 30B6E180238AA for ; Wed, 30 Dec 2020 07:09:16 +0000 (UTC) X-HE-Tag: soap31_1406d2c274a3 X-Filterd-Recvd-Size: 6559 Received: from mail-ej1-f49.google.com (mail-ej1-f49.google.com [209.85.218.49]) by imf24.hostedemail.com (Postfix) with ESMTP for ; Wed, 30 Dec 2020 07:09:15 +0000 (UTC) Received: by mail-ej1-f49.google.com with SMTP id ce23so20843757ejb.8 for ; Tue, 29 Dec 2020 23:09:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=l8j4LBcYSM3A9J/lyobN/Td5iPDnCHBqIY/lSxbOPoI=; b=an2vDV1/CDDLLSUvQH6Uvml6j8CxlkgAoU852XXsrwcYFpo5Ao01qFD5HAOuScQ+qa QT3/PEoDuXvZdIc3jQfO05AoGVp8vIC6pI0hZBL+jA0VmErjD1qwAbj8HHm6s0ZO4EWi QcYdggbqXxbShmXxf5mFh1jWSRWQe8ZIiP2WbCTnMFvvUFncKOinCVEORK9dYWS7Srq2 F2tyiD7J/Pq/Dp/lkoub23RdxuPhavSZPsXlGQGJ13t08izTxWrSz97txktHmXo2EqrK lwl2krXf/q/AAWhJkgwnSu3ndkhVAy3gtvXeReSa5rG/wgPR/MHX2nNOcHFN5YUyI5Ot xfDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=l8j4LBcYSM3A9J/lyobN/Td5iPDnCHBqIY/lSxbOPoI=; b=juiwLvC5ljAsNPfLf9sWIh9euUxRmz4R11XJ4fKyrKn0cggjUAHN6BsnTvDVVMcWXh Lp+WnxDVufMP1hjzEc1Xkn29iGJUQIvvCQ6kC6o2htdCGXUhSrQObwFjqCw/9K5GEh23 b4tSJTtX4ci4fh2GSNlT05z5H3yfochLQeze1xRYhVGEzKyxa7dxGCJMLwTJFvm0BbM8 wDswOuPPb/0nxV0WI/cqhRHRIN17DFGnRuJ6mX4nTMiGLfoDayps3bFdiTKJQ/EYz+3s matG67TCKKOLGa5Z9Gnl8eNz6kQeQka13mVra/8do6ii/thmlIOXYQnDIkxTEZuG0o2C 9dnw== X-Gm-Message-State: AOAM533ob76ZFWF4ExSkJPHyk2Pcm5FdvGWCWVpuO6b9GFvv0NCjQsbM 9Kg6WcgaLC+KSiUX7fqcVzTGZIvRgKYcVKnk3fcA X-Google-Smtp-Source: ABdhPJxSggzhnaXhPdSiv6Fgwwsm7jrYKdGJWgqP2yGg8R6DbCqwUHnGXSyfiUFMd9tVV3XFaaGE9JgwBqfN+ujcONk= X-Received: by 2002:a17:907:961e:: with SMTP id gb30mr46516203ejc.197.1609312154151; Tue, 29 Dec 2020 23:09:14 -0800 (PST) MIME-Version: 1.0 References: <20201222145221.711-1-xieyongji@bytedance.com> <0e6faf9c-117a-e23c-8d6d-488d0ec37412@redhat.com> <2b24398c-e6d9-14ec-2c0d-c303d528e377@redhat.com> <1356137727.40748805.1609233068675.JavaMail.zimbra@redhat.com> In-Reply-To: From: Yongji Xie Date: Wed, 30 Dec 2020 15:09:03 +0800 Message-ID: Subject: Re: Re: [RFC v2 09/13] vduse: Add support for processing vhost iotlb message To: Jason Wang Cc: "Michael S. Tsirkin" , Stefan Hajnoczi , sgarzare@redhat.com, Parav Pandit , akpm@linux-foundation.org, Randy Dunlap , Matthew Wilcox , viro@zeniv.linux.org.uk, axboe@kernel.dk, bcrl@kvack.org, corbet@lwn.net, virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, kvm@vger.kernel.org, linux-aio@kvack.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Dec 30, 2020 at 2:11 PM Jason Wang wrote: > > > On 2020/12/29 =E4=B8=8B=E5=8D=886:26, Yongji Xie wrote: > > On Tue, Dec 29, 2020 at 5:11 PM Jason Wang wrote: > >> > >> > >> ----- Original Message ----- > >>> On Mon, Dec 28, 2020 at 4:43 PM Jason Wang wrot= e: > >>>> > >>>> On 2020/12/28 =E4=B8=8B=E5=8D=884:14, Yongji Xie wrote: > >>>>>> I see. So all the above two questions are because VHOST_IOTLB_INVA= LIDATE > >>>>>> is expected to be synchronous. This need to be solved by tweaking = the > >>>>>> current VDUSE API or we can re-visit to go with descriptors relayi= ng > >>>>>> first. > >>>>>> > >>>>> Actually all vdpa related operations are synchronous in current > >>>>> implementation. The ops.set_map/dma_map/dma_unmap should not return > >>>>> until the VDUSE_UPDATE_IOTLB/VDUSE_INVALIDATE_IOTLB message is repl= ied > >>>>> by userspace. Could it solve this problem? > >>>> > >>>> I was thinking whether or not we need to generate IOTLB_INVALIDAT= E > >>>> message to VDUSE during dma_unmap (vduse_dev_unmap_page). > >>>> > >>>> If we don't, we're probably fine. > >>>> > >>> It seems not feasible. This message will be also used in the > >>> virtio-vdpa case to notify userspace to unmap some pages during > >>> consistent dma unmapping. Maybe we can document it to make sure the > >>> users can handle the message correctly. > >> Just to make sure I understand your point. > >> > >> Do you mean you plan to notify the unmap of 1) streaming DMA or 2) > >> coherent DMA? > >> > >> For 1) you probably need a workqueue to do that since dma unmap can > >> be done in irq or bh context. And if usrspace does't do the unmap, it > >> can still access the bounce buffer (if you don't zap pte)? > >> > > I plan to do it in the coherent DMA case. > > > Any reason for treating coherent DMA differently? > Now the memory of the bounce buffer is allocated page by page in the page fault handler. So it can't be used in coherent DMA mapping case which needs some memory with contiguous virtual addresses. I can use vmalloc() to do allocation for the bounce buffer instead. But it might cause some memory waste. Any suggestion? > > > It's true that userspace can > > access the dma buffer if userspace doesn't do the unmap. But the dma > > pages would not be freed and reused unless user space called munmap() > > for them. > > > I wonder whether or not we could recycle IOVA in this case to avoid the > IOTLB_UMAP message. > We can achieve that if we use vmalloc() to do allocation for the bounce buffer which can be used in coherent DMA mapping case. But looks like we still have no way to avoid the IOTLB_UMAP message in vhost-vdpa case. Thanks, Yongji