From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 838C7EE4993 for ; Wed, 23 Aug 2023 13:15:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1F1E390002F; Wed, 23 Aug 2023 09:15:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1A24390002C; Wed, 23 Aug 2023 09:15:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 06ACA90002F; Wed, 23 Aug 2023 09:15:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id EA7E390002C for ; Wed, 23 Aug 2023 09:15:35 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id BCAF54005F for ; Wed, 23 Aug 2023 13:15:35 +0000 (UTC) X-FDA: 81155416230.23.39D6DBA Received: from mail-vk1-f182.google.com (mail-vk1-f182.google.com [209.85.221.182]) by imf19.hostedemail.com (Postfix) with ESMTP id B9A041A002B for ; Wed, 23 Aug 2023 13:15:32 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=USkM1wPn; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf19.hostedemail.com: domain of tfiga@chromium.org designates 209.85.221.182 as permitted sender) smtp.mailfrom=tfiga@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692796532; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/G4PBhoXP1LYQ2AlkCcJ3ZNZ43XaQmRKac+McL4CLhE=; b=jI+eCTna+Sj9AbNACR3LhSH3iUjIRZXAX02z4CE/YYHlCjw/8iG11CyNezqKwbh6nYwtFc esHWj0bSpMhy+dw4eegIQ3m5CUH7Uk/hl8rn93LlVZ3hph4XsjNPtdCrEBsL94UMC5GTT9 iDHPrEfJF1LP3JWx0OC5NsruY/Yau+E= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=USkM1wPn; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf19.hostedemail.com: domain of tfiga@chromium.org designates 209.85.221.182 as permitted sender) smtp.mailfrom=tfiga@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692796532; a=rsa-sha256; cv=none; b=FTouSM+xw9wch4vdNtJrUwQ2aN7AmshST1WN98njVxivx/y0nC49vdL8yqRfPZenw59xUw 1ZBHZ6TS0gIpEvsmV1RhsCTwqj9/IiD0MH6SSqHL/FzUlzL0EZcc74sX8S9CsyfI8Ptm+N 7JmcRXihjIFhNP/KSwJkr8PqgNRVdwI= Received: by mail-vk1-f182.google.com with SMTP id 71dfb90a1353d-48d14d11756so1224349e0c.2 for ; Wed, 23 Aug 2023 06:15:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1692796530; x=1693401330; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=/G4PBhoXP1LYQ2AlkCcJ3ZNZ43XaQmRKac+McL4CLhE=; b=USkM1wPnbKAO9A1ucEyk5mzrVyjGOHAlWrDkX0ej6NRS7PXcnImyFy/ipTLZcBeEof AzJNhc0fciwjeA/z6TiZe6ecKZQy4PGEEXAe2JWSSQ+0pZmZkizYS+GKwbfDLg6/ggCd NuwXkw+aDJxq4kFi8yqGWIwKjLSL3Y52/MREo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692796530; x=1693401330; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/G4PBhoXP1LYQ2AlkCcJ3ZNZ43XaQmRKac+McL4CLhE=; b=JqtRnOGQ/TQCQctZbGEkS85eFASzzw2Nn3sVKRHcPdMZqC4U/pQmP2E1bR9gApROXK 2WqPk7fukePOvKTZJwS8gMMrFzQoDOIh5EZmIyRrEJcGzJ2dWovyCo4I34ZiqRUNuw5d eDRbcx3ZCyPEylSGLjCTFuuQvPJOerAHPTLXYIjfE3bEGCXyv8UP9QK9cNXnlgYcq9kX kvW8lF35jL+bMWetkE8VZH1ENdJeys6lPcOlPZsHistDoZABNvklKDO27RTxyGhhsYbK hryqUx+xJBE8gz2a2Oh9LCE1XuqUyJehoXBLr4Lg6x0ObHF67zjtJokiGJRqmCatRak7 vRBg== X-Gm-Message-State: AOJu0YyapC3DhloWID4VZQojM6c6GMalUITP67pc72BienNL0hkM7z+2 MfpQnz6bBkYz26sAYa58NbkL+tx/LAexAfBdXRew9g== X-Google-Smtp-Source: AGHT+IHOCd/6cAw9s0OrH8Ans0NR9/Ap5Q3DKyFaPl5KzQNl6KTsqGb6zO4Bt7KQ97v6UiWb+nu+ag== X-Received: by 2002:a1f:c985:0:b0:487:1926:5aab with SMTP id z127-20020a1fc985000000b0048719265aabmr10203293vkf.15.1692796530330; Wed, 23 Aug 2023 06:15:30 -0700 (PDT) Received: from mail-vs1-f41.google.com (mail-vs1-f41.google.com. [209.85.217.41]) by smtp.gmail.com with ESMTPSA id e22-20020ac5c7d6000000b004893a9acd05sm60833vkn.47.2023.08.23.06.15.29 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 23 Aug 2023 06:15:29 -0700 (PDT) Received: by mail-vs1-f41.google.com with SMTP id ada2fe7eead31-44d4c3fa6a6so1403484137.0 for ; Wed, 23 Aug 2023 06:15:29 -0700 (PDT) X-Received: by 2002:a67:f988:0:b0:44d:63a3:4be4 with SMTP id b8-20020a67f988000000b0044d63a34be4mr4337218vsq.29.1692796528899; Wed, 23 Aug 2023 06:15:28 -0700 (PDT) MIME-Version: 1.0 References: <029b982f-da62-4fa8-66c4-ab11a515574a@synaptics.com> In-Reply-To: From: Tomasz Figa Date: Wed, 23 Aug 2023 22:15:11 +0900 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [RFC]: shmem fd for non-DMA buffer sharing cross drivers To: Hsia-Jun Li Cc: linux-mm@kvack.org, dri-devel@lists.freedesktop.org, Linux Media Mailing List , hughd@google.com, akpm@linux-foundation.org, Simon Ser , Hans Verkuil , daniels@collabora.com, ayaka , linux-kernel@vger.kernel.org, Nicolas Dufresne Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: B9A041A002B X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: mbr1gw9zdssdm9qr8nqjh7jtab3gg97q X-HE-Tag: 1692796532-3263 X-HE-Meta: U2FsdGVkX18Z1sQUCzJPcuDrE8tb7lICfj5ZRCxu3pF55UATxl1cFm+z7zWjFSN+UqLLUFkxx/f5D7WBMbiQtZHQUPL11vBJxdhkD547eianjCqy+mcdSmlPHe2Z9HRO19q56jzgTfGGXGdx9s0jfm5JiF8ninhI8ZdZQfVMRW48akuAvCYmIt3SLBhwGpGJaGu4qv9yhUPvdAa7g4aW2v4qREi11T/AjgBgsKh6NVVA1obkEwKcM6s9FgWa0LhAP+NWsitOiqx7MU1tORgA6SnjoXnL2+Y7Sb+S188OAnwZAwxvylUeAx4o1mcC5qOgWShkpPPo0/+NquIV+bsLIL/hvEG351ly/0lDTr08Gd3Fb5Re6aj/F28w9p04tJZAR80K7C0pV0tgYSwlXfW7AFzdUmyig5MOWAI9hrs8qTXZhsWlMqFTPzM/GGvGxyhuX2V6na+GQLsU9dOfTeAbKkOlDXBfft7sq2GfFESSxVn1lbFuPYZ7WPNEOjFGaSqkk++NUFER1AdrpwbE1AkgB8/JjFqOy3dzXgikaibWsPKOjbONhE4ssJEXcbYkPqNxTfkRz3OZzvaCJkJEaDIkJsc7AiWriWmJZOsUZfn46E7a3GW2zv9hZ93wmmPq/uT5Ezahf9H2dLwpOrhQ8kO2i1J9AKCkSOoq9dyVYmPq7tHCuJ4f3MFabbz/eOCNRipyVmTTKNjoTvJK23IY655QIf/+0M+pho76X00nBkAJJUqZGRf6Qqq7M1J+5l9axrULICOidDq07t3lU12hJXc+DLhJZp1S6SCDuLLWbTI/LMRe9WyKLXC0KGXfVUVD4tzin/E4NBrL0Nxu/gwNpdPYDa+qlet67UF49rn/QokSQZuJ9UN6VraILkp0pi4hrPIXnnQs3M2IcNoSGTF/yd/coFPizt4KYQn+u/6JSMjiH3H6tT6XYw6L/jJdK7gnNEeRmRFhPEJbGqm2VBuyUMg b4ge8Wqm cDV0TR7VL3WtlYUG+wmLum/EFMkAittAe574JyocEeoieMokkXYG1tN4nKwWxcPy7CXCrYkcVEdgkm05PRkFE2S/d4p8Q9BsNmiecKUqPhpC5TjvWWHj48cgGPqi939p0U4OBqUomw5AmLPP9EegwbtoCmbw+8vT8rm6wETU5mttJTiJDkJecsIs8tcsGuB9ahVmu1IPVDqL625hyGgcj3o5w4gLIBuJv+qyf6j+O071P8v5YwpTjrDksMlBJVNjyiqgtOk3rV9jU/vvQYBL/6lZXH6h5awunyL4Vk1kzt1qdJ+joGNs3F5JI/jbabIiRq17ss2d96+0GI4xDVPxLzTQaPgt+lK1/GAGYrSpaCPnp84hHTlhw1YQhswS0s921G4vkernsQTXJUXKlP+SOHh0y5fq9M5QyvllY51DcxHrzzrQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Aug 23, 2023 at 4:11=E2=80=AFPM Hsia-Jun Li wrote: > > > > On 8/23/23 12:46, Tomasz Figa wrote: > > CAUTION: Email originated externally, do not click links or open attach= ments unless you recognize the sender and know the content is safe. > > > > > > Hi Hsia-Jun, > > > > On Tue, Aug 22, 2023 at 8:14=E2=80=AFPM Hsia-Jun Li wrote: > >> > >> Hello > >> > >> I would like to introduce a usage of SHMEM slimier to DMA-buf, the maj= or > >> purpose of that is sharing metadata or just a pure container for cross > >> drivers. > >> > >> We need to exchange some sort of metadata between drivers, likes dynam= ic > >> HDR data between video4linux2 and DRM. > > > > If the metadata isn't too big, would it be enough to just have the > > kernel copy_from_user() to a kernel buffer in the ioctl code? > > > >> Or the graphics frame buffer is > >> too complex to be described with plain plane's DMA-buf fd. > >> An issue between DRM and V4L2 is that DRM could only support 4 planes > >> while it is 8 for V4L2. It would be pretty hard for DRM to expend its > >> interface to support that 4 more planes which would lead to revision o= f > >> many standard likes Vulkan, EGL. > > > > Could you explain how a shmem buffer could be used to support frame > > buffers with more than 4 planes? > > If you are asking why we need this: I'm asking how your proposal to use shmem FD solves the problem for those c= ases. > 1. metadata likes dynamic HDR tone data > 2. DRM also challenges with this problem, let me quote what sima said: > "another trick that we iirc used for afbc is that sometimes the planes > have a fixed layout > like nv12 > and so logically it's multiple planes, but you only need one plane slot > to describe the buffer > since I think afbc had the "we need more than 4 planes" issue too" > > Unfortunately, there are vendor pixel formats are not fixed layout. > > 3. Secure(REE, trusted video piepline) info. > > For how to assign such metadata data. > In case with a drm fb_id, it is simple, we just add a drm plane property > for it. The V4L2 interface is not flexible, we could only leave into > CAPTURE request_fd as a control. > >> > >> Also, there is no reason to consume a device's memory for the content > >> that device can't read it, or wasting an entry of IOMMU for such data. > > > > That's right, but DMA-buf doesn't really imply any of those. DMA-buf > > is just a kernel object with some backing memory. It's up to the > > allocator to decide how the backing memory is allocated and up to the > > importer on whether it would be mapped into an IOMMU. > > > I just want to say it can't be allocated at the same place which was for > those DMA bufs(graphics or compressed bitstream). > This also could be answer for your first question, if we place this kind > of buffer in a plane for DMABUF(importing) in V4L2, V4L2 core would try > to prepare it, which could map it into IOMMU. > V4L2 core will prepare it according to the struct device that is given to it. For the planes that don't have to go to the hardware a struct device could be given that doesn't require any DMA mapping. Also you can check how the uvcvideo driver handles it. It doesn't use the vb2 buffers directly, but always writes to them using CPU (due to how the UVC protocol is designed). > >> Usually, such a metadata would be the value should be written to a > >> hardware's registers, a 4KiB page would be 1024 items of 32 bits regis= ters. > >> > >> Still, I have some problems with SHMEM: > >> 1. I don't want the userspace modify the context of the SHMEM allocate= d > >> by the kernel, is there a way to do so? > > > > This is generally impossible without doing any of the two: > > 1) copying the contents to an internal buffer not accessible to the > > userspace, OR > > 2) modifying any of the buffer mappings to read-only > > > > 2) can actually be more costly than 1) (depending on the architecture, > > data size, etc.), so we shouldn't just discard the option of a simple > > copy_from_user() in the ioctl. > > > I don't want the userspace access it at all. So that won't be a problem. In this case, wouldn't it be enough to have a DMA-buf exporter that doesn't provide the mmap op? > >> 2. Should I create a helper function for installing the SHMEM file as = a fd? > > > > We already have the udmabuf device [1] to turn a memfd into a DMA-buf, > > so maybe that would be enough? > > > > [1] https://elixir.bootlin.com/linux/v6.5-rc7/source/drivers/dma-buf/ud= mabuf.c > > > It is the kernel driver that allocate this buffer. For example, v4l2 > CAPTURE allocate a buffer for metadata when VIDIOC_REQBUFS. > Or GBM give you a fd which is assigned with a surface. > > So we need a kernel interface. Sorry, I'm confused. If we're talking about buffers allocated by the specific allocators like V4L2 or GBM, why do we need SHMEM at all? Best, Tomasz > > Best, > > Tomasz > > > >> > >> -- > >> Hsia-Jun(Randy) Li > > -- > Hsia-Jun(Randy) Li