From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 711B9C71133 for ; Fri, 25 Aug 2023 12:20:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BCB3590001A; Fri, 25 Aug 2023 08:20:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BA2008E0015; Fri, 25 Aug 2023 08:20:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A929590001A; Fri, 25 Aug 2023 08:20:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 992DA8E0015 for ; Fri, 25 Aug 2023 08:20:26 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 72839C04AD for ; Fri, 25 Aug 2023 12:20:26 +0000 (UTC) X-FDA: 81162534852.18.E4803C7 Received: from mail-ej1-f51.google.com (mail-ej1-f51.google.com [209.85.218.51]) by imf06.hostedemail.com (Postfix) with ESMTP id 8B308180031 for ; Fri, 25 Aug 2023 12:20:24 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=fooishbar-org.20221208.gappssmtp.com header.s=20221208 header.b=fWg8e+jR; dmarc=none; spf=none (imf06.hostedemail.com: domain of daniel@fooishbar.org has no SPF policy when checking 209.85.218.51) smtp.mailfrom=daniel@fooishbar.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692966024; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kWYyngDJ3qSSpaOwkQKw/6S3U2XYLWWAXr/ethxyWfY=; b=1oY+5TXR7wxJGDvBd+v92SbeWSvtT+dcwaNRRJqtE7Bu6A/TAJVS8ckoXFrvHwYbZOC+wO vXq74etBqlJZTvWi2L2T9G649dk1DXhO2BBjOtihapuxL6RZoni0b3TS1vagRhXNkp4Tbo nZdvzKbHUBppTGbYLFSDYm8AU6+HQRw= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=fooishbar-org.20221208.gappssmtp.com header.s=20221208 header.b=fWg8e+jR; dmarc=none; spf=none (imf06.hostedemail.com: domain of daniel@fooishbar.org has no SPF policy when checking 209.85.218.51) smtp.mailfrom=daniel@fooishbar.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692966024; a=rsa-sha256; cv=none; b=R1OiaT8+UMHPWMtOrL1YXcnLWp9kU/08lYvcElcARFgUZt0J7YUf9qXHu4yfmOtZAeWced 2cdrRlCeajDBa2hMCUgRg3DcH6yPCHIOZkzlXrO4HdqCXkMFOrT762ddr6jCdVs8TMLrKp 9bWD+MIHR3CxhtpdrzvzUiY9Dx2ydu4= Received: by mail-ej1-f51.google.com with SMTP id a640c23a62f3a-9a2a4a5472dso218548466b.1 for ; Fri, 25 Aug 2023 05:20:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fooishbar-org.20221208.gappssmtp.com; s=20221208; t=1692966023; x=1693570823; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=kWYyngDJ3qSSpaOwkQKw/6S3U2XYLWWAXr/ethxyWfY=; b=fWg8e+jR7J7GfrbzGYj+LejpRt6q8kD5MdO+pdHY0xIB0Y/p2244yhesvkI73MD8SW KuTlzEdwoX1d5TdleWVRkYyvjUdAZ8FxqjAP8Px6CgSSyDWmMSaKzWIgYNMNKd1fTszr ma8rdIxjXD+QIuwdkV5IgjS+/HdBPmY/9l+sGmVTatchUiVSRC4VmQG34lRPMrIj8uEH KFW2vJMnZkPFqML2Qsd3K1+qpz4N+FYXUP8iRpORRpKmsVuSGC1PVfHsJ2IL5HNK1XyK qofYpV/Rc7LlYZghHxTQBiO8b3trAn5cuc15cfdhzSsydx0Gj3nFYvYmdsyOzdvcTHE8 IpkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692966023; x=1693570823; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=kWYyngDJ3qSSpaOwkQKw/6S3U2XYLWWAXr/ethxyWfY=; b=h7l9OQ6xY0tIhUTMVlPQT5U60KMOUTskWOrFEg+a1FbeiBXsATvnqvipEdeyediOG5 0uhdqGZLFDkPn95EXn9mePxNDJIkoYzBZCghfI5KSKerPnEn4dmebBbppmo8gBB+cfHo LhzfJ3a2U77a9W6IsO4kYowuFRFgc1N11pAn4pQLONLQzD3sT+7KuGImAP90GHqfODru P4mqFSDFL9cfu33NbchST4WZ1gnBizyWzgqSlLCSLHABvCX/1IxchdiY35hKUlso2yF0 /nSOW//shx3X4rBtP++vG5cBD79EwN9UJ0MWs61HUTkHOXCDQvd4WCWeOn2YA+HqOYxp 3rFQ== X-Gm-Message-State: AOJu0Yw6PLfQm9uCqd85f69WBS9VtYDpw2ytkzzIOqTRgCS1jfWw+wpG GqT5Kx9AXBRVxby/qT3+UOO6gMArzlKUPetzB5Perw== X-Google-Smtp-Source: AGHT+IErw8qnClNuZp9zL2W3R5DxEJmB8WAy3evsgCLSL46uQHxCzXGTVd96j/AToUUTTaIwJ1VU4czxprtRlgB5e0w= X-Received: by 2002:a17:907:7da3:b0:982:a022:a540 with SMTP id oz35-20020a1709077da300b00982a022a540mr19879140ejc.11.1692966022330; Fri, 25 Aug 2023 05:20:22 -0700 (PDT) MIME-Version: 1.0 References: <029b982f-da62-4fa8-66c4-ab11a515574a@synaptics.com> <20230825104052.4573ab7b@eldfell> <65432c20-a6fd-141c-2ced-a7e6599a1e7c@synaptics.com> In-Reply-To: <65432c20-a6fd-141c-2ced-a7e6599a1e7c@synaptics.com> From: Daniel Stone Date: Fri, 25 Aug 2023 13:20:09 +0100 Message-ID: Subject: Re: [RFC]: shmem fd for non-DMA buffer sharing cross drivers To: Hsia-Jun Li Cc: Pekka Paalanen , daniels@collabora.com, ayaka , hughd@google.com, linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org, Tomasz Figa , linux-mm@kvack.org, Hans Verkuil , akpm@linux-foundation.org, Nicolas Dufresne , Linux Media Mailing List Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 8B308180031 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 36kjctbbq8k1sydc59wcz89ss788e6ft X-HE-Tag: 1692966024-664878 X-HE-Meta: U2FsdGVkX1+HQt93ieyfx+OGMT+cFhqJS2thjp2gce97DO79xe9yt9/HznnPLcqA2TMkMRZymWHZROJ5cRmG5YIBuT88eZ0TIfm+VlgD6xAVp/1U2XjF7xqPEbDFH+YYilZgNRW2OvwWqK92gHryV1LBTYMLwyVqocc7E5Qez6CGrfWNznqJwMFxORDO3kEqTjQ/C7E06F0EYUGZYZM1Qjs/P/AxkdgqcdWvj2OS74JKbUSInZ7H2g56j0qnPzBV6M73tcnbBUmPVtO+0KRGNP1GNQJJuAyFwdS41ZUPni20D54WNqiRoWZRrdhRBP+YJuYULQucmHCb0Nc/gmJo3bmbbfENMhx6tsHxepFU4UxUeY/dpYBO+Fqx/pyvmrYdFc0684qcNHc7iyB7sSCNf5l7ClhmRhNYwlNM0i6fBOgxXTsYiCuD2Dbgg0LLy/x7/DWvA3U7EDJNcDZS1ozU2VsGLwYoqEEIN9UBzHl5kO6DVxwClEcVSM7BRqQsB/ZseQNznKkDdd3nAkDbac1NpST6cKHBdBFOgapHyl+hRx8lMteHCHOxN8oK56Vvx2i/l7npjGVWf8n0Gn1jhLlAJxaLPlaBv5wF4kXOL1QVJ2O22i8/zxgK3Ck97IIfjHRQ116DguguWdU+dGVJI1cI5hhPax+Gc775JxfnPnc6xtFQlEgyFmI76uh4b36t+WUpGOcoZvEFAaucVQ4JpCpZhT7sobNtNKMV1VFa7xi5r1zroZPhnX3pZz+axx/R1c7nPBpWSVrVL5Sht2EV192qEkAJfx4A5xan9X7WLMYNvhQcNW3FoUF2eZks/ky76qgKhBxQQ94UUO6ViaO6DchZpgy7s1YblkQPpScH6jqrSPW8NMJ6uvivNObK7VVj7QGC0GlDQlzpe02bcR+7CaDwnV91/nlj+kwiLlU89sAu4/u0HsGoVPa+QzeGG74Zp0p+RJKvqwKiTXbpB0IeJJx hxaFsbaT S/kRecE04mC2TrE62nbKz3XzPeCACRxgoyc9WrUE+R3aCl8f7Zb/oxjjrYYtHE6WQnCoVy8v7FiuclBcfkKpkb59R2eD6bETvfjrL8PC8sG6Nl55TfMYbv/AJPOYcaxcZVFYFt0S8g3AMWNkEWjElwtSfVUlzt0az5bEveqFuz4HL15UEsJKG4gcPPFY4x+FDci2RyD6tiEHLKY3Sqd9oZ0+9U9/1WRThT4GD1d3VZTtKxUgQ8+nCvcGTRpuDRPpQlSZNwiFxUkRRd3Jj1gEiyzJrvvhpokwDDo5JPROoHbfAdv3Yheu71Svmy8hx5D/FK7W3ekYUC/ba12w= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000007, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi, On Fri, 25 Aug 2023 at 08:56, Hsia-Jun Li wrote: > On 8/25/23 15:40, Pekka Paalanen wrote: > > if userspace cannot access things like an image's HDR metadata, then it > > will be impossible for userspace to program KMS to have the correct > > color pipeline, or to send intended HDR metadata to a video sink. > > > > You cannot leave userspace out of HDR metadata handling, because quite > > probably the V4L2 buffer is not the only thing on screen. That means > > there must composition of multiple sources with different image > > properties and metadata, which means it is no longer obvious what HDR > > metadata should be sent to the video sink. > > > > Even if it is a TV-like application rather than a windowed desktop, you > > will still have other contents to composite: OSD (volume indicators, > > channels indicators, program guide, ...), sub-titles, channel logos, > > notifications... These components ideally should not change their > > appearance arbitrarily with the main program content and metadata > > changes. Either the metadata sent to the video sink is kept static and > > the main program adapted on the fly, or main program metadata is sent > > to the video sink and the additional content is adapted on the fly. > > > > There is only one set of HDR metadata and one composited image that can > > be sent to a video sink, so both must be chosen and produced correctly > > at the source side. This cannot be done automatically inside KMS kernel > > drivers. > > There may be some misunderstanding. > Let suppose this HDR data is in a vendor specific format. > Both upstream(decoder) and downstream(DRM) hardware devices are coming > from the same vendor. > Then we just need to delivery the reference to this metadata buffer from > the upstream to downstream, both of drivers know how to handle it. > > Despite the userspace, we just need to extend a wayland protocol that > making wayland compositor know how to receive the reference to the > metadata and set it to the DRM plane. > > If you want a common HDR formats for all HDR variants(HDR10+, DV), I am > not against it. But it won't make the userspace be able to fill the HDR > metadata even the HDR data comes from the bitstream(likes SEI). We must > consider the case of Secure Video Path(Digital Right), the bitstream is > not accessible from (REE) userspace nor linux kernel, the downstream > must take what the upstream feed. To summarise from IRC, so it's properly documented: the community will not accept this. The mechanism (shmem, dmabuf, copy_from_user, whatever) is _not_ the problem. The problem is the concept. There have been incredibly extensive discussions on this list about colour management and HDR, summarised in documentation in the DRM repository, as well as three talks at the last XDC. This design is the result of discussion between many community participants - including hardware vendors - who have all come up with a design which prioritises transparency and explicit operation. What you are suggesting is exactly the opposite of this. A design in which opaque magic blobs are passed around and the kernel does unknown things based on the contents of those blobs, contradicts this design. (This is different to compression, where even if the format is proprietary, the effect is well-understood - for the current compression mechanisms, it is a lossless transform.) The old Android Display Framework (ADF) was based around the same design with blobs of opaque driver-specific data, where generic code - either in the kernel or in userspace - could not understand the effect of these blobs. This design was rejected, and we made a clear choice to follow the DRM design principles instead. Upstream will not accept any design which hides magic away. 'GKI is hard', 'other vendors won't let us', etc, are not good enough reasons to change our mind on this fundamental principle. Cheers, Daniel