From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.5 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0734FC4363D for ; Thu, 24 Sep 2020 11:06:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7ADE6239A1 for ; Thu, 24 Sep 2020 11:06:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7ADE6239A1 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E6A296B00B0; Thu, 24 Sep 2020 07:06:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E1A2B900017; Thu, 24 Sep 2020 07:06:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D302D6B00B2; Thu, 24 Sep 2020 07:06:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0177.hostedemail.com [216.40.44.177]) by kanga.kvack.org (Postfix) with ESMTP id BFF696B00B0 for ; Thu, 24 Sep 2020 07:06:30 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 836B6180AD801 for ; Thu, 24 Sep 2020 11:06:30 +0000 (UTC) X-FDA: 77297676540.21.tramp35_370b0592715e Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin21.hostedemail.com (Postfix) with ESMTP id 60D6B180442C4 for ; Thu, 24 Sep 2020 11:06:30 +0000 (UTC) X-HE-Tag: tramp35_370b0592715e X-Filterd-Recvd-Size: 5535 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf43.hostedemail.com (Postfix) with ESMTP for ; Thu, 24 Sep 2020 11:06:29 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B689B152B; Thu, 24 Sep 2020 04:06:28 -0700 (PDT) Received: from [10.57.48.76] (unknown [10.57.48.76]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 539263F8C6; Thu, 24 Sep 2020 04:06:26 -0700 (PDT) Subject: Re: IOVA allocation dependency between firmware buffer and remaining buffers To: Marek Szyprowski , Thierry Reding Cc: Joerg Roedel , Shaik Ameer Basha , Ajay kumar , Linux IOMMU , linux-mm@kvack.org, Rob Clark , jean-philippe@linaro.org, will@kernel.org, hch@lst.de, baolu.lu@linux.intel.com, Shaik Ameer Basha References: <59cda41f-170c-a1ad-a345-bc38b9ed4d73@arm.com> <11584d09-5995-6133-3bd3-8f7a0afd0e01@samsung.com> <20200924082830.GB27174@8bytes.org> <37e767b8-8ec4-ae80-ea0d-1caf3cdab8fa@samsung.com> <20200924101640.GE2483160@ulmo> <832be601-c016-70b7-2b59-5f4915c53f85@samsung.com> From: Robin Murphy Message-ID: <46f10f99-5da5-257a-4a02-984ff8ed8c6f@arm.com> Date: Thu, 24 Sep 2020 12:06:24 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:78.0) Gecko/20100101 Thunderbird/78.2.2 MIME-Version: 1.0 In-Reply-To: <832be601-c016-70b7-2b59-5f4915c53f85@samsung.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-GB Content-Transfer-Encoding: 7bit X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 2020-09-24 11:47, Marek Szyprowski wrote: > Hi Robin, > > On 24.09.2020 12:40, Robin Murphy wrote: >> On 2020-09-24 11:16, Thierry Reding wrote: >>> On Thu, Sep 24, 2020 at 10:46:46AM +0200, Marek Szyprowski wrote: >>>> On 24.09.2020 10:28, Joerg Roedel wrote: >>>>> On Wed, Sep 23, 2020 at 08:48:26AM +0200, Marek Szyprowski wrote: >>>>>> It allows to remap given buffer at the specific IOVA address, >>>>>> although >>>>>> it doesn't guarantee that those specific addresses won't be later >>>>>> used >>>>>> by the IOVA allocator. Probably it would make sense to add an API for >>>>>> generic IOMMU-DMA framework to mark the given IOVA range as >>>>>> reserved/unused to protect them. >>>>> There is an API for that, the IOMMU driver can return IOVA reserved >>>>> regions per device and the IOMMU core code will take care of mapping >>>>> these regions and reserving them in the IOVA allocator, so that >>>>> DMA-IOMMU code will not use it for allocations. >>>>> >>>>> Have a look at the iommu_ops->get_resv_regions() and >>>>> iommu_ops->put_resv_regions(). >>>> >>>> I know about the reserved regions IOMMU API, but the main problem here, >>>> in case of Exynos, is that those reserved regions won't be created by >>>> the IOMMU driver but by the IOMMU client device. It is just a result >>>> how >>>> the media drivers manages their IOVA space. They simply have to load >>>> firmware at the IOVA address lower than the any address of the used >>>> buffers. >>> >>> I've been working on adding a way to automatically add direct mappings >>> using reserved-memory regions parsed from device tree, see: >>> >>> https://lore.kernel.org/lkml/20200904130000.691933-1-thierry.reding@gmail.com/ >>> >>> Perhaps this can be of use? With that you should be able to add a >>> reserved-memory region somewhere in the lower range that you need for >>> firmware images and have that automatically added as a direct mapping >>> so that it won't be reused later on for dynamic allocations. >> >> It can't easily be a *direct* mapping though - if the driver has to >> use the DMA masks to ensure that everything stays within the >> addressable range, then (as far as I'm aware) there's no physical >> memory that low down to equal the DMA addresses. >> >> TBH I'm not convinced that this is a sufficiently common concern to >> justify new APIs, or even to try to make overly generic. I think just >> implementing a new DMA attribute to say "please allocate/map this >> particular request at the lowest DMA address possible" would be good >> enough. Such a thing could also serve PCI drivers that actually care >> about SAC/DAC to give us more of a chance of removing the "try a >> 32-bit mask first" trick from everyone's hotpath... > > Hmm, I like the idea of such DMA attribute! It should make things really > simple, especially in the drivers. Thanks for the great idea! I will try > to implement it then instead of the workarounds I've proposed in > s5p-mfc/exynos4-is drivers. Right, I think it's fair to draw a line and say that anyone who wants a *specific* address needs to manage their own IOMMU domain. In the backend I suspect it's going to be cleanest to implement a dedicated iova_alloc_low() (or similar) function in the IOVA API that sidesteps all of the existing allocation paths and goes straight to the rbtree. Robin.