From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.5 required=3.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 96810C4363D for ; Thu, 24 Sep 2020 14:14:34 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EE765212CC for ; Thu, 24 Sep 2020 14:14:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Ff/CN0yz" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EE765212CC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 43C6C6B005C; Thu, 24 Sep 2020 10:14:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3EDC78E0001; Thu, 24 Sep 2020 10:14:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3033B6B0062; Thu, 24 Sep 2020 10:14:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0077.hostedemail.com [216.40.44.77]) by kanga.kvack.org (Postfix) with ESMTP id 1CE936B005C for ; Thu, 24 Sep 2020 10:14:33 -0400 (EDT) Received: from smtpin15.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id CD73A180AD804 for ; Thu, 24 Sep 2020 14:14:32 +0000 (UTC) X-FDA: 77298150384.15.key72_251241d2715f Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin15.hostedemail.com (Postfix) with ESMTP id 809001814B0C8 for ; Thu, 24 Sep 2020 14:14:32 +0000 (UTC) X-HE-Tag: key72_251241d2715f X-Filterd-Recvd-Size: 7962 Received: from mail-wr1-f65.google.com (mail-wr1-f65.google.com [209.85.221.65]) by imf08.hostedemail.com (Postfix) with ESMTP for ; Thu, 24 Sep 2020 14:14:31 +0000 (UTC) Received: by mail-wr1-f65.google.com with SMTP id a17so4009293wrn.6 for ; Thu, 24 Sep 2020 07:14:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:sender:from:date:message-id :subject:to:cc; bh=oKmEvP0fvH/MtuuMfd1/7oF0tm9aotVrqabNoBoq5kU=; b=Ff/CN0yzxu8BrvNMeqciq+FVYYVX2FOGfPQXwTLGfe0lQrivZ0ncLZjnJELMlFDvqr 3L6kB8fT+Wb+X8vLs55KlIIyhJ3dJFr3Rj2DnyYz3Q20kFR5i2oziSiprcCaKqEXGFPy DCbOu9gcA3McuJjQY1+gmJb7WWf5qodLH1cN7EwBuRzkAnnvzXy6Tg66FHUh+prLRl22 WU5UApmatapBNDHAADap1oxiGIPFpwFCnfz3pw3oFzr7izxHT1er/bWd7jQIB2frcuzi +b+RQNwSPaHQNlYq5+CxZ6HhbA9rDNzQNN7uVUnOWZ1j8rmwXx0L+XdP6pWlMFgk+gEj eXVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:sender:from :date:message-id:subject:to:cc; bh=oKmEvP0fvH/MtuuMfd1/7oF0tm9aotVrqabNoBoq5kU=; b=T1VhBYZcuEvJrjl5QQA25TMgKDBDBXS5ejTiUUQ4cibbPrMC5BwJCCFdvlnl8wVfSl yGHP1KvQ64Hz/tyyT7Fvr5iV9kFAbWk4Td17cj7a4ioieL1rAlotaGK1YTfma23GlI3v yRBXGHBvfTK5bOIOMz4fq8+Uqm87Cab4HLgbhCzzTAZbDIVhPm86kX2BmgeyaH0pnRzE Izd21K75rS50OG6dIA+lBw5PoXmthFhWqDNzCaaxxJMP4P+V88IwM6QO0pxRgcb18leH kF7WHubYlSq7PHuvNeNzn7sItyWupdyjU27KB653WlNqdgXuIuVq64DeYU0j1RMKkK5u O13w== X-Gm-Message-State: AOAM5308heLJALX847bM803Dud8QRRR6LysGKaHzZInEnTv8klIb2yC/ 0gPEsyECy+UWYyDgxS4ImiQMEl3p3ew//1iA/ks= X-Google-Smtp-Source: ABdhPJyNTqcd3aVlW3iItxA8DlolxWBS2LKF38N3Kl4q4kAwUYPjAmtzBpIhGbYdl5I4ht9nYkAIBiX3VLg1AotBuhw= X-Received: by 2002:a5d:5261:: with SMTP id l1mr5339778wrc.193.1600956870566; Thu, 24 Sep 2020 07:14:30 -0700 (PDT) MIME-Version: 1.0 References: <59cda41f-170c-a1ad-a345-bc38b9ed4d73@arm.com> <11584d09-5995-6133-3bd3-8f7a0afd0e01@samsung.com> <20200924082830.GB27174@8bytes.org> <37e767b8-8ec4-ae80-ea0d-1caf3cdab8fa@samsung.com> <20200924101640.GE2483160@ulmo> <832be601-c016-70b7-2b59-5f4915c53f85@samsung.com> <46f10f99-5da5-257a-4a02-984ff8ed8c6f@arm.com> In-Reply-To: <46f10f99-5da5-257a-4a02-984ff8ed8c6f@arm.com> X-Google-Sender-Delegation: ameersk@gmail.com From: Shaik Ameer Basha Date: Thu, 24 Sep 2020 19:44:19 +0530 X-Google-Sender-Auth: HhYDNEdiSEFakz822RU-QAajhv0 Message-ID: Subject: Re: IOVA allocation dependency between firmware buffer and remaining buffers To: Robin Murphy Cc: Marek Szyprowski , Thierry Reding , Joerg Roedel , Ajay kumar , Linux IOMMU , linux-mm@kvack.org, Rob Clark , jean-philippe@linaro.org, will@kernel.org, hch@lst.de, baolu.lu@linux.intel.com, Shaik Ameer Basha Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi Robin and Marek, On Thu, Sep 24, 2020 at 4:36 PM Robin Murphy wrote: > > On 2020-09-24 11:47, Marek Szyprowski wrote: > > Hi Robin, > > > > On 24.09.2020 12:40, Robin Murphy wrote: > >> On 2020-09-24 11:16, Thierry Reding wrote: > >>> On Thu, Sep 24, 2020 at 10:46:46AM +0200, Marek Szyprowski wrote: > >>>> On 24.09.2020 10:28, Joerg Roedel wrote: > >>>>> On Wed, Sep 23, 2020 at 08:48:26AM +0200, Marek Szyprowski wrote: > >>>>>> It allows to remap given buffer at the specific IOVA address, > >>>>>> although > >>>>>> it doesn't guarantee that those specific addresses won't be later > >>>>>> used > >>>>>> by the IOVA allocator. Probably it would make sense to add an API for > >>>>>> generic IOMMU-DMA framework to mark the given IOVA range as > >>>>>> reserved/unused to protect them. > >>>>> There is an API for that, the IOMMU driver can return IOVA reserved > >>>>> regions per device and the IOMMU core code will take care of mapping > >>>>> these regions and reserving them in the IOVA allocator, so that > >>>>> DMA-IOMMU code will not use it for allocations. > >>>>> > >>>>> Have a look at the iommu_ops->get_resv_regions() and > >>>>> iommu_ops->put_resv_regions(). > >>>> > >>>> I know about the reserved regions IOMMU API, but the main problem here, > >>>> in case of Exynos, is that those reserved regions won't be created by > >>>> the IOMMU driver but by the IOMMU client device. It is just a result > >>>> how > >>>> the media drivers manages their IOVA space. They simply have to load > >>>> firmware at the IOVA address lower than the any address of the used > >>>> buffers. > >>> > >>> I've been working on adding a way to automatically add direct mappings > >>> using reserved-memory regions parsed from device tree, see: > >>> > >>> https://lore.kernel.org/lkml/20200904130000.691933-1-thierry.reding@gmail.com/ > >>> > >>> Perhaps this can be of use? With that you should be able to add a > >>> reserved-memory region somewhere in the lower range that you need for > >>> firmware images and have that automatically added as a direct mapping > >>> so that it won't be reused later on for dynamic allocations. > >> > >> It can't easily be a *direct* mapping though - if the driver has to > >> use the DMA masks to ensure that everything stays within the > >> addressable range, then (as far as I'm aware) there's no physical > >> memory that low down to equal the DMA addresses. > >> > >> TBH I'm not convinced that this is a sufficiently common concern to > >> justify new APIs, or even to try to make overly generic. I think just > >> implementing a new DMA attribute to say "please allocate/map this > >> particular request at the lowest DMA address possible" would be good > >> enough. Such a thing could also serve PCI drivers that actually care > >> about SAC/DAC to give us more of a chance of removing the "try a > >> 32-bit mask first" trick from everyone's hotpath... > > > > Hmm, I like the idea of such DMA attribute! It should make things really > > simple, especially in the drivers. Thanks for the great idea! I will try > > to implement it then instead of the workarounds I've proposed in > > s5p-mfc/exynos4-is drivers. > > Right, I think it's fair to draw a line and say that anyone who wants a > *specific* address needs to manage their own IOMMU domain. > > In the backend I suspect it's going to be cleanest to implement a > dedicated iova_alloc_low() (or similar) function in the IOVA API that > sidesteps all of the existing allocation paths and goes straight to the > rbtree. This is the place we started with.. But our solution was to provide an API which limits the allocation range per device (dynamically) based on the driver request.. Something like, limit_iova_alloc_range(dev, low_iova, high_iova); /* via helpers */ When multiple devices use the same IOVA space, how we can handle api's like " iova_alloc_low()" ? And providing APIs like " limit_iova_alloc_range()" may cater similar future requirements from drivers instead of worrying about high/low/mid etc. Again, flexibility should be there with user drivers to request the range they want at any point of time... Please let us know your inputs. > > Robin.