From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D46A4C64ED6 for ; Tue, 28 Feb 2023 02:17:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 41AAD6B007D; Mon, 27 Feb 2023 21:17:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3CB566B007E; Mon, 27 Feb 2023 21:17:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 292766B0080; Mon, 27 Feb 2023 21:17:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 1A0A66B007D for ; Mon, 27 Feb 2023 21:17:26 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id BF96A160D9B for ; Tue, 28 Feb 2023 02:17:25 +0000 (UTC) X-FDA: 80515088850.26.7E4D9EE Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf09.hostedemail.com (Postfix) with ESMTP id B89DD140006 for ; Tue, 28 Feb 2023 02:17:23 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hOdR6h1V; spf=pass (imf09.hostedemail.com: domain of dakr@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dakr@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1677550643; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=L1Mb2daXfEUQHvXVW4hyDc03Jb2/hD1w1K+XnRGqyVE=; b=uiwd9H4HvKtEB6ijxvB/ccUDOIMTaJLzItu+xCaE51a1325PL4dm5lPKKrD/IQoIw2H2G3 mAX1/FrZvJ39nGOyfMmC+CqLLi+KykXp0NYwJms+0tybhs3StBeKkwjl3QI4Mv9amUuuMj RqtQJU5jTNIyceFMHA6OyoLfngIj/YA= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=hOdR6h1V; spf=pass (imf09.hostedemail.com: domain of dakr@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dakr@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1677550643; a=rsa-sha256; cv=none; b=VR9Hh8veb1wEA4ULugmfY949HdD7B65dSh13NhkFfIxvjrjdW8ckSH59eoY7F0c3HZNIpz sL52FZtnEPlM5PMNPh1wWaP+b+ZLiZwk8rP2xYKOgG3QceN4dg3eovqbQ9z24UhilrcQnD Ok/28tielBJEi1cSko8PON58g0Y1zAI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677550643; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=L1Mb2daXfEUQHvXVW4hyDc03Jb2/hD1w1K+XnRGqyVE=; b=hOdR6h1VRk22MNsAfJQdRdediUGgYeJVNpSvw9PPsZCL9XhCJYLUaov4TTUO8EuysfTRHw H3yKHZGzqenxUwSRd4ScqcWft8KpSctGG1wfmsXYbmb95QyiAMcXIjSqyPmOO3eXBT11L9 X0XOt8nOYry/QCO0q7n1727GecNO9RE= Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-249-bheThrW6O068_FWgE2O-kA-1; Mon, 27 Feb 2023 21:17:21 -0500 X-MC-Unique: bheThrW6O068_FWgE2O-kA-1 Received: by mail-ed1-f69.google.com with SMTP id ec13-20020a0564020d4d00b004a621e993a8so11462886edb.13 for ; Mon, 27 Feb 2023 18:17:20 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:subject:to :from:date:message-id:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=aaG8/Bg6dhlDDPoj6nBl4VpMZT/wui8WldswSe4hBs4=; b=53ESYQoQrJlofXZ8Ydyu/qfwKbkzI8NIXcIPse/ynlSATspqOK/Y+15RP+3s2scrqG 2acfCGQLN7UM3/Jb5bkgmzVDZj6zERxhJzu4PnoBUbAg5myvXr9ULRmJRPvWULN979fj PQhvd237dGprBx47IDcJfOb9nSJ7dDAoWkZBS+nVkuOmZqh4jOeD7qmsMR6uuPMvk2U4 I79xYOQXd+cFmftoFHUeVDHJWm6eyIL25lY36Qtg7Ku9GsCJTVaN6shg8hrjF7Lu3fma AL5UlSz+iJvpkzfsDQCkZ2dspDcuvIQ5GDDu8X2UqLLcJB1hr7zpGjw2n8/gkfUZU36s 7Gjw== X-Gm-Message-State: AO0yUKV9TEfUYCl7isSWJSreVdNb3CE13jiUSEQPs7INf1qf6vAehgvD JgS9X7sEMpivmzJgkXhiTyKluxGGDFegzYtSLGnhmhnlP2WuZ2Uxdf0gAmqRiZNJzg3/efHSYXI RtntOzP3WYG0= X-Received: by 2002:a17:906:714e:b0:8b2:8876:6a3c with SMTP id z14-20020a170906714e00b008b288766a3cmr878942ejj.29.1677550640021; Mon, 27 Feb 2023 18:17:20 -0800 (PST) X-Google-Smtp-Source: AK7set9tTXKhn1koDlG1F1zP1IAfope6sN3htqLi1+Nr+TLZ9X3jnhKZaiB/RADncuz0n/Vy80QyHA== X-Received: by 2002:a17:906:714e:b0:8b2:8876:6a3c with SMTP id z14-20020a170906714e00b008b288766a3cmr878917ejj.29.1677550639717; Mon, 27 Feb 2023 18:17:19 -0800 (PST) Received: from WINDOWS. ([2a02:810d:4b3f:de78:41c:160e:6481:ed36]) by smtp.gmail.com with ESMTPSA id s17-20020a170906501100b008b17aa6afc8sm3927141ejj.30.2023.02.27.18.17.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Feb 2023 18:17:18 -0800 (PST) Message-ID: <63fd642e.170a0220.f67f6.e248@mx.google.com> X-Google-Original-Message-ID: Date: Tue, 28 Feb 2023 03:17:15 +0100 From: Danilo Krummrich To: "Liam R. Howlett" , airlied@gmail.com, daniel@ffwll.ch, tzimmermann@suse.de, mripard@kernel.org, corbet@lwn.net, christian.koenig@amd.com, bskeggs@redhat.com, matthew.brost@intel.com, boris.brezillon@collabora.com, alexdeucher@gmail.com, ogabbay@kernel.org, bagasdotme@gmail.com, willy@infradead.org, jason@jlekstrand.net, dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Dave Airlie Subject: Re: [PATCH drm-next v2 05/16] drm: manager to keep track of GPUs VA mappings References: <20230217134422.14116-1-dakr@redhat.com> <20230217134422.14116-6-dakr@redhat.com> <20230221182050.day6z5ge2e3dxerv@revolver> MIME-Version: 1.0 In-Reply-To: <20230221182050.day6z5ge2e3dxerv@revolver> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: multipart/mixed; boundary="xgWBGL0J166I1n9/" Content-Disposition: inline X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: B89DD140006 X-Rspam-User: X-Stat-Signature: cmeytdgd8jy6c4sfjw3bjtqof59qc3nw X-HE-Tag: 1677550643-516031 X-HE-Meta: U2FsdGVkX18esp02FwEVMxW/cwqMAC7MITbzNWnMNz5yPfVzo1sC1cfnIzAP1KnkVgR4Xa+7dQHf9XdDhZ8Dw0Y7TCxq0XjFzO1iuWvcGr7w5cAQ25Vca+Y182VzlUwW98Fei5vczSb4Echt2YOnGVKkPCOvrgmDLEvwa+bAbUced9xd6YWMX9+1JddxAbnD6iSfTVT16SbZJ7DtKD3vaRHlCr7rvm/8Sx8Wk5Fc8+8wwf+KuEE+ydLJpahL5XS8jyxc4bc56Zq+9A/cTPQUTav5nfE5C+v0/wIrTb/5VD7ppicFEmi9scH+tuxJdbiVIHpOnC35TXq5xA3temH0uEJdqDcCS/mcLZpAV0wrpoVaJt3VBdmPcidb2t7p5elTovSRKBwAlnOFgVtS/Gv0CesQKMXeo/2TYmgnvJilMfQ1iX7So5T4MsKcTKtBSBLQVXtR9XkVMIjP6yAut9PW23HCJ/mjh9AAvC6yDp8gD5ERfeMKPrlcXf+gKXg1ioy4dfp95YRptFCyF6PVZko/ULibAL1WNRmuNXmVapX3mgs15r3nSFvsCTTQ/tAfXbUXWYj4/4Brk1AxGNOXxk2dyb0Ly2CeeyyEybuCWv/LH5jLZG7qyNBqP2c2FMVPR94DH4UqetuMuN2ZfInoLi9Drw3U1ztRprTfWk2OYDEIvUGPcEfU3QV5tn9WzpllZoUpHD08SpbuH1sJP2GfTfJcOfIEzBu8yv+3MzdOZrowRh+7hyAP16lG8sAmcfaR685MBA9P78p6x1bwLegVxgqLaqFmWLKX1L3o4nlWfr2e5HD08pwtPxArqlNXwhnuuQdAFoGc41p9HBwEWRAAOBOmapcBzxPLNrJ2xoJ1IDLj8lM+Ped0k2aaXo4SAMhHBt400Tjc060xAOh2fC6fI3s+92E7LKia3Rqsv17ogYkAYttPVnFriBguVIGmmlJlQ5DmDp1DedpRwp4vY/hS/U0 fmcERpWn AYu3oKk898PqfVL286CBapvMvbEQpJ8aUwu3zXTIcWwf0Qk6T0o4Q7k89vnISRbaU2af5S1gg8NrTHnRWWrkAOtco7McvaRJaxuPQxOmvpPdOI1om1n08Jy9ySm92G6bqFVdWAaEMxoGpAmHnZBWVBQaXfnU1CNTT+jWFa66AehVuoxQpDMJ1/hUcTOfOyLbTMkX79MRggj27iSMyhZZU8mq+aRSUdSSfZSl/R0i/NQmwtcAQMlieWIM2JsaShctrMFTVQk7QxlOp8cQHBnlncnbucwvT+GB/1RHxGHyWpafLAxSjan61IssQSELCMBpvLvrjybPG4M025JDJE4zAIL7B6Te3NCDgrOo0/gJFHscOQzJWEJ/1QzL4RDral+ncQCFq/uzU8As3/R2gtplZ2rPAQP9gTuuEk60XjNMSY/Bi0d1yCPEOQ+cdKH/+Qxpyw+6Wn6T8S310Dl1VsNrmoQaPiRO/EXpwpAdga0QlEeuBKwNeHj8bcjbqfA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: --xgWBGL0J166I1n9/ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tue, Feb 21, 2023 at 01:20:50PM -0500, Liam R. Howlett wrote: > * Danilo Krummrich [230217 08:45]: > > Add infrastructure to keep track of GPU virtual address (VA) mappings > > with a decicated VA space manager implementation. > > > > New UAPIs, motivated by Vulkan sparse memory bindings graphics drivers > > start implementing, allow userspace applications to request multiple and > > arbitrary GPU VA mappings of buffer objects. The DRM GPU VA manager is > > intended to serve the following purposes in this context. > > > > 1) Provide infrastructure to track GPU VA allocations and mappings, > > making use of the maple_tree. > > > > 2) Generically connect GPU VA mappings to their backing buffers, in > > particular DRM GEM objects. > > > > 3) Provide a common implementation to perform more complex mapping > > operations on the GPU VA space. In particular splitting and merging > > of GPU VA mappings, e.g. for intersecting mapping requests or partial > > unmap requests. > > > > Suggested-by: Dave Airlie > > Signed-off-by: Danilo Krummrich > > --- > > Documentation/gpu/drm-mm.rst | 31 + > > drivers/gpu/drm/Makefile | 1 + > > drivers/gpu/drm/drm_gem.c | 3 + > > drivers/gpu/drm/drm_gpuva_mgr.c | 1704 +++++++++++++++++++++++++++++++ > > include/drm/drm_drv.h | 6 + > > include/drm/drm_gem.h | 75 ++ > > include/drm/drm_gpuva_mgr.h | 714 +++++++++++++ > > 7 files changed, 2534 insertions(+) > > create mode 100644 drivers/gpu/drm/drm_gpuva_mgr.c > > create mode 100644 include/drm/drm_gpuva_mgr.h > > > > diff --git a/Documentation/gpu/drm-mm.rst b/Documentation/gpu/drm-mm.rst > > index a52e6f4117d6..c9f120cfe730 100644 > > --- a/Documentation/gpu/drm-mm.rst > > +++ b/Documentation/gpu/drm-mm.rst > > @@ -466,6 +466,37 @@ DRM MM Range Allocator Function References > > .. kernel-doc:: drivers/gpu/drm/drm_mm.c > > :export: > > > ... > > > + > > +/** > > + * drm_gpuva_remove_iter - removes the iterators current element > > + * @it: the &drm_gpuva_iterator > > + * > > + * This removes the element the iterator currently points to. > > + */ > > +void > > +drm_gpuva_iter_remove(struct drm_gpuva_iterator *it) > > +{ > > + mas_erase(&it->mas); > > +} > > +EXPORT_SYMBOL(drm_gpuva_iter_remove); > > + > > +/** > > + * drm_gpuva_insert - insert a &drm_gpuva > > + * @mgr: the &drm_gpuva_manager to insert the &drm_gpuva in > > + * @va: the &drm_gpuva to insert > > + * @addr: the start address of the GPU VA > > + * @range: the range of the GPU VA > > + * > > + * Insert a &drm_gpuva with a given address and range into a > > + * &drm_gpuva_manager. > > + * > > + * Returns: 0 on success, negative error code on failure. > > + */ > > +int > > +drm_gpuva_insert(struct drm_gpuva_manager *mgr, > > + struct drm_gpuva *va) > > +{ > > + u64 addr = va->va.addr; > > + u64 range = va->va.range; > > + MA_STATE(mas, &mgr->va_mt, addr, addr + range - 1); > > + struct drm_gpuva_region *reg = NULL; > > + int ret; > > + > > + if (unlikely(!drm_gpuva_in_mm_range(mgr, addr, range))) > > + return -EINVAL; > > + > > + if (unlikely(drm_gpuva_in_kernel_region(mgr, addr, range))) > > + return -EINVAL; > > + > > + if (mgr->flags & DRM_GPUVA_MANAGER_REGIONS) { > > + reg = drm_gpuva_in_region(mgr, addr, range); > > + if (unlikely(!reg)) > > + return -EINVAL; > > + } > > + > > ----- > > > + if (unlikely(drm_gpuva_find_first(mgr, addr, range))) > > + return -EEXIST; > > + > > + ret = mas_store_gfp(&mas, va, GFP_KERNEL); > > mas_walk() will set the internal maple state to the limits to what it > finds. So, instead of an iterator, you can use the walk function and > ensure there is a large enough area in the existing NULL: > > /* > * Nothing at addr, mas now points to the location where the store would > * happen > */ > if (mas_walk(&mas)) > return -EEXIST; > For some reason mas_walk() finds an entry and hence this function returns -EEXIST for the following sequence of insertions. A = [0xc0000 - 0xfffff] B = [0x0 - 0xbffff] Interestingly, inserting B before A works fine. I attached a test module that reproduces the issue. I hope its just a stupid mistake I just can't spot though. > /* The NULL entry ends at mas.last, make sure there is room */ > if (mas.last < (addr + range - 1)) > return -EEXIST; > > /* Limit the store size to the correct end address, and store */ > mas.last = addr + range - 1; > ret = mas_store_gfp(&mas, va, GFP_KERNEL); > --xgWBGL0J166I1n9/ Content-Type: text/x-c; charset=us-ascii Content-Disposition: attachment; filename="maple.c" /* SPDX-License-Identifier: GPL-2.0 */ #if 1 #include #include #include #include #include #include #include #include #include #include #include #include #include #endif struct maple_tree mt; struct va { u64 addr; u64 range; }; static int va_store(struct va *va) { void *entry = NULL; u64 addr = va->addr; u64 range = va->range; u64 last = addr + range - 1; MA_STATE(mas, &mt, addr, addr); int ret; mas_lock(&mas); if ((entry = mas_walk(&mas))) { pr_err("addr=%llx, range=%llx, last=%llx, mas.index=%lx, mas.last=%lx, entry=%px - exists\n", addr, range, last, mas.index, mas.last, entry); ret = -EEXIST; goto err_unlock; } if (mas.last < last) { pr_err("addr=%llx, range=%llx, last=%llx, mas.index=%lx, mas.last%lx, va=%px - not enough space\n", addr, range, last, mas.index, mas.last, va); ret = -EEXIST; goto err_unlock; } mas.last = last; ret = mas_store_gfp(&mas, &va, GFP_KERNEL); if (ret) { pr_err("mas_store_gfp() failed\n"); goto err_unlock; } mas_unlock(&mas); pr_info("addr=%llx, range=%llx, last=%llx, mas.index=%lx, mas.last=%lx, va=%px - insert\n", addr, range, last, mas.index, mas.last, va); return 0; err_unlock: mas_unlock(&mas); return ret; } static int __init maple_init(void) { struct va kernel_node = { .addr = 0xc0000, .range = 0x40000 }; struct va node = { .addr = 0x0, .range = 0xc0000 }; mt_init(&mt); va_store(&kernel_node); va_store(&node); return 0; } static void __exit maple_exit(void) { mtree_lock(&mt); __mt_destroy(&mt); mtree_unlock(&mt); } module_init(maple_init); module_exit(maple_exit); MODULE_LICENSE("GPL v2"); MODULE_AUTHOR("Danilo Krummrich"); MODULE_DESCRIPTION("Maple Tree example."); MODULE_VERSION("0.1"); --xgWBGL0J166I1n9/--