From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 369DEC61DA4 for ; Thu, 9 Mar 2023 09:48:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C6D34280003; Thu, 9 Mar 2023 04:48:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C1F4C6B0075; Thu, 9 Mar 2023 04:48:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AE5E4280003; Thu, 9 Mar 2023 04:48:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 9F9746B0072 for ; Thu, 9 Mar 2023 04:48:49 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 726CA16074F for ; Thu, 9 Mar 2023 09:48:49 +0000 (UTC) X-FDA: 80548885578.16.B89AECE Received: from madras.collabora.co.uk (madras.collabora.co.uk [46.235.227.172]) by imf09.hostedemail.com (Postfix) with ESMTP id 6B64A140014 for ; Thu, 9 Mar 2023 09:48:46 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=collabora.com header.s=mail header.b=BlA9bNQG; spf=pass (imf09.hostedemail.com: domain of boris.brezillon@collabora.com designates 46.235.227.172 as permitted sender) smtp.mailfrom=boris.brezillon@collabora.com; dmarc=pass (policy=quarantine) header.from=collabora.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678355326; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3mJYp+jPcRr/qJ9vgevsSUpWy/P45kAM7qwgHAE9gzA=; b=fd4ohfD+3CT4Aq7qUT49DA5qLshUnvJ2T/XuqRyNFCasUUjRsZt+BGLAr/bdsNhQ8uiwOa qwsvTQLHHP9/nJcwg6XnsoWlgEGc8CXop60D07GoTGaSB8f57aEGi6LYwcuE4snEmQ8UNb dO/bFvziYwMqgAxj0Uv4d9g772KGluc= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=collabora.com header.s=mail header.b=BlA9bNQG; spf=pass (imf09.hostedemail.com: domain of boris.brezillon@collabora.com designates 46.235.227.172 as permitted sender) smtp.mailfrom=boris.brezillon@collabora.com; dmarc=pass (policy=quarantine) header.from=collabora.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678355326; a=rsa-sha256; cv=none; b=28L8+xGORgGhjlfQmX/0jwgeQCPNbpoaIjaXDTQjzv6e29dAGAjwfhamWVjXHMjk0gPfH6 pse4/HUJF4sNj278hwe1/O9hU2cZSjoX1x339hv3RuB7soAr24UluxrSDNKmhexuNOzXe2 q/BE8WpErHjZwjkkO4fzhxaHi9oHyjw= Received: from localhost (unknown [IPv6:2a01:e0a:2c:6930:5cf4:84a1:2763:fe0d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: bbrezillon) by madras.collabora.co.uk (Postfix) with ESMTPSA id 3F5B56603007; Thu, 9 Mar 2023 09:48:44 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1678355324; bh=A50/72yir9yl+4D3pjJl/L2YPO9X4ebuwpQ6QWEfGK8=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=BlA9bNQGTwENVcuchUmZU0zKRIQtfqHMv+XEHNt1n1px6HtnR7et8gClwpnnVdkIn IsLWvioyN29w7AqACudpFr57ZctauG6ZIgJFd8LftSXgf0CS38ZZzV83iRogQTFBWL sF11OCF68vbVN1Q1lGeMs89xhkNjZOMZN8s386GpenrH5dhZ3C9kzlB0ljekHfrbZ+ aTBlrwC1DbsFNIaqc0c5qjzOEExd2mkcY8NUCtPlSfxkj0CgG9FVWYKTW4BaEycQ9Z SR9E5bo+zEr5A5xTgQqQyb0UFurgErllvE2vblGYI1jIncZB+DGGb/blpuN+cDwku2 /qsm7DdJQybZA== Date: Thu, 9 Mar 2023 10:48:41 +0100 From: Boris Brezillon To: Danilo Krummrich Cc: airlied@gmail.com, daniel@ffwll.ch, tzimmermann@suse.de, mripard@kernel.org, corbet@lwn.net, christian.koenig@amd.com, bskeggs@redhat.com, Liam.Howlett@oracle.com, matthew.brost@intel.com, alexdeucher@gmail.com, ogabbay@kernel.org, bagasdotme@gmail.com, willy@infradead.org, jason@jlekstrand.net, dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH drm-next v2 00/16] [RFC] DRM GPUVA Manager & Nouveau VM_BIND UAPI Message-ID: <20230309104841.7c03d5b4@collabora.com> In-Reply-To: <20230309101243.1150506f@collabora.com> References: <20230217134422.14116-1-dakr@redhat.com> <20230309101243.1150506f@collabora.com> Organization: Collabora X-Mailer: Claws Mail 4.1.1 (GTK 3.24.36; x86_64-redhat-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 6B64A140014 X-Rspam-User: X-Stat-Signature: uus3iwy6fcu1mqdhtbh3qtik6efw1j3n X-HE-Tag: 1678355326-787834 X-HE-Meta: U2FsdGVkX19UuVUA0UhZW88p89RWRqJLqrusX/O7843mdYKEyn2aOodJAylfFfYYE7EX05XiyZHuysRxsZhmnvgSJNknaaE96BtrQUp4B7pvn9/LlT5xTSejOg409UM49f4hkCVsNZNhB7AmhLn8gZk4eiealztlOQJ9jEFQAb4WiRyKiKosTy3oFEJ7RY+RrW3FAxCvrHUZ6H8QqPgHFFFcV0VWXmnlqH7hCR5gzAyoq5meQRR55b7EfKi97d87qSv9KV/IkSX36hMux1wuFFpkLH2crFqZGoRMiT2/KTpeOr5RxCZE8XQ7+xQ2VwdrDM2LyvrfuITsJGIebaeJPBbFj9UTFuSZYIBm4rw/R3kUaW90blZPlkWENpTTxekl2RRNGQhDC9+xCaAonj6SR0rUz99+OnuhvT5mRL/q3owIBQKJOSc6NkkCSv8yKqZDUYFnKcWmyIkTIG4CJFoZBWiQU4EBJ9n+vc3rCZX6l2g3CQSukwKky7BYEk98qEmDA2OQ57sJwkRVDy57IJTCvSEmquHif+Msn5wnavmVjsnagdnlTVmemkGQ+KifRm/cZy3EG9hyiu1rKroo+DpQs7Vfiao2ORzSRKIHQbctK+Xj9b9LfUMzVVmOSiQRzRtD7XzO55wZXjQXv6msaWWiYINPsviXKEVKIiAhb7QSFF9BHH6eHmZVCaskElhmd0ddJ2zzTCGDVrXHp8AgNw3s4iIDr2AuatMyjQipOQE8sAwnV1+F1dW8/lQbSOdn0NtPBhXQrarFUUmaXc9Z+d9w4XnnHipx9KTiCBFA233mMu0e0OyZJouJh+oDp/meCyZsRsHzmtkaKAC8O2oODwSxZz8uNCsijDmA48uq3STuYdFMDjqfpMcMgnJ8PU2GEmuFEjruTQZNTpXr0lXR9iJ8LneacLQh/2OjNm0EH5QWRJYsnz6/FJdUr2WAWjF1PRHf/Bb9BTlz1HthkyJ1M26 cZKXFuTq Ab29GzVE01LDBc9FxxOpse/BQyTML1jSPvl2/jXQiNpBWvg01fP0R3jispccYuZET3OhL4e5zHhZ11jWtwpIRENjzqSqtB7/oOz7UzNPpW5Cuh81j04VUDdaPz3vVN0f7j5OnZrYQkL5Yn8NfecyfpsV2bYgirVgjkYLKMim7iOsuQo8x9VMtamAbhscOMzJhj+6E/JiTE/chk2pTl6LubyPDWN5zr0FuOqJeKRMXoens92XM0YY9XkaQygj3VjzNIiDY1EhWjknb55gW3fXZdXMLE+uX9PYJ26gb5lCLgirP339QGwsG/ddmTjNXR9o7+RcdoX4ZFm7wbtxcNf90oJLqsfDYo55aNtWna5kVy9IRa8RMudMI17a1h8+b9IW81ymM3Er7M1TKUiY56XhMZ/6+ASH+RGWC26ss X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, 9 Mar 2023 10:12:43 +0100 Boris Brezillon wrote: > Hi Danilo, > > On Fri, 17 Feb 2023 14:44:06 +0100 > Danilo Krummrich wrote: > > > Changes in V2: > > ============== > > Nouveau: > > - Reworked the Nouveau VM_BIND UAPI to avoid memory allocations in fence > > signalling critical sections. Updates to the VA space are split up in three > > separate stages, where only the 2. stage executes in a fence signalling > > critical section: > > > > 1. update the VA space, allocate new structures and page tables > > Sorry for the silly question, but I didn't find where the page tables > pre-allocation happens. Mind pointing it to me? It's also unclear when > this step happens. Is this at bind-job submission time, when the job is > not necessarily ready to run, potentially waiting for other deps to be > signaled. Or is it done when all deps are met, as an extra step before > jumping to step 2. If that's the former, then I don't see how the VA > space update can happen, since the bind-job might depend on other > bind-jobs modifying the same portion of the VA space (unbind ops might > lead to intermediate page table levels disappearing while we were > waiting for deps). If it's the latter, I wonder why this is not > considered as an allocation in the fence signaling path (for the > bind-job out-fence to be signaled, you need these allocations to > succeed, unless failing to allocate page-tables is considered like a HW > misbehavior and the fence is signaled with an error in that case). Ok, so I just noticed you only have one bind queue per drm_file (cli->sched_entity), and jobs are executed in-order on a given queue, so I guess that allows you to modify the VA space at submit time without risking any modifications to the VA space coming from other bind-queues targeting the same VM. And, if I'm correct, synchronous bind/unbind ops take the same path, so no risk for those to modify the VA space either (just wonder if it's a good thing to have to sync bind/unbind operations waiting on async ones, but that's a different topic). > > Note that I'm not familiar at all with Nouveau or TTM, and it might > be something that's solved by another component, or I'm just > misunderstanding how the whole thing is supposed to work. This being > said, I'd really like to implement a VM_BIND-like uAPI in pancsf using > the gpuva_manager infra you're proposing here, so please bare with me > :-). > > > 2. (un-)map the requested memory bindings > > 3. free structures and page tables > > > > - Separated generic job scheduler code from specific job implementations. > > - Separated the EXEC and VM_BIND implementation of the UAPI. > > - Reworked the locking parts of the nvkm/vmm RAW interface, such that > > (un-)map operations can be executed in fence signalling critical sections. > > > > Regards, > > Boris >