From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0F5D6C1744E for ; Tue, 12 Nov 2019 20:23:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id A885B214E0 for ; Tue, 12 Nov 2019 20:23:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="b6CK4yU3" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A885B214E0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 4F48A6B000A; Tue, 12 Nov 2019 15:22:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4796B6B0010; Tue, 12 Nov 2019 15:22:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 11A346B000C; Tue, 12 Nov 2019 15:22:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0008.hostedemail.com [216.40.44.8]) by kanga.kvack.org (Postfix) with ESMTP id E665C6B000A for ; Tue, 12 Nov 2019 15:22:52 -0500 (EST) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with SMTP id 9145D37E7 for ; Tue, 12 Nov 2019 20:22:52 +0000 (UTC) X-FDA: 76148748984.14.dolls40_190a0eee54961 X-HE-Tag: dolls40_190a0eee54961 X-Filterd-Recvd-Size: 11182 Received: from mail-qk1-f194.google.com (mail-qk1-f194.google.com [209.85.222.194]) by imf34.hostedemail.com (Postfix) with ESMTP for ; Tue, 12 Nov 2019 20:22:51 +0000 (UTC) Received: by mail-qk1-f194.google.com with SMTP id i19so15722364qki.2 for ; Tue, 12 Nov 2019 12:22:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pTAE3C6UkPphbB8bkoC7OIstj7M49YCEG83oq3fCqt8=; b=b6CK4yU3UoL1CgOKkjCuHrBDiEHENtFT8feX58DmCzY3bqiu4ZK/Od2V3upqZFZ5q7 zm5H9MRQbRsh0K4H8EnHPsoAOVpkNGbeQTWESbKT7P1dzjMaa36fJQDItWwIT4VRKg3R Gas8YxYUJLlyuUQYpfr0WFPgm8oZMHWzG77Crk8koct7WiOYQzWm92xQWPw7pPLdfWH2 NYRVu5/wK0lZF5PCsWAVQL7hRAsVZrhf1u7s5+qbe9Y9tLc5b1GA1xrg+xi8AQKDdxZe ibyjaFHn3hUIJ+4muJB7bymx2DOpJUqTbrvG73Ef8vCtg1rq/D41eg/PfUtqDhj5VWop W2PA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pTAE3C6UkPphbB8bkoC7OIstj7M49YCEG83oq3fCqt8=; b=H9wk3D9erwHNDwq9dLWA35xEHqNV9Z/AzL4CnNyK+bu1LBYVwQnmzyhAGAzDy2FCmq uIlwtf1CslT4c0/ehLmT8wtDuG7eVhEi+MUUycOsx/xtffV9ZlZGvweyuHBC5ZQ4scXZ iz6wHSb0/3wu74ae9WmW/2vXutdrDOzR8BM0oYDFJFQdsdXLyMRrb833oipukY9v0pE6 KhT6Rs2AQabQiug9zWxFewbypTZ0qgIs6hZaAFQ588UFZzfL22aEXArncVaQ5P2nkKb8 EFbKabkM+E9099cEEpzXCYLWhL+9dNXoWXl4SE8KFBEAofgC8XXayq3ttffr/cli4AC0 GsBg== X-Gm-Message-State: APjAAAVXvLm7Oa9PgZB+a3FfLC4LJsQng48xBrLMH6Txo7kmVNgy+RLE e9xMJNZDyKe5qecP0o4w2IcGLyv8z3E= X-Google-Smtp-Source: APXvYqzaFmdsuGMd+sc3S8UlqP5hfIuFeBKJzgQIPlueQUEvMNlEoPp1r+TNgyLS5EenfDQ/yuhiOA== X-Received: by 2002:a37:9e89:: with SMTP id h131mr17485213qke.477.1573590171112; Tue, 12 Nov 2019 12:22:51 -0800 (PST) Received: from ziepe.ca (hlfxns017vw-142-162-113-180.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.162.113.180]) by smtp.gmail.com with ESMTPSA id h27sm11695982qtk.37.2019.11.12.12.22.47 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 12 Nov 2019 12:22:48 -0800 (PST) Received: from jgg by mlx.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1iUcgZ-0003kG-Fs; Tue, 12 Nov 2019 16:22:47 -0400 From: Jason Gunthorpe To: linux-mm@kvack.org, Jerome Glisse , Ralph Campbell , John Hubbard , Felix.Kuehling@amd.com Cc: linux-rdma@vger.kernel.org, dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Alex Deucher , Ben Skeggs , Boris Ostrovsky , =?UTF-8?q?Christian=20K=C3=B6nig?= , David Zhou , Dennis Dalessandro , Juergen Gross , Mike Marciniszyn , Oleksandr Andrushchenko , Petr Cvek , Stefano Stabellini , nouveau@lists.freedesktop.org, xen-devel@lists.xenproject.org, Christoph Hellwig , Jason Gunthorpe Subject: [PATCH v3 08/14] nouveau: use mmu_notifier directly for invalidate_range_start Date: Tue, 12 Nov 2019 16:22:25 -0400 Message-Id: <20191112202231.3856-9-jgg@ziepe.ca> X-Mailer: git-send-email 2.24.0 In-Reply-To: <20191112202231.3856-1-jgg@ziepe.ca> References: <20191112202231.3856-1-jgg@ziepe.ca> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Jason Gunthorpe There is no reason to get the invalidate_range_start() callback via an indirection through hmm_mirror, just register a normal notifier directly. Tested-by: Ralph Campbell Signed-off-by: Jason Gunthorpe --- drivers/gpu/drm/nouveau/nouveau_svm.c | 95 ++++++++++++++++++--------- 1 file changed, 63 insertions(+), 32 deletions(-) diff --git a/drivers/gpu/drm/nouveau/nouveau_svm.c b/drivers/gpu/drm/nouv= eau/nouveau_svm.c index 668d4bd0c118f1..577f8811925a59 100644 --- a/drivers/gpu/drm/nouveau/nouveau_svm.c +++ b/drivers/gpu/drm/nouveau/nouveau_svm.c @@ -88,6 +88,7 @@ nouveau_ivmm_find(struct nouveau_svm *svm, u64 inst) } =20 struct nouveau_svmm { + struct mmu_notifier notifier; struct nouveau_vmm *vmm; struct { unsigned long start; @@ -96,7 +97,6 @@ struct nouveau_svmm { =20 struct mutex mutex; =20 - struct mm_struct *mm; struct hmm_mirror mirror; }; =20 @@ -251,10 +251,11 @@ nouveau_svmm_invalidate(struct nouveau_svmm *svmm, = u64 start, u64 limit) } =20 static int -nouveau_svmm_sync_cpu_device_pagetables(struct hmm_mirror *mirror, - const struct mmu_notifier_range *update) +nouveau_svmm_invalidate_range_start(struct mmu_notifier *mn, + const struct mmu_notifier_range *update) { - struct nouveau_svmm *svmm =3D container_of(mirror, typeof(*svmm), mirro= r); + struct nouveau_svmm *svmm =3D + container_of(mn, struct nouveau_svmm, notifier); unsigned long start =3D update->start; unsigned long limit =3D update->end; =20 @@ -264,6 +265,9 @@ nouveau_svmm_sync_cpu_device_pagetables(struct hmm_mi= rror *mirror, SVMM_DBG(svmm, "invalidate %016lx-%016lx", start, limit); =20 mutex_lock(&svmm->mutex); + if (unlikely(!svmm->vmm)) + goto out; + if (limit > svmm->unmanaged.start && start < svmm->unmanaged.limit) { if (start < svmm->unmanaged.start) { nouveau_svmm_invalidate(svmm, start, @@ -273,19 +277,31 @@ nouveau_svmm_sync_cpu_device_pagetables(struct hmm_= mirror *mirror, } =20 nouveau_svmm_invalidate(svmm, start, limit); + +out: mutex_unlock(&svmm->mutex); return 0; } =20 -static void -nouveau_svmm_release(struct hmm_mirror *mirror) +static void nouveau_svmm_free_notifier(struct mmu_notifier *mn) +{ + kfree(container_of(mn, struct nouveau_svmm, notifier)); +} + +static const struct mmu_notifier_ops nouveau_mn_ops =3D { + .invalidate_range_start =3D nouveau_svmm_invalidate_range_start, + .free_notifier =3D nouveau_svmm_free_notifier, +}; + +static int +nouveau_svmm_sync_cpu_device_pagetables(struct hmm_mirror *mirror, + const struct mmu_notifier_range *update) { + return 0; } =20 -static const struct hmm_mirror_ops -nouveau_svmm =3D { +static const struct hmm_mirror_ops nouveau_svmm =3D { .sync_cpu_device_pagetables =3D nouveau_svmm_sync_cpu_device_pagetables= , - .release =3D nouveau_svmm_release, }; =20 void @@ -294,7 +310,10 @@ nouveau_svmm_fini(struct nouveau_svmm **psvmm) struct nouveau_svmm *svmm =3D *psvmm; if (svmm) { hmm_mirror_unregister(&svmm->mirror); - kfree(*psvmm); + mutex_lock(&svmm->mutex); + svmm->vmm =3D NULL; + mutex_unlock(&svmm->mutex); + mmu_notifier_put(&svmm->notifier); *psvmm =3D NULL; } } @@ -320,7 +339,7 @@ nouveau_svmm_init(struct drm_device *dev, void *data, mutex_lock(&cli->mutex); if (cli->svm.cli) { ret =3D -EBUSY; - goto done; + goto out_free; } =20 /* Allocate a new GPU VMM that can support SVM (managed by the @@ -335,24 +354,33 @@ nouveau_svmm_init(struct drm_device *dev, void *dat= a, .fault_replay =3D true, }, sizeof(struct gp100_vmm_v0), &cli->svm.vmm); if (ret) - goto done; + goto out_free; =20 - /* Enable HMM mirroring of CPU address-space to VMM. */ - svmm->mm =3D get_task_mm(current); - down_write(&svmm->mm->mmap_sem); + down_write(¤t->mm->mmap_sem); svmm->mirror.ops =3D &nouveau_svmm; - ret =3D hmm_mirror_register(&svmm->mirror, svmm->mm); - if (ret =3D=3D 0) { - cli->svm.svmm =3D svmm; - cli->svm.cli =3D cli; - } - up_write(&svmm->mm->mmap_sem); - mmput(svmm->mm); + ret =3D hmm_mirror_register(&svmm->mirror, current->mm); + if (ret) + goto out_mm_unlock; =20 -done: + svmm->notifier.ops =3D &nouveau_mn_ops; + ret =3D __mmu_notifier_register(&svmm->notifier, current->mm); if (ret) - nouveau_svmm_fini(&svmm); + goto out_hmm_unregister; + /* Note, ownership of svmm transfers to mmu_notifier */ + + cli->svm.svmm =3D svmm; + cli->svm.cli =3D cli; + up_write(¤t->mm->mmap_sem); mutex_unlock(&cli->mutex); + return 0; + +out_hmm_unregister: + hmm_mirror_unregister(&svmm->mirror); +out_mm_unlock: + up_write(¤t->mm->mmap_sem); +out_free: + mutex_unlock(&cli->mutex); + kfree(svmm); return ret; } =20 @@ -494,12 +522,12 @@ nouveau_range_fault(struct nouveau_svmm *svmm, stru= ct hmm_range *range) =20 ret =3D hmm_range_register(range, &svmm->mirror); if (ret) { - up_read(&svmm->mm->mmap_sem); + up_read(&svmm->notifier.mm->mmap_sem); return (int)ret; } =20 if (!hmm_range_wait_until_valid(range, HMM_RANGE_DEFAULT_TIMEOUT)) { - up_read(&svmm->mm->mmap_sem); + up_read(&svmm->notifier.mm->mmap_sem); return -EBUSY; } =20 @@ -507,7 +535,7 @@ nouveau_range_fault(struct nouveau_svmm *svmm, struct= hmm_range *range) if (ret <=3D 0) { if (ret =3D=3D 0) ret =3D -EBUSY; - up_read(&svmm->mm->mmap_sem); + up_read(&svmm->notifier.mm->mmap_sem); hmm_range_unregister(range); return ret; } @@ -587,12 +615,15 @@ nouveau_svm_fault(struct nvif_notify *notify) args.i.p.version =3D 0; =20 for (fi =3D 0; fn =3D fi + 1, fi < buffer->fault_nr; fi =3D fn) { + struct mm_struct *mm; + /* Cancel any faults from non-SVM channels. */ if (!(svmm =3D buffer->fault[fi]->svmm)) { nouveau_svm_fault_cancel_fault(svm, buffer->fault[fi]); continue; } SVMM_DBG(svmm, "addr %016llx", buffer->fault[fi]->addr); + mm =3D svmm->notifier.mm; =20 /* We try and group handling of faults within a small * window into a single update. @@ -609,11 +640,11 @@ nouveau_svm_fault(struct nvif_notify *notify) /* Intersect fault window with the CPU VMA, cancelling * the fault if the address is invalid. */ - down_read(&svmm->mm->mmap_sem); - vma =3D find_vma_intersection(svmm->mm, start, limit); + down_read(&mm->mmap_sem); + vma =3D find_vma_intersection(mm, start, limit); if (!vma) { SVMM_ERR(svmm, "wndw %016llx-%016llx", start, limit); - up_read(&svmm->mm->mmap_sem); + up_read(&mm->mmap_sem); nouveau_svm_fault_cancel_fault(svm, buffer->fault[fi]); continue; } @@ -623,7 +654,7 @@ nouveau_svm_fault(struct nvif_notify *notify) =20 if (buffer->fault[fi]->addr !=3D start) { SVMM_ERR(svmm, "addr %016llx", buffer->fault[fi]->addr); - up_read(&svmm->mm->mmap_sem); + up_read(&mm->mmap_sem); nouveau_svm_fault_cancel_fault(svm, buffer->fault[fi]); continue; } @@ -704,7 +735,7 @@ nouveau_svm_fault(struct nvif_notify *notify) NULL); svmm->vmm->vmm.object.client->super =3D false; mutex_unlock(&svmm->mutex); - up_read(&svmm->mm->mmap_sem); + up_read(&mm->mmap_sem); } =20 /* Cancel any faults in the window whose pages didn't manage --=20 2.24.0