From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97479C433DB for ; Tue, 9 Feb 2021 13:35:26 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C45E964EDC for ; Tue, 9 Feb 2021 13:35:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C45E964EDC Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ziepe.ca Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 263F66B0005; Tue, 9 Feb 2021 08:35:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 214C76B006C; Tue, 9 Feb 2021 08:35:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 104A56B006E; Tue, 9 Feb 2021 08:35:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0138.hostedemail.com [216.40.44.138]) by kanga.kvack.org (Postfix) with ESMTP id EA9826B0005 for ; Tue, 9 Feb 2021 08:35:24 -0500 (EST) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id A6DA3180357F3 for ; Tue, 9 Feb 2021 13:35:24 +0000 (UTC) X-FDA: 77798826168.13.dirt31_000331827607 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin13.hostedemail.com (Postfix) with ESMTP id 741CC18140B89 for ; Tue, 9 Feb 2021 13:35:24 +0000 (UTC) X-HE-Tag: dirt31_000331827607 X-Filterd-Recvd-Size: 5924 Received: from mail-qk1-f169.google.com (mail-qk1-f169.google.com [209.85.222.169]) by imf17.hostedemail.com (Postfix) with ESMTP for ; Tue, 9 Feb 2021 13:35:23 +0000 (UTC) Received: by mail-qk1-f169.google.com with SMTP id 19so7704038qkc.13 for ; Tue, 09 Feb 2021 05:35:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=25WUiLOJuw+GE+OUcQPIenFzwQPp8UMH3H3KE2iMm0w=; b=mzalXTaSQvqsMyTlcg2ocFEUjvAp99/a77nvoAG+uBoekFj7jweXCNHWlDPY2podsm yk9Mj1F2cz110lWiEY7yQblSp7LnblfxSgv5BUgT9Flc4ccAx72uC8Y2THXcbcbD6kGD MyoZYCBgQenxcNlyO8fljyQ6Jq2M8pigmK0+JOjqZBy3biXjyoLm+p+yERZ9wYNk+Bnn o52J19oTF3wLoOgSdYnYV1qzXs2BuOw4ltcx4+q08fA7o9lTx2BseR4h0neV0CMUVA5F HAUa4RtablrEiHfxqQTuyRNKLDapPyjHXbP+MKYcA1EGo2CxvAnP3VaRLHFNrbDuXCkv efLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=25WUiLOJuw+GE+OUcQPIenFzwQPp8UMH3H3KE2iMm0w=; b=FL7bBtpv1xAzoGFDclTeuJegX7XzMU/VHUddCk8ZHgUEGsbtYqSeSNjlyhDY+/VlEk v6sJ+WDzNXqayZkXmjBHCWHNUabLtKUor+OPeJR9bF9tMh33JMECEYiMIbeqDqtkgIuw KVLi+2jKcyPRRDJSHrkEFa0bfY5ntQwcivwCxZ9FCUM5MV7RF9NHRxBhDxkcrt4qrRoi Upf4OWErYtWDyS7ui54A3F3y/TxN+Gl3wos7iA6yOZsdwbX1aqgSVdsNkN1ub02+ikQp lPmIP8h9eIrfxXyaL9X1VHmOtb1aA/Zp1JNWzlPPdOyY3WKZ5DNvTEsMRjyjgGxz8V3T XL/g== X-Gm-Message-State: AOAM5335jQlkHlS3aHJLfl3FBVZZZvBk31fHc0t0bGBoe4909i+KIOLT N20QS0QbVbaKqc+A1CnfMaJZfQ== X-Google-Smtp-Source: ABdhPJzbjksA775ifW6lQbAm7tuYtcAJCYrBFcFhoKRtCHGrYI9QO0U12fgq+gl/FMuI2/wmaQjtRQ== X-Received: by 2002:a05:620a:6d7:: with SMTP id 23mr8251839qky.460.1612877721350; Tue, 09 Feb 2021 05:35:21 -0800 (PST) Received: from ziepe.ca (hlfxns017vw-142-162-115-133.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.162.115.133]) by smtp.gmail.com with ESMTPSA id e66sm6901615qkd.82.2021.02.09.05.35.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Feb 2021 05:35:20 -0800 (PST) Received: from jgg by mlx with local (Exim 4.94) (envelope-from ) id 1l9TAm-005RH4-8s; Tue, 09 Feb 2021 09:35:20 -0400 Date: Tue, 9 Feb 2021 09:35:20 -0400 From: Jason Gunthorpe To: Alistair Popple Cc: Daniel Vetter , Linux MM , Nouveau Dev , Ben Skeggs , Andrew Morton , Linux Doc Mailing List , Linux Kernel Mailing List , kvm-ppc@vger.kernel.org, dri-devel , John Hubbard , Ralph Campbell , Jerome Glisse Subject: Re: [PATCH 0/9] Add support for SVM atomics in Nouveau Message-ID: <20210209133520.GB4718@ziepe.ca> References: <20210209010722.13839-1-apopple@nvidia.com> <3426910.QXTomnrpqD@nvdebian> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3426910.QXTomnrpqD@nvdebian> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Feb 09, 2021 at 11:57:28PM +1100, Alistair Popple wrote: > On Tuesday, 9 February 2021 9:27:05 PM AEDT Daniel Vetter wrote: > > > > > > Recent changes to pin_user_pages() prevent the creation of pinned pages in > > > ZONE_MOVABLE. This series allows pinned pages to be created in > ZONE_MOVABLE > > > as attempts to migrate may fail which would be fatal to userspace. > > > > > > In this case migration of the pinned page is unnecessary as the page can > be > > > unpinned at anytime by having the driver revoke atomic permission as it > > > does for the migrate_to_ram() callback. However a method of calling this > > > when memory needs to be moved has yet to be resolved so any discussion is > > > welcome. > > > > Why do we need to pin for gpu atomics? You still have the callback for > > cpu faults, so you > > can move the page as needed, and hence a long-term pin sounds like the > > wrong approach. > > Technically a real long term unmoveable pin isn't required, because as you say > the page can be moved as needed at any time. However I needed some way of > stopping the CPU page from being freed once the userspace mappings for it had > been removed. The issue is you took the page out of the PTE it belongs to, which makes it orphaned and unlocatable by the rest of the mm? Ideally this would leave the PTE in place so everything continues to work, just disable CPU access to it. Maybe some kind of special swap entry? I also don't much like the use of ZONE_DEVICE here, that should only be used for actual device memory, not as a temporary proxy for CPU pages.. Having two struct pages refer to the same physical memory is pretty ugly. > The normal solution of registering an MMU notifier to unpin the page when it > needs to be moved also doesn't work as the CPU page tables now point to the > device-private page and hence the migration code won't call any invalidate > notifiers for the CPU page. The fact the page is lost from the MM seems to be the main issue here. > Yes, I would like to avoid the long term pin constraints as well if possible I > just haven't found a solution yet. Are you suggesting it might be possible to > add a callback in the page migration logic to specially deal with moving these > pages? How would migration even find the page? Jason