From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 68E27C02193 for ; Tue, 28 Jan 2025 13:20:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6BFFA28022D; Tue, 28 Jan 2025 08:20:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 670D828022B; Tue, 28 Jan 2025 08:20:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 510F728022D; Tue, 28 Jan 2025 08:20:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 3174928022B for ; Tue, 28 Jan 2025 08:20:39 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id A7EF481343 for ; Tue, 28 Jan 2025 13:20:38 +0000 (UTC) X-FDA: 83056920156.08.C54FCD2 Received: from mail-qk1-f178.google.com (mail-qk1-f178.google.com [209.85.222.178]) by imf10.hostedemail.com (Postfix) with ESMTP id 8DAD9C001D for ; Tue, 28 Jan 2025 13:20:36 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=ziepe.ca header.s=google header.b=hXnBZLax; spf=pass (imf10.hostedemail.com: domain of jgg@ziepe.ca designates 209.85.222.178 as permitted sender) smtp.mailfrom=jgg@ziepe.ca; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738070436; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GOWXoYYRnWMsoeKK+VVuf/lMuv2V7GO+mfTTLFXfUSE=; b=gd+Pxhz0/XM2iL/83JX4BrVHALuqcrDwwqpMRIhGrJlOw1ap8i3YvS3NJRndZ89/EuyhWi X0jqk1KjAHrKQ/TxOFAD+/UnrSiBUtpBky0TTw60sVATyTbIi6uHC9v0PineUKGpIny6HE xOVgxSQ6P9kz7Fx+o6X+S6tb+wlqn8s= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738070436; a=rsa-sha256; cv=none; b=rngcakV5Ok4eeLHjxvlCSpMA7uc1a7ubOQvYDBuvShewNGYWc4BqRKHsdBLkNpzhjuU0pp Y5UOrIkcGfXsA5VWGGsIMgZIvdxhQ9XkPiqybnwBBDtW4/V8XEvSsN3fuxr7xExkBGPp+C VZ+qYPwVGQ/26/lZH1PZQxjN2ugCBtU= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=ziepe.ca header.s=google header.b=hXnBZLax; spf=pass (imf10.hostedemail.com: domain of jgg@ziepe.ca designates 209.85.222.178 as permitted sender) smtp.mailfrom=jgg@ziepe.ca; dmarc=none Received: by mail-qk1-f178.google.com with SMTP id af79cd13be357-7b6e9317a2aso575991485a.0 for ; Tue, 28 Jan 2025 05:20:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; t=1738070435; x=1738675235; darn=kvack.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=GOWXoYYRnWMsoeKK+VVuf/lMuv2V7GO+mfTTLFXfUSE=; b=hXnBZLaxijWnjWLn5bLDlZ57day1NdwPk3J0ZdR0mWXyT79S8o62QAL4zVvfLwRzL3 lKwy7cgu6OX+shmt/R1xv4hPd7IAKL5raD/roRr6Xz7wTO/A+a1JqrPMGy1PPyDd0VZj /YgPQbaT733TTj9hI4AAGhvzH/LMTzJMuulb4wtHfzO7HQNPKQt/66E6rcHNoI707LrR WOaS9c6vvH8zxRiQq1v8/2qfWiSnz4/Y5FNLoNXJHsCkOG6+IJKa8ws8GiLCkIVO5yZv S+r1s+IQ7Rz197xc6dpzdsGOpswP5tZzbRgrEMc6EtdQatu5RKDoZk5ozEEqpRrE9S1d rBiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738070435; x=1738675235; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=GOWXoYYRnWMsoeKK+VVuf/lMuv2V7GO+mfTTLFXfUSE=; b=K9r3UDWr+cCoWGmdOk4N9K4kTxe84MA1bVirlOjNP2Z8prI6qiOS9/AVkz1gc45V2H oWaUGoHtRBvv1xPSjhzG7veZosfd34vliIGF4Nx1K9dsgRuRifoVhkjdbKTOCS4BQoCq /XOsgx3l+b5oYW9Fn+8igCl2fqcd0Xyl2OO7qDPgjts4ODQ2p2ZYDlTHIudS1Q5P2djV zYQC8Bfb57zPgs22u0P9T9o1l6NkX1c4Wojvl2aeYUW/jDf6I3IoubQW1Uk9GUhbcNgC K1Ik8UpjfX14MvHWkGTdB0BtJcmKRZpAnhoSbBbxJfrfRJswbr55Bcw3G3gzLah146e3 fxCQ== X-Forwarded-Encrypted: i=1; AJvYcCWiVXWmKRvmeTp5psUtgtAd4ka/G3TPNMfVE6mGtM/7AgWf2hszTQUh/C/7NWpU09xGV/BNmZvPMQ==@kvack.org X-Gm-Message-State: AOJu0YwNKDb4NqlZArCSmFYFd4f0OGnUGwAZoOTu0KdaZRwAND7P10kv daqZ4in3RG21WPioCbDcoGDU7T9shGpz9N5v3d9ejJi1V5CTm2/nUux1Avmde80= X-Gm-Gg: ASbGncuFztRdyF8+onLC9MCB4vMeT8QsnWXUwEqsjXNTPl+ObfwmWxgFTwx+xk/Jnoi IWYYXOqR97AqM2Ozxoa+3uWzgHb/yTVqSkARwAFdhDUhUYgIYGG49DJxHeQxNamu/XOjsh+vgwn RfAAOs1WoZK7mskREvoEw+xZJE9ogLPLMiQZnK3cKtuqdwhQvrbdBT0IeBr6hsfKMF7OAa9my1o w1S6CVxovEcwOSs9aVSOaO+yMUsG5ey0Dh8aBecMh/3JZ6g8OHyzetjLo4n3PCZGFLhsensyXhs MqN3J8lvtATsEg/KcpLINvgAN2RVNKntuPfn7wjLEMknFHqANvL+mzo2OomWd0mNZ/w97vaMGEU = X-Google-Smtp-Source: AGHT+IFfqsRD0FOVkdIg887UXRm977J4ky26yoqHXaGeXqELEj2EneWq+CuXUYKkdzvufPHaZSVAIg== X-Received: by 2002:a05:620a:3910:b0:7b7:142d:53a8 with SMTP id af79cd13be357-7bff3fb4946mr402618885a.19.1738070435489; Tue, 28 Jan 2025 05:20:35 -0800 (PST) Received: from ziepe.ca (hlfxns017vw-142-68-128-5.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.128.5]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7be9aeedc68sm512887085a.74.2025.01.28.05.20.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Jan 2025 05:20:34 -0800 (PST) Received: from jgg by wakko with local (Exim 4.97) (envelope-from ) id 1tclW6-00000007TbE-14aw; Tue, 28 Jan 2025 09:20:34 -0400 Date: Tue, 28 Jan 2025 09:20:34 -0400 From: Jason Gunthorpe To: Thomas =?utf-8?Q?Hellstr=C3=B6m?= Cc: Yonatan Maman , kherbst@redhat.com, lyude@redhat.com, dakr@redhat.com, airlied@gmail.com, simona@ffwll.ch, leon@kernel.org, jglisse@redhat.com, akpm@linux-foundation.org, GalShalom@nvidia.com, dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, linux-mm@kvack.org, linux-tegra@vger.kernel.org Subject: Re: [RFC 1/5] mm/hmm: HMM API to enable P2P DMA for device private pages Message-ID: <20250128132034.GA1524382@ziepe.ca> References: <20241201103659.420677-1-ymaman@nvidia.com> <20241201103659.420677-2-ymaman@nvidia.com> <7282ac68c47886caa2bc2a2813d41a04adf938e1.camel@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <7282ac68c47886caa2bc2a2813d41a04adf938e1.camel@linux.intel.com> X-Rspamd-Queue-Id: 8DAD9C001D X-Stat-Signature: 8cizi4r11k4na8bnpbzgaih8nmkfhk3y X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1738070436-318625 X-HE-Meta: U2FsdGVkX19MtCxsMJrEp4YP5ixXFOemh4yEcrgVO5ZMtFVkwd+J1xAdQphHOvfpxf+8LaNDMlK9z+VfRrtnWWmbU5+6JxW8XqieQUr+J+X3CgUT/ltD/xZ2D2mU/e+I1nC+1leGFUit+lzmnIyiMgskLXDO7dOBFDwDTy3/m38i2Gxzrx4jd3G3RMrA0lNqr/Pw/FImQtSNmdJHIQ815r65M24cGD8h3rfQSxURsvA1a4d3Uf1SQx7PTIIrv9ftv/SoJXElJRyabQyTlGQKNujSMKHyl/7QPTbhLGWek9NTUSAbWXeCXkaX6myJy9nPj/jWALTERez0DxUsNG/ulF8jJNHnYmCBZ4D9TRAOSQtiHeUd9rpf5V/k3MWKvkD5cSUowfyBbhjWbkdgFjR8ntCP2L02PTiHFMmVt1qPQqk4EJJxxswh0lazm6BWR/nxNAMoVritf7sxx425rqBuuWEqp03H1Dv5QkCbLpnQ10Kbvs+MF/hmIegKysm0uti35JghvOH/7YsjWfJMJgkmYA22TUDESjESoPzv8qhuE09Pu9txnYdIzXMwuhjbeSUSmudEkY+D7BYB/wWqKVssQHyHvR/KCNtkY6QqzX3Rfw42rNeNEWXSPIrToS4H8Fboe+cYMsyYHUZMsjb7zo5xFY181m4lUugAENrhcA0dii26TIova17Gti6VmkUiC5xDyAquLQb28xxVpsjsB0YDGMbIy4rN9EQ+Rw6i6B7BxOInf0O6XeZnAUbBE23/15bKYSsBbQjToVPrwawiTRjiLMsmolPKtlG2nvYdcuHQgcJXwXyALFZ1vVRHFxJi8bG9NKb/XjfgdBPFQv6Ij6f58vxOiLRC6/dz++EhgcGhJT+PtG5aPee77h460AbGEcGx3K0ptef+vV+KKbhdkwQh8JQCw7e53jiLN4hD0OfgkFeaQTKFk3RaACNgMIQsixyw7bVXMhY07nQ0/wOfblq 1DSAOpSh 8mZyuVFlxkxSweQbloFRpA0oWS5mq81A1qTXfhFVCckOcVwn4qESAf+wMT+CkkwXGd2pGhiipum0aJXUUmQtVLMo4kLU7R/dM0emPIW7o3BVh0BQ9Yv5cwdm8fxXJ3Kp8u5O1Hnc9WA3QGHp5wRnVybnmF8I/2drXXfr8N/ajGid9zzjvk4sF11JAwZUGzRcpRpI+b26ZOVhxZTXI3HTEdmaFnOypmTwjgpw1IcCV2u0rUzOUeTMc4EF5ohY+ahacVSfqreLcvz2hrsoXwRKKCHXQbK81ugDTnrQkE/mD5fEKB91c7Mkdi/K3Cg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000027, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Jan 28, 2025 at 09:51:52AM +0100, Thomas Hellström wrote: > How would the pgmap device know whether P2P is actually possible > without knowing the client device, (like calling pci_p2pdma_distance) > and also if looking into access control, whether it is allowed? The DMA API will do this, this happens after this patch is put on top of Leon's DMA API patches. The mapping operation will fail and it will likely be fatal to whatever is going on. get_dma_pfn_for_device() returns a new PFN, but that is not a DMA mapped address, it is just a PFN that has another struct page under it. There is an implicit assumption here that P2P will work and we don't need a 3rd case to handle non-working P2P.. > but leaves any dma- mapping or pfn mangling to be done after the > call to hmm_range_fault(), since hmm_range_fault() really only needs > to know whether it has to migrate to system or not. See above, this is already the case.. > One benefit of using this alternative > approach is that struct hmm_range can be subclassed by the caller and > for example cache device pairs for which p2p is allowed. If you want to directly address P2P non-uniformity I'd rather do it directly in the core code than using a per-driver callback. Every driver needs exactly the same logic for such a case. Jason