From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19E38C02199 for ; Tue, 4 Feb 2025 19:16:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8905A280001; Tue, 4 Feb 2025 14:16:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 83FFC6B0085; Tue, 4 Feb 2025 14:16:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7075E280001; Tue, 4 Feb 2025 14:16:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 522AF6B0083 for ; Tue, 4 Feb 2025 14:16:06 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 0F4901603EE for ; Tue, 4 Feb 2025 19:16:06 +0000 (UTC) X-FDA: 83083217532.23.3B6ABE6 Received: from mail-qk1-f179.google.com (mail-qk1-f179.google.com [209.85.222.179]) by imf13.hostedemail.com (Postfix) with ESMTP id EFE9620008 for ; Tue, 4 Feb 2025 19:16:03 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=ziepe.ca header.s=google header.b=a1LC9rID; spf=pass (imf13.hostedemail.com: domain of jgg@ziepe.ca designates 209.85.222.179 as permitted sender) smtp.mailfrom=jgg@ziepe.ca; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738696564; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RbSThg2+IgOzUPVYO+O03rGdKt6FNxSQQgDDlA3zKuo=; b=ANSBPO0wG0kLEuLCQVRwQ2TYSodoBkixi13/qAw8E51tHg2QjLxtUO1j+Rk88tB0mUHsXB pUGWD2RbMe5wLtk4vBFuL1iIyOQZfJTtRsM+diwfM6+6AdBMGBXl688P/+NVKjEnXiKRt8 K55SuphNb1l4Zzfk0OM12vJnQOXZLjc= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738696564; a=rsa-sha256; cv=none; b=3hv4b4osUH7IdjqckGNE/m9txHs0XCgOiDg91iPLDauTUbNouyOXtQ8OxfLqUH84tW3UrH dJIrswzCx0kF2Z09agbrh0z9sk2oKQcKOld/Q8bNdNg8DpJxFqquvE5TchdeXdYsYu7C5M pE6KFMFSRH2gsTobCBPoIQj2pB9Ec+o= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=ziepe.ca header.s=google header.b=a1LC9rID; spf=pass (imf13.hostedemail.com: domain of jgg@ziepe.ca designates 209.85.222.179 as permitted sender) smtp.mailfrom=jgg@ziepe.ca; dmarc=none Received: by mail-qk1-f179.google.com with SMTP id af79cd13be357-7b702c3c021so576461185a.3 for ; Tue, 04 Feb 2025 11:16:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; t=1738696563; x=1739301363; darn=kvack.org; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=RbSThg2+IgOzUPVYO+O03rGdKt6FNxSQQgDDlA3zKuo=; b=a1LC9rIDkyt3UBTc9SIUf6rPAnearHKgIM4l8SMoEhJCir8LhnziJIwsRs42sQ7sAH leCYNFhHrLH5druJQpkRRNHh81WMhG/J812dbInxx65JI1YHTp21SUiz1B8+clUIuwc/ Yo98tPTvjKEAmdQnzUkGe8GL/9DuXBj60Q1SdIOXrdU5Ipiuqeq05JWGvCPysf9DtG6e eC1mnpetwMs57JtcamQsf0rKLDjvehTo9vcAQSEGX8SRfz4QlTzGwvFb3KSYcQfkPfeN t4Oc3O2KL3sQEjxvrmoqTuCZ/hHNoxE4ymTIuSUf7LujFvlUkOSaCuDnmRhYZNP/Y4H5 CmLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738696563; x=1739301363; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=RbSThg2+IgOzUPVYO+O03rGdKt6FNxSQQgDDlA3zKuo=; b=etZqLnWszJgo9xDT8f+O3lduiuhOYeufbeGk/JqLZUYqmGKZGAv+WKc+o6xxI57IwC OYz2nccHsgJPQZAouZmMNGT7o2hxv7Y5HOjIhWCc++zDeT9ZcI7Hxz/VS5sj3Now46rZ hB+eHdnplQ+/1Oxvcj4Tx6S+37RG7/KJI8/qsUQCVIKErF8kKZus3n9MgZDdKuFxFQck GleQ2hplCgeo1cQsFzZS9x9LGVTIyrBpMJK20cc+QaRqvZV1AHbqkg610+UasrE3ccW/ L5s67IwSkf6jM4XaCBiwlwrJjSGJpkI6lZrjFXDN+ntgdJcFQcLvs3MCkP/Ez0YfYPdy zlLQ== X-Forwarded-Encrypted: i=1; AJvYcCXCDcQKpgCpk25zMmZICX0Pc+LlruuQOxCY+guDQjpLVb7AqiMDvlEWeenPYleeWzFrxxeE5tytnQ==@kvack.org X-Gm-Message-State: AOJu0YxCKnD08XJnILwutFOQohx9JZoBnITRO1ttDeSm4/PFaBdkU2Pr xu+Lb5XUSEOUR5uc+kcUUPCHQEwhHVipIfjVTuji8kIrIeqfKawVdGFjrC3gIAc= X-Gm-Gg: ASbGncu5bdcZ+kOzpkzmKvE8e+d6DfOQDHAbtr4dxqQ7Yzp5srxD862U3sAhXSC6njO tTVE7m/vt9ufNUyMunOmO7qldaCFY6yvD+vuNxab9ho9Bpy4MgqElwD6KJ1rb+zP2pbVN40Qnh1 wYwEq0JobgtGGJKjqwDEToHDoR+5bmMPDdYGztRz5h9QAh8u4tMsd4LBKYY36keTrUb5gS03uq6 +7ESRXCRkcCbSKXNBW/k0MVePpIlY3FukgF1LVsKGp28icJfy0oUxv/msGHfELK6Wya1yaAEHIS MURir2XTseGIe2V4bIRbE23VgS9CDstlxsi2UHj63OfV9QOoT+OQoSBLNDP8NG1h X-Google-Smtp-Source: AGHT+IFDaDGZr7b/tFbMEfs7niyU8J8dtoT91Z8F6Kt1MP6P2VK98JCkXIPoyuzy/DdpTLj+ZRuWhQ== X-Received: by 2002:a05:620a:2710:b0:7b6:ebc6:181a with SMTP id af79cd13be357-7bffcd9997amr4139068785a.41.1738696563125; Tue, 04 Feb 2025 11:16:03 -0800 (PST) Received: from ziepe.ca (hlfxns017vw-142-68-128-5.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.128.5]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7c00a8d97c2sm670296085a.64.2025.02.04.11.16.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Feb 2025 11:16:01 -0800 (PST) Received: from jgg by wakko with local (Exim 4.97) (envelope-from ) id 1tfOOv-0000000CUOy-1LgY; Tue, 04 Feb 2025 15:16:01 -0400 Date: Tue, 4 Feb 2025 15:16:01 -0400 From: Jason Gunthorpe To: Thomas =?utf-8?Q?Hellstr=C3=B6m?= Cc: Yonatan Maman , kherbst@redhat.com, lyude@redhat.com, dakr@redhat.com, airlied@gmail.com, simona@ffwll.ch, leon@kernel.org, jglisse@redhat.com, akpm@linux-foundation.org, GalShalom@nvidia.com, dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, linux-mm@kvack.org, linux-tegra@vger.kernel.org Subject: Re: [RFC 1/5] mm/hmm: HMM API to enable P2P DMA for device private pages Message-ID: <20250204191601.GK2296753@ziepe.ca> References: <20250129134757.GA2120662@ziepe.ca> <20250130132317.GG2120662@ziepe.ca> <20250130174217.GA2296753@ziepe.ca> <20250203150805.GC2296753@ziepe.ca> <7b7a15fb1f59acc60393eb01cefddf4dc1f32c00.camel@linux.intel.com> <20250204132615.GI2296753@ziepe.ca> <3e96aef8009be69858a69d3e49a2bd7fc7d06f5f.camel@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <3e96aef8009be69858a69d3e49a2bd7fc7d06f5f.camel@linux.intel.com> X-Stat-Signature: xceskyuyz8dw3urhocpsm8cd4nbhz1nm X-Rspam-User: X-Rspamd-Queue-Id: EFE9620008 X-Rspamd-Server: rspam03 X-HE-Tag: 1738696563-805720 X-HE-Meta: U2FsdGVkX19T4PLkaytrUxgYBgdKYTcSYjAJBUBqWjAEjWF++6D2XhG0RT5e/1YJ/HGJlYTjOoXLUJRGzUGI8HKHDKBSENFw22pcQ8XkB5STf/nsZEf3D5Z3TWBpU9wHUFzy03CpM3PTA3IEZOITW7LWt0M4esPy98WUHPsLv5qvR/ZTMn5/xtIZon2vmV/HAsYEOFJuuGWqdMYUKgtt/YUGFNV7W+uhOwUvkf/d6d/jbVnGgNGZdJ/ehFq4YVo360UAdDBzu5r+cbBDriM38uuEM23h+/nwqvump4NBnqyPhlHF2Fp6dCMJhRcFmHBCOKY30Q4soS9sQ0sqyaH6GLK88mHEH960ZqgKR8HrqXOGgLkIHUadyVA/ai5Q4UL2/WrD+ti5bCxgb6EPJ25bgoXBUQ3h0ZBz9BRFHSz1Uyru9QGgIbBppEraOTxNIr73v8VgjhDp1OKJ9qq6qcGhxTslHyfmAob94NIHhwYJM8DYaphjjUlLa8R5RqWjJiLDC25tboAcjYxvQw3czz8a3KYYcWtD2s0qfiy04E34yGOOp/cKt/iq5Xw4eodczdg69BbvzwVRbs8y3kWrNGLeELYS46qV7bSbsgAPftTScKVDnMLmCRiOKxoGUyXeJpHyCHsKIgZBvHaM/v3KWbI2j1hAz7gvu+HRmJZCRxZnoOZrYzZswi64bWWsGY3p2U2h4IgEt/iM+2fI2mcShiG9UporDjRqvvbrxaBcQFbUz4+uwfFxfW7RSdzHo14UHoAX1XojVShRif7L5OjLs6/ffN8INeMNWsiFaTnoEkd/pu36wINTTJprW6oUdxTLUjIg/bkaesbFq+OVX3612dUGLv+ODuXIWf2+zQj8ytIiYjziDBBA++7uVL+6hA60SxiXj0gph96y+4Qo3GrRXT1AoY2dRD6jhzBgWZORAuMJjnbEaDVQTiu3BJSOs6HqJemCuVwy92jZ8HNgu5T7CeC c4SvpUI9 6hhbuoArgi0mI/sGh6iwElnpz25icOdx+nZ9FCJbR4YJLpF3dUd2XKY+c5kl4kl4VvWZo1s1U6RGT8dzEpbJKTWLtDMyyLWuB5V039uiUQ07I+XMLbnf9HMLP2JsLA6x0tw9+8dJpvTYl4LwzqfRocUcygC3FBufi/8v5qbZ+zfVaXeX+ef291OyzuT4WgP9c8JskwaORwzBGtR5pTSYuUc6vZGzFOK1d89Ffc8LtvJRMao2T2dcDMuhKnT2fAd3JJCvJhLTVjFJ/yTsXdxoiPs+0AWkuHldyuxTKymlf2p599uMarZn+nFIsyw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Feb 04, 2025 at 03:29:48PM +0100, Thomas Hellström wrote: > On Tue, 2025-02-04 at 09:26 -0400, Jason Gunthorpe wrote: > > On Tue, Feb 04, 2025 at 10:32:32AM +0100, Thomas Hellström wrote: > > > > > > > > 1) Existing users would never use the callback. They can still rely > > > on > > > the owner check, only if that fails we check for callback > > > existence. > > > 2) By simply caching the result from the last checked dev_pagemap, > > > most > > > callback calls could typically be eliminated. > > > > But then you are not in the locked region so your cache is racy and > > invalid. > > I'm not sure I follow? If a device private pfn handed back to the > caller is dependent on dev_pagemap A having a fast interconnect to the > client, then subsequent pfns in the same hmm_range_fault() call must be > able to make the same assumption (pagemap A having a fast > interconnect), else the whole result is invalid? But what is the receiver going to do with this device private page? Relock it again and check again if it is actually OK? Yuk. > > > 3) As mentioned before, a callback call would typically always be > > > followed by either migration to ram or a page-table update. > > > Compared to > > > these, the callback overhead would IMO be unnoticeable. > > > > Why? Surely the normal case should be a callback saying the memory > > can > > be accessed? > > Sure, but at least on the xe driver, that means page-table repopulation > since the hmm_range_fault() typically originated from a page-fault. Yes, I expect all hmm_range_fault()'s to be on page fault paths, and we'd like it to be as fast as we can in the CPU present case.. Jason