From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34BD8C02193 for ; Tue, 4 Feb 2025 22:02:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9AAB46B007B; Tue, 4 Feb 2025 17:02:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 934306B0082; Tue, 4 Feb 2025 17:02:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7D4836B0083; Tue, 4 Feb 2025 17:02:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 6101B6B007B for ; Tue, 4 Feb 2025 17:02:00 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id C67A846FFE for ; Tue, 4 Feb 2025 22:01:59 +0000 (UTC) X-FDA: 83083635558.07.FCF901C Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.14]) by imf11.hostedemail.com (Postfix) with ESMTP id DD9CE4000C for ; Tue, 4 Feb 2025 22:01:56 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=SdO8o5t8; spf=none (imf11.hostedemail.com: domain of thomas.hellstrom@linux.intel.com has no SPF policy when checking 192.198.163.14) smtp.mailfrom=thomas.hellstrom@linux.intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1738706517; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=lbPHclbARFkQmvsu0zJqrdw8H+4xG9Vq8ZiG/BDW7Jo=; b=eqfP9s/dRi58itTkL5AJDUliMrFWaAk6cmGTk+QYGqfuB5eztARF1x55wY+/28/rk30nYV hMYIP/szXWcWK4kSUVLfyOfyeQqMefn02FbkJLo0Hs2Y1vnlhZp9bGwRDerE2CInEtHoaV 4HbuZnL2N7xp0tMmCz1tk3uQOTUd8CM= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=SdO8o5t8; spf=none (imf11.hostedemail.com: domain of thomas.hellstrom@linux.intel.com has no SPF policy when checking 192.198.163.14) smtp.mailfrom=thomas.hellstrom@linux.intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1738706517; a=rsa-sha256; cv=none; b=8k7EksLWyb1/wuavjGBLOO6W1tsiADTHJHQeimPNi+uwQsXK146a09hWuVa6Pocrda84hU i6/o8Zlot3IIAjZ9umtIQ8X3gjFP+sn5d/S5wEZmZ5Dfx8Vl6dY0Vk1mgzsQNe+vDYORgJ es8iP8yx3YGbqPKsAaq4U207mHDeaDw= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738706517; x=1770242517; h=message-id:subject:from:to:cc:date:in-reply-to: references:content-transfer-encoding:mime-version; bh=lbPHclbARFkQmvsu0zJqrdw8H+4xG9Vq8ZiG/BDW7Jo=; b=SdO8o5t8l25nWlUS7nRVK96gD06y5ImWdrlIRyj6biZZVYr2V8sVqGuL Tn9Iomr8JyF3p3eZimWCMD10jwov8R20ndKXh7cdvyAPG0SDufnBFZtcn Ycq8+jF2K8gWtTv8jK/wcxlRUEDsMdfaFNm0vROumUvkTWMaZzKAyua38 B2MDNmE3eD+iip6J+dwF7xNsNOZ+QX21Qyhrs/MQoTo2vn8F3CRxP9RKl lhLjJETaIfqsBFbXhhesBfZwgBB8zWYrMoCeu8VOfT2qrfVYbASSyjlMj hsIrFiugFuMFTFoysx2bML+DtydZrQJem8CBYOH8Ly18Fax5pZ+Pn5+ka w==; X-CSE-ConnectionGUID: XdozRTBSSne2qo1GtdlZ9g== X-CSE-MsgGUID: bDQhyDErTCaDkv9WQgMTaQ== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="39517118" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="39517118" Received: from fmviesa010.fm.intel.com ([10.60.135.150]) by fmvoesa108.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 14:01:55 -0800 X-CSE-ConnectionGUID: NAx9oX9yRyOex+aeBj7lDg== X-CSE-MsgGUID: DNsEcHZsSNySIyoK+sYkqw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="111263801" Received: from lfiedoro-mobl.ger.corp.intel.com (HELO [10.245.246.144]) ([10.245.246.144]) by fmviesa010-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 14:01:28 -0800 Message-ID: <60b7e29853cb33d45d10101e494c7ddbe6a5abd6.camel@linux.intel.com> Subject: Re: [RFC 1/5] mm/hmm: HMM API to enable P2P DMA for device private pages From: Thomas =?ISO-8859-1?Q?Hellstr=F6m?= To: Jason Gunthorpe Cc: Yonatan Maman , kherbst@redhat.com, lyude@redhat.com, dakr@redhat.com, airlied@gmail.com, simona@ffwll.ch, leon@kernel.org, jglisse@redhat.com, akpm@linux-foundation.org, GalShalom@nvidia.com, dri-devel@lists.freedesktop.org, nouveau@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, linux-mm@kvack.org, linux-tegra@vger.kernel.org Date: Tue, 04 Feb 2025 23:01:25 +0100 In-Reply-To: <20250204191601.GK2296753@ziepe.ca> References: <20250129134757.GA2120662@ziepe.ca> <20250130132317.GG2120662@ziepe.ca> <20250130174217.GA2296753@ziepe.ca> <20250203150805.GC2296753@ziepe.ca> <7b7a15fb1f59acc60393eb01cefddf4dc1f32c00.camel@linux.intel.com> <20250204132615.GI2296753@ziepe.ca> <3e96aef8009be69858a69d3e49a2bd7fc7d06f5f.camel@linux.intel.com> <20250204191601.GK2296753@ziepe.ca> Organization: Intel Sweden AB, Registration Number: 556189-6027 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.54.3 (3.54.3-1.fc41) MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: DD9CE4000C X-Stat-Signature: jbhiu5h86yjm7t8i9ym8dgq14nc975jo X-Rspam-User: X-HE-Tag: 1738706516-213120 X-HE-Meta: U2FsdGVkX18v4SNVQYQRyT9VzQZqlZJsg4o6v+ICJgeFbS1b7lc+NYWIaR8/40CNgBhoTR2pHB9lU9FrQvCS+Bav4Us9zFTCK9oyBVaJjh+LdrZgCFHRcpY+xftdyJivFxvp1eUCwyVO/HpoTMnrz/qLVhDuiR30Vn4xz3/vkFXf1mOebGv8FTiaWNNBYQxXgxNMvQ0Erg1WxexqpReAwqdyr5t5f4+LPZNkbclHuOb5DPUjDJlEADSikvsk9K4Nk2XfPJCPUjEqj+m/C7NpMJzZoqwo2Xn3EsgrS2WGQfYVTMOzCiStDslfzIBCyd400BnQ+VXv6NbTPFLmpVtmQFuxdwzy8Gw/OOIoahluhtUAWnFDlLh3RnwFO4CIRAAVWADKY869DdhbwbFrjSDQTq9IVZwioh6vruEJF6NYarNX46WeKYGAXz5iCjyHQPpIuSR9u9bRl1yw5OhR7jyTgNwl7ZkGY9HZ3MRYPVRW3GPc7+Kg2Qj5Yk2K9UeJ57xPSNjcF8jJjXcZOPL2kLwDyH55NgUaJ0lHCtyy+TW+LeRK/ETaXnAZN1/6YPHj6X5renwNkieFa0lXX5gSWTNOR0re4UbugZpBgS+RaKkMrTlFgoCAl0aJ/r9B9s6iOVV7wdJYjut8OysTrRD26PtXKs/4yHcEwITGPz54JfVnphNCJLkC2meKYt2YuXh7YZkAPUPKPE3z65xv8FA536XarJ8fjdRceWX2sEuVT1Fz69Wk+iZKY5cmHJNEzS4h6e8FibAECB+MylhVglm8xTNvxJ0L2sooQfYSzuIfs8RUviKs4tfADne5A65Nl/1sbdpyzBNDVKwntzsGXK7bREe5JajxESw+t3hJr6jX3/TrQTW254mCaC4jAB6PHwf7oulU/RW/wUjxqoQWoy67Lt+AC9n6WpuNu4MHJZ/FMlxRry8SxN8lG7XcDOyxwDLLsxq0eVLEKL/4ZMMXDwUR9qp SNEfI0c4 WSzmnhkFm+QqCik1fYSnG5lKBbJncVR1p85ORNpnYUd1mD9lppde2SKcKILIRPGRdYjUbm5ynqsbrPVHrd2gfE0iN+FXWlz2iXMr8LXKgcwywjqcr1O4YsoLdFQXFxEmp7LJ1gutCC50LGhdHNf7t0+jj1K7t9XWcUfTVvX8EDydiRBm1nc1L6/dXYsa9YAWzXZARej5fAudrz779+QgROtKsB2rg6Mg2rlc78kznQebveVc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, 2025-02-04 at 15:16 -0400, Jason Gunthorpe wrote: > On Tue, Feb 04, 2025 at 03:29:48PM +0100, Thomas Hellstr=C3=B6m wrote: > > On Tue, 2025-02-04 at 09:26 -0400, Jason Gunthorpe wrote: > > > On Tue, Feb 04, 2025 at 10:32:32AM +0100, Thomas Hellstr=C3=B6m wrote= : > > > >=20 > > >=20 > > > > 1) Existing users would never use the callback. They can still > > > > rely > > > > on > > > > the owner check, only if that fails we check for callback > > > > existence. > > > > 2) By simply caching the result from the last checked > > > > dev_pagemap, > > > > most > > > > callback calls could typically be eliminated. > > >=20 > > > But then you are not in the locked region so your cache is racy > > > and > > > invalid. > >=20 > > I'm not sure I follow? If a device private pfn handed back to the > > caller is dependent on dev_pagemap A having a fast interconnect to > > the > > client, then subsequent pfns in the same hmm_range_fault() call > > must be > > able to make the same assumption (pagemap A having a fast > > interconnect), else the whole result is invalid? >=20 > But what is the receiver going to do with this device private page? > Relock it again and check again if it is actually OK? Yuk. I'm still lost as to what would be the possible race-condition that can't be handled in the usual way using mmu invalidations + notifier seqno bump? Is it the fast interconnect being taken down? /Thomas >=20 > > > > 3) As mentioned before, a callback call would typically always > > > > be > > > > followed by either migration to ram or a page-table update. > > > > Compared to > > > > these, the callback overhead would IMO be unnoticeable. > > >=20 > > > Why? Surely the normal case should be a callback saying the > > > memory > > > can > > > be accessed? > >=20 > > Sure, but at least on the xe driver, that means page-table > > repopulation > > since the hmm_range_fault() typically originated from a page-fault. >=20 > Yes, I expect all hmm_range_fault()'s to be on page fault paths, and > we'd like it to be as fast as we can in the CPU present case.. >=20 > Jason