From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57679D37489 for ; Thu, 17 Oct 2024 13:05:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9B4F26B007B; Thu, 17 Oct 2024 09:05:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 965526B0082; Thu, 17 Oct 2024 09:05:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 806466B0083; Thu, 17 Oct 2024 09:05:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 5CB5E6B007B for ; Thu, 17 Oct 2024 09:05:44 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 51E39AC816 for ; Thu, 17 Oct 2024 13:05:22 +0000 (UTC) X-FDA: 82683115746.01.57928A9 Received: from mail-qt1-f179.google.com (mail-qt1-f179.google.com [209.85.160.179]) by imf19.hostedemail.com (Postfix) with ESMTP id D98AC1A0005 for ; Thu, 17 Oct 2024 13:05:29 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=ziepe.ca header.s=google header.b=M1zXBJQx; dmarc=none; spf=pass (imf19.hostedemail.com: domain of jgg@ziepe.ca designates 209.85.160.179 as permitted sender) smtp.mailfrom=jgg@ziepe.ca ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729170294; a=rsa-sha256; cv=none; b=2Z8SK6aquVanDSCDMXmxHJlbc+kKfmGY0ziBGsbORDSQm6+i+sjbRxVmHj3wswQYuXBRII FjiBEl62Mw89ZW6U5JUqqZYLdtqoqRGkmhCndhKsbr3sxQ90A7IPOCt4B2j62vylzc34Xd ArnU7ezY3pnjt030cHyBI2paWpSI070= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=ziepe.ca header.s=google header.b=M1zXBJQx; dmarc=none; spf=pass (imf19.hostedemail.com: domain of jgg@ziepe.ca designates 209.85.160.179 as permitted sender) smtp.mailfrom=jgg@ziepe.ca ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729170294; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Lk++0v4zwR7xUpep4iNg0I6MKOHA7nEjzuqiKjJzkok=; b=yKOijroHSuWBySVs9fqXsb3uEmaVqAQIeRInvcMwepzobjnbTG/xWP7lFXHTKg6DkOMBbO HS39dJrYAadz88ursuMRMZbDE+4dKDlJmsKBk07IpeX8hrC+KzL1eJojH1M+7mwVde9yW5 YO6Xf1Gmi8lodKjL/INyHUKGlKXT75I= Received: by mail-qt1-f179.google.com with SMTP id d75a77b69052e-460963d6233so5628721cf.2 for ; Thu, 17 Oct 2024 06:05:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; t=1729170341; x=1729775141; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Lk++0v4zwR7xUpep4iNg0I6MKOHA7nEjzuqiKjJzkok=; b=M1zXBJQxp3Ch+VkjUx+BVVZgaCwbGDH8SFml5ymOX0PbtpT2qmi/7jHNVNOBxPbtzV FUo9wu2xSIG5cpvCNR5QOMsqXQUOXXCVvCk9xZTofSU//root/gcLqnl5RxnTmOL3tdH LFlmHUsoJHJlOFDxWYUwCmOuyBGiDQTjRPOJuHHcoad6RfFE7EBxhbl/N4IGA92KIEZ8 cTIl5GVQQHk/D/fKhFGpygSoUa7iXUQmmp2IiVsOZe3IuSjPGyfIH/gYArg1YMeC+LB8 MTNtJAzHpMMVrWf2wRreb8iybGqAI4ARPT5x/rLDM4zbr+IiaYrJyavVj2gn9MDLrXjQ nSRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729170341; x=1729775141; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Lk++0v4zwR7xUpep4iNg0I6MKOHA7nEjzuqiKjJzkok=; b=jurL1b4NzBOE5dcRwsPTeaSAoKqZNIAlfjffbg1sx/H+SyLucD1LFMwzXN6UA9dUqk fTYnOmafi8atjEmzqpWlzh43rcons4DhZzDNfuNfU2npueBYnsYpijp43YAY4TqJAlcw tXF1HQPKLVlOCaRVlQ64X60MBXrkf8QF+nq+/l0rSaOVyeICKsGlG7BkGKGgIYHMJwwD u+wAfnx1xcg8aC/nZx+0s9leSnNMpyVgfdje2ADHogrRkMGlYXk+toRqJRE7O/whOHUa oStMgzATn9aUjIVN0hpBD40+ldM7zeCkXGMwyIRF11Wrs7VwDZRPxp43DaP74eL8rkz8 Nmxw== X-Forwarded-Encrypted: i=1; AJvYcCW8S9pQXk1GqlIp9JQZQQjJhaUZJNGWkCympYMcVaxBROriv0Aq7zVEaW4Um6k/K9wnV4ZKUQ93/Q==@kvack.org X-Gm-Message-State: AOJu0YwsYiRXBC5fzLYMv8suqdJ41DwbraYvyVXz6Hoi+8pUj34m1WV/ sGsXcxHmjfsb17XhpWViAR13GN9FDi1iolP5F1n0GCO06kTq6/A+j4Z1qPICBT8= X-Google-Smtp-Source: AGHT+IG5n5S3zHSv/JX5s/LYaf+o3YrOaG8HklVpPKUbKJ/yS6kcQbfiEDJXO6iYDC0yAlGNuOhLug== X-Received: by 2002:a05:6214:2b93:b0:6cb:d094:d1c4 with SMTP id 6a1803df08f44-6cbf009b918mr251899446d6.30.1729170340973; Thu, 17 Oct 2024 06:05:40 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-128-5.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.128.5]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6cc2290fa01sm27747076d6.24.2024.10.17.06.05.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Oct 2024 06:05:40 -0700 (PDT) Received: from jgg by wakko with local (Exim 4.95) (envelope-from ) id 1t1QCB-003suE-SU; Thu, 17 Oct 2024 10:05:39 -0300 Date: Thu, 17 Oct 2024 10:05:39 -0300 From: Jason Gunthorpe To: Christoph Hellwig Cc: Yonatan Maman , nouveau@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, linux-mm@kvack.org, herbst@redhat.com, lyude@redhat.com, dakr@redhat.com, airlied@gmail.com, simona@ffwll.ch, leon@kernel.org, jglisse@redhat.com, akpm@linux-foundation.org, dri-devel@lists.freedesktop.org, apopple@nvidia.com, bskeggs@nvidia.com, Gal Shalom Subject: Re: [PATCH v1 1/4] mm/hmm: HMM API for P2P DMA to device zone pages Message-ID: <20241017130539.GA897978@ziepe.ca> References: <20241015152348.3055360-1-ymaman@nvidia.com> <20241015152348.3055360-2-ymaman@nvidia.com> <20241016154428.GD4020792@ziepe.ca> <20241016174445.GF4020792@ziepe.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspam-User: X-Rspamd-Queue-Id: D98AC1A0005 X-Rspamd-Server: rspam01 X-Stat-Signature: got33bpmqmdo4hz1zw69bujyr15854b3 X-HE-Tag: 1729170329-109606 X-HE-Meta: U2FsdGVkX18LFO8Z0m0Paa8o+6h300MhVy4jwDpQbPx7ddx/v/PEjdFEHzh85KY8S2bYibFuP1lfPkQzJ0reztkU6n8Af/mPeDPOroEN/xbvN5vvfDKCbRfFRTdpf4r/TmwiFzP0KPAytui5WOKNQDkVobt/9XgttfnbJ+t1cuh/u8UcuAtsPB4c6zg/+VJqpN1LbLzSlgwA2nCgUzL9z1wsTO+zBgd9CA05whyaY6td5zX5JGACJsY5/BtCcwyekIabObECKbKPCHimxeBNWktTR74KWCY+3deesdy7gn87ule/XVHsF0630G2fOG2UW0Th+SoECB08YINN8zqWvoYMWqkGUfhPIZD6pgmnYom1WkyzM1MgBZuIBmiIp/WPE+aztsEayaSlG4/JqO1Jpo2N3d6m4liFvDP6qkuMCSOAczOBvYmef0Ruy0TrLQEBPEh20nnPaQSEN/F14HbTho1t0qw09XU7qjO/4ZClVjTF8HuMLTSUruoPEVAL+8VsDVzUbw5ECl2D6t70pSuESNnPclnkRHvc2tZjDqbLuPjC+3RtS1I2qGKDu/z3JuOa7rVWeRx1q1s8DatzLw+ykwdvhBw2WXh+I/AOu7g7fSku2SiR+W9UERAFdCqOVZwZQw88FVkriIVW0fj+DqJnd35q2P0IXZ/dpOnnZ21ndoZNyh3TWXEo0rF4K1uPiCtdiCPgN4YP6l+ZP8j+12E1zQiyZwvfOaTg0AyzVe2D/scFIXmARDkkvityVkel5Cadj+KJzNdLciCCFeIzhil9P9wDv+Tj+Q6awkU4FzFo90C0HqtsClPTfTo7I8RmXQQjdT8CehkAgeTLZF1abA8eOykIxU/Jwr+l/ek3usxsksfMYJD0gjv4bsNoiiheaH5qyHJbmxHPFkWT1caMNQwbn0eaC1MUdDrY59SwSWOJkNuxF5C5is+vCIUZmRXiCVnXOpI0dV33JST6yHi5BY9 4itVW6UP 2hwciYDmxtrj8J+QHVMeUHFBdmtsWY0JtqIiZm7eXR77tCG6fwV8xOXLgOQNqIJESmDCUijcIbDdBGmOHTDF7BTc9vOOePKM5a7oP8Fa4QiC+fEb02TG62tMWT1OADehxk++9UzZCGw8gxSd4m+7oo5+zjA8haQv//4bzzKLp7HTIw6etrQLRAFzOnOxzUJcGl020n6V4UAci6RJtaF6FIGVpJYcPsnwSMKDktFDFSAhr0OHTEXXXKyD5lz04SiDHOZwP1a3QRFwomTs4wDcfnmSUJmkMLK6onmb0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000004, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Oct 17, 2024 at 04:58:12AM -0700, Christoph Hellwig wrote: > On Wed, Oct 16, 2024 at 02:44:45PM -0300, Jason Gunthorpe wrote: > > > > FWIW, I've been expecting this series to be rebased on top of Leon's > > > > new DMA API series so it doesn't have this issue.. > > > > > > That's not going to make a difference at this level. > > > > I'm not sure what you are asking then. > > > > Patch 2 does pci_p2pdma_add_resource() and so a valid struct page with > > a P2P ZONE_DEVICE type exists, and that gets returned back to the > > hmm/odp code. > > > > Today odp calls dma_map_page() which only works by chance in limited > > cases. With Leon's revision it will call hmm_dma_map_pfn() -> > > dma_iova_link() which does call pci_p2pdma_map_type() and should do > > the right thing. > > Again none of this affects the code posted here. It reshuffles the > callers but has no direct affect on the patches posted here. I didn't realize till last night that Leon's series did not have P2P support. What I'm trying to say is that this is a multi-series project. A followup based on Leon's initial work will get the ODP DMA mapping path able to support ZONE_DEVICE P2P pages. Once that is done, this series sits on top. This series is only about hmm and effectively allows hmm_range_fault() to return a ZONE_DEVICE P2P page. Yonatan should explain this better in the cover letter and mark it as a RFC series. So, I know we are still figuring out the P2P support on the DMA API side, but my expectation for hmm is that hmm_range_fault() returing a ZONE_DEVICE P2P page is going to be what we want. > (and the current DMA series lacks P2P support, I'm trying to figure > out how to properly handle it at the moment). Yes, I see, I looked through those patches last night and there is a gap there. Broadly I think whatever flow NVMe uses for P2P will apply to ODP as well. Thanks, Jason