From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 152A4C3271E for ; Mon, 8 Jul 2024 23:57:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 527C46B0098; Mon, 8 Jul 2024 19:57:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4D75F6B0099; Mon, 8 Jul 2024 19:57:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 351046B009A; Mon, 8 Jul 2024 19:57:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 16F3E6B0098 for ; Mon, 8 Jul 2024 19:57:26 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 8A747121633 for ; Mon, 8 Jul 2024 23:57:25 +0000 (UTC) X-FDA: 82318249650.06.E4823D5 Received: from mail-yb1-f179.google.com (mail-yb1-f179.google.com [209.85.219.179]) by imf12.hostedemail.com (Postfix) with ESMTP id 9030D4000C for ; Mon, 8 Jul 2024 23:57:23 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=ziepe.ca header.s=google header.b=Ez2cn7VB; spf=pass (imf12.hostedemail.com: domain of jgg@ziepe.ca designates 209.85.219.179 as permitted sender) smtp.mailfrom=jgg@ziepe.ca; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720483020; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=w81yhK/4eC4FICM5BdCB5730Wjijfv+GUBmbab3qDKE=; b=GwPyKek1oBgL2N3zG5y06NtvT9D/9v0rhoPffPdpT1k60g6nHHcOtam1V9pGt7stEd4EXx ZFmGVXuNBT2113mUw1fcfR8ju2WB1+AqoLDi8UvjauhPn7KVbjVml5/SOaa2c12xe+iPm1 fwn3ul1VfmlaemmKtWZzyDuIqmNHX70= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=ziepe.ca header.s=google header.b=Ez2cn7VB; spf=pass (imf12.hostedemail.com: domain of jgg@ziepe.ca designates 209.85.219.179 as permitted sender) smtp.mailfrom=jgg@ziepe.ca; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720483020; a=rsa-sha256; cv=none; b=ejFvp63RrTQURdhQUVPDP5A2jenuyzWcuKpSaCX0R9AHzZYOwb0yAyiAFvOdnK7yB6iVWy WpZzeq7YXKzrshADz6ydJCh9UqGkIgByqXbZK8nc32cbaxjpkJ+ETTc6UpcvmOeitadQ+U k2+ai+bK7XzvD/K8V5D00G1qtWWsSWo= Received: by mail-yb1-f179.google.com with SMTP id 3f1490d57ef6-df481bf6680so4394086276.3 for ; Mon, 08 Jul 2024 16:57:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; t=1720483042; x=1721087842; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=w81yhK/4eC4FICM5BdCB5730Wjijfv+GUBmbab3qDKE=; b=Ez2cn7VBzdlgAYxSPifhG8wRGVJEgdFCXkFB89zcm2GU6yqX5uK9+IDEbvzjkjlCkc 4PJ5Ogl4O7gB5IVyBEWa2aVKvCJEwQyclrE+pXt8vfZUcu6Xl9Aas+ximKH4KLgmvgot FXIh34hC64i7MqyMZgrf/TS0SWrVR9ESuZQmO2gAZ+7rM/+q+zSEBDFfgnbNYlr8MQkT 5Vrq+VojQnugFsV33c3TP5CbQHCRiO8u1YjjH2TZVdVBm2tPVw8wdmhkOJZM5AstjqlF QiSbRU1dqYo4N0bDN8vAnkz75a0hg6F5gwQZIHw/+AXkvF2wat+do6LsR8DI0ll1Ve1Z J+rQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720483042; x=1721087842; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=w81yhK/4eC4FICM5BdCB5730Wjijfv+GUBmbab3qDKE=; b=dDFvg4YHAF44P2fGZiYiubbwZzPyjwctTWLcTLgGJElKGvvv6iN84DsFGr0aORJW0S DRmvdSairKb1uYhzxbYTM/5FVpcfb/DCUV3EudVEyW+6/ZzV3J8krRK9xqXaFboAS3Uk +Tb2EcWcImtxGBOqkqbouvHx4qMpslWxaM4v063w69KXnsT1fHiFD0C9OIBKa39Vv4i8 GA2klCl9k5AaELQYrpuBRWjfg/jom1FkuG5RZTnqwMOV9QNtB3vpwlni90nSlpSatMXU l1PX75Bop+MidFfPSwpgsd+zjmlY/0s0N1wRC7M0+SKNmSbM4syDUVorA7qpzi3E/3yc nBNw== X-Forwarded-Encrypted: i=1; AJvYcCWQhKASlASe8p6KnlugFI6u98qC7hdJe6JxDbpz7L7qwysoTwivgxJlrrC2EmYuYVGIBBIctuJB3kjIT+pWA/RyWqk= X-Gm-Message-State: AOJu0YzXdRgl6jw786ZJLvM2hVoYnm9UoBnGpzWXkjtoZSMVQk0pOcx7 1h3PRxk7TjHTr+RWHSqL8tS9WI1tWdh31u/GxJSHHEMnT4Ssbv86P1Lz3i64gT0= X-Google-Smtp-Source: AGHT+IFVd0YPFm+CFj+OFAbn4VlgzR0c7GB9Ja/EMJb3su7krIj2ap0s4sP+8C7C1jj96rzmX8e+3g== X-Received: by 2002:a25:d694:0:b0:e03:b14e:350f with SMTP id 3f1490d57ef6-e041b125e84mr1427037276.50.1720483042570; Mon, 08 Jul 2024 16:57:22 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-68-80-239.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.68.80.239]) by smtp.gmail.com with ESMTPSA id 6a1803df08f44-6b61ba8ad1esm3902106d6.105.2024.07.08.16.57.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jul 2024 16:57:22 -0700 (PDT) Received: from jgg by wakko with local (Exim 4.95) (envelope-from ) id 1sQyET-0014tP-5Q; Mon, 08 Jul 2024 20:57:21 -0300 Date: Mon, 8 Jul 2024 20:57:21 -0300 From: Jason Gunthorpe To: Christoph Hellwig Cc: Leon Romanovsky , Jens Axboe , Robin Murphy , Joerg Roedel , Will Deacon , Keith Busch , "Zeng, Oak" , Chaitanya Kulkarni , Sagi Grimberg , Bjorn Helgaas , Logan Gunthorpe , Yishai Hadas , Shameer Kolothum , Kevin Tian , Alex Williamson , Marek Szyprowski , =?utf-8?B?SsOpcsO0bWU=?= Glisse , Andrew Morton , linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, iommu@lists.linux.dev, linux-nvme@lists.infradead.org, linux-pci@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org Subject: Re: [RFC PATCH v1 00/18] Provide a new two step DMA API mapping API Message-ID: <20240708235721.GF14050@ziepe.ca> References: <20240705063910.GA12337@lst.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240705063910.GA12337@lst.de> X-Stat-Signature: 7pz5sdiofbh8688841acxadrs57dy1ad X-Rspam-User: X-Rspamd-Queue-Id: 9030D4000C X-Rspamd-Server: rspam02 X-HE-Tag: 1720483043-393746 X-HE-Meta: U2FsdGVkX18Y8DfRJ1Ob2W/qYzxGWWKh42E1um2M7GS0m1+MsM1tND2nnpRGP5qFiO2yh4IxLKsAnQDY9MK3r3ZtxQcg7zhZ2qiETOHhQvk2w1ylYbYWwXZDFKDVyDwaImFRpM6ySLY5+oh8Xvc1BhjCBk/B8dVTCD1T7uIyHJ7jkdElrcJ/zZvrybrx0bF7wiasVkaaZOooYUGl378TXG3dD9Th/3cJgWlG5NxS5hs8mIEMA9pQgHAZkXaf35Y/NTEAR5pr+fdLOJtVnlYHnAVxClhrjxDDnNVfb2Wfb+oOHiPDtn4NRf9Z8PFH9aW3IVmlcBeHE1eNl56p6oAf7mU2NQ+2XzWAhurOw6tvauOOYmN36cDYgdi6lZ/4wPiWfAvI1bmiQ7b6zxBkp48rZnLPUeIHbt1O9IrvfetyjwZyd9X5hQ7gpebHUCw5p0zMZdL9JDIIAdmLgegx5rl/pSxH8WtiFxiSQOHyiuQxBSZZ1yXXlvre68Ju8hAs2Om9xJ7K5NHVcjhaGDdLnBPTZzd2qx4xE82B39Fud4NdmXJITK/tKHopYU+Ob+WCq2xsN6nr4o1HKnZiYDDufLYvZ36dTcW7idObvrOB1JRy4/lcq+O3K/oEmM2Denw4Y9+S1ynGSuTEj9+KtCy+m3Nrf8gwR2klZU8+k56Sk9PUTCNTSc7A6qKlEbgApnybpjp+j+TUenWhbTmwzhdfqKHC3+Of31ghmWjiJgK1dbDMWPfvo0vfVgY4l+K4zqd3AFEpimlTAId+MV8dfoF6UuraJuXTnzY/8vdTVgQPeDx9kRjcjbbR2yb12t36EFBOOZEBrt+/IdOxalC2/RMWxogmKUwdo+tgPhQJ/Q48R/iQmuh9JAygvldJ+CBg/qV8M6pdYKadHPVZoqdDxuXNX8ff3NjySvJrSE2PuPVN89fcptZiKf6tyQMMd3ZgcaMwiwyqwndhB8ky9LyrC+I2tHf 3uoEagdR 5N6b42vm8H7teKxb1bh+xYAtICkz02MMY2bcDh9Joz3cXjOzkppzWWp20ixJxRyClzgtNzb05GOq42LChpA4AIDn1plCAabDjx2dGsfZEHJ5MVRgZIoF62hE2R+AHqlaqY9zKnPtkfCinyCxfaqfw4oG0AJvtvLSI67UvvTOKei54S8YdO2xxgZS/zAvDTLr8jVBZ0jzP1R7/ia4/vbXvGKuq2qZs27jwmkRdfO3AjHnbLktapmCKS8r96DSSUaTk4K2TpZGkK+orYrE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000026, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Jul 05, 2024 at 08:39:10AM +0200, Christoph Hellwig wrote: > Review from the NVMe driver consumer perspective. I think if all these > were implement we'd probably end up with less code than before the > conversion. One of the topics that came up with at LSF is what to do with the dma_memory_type data? The concept here was that the new DMA API would work on same-type memory only, meaning the caller would have to split up by type. I understand the block stack already does this using P2P and !P2P, but that isn't quite enough here as we want to split principally based on IOMMU or !IOMMU. Today that boils down to splitting based on a few things, like grouping encrypted memory, and grouping by P2P provider. So the idea was the "struct dma_memory_type" would encapsulate whatever grouping was needed and the block layer would drive its splitting on this, same structs can be merged. I didn't notice this got included in this RFC.. The trivial option is to store the dma_memory_type per-BIO and drive the de-duplication like that. The other could be to derive it from first entry in the bio_vect every time a new page is added. This same-type design is one of the key elements of this API arrangement - for RDMA I expect we will need a data structure storing a list of types with a list of pages, and we will DMA map type by type. Jason