From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2723C369CB for ; Wed, 23 Apr 2025 17:29:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5CE696B0025; Wed, 23 Apr 2025 13:29:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 57D8D6B0026; Wed, 23 Apr 2025 13:29:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 41F1A6B0027; Wed, 23 Apr 2025 13:29:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 24C396B0025 for ; Wed, 23 Apr 2025 13:29:00 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 97F9F1CFEE9 for ; Wed, 23 Apr 2025 17:29:00 +0000 (UTC) X-FDA: 83365994040.16.EB347CD Received: from mail-qt1-f178.google.com (mail-qt1-f178.google.com [209.85.160.178]) by imf30.hostedemail.com (Postfix) with ESMTP id AE31E80012 for ; Wed, 23 Apr 2025 17:28:58 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=ziepe.ca header.s=google header.b=bNwaBMRk; spf=pass (imf30.hostedemail.com: domain of jgg@ziepe.ca designates 209.85.160.178 as permitted sender) smtp.mailfrom=jgg@ziepe.ca; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1745429338; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IO21pmrwIRca1fPgD174SCTD3P5B2ca9Ykqyd/DD35Q=; b=ZD4MvTnOgLH8p+Yn7CVr/SFLfFzH4wqgCER4PJ5GXenynVGlEHG2+ma7CVsokqD5YVD6py gEISQNSKaZn8zTzUJmjSTHZ1B+U5YYeBNVvXk/Z/EUQ34aNIPSPCMyzndZpZn2MrSOu9hE Pj+ZoTuxdEmMoMgWf3Y8Nd0mBH8CQG8= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=ziepe.ca header.s=google header.b=bNwaBMRk; spf=pass (imf30.hostedemail.com: domain of jgg@ziepe.ca designates 209.85.160.178 as permitted sender) smtp.mailfrom=jgg@ziepe.ca; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1745429338; a=rsa-sha256; cv=none; b=N2sOntEsN162to16qr4Vpqd0vtzVCL3uWajsbj/sm+ybCWV9tRMQS1SGFQaJh3yn4kQAHy 3KpaMvcqKSOc2KVfo7Qz6/BOCkeiNdpFD7mcvh/ezxoF8uhBKSlFk4gDF3HbD3f81x2WFZ J6w5slRUVily/obF6GroWlNNaQmOonY= Received: by mail-qt1-f178.google.com with SMTP id d75a77b69052e-476a304a8edso1008491cf.3 for ; Wed, 23 Apr 2025 10:28:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; t=1745429338; x=1746034138; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=IO21pmrwIRca1fPgD174SCTD3P5B2ca9Ykqyd/DD35Q=; b=bNwaBMRkSbpnA9e1Wh8rReKzCTX53tPyU6Br1MIMklnOGup9BDpaHd11A/8flF367K 5M5+BLSBP8qVZqmTW4crEJX1gAIJhh9Ezf9WfzCLjS8VL/FOExqEE5TRKjPULqLkIGsX 6wWRJqt2PSpbIIw5C6xhR9QL9V4dT6bhs1+EPC47JtH7o7Ge6CJ2M0bXce2b8juZJIPU NzG7OYLiXQBz6uDg7rwXjw0PWlx98VUbyuFLf6yKTLXns0p3hy2awehEZfsWyruTvEK5 Q5RNh+KgXoU5eIgELF7j1do0S+1zVKMOxKq1Vjy++Q5sc0c8KbuIWaf5K9IGo32oUrhE HOcQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745429338; x=1746034138; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=IO21pmrwIRca1fPgD174SCTD3P5B2ca9Ykqyd/DD35Q=; b=Qq9pB2J/HMVf1uu27GysyDNbPw7Ebk+FySYmkdCSlCnRu8SgdYCdtZhYct2DSrPYo+ QtfArQUAaCIyDNS4xu9cNpsjKLWiH9tN017kC9mRy/FKLVJ97rGYSbnHbNxYfZAyZGjP 8gT+1cIu1oSGPc/yn+QnUSaiT8yG8CQuCz3zNDh0XrMiKLlDxikVe/X80QwlzRvMNYFG uPIWhdc0PJgHXQ43QL1VaeZfpmHYCJ5FYeSstdkIpXoChiovJ+C2sw8iSmZbTy7qLoVG 0E/vD9Qu6QZotwfISMroMRrAGGjCkAvLbCOxTzw5FCZCoULF57UIWmWeBCoTVbjvJMCx 1+aA== X-Forwarded-Encrypted: i=1; AJvYcCWb5wgMRug8+eRuyJGABzb3O6aAuYqjSDtAHx7ZA76MzWkczocMlVvWRBnG17YfeOAMBlVVexn8Dw==@kvack.org X-Gm-Message-State: AOJu0YyBLmQQK9rGowiy/Qz8RPA0HC8FSfXwzpzBlyUUGbkmGc26SgL8 4HP3ekRQQoc/53B0bzN9rj9fXp74z5yx6qAmF7AjnUKI5OsaKr8XUGkDWit0+jY= X-Gm-Gg: ASbGncshtboSgIVaSQHBdj9ZklivlZ2pTC4RYinVkimIjLiKYiNK5kX+46MUVMfE63g OszfMSlzbRl7iKqEkIt87uOg2P/b8WAXuaQFejz3KENuMvuWo0cTfDQ1YuQ5L83+zySQpzFIR2N DrSzfHHGaL5uqdv6uGoE0uh181AORIw7biIXcU6I+Xjr4FgOUnv6J5f8TNvO/BeWjkazplc3jx+ xXJebVVpCS4/1Ra1VGxqRAGYNojptKyRNbdaOGHDkAwNrXwfnRNgPwpZAFTFUGKkIg2spkSqN4X YTAJgTR/ILaShogHwJ68pUxUQucYCfb7/Qq6Ui8nE1z1HvgkLJOP9EJMfqpXLX7UETHpNcyBsij 7s42AlFnkBVlBW/R5788= X-Google-Smtp-Source: AGHT+IFhccZU7hwIpoqF9kdgyBZNm8CDnNWVmBsNPQQ6+B1QKTjLAh4Uz8aWpAuaS2cilFrr6VCiig== X-Received: by 2002:a05:622a:18a6:b0:477:5c21:2e1f with SMTP id d75a77b69052e-47e77a9e41emr774971cf.34.1745429337818; Wed, 23 Apr 2025 10:28:57 -0700 (PDT) Received: from ziepe.ca (hlfxns017vw-142-167-219-86.dhcp-dynamic.fibreop.ns.bellaliant.net. [142.167.219.86]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-47ae9c3b5c3sm71026671cf.21.2025.04.23.10.28.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Apr 2025 10:28:57 -0700 (PDT) Received: from jgg by wakko with local (Exim 4.97) (envelope-from ) id 1u7du4-00000007LYP-3LLk; Wed, 23 Apr 2025 14:28:56 -0300 Date: Wed, 23 Apr 2025 14:28:56 -0300 From: Jason Gunthorpe To: Leon Romanovsky Cc: Marek Szyprowski , Jens Axboe , Christoph Hellwig , Keith Busch , Leon Romanovsky , Jake Edge , Jonathan Corbet , Zhu Yanjun , Robin Murphy , Joerg Roedel , Will Deacon , Sagi Grimberg , Bjorn Helgaas , Logan Gunthorpe , Yishai Hadas , Shameer Kolothum , Kevin Tian , Alex Williamson , =?utf-8?B?SsOpcsO0bWU=?= Glisse , Andrew Morton , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, linux-rdma@vger.kernel.org, iommu@lists.linux.dev, linux-nvme@lists.infradead.org, linux-pci@vger.kernel.org, kvm@vger.kernel.org, linux-mm@kvack.org, Niklas Schnelle , Chuck Lever , Luis Chamberlain , Matthew Wilcox , Dan Williams , Kanchan Joshi , Chaitanya Kulkarni Subject: Re: [PATCH v9 11/24] mm/hmm: provide generic DMA managing logic Message-ID: <20250423172856.GM1213339@ziepe.ca> References: <3abc42885831f841dd5dfe78d7c4e56c620670ea.1745394536.git.leon@kernel.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3abc42885831f841dd5dfe78d7c4e56c620670ea.1745394536.git.leon@kernel.org> X-Stat-Signature: bmycxjnp8ugw79tgoimu7pu5qfw3j119 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: AE31E80012 X-Rspam-User: X-HE-Tag: 1745429338-973919 X-HE-Meta: U2FsdGVkX1/TFYB3R9aUM33/tb8T/IqLe8oJTKzJmgcCNB1p2UdlxU36AaaU9GFCF4V1XSKvI1REuL4YcRzp4CPIXmtsc31AAkCCdnovtQ7oEdrB4aVka76cMfaIe8UMNzO+BG24mBhUCVfXsyjEY9ObpXiiR7DZPWak3eS+r7q8T2xI6fg5IN1agbkKOOv568FhvvL5Bnu7x3nvvWl7S57DqX0j5+go186egLNTgiRYlAN+Bx4zRGmYZxUrvj3Z+2asBtUCpQ4ySy0sXoHmf9/T2nbzxPMLjk/1acLohZW2/Uf2OEJcmhHLJIHFFab9E10AbBz/rlDJdp29Hg5a/oYoHCBYgTFFl3nOJ1L2dy2yLZVHc1ygSfXOap3ck5waNcz/JvOBFkTZIO51kKnYMslSobHFf72kcZeCOO79vttlWmnKmuuN9O7u7IniXC+ZM69vAEoLhJ4Jgrrs0lB72i7GxZLNfDgYI3JSSnyBYPYEiq3m9bs66IrgtxaHXsSUScv2ewiVhCwnB656okEC3wrcPrNRmqbFEGLOcKVZ0lG8cMjr4RCQ2lj6xSjj/tN9d7V97x7KmvGadp4M6uGDIRHOzhy2yYrKWKGcRx7zTrTvuXN6rf+w3OF+T7iP5AFAPQjAJ923y9QiMGB6T+/EDh0ItEmrO41FYLZV21OfqtSLXmP931UblzTK92wlAcLi30ZOLoCSPuam7WSB+MzgKPz1fjAskhia5UVIkdHnwzusE4VN0mkbb7yvbEpfHEY016yBGrNj5jO6QJGJTZ5fH9IUnm83wBS2Cn8nlnpFX2PMrS7EJ9CjHlfSplYRGL0oraPwvZiYN43UJ/x5Df/wy1CkyJWxiIOexJU5YYKd2tOyPuq1oug2kRwAvAKR26Kc0VoxZmRnSc2r3/Njn6vG1PYOmC0649pwMDh86yLxMw2WAnfCU7URG8zs11wSfm+QU2KFX1dAkZWOVDzIEqO nFdeJU60 mhaqhzmvtkmTynqPeX/s/fOypLyh2ITiEZ+sig51VvHhgTVxSAa27we6gzcfwCmA4REa8yVoiHQqy47uk3rmOLYyo73+zkUhehfDEdlwAaYhlZuFgfaeg+A5nX0zYn0RETy4xJNxj23f1n7mUgYDNA/XAiO7Li9uh/0xpLX8rTdsY3Xiqhg7A6EtdaQVtB6Pk8bNahNI/+XA0WSR2tnP9hB091U7wWsNss92tDAdm2zkfSgn4rplfptQvlCkYZmkCA4t1fDytv1JCkYKCPlWKNDPTL4o9Dp6DbGR05v/Bn0jeIc6q2cAZmqubGA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Apr 23, 2025 at 11:13:02AM +0300, Leon Romanovsky wrote: > From: Leon Romanovsky > > HMM callers use PFN list to populate range while calling > to hmm_range_fault(), the conversion from PFN to DMA address > is done by the callers with help of another DMA list. However, > it is wasteful on any modern platform and by doing the right > logic, that DMA list can be avoided. > > Provide generic logic to manage these lists and gave an interface > to map/unmap PFNs to DMA addresses, without requiring from the callers > to be an experts in DMA core API. > > Tested-by: Jens Axboe I don't think Jens tested the RDMA and hmm parts :) > + /* > + * The HMM API violates our normal DMA buffer ownership rules and can't > + * transfer buffer ownership. The dma_addressing_limited() check is a > + * best approximation to ensure no swiotlb buffering happens. > + */ This is a bit unclear, HMM inherently can't do cache flushing or swiotlb bounce buffering because its entire purpose is to DMA directly and coherently to a mm_struct's page tables. There are no sensible points we could put the required flushing that wouldn't break the entire model. FWIW I view that fact that we now fail back to userspace in these cases instead of quietly malfunction to be a big improvement. > +bool hmm_dma_unmap_pfn(struct device *dev, struct hmm_dma_map *map, size_t idx) > +{ > + struct dma_iova_state *state = &map->state; > + dma_addr_t *dma_addrs = map->dma_list; > + unsigned long *pfns = map->pfn_list; > + unsigned long attrs = 0; > + > +#define HMM_PFN_VALID_DMA (HMM_PFN_VALID | HMM_PFN_DMA_MAPPED) > + if ((pfns[idx] & HMM_PFN_VALID_DMA) != HMM_PFN_VALID_DMA) > + return false; > +#undef HMM_PFN_VALID_DMA If a v10 comes I'd put this in a const function level variable: const unsigned int HMM_PFN_VALID_DMA = HMM_PFN_VALID | HMM_PFN_DMA_MAPPED; Reviewed-by: Jason Gunthorpe Jason