From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B8ECC64EC7 for ; Tue, 28 Feb 2023 20:59:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DF2206B0078; Tue, 28 Feb 2023 15:59:56 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DA27A6B007B; Tue, 28 Feb 2023 15:59:56 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C1BD66B007D; Tue, 28 Feb 2023 15:59:56 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id AED7E6B0078 for ; Tue, 28 Feb 2023 15:59:56 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 7272C160B57 for ; Tue, 28 Feb 2023 20:59:56 +0000 (UTC) X-FDA: 80517917592.18.1900266 Received: from mail-yw1-f179.google.com (mail-yw1-f179.google.com [209.85.128.179]) by imf07.hostedemail.com (Postfix) with ESMTP id B17D840022 for ; Tue, 28 Feb 2023 20:59:53 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=fF7yR1zi; spf=pass (imf07.hostedemail.com: domain of tjmercier@google.com designates 209.85.128.179 as permitted sender) smtp.mailfrom=tjmercier@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1677617993; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=H15Wgrg45uphokJUqJE+ge3DLPSULLnvrd4PVO7cLLc=; b=UKcZimonVlgGH93rq024cuEJ04NQyYT5JSi4fOCZfKk3r/CSLo06VQhDYR8gOGWR11AWj2 BLeteS/CQtRWKcSkvZbgUeKqXlK+bdm7mPjQB02uJV0/ha7I1zLxR03baxyFa1gjo3Sh9V Z+eIB5QpycVJJU15fzyuiADYgaxXpQo= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=fF7yR1zi; spf=pass (imf07.hostedemail.com: domain of tjmercier@google.com designates 209.85.128.179 as permitted sender) smtp.mailfrom=tjmercier@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1677617993; a=rsa-sha256; cv=none; b=65uKBB/4pFI5ZhQpwtES+y/vFGJP0jBRS3Qnj9nYYbEAhhxIjEWQMgDlpuhG4V/3Xm2NdM S03po7TeS/W6se64atHSGSRjxiAxXE6iT9bnhXGftPE7CCgWFcOj2YQNGKQDrofkWPQLMy uCjGjgrCG9GeebqA5DM2OC4rlD45nHA= Received: by mail-yw1-f179.google.com with SMTP id 00721157ae682-536bbe5f888so309521067b3.8 for ; Tue, 28 Feb 2023 12:59:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=H15Wgrg45uphokJUqJE+ge3DLPSULLnvrd4PVO7cLLc=; b=fF7yR1zieU4gkviG5qrWrdfSWhtKPGGWlvtMBXv4KHBDu0aJ3gWsucs0SnY/VXu9xI 98ilQKazhvCJdc/WBewUB+5SWcdhorii7J1pOeAzos6f/dQqHZdg02M4AtOUfeTq8eOO f09htTPMG/pkdx7lHQGqeX5kSZIxN5OkkPfc77FKJnyPHYn11vG35NdgnRPHPkAFKcea XgImIWOsGyc18s8CqlKSzyo64N6r/4hd/TI5CyRtFIgPb+s+gMlpDT8KVNO9m7c21yRW SoYMoaCOLdsdFi4o18FJKo606Yo0hMRNZOz6SykPjzCGVDwzVDiSukBxaNiAwswCzVE4 k7xA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=H15Wgrg45uphokJUqJE+ge3DLPSULLnvrd4PVO7cLLc=; b=28AV1K1LhQH1aLqF2X2rnTAqELBXeqcKPpg3Nas664mxITEqJsZZuGxxjRdGF8RNB6 oNjnpA14I/g58BnhJboS75VWcCRLXfWxkj/sWJKuHa7FTN7teAArXv+e0kY2DWB9sWRi A2WLA5jMDO0aXUmMqAlnSLwGPXyYyDRpb1cLDrWaXmDDgT5b1ys5zIkcaJrkPAtSPNKJ JIN92LScLzkdtgQ2szalaTLzcguKAEp8P0yu+ue8tTZT8XYQ+B0u4XJ+jkW1tsCiUvvq 4+HhYNi6K7g+3aPglRg6ZGjAPAs2kyj+8YrVXxk+kGMmD9uyGuX7YWdPH3tvLxJML9bv LvbQ== X-Gm-Message-State: AO0yUKWqZiKjAR9C/R8xivC8LCKcB5WZTDMzXlk496Dc1lV/lwix7SQW 6EWjrgoL7lVzLZKHY5euS5A97KimPK4RUFr5ZDHpUA== X-Google-Smtp-Source: AK7set+d6vs4ymgWzsVBho7zIKcb2AoIn1quNMhQ7GG855RnJ+2d+gNas87Vdn9aekeC5XgvBudues+ObsH6EIGw7ZY= X-Received: by 2002:a5b:8b:0:b0:90d:af77:9ca6 with SMTP id b11-20020a5b008b000000b0090daf779ca6mr2178238ybp.7.1677617992660; Tue, 28 Feb 2023 12:59:52 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: "T.J. Mercier" Date: Tue, 28 Feb 2023 12:59:41 -0800 Message-ID: Subject: Re: [LSF/MM/BPF proposal]: Physr discussion To: Jason Gunthorpe Cc: lsf-pc@lists.linuxfoundation.org, linux-mm@kvack.org, iommu@lists.linux.dev, linux-rdma@vger.kernel.org, Matthew Wilcox , Christoph Hellwig , Joao Martins , John Hubbard , Logan Gunthorpe , Ming Lei , linux-block@vger.kernel.org, netdev@vger.kernel.org, dri-devel@lists.freedesktop.org, nvdimm@lists.linux.dev Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: B17D840022 X-Rspam-User: X-Stat-Signature: c8o1weoipejfb6cn9aqt1ucxto3d5n47 X-HE-Tag: 1677617993-810304 X-HE-Meta: U2FsdGVkX18zAgsTBH/K/1dFeDepStimg8Ep8uzWcWIGzeXoWj3iRr3iNdRKiWY5JKxzKdyq1VvDRbnkyp20vLZIF7e8kCRKUQb4IDRMnDzpt0kVoDQclyYd5fmyuNRNLFFQ2twi7sbCmkEQY+OQ9MjY5L+Ki7A9hbxVLULa4S0023ZSdPTdWmhEBcP3cOKVyA0qcpaCyL+5DElum947i6CWJ0q1/EDikaiBqbdHqMHdKbmy+ENfrgcWpEfzsBkD1Rv++xZRd2CbzikQiRJCRrck+jO3BF6JEv2fFvlsnYNrD5Ny3lSNpX094JCFVb99nMZdFQ4P0f2n7pCgRb+N2ZZTSmShWQY8V01d1P6xOxNiurjzQM6NgJnuUR9NABPXvkRIJ8yMKAN8joDjYxP6mm391lut1W0PAj1SABNcsEOZAxlDHXQf5Ct/aXesiu26HHSzktf4Q4Q3yGWB+WVgREMrhGwTeGQqzh2HpeirZl/RuMbsNlwiQV33XMag13C3Y428D+rGAVvc9ZWpoUDFUlZ82Cp1GiwKaxo3Q/buPWvi9u+9JVVEpJXIB7i71qOfq5aKlxy42IDvDuAlKQcj7Wn/YeKPo8ZGSce3/oe7MKB1ph46sV47amB7q1m+9+CZE6Ss8ZTqD19YBVglkiYUJuzd/ZtwnmmRKZ9S/vTkEH4RFVIQ9aUrEmnItyNTNTTBlJzW6X8yVfu7IyYjk/t2RBkD13LNeKsLzaMvjqRF8bKDya8/ne1Qy22zrMizLHI1uR05o0ew3g5wYFrlFrY3ywPxUCVRyie3yVl0wIrkJtqR5EsEitfnVarmkvs0UKbRJYfCKRP6Dl4JGQr0EFXzoga6NSwfXComCHWuCfreswT9G9tqf4hbwJwOQzEpaTMU8RGZdbzLcyIz7IzTf3dn6rcM/w3vDkymZXW9/4GjKdL9AdcnlhA4FYcm6waGzxwBxCQUKYmoStJxk2AnLTF CkpFmZug jIld51nixILgnimvJfvETVtzCGq66Y5ta2JXo5XmBQmQfjd1P3UyPN9VxWPdyUBSHj7PkXo0VVMN/LSIgbB6PYN7zAPSTpmzoxIiglvczFbAbICaU9EkkI5gLlzEqz4cnGzcKoWsupLAlfI+yz8wOBOhqhbnyR3rxepKYRlWCbBt7JPl9b2oWYbvjcpy8f4r+ZoHRXFSTKgywYobI+qNHaPRu2QEP9Y35sEFXVOVe4BTNuSOYbUg/aQElI1zGIkwx+jiZybP4E2mNw6qZ5/A6JYwoPkDDvaAbMdKARkvut+UGK0aHpjypWPnrHU3Atg7itqsJjMbR5Wb2D1R+y5ixgei9XBXO4o0TOYS6c0MvEQ+gxXSlUXFSVF4/VopOz0j0fS/B X-Bogosity: Ham, tests=bogofilter, spamicity=0.000004, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sat, Jan 21, 2023 at 7:03 AM Jason Gunthorpe wrote: > > I would like to have a session at LSF to talk about Matthew's > physr discussion starter: > > https://lore.kernel.org/linux-mm/YdyKWeU0HTv8m7wD@casper.infradead.org/ > > I have become interested in this with some immediacy because of > IOMMUFD and this other discussion with Christoph: > > https://lore.kernel.org/kvm/4-v2-472615b3877e+28f7-vfio_dma_buf_jgg@nvidia.com/ > > Which results in, more or less, we have no way to do P2P DMA > operations without struct page - and from the RDMA side solving this > well at the DMA API means advancing at least some part of the physr > idea. > > So - my objective is to enable to DMA API to "DMA map" something that > is not a scatterlist, may or may not contain struct pages, but can > still contain P2P DMA data. From there I would move RDMA MR's to use > this new API, modify DMABUF to export it, complete the above VFIO > series, and finally, use all of this to add back P2P support to VFIO > when working with IOMMUFD by allowing IOMMUFD to obtain a safe > reference to the VFIO memory using DMABUF. From there we'd want to see > pin_user_pages optimized, and that also will need some discussion how > best to structure it. > > I also have several ideas on how something like physr can optimize the > iommu driver ops when working with dma-iommu.c and IOMMUFD. > > I've been working on an implementation and hope to have something > draft to show on the lists in a few weeks. It is pretty clear there > are several interesting decisions to make that I think will benefit > from a live discussion. > > Providing a kernel-wide alternative to scatterlist is something that > has general interest across all the driver subsystems. I've started to > view the general problem rather like xarray where the main focus is to > create the appropriate abstraction and then go about transforming > users to take advatange of the cleaner abstraction. scatterlist > suffers here because it has an incredibly leaky API, a huge number of > (often sketchy driver) users, and has historically been very difficult > to improve. > > The session would quickly go over the current state of whatever the > mailing list discussion evolves into and an open discussion around the > different ideas. > > Thanks, > Jason > Hi, I'm interested in participating in this discussion!