From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2EA0C27C76 for ; Sat, 21 Jan 2023 15:03:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E2E026B0072; Sat, 21 Jan 2023 10:03:20 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DDE486B0073; Sat, 21 Jan 2023 10:03:20 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C578A6B0074; Sat, 21 Jan 2023 10:03:20 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B24976B0072 for ; Sat, 21 Jan 2023 10:03:20 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 83387807D7 for ; Sat, 21 Jan 2023 15:03:20 +0000 (UTC) X-FDA: 80379124560.03.FF07EBC Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2061.outbound.protection.outlook.com [40.107.244.61]) by imf09.hostedemail.com (Postfix) with ESMTP id 0C3BD140022 for ; Sat, 21 Jan 2023 15:03:15 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=WlWaU14N; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf09.hostedemail.com: domain of jgg@nvidia.com designates 40.107.244.61 as permitted sender) smtp.mailfrom=jgg@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674313397; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=y5bkh5dkw3jlgawWlJroImGy5kmIR021Jist+9JrbGo=; b=UgbiO3bEULPhQj2APtYSNkiqY0qv3YJo+42DQdXp/sAYYn0Mc4dxJZD0pvjqZ3u0P4vX9T 7/YiOCRZRjzfLxqQobWaznfbnZ19IQTCHw0II26LgsrOfcw2q4KwVQoe1KlgSHAtBmX/BT U8jVHfU5NOWK9XqD0aFSD8ZhlmaA+wM= ARC-Authentication-Results: i=2; imf09.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=WlWaU14N; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf09.hostedemail.com: domain of jgg@nvidia.com designates 40.107.244.61 as permitted sender) smtp.mailfrom=jgg@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1674313397; a=rsa-sha256; cv=pass; b=w7tVIdvjK7km1345U6S4zJR2tWIO9eU87OJhC4TSJVgpkmsbY/IufQr4N4hqa/LJemi7/+ KGTs69zoZSdePAw/qiWWNAHtC/mHwrbH+MmoFCuwTU/eZqqyZbwy2qmQUNjklsCFZsJN6u dZYNJ1ybf4L1Qc6bvTh52IhBaWmk59I= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=da1GLpMN+46623RGkXrjz+yydCdg6wQYCSYysaqdxIXihRB69lZ0Lk1uO2+E/zqcSAhmVsFTZGegx3PWyP4OoTNVLLOm72qvWtOIirKnWecIZ71QK2uO2w2GOH9lLGkyPEzhg1tSPsoduaRKgWd+MDd2lhOftGH95AttK4fkCimS+1Dp7uRbtU62/czuVQxXxXuNNChFz7BAaD1viKTKBoIF/3fTyXzBeEMgJegYme9ENwimwpIp3Svk3sQaZK9MmlYAHq+Y0ZlqeC7w2ADZO192A14jMAy4m5e8G+RoNGFBQTq1VmQUWYHKDuLeZAwG9dXqvfo8zbpQ+kgshRp/5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=y5bkh5dkw3jlgawWlJroImGy5kmIR021Jist+9JrbGo=; b=ivFIrrVt6dbyrxwO/YG3jPrxWRrbG5p128Ft+aVBofK2X/z1/y7S2kCauDMog3DAGvJoIkwudJmrWlR7cIGrgd3sTw7ID8EZYj/3IBJduidc73TwOL3XHnVZxJSEKD2Vtn1S3/5AtBFsQIg9WN0jD3CQXxCCDJmwtYkURdpxPT0G9tbUkbDrxLiKNmH23pC4dVHphJnW3ol2HalDWH8p+ugAr5Is1I5DBUjlpms0KLwQiZqPot57kyff6McRjGCFMNGSmeVb2r1TuIVRGkDHJrt/kNej12VIffv/fOhXfsC1cdhmbmpAmewLbzvQxXYoa3TQ//QTmOTV/8H1hDQJfA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=y5bkh5dkw3jlgawWlJroImGy5kmIR021Jist+9JrbGo=; b=WlWaU14NPkou5u/h/G8mNCzCiNN23WWd0/cQxRfqIwGTK81QH635qtVBhSThWThhf/VZbxqgksIGLuzSBHIX7B8XTuoe2zvrarYi6FsklHtm/jW9fbSPyYZmloYnlW0q4gG/1DSKS78Q2uSgYFEkxDgAmuxmZSg8JIQHVLFOyXfW+xc2qSfa6fYgjvBB7lMGhY540XscJfnFN+x/UF3y8Ui5UuugNoOw0CHfmWIEnqYBpYL1xY0+edmlBxCwRKgxzhArCqS+kIUrfW12VaMX7rjmThoKlip+TLjgz1/JT06qlp2oZS6l1a1BwjIOaHDyMdgdLyj3sD7dni9VYRXF4w== Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by MW4PR12MB7167.namprd12.prod.outlook.com (2603:10b6:303:225::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6002.27; Sat, 21 Jan 2023 15:03:13 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::f8b0:df13:5f8d:12a]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::f8b0:df13:5f8d:12a%9]) with mapi id 15.20.6002.013; Sat, 21 Jan 2023 15:03:06 +0000 Date: Sat, 21 Jan 2023 11:03:05 -0400 From: Jason Gunthorpe To: lsf-pc@lists.linuxfoundation.org, linux-mm@kvack.org, iommu@lists.linux.dev, linux-rdma@vger.kernel.org Cc: Matthew Wilcox , Christoph Hellwig , Joao Martins , John Hubbard , Logan Gunthorpe , Ming Lei , linux-block@vger.kernel.org, netdev@vger.kernel.org, linux-mm@kvack.org, linux-rdma@vger.kernel.org, dri-devel@lists.freedesktop.org, nvdimm@lists.linux.dev Subject: [LSF/MM/BPF proposal]: Physr discussion Message-ID: Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-ClientProxiedBy: MN2PR02CA0027.namprd02.prod.outlook.com (2603:10b6:208:fc::40) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|MW4PR12MB7167:EE_ X-MS-Office365-Filtering-Correlation-Id: d602da7c-5402-4b94-bfa4-08dafbc09a19 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: CRMnbebuCwY2XUAUG4zSwIe1enDOU/bfPiLcADCdQWxpsMCkdoJxHYf+cjIoUZr0NB9zAFxBWbxtCvRqATlQWLYjPoQbGR8HtbMM3sCogJQZGMHYmL6H8sT8yeRg/U4YDYLGC43S75B5xNvJvace83NECE8PxAVPWQTh66/lCHMId+12NsKq6ajdT6rZVUPnh0dAhynnwoyHpI8WlD9qNjK9z9vDSJz2t1cLg4zeTdAFmzzEhw3GjPPvBY9/hZThbMe//HOTRrpyBpJnewnFpcN1/4iwos+01yYdpe348F/aDtA0x1xNlUAtoR5LJt1/9xgC3tYUH+HK3ll9/BCI8b4KbG47SVArAXz4bQrj9/hMW4weqCmZMxoz6FtIFXtY9VOqoFiEU4/4zN1zLyo7WYCWr+dyduK4bW3EK8a2dR5SXWckscDqA1/r83Zb5RlY2yfUDTmzsPNYjFTeqABjjZqqUKeAWHgHpbWdBhO9u+1u/DpF26/zzKGpzUOQbT42SNynbGmzgp10wEvDuoOex5gc6+4ZBx7k5zASKCgX05U5vmq1huJm22wAMdZwJ5Dm2eSegKXO7P/ishk6d2LScQLI0X8jSEiV57uO2KqFRYECJcOj20aKI3PptY2nldJYVyh/pHQFWjeVm8f2C98qx3GyHxMlkKjiD0TOmEejlk6f/U12cIzVM1mo7u4AWftwlhdEaPVG+jHkNTqsSc2CeT69J1112T4exZLRVgIfpYMxlAwJEQKbbJTEmq/V7MnJZ3jvlTEfGNnscM9llCIzm5dWWJjLx0L9dLXkAN5TTKUw2hcPyVvJrhyire921RQO X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230022)(4636009)(39860400002)(366004)(376002)(136003)(396003)(346002)(451199015)(2616005)(36756003)(6512007)(6486002)(966005)(478600001)(26005)(186003)(5660300002)(7416002)(66476007)(6506007)(41300700001)(2906002)(8936002)(54906003)(4326008)(316002)(66946007)(8676002)(66556008)(86362001)(38100700002)(67856001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?F3BtidbfOpqNrP0AWXf8iW9XqvsJOhqseIZ2Oll+rGUOB/hTWmy3UCsqFod7?= =?us-ascii?Q?EHZJ48jGqvFDAuVpNk3DepjhbbhpO1E0BMOBmAfJ4rKCqKGwS2KpsrM9LWFR?= =?us-ascii?Q?7eJxJthEf/0oBMBDcvB3Jt8cnGJ7A586GXqzmVnJNd6DzfGCgB1d79l6RcS8?= =?us-ascii?Q?hCRwH6IaEUT2KnG8Ds7MBt0UEpJ2jYR7MtjV33rMMXx2AmfyYIvKsWDfzRUe?= =?us-ascii?Q?ZpHG62etWXCkt+DvDL7X4tDN/YG9gxM/xej6jWOQeXAX78AfXpKo0tSw9i1V?= =?us-ascii?Q?bflChshYSj51Y69JQBNf2y2XYY+VSBRPSrQFfwgWBPi2St2owQOjbTDYVfFv?= =?us-ascii?Q?Z8RH+DpQoG9IpJ7RQ0Ra1Xs2NWuODxG0RknEASsZahiaRgzmLJVqIBVRNBOg?= =?us-ascii?Q?eYULA7UsBkIaZiM/L7HahjyEL5fTXiRkqbA5y+dWwnFW/YvIauyIBc0pZqU4?= =?us-ascii?Q?5LoKOMezuI6aVGejSzTfIN/vZG4lg4w082yiC53xU4Jq5XG0ka9OkpE94Uc2?= =?us-ascii?Q?e9Dcte51ghX0eXrxLMrlcyOX/SkFHxAaxIV8srbzj56mA8tM0zCACEBFoy2G?= =?us-ascii?Q?LSWLWCPj4B+KdOquV84fsUN3l9UpNXFAjuqApmyTofj1d6PTLqia+phcgrOx?= =?us-ascii?Q?2DyS8lXJEoykDS4JCNH42JwSfCnOOALsNjcFOk1PY6J8nPnLzH7Ct13SDcI3?= =?us-ascii?Q?/j5DvU+OlUxeZYitNQN9HH9TWwzDKx0HFfIykfSAw25WAq/PnnF3sqJDJnaz?= =?us-ascii?Q?osvsWOEJ2zQcS6bb4dDIsNPBRnENan6Zp/XwTNOH0mnPC6GiWlMatL8TtmBZ?= =?us-ascii?Q?qVw3R+jjsbumSmMWEzkT0q1fKGwaV+5GQbVLF/yCQHhFAU3m/wKRwwk3cWtp?= =?us-ascii?Q?CQ1QmGSKvKwkYF0ANWtCGKsU1xr3TuAyQFpHPBBrJwVMsTD+CXmyXZFBIRUN?= =?us-ascii?Q?gc36PyLroat6wu2kmYVAvkRu484uJOXLT5uGjWXaNmNfFia1RSzHcyD0oXKT?= =?us-ascii?Q?FzzS0D0bMDG4Zo/DaWXDx1tixhc2iz53HaKOVZuRh+BE82yGW05HhJ2BGONG?= =?us-ascii?Q?6VEQKTSz6akReE6feMADA0IxXewPkJZncMebPBJWvQpnMpgzdaYnZHeNyxPQ?= =?us-ascii?Q?gjDqWDs1xtnqJW6m8IXXQOizVpC6e1EthJul59fkqSAfP3pb5nrIpGjAMqkk?= =?us-ascii?Q?9G3rw7HbWISHlhPUVGFWBY9EskHVQdlQavSiyhtU2IBTaU+AcRU3QALi6x9H?= =?us-ascii?Q?RgCwVdgoqmKN1w4Y3VoRSp8wWoutCeskSGpDtrLPA/fJhR46wJRYgB6zpriX?= =?us-ascii?Q?ue5d/zzAGN4yGE3RaztmUfL8jYn8niqrz/EbYRz/fF6pGHr/OpQkQ6VrjYz5?= =?us-ascii?Q?q2j506TjFvq0PWNP4GhYaBTcqxIu1EIc48oKTruujAlv2nEzEtyYvhchKkmt?= =?us-ascii?Q?RV1IC9yWhf6cThBjmr+gQLmSj25bh9yXPeOLST+fvT7lYe4lW3xsrZn/Sn1S?= =?us-ascii?Q?UU8FwZw8HaCkkoKaXuZ3BM8/Da4FHDmanqT2JIxQWrbewVODe1pj5aV6pYfJ?= =?us-ascii?Q?jsHWLEC2J8rISIaIWyRmJPK3PbxUJC1AIS4lXOBs?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: d602da7c-5402-4b94-bfa4-08dafbc09a19 X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jan 2023 15:03:06.5736 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 7FLLeQpTT5r0fWJNmbBRqDe+jOyxVxW50i+u69QNjWEcPeRiCqI2DsfTswcC9pgN X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB7167 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 0C3BD140022 X-Stat-Signature: 1jx9r5mwmexbxtz9795k4tesarhugxnw X-Rspam-User: X-HE-Tag: 1674313395-144454 X-HE-Meta: U2FsdGVkX1+I30VlCihfdtCpIenTA/AWZFyemvG+mDhvsEs5Vp9gRCLq82M0Q7hHtbtjau0YIKq1hdKfkXRHdAqRKFkV9MtamMYVKHjY8DPaCvJ+p/wUJpHpATr6CcLJ9jaVbdgLhh097rDr/6k5hLqkcrke13Q9j0+Yfy/7+Z8P2u0yuQDXOyJREHlQf0gTrIhZzyLe8JC7CVJyyDP2tm5sw86nahwWI6UlECMEgkBOR2Lqm5fbX6BWQbqRlP/sQneiZzbc5W1pjzAmPCSFhCv/F1NLN850PMpi2XdJcptg5U19zqhNHCKlDkrByjrFceBL16CVmdr8lMwAop0nb8+IvqYybluYfJcW/xs+EkMVnSDPcHshoMQBtUKGmQMkRC4LX/5IaMFYGVhIZNYAyD7+96dIHNpFABSW7VFiJKDnMMuNRghH6mL/UXIHotS2436QF5XctfjmnuT/m4hCV3Ub4JNmF4D7UucHLrgvMASy6m56ii8WLwiipc8rEafD5IfkP2LKE9wqeE76/Ijrjr+09gnnUqiiX+Qo4xfDL5cvmHSyLSy1YhWlb1QzdM3DCf6nhZgyXcW1T7MWie9FNozJaQIiH7tsEzxHQ5ry5wJExBCdttuKhiNlhm/tzk47skJ8QXwJXfnZiWYcC9icvxmb1+n+IqRh+tqn0UQQy2CQzgVt+aU2QcxH3HSusvDfrfO3zQt17flSmjJCS5yjyz1rHGIp55br3SP+3smna0a2gH4FmBzHURdsM7Q3eRAeA4eq55k2pWTqEeRQr25es3YNvbLvPKpgbfOcybwlcHmMDJHi3CQ/JqYTTdWYv/qIDmh2Nv2msoz2I4+ms2PxIB4NrXZQI6pVjms56gFM+R8sh23o34x1b9K92LPVu0rWgRFiTaofuWML0YEbSBhAg26+1qPVVGNVZoGQlFOeNwuISCP4wWetWuCqiBIMtYDgqR8Y+c5XbZ4LftyU19r PuoF3fjV Cp9cGStvaNMMVE4YmRuPcNPZCyrQqij/iO4cpy5Zdinkmcy5Sbu+JygUFN4dtuAMhvE4gaI+rUSLbqAZL639YY62UbWWwg1z8u8Xf+cROZaAO2K7ePPog8VTqB2Tf08wqfcvtTm83gtto+DDx0D/X7CEuQJ2myB7R8gzQC0CxLypiuG4EDG64gINWXsmKEJz8sv+jED2//LwZGYOnp1G2WKaZnDS7IrMLRR+lNuW2BS+0t+bGcI1rGz+5jcTiufw9u5ymKUBa4m3/KgKpxl4AntO6YJ0+wHarMJv1EF857aN9Ioop1OTs3TMTQlBmZF3U7Nh2lUeG7oCZE+u6437UcvJZ9ulh2pGXa5qEP5I6r9AGHuPm3s7TfiXufA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: I would like to have a session at LSF to talk about Matthew's physr discussion starter: https://lore.kernel.org/linux-mm/YdyKWeU0HTv8m7wD@casper.infradead.org/ I have become interested in this with some immediacy because of IOMMUFD and this other discussion with Christoph: https://lore.kernel.org/kvm/4-v2-472615b3877e+28f7-vfio_dma_buf_jgg@nvidia.com/ Which results in, more or less, we have no way to do P2P DMA operations without struct page - and from the RDMA side solving this well at the DMA API means advancing at least some part of the physr idea. So - my objective is to enable to DMA API to "DMA map" something that is not a scatterlist, may or may not contain struct pages, but can still contain P2P DMA data. From there I would move RDMA MR's to use this new API, modify DMABUF to export it, complete the above VFIO series, and finally, use all of this to add back P2P support to VFIO when working with IOMMUFD by allowing IOMMUFD to obtain a safe reference to the VFIO memory using DMABUF. From there we'd want to see pin_user_pages optimized, and that also will need some discussion how best to structure it. I also have several ideas on how something like physr can optimize the iommu driver ops when working with dma-iommu.c and IOMMUFD. I've been working on an implementation and hope to have something draft to show on the lists in a few weeks. It is pretty clear there are several interesting decisions to make that I think will benefit from a live discussion. Providing a kernel-wide alternative to scatterlist is something that has general interest across all the driver subsystems. I've started to view the general problem rather like xarray where the main focus is to create the appropriate abstraction and then go about transforming users to take advatange of the cleaner abstraction. scatterlist suffers here because it has an incredibly leaky API, a huge number of (often sketchy driver) users, and has historically been very difficult to improve. The session would quickly go over the current state of whatever the mailing list discussion evolves into and an open discussion around the different ideas. Thanks, Jason