From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2B96C001DF for ; Tue, 25 Jul 2023 12:37:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C929A8D0001; Tue, 25 Jul 2023 08:37:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C42DE6B0074; Tue, 25 Jul 2023 08:37:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ABBE68D0001; Tue, 25 Jul 2023 08:37:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 9BE066B0071 for ; Tue, 25 Jul 2023 08:37:07 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 5E59FA0D9B for ; Tue, 25 Jul 2023 12:37:07 +0000 (UTC) X-FDA: 81050084094.20.6C3D02C Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2059.outbound.protection.outlook.com [40.107.244.59]) by imf13.hostedemail.com (Postfix) with ESMTP id 73CFC20016 for ; Tue, 25 Jul 2023 12:37:04 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=Tf08WFIV; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf13.hostedemail.com: domain of jgg@nvidia.com designates 40.107.244.59 as permitted sender) smtp.mailfrom=jgg@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690288624; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=FypKwbVjkl2qcC9UTGF5pGp+iU2awiJpkdqg2q+b5OE=; b=N87S8jcu/zxOhN2wyaOJbRTLcHSOkh+cSXAyKzriqXD4o4se+QsL07Aljg92byO9aAYZMp MTJhSJtOEtY4TXR480FWBlmjRvh+9d2GAASnn2YmW8Tm2Ic8Ceh3LBAlRbKutTL2uj5nRw 9MpQWUR2+MUNpZjgtOHrRz2ra6ZEcZc= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1690288624; a=rsa-sha256; cv=pass; b=s8XRF3TmDFhdaaQ5StjcpVbRnrnCzACMXAeLBWskYbSzpQX20SWGCKHbC8J9CjzL7hvhpK +49/5n5xm3dmzTB6VAl8a4MZvJrvVUEjdEvU2yPil6AC7kMe7aXDTaOIFnhNaZWeK2YfdK dSpZhx8ScCRfOJlcYUZl6fXgahtMJLE= ARC-Authentication-Results: i=2; imf13.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=Tf08WFIV; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf13.hostedemail.com: domain of jgg@nvidia.com designates 40.107.244.59 as permitted sender) smtp.mailfrom=jgg@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WH+Yr9r46ABxtLcwW2GX51Bz1tEJOQAFsEpl1scgxfA4qMi9AK4N0lTw0Gnc1y15gDqsH1SVmYI5uUuPMp3a2vUlHlyaXRX3A1WJ5Wk4urZ5J0TuXVYtsOw+0YPErIfVpfiwT4NM9BEjYhQ99jX5sL39uwjv65yQSZ4nnJA77Cb5YdtTYErXz3itTTH4mZpnT02YwGYYXF2Yt7+LoQZmq5HcClThzHjqp+xtYXJcxKlXtiVyERwBSZshbRx/cMczxRok2TeY0pWq3OJh/BFffsyFE/e4h0Q6tQ5lKQcijNztTiBWgVnhFEEwXZQkJo7vtQumU5xNyiy+tyVI8OeXUQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=FypKwbVjkl2qcC9UTGF5pGp+iU2awiJpkdqg2q+b5OE=; b=n2zFhFr/5xjMKmhejUGV0qd9pSf/qaV9C/C2/qP9zKJw9YGq0yHuD/cdvspq/O4lqbAXo0wKqS51Sj7XkVIxHnIFOdhJwhOgyh1Vssok8EjV93jvH2EwATrcAk4XsiasAsdg95j2Z5fcn6+dO/8vU8DoCmvRdlRFYJBe8z29aKlZjkvQ4Pwj8fWPsaTlwn6EJftt9wBTpLxtEqfmCbFKBkoR6KzrbBlYwCY3UP7prbpZ+hZAPDeGtxKeZ64a6oz/cJJ38JmXWY5GZi5R2OjhpEMDC5xcBs+xzUFBgE8NGJfKGOX8eaX7ZmZ1SG7H1BO3Da/CLTTrS3V0aGR3zMPREw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=FypKwbVjkl2qcC9UTGF5pGp+iU2awiJpkdqg2q+b5OE=; b=Tf08WFIVnHk9KS5fF91ZalKYQ2iAR/OMAa+LecPzJM53V9Q8VvCQ7lC47T0KVF7tHwvzxtJvtHlFe/fA78RXKKrqOwQkTidOYTRBMWgcNfioGD+ZQ86hFk7RGGoXC00/JQ7IKepOWRiRSZqIdz7Mc50uwrQ5eWySjHJPin2nPLJxmPurONAok4TJBMpUl5ZamDAVgZ6JVnFksaESJ6cPW96hZX+0a/5x0h49011wmBROHO3PMPyXf+Vybj9FdoI8FBOvo/486FSEE9mjesVFcyhLd5O6Y1nSvoRYSm8vO9q7QZ20dU1VA7XsIwQKUXYjQLsrCQzFdc04Pjzna7PHIg== Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by PH8PR12MB6794.namprd12.prod.outlook.com (2603:10b6:510:1c5::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6609.33; Tue, 25 Jul 2023 12:37:01 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::5111:16e8:5afe:1da1]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::5111:16e8:5afe:1da1%6]) with mapi id 15.20.6609.032; Tue, 25 Jul 2023 12:37:01 +0000 Date: Tue, 25 Jul 2023 09:36:59 -0300 From: Jason Gunthorpe To: "Kasireddy, Vivek" Cc: Alistair Popple , Gerd Hoffmann , "Kim, Dongwon" , David Hildenbrand , "Chang, Junxiao" , Hugh Dickins , Peter Xu , "linux-mm@kvack.org" , "dri-devel@lists.freedesktop.org" , Mike Kravetz Subject: Re: [RFC v1 1/3] mm/mmu_notifier: Add a new notifier for mapping updates (new pages) Message-ID: References: <20230718082858.1570809-1-vivek.kasireddy@intel.com> <20230718082858.1570809-2-vivek.kasireddy@intel.com> <87jzuwlkae.fsf@nvdebian.thelocal> <87pm4nj6s5.fsf@nvdebian.thelocal> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: MN2PR05CA0042.namprd05.prod.outlook.com (2603:10b6:208:236::11) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|PH8PR12MB6794:EE_ X-MS-Office365-Filtering-Correlation-Id: c5036177-9159-41c1-4a7d-08db8d0bd805 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: PX5eO0TKF2Q02MG5td/uG44alTo8BczDKXPV6AlhEiKsNe4CVjYSitHXS/+zz9Qm9clXs9OOdY6bXqXSGgptQ2HKZrCcGMCSPpp6NkkcVyylmUxWTLXGtYeZGSgaWPO2GCQr4Qu3XTAb1Ul38eW5zZDIcp13jpvb6P5BaB3Dxx81rvgrs1Cz59+1jWWtUQEBwfDBlW7UQgPdmcruS4b/Kv0G5tVQqDjHNuaj4oPQwoOnYLBvQPxlpeIe2uzsfz3zoD/sNs3clLN3YIt9h0wTt+JTCx5clbwi/Qe/BzYH6znxOGRYzoJLKqeOSdHX+HVm+G4gBgX4pzFeUbFF8B6Fy46kyVoX/yQQG4u6gKsX27dsX2eJqYv6ng/o7pGGr7uqA9WqcD6MerRaaOpXMMvvKY86gIaJnnz7CNzRYeFTW332AsffUnEf2RDtCC5gAmKk235719Mnp49z69dtCeRGH4IlWSkSF18zKQguGkBTFZCeWntKO/OoawXf6am+rPF14bQ5eeuTAnEO8YavHxkwz0LdPZ2hT9jBgC6YvXIi/HansdtsnDV+3ZrmUgMwbzAN X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(4636009)(366004)(396003)(136003)(346002)(376002)(39860400002)(451199021)(6512007)(2616005)(26005)(6506007)(186003)(41300700001)(66476007)(316002)(6486002)(36756003)(7416002)(5660300002)(38100700002)(15650500001)(4326008)(66556008)(6916009)(66946007)(2906002)(8936002)(8676002)(83380400001)(86362001)(54906003)(478600001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?/82R9mLtyRwlzBTvEI7I1flVmALUHIMBYG7dRnq51EipRnffopkVBnXWeYgK?= =?us-ascii?Q?fRdKuEAisexPMY5K3ZdrTaC/1cHyAvnfXW45jf/vrjVSRwXSfBynSCxpRld4?= =?us-ascii?Q?e/+4yK9jrH5vtmqO+klsTdV+uzkoVxf1WYK8UNN6HmwEbI4hay/Zoly3ByXb?= =?us-ascii?Q?XDPg/t0XkKgyeg2qHJvMm1titM0hjnkBPl9e4f20i4FigBj0BmLQ43IYPivi?= =?us-ascii?Q?PPU1ANH36X2ESrLg+Pb/kgoLe90WeEnrOWF8ACGHNKsT+1Y+vmjSu4x+9K8T?= =?us-ascii?Q?qSK5AnXr2ila8rtU4Drd+V2Fehy/vtpbUi9QcuhrDsppLY3+CtZjeTxlLBG1?= =?us-ascii?Q?yQC5PoUEWUDaJwS9LWVZTEJs1PXoLdB6+KNbqQU0WK8C7UwOlEVD82YYdIg6?= =?us-ascii?Q?Mg2W4Fe71DAavWJ+gHuOPg3rczb3hQhLaWCoWipj32+7qrkQujXMR8167mS5?= =?us-ascii?Q?tKnbsKZRUOdhfeUFL3SPmOJn1iFj4PWkCaOjAkXY37tIVNXxcNQAfzvkoVX9?= =?us-ascii?Q?+7JLG9QNRjsFhO4hQpZGj4kMtR0+TsFrNReH5/c/d2iZSWKwxmf9oLG2ckXS?= =?us-ascii?Q?xeIeiGQxvYZR2D2lqX+o2SJ6SJU1hUTBSPluZzKzAAjc8VIE6jtUR9E/cKVb?= =?us-ascii?Q?PI6EwLcuhoNlCSX5QPHbdd+SI0lYSCX8yDvmsHaqwPvRUULBiiukn8BrtLXf?= =?us-ascii?Q?bx6WPC5/MRy8BOEZKGRHG3qtU0C3BCEq4vZlYUHUgJVnt24a7hfK2lvm0kHC?= =?us-ascii?Q?FHg784bNIXYirBaiRxQaEGRTkmITmurrlPAQkccOkMuhUVRq0bLqX2UsK2/r?= =?us-ascii?Q?T9vqHQOU6VPmFVmFxljJudkLtr7Xq22MXz56AeAh/WJvzcTa/jMfaPRqRNVV?= =?us-ascii?Q?bybmY5toD0QiA9Gv4Flj+/TdO0E6w+PYl14HRKhIMfzPFmTDQHEl1IvNsmgO?= =?us-ascii?Q?McXs+QAOBc3cnFlcBVTvrhun0/ghkGs5KPYAJt/zXpTHIVFF9L6JMnnWsGVg?= =?us-ascii?Q?3nHx/Zpbv3RrB8+Pz5mEgqZvWPdRkecZmyopUsJft4g95JE9osg3tVIhcuEH?= =?us-ascii?Q?lO0y0El8oj0R8dk9IgJa0sWHftkFl8UANQNpqhcJw23AGJP7dHQnlhIBZJcv?= =?us-ascii?Q?VDgkXpFi70tdudztwKACSUOnh4SgphrMOujAMpLLANokLGr60fTH9aaENr0e?= =?us-ascii?Q?Ub7PMvqRkGuvAiZc+BRabzMsg4p5zv/4UsssNnQdvymRML4UM2aKVxDZM2Yy?= =?us-ascii?Q?bZg2JuY5Ald/FORE1JO0lD8Sj+S/+3zgFyDGPXc92bm7GtmutRx/FXwArX/A?= =?us-ascii?Q?7s96iVaKZsHj+C4Inqq64twm+9GS1KlLduL38hAJ1eY04xt6kPHVXnnGxtPm?= =?us-ascii?Q?inIs9A4JRjzaHDg5ZT5mOkH/TbPBML8KY8zW50VXpeN08XdBKhUZRmZAldVg?= =?us-ascii?Q?45ukvOWUvIBdjlFbfTGnR9Yo7x8JrLOMjc3/Ie4/LaZAPAGxZkCOAl2DsoqQ?= =?us-ascii?Q?uzTMFcw/yNPVsyJhCLB4qmqX39j3owMQ/0At8xVCkAxq2ArPlTPLtKHyfCvO?= =?us-ascii?Q?Olr05e9oCWBjOltPQBwuQwdpOrezPIELY5iU/OXQ?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: c5036177-9159-41c1-4a7d-08db8d0bd805 X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Jul 2023 12:37:01.3909 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: hAyCTJ16dKZiv7wq6bl+nUTT056rnEv5oOZ1lxDu7A6CFKYfw4VxDg1FR9ifrxnc X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR12MB6794 X-Stat-Signature: rnd97hrubcqj6593beeqwwaw1pxtncse X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 73CFC20016 X-Rspam-User: X-HE-Tag: 1690288624-827132 X-HE-Meta: U2FsdGVkX18uN/qqfGyWUZhk9hCjMZa/po37JVVexEBNd1TpXiI6pHm79qTYTWLvXXopFjXKU+vvROCIqbw+CsK1W/3Lp6C7uqHZS7PUcJX/pCvLbMURAcDGSzCGRPGPM/B2XcnEb/GAzPwKFSXpEt5FykqViBPhXaqp2+gr6C/aO2gSi8lnHrfwmqWTd2+c+kq/xHggy+0mCXC7+Ezs0JdemX9PdsIeiJw+SZ74WrvLQZChroL9lb5gQqbT0GLv7J+r3rQS4Ne/boQa983aoGK+CfxY8I0ezdLlVhHR898CiRNJl55QZBUXMMWGCYqn2Gvpcu8N+ehw+AeCSHvltD6BoyA6rqPICNJ7EVVpbTzxQIoyDo5fSOlDmjwIF0TC5nxLDYwbmvPBiNaKcT81Tvg9wVf7/DJGFHxm4FPpZ9bxN8OZqoJRnwL4h35Dnk1LnF6oWhno/OgwPvfQgSHeHxqlP0vdj/gWQLEXDIfY+XRvWS3y6ItJwVq9CdOvOBUxbmHYgMA6Sopxtj1Z+atVMaXCHUm6RLrgybjtFv7EjHk1+cIMwc2Ya78pmhlnYbiAgRcbkAIB7n1wr/O6bqtQlh3kOvB1s01hBNINdQUfu/xdtmgZ6BJC2YPHPJaxEy/8ahb4qRnCejEX8Tpk8mHYWGXtbWIG+xab6EtBDZlQaGUMiSdL4xBHcOwjA/HHVMXLm7wql4cFjDRU37Oy5KgbQpSnSJPIZuYHEi0VAamon6WLkFvYx476uLOOEk2jWdmJqNlq2uaY3u20xTpm2tUEEK9Tn2kMFpeHlKacAnz2Puv/O+2Ebj7rtMG/Dvxe5lJbEbdKQ6VRDUpw1GMLkkzDhPiij/0HUYhFBpL2XcJ8S9dV+9LZ2GSicsXX8DA1ZtgRif96kkm5crg6i+1byHuYI5uksOHWtDIqsTvRR8ceJZIfMzYVzrX9UleuPLea8hSmpRswf0GNhxCp2WJFj9Z zDiMFB/j 8Y2j5QEpWQYOKiAkUZOkbp7Asb5JkcFk2V8swXGvQJpbf2R6PZ7HMsQzpemzHEfRj0CaBjmVjtJVdMnVigkGD0Al2ZkOnNdC1Vz5j6Z+f1WYZ4x/KRGfYr/moHW1a5EtJqDYCiRU6K6NdKlRsr3TBADsOy177L8VsphK79McdwJADNSFOlFvZP9kiyNLC0ulJwcfnTPmujRM1EPLe7sKkF+iUmdBzwqFf7NqHLFtKMIV4Urh+OsWlfgxmEEdiUeg+re8SYYxkitXzuBmHctiTwvtV9NeIl69R7MzoOF9zdUiP0ic= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Jul 24, 2023 at 08:32:45PM +0000, Kasireddy, Vivek wrote: > Hi Jason, > > > > > On Mon, Jul 24, 2023 at 07:54:38AM +0000, Kasireddy, Vivek wrote: > > > > > > I'm not at all familiar with the udmabuf use case but that sounds > > > > brittle and effectively makes this notifier udmabuf specific right? > > > Oh, Qemu uses the udmabuf driver to provide Host Graphics components > > > (such as Spice, Gstreamer, UI, etc) zero-copy access to Guest created > > > buffers. In other words, from a core mm standpoint, udmabuf just > > > collects a bunch of pages (associated with buffers) scattered inside > > > the memfd (Guest ram backed by shmem or hugetlbfs) and wraps > > > them in a dmabuf fd. And, since we provide zero-copy access, we > > > use DMA fences to ensure that the components on the Host and > > > Guest do not access the buffer simultaneously. > > > > So why do you need to track updates proactively like this? > As David noted in the earlier series, if Qemu punches a hole in its memfd > that goes through pages that are registered against a udmabuf fd, then > udmabuf needs to update its list with new pages when the hole gets > filled after (guest) writes. Otherwise, we'd run into the coherency > problem (between udmabuf and memfd) as demonstrated in the selftest > (patch #3 in this series). Holes created in VMA are tracked by invalidation, you haven't explained why this needs to also see change. BTW it is very jarring to hear you talk about files when working with mmu notifiers. MMU notifiers do not track hole punches or memfds, they track VMAs and PTEs. Punching a hole in a mmapped memfd will invalidate the convering PTEs. > > Trigger a move when the backing memory changes and re-acquire it with > AFAICS, without this patch or adding new change_pte calls, there is no way to > get notified when a new page is mapped into the backing memory of a memfd > (backed by shmem or hugetlbfs) which happens after a hole punch followed > by writes. Yes, we have never wanted to do this because is it racy. If you still need the memory mapped then you re-call hmm_range_fault and re-obtain it. hmm_range_fault will resolve all the races and you get new pages. > We can definitely get notified when a hole is punched via the > invalidate notifiers though, but as I described earlier this is not very helpful > for the udmabuf use-case. I still don't understand why, or what makes udmabuf so special compared to all the other places tracking VMA changes and using hmm_range_fault. Jason