From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F904C001E0 for ; Sun, 30 Jul 2023 23:09:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7081F28000D; Sun, 30 Jul 2023 19:09:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6B729280006; Sun, 30 Jul 2023 19:09:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5577728000D; Sun, 30 Jul 2023 19:09:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 434F6280006 for ; Sun, 30 Jul 2023 19:09:44 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 053021A09ED for ; Sun, 30 Jul 2023 23:09:43 +0000 (UTC) X-FDA: 81069822288.01.367015F Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2040.outbound.protection.outlook.com [40.107.236.40]) by imf24.hostedemail.com (Postfix) with ESMTP id 353E2180010 for ; Sun, 30 Jul 2023 23:09:41 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=JtKLoIAU; dmarc=pass (policy=reject) header.from=nvidia.com; spf=pass (imf24.hostedemail.com: domain of jgg@nvidia.com designates 40.107.236.40 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690758581; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=S5ECq/IPXrJDIh/ws7hb6ukLjCZ1Em84bICo+I2nT8s=; b=GGj/qDjKsV39metoDJFr4X8rVQw2AvuJn3cZtcLV5hbMf9SxFQB04pW9fZykcUyIhBreX+ nBspRHtgQSIXuSBfqU/C3wjPIOvxSZtulcEPq2+1fbeAdxQ68Xg8NiScrR2ikH/qDbGUP4 YbW7ARsvyg2E2XNGHHQYLKZCicdFmus= ARC-Authentication-Results: i=2; imf24.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=JtKLoIAU; dmarc=pass (policy=reject) header.from=nvidia.com; spf=pass (imf24.hostedemail.com: domain of jgg@nvidia.com designates 40.107.236.40 as permitted sender) smtp.mailfrom=jgg@nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1690758581; a=rsa-sha256; cv=pass; b=tmSM66y6OKOYKH60sxwnUJSMa2+D+P48wvuWictIso92KE6KZy/1L7NtFVmoR/xh/Nykm8 3WQbPkBCUo8mFsL/xqxycq8kz4Ocy6p441rVnBU+DOYNwhq8pzqlpJJ2ZIIlKjElqd4xQa X9VTrchK+Bw3hNNUcTGWa5clM67KMSM= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BfvTQ3YV9cc3DdpMAMam9Afh4OG12EaHo+rJCuy7COLq28N/+uf19NiAqRdtPHfcRChR/llWrfMKLn8b6/YiJtYz0zSg0ViHUTBYjGOzzR9xe562Kpy+efZL/7+PhpFVV43c195a1697+BWpqmogIOQsV7hr2bVPl7iNAn/7/F/AK8/OnYW3ZwbIg5OPZNSQ0jeVusLYCZ82K9yTBwFIPAVOsPErkkp/z1enTOBDvdr4Ll9xSl4EVdlwYwDZIvro1b5Bo5HAUODUOSXP9R9+mDJn0PCA9JcChTYBtZdDcxqXatcVxiKMAm9Qth/vkLCfoumkaa6ekP6Kt5M1BtJVlg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=S5ECq/IPXrJDIh/ws7hb6ukLjCZ1Em84bICo+I2nT8s=; b=gf1oHBlL128ioopkjll/psVzd7XGvO+Qx5gy7MbtNDYd4W01j1zvxh083GyKmZbxx2ekTyCZISGmjHgpuHDaIcMJmdvZP4oFM4BeIWBBzSuU5/NT2kcdPK1L/ai07Npx1CE+V8nzIxNvfr3Vf9YgUfNnMc3mNCev9DiKpmfXlMgZQe67ZQZeiT72aEDSAHCSniUBeb3jX7q/tnU46mYsfSwZa4/4haZ7lXU8VM73KZ/a2/GymUEswEhAO7t2A62MVemVt3jELojIdt2KUTWAODp9UL4+ZRSofVGsctwGlL4pnqJ6KDoUZOfiYjaDKxjk+GvVa+rdAmlOyI0CZlwlLQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=S5ECq/IPXrJDIh/ws7hb6ukLjCZ1Em84bICo+I2nT8s=; b=JtKLoIAUqcxSDW2BCJQSFPGCVwpOg6duIG+AU0k8wtyLIV2L/PhcHVTKEnENrhL5N8A5qTLEO5HIP8bE0hAdHRQb9xAH66DyUrNLgG2x1TjZefnZhL4wRMrTVUOZATf8dTsbJ55JhM/Ugl//DhpRgaEqlW7wH4hhx53oR5nliT+90U5CRtMHNQ0ZIy5FfLNxY1EOAg27X+o34o33qq+DgKlSW6QcirKvxNLd5sON3ztPqSLBTa6TBSNtzreYg0NvtRALE+WKV7PBNuPbG03yaGTxkGyZJIhoa56Uqc6AAJcOO89kCKmM5no6s0wwNkefKCGO93RpPYCs7Nnyv6vEjw== Received: from LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) by DM4PR12MB5748.namprd12.prod.outlook.com (2603:10b6:8:5f::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6631.43; Sun, 30 Jul 2023 23:09:38 +0000 Received: from LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::5111:16e8:5afe:1da1]) by LV2PR12MB5869.namprd12.prod.outlook.com ([fe80::5111:16e8:5afe:1da1%6]) with mapi id 15.20.6631.043; Sun, 30 Jul 2023 23:09:38 +0000 Date: Sun, 30 Jul 2023 20:09:37 -0300 From: Jason Gunthorpe To: "Kasireddy, Vivek" Cc: Alistair Popple , Gerd Hoffmann , "Kim, Dongwon" , David Hildenbrand , "Chang, Junxiao" , Hugh Dickins , Peter Xu , "linux-mm@kvack.org" , "dri-devel@lists.freedesktop.org" , Mike Kravetz Subject: Re: [RFC v1 1/3] mm/mmu_notifier: Add a new notifier for mapping updates (new pages) Message-ID: References: <87pm4nj6s5.fsf@nvdebian.thelocal> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: CH0PR03CA0273.namprd03.prod.outlook.com (2603:10b6:610:e6::8) To LV2PR12MB5869.namprd12.prod.outlook.com (2603:10b6:408:176::16) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: LV2PR12MB5869:EE_|DM4PR12MB5748:EE_ X-MS-Office365-Filtering-Correlation-Id: 99d7181f-483d-46e4-5914-08db91520c87 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: cHZvrWLXgGViDDQatDfe9MdtRbKSN9xT2hsNce71E9a0vMjChbiWpxsXUnSjAc2nTbv9n7Eyid43APVKv9J3ZLv88OIRbQiBD4M/Mf4kgAcM0IS52jFh0zmqQ4W8SotkMDqk8uZyAFQK8izxkuI/Goa1WmoD7l6jdxBDSvV9hzUrpVML6VS+nD8HK8Tg2qD3Yvam3uwShvFeGE9ck5d+MUMKIOBBomXfDMzQrBo+okdoO4ApF6ozufLyVIK5AGrTDZ6apfQDQpXaJW6Ynqb+pNeIGDcwgfTvOuWpAmkz52lw8T6Hd9a6iS9tLSoioPDbQiBzODG0hcdOa/AMXaaKdCCwHFUAGdlxGVf6PRNajamv/OgZYVyCMOZ3h31H2bIYA+iGjngYYVdx84wXSVJHVVYu3XeOd0CT4HyrdMFfA7R0H8JYxWN1PIYunAwtCbb6ZpL3CSvu3QAZQ0+VXwKDptKF8MgChBpeseW53NmmrMlI77iXMVY43PVZPm67dWTO9oyBb56AQ6lbo6qVyhiQrRJ0lIw/cUEoxdp+m6XGozzMbuu/tE+flc4xwiUlNAks X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:LV2PR12MB5869.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(4636009)(396003)(39860400002)(136003)(376002)(366004)(346002)(451199021)(38100700002)(86362001)(36756003)(6512007)(478600001)(6486002)(2616005)(186003)(6506007)(26005)(8676002)(8936002)(7416002)(5660300002)(4326008)(6916009)(66946007)(66556008)(66476007)(2906002)(54906003)(41300700001)(316002)(83380400001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?I9zydEy3vtsOIvOojuTSrQcddVwJhqQOUbh/+un7hDSYZywXXAvxiixfjcEq?= =?us-ascii?Q?tVtRO8xeefT1772v2MNqeDPd0RrHDRH85FsUL5C6xJ6I3/gfEmVIefH4CDC/?= =?us-ascii?Q?2pF4CebNQsxJDce+AZ1YECgu5/n+JFcletdUVJlESL19vPxKzCK+Ab25LFYe?= =?us-ascii?Q?1ip3XqtrUDwgJR/wPdIxcaPBW/R3gZwdMts9pJDMBJlB2OcYk+ea0DjTQgqP?= =?us-ascii?Q?PRbI+llkSGxrCBE0Q/Pa8fYaHnKVAY50BLIjFumkW8QM5DM9oIXGz9PqPpro?= =?us-ascii?Q?SSElQdsLx6nVtJnrvXvJH7QDj7ca8v+M9CirksahogE0X4/ch7kNXS0XKV6C?= =?us-ascii?Q?cumy+NzOVss4EJgRzIrn53L1JcQzSaVbAIwnS89Lx1AUt4YPLD1duOWX8A9J?= =?us-ascii?Q?poby9Rz1MViXT7DEeNOYX81rdPJJ+TPVr0e3NAgJZFEM60koO2eQScMOM5CE?= =?us-ascii?Q?dbHK/F19QKJFp3yE3+nr0Xpldslm8ob5oZ/nPLLRs3SYKvraItSRdjM5A1au?= =?us-ascii?Q?jyvq13B5F23DPzkueeUGmOZr/AlAnAWCo5GMqZH8jGhIZTST1bEOp+9gMqAr?= =?us-ascii?Q?8x8SPFI64lJTJ8HC11sDZGmHg+BDB795tQu9HXdcPHJHqBWPxeGMAKWYIclq?= =?us-ascii?Q?E3Yref93Oe9bJQNjXXDiQ07yp5ki6gNGbNpPLR4irdiRirwuQgtFFi9bWnUW?= =?us-ascii?Q?z4YMr7vbF7JbeR4B3eo8+lBc90NsQb0HtiZ026sXc9U19wqw07Pl5rJs8XZn?= =?us-ascii?Q?29d5DceCBbhUgyZD8Kv1N4A+fRafNrjjpFFReZGmj476bcwopkQ+mI7rhEO5?= =?us-ascii?Q?RkNOc/Uyd74q7MWEwwGTiyTWu8w4hYgB6MeU7NbzO62VnnlsqSTLNYLgQuWC?= =?us-ascii?Q?VxQfdKtaYj5T1yyzGxMgmL4EWuEs06ZMsRvTUKt/4EaKroJt/7mi9HUglPkI?= =?us-ascii?Q?KBOVYj2CAUOZSqhooYLIljD1ifsyVqj1N50oQK6oQF6v/7bFFJSq+9Rz4SeM?= =?us-ascii?Q?wWdDRdk9+lEFH++OBHrG/P7cQ7AN1a3tEBVhvChMd/nxP99RBPe5cbrz+EhW?= =?us-ascii?Q?KOcVU2mSxTw0hoEHeTNTpIrCXh2fap7vlYNFloyPAU0SyS4eRBa1mc+g4Kfg?= =?us-ascii?Q?EFFvwyo7+4DVWtrb82iZK/n5WJrI4f34MUwucWlk024sn2n7izvi9UvMqblS?= =?us-ascii?Q?94hYe45QdkZ8lsbEs0RrdNfcRtLTWZEoPOqexOrpBgsTDm3FcPOZOewuTVUh?= =?us-ascii?Q?zakoNuvTV081tp+McAdQNqAOUIIg0aqckZ6MrqGGC6ccTDQbpZ5WdU4cGQQ/?= =?us-ascii?Q?lHCshpu2MLJ2Jt84YGM+xj3yEg1kuio3e35w/bFg55I12fEpUfM7PZ/Ktz5E?= =?us-ascii?Q?obArR/REnpF8iah5Ljaln/Z75u+kTuQt+GClYUCrjzbzYB1kcm+5XITn54y9?= =?us-ascii?Q?aaYiHZIP6EZg9vi828xkmB85cVBtRwY22EeyJZn9v/y+fVWtWmo0cA4aNAVi?= =?us-ascii?Q?PytttnjdOkmktAj9n9dYFMrepwecDT673h2YQEH/NLToHXnmeb4NIx0ZtBOx?= =?us-ascii?Q?9ujpc+D/7Yog0rbjeWXsgyN7QrzAUK/zu+Uuqzmm?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 99d7181f-483d-46e4-5914-08db91520c87 X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jul 2023 23:09:38.8148 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Kezs92cgSM0hh4Ucy7HC0nG+cLGwrH2lR236mSVGOEN7SWMEyxZrmR4MCdtcQO8r X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5748 X-Rspamd-Queue-Id: 353E2180010 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 94sgc75wgx1wwarurf6yt5caa3ssa3y8 X-HE-Tag: 1690758581-957619 X-HE-Meta: U2FsdGVkX1/GMxjyPfBa49hp73ayVTe8dCD2StqCgJ2BQzMvd3W0xJLzti9vJfIcM650DsQKKvCUMVEVHaf437V9kFdcm8ra0mzQHnNHXSvd25mhswsGexQBhQt2xhFnTpgtJFr7+fjB2/YwON+b7bBpiZBJ7FeK5R0LHrw41xNUx84DR9U5F0+N26iKYSOaEd9YQbe1KCIpntx56wACv7ObHygXHY97BPu3/ccq8sxyZBWHcOdmRnskDubeSWUsnkBcmG8uXiGsNfTPbK2oolyu7TtVtJxVOKZJVBsB+sXSwCe69z1p7Lw0iJA6at0UdO+R6eD4j9vbfwosZG8MHR+CQy9ZktGd06e6pFU6ZvqpuoRx5xm716y/5ZC/kSKppNCV5ZzG+4ros7KKL6v4AkJ7y5SWRhXidVANAkD7MPGjOt3vdL6qzFmRW36EJa8mZWsagUWW123EGJCEL9q0jyxwoOW+bxJimzkSpgOEc54OA/1cAZlxC3r7llRs56DPxnuzeDOr3g3+23ceIz9cTB1C6EeuwC4ZwY9OkSFIGQLwAXQdb3JhjMW6ucmt2+5J2qihtrA/2VfhsevxBqXOKJ+c535SuwatUElcSBCNLdY2I2RKNrpTQZIMd1ibV22DrTCzTiTKyrO4VyNR+4xKjTq+eSC3xztQ4rnuUwVykjmFVfou+cS8NkAemVle5sXfaaiPSQrkLV4Dsk5JZcemYZI28lBqM2gC7g920QVLkv19ffaZrSQ/tBOzWrWg62Uu7oL2MC++00ihvebVh50kGOJooQylv8wErGfhTOYEZ70k0a2Xb24qlZlsDrguiyrhQyW/Lo4XnU8jt/PnRBRQkv7unDgwKOLYeeZHweEBGrc0jbV3I5hDM5BXEjxF1/ui7UQ+U9n0DK2qyDXJ5/RyOhOzd+iIjkg6/oCeShM47i/TS0S2W6APAVLll33xVN0+W0qrwtkJhH6IIX8shOb edYmscyW TfiDqYs2eAqeX1/GW8QmdqVZVb+SO9IXlCu4RyiwHnWaAOQ0Ep8fzfRjk0H/pvSVIgEef+72GwLLau3LNs0k5lWGvYJMB4snJJ+QHni6U9EVme3nu79OJbXMnVJAgaVt27HAYlW+pXhDbKphfKK6KVjF9QkDHbT6CVXF/PpgEjcBJ+VdOXPJumGKl675gdXIuIGAiPpRaR0AHx944xs5o8c/MkkBsYijIc+DfIv3nljGjXwE+fytuGKoYUwSeerg6Jo+4B097lPLV8Pv2IxWtXL80dZCB5JbCEKt7R1XYqrpzo7fGFalrPSd6c9gwNpYd2vWnwx505oUPl30= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000456, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sat, Jul 29, 2023 at 12:46:59AM +0000, Kasireddy, Vivek wrote: > > Later the importer decides it needs the memory again so it again asks > > for the dmabuf to be present, which does hmm_range_fault and gets > > whatever is appropriate at the time. > Unless I am missing something, I think just doing the above still won't solve > the problem. Consider this sequence: > write_to_memfd(addr1, size, 'a'); > buf = create_udmabuf_list(devfd, memfd, size); > addr2 = mmap_fd(buf, NUM_PAGES * NUM_ENTRIES * getpagesize()); > read(addr2); > write_to_memfd(addr1, size, 'b'); > punch_hole(memfd, MEMFD_SIZE / 2); > -> Since we can process the invalidate at this point, as per your suggestion, > we can trigger dmabuf move to let the importers know that the dmabuf's > backing memory has changed (or moved). > > read(addr2); > -> Because there is a hole, we can handle the read by either providing the > old pages or zero pages (if using hmm_range_fault()) to the > importers. You never provide the old pages. After trunctate the only correct value to read is zero. > Maybe it is against convention, but I think it makes sense to provide old > pages (that were mapped before the hole punch) because the importers > have not read the data in these pages ('b' above) yet. Nope. > And, another reason to provide old pages is because the data in > these pages is shown in a window on the Host's screen so it > doesn't make sense to show zero page data. So why did you trucate it if you want to keep the data? > -> write_to_memfd(addr1, size, 'c'); > As the hole gets refilled (with new pages) after the above write, AFAIU, we > have to tell the importers again that since the backing memory has changed, > (new pages) they need to recreate their mappings. But herein lies the problem: > from inside the udmabuf driver, we cannot know when this write occurs, so we > would not be able to notify the importers of the dmabuf move. You get another invalidate because the memfd removes the zero pages that hmm_range_fault installed in the PTEs before replacing them with actual writable pages. Then you do the move, and another hmm_range_fault, and basically the whole thing over again. Except this time instead of returning zero pages it returns actual writable page. Jason