From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D526BD3E79C for ; Thu, 11 Dec 2025 07:06:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 441956B0008; Thu, 11 Dec 2025 02:06:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 419686B000A; Thu, 11 Dec 2025 02:06:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 308D06B000C; Thu, 11 Dec 2025 02:06:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 200216B0008 for ; Thu, 11 Dec 2025 02:06:32 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 6A16C8A989 for ; Thu, 11 Dec 2025 07:06:31 +0000 (UTC) X-FDA: 84206306982.15.AA00ADA Received: from SN4PR0501CU005.outbound.protection.outlook.com (mail-southcentralusazon11011016.outbound.protection.outlook.com [40.93.194.16]) by imf03.hostedemail.com (Postfix) with ESMTP id 6BD9F2000C for ; Thu, 11 Dec 2025 07:06:28 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=MQcl3Ad+; spf=pass (imf03.hostedemail.com: domain of ankita@nvidia.com designates 40.93.194.16 as permitted sender) smtp.mailfrom=ankita@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1765436788; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=q5YHI6AFwaCUcqJFzuoVIUcmYY3IUXGB+1xa6MXCpOI=; b=3Vs62ACFydPNWU23qPu2R9WzsI/tq/Wh2Dc1wQSsmdnMVuOjucF/Zrm1I5mYHU+WhAFt4W EiAkCjaEdNM9k13iNDuBcGbAdtz/Ggbv28HivRO1ds1ZalnykQkIIV/6MEmK300kN481Dm yYPkjx6yD83eF5voARPDR/nI8Y3TiBY= ARC-Authentication-Results: i=2; imf03.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=MQcl3Ad+; spf=pass (imf03.hostedemail.com: domain of ankita@nvidia.com designates 40.93.194.16 as permitted sender) smtp.mailfrom=ankita@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1765436788; a=rsa-sha256; cv=pass; b=iK99nhlPype9JqoK28vSGlGQryTSlSS46fsaZ5+Oum2lHk64u7QSQNU7OaurZJq/Bglqwv 058XxNM2DNAMmFaDvTzxl/lnds91UbZJ96/8o4NaYlrCJtlaKBs9M/8k15+uXMU7OHtnXs ar3FZV1umM0k60vel4h4zhjCP7SVJAM= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=fTCxnTecVk9BBOX+Nk8JmVlj6i7xNXkm3URK0FwJcEtrOyu7OUYpMDl4L1QqLTbLMK1m1qGc05LvlLOoFUj+4QtIEcmdhZ/PtpP/Ihrw0nTB88GQ0+EnpNLSK1Ghh5mQqz1T4aZaEibfGwghRmNGg3LLteQbYkXv+8kehFUySBA8sjXlwFk2Je+fakBF+d9lkiHGu+IBKJA1xh2SWtLvCtUT1+OvbjuDKGkF8HOpKRFVOT1AAkxoCtKkcwevtdDml9qmm3q7KKAWRmuWT8OxPY5v9rZVJ+RBvYYkwUZ9QETm1Tnicx0yXCQbQYJhfGwcCQNtCO/Fh43RAXLu4l/2yg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=q5YHI6AFwaCUcqJFzuoVIUcmYY3IUXGB+1xa6MXCpOI=; b=ICDY4UfP/7uwn/jGgMhoZYx+yT9t1adlPaPreyEv7zhNs8Cw2eFJOIlEgPT3WdWDz4/ZDSvy0aUK/I9y+zyVcqt8SrI3wvG+/FWlWfqcuRUo7KoStb08F02BgJJMaO3JMXn2U2eMS+DXUq1nIzxTkibmdcGmb2NQC5Y45z4l+Y5vfLW4uSCDEKuodjW1CwW85XqXnvUPYg7yi4BKoRDaFEjPJshL1+5KALcHZYrcd9mH75qKyZ5wElEgdDg3v47ibqJSsZJPIXIs2Eh1WQhuSXgQSbHanD4qa8e6/W5G/8ohtq3Lm8TyQCGbmrmNXfpoSjZc/bIPCIcDTqLsLZdsKQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=q5YHI6AFwaCUcqJFzuoVIUcmYY3IUXGB+1xa6MXCpOI=; b=MQcl3Ad+RikandLTfNbG8XanV/1t2ABOTVmAdoyplXbA0ZMjLwMgSVw6ZuJqBhduRd4Uq/97YOgenUhyuCwZsmTTnKTHh65lj4idrywdmI+xG5b8oTnJ9Qepf6Rz/IqJ93sLV05EQ40XdeO51jL1qMmcPba3YZV1GeeoeCDDTN/3R9LMGR7JfWDAy1CzuNsOWZynWRzdnWvBapgbNFfrLs64KtArpoax02MEeswi0I5nVkd+4YXi6EkWh6lMKOmIQT2Je60T7hPGkUIN4YrPRTNay52XyVO+m7gDEKaW5ikSKAaIL8QUcPi/XIG57Eiu7afuuDU9pYYYsok/XPMWpw== Received: from PH7PR03CA0002.namprd03.prod.outlook.com (2603:10b6:510:339::21) by DM4PR12MB6253.namprd12.prod.outlook.com (2603:10b6:8:a6::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9412.9; Thu, 11 Dec 2025 07:06:23 +0000 Received: from CY4PEPF0000E9D0.namprd03.prod.outlook.com (2603:10b6:510:339:cafe::54) by PH7PR03CA0002.outlook.office365.com (2603:10b6:510:339::21) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.9388.15 via Frontend Transport; Thu, 11 Dec 2025 07:06:11 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by CY4PEPF0000E9D0.mail.protection.outlook.com (10.167.241.135) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.9388.8 via Frontend Transport; Thu, 11 Dec 2025 07:06:22 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 10 Dec 2025 23:06:07 -0800 Received: from rnnvmail201.nvidia.com (10.129.68.8) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20; Wed, 10 Dec 2025 23:06:06 -0800 Received: from localhost.nvidia.com (10.127.8.12) by mail.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.20 via Frontend Transport; Wed, 10 Dec 2025 23:06:06 -0800 From: To: , , , , , , , , , CC: , , , , , , , Subject: [PATCH v1 3/3] vfio/nvgrace-gpu: register device memory for poison handling Date: Thu, 11 Dec 2025 07:06:03 +0000 Message-ID: <20251211070603.338701-4-ankita@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20251211070603.338701-1-ankita@nvidia.com> References: <20251211070603.338701-1-ankita@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000E9D0:EE_|DM4PR12MB6253:EE_ X-MS-Office365-Filtering-Correlation-Id: e4dd5f22-1e56-4ceb-f6df-08de3883caeb X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|82310400026|1800799024|36860700013|921020|13003099007; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?a6CddOKx4comeCdtnmcuHRIwDdnl9XKeAqvwgVmi+YcJPrTGeKjU2+Xwtqxz?= =?us-ascii?Q?TBS59bB+J3liK+cXIQS1S6hsdbTMZM04x+1KL1nFBYiAtXj3n2tLuhQY9wYt?= =?us-ascii?Q?5EQrvQX5phmT8hZEu9A8BByURzxYCBOWzmjlVvq4S0ccFuvRasjJ4S/k9y5D?= =?us-ascii?Q?Wxki7iJx+C6XuPSNJPH/XnMnjI6OkvbhO10lzBKJlkGnZaI9W+NqRAxRF9LK?= =?us-ascii?Q?cL/V9QV8hN9u0rbVRGIJmfIrfI7u7cDPDuCyo6cSL3/JAYxpiVfzHq+G3Knu?= =?us-ascii?Q?BYa1yses/lmwWHJPd1yiKkH9an/KhVaBFwZiAHiQFuSWgInf/UdUq3lRR+n1?= =?us-ascii?Q?/K3GlpaAeS1WZXcTr0owZv1fz0H8LoQrIm+d8GLZkFDotw5DAxd+9MtTKKfU?= =?us-ascii?Q?KGjkE20dUnZ2GCe/6ibQldiEPk7kbC8E9ZWwVXg+U9w5AwSBSbKAjL0SkjjF?= =?us-ascii?Q?AUmsxSwDAEUnqftOhDRfeRsQtC6kIqx9GxEgfp32r0mqW8bjobFfCHW74q94?= =?us-ascii?Q?OWp3qgUSgux3TkbWGJniWz3wFP8Ad5zpbSxiDcxXBvanhzWPf6QYeJ2TAqSi?= =?us-ascii?Q?9raYMAbE8guOtviL/jDC0e3JiJxFs0MEfiQyLRopzxDudXdo97ANdOj8IswH?= =?us-ascii?Q?uYyS8OtsleRXjGC3aKr2LIJhLnWBFcJYFWO3shYyfu7h3DMExB5qSmUbxu+V?= =?us-ascii?Q?NmbD9sUiz0TyGjr1fkms/9HYZtpT0OyBT0p6jBCjUiXVhGxx70iHmob9RI8P?= =?us-ascii?Q?BPG4+pTQ2OsNLuFXGHWxHUhOJbojwf8G7HgChAJOJyffZ2PCzRSLW4jqqjeT?= =?us-ascii?Q?LVmLpAS0M0p2qGCBB4feVaSkSM1gxiltExfvi6G0YaTIeZRRtg3I+CppKhi6?= =?us-ascii?Q?2YRmFizmrLGBoFK/IJkWLwcEP2Yx63l/yjCw42XhQAHs0vthzW3Hc7O9HmfQ?= =?us-ascii?Q?DuoqCwnGlGoE7suv5F2ZHfXDkR3QuJEpcl73VKd1T73ygOtcv228vkc5XPTp?= =?us-ascii?Q?TDOiqDSJs/Au+bs+xI4heoKtnZU08OGggVtA3x38msjO8vfuhUz+v2rSdNPQ?= =?us-ascii?Q?mYyRf9Td/qbwg5VpS6yGSGZbN3L7B4v/WRnh7V4y9PorzsXqBI3t9bWQzx+w?= =?us-ascii?Q?znTgJJw/W8pPaMjImf0SLMwoFxG5Zr8CQzRS0WLQHVaE2AChH85Q2sOk0YW1?= =?us-ascii?Q?qzw6GJE5yGE1Pa5BtgKsM4gRWwahCNUpijusVm8g95ds28+c/Wy7XUjqjTak?= =?us-ascii?Q?21nuD6MJ0r1dZe4C2wHmRJvEuJw/V3KWtTy+/ozoRjEdVe7UJvNIiEv8b/cu?= =?us-ascii?Q?A+fZUoKcP59jVG2GHL8Mnk15xGpLyrlBcM+Gjv5bmaLFU5UDFIt1RGaivxAJ?= =?us-ascii?Q?dMuGp5ZxGL0OogQPMtPXpJQGX9SC7I3Nv4ajpr7wvdrr6FmbuVKk/PawAMmP?= =?us-ascii?Q?p5FYGRdut3/k7/2hK+Ut4QHnNwVfXaQtM26Ym9hECcKCDuV3nRs0kNqEgr7m?= =?us-ascii?Q?DHwvYPO7XsJRgkOZxRYWyzz98+Wr3yLktux+QZNJiDMq8sxbQPtowcXBtVZR?= =?us-ascii?Q?BBrHvosO20Q899AW2T3xFi42AKvwVaa9c3kc74uz?= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(376014)(82310400026)(1800799024)(36860700013)(921020)(13003099007);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Dec 2025 07:06:22.8300 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e4dd5f22-1e56-4ceb-f6df-08de3883caeb X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000E9D0.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6253 X-Stat-Signature: cqmpamexio44x1d8wpxp4r45e6n33fwn X-Rspamd-Queue-Id: 6BD9F2000C X-Rspam-User: X-Rspamd-Server: rspam09 X-HE-Tag: 1765436788-322509 X-HE-Meta: U2FsdGVkX1/dKYLMJZYdn+dVHKIbvEiipQqnVZyKiMAl62Ti4fahhsS7NCsgKpbRskYYu/FBihXnwV9FOZdNg30mexsmzHE6rXb3ibXZKhtid7ToYJmDsmuODtoz5incAYshxPvwVl4p2/0UWZybyvVno1jdDdMBHDUaYFp5EkyQhU9v/P90op3xljxj4o6Ah+8yjOCIae19+eGdro0DuYu9/C0+5c84PUTQAW8OjUwS+hp22zoaYEFbcUidKxM/bl8jHBMtQmvKhF11sxPSct3AL4nQBv8dKCXxRCUjtciv1fgwObf4pgqGgDDZ1gcFSXoGcX6vAYGe0TnMsOL1We1y5fmH7rUnbTMqkK1bbSLNFllCrg7DK6YO8+rp9A6dQO2tKQ3pKMX90pKk0SrhoNJhAXmT8/5k2znkKFSbhdXa0NuaV2HX9GO5N2WO6V3wsC/hi1UhvgOd1lmf9q0bqUI8c4tetkt2HVUFOVbMUT8bhvxigK/LXS5FeVpzpVR+8BUWvLrSd9LbY4TSx49bmk6hbTdwJJkrvrXNO6W9LtHUywiTUxMdQR+V/6Azi9HYY0rwc397y/0fBtjs+sFB32+2dwZguEca2GO6zUHaQ46zCQ+xkTc1sAdlvnA6oyjY5OJVQzgvLe/vK0Z2FlXQcT8glg6h3cJgzlJ2zCFTRfObHBFSz2ct3PLgtmV1UVr53+oFYW5Ru4y7yf+FTww3H4xucmp78+iNDtL2ARp6gZDUOwuYLivNtfJXF0CFsUjhhk31PLnesDPIab8SFV9uuw/2wh76JosVC8VLNYl9g5Thfew1sZSHKW456V40tv0cRNoHeQZPyYRxcu6JvuC6nfqIjzJfTcg50FqIUHoLOc4h5LvodxqgnmFoG+j72OX6uFs8UaKH4QXENpnwIPRg+8yDcFoqTyp+lxqEXw0k7+LMworiqCoyRM2XeeaoVinMfPGsC1+PxVVWu6AED7B IEhTtYZi DGNIGePTIgkDno16PUo8fcEp32mBxahovujBN1TEVySvWwAioPN167ElWLGIF1OJW8gyR/G3ZBLeCiqEG8J+u8I06BrjaJQfYyFH2wSNib/49CC678ODkQlfkVgGAobvoq4XV0ZHS0eRTbJ1JDx36AU/afGVU+06u+Vy+K9mV3DAFQ3Ivp8oTmWYQP451c2UWaOMAEyE6gSWltJiCzYYX0hU9qfjXaYYGF9dou3GYXdEgxCejZm63vXUYkU0XHfocntp9KjLcgMqICb66ldXjUG3KTUGyv4m+h21VyJZNNtsQ9748vn8Mb9s1ga8d5loZGBuQOOOpDBMGv4/o2WjPCr3RStJjd2+TInlmrY3WLZraNeXpg9/fjIsvW9MsrlfrhONm1rHrf3Elxi94OwZRccWA6ATZKD35nhS26hSfiaYxzGa/dFHNM0gwhEXs1GGf/oIi4cjCJSCXMpCDlRed5HrcG+NSayjAmWQBKJAn+wiS33E= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Ankit Agrawal The nvgrace-gpu module [1] maps the device memory to the user VA (Qemu) without adding the memory to the kernel. The device memory pages are PFNMAP and not backed by struct page. The module can thus utilize the MM's PFNMAP memory_failure mechanism that handles ECC/poison on regions with no struct pages. The kernel MM code exposes register/unregister APIs allowing modules to register the device memory for memory_failure handling. Make nvgrace-gpu register the GPU memory with the MM on open. The module registers its memory region, the address_space with the kernel MM for ECC handling and implements a callback function to convert the PFN to the file page offset. The callback functions checks if the PFN belongs to the device memory region and is also contained in the VMA range, an error is returned otherwise. Link: https://lore.kernel.org/all/20240220115055.23546-1-ankita@nvidia.com/ [1] Suggested-by: Alex Williamson Suggested-by: Jason Gunthorpe Signed-off-by: Ankit Agrawal --- drivers/vfio/pci/nvgrace-gpu/main.c | 116 +++++++++++++++++++++++++++- 1 file changed, 112 insertions(+), 4 deletions(-) diff --git a/drivers/vfio/pci/nvgrace-gpu/main.c b/drivers/vfio/pci/nvgrace-gpu/main.c index 84d142a47ec6..fdfb961a6972 100644 --- a/drivers/vfio/pci/nvgrace-gpu/main.c +++ b/drivers/vfio/pci/nvgrace-gpu/main.c @@ -9,6 +9,7 @@ #include #include #include +#include /* * The device memory usable to the workloads running in the VM is cached @@ -49,6 +50,7 @@ struct mem_region { void *memaddr; void __iomem *ioaddr; }; /* Base virtual address of the region */ + struct pfn_address_space pfn_address_space; }; struct nvgrace_gpu_pci_core_device { @@ -88,6 +90,83 @@ nvgrace_gpu_memregion(int index, return NULL; } +static inline int pfn_memregion_offset(struct nvgrace_gpu_pci_core_device *nvdev, + unsigned int index, + unsigned long pfn, + u64 *pfn_offset_in_region) +{ + struct mem_region *region; + unsigned long start_pfn, num_pages; + + region = nvgrace_gpu_memregion(index, nvdev); + if (!region) + return -ENOENT; + + start_pfn = PHYS_PFN(region->memphys); + num_pages = region->memlength >> PAGE_SHIFT; + + if (pfn < start_pfn || pfn >= start_pfn + num_pages) + return -EINVAL; + + *pfn_offset_in_region = pfn - start_pfn; + + return 0; +} + +static inline struct nvgrace_gpu_pci_core_device * +vma_to_nvdev(struct vm_area_struct *vma); + +static int nvgrace_gpu_pfn_to_vma_pgoff(struct vm_area_struct *vma, + unsigned long pfn, + pgoff_t *pgoff) +{ + struct nvgrace_gpu_pci_core_device *nvdev; + unsigned int index = + vma->vm_pgoff >> (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT); + u64 vma_offset_in_region = vma->vm_pgoff & + ((1U << (VFIO_PCI_OFFSET_SHIFT - PAGE_SHIFT)) - 1); + u64 pfn_offset_in_region; + int ret; + + nvdev = vma_to_nvdev(vma); + if (!nvdev) + return -EPERM; + + ret = pfn_memregion_offset(nvdev, index, pfn, &pfn_offset_in_region); + if (ret) + return ret; + + /* Ensure PFN is not before VMA's start within the region */ + if (pfn_offset_in_region < vma_offset_in_region) + return -EINVAL; + + /* Calculate offset from VMA start */ + *pgoff = vma->vm_pgoff + + (pfn_offset_in_region - vma_offset_in_region); + + return 0; +} + +static int +nvgrace_gpu_vfio_pci_register_pfn_range(struct vfio_device *core_vdev, + struct mem_region *region) +{ + int ret; + unsigned long pfn, nr_pages; + + pfn = PHYS_PFN(region->memphys); + nr_pages = region->memlength >> PAGE_SHIFT; + + region->pfn_address_space.node.start = pfn; + region->pfn_address_space.node.last = pfn + nr_pages - 1; + region->pfn_address_space.mapping = core_vdev->inode->i_mapping; + region->pfn_address_space.pfn_to_vma_pgoff = nvgrace_gpu_pfn_to_vma_pgoff; + + ret = register_pfn_address_space(®ion->pfn_address_space); + + return ret; +} + static int nvgrace_gpu_open_device(struct vfio_device *core_vdev) { struct vfio_pci_core_device *vdev = @@ -114,14 +193,28 @@ static int nvgrace_gpu_open_device(struct vfio_device *core_vdev) * memory mapping. */ ret = vfio_pci_core_setup_barmap(vdev, 0); - if (ret) { - vfio_pci_core_disable(vdev); - return ret; + if (ret) + goto error_exit; + + if (nvdev->resmem.memlength) { + ret = nvgrace_gpu_vfio_pci_register_pfn_range(core_vdev, &nvdev->resmem); + if (ret && ret != -EOPNOTSUPP) + goto error_exit; } - vfio_pci_core_finish_enable(vdev); + ret = nvgrace_gpu_vfio_pci_register_pfn_range(core_vdev, &nvdev->usemem); + if (ret && ret != -EOPNOTSUPP) + goto register_mem_failed; + vfio_pci_core_finish_enable(vdev); return 0; + +register_mem_failed: + if (nvdev->resmem.memlength) + unregister_pfn_address_space(&nvdev->resmem.pfn_address_space); +error_exit: + vfio_pci_core_disable(vdev); + return ret; } static void nvgrace_gpu_close_device(struct vfio_device *core_vdev) @@ -130,6 +223,11 @@ static void nvgrace_gpu_close_device(struct vfio_device *core_vdev) container_of(core_vdev, struct nvgrace_gpu_pci_core_device, core_device.vdev); + if (nvdev->resmem.memlength) + unregister_pfn_address_space(&nvdev->resmem.pfn_address_space); + + unregister_pfn_address_space(&nvdev->usemem.pfn_address_space); + /* Unmap the mapping to the device memory cached region */ if (nvdev->usemem.memaddr) { memunmap(nvdev->usemem.memaddr); @@ -247,6 +345,16 @@ static const struct vm_operations_struct nvgrace_gpu_vfio_pci_mmap_ops = { #endif }; +static inline struct nvgrace_gpu_pci_core_device * +vma_to_nvdev(struct vm_area_struct *vma) +{ + /* Check if this VMA belongs to us */ + if (vma->vm_ops != &nvgrace_gpu_vfio_pci_mmap_ops) + return NULL; + + return vma->vm_private_data; +} + static int nvgrace_gpu_mmap(struct vfio_device *core_vdev, struct vm_area_struct *vma) { -- 2.34.1