From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F36A1C7115C for ; Sat, 21 Jun 2025 04:21:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9A1786B0096; Sat, 21 Jun 2025 00:21:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 92E5E6B0098; Sat, 21 Jun 2025 00:21:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 77DA86B0099; Sat, 21 Jun 2025 00:21:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 5E3266B0096 for ; Sat, 21 Jun 2025 00:21:35 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id D425C81E4C for ; Sat, 21 Jun 2025 04:21:34 +0000 (UTC) X-FDA: 83578108908.04.83C89E5 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2061.outbound.protection.outlook.com [40.107.94.61]) by imf01.hostedemail.com (Postfix) with ESMTP id D62A340004 for ; Sat, 21 Jun 2025 04:21:31 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=lEG7AklU; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf01.hostedemail.com: domain of ankita@nvidia.com designates 40.107.94.61 as permitted sender) smtp.mailfrom=ankita@nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1750479692; a=rsa-sha256; cv=pass; b=btBFr/WZ4YTFk/PqsbuWc53FppzKMX7sUgbHZ+zmxtqI+GC4mdgkhZj/WLan1C28gVFqDs enhBEbJ0fQ3h0pGlUV8iPEHCZRieqxqg4SloTU6x0wYBXN8njFcIVdbSpmnSdNeqWLr6lo H/tXY7yFzFsmQ5M6jsApQWbMTfCio2I= ARC-Authentication-Results: i=2; imf01.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=lEG7AklU; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf01.hostedemail.com: domain of ankita@nvidia.com designates 40.107.94.61 as permitted sender) smtp.mailfrom=ankita@nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1750479692; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WJcob8QYfo5hlHn0Ev0TZ2EMAwugli1iZYAf5iub76M=; b=tjy/kNYhsPiThYPPqXaIpUwLwRQh3ahE0lPTZl/87m1xBZvgz/L84U0RUD9hOeUu+PepTd Ord3butvosY1MjuayofTb+RIT9PR5ZRcN18+J7NKku/7eHMqsDjwzD7mitGfmZNasByP5t QlCP1ww8aFOpoSd3Li+UkfSCRK0xjvA= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ymliTTkXYLzRyf00J6BU5YXUZ5lKoHVZDwUC4jejHPhX3F+sV09VqgZEIKNja4gln9MOoXyeH7V0cYRDiu5uqDjPFkXURIn2iAsAN2GZSPtxx/zR1IrPVpBFEwX9n1GyPne4pe+fzIE/DXvTxUUVUMWDj6ItRMZoCZrci1UBb3d4Vbx6JLHemF2E0L0Es7lMRyD6v6KLwp6Hm+v6/0dm7Cci5fehpIvQ1AYbmSOp5qsOHZgYXI4NGKukGiiCWL8FaGuPAhjDq2gYAuIZ6VROHHc+0nCBakwRcxkQot4sPdxWO0ru4HU/grgse+1ex0iIRDyp/XlFT88uTV9wf/tsVg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=WJcob8QYfo5hlHn0Ev0TZ2EMAwugli1iZYAf5iub76M=; b=dHuGeZHdCrPFv1r2xG8mzVoZ8aKTmXKff+Q/LLhWUxNzX9+Y/Wi7m+Ycsa7r/L9Vkm2480hWj+jzcKRpYHr9pETTkH2Fl6XGM2WTsgUWAYuHoK3sk1z4SA+Tx9l9Er5DmwysbN2/qqrJ9Un01XsF7vR+4mpA1duCDko0xdKhw+qLQYGhzL/J5ioIedYilKPVQJ7tt21YOwOE+KdeIbrpVNYIMgnGnPgZqMT/hA7rUwfiLIBSaXEdUQyo4SiNTCxKAsTtpPSaKQwv47rJBNLtcKSASqOLSB+vjj8fPb9UHQ5yczWXct5AbLccEkSiDE1bGKKRxa3PiC9xbl58JPr/KA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=redhat.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WJcob8QYfo5hlHn0Ev0TZ2EMAwugli1iZYAf5iub76M=; b=lEG7AklUf2ckzhoNUCjZ0fiO83lvuNbGKDEaMO9PfXjXPzUUi6klsB8fbSaVQiI7qTbJYxMMbYuooWlCmFgp2qcpys2uxCWkdNK6JdCTbrRsnRCMDn7LKc1yjZuQM7CjYTlN+hZvANxHehra2qiFnuia+d8yAHrlcrHTRIktuGLgf8XzCDlBYUKXETVUn8zlq3TgMojcK+lzh7zIIhscPYZnsfWd39026H62OMSLk++Qe08bUmlI6EYoLL9/OMnpAIr84rXRpRFeKT+1l2/P3ZGNw2ZRvmeTUW+C2pq6+p7hsGDvis4ve9X4+DcKBIVR31FLQProCWlISAPaW2+ypw== Received: from CH2PR14CA0038.namprd14.prod.outlook.com (2603:10b6:610:56::18) by DS0PR12MB8344.namprd12.prod.outlook.com (2603:10b6:8:fe::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8835.29; Sat, 21 Jun 2025 04:21:25 +0000 Received: from CH2PEPF0000013F.namprd02.prod.outlook.com (2603:10b6:610:56:cafe::6c) by CH2PR14CA0038.outlook.office365.com (2603:10b6:610:56::18) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8857.26 via Frontend Transport; Sat, 21 Jun 2025 04:21:25 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by CH2PEPF0000013F.mail.protection.outlook.com (10.167.244.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8857.21 via Frontend Transport; Sat, 21 Jun 2025 04:21:25 +0000 Received: from drhqmail201.nvidia.com (10.126.190.180) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Fri, 20 Jun 2025 21:21:14 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail201.nvidia.com (10.126.190.180) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Fri, 20 Jun 2025 21:21:13 -0700 Received: from localhost.nvidia.com (10.127.8.12) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14 via Frontend Transport; Fri, 20 Jun 2025 21:21:13 -0700 From: To: , , , , , , , , , , , , , , CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v9 5/6] KVM: arm64: Allow cacheable stage 2 mapping using VMA flags Date: Sat, 21 Jun 2025 04:21:10 +0000 Message-ID: <20250621042111.3992-6-ankita@nvidia.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250621042111.3992-1-ankita@nvidia.com> References: <20250621042111.3992-1-ankita@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-NV-OnPremToCloud: AnonymousSubmission X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH2PEPF0000013F:EE_|DS0PR12MB8344:EE_ X-MS-Office365-Filtering-Correlation-Id: ed5e89f3-bce4-4bbf-20ed-08ddb07b1644 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|376014|7416014|36860700013|1800799024|921020; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?Lc19nwvt/DEKAM8dXygyOKID5wi8w28BheC4i35UyJwJQvIHFqNZG1WGMChC?= =?us-ascii?Q?u1sdElFBuFl+i2bSxOAjOXzy274XjyPXRRpsQjAYp3+//Llt1R8RSsYZedH1?= =?us-ascii?Q?4Uz37G7CWDQe3OZGkVHkp9+9GR3egwyAhPRgAn31OuBqOSkNjyVRYoi/jVS9?= =?us-ascii?Q?FjCwDoHlZSq64E6lxLdi1JiFgHaHR9+Pv0hNHwlA+I33Mfvu4buIzMi6aCRK?= =?us-ascii?Q?ZlXwYGIOwYAPxrvFTkKZ73IAGSs+62itl04WcK5kEuX+e5+zFvfkOk7EhGiB?= =?us-ascii?Q?M2VUVGL+PpsrFlE8AXdYxnQtQPk48lx96wiThOqvKOIPxwXwofMLkM1HPfh/?= =?us-ascii?Q?xi2Sb03QU80tQBEEJiUnrr1NPKm1nrnesO/Hq4UfGIJETnz0mEYceadvukRz?= =?us-ascii?Q?otEOew8bCJ+eF0VgiSm7zITWNRK6jBuumKnPCohOWiKkW/jYtQX0m4aK/ps4?= =?us-ascii?Q?VRqPNPTJheMEX4nDK1j7cnhM271w7Ng9Gm0P1A8SEm5uMqSMK7tRPABL476r?= =?us-ascii?Q?SBfbyN6I3Qn7eQZi6Kabvfo6LKqxjy0A3eoSYHiJ77EOXYA6WYM8CC0k7lZz?= =?us-ascii?Q?hcIoA+OMf5xlujgeBXGdnBgbhtKsKGIoj48FQftI0kpAswYtXqK0yLhjt6K2?= =?us-ascii?Q?C/J7qB91RitUOjT2/iDpDT7d24WcralfqdYUJld2CtHr6Ws1D5RzgAsXSF8V?= =?us-ascii?Q?dsHtKTzyC0p1XQixxwXIQ0EfoLGDbWSMvV3i7LMuygJzpw5xDn/wwZJMGxfT?= =?us-ascii?Q?z/h178tJcE1wTbSzRWOSLwVvW54/nZFjC9JLKLfykkwVAQIUzWhO5Pa2gw/D?= =?us-ascii?Q?zZcLEj8RUzEZ08KW0b5suVkbIwTqxpUPh/lBzBrpd2+yOiUK8RNEzHLIvG7W?= =?us-ascii?Q?kW+xgEpDx2v/gN0B93Pg1qV1tKWmFbp0uP4UpLvIQpJDEZH8xvbJbhaGtYyn?= =?us-ascii?Q?hc0Znzwf9wuTsdWyACWg59O+oVthQGHP9XZETMAv418/ebRqHZPKO6pV26Aw?= =?us-ascii?Q?GW9ZU1Yhs74DBxoGXWUNglOQ2bJ/V3z+cZHZv7o2dk7CdO4JIyXVz07omHK4?= =?us-ascii?Q?s8mjYOih7pmSKFBkfAXvzYuNtIktQTo4GxGA7Vxv23XiFecLP433dkLwt/jU?= =?us-ascii?Q?vD4nny5ZtpC7HF62ir3YhK+iod1qJHYTpUgaO7MXC5IK1fK8srHYAez7NxBN?= =?us-ascii?Q?w0XtJhOzoP0myWoN+XOMce8WceOcv4r4FjfSaqyye34pfJLDm5qUsk6jWYuJ?= =?us-ascii?Q?hQYleu+ocA97r1udHYEzmyN0cMc/c0C2StJyOPA9dGWxfAiT6YIXC4Y/dVzR?= =?us-ascii?Q?877xqWnElgsxAvOGOdXosq0zrWyMQd42ORyMDmiOKs+sJEYRiellTjFUAiZy?= =?us-ascii?Q?gHUGEqXTSk7jl2DO5RGXLdtJKTYIiwcGqoNaISnGrpRiuAy6TLV4+UY3NMah?= =?us-ascii?Q?qEyVnfPEamSq8/o99HEV+6fg5H0X1fFLULesyYnaFEU8rBnr8hespWYM2G2p?= =?us-ascii?Q?4l8hrLeDxKtntw41BK9IhodQvhbywsw7Jv1ZrnEBus4/4uDswL94HwDELw?= =?us-ascii?Q?=3D=3D?= X-Forefront-Antispam-Report: CIP:216.228.118.232;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge1.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(376014)(7416014)(36860700013)(1800799024)(921020);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jun 2025 04:21:25.6850 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ed5e89f3-bce4-4bbf-20ed-08ddb07b1644 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.232];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CH2PEPF0000013F.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8344 X-Rspam-User: X-Rspamd-Queue-Id: D62A340004 X-Rspamd-Server: rspam10 X-Stat-Signature: 9wftabi1619ftk1uft1uy8k7uwamh4p1 X-HE-Tag: 1750479691-246998 X-HE-Meta: U2FsdGVkX19LtJE2jzS3TGTi51QXJttARJwkn7jzPnKUVRmev22tp/z7CoV3DhZNNMVExvEzVpAEBMi/tXc+UDbghvq9+y7rYyeqtqz87jXcD1/fsqyXxhstAzP9MdNPhJCrBdmInfJmhQyOd00042od/CbV008FvrEW+Yji7+1ykRRBqdvggZGriajNrIiGFA8YLP4AFXQtOrJqDnQdMxS8TAxl4x6vN39CYdRnB0gxkpDDIF/frRtwLR99aBERBpZrXCIte5FV1QLp19zVA4oojrw7gGV1fd2N7uI+tBYszUGiqFvwwMxLUTpygkrZVb11N5aj6PlZeLPlmcpUTu1JMWPCMMjhJoY6djyOrYWehm1hFNuhNWUQC8fEAUkmdzDxZou1obBqVINcuB3ArZMkKDTT4YY5U6T0J28omtmaj0nlizw3XbrUeJ8LcSkmGJc6KrxlkBO0aGgD9cFXhiSu1DzrTpwxMRvq4LRFeFJua6Q5kvTRTgEuIjQqUT7ojsenvyBNBsH/0M+G5Cw303r2Rfix4OMR9xophxl/Z5t0pp7mEYkRZ0XIYfmnjgxgSUA1WMjzUGDhgTGRTlLCXu7GeVLy+JFYCePJlJWfnKTQ4MEP6qqQr7ZPQg+wKpSbcAU4jbQ7a8W6mXRPUHOnpTsLC9bI3TjenuqIPPzvbBOxHq7CDALHOCqE6fse+c9nzhjtUTPrdJkj9vn5yENjIcE8ywV+nSuL5a8oFcBgmWKJ+FafemOaSMADR5RqbAJCWbFKJBsAzEAccHXRBFuwzjhD2mzmgShQvjfRlBzNWtU7Qk+0eauROFwQhR74VIaOiPTiGpjXr6CntTyAO+3KOTsTd2JPyvdU37OAOma3eo93LbQ5EvElLJ1W5QSPr4RNT2x/1Jzgxq8gLS9qFFLMHuRLNft4IR1oZqG0tCqCrOFSim9sxA3LgSE1CBKjvkXJkqYKS2p82P5/yqJ3LG8 VqJFF1sx BLOpSYfuwzQAfT8jRmp24RkIcUXBljBI1CLJobcey3ODv51VLsIL56dE5Vcah7Zg2LBF25coH7HsOWN587bGy93R5v83yReYbMlK/R6k9fGnWc4H5ihWEisYO0g19tgapYCuc4xdpif1RU8CcAIBcmz7vbbcy3IvXOE23ZDSncSFnDAg51gFOLj+eJaB+/uFO3E+PUgSWcA0Rho7nclUXavxmfA+oLA/CNbLh2GHmSj219/JZ/o8fBZeR0l1ZZ60sF83sxCTQcSFndZczKSwaatpwF7rmQibUWhpxi8veeNz/+/ps7v8acEoRaKPQMLTfCM0SFBJZ/1JAkVUhX7ONJOeZQNWwXbgJm+j+N7SmwoTntbhTdsdYBk1Q0KfIhplD8WNKt68q63CpmgNE4/kOnBP7bx+aa2LfOf/bsj6r9pL0ZDDJdHMyigy6tuyrwT09DSO4ywEy70mNGqeaMtN6LQh1FN3BNRnJz5uqApoC0nWHur7YW5QBYnEGiv2gXJoRqqrq728mGf31fTtyTvGyVYzT8g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Ankit Agrawal Today KVM forces the memory to either NORMAL or DEVICE_nGnRE based on pfn_is_map_memory (which tracks whether the device memory is in the kernel map) and ignores the per-VMA flags that indicates the memory attributes. The KVM code is thus restrictive and allows only for the memory that is added to the kernel to be marked as cacheable. The device memory such as on the Grace Hopper/Blackwell systems is interchangeable with DDR memory and retains properties such as cacheability, unaligned accesses, atomics and handling of executable faults. This requires the device memory to be mapped as NORMAL in stage-2. Given that the GPU device memory is not added to the kernel (but is rather VMA mapped through remap_pfn_range() in nvgrace-gpu module which sets VM_PFNMAP), pfn_is_map_memory() is false and thus KVM prevents such memory to be mapped Normal cacheable. The patch aims to solve this use case. Note when FWB is not enabled, the kernel expects to trivially do cache management by flushing the memory by linearly converting a kvm_pte to phys_addr to a KVA, see kvm_flush_dcache_to_poc(). The cache management thus relies on memory being mapped. Moreover ARM64_HAS_CACHE_DIC CPU cap allows KVM to avoid flushing the icache and turns icache_inval_pou() into a NOP. These two capabilities are thus a requirement of the cacheable PFNMAP feature. Make use of kvm_arch_supports_cacheable_pfnmap() to check them. A cachebility check is made by consulting the VMA pgprot value. If the pgprot mapping type is cacheable, it is safe to be mapped S2 cacheable as the KVM S2 will have the same Normal memory type as the VMA has in the S1 and KVM has no additional responsibility for safety. Checking pgprot as NORMAL is thus a KVM sanity check. It is possible to have COW VM_PFNMAP when doing a MAP_PRIVATE /dev/mem mapping on systems that allow such mapping. Add check for COW VM_PFNMAP and refuse such mapping. No additional checks for MTE are needed as kvm_arch_prepare_memory_region() already tests it at an early stage during memslot creation. There would not even be a fault if the memslot is not created. CC: Oliver Upton CC: Sean Christopherson Suggested-by: Jason Gunthorpe Suggested-by: Catalin Marinas Suggested-by: David Hildenbrand Signed-off-by: Ankit Agrawal --- arch/arm64/kvm/mmu.c | 61 ++++++++++++++++++++++++++++++++++++-------- 1 file changed, 50 insertions(+), 11 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index d8d2eb8a409e..ddcbbcc8a9ae 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1681,18 +1681,53 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, if (is_error_noslot_pfn(pfn)) return -EFAULT; + /* + * Check if this is non-struct page memory PFN, and cannot support + * CMOs. It could potentially be unsafe to access as cachable. + */ if (vm_flags & (VM_PFNMAP | VM_MIXEDMAP) && !pfn_is_map_memory(pfn)) { /* - * If the page was identified as device early by looking at - * the VMA flags, vma_pagesize is already representing the - * largest quantity we can map. If instead it was mapped - * via __kvm_faultin_pfn(), vma_pagesize is set to PAGE_SIZE - * and must not be upgraded. - * - * In both cases, we don't let transparent_hugepage_adjust() - * change things at the last minute. + * COW VM_PFNMAP is possible when doing a MAP_PRIVATE + * /dev/mem mapping on systems that allow such mapping. + * Reject such case. */ - s2_force_noncacheable = true; + if (is_cow_mapping(vm_flags)) + return -EINVAL; + + if (is_vma_cacheable) { + /* + * Whilst the VMA owner expects cacheable mapping to this + * PFN, hardware also has to support the FWB and CACHE DIC + * features. + * + * ARM64 KVM relies on kernel VA mapping to the PFN to + * perform cache maintenance as the CMO instructions work on + * virtual addresses. VM_PFNMAP region are not necessarily + * mapped to a KVA and hence the presence of hardware features + * S2FWB and CACHE DIC are mandatory for cache maintenance. + * + * Check if the hardware supports it before allowing the VMA + * owner request for cacheable mapping. + */ + if (!kvm_arch_supports_cacheable_pfnmap()) + return -EFAULT; + + /* Cannot degrade cachable to non cachable */ + if (s2_force_noncacheable) + return -EINVAL; + } else { + /* + * If the page was identified as device early by looking at + * the VMA flags, vma_pagesize is already representing the + * largest quantity we can map. If instead it was mapped + * via __kvm_faultin_pfn(), vma_pagesize is set to PAGE_SIZE + * and must not be upgraded. + * + * In both cases, we don't let transparent_hugepage_adjust() + * change things at the last minute. + */ + s2_force_noncacheable = true; + } } else if (logging_active && !write_fault) { /* * Only actually map the page as writable if this was a write @@ -2269,8 +2304,12 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, break; } - /* Cacheable PFNMAP is not allowed */ - if (kvm_vma_is_cacheable(vma)) { + /* + * Cacheable PFNMAP is allowed only if the hardware + * supports it. + */ + if (kvm_vma_is_cacheable(vma) && + !kvm_arch_supports_cacheable_pfnmap()) { ret = -EINVAL; break; } -- 2.34.1