From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5BDCAC369C2 for ; Tue, 22 Apr 2025 23:36:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 911B56B0005; Tue, 22 Apr 2025 19:36:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8995A6B0007; Tue, 22 Apr 2025 19:36:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 69CE36B0008; Tue, 22 Apr 2025 19:36:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 4CB426B0005 for ; Tue, 22 Apr 2025 19:36:08 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 0355A1D0EE1 for ; Tue, 22 Apr 2025 23:36:07 +0000 (UTC) X-FDA: 83363290416.02.43EC2C9 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2068.outbound.protection.outlook.com [40.107.94.68]) by imf17.hostedemail.com (Postfix) with ESMTP id 2D3054000D for ; Tue, 22 Apr 2025 23:36:04 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=YwV+Ee9J; spf=pass (imf17.hostedemail.com: domain of jgg@nvidia.com designates 40.107.94.68 as permitted sender) smtp.mailfrom=jgg@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1745364965; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HlygJG8kSWtI1KaCbzDPKc6kOHVZQjMz3VIqH+Xkceg=; b=qbVo8UGkgS7ICMnzzbZ1NJLh1bYZeaFcjp8aw3NKwBLCb8ZsoQQM/iPQv1qnDILwozsdCc IgHdY5r5x4V/y2QcpUn16IMYTrsSOar8Sols3TGKMNOcYDxOPaEBmz/NLfWdF/MXmWj0CM +oaAxcdgOMjmbEXixtDuAfj6Bv4Zs2o= ARC-Authentication-Results: i=2; imf17.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=YwV+Ee9J; spf=pass (imf17.hostedemail.com: domain of jgg@nvidia.com designates 40.107.94.68 as permitted sender) smtp.mailfrom=jgg@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1745364965; a=rsa-sha256; cv=pass; b=TSgfE+3XbgVp8K+PIVbk8a35EDoaHqJMHynNC7L7euYb9wihUhaaYH3wpU3bc/wkhqpvLz Bc/FQIyc8b89lRbiSUlRGcZZu2liclpomWR/IbkchFwZmXbLoX2WxV7RcPBktg6zJwnzJz FdkMLe2aJQQtlKNN92VL2O2yFk+dJKk= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=b00a+D2U6U0ku1AMpbcitEQLoseK1+PbZmztu6hLOFjLQ1cnS+h9C6zH8a2h5n0/S4wJrjyHRgLNbw7JWoUADo5u7pI+Lmshj9h0L6KQ/qNjuqHOX5G2Vw7i2XKeGsLCACwu/9HA4pfreQ1iujlOwTxSMTmgcdIC+WI+NJFDiAyqYLzuxwMAIUkdGUrU1GhxqriG4nl7kkO1U11o3vJZiFSL+hrqy5NcEboVjCwn1CEIHxIouQkdWErJkkfXHMYLsJhpIa6+BSbWPyUww9BRh+4FPbhk7jDsJlA6gbVSnqL8nzi6WFJ2ncTPt8YP4XH6KUBTgVdz/EYiMNif0+90cw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=HlygJG8kSWtI1KaCbzDPKc6kOHVZQjMz3VIqH+Xkceg=; b=MDSEacMHReG23dRn+7SXCBzLFOiJOvdSVc/Afghs1ESKch+HUGreF+2yD5Mx+zIywxQ2DzeMUTPl5eSedKqfBJ3PTTEYaYgqF2s3/xn55Zw7xABq1czH43i1z5qTil5/33lEbZLbYzRHnkNS0qXS7kjDKHa7bCgBOfoxiOOjEF8/MquiB8yogyuIM65t2jSkE71LMjGVGKyb0ZUfzmK80Kt/6qxyU320CWB2khZdiFZnMZC1YymwMsvX8tmir8txIT3KTt+BrwfVmPKJmWATj5OY4wMldVYs9UjVwT8GBXNJO7k1H2FEDHYMR5JsO88QzbR1ntoCQdvJRz1fQ0/Mww== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HlygJG8kSWtI1KaCbzDPKc6kOHVZQjMz3VIqH+Xkceg=; b=YwV+Ee9J31HEhhIFtiXCk2Q9FjKBa0bWL54iXRIXaPtNluGBkarqSQq6Bow9q+Bcz+XTai53BFQYAhbmKF2yVi6EG3WyZdTfDSOTHqSTy6tlD6WS4nWrtWVzj18MLERVlkIdw7PunAECWMtnP4ADGYF9Flt1M9pWVGO7eI9ZLeNbQiiE2xiyKixZhDEkvjTIQo1QcK8HyfmX1hD95h15cDR5YxYpffkXRyz57umwrqxXqwO2+RLPNRs9FoiYrx9POdEzTdtv8plw5xARq3KnLKubwDr7gqMQlzkYgBdQHMt4peEuWitdqtoiuHZ8NxdYtmlanOmug9FsS8kuP6m78Q== Received: from CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) by IA1PR12MB8407.namprd12.prod.outlook.com (2603:10b6:208:3d9::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8655.35; Tue, 22 Apr 2025 23:35:58 +0000 Received: from CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732]) by CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732%4]) with mapi id 15.20.8632.030; Tue, 22 Apr 2025 23:35:58 +0000 Date: Tue, 22 Apr 2025 20:35:56 -0300 From: Jason Gunthorpe To: Oliver Upton Cc: Catalin Marinas , Ankit Agrawal , Sean Christopherson , Marc Zyngier , "joey.gouly@arm.com" , "suzuki.poulose@arm.com" , "yuzenghui@huawei.com" , "will@kernel.org" , "ryan.roberts@arm.com" , "shahuang@redhat.com" , "lpieralisi@kernel.org" , "david@redhat.com" , Aniket Agashe , Neo Jia , Kirti Wankhede , "Tarun Gupta (SW-GPU)" , Vikram Sethi , Andy Currid , Alistair Popple , John Hubbard , Dan Williams , Zhi Wang , Matt Ochs , Uday Dhoke , Dheeraj Nigam , Krishnakant Jaju , "alex.williamson@redhat.com" , "sebastianene@google.com" , "coltonlewis@google.com" , "kevin.tian@intel.com" , "yi.l.liu@intel.com" , "ardb@kernel.org" , "akpm@linux-foundation.org" , "gshan@redhat.com" , "linux-mm@kvack.org" , "ddutile@redhat.com" , "tabba@google.com" , "qperret@google.com" , "kvmarm@lists.linux.dev" , "linux-kernel@vger.kernel.org" , "linux-arm-kernel@lists.infradead.org" Subject: Re: [PATCH v3 1/1] KVM: arm64: Allow cacheable stage 2 mapping using VMA flags Message-ID: <20250422233556.GB1648741@nvidia.com> References: <20250331145643.GF10839@nvidia.com> <20250407161540.GG1557073@nvidia.com> <20250422135452.GL823903@nvidia.com> <20250422170324.GB1645809@nvidia.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: MN0PR02CA0027.namprd02.prod.outlook.com (2603:10b6:208:530::30) To CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB8659:EE_|IA1PR12MB8407:EE_ X-MS-Office365-Filtering-Correlation-Id: 2deeafad-6380-42f3-4b09-08dd81f66ef3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?wFUOLerbKKVxL/ebfLnMIv/ADXb2Q9JEHWyNMLqOuNWyXYsLJrut7FeSSPs2?= =?us-ascii?Q?rtKh8vqJDvxJLAQAljT0wnJZAJYL4jxC5QYJX9PuDJGebca0olxTQjwWTPvF?= =?us-ascii?Q?9e1weMlZAa1C3blEo4/1qkVAWpDZsQtkFF6IN+ttJEGdn/4CX9jt9HPSOfog?= =?us-ascii?Q?Ahcd+588Qt7pHey1sxcdMOj2B60ZRX1i8lESVx56zoll7lbu7r+LqFVmEAF+?= =?us-ascii?Q?emFR0lfs5J8UBbxkIqn8OJH/T9VaN8GdQMj5quDpA97pLgtz5Z8DhuvR4ujk?= =?us-ascii?Q?LGy3PBxVWY1LqdyblleYeqJy2rrAPZgaDNWGb1RQofeelPylkACPatCeaxvY?= =?us-ascii?Q?kOPrSYRd9yslzOs5rNU8SCZcXHIOjLwLUnSo0uWMEtEA8faOdHWj8LjQ/wUt?= =?us-ascii?Q?vc2HZAySTLuBdfVV9ciXp4xjrBriXS83oHdluXdYIG200Z+dyjk/r2LZRhGE?= =?us-ascii?Q?geUNlXKGcXefNKlAtyQSWdhyuH7iyB6DN2Pfkp+7sTm1TMlsHITxXvZQwc4c?= =?us-ascii?Q?gzEVH0YeAVIURqaTy1XD4VlFLqPZkIrq5AvcLuyWbJg2eMQWsaRxoqHYeWXe?= =?us-ascii?Q?i0oGL5Zk5KjjWfWhivV4EfK0fxq8JX76lOpg8nLOy3dBs8eGKTdJhGee+4KY?= =?us-ascii?Q?YoGqQCbbOs8qHWD13qhjR7fp8s2joimiTYxxOx43pjehBRpZq/eJMHjSSDKX?= =?us-ascii?Q?JL5BDDa5Zvw84JQKxS18yAbzI0WNefFmGJQApELy0o2xP0VP74kj2tg44Vc3?= =?us-ascii?Q?oyH2A1ThZJwcoNCa36g9k6r6WWKKxjsezApFlcHFSzUCPKdJN99rNXoZi1V/?= =?us-ascii?Q?aMBGfkEwiEW+8WitxuA35wWd7dAWwvQfQzD+jpBBQpz8rxcJels2tyv8ZZ+Q?= =?us-ascii?Q?Cm9s0wajI1zu14M5p7MOMKDcCWJmgRKJJIuAVA7PxegLCznr1axM5SYO21zu?= =?us-ascii?Q?bHe7hboPqwYKDe8J1IHz5yaukACmdseDHUTapwoVJ+wqnaTM6610TkMHWYeZ?= =?us-ascii?Q?xOpguhWIh/mBfvnuOqwjE/cuhUXHzAz9XkMXHuvVAgQn1Xr9s7S5QuzApjqB?= =?us-ascii?Q?0ea9EXdARLakmYYnUwQS89X7miisrkZ0OSAm0niTk6xaQPegelWgCWQn4Gdc?= =?us-ascii?Q?h/EOXV1iNB006VENw08jyBxO0xOCTXMc6+Tfr64wX43CDJWXuXKshg9nV9U5?= =?us-ascii?Q?VOSF80qqlrfvpsHcwP/8Nu2sxWD3iCmtarPqNOLp774FiRzFIr6uo4aa8bdY?= =?us-ascii?Q?n0DLVKlLMmhGhzbArW1oqjGfQzTbfT2lACmbroH18akz2DuRiBmYRBf9GqmM?= =?us-ascii?Q?2Tpt0EyJ/1lP1j21b4srleVRONX6rxJVq3RW+EZxfTTl0iraiIUlJPpWpslD?= =?us-ascii?Q?KuqfhaJymBo+NQgSf0iKuXU76hOuS044nM6QG0C7AI+BTO27G0DfwrZToooi?= =?us-ascii?Q?h8fyc3nXsvg=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH3PR12MB8659.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?PeQq2H1WXdzKei0NZbjUlczjYgjj0+oZcnMXV7XtmBglzwMeUnNVYt4mCxhQ?= =?us-ascii?Q?qNqdyCOH7E9h2Diab7KArKbB9jWztxrXf/OfMo5ncdunKnB1kZ2fldfP3eKI?= =?us-ascii?Q?+uDFo2Eb6Be97Z/4ZeTXDQoSgvW/gfFUWXd3kYPYsk1MdCKWDXoqmL4wBveJ?= =?us-ascii?Q?Za8yUgpPCvC9wxek2H/hP2fRcXAr+cigWjl/1D3HbL2zhIuT5G2/a4HYiadv?= =?us-ascii?Q?r3Kq+sc13Ht7VIKSdPMsta3q7Sx0Pg6iU4WBARh5EJyiah9IFWJlvO4cDSCj?= =?us-ascii?Q?BmmxaZkEQumUQTr1Ucxd+8UBSGHrGTo4g4PGQAJtrMgm7B9hCAxOyAcvGEVz?= =?us-ascii?Q?BJIWRd5rEXUYDPnjysu3WKj7F/CGsahYrsDNk5qLuaXZD3Zxw/OUoTmXSjR1?= =?us-ascii?Q?ZkuG2SUuFjyiGdM4iJXY731YGXa/xPpQrBr8OmOnkmgAt9IqcHgKW2YNRPG4?= =?us-ascii?Q?xelYePc9gs/OnY2n/ObC3z2epqqVqFZsCxKk1O/MxnaRoeCNYa7vqRql4ex6?= =?us-ascii?Q?WIj7LM5YEunUgySqXNNz9mQMjKhDwaVyT2gm7IfvbAdjvMXv5ESViJFi07b9?= =?us-ascii?Q?t7rekYC6iO8bCZj8rnOfWcLn1hQH2iLo97nq01sNnFMtgGXhNpadu/lYx+ZD?= =?us-ascii?Q?eAYcUzOPK2G8wKXzl+MX4KHnErKWCwXN6w3kCvOhIKC6FmEJPRikeKY2wC7w?= =?us-ascii?Q?6v4Tj5+mY/VoAIFb4QLCO86aCcK9kwO3JC3EJ/jRDObyzBO4xY/Lzcnl5fev?= =?us-ascii?Q?CHxyGEtx9yb9/GUtariWEjrkZVp175WgWMwYI2aDm5Otn6ILxP58J65y6zMk?= =?us-ascii?Q?yKbmQFVo4XbN3awuamPEgHixzFQ3B8fyDKW2MuZtpq5QcJTr/oSUrhee6xz7?= =?us-ascii?Q?4gjU63Zu3v+n0ypVbwPYJXtxj8alzDaRk8LvfNvYBGW1mk4MhfY80ib82RaM?= =?us-ascii?Q?U4hYxmXWi/lR8PIe9frtvfrng/EQfH4FnTrMPxX5gK41lyN3DMhJPE+3WTtI?= =?us-ascii?Q?sl3QyfV9xlA45Svmm/4phbvwU8/eY2fpOE2pWynhg6FBNYUvl1ibihpNOyf9?= =?us-ascii?Q?bEPu7hRnQowSemVPAFW7R+UOCzn5pGfLmc48hNQPV3NnjrWNrgX5vfvz9Lb+?= =?us-ascii?Q?DVcCXxm+fEzablZnf9Bcsz9b7Tg/dtAVpIcPuimPBOAwE61KcUdEy6bo6QJy?= =?us-ascii?Q?Xg9TTdsIAWVgTqqvkoXliEeLUrpqhFoC7IGHCENgd87skcuLEP5yUR3xPhrS?= =?us-ascii?Q?sr4TN/1FgGXNF2NfyERHRa02r2m4Pp8hY5Sos8oG5estud4xdGC8qaNOXr3C?= =?us-ascii?Q?l2R6Y8ngHqTZh05wQPAB7V7SYo4L4SsQN+VHqVFm52lhikUS9UdvGmUE2iYd?= =?us-ascii?Q?anW2UT9pG18op1MYm2IQKY/p/BSdO1t7tgjxhdjOAFD1TdhHinyK0l70xgNQ?= =?us-ascii?Q?+DNzlBJUh9nGrTAym3B3srSG3Y0vkr4mHsMF0dVUfEaOeCfHjLjphlpBK4YV?= =?us-ascii?Q?YPKJYqLtyRHUqq7uuZ4exGOUQj23xrJoIKaHfz1wB6UwSjOsXpqlnQOfLhVu?= =?us-ascii?Q?B+tJjnNQCDSpr9lI+202M75JhO5lLzCkJmm7AG6Y?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2deeafad-6380-42f3-4b09-08dd81f66ef3 X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB8659.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Apr 2025 23:35:58.2902 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: j6NmobJ1rYOVoncfNDFAKdbJioU7o5hnr4FrB0wvIzH2/xrKcIF3YrMuFU/t89MI X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB8407 X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 2D3054000D X-Stat-Signature: yzgf4scou93rek7z5bokh36ji4okg1t3 X-HE-Tag: 1745364964-553486 X-HE-Meta: U2FsdGVkX19rlqXlGXQ6iMbk7rI6kDDlstbJHSHnkPXEbhpnHBZJ4uZ9Xj5w7FU7C51v9cDNRHsHFGPGMr0aL0kpk772PNH/03dFNchEG2GcL5UQV2ZDbTyHpnrUz+tvi60Vmx8sRkY+54PcbXPQW/FdYbyOO/FgmW/k/GbSjVC/xjjr52tRflLDp7UUrMtuNQb87WrpcTww00LNAHbqP4Urn7AYLoPWmQt6WrZ1GITx15ZBz0lkR976GExGEGCnQcZMMc2NyuLBThrfewpMPa20lPtDv/Y2lYjztW55SjyG3qvicFdqKkQGz1255dSQDQGYfaIWi5sV4xQr7Y1raq1TZkt6KlAtpX8bK68DnKj7PhNTC0KJRbUyfa6XpHQ9t5oNFASt4Rwr5cQtU4Do9Rk1B6t2eqO5oe3UKJRjYaRduQbi+jkK+EJU+el1+FdW9wXIdNc6ak1sZQ7WjoESEQla582qR6r9p4kLxO/hw3yhIZ5/uGo8sFw28ezSAv6grYp0hUCz8fbTSjSkzfn7TsK1GwmZk/tuVoDXBJGlRSjSGejx4S5Ei4iLiGuaeva5xhrI3GCX10wsuF43JN/X1boYuTQWW/N5ieE6fDpAOqfY2lWp8b5nlfgwSaYGvLbsJald8j14OhKJSw1cF7MxAWqH880DHxk+9sv0coYEvcHfWWHkonL0piX7urIoPjP4qVLNLOKgWJy9ORT85KuC2nAgo7UhvTFp2M6IobyCPIl3gEK+gF15abdeBAl/5n57z7zDZyzFcOGGHqODLk3nCN0BPzXa36cmfNJ04mIOwL/l2KPBdvK9buPsND7ef9PDMyD6NUoHgKoiovdv6NkIQpa+O7jFBEh9QKqn7cpc4L+bPmcvqP/rgqV4zgYa4ALnj65OwSp/SHpnoahln4ZFLrqO0aqz9bBjmRSXP8qbP+nh2X2w9mHI9vlGSt/nV+eOnML9Zpc9HhElSp/jpwq jKKkzUKX S8k9NFDYi7Q3wNAES5JZ+ZkdgGPKncGqnGjIntnSX3px/YDl3+c6RYNOXcGbz8u9RA66JBub2pjgsqBuD7iOTuih5IAIbelf1xPsonmdshlVSCO+Uph7M8A5Ad1WKQkTInpiEXhFSLO/JknQkEOYyRwwtA46PtZp2A/iRqFzMzp/2W25EdxgEOHkShSw/KJ0Q+pvWluJoaQ75F4SGm0+skWIEzl8OzZnyNdngnGEypsLRKcqPl7wWvdF1gf7xQKFM8PVZp1CGDMi4R1ldu1l4Vc26pooQQnJUBd9jTptQN80auO0BorhhhRMVlEINXQ9rcEyrPk2R96tceyYi9dwepJKRoX3d9FhocTjsKwxJ9KLeem8zti1trdLSvM6hXbDjuj+zS8syx+/Kfp2GE6crAsCc0de+wGQAeVZOlakrQ1P71MEDynqyn7jZeDh4FdUq6rUF X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Apr 22, 2025 at 02:28:18PM -0700, Oliver Upton wrote: > So, if a VMM wants to do migration of VMs on !S2FWB correctly, it'd > probably want to know it can elide CMOs on something that actually bears > the feature. OK > > > - The memslot flag says userspace expects a particular GFN range to guarantee > > > Write-Back semantics. This can be applied to 'normal', kernel-managed memory > > > and PFNMAP thingies that have cacheable attributes at host stage-1. > > > > Userspace doesn't actaully know if it has a cachable mapping from VFIO > > though :( > > That seems like a shortcoming on the VFIO side, not a KVM issue. What if > userspace wants to do atomics on some VFIO mapping, doesn't it need to > know that it has something with WB? VFIO is almost always un cachable. So far there is only one device merged, and CXL is coming, that would even support/require cachable. VFIO always had this sort of programming model where the userspace needs to know details about the device it is using so it hasn't really been an issue so far that the kernel doesn't tell the userspace what cachability it got.. We could add it but now we are adding new kernel code, qemu code and kvm code that is actually pretty pointless. > > I don't really see a point in this. If the KVM has the cap then > > userspace should assume the S2FWB behavior for all cachable memslots. > > Wait, so userspace simultaneously doesn't know the cacheability at host > stage-1 but *does* for stage-2? No, it doesn't know either. The point is the VMM doesn't care about any of this. It just wants to connect KVM to VFIO and have the kernel internally negotiate the details of how that works. There is zero value in the VMM being aware that KVM/VFIO is using cachable or non-cachable mappings because it will never touch this memory anyhow, and arguably it would be happier if it wasn't even in a VMA in the first place. > This is why I contend that userspace needs a mechanism to discover > the memory attributes on a given memslot. Without it there's no way > of knowing what's a cacheable memslot. If it cares about this then it should know by virtue of having put a cachable VMA into the memslot. > Along those lines, how is the VMM going to describe that cacheable > PFNMAP region to the guest? Heh. It creates a virtual PCI device in the guest and the space is mapped to a virtual BAR. When the guest driver binds to this device it will map the virtual BAR as cachable instead of as IO because it knows to do that based on the vPCI device ID. If something goes weird and the guest tries to use UC instead of cachable the S2FWB will block it and the guest will probably malfunction. CXL will probably have the VMM understand things a bit more and will generate the various ACPI tables CXL uses. > > What should happen if you have S2FWB but don't pass the flag? For > > normal kernel memory it should still use S2FWB. Thus for cachable > > PFNMAP it makes sense that it should also still use S2FWB without the > > flag? > > For kernel-managed memory, I agree. Accepting the flag for a memslot > containing such memory would solely be for discoverability. > > OTOH, cacheable PFNMAP is a new feature and I see no issue compelling > the use of a new bit with it. Feels weird to me. Now the VMM has to discover if the KVM supports the new flag and only use it on new KVM versions just to accomplish.. nothing? > The entire reason I'm dragging my feet about this is I'm concerned we've > papered over the complexity of memory attributes (regardless of > provenance) for way too long. KVM's done enough to make this dance 'work' > in the context of kernel-managed memory, but adding more implicit KVM > behavior for cacheable thingies makes the KVM UAPI even more > unintelligible (as if it weren't already). It is very complex.. I'm not sure adding a flag in this case is making it any simpler though. If you had a flag from day 0 that said 'this is a MMIO mapping do MMIO stuff' that would be alot clearer about how it should be used. But here we have a flag that doesn't seem well defined. When should the VMM set this flag? > So this flag isn't about giving userspace any degree of control over > memory attributes. Just a way to know for things it _expects_ to be > treated as cacheable can be guaranteed to use cacheable attributes in > the VM. Can you get some agreement with Sean? He seems very strongly opposed to this direction. Jason