From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F2EFC02183 for ; Thu, 16 Jan 2025 18:14:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A9C016B0082; Thu, 16 Jan 2025 13:14:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A24BD6B0083; Thu, 16 Jan 2025 13:14:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 877DA6B0085; Thu, 16 Jan 2025 13:14:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 668F96B0082 for ; Thu, 16 Jan 2025 13:14:07 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 15E59141023 for ; Thu, 16 Jan 2025 18:14:07 +0000 (UTC) X-FDA: 83014114134.14.B9B1F14 Received: from SJ2PR03CU002.outbound.protection.outlook.com (mail-westusazolkn19013086.outbound.protection.outlook.com [52.103.2.86]) by imf11.hostedemail.com (Postfix) with ESMTP id 3E1194000B for ; Thu, 16 Jan 2025 18:14:04 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=outlook.com header.s=selector1 header.b=HdHbfIHr; dmarc=pass (policy=none) header.from=outlook.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf11.hostedemail.com: domain of mhklinux@outlook.com designates 52.103.2.86 as permitted sender) smtp.mailfrom=mhklinux@outlook.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1737051244; a=rsa-sha256; cv=pass; b=uxjBWfh6aFGaY8RLFE/u3sC4wb1lI6AO97ZyxndQplTPEwnLmekSOYmd5pbNPnIsRgDjJw hj6Wb7fQaKoe8eykXaH+tqzD4vJrB51c6zJLL9QcR8PELdj64DoVR/72QlKy7oRiKCDARC +Xtvwv6uk7dW5oq06sFGXJp9Npm8w28= ARC-Authentication-Results: i=2; imf11.hostedemail.com; dkim=pass header.d=outlook.com header.s=selector1 header.b=HdHbfIHr; dmarc=pass (policy=none) header.from=outlook.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf11.hostedemail.com: domain of mhklinux@outlook.com designates 52.103.2.86 as permitted sender) smtp.mailfrom=mhklinux@outlook.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737051244; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=99AFz1Zi2ODv3X/ErQO01K6Jci2Jgvix0oHLtiavSzE=; b=GmkiHpOWm0t1JujPsVYS5L3jRfvQGKzBJMAhLJnH3Iawt/7u82JAHjFzSoG8bXpCw0HV6V ucThtGuGi5zIMvyjq9sZ3F6umGnR0gXK86WEc6cAoVbJ9z80OFxhls+wnFNaloH8aGdaRx CpOaP9Q7nZ99+AnET3Mqk98fBajKBgs= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=e8QNx4EMzja82x3zvv0ng3gljWp+EmcVrgWy26qvguNSJvF2s/qGpz0SL0A9FjwTKctREfDvfsWcLlG9qwIGeMFNQpSPVxXEeWybyFaXBEsRAClTw0RLJkAfkBuGqc6V7Uo35tFPAaIGLBbY71D65Jo+jDP1nLFmdv9Lr1lMyHOIZIW7sa6HBrwwgOEFyu57FwJwhYcAbGB2Kd7EsEyactJgHTP4tedH0gKLG8kpXRplbiNAMP38WtOE2OiW6V6ibs94ytX5GofdiPABNCsHrEA8SayupTToIGZZuOOqAqd6ZCZaplQd+sH83KpNS68IqWU4qEggFAqhq2/ndEKj5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=99AFz1Zi2ODv3X/ErQO01K6Jci2Jgvix0oHLtiavSzE=; b=x2csC9KIfdLRrH6DiGSa0eHgQd0TSW2tNkb+Dakuxb3BN8vLz4Nb74k0cPKsZqe061TCeurHdT4h6XXw85bQ1+aldzQZPb0wPSOVmIF0sazxp96j3p50JeC/Dp7535eu/MR3dTXjQPvqW49Z7LUXS0q5XuGAkdkGUHyhuOadyIo9N/fzr1/X3gPBPg3PDhzfbQEHBJTuSST86yMCgmREt7pDOsZpbZr1fVzQ7j8DCJy8vIAulzYO26peBEQQIRHWDurYG3qSyvMHQydxOp8Mr1jhNP1Ji4RDmuqwR3GefeO0XmfpCVVFXOoDtkea+AMxpZA3NNxAqqCqeh5xfPJVww== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outlook.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=99AFz1Zi2ODv3X/ErQO01K6Jci2Jgvix0oHLtiavSzE=; b=HdHbfIHr/Gy9HMKNN3pZbgbp4v27AzW424bKCZv5cCkoPQ5ViXr9E1aVBoVd1pY1H9QwYW+DiApK4BzRW3forRLor7KpCfXRcnyP22WCjgnLc7o3zxafRslMp0FCPfS4Mm/NOiXgG9pkTNT4pdwgWkGkNckz/b+b3stH1ts0baq03qCD2p6yIsgIRaHXHzIe69xdZ4ZI8cve9UtlXalWCZ3kOxfgWsehtRN0MVe1Nd1tnteQOuKmoG9EDhnhxQUOZaq9tMdrmABpnGXe1aM/ECkFjkHCEVBljuKYZ4bkuDxcZR8/9COjxuKweCB7y82CsytTxAFCp/Yw2Zg6ftqsIg== Received: from SN6PR02MB4157.namprd02.prod.outlook.com (2603:10b6:805:33::23) by SJ0PR02MB7632.namprd02.prod.outlook.com (2603:10b6:a03:321::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8356.14; Thu, 16 Jan 2025 18:14:00 +0000 Received: from SN6PR02MB4157.namprd02.prod.outlook.com ([fe80::cedd:1e64:8f61:b9df]) by SN6PR02MB4157.namprd02.prod.outlook.com ([fe80::cedd:1e64:8f61:b9df%4]) with mapi id 15.20.8356.010; Thu, 16 Jan 2025 18:14:00 +0000 From: Michael Kelley To: "riel@surriel.com" , "x86@kernel.org" CC: "linux-kernel@vger.kernel.org" , "bp@alien8.de" , "peterz@infradead.org" , "dave.hansen@linux.intel.com" , "zhengqi.arch@bytedance.com" , "nadav.amit@gmail.com" , "thomas.lendacky@amd.com" , "kernel-team@meta.com" , "linux-mm@kvack.org" , "akpm@linux-foundation.org" , "jannh@google.com" , "andrew.cooper3@citrix.com" Subject: RE: [PATCH v5 00/12] AMD broadcast TLB invalidation Thread-Topic: [PATCH v5 00/12] AMD broadcast TLB invalidation Thread-Index: AQHbZ775s6MDDs1YUUKZAICL5pC9kbMZrNHw Date: Thu, 16 Jan 2025 18:14:00 +0000 Message-ID: References: <20250116023127.1531583-1-riel@surriel.com> In-Reply-To: <20250116023127.1531583-1-riel@surriel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-traffictypediagnostic: SN6PR02MB4157:EE_|SJ0PR02MB7632:EE_ x-ms-office365-filtering-correlation-id: d59e9ba5-b3a2-43ae-3107-08dd36598d20 x-microsoft-antispam: BCL:0;ARA:14566002|8062599003|461199028|19110799003|8060799006|15080799006|18061999003|1602099012|10035399004|102099032|3412199025|440099028|4302099013; x-microsoft-antispam-message-info: =?us-ascii?Q?L4MC/qhik7cZfbzzGJCqs5UOt05nNGZ6REyG8FKjf2c8w7RcJJBg0epBjsJx?= =?us-ascii?Q?fN0cmRPZSpCzqAhpFqJNTg9oWHHP+NFOi7muVuVcn7n5tvT4eEj6smMe+7wT?= =?us-ascii?Q?rlHkiWg0FHAb5jxptzDHQ8Re6aEe/eXLqKiRxtJgJAuFu2R7DHIg6hEL6rHJ?= =?us-ascii?Q?rS5KxyAiCYFRPwEJ3FiuCkzELFkkcTpJJ0HHYANajQLQqHs3HppFOzk2rgB4?= =?us-ascii?Q?RdCsOlAZZpJGWs1a9vMs3fzX3CbBOg/dVZR7fa8Zxkn/+9a1Aj92+i4vf0GK?= =?us-ascii?Q?pyQz6XiyqskhDgIqyq9x6ExPtCyeVNWGngit4vFt7s/+ZBfi5gsOEsSyaMnW?= =?us-ascii?Q?0i0Ztrvqvu+D7Ay+h8h/SsvRarchTHno7rgnsbpLBEk7ib7dy/5vuBUVCcYo?= =?us-ascii?Q?fE+e/+gA3FhrA244TTtMCejeASZQtyjuRG/JZBYRqa+hzoPMNyFOp58PJ2jz?= =?us-ascii?Q?pTlh6g8JGZAWLmULFRjyRtVyBlscHY300cGqZ13ZUHuE/ssPPUXWu6qZmoYs?= =?us-ascii?Q?W+8W8FIGuwu3iOAPGLxFyXxfrqFkZpv6H7MI+WgL3j3V7Pho+JWWCyEMJ+c+?= =?us-ascii?Q?dkQetMRv9p53dYCbyiGDfunuEiOPQX4Rt8W0QtYElDTXsrriwhEYx13rFiOQ?= =?us-ascii?Q?UJPiBwZDdwEEBdQvtjJoaF1fXu+4A+xDpQ6fqMdpkDYf3zVY5BMC5OoUjkSJ?= =?us-ascii?Q?+GkXn4Oz3gGISIQMLyg5RiJpf0mzCQLfDJNFe+SaSDlM1/QDFO6bOZ6tD5m/?= =?us-ascii?Q?Ow+F4PDyYMKfwLTZqtuvYTGv3xZHrxieHfSMrc0dfqo8ACnIWMcgrXxqPN6D?= =?us-ascii?Q?F4yVKKcl2jjpk9PxwyMNmQJiSitRjFbiOU95UK4Envo5AmvfGQJSyvSL6DiG?= =?us-ascii?Q?qNIBgPL026s271uQlrA0ZPI+Y7E6kpXjArIlrPNogcQqLpT1QBLdTto+VqVg?= =?us-ascii?Q?8idfJlHqvW425kdA9nBLLSoqDLWmpmwoVwio9bAvGMDYjNEcS4p9B7WEhcfe?= =?us-ascii?Q?nHft+ZKp0BBToLoTEzqXjHpcUym8dEidfCEPOpe9+ozk2O90rMuRxOfr7Mws?= =?us-ascii?Q?wAw/tUYxSam/4yFxHfp105NcyrQ7PeRFjhL/NtRV++dW4oCyCHrJmPHo22/9?= =?us-ascii?Q?fdEH90XhIB4I0+NAry6NjjNs2Xu+1F950s+EwysHCkUqJ62unTQKH6LV3tEa?= =?us-ascii?Q?fd5MQvMRN/PWT5opZ75X0CQioaLaJSw4fuNkdw=3D=3D?= x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?LTyY4+B8/2woefrABZ/8j5QB3u0y4wsHSl8KnXcK9hj1HgDmNZi0OpFfGe4O?= =?us-ascii?Q?ehLVB2cCOD77HXw24jK1EYE5wDlfFqPyK60OipEHVQCLImhYfTOeRpXfDrN3?= =?us-ascii?Q?Mp5vl5HZQ448beLcLz+NS7yD/VgJt9bLtu5kSiiyzUvd9tBMZ83/9j6s1AAK?= =?us-ascii?Q?3c0Qpbo8jGdqv23rIk5Alf8gMJVQuoUfsqrMzOw6jWTWSbjIMrG5ipp6oUtz?= =?us-ascii?Q?B/ekBpqV3NFvAtm0Sam2KKqfN55lzH9DxfHF3rxWCjBG/5VJSUMO9Pb4+sNI?= =?us-ascii?Q?o1UwXYKMVbYnj3ZWqVZSlaWky134XcpRRFC4BwMkDf4WAzJ5uA2/3xhMJuZP?= =?us-ascii?Q?+BoDlXzKC3umYKQ/siYPA75pqpnGdLJ1qM6+RT4B6lPizx9y8rkOxteuqt4B?= =?us-ascii?Q?84DAzqTpH2AgJo4fK2qo7q0csVEe5SVgoHb4BAVOP4Ik/RR7/fxODSnfJAwa?= =?us-ascii?Q?GvweQ/O2OwisJBj07qVzqDzlnoHKr3xY5GDcdZFplvgyXf+35j3oT2WXVXEM?= =?us-ascii?Q?iD3gAQhwTOGIgP552XGSCl2ahlmSSN0prLyPFWPi8/dfrN625J1DtB511cDs?= =?us-ascii?Q?OBvkaDk9ULPF7hs15Lmh62fbKRUeRt58t1fBIYOLKzCXXYeSAYWYZcUgbeNu?= =?us-ascii?Q?1v3il0Lq9c6HiLqoS8jE4qkRz8mP8VtOWF/LpB2WyHynHRDeUgsH1j9ZwEef?= =?us-ascii?Q?7uo6sy1rAxh468405aeMccDkeE0vefrx89v6r2qBjXi0pudPkz3Mo+DUXJnW?= =?us-ascii?Q?46OoWnBb9oL/DUC4/vU/PIppWpEBhjKCjtMqGMLIiAf4UzbOdXBM8XK99Y+w?= =?us-ascii?Q?JV/4M3aOQCLhR0z4IZGPYVlCFU0tL2ZFa6+Q4+DxyEWJJzsh188OEmopGGNC?= =?us-ascii?Q?T8yXql3KKh7Pq82GaD1Y7sJnwCX9h2j6pZ4JHYZWTmWFPA7+As3y5CYLmHCE?= =?us-ascii?Q?I1CX7qySgPovmKy5b/r7HAx1auzsYsjnnLq6JOuiswYdU3TutOiDHX+Zxela?= =?us-ascii?Q?ml17Q2wfi3ONz4J8L97gyFwxSiPVtJlhBjCv6clg0ozn2lz7soSU/F5sWRz5?= =?us-ascii?Q?lQULEuOPAt1VadSSi5EQ3XhQu4PE4tfCfpz/h8RkRm+UW05ZHYSXIwWWC1Zx?= =?us-ascii?Q?O268s3Xof3aosVEHKpWK6/quwEaXrNQSutS+TgKb6cFDzyeJZM1AYR8l0uB3?= =?us-ascii?Q?VaJx5vy8iTBm70889nioeaMgT+3Ou34N39jR4ca1yH5IvWM+R5rIB33Qvhg?= =?us-ascii?Q?=3D?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SN6PR02MB4157.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: d59e9ba5-b3a2-43ae-3107-08dd36598d20 X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Jan 2025 18:14:00.3583 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR02MB7632 X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 3E1194000B X-Stat-Signature: wa9opwtihme3bt747jmm6dqkgqrx6ftr X-HE-Tag: 1737051244-242337 X-HE-Meta: U2FsdGVkX18Pwv/xRlrQeCGNKrU9Z6haikCKXps7be6WLtoC3UFMcrIz7rLFI4/7g5z6ooUtoRJbfFijKzUnbDjKR/pjikARX9Rsv4YEpFTO6oWKrfy4Ec/idqU9CDb5SkDM55x/Nmlve6Si1YExiv9rXDj6djzO7/G6vQtHJsRomV4LYKuQWDv0f6/4O5XOFrot0cCX4Im/ueoGhnP2fVYgLaC4TZeLhKpJPj9TwXPkMp+uAkR5bNfVXrXdCxjPS75rO1gS3JD5XsFcNmTYlR6OnuiaD6KewHJPsLX1Rx7AoLzgCcQ7x8NUYCwbxhGK/QvjtR3p5v2bgRsK4Ebw34teeQrBoNtMqkOSuJU5L8W4UnBlUTlyJDoIXvEzA/h4QstGjUq5Xy7PU1skK4s14tnWAc49ZpfwIRydHNgTDKE+LwvIdi2BQwuQbLvxvptCpRAucnj21FSD82IjgWvFm4IRUOV/nZ/J3nFpxiykxCgJNjzF8196snSf1ED+RZo3t82D3w246Vhi2E+9j8/aV6prwtrDffyqCNJEm9mEQoelOBvVD6VIZHbeafWD+NTROVFNlVuBCjqGJoqmXvWnnyM8Zde7w5OaccVU0hTfPAybvB668kzEGxywu0nwsx0zowBP2MwcITHrSoxgxEUD18/YiHIj3leM6QCy5mDYv5eNDG6p1um9Sex4c4giZYM0mSHv0gRlSTDCnp1NBIIqH39Uz6/gbLxzdSbhjVoVCLzgo0nGK1wzwW5am6lP7+PgaS+S4sYG2ze9aN3kRh/7/cou8MZZHkBeGFkwkoX+WITlzyoJcX9LX6AkqhoimNirwjisoKFcl/8N1e9C1i1TYOEJGqZuNvWrgjOKBWOumrT4GhS9v86RUREjdjzp8K2X/26UX5JlBhOQ+yLDep+lhzUjpdw4fNzFibeNwp5BgqO8kWrTFfJqJ5IoMiktqH4NHAbJrGnWjrRuXzvCtAL 7PGe/m1n EmPmwiN5rmIA3puaVUaFtKfp243qzFNbVpYevGzVpxvorrLltDqzJhhCE1HZMWKl0VvkC8+eFXn4gxxJTnFqmIZXBxD19LUoppDo36bs2vEIhu1lLgDM8ldXSICKkNuucSQEGHzYZklAYuRMmBIubUBhCR+OOJQfNTF3zb84Mq+1iQpZ1b+X/riDwVctuB6b7tq2oE+KkY1uOyRS3tTiXg7ghN7E44fzazpvGnpKs9mkaS73mel/BvHe1PChq6JXh4xcxgqm3zvTW1hMH4Hot4HJazEk0qMANt8lq45y7VblNO6IloppuNAG7G0cfKpj3RY6MzL5WbYlunqnndUe9FN+7RSbZRdM5TsSOjmtceoz662+m4QkXed5Y8zuWIQScUBv0p/DrOsxpIloeS23yQTC2Plcp1keVXkbJH7zoNABi4fITQyXQq8/8SvWn26SL95y+riAR/yDqU8vz+SkqU3GWxO1d8Gbx853xs2zz3Atl3pCR+xhsKD7mHXCvRNfeoNGGRd6gMJgmd1xlHa9RIT3IPwwzcDABksZw2ZsrMi1jAyJ6/kXVahH4dvDFxa4y9hlguNPj2TL8NR9MDi7kYvQtYlWsiF537k4FJ9GniP0rOICsrYSVK9EKdl+HO/jy5iPZX+vkwdVV6w+0OMfCEcQs9FcV8CXaxPMCl1iQBsNlPt6a8TMruPbQLNSVeW1zglYQlOzjIDdBkXDeLOJWsoS0raIZo2KOK7gERcBWNmEydQ7+DR7VQAclygKQ3UJpw5FT0X51LrD9VsXNmBZyi2nPfA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: riel@surriel.com Sent: Wednesday, January 15, 2025= 6:30 PM >=20 > Add support for broadcast TLB invalidation using AMD's INVLPGB instructio= n. >=20 > This allows the kernel to invalidate TLB entries on remote CPUs without > needing to send IPIs, without having to wait for remote CPUs to handle > those interrupts, and with less interruption to what was running on > those CPUs. >=20 > Because x86 PCID space is limited, and there are some very large > systems out there, broadcast TLB invalidation is only used for > processes that are active on 3 or more CPUs, with the threshold > being gradually increased the more the PCID space gets exhausted. >=20 > Combined with the removal of unnecessary lru_add_drain calls > (see https://lkml.org/lkml/2024/12/19/1388) this results in a > nice performance boost for the will-it-scale tlb_flush2_threads > test on an AMD Milan system with 36 cores: >=20 > - vanilla kernel: 527k loops/second > - lru_add_drain removal: 731k loops/second > - only INVLPGB: 527k loops/second > - lru_add_drain + INVLPGB: 1157k loops/second >=20 > Profiling with only the INVLPGB changes showed while > TLB invalidation went down from 40% of the total CPU > time to only around 4% of CPU time, the contention > simply moved to the LRU lock. >=20 > Fixing both at the same time about doubles the > number of iterations per second from this case. >=20 > Some numbers closer to real world performance > can be found at Phoronix, thanks to Michael: >=20 > https://www.phoronix.com/news/AMD-INVLPGB-Linux-Benefits >=20 > The code is now in a state where I am not sure what else needs to > be done before it can be merged. If you can think of something, > please let me know ;) Rik -- We had an earlier thread about INVLPGB/TLBSYNC in a VM [1]. It turns out that Hyper-V in the Azure public cloud enables INVLPGB/TLBSYNC in Confidential VMs (CVMs, which conform to the Linux concept of a CoCo VM) running on AMD processors using SEV-SNP. The CPUID instruction in a such a VM reports the enablement as expected. The instructions are *not* enabled in general purpose VMs running on the same AMD processors. The enablement is a natural outgrowth of CoCo VM's wanting to be able to avoid a dependency on the untrusted hypervisor to perform TLB flushes. Of course, Linux hasn't been updated to make use of the instructions in this scenario, and your patch set doesn't use the instructions in all situations. So CoCo VMs may still use the paravirtualization that makes hypercalls to do TLB flushes. It's future work to *always* use INVLPGB (if available) in a CoCo VM. For a couple of days, I've been running your v4 patch set in an Azure CVM, just to make sure nothing bad happens. From a basic testing standpoint, no issues. As expected, INVLPGB is used in some cases, and the existing paravirt hypercalls are used in other cases. But I have not fully reviewed your code looking for potential VM-only issues. All of this is to say "Don't exclude the VM scenario from your thinking." The scenario exists in real life today. I don't have any specific code changes needed for the scenario. Michael [1] https://lore.kernel.org/lkml/00294e7e-dcd8-f940-372e-070b8d174582@amd.c= om/ >=20 > v5: > - use byte assembly for compatibility with older toolchains (Borislav, M= ichael) > - ensure a panic on an invalid number of extra pages (Dave, Tom) > - add cant_migrate() assertion to tlbsync (Jann) > - a bunch more cleanups (Nadav) > - key TCE enabling off X86_FEATURE_TCE (Andrew) > - fix a race between reclaim and ASID transition (Jann) > v4: > - Use only bitmaps to track free global ASIDs (Nadav) > - Improved AMD initialization (Borislav & Tom) > - Various naming and documentation improvements (Peter, Nadav, Tom, Dave= ) > - Fixes for subtle race conditions (Jann) > v3: > - Remove paravirt tlb_remove_table call (thank you Qi Zheng) > - More suggested cleanups and changelog fixes by Peter and Nadav > v2: > - Apply suggestions by Peter and Borislav (thank you!) > - Fix bug in arch_tlbbatch_flush, where we need to do both > the TLBSYNC, and flush the CPUs that are in the cpumask. > - Some updates to comments and changelogs based on questions. >=20