From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BA20E77188 for ; Fri, 10 Jan 2025 06:08:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F40C98D0002; Fri, 10 Jan 2025 01:08:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EEF778D0001; Fri, 10 Jan 2025 01:08:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D67FE8D0002; Fri, 10 Jan 2025 01:08:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id B59038D0001 for ; Fri, 10 Jan 2025 01:08:18 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 29C9480817 for ; Fri, 10 Jan 2025 06:08:18 +0000 (UTC) X-FDA: 82990512276.02.180C6B9 Received: from mail-wm1-f54.google.com (mail-wm1-f54.google.com [209.85.128.54]) by imf05.hostedemail.com (Postfix) with ESMTP id 3027D10000A for ; Fri, 10 Jan 2025 06:08:15 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=CeVJwt7a; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf05.hostedemail.com: domain of nadav.amit@gmail.com designates 209.85.128.54 as permitted sender) smtp.mailfrom=nadav.amit@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736489296; a=rsa-sha256; cv=none; b=KrUVAw1FETftOANzyvuNcFRgfmCs+maX0kFisaxenPbZR4ujZ1OgX31rnaSDqA8GN1j75L cZqZZdTdnTOSepsDV7ZVv3zdpslAUoYfnHbQcOgFkmjn6Dx/UFzKQlib5/uWB9eTPAYF5I n79ygnCnJcHZJQBJoIlP2Z8bSTOBoQ4= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=CeVJwt7a; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf05.hostedemail.com: domain of nadav.amit@gmail.com designates 209.85.128.54 as permitted sender) smtp.mailfrom=nadav.amit@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736489296; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=AIF1eAxxJoLk/2NWoF0H87GjzPYtUPj+T1xrd8DQ6Jg=; b=VtOfJpUSR3VMmHDq1czbRbVAoNGE5/NTm2w5Y1CIRA+cLhFOeqxUWq/LGZsiZn5ZSFZqd/ Dar+4TQ336cuuyyim6mcvWVK0ZcG+6DpENdQJFnTfApNl9wPVcfFCDb+1igk4Z2eWWptAc IxTEZvOQ3mv4mr3PWnHIG0LFoENs/+k= Received: by mail-wm1-f54.google.com with SMTP id 5b1f17b1804b1-43618283d48so12610155e9.1 for ; Thu, 09 Jan 2025 22:08:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736489295; x=1737094095; darn=kvack.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=AIF1eAxxJoLk/2NWoF0H87GjzPYtUPj+T1xrd8DQ6Jg=; b=CeVJwt7aTJSEiR8sa9Ob1yzEpLi9ZAPROazJW75t6b2NfDf753RPtljUNlvlKrb+SG RyN2DIypq8BjB9cp+idQKEDtUOTVUBWtjs57RKB9lKL1x2XyFOY88yJoSFVOVO2dL3Oq YpmkwlLAnlbQCsUBj0H8R1h8gwhZsTJe02upg/tvuqAw3WPs4ArbuRvQ8j/0KY9NTLaW Wxr8rtfOxMbdK1JYRYDSf4hbHy6gy61a407O8OwQ9RtdRF10rq+LGBy81MLKV7AEKxn7 dqL5UR9Uilp9OVxsv1erN7JJRbTY5E7TcnEs0fXVtXgsYEWiUN1hCIzdxei3L5NxPhKZ IASg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736489295; x=1737094095; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AIF1eAxxJoLk/2NWoF0H87GjzPYtUPj+T1xrd8DQ6Jg=; b=pPfPSyM/EArpgGmc2MwZJQzNh69xYiqMdFq3w1GCZBCimyUsqNhnZI5Y1TmRouXAn2 9teOOmSJbfuhROWhJ3q1C/BMR+SaYN2fV1j0FO5HAOMzExg011MRBsiuDsO/PsiG5YgY ldRjNGJz6jtwnoBHvpx5LqkYkMJaCyq86uLct3lJ8A24PCzep5vaKcGL0QsnEwamShRv nVmBOQgn9z0rAFFPCz1ktPl707gsQYv9ni9E6xEe9BGbzqVhqCgrDljaIKN2trFUmkPC N6AgWSplTZRfn3xd8FQVpwgvKV+ZaEDbo/BmDtMRAuXYUIYtMmDEwK7J4dCvMTDyC2Mi C9xA== X-Forwarded-Encrypted: i=1; AJvYcCVavk27ppFG+E4ukh0uUjK9QagQKpGxQSVgA/UlvWPIe86pW7t5K67ZUAqceXsz+fjIQF33jCcfJQ==@kvack.org X-Gm-Message-State: AOJu0Yzzct8BUa/BhQJ7tCBuUMxq9TJdsnIlBMd2G9FBlrqJ/cjjNKgx eXi1UfaxsKxgyy1eju8tOMny4pNbRfBu6elokzMp+lGbZj6hlZyF X-Gm-Gg: ASbGncsqG4wcl3Hbbrs4xph4O4M6vRarbCQz2L0XaA7tztmTItb/yJLAgaQ2dA5egLy 0U63yKHF4+aNF2lfnNYgBMK3rKk1rOTB9vlTNd+NX4++MaZyAoi9b8Bd5V9T7//5wiRprowE8EA x9u4N1JFAXjGOLGEhseaJYXFKaxSlg0R4a+NuAWIkLMwBLsucei7vGHK/nGMSs/2jadcMa82JVP xcAIEpqiwY5BpezD1IfGB0bXrSttx54Q9eudV2/BzQlgi6OYO1nXT364bwK6OeZ5HBmYg== X-Google-Smtp-Source: AGHT+IFpXyaxjC8xi/oV4JH+nccKAN6bFaS024FqLgGassM9bqlo+KptiQ6c6GwwWuoItH5iZH5Xxw== X-Received: by 2002:a05:600c:3b0c:b0:434:fbda:1f36 with SMTP id 5b1f17b1804b1-436e26e51a0mr82585625e9.20.1736489294392; Thu, 09 Jan 2025 22:08:14 -0800 (PST) Received: from smtpclient.apple ([5.29.8.141]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-436e9e37c2esm41296195e9.28.2025.01.09.22.08.12 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 09 Jan 2025 22:08:13 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.300.87.4.3\)) Subject: Re: [PATCH 06/12] x86/mm: use INVLPGB for kernel TLB flushes From: Nadav Amit In-Reply-To: <426011a9-1fbc-415c-bac7-df5d67417df3@intel.com> Date: Fri, 10 Jan 2025 08:07:56 +0200 Cc: Rik van Riel , the arch/x86 maintainers , Linux Kernel Mailing List , kernel-team@meta.com, Dave Hansen , luto@kernel.org, peterz@infradead.org, Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , Andrew Morton , zhengqi.arch@bytedance.com, "open list:MEMORY MANAGEMENT" Content-Transfer-Encoding: quoted-printable Message-Id: References: <20241230175550.4046587-1-riel@surriel.com> <20241230175550.4046587-7-riel@surriel.com> <855298e6e981378c3afeab93b8c3cb821a7a5b88.camel@surriel.com> <426011a9-1fbc-415c-bac7-df5d67417df3@intel.com> To: Dave Hansen X-Mailer: Apple Mail (2.3826.300.87.4.3) X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 3027D10000A X-Stat-Signature: 54njzmphg6s68iiy6s66xeaukiw66hey X-Rspam-User: X-HE-Tag: 1736489295-246634 X-HE-Meta: U2FsdGVkX1+Yljkyc9dcIrBY8s/lXptCrx2fJ2BUfSq1z3T7ZX1JxXptHu2kzVwOBoDFsIp5Sa/PXtilq2n4OK16X805zqjQtqTxQzDx8URKGDXnTT157IQwCWejm6IGWUV+XpB1Pj/4HguD6uD5haohRLv4TL2teLrWPyHt0lo6l4utgeW+xZlWLoVVwkC3AzEsEr/mJ67lKslNyLRS2/1J72ONkXWwpAE5I/Ec1GQy3E72VtIQM8f9exiR/d+//oukLKZZSBtOHtDLBP4CP8R8vGwgClLskPzcIi7FGrafL64HSLf94x0ZCK2WFrB8SI/Aw9B2yU1brtdkCbAK9MYATFpBN4FhO7cT3LjTqp18cdGDpHGU5ufj4EYJOvMTSOawVhDmNoLsKP+6M+mR0IjZ+eaWbanYWx5BwDnm9f+ArivdEjYoCRAtLptfOBofnl4lilRAB71CAkLcn+N1nZ2+hdr1ZOG1HZ6P/LwnYgZj6h4Fh6PxQgNmZhTzhSuxeNX4Tykn7O5gxZOBPpsVYX0EI9xT/Ss5eAw4sv57QXvK7QsbhhjRDx5JHK8X59gqNP3o+JiAhiQtEgFc6zVLjbjzKAq0DKMLrGd+k08BPfJhLf63+zviYj7ZvHiywOAukFI1pmY7nx3lAN9gJJf11CFC1WYl+QlQYuk+Txy5L3I9X/AmM+5vd2DqPqG+4dr5F6SKTXpU2q01FuaHT2efgEt8MofJNClVF/yJxW1xxUTfT0m5rMuOXuvkEcJT1VLlnYxCl+ebD/OSxgnwMdRogdW4atwvN8vk3/ZO9nQU6ahwkoIefcUwelGa0HLYtCNOjy2sUKMqToZKlDR3KFUlaQ4KZmdW3Pnqi3TUgrYsOAaymckOal3W53XJnDFdoA4vecxCHtXkl9ABh/CLqznOXcgDkr3FE122VnsauKrj/+1/mN+CTF8meG7oRVG+iXO1rL7SnNEtj1X+DhcFCeZ tHa1OE0S zisVepQa4y784ZTDXsvkzc8Pf20qSHoaWr0izFm9gk18uJtb0FeVKdu6iVks9JVo76AwR2dM+U2zX5wBXc0R4FQxlK2JkTdf7pjEYUrDRh0X21+z30Lm04Q9NsF7I9Hc/tSsfu9CGabqGdbW6V6qQYNI5Wt7Y66vP8uI27Z5gWKoruI5siQwJqXL6RNy13PyQuSY6l4v9u8JV4Y/ujyrBi2QAOTyzMQiIP21AnzIF74cKTuUeDBvXLIqMkF8Re49srd9/yCkoSal8ii1TOQIqfseUrNRWOevWcGGA6ZTAAex1tw9n3QG2KV7dyyuJR4GTmycHkUP6dluNoblxA1zqNPUqMVynydxc70u3IWIQ/XBMZ+ZqGJX3+aRbzy/WXVmSglnzUvQUU1GNzRqYq3ufVpeO6j0OcQVvjX5z9AgCGgFmwOfVh3BPN2+sigq6Gk8bEhVH X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: > On 9 Jan 2025, at 23:18, Dave Hansen wrote: >=20 > But actually I think INVLPGB is *WAY* better than INVLPG here. INVLPG > doesn't have ranged invalidation. It will only architecturally > invalidate multiple 4K entries when the hardware fractured them in the > first place. I think we should probably take advantage of what INVLPGB > can do instead of following the INVLPG approach. >=20 > INVLPGB will invalidate a range no matter where the underlying entries > came from. Its "increment the virtual address at the 2M boundary" mode > will invalidate entries of any size. That's my reading of the docs at > least. Is that everyone else's reading too? This is not my reading. I think that this reading assumes that besides the broadcast, some new =E2=80=9Crange flush=E2=80=9D was added to the = TLB. My guess is that this not the case, since presumably it would require a different TLB structure (and who does 2 changes at once ;-) ). My understanding is therefore that it=E2=80=99s all in microcode. There = is a =E2=80=9Cstride=E2=80=9D and =E2=80=9Cnumber=E2=80=9D which are used by = the microcode for iterating and on every iteration the microcode issues a TLB invalidation. This invalidation is similar to INVLPG, just as it was always done (putting aside the variants that do not invalidate the PWC). IOW, the page-size is not given as part of the INVLPG and not as part of INVLPGB = (regardless of the stride) for whatever entries used to invalidate a given address. I think my understanding is backed by the wording of "regardless of the page size=E2=80=9D appearing for INVLPG as well in AMD=E2=80=99s manual. My guess is that invalidating more entries will take longer time, maybe not on the sender, but at least on the receiver. I also guess that in certain setups - big NUMA machines - INVLPGB might perform worse. I remember vaguely ARM guys writing something about such behavior.