From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A292C02182 for ; Tue, 21 Jan 2025 10:45:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B4C75280002; Tue, 21 Jan 2025 05:45:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AF9CA280001; Tue, 21 Jan 2025 05:45:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9C146280002; Tue, 21 Jan 2025 05:45:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 780CB280001 for ; Tue, 21 Jan 2025 05:45:23 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id D0FB380877 for ; Tue, 21 Jan 2025 10:45:22 +0000 (UTC) X-FDA: 83031127284.17.6959CC0 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf29.hostedemail.com (Postfix) with ESMTP id 2B14B120013 for ; Tue, 21 Jan 2025 10:45:20 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Ix8NA+lk; spf=none (imf29.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737456321; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=r5vpCvDvnd04BK2QN6O+XOM0rF9uo+eThCy305D/h4g=; b=Bq36KfXG//NmkwWN3mITAIYK5PuEbmCXGhNSeY80YfnZwPiVd7oYkczH663ndO17a1iu2s dEJF46xhg7xsYIQGQeVTHCZVmuow1TKp5hK1Jff6a4XlibZ/0z9xfqM2PiIHDkErP4g1Ni KPURLlCQK7rqSY3+BzwDcse61829Bd8= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737456321; a=rsa-sha256; cv=none; b=iIgJOPLJsbWdbYobrZ97O2GCVecLqSlXBuhHy4ncgmjpI+uXcjjhd3Mtp6CN3DDHHAnuiz 7jXldt7j8wDfnqnA7GlcaLXal/nQvoUK5nAhaEt3p+RnQB21jkbPWi5i6pQyMJED/BOK1X tALQz6wuM9AlgC/Y2Up4UuoqlohX/Jw= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Ix8NA+lk; spf=none (imf29.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=peterz@infradead.org; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=r5vpCvDvnd04BK2QN6O+XOM0rF9uo+eThCy305D/h4g=; b=Ix8NA+lk0iihGFXlWiDcV9WNaQ Ze2y2xSgj+/qDyuVOMPc4LAx/y/ZVvtNkvWuteFysuY8tYiVHiZmf9n8fsEteh+9m1bO41uZ8V+LL zCSMAtDNyfVdlHEYPvtzAhuabwuVxWju4Vdx0sjDuBjR1GaTMPj9axVsavEqTvFtal9ryrKB3+e5Y dg7euTE2OiFtElwlVO9gJT9QKGHsBiFrTn/+DG01ccaPez9+IvXq2rgETmqYh1LeeMmMaV5q/gPYq peni83KOJD6Du2SVyWXKdpEWsNHN7z0GdkGwsR5Kj2Lk7kaVsU9TPRtvJ8JfejBQVVb+j4LRICL3t sN3s5VMA==; Received: from 77-249-17-89.cable.dynamic.v4.ziggo.nl ([77.249.17.89] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1taBkv-00000007g1E-344r; Tue, 21 Jan 2025 10:45:13 +0000 Received: by noisy.programming.kicks-ass.net (Postfix, from userid 1000) id 4BBD7300619; Tue, 21 Jan 2025 11:45:13 +0100 (CET) Date: Tue, 21 Jan 2025 11:45:13 +0100 From: Peter Zijlstra To: Andrew Cooper Cc: Michael Kelley , "riel@surriel.com" , "x86@kernel.org" , "linux-kernel@vger.kernel.org" , "bp@alien8.de" , "dave.hansen@linux.intel.com" , "zhengqi.arch@bytedance.com" , "nadav.amit@gmail.com" , "thomas.lendacky@amd.com" , "kernel-team@meta.com" , "linux-mm@kvack.org" , "akpm@linux-foundation.org" , "jannh@google.com" Subject: Re: [PATCH v5 00/12] AMD broadcast TLB invalidation Message-ID: <20250121104513.GB7145@noisy.programming.kicks-ass.net> References: <20250116023127.1531583-1-riel@surriel.com> <20250116223748.GA33629@noisy.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 2B14B120013 X-Stat-Signature: hqecqrt9yccib6qtskqmog3p6md3u9g9 X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1737456320-79938 X-HE-Meta: U2FsdGVkX1+6A0pXoXAAExnVwfKhfseO9exokOH99Ux3Mhy9Qr5E9Khx0Vgwx+qkupJQWYVrQcT6YO0WBwrtABihSWuG6L2bUqlxMXDBKra7EyOtjzidF+5XDtip6cONleSGoAl7ybgG4R/5YE8YTRi4D/eVsnQePnC/3225T5mdoKE3US/EvFhipQHregLq0bhb1EHEa4DsMAPWMGolMNzEZUM04YUkay3LJpf0qHhGzPvmdml0Q3gkKibWent6RSDrqNwMY75t5XzRtDmW5cxdmmmQfMlq4S6hNshATAOytPYKX9NrkvcRQnFxVNfLsbH2kQuP3tk7o1PvvwTNlnihZ9gJUUfWdjAv6drM507rRjQWlFpswx2aOKRQ9k24nRTmGCh2eVpFthIOHk9UZPg1gvWktlL4NoUhksX60eUnOtnCLJdqxW0H3gFiKkpXq1n8vlVkX5SCraNtwaUSxT8/R2IO11yeSGwaCixuhOlnYS/MdK9yTw4eN2GQ0ochOyNIQD/1b3sYiupdB0i+8du/1vw5otVLJ9tV/jn3caYJVgeSJQOhtX5PwSw58sJ+EkG6FLTwc0oIq7T8kYm0OrlVmldhotfTnS3MqyCDZJqBOyK610pzcwPzufZrMd+jwrbsCoL6XqvqTPDnUAWpLhBWx49rYLEllbr2kr6TZtokk8p02WiVygdsg3HwnAzJ7Qgv28cYdR+svGhcsgHgXU7IGFQPXKaagqjFbBylanGBH1bmHcfS9GUvzLyyHGbdA30vg1hQ4aOLrLVagz3WQfZ1cEReHAi1uj4VJpUyfcJn8hsv6IcXDOIJATCbA4mBcRG4F5xXU8EGlWCSbqW/f5FrJ8bzyn2ZTjEbQrht7X21ozhJNvLsAh8a3O9ipmxAkjoTISvSmTMrRmAmtq55BgO25PJbmuPWc+ed154imXeRJWh1ypcFGJSXMcK52Z/7X4JKIcXZdBbwi6h11kJ K1F6wuUg 2kxD0ScEtCt8IpHcIps7xjjZ0wv7cyKwQsEl3IfgUg5GhdtRyuByDVkXhJstqUPK7VSbm7cenaBa76Po3EVNwEUBoPrhKgXdQispeXwaEmlYWcatq/GRI7pf6NFN1tHAZe1IMu8II4RRATC3FpcbsBdpyJaMAXzMAFBe5PsyhVjgphMazjqkPM7COsAKKqIvGMwl8QSrc1YPXT+v6P5YNaQBYda8aQJzdzF2yzZsvD4CfqIRYkCEUDyyVEGe5fovHWSVpRIR2efMRMF8My1BqZ1E/rK8cgpP5wIXedeRi0eaheBIJxjvoodbQEg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Jan 17, 2025 at 12:00:33AM +0000, Andrew Cooper wrote: > On 16/01/2025 10:37 pm, Peter Zijlstra wrote: > > On Thu, Jan 16, 2025 at 06:14:00PM +0000, Michael Kelley wrote: > >> So CoCo > >> VMs may still use the paravirtualization that makes hypercalls to do > >> TLB flushes. It's future work to *always* use INVLPGB (if available) > >> in a CoCo VM. > > That would place a limit on the number of CPUs, to be no larger than the > > number of available ASIDs. > > Can you please be specific between PCID (the x86 architectural thing > commonly called ASID) or ASID (the thing named by the AMD architecture). > > INVLPGB instruction under virt can use PCIDs to its hearts content, but > ASIDs are rewritten behind the scenes because VM does not usually know > the ASID the VMM assigned to it. Sorry, I had to re-read the documents again to find out what you were talking about since I habitually skip all virt bits when reading. Urgh @ AMD for adding ASID, although I think I understand what they're doing. Anyway, PCID, the thing commonly called ASID by pretty much all other architectures using broadcast TLBI. Since ASIDs be global and independent execution of CPUs means you then need at least one ASID per CPU (when they're all running a different process), you must not have more CPUs than available ASIDs, or you're into trouble. Since PCID space is 12 bits, we cannot have more than 4k CPUs (minus a few for that other PCID crap we do), or half that when PTI. And while that sounds like a lot -- its getting easier to hit every passing year, we're having how many CPUs on a single Epyc now? 192*2 or somesuch nonsense IIRC. So if someone wants to mandate Broadcast TLBI, someone needs to figure out what to do when we hit this limit.