From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03BF9C02182 for ; Wed, 22 Jan 2025 08:38:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1FEF26B0082; Wed, 22 Jan 2025 03:38:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1AD7F6B0083; Wed, 22 Jan 2025 03:38:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 09D016B0085; Wed, 22 Jan 2025 03:38:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id E10726B0082 for ; Wed, 22 Jan 2025 03:38:48 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 97096B1279 for ; Wed, 22 Jan 2025 08:38:48 +0000 (UTC) X-FDA: 83034437136.29.D8E2F1E Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf17.hostedemail.com (Postfix) with ESMTP id B114840014 for ; Wed, 22 Jan 2025 08:38:45 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Vf8J43Sz; spf=none (imf17.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737535126; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8+DQwNia3kWOeUROafPpORc26/Dzm182xsVdRuaDuec=; b=MhqTU85mqgrXGACJKhsXx0VOAGDf1qoGncZa5OuLRhKbUCXmoTDwnZHfThOPBMDbDRAWQc HphBSgbKwZo/YQyyeAHcFip9vtQj3NbMrhMqUwsUqi7CVFClBtBXZbWzfQpoW3NJMX2LHp M3R3DOERyHseX/fDz/gY4EUjtmggeGA= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Vf8J43Sz; spf=none (imf17.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737535126; a=rsa-sha256; cv=none; b=P530c4dLsYVpd/ZK0b2V+wl0ggmI4mqGXxjX6ObcdMDuF473RlQNO9MeBS5eReqOl1DpZr gLvx0ZJlb3Wx3PQoM4y3z0EUtmHUkerrpxMkopkuruWzau5/RUlJSkQE94nItGJ2WaZu9u 8ueBzC61ZP4Oes4Fm3ZPSqY5yNZ3WN0= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=8+DQwNia3kWOeUROafPpORc26/Dzm182xsVdRuaDuec=; b=Vf8J43SzDLrIqCrc4kq9Y5NlbK DRfUoIjR4AlIO1OZpmjdm1cEtaCtRyFPWYOHorZEZl7oyJ3E32n7yZoAfmCJawKLyYpYl1cnIh3y5 ASGPF/o9OZTvShLuxuNcNlPsZvakttDr7IsF+Jq1/477SyhpqsWAi9yVG34vosaumdk8L8wl4yuOn CLOdJmDdPMrqndK43nEOYNEzQvhDYDWl4N6g63EUxj73yy9KFvKbHgQduVye8+HsSsUQJyEmQf3D8 zyNdtNWHUqjBy7Gt5sAhEp1k5Wv72w8ymbDy17XNXU2sbYGoe9cxtdy0FIjhqTlQ9V0xIC3181v6Y g/BZavwQ==; Received: from 77-249-17-89.cable.dynamic.v4.ziggo.nl ([77.249.17.89] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1taWFw-00000000xlm-0aRv; Wed, 22 Jan 2025 08:38:36 +0000 Received: by noisy.programming.kicks-ass.net (Postfix, from userid 1000) id 3E4D9300599; Wed, 22 Jan 2025 09:38:35 +0100 (CET) Date: Wed, 22 Jan 2025 09:38:35 +0100 From: Peter Zijlstra To: Rik van Riel Cc: x86@kernel.org, linux-kernel@vger.kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, zhengqi.arch@bytedance.com, nadav.amit@gmail.com, thomas.lendacky@amd.com, kernel-team@meta.com, linux-mm@kvack.org, akpm@linux-foundation.org, jannh@google.com, mhklinux@outlook.com, andrew.cooper3@citrix.com Subject: Re: [PATCH v6 09/12] x86/mm: enable broadcast TLB invalidation for multi-threaded processes Message-ID: <20250122083835.GE7145@noisy.programming.kicks-ass.net> References: <20250120024104.1924753-1-riel@surriel.com> <20250120024104.1924753-10-riel@surriel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20250120024104.1924753-10-riel@surriel.com> X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: B114840014 X-Stat-Signature: 1zprrf8kegnqxtjwsehxjb68b8jp4umw X-Rspam-User: X-HE-Tag: 1737535125-837924 X-HE-Meta: U2FsdGVkX18FQyiyyyDNx/WPmm3CKWDoT3glKFeMSAJhzTiYHazPx4T7MKyyqteTTge582g1nUcSzspGB1yMjOhJ8jl9Gv+aikwnByUyEFLYvbcLrbxK7vcqj9BXDTgy9UQll6S2Ydr4+BzfLQK1tyO4+iVrjk1KK3zks/JrkljBol9er3HVACS20Cy4bhDcfCEilw2F8+7drhkhGyKn01PUlqH+x5N1GGCh6ue/XZMQ/fRDNMUF4fxfKf5mKNgCKtoognsilCK/Lxh0m1hOWWLRO8jzWscsFpDTssmEoIIH/8aR/TuzxO2kS6ZHgUGKjNIpDqkyKlj6cPf1ZpbYL2ax5sC8afNCELRqYFe70uq45WAMyS6rPTggyW4uBVFKVa3X8nR74p3tdeGNSENZp8wjpWlMLJXEmj41qXQxecVZQUHF5bRxUMpMjmnCe9ltdkBkvr9XtO6iiPVLYhfTbIulVCQ3niIT3xnt8gQEW1naMN8HhB2lDbjt+SneNYambWUTUEou1SlEStpLHenehKypT39uz730v4x47UDne8vf1Q0BqmffJe7+YyIS85zEya9zDDlyiRvfhhP+kNCESyyvLUBEsGz6sw7Z7pEXFzyTcV610B/a0hrXz2UUqFHCmoL3p22mNbd0cn4nujDKYZgeN8vbrKLgZerPZrOtrsITUi9DCmf2OniAVUPw10OTiFH5QW69Jpj5h5z9MzxebEu5FfSfwOWJBn54JUfYG/LLQK5S+ndOUxf8laXkRmz/IBtDgN6tYrAmPlGqkFGwklis6I/RytIPBfZMnEOdrH+Gf/BaaChKXXe/2ZI7l6ASS5RjT2dBG7CwZkZOI19DH88QBZP3j1TPpkIYFBiSxgUGVCvCRsGi7GfFOkDjhsE/2FqFdnwSGPjmeJO41OIfY0bvBMCVk8ycagxudbWWYv6KkD+lwor+L+c7jwRHIvfGKNVoGvkTj/DFSh/mPjD ovPxemoZ LPe5nkzsrdC4bepJUv6X5CDMOX3seu2794k2SpNHjrqUQDaBASUy6XrBgVvijus0QjFJcj3Vd3BqsecUIbBUBdXpf/GPWI2iclr/rI/RFFJ04K0piPWRTXz4sDShCxdo9Z0kT5lGow8gqNMjltAhCF4S7Ii4d1qGepRUJtmpV+3GP/bphAD99l2rmeXruXZ8rNhb1XnV/AXARy5NQfuuslMLP2mKD2/VEwySilZl7Y4HR2Q0it6ez8iOZF+JOVtmMCwlFtHHGbsCO7AU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Sun, Jan 19, 2025 at 09:40:17PM -0500, Rik van Riel wrote: > +#ifdef CONFIG_X86_BROADCAST_TLB_FLUSH > +/* > + * Logic for broadcast TLB invalidation. > + */ > +static DEFINE_RAW_SPINLOCK(global_asid_lock); > +static u16 last_global_asid = MAX_ASID_AVAILABLE; > +static DECLARE_BITMAP(global_asid_used, MAX_ASID_AVAILABLE) = { 0 }; > +static DECLARE_BITMAP(global_asid_freed, MAX_ASID_AVAILABLE) = { 0 }; > +static int global_asid_available = MAX_ASID_AVAILABLE - TLB_NR_DYN_ASIDS - 1; > + > +static void reset_global_asid_space(void) > +{ > + lockdep_assert_held(&global_asid_lock); > + > + /* > + * A global TLB flush guarantees that any stale entries from > + * previously freed global ASIDs get flushed from the TLB > + * everywhere, making these global ASIDs safe to reuse. > + */ > + invlpgb_flush_all_nonglobals(); > + > + /* > + * Clear all the previously freed global ASIDs from the > + * broadcast_asid_used bitmap, now that the global TLB flush > + * has made them actually available for re-use. > + */ > + bitmap_andnot(global_asid_used, global_asid_used, > + global_asid_freed, MAX_ASID_AVAILABLE); > + bitmap_clear(global_asid_freed, 0, MAX_ASID_AVAILABLE); > + > + /* > + * ASIDs 0-TLB_NR_DYN_ASIDS are used for CPU-local ASID > + * assignments, for tasks doing IPI based TLB shootdowns. > + * Restart the search from the start of the global ASID space. > + */ > + last_global_asid = TLB_NR_DYN_ASIDS; > +} > + > +static u16 get_global_asid(void) > +{ > + lockdep_assert_held(&global_asid_lock); > + > + do { > + u16 start = last_global_asid; > + u16 asid = find_next_zero_bit(global_asid_used, MAX_ASID_AVAILABLE, start); > + > + if (asid >= MAX_ASID_AVAILABLE) { > + reset_global_asid_space(); > + continue; > + } > + > + /* Claim this global ASID. */ > + __set_bit(asid, global_asid_used); > + last_global_asid = asid; > + global_asid_available--; > + return asid; > + } while (1); > +} Looking at this more... I'm left wondering, did 'we' look at any other architecture code at all? For example, look at arch/arm64/mm/context.c and see how their reset works. Notably, they are not at all limited to reclaiming free'd ASIDs, but will very aggressively take back all ASIDs except for the current running ones. And IIRC more architectures are like that (at some point in the distant past I read through the tlb and mmu context crap from every architecture we had at that point -- but those memories are vague). If we want to move towards relying on broadcast TBLI, we'll need to go in that direction. Also, as argued in the old thread yesterday, we likely want more PCID bits -- in the interest of competition we can't be having less than ARM64, surely :-) Anyway, please drop the crazy threshold thing, and if you run into falling back to IPIs because you don't have enough ASIDs to go around, we should 'borrow' some of the ARM64 code -- RISC-V seems to have borrowed very heavily from that as well.