From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D885C02198 for ; Wed, 12 Feb 2025 09:54:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 75F5C280002; Wed, 12 Feb 2025 04:54:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 70FC6280001; Wed, 12 Feb 2025 04:54:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5D7D4280002; Wed, 12 Feb 2025 04:54:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 3F8CF280001 for ; Wed, 12 Feb 2025 04:54:50 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id DC4001408FC for ; Wed, 12 Feb 2025 09:54:49 +0000 (UTC) X-FDA: 83110833498.05.47B2B15 Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf23.hostedemail.com (Postfix) with ESMTP id A54FE140004 for ; Wed, 12 Feb 2025 09:54:46 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Pq0yP9Uc; spf=none (imf23.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739354088; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6c+WjwuOjtZ/tCDydLZsLysHBEZRfMiTVscRPlLORcI=; b=vTxyewquPajF/t/Pauhh0EoQ46WXQktFRakI15c/9eHowsSThUK6z3nrVylMnSxRIZEmzy MMjcvA4Ai6v09h7sNlBRp62odWwCz3E1tcB5mZ6jgVX1ohLvGicm+NJPMoCDrnLgh0ddZx BjnCyzhrtJjTbdebOxZS/tr2WHFt7jU= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Pq0yP9Uc; spf=none (imf23.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739354088; a=rsa-sha256; cv=none; b=H73I5ja/0ui/nl/L1U69UaiXMNs9iGmW0zXrmYDcLEPI2MgCYhpN+hrZ+B53H694jYOhlU Y8+rW1ubyepGvFxQf0qTG8Sgq2Bh6eBE1ZklFZCVYg4pe1dzfVmblbFqQ+oGze111pXPoe ofZ74HNg30sEtVnQo5wa3Kilp01KfAY= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=6c+WjwuOjtZ/tCDydLZsLysHBEZRfMiTVscRPlLORcI=; b=Pq0yP9Ucvm22cCmGGU2MzGS7vX pnvfMziMyGrKPf+LpLEw5sbsHtTw6kmtptlFPhxfwwCfog8sJkZ2yrluAR9ouMu/QetwtttEb0TI3 lR6ESkwaqzXKWrb60nxyzjJ0mN9Hq1ZDEeow40V/O4dNQs0amTR6u7R5Doi5XAEOptzSpKmRAfk2b oKjE/qMWuTSkL0+fUYpeX1QO2+wCR1Cle5A3MjhW2MV79Skg2QykLcY/pGy02Nq5iXyFIRVhfcCwX NAANbHouzC2yj9uUeoBbmAVd6UxSutHSm2lCUw9Fa6/0+O7fhqyk4gB7BD22kIB3Z8lAPf8kQHlAR wZS+bVXw==; Received: from 77-249-17-252.cable.dynamic.v4.ziggo.nl ([77.249.17.252] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.98 #2 (Red Hat Linux)) id 1ti9S3-00000004ByC-13YM; Wed, 12 Feb 2025 09:54:39 +0000 Received: by noisy.programming.kicks-ass.net (Postfix, from userid 1000) id CCE93300318; Wed, 12 Feb 2025 10:54:38 +0100 (CET) Date: Wed, 12 Feb 2025 10:54:38 +0100 From: Peter Zijlstra To: Rik van Riel Cc: x86@kernel.org, linux-kernel@vger.kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, zhengqi.arch@bytedance.com, nadav.amit@gmail.com, thomas.lendacky@amd.com, kernel-team@meta.com, linux-mm@kvack.org, akpm@linux-foundation.org, jackmanb@google.com, jannh@google.com, mhklinux@outlook.com, andrew.cooper3@citrix.com, Manali Shukla Subject: Re: [PATCH v10 09/12] x86/mm: enable broadcast TLB invalidation for multi-threaded processes Message-ID: <20250212095438.GC19118@noisy.programming.kicks-ass.net> References: <20250211210823.242681-1-riel@surriel.com> <20250211210823.242681-10-riel@surriel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20250211210823.242681-10-riel@surriel.com> X-Rspam-User: X-Rspamd-Queue-Id: A54FE140004 X-Stat-Signature: mrcuxznb46su7segywbx6s3xzcwwghbi X-Rspamd-Server: rspam03 X-HE-Tag: 1739354086-10890 X-HE-Meta: U2FsdGVkX1+iI0I+wi11aB4uSgZcgTuTY+uwyGGE5cW57iQ0dihKGkKJ9twuAJp79L83N+2XtDmH0ifKSPDzfY8Bna63G3rwKdfDveYickRteHV2HyzvdeNJVoPSC2ZfhMdX2+BRMxTZvcVQs2GO+gN/XLdWBysoCZR9rPIoOMRCyveAYcj8jq4dx2Hk0cKCt9WV3w1Xg383YpUk4x/8LIEAZK608H2GSsB0KwggOMaNM4pBtXY5yHLT4n6Fr5zTJ+aHbHOSdqYGLxz3c3OJxwjkVKa/a/R2Nwx0k7N9geW3OuqcE6vi6cQ8QH35PdBOzxSaukxp3MOaERH31LddWHmawdFJ1QWFhhk+FdnrScJ3N5V9v0vxN1NlL7dKDyFyxYnfoNPkFR4gePggqpCdD86LBc5PvXDFaWkUGMM2UV3blb6MfonjOuYQPgS/wJBh9o2S8To8QIO+Az9Yhz5KsCzflj/I5hU/fOdzVjv36Y1w1Cji7PEIv0St9jIvgmxgsB99Y1Tuk55H2yU/j/PEGORziK9Ws/CBhxbZWCwY/I6TXr5Z/jwB+EfMGX+E5CBVXc2f4zas34dXzRuf5MSXLeb1BEQo2pbgAhFQWGFd5pKervr/HQdHx5PqIMDBATvKp5jW3/CPUFiCxKgTBaJYyMyWAqoAQuAL4CRle3EGSM/4ROluRErEMN5EbbW7SQqSSsrwL9fpk80symiSdXpvkeipU9UG2ED3aZIyBm9Drca4eM+rTP+PTJCJOcXu4Z31aGN4rf2/ZpxJcSzhrlr68BkP4H8ZkOV91D9fn0qI7O0YTHtfgnMUWvlJdLcx3BNhWFcb1+4TmT6HwXHA60PCkRmXIudtPWQU/osayYzWt3GofsZJTM4JxmUACC95AtnpZ473wTLZg5r7FamYJGYAF1YFQFA9vMgj9fgsF0KeZVfOQDtQIeWszNWlDMeJ/XBP+CkXhQ0XGbPyirf1W/4 5GD9m+ux JTwprXgZySCivtPRfOCbrsxHdF2TX7j7gWHiV8qxpjNpAZOEAfaSBrOFX91kIJB2gGgjyiwSPpfx4WJpn52jukP3VGbhAwuvYc7YchNSGr2vYrMe910bt4xey6KTVTcQHENiMlzvTTE8HFitwywR+dbDfhq/Sc+aih1ZpCKCBhWcgg9U4Rh4DniPZ2OiiL3QmatvB/NHN8d2X55rdg35rZKlvhT6paEOzBQD0vyRpHkrHRbwVghh3YTQgCDtsQS+L+Y+3FujHk4TC6I0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Feb 11, 2025 at 04:08:04PM -0500, Rik van Riel wrote: > +static void broadcast_tlb_flush(struct flush_tlb_info *info) > +{ > + bool pmd = info->stride_shift == PMD_SHIFT; > + unsigned long maxnr = invlpgb_count_max; > + unsigned long asid = info->mm->context.global_asid; > + unsigned long addr = info->start; > + unsigned long nr; > + > + /* Flushing multiple pages at once is not supported with 1GB pages. */ > + if (info->stride_shift > PMD_SHIFT) > + maxnr = 1; How does this work? Normally, if we get a 1GB range, we'll iterate on the stride and INVLPG each one (just like any other stride). Should you not instead either force the stride down to PMD level or force a full flush? > + > + /* > + * TLB flushes with INVLPGB are kicked off asynchronously. > + * The inc_mm_tlb_gen() guarantees page table updates are done > + * before these TLB flushes happen. > + */ > + if (info->end == TLB_FLUSH_ALL) { > + invlpgb_flush_single_pcid_nosync(kern_pcid(asid)); > + /* Do any CPUs supporting INVLPGB need PTI? */ > + if (static_cpu_has(X86_FEATURE_PTI)) > + invlpgb_flush_single_pcid_nosync(user_pcid(asid)); > + } else do { > + /* > + * Calculate how many pages can be flushed at once; if the > + * remainder of the range is less than one page, flush one. > + */ > + nr = min(maxnr, (info->end - addr) >> info->stride_shift); > + nr = max(nr, 1); > + > + invlpgb_flush_user_nr_nosync(kern_pcid(asid), addr, nr, pmd); > + /* Do any CPUs supporting INVLPGB need PTI? */ > + if (static_cpu_has(X86_FEATURE_PTI)) > + invlpgb_flush_user_nr_nosync(user_pcid(asid), addr, nr, pmd); > + > + addr += nr << info->stride_shift; > + } while (addr < info->end); > + > + finish_asid_transition(info); > + > + /* Wait for the INVLPGBs kicked off above to finish. */ > + tlbsync(); > +}