From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2BE3C64ED8 for ; Mon, 27 Feb 2023 13:33:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 747E46B0072; Mon, 27 Feb 2023 08:33:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6F73E6B0073; Mon, 27 Feb 2023 08:33:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 60D556B0074; Mon, 27 Feb 2023 08:33:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 526D06B0072 for ; Mon, 27 Feb 2023 08:33:18 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 2B35E1A0591 for ; Mon, 27 Feb 2023 13:33:18 +0000 (UTC) X-FDA: 80513163276.01.0D0FDD6 Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) by imf02.hostedemail.com (Postfix) with ESMTP id DC4268000E for ; Mon, 27 Feb 2023 13:33:15 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=I0RZYM2u; spf=none (imf02.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1677504796; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=lYtdb3OvoePD9FKa/wblmpog8oSZWnWeDVgUCfTs9As=; b=loRGc5L4cT+rxogjsVrbIeoruRa2DTLTsjBcxX1l+hE8I4AmLtb6VSTDElbK1sVbNB+I4X qMdDY8yBbwpmuVPGxrNwg+3d5XUWbbJaeiexDDwkn422+oeltPPrSUEJm9G20kMlP6daNU fXKivErTbrpQcwDSXpJnVD+8XzxpS20= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=I0RZYM2u; spf=none (imf02.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=peterz@infradead.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1677504796; a=rsa-sha256; cv=none; b=WQL/478mbCUN0q1KKKwqEr2m9whhlLKQXMZ7CrLtlVmC72Q2K3n0gZqFgjX3wX7hkbORQG C0VFW8Vsmf5CPa65tbTpftTiABPOpFkks/6TSAwJqdvWP10QYm49Fn9SjfFOClrv6VP+Sb 1ddFMEV+otNP+K3cD8+vFeQ31lYrgv8= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=lYtdb3OvoePD9FKa/wblmpog8oSZWnWeDVgUCfTs9As=; b=I0RZYM2uTHooxcHaCkgIjRt6S4 Q+Cio3xlxnE2D8zBu9EwrwmOHggoJ4gPAUNYZLjzw6o6dohV+xL162RX8FtA3I/WRk1wFdNPEm/yr yrKFjbWHC1ZMcqj/0irRPq39VpTS5bCGOh+VFsbhmQUAls+W6xDUTrdPqBkP6MdJ6Rvt2Hjd19xYC 9leBtPob2IRbId2+FqrKY9fTkZ6D+RbZusAK7d7eGmHpRP7Hxvnw1VZqOvC4MIc5pzaGzxjDYi/yr tlalojHEEtgTFimWnPIYIitNI7zGfGggSUik5O456sxE5vEq3hLP19ZxR53wd51AF/rpo5SH2HOqY 3ZlKpOkQ==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1pWdcq-00EEZC-0g; Mon, 27 Feb 2023 13:33:08 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 4A9E8300293; Mon, 27 Feb 2023 14:33:06 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 1000) id 2942823B24806; Mon, 27 Feb 2023 14:33:06 +0100 (CET) Date: Mon, 27 Feb 2023 14:33:06 +0100 From: Peter Zijlstra To: Andrew Morton Cc: Nicholas Piggin , Linus Torvalds , Nadav Amit , Dave Hansen , Andy Lutomirski , Catalin Marinas , Will Deacon , Rik van Riel , linux-arch@vger.kernel.org, linux-mm@kvack.org, linuxppc-dev@lists.ozlabs.org, Andy Lutomirski Subject: Re: [PATCH v7 5/5] powerpc/64s: enable MMU_LAZY_TLB_SHOOTDOWN Message-ID: References: <20230203071837.1136453-1-npiggin@gmail.com> <20230203071837.1136453-6-npiggin@gmail.com> <20230226141238.6ec5fdf7d75dcf2cd4c58ba0@linux-foundation.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230226141238.6ec5fdf7d75dcf2cd4c58ba0@linux-foundation.org> X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: DC4268000E X-Stat-Signature: 8775cb88nxtmrs7su7oub43nmx3s5yqo X-Rspam-User: X-HE-Tag: 1677504795-733802 X-HE-Meta: U2FsdGVkX1/Y4VYvKoZrzGmIpqqst1eCsk1LcUc5SWDhYDK7ORtNRa0SP2qyMu2TEjVdCq7YRhRnhJeS/E0y/Omu0S/sBg1NcsKo6dOmban9VEUdzEj9IZ4NToo9NyIQfJfD9SGYkvdLQKdznWzOXMIq+o2/8hfyZ2vqNd6wMzgPR2vGWH9M6HaYnZh8//MF0NS6k+t39bMDtAIBpXGKthDf38sN0ibYnrgAVkCkPN0zXIvmUbsaw60vLtyfNTqCj4aoRwu1NibB4l+AZiPib4ybxwHEc+DWxRxETLb2Z+0DOhcjD+NyOY5fMxQjiHz5bp23X/69jutjKmvE3k7bKZEkfTz+/3YOtkAfsE0Pp1BkW+QS8BDi3qbu1iFTBxLhvuFgJdaRp496SLXCxNGo47DCKluLxPyYK/Z7HzbcgnWK0ZA4F+pgu+GzrE4UUHTh7Bu8yA/oYBtux/eVx69Q57PAcXZgZ1D/Qg/HM21aZFF4Gse4rwmcKFEFUNer+fxWSeYNut+lXjh5Yoce68L7KL/iKoUvt4p/DbwcvYzAoEqOrlJ04Hkc0aZ59WBiD7z16qafJQsemAsB1K7Dm1avec8n7AYGUNzeRz9YJkQWrOpVJMZSOfyZdMcSbb+okdaKhhbYgGtF461xRLwQCIwJw7HYfdfOQ/7fBn0fcQieQJNrFrCW+VuC0vVF8/zic5QFqDX1iNVQ0ZW66C+H1AdQVj1Dt5WwpgWvJvLQBcj1MUgXUmMonkARUaFow4Aej0Ab8O/4vA7WpNlZtUYKXNs3FBz17tGtXA0HcQItAiEpQDaVD7ayxLDTkpstBa08VPXYBqbqWJxAwcOUe1j+SlTyN96nMtNyDqRQr0xJzzoTR+xMpDknTHwZ5zDkr0K9uoR3CKcgYfeo1SkmceMeihsfSdjpgQtAOEt5UcPl3LwtIFQioSM6Bz5Q0x2V2lmIVngz/BWXbkjSS59pPAYlotD V9Pv0Zc9 nAKurZ1CYfkS6IUiLmNFUahIu6Eu+Afnwyd015kZzzf++ZZATj5D5+t/GDl8hRFBMOTV99htB/xipKN5ZnCg693aommcSIv0+QMUQZpStXg31L62/+QIM01yZ9jT50miGEkvA+Mw7l3lQ98GevOatcVyZAf83ybFeyEkNXRxfIyOfPE+7WKRQM/ODf5Z2sXh8igBXbCaRgpZ3JcXj1QAq7p3Fj2s2Uuo9CdYSWLEXIPYztXSko6I/1sPZX1jVVeZMT1AydfTMPsSDuy4i8WISAYRStixAO/nLp2YUxx32QJ5IBlyyD5uIqDSJTr/JBwykH5U7v3Xx92tzQt+Zr11gFV5e2CQuIa6v6j08 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sun, Feb 26, 2023 at 02:12:38PM -0800, Andrew Morton wrote: > On Fri, 3 Feb 2023 17:18:37 +1000 Nicholas Piggin wrote: > > > On a 16-socket 192-core POWER8 system, the context_switch1_threads > > benchmark from will-it-scale (see earlier changelog), upstream can > > achieve a rate of about 1 million context switches per second, due to > > contention on the mm refcount. > > > > 64s meets the prerequisites for CONFIG_MMU_LAZY_TLB_SHOOTDOWN, so enable > > the option. This increases the above benchmark to 118 million context > > switches per second. > > Is that the best you can do ;) > > > This generates 314 additional IPI interrupts on a 144 CPU system doing > > a kernel compile, which is in the noise in terms of kernel cycles. > > > > ... > > > > --- a/arch/powerpc/Kconfig > > +++ b/arch/powerpc/Kconfig > > @@ -265,6 +265,7 @@ config PPC > > select MMU_GATHER_PAGE_SIZE > > select MMU_GATHER_RCU_TABLE_FREE > > select MMU_GATHER_MERGE_VMAS > > + select MMU_LAZY_TLB_SHOOTDOWN if PPC_BOOK3S_64 > > select MODULES_USE_ELF_RELA > > select NEED_DMA_MAP_STATE if PPC64 || NOT_COHERENT_CACHE > > select NEED_PER_CPU_EMBED_FIRST_CHUNK if PPC64 > > Can we please have a summary of which other architectures might benefit > from this, and what must they do? > > As this is powerpc-only, I expect it won't get a lot of testing in > mm.git or in linux-next. The powerpc maintainers might choose to merge > in the mm-stable branch at > git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm if this is a > concern. I haven't really had time to page all of this back in, but x86 is very close to be able to use this, it mostly just needs cleaning up some accidental active_mm usage. I've got a branch here: https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/log/?h=x86/lazy That's mostly Nick's patches with a bunch of Andy's old patches stuck on top. I also have a pile of notes, but alas, not finished in any way.