From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B37EC77B7A for ; Wed, 17 May 2023 00:23:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E3E76900004; Tue, 16 May 2023 20:23:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DEF0D900003; Tue, 16 May 2023 20:23:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CB68D900004; Tue, 16 May 2023 20:23:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id BAA71900003 for ; Tue, 16 May 2023 20:23:12 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 70F9DC02B9 for ; Wed, 17 May 2023 00:23:12 +0000 (UTC) X-FDA: 80797847424.21.30C88E3 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by imf08.hostedemail.com (Postfix) with ESMTP id 92D0D160019 for ; Wed, 17 May 2023 00:23:10 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=linutronix.de header.s=2020 header.b=4Ty7eDNA; dkim=pass header.d=linutronix.de header.s=2020e header.b=m+qtwAhX; spf=pass (imf08.hostedemail.com: domain of tglx@linutronix.de designates 193.142.43.55 as permitted sender) smtp.mailfrom=tglx@linutronix.de; dmarc=pass (policy=none) header.from=linutronix.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684282990; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QSz/tG9SR6kUBVL7qlRk6WJM8xEw2Oijp2gSEJgARzI=; b=aXWEIJm/ASrFQgN5+uo09H+EKEVzCXC0l2x+JAm5bDTI0z6sSoYCKY1RadX9DTDwlnyWqQ 8x0Rbk/czM6GiTnM5TdhybLbFBzepUI7Kw3MrSWxXF/HUqch2TPoHfccCbpMT6X8cRjbZS ch8/T/wI7EkkVrE80dN2kFuGeT0Eplo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684282990; a=rsa-sha256; cv=none; b=DVP0M2sBVrywfYVKg8jwU6nh3TPb/H6Xqp0NTQ3v0t5qH7UHNbZ/aZeEgj7CTuMhxpWq// 0bwwcnyQjfbKM9optQWH7vOIzgwGZYqK6LW0x5H3mBPEorFFkHOku7XpBLmKEHYVhjqKkm csdHqhX+p2dDh2oI9tFCfs3H825Nbnw= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=linutronix.de header.s=2020 header.b=4Ty7eDNA; dkim=pass header.d=linutronix.de header.s=2020e header.b=m+qtwAhX; spf=pass (imf08.hostedemail.com: domain of tglx@linutronix.de designates 193.142.43.55 as permitted sender) smtp.mailfrom=tglx@linutronix.de; dmarc=pass (policy=none) header.from=linutronix.de From: Thomas Gleixner DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1684282988; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=QSz/tG9SR6kUBVL7qlRk6WJM8xEw2Oijp2gSEJgARzI=; b=4Ty7eDNA6G/w9N9hxKTMmX6EGof4VAi17cFCItKWdwL+F4oJAXgqx4irPgjKn2X0fkvxhc c8j10WfyjvHnhOQbTcsr5w5BWWHr+P8hqlDdFDZuQ4krRm0yujgYab7FlIbFK8MJELpJMd 4qoe+Z85gbBKOoyjn7VmnwC2aDClGqIO7zhLiE4GD99/DAv+Q7GZQjjSRVCkBKd9Nf/m8z ZzrLBT4P0AK/crjbyWbvkhAGIvqTnIxeo2UeSo8hsm8eXXtx6Brv0vx5z060UqTINalqAk TFhfxvLM15pSoWbTKlSvUcQomfpIZoWniUN+DwUeAbysrT1UytnqVJ+RUIX/gQ== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1684282988; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=QSz/tG9SR6kUBVL7qlRk6WJM8xEw2Oijp2gSEJgARzI=; b=m+qtwAhX58bp3HAuAdreKyAxJhJiMXAzTIEnBpjaCUKndlmNwr++AogOwgR1YpnMJzbuAV JDNaohGhgXuVMHCQ== To: Nadav Amit Cc: Uladzislau Rezki , "Russell King (Oracle)" , Andrew Morton , linux-mm , Christoph Hellwig , Lorenzo Stoakes , Peter Zijlstra , Baoquan He , John Ogness , linux-arm-kernel@lists.infradead.org, Mark Rutland , Marc Zyngier , x86@kernel.org Subject: Re: Excessive TLB flush ranges In-Reply-To: <87bkik6pin.ffs@tglx> References: <87a5y5a6kj.ffs@tglx> <87353x9y3l.ffs@tglx> <87zg658fla.ffs@tglx> <87r0rg93z5.ffs@tglx> <87cz308y3s.ffs@tglx> <87y1lo7a0z.ffs@tglx> <87o7mk733x.ffs@tglx> <7ED917BC-420F-47D4-8956-8984205A75F0@gmail.com> <87bkik6pin.ffs@tglx> Date: Wed, 17 May 2023 02:23:07 +0200 Message-ID: <87353v7qms.ffs@tglx> MIME-Version: 1.0 Content-Type: text/plain X-Stat-Signature: wm9tqpsk9j3aum9wuh6wpfmtdtbyrdt1 X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 92D0D160019 X-HE-Tag: 1684282990-880665 X-HE-Meta: U2FsdGVkX19c0kUBfTkqb4iYPV4flIWanX0tqH1MHsBsVdp9ADyI9g9ov1UxzpSisP9inMwitf2lgPo8OQkB6fUK7JHZJ0v011SVJc/BybZd9xLnMUR5DDbTaz5B2IiGipuTWtUF2qE3NgEiF15PigCPbGmwGNmhXa5TLtW5P2nF57WfXMkzVBYgA/eESv6viRInlSohDfY3nvh0OJ/dIFgtJlnzhw1bgkJWlrAo4GiQcZ0hKd2kQV/K+57lKIi6wJSBp7omO41uRtKTCu0cN5sYygP9I2NLm2fkMTd3yGt55HxL20dMI5zNqlQwY1XNCbilX76SJ6Z90/XxPHQ5hjHeCezML2F16KzPiS4A8Lk5VU+Y9fSOoznQrQPaMKi/2LmI5z6VKdkjLTXFFfF6HLzr+MmjDwHxT7c6Ih2JEJeLap5gFSu1gt+s+AzC45wLPeOSh7DKik4yM85Er+qTSy43jN0ptdpbce+occhNfKQRtdieBDayWWnkJqEiJndcdD+1PJyp9njw8CFEQ6YYwWP1sTS/bl1FM+Nt5m7ss00EEYVfi+BJdd7Zi1x549J3j0BffCBTTR7ZISjCVEkPYSgG96nayPZvUUvNdBNqX4bl7Qr+s0Q9ZDz1mcF+5xolfbkgUPI2uPP8ZMmVqVMa0oZn66JFx/d4B2idYkIoeU4TiiX8mUyrmD7reaTNjJ0d2GirpsXDc+GSIXSLpb4fudY1gyfKZm1WVFJdQR71l1dMcuJlUfInn+b28ylV9YoIVaTc8NE7Pi0xLbATj5fH5cnkI9JkMAOW0jcKCy6pXDZrK27w2dTXz7XhFo2WNC2+tDEbaGBbmXX5qkZWnTLh+JgXrC7WuOwzhwSKcttkQ/dxFO8DhkBqYoo5W56aLUiz9rGu0Nfe4uJy6ZRCwoiAX8rc1YPcgLKqcik7uoY95/q3VLVFwho7JxoN690iZ9bHu0zEQr7Wpudg5VY/TsB 8bLAbGXt dw0pH/mFqNIyBvmtG0anyIdbxGVSvJS3XOZaAbqrrydO8m0nci3PYfKsI9U4py64RuXsiFm4+C0Ynv5pIvpDSK1g8rgyR7ayNfDRWLzAAs0MzZNzKXiLLW1b6dA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, May 16 2023 at 21:32, Thomas Gleixner wrote: > On Tue, May 16 2023 at 10:56, Nadav Amit wrote: >>> On May 16, 2023, at 7:38 AM, Thomas Gleixner wrote: >>> >>> There is a world outside of x86, but even on x86 it's borderline silly >>> to take the whole TLB out when you can flush 3 TLB entries one by one >>> with exactly the same number of IPIs, i.e. _one_. No? >> >> I just want to re-raise points that were made in the past, including in >> the discussion that I sent before and match my experience. >> >> Feel free to reject them, but I think you should not ignore them. > > I'm not ignoring them and I'm well aware of these issues. No need to > repeat them over and over. I'm old but not senile yet. Just to be clear. This works the other way round too. It makes a whole lot of a difference whether you do 5 IPIs in a row which all need to get a cache line updated or if you have _one_ which needs a couple of cache lines updated. INVLPG is not serializing so the CPU can pull in the next required cache line(s) on the VA list during that. These cache lines are _not_ contended at that point because _all_ of these data structures are not longer globally accessible (mis-speculation aside) and therefore not exclusive (misalignment aside, but you have to prove that this is an issue). So just dismissing this on 10 years old experience is not really helpful, though I'm happy to confirm your points once I had the time and opportunity to actually run real testing over it, unless you beat me to it. What I can confirm is that it solves a real world problem on !x86 machines for the pathological case at hand On the affected contemporary ARM32 machine, which does not require IPIs, the selective flush is way better than: - the silly 1.G range one page by one flush (which is silly on its own as there is no range check) - a full tlb flush just for 3 pages, which is the same on x86 albeit the flush range is ~64GB there. The point is that the generic vmalloc code is making assumptions which are x86 centric on not even necessarily true on x86. Whether or not this is benefitial on x86 that's a completey separate debate. There is also a debate required whether a wholesale "flush on _ALL_ CPUs' is justified when some of those CPUs are completely isolated and have absolutely no chance to be affected by that. This process bound seccomp/BPF muck clearly does not justify to kick isolated CPUs out of their computation in user space just because... Thanks, tglx