From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 457E3C02188 for ; Mon, 27 Jan 2025 10:37:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A0A8528013D; Mon, 27 Jan 2025 05:37:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9936128013A; Mon, 27 Jan 2025 05:37:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7E6AB28013D; Mon, 27 Jan 2025 05:37:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 5BE3E28013A for ; Mon, 27 Jan 2025 05:37:04 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id BAEEBB07EC for ; Mon, 27 Jan 2025 10:37:03 +0000 (UTC) X-FDA: 83052879126.05.F0CD627 Received: from mail-lf1-f43.google.com (mail-lf1-f43.google.com [209.85.167.43]) by imf22.hostedemail.com (Postfix) with ESMTP id C07B4C0002 for ; Mon, 27 Jan 2025 10:37:01 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=SDGzcU1n; spf=pass (imf22.hostedemail.com: domain of urezki@gmail.com designates 209.85.167.43 as permitted sender) smtp.mailfrom=urezki@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737974221; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=W4QFE15tzxGQeI7M/zW2nqDheIV1bS+m/FQTTUzpTLk=; b=eq+ccPeNafPBIbdGTwK7auYLZoYq778d3IpkYQvyG+y7YKXWafR33vpf0u7/bZ+I2+wKD/ 0kPjH1uPgDZiQLwxpO3IgeQs01UOHF5LGE3WIwtg6DP/aAuGjr/dYzcOA4DTV5uKptSmue blU5OekPejlC0k41jFYcGzdqMZWHv0g= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737974221; a=rsa-sha256; cv=none; b=VAH+qCueg++87LXsjfa607Aw1Y4upJUj0Wh29P8vzOctb92XIwh2DEnOpAGajBmEWUZ2uP 3HrskMu8BMZHO3hlSwRIHxATa3wQ/fUBmsAuAnAcFsRA37iGQnJl5mDMs93IgODsGr72bf eBxk8hU5SOkGzN8EOR51HIRPojNn6eo= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=SDGzcU1n; spf=pass (imf22.hostedemail.com: domain of urezki@gmail.com designates 209.85.167.43 as permitted sender) smtp.mailfrom=urezki@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-lf1-f43.google.com with SMTP id 2adb3069b0e04-54024ecc33dso4304717e87.0 for ; Mon, 27 Jan 2025 02:37:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1737974220; x=1738579020; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:date:from:from:to:cc:subject:date:message-id:reply-to; bh=W4QFE15tzxGQeI7M/zW2nqDheIV1bS+m/FQTTUzpTLk=; b=SDGzcU1nwH05Yg2ref8lWeTImu3osAVQQ756gvlCowjD7u3zhlkFbUwYioRFuJO1oW dCXZQStNK/7bHHJW/pkjWAl5Xzp3fWSJCZkNsqZFAPWOXI3y0lGEKw2WShmdHM1K1E5z zQ5tugXVMbcw6dN7OjvDm0jm2uQmZ3uDOa7XpkvRy/+wFY/VR8tZ/ZfnU57MAMwUYjRH J6+9qsiLp1Mm1lPh2i07akd/i9tthKZFEjXbxJ5vNjLRzVD8xRkjN4Dy2TmA+tvkDjsw d+81swsc92BQ/z5NFxRA2mScNjBYZwVPfsLC5Ny9VlCKXSf9bJwYMI02XXnMIRcFuo2h rk4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737974220; x=1738579020; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:date:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=W4QFE15tzxGQeI7M/zW2nqDheIV1bS+m/FQTTUzpTLk=; b=TohKZTN6V8EFAOMZAuKID1F228zUBfe+utP0sagA8QcnNhwwMiBFtgbNBZQPTaDkgp nAZDiMz3uxFQxFqFlFni7w5i1gBVRFLV4A3haWp4OpWhJL4kOPon4xVNZlgu9FGgP2rY n1V3c86lePyAmvdVbnCCOslN7VeuecIRKWiG4oT4SwWT/CXgxGgBQzdCgO/zxwuHRAmT zMiYw+E7heMMwb1lpNWQUDd1TAkzc/kGMacvXjGP+9+nqdmmSIAddGlV9T4cPf6WlO6Y ckNA7sK/z0elNyL6x4VT4wpnXKM+Wop0/TyLmCNc6uwSXhRI0HEsAvkaIckKBhUaJfua zFrQ== X-Forwarded-Encrypted: i=1; AJvYcCUFv+Im9TaEQnbWfM2MUsxXYzJhOvKZQbvrsgk5ZDNO557o6WabcHyGKLqoV0YmUHtj8zI9USlAEA==@kvack.org X-Gm-Message-State: AOJu0YxdcgbFfrNbcQJ6ttwcA8D3Fb/+Ok1YHHClEPLk7ElzjMc3xIZY xuwepF6Ov5ujltMlY0/0VoPpA2m+C36M5oRmsO6OTxMt/Hdg0SI8 X-Gm-Gg: ASbGncvbuG00dVRyU2iEDFlbJVVJscQ421Wb991unI1as6OzlZUynec7OKd32N0UnqY 82KMVfZ+O8oDCRR4u+Hd9hm1MX+D2j2VzmFK4f58FRcmmqGe4EgdnmDrLsFsRABMLOX0HZj5PcE 6BymUAH47bAF9sh5OuZ729NEudTjZqZR86ABEUnV5r90u4GmaN1f8T9e73XpoGCH4oQdV86+zWS TDYLMOoca/SCMUoYgPD2+aa3Hz3HFStsbvvliJ/X/mTz34X0B4NUN4+FTgx6fzfC0AIUipugG9h 3OmWVOdWMcGhESw6QTgVtLiF2UKvj8kHjow= X-Google-Smtp-Source: AGHT+IEcn2mDT5d0SWT5jg0poO+BtsvQPwWj5ydN+QiFTcCpstqFwzzlc4zIAZ/gMSqSpGuiyMrqQQ== X-Received: by 2002:a19:8c1d:0:b0:543:baa9:a48a with SMTP id 2adb3069b0e04-543baa9a4bcmr6967602e87.27.1737974219462; Mon, 27 Jan 2025 02:36:59 -0800 (PST) Received: from pc636 (host-217-213-93-172.mobileonline.telia.com. [217.213.93.172]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-3076bc1956csm14103761fa.70.2025.01.27.02.36.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Jan 2025 02:36:58 -0800 (PST) From: Uladzislau Rezki X-Google-Original-From: Uladzislau Rezki Date: Mon, 27 Jan 2025 11:36:51 +0100 To: Valentin Schneider Cc: Uladzislau Rezki , Jann Horn , linux-kernel@vger.kernel.org, x86@kernel.org, virtualization@lists.linux.dev, linux-arm-kernel@lists.infradead.org, loongarch@lists.linux.dev, linux-riscv@lists.infradead.org, linux-perf-users@vger.kernel.org, xen-devel@lists.xenproject.org, kvm@vger.kernel.org, linux-arch@vger.kernel.org, rcu@vger.kernel.org, linux-hardening@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, bpf@vger.kernel.org, bcm-kernel-feedback-list@broadcom.com, Juergen Gross , Ajay Kaher , Alexey Makhalov , Russell King , Catalin Marinas , Will Deacon , Huacai Chen , WANG Xuerui , Paul Walmsley , Palmer Dabbelt , Albert Ou , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" , Peter Zijlstra , Arnaldo Carvalho de Melo , Namhyung Kim , Mark Rutland , Alexander Shishkin , Jiri Olsa , Ian Rogers , Adrian Hunter , "Liang, Kan" , Boris Ostrovsky , Josh Poimboeuf , Pawan Gupta , Sean Christopherson , Paolo Bonzini , Andy Lutomirski , Arnd Bergmann , Frederic Weisbecker , "Paul E. McKenney" , Jason Baron , Steven Rostedt , Ard Biesheuvel , Neeraj Upadhyay , Joel Fernandes , Josh Triplett , Boqun Feng , Mathieu Desnoyers , Lai Jiangshan , Zqiang , Juri Lelli , Clark Williams , Yair Podemsky , Tomas Glozar , Vincent Guittot , Dietmar Eggemann , Ben Segall , Mel Gorman , Kees Cook , Andrew Morton , Christoph Hellwig , Shuah Khan , Sami Tolvanen , Miguel Ojeda , Alice Ryhl , "Mike Rapoport (Microsoft)" , Samuel Holland , Rong Xu , Nicolas Saenz Julienne , Geert Uytterhoeven , Yosry Ahmed , "Kirill A. Shutemov" , "Masami Hiramatsu (Google)" , Jinghao Jia , Luis Chamberlain , Randy Dunlap , Tiezhu Yang Subject: Re: [PATCH v4 29/30] x86/mm, mm/vmalloc: Defer flush_tlb_kernel_range() targeting NOHZ_FULL CPUs Message-ID: References: <20250114175143.81438-1-vschneid@redhat.com> <20250114175143.81438-30-vschneid@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: C07B4C0002 X-Stat-Signature: mhm94wzfmigffgsobftb8wg81wzux84t X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1737974221-169018 X-HE-Meta: U2FsdGVkX1/yAwUst5WeEs8xb9qkxO3znXW2MclfH+xYUnyrNrtnr+y6SQzYJe2p/KNHN6fuK/JlTLvDLP/Q7DjI8bECbvdFvWm+1atAJFR+P1uS+6fFtw8M7by5jfk9y3qgeXOHU2m6cy8M/ErY8LyTtqhRLuo1s5xZcFJyn0i5LBQu+SMvGzZ66vyDJOIeQ3SnsKZnLipJ2hIpZA8XIp9zjcgJIA8UqunLKnC4rI6au3mLTFlF8mNpzpvfzRqFKUaKm0e7f+ZV3yssNowoSl3WplAV6Y9nTtjt/GAGLARtPmzu/W6HpHVX6mO9//hFqjKuVaGNIqF2fY95OvbX8t8nVilEYQVrx331+OgoOaynLVXb2ztm0j4lQxX7S///YZs617wfabyPQ+S1DhtrVnQNf01KMgizyCTpJ3Z77gCZLObXcR9uzKaAa0f0hVZFB7tnK1UhxOg9hAs9pZwfg/BGyQgXPkuyeErziShrvD281TeJbnviXWBYDYVYLOvGlIpwI0q87uFbgsZ1kGqY6OTajSxNvngMp5jy8vnlCaTm/3luv1RwrWFK4y3H4JzFyy8a+IGJFYzTaWOJr0sdzhzuJosfwl+JYY0pqCgr5Uoaxuse14b0vU7asFAy6hk2j8jOluYKZ3fmjgq7qh6ouqOEInIE4VOl2LJnjvp5m609J2OspT+a7YmoF75HcFHfrLypN6UKLAslqoSlI35GGTRlc6U/dcguLYFwsLgeKD11uj3oS1QRXIb+R7Mx9Sbe7k1k/KOTjrAVUXmMiJ/a9nJmvLcZfH2oa/SV5ekgN5CA+X4ofjJDKF66hTc4e9OReerLhvyMN6OAyWqNOEUWYXHdbs3SjSIX71fYC7l1+Frtr2QKDSuNXGu+g0MZT1qAtQLqy3ngo5HvipEhEkuDrVd/ppfbIhqA3JTuGHk6Xel/0ULwWSzFuRgmREmqYnG4ggqesjvHWOhOLf8EuW7 qxWgCQ0s Krn26l6GmO9VwxuTxbup7y+BFGDbjfDcrOwcfLFcaHpQwklOTkxGshY6vPZilp7I2c/MTIDi6jwxlxQyBbmX4BpGQ0usthpSaCm5NSuMaw4bVmtKZaNrPWMqNomoScrIsa6RdYrPt4dw86N4UNsxmAN84ascMt+0F0tRjLZGJbkKk5RmQLgHaPE/hCJmNC8+pmsl9lsl6Qh6T5YVb66NyKKw6IzQoSyKGGA/EtMnuUfmCJUMeaMyCK2+wWt0f4UkiOYX2kQgtrV/Fe1HCuccHJppwvH5RBSCbSVKlVaMjTT40uQEah7mjwGu+I68l+IENddRo5zNi+8kxG98gpKj4fPu4lSMFhUjsrhp26uh2s4E4SIMj0qrfkWx54/d4oS6gHIzyvVP7wwqsZho12LTSaiztoHZ39F0P+MZn9fWp8cF2Y+u7PUV+wEfdenndEFQvZaPvdSDRiL8BpTJKtqYhG42zIGk2hfO9fh32AHWGbyTOSH68RoIukDZlzxlS8ik69ziL X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Jan 24, 2025 at 04:22:19PM +0100, Valentin Schneider wrote: > On 21/01/25 18:00, Uladzislau Rezki wrote: > >> > > >> > As noted before, we defer flushing for vmalloc. We have a lazy-threshold > >> > which can be exposed(if you need it) over sysfs for tuning. So, we can add it. > >> > > >> > >> In a CPU isolation / NOHZ_FULL context, isolated CPUs will be running a > >> single userspace application that will never enter the kernel, unless > >> forced to by some interference (e.g. IPI sent from a housekeeping CPU). > >> > >> Increasing the lazy threshold would unfortunately only delay the > >> interference - housekeeping CPUs are free to run whatever, and so they will > >> eventually cause the lazy threshold to be hit and IPI all the CPUs, > >> including the isolated/NOHZ_FULL ones. > >> > > Do you have any testing results for your workload? I mean how much > > potentially we can allocate. Again, maybe it is just enough to back > > and once per-hour offload it. > > > > Potentially as much as you want... In our Openshift environments, you can > get any sort of container executing on the housekeeping CPUs and they're > free to do pretty much whatever they want. Per CPU isolation they're not > allowed/meant to disturb isolated CPUs, however. > > > Apart of that how critical IPIing CPUs affect your workloads? > > > > If I'm being pedantic, a single IPI to an isolated CPU breaks the > isolation. If we can't quiesce IPIs to isolated CPUs, then we can't > guarantee that whatever is running on the isolated CPUs is actually > isolated / shielded from third party interference. > I see. I thought you are fixing some issue. I do not see a straight forward way how to remove such "distortion". Probably we can block the range which we defer for flushing. But it also can be problematic because of other constraints. Thanks! -- Uladzislau Rezki