From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50A93C4332F for ; Wed, 8 Nov 2023 15:38:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6DE746B027E; Wed, 8 Nov 2023 10:38:17 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 665376B0281; Wed, 8 Nov 2023 10:38:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 505796B0283; Wed, 8 Nov 2023 10:38:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 3D5936B027E for ; Wed, 8 Nov 2023 10:38:17 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 0B0921606E1 for ; Wed, 8 Nov 2023 15:38:17 +0000 (UTC) X-FDA: 81435193434.06.B90DF3F Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by imf11.hostedemail.com (Postfix) with ESMTP id D374E40021 for ; Wed, 8 Nov 2023 15:38:14 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=linutronix.de header.s=2020 header.b=bCHL0Uzo; dkim=pass header.d=linutronix.de header.s=2020e header.b=IiM8I42O; spf=pass (imf11.hostedemail.com: domain of tglx@linutronix.de designates 193.142.43.55 as permitted sender) smtp.mailfrom=tglx@linutronix.de; dmarc=pass (policy=none) header.from=linutronix.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699457895; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=18UXkYkGm08AQHZBxS21cLjRT0YoIXe2siFB63ueLa0=; b=sphvPEEeeM7uKIwrl1G4qyXAJKWTBe+GmyQ8wFOx47R+J1XfdM1JjQ4GttZSbc24bGZ5h4 sw4Qv31PyJpTlh7v12aca9pQy3i+4gK7sA4HhHB5svDgV6EAK6S72mNlIaThZ/t4YBbK7Y zx34aUlPBc+FCIrQ6shI3of9ORfgQH0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699457895; a=rsa-sha256; cv=none; b=JgPWY9MJbRkxWbUSXoEPpBV71v/KK0lzEgGQzrV7zXcIUgYDFWf/5hOfyVMZ1pSdTVEsY2 kP4PWnqUZSEhTVF7aqwknqW3W987Jk7sgsVSHTfIZkqiPPbPDBD2bNUrP9+hc4JIXy1PaF SlYh9rO3IhbAk5sYoS/+Zj9cI0afC/M= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=linutronix.de header.s=2020 header.b=bCHL0Uzo; dkim=pass header.d=linutronix.de header.s=2020e header.b=IiM8I42O; spf=pass (imf11.hostedemail.com: domain of tglx@linutronix.de designates 193.142.43.55 as permitted sender) smtp.mailfrom=tglx@linutronix.de; dmarc=pass (policy=none) header.from=linutronix.de From: Thomas Gleixner DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1699457891; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=18UXkYkGm08AQHZBxS21cLjRT0YoIXe2siFB63ueLa0=; b=bCHL0UzoqJ4k+ydiCqDSGPqPJk1wUNFPASj3NywFEUtwqv3YAgQgmza2shT9fwGWz4dSLf Z47yBjsBmo4xTShOD953CfYU6WeU9iYOa32v04j4K7e71oWQxN4bZ5ops6ySLFi0PboAqF 7+oBln+ji5AXRmrwT7LXevHuY42IMgcOgLtb6uE8yQ8NwA5EBpqkSc8yZajxkATB069U05 FC6m3hTwoyC3wyFG8hWCOl12LOC+x6gXW/KP17XQZfdlrPimDTrUQBzWlc2w1RygMPXGdW 06kgKW5NgVkFArzzednxNhbmALGRJvCRv79LVSG28eofYlglCyoIDzN/BqaGAg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1699457891; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=18UXkYkGm08AQHZBxS21cLjRT0YoIXe2siFB63ueLa0=; b=IiM8I42Oi9ibnNKbFzDNnHjXg2zB7ZBR8wakKl39Y2K3ylt3Glo4UPGTNrWoNgYvRHdAJK 9woTxhafpSOZwGCA== To: Peter Zijlstra , Ankur Arora Cc: linux-kernel@vger.kernel.org, torvalds@linux-foundation.org, paulmck@kernel.org, linux-mm@kvack.org, x86@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, jgross@suse.com, andrew.cooper3@citrix.com, mingo@kernel.org, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@aculab.com, richard@nod.at, mjguzik@gmail.com Subject: Re: [RFC PATCH 00/86] Make the kernel preemptible In-Reply-To: <20231108101330.GK3818@noisy.programming.kicks-ass.net> References: <20231107215742.363031-1-ankur.a.arora@oracle.com> <20231108085156.GD8262@noisy.programming.kicks-ass.net> <87bkc4lfxp.fsf@oracle.com> <20231108101330.GK3818@noisy.programming.kicks-ass.net> Date: Wed, 08 Nov 2023 16:38:11 +0100 Message-ID: <877cmsgsrg.ffs@tglx> MIME-Version: 1.0 Content-Type: text/plain X-Stat-Signature: 5hkebu6xxchss9p1hafhuiczyggxam7x X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: D374E40021 X-Rspam-User: X-HE-Tag: 1699457894-268339 X-HE-Meta: U2FsdGVkX1+eMgkCwoidQpEh0IMGq3OVwoe5F1DJtpZcrEDhrfXE6lGwwbJ9l8+GQFJRLhpdB44hLP06yQKuBP9G/c1P3Pkkt6HEg17tEVequ0hxiimWjTW+H/xaVS/5+AfWeZ6nkPVS9nVYD5oqkhJwlYSsgrAjNowHq4KWWe+6QCETo1XaHf0/ds6hoPzq7CG4In8eFc5/3eruzko4erpQRrUJvQKHkDA3iMBgXlrE4E/bLn1V125a6U/QjmaW5z2aOyb8L1ymR5jwpjBGjWle0Q84sW1+ijQLp92aUHai5wPX9RxYET/gXZ095Qcu4BenzDUWNezJ7oyACAMy1kWtUZ1QokqqCOnXjI0l/Gj09rjylwnHaVvJi/Iq+Cfywnx+kRrNseYoVQEnOYOp6jywbQWIz5z32P+ThCSRnvCrF1yNpwzGTZrRwWvd3GoPsxDZfYCJjVyKEY0TPHKGrk+2UnAYD1WJlRB2Zj/ULjG5AgTz/PgilFbRuIHCgNdIkOzSHesvmulFW0bkgodIxXSclAwWO2cRauF+qgnBnP3uoe5xrDqRo3hITVuEEnccWZuOz1nZq0+H9TxItoy5/IvHXI4UTzzd2d6Yo5MOZbBy/RJ2WU5NTtdAXHjw237CMTXW8S1tMOixs9JiaCTgNZMaYbYVT/xBoB4CcpvandqJc+Dl3owVcS/1jfSG9NjBHhxbrgl7E5+JXa9bFKRpqkPY+FZvEe/JTjW5l7gKzHRsoiJyQ0+nNuSTwfOij4lwbt1eXIqTwfXql60Pra1eiP3w2VYSy5irqtpS8rIc09JGWiwgGVScpHfnAv5bfAIXQysHbp3WDlBrhjPH9yrDd7NMMgV6gRrQ1BXFuxVqYmdNujrydD/dM5AKHizp3q1tvnMomOE6BOLLFZNvhFbWhKrOUglTbowv43hM1ZU+HzeyITIm3nGgwCbtxAd+x6ApjLWX84TnWPbDXmUy7Nt VWVvHz75 kMjusfM8Irwrxd5kneBsAC9V6J3ahprlUtiWF54C38qJuq1/R0SVYvMlLrYaXQG/rmzOvynsnF/JEB866dq2cVs2lwkvnwned2H/cOVzJWec0WOtyMK9gkxnORCoESndFTF9QOvktTA6Agcc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Nov 08 2023 at 11:13, Peter Zijlstra wrote: > On Wed, Nov 08, 2023 at 02:04:02AM -0800, Ankur Arora wrote: > I'm not understanding, those should stay obviously. > > The current preempt_dynamic stuff has 5 toggles: > > /* > * SC:cond_resched > * SC:might_resched > * SC:preempt_schedule > * SC:preempt_schedule_notrace > * SC:irqentry_exit_cond_resched > * > * > * NONE: > * cond_resched <- __cond_resched > * might_resched <- RET0 > * preempt_schedule <- NOP > * preempt_schedule_notrace <- NOP > * irqentry_exit_cond_resched <- NOP > * > * VOLUNTARY: > * cond_resched <- __cond_resched > * might_resched <- __cond_resched > * preempt_schedule <- NOP > * preempt_schedule_notrace <- NOP > * irqentry_exit_cond_resched <- NOP > * > * FULL: > * cond_resched <- RET0 > * might_resched <- RET0 > * preempt_schedule <- preempt_schedule > * preempt_schedule_notrace <- preempt_schedule_notrace > * irqentry_exit_cond_resched <- irqentry_exit_cond_resched > */ > > If you kill voluntary as we know it today, you can remove cond_resched > and might_resched, but the remaining 3 are still needed to switch > between NONE and FULL. No. The whole point of LAZY is to keep preempt_schedule(), preempt_schedule_notrace(), irqentry_exit_cond_resched() always enabled. Look at my PoC: https://lore.kernel.org/lkml/87jzshhexi.ffs@tglx/ The idea is to always enable preempt count and keep _all_ preemption points enabled. For NONE/VOLUNTARY mode let the scheduler set TIF_NEED_RESCHED_LAZY instead of TIF_NEED_RESCHED. In full mode set TIF_NEED_RESCHED. Here is where the regular and the lazy flags are evaluated: Ret2user Ret2kernel PreemptCnt=0 need_resched() NEED_RESCHED Y Y Y Y LAZY_RESCHED Y N N Y The trick is that LAZY is not folded into preempt_count so a 1->0 counter transition won't cause preempt_schedule() to be invoked because the topmost bit (NEED_RESCHED) is set. The scheduler can still decide to set TIF_NEED_RESCHED which will cause an immediate preemption at the next preemption point. This allows to force out a task which loops, e.g. in a massive copy or clear operation, as it did not reach a point where TIF_NEED_RESCHED_LAZY is evaluated after a time which is defined by the scheduler itself. For my PoC I did: 1) Set TIF_NEED_RESCHED_LAZY 2) Set TIF_NEED_RESCHED when the task did not react on TIF_NEED_RESCHED_LAZY within a tick I know that's crude but it just works and obviously requires quite some refinement. So the way how you switch between preemption modes is to select when the scheduler sets TIF_NEED_RESCHED/TIF_NEED_RESCHED_LAZY. No static call switching at all. In full preemption mode it sets always TIF_NEED_RESCHED and otherwise it uses the LAZY bit first, grants some time and then gets out the hammer and sets TIF_NEED_RESCHED when the task did not reach a LAZY preemption point. Which means once the whole thing is in place then the whole PREEMPT_DYNAMIC along with NONE, VOLUNTARY, FULL can go away along with the cond_resched() hackery. So I think this series is backwards. It should add the LAZY muck with a Kconfig switch like I did in my PoC _first_. Once that is working and agreed on, the existing muck can be removed. Thanks, tglx