From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 162F7C61D90 for ; Tue, 21 Nov 2023 15:32:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AB47E6B0484; Tue, 21 Nov 2023 10:32:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A64CA6B0485; Tue, 21 Nov 2023 10:32:55 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 90D806B0486; Tue, 21 Nov 2023 10:32:55 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 7D92A6B0484 for ; Tue, 21 Nov 2023 10:32:55 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 4BC751CAABE for ; Tue, 21 Nov 2023 15:32:55 +0000 (UTC) X-FDA: 81482354310.23.14D9A70 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf05.hostedemail.com (Postfix) with ESMTP id 05CC3100004 for ; Tue, 21 Nov 2023 15:32:51 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=UARqvyaw; spf=pass (imf05.hostedemail.com: domain of "SRS0=TAUs=HC=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org" designates 139.178.84.217 as permitted sender) smtp.mailfrom="SRS0=TAUs=HC=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org"; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1700580772; a=rsa-sha256; cv=none; b=DAj6+TdWbdJq9Qjf2YD9FgcWdimkODnQrWa8P8D6glOX4cIK060djPa5K/Y/nhVJVgy7ZE uqnuyxiLNmDf2hnMnRHhHsggXIG/m3ND7zvvAXOzTtLgQbqWvcB45gU287/cpyHHn7g9LZ gdu2j4ICDgfhUhHA6WzJT1Of4iiX3xY= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=UARqvyaw; spf=pass (imf05.hostedemail.com: domain of "SRS0=TAUs=HC=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org" designates 139.178.84.217 as permitted sender) smtp.mailfrom="SRS0=TAUs=HC=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org"; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1700580772; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Kl0O+lt4r3bwWQ9LkjfY7qAR6cI2tnpj+xCGPJ/+Jqk=; b=62w/bjF+sc+S3Ixxa2tNAWFCwMfkZUZKT/3XuoAyWOiwCFXxoPTxLETh2BELrNNfK7OLOs 3Zv14OFyma6tBsT4Bs2VAGFWO56k0cVBrDwBaSVAToMc7QsecMQpCi02FSRzU67DjX8H16 0At0mQJYuCopmeYdfs68MT4Lcvhld0w= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id F345561798; Tue, 21 Nov 2023 15:32:50 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 97346C433C7; Tue, 21 Nov 2023 15:32:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1700580770; bh=Ln95rqmdPgQ/SblMckJHRPBdwum+BpDOGzWgm9k+tYM=; h=Date:From:To:Cc:Subject:Reply-To:References:In-Reply-To:From; b=UARqvyaw5z+sfB8F6cDkXbEtmkyPZDEsqpKXzXi9rlh0V4O4HVCUZQNuuN+gEgIAM 6zYh7Y2Hq7iZ9Cfy5pbk1bfFKB3l/2MXEEmvUOM8EfS51MzOjHeCUT49WUDSRNhWqg qxJI+4aYelvtrgYI2Wyo6Vsz0ybQ4ZpJXICrWzE1120QV7ZUXYDOrS6vawMB8OqYra tIFj6T49+kTKZehEY+D9MtimftiCzP4I1T7NKtIyoaLAhPpp8GYUmk1nFGGveIrbXw a6MkCQRNdnh86M2pSHOLyaHZxP7rNFuVnSiZgQlpnD2nyrXgNIj4V1uU6IJXnEEFBg kZOZ++GAdpeEA== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 30FECCE04BD; Tue, 21 Nov 2023 07:32:50 -0800 (PST) Date: Tue, 21 Nov 2023 07:32:50 -0800 From: "Paul E. McKenney" To: Z qiang Cc: Ankur Arora , linux-kernel@vger.kernel.org, tglx@linutronix.de, peterz@infradead.org, torvalds@linux-foundation.org, linux-mm@kvack.org, x86@kernel.org, akpm@linux-foundation.org, luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, boris.ostrovsky@oracle.com, konrad.wilk@oracle.com, jgross@suse.com, andrew.cooper3@citrix.com, mingo@kernel.org, bristot@kernel.org, mathieu.desnoyers@efficios.com, geert@linux-m68k.org, glaubitz@physik.fu-berlin.de, anton.ivanov@cambridgegreys.com, mattst88@gmail.com, krypton@ulrich-teichert.org, rostedt@goodmis.org, David.Laight@aculab.com, richard@nod.at, mjguzik@gmail.com Subject: Re: [RFC PATCH 48/86] rcu: handle quiescent states for PREEMPT_RCU=n Message-ID: Reply-To: paulmck@kernel.org References: <20231107215742.363031-1-ankur.a.arora@oracle.com> <20231107215742.363031-49-ankur.a.arora@oracle.com> <2027da00-273d-41cf-b9e7-460776181083@paulmck-laptop> <87lear4wj6.fsf@oracle.com> <46a4c47a-ba1c-4776-a6f8-6c2146cbdd0d@paulmck-laptop> <31d50051-e42c-4ef2-a1ac-e45370c3752e@paulmck-laptop> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 05CC3100004 X-Stat-Signature: gkiicxpa1wu54e78rereuryzme1utkar X-Rspam-User: X-HE-Tag: 1700580771-734360 X-HE-Meta: U2FsdGVkX1/EYL4Lu+VzEinyFeKvrYbfsfF1BwwKO47cRB4eZd4YUdDGHMLbtMOEhjzXSwxbQujuyzdHsP9domq+TFt+6SNejZjcWLMvKKXGMB6+fbCzNFgZUAhwoi7OZFmkjdjMrKdBGKp9ajWCYiTKYs4cY2hXkAfHpVbCa0SxsF4GbS3FO2G9Uv17DeAlfvSzcNNXcbanyFXkdK/O7tariSAhPuw2+ZRFbjie+GIei9o2xvW3BAgvb15ZomcVUc3NixAGccTIMs6l1p9ObOQxDpDxjfsyUMoC2q4D3fM/RFB6uZyxo7+7k6bRcSwVdoPEUK1+lojKMxi9q/INhG8tqlx6y1tToYIRfeyD+/cf6QaSr02slxrDEUuZKWrfxVgOWb+1rnG+7uvo5BVnMwFdP5glGBfOSdUfEVYpd512qdyqz2Wfzdz8Fd9bhW+mqEFy/Il9mjhn0ufBABnTZr6dtIw+pODIl3f/sFr60cbBbjM7amljfTIYra3fMM/cWNYcLn5fDGd5NF53cvsfWpvu5UCeLXA9t2kND6jRUyoFiWki7imBlHgeW8JriGff3TnzsoW6QPCRYadP2dgjwuKD57tYB+axk5+sXeaumULH8vuJwLWzKE7PKHLeYTrCOdb26bJASWfy0MOCzNcog2zATPDL74KOl0rbSrNuJl73ukqd9Hv0P8v40TqsqED/hK6ZZxnNz3R168Li3Wim7BZ3enaQ33cs76FkzAQ06V5BwwhNm36BCTHFzY8G/qqZQ7mgWR6Da/zIJQbKGw2UbOw6Q8TW/5r8d9aD0PScDP4wskH0MY5NPG1aw473wkxnKqRwe3c5O1x6XOXMDkUBOf1cQ+4hGJXBkJgU12+/mgKt24XuoYNjr5HBOe2iwvhKqd20Ybo+hF5ptwKNZ4rCKh12kA6opTTV+uU6SO53YSPPoYwV9j3Vb7+O5P61a5yFrOCCVHOts7s+Htqn2CY 2c+9VeTb /qTHzLoiTa1A3IEPWn8hEpSFCs+0LKQoK0mYI+t+8u6T2H8w+kiO6Xv3850NGXcKkpL/t4pHXjXV/lzYRHv9PsINBCWL4xSeLBICdKzz65bkw1yfboZde8OclwSs4Op0pvG7mjlcoVEjXpxOmYZktmG422zqtaeJ2G4kNbzntgmsP0DzxC/dKobxqhuwN3TBwYiV1T7kQyBozS0Q1g8tyAcc6d71vPpJANET5Yxh/CB3ycqfXSqiIrx+MFN/1xMDaikNTa15r8MLJxmlT2BTyZxdwuFpUmj/HVrdyr96X4N11MOP4DexSG9ZBaomu+7mvzxbDwoKptIZl/0irciz2YKkMOPxDOeV4YeFe09bnoWHxmt07wDnQgV96b6hJQtatS3aMpXJCfTnltt1tzRjAVOYH2x/+y/WaIj/e/PMO1Age94B70s7EtZbNFjjomI0ydsPx8TYaCHVjOCzbBdyXvIJzDPDdFRL2DdtO+Ic3QE7L89qVzMg708Qz2evCim20PGS33nJSI0MIa9+boHtvCnI3F6CaAIYW7QipWqw8vqgDE66zNHTagQG23boIG5WZWfSD1KThUulGc1CYE18Yy0dhWgqS2aSwHVTdeA8qg3B7rTvmwIScOpN57AgIBLxDywCuaR+uZtD/JRl25D97Unmj3gtLNEdHhiMEU+z9cHyGh4U6SlkzOeW8c4f2UKhgPVZbmkZXHcLYWHmNMBTZpbliNvdOBn01KrvlPBo8qXaI8So= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Nov 21, 2023 at 02:13:53PM +0800, Z qiang wrote: > > > > On Mon, Nov 20, 2023 at 09:17:57PM -0800, Paul E. McKenney wrote: > > > On Mon, Nov 20, 2023 at 07:26:05PM -0800, Ankur Arora wrote: > > > > > > > > Paul E. McKenney writes: > > > > > On Tue, Nov 07, 2023 at 01:57:34PM -0800, Ankur Arora wrote: > > > > >> cond_resched() is used to provide urgent quiescent states for > > > > >> read-side critical sections on PREEMPT_RCU=n configurations. > > > > >> This was necessary because lacking preempt_count, there was no > > > > >> way for the tick handler to know if we were executing in RCU > > > > >> read-side critical section or not. > > > > >> > > > > >> An always-on CONFIG_PREEMPT_COUNT, however, allows the tick to > > > > >> reliably report quiescent states. > > > > >> > > > > >> Accordingly, evaluate preempt_count() based quiescence in > > > > >> rcu_flavor_sched_clock_irq(). > > > > >> > > > > >> Suggested-by: Paul E. McKenney > > > > >> Signed-off-by: Ankur Arora > > > > >> --- > > > > >> kernel/rcu/tree_plugin.h | 3 ++- > > > > >> kernel/sched/core.c | 15 +-------------- > > > > >> 2 files changed, 3 insertions(+), 15 deletions(-) > > > > >> > > > > >> diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h > > > > >> index f87191e008ff..618f055f8028 100644 > > > > >> --- a/kernel/rcu/tree_plugin.h > > > > >> +++ b/kernel/rcu/tree_plugin.h > > > > >> @@ -963,7 +963,8 @@ static void rcu_preempt_check_blocked_tasks(struct rcu_node *rnp) > > > > >> */ > > > > >> static void rcu_flavor_sched_clock_irq(int user) > > > > >> { > > > > >> - if (user || rcu_is_cpu_rrupt_from_idle()) { > > > > >> + if (user || rcu_is_cpu_rrupt_from_idle() || > > > > >> + !(preempt_count() & (PREEMPT_MASK | SOFTIRQ_MASK))) { > > > > > > > > > > This looks good. > > > > > > > > > >> /* > > > > >> * Get here if this CPU took its interrupt from user > > > > >> diff --git a/kernel/sched/core.c b/kernel/sched/core.c > > > > >> index bf5df2b866df..15db5fb7acc7 100644 > > > > >> --- a/kernel/sched/core.c > > > > >> +++ b/kernel/sched/core.c > > > > >> @@ -8588,20 +8588,7 @@ int __sched _cond_resched(void) > > > > >> preempt_schedule_common(); > > > > >> return 1; > > > > >> } > > > > >> - /* > > > > >> - * In preemptible kernels, ->rcu_read_lock_nesting tells the tick > > > > >> - * whether the current CPU is in an RCU read-side critical section, > > > > >> - * so the tick can report quiescent states even for CPUs looping > > > > >> - * in kernel context. In contrast, in non-preemptible kernels, > > > > >> - * RCU readers leave no in-memory hints, which means that CPU-bound > > > > >> - * processes executing in kernel context might never report an > > > > >> - * RCU quiescent state. Therefore, the following code causes > > > > >> - * cond_resched() to report a quiescent state, but only when RCU > > > > >> - * is in urgent need of one. > > > > >> - * / > > > > >> -#ifndef CONFIG_PREEMPT_RCU > > > > >> - rcu_all_qs(); > > > > >> -#endif > > > > > > > > > > But... > > > > > > > > > > Suppose we have a long-running loop in the kernel that regularly > > > > > enables preemption, but only momentarily. Then the added > > > > > rcu_flavor_sched_clock_irq() check would almost always fail, making > > > > > for extremely long grace periods. > > > > > > > > So, my thinking was that if RCU wants to end a grace period, it would > > > > force a context switch by setting TIF_NEED_RESCHED (and as patch 38 mentions > > > > RCU always uses the the eager version) causing __schedule() to call > > > > rcu_note_context_switch(). > > > > That's similar to the preempt_schedule_common() case in the > > > > _cond_resched() above. > > > > > > But that requires IPIing that CPU, correct? > > > > > > > But if I see your point, RCU might just want to register a quiescent > > > > state and for this long-running loop rcu_flavor_sched_clock_irq() does > > > > seem to fall down. > > > > > > > > > Or did I miss a change that causes preempt_enable() to help RCU out? > > > > > > > > Something like this? > > > > > > > > diff --git a/include/linux/preempt.h b/include/linux/preempt.h > > > > index dc5125b9c36b..e50f358f1548 100644 > > > > --- a/include/linux/preempt.h > > > > +++ b/include/linux/preempt.h > > > > @@ -222,6 +222,8 @@ do { \ > > > > barrier(); \ > > > > if (unlikely(preempt_count_dec_and_test())) \ > > > > __preempt_schedule(); \ > > > > + if (!(preempt_count() & (PREEMPT_MASK | SOFTIRQ_MASK))) \ > > > > + rcu_all_qs(); \ > > > > } while (0) > > > > > > Or maybe something like this to lighten the load a bit: > > > > > > #define preempt_enable() \ > > > do { \ > > > barrier(); \ > > > if (unlikely(preempt_count_dec_and_test())) { \ > > > __preempt_schedule(); \ > > > if (raw_cpu_read(rcu_data.rcu_urgent_qs) && \ > > > !(preempt_count() & (PREEMPT_MASK | SOFTIRQ_MASK))) \ > > > rcu_all_qs(); \ > > > } \ > > > } while (0) > > > > > > And at that point, we should be able to drop the PREEMPT_MASK, not > > > that it makes any difference that I am aware of: > > > > > > #define preempt_enable() \ > > > do { \ > > > barrier(); \ > > > if (unlikely(preempt_count_dec_and_test())) { \ > > > __preempt_schedule(); \ > > > if (raw_cpu_read(rcu_data.rcu_urgent_qs) && \ > > > !(preempt_count() & SOFTIRQ_MASK)) \ > > > rcu_all_qs(); \ > > > } \ > > > } while (0) > > > > > > Except that we can migrate as soon as that preempt_count_dec_and_test() > > > returns. And that rcu_all_qs() disables and re-enables preemption, > > > which will result in undesired recursion. Sigh. > > > > > > So maybe something like this: > > > > > > #define preempt_enable() \ > > > do { \ > > > if (raw_cpu_read(rcu_data.rcu_urgent_qs) && \ > > > !(preempt_count() & SOFTIRQ_MASK)) \ > > > > Sigh. This needs to include (PREEMPT_MASK | SOFTIRQ_MASK), > > but check for equality to something like (1UL << PREEMPT_SHIFT). > > > > For PREEMPT_RCU=n and CONFIG_PREEMPT_COUNT=y kernels > for report QS in preempt_enable(), we can refer to this: > > void rcu_read_unlock_strict(void) > { > struct rcu_data *rdp; > > if (irqs_disabled() || preempt_count() || !rcu_state.gp_kthread) > return; > rdp = this_cpu_ptr(&rcu_data); > rdp->cpu_no_qs.b.norm = false; > rcu_report_qs_rdp(rdp); > udelay(rcu_unlock_delay); > } > > The rcu critical section may be in the NMI handler needs to be considered. You are quite right, though one advantage of leveraging preempt_enable() is that it cannot really enable preemption in an NMI handler. But yes, that might need to be accounted for in the comparison with preempt_count(). The actual condition needs to also allow for the possibility that this preempt_enable() happened in a kernel built with preemptible RCU. And probably a few other things that I have not yet thought of. For one thing, rcu_implicit_dynticks_qs() might need adjustment. Though I am currently hoping that it will still be able to enlist the help of other things, for example, preempt_enable() and local_bh_enable(). Yes, it is the easiest thing in the world to just whip out the resched_cpu() hammer earlier in the grace period, and maybe that is the eventual solution. But I would like to try avoiding the extra IPIs if that can be done reasonably. ;-) Thanx, Paul > Thanks > Zqiang > > > > > > > Clearly time to sleep. :-/ > > > > Thanx, Paul > > > > > rcu_all_qs(); \ > > > barrier(); \ > > > if (unlikely(preempt_count_dec_and_test())) { \ > > > __preempt_schedule(); \ > > > } \ > > > } while (0) > > > > > > Then rcu_all_qs() becomes something like this: > > > > > > void rcu_all_qs(void) > > > { > > > unsigned long flags; > > > > > > /* Load rcu_urgent_qs before other flags. */ > > > if (!smp_load_acquire(this_cpu_ptr(&rcu_data.rcu_urgent_qs))) > > > return; > > > this_cpu_write(rcu_data.rcu_urgent_qs, false); > > > if (unlikely(raw_cpu_read(rcu_data.rcu_need_heavy_qs))) { > > > local_irq_save(flags); > > > rcu_momentary_dyntick_idle(); > > > local_irq_restore(flags); > > > } > > > rcu_qs(); > > > } > > > EXPORT_SYMBOL_GPL(rcu_all_qs); > > > > > > > Though I do wonder about the likelihood of hitting the case you describe > > > > and maybe instead of adding the check on every preempt_enable() > > > > it might be better to instead force a context switch in the > > > > rcu_flavor_sched_clock_irq() (as we do in the PREEMPT_RCU=y case.) > > > > > > Maybe. But rcu_all_qs() is way lighter weight than a context switch. > > > > > > Thanx, Paul