From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4BE4C07E9D for ; Tue, 27 Sep 2022 16:17:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 41A118E00D8; Tue, 27 Sep 2022 12:17:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3C8BF8E00C1; Tue, 27 Sep 2022 12:17:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 26A188E00D8; Tue, 27 Sep 2022 12:17:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 16A0A8E00C1 for ; Tue, 27 Sep 2022 12:17:19 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id DD2A9160FDE for ; Tue, 27 Sep 2022 16:17:18 +0000 (UTC) X-FDA: 79958370156.15.7C51937 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf15.hostedemail.com (Postfix) with ESMTP id 4CA02A000E for ; Tue, 27 Sep 2022 16:17:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1664295437; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=LBj3weUn5blVY9W7yPJRML2CDkn+Cxoar3VM+26p9ec=; b=NA6zoHJSPIv9c8vPWv0MB+UhWIo+MVyyRD1KLIw4IyYTA0FdjKhplaExbzO60BO8YzorZA 3mJUBHPzcxyAxbmioku5QEx6eB0Sixae3SecGe77lg7f912yW3xtZbJtVaHq2Z5ZEe2AWp d/60Ejbs1m1YXOrkoql0QKpxz5Mi61Y= Received: from mail-pj1-f72.google.com (mail-pj1-f72.google.com [209.85.216.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-359-6VwvTIznMMqPorWa8zJ24A-1; Tue, 27 Sep 2022 12:17:16 -0400 X-MC-Unique: 6VwvTIznMMqPorWa8zJ24A-1 Received: by mail-pj1-f72.google.com with SMTP id fa8-20020a17090af0c800b00202dd39c035so3891711pjb.7 for ; Tue, 27 Sep 2022 09:17:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date; bh=LBj3weUn5blVY9W7yPJRML2CDkn+Cxoar3VM+26p9ec=; b=OhnD2Rj5+vXYwynzgdBpjQwzkSCFMNu1KB/DnxZ/IVgmRNDdsae+jBlMVdvhZ7w3tL L2y3aeVyMVeFkXK1+tEw5mItjjfhWOxsBDc3IelXLIxFL+kRu7YfDhdQ5XAevC5Hq8Ig jsKbh0BE5GgVhB/0Gl166MF8SNpflpNYDXSapJunOJ4c5pLorYf7kw8PEwOT0/p2D3IK OJS+tLiQJAuoa5XYznifA17je4OdGsf95BQLyd1ZCNKUkEe0w1oi8GSVxg0YqhN3HnZC Yu8TM8hTcgkPFpkaCWu9pKocdAouI0v6ll20phPzT0jRck8eEQqMVR820L5bUgyL/+4S wsug== X-Gm-Message-State: ACrzQf2wvjEpKKKxu166Z4hyI0dlXgN3arCCr6emgK7E4luD+fazWmml 95l9UhKWJzyya7uE2aCb6M8Cdbp4lE8oeLTdP1KMHRMN9qRgjj8Jbn8t7qrtWyqN1POwWAzOvCq lxonoa/zBUD1L36GgSSXQs0A4gxU= X-Received: by 2002:a17:90b:1b07:b0:203:5860:b441 with SMTP id nu7-20020a17090b1b0700b002035860b441mr5345912pjb.103.1664295434015; Tue, 27 Sep 2022 09:17:14 -0700 (PDT) X-Google-Smtp-Source: AMsMyM43erDVvXpiXgbw0gH+6F1oO9GIbLJJtG0+ZhElQ2zamI/oKGDErF922Gq/I14277j6s2dDDV3LI3JWxFFofas= X-Received: by 2002:a17:90b:1b07:b0:203:5860:b441 with SMTP id nu7-20020a17090b1b0700b002035860b441mr5345875pjb.103.1664295433692; Tue, 27 Sep 2022 09:17:13 -0700 (PDT) MIME-Version: 1.0 References: <20220924152227.819815-1-atomlin@redhat.com> <20220924152227.819815-5-atomlin@redhat.com> In-Reply-To: <20220924152227.819815-5-atomlin@redhat.com> From: Rafael Folco Date: Tue, 27 Sep 2022 13:17:02 -0300 Message-ID: Subject: Re: [PATCH v8 4/5] tick/nohz_full: Ensure quiet_vmstat() is called on exit to user-mode when the idle tick is stopped To: Aaron Tomlin Cc: frederic@kernel.org, mtosatti@redhat.com, cl@linux.com, tglx@linutronix.de, mingo@kernel.org, peterz@infradead.org, pauld@redhat.com, neelx@redhat.com, oleksandr@natalenko.name, atomlin@atomlin.com, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: multipart/alternative; boundary="000000000000207e9b05e9aafa51" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1664295438; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LBj3weUn5blVY9W7yPJRML2CDkn+Cxoar3VM+26p9ec=; b=1W+61pscrohgXICg8qNG9/1KhQODzkjDT6LTrs+J9JqJG/C99ZM8HBEJ8zXjTnrh5k3Ban pCrT9Wjc/kGWYdGZuxkEl1NmZRBhLOlQyWNFYMfvjJGlnZDnssKUkfGL7PyPcPqU8I75Co C1fP8bgS+3hmACfSWEvllUM2MrxdXw4= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=NA6zoHJS; spf=pass (imf15.hostedemail.com: domain of rfolco@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=rfolco@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1664295438; a=rsa-sha256; cv=none; b=M/yK+sjbM10KZFU6hlEZ1fgxug5KKr7v1W9Wj6RgnmGmjCDRIc0MBQwJf0s77FxOJT10U2 f+48D5wE7mcOZQEcAHkpksf4HVT/YvJogk8hk3yq1qxbD3o4kOTwBwcv25AvXxyK/KPJAc GK05WV3wKozaf6TAjz0dt+TtLD/LZBk= X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 4CA02A000E Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=NA6zoHJS; spf=pass (imf15.hostedemail.com: domain of rfolco@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=rfolco@redhat.com; dmarc=pass (policy=none) header.from=redhat.com X-Stat-Signature: orodx55px9i7csebt5m5oujdbp8s1u7e X-Rspam-User: X-HE-Tag: 1664295438-218149 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: --000000000000207e9b05e9aafa51 Content-Type: text/plain; charset="UTF-8" Tested this patch w/ nohz_full setup and oslat 8h run on isolated cpus, max latency is 7us versus 15us without the patch. Thanks. On Mon, Sep 26, 2022 at 8:02 PM Aaron Tomlin wrote: > This patch ensures CPU-specific vmstat differentials do not remain > when the scheduling-tick is stopped and before exiting to user-mode > in the context of nohz_full only. > > A trivial test program was used to determine the impact of the proposed > changes and under vanilla. The mlock(2) and munlock(2) system calls was > used solely to modify vmstat item 'NR_MLOCK'. The following is an average > count of CPU-cycles across the aforementioned system calls: > > Vanilla Modified > > Cycles per syscall 8461 8690 (+2.6%) > > Signed-off-by: Aaron Tomlin > --- > include/linux/tick.h | 5 +++-- > kernel/time/tick-sched.c | 15 +++++++++++++++ > 2 files changed, 18 insertions(+), 2 deletions(-) > > diff --git a/include/linux/tick.h b/include/linux/tick.h > index bfd571f18cfd..a2bbd6d32e33 100644 > --- a/include/linux/tick.h > +++ b/include/linux/tick.h > @@ -11,7 +11,6 @@ > #include > #include > #include > -#include > > #ifdef CONFIG_GENERIC_CLOCKEVENTS > extern void __init tick_init(void); > @@ -272,6 +271,7 @@ static inline void tick_dep_clear_signal(struct > signal_struct *signal, > > extern void tick_nohz_full_kick_cpu(int cpu); > extern void __tick_nohz_task_switch(void); > +void __tick_nohz_user_enter_prepare(void); > extern void __init tick_nohz_full_setup(cpumask_var_t cpumask); > #else > static inline bool tick_nohz_full_enabled(void) { return false; } > @@ -296,6 +296,7 @@ static inline void tick_dep_clear_signal(struct > signal_struct *signal, > > static inline void tick_nohz_full_kick_cpu(int cpu) { } > static inline void __tick_nohz_task_switch(void) { } > +static inline void __tick_nohz_user_enter_prepare(void) { } > static inline void tick_nohz_full_setup(cpumask_var_t cpumask) { } > #endif > > @@ -308,7 +309,7 @@ static inline void tick_nohz_task_switch(void) > static inline void tick_nohz_user_enter_prepare(void) > { > if (tick_nohz_full_cpu(smp_processor_id())) > - rcu_nocb_flush_deferred_wakeup(); > + __tick_nohz_user_enter_prepare(); > } > > #endif > diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c > index b0e3c9205946..634cd0fac267 100644 > --- a/kernel/time/tick-sched.c > +++ b/kernel/time/tick-sched.c > @@ -26,6 +26,7 @@ > #include > #include > #include > +#include > > #include > > @@ -519,6 +520,20 @@ void __tick_nohz_task_switch(void) > } > } > > +void __tick_nohz_user_enter_prepare(void) > +{ > + struct tick_sched *ts; > + > + if (tick_nohz_full_cpu(smp_processor_id())) { > + ts = this_cpu_ptr(&tick_cpu_sched); > + > + if (ts->tick_stopped) > + quiet_vmstat(); > + rcu_nocb_flush_deferred_wakeup(); > + } > +} > +EXPORT_SYMBOL_GPL(__tick_nohz_user_enter_prepare); > + > /* Get the boot-time nohz CPU list from the kernel parameters. */ > void __init tick_nohz_full_setup(cpumask_var_t cpumask) > { > -- > 2.37.1 > > > -- Folco --000000000000207e9b05e9aafa51 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Tested this patch w/ nohz_full setup and oslat 8h run on i= solated cpus, max latency is 7us versus 15us without the patch.=C2=A0
T= hanks.

On Mon, Sep 26, 2022 at 8:02 PM Aaron Tomlin <atomlin@redhat.com> wrote:
This patch ensures CPU-specific v= mstat differentials do not remain
when the scheduling-tick is stopped and before exiting to user-mode
in the context of nohz_full only.

A trivial test program was used to determine the impact of the proposed
changes and under vanilla. The mlock(2) and munlock(2) system calls was
used solely to modify vmstat item 'NR_MLOCK'. The following is an a= verage
count of CPU-cycles across the aforementioned system calls:

=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Vanilla=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Modified

=C2=A0 Cycles per syscall=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 8= 461=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 86= 90=C2=A0 =C2=A0 (+2.6%)

Signed-off-by: Aaron Tomlin <atomlin@redhat.com>
---
=C2=A0include/linux/tick.h=C2=A0 =C2=A0 =C2=A0|=C2=A0 5 +++--
=C2=A0kernel/time/tick-sched.c | 15 +++++++++++++++
=C2=A02 files changed, 18 insertions(+), 2 deletions(-)

diff --git a/include/linux/tick.h b/include/linux/tick.h
index bfd571f18cfd..a2bbd6d32e33 100644
--- a/include/linux/tick.h
+++ b/include/linux/tick.h
@@ -11,7 +11,6 @@
=C2=A0#include <linux/context_tracking_state.h>
=C2=A0#include <linux/cpumask.h>
=C2=A0#include <linux/sched.h>
-#include <linux/rcupdate.h>

=C2=A0#ifdef CONFIG_GENERIC_CLOCKEVENTS
=C2=A0extern void __init tick_init(void);
@@ -272,6 +271,7 @@ static inline void tick_dep_clear_signal(struct signal_= struct *signal,

=C2=A0extern void tick_nohz_full_kick_cpu(int cpu);
=C2=A0extern void __tick_nohz_task_switch(void);
+void __tick_nohz_user_enter_prepare(void);
=C2=A0extern void __init tick_nohz_full_setup(cpumask_var_t cpumask);
=C2=A0#else
=C2=A0static inline bool tick_nohz_full_enabled(void) { return false; }
@@ -296,6 +296,7 @@ static inline void tick_dep_clear_signal(struct signal_= struct *signal,

=C2=A0static inline void tick_nohz_full_kick_cpu(int cpu) { }
=C2=A0static inline void __tick_nohz_task_switch(void) { }
+static inline void __tick_nohz_user_enter_prepare(void) { }
=C2=A0static inline void tick_nohz_full_setup(cpumask_var_t cpumask) { } =C2=A0#endif

@@ -308,7 +309,7 @@ static inline void tick_nohz_task_switch(void)
=C2=A0static inline void tick_nohz_user_enter_prepare(void)
=C2=A0{
=C2=A0 =C2=A0 =C2=A0 =C2=A0 if (tick_nohz_full_cpu(smp_processor_id()))
-=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rcu_nocb_flush_defe= rred_wakeup();
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0__tick_nohz_user_en= ter_prepare();
=C2=A0}

=C2=A0#endif
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index b0e3c9205946..634cd0fac267 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -26,6 +26,7 @@
=C2=A0#include <linux/posix-timers.h>
=C2=A0#include <linux/context_tracking.h>
=C2=A0#include <linux/mm.h>
+#include <linux/rcupdate.h>

=C2=A0#include <asm/irq_regs.h>

@@ -519,6 +520,20 @@ void __tick_nohz_task_switch(void)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 }
=C2=A0}

+void __tick_nohz_user_enter_prepare(void)
+{
+=C2=A0 =C2=A0 =C2=A0 =C2=A0struct tick_sched *ts;
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0if (tick_nohz_full_cpu(smp_processor_id())) { +=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ts =3D this_cpu_ptr= (&tick_cpu_sched);
+
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0if (ts->tick_sto= pped)
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0quiet_vmstat();
+=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0rcu_nocb_flush_defe= rred_wakeup();
+=C2=A0 =C2=A0 =C2=A0 =C2=A0}
+}
+EXPORT_SYMBOL_GPL(__tick_nohz_user_enter_prepare);
+
=C2=A0/* Get the boot-time nohz CPU list from the kernel parameters. */
=C2=A0void __init tick_nohz_full_setup(cpumask_var_t cpumask)
=C2=A0{
--
2.37.1




--
Folco
--000000000000207e9b05e9aafa51--