From: Geert Uytterhoeven <geert@linux-m68k.org>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Frederic Weisbecker <frederic@kernel.org>,
Guo Ren <guoren@kernel.org>,
"Rafael J. Wysocki" <rafael.j.wysocki@intel.com>,
Kajetan Puchalski <kajetan.puchalski@arm.com>,
Tony Lindgren <tony@atomide.com>,
Ulf Hansson <ulf.hansson@linaro.org>,
Ingo Molnar <mingo@kernel.org>,
linux@armlinux.org.uk, linux-imx@nxp.com,
linux-kernel@vger.kernel.org, linux-omap@vger.kernel.org,
linux-samsung-soc@vger.kernel.org,
linux-perf-users@vger.kernel.org, linux-pm@vger.kernel.org,
linux-clk@vger.kernel.org, linux-arm-msm@vger.kernel.org,
linux-tegra@vger.kernel.org, linux-arch@vger.kernel.org,
linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org,
kasan-dev@googlegroups.com, linux-renesas-soc@vger.kernel.org
Subject: Re: [PATCH v3 07/51] cpuidle,psci: Push RCU-idle into driver
Date: Tue, 7 Mar 2023 17:40:08 +0100 (CET) [thread overview]
Message-ID: <ff338b9f-4ab0-741b-26ea-7b7351da156@linux-m68k.org> (raw)
In-Reply-To: <20230112195539.760296658@infradead.org>
Hoi Peter,
(reduced the insane CC list)
On Thu, 12 Jan 2023, Peter Zijlstra wrote:
> Doing RCU-idle outside the driver, only to then temporarily enable it
> again, at least twice, before going idle is daft.
>
> Notably once implicitly through the cpu_pm_*() calls and once
> explicitly doing ct_irq_*_irqon().
>
> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
> Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
> Reviewed-by: Guo Ren <guoren@kernel.org>
> Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
> Tested-by: Kajetan Puchalski <kajetan.puchalski@arm.com>
> Tested-by: Tony Lindgren <tony@atomide.com>
> Tested-by: Ulf Hansson <ulf.hansson@linaro.org>
Thanks for your patch, which is now commit e038f7b8028a1d1b ("cpuidle,
psci: Push RCU-idle into driver") in v6.3-rc1.
I have bisected a PSCI checker regression on Renesas R-Car Gen3/4 SoCs
to commit a01353cf1896ea5b ("cpuidle: Fix ct_idle_*() usage") (the 7
commits before that do not compile):
psci_checker: PSCI checker started using 2 CPUs
psci_checker: Starting hotplug tests
psci_checker: Trying to turn off and on again all CPUs
psci: CPU0 killed (polled 0 ms)
Detected PIPT I-cache on CPU0
CPU0: Booted secondary processor 0x0000000000 [0x411fd073]
psci_checker: Trying to turn off and on again group 0 (CPUs 0-1)
psci: CPU0 killed (polled 0 ms)
Detected PIPT I-cache on CPU0
CPU0: Booted secondary processor 0x0000000000 [0x411fd073]
psci_checker: Hotplug tests passed OK
psci_checker: Starting suspend tests (10 cycles per state)
psci_checker: CPU 0 entering suspend cycles, states 1 through 1
psci_checker: CPU 1 entering suspend cycles, states 1 through 1
------------[ cut here ]------------
WARNING: CPU: 1 PID: 177 at kernel/context_tracking.c:141 ct_kernel_exit.constprop.0+0xd8/0xf4
Modules linked in:
CPU: 1 PID: 177 Comm: psci_suspend_te Not tainted 6.2.0-rc1-salvator-x-00052-ga01353cf1896 #1415
Hardware name: Renesas Salvator-X 2nd version board based on r8a77965 (DT)
pstate: 604000c5 (nZCv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : ct_kernel_exit.constprop.0+0xd8/0xf4
lr : ct_kernel_exit.constprop.0+0xc8/0xf4
sp : ffffffc00b73bd30
x29: ffffffc00b73bd30 x28: ffffff807fbadc90 x27: 0000000000000000
x26: 0000000000000000 x25: 0000000000000000 x24: 0000000000000000
x23: ffffff800981e140 x22: 0000000000000001 x21: 0000000000010000
x20: ffffffc0086be1d8 x19: ffffff807fbac070 x18: 0000000000000000
x17: ffffff80083d1000 x16: ffffffc00841fff8 x15: ffffffc00b73b990
x14: ffffffc00895be78 x13: 0000000000000001 x12: 0000000000000000
x11: 00000000000001aa x10: 00000000ffffffea x9 : 000000000000000f
x8 : ffffffc00b73bb68 x7 : ffffffc00b73be18 x6 : ffffffc00815ff34
x5 : ffffffc00a6a0c30 x4 : ffffffc00801ce00 x3 : 0000000000000000
x2 : ffffffc008dc3070 x1 : ffffffc008dc3078 x0 : 0000000004208040
Call trace:
ct_kernel_exit.constprop.0+0xd8/0xf4
ct_idle_enter+0x18/0x20
psci_enter_idle_state+0xa4/0xfc
suspend_test_thread+0x238/0x2f0
kthread+0xd8/0xe8
ret_from_fork+0x10/0x20
irq event stamp: 0
hardirqs last enabled at (0): [<0000000000000000>] 0x0
hardirqs last disabled at (0): [<ffffffc0080798b0>] copy_process+0x608/0x13dc
softirqs last enabled at (0): [<ffffffc0080798b0>] copy_process+0x608/0x13dc
softirqs last disabled at (0): [<0000000000000000>] 0x0
---[ end trace 0000000000000000 ]---
------------[ cut here ]------------
WARNING: CPU: 1 PID: 177 at kernel/context_tracking.c:186 ct_kernel_enter.constprop.0+0x78/0xa4
Modules linked in:
CPU: 1 PID: 177 Comm: psci_suspend_te Tainted: G W 6.2.0-rc1-salvator-x-00052-ga01353cf1896 #1415
Hardware name: Renesas Salvator-X 2nd version board based on r8a77965 (DT)
pstate: 604000c5 (nZCv daIF +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
pc : ct_kernel_enter.constprop.0+0x78/0xa4
lr : ct_kernel_enter.constprop.0+0x68/0xa4
sp : ffffffc00b73bd30
x29: ffffffc00b73bd30 x28: ffffff807fbadc90 x27: 0000000000000000
x26: 0000000000000000 x25: 0000000000000000 x24: 0000000000000000
x23: ffffff800981e140 x22: 0000000000000001 x21: 00000000ffffffa1
x20: ffffffc0086be1d8 x19: 00000000000000c0 x18: 0000000000000000
x17: ffffff80083d1000 x16: ffffffc00841fff8 x15: ffffffc00b73b990
x14: ffffffc00895be78 x13: ffffff800e325180 x12: ffffffc076de9000
x11: 0000000034d4d91d x10: 0000000000000008 x9 : 0000000000001000
x8 : ffffffc008012800 x7 : 0000000000000000 x6 : ffffff807fbac070
x5 : ffffffc008dc3070 x4 : 0000000000000000 x3 : 000000000001a9fc
x2 : 0000000000000003 x1 : ffffffc008dc3070 x0 : 0000000004208040
Call trace:
ct_kernel_enter.constprop.0+0x78/0xa4
ct_idle_exit+0x18/0x38
psci_enter_idle_state+0xdc/0xfc
suspend_test_thread+0x238/0x2f0
kthread+0xd8/0xe8
ret_from_fork+0x10/0x20
irq event stamp: 0
hardirqs last enabled at (0): [<0000000000000000>] 0x0
hardirqs last disabled at (0): [<ffffffc0080798b0>] copy_process+0x608/0x13dc
softirqs last enabled at (0): [<ffffffc0080798b0>] copy_process+0x608/0x13dc
softirqs last disabled at (0): [<0000000000000000>] 0x0
---[ end trace 0000000000000000 ]---
psci_checker: Failed to suspend CPU 1: error -1 (requested state 1, cycle 0)
psci_checker: CPU 0 suspend test results: success 0, shallow states 10, errors 0
mmcblk0rpmb: mmc0:0001 BGSD3R 4.00 MiB, chardev (243:0)
psci_checker: CPU 1 suspend test results: success 0, shallow states 9, errors 1
psci_checker: 1 error(s) encountered in suspend tests
psci_checker: PSCI checker completed
> ---
> drivers/cpuidle/cpuidle-psci.c | 9 +++++----
> 1 file changed, 5 insertions(+), 4 deletions(-)
>
> --- a/drivers/cpuidle/cpuidle-psci.c
> +++ b/drivers/cpuidle/cpuidle-psci.c
> @@ -69,12 +69,12 @@ static int __psci_enter_domain_idle_stat
> return -1;
>
> /* Do runtime PM to manage a hierarchical CPU toplogy. */
> - ct_irq_enter_irqson();
> if (s2idle)
> dev_pm_genpd_suspend(pd_dev);
> else
> pm_runtime_put_sync_suspend(pd_dev);
> - ct_irq_exit_irqson();
> +
> + ct_idle_enter();
>
> state = psci_get_domain_state();
> if (!state)
> @@ -82,12 +82,12 @@ static int __psci_enter_domain_idle_stat
>
> ret = psci_cpu_suspend_enter(state) ? -1 : idx;
>
> - ct_irq_enter_irqson();
> + ct_idle_exit();
> +
> if (s2idle)
> dev_pm_genpd_resume(pd_dev);
> else
> pm_runtime_get_sync(pd_dev);
> - ct_irq_exit_irqson();
>
> cpu_pm_exit();
>
> @@ -240,6 +240,7 @@ static int psci_dt_cpu_init_topology(str
> * of a shared state for the domain, assumes the domain states are all
> * deeper states.
> */
> + drv->states[state_count - 1].flags |= CPUIDLE_FLAG_RCU_IDLE;
> drv->states[state_count - 1].enter = psci_enter_domain_idle_state;
> drv->states[state_count - 1].enter_s2idle = psci_enter_s2idle_domain_idle_state;
> psci_cpuidle_use_cpuhp = true;
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
next prev parent reply other threads:[~2023-03-07 16:40 UTC|newest]
Thread overview: 64+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-01-12 19:43 [PATCH v3 00/51] cpuidle,rcu: Clean up the mess Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 01/51] x86/perf/amd: Remove tracing from perf_lopwr_cb() Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 02/51] x86/idle: Replace x86_idle with a static_call Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 03/51] cpuidle/poll: Ensure IRQ state is invariant Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 04/51] cpuidle: Move IRQ state validation Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 05/51] cpuidle,riscv: Push RCU-idle into driver Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 06/51] cpuidle,tegra: " Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 07/51] cpuidle,psci: " Peter Zijlstra
2023-03-07 16:40 ` Geert Uytterhoeven [this message]
2023-03-20 14:56 ` Mark Rutland
2023-01-12 19:43 ` [PATCH v3 08/51] cpuidle,imx6: " Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 09/51] cpuidle,omap3: " Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 10/51] cpuidle,armada: " Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 11/51] cpuidle,omap4: " Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 12/51] cpuidle,dt: " Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 13/51] cpuidle: Fix ct_idle_*() usage Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 14/51] cpuidle,cpu_pm: Remove RCU fiddling from cpu_pm_{enter,exit}() Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 15/51] acpi_idle: Remove tracing Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 16/51] cpuidle: Annotate poll_idle() Peter Zijlstra
2023-01-20 9:56 ` Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 17/51] objtool/idle: Validate __cpuidle code as noinstr Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 18/51] cpuidle,intel_idle: Fix CPUIDLE_FLAG_IRQ_ENABLE *again* Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 19/51] cpuidle,intel_idle: Fix CPUIDLE_FLAG_INIT_XSTATE Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 20/51] cpuidle,intel_idle: Fix CPUIDLE_FLAG_IBRS Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 21/51] arch/idle: Change arch_cpu_idle() IRQ behaviour Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 22/51] x86/tdx: Remove TDX_HCALL_ISSUE_STI Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 23/51] arm,smp: Remove trace_.*_rcuidle() usage Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 24/51] arm64,smp: " Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 25/51] printk: " Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 26/51] time/tick-broadcast: Remove RCU_NONIDLE usage Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 27/51] cpuidle,sched: Remove annotations from TIF_{POLLING_NRFLAG,NEED_RESCHED} Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 28/51] cpuidle,mwait: Make noinstr clean Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 29/51] cpuidle,tdx: Make tdx " Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 30/51] cpuidle,xenpv: Make more PARAVIRT_XXL " Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 31/51] cpuidle,nospec: Make " Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 32/51] cpuidle,acpi: " Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 33/51] trace: Remove trace_hardirqs_{on,off}_caller() Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 34/51] trace: WARN on rcuidle Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 35/51] trace,hardirq: No moar _rcuidle() tracing Peter Zijlstra
2023-01-17 4:24 ` Masami Hiramatsu
2023-01-17 8:53 ` Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 36/51] cpuidle,omap3: Use WFI for omap3_pm_idle() Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 37/51] cpuidle,omap3: Push RCU-idle into omap_sram_idle() Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 38/51] cpuidle,omap4: Push RCU-idle into omap4_enter_lowpower() Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 39/51] arm,omap2: Use WFI for omap2_pm_idle() Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 40/51] cpuidle,powerdomain: Remove trace_.*_rcuidle() Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 41/51] cpuidle,clk: " Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 42/51] ubsan: Fix objtool UACCESS warns Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 43/51] intel_idle: Add force_irq_on module param Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 44/51] entry,kasan,x86: Disallow overriding mem*() functions Peter Zijlstra
2023-01-12 19:43 ` [PATCH v3 45/51] sched: Always inline __this_cpu_preempt_check() Peter Zijlstra
2023-01-12 19:44 ` [PATCH v3 46/51] arm64,riscv,perf: Remove RCU_NONIDLE() usage Peter Zijlstra
2023-01-12 19:44 ` [PATCH v3 47/51] cpuidle: Ensure ct_cpuidle_enter() is always called from noinstr/__cpuidle Peter Zijlstra
2023-01-12 19:44 ` [PATCH v3 48/51] cpuidle,arch: Mark all ct_cpuidle_enter() callers __cpuidle Peter Zijlstra
2023-01-12 19:44 ` [PATCH v3 49/51] cpuidle,arch: Mark all regular cpuidle_state::enter methods __cpuidle Peter Zijlstra
2023-01-12 19:44 ` [PATCH v3 50/51] cpuidle: Comments about noinstr/__cpuidle Peter Zijlstra
2023-01-12 19:44 ` [PATCH v3 51/51] context_tracking: Fix noinstr vs KASAN Peter Zijlstra
2023-01-13 18:06 ` [PATCH v3 00/51] cpuidle,rcu: Clean up the mess Paul E. McKenney
2023-01-16 16:59 ` Mark Rutland
2023-01-17 10:26 ` Peter Zijlstra
2023-01-17 12:39 ` Sudeep Holla
2023-01-17 13:16 ` Mark Rutland
2023-01-17 14:21 ` Sudeep Holla
2023-01-17 15:35 ` Mark Rutland
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ff338b9f-4ab0-741b-26ea-7b7351da156@linux-m68k.org \
--to=geert@linux-m68k.org \
--cc=frederic@kernel.org \
--cc=guoren@kernel.org \
--cc=kajetan.puchalski@arm.com \
--cc=kasan-dev@googlegroups.com \
--cc=linux-arch@vger.kernel.org \
--cc=linux-arm-msm@vger.kernel.org \
--cc=linux-clk@vger.kernel.org \
--cc=linux-imx@nxp.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-omap@vger.kernel.org \
--cc=linux-perf-users@vger.kernel.org \
--cc=linux-pm@vger.kernel.org \
--cc=linux-renesas-soc@vger.kernel.org \
--cc=linux-samsung-soc@vger.kernel.org \
--cc=linux-tegra@vger.kernel.org \
--cc=linux-trace-kernel@vger.kernel.org \
--cc=linux@armlinux.org.uk \
--cc=mingo@kernel.org \
--cc=peterz@infradead.org \
--cc=rafael.j.wysocki@intel.com \
--cc=tony@atomide.com \
--cc=ulf.hansson@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox