From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E454EC83F03 for ; Wed, 2 Jul 2025 22:15:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 628706B00B3; Wed, 2 Jul 2025 18:15:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5D8026B00B4; Wed, 2 Jul 2025 18:15:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4A1998D0001; Wed, 2 Jul 2025 18:15:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 36D3D6B00B3 for ; Wed, 2 Jul 2025 18:15:12 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 903A78041B for ; Wed, 2 Jul 2025 22:15:11 +0000 (UTC) X-FDA: 83620731222.15.A6CD84A Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by imf07.hostedemail.com (Postfix) with ESMTP id B0A4C40004 for ; Wed, 2 Jul 2025 22:15:09 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=linutronix.de header.s=2020 header.b=KX7tUCqj; dkim=pass header.d=linutronix.de header.s=2020e header.b=a7Ho1+rH; dmarc=pass (policy=none) header.from=linutronix.de; spf=pass (imf07.hostedemail.com: domain of tglx@linutronix.de designates 193.142.43.55 as permitted sender) smtp.mailfrom=tglx@linutronix.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1751494510; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:dkim-signature; bh=RZb4TuPSsxuStgzyCzXshm9KXsVdzW7EFuWMfv5so1U=; b=FNCAmMQR5Bhdi6MkGTT8E3zTu8dsJuTznH3Z7oryjtgCbMP641TOmwSbSzYeNYw/UirWvL FMHdB3aR3mwppSVn3hE3N/Sov0MD9IhQC02vZ/QD1phncN77Qh/8mMlcmaAHfB2lecda8c pKSswSTNcJ8c8BDqte3fmujnetZuas4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1751494510; a=rsa-sha256; cv=none; b=ebMUhz5IjYWDRsdyW8ncvBgq5Z7psxgdkYT7xPc9yAHWVTbgTcmJVHVNe1Xq+WSClw4M7K gkNeoMEm8mQX88zDjFnVVUWZs0aq5CKEPHM4ouH3KwgHBM64xxYiQ1Ah14CFmuq1jiUqEa vpgJHpV85M498tQrVRJM3hJWBPFpwDc= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=linutronix.de header.s=2020 header.b=KX7tUCqj; dkim=pass header.d=linutronix.de header.s=2020e header.b=a7Ho1+rH; dmarc=pass (policy=none) header.from=linutronix.de; spf=pass (imf07.hostedemail.com: domain of tglx@linutronix.de designates 193.142.43.55 as permitted sender) smtp.mailfrom=tglx@linutronix.de From: Thomas Gleixner DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1751494504; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=RZb4TuPSsxuStgzyCzXshm9KXsVdzW7EFuWMfv5so1U=; b=KX7tUCqj21OraL63r5pvq5dJc/03gkwozim1iAF8O3ywsrEiPyNGUnCLrtjIqFcMk0iyE1 c71xp4zV/YwgZf6EYjJp1RUc+7qYqCwUf4YY/8KUvwuoHG2jhRQD0GelH903jwHsZ2BqCK CNAUdi3Z5u/HlmYQlak8G44RsnG4Lnmsfw0OhK5LkoI4vZpjEen37DpZPtWvEHXUudeeJA leezf2GoSccNgIqKKZ51k/jRJ8HIp/28xQIXg7OvZtqt3l7lIlalH5JFuPgogAZn19Yt4R FvoMn+PHtOPLcg+/k1JXneDzLK1WZybM0CiDB8fMRdVCMgTLzADH4CetAZ7sMg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1751494504; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to; bh=RZb4TuPSsxuStgzyCzXshm9KXsVdzW7EFuWMfv5so1U=; b=a7Ho1+rH9oOKcbkjPsQkPpdAUiuFOy9DBYfdBonsQ99y1sNYKv9l3L7yQns1dIrXaiWJGr CW2jEtebTMJpqGCA== To: Christoph Lameter via B4 Relay , Anna-Maria Behnsen , Frederic Weisbecker , Ingo Molnar Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, sh@gentwo.org, Darren Hart , "Christoph Lameter (Ampere)" Subject: Re: [PATCH] Skew tick for systems with a large number of processors In-Reply-To: <20250702-tick_skew-v1-1-ff8d73035b02@gentwo.org> Date: Thu, 03 Jul 2025 00:15:04 +0200 Message-ID: <87sejew87r.ffs@tglx> MIME-Version: 1.0 Content-Type: text/plain X-Stat-Signature: 785h6opigcqryh1u3j9yxrfijasydy9s X-Rspamd-Queue-Id: B0A4C40004 X-Rspam-User: X-Rspamd-Server: rspam07 X-HE-Tag: 1751494509-666388 X-HE-Meta: U2FsdGVkX19uC5uWnmOgaONu5LXC3LW5SAawZb+aGQrKWk0VayBQmQRVQ06aSOG6dbYKEQd+VGCruuPOLj7Urc1L57CnhT2CTuATB3TTUqKNS8Syha1tg0nqTEMFRK9J2TAOCdK4dPrzMvQNtjHzjNzS3ADKDBdAJk6Ur/ocn8uiV/utbUvtAgX9u35lKWcqblncPaqXl+0HS/dQcJylrCDMDNpkUIuis8FgyVsq7SxJCa+O9MV0OdUOX15TyRL/wsYimxMWxFUVgO7fOaYmrYkeZhtijJ87ccZLRKhjfU9vYE/42kWz82xb90ixsyygR1/DbB6swVns5nDyImTcXSicrzHQSTNKl0ZieOcmdU+inRceoZOwdCD5BkbXtYr5szZw3q5qwmvi2JDUIZXNXFrI+77dM+qIvfRbag6861Bx/J+D6rVHoIg7QrZIcw0nkZjeCGRZc3G+MHjsQLxlIq7TO7bsgwNJo76TFHUCGL2uRf1DkaKspRxefe7mxxqd67MHX0FVCr9N4/vctoiTwdoWoATIgwU2NIGH8RpWhAvmam4TtXz6Nf8YtIFf+GCdrrL/iIXHFCFiA7alPA01Wxjs53O+kgwFxd0M7TU4b+9SnigzOeOQCXUleXqWPD1I+w0nR2KAuEYRilArLRV3UMrI8Z6z5b8TXolwppU/Y159btTeImneLpWjdEEPzlzxv6eaFqXqr8q4f+aDFqaOrPvWsnBrGxQtpQTBpv6Qr+byc413w1zdOQOYoDv3FdvIE6kvLmHTHQyX3HHulCrbpqHsU6YmxSqFFB6d+uNWqmiHmAQsGdP2iLhWbItEPSp/Vb9Zdw8KvEKRJVv0K7ntWDD1KqJwG4VId7Q04TPKiY0PXjJ9vXEE40Pb9IsszMDCMUwff8JnHaowJyGA1hcuhaE4GYnR1TZmEKor+KdjIq3ezewulMPz1c/pB8VHBJ761QbeydcVC32ct59mMJG 5CPH5MPo 1T53MQfvO0kt2zL5rzoWaqksIINVnAXhA6D28FZCPSIQPArc6AAz+wyrzcduxtv30eHmcD2zY86GJsRgrZTbo/s8kWwDX1qjzCJODE7wOqPZDhyz/5Yb9GPsjgelNbZnMLt3SqDgnVdRfLfk87TMGUQ+5Pe5e9W3IHckKmqZk+6Dh72u6X2BzoorzaRmBTTn+AShyD8Lnx5LFfHFlGg6o8+hxqpVi3QbEvm+i5V6LUtyL7FvqBm6Em6/+nGIh+DQLVuhGs6qmzakVe3gdPPZrpDhWp1skT63LW0o4knsYXu4Ev6h2NVU3H66K7MqLtQm74CMLOxsVZ+Dwu7VENLoSRtwSIVXNtfFop5CxJ928i9AEJ6ccF2zafj83gvddKFOWKcjL X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Christoph! On Wed, Jul 02 2025 at 12:42, Christoph Lameter via wrote: Subject starts with a subsystem followed by a colon and then the short log. That has been that way forever and is clearly documented. You're not really new to kernel development and I pointed that out to you before: https://lore.kernel.org/all/87o74m1oq7.ffs@tglx No? > From: "Christoph Lameter (Ampere)" > > Synchronized ticks mean that all processors will simultaneously process > ticks and enter the scheduler. So the contention increases as the number > of cpu increases. The contention causes latency jitter that scales with > the number of processors. > > Staggering the timer interrupt also helps mitigate voltage droop related > issues that may be observed in SOCs with large core counts. > See https://semiengineering.com/mitigating-voltage-droop/ for a more > detailed explanation. > > Switch to skewed tick for systems with more than 64 processors. This lacks a proper explanation why that does not have any negative side effects on existing deployments and application scenarios. > --- a/kernel/Kconfig.hz > +++ b/kernel/Kconfig.hz The tick related Kconfig options are in kernel/time/Kconfig > + > +config TICK_SKEW_LIMIT > + int > + default 64 That wants a range 0 NR_CPUS or such > + help > + If the kernel is booted on systems with a large number of cpus then the > + concurrent execution of timer ticks causes long holdoffs due to > + serialization. Synchrononous executions of interrupts can also cause > + voltage droop in SOCs. So switch to skewed mode. This mechanism What does 'So switch to skewed mode.' help the user to select any useful value? This wants to have a proper explanation for picking a value which is understandable by mere mortals and not some useless "expert" word salad. > + can be overridden by specifying "tick_skew=x" on the kernel command line. Neither does it explain how that override affects the chosen value nor update the documentation of the command line value to make users aware of this behavioural change. For the casual reader this suggests, that tick_skew=x allows to change that number on the kernel command line, which it does not. > -static int sched_skew_tick; > +static int sched_skew_tick = -1; What's this magic -1 here? Can we please have some obvious and understandable define for this? > static int __init skew_tick(char *str) > { > @@ -1572,6 +1572,16 @@ void tick_setup_sched_timer(bool hrtimer) > { > struct tick_sched *ts = this_cpu_ptr(&tick_cpu_sched); > > + /* Figure out if we should skew the tick */ > + if (sched_skew_tick < 0) { This is incompatible with the existing code, which is unfortunately stupid already. Today 'tick_skew=-1' causes the tick to be skewed. Now it gets a different meaning. Not that it matters much, but change logs are supposed to mention user visible behavioural differences and argue why they don't matter, no? > + if (num_possible_cpus() >= CONFIG_TICK_SKEW_LIMIT) { > + sched_skew_tick = 1; > + pr_info("Tick skewed mode enabled. Possible cpus %u > %u\n", > + num_possible_cpus(), CONFIG_TICK_SKEW_LIMIT); I'm not convinced that this is useful, but that's the least of the issues. > + } else The else clause wants curly brackets for symmetry. > + sched_skew_tick = 0; The above aside. As you completely failed to provide at least the minimal historical background in the change log, let me fill in the blanks. commit 3704540b4829 ("tick management: spread timer interrupt") added the skew unconditionally in 2007 to avoid lock contention on xtime lock. commit af5ab277ded0 ("clockevents: Remove the per cpu tick skew") removed it in 2010 because the xtime lock contention was gone and the skew affected the power consumption of slightly loaded _large_ servers. commit 5307c9556bc1 ("tick: Add tick skew boot option") brought it back with a command line option to address contention and jitter issues on larger systems. So while you preserved the behaviour of the command line option in the most obscure way, you did not even make an attempt to explain why this change does not bring back the issues which caused the removal in commit af5ab277ded0 or why they are irrelevant today. "Scratches my itch" does not work and you know that. This needs to be consolidated both on the implementation side and also on the user side. Thanks for making me do your homework, tglx