From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4AE75C02198 for ; Mon, 10 Feb 2025 14:10:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0177A6B0082; Mon, 10 Feb 2025 09:10:33 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F08F86B0083; Mon, 10 Feb 2025 09:10:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA95D6B0085; Mon, 10 Feb 2025 09:10:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id BE3876B0082 for ; Mon, 10 Feb 2025 09:10:32 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id E71D61A12F6 for ; Mon, 10 Feb 2025 14:07:43 +0000 (UTC) X-FDA: 83104213248.26.47491EC Received: from mail-wm1-f45.google.com (mail-wm1-f45.google.com [209.85.128.45]) by imf28.hostedemail.com (Postfix) with ESMTP id E4702C000D for ; Mon, 10 Feb 2025 14:07:41 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=joelfernandes.org header.s=google header.b=oR5vaz9B; spf=pass (imf28.hostedemail.com: domain of joel@joelfernandes.org designates 209.85.128.45 as permitted sender) smtp.mailfrom=joel@joelfernandes.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739196462; a=rsa-sha256; cv=none; b=Xq4WG4JRdU9LCK+GmgngDpfcOIPTCixVSX2Irfl0nq1RhQJ+IS4+aSKBhqlarOFWoVBQIt fpr78Vf9owCL5G120cGzs7KGSO8lA+Ig47ED9lYGRq2s9hcqQh9vbFB4iGApcws1zHRfER SPsy1K+Ju4w28LQZu2M9bVe+hdhOSZE= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=joelfernandes.org header.s=google header.b=oR5vaz9B; spf=pass (imf28.hostedemail.com: domain of joel@joelfernandes.org designates 209.85.128.45 as permitted sender) smtp.mailfrom=joel@joelfernandes.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739196462; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=p0mkagQrLShZVdk5H2255n0SRNzKdf8QaJ4Fo2H5hYY=; b=My22JuXoaI2pA+HxWZwyMnWGP0dcDs4FRRFrAMqURKzNldb8UKaSFkJQUR2e2QbLest2vN LbdQLqieU2nsEZ2YkR4CkeUGy+ZNXMIPRQbtS41GVniCCa6+R0oZAPPMcfJgIrq3S6MHz0 td2lXoAkHe08EN04k91ioQjTvb/2nP0= Received: by mail-wm1-f45.google.com with SMTP id 5b1f17b1804b1-439307d83f0so12169705e9.3 for ; Mon, 10 Feb 2025 06:07:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=joelfernandes.org; s=google; t=1739196459; x=1739801259; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=p0mkagQrLShZVdk5H2255n0SRNzKdf8QaJ4Fo2H5hYY=; b=oR5vaz9BR7YOAzWHy303QM9AfEx4dnPfQf+XAtRem8td/KNMwHbfBoN2UM9FG2qFd8 dOQzYk95/tVFis5bsIzKShgdpbzDaFSarcKH19/Q465ijDTYMRxRIJdbWHHDQzo4yena j0685XFxhMK32dz9U1QNjheC3vOISqty/ZrBk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739196459; x=1739801259; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=p0mkagQrLShZVdk5H2255n0SRNzKdf8QaJ4Fo2H5hYY=; b=ChD11RqSFprJICUrgnPSs2XwJFfqP1jYnCaEhLe3GeHyXVmAwxxI2Ns3/im9nB0b5U Erkee888XAEBQJ9bJFKcmQw2erDFSC+pcjfVsSzSk+m3cj4xdTtFLsBp03o5OQ9780BW huZuPH1K/ax3Mk68s/s2903spMCTy/3wEx684ZoTKwRa+tx+PKcAnJH21oyp13yA56ZT b3srvp9GYSg3YV2big1WSH21DkOkqm2RjehGZ8V7Tv+pdcrOljg0cdfG14FxxtDdEC+B 6I4NqWC5nBRtc1Ujj5D3wWgqCF/uo1eO7/ELrF3vtIUxKCJUiVTcIp60qvf4QIwvUP1p WpXg== X-Forwarded-Encrypted: i=1; AJvYcCUci6hw70fVYOJo5ooC2ZYWIZMWoqm10now4/jq9WD/bL+JjnwTxGt29pXyTfD2U9qBg1JgD0dJdA==@kvack.org X-Gm-Message-State: AOJu0YxiGzHilrOfZeAGNbUQYPhm5+FvetIWlyJUFKPaY+/kJuNGI0GU xDzg3HZFCEl3oKOUy6vq5yRBIX6vbcPGhwRmRea3BRrQKoNyFlVGabs62Fgv1K+u5XsAfLMEQky E+ze6cf17HFOfzbPOny7mSVC3L5rixva5vIKoeA== X-Gm-Gg: ASbGncsKqFXVCkBsY3OxHXnq2v34ocLI5iOBQbEqUl3EBank9wpWc2Dak7ahyfapuIz YnuVUOJvZpMZRlu5W/bV08JQD0IQz0El12Q26JUe8ArgOuJv6k6cLmFEP1RcI6NHydD/U7tUU X-Google-Smtp-Source: AGHT+IGxIlrUl6KkSiVWsX9ssbEDQy6PoNPBsPKG4NuEx7YWW6vGApblEwLGUcefq/4kHgdnJjr16lfmTeY8Hcu8wsA= X-Received: by 2002:a05:600c:1909:b0:434:fafe:edb with SMTP id 5b1f17b1804b1-439249b04d2mr105436075e9.24.1739196458293; Mon, 10 Feb 2025 06:07:38 -0800 (PST) MIME-Version: 1.0 References: <9DA1FAE6-A008-4785-BDF9-541457E29807@joelfernandes.org> <20250204220418.35949317@gandalf.local.home> <20250205081635.397eacb0@gandalf.local.home> <20250206083039.0916ad24@gandalf.local.home> In-Reply-To: <20250206083039.0916ad24@gandalf.local.home> From: Joel Fernandes Date: Mon, 10 Feb 2025 09:07:27 -0500 X-Gm-Features: AWEUYZnPqUrM38HMt3sXrxskO2VmAyr4EFBG6mWi16NIsKaWAigNoh4M4L1D3xA Message-ID: Subject: Re: [RFC][PATCH 1/2] sched: Extended scheduler time slice To: Steven Rostedt Cc: Prakash Sangappa , Peter Zijlstra , linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, Thomas Gleixner , Ankur Arora , Linus Torvalds , linux-mm@kvack.org, x86@kernel.org, Andrew Morton , luto@kernel.org, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, willy@infradead.org, mgorman@suse.de, jon.grimm@amd.com, bharata@amd.com, raghavendra.kt@amd.com, Boris Ostrovsky , Konrad Wilk , jgross@suse.com, Andrew.Cooper3@citrix.com, Vineeth Pillai , Suleiman Souhlal , Ingo Molnar , Mathieu Desnoyers , Clark Williams , bigeasy@linutronix.de, daniel.wagner@suse.com, Joseph Salisbury , broonie@gmail.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: E4702C000D X-Stat-Signature: c47uupbacym843bphz6xicrqd8aeiiif X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1739196461-383961 X-HE-Meta: U2FsdGVkX1/lfJ0c4iNoKSjrTa5g0h7G4/paCvbg0Bdbozc/W/n2iDp5zYiN4tnBthMJ1ssQs5G3pWRKNbCLbroycr3HdVgyB9YNJNQCspi/02blzJ2wUIwyeyl0b/uJ9i3wLWy6NYZ+ymRGYizdnqf3O+5JtzKzgkIh27BbbBXL/9G/LbeJDCQi4231P9FWJusovXvMqG3MVIWUWCqHUaN9haKwCzgxgh5DPN87xWWIZIE5OX+Z5pMZk/4pUM5UUOskPeh2TZpY68APa87yP+Ruy4VTc3rqFi1ddktFcypgebSu8oH5jYTxNC0no4XK4sfkNr67ZknP3cX+nJrfJ8iHioUjb7wWw0kJxnHQAQukeZ1jLzwgCntPyoixemu853mZ4pIkkGI8bGek+tIOmyx8/4K4DwwUl9BIqrKQ/Qzb0cT3gfZ1TC8yEJvKVSXwNU+fwJ7VGPKRz3XKL+n70dyIz9+e2ytuEvugjRYpnAAScA+EJoXdT8H4/wumNxoEfoSjwNK4Y52P5V7ZbLp/uIDpWiHARyI3LrPekl8U9vtgp0IgP3U9TyxdyHAyPEfsdj7nUes3h+bdiHoqtxKsghO+XblEsbyw08sonOJxmEZisXqEJCOXQar3udOFsGPgC2Uxn4ThCTiyRJnoLPw30UoQdK255HSonPlKEX4pXsmdNSnUHjoPIKZBHnEPlML6txiPxTUKXZIbLjMCkmzEZH4zHC39SFZB9yFw3FwsmRWWhjx7ZSqrhPGxJKdB6B/nqd2y7revsNR5VO/4BFzlilTxiRtTvt0uj3+uSz1nbTfph7wLKXFwxSFUfOA8I6H3h7WG+zL3lSACN0qFB0yJtcE/bNN0HUZEXUmDK1yeBhoyuQmamozSL8OnUqbwLBD7F5NrtFemMhWaX0kcvDOnNWJnnvWz0UbtPDY2oCcUCGcKiyA4OsOruQNINMkLQ6YZFfl9jzog6r7J+8GLtTz CmJhDJa1 l/iho/dgLFxdqs3B8zu7mSPhDmlAgGupAiodF9v/at9NnAG/TbUhA8eeEswZz9gJdBF7TUn5QFT0q3Ip9K7iOIaaCNpdixA1YJw0X41NZZ5hVtP+XAOHRGDm3Mp8js1m5sqGSKCO1wgWWaIRRy94Ceu1JDMvU8X09nIkrpaOrSmEiM7aTOEvVa95DvIFvgOUsRmB2mcU9QVcg9pCZ7gb+0EMT6X/NfhpHlsozPPxe8Ves3LoBsHvTIcCmdqz2E55LvNmBHh+6jlhM3BZ72nt1lcgfoLvVFoCHR5wLl+htX3PEKpUEkZlMkQEbAjQP71KtZZcBU8qn2WTq7qpFxtBM5T8524kXPTbOpp3ZTxwNKTkpbYtZYsOylkArmBOyt4i1sKvs X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Feb 6, 2025 at 8:30=E2=80=AFAM Steven Rostedt = wrote: > > On Wed, 5 Feb 2025 22:07:12 -0500 > Joel Fernandes wrote: > > > > > > RT tasks don't have a time slice. They are affected by events. An ext= ernal > > > interrupt coming in, or a timer going off that states something is > > > happening. Perhaps we could use this for SCHED_RR or maybe even > > > SCHED_DEADLINE, as those do have time slices. > > > > > > But if it does get used, it should only be used when the task being > > > scheduled is the same SCHED_RR priority, or if SCHED_DEADLINE will no= t fail > > > its guarantees. > > > > > > > Right, it would apply still to RR/DL though... > > But it would have to guarantee that the RR it is delaying is of the same > priority, and that delaying the DL is not going to cause something to mis= s > its deadline. See Peter comment: "Then pick another number; RT too has a max scheduling latency number (on some random hardware). If you stay below that, all is fine.". > > 3. Overloading the purpose of LAZY: My understanding is, the purpose > > of LAZY is to let the scheduler decide if it wants to preempt based on > > preemption mode. It is not based on any hint, just on the preemption > > mode. I guess you are overloading LAZY by making LAZY flag also extend > > userspace timeslice (versus say making the time-slice extension hint > > its own thing...). > > I already replied about that. Note, LAZY was created in PREEMPT_RT for th= is > very purpose (but in the kernel), and ported to vanilla for a slightly > different purpose. > > Here's the history: > > PREEMPT_RT would convert spin_locks in the kernel to sleeping mutexes. > > This made RT tasks respond much faster to events. > > But non-RT (SCHED_OTHER) started suffering performance issues. > > When looking at the performance issues, we found that it was due to tas= ks > holding these sleeping spin_locks and being preempted. That is, the > preemption of holding spin_locks was causing more contention and slowin= g > things down tremendously. > > To first handle this, adaptive mutexes was introduced. These would spin > if the owner of the lock was still running, and would go to sleep if th= e > owner goes to sleep. This helped things quite a bit, but PREEMPT_RT was > still suffer a performance deficit compared to non-RT. > > This was because of the timer tick on SCHED_OTHER tasks that could > preempt a task holding a spin lock. > > NEED_RESCHED_LAZY was introduced to remedy this. It would be set for > SCHED_OTHER tasks and NEED_RESCHED for RT tasks. If the task was holdin= g > a sleeping spin lock, the NEED_RESCHED_LAZY would not preempt the runni= ng > task, but NEED_RESCHED would. If the SCHED_OTHER task was not holding a > sleeping spin_lock it would be preempted regardless. > > This improved the performance of SCHED_OTHER tasks in PREEMPT_RT to be as > good as what was in vanilla. > > You see, LAZY was *created* for this purpose. Of letting the scheduler kn= ow > that the running task is in a critical section and the timer tick should > not preempt a SCHED_OTHER task. > I just wanted to extend this to SCHED_OTHER in user space too. Currently it does not "let anyone know" it is running in a critical section though. Various paths (update_curr(), wake up) just do a 'lazy' resched until the timer tick has elapsed, or the task returns to usermode/idle at which point schedule() is called. And it does this only for FAIR tasks. That can well happen even if the currently running task is not in a critical section in the kernel at all. Sure, it may benefit critical sections in the upstream kernel but where is that explicit? I still feel we should not overload this in-kernel mechanism for userspace locking and complicate things. > > Yes, I have worked on RT projects before -- you would know better > > than anyone. :-D. But admittedly, I haven't got to work much with > > PREEMPT_RT systems. > > Just using RT policy to improve performance is not an RT project. I'm > talking about projects that if you miss a deadline things crash. Where th= e > project works very hard to make sure everything works as intended. No no no, I have done way more than applying just the RT policy. So that means you do not know me that well;-).. I have worked on audio driver latency, low latency audio, latency issues in vmalloc code, preempt tracers, irq tracepoints , wake up latency tracers and various scheduler overhead debug =E2=80=94 many of those issues dealt with sub millisecond latency.. I also dealt with cpu idle issues in the hardware causing real time latency problems (see my past talks if interested). I was partly a hardware engineer when I started my career and have built circuits. I have Electronics and Computer engineering degrees. - Joel