From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1597C433FE for ; Sat, 8 Oct 2022 10:34:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A59396B0072; Sat, 8 Oct 2022 06:34:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A07036B0073; Sat, 8 Oct 2022 06:34:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8CF366B0074; Sat, 8 Oct 2022 06:34:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 777F16B0072 for ; Sat, 8 Oct 2022 06:34:53 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 3DF161C61DE for ; Sat, 8 Oct 2022 10:34:53 +0000 (UTC) X-FDA: 79997424066.26.548A643 Received: from r3-23.sinamail.sina.com.cn (r3-23.sinamail.sina.com.cn [202.108.3.23]) by imf29.hostedemail.com (Postfix) with ESMTP id 3A4A6120021 for ; Sat, 8 Oct 2022 10:34:50 +0000 (UTC) Received: from unknown (HELO localhost.localdomain)([114.249.60.223]) by sina.com (172.16.97.23) with ESMTP id 634151EB0002A7DD; Sat, 8 Oct 2022 18:33:17 +0800 (CST) X-Sender: hdanton@sina.com X-Auth-ID: hdanton@sina.com X-SMAIL-MID: 30070154919543 From: Hillf Danton To: Youssef Esmat Cc: Vincent Guittot , peterz@infradead.org, dietmar.eggemann@arm.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH v5 7/7] sched/fair: Add latency list Date: Sat, 8 Oct 2022 18:34:39 +0800 Message-Id: <20221008103439.107-1-hdanton@sina.com> In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1665225293; a=rsa-sha256; cv=none; b=w7k9LL7FgxOcHvvUOfXmMfDH1FKswbchT8cbV9c/1q1qrQaCZk4ifS6RlgPSw9l+6R/Kwo DIoc+vzDVMkKtEbJKa1vOgEQSsRZacxnG/CiO/Yv9dNLYhDyJNLtcsEUFby84unbepATjT ZYdgblyX2NdF+CQNQlcx7XMjBf77aUQ= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=none; spf=pass (imf29.hostedemail.com: domain of hdanton@sina.com designates 202.108.3.23 as permitted sender) smtp.mailfrom=hdanton@sina.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1665225293; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=idhOroyxfNtVwS82nJlB3icGrIbRnBqqDLsdvFdjtPc=; b=fG8xtpRQhLgjoEgQWWBfRky8Paz72v8wmksK1k4TcCNCdm3Lv91ysssSwe1BdMfKmJWd5G 981OreBpKH2KQEBodLBW6wT5FDo+I5RfMUzuj1UV7BxQdMz18v+ZW2wWPqBQkloUxw8tAd LmAcuesKb1A8BNzxfSx057pYj3+5f5g= Authentication-Results: imf29.hostedemail.com; dkim=none; spf=pass (imf29.hostedemail.com: domain of hdanton@sina.com designates 202.108.3.23 as permitted sender) smtp.mailfrom=hdanton@sina.com; dmarc=none X-Stat-Signature: kji915h88ecm6y6jjti8gc3k5j6jesnw X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 3A4A6120021 X-Rspam-User: X-HE-Tag: 1665225290-145604 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 7 Oct 2022 20:04:51 -0500 Youssef Esmat > Hi Vincent, > > On Sun, Sep 25, 2022 at 9:39 AM Vincent Guittot wrote: > > > > Add a rb tree for latency sensitive entities so we can schedule the most > > sensitive one first even when it failed to preempt current at wakeup or > > when it got quickly preempted by another entity of higher priority. > > > > In order to keep fairness, the latency is used once at wakeup to get a > > minimum slice and not during the following scheduling slice to prevent > > long running entity to got more running time than allocated to his nice > > priority. > > > > The rb tree nebales to cover the last corner case where latency > > sensitive entity can't got schedule quickly after the wakeup. > > > > hackbench -l 10000 -g $group & > > cyclictest --policy other -D 5 -q -n > > latency 0 latency -20 > > group min avg max min avg max > > 0 17 19 29 17 18 30 > > 1 65 306 7149 64 83 208 > > 4 50 395 15731 56 80 271 > > 8 56 781 41548 54 80 301 > > 16 60 1392 87237 59 86 490 > > > > group = 0 means that hackbench is not running. > > > > Both avg and max are significantly improved with nice latency -20. If we > > add the histogram parameters to get details of latency, we have : > > > > hackbench -l 10000 -g 16 & > > cyclictest --policy other -D 5 -q -n -H 20000 --histfile data.txt > > latency 0 latency -20 > > Min Latencies: 60 61 > > Avg Latencies: 1077 86 > > Max Latencies: 87311 444 > > 50% latencies: 92 85 > > 75% latencies: 554 90 > > 85% latencies: 1019 93 > > 90% latencies: 1346 96 > > 95% latencies: 5400 100 > > 99% latencies: 19044 110 > > > > Signed-off-by: Vincent Guittot > > --- > > The ability to boost the latency sensitivity of a task seems very > interesting. I have been playing around with these changes and have > some observations. > > I tried 2 bursty tasks affinitized to the same CPU. The tasks sleep > for 1ms and run for 10ms in a loop. I first tried it without adjusting > the latency_nice value and took perf sched traces: > > latency_test:7040 | 2447.137 ms | 8 | avg: 6.546 ms | > max: 10.674 ms | max start: 353.809487 s | max end: 353.820161 s > latency_test:7028 | 2454.777 ms | 7 | avg: 4.494 ms | > max: 10.609 ms | max start: 354.804386 s | max end: 354.814995 s > > Everything looked as expected, for a 5s run they had similar runtime > and latency. > > I then adjusted one task to have a latency_nice of -20 (pid 8614 > below) and took another set of traces: > > latency_test:8618 | 1845.534 ms | 131 | avg: 9.764 ms | > max: 10.686 ms | max start: 1405.737905 s | max end: 1405.748592 s > latency_test:8614 | 3033.635 ms | 16 | avg: 3.559 ms | > max: 10.467 ms | max start: 1407.594751 s | max end: 1407.605218 s > > The task with -20 latency_nice had significantly more runtime. The > average latency was improved but the max roughly stayed the same. As > expected the one with latency_nice value of 0 experienced more > switches, but so did the one with latency_nice of -20. Hey Youssef See if revert works again in this case in terms of fixing regression. Hillf +++ b/kernel/sched/fair.c @@ -4571,7 +4571,7 @@ check_preempt_tick(struct cfs_rq *cfs_rq { unsigned long ideal_runtime, delta_exec; struct sched_entity *se; - s64 delta; + s64 delta, d2; ideal_runtime = sched_slice(cfs_rq, curr); delta_exec = curr->sum_exec_runtime - curr->prev_sum_exec_runtime; @@ -4595,12 +4595,12 @@ check_preempt_tick(struct cfs_rq *cfs_rq se = __pick_first_entity(cfs_rq); delta = curr->vruntime - se->vruntime; - delta -= wakeup_latency_gran(curr, se); + d2 = delta - wakeup_latency_gran(curr, se); if (delta < 0) return; - if (delta > ideal_runtime) + if (delta > ideal_runtime || d2 > ideal_runtime) resched_curr(rq_of(cfs_rq)); }