From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E090C54EE9 for ; Tue, 20 Sep 2022 15:18:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 93C41940009; Tue, 20 Sep 2022 11:18:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8ECF2940007; Tue, 20 Sep 2022 11:18:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7B383940009; Tue, 20 Sep 2022 11:18:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 6CBBD940007 for ; Tue, 20 Sep 2022 11:18:03 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 2F13B1603CC for ; Tue, 20 Sep 2022 15:18:03 +0000 (UTC) X-FDA: 79932819246.05.9424ADB Received: from mail-lf1-f50.google.com (mail-lf1-f50.google.com [209.85.167.50]) by imf14.hostedemail.com (Postfix) with ESMTP id CF31D100038 for ; Tue, 20 Sep 2022 15:18:02 +0000 (UTC) Received: by mail-lf1-f50.google.com with SMTP id u18so4329427lfo.8 for ; Tue, 20 Sep 2022 08:18:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date; bh=5se1nLah5wB7N2z+/BXJGVEJry5l8HVmuf5730Cxm1c=; b=t/GsXU0TSldX02gXKmfWhpOaf9w10U4rMd7iYYtyHToeJwoAZbn1VARtKPcjIrzJax vGhTZMfwXH44f/9DBonR/qhyrV320zwtnmLK2WTYEWn+q1tCsYt1asWA6V3ZWRWICvT9 n/55/9QYwqCkn3T/l0/VJQoPch7nuxBwgX4gZZEWn57lme27FqZqX8ZiwEnTB8wVP7kV P1kzzHbAzmAAOFqb7wei+kL0Lx8fu+zQsUYlzMuEsZE1L6SJPfl7nVuZZe0OxGSB8fnR yfPlsNep6MPbUZ4WnPa2IaZwXwbuzYybnF4/Dw20vnXTjoJwrM7Q+fBfUdASo6Xx3kz2 9LYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date; bh=5se1nLah5wB7N2z+/BXJGVEJry5l8HVmuf5730Cxm1c=; b=bjWW5AnNt61TRzxCUNJPw3cSBjNZvFAtlm1UwPfCCzQNflrf6zo2MtYm6Imh+MvQP0 fVamu+9hKnxcc5k2L4zQXDdAto3RrtWp+st0JPGTAfufAJ/al/T2kHSPxjLm193g4HJp cVmcqwpnetDMxe/Bj34g945FcVeI7x7TN0T70vPuGZx0cRaBnk4cO8DJGJZiQbKkOIAu IkdbtrwnFnlS0Mbn/KrHwA3N4Aq+0Ud5ifNObE3hLhujE4gvwd+jbfXq0CEGb4Tgl0B/ jtiurTIUYkd4DQ8RShItCXwChcMa/RfN1uppVYwNCFz0R6TnW36mHexGSpAFzfQ54JvR XqiA== X-Gm-Message-State: ACrzQf2RKpmDGEdFWHF1MLuLXX85NKNYnVWBnCECzacPfQR1m1sAUz6m DD5j5ggZXwbAxLMuMKs05kIjRajEpjs5iaJC9Bt7Cg== X-Google-Smtp-Source: AMsMyM7g2GB6gWS/U1QhMJ4bhZeEHRjSTibIwVDLMHasj5Gw1dMEzmBKIGssqwSpPOnFUDfFvoU6IVNBam3WpyHqSMY= X-Received: by 2002:a05:6512:2286:b0:49e:eb:ea19 with SMTP id f6-20020a056512228600b0049e00ebea19mr8571266lfu.645.1663687080917; Tue, 20 Sep 2022 08:18:00 -0700 (PDT) MIME-Version: 1.0 References: <20220916080305.29574-6-vincent.guittot@linaro.org> <20220916120245.2951-1-hdanton@sina.com> <20220917225819.817-1-hdanton@sina.com> <20220920113238.1176-1-hdanton@sina.com> In-Reply-To: <20220920113238.1176-1-hdanton@sina.com> From: Vincent Guittot Date: Tue, 20 Sep 2022 17:17:49 +0200 Message-ID: Subject: Re: [PATCH v4 5/8] sched/fair: Take into account latency priority at wakeup To: Hillf Danton Cc: peterz@infradead.org, mgorman@suse.de, valentin.schneider@arm.com, parth@linux.ibm.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1663687082; a=rsa-sha256; cv=none; b=MAIGK3CT7sbKiKpe2OXWBVYPOAA56FK1B1KgeaMUOzF2G+Rx4auJOCKSe8MGeqYlZwTF9+ xtlFKUfEt1BQXjaHFPTSJ1l6JvpIjubqb2YSFt6BemQ1q40+vpERB3uq6WxTEXggC8BMHo l9a4WqHK0ZZfDPASkN3M0QP8Ny1OHt4= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=linaro.org header.s=google header.b="t/GsXU0T"; dmarc=pass (policy=none) header.from=linaro.org; spf=pass (imf14.hostedemail.com: domain of vincent.guittot@linaro.org designates 209.85.167.50 as permitted sender) smtp.mailfrom=vincent.guittot@linaro.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1663687082; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5se1nLah5wB7N2z+/BXJGVEJry5l8HVmuf5730Cxm1c=; b=n4iVPS7RUAYOdxfMamDdaSShPun05iJsvdQOO9/FyxCPV5N6j+sXv6JJCEeSDAowWudhU8 X7b+LkwDhKSSRf231V48ZhgpG4GtKoKzEnqVbdgMZEfI251rGN6dhqHyRpY+ReaiBscQTu HrIM7Xqo2/6B9FtvIgEMY0RDSOXIVdw= X-Stat-Signature: tftx6dtn7georubbqwu913ge46sro1bk X-Rspamd-Queue-Id: CF31D100038 X-Rspam-User: Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=linaro.org header.s=google header.b="t/GsXU0T"; dmarc=pass (policy=none) header.from=linaro.org; spf=pass (imf14.hostedemail.com: domain of vincent.guittot@linaro.org designates 209.85.167.50 as permitted sender) smtp.mailfrom=vincent.guittot@linaro.org X-Rspamd-Server: rspam03 X-HE-Tag: 1663687082-532515 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, 20 Sept 2022 at 13:32, Hillf Danton wrote: > > On 18 Sep 2022 12:46:00 +0200 Vincent Guittot wrote: > > On Sun, 18 Sept 2022 at 00:58, Hillf Danton wrote: > > > > > > On 16 Sep 2022 15:36:53 +0200 Vincent Guittot wrote: > > > > > > > > Hi Hillf, > > > > > > > > On Fri, 16 Sept 2022 at 14:03, Hillf Danton wrote: > > > > > > > > > > Hello Vincent > > > > > > > > > > On 16 Sep 2022 10:03:02 +0200 Vincent Guittot wrote: > > > > > > > > > > > > @@ -4606,6 +4608,7 @@ check_preempt_tick(struct cfs_rq *cfs_rq, struct sched_entity *curr) > > > > > > > > > > > > se = __pick_first_entity(cfs_rq); > > > > > > delta = curr->vruntime - se->vruntime; > > > > > > + delta -= wakeup_latency_gran(curr, se); > > > > > > > > > > > > if (delta < 0) > > > > > > return; > > > > > > > > > > What is derived from the latency nice you added is the runtime granulaity > > > > > which has a role in preempting the current task. > > > > > > > > > > Given the same defination of latency nice as the nice, the runtime granularity > > > > > can be computed without introducing the latency nice. > > > > > > > > > > Only for thoughts now. > > > > > > > > > > Hillf > > > > > > > > > > +++ b/kernel/sched/fair.c > > > > > @@ -4569,7 +4569,7 @@ dequeue_entity(struct cfs_rq *cfs_rq, st > > > > > static void > > > > > check_preempt_tick(struct cfs_rq *cfs_rq, struct sched_entity *curr) > > > > > { > > > > > - unsigned long ideal_runtime, delta_exec; > > > > > + unsigned long ideal_runtime, delta_exec, granu; > > > > > struct sched_entity *se; > > > > > s64 delta; > > > > > > > > > > @@ -4594,6 +4594,14 @@ check_preempt_tick(struct cfs_rq *cfs_rq > > > > > return; > > > > > > > > > > se = __pick_first_entity(cfs_rq); > > > > > + > > > > > + granu = sysctl_sched_min_granularity + > > > > > + (ideal_runtime - sysctl_sched_min_granularity) * > > > > > + (se->latency_nice + 20) / LATENCY_NICE_WIDTH; > > > > > > > > There is no latency_nice field in se but a latency_offset instead > > > > > > > > Also at this step, we are sure that curr has run at least > > > > sysctl_sched_min_granularity and we want now to compare curr vruntime > > > > with first se's one. We take the latency offset into account to make > > > > sure we will not preempt curr too early > > > > > > > > > + > > > > > + if (delta_exec < granu) > > > > > + return; > > > > > + > > > > > delta = curr->vruntime - se->vruntime; > > > > > > > > > > if (delta < 0) > > > return; > > > > > > if (delta > ideal_runtime) > > > resched_curr(rq_of(cfs_rq)); > > > > > > After another look, curr is not preempted without the gap in vruntime > > > between curr and the first entity growing more than ideal runtime, while > > > > Curr can be preempted as it has run more than the ideal time (1st > > test). This one is to make sure that the diff does not become too > > large. Here we reuse the same comparison as wakeup to make sure that a > > newly curr will get a chance to run its ideal time after having > > preempted at wakeup the previous current > > IIUC it would take two checks to preempt correctly - diff in vruntime > is checked first to avoid preempting too early, then it is checked again > with latency_offset taken into account to avoid preempting too late. The 1st test in check_preempt_tick() : if (delta_exec > ideal_runtime) ensures that a resched happens after the current run is slice The 2nd test : if (delta_exec < sysctl_sched_min_granularity) ensures that current will run at least 3ms The 3rd one : if (delta > ideal_runtime) is there to make sure that there is not too much diff between the vruntime. But we are comparing virtual runtime with execution time and as Peter mentioned in another thread that's weird. [1] will fix it in addition to ensure max runtime. Then, current might have preempted first_entity few ms before thanks to its latency_offset. If the tick happens quickly after the preemption, delta might be above ideal_runtime whereas current has run its ideal time yet [1] https://lore.kernel.org/all/20220916131538.24706-1-vincent.guittot@linaro.org/ > > +++ b/kernel/sched/fair.c > @@ -4571,7 +4571,7 @@ check_preempt_tick(struct cfs_rq *cfs_rq > { > unsigned long ideal_runtime, delta_exec; > struct sched_entity *se; > - s64 delta; > + s64 delta, d2; > > ideal_runtime = sched_slice(cfs_rq, curr); > delta_exec = curr->sum_exec_runtime - curr->prev_sum_exec_runtime; > @@ -4595,11 +4595,9 @@ check_preempt_tick(struct cfs_rq *cfs_rq > > se = __pick_first_entity(cfs_rq); > delta = curr->vruntime - se->vruntime; > + d2 = delta - wakeup_latency_gran(curr, se); > > - if (delta < 0) > - return; > - > - if (delta > ideal_runtime) > + if (delta > ideal_runtime || d2 > ideal_runtime) > resched_curr(rq_of(cfs_rq)); > } >