From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A51F7C433EF for ; Mon, 8 Nov 2021 09:21:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2BA3D6125F for ; Mon, 8 Nov 2021 09:21:19 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 2BA3D6125F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 9E6016B0073; Mon, 8 Nov 2021 04:21:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 994346B0074; Mon, 8 Nov 2021 04:21:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 882D06B0075; Mon, 8 Nov 2021 04:21:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0120.hostedemail.com [216.40.44.120]) by kanga.kvack.org (Postfix) with ESMTP id 799586B0073 for ; Mon, 8 Nov 2021 04:21:18 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 305C377817 for ; Mon, 8 Nov 2021 09:21:18 +0000 (UTC) X-FDA: 78785219436.03.08EA31A Received: from mail-qv1-f49.google.com (mail-qv1-f49.google.com [209.85.219.49]) by imf11.hostedemail.com (Postfix) with ESMTP id C3044F000218 for ; Mon, 8 Nov 2021 09:21:17 +0000 (UTC) Received: by mail-qv1-f49.google.com with SMTP id b17so11486829qvl.9 for ; Mon, 08 Nov 2021 01:21:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=44pYJW76BNvqnwNFsAADAvCHu/ye4pCtZSvD5oLz7Vo=; b=man9nIqCDwUVnQNAWgFdoB2KVK30SLTwTXmMoapBLTbDq7xcZqKx6iuNWv0W+05VyH awidQxm+pD/aU3/NxEJLPifZURF93KlhwJyoHd07aT8amXcsz+SgEf4ZvoeJgHhp3bB5 jfmyqjtalwH/1IngvKdVR5VGSnEzU9aESvbJLoEJE7+GiKiyzOtzwi0i0EkJGlgar6ik h/CM27NsL+AwRGpJEXUUFBn8qJJt7D0HiN0GpTE4nOmiPARCD8ipDO5skB6QJ0Fm0MgA UsJhTs1bU2/5FPyn/T3qhHe2r4IHbiwQAL4cgjBLrVSZg3VysUsT9/OhlpmTi1uWu2FV GRLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=44pYJW76BNvqnwNFsAADAvCHu/ye4pCtZSvD5oLz7Vo=; b=PFrFZVixX/nfYCd+GH87QODQvrRvL9fZpz1sftJMpWN/YOG9iGUUZ6H/qV3KbTCgZi D2MgM3RdZf7SgovhL3LCqKfwpOeq/tP95xF079awekjR4Hsb90iS8S6X1lWAKsHIpL9d 9h3LgTOzM3m4wuhbCeDoyIa3WDG//m3KE3WfmIulkwoyJgDi9YPUQ20/8Wzdi/pkGIOa rE9arkko/fQkoSzdoToz2+rkK7ngNDAXAuwBl9JBI3uQcHMJciPVwcvk1xpgG3PIEob+ Ubu3MQidbPSvHWPzoWrqOT+SGpROdWqQ/Tu6PKBwfLMkCwf8N8XFUhxz0hDGN64tL1Bq Z2Vg== X-Gm-Message-State: AOAM533fcH6gWMQxgnUelNJPKzOjtJqwWvFu6NIAfzz3h8qJM96VhaD2 glWBkGsI4hL8xqWIQbVCR99ZcsYI/IZ8B1yfBbo= X-Google-Smtp-Source: ABdhPJz03DiS70ytD4xBMmawGUMpmbGTktECHfQ7kfsODxQJO4onqlNyaKjiORHRtVwTOUSd5u1CvlhbZZD5dwMEx0w= X-Received: by 2002:a05:6214:d8e:: with SMTP id e14mr17462332qve.37.1636363277093; Mon, 08 Nov 2021 01:21:17 -0800 (PST) MIME-Version: 1.0 References: <1634278612-17055-1-git-send-email-huangzhaoyang@gmail.com> <78b3f72b-3fe7-f2e0-0e6b-32f28b8ce777@arm.com> <85c81ab7-49ed-aba5-6221-ea6a8f37f8ad@arm.com> In-Reply-To: From: Zhaoyang Huang Date: Mon, 8 Nov 2021 17:20:56 +0800 Message-ID: Subject: Re: [Resend PATCH] psi : calc cfs task memstall time more precisely To: Xuewen Yan Cc: Dietmar Eggemann , Johannes Weiner , Andrew Morton , Michal Hocko , Vladimir Davydov , Zhaoyang Huang , "open list:MEMORY MANAGEMENT" , LKML , Peter Zijlstra , Vincent Guittot , xuewen.yan@unisoc.com, Ke Wang Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: C3044F000218 X-Stat-Signature: dfx3w6jwi5a9imhs6pc4j1up6wcde8ce Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=man9nIqC; spf=pass (imf11.hostedemail.com: domain of huangzhaoyang@gmail.com designates 209.85.219.49 as permitted sender) smtp.mailfrom=huangzhaoyang@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-HE-Tag: 1636363277-563669 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Nov 8, 2021 at 4:49 PM Xuewen Yan wrote: > > Hi Dietmar > > On Sat, Nov 6, 2021 at 1:20 AM Dietmar Eggemann > wrote: > > > > On 05/11/2021 06:58, Zhaoyang Huang wrote: > > >> I don't understand the EAS (probably asymmetric CPU capacity is meant > > >> here) angle of the story. Pressure on CPU capacity which is usable for > > >> CFS happens on SMP as well? > > > Mentioning EAS here mainly about RT tasks preempting small CFS tasks > > > (big CFS tasks could be scheduled to big core), which would introduce > > > more proportion of preempted time within PSI_MEM_STALL than SMP does. > > > > What's your CPU layout? Do you have the little before the big CPUs? Like > > Hikey 960? > > > > root@linaro-developer:~# cat /sys/devices/system/cpu/cpu*/cpu_capacity > > 462 > > 462 > > 462 > > 462 > > 1024 > > 1024 > > 1024 > > 1024 > > > > And I guess rt class prefers lower CPU numbers hence you see this? > > > our CPU layout is: > xuewen.yan:/ # cat /sys/devices/system/cpu/cpu*/cpu_capacity > 544 > 544 > 544 > 544 > 544 > 544 > 1024 > 1024 > > And in our platform, we use the kernel in mobile phones with Android. > And we prefer power, so we prefer the RT class to run on little cores. > > > > >> > > >> This will let the idle task (swapper) pass. Is this indented? Or do you > > >> want to only let CFS tasks (including SCHED_IDLE) pass? > > > idle tasks will NOT call psi_memstall_xxx. We just want CFS tasks to > > > scale the STALL time. > > > > Not sure I get this. > > > > __schedule() -> psi_sched_switch() -> psi_task_change() -> > > psi_group_change() -> record_times() -> psi_memtime_fixup() > > > > is something else than calling psi_memstall_enter() or _leave()? > > > > IMHO, at least record_times() can be called with current equal > > swapper/X. Or is it that PSI_MEM_SOME is never set for the idle task in > > this callstack? I don't know the PSI internals. According to my understanding, PSI_MEM_SOME represents the CORE's state within which there is at least one task trapped in memstall path(only counted in by calling PSI_MEMSTALL_ENTER). record_times is responsible for collecting the delta time of the CORE since it start. What we are doing is to make the delta time more precise. So idle task is irrelevant for these. > > > > >> > > >> if (current->sched_class != &fair_sched_class) > > >> return growth_fixed; > > >> > > >>>>>> + > > >>>>>> + if (current->in_memstall) > > >>>>>> + growth_fixed = div64_ul((1024 - rq->avg_rt.util_avg - rq->avg_dl.util_avg > > >>>>>> + - rq->avg_irq.util_avg + 1) * growth, 1024); > > >>>>>> + > > >> > > >> We do this slightly different in scale_rt_capacity() [fair.c]: > > >> > > >> max = arch_scale_cpu_capacity(cpu_of(rq) /* instead of 1024 to support > > >> asymmetric CPU capacity */ > > > Is it possible that the SUM of rqs' util_avg large than > > > arch_scale_cpu_capacity because of task migration things? > > > > I assume you meant if the rq (cpu_rq(CPUx)) util_avg sum (CFS, RT, DL, > > IRQ and thermal part) can be larger than arch_scale_cpu_capacity(CPUx)? > > > > Yes it can. > > > > Have a lock at > > > > effective_cpu_util(..., max, ...) { > > > > if (foo >= max) > > return max; > > > > } > > > > Even the CFS part (cpu_rq(CPUx)->cfs.avg.util_avg) can be larger than > > the original cpu capacity (rq->cpu_capacity_orig). > > > > Have a look at cpu_util(). capacity_orig_of(CPUx) and > > arch_scale_cpu_capacity(CPUx) both returning rq->cpu_capacity_orig. > > > > Well, your means is we should not use the 1024 and should use the > original cpu capacity? > And maybe use the "sched_cpu_util()" is a good choice just like this: > > + if (current->in_memstall) > + growth_fixed = div64_ul(cpu_util_cfs(rq) * growth, > sched_cpu_util(rq->cpu, capacity_orig_of(rq->cpu))); > > Thanks! > > BR > xuewen