From: Ivan Babrou <ivan@cloudflare.com>
To: Shakeel Butt <shakeelb@google.com>
Cc: "Michal Koutný" <mkoutny@suse.com>,
"Daniel Dao" <dqminh@cloudflare.com>,
kernel-team <kernel-team@cloudflare.com>,
"Linux MM" <linux-mm@kvack.org>,
"Johannes Weiner" <hannes@cmpxchg.org>,
"Roman Gushchin" <guro@fb.com>, "Feng Tang" <feng.tang@intel.com>,
"Michal Hocko" <mhocko@kernel.org>,
"Hillf Danton" <hdanton@sina.com>,
"Andrew Morton" <akpm@linux-foundation.org>,
"Linus Torvalds" <torvalds@linux-foundation.org>
Subject: Re: Regression in workingset_refault latency on 5.15
Date: Wed, 2 Mar 2022 14:33:41 -0800 [thread overview]
Message-ID: <CABWYdi09io3jt4-ZoQffS+JbuKQoD32epx4682_2obY5Ap=mzw@mail.gmail.com> (raw)
In-Reply-To: <CABWYdi1sqSj7tb397FxS1KVURUF-RyduZj2Ubia7Gitocw8hsw@mail.gmail.com>
On Tue, Mar 1, 2022 at 7:40 PM Ivan Babrou <ivan@cloudflare.com> wrote:
>
> On Tue, Mar 1, 2022 at 6:50 PM Shakeel Butt <shakeelb@google.com> wrote:
> >
> > On Tue, Mar 01, 2022 at 04:48:00PM -0800, Ivan Babrou wrote:
> >
> > [...]
> >
> > > Looks like you were right that targeted flush is not going to be as good.
> >
> >
> > Thanks a lot. Can you please try the following patch (independent of other
> > patches) as well?
>
> Sure thing, I'll come back with the numbers tomorrow.
Measurements from different configurations:
* Baseline 5.15.25:
$ sudo /usr/share/bcc/tools/funclatency -d 60 workingset_refault
Tracing 1 functions for "workingset_refault"... Hit Ctrl-C to end.
nsecs : count distribution
0 -> 1 : 0 | |
2 -> 3 : 0 | |
4 -> 7 : 0 | |
8 -> 15 : 0 | |
16 -> 31 : 0 | |
32 -> 63 : 0 | |
64 -> 127 : 0 | |
128 -> 255 : 0 | |
256 -> 511 : 15931 |****************************************|
512 -> 1023 : 7040 |***************** |
1024 -> 2047 : 1566 |*** |
2048 -> 4095 : 3844 |********* |
4096 -> 8191 : 1689 |**** |
8192 -> 16383 : 115 | |
16384 -> 32767 : 54 | |
32768 -> 65535 : 42 | |
65536 -> 131071 : 14 | |
131072 -> 262143 : 4 | |
262144 -> 524287 : 4 | |
524288 -> 1048575 : 92 | |
1048576 -> 2097151 : 2322 |***** |
2097152 -> 4194303 : 8244 |******************** |
4194304 -> 8388607 : 775 |* |
8388608 -> 16777215 : 73 | |
avg = 776487 nsecs, total: 32464181849 nsecs, count: 41809
* Delayed flush:
$ sudo /usr/share/bcc/tools/funclatency -d 60 workingset_refault
Tracing 1 functions for "workingset_refault"... Hit Ctrl-C to end.
nsecs : count distribution
0 -> 1 : 0 | |
2 -> 3 : 0 | |
4 -> 7 : 0 | |
8 -> 15 : 0 | |
16 -> 31 : 0 | |
32 -> 63 : 0 | |
64 -> 127 : 0 | |
128 -> 255 : 0 | |
256 -> 511 : 16271 |****************************************|
512 -> 1023 : 8147 |******************** |
1024 -> 2047 : 2743 |****** |
2048 -> 4095 : 3964 |********* |
4096 -> 8191 : 7137 |***************** |
8192 -> 16383 : 4039 |********* |
16384 -> 32767 : 352 | |
32768 -> 65535 : 214 | |
65536 -> 131071 : 80 | |
131072 -> 262143 : 15 | |
262144 -> 524287 : 0 | |
524288 -> 1048575 : 1 | |
avg = 3248 nsecs, total: 139586555 nsecs, count: 42963
* Targeted flush:
$ sudo /usr/share/bcc/tools/funclatency -d 60 workingset_refault
Tracing 1 functions for "workingset_refault"... Hit Ctrl-C to end.
nsecs : count distribution
0 -> 1 : 0 | |
2 -> 3 : 0 | |
4 -> 7 : 0 | |
8 -> 15 : 0 | |
16 -> 31 : 0 | |
32 -> 63 : 0 | |
64 -> 127 : 0 | |
128 -> 255 : 0 | |
256 -> 511 : 25298 |****************************************|
512 -> 1023 : 13334 |********************* |
1024 -> 2047 : 1519 |** |
2048 -> 4095 : 4265 |****** |
4096 -> 8191 : 2339 |*** |
8192 -> 16383 : 162 | |
16384 -> 32767 : 73 | |
32768 -> 65535 : 70 | |
65536 -> 131071 : 23 | |
131072 -> 262143 : 1 | |
262144 -> 524287 : 3 | |
524288 -> 1048575 : 55 | |
1048576 -> 2097151 : 1409 |** |
2097152 -> 4194303 : 8306 |************* |
4194304 -> 8388607 : 1139 |* |
8388608 -> 16777215 : 89 | |
avg = 589361 nsecs, total: 34233041412 nsecs, count: 5808
* Your most recent patch:
$ sudo /usr/share/bcc/tools/funclatency -d 60 workingset_refault
Tracing 1 functions for "workingset_refault"... Hit Ctrl-C to end.
nsecs : count distribution
0 -> 1 : 0 | |
2 -> 3 : 0 | |
4 -> 7 : 0 | |
8 -> 15 : 0 | |
16 -> 31 : 0 | |
32 -> 63 : 0 | |
64 -> 127 : 0 | |
128 -> 255 : 0 | |
256 -> 511 : 37051 |****************************************|
512 -> 1023 : 17144 |****************** |
1024 -> 2047 : 1051 |* |
2048 -> 4095 : 2705 |** |
4096 -> 8191 : 1911 |** |
8192 -> 16383 : 184 | |
16384 -> 32767 : 93 | |
32768 -> 65535 : 70 | |
65536 -> 131071 : 30 | |
131072 -> 262143 : 21 | |
262144 -> 524287 : 216 | |
524288 -> 1048575 : 1100 |* |
1048576 -> 2097151 : 6522 |******* |
2097152 -> 4194303 : 5903 |****** |
4194304 -> 8388607 : 467 | |
8388608 -> 16777215 : 19 | |
avg = 403261 nsecs, total: 30037729036 nsecs, count: 74487
Looks like there are plenty of longer pauses still.
I can also get the numbers from 5.10 for you if you want.
next prev parent reply other threads:[~2022-03-02 22:33 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-02-23 13:51 Daniel Dao
2022-02-23 15:57 ` Shakeel Butt
2022-02-23 16:00 ` Shakeel Butt
2022-02-23 17:07 ` Daniel Dao
2022-02-23 17:36 ` Shakeel Butt
2022-02-23 19:28 ` Ivan Babrou
2022-02-23 20:28 ` Shakeel Butt
2022-02-23 21:16 ` Ivan Babrou
2022-02-24 14:46 ` Daniel Dao
2022-02-24 16:58 ` Shakeel Butt
2022-02-24 17:34 ` Daniel Dao
2022-02-24 18:00 ` Shakeel Butt
2022-02-24 18:52 ` Shakeel Butt
2022-02-25 10:23 ` Daniel Dao
2022-02-25 17:08 ` Ivan Babrou
2022-02-25 17:22 ` Shakeel Butt
2022-02-25 18:03 ` Michal Koutný
2022-02-25 18:08 ` Ivan Babrou
2022-02-28 23:09 ` Shakeel Butt
2022-02-28 23:34 ` Ivan Babrou
2022-02-28 23:43 ` Shakeel Butt
2022-03-02 0:48 ` Ivan Babrou
2022-03-02 2:50 ` Shakeel Butt
2022-03-02 3:40 ` Ivan Babrou
2022-03-02 22:33 ` Ivan Babrou [this message]
2022-03-03 2:32 ` Shakeel Butt
2022-03-03 2:35 ` Shakeel Butt
2022-03-04 0:21 ` Ivan Babrou
2022-03-04 1:05 ` Shakeel Butt
2022-03-04 1:12 ` Ivan Babrou
2022-03-02 11:49 ` Frank Hofmann
2022-03-02 15:52 ` Shakeel Butt
2022-03-02 10:08 ` Michal Koutný
2022-03-02 15:53 ` Shakeel Butt
2022-03-02 17:28 ` Ivan Babrou
2022-02-24 9:22 ` Thorsten Leemhuis
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CABWYdi09io3jt4-ZoQffS+JbuKQoD32epx4682_2obY5Ap=mzw@mail.gmail.com' \
--to=ivan@cloudflare.com \
--cc=akpm@linux-foundation.org \
--cc=dqminh@cloudflare.com \
--cc=feng.tang@intel.com \
--cc=guro@fb.com \
--cc=hannes@cmpxchg.org \
--cc=hdanton@sina.com \
--cc=kernel-team@cloudflare.com \
--cc=linux-mm@kvack.org \
--cc=mhocko@kernel.org \
--cc=mkoutny@suse.com \
--cc=shakeelb@google.com \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox