Re: Regression in workingset_refault latency on 5.15

[Daniel Dao <dqminh@cloudflare.com>]

On Thu, Feb 24, 2022 at 6:52 PM Shakeel Butt <shakeelb@google.com> wrote:
>
> On Thu, Feb 24, 2022 at 10:00:55AM -0800, Shakeel Butt wrote:
> > On Thu, Feb 24, 2022 at 9:34 AM Daniel Dao <dqminh@cloudflare.com> wrote:
> [...]
> > Anyways I am thinking of introducing mem_cgroup_flush_stats_asyn()
> > which will schedule flush_memcg_stats_dwork() without delay. Let me
> > prepare the patch based on 5.15-stable for you to test.
>
>
> Can you please try the following patch and let me the results?
>
> diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
> index d9b8df5ef212..cd732d7e00ca 100644
> --- a/include/linux/memcontrol.h
> +++ b/include/linux/memcontrol.h
> @@ -1002,6 +1002,7 @@ static inline unsigned long
> lruvec_page_state_local(struct lruvec *lruvec,
>   }
>
>   void mem_cgroup_flush_stats(void);
> +void mem_cgroup_flush_stats_async(void);
>
>   void __mod_memcg_lruvec_state(struct lruvec *lruvec, enum node_stat_item
> idx,
>                               int val);
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index 32ba963ebf2e..0f298cbd4763 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -682,6 +682,14 @@ void mem_cgroup_flush_stats(void)
>                 __mem_cgroup_flush_stats();
>   }
>
> +void mem_cgroup_flush_stats_aync(void)
> +{
> +       if (atomic_read(&stats_flush_threshold) > num_online_cpus()) {
> +               mod_delayed_work(system_unbound_wq, &stats_flush_dwork, 0);
> +               atomic_set(&stats_flush_threshold, 0);
> +       }
> +}
> +
>   static void flush_memcg_stats_dwork(struct work_struct *w)
>   {
>         __mem_cgroup_flush_stats();
> diff --git a/mm/workingset.c b/mm/workingset.c
> index d5b81e4f4cbe..86d43bfc5c63 100644
> --- a/mm/workingset.c
> +++ b/mm/workingset.c
> @@ -352,7 +352,7 @@ void workingset_refault(struct page *page, void *shadow)
>
>         inc_lruvec_state(lruvec, WORKINGSET_REFAULT_BASE + file);
>
> -       mem_cgroup_flush_stats();
> +       mem_cgroup_flush_stats_async();
>         /*
>          * Compare the distance to the existing workingset size. We
>          * don't activate pages that couldn't stay resident even if

I think this looks good so far. Some stats:

1) More time spent in flush_memcg_stats_dwork

$  sudo /usr/share/bcc/tools/funclatency -d 60 flush_memcg_stats_dwork
Tracing 1 functions for "flush_memcg_stats_dwork"... Hit Ctrl-C to end.

     nsecs               : count     distribution
         0 -> 1          : 0        |                                        |
         2 -> 3          : 0        |                                        |
         4 -> 7          : 0        |                                        |
         8 -> 15         : 0        |                                        |
        16 -> 31         : 0        |                                        |
        32 -> 63         : 0        |                                        |
        64 -> 127        : 0        |                                        |
       128 -> 255        : 0        |                                        |
       256 -> 511        : 21       |                                        |
       512 -> 1023       : 115      |                                        |
      1024 -> 2047       : 47       |                                        |
      2048 -> 4095       : 86       |                                        |
      4096 -> 8191       : 48       |                                        |
      8192 -> 16383      : 2        |                                        |
     16384 -> 32767      : 2        |                                        |
     32768 -> 65535      : 1        |                                        |
     65536 -> 131071     : 52       |                                        |
    131072 -> 262143     : 400      |**                                      |
    262144 -> 524287     : 2172     |****************                        |
    524288 -> 1048575    : 4193     |*******************************         |
   1048576 -> 2097151    : 5357     |****************************************|
   2097152 -> 4194303    : 1834     |*************                           |
   4194304 -> 8388607    : 485      |***                                     |
   8388608 -> 16777215   : 3        |                                        |

avg = 1346983 nsecs, total: 19959597708 nsecs, count: 14818


2) Less time spent in workingset_refault

~$ sudo /usr/share/bcc/tools/funclatency -d 60 workingset_refault
Tracing 1 functions for "workingset_refault"... Hit Ctrl-C to end.

     nsecs               : count     distribution
         0 -> 1          : 0        |                                        |
         2 -> 3          : 0        |                                        |
         4 -> 7          : 0        |                                        |
         8 -> 15         : 0        |                                        |
        16 -> 31         : 0        |                                        |
        32 -> 63         : 0        |                                        |
        64 -> 127        : 0        |                                        |
       128 -> 255        : 0        |                                        |
       256 -> 511        : 10318    |****************************************|
       512 -> 1023       : 2877     |***********                             |
      1024 -> 2047       : 1089     |****                                    |
      2048 -> 4095       : 4785     |******************                      |
      4096 -> 8191       : 4528     |*****************                       |
      8192 -> 16383      : 1270     |****                                    |
     16384 -> 32767      : 82       |                                        |
     32768 -> 65535      : 80       |                                        |
     65536 -> 131071     : 20       |                                        |
    131072 -> 262143     : 2        |                                        |
    262144 -> 524287     : 0        |                                        |
    524288 -> 1048575    : 1        |                                        |

avg = 2705 nsecs, total: 67782139 nsecs, count: 25052