Re: phantom memory in a cgroup (was [BUG] ZSwap leaks memory upon being disabled)

[Yosry Ahmed <yosryahmed@google.com>]

On Wed, Oct 30, 2024 at 7:41 AM Konstantin Kharlamov <Hi-Angel@yandex.ru> wrote:
>
> On Mon, 2024-10-28 at 01:13 +0300, Konstantin Kharlamov wrote:
> > On Sun, 2024-10-27 at 12:31 -0700, Yosry Ahmed wrote:
> > > One thing you can do is take a snapshot of memory.stat when
> > > memory.swap.current is at a high value (for sddm), then swapoff,
> > > then
> > > take another snapshot of memory.stat.
> > >
> > > We should see an increase in either anon or shmem, which will tell
> > > us
> > > which type of memory was swapped out.
> >
> > Okay. I will have to wait, because the session got killed by OOM. But
> > I
> > think it's gonna reproduce in just a few days, my new workflow seems
> > to
> > be triggering that a lot.
>
> Done. I missed one cycle, which again got my session killed by OOM 😅
> Now I caught this in time. The information was retrieved by:
>
>     (systemctl status sddm && cat /sys/fs/cgroup/system.slice/sddm.service/memory.stat) > ~/Projects/cgroups-mem-leak/"$(date -R)".log
>
> I wasn't sure how to represent it in email, and decided to post a diff
> of "before `swapoff -a`" and "after …", to be viewed with `diffr` or
> with `perl /path/to/diff-highlight` of git or similar.
>
> Diff follows:
>
> --- "Wed, 30 Oct 2024 17:27:38 +0300.log"       2024-10-30 17:27:38.401290017 +0300
> +++ "Wed, 30 Oct 2024 17:28:12 +0300.log"       2024-10-30 17:28:12.397695798 +0300
> @@ -6,8 +6,8 @@
>               man:sddm.conf(5)
>     Main PID: 710 (sddm)
>        Tasks: 9 (limit: 18621)
> -     Memory: 1.2G (peak: 2.8G swap: 1.7G swap peak: 3.2G zswap: 58.3M)
> -        CPU: 6h 10min 7.847s
> +     Memory: 2.8G (peak: 2.8G swap: 0B swap peak: 3.2G)
> +        CPU: 6h 10min 14.748s
>       CGroup: /system.slice/sddm.service
>               ├─710 /usr/bin/sddm
>               └─746 /usr/lib/Xorg -nolisten tcp -background none -seat seat0 vt2 -auth /run/sddm/xauth_GXHGRA -noreset -displayfd 20
> @@ -22,36 +22,36 @@
>  окт 28 09:22:36 dell-g15 sddm-helper[925]: Writing cookie to "/tmp/xauth_RKKlcB"
>  окт 28 09:22:36 dell-g15 sddm-helper[925]: Starting X11 session: "" "/usr/share/sddm/scripts/Xsession \"env KDEWM=/usr/bin/i3 /usr/bin/startplasma-x11\""
>  окт 28 09:22:36 dell-g15 sddm[710]: Session started true
> -anon 42807296
> -file 1150423040
> -kernel 79376384
> +anon 93822976

Anonymous memory increased, but not by too much.

> +file 2957750272
> +kernel 18210816
>  kernel_stack 147456
>  pagetables 7204864
>  sec_pagetables 0
>  percpu 2184
>  sock 0
>  vmalloc 12288
> -shmem 1150795776
> -zswap 61173438
> -zswapped 1751408640
> +shmem 2958123008

shmem increased by a lot (~1.8G).

So this looks like it could be the answer to your question about where
the swap usage is coming from. I would try to find what tmpfs files
are used by this application.

> +zswap 0
> +zswapped 0
>  file_mapped 4108288
>  file_dirty 0
>  file_writeback 0
> -swapcached 17666048
> +swapcached 0
>  anon_thp 2097152
>  file_thp 0
>  shmem_thp 0
> -inactive_anon 445014016
> -active_anon 625201152
> +inactive_anon 589209600
> +active_anon 2321489920
>  inactive_file 2895872
>  active_file 2244608
> -unevictable 140836864
> -slab_reclaimable 8166656
> -slab_unreclaimable 2618128
> -slab 10784784
> -workingset_refault_anon 69854
> +unevictable 141144064
> +slab_reclaimable 8169032
> +slab_unreclaimable 2625208
> +slab 10794240
> +workingset_refault_anon 177253
>  workingset_refault_file 12496
> -workingset_activate_anon 33476
> +workingset_activate_anon 41579
>  workingset_activate_file 2372
>  workingset_restore_anon 12558
>  workingset_restore_file 2132
> @@ -64,14 +64,14 @@
>  pgsteal_kswapd 1243374
>  pgsteal_direct 348876
>  pgsteal_khugepaged 9149
> -pgfault 626853
> +pgfault 626941
>  pgmajfault 11521
>  pgrefill 560417
>  pgactivate 85087
>  pgdeactivate 0
>  pglazyfree 0
>  pglazyfreed 0
> -zswpin 87568
> +zswpin 515158
>  zswpout 1395410
>  zswpwb 211559
>  thp_fault_alloc 8