/proc/PID/status shows VmSwap > 0 after swapoff

[Florian Schmidt <florian.schmidt@nutanix.com>]

Hey,

TL;DR: is it expected behavior that the "VmSwap" output in 
/proc/PID/status can be > 0, even without swap, and if so, why?


While experimenting with swap, I've run into a behavior that I can't 
quite explain. My setup (tested on 5.2.4): I reboot a machine with a 
swap partition; then I produce memory pressure by allocating lots of 
memory and writing to each page of allocated memory. So far, so good: 
memory is being swapped out, and that information is reflected in 
several respective files in /proc:

$ cat /proc/swaps
Filename                     Type            Size    Used    Priority
/dev/dm-1                    partition       4194300 5644    -2

$ grep Swap /proc/meminfo
SwapCached:          600 kB
SwapTotal:       4194300 kB
SwapFree:        4188656 kB

$ grep VmSwap /proc/[0-9]*/status
/proc/1/status:VmSwap:       464 kB
/proc/257/status:VmSwap:               0 kB
/proc/272/status:VmSwap:               0 kB
/proc/498/status:VmSwap:             176 kB
/proc/499/status:VmSwap:             292 kB
/proc/528/status:VmSwap:              96 kB
/proc/531/status:VmSwap:              56 kB
/proc/535/status:VmSwap:              24 kB
/proc/537/status:VmSwap:              20 kB
/proc/541/status:VmSwap:               8 kB
/proc/542/status:VmSwap:               4 kB
/proc/548/status:VmSwap:             264 kB
/proc/561/status:VmSwap:              76 kB
/proc/564/status:VmSwap:               0 kB
/proc/565/status:VmSwap:             428 kB
/proc/578/status:VmSwap:             156 kB
/proc/579/status:VmSwap:            1272 kB
/proc/587/status:VmSwap:             196 kB
/proc/588/status:VmSwap:             452 kB
/proc/589/status:VmSwap:             908 kB
/proc/599/status:VmSwap:               0 kB
/proc/606/status:VmSwap:               0 kB


Now I disable swap (swapoff -a), which, as far as I understand, should 
force all swapped out pages to be swapped back in. The swap is indeed gone:

$ grep Swap /proc/meminfo
SwapCached:            0 kB
SwapTotal:             0 kB
SwapFree:              0 kB

However, some processes claim to still have swapped out memory:

$ grep VmSwap /proc/[0-9]*/status
/proc/1/status:VmSwap:         0 kB
/proc/257/status:VmSwap:               0 kB
/proc/272/status:VmSwap:               0 kB
/proc/498/status:VmSwap:               0 kB
/proc/499/status:VmSwap:               0 kB
/proc/528/status:VmSwap:              12 kB
/proc/531/status:VmSwap:               8 kB
/proc/535/status:VmSwap:              16 kB
/proc/537/status:VmSwap:               0 kB
/proc/541/status:VmSwap:               0 kB
/proc/542/status:VmSwap:               0 kB
/proc/548/status:VmSwap:               0 kB
/proc/561/status:VmSwap:               0 kB
/proc/564/status:VmSwap:               0 kB
/proc/565/status:VmSwap:               0 kB
/proc/578/status:VmSwap:               0 kB
/proc/579/status:VmSwap:               0 kB
/proc/587/status:VmSwap:               0 kB
/proc/588/status:VmSwap:               0 kB
/proc/589/status:VmSwap:               0 kB
/proc/599/status:VmSwap:               0 kB
/proc/606/status:VmSwap:               0 kB

How can this be? I understand that VmSwap doesn't account for shmem, so 
the sum of all VmSwaps might be *smaller* than (SwapTotal - SwapFree), 
but the other way round? A quick look into the source code tells me 
VmSwap is read from each process's MM_SWAPENTS, and while that one is 
only updated in a few places, this leads far enough into the deep end of 
the mm subsystem that I'm not sure I can immediately understand all 
situations in which that value is updated.

So before I invest the time to dive into this code, I thought I'd ask 
here: is that behavior (VmSwap output in proc can be > 0 even without 
swap) expected, and if so, why?

Cheers,
Florian