Re: Question about the laziness of MADV_FREE

["Niklas Hambüchen" <mail@nh2.me>]

Hello Michal,

thanks for the swift reply and patch!

> We batch multiple pages to become really lazyfree. This means that those
> pages are sitting on a per-cpu list (see mark_page_lazyfree). So the
> the number drift depends on the number of CPUs.

Is there an upper bound that I can rely on in order to judge how far off the accounting is (perhaps depending on the number of CPUs as you say)?
For example, if the drift is bounded to, say 10%, that would probably be fine, while if it could be off by 2x or so, that would make system inspection tough.

>> For my investigation it would be very useful if I could get accurate accounting.
>> How much work would the "If this is not desirable please file a bug report" bit entail?
> 
> What would be the reason to get the exact number?

Mainly to debug situations where programs run out of memory.
Quite similar to the third point on https://lore.kernel.org/patchwork/cover/755741/.

In such situations, the first thing people usually do is to look at RES and see if things are off.
The fact that RES may still showing memory usage from before can already send one down the wrong investigation path very quickly.
For example, my process takes up to 50 GB when processing some data, and MADV_FREEs it all when it's done and idling.
Due to the lazy freeing, RES will continue to show up as 50 GB even when idle, which may make people suspect a memory leak when there really is none.

In this specific case, one can at least consult proc's LazyFree to figure this out, but if you cannot rely on that number to be accurate either (and the docs not saying how inaccurate it is), it's easy to feel lost about it.

Thanks!