Re: Caching/buffers become useless after some time

[Vlastimil Babka <vbabka@suse.cz>]

Forgot to answer this:

On 10/31/18 3:53 PM, Marinko Catovic wrote:
> Well caching of any operations with find/du is not necessary imho
> anyway, since walking over all these millions of files in that time
> period is really not worth caching at all - if there is a way you
> mentioned to limit the commands there, that would be great.
> Also I want to mention that these operations were in use with 3.x
> kernels as well, for years, with absolutely zero issues.

Yep, something had to change at some point. Possibly the
reclaim/compaction loop. Probably not the way dentries/inodes are being
cached though.

> 2 > drop_caches right after that is something I considered, I just had
> some bad experience with this, since I tried it around 5:00 AM in the
> first place to give it enough spare time to finish, since sync; echo 2
>> drop_caches can take some time, hence my question about lowering the
> limits in mm/vmscan.c, void drop_slab_node(int nid)
> 
> I could do this effectively right after find/du at 07:45, just hoping
> that this is finished soon enough - in one worst case it took over 2
> hours (from 05:00 AM to 07:00 AM), since the host was busy during that
> time with find/du, never having freed enough caches to continue, hence

Dropping caches while find/du is still running would be
counter-productive. If done after it's already finished, it shouldn't be
so disruptive.

> my question to let it stop earlier with the modification of
> drop_slab_node ... it was just an idea, nevermind if you believe that
> it was a bad one :)

Finding a universally "correct" threshold could easily be impossible. I
guess the proper solution would be to drop the while loop and
restructure the shrinking so that it would do a single pass through all
objects.