Re: Balloon pressuring page cache

[David Hildenbrand <david@redhat.com>]

On 30.01.20 16:20, Michael S. Tsirkin wrote:
> On Thu, Jan 30, 2020 at 04:02:34PM +0100, David Hildenbrand wrote:
>> On 29.01.20 20:11, Tyler Sanderson wrote:
>>>
>>>
>>> On Wed, Jan 29, 2020 at 2:31 AM David Hildenbrand <david@redhat.com
>>> <mailto:david@redhat.com>> wrote:
>>>
>>>     On 29.01.20 01:22, Tyler Sanderson via Virtualization wrote:
>>>     > A primary advantage of virtio balloon over other memory reclaim
>>>     > mechanisms is that it can pressure the guest's page cache into
>>>     shrinking.
>>>     >
>>>     > However, since the balloon driver changed to using the shrinker API
>>>     >
>>>     <https://github.com/torvalds/linux/commit/71994620bb25a8b109388fefa9e99a28e355255a#diff-fd202acf694d9eba19c8c64da3e480c9> this
>>>     > use case has become a bit more tricky. I'm wondering what the intended
>>>     > device implementation is.
>>>     >
>>>     > When inflating the balloon against page cache (i.e. no free memory
>>>     > remains) vmscan.c will both shrink page cache, but also invoke the
>>>     > shrinkers -- including the balloon's shrinker. So the balloon driver
>>>     > allocates memory which requires reclaim, vmscan gets this memory by
>>>     > shrinking the balloon, and then the driver adds the memory back to the
>>>     > balloon. Basically a busy no-op.
>>>     >
>>>     > If file IO is ongoing during this balloon inflation then the page
>>>     cache
>>>     > could be growing which further puts "back pressure" on the balloon
>>>     > trying to inflate. In testing I've seen periods of > 45 seconds where
>>>     > balloon inflation makes no net forward progress.
>>>     >
>>>     > This wasn't a problem before the change to the shrinker API since
>>>     forced
>>>     > balloon deflation only occurred via the OOM notifier callback
>>>     which was
>>>     > invoked only after the page cache had depleted.
>>>     >
>>>     > Is this new busy behavior working as intended?
>>>
>>>     Please note that the shrinker will only be registered in case we have
>>>     VIRTIO_BALLOON_F_DEFLATE_ON_OOM - (which is AFAIK very rare) - to
>>>     implement automatic balloon deflation when the guest is under memory
>>>     pressure.
>>>
>>>
>>>     Are you actually experiencing issues with that or did you just stumble
>>>     over the code?
>>>
>>>
>>> We have a use case that is encountering this (and that registers
>>> DEFLATE_ON_OOM). We can work around this, but it does seem inefficient.
>>> I understand there were good reasons for moving away from the OOM
>>> notifier callback, but I'm wondering if the balloon driver could specify
>>> a "nice" level to the shrinker API that would cause it to be reclaimed
>>> from only as a last resort?
>>>  
>>
>> Cc-ing linux-mm, Michal and Michael.
> 
> 
> Interesting.  VIRTIO_BALLOON_F_DEFLATE_ON_OOM is really
> underspecified in a bunch of ways.
> 
> I'll wait to see what does Michal say from Linux POV.


Just wondering, does implying that we are using the shrinker that a

echo 3 > /proc/sys/vm/drop_caches

Will deflate the whole balloon? If yes, than that's *really* not desired.

-- 
Thanks,

David / dhildenb