From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.3 required=3.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47F2AC2D0DB for ; Thu, 30 Jan 2020 20:00:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id EDD6820708 for ; Thu, 30 Jan 2020 19:59:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="jVU4wHQp" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EDD6820708 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 760E66B037F; Thu, 30 Jan 2020 14:59:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 711806B0380; Thu, 30 Jan 2020 14:59:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 626A96B0381; Thu, 30 Jan 2020 14:59:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0044.hostedemail.com [216.40.44.44]) by kanga.kvack.org (Postfix) with ESMTP id 4DDA46B037F for ; Thu, 30 Jan 2020 14:59:59 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 09F17181AEF07 for ; Thu, 30 Jan 2020 19:59:59 +0000 (UTC) X-FDA: 76435366518.22.screw65_3a013be9e9039 X-HE-Tag: screw65_3a013be9e9039 X-Filterd-Recvd-Size: 10165 Received: from mail-vs1-f51.google.com (mail-vs1-f51.google.com [209.85.217.51]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Thu, 30 Jan 2020 19:59:58 +0000 (UTC) Received: by mail-vs1-f51.google.com with SMTP id t12so2856491vso.13 for ; Thu, 30 Jan 2020 11:59:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=2eQCO1Ierwus9UuWwTZvBWAfmWkp7gyT6B9P6N5VYTA=; b=jVU4wHQpvNcASRS0EmkC6VWvHs6TPQqGiBDDkTwmF3xJVikkI1SSSdBlXhzM1b0/mG ykDA393vKGtJ3jwbYCy11RmDtun+QGjoQqkkr/hyI/ogCj94HEN/SIlQlH2LpSF3zbU9 Hoq7j6M3PJe8PGkcUUEr16HYwmeGxkUpyB3haLapYvdb2moeepXZLZJ7RbtWWas7m+l8 5fu8qkgHDPvMEsDehkS2D0n15H6LRwcvKDHaBBmLxI3oLhp3KPpcpjeTOn0LzGisvFqh oma8xZMeoLmFKPzRaDmPTXO4GY+8L+cV15n+TFFv+MCSElI7xZ7AhuGvCgI/IMA0JQGr wdCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=2eQCO1Ierwus9UuWwTZvBWAfmWkp7gyT6B9P6N5VYTA=; b=fDkOjvdlJ0RvEImMBjyhIR+8s610thf3FqlTMETSkrz2gf8+GB03I4FtfAxlsqMuma 8g6/2DXpeQ/niCwRS05fQFo5WqWvx4MtImE6Y4EhYq6P4tliHoHWLXHzN+T8XCNx8EiS UgrPVshlBSO0d3nlbQLF7QPX62HwCtmTgNuk29I6HjGAv53uX57X87DkTB5IRrNzhK83 AE5WBa6gfI111lBifo1liwFjQYdXgMT5bmjqAAvwg+O63c0BgRi7sNzMMSR5xAwD+5t7 YlPTijY+SMSzT9IvMRXn+nWRgTpsjf5kUOV0NJQEwY3dzjxeRbYKLDxT/Ln8ZDQX99Ir fsTA== X-Gm-Message-State: APjAAAWyo9EjXzbAyGJTOlsB8oBzBImU0TSZT6wLV9JDwA0C9/PzWD9P Q0mnegTiDxTvaEs1Vg8m9kS+Yeejvm0G8rUq7outPA== X-Google-Smtp-Source: APXvYqwHHkpXItZTvi6VZUTeId6Wyiby1+qRXx3YMDo6RV+1fgrQepnwXNMd3jYXmtjL16xzQJW7SnCzS9T51EGZANI= X-Received: by 2002:a05:6102:2159:: with SMTP id h25mr4512414vsg.160.1580414397772; Thu, 30 Jan 2020 11:59:57 -0800 (PST) MIME-Version: 1.0 References: <91270a68-ff48-88b0-219c-69801f0c252f@redhat.com> <75d4594f-0864-5172-a0f8-f97affedb366@redhat.com> <286AC319A985734F985F78AFA26841F73E3F8A02@shsmsx102.ccr.corp.intel.com> In-Reply-To: <286AC319A985734F985F78AFA26841F73E3F8A02@shsmsx102.ccr.corp.intel.com> From: Tyler Sanderson Date: Thu, 30 Jan 2020 11:59:46 -0800 Message-ID: Subject: Re: Balloon pressuring page cache To: "Wang, Wei W" Cc: David Hildenbrand , "virtualization@lists.linux-foundation.org" , David Rientjes , "linux-mm@kvack.org" , Michal Hocko , "Michael S. Tsirkin" Content-Type: multipart/alternative; boundary="000000000000c75619059d60e67e" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: --000000000000c75619059d60e67e Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Jan 30, 2020 at 7:31 AM Wang, Wei W wrote: > On Thursday, January 30, 2020 11:03 PM, David Hildenbrand wrote: > > On 29.01.20 20:11, Tyler Sanderson wrote: > > > > > > > > > On Wed, Jan 29, 2020 at 2:31 AM David Hildenbrand > > > wrote: > > > > > > On 29.01.20 01:22, Tyler Sanderson via Virtualization wrote: > > > > A primary advantage of virtio balloon over other memory reclaim > > > > mechanisms is that it can pressure the guest's page cache into > > > shrinking. > > > > > > > > However, since the balloon driver changed to using the shrinker > API > > > > > > > > > > e99a28e355255a#diff-fd202acf694d9eba19c8c64da3e480c9> this > > > > use case has become a bit more tricky. I'm wondering what the > > intended > > > > device implementation is. > > > > > > > > When inflating the balloon against page cache (i.e. no free > memory > > > > remains) vmscan.c will both shrink page cache, but also invoke > the > > > > shrinkers -- including the balloon's shrinker. So the balloon > driver > > > > allocates memory which requires reclaim, vmscan gets this memor= y > > by > > > > shrinking the balloon, and then the driver adds the memory back > to > > the > > > > balloon. Basically a busy no-op. > > Per my understanding, the balloon allocation won=E2=80=99t invoke shrinke= r as > __GFP_DIRECT_RECLAIM isn't set, no? > I could be wrong about the mechanism, but the device sees lots of activity on the deflate queue. The balloon is being shrunk. And this only starts once all free memory is depleted and we're inflating into page cache. > > > > > > > > > > If file IO is ongoing during this balloon inflation then the pa= ge > > > cache > > > > could be growing which further puts "back pressure" on the > balloon > > > > trying to inflate. In testing I've seen periods of > 45 seconds > where > > > > balloon inflation makes no net forward progress. > > I think this is intentional (but could be improved). As inflation does no= t > stop when the allocation fails (it simply sleeps for a while and resumes.= . > repeat till there are memory to inflate) > That's why you see no inflation progress for long time under memory > pressure. > As noted above the deflate queue is active, so it's not just memory allocation failures. > > > Best, > Wei > --000000000000c75619059d60e67e Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Thu, Jan 30, 2020 at 7:31 AM Wang,= Wei W <wei.w.wang@intel.com= > wrote:
On T= hursday, January 30, 2020 11:03 PM, David Hildenbrand wrote:
> On 29.01.20 20:11, Tyler Sanderson wrote:
> >
> >
> > On Wed, Jan 29, 2020 at 2:31 AM David Hildenbrand <david@redhat.com
> > <mailto:= david@redhat.com>> wrote:
> >
> >=C2=A0 =C2=A0 =C2=A0On 29.01.20 01:22, Tyler Sanderson via Virtual= ization wrote:
> >=C2=A0 =C2=A0 =C2=A0> A primary advantage of virtio balloon ove= r other=C2=A0memory reclaim
> >=C2=A0 =C2=A0 =C2=A0> mechanisms is that it can=C2=A0pressure t= he guest's page cache into
> >=C2=A0 =C2=A0 =C2=A0shrinking.
> >=C2=A0 =C2=A0 =C2=A0>
> >=C2=A0 =C2=A0 =C2=A0> However, since the balloon driver changed= to using the shrinker API
> >=C2=A0 =C2=A0 =C2=A0>
> >
> <https://github.com/torva= lds/linux/commit/71994620bb25a8b109388fefa9
> e99a28e355255a#diff-fd202acf694d9eba19c8c64da3e480c9>=C2=A0this
> >=C2=A0 =C2=A0 =C2=A0> use case has become a bit more tricky. I&= #39;m wondering what the
> intended
> >=C2=A0 =C2=A0 =C2=A0> device implementation is.
> >=C2=A0 =C2=A0 =C2=A0>
> >=C2=A0 =C2=A0 =C2=A0> When inflating the balloon against page c= ache (i.e. no free memory
> >=C2=A0 =C2=A0 =C2=A0> remains) vmscan.c will both shrink page c= ache, but also invoke the
> >=C2=A0 =C2=A0 =C2=A0> shrinkers -- including the balloon's = shrinker. So the balloon driver
> >=C2=A0 =C2=A0 =C2=A0> allocates memory which requires reclaim, = vmscan gets this memory
> by
> >=C2=A0 =C2=A0 =C2=A0> shrinking the balloon, and then the drive= r adds the memory back to
> the
> >=C2=A0 =C2=A0 =C2=A0> balloon. Basically a busy no-op.

Per my understanding, the balloon allocation won=E2=80=99t invoke shrinker = as __GFP_DIRECT_RECLAIM isn't set, no?
I could be = wrong about the mechanism, but the device sees lots of activity on the defl= ate queue. The balloon is being shrunk. And this only starts once all free = memory is depleted and we're inflating into page cache.


> >=C2=A0 =C2=A0 =C2=A0>
> >=C2=A0 =C2=A0 =C2=A0> If file IO is ongoing during this balloon= inflation then the page
> >=C2=A0 =C2=A0 =C2=A0cache
> >=C2=A0 =C2=A0 =C2=A0> could be growing which further puts "= ;back pressure" on the balloon
> >=C2=A0 =C2=A0 =C2=A0> trying to inflate. In testing I've se= en periods of > 45 seconds where
> >=C2=A0 =C2=A0 =C2=A0> balloon inflation makes no net forward pr= ogress.

I think this is intentional (but could be improved). As inflation does not = stop when the allocation fails (it simply sleeps for a while and resumes.. = repeat till there are memory to inflate)
That's why you see no inflation progress for long time under memory pre= ssure.
As noted above the deflate queue is active, so = it's not just memory allocation failures.
=C2=A0


Best,
Wei
--000000000000c75619059d60e67e--