From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.3 required=3.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DFDFAC3B1BF for ; Fri, 14 Feb 2020 20:49:03 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 89D1122314 for ; Fri, 14 Feb 2020 20:49:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="tz8lM3nO" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 89D1122314 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A2E916B0689; Fri, 14 Feb 2020 15:48:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9E2116B068D; Fri, 14 Feb 2020 15:48:55 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 880736B068C; Fri, 14 Feb 2020 15:48:55 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0094.hostedemail.com [216.40.44.94]) by kanga.kvack.org (Postfix) with ESMTP id 64B5D6B0689 for ; Fri, 14 Feb 2020 15:48:55 -0500 (EST) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 147934DB4 for ; Fri, 14 Feb 2020 20:48:55 +0000 (UTC) X-FDA: 76489921830.12.grain82_30b3b81913f26 X-HE-Tag: grain82_30b3b81913f26 X-Filterd-Recvd-Size: 9805 Received: from mail-vk1-f194.google.com (mail-vk1-f194.google.com [209.85.221.194]) by imf49.hostedemail.com (Postfix) with ESMTP for ; Fri, 14 Feb 2020 20:48:54 +0000 (UTC) Received: by mail-vk1-f194.google.com with SMTP id b2so2957275vkk.4 for ; Fri, 14 Feb 2020 12:48:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=cVD4wX404XOWnpP+jP2jRcvAhUXcMGtZ+78P2hMnNVI=; b=tz8lM3nO+h1J22d+uWHUN+dZWZp9fYiEJdHGM5W8BUYDc/F6+tVhUwcGkml9sx2TyC eLCSbRJUZ6JUyspEA4vnlHDT5cTcXvxVfDRG1NcasXs3Wf2GYJgLgu+r3Anhsc3ZJjHU 9dg+JCCjEaMop9m7SAChX2ALwpgOojjKfycH61DdARxbXZyD8s1940zjfAUwBarZJk3s xwkX+lxgyqLxtR1Vh05rln8vauMpBITgAZJiQzax6SgqgpcsDzRnvFTu6eu9u5kSuJ1y 8sByLMvP5sThmn4MjLanEY5e7Qee/y+W1p+AiOzhGeflzs1QHtRYUe+nY8fiWZtHHmhu qYkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=cVD4wX404XOWnpP+jP2jRcvAhUXcMGtZ+78P2hMnNVI=; b=rSRjKpWCijGfU8EYmgOuM6eTOAyxsuR6S+KrYvJ/HdeQ0K7vFXRpckGSE6/qyNpHhm GtfapOMFP3jy8IEC6Cy4JXDnhUTQ02mkDEZ5LBpGuz1OxS0OI4cJ1Bta71EFTGhHEOqW ZnDi+23mwyaeSh6tac+UpeqkC6GKZCCrqHqdJd3HoYWzSJ4jRhbh4c23HKkV8Xlq0she sbPgrUjC91MxOkaOpBkudYEZvGJdpUucrlJR/UXrh46Z6hVdNkrCakeJJbyEkbIfV+A+ Jr8S6+C0zPERE68A+41A72xIZZ4omh6N/MQ85iUUyTm9r3cysrlgQJdl8P67tPACQQB4 yTOA== X-Gm-Message-State: APjAAAX0ccK22bqqECZ/YeFWKPHfi7T3k2Rl82Hsx9NobAP2GrBMdDZM oekaCAHEd7c6iy99oj+QMjjt5RWgMDH9ITDXy54Geg== X-Google-Smtp-Source: APXvYqz3bXhGEDPtdmDlO1dBPFCoEBtqZ+I1h3jNaZe+jf+wvfS4Vaj5sQvhiC0g/78U43tAtpb5c2DIeh/fLfv/9FY= X-Received: by 2002:a1f:a484:: with SMTP id n126mr407782vke.58.1581713333937; Fri, 14 Feb 2020 12:48:53 -0800 (PST) MIME-Version: 1.0 References: <20200205163402.42627-1-david@redhat.com> <20200205163402.42627-4-david@redhat.com> <20200214140641.GB31689@dhcp22.suse.cz> <802f93b1-1588-bd2c-8238-c12ec7f7ae9e@redhat.com> In-Reply-To: <802f93b1-1588-bd2c-8238-c12ec7f7ae9e@redhat.com> From: Tyler Sanderson Date: Fri, 14 Feb 2020 12:48:42 -0800 Message-ID: Subject: Re: [PATCH v1 3/3] virtio-balloon: Switch back to OOM handler for VIRTIO_BALLOON_F_DEFLATE_ON_OOM To: David Hildenbrand Cc: Michal Hocko , linux-kernel@vger.kernel.org, linux-mm@kvack.org, virtualization@lists.linux-foundation.org, "Michael S . Tsirkin" , Wei Wang , Alexander Duyck , David Rientjes , Nadav Amit Content-Type: multipart/alternative; boundary="000000000000685d73059e8f55b9" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: --000000000000685d73059e8f55b9 Content-Type: text/plain; charset="UTF-8" Regarding Wei's patch that modifies the shrinker implementation, versus this patch which reverts to OOM notifier: I am in favor of both patches. But I do want to make sure a fix gets back ported to 4.19 where the performance regression was first introduced. My concern with reverting to the OOM notifier is, as mst@ put it (in the other thread): "when linux hits OOM all kind of error paths are being hit, latent bugs start triggering, latency goes up drastically." The guest could be in a lot of pain before the OOM notifier is invoked, and it seems like the shrinker API might allow more fine grained control of when we deflate. On the other hand, I'm not totally convinced that Wei's patch is an expected use of the shrinker/page-cache APIs, and maybe it is fragile. Needs more testing and scrutiny. It seems to me like the shrinker API is the right API in the long run, perhaps with some fixes and modifications. But maybe reverting to OOM notifier is the best patch to back port? On Fri, Feb 14, 2020 at 6:19 AM David Hildenbrand wrote: > >> There was a report that this results in undesired side effects when > >> inflating the balloon to shrink the page cache. [1] > >> "When inflating the balloon against page cache (i.e. no free memory > >> remains) vmscan.c will both shrink page cache, but also invoke the > >> shrinkers -- including the balloon's shrinker. So the balloon > >> driver allocates memory which requires reclaim, vmscan gets this > >> memory by shrinking the balloon, and then the driver adds the > >> memory back to the balloon. Basically a busy no-op." > >> > >> The name "deflate on OOM" makes it pretty clear when deflation should > >> happen - after other approaches to reclaim memory failed, not while > >> reclaiming. This allows to minimize the footprint of a guest - memory > >> will only be taken out of the balloon when really needed. > >> > >> Especially, a drop_slab() will result in the whole balloon getting > >> deflated - undesired. > > > > Could you explain why some more? drop_caches shouldn't be really used in > > any production workloads and if somebody really wants all the cache to > > be dropped then why is balloon any different? > > > > Deflation should happen when the guest is out of memory, not when > somebody thinks it's time to reclaim some memory. That's what the > feature promised from the beginning: Only give the guest more memory in > case it *really* needs more memory. > > Deflate on oom, not deflate on reclaim/memory pressure. (that's what the > report was all about) > > A priority for shrinkers might be a step into the right direction. > > -- > Thanks, > > David / dhildenb > > --000000000000685d73059e8f55b9 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Regarding Wei's patch that modifies the shrinker = implementation, versus this patch which reverts to OOM notifier:
= I am in favor of both patches. But I do want to make sure a fix gets back p= orted to 4.19 where the performance regression was first introduced.
<= div>My concern with reverting to the OOM notifier is, as mst@ put it (in th= e other thread):
"when linux hit= s OOM=C2=A0all kind of error path= s are being hit, latent bugs start triggering,=C2=A0latency goes up drastically."
= The guest could be in a lot of pain before the OOM notifier is invoked, and= it seems like the shrinker API might allow more fine grained control of wh= en we deflate.

On the other hand, I'm not tota= lly convinced that Wei's patch is an expected use of the shrinker/page-= cache APIs, and maybe it is fragile. Needs more testing=C2=A0and scrutiny.<= /div>

It seems to me like the shrinker API is the right = API in the long run, perhaps with some fixes and modifications. But maybe r= everting to OOM notifier is the best patch to back port?

On Fri, Feb 14, 202= 0 at 6:19 AM David Hildenbrand <david@redhat.com> wrote:
>> There was a report that this results= in undesired side effects when
>> inflating the balloon to shrink the page cache. [1]
>>=C2=A0 =C2=A0 =C2=A0 "When inflating the balloon against page = cache (i.e. no free memory
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0remains) vmscan.c will both shrink page = cache, but also invoke the
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0shrinkers -- including the balloon's= shrinker. So the balloon
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0driver allocates memory which requires r= eclaim, vmscan gets this
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0memory by shrinking the balloon, and the= n the driver adds the
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0memory back to the balloon. Basically a = busy no-op."
>>
>> The name "deflate on OOM" makes it pretty clear when def= lation should
>> happen - after other approaches to reclaim memory failed, not whil= e
>> reclaiming. This allows to minimize the footprint of a guest - mem= ory
>> will only be taken out of the balloon when really needed.
>>
>> Especially, a drop_slab() will result in the whole balloon getting=
>> deflated - undesired.
>
> Could you explain why some more? drop_caches shouldn't be really u= sed in
> any production workloads and if somebody really wants all the cache to=
> be dropped then why is balloon any different?
>

Deflation should happen when the guest is out of memory, not when
somebody thinks it's time to reclaim some memory. That's what the feature promised from the beginning: Only give the guest more memory in
case it *really* needs more memory.

Deflate on oom, not deflate on reclaim/memory pressure. (that's what th= e
report was all about)

A priority for shrinkers might be a step into the right direction.

--
Thanks,

David / dhildenb

--000000000000685d73059e8f55b9--