From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25F59C4727F for ; Mon, 5 Oct 2020 13:51:37 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id ADDE520774 for ; Mon, 5 Oct 2020 13:51:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ADDE520774 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=canonical.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C58236B005C; Mon, 5 Oct 2020 09:51:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C0AB86B005D; Mon, 5 Oct 2020 09:51:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AD1F98E0001; Mon, 5 Oct 2020 09:51:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0137.hostedemail.com [216.40.44.137]) by kanga.kvack.org (Postfix) with ESMTP id 7E94B6B005C for ; Mon, 5 Oct 2020 09:51:35 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 10B051EE6 for ; Mon, 5 Oct 2020 13:51:35 +0000 (UTC) X-FDA: 77338009350.21.comb20_0b09cf6271be Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin21.hostedemail.com (Postfix) with ESMTP id DC17B180442C0 for ; Mon, 5 Oct 2020 13:51:34 +0000 (UTC) X-HE-Tag: comb20_0b09cf6271be X-Filterd-Recvd-Size: 5449 Received: from youngberry.canonical.com (youngberry.canonical.com [91.189.89.112]) by imf29.hostedemail.com (Postfix) with ESMTP for ; Mon, 5 Oct 2020 13:51:34 +0000 (UTC) Received: from mail-wr1-f69.google.com ([209.85.221.69]) by youngberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1kPQto-0000xL-QL for linux-mm@kvack.org; Mon, 05 Oct 2020 13:51:32 +0000 Received: by mail-wr1-f69.google.com with SMTP id w7so4057730wrp.2 for ; Mon, 05 Oct 2020 06:51:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=bQQp2PNEq0FY7Hqco5htUGBXM0/SRgDQIUWJr9BdqaM=; b=RcbPJ4goqPY/3GnR0KPHazMr4sOd2kmaMv6LOA0TGsmSqlKAPaqm84ggUj7cG9F1eU py/7JkQHwlHWCrZ8bMt3rR90ZO3rbUPlNKlnAY2K8WRWfDCX8jxgCz02Qjy9ORDi2o4L 260fNHuB9pjOf0zDYIaRjV97RRcZ7RqFwP+LnHxmftJMmPc4ao0vmQWYHlBg1yLogBgE SOaaY07G+2zCGacpdDAuLYYsDUk7OtYwGYLS29O/bBihvN6NuXFagHTdGX05iFic/Qgl n03EFnXtvzCWC6t6xK4rlMsk/usbsJgupQkXyf2mq2BAQVKSrF1C0VpykGuNryhot/Gg fHLA== X-Gm-Message-State: AOAM530cRJAJvlvFx4kaRpr7rySePqONCyojWOGFyjsAZfjWI3djjWme zMcDJ5JOc3+YUVTQV7Z1knyjdK46t4XEWilabAAEnbyzhdyrjz7DPxmdavHid/RXif69BYMhRVx 7pflwmw9KoEBMVvTX19qANoef7CNl X-Received: by 2002:a1c:28a:: with SMTP id 132mr17121829wmc.9.1601905892395; Mon, 05 Oct 2020 06:51:32 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwofc6USifXN7h+C0hcGmEQa4n+RfAxxI8ZWg7rnK634gMSN9Wa915TnRbfzQlf/I65n7VhQw== X-Received: by 2002:a1c:28a:: with SMTP id 132mr17121788wmc.9.1601905892065; Mon, 05 Oct 2020 06:51:32 -0700 (PDT) Received: from localhost (host-79-36-133-218.retail.telecomitalia.it. [79.36.133.218]) by smtp.gmail.com with ESMTPSA id t202sm13779805wmt.14.2020.10.05.06.51.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 05 Oct 2020 06:51:31 -0700 (PDT) Date: Mon, 5 Oct 2020 15:51:30 +0200 From: Andrea Righi To: Chris Down Cc: Michal Hocko , Vladimir Davydov , Li Zefan , Tejun Heo , Johannes Weiner , Andrew Morton , Luigi Semenzato , "Rafael J . Wysocki" , cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org Subject: Re: [PATCH RFC v2] Opportunistic memory reclaim Message-ID: <20201005135130.GA850459@xps-13-7390> References: <20201005081313.732745-1-andrea.righi@canonical.com> <20201005112555.GA108347@chrisdown.name> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20201005112555.GA108347@chrisdown.name> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Oct 05, 2020 at 12:25:55PM +0100, Chris Down wrote: > Andrea Righi writes: > > This feature has been successfully used to improve hibernation time of > > cloud computing instances. > > > > Certain cloud providers allow to run "spot instances": low-priority > > instances that run when there are spare resources available and can be > > stopped at any time to prioritize other more privileged instances [2]. > > > > Hibernation can be used to stop these low-priority instances nicely, > > rather than losing state when the instance is shut down. Being able to > > quickly stop low-priority instances can be critical to provide a better > > quality of service in the overall cloud infrastructure [1]. > > > > The main bottleneck of hibernation is represented by the I/O generated > > to write all the main memory (hibernation image) to a persistent > > storage. > > > > Opportunistic memory reclaimed can be used to reduce the size of the > > hibernation image in advance, for example if the system is idle for a > > certain amount of time, so if an hibernation request happens, the kernel > > has already saved most of the memory to the swap device (caches have > > been dropped, etc.) and hibernation can complete quickly. > > Hmm, why does this need to be implemented in kernelspace? We already have > userspace shrinkers using memory pressure information as part of PID control > already (eg. senpai). Using memory.high and pressure information looks a lot > easier to reason about than having to choose an absolute number ahead of > time and hoping it works. senpai is focused at estimating the ideal memory requirements without affecting performance. And this covers the use case about reducing memory footprint. In my specific use-case (hibernation) I would let the system use as much memory as possible if it's doing any activity (reclaiming memory only when the kernel decides that it needs to reclaim memory) and apply a more aggressive memory reclaiming policy when the system is mostly idle. I could probably implement this behavior adjusting memory.high dynamically, like senpai, but I'm worried about potential sudden large allocations that may require to respond faster at increasing memory.high. I think the user-space triggered memory reclaim approach is a safer solution from this perspective. -Andrea