From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D56DC4167B for ; Mon, 27 Nov 2023 21:01:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9CB4F6B02AA; Mon, 27 Nov 2023 16:01:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 979A76B02AE; Mon, 27 Nov 2023 16:01:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 841616B02C9; Mon, 27 Nov 2023 16:01:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 73BF56B02AA for ; Mon, 27 Nov 2023 16:01:05 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 4832DB5BC1 for ; Mon, 27 Nov 2023 21:01:05 +0000 (UTC) X-FDA: 81504954090.29.F38D8AB Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf04.hostedemail.com (Postfix) with ESMTP id 7E9AC40018 for ; Mon, 27 Nov 2023 21:01:01 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=yzDbR+kT; dmarc=none; spf=pass (imf04.hostedemail.com: domain of akpm@linux-foundation.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701118862; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LY6AvkeA6OKl7ZBG8XXTWi8GeuV0aqlfHr7oz74WW2o=; b=ZVPxAUCZHLuWqNIowRJhd7y/f2nxxfoh1yxGkJGC0U6vJMefFPXNLZIpC2Xx2nFMdnZIz0 7sIZVzq9b4XQSPi1ads5d6CXhOQ0p7uK0aqL0hpzk1v/Ft+ah1rCvc520/WLL19gMaYtpx QrSvTOpxw0xCxZYwq7Hv8IF8r0AIdqs= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=yzDbR+kT; dmarc=none; spf=pass (imf04.hostedemail.com: domain of akpm@linux-foundation.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701118862; a=rsa-sha256; cv=none; b=g1vFKXYO67+3ydrH9i/caPKdZEVXS5ytS+/oQPgo4h2KQDrJmeDSA63WQMZWDQC1aK3tp4 44ZELH/YBX+RUCyYpOYF1KKLLy20irlBNfeHEwrxMTJGZRfSpH9on9YcDD4H2YjQzmB2N5 iQRHWU7jYfSYESO+HO0HvEe8gowCBnM= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 08CDECE1250; Mon, 27 Nov 2023 21:00:58 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id A1E8DC433C9; Mon, 27 Nov 2023 21:00:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1701118857; bh=d0FagZci1PXFMezi/BonwyYv+7yV34Dht6vxRLQOKK8=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=yzDbR+kT8L1Ms6G8xL0TrV5ITh7u9pHV2Tf3NB55xhtilyqokg+gTeQDHkcIheVm8 DFA9Di1naDUQ7+V1qm6//jzz5SoG4oQI202ZHvMMcRWO/D2v+XJx6r6wBn7BU8ic6i nNpDcKvEsu+FmI1qaMNFFOqckiGwHehyfX1VN2tA= Date: Mon, 27 Nov 2023 13:00:55 -0800 From: Andrew Morton To: Nhat Pham Cc: hannes@cmpxchg.org, cerasuolodomenico@gmail.com, yosryahmed@google.com, sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, chrisl@kernel.org, linux-mm@kvack.org, kernel-team@meta.com, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, shuah@kernel.org Subject: Re: [PATCH v6 6/6] zswap: shrinks zswap pool based on memory pressure Message-Id: <20231127130055.30c455906d912e09dcb7e79b@linux-foundation.org> In-Reply-To: <20231127193703.1980089-7-nphamcs@gmail.com> References: <20231127193703.1980089-1-nphamcs@gmail.com> <20231127193703.1980089-7-nphamcs@gmail.com> X-Mailer: Sylpheed 3.8.0beta1 (GTK+ 2.24.33; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 7E9AC40018 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: raoofk1x6tssriwnr669n8qna4bcumrn X-HE-Tag: 1701118861-179881 X-HE-Meta: U2FsdGVkX18EytLqvZOYzEd/Zm/ippni5GHbBuw6jPyhEwER1lfM6VUvj0xHsyC3r8hR950FW5Rc6C7stqs25CZbnxw8qKvJmNgIr0i8Ef72UJ+c36kIjc3grWJo77HHIasq1oDSsTfKvkw/EWhTRMsl25MbVwpbidlrp3J75LlWNXvDBfCys9GRfa9HQm8Do50TFscK3KB27a0RTDVPDdUXVF2n+mqpD6jLDjk3V074hZin7hMGRP7LQ7BFwxQAnDpnSCCUpl+NGFmTBFfuXtd8CoCCMkv+8yyUUhwp34F4ZTOZCgJQHzEqlapYTM8tZ6LfNmeTq3hlF4c07HAO6/RoaVjKqL88pgJ484qM4ysdvcuudK7xqCVipx2puX0ixOO2dVhC0Ar9EfaE2cjfSNLCVTKNaAgagWZEQZArlD81dLIB6/SyOzqaua5oliATiS4Iq2LEfirfn6nfzw/7UXITgK/WZJFR9IWCezV8ZkoXN4QzXYEe27fhOUxNi3/HCHI1XuYX+lJ1u+gc3aKsuOmRjXlqjhaYClytzE4xUyx3HIRhkBe43hwRlaYlvCVeCdvY9uU/oML94hBR4IkaqvQGwS3n8UUsHYPiProMGMRosJ6O3RO/bfmzRSp/i7L3uoeKIXDC+RVopjX0nS2LLs2H0fQSGZjquyANwScBjIdZYNqJTJrqGvBJKPgTNbNykhRUzYL/a4NjeoKjhalNmtw6QnLKncO70xM/LgbDHx6QSrayAdI8wbviHo6vCVgp4O1ZBxwxmHiG18AcESlhTJ/N17uD3YFLsB4BDJfOlxSfuyzZd/M7X6paurrYwOdi5pfMKM55qiZyhxSR9OoexzcHC3EEgguEeBlFu8eoQxZ3ksPfE2IgDRhpTuKJP+v4mHo93QmmtJ5oXJM0WGvmZnTwj9NshbAhpohabmVc4bLd3rU7468Qap8hO5U4gxmY3pS3wDDpsIl22DE2Nq1 eqCKuXmI ruIyViDI7fJl+533kL2mFkR4yXvX/V327MqSG/3ViY183rhduqUslh6k5LH7+BXYdSoBuQYYfsmZLKRjPvASpkAHQxCB5OSn5+e15jglEUv7PWoqIJEI3yfH1Z5G/uQDWt2P/UUjOKEST2uBTXyVX0djvFuO6PT2Ja6pyNa1nEOn3/zpJYswj1Qzb4Qf6fDGK56X5DK0kdoCVn+qZ8xVOzl/xTUncqFucCXjU19mPgxfdcgA+rZ3nnxfyQEGWWJfXERJ0QLqmvKMOQvkMz/OZpPA96jC+yFFuPtGm X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, 27 Nov 2023 11:37:03 -0800 Nhat Pham wrote: > Currently, we only shrink the zswap pool when the user-defined limit is > hit. This means that if we set the limit too high, cold data that are > unlikely to be used again will reside in the pool, wasting precious > memory. It is hard to predict how much zswap space will be needed ahead > of time, as this depends on the workload (specifically, on factors such > as memory access patterns and compressibility of the memory pages). > > This patch implements a memcg- and NUMA-aware shrinker for zswap, that > is initiated when there is memory pressure. The shrinker does not > have any parameter that must be tuned by the user, and can be opted in > or out on a per-memcg basis. > > Furthermore, to make it more robust for many workloads and prevent > overshrinking (i.e evicting warm pages that might be refaulted into > memory), we build in the following heuristics: > > * Estimate the number of warm pages residing in zswap, and attempt to > protect this region of the zswap LRU. > * Scale the number of freeable objects by an estimate of the memory > saving factor. The better zswap compresses the data, the fewer pages > we will evict to swap (as we will otherwise incur IO for relatively > small memory saving). > * During reclaim, if the shrinker encounters a page that is also being > brought into memory, the shrinker will cautiously terminate its > shrinking action, as this is a sign that it is touching the warmer > region of the zswap LRU. > > As a proof of concept, we ran the following synthetic benchmark: > build the linux kernel in a memory-limited cgroup, and allocate some > cold data in tmpfs to see if the shrinker could write them out and > improved the overall performance. Depending on the amount of cold data > generated, we observe from 14% to 35% reduction in kernel CPU time used > in the kernel builds. > > ... > > --- a/include/linux/mmzone.h > +++ b/include/linux/mmzone.h > @@ -22,6 +22,7 @@ > #include > #include > #include > +#include > #include > > /* Free memory management - zoned buddy allocator. */ > @@ -641,6 +642,7 @@ struct lruvec { > #ifdef CONFIG_MEMCG > struct pglist_data *pgdat; > #endif > + struct zswap_lruvec_state zswap_lruvec_state; Normally we'd put this in #ifdef CONFIG_ZSWAP. > --- a/include/linux/zswap.h > +++ b/include/linux/zswap.h > @@ -5,20 +5,40 @@ > #include > #include > > +struct lruvec; > + > extern u64 zswap_pool_total_size; > extern atomic_t zswap_stored_pages; > > #ifdef CONFIG_ZSWAP > > +struct zswap_lruvec_state { > + /* > + * Number of pages in zswap that should be protected from the shrinker. > + * This number is an estimate of the following counts: > + * > + * a) Recent page faults. > + * b) Recent insertion to the zswap LRU. This includes new zswap stores, > + * as well as recent zswap LRU rotations. > + * > + * These pages are likely to be warm, and might incur IO if the are written > + * to swap. > + */ > + atomic_long_t nr_zswap_protected; > +}; > + > bool zswap_store(struct folio *folio); > bool zswap_load(struct folio *folio); > void zswap_invalidate(int type, pgoff_t offset); > void zswap_swapon(int type); > void zswap_swapoff(int type); > void zswap_memcg_offline_cleanup(struct mem_cgroup *memcg); > - > +void zswap_lruvec_state_init(struct lruvec *lruvec); > +void zswap_lruvec_swapin(struct page *page); > #else > > +struct zswap_lruvec_state {}; But instead you made it an empty struct in this case. That's a bit funky, but I guess OK. It does send a careful reader of struct lruvec over to look at the zswap_lruvec_state definition to understand what's going on. > static inline bool zswap_store(struct folio *folio) > { > return false; > @@ -33,7 +53,8 @@ static inline void zswap_invalidate(int type, pgoff_t offset) {} > static inline void zswap_swapon(int type) {} > static inline void zswap_swapoff(int type) {} > static inline void zswap_memcg_offline_cleanup(struct mem_cgroup *memcg) {} > - > +static inline void zswap_lruvec_init(struct lruvec *lruvec) {} > +static inline void zswap_lruvec_swapin(struct page *page) {} Needed this build fix: --- a/include/linux/zswap.h~zswap-shrinks-zswap-pool-based-on-memory-pressure-fix +++ a/include/linux/zswap.h @@ -54,6 +54,7 @@ static inline void zswap_swapon(int type static inline void zswap_swapoff(int type) {} static inline void zswap_memcg_offline_cleanup(struct mem_cgroup *memcg) {} static inline void zswap_lruvec_init(struct lruvec *lruvec) {} +static inline void zswap_lruvec_state_init(struct lruvec *lruvec) {} static inline void zswap_lruvec_swapin(struct page *page) {} #endif _