From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C219CDB474 for ; Tue, 17 Oct 2023 17:44:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 91B0A80053; Tue, 17 Oct 2023 13:44:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8A31380045; Tue, 17 Oct 2023 13:44:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7447780053; Tue, 17 Oct 2023 13:44:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 5E40A80045 for ; Tue, 17 Oct 2023 13:44:10 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 31F9DC0E07 for ; Tue, 17 Oct 2023 17:44:10 +0000 (UTC) X-FDA: 81355677060.01.C6CFEAE Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf02.hostedemail.com (Postfix) with ESMTP id 3D1378000F for ; Tue, 17 Oct 2023 17:44:08 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf02.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1697564648; a=rsa-sha256; cv=none; b=tvl3WY5qISUzWoxh8uslw99ZZTBu7PJx2cXT1esA+yeo0Zvj+FVMcQ9YkoKgzDTUc5U87o lrHGL5NGyLq4NmO1xM2ZLX7Ip+VrYHc+2SmmHcb/2tgdXVWkpFz2DtsPaVhLs4vJ8uO/fW GIVA9HLc5p3Fx+Rf723rCKBIeyyFFE4= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf02.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1697564648; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eDSHMy/oTqLr4d6BTj1alhBA9LqHs15pZRa+uhLIg0A=; b=uVxhSXuGx1H+S4o9vUsEp+ZTeP3khB8OzYhWGgCTc1QG5xEPsRDfe/0sS7JhO3z3cal+Kj 86dQcdTHSeBD3SHWgLdzn1aGo7lQgKxLzUYckwKczq/GPJigjfMT/sC5BwYJ9a4K7yhYqO DrlT0QVl5oG3kA51x1j10K3r8iLYAQc= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C66182F4; Tue, 17 Oct 2023 10:44:47 -0700 (PDT) Received: from [10.57.66.147] (unknown [10.57.66.147]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 0E7AA3F64C; Tue, 17 Oct 2023 10:44:04 -0700 (PDT) Message-ID: <21606fe5-fb9b-4d37-98ab-38c96819893b@arm.com> Date: Tue, 17 Oct 2023 18:44:03 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 1/2] zswap: make shrinking memcg-aware Content-Language: en-GB To: Nhat Pham , akpm@linux-foundation.org Cc: hannes@cmpxchg.org, cerasuolodomenico@gmail.com, yosryahmed@google.com, sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, linux-mm@kvack.org, kernel-team@meta.com, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org References: <20230919171447.2712746-1-nphamcs@gmail.com> <20230919171447.2712746-2-nphamcs@gmail.com> From: Ryan Roberts In-Reply-To: <20230919171447.2712746-2-nphamcs@gmail.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 3D1378000F X-Stat-Signature: m1yt95hhzjtpw955e7ij4n7n97p77a61 X-Rspam-User: X-HE-Tag: 1697564648-219985 X-HE-Meta: U2FsdGVkX18fGxzAzBCXrsgp+QSgKybOnbRhEVYZ1dncBV6l2UYs5ApfARVWr7xC/WiF7t/N2oH+6AW+oIjRh2i2efRGp2d2NJdUcfhXjLIZrOohHxIyU3qwkmrNNvQ4Y1I2zFnKFrQkPRa1sg3bTYSHhKvbHbIijafqxdZc8iEJJu9Ug0/wHGanhESwZ89LOZxlPkLr5VAywDsLjpW7UdtbtmiDONqCa8AMIq7P+96h9jKKLaN6Ihbz2XDsAkP6uKWd04L0tRphG6yqzOlA4bgKbc2hmpcQc9OtPbJmoVHS628XpcaDPiAbNWx/qWDUD7b4kD5iv6C/Hs+Fz7aq92DrW+Zt8Hh0eK8/Tew1lJbKQt+Er5ZZXJ5OwFqAxx+B04803oAc/xOO19V4lEGsDcPN2Eyf1SNWTndoscCYAtEj5iK5YGfXW7QlERIHSwX1fDM7SdMj+8n0A76uAOJQoqK+WbysWvaktxn2+nuDReKH2kgAZ9+WUS21dsYmrUh9ycv4Cv3F/EHpCVp5uIqplXn7EEhcVp89eWVMEY5sxfpIxz16vtNQJP9F+7LmKbuxUAbdfdhWoiEvzturXyFk7E87DoatKKmhGcYEYV4Gz0vV4LNCiAApcpKazziq+o+kvGDUrS6oNSzAT10WDxPdseeV0p5n06MO0P5QEKPsGqSyFXhXVSiKPACkfEbSPyDbgeCigLzkClQUagocCEyTMrasvM9HmfxYaHovHFDd6hrSIegnHY3pf+6ofnLjs8k0+gk+bYU8wVa6PsgwDupb62AiWGT0cqAco7oZXm1k9Rb718ZbBhyAzWS9ebALxXMS5dzvWZtjKxmaE335jOAfu1wUi18Be4GRO8mjKbpch2xQAn5FI8LYE/IkbBB8zY9O1EiFt5kr156F78kJDT57WK2MO7BoBgUT6kJN8RPZkr6IeVOTmwYMtHehcZJO1cUgeLnk+Cm+nUomm78OfGx y7YdlqbJ 1t883JzYxVmN+OBRtWrF9i6yRZ7heMoUMl87O3+G4UEil0UDaStPkt0bdXAHCfX9zsAHhWactVmMjG//cESxfsqbRN51tpHoMBf1uA+gk+97g7idbW48ysceIVAKRHbA6iofUUev1NAUmx34SgHZyRdHCoBDvcRNbhz5X3yVYBbzZ1t4s3cguTHajrfNqpLNW4LUsw8GcToCzjSAoDGRp407//mZjbfcu/r1c X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 19/09/2023 18:14, Nhat Pham wrote: > From: Domenico Cerasuolo > > Currently, we only have a single global LRU for zswap. This makes it > impossible to perform worload-specific shrinking - an memcg cannot > determine which pages in the pool it owns, and often ends up writing > pages from other memcgs. This issue has been previously observed in > practice and mitigated by simply disabling memcg-initiated shrinking: > > https://lore.kernel.org/all/20230530232435.3097106-1-nphamcs@gmail.com/T/#u > > This patch fully resolves the issue by replacing the global zswap LRU > with memcg- and NUMA-specific LRUs, and modify the reclaim logic: > > a) When a store attempt hits an memcg limit, it now triggers a > synchronous reclaim attempt that, if successful, allows the new > hotter page to be accepted by zswap. > b) If the store attempt instead hits the global zswap limit, it will > trigger an asynchronous reclaim attempt, in which an memcg is > selected for reclaim in a round-robin-like fashion. > > Signed-off-by: Domenico Cerasuolo > Co-developed-by: Nhat Pham > Signed-off-by: Nhat Pham > --- > include/linux/list_lru.h | 39 +++++++ > include/linux/memcontrol.h | 5 + > include/linux/zswap.h | 9 ++ > mm/list_lru.c | 46 ++++++-- > mm/swap_state.c | 19 ++++ > mm/zswap.c | 221 +++++++++++++++++++++++++++++-------- > 6 files changed, 287 insertions(+), 52 deletions(-) > [...] > @@ -1199,8 +1272,10 @@ bool zswap_store(struct folio *folio) > struct scatterlist input, output; > struct crypto_acomp_ctx *acomp_ctx; > struct obj_cgroup *objcg = NULL; > + struct mem_cgroup *memcg = NULL; > struct zswap_pool *pool; > struct zpool *zpool; > + int lru_alloc_ret; > unsigned int dlen = PAGE_SIZE; > unsigned long handle, value; > char *buf; > @@ -1218,14 +1293,15 @@ bool zswap_store(struct folio *folio) > if (!zswap_enabled || !tree) > return false; > > - /* > - * XXX: zswap reclaim does not work with cgroups yet. Without a > - * cgroup-aware entry LRU, we will push out entries system-wide based on > - * local cgroup limits. > - */ > objcg = get_obj_cgroup_from_folio(folio); > - if (objcg && !obj_cgroup_may_zswap(objcg)) > - goto reject; > + if (objcg && !obj_cgroup_may_zswap(objcg)) { > + memcg = get_mem_cgroup_from_objcg(objcg); > + if (shrink_memcg(memcg)) { > + mem_cgroup_put(memcg); > + goto reject; > + } > + mem_cgroup_put(memcg); > + } > > /* reclaim space if needed */ > if (zswap_is_full()) { > @@ -1240,7 +1316,11 @@ bool zswap_store(struct folio *folio) > else > zswap_pool_reached_full = false; > } > - > + pool = zswap_pool_current_get(); > + if (!pool) { > + ret = -EINVAL; > + goto reject; > + } Hi, I'm working to add support for large folios within zswap, and noticed this piece of code added by this change. I don't see any corresponding put. Have I missed some detail or is there a bug here? > /* allocate entry */ > entry = zswap_entry_cache_alloc(GFP_KERNEL); > if (!entry) { > @@ -1256,6 +1336,7 @@ bool zswap_store(struct folio *folio) > entry->length = 0; > entry->value = value; > atomic_inc(&zswap_same_filled_pages); > + zswap_pool_put(pool); I see you put it in this error path, but after that, there is no further mention. > goto insert_entry; > } > kunmap_atomic(src); > @@ -1264,6 +1345,15 @@ bool zswap_store(struct folio *folio) > if (!zswap_non_same_filled_pages_enabled) > goto freepage; > > + if (objcg) { > + memcg = get_mem_cgroup_from_objcg(objcg); > + lru_alloc_ret = memcg_list_lru_alloc(memcg, &pool->list_lru, GFP_KERNEL); > + mem_cgroup_put(memcg); > + > + if (lru_alloc_ret) > + goto freepage; > + } > + > /* if entry is successfully added, it keeps the reference */ > entry->pool = zswap_pool_current_get(); The entry takes it's reference to the pool here. Thanks, Ryan