From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2929C4167B for ; Wed, 6 Dec 2023 06:44:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3488C6B007D; Wed, 6 Dec 2023 01:44:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2A9E56B007E; Wed, 6 Dec 2023 01:44:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 14AC76B0080; Wed, 6 Dec 2023 01:44:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id F236A6B007D for ; Wed, 6 Dec 2023 01:44:00 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id BB43C120401 for ; Wed, 6 Dec 2023 06:44:00 +0000 (UTC) X-FDA: 81535453440.03.B7754B9 Received: from out-174.mta1.migadu.com (out-174.mta1.migadu.com [95.215.58.174]) by imf18.hostedemail.com (Postfix) with ESMTP id B24491C0019 for ; Wed, 6 Dec 2023 06:43:58 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=R3EXQyhc; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf18.hostedemail.com: domain of chengming.zhou@linux.dev designates 95.215.58.174 as permitted sender) smtp.mailfrom=chengming.zhou@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701845039; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=L68GGlTplESV/Uk34Q5KhUMIcQyrsh0KKX3y4sPM0Qw=; b=W9SmZD/VXvd7cdIpRzdq5+jDBruaI6Nm87neqwDDGeYX6DEBVTQcPY16MGZBacT2j6RENo Pzv+93yMUuRoc40deJ6kN5RhWQrpf+wXfe152kroFwmVNY2j+JrTSwwCsN+5QYV8+ES1d0 x3OHcXgGJWQDOY/vfNVsYiWCEa/VU2M= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=R3EXQyhc; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf18.hostedemail.com: domain of chengming.zhou@linux.dev designates 95.215.58.174 as permitted sender) smtp.mailfrom=chengming.zhou@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701845039; a=rsa-sha256; cv=none; b=3avhq8MmV51qEBM22DVGrgBE5OGO6ghU+hS+DcPlZ9c6PH+w1hqJDCBgm0NYoZBavF3Ajr dP5uFUqLEqirNqM10xRkXfcJ/RJ9jrw/+2NoUJ+eDPnyLzcydPbaT+1SbZrD5oR1lOzjvk fl+SenyXywlVxjr3XcCmloT1wV1XocE= Message-ID: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1701845035; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=L68GGlTplESV/Uk34Q5KhUMIcQyrsh0KKX3y4sPM0Qw=; b=R3EXQyhcv5+RGJmgumggdbkeOKhqUnpkHITAo3gTVWgV17Jj4KhKB2fUTP9P/3FkyHK1Yx 4lvSFddDjHbUXiiM427uElO/cTmNnrqqMrLaXr2/LnerjMVGe9Fbg6nUBB8vEopa5YlL9+ lt0APH3XACSf68CH9mKii8xsDNYXXp0= Date: Wed, 6 Dec 2023 14:43:48 +0800 MIME-Version: 1.0 Subject: Re: [PATCH v8 6/6] zswap: shrinks zswap pool based on memory pressure Content-Language: en-US To: Yosry Ahmed Cc: Nhat Pham , akpm@linux-foundation.org, hannes@cmpxchg.org, cerasuolodomenico@gmail.com, sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com, mhocko@kernel.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, chrisl@kernel.org, linux-mm@kvack.org, kernel-team@meta.com, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, shuah@kernel.org References: <20231130194023.4102148-1-nphamcs@gmail.com> <20231130194023.4102148-7-nphamcs@gmail.com> X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Chengming Zhou In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_OUT X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: B24491C0019 X-Stat-Signature: nxmnsca94rohpbopqnjcx667myogitpt X-HE-Tag: 1701845038-89956 X-HE-Meta: U2FsdGVkX189jCepZYsPKD3re/37zliCf5t9Hjoe/kwVuAnqIcDTAkhl6Y7BbJG2aJLWA8c4SLJ51Z96d602xMnNZ9skAAp9BQkkc2yr04AvkI3Wqj+kU3l3GmSoUJuZ09LRyIz5RPK4qRdoUZSZBSJaIPQjiYyy3lz4z3cn4xlznR4bJXOsHTjZnObZPCQJkWIJFeyS2LZSXnM0Y1IA+yVV0ScWp6hzoDz595c9IJ9qlI+bYhV72K0dQgbfkv9taS+5H5IRDxHRwFgSofEac1E7BUdq2ulQl0I/lYou4rK15u9VBJzU8kPx8al4p5j0NLID2AbExjOiDdQDc/Tbrg1BHfxzrYPKrCeZx+FVcG/WJ9yo4gBy4kGQ2NYPJ3QLudmql3ehaZNAwnLp4n7N6eRx2kLUaMhKn/rXTunZ8LtENNGaSdudoWFh7sQmgCGLkdx1v5IoEG20oNfAo/1wlWOunIjEYJVFZHRD1XJJG09xe0yepYM5Vugy2apgaYBF8yHPp9XOVh1mDYZCHd2Shgd8bBWib1Jf+k6l6xRYtJ7LyNWZMOqe5RutMUj1xvd4C3cWlUzWLE3d3VdogceU0sbhdxeD8gWdgd1x21HW5WjnjbETGq8R+wJW5jhvP8ahUHv9XLxZSogvZXthuGvaGDhPrJ7aDXGVjwFcIN4tOhcRF0SfAJxY/60KvNrIr8RXUM+TnUh+HOSo5kYKXZMfHIM8Ev6A1fJx7/My3f7mpDdAAgNKRgh2G3aYoQq7Zs/KXbbJltVVbk6S825AJmccuZuD4zYBni/yWa2w3ymFGuQmV+DkEnTUs/mVfSAciGd9Furcm4Vk+HUNZD3gOTuD4OGW+z4t13QJ8znfeyQkAyvh9IOKjN/OYwk4FEYwKUx12CSAc7AHSlktq6dzfZBFF6MiJnt0qp3wDfAXIdLOwP7IkIYXCD9mNQYxNi89n+blFT9cjPEq/y8l6wWQgtn w7EqMNs/ QIzuwEUTodjCelCysbIb7gOIR5bFrprNaP9PcyXsyrP2wlPocm/ywlPAhVzQxbtAXwyo1OMU6HlhVStg= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2023/12/6 13:59, Yosry Ahmed wrote: > [..] >>> @@ -526,6 +582,102 @@ static struct zswap_entry *zswap_entry_find_get(struct rb_root *root, >>> return entry; >>> } >>> >>> +/********************************* >>> +* shrinker functions >>> +**********************************/ >>> +static enum lru_status shrink_memcg_cb(struct list_head *item, struct list_lru_one *l, >>> + spinlock_t *lock, void *arg); >>> + >>> +static unsigned long zswap_shrinker_scan(struct shrinker *shrinker, >>> + struct shrink_control *sc) >>> +{ >>> + struct lruvec *lruvec = mem_cgroup_lruvec(sc->memcg, NODE_DATA(sc->nid)); >>> + unsigned long shrink_ret, nr_protected, lru_size; >>> + struct zswap_pool *pool = shrinker->private_data; >>> + bool encountered_page_in_swapcache = false; >>> + >>> + nr_protected = >>> + atomic_long_read(&lruvec->zswap_lruvec_state.nr_zswap_protected); >>> + lru_size = list_lru_shrink_count(&pool->list_lru, sc); >>> + >>> + /* >>> + * Abort if the shrinker is disabled or if we are shrinking into the >>> + * protected region. >>> + * >>> + * This short-circuiting is necessary because if we have too many multiple >>> + * concurrent reclaimers getting the freeable zswap object counts at the >>> + * same time (before any of them made reasonable progress), the total >>> + * number of reclaimed objects might be more than the number of unprotected >>> + * objects (i.e the reclaimers will reclaim into the protected area of the >>> + * zswap LRU). >>> + */ >>> + if (!zswap_shrinker_enabled || nr_protected >= lru_size - sc->nr_to_scan) { >>> + sc->nr_scanned = 0; >>> + return SHRINK_STOP; >>> + } >>> + >>> + shrink_ret = list_lru_shrink_walk(&pool->list_lru, sc, &shrink_memcg_cb, >>> + &encountered_page_in_swapcache); >>> + >>> + if (encountered_page_in_swapcache) >>> + return SHRINK_STOP; >>> + >>> + return shrink_ret ? shrink_ret : SHRINK_STOP; >>> +} >>> + >>> +static unsigned long zswap_shrinker_count(struct shrinker *shrinker, >>> + struct shrink_control *sc) >>> +{ >>> + struct zswap_pool *pool = shrinker->private_data; >>> + struct mem_cgroup *memcg = sc->memcg; >>> + struct lruvec *lruvec = mem_cgroup_lruvec(memcg, NODE_DATA(sc->nid)); >>> + unsigned long nr_backing, nr_stored, nr_freeable, nr_protected; >>> + >>> +#ifdef CONFIG_MEMCG_KMEM >>> + cgroup_rstat_flush(memcg->css.cgroup); >>> + nr_backing = memcg_page_state(memcg, MEMCG_ZSWAP_B) >> PAGE_SHIFT; >>> + nr_stored = memcg_page_state(memcg, MEMCG_ZSWAPPED); >>> +#else >>> + /* use pool stats instead of memcg stats */ >>> + nr_backing = get_zswap_pool_size(pool) >> PAGE_SHIFT; >>> + nr_stored = atomic_read(&pool->nr_stored); >>> +#endif >>> + >>> + if (!zswap_shrinker_enabled || !nr_stored) >> When I tested with this series, with !zswap_shrinker_enabled in the default case, >> I found the performance is much worse than that without this patch. >> >> Testcase: memory.max=2G, zswap enabled, kernel build -j32 in a tmpfs directory. >> >> The reason seems the above cgroup_rstat_flush(), caused much rstat lock contention >> to the zswap_store() path. And if I put the "zswap_shrinker_enabled" check above >> the cgroup_rstat_flush(), the performance become much better. >> >> Maybe we can put the "zswap_shrinker_enabled" check above cgroup_rstat_flush()? > > Yes, we should do nothing if !zswap_shrinker_enabled. We should also > use mem_cgroup_flush_stats() here like other places unless accuracy is > crucial, which I doubt given that reclaim uses > mem_cgroup_flush_stats(). > Yes. After changing to use mem_cgroup_flush_stats() here, the performance become much better. > mem_cgroup_flush_stats() has some thresholding to make sure we don't > do flushes unnecessarily, and I have a pending series in mm-unstable > that makes that thresholding per-memcg. Keep in mind that adding a > call to mem_cgroup_flush_stats() will cause a conflict in mm-unstable, My test branch is linux-next 20231205, and it's all good after changing to use mem_cgroup_flush_stats(memcg). > because the series there adds a memcg argument to > mem_cgroup_flush_stats(). That should be easily amenable though, I can > post a fixlet for my series to add the memcg argument there on top of > users if needed. > It's great. Thanks!