From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B47F7C48260 for ; Fri, 16 Feb 2024 08:49:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 410CA8D000E; Fri, 16 Feb 2024 03:49:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3CAD38D0001; Fri, 16 Feb 2024 03:49:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 212698D000E; Fri, 16 Feb 2024 03:49:36 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 0CDDA8D0001 for ; Fri, 16 Feb 2024 03:49:36 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id D35C416032F for ; Fri, 16 Feb 2024 08:49:35 +0000 (UTC) X-FDA: 81797043510.30.E44964E Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) by imf05.hostedemail.com (Postfix) with ESMTP id 18720100009 for ; Fri, 16 Feb 2024 08:49:33 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="h1X6/LLZ"; spf=pass (imf05.hostedemail.com: domain of zhouchengming@bytedance.com designates 209.85.214.178 as permitted sender) smtp.mailfrom=zhouchengming@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1708073374; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ef4J5UO4FCUkGSsVpOjeciLsm69YrWZrK7HVqGKOCu8=; b=laYG7xJLI0slcGqGzsqkHg96Q7hO2Yt33BeQomT8PgsaHtBWsbBrFf4vpz4Rwe35yGGrCy CNcw9ViZ3mGcuuHZyrk346bTQrPRSXW34a/xwmLdRVozwLMHXFJEWyWrikHviRw7kG/U/i nkIDWHib+eFScFgpwUsQuEscB2TTFmE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1708073374; a=rsa-sha256; cv=none; b=FMSsI+jn10+vphCaZI1FZTBeITDAJpO9RMhBG/wmm5gZI3PU7JQpkRYIIZgIsgTz3OGT4Z hIE7nIQCjisPtcDbejLrpyQ1clrj9IF/bTox12bEDJFXfFiyvOaR+mSbesx645Orn4NQ9d VS0BKhnbLzwnm52FKR6WT2CXzDWYCkc= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="h1X6/LLZ"; spf=pass (imf05.hostedemail.com: domain of zhouchengming@bytedance.com designates 209.85.214.178 as permitted sender) smtp.mailfrom=zhouchengming@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-1d934c8f8f7so18035745ad.2 for ; Fri, 16 Feb 2024 00:49:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1708073373; x=1708678173; darn=kvack.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=ef4J5UO4FCUkGSsVpOjeciLsm69YrWZrK7HVqGKOCu8=; b=h1X6/LLZGpzLs10yjBNoFIhN0BKtW3I5aE3IfH3vsThse53CzxcLdLOJZf4MzoEPu9 yvO3B7ChdGnOz2Mzl2Nl9mERmadeqZUsU2MDxUIm3rSDavYoQupD1Gx66yyhHc5qCf6P pPvhERuEyTBwaPkOrGGSgblASxpdXM4zjUSYm1v85Qq60hSzHwAGsfYrHa2VUDRdX7wt oFPuF102tksssy2oH5zslZgvkgP4A3y98FCffYsAeA8miZU4UBoaQrge82lYLD37eNjG rJdnymABJ3uwihUCKPhm6sBHBAUxdFIHYubwppFrD+34cQuslVxXJvIxNao+KhURYg6F GtRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708073373; x=1708678173; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ef4J5UO4FCUkGSsVpOjeciLsm69YrWZrK7HVqGKOCu8=; b=e2SqeohhOdZb8nrftR8KpoHVXR0FLJbIMokclrplZBRDD952YJ4j7DCOjU2q2IGEHF fViGkFANVy4WFfzkQ6sSPz/xg+wZh01ev0/Nftwe6H/8dTW+OcwdChPSW0szpgf/GuMz APf0YrArRfMhZwxSbCx2wFyk8hCOdsWfajEUgFTNNzIXwV2nAxotkcp8pi0PrfvkKPvS pO4QKirSuXWH/QOisiYti/Xkv1rK90hz+IC7qhUdumxa7hFqB+0l7PQad5AZeVsRaN+D lUEYQ2t/zIC8LMAmouope7gzbiftm+qJ0b9Vy75pSD+okVyF7yGEeUKC0JF7bGev518m o+5Q== X-Forwarded-Encrypted: i=1; AJvYcCUVxINqwPfKZgzReptM1X/I2qQatRLaGC9tvzxHUnnC73pQ4DSyDdYhIOOQ3ZIxbyFg1phZyF8cCJWlvGag3Vr+8no= X-Gm-Message-State: AOJu0YzAvbczjP3nce5vZ+9lZRl/XY31LNYq1BIhGl9Vl0S9IDM/T26y npSbd78NT2v6d/ghfyqfB2IDdrd7cNou5oYxi82K8zBIBBrQpqrf28K9bEHNHJs= X-Google-Smtp-Source: AGHT+IFfZdn6OadDErDuWgbMNaXtVYv3uOoUsCK9yeWZpHnbz45NwSh2/3MVCHzMg/GqBeE2vQzRUQ== X-Received: by 2002:a17:902:ec82:b0:1da:498:24be with SMTP id x2-20020a170902ec8200b001da049824bemr5056656plg.40.1708073372919; Fri, 16 Feb 2024 00:49:32 -0800 (PST) Received: from [10.4.195.175] ([139.177.225.229]) by smtp.gmail.com with ESMTPSA id u20-20020a170902e21400b001db4c89aea5sm2529069plb.158.2024.02.16.00.49.30 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 16 Feb 2024 00:49:32 -0800 (PST) Message-ID: Date: Fri, 16 Feb 2024 16:49:28 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 2/2] mm/zswap: change zswap_pool kref to percpu_ref Content-Language: en-US To: Yosry Ahmed Cc: Johannes Weiner , Andrew Morton , Nhat Pham , linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <20240210-zswap-global-lru-v2-0-fbee3b11a62e@bytedance.com> <20240210-zswap-global-lru-v2-2-fbee3b11a62e@bytedance.com> From: Chengming Zhou In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 18720100009 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 1gbkt33djdc5py4cmht5auyayzhtabcj X-HE-Tag: 1708073373-171358 X-HE-Meta: U2FsdGVkX182tHVqYwOpCyG8M0n++su9o6oZN/HWCZ6KKtWcmd6Dt3iWj7f6u1+N7FCzgO71uQotiVtzjaUzcvQYAK/bCneIBlBIL1mzqRlthcIy20f29avia57apuB9GT4+NzTpTjajjBP7bfRHqLr7qJp/UPqi/oRBW5CuJrdyawQ4L3CjICUELeAVihKGjqGrJ+I13OFNFY3nlN0nOmuMAYKTDqI+cwGsppkw0RSU53uKSxbEDbtbrgqVyg5GF1JwSbC2Xafwi66jfI88fjLXOZqYB4j6NyOSwcWGcJt/3IctU1kBNqg6fHAPvQpB2zWYoWaqTf/5mh6XjW8ZX5qR4lCt1rLIfiEnZO/HN3RqwVhYGut/ms32EYLsxBjrHIMh/Y3MDtG2EO6g40Cv7yemQvAUM8i3edZxxSrd87bNj/Q2+/RBAT+qzW6jV7Dh0UOg+8lsR5JFEb3KgcfGQZ6qlpsRFqU/hUapS88WMkYjzviwJiMl0UZI7ld4PVxNkxp47ZrSjztC0qUHKXmXtTUzVjgkVtxFS1iV6Pug8OCsDm5T5b3cUMtEf5T63AlWJ47/X+cQPoEELd0h31PxIxdch6+E80capBxanzCue6LGpOXsFvPIHNiIi0+IC70zzZF0+D/UIWOv3XiC6pu2dGP2vzKC7H4P8N+y+fwUtbviJv1swNsd/SNqicUuw2M+9zzIf+RSHKF6GDf0gdykezYUwKVVBIKcNNfllF4ZJYXIEoDFewVuRXC6vvGmEd9EPJWFM1xtn8Hug8SOEhjfLJ20EerPtCDLgkc8JQdhlNNEMvNl07EjYzjOA2JazoeFaaMk1dLUwarOkmQp1JkL0bgPltDw0VbXgVpPW2ABAYtCoOqE7BUZHcAR5elCw8vPC1kUSASiQDklbLhN76raIzK2DcfWnUi0EyEUG2H8Resuuv8AMTqOai4EUyUI+aGlflpUaBvrMzy9eTZ3/jX PG/oJpJT DD8YMs5MYNskVh18iryk56J4m8Iis1ZaS3zrhB9qhPANbRfXX4DyuVyqf9PocRs/UsY29BXzg6rndXCMtQTv1n5mqYKQS+IepfgWS+suiN2bcvB0IsFqo3XcWEQZjQCMxuPnyMLApD9obReqp18rA41enfXtq9ZtCznJXCONkciBoxmkiwN/MpMA1AA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024/2/15 04:10, Yosry Ahmed wrote: > On Wed, Feb 14, 2024 at 08:54:38AM +0000, Chengming Zhou wrote: >> All zswap entries will take a reference of zswap_pool when >> zswap_store(), and drop it when free. Change it to use the >> percpu_ref is better for scalability performance. >> >> Although percpu_ref use a bit more memory which should be ok >> for our use case, since we almost have only one zswap_pool to >> be using. The performance gain is for zswap_store/load hotpath. >> >> Testing kernel build (32 threads) in tmpfs with memory.max=2GB. >> (zswap shrinker and writeback enabled with one 50GB swapfile, >> on a 128 CPUs x86-64 machine, below is the average of 5 runs) >> >> mm-unstable zswap-global-lru >> real 63.20 63.12 >> user 1061.75 1062.95 >> sys 268.74 264.44 >> >> Signed-off-by: Chengming Zhou >> --- >> mm/zswap.c | 31 ++++++++++++++++++++++--------- >> 1 file changed, 22 insertions(+), 9 deletions(-) >> >> diff --git a/mm/zswap.c b/mm/zswap.c >> index dbff67d7e1c7..f6470d30d337 100644 >> --- a/mm/zswap.c >> +++ b/mm/zswap.c >> @@ -173,7 +173,7 @@ struct crypto_acomp_ctx { >> struct zswap_pool { >> struct zpool *zpools[ZSWAP_NR_ZPOOLS]; >> struct crypto_acomp_ctx __percpu *acomp_ctx; >> - struct kref kref; >> + struct percpu_ref ref; >> struct list_head list; >> struct work_struct release_work; >> struct hlist_node node; >> @@ -304,6 +304,7 @@ static void zswap_update_total_size(void) >> /********************************* >> * pool functions >> **********************************/ >> +static void __zswap_pool_empty(struct percpu_ref *ref); >> >> static struct zswap_pool *zswap_pool_create(char *type, char *compressor) >> { >> @@ -357,13 +358,18 @@ static struct zswap_pool *zswap_pool_create(char *type, char *compressor) >> /* being the current pool takes 1 ref; this func expects the >> * caller to always add the new pool as the current pool >> */ >> - kref_init(&pool->kref); >> + ret = percpu_ref_init(&pool->ref, __zswap_pool_empty, >> + PERCPU_REF_ALLOW_REINIT, GFP_KERNEL); >> + if (ret) >> + goto ref_fail; >> INIT_LIST_HEAD(&pool->list); >> >> zswap_pool_debug("created", pool); >> >> return pool; >> >> +ref_fail: >> + cpuhp_state_remove_instance(CPUHP_MM_ZSWP_POOL_PREPARE, &pool->node); >> error: >> if (pool->acomp_ctx) >> free_percpu(pool->acomp_ctx); >> @@ -436,8 +442,9 @@ static void __zswap_pool_release(struct work_struct *work) >> >> synchronize_rcu(); >> >> - /* nobody should have been able to get a kref... */ >> - WARN_ON(kref_get_unless_zero(&pool->kref)); >> + /* nobody should have been able to get a ref... */ >> + WARN_ON(percpu_ref_tryget(&pool->ref)); > > Just curious, was there any value from using kref_get_unless_zero() over > kref_read() here? If not, I think percpu_ref_is_zero() is more > intuitive. This also seems like it fits more as a debug check. Agree, percpu_ref_is_zero() is better for debug. > >> + percpu_ref_exit(&pool->ref); >> >> /* pool is now off zswap_pools list and has no references. */ >> zswap_pool_destroy(pool); >> @@ -445,11 +452,11 @@ static void __zswap_pool_release(struct work_struct *work) >> >> static struct zswap_pool *zswap_pool_current(void); >> >> -static void __zswap_pool_empty(struct kref *kref) >> +static void __zswap_pool_empty(struct percpu_ref *ref) >> { >> struct zswap_pool *pool; >> >> - pool = container_of(kref, typeof(*pool), kref); >> + pool = container_of(ref, typeof(*pool), ref); >> >> spin_lock(&zswap_pools_lock); >> >> @@ -468,12 +475,12 @@ static int __must_check zswap_pool_get(struct zswap_pool *pool) >> if (!pool) >> return 0; >> >> - return kref_get_unless_zero(&pool->kref); >> + return percpu_ref_tryget(&pool->ref); >> } >> >> static void zswap_pool_put(struct zswap_pool *pool) >> { >> - kref_put(&pool->kref, __zswap_pool_empty); >> + percpu_ref_put(&pool->ref); >> } >> >> static struct zswap_pool *__zswap_pool_current(void) >> @@ -603,6 +610,12 @@ static int __zswap_param_set(const char *val, const struct kernel_param *kp, >> >> if (!pool) >> pool = zswap_pool_create(type, compressor); >> + else { >> + /* Resurrect percpu_ref to percpu mode. */ >> + percpu_ref_resurrect(&pool->ref); > > I think this is not very clear. The previous code relied on the ref from > zswap_pool_find_get() to replace the initial ref that we had dropped > before. This is not needed with percpu_ref_resurrect() because it > already restores the initial ref dropped by percpu_ref_kill(). > > Perhaps something like: > /* > * Restore the initial ref dropped by percpu_ref_kill() > * when the pool was decommissioned and switch it again > * to percpu mode. > / > Ok, will add this comment, it's clearer. Thanks!