From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA6EEC4167B for ; Wed, 6 Dec 2023 20:42:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 262456B0093; Wed, 6 Dec 2023 15:42:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2129C6B0096; Wed, 6 Dec 2023 15:42:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0DA976B0098; Wed, 6 Dec 2023 15:42:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id F18706B0093 for ; Wed, 6 Dec 2023 15:42:37 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id C1FC3A0234 for ; Wed, 6 Dec 2023 20:42:37 +0000 (UTC) X-FDA: 81537566754.05.F604414 Received: from mail-wm1-f52.google.com (mail-wm1-f52.google.com [209.85.128.52]) by imf07.hostedemail.com (Postfix) with ESMTP id EBEBF40021 for ; Wed, 6 Dec 2023 20:42:35 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=4nwdN8RZ; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf07.hostedemail.com: domain of yosryahmed@google.com designates 209.85.128.52 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701895356; a=rsa-sha256; cv=none; b=c8omZ0SHEL3hfMlbuiKRL1TWZFv2amx1aeOVbCUCUvwis0DrE2H9d0nA7yYGMtfH2LokVa YkHrB41o8TEj3l76phxCBo+YSLyFleWtwx0vp+I7EwnS6TYhJDOc3NlWOtTW4kZnHl6Z0e zs01UbIZb/8zDXkPhkSDxNZ23euYzBE= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=4nwdN8RZ; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf07.hostedemail.com: domain of yosryahmed@google.com designates 209.85.128.52 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701895356; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=DY1IC2rhyb/Bn5x0Ih7ThDloh0MTXdgqLcG6ncgKEKc=; b=0yJM02cnIcHeF60hqnEmtqJhtv6e+IMXu5agAXhNTVNr4qnRrvV9DOUyJI/Q+hIJO2qnD1 0EUaPpprFA5TVZLqUozKr5uhui+FpF49k3UXScew0jytHJ0aW4eXoTkjVOpaRZWjhvwrl9 +IBO/irKvTuFMbjx3Ic7qpvm5PgtODg= Received: by mail-wm1-f52.google.com with SMTP id 5b1f17b1804b1-40c18e9d7c0so2554915e9.0 for ; Wed, 06 Dec 2023 12:42:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1701895354; x=1702500154; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=DY1IC2rhyb/Bn5x0Ih7ThDloh0MTXdgqLcG6ncgKEKc=; b=4nwdN8RZ3Pc9szeKJmU1E86Z6NcVUGYTYsw2q//NpvyyI1QBUO3B2TY1b4C04xI1mg 9/g7HE1jr9+c/vHjIy3+K2h05YHihssjhHZRf5A1DkMct8kFesmjCPX+83VZOCCb01S7 HpDBs2Udrm4+tEdcYBRwXWV+3lutOq0qnwqOqzUgho2DSb58gAeQLCwhqPF7pUnhJ3ar Ey6xuwnb4cCAvv6XdlWTJVER4aLFHLUsx7zIeaeppYjY+mUWbT7dDUxUmUJxheczMa78 IAwpeiNeOH4eo6cWFzHqr64OLeDnvrKtcPI0wEIuflXWHxw4wc5Dkh2d46ayMvlVky76 PVEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701895354; x=1702500154; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DY1IC2rhyb/Bn5x0Ih7ThDloh0MTXdgqLcG6ncgKEKc=; b=f/6tMHVqLVJMJHbIx65aJqCljyPuVTQ4Q3EYAA7vTHqFM5Tgv7Wrd5Ta34JlVbftJN PeHPM6aS+3hCkBRPrZ7K3eWQ69gs7ztBU+QwuQ3LcbnNOotHrH4h4FjB/DYLIceP8QFL dZGdIKzc5OCrOi/0A2q7BA21K5byZCQoPF78Dkn7+HMoL/+dL3ZHGJUnSmQWbrSm4Ulo qf4PBWgWQJpB6/n6BXsijmEgc+BuDycpDelqLOVLCN2+EzlqcQo4gWlJlxZkoti4lvxw 55BZKpHxGZ6eYmwRjDHtg84Pt8zYcjcz6XbIdMV6ONEA3J6u/SJU0hgdfSj7okyygPAh T8qg== X-Gm-Message-State: AOJu0YzaW859fkS5gEktr1FLncfzqTN6ovR1/Un4qXWuh+VjItTs//hz p0/98Dhf4JJ2eWT5EYyrIWVctSbqq9bgJIo3XCtWPA== X-Google-Smtp-Source: AGHT+IHxFajjYl5fbmWvZOaXd0l+GmbiWxkRPg0YrqfXoTyFy1Mdd4gYd2znDMOPPmqC7xLlqwEAjDz+tATZ+x4IvvE= X-Received: by 2002:a05:600c:4f95:b0:40b:3e66:f5ff with SMTP id n21-20020a05600c4f9500b0040b3e66f5ffmr928503wmq.24.1701895354074; Wed, 06 Dec 2023 12:42:34 -0800 (PST) MIME-Version: 1.0 References: <20231206-zswap-lock-optimize-v1-0-e25b059f9c3a@bytedance.com> In-Reply-To: From: Yosry Ahmed Date: Wed, 6 Dec 2023 12:41:54 -0800 Message-ID: Subject: Re: [PATCH 0/7] mm/zswap: optimize the scalability of zswap rb-tree To: Nhat Pham Cc: Chengming Zhou , Vitaly Wool , Johannes Weiner , Michal Hocko , Seth Jennings , Dan Streetman , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Chris Li Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: EBEBF40021 X-Stat-Signature: tc78aa4hbx174b76j4up8ps8c84qgfpz X-HE-Tag: 1701895355-812414 X-HE-Meta: U2FsdGVkX1/OgZqXTcxxgs3TyeqmSRYTITIaoTLERRMrlXoMfAr34Y7jIcmC6jnBNmWIOmcIAh7PpJit2i2G08HH2PRI2HIn/d24Qz4ghVjlrbY+5f6XbSDMo6zLZDR21FO7Ztt8FGZLl7KtpozbLG0N0nPNnBBuwf4u0qmLn/AkFAH9mUhyiuC4yhFR9U8VGxtIZjb9GikO7KNGOfYfuxnrWAFEJnNYIDQ3sjIJgkFJMVcmLpb1CzpawXLLBiF617NU42L/xrTNsmEdig8EdpXE/BHg7NsRtVSE4T4ACgVbFysQ06y2uAa6kiQhZ9CUgGwWhw0GL5/j+ioGx/X0Ipp8Eu2H4kLpMuCmyRHMVPjoD3xxaqkqNUigqeHFDOsdwfygBJ4Uw3fNfuNaie7XAxlOirYWnRntOKilyfC1Xd4rE0QCiE6hPcZUKvARIqFa+GHmcXiFw2jYIti5W8IY5PrdJL4rOmxaFTfnCPOR1HyhC0vF+lr3z8D4phL3Q1vBtuBlirbRoSFsNXD53fysxX+28r4ExbzlCTBFB1m/l373QmQj8rZelrWG4w6yUyAxJnGk1sHF7O5tWo2FC8DYg0gefw0BWVAc+/dq8Kz/jYq12FuC4D0OX46SiJr1+U+7eufQoqa6w+bCgQdr6ouC8bg6o68KLCZAbk45AxPc/tofPLYRSuZqK9UJPyL39/qeDufQFcAbBFin1gNnWW204QC9zaCRxLNrq6CCELf0yYOHYVcxqcMohUjg4qTFJPZ4mhPD31TwPiLx8x9inA1KagYLW1483Om0AmiDQBF6HLb5fxYRw0ZNmrf/jd2+JgQbO7mg/GerUFtLsbIQzDb0fypqqrzo+4xirn4RU1wcRUoALvp0uOYOqHL6TVg9vPaQw3029kYg9WcOF9LvYMDt2vzW5tAq+OZ0PnM+WKwLLU6/0VftDRV7JqLll0vNi+rh8BO7U0G+wkLoHqx/W5/ pRkbEfj8 Q02GzsF9GoDcyME9RbQO8kgXSnYvEtg9ZM7JToSbPK8Cc5jp3L4XUeWafUPbmBXDbOjSjBGSlgXFNpc/lSO/m13uvsAr5TGnfAkdZo9U/tw0zZK5OlND0vyoj+VM2cVA7Hoavu07HJKhWgD5BqYqeeNRiQxCigtqWX1deIHmo8pG9LeFx0ZGi1qT50HUJQfVm9tjCnKW/+DfKYDd09KCcne9zt4/t7hF1MgLNX0+duTgPBP/S0L80ED3zXqFJOBh4uVbJTD0M5z7UkdubTijfyvVqL+0FotVnMmvZCfhSyRZfEZMNE+DzTEwadecJuQi5YJlf85w+C65YjXtFK3wsNUWiAlmE4Gwq5k3OwON3zdNTrsN8YkMQ0tb1dMubLYjzTKtqZS98SpKxoqipVnBbfMCB88Vjy7tclsqDtSoE99uOqjgSgmZ+Pf62Jih9AKkfjzYNHPm1GzJ0gxoKm+A7jNqoR7MiG/Cu+/cLOtlcWqxX83G7820PcTcPL05duHZHJtSE10TE6Kdt5u+o1U8AJn6Of5v6sT3X1LJlVcS5tKDQWlSB4TDNVks5xxnncwwpJ3AXQS406ZXvivIBGLbZX5ENAIhySALLAlTdKM47WE14obmmQXuyRr/UnwB43z6DpKj95PANGlBEikoVdWCfzslrOXu5t/0BgKZa/TLAWjaT2Us= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Dec 6, 2023 at 9:24=E2=80=AFAM Nhat Pham wrote: > > + Chris Li > > Chris, I vaguely remember from our last conversation that you have > some concurrent efforts to use xarray here right? If I recall correctly, the xarray already reduces the lock contention as lookups are lockless, but Chris knows more here. As you mentioned in a different email, it would be nice to get some data so that we can compare different solutions. > > On Wed, Dec 6, 2023 at 1:46=E2=80=AFAM Chengming Zhou > wrote: > > > > Hi everyone, > > > > This patch series is based on the linux-next 20231205, which depends on > > the "workload-specific and memory pressure-driven zswap writeback" seri= es > > from Nhat Pham. > > > > When testing the zswap performance by using kernel build -j32 in a tmpf= s > > directory, I found the scalability of zswap rb-tree is not good, which > > is protected by the only spinlock. That would cause heavy lock contenti= on > > if multiple tasks zswap_store/load concurrently. > > > > So a simple solution is to split the only one zswap rb-tree into multip= le > > rb-trees, each corresponds to SWAP_ADDRESS_SPACE_PAGES (64M). This idea= is > > from the commit 4b3ef9daa4fc ("mm/swap: split swap cache into 64MB trun= ks"). > > > > Although this method can't solve the spinlock contention completely, it > > can mitigate much of that contention. > > > > Another problem when testing the zswap using our default zsmalloc is th= at > > zswap_load() and zswap_writeback_entry() have to malloc a temporary mem= ory > > to support !zpool_can_sleep_mapped(). > > > > Optimize it by reusing the percpu crypto_acomp_ctx->dstmem, which is al= so > > used by zswap_store() and protected by the same percpu crypto_acomp_ctx= ->mutex. > > > > Thanks for review and comment! > > > > To: Andrew Morton > > To: Seth Jennings > > To: Dan Streetman > > To: Vitaly Wool > > To: Nhat Pham > > To: Johannes Weiner > > To: Yosry Ahmed > > To: Michal Hocko > > Cc: linux-kernel@vger.kernel.org > > Cc: linux-mm@kvack.org > > Signed-off-by: Chengming Zhou > > > > --- > > Chengming Zhou (7): > > mm/zswap: make sure each swapfile always have zswap rb-tree > > mm/zswap: split zswap rb-tree > > mm/zswap: reuse dstmem when decompress > > mm/zswap: change dstmem size to one page > > mm/zswap: refactor out __zswap_load() > > mm/zswap: cleanup zswap_load() > > mm/zswap: cleanup zswap_reclaim_entry() > > > > include/linux/zswap.h | 4 +- > > mm/swapfile.c | 10 ++- > > mm/zswap.c | 233 +++++++++++++++++++++---------------------= -------- > > 3 files changed, 106 insertions(+), 141 deletions(-) > > --- > > base-commit: 0f5f12ac05f36f117e793656c3f560625e927f1b > > change-id: 20231206-zswap-lock-optimize-06f45683b02b > > > > Best regards, > > -- > > Chengming Zhou