From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16706C47258 for ; Wed, 17 Jan 2024 23:41:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 917276B0089; Wed, 17 Jan 2024 18:41:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8A05B6B0092; Wed, 17 Jan 2024 18:41:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 740AF6B0093; Wed, 17 Jan 2024 18:41:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 5D74A6B0089 for ; Wed, 17 Jan 2024 18:41:54 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 2DD0B140239 for ; Wed, 17 Jan 2024 23:41:54 +0000 (UTC) X-FDA: 81690428148.27.F45EF6C Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf20.hostedemail.com (Postfix) with ESMTP id 4EB661C0008 for ; Wed, 17 Jan 2024 23:41:51 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=JVwTYqsN; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf20.hostedemail.com: domain of chrisl@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=chrisl@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705534911; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Nf1/s8K10TSDoTvcJ7yzFPVYZ5HobJbsF9cEHByGorE=; b=XIuDcdaBz3sfQZvwuPQexYRsiSYilcSrgc5LOrHasNFleBtXLbsvlRyd+f0qyLJJJ6uc/R SxyX/u2eACEBcHvjCnfV7yXnVSsx9kl/eWpR8XSR9xEYmj2UI7pGwvzxumvu8hUxsqsD8S 9qsMakGberIB3zmEkXVrSDfHDQK51nI= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=JVwTYqsN; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf20.hostedemail.com: domain of chrisl@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=chrisl@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705534911; a=rsa-sha256; cv=none; b=5VZCdCJLSJ7M9wW0DIkDVl2wJl+JHs/scFjqS0faWEo8QbO2qOXz0V/GjOlUr50hUOr1Cx kgq7otVMZfdFK7JC40BRiaPFnVlJzQxGqxd4yv0mKLgTYm8IbWQR8OLBfO5BAKfI4aVdvE pbgLWiqU/WXK78NYkEJZbxbbAr1yeig= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by ams.source.kernel.org (Postfix) with ESMTP id 55206B815DE for ; Wed, 17 Jan 2024 23:41:49 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 97EBBC43394 for ; Wed, 17 Jan 2024 23:41:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1705534908; bh=cY79/Cfp9xIflNIDwfUmZ2vm5pn0cwkYXSHLUwKaasQ=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=JVwTYqsNhQmI0YoHxpkxpGD41oMoBmdB5gB6csq63I1W8k3eO4j97Q59XcVo+y3J9 p06o/RNx3V4rGt6wYHkHMCFqD+1MtadTpTH7CstvD8mBnOYGppldPi1rDre7hoyooH xOHFM4HZqRT+olNjR75ncyorWJtChHNy0rGjmjmaxi7SAXs0XTWGWo/WPITTk++I00 3xcKhb6zFSguSfcNla8ajfSCKMuSzkBd9VB76r2idmf9jpOs1PeLjD0yPopxBBmxa+ 7jSwy7dIgDNhcZrt1InB7Eoq5PqzFjnOkMaauBmU431Nmw8xwuPRQa74xZTVAyLjtt DQeEXKfajnkzw== Received: by mail-pj1-f44.google.com with SMTP id 98e67ed59e1d1-2900c648b8bso573328a91.3 for ; Wed, 17 Jan 2024 15:41:48 -0800 (PST) X-Gm-Message-State: AOJu0Yw8YcqYdrQaUHKTn6NJlfGkdVWvJ4e93IipjyA6p0SmiXgn7Xxt OR9pZHLKRu2Mhz9RlruQO719P+LCewfR83nriUKfPL6hZc1kpV8wX57Yd7EzJdbiAqhiYG5pdRa 7b1gxRZJmbiyTBO3p69w1ag3ecQksQOmZVXEF X-Google-Smtp-Source: AGHT+IFs7Y9kVpqYyUhwzuMYD6tF9HAEqcY8re5F/bDLJ7v+vAmuGFeD34dRqhpQ93irvy0QmLM3iQHxGmMhGiTY8fI= X-Received: by 2002:a17:90a:12ca:b0:28e:89d2:87ba with SMTP id b10-20020a17090a12ca00b0028e89d287bamr43131pjg.12.1705534907954; Wed, 17 Jan 2024 15:41:47 -0800 (PST) MIME-Version: 1.0 References: <20240117-b4-zswap-lock-optimize-v1-0-23f6effe5775@bytedance.com> In-Reply-To: From: Chris Li Date: Wed, 17 Jan 2024 15:41:35 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH 0/2] mm/zswap: optimize the scalability of zswap rb-tree To: Yosry Ahmed Cc: Chengming Zhou , Andrew Morton , linux-kernel@vger.kernel.org, Johannes Weiner , linux-mm@kvack.org, Nhat Pham Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 4EB661C0008 X-Stat-Signature: fnugihemax6wmik963rzf5ng8us6bijz X-Rspam-User: X-HE-Tag: 1705534911-941937 X-HE-Meta: U2FsdGVkX19RDfP0u3uX952KhotwWd+337AbO7we12+hJoNVNGF63CSCRk+0GRZzuFVqz1Y5aqAp1weSLL+w7rmMHLkfWv7DXNyBKwo9w96yo3jhqvYX0VH6T6TyAAA7Ql1Nj+ZBsugSOHJV5Iw9tkaPZy8MnhhUaWXlDNH9ufExjZ6lIxRk9AT4F3TpUSptMTmoAFQ8wtNZYXS8WaJhN04OfLjz1zbjwu7tbrGcWz0rTXFH3qm129HdNeBjkyGQbducyU4g1+OlKnh7OFLFiP/W6b3yhG9fBj0gmp66PPWHZRwezd1MgBnwXAZfsulmxqrQGTAXOTZUATmItANcOT7BYlZKpxYcPu5/+1vyeMFCq7iIcSJ5MKt4Bpv6XbvwrqQ5x3oRuXcb/MvLKan8K2aWGmQhSKYZMzxOxtqhUE4to7EI831EbuE9wuf4/y0SOoNVAYcK+7ZIWe0DbSw5zrZmZoujleVIUJeNqz25GXV94kOFjVQZst0DUgPg3OkoHvqU/FIa8hXPYwRz6qt7jROs0L2Cl05fPAjw2Kj4Xa2bTJYlNHtYEFrKdbIWkslPkLJJYsrn1tmSypD80DIcJ7L3NBS5taYOX3T6MeDscPnMnEIoyOEy1/mMaCn3c9xMEf7GWSGtRnezVrEKsLyXMc0/rAudF/+w7zfR6iDUAfm5RIwgZN0KYDMf580IZrh1PDApZlR6Gczcax4RKuAOtFiJ74jB6V6X5IaI+p2sfoZFf2YZ+BepRdWdHkB67EcCxsN5xUPSO+nan9HZbjF4toecBAftgtnVy5f5+ILecg4etV9n03NGT/1bi4AXucqi/EJHL6s0ZEbyEt10iYglMUGNBCtsoFNUW5SCcpaLNpBVq73HhGDBbOUEw3b3wUuNQxd9hUmoGh/MW5PWibY4267bnBnAF2/g X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi Yosry and Chengming, On Wed, Jan 17, 2024 at 10:38=E2=80=AFAM Yosry Ahmed wrote: > > On Wed, Jan 17, 2024 at 1:23=E2=80=AFAM Chengming Zhou > wrote: > > > > When testing the zswap performance by using kernel build -j32 in a tmpf= s > > directory, I found the scalability of zswap rb-tree is not good, which > > is protected by the only spinlock. That would cause heavy lock contenti= on > > if multiple tasks zswap_store/load concurrently. > > > > So a simple solution is to split the only one zswap rb-tree into multip= le > > rb-trees, each corresponds to SWAP_ADDRESS_SPACE_PAGES (64M). This idea= is > > from the commit 4b3ef9daa4fc ("mm/swap: split swap cache into 64MB trun= ks"). > > > > Although this method can't solve the spinlock contention completely, it > > can mitigate much of that contention. Below is the results of kernel bu= ild > > in tmpfs with zswap shrinker enabled: > > > > linux-next zswap-lock-optimize > > real 1m9.181s 1m3.820s > > user 17m44.036s 17m40.100s > > sys 7m37.297s 4m54.622s > > > > So there are clearly improvements. And it's complementary with the ongo= ing > > zswap xarray conversion by Chris. Anyway, I think we can also merge thi= s > > first, it's complementary IMHO. So I just refresh and resend this for > > further discussion. > Sorry I have been radio silent busying on a few refreshments of the xarray on the recent kernel tree. There is an assertion triggered on xarray and the rb tree does not agree with each other. It takes some time to debug. I ironed that out, also glad the assert did catch a bug. Currently the xarray patch should have everything it takes to use RCU read lock. However taking out the tree spinlock is more work than previously. If we are going to remove the tree spinlock, I think we should revert back to doing a zswap tree lookup and return the zswap entry with reference increased. The tree mapping can still decouple from the zswap entry reference count drop to zero. Anyway, my V1 of the xarray patch will not include removing the tree spinlock. > The reason why I think we should wait for the xarray patch(es) is > there is a chance we may see less improvements from splitting the tree > if it was an xarray. If we merge this series first, there is no way to > know. > > Chris, do you intend to send the xarray patch(es) anytime soon? Thanks for the heads up. Let me send it out now. Chris