From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0D618C10F05 for ; Wed, 6 Dec 2023 17:24:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7709D6B007B; Wed, 6 Dec 2023 12:24:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 720F06B0080; Wed, 6 Dec 2023 12:24:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6365A6B0081; Wed, 6 Dec 2023 12:24:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 539076B007B for ; Wed, 6 Dec 2023 12:24:46 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 2327440159 for ; Wed, 6 Dec 2023 17:24:46 +0000 (UTC) X-FDA: 81537068172.12.7369945 Received: from mail-io1-f52.google.com (mail-io1-f52.google.com [209.85.166.52]) by imf15.hostedemail.com (Postfix) with ESMTP id 5C68FA0007 for ; Wed, 6 Dec 2023 17:24:43 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="F/F+NReW"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf15.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.166.52 as permitted sender) smtp.mailfrom=nphamcs@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701883483; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YFtw+JoIilNzXGrK1+eC18U9vk6uZG0kOBjgadIHka0=; b=h3mIicViIElShvBDEzuyl6xrm4miSN69rNZCSqcggW0BkSWZpF80lSz7R1Djm1U33qgKFB e7rDPPNvVpAvJIH3mDBxiyFtYjArLBdhgp3vR0XLtW3Icm+osKay0Ue6Pss5ZmIfxyB8S7 yPZKMpsgSHg5yels/wk+1n5E8kBzVoE= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="F/F+NReW"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf15.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.166.52 as permitted sender) smtp.mailfrom=nphamcs@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701883483; a=rsa-sha256; cv=none; b=sqEiE5mKsyQRlzRtDzgclwP3sf3/O0eHfViGO6Gnt20KGzWa4n3s/6P/QrZdccwcJ5CUUK jNBeRurFKF5LwBRZguBsr8KvDjqSzqEcZEPiRg7GWIdsbCHuEstc80aiBRhxtigeegWXLn rAwqx8QTZw5byrEsXx/B2OaLOjc+BXw= Received: by mail-io1-f52.google.com with SMTP id ca18e2360f4ac-7b389399dfdso377633239f.2 for ; Wed, 06 Dec 2023 09:24:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701883482; x=1702488282; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=YFtw+JoIilNzXGrK1+eC18U9vk6uZG0kOBjgadIHka0=; b=F/F+NReWcMQmp+4jBcYeNyMnIvr85AftQaGUkf6kzhkkUfvlwtRYKTdJoeanCXpePP 89treADA7ws0faIQiqygJ5urRBVywepAlxJteZ0euauXlzLo4d4LTZ7gDRYRdYz15dOX bfu8yTfIiBV4pDvyDIuGY77jYVnPFQzMpWLVdDKyeh+IoXUtsGDt2iyetMfSaLYrkLBO vjnzYhl8hoRoOV/lKZDp2/8Jr9iFgi+o002mdsG1gx4XhAtMmYl63/r5uht2z9r8coWt jKYdat5W1zwKviqg/5nuUYaXeRSHiELxtQ+bqDWKbayQS9RSfeh8/n7gy79KuuJ473TU zYyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701883482; x=1702488282; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YFtw+JoIilNzXGrK1+eC18U9vk6uZG0kOBjgadIHka0=; b=LB3ByBSJLhUC6ZWTbNkwviNZp+S0V90OhuBvawmHP344xWK1HUcPG0C+/crpAWhPBf TJ179ymzJHsE3ixntnxY6+YBH0FOF/HaeIC/Zl3bBAqUxUsAMECZv7H6o1qBt6K2TAL3 IlVFu4fMcZhFrhH2djfU9z3bqANA8ZjerzF6SnG2ZdN4GlutNM19S772JKfihQCEhaGJ w+juk314yaQliexRWJuqBQd7jgCy0qB0skDI4ts/Qw9/G6Mi6AOUpcKkxL/qgccnkaX7 PoU+xfhWLWxhnAw9f7fPjQDNkxesG/qgbEdmvV8lLF0dtuDjxD3sOF7pfnqwM/FfxaxM 7JFQ== X-Gm-Message-State: AOJu0YzlahinJgYot6yHCS7sn0zwGTJiBKf/X1xAK1B3FM46VmK38uIE qXIYZn5KmNN4ZmJ1cU1GQ8e5O6jyYLVTIb5Y1N4= X-Google-Smtp-Source: AGHT+IH7S9uNub8N0q1SiSQSHC7lCi8nYYBBtFWhavsWZKjHB+uJudW8tQXA6tPZhyR9gH09WSBlAsXlpppM4xUVtFA= X-Received: by 2002:a05:6602:29ca:b0:7b3:95d6:62df with SMTP id z10-20020a05660229ca00b007b395d662dfmr1830560ioq.21.1701883482412; Wed, 06 Dec 2023 09:24:42 -0800 (PST) MIME-Version: 1.0 References: <20231206-zswap-lock-optimize-v1-0-e25b059f9c3a@bytedance.com> In-Reply-To: <20231206-zswap-lock-optimize-v1-0-e25b059f9c3a@bytedance.com> From: Nhat Pham Date: Wed, 6 Dec 2023 09:24:31 -0800 Message-ID: Subject: Re: [PATCH 0/7] mm/zswap: optimize the scalability of zswap rb-tree To: Chengming Zhou Cc: Vitaly Wool , Johannes Weiner , Michal Hocko , Seth Jennings , Dan Streetman , Andrew Morton , Yosry Ahmed , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Chris Li Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 5C68FA0007 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: zgqq59n7abxad4wk3g5896dc1i8rz36b X-HE-Tag: 1701883483-264910 X-HE-Meta: U2FsdGVkX18r2xlFIFgaPtjj7OgK8V5Vj2GJdFSEHhm+PupMZoc4UTjejCgE8qetlVUFgxHFt5iQIfhmR6yrO/e7EhddHPprTGRH6a73ZJm4gA4Zd8zWuG/rXb/AXVQvSCTZLUN/Hdkxq1mxmFGrwkiGi8jAFTVy16H/sEjI8ISEIiwZkS+1UmzPszbRdK7uhNf8ktJCGzqw7iEvBnDRUEHYSQ80yUWPokLNuHBxwGA8exB3VT6eLpbnDtNUu0gkzeiBCO1X6EnzpA8f4EjoJABhjiPAwgcd+RZA4P+fBCL3CeN/EXX86ls01ic/OdTVIgghlH9KWXYQVXNGOVKnOJ/T1nqn849UM5Qnv+FzmfLt+UMfjvietj0HXFHKhJIXBStzEvFiZM3BnFhHF68r/gSkvRTY/O/jB9HWBWjsM9PQiBo1SpuxskYQHpvsbmxYStRGfBfPPAhfvlp3PCsJBmRF9z9PpJAMV7Rgf9ZKmbbQayWrhWc2E8XEJMTuzGmZRGDwLDuIGy7mE1ffqW3ZkKhyKEQZRp7Cm+h8IYE7KRcu8BBEhj4iwx15xBDPwze5by3OXYDj/eTImfdI59VJuUEfWhI/b6/+lIpp7ecvYzUeMFHBMJnYaqGuQzX6eAh6i8JGTnkfKDdioX4Ts2yPrlk3DaxZl+NxEbMs3UGzvMKh5mAUuw0PCarwGzTRPkdy1WG7zKOz4PXThT3SfL4Zscci0weXI/CULy8IXGZYDSP4o0eBM27JjmhSS8GnJq9v/DV6500HvTB8AYU4nhlGQn8nuVTuoNfLZ6HIabBHGOx5EGElQFKIB4hTtX6yGcu18HK33zJdyk0UD+lp5+sc80sREOvt84FnW9BS5rrbrrtpUzl1Nqe5uFZpRZlyxxPcNyrBOKxt41dJzFsqDYpRjT9VORjT1GRcLZBckCgWN+oPLixcePDv+zqzlgytDVWT4CM6q6mjuHmk+9gFKQg jlkOEzNT fpfeM9URthhUXucblBcK0LLlAuN9A8dhT8rMtjEav+DyG/41S0pItxVjGxhTnu34sErYIfq6ieCWATKel0lfrAWHZucmR2tRcyRh+u21slzWyVGNMkCqyKYdg/+qyAHhkI9QWchUFqDRAewD/xLkvF14bsDDf+lJHb0KHyEnwwDQVmiYmW1AdNQrvVeLZMo26PCSI9AMsMm0x9rFUFKPETrWtLc+j6IeQHHgmKOhywKYefhxfhJu38AaH3rYa+u9WwF3Uhd/djbCefU/UZa6pTUBg2pVm8p83yLup0ztyPlwEMa/+S31X99QLRkEZQ2O/9AXTf6vfPBJiqC3d2pcJl8U5Mnb9BI+yO71xV1yrYsrhNngib8g2RYHWp+KZyRb76ilmVbKf+uRQtsflfgGWg9cDb0irGikrjPDlT2S0OM83/jZZTkcOcPl+TbsFgP2/ERaFZQeHqDaO62kU4zuH3YKGtKhmWdcO4bEQXKJwNb1GunJlqbHJQHDkSKyak3jsfX1Pgz83Ubhel/NXG+KP401l3eTvxWPClUMg40kGH6bEc+NlgvlHwh9gA+1vvLOFwu10Di1TrQkZZGercsfCD35pEMzpWKZXZM3pa+Ih79TLEo64pV+PXnK9deBXASxZbLZ72mJ0CFAd6CZwYf8IE8YVPZLljOej4Y9r8Vbu98+fCEdhgmXiOOFdlY4C5fiJwGsH5UyaKxkP9ohLrBQAhNOOnP1PvXLf/lH6y3YGxofC20hNEhdgR4KFgUjqUPlDywciP8iUQ/AaCUGMRi1ItwzROY53yKsvoUJw73qcZkXEC5U= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: + Chris Li Chris, I vaguely remember from our last conversation that you have some concurrent efforts to use xarray here right? On Wed, Dec 6, 2023 at 1:46=E2=80=AFAM Chengming Zhou wrote: > > Hi everyone, > > This patch series is based on the linux-next 20231205, which depends on > the "workload-specific and memory pressure-driven zswap writeback" series > from Nhat Pham. > > When testing the zswap performance by using kernel build -j32 in a tmpfs > directory, I found the scalability of zswap rb-tree is not good, which > is protected by the only spinlock. That would cause heavy lock contention > if multiple tasks zswap_store/load concurrently. > > So a simple solution is to split the only one zswap rb-tree into multiple > rb-trees, each corresponds to SWAP_ADDRESS_SPACE_PAGES (64M). This idea i= s > from the commit 4b3ef9daa4fc ("mm/swap: split swap cache into 64MB trunks= "). > > Although this method can't solve the spinlock contention completely, it > can mitigate much of that contention. > > Another problem when testing the zswap using our default zsmalloc is that > zswap_load() and zswap_writeback_entry() have to malloc a temporary memor= y > to support !zpool_can_sleep_mapped(). > > Optimize it by reusing the percpu crypto_acomp_ctx->dstmem, which is also > used by zswap_store() and protected by the same percpu crypto_acomp_ctx->= mutex. > > Thanks for review and comment! > > To: Andrew Morton > To: Seth Jennings > To: Dan Streetman > To: Vitaly Wool > To: Nhat Pham > To: Johannes Weiner > To: Yosry Ahmed > To: Michal Hocko > Cc: linux-kernel@vger.kernel.org > Cc: linux-mm@kvack.org > Signed-off-by: Chengming Zhou > > --- > Chengming Zhou (7): > mm/zswap: make sure each swapfile always have zswap rb-tree > mm/zswap: split zswap rb-tree > mm/zswap: reuse dstmem when decompress > mm/zswap: change dstmem size to one page > mm/zswap: refactor out __zswap_load() > mm/zswap: cleanup zswap_load() > mm/zswap: cleanup zswap_reclaim_entry() > > include/linux/zswap.h | 4 +- > mm/swapfile.c | 10 ++- > mm/zswap.c | 233 +++++++++++++++++++++-----------------------= ------ > 3 files changed, 106 insertions(+), 141 deletions(-) > --- > base-commit: 0f5f12ac05f36f117e793656c3f560625e927f1b > change-id: 20231206-zswap-lock-optimize-06f45683b02b > > Best regards, > -- > Chengming Zhou