From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46CB9C4167B for ; Thu, 7 Dec 2023 18:15:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5DFF36B0072; Thu, 7 Dec 2023 13:15:48 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 58E646B0074; Thu, 7 Dec 2023 13:15:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 47D066B0075; Thu, 7 Dec 2023 13:15:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 362846B0072 for ; Thu, 7 Dec 2023 13:15:48 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id BFBB31C0674 for ; Thu, 7 Dec 2023 18:15:47 +0000 (UTC) X-FDA: 81540825534.14.F35FA40 Received: from mail-io1-f53.google.com (mail-io1-f53.google.com [209.85.166.53]) by imf29.hostedemail.com (Postfix) with ESMTP id E3A9A12002D for ; Thu, 7 Dec 2023 18:15:44 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="BoB6ftT/"; spf=pass (imf29.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.166.53 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701972944; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=d4OjwNy8mbYcl7XPkSSgPHOa+dg/ygA00wlk4vTz/nQ=; b=DEYMgYLWXcuohVIysxmibeZqERCGitQCTcXddRXpLAK2jUzDoIedA1KcI2kBT5U07ZW10U i0AOifLT/UY0jQcOfGEFl5Yl4BdvxvMMk7CEbmEyaxDI4S0FqUgfnu0Qhl3pvO0vrRFufG 2dl9lcSDYQg1kTXDdBPiJ+Ak+enRLrs= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701972944; a=rsa-sha256; cv=none; b=BuzUgmwU/YBcrhzVSLbO8W8hd08A7hVGcdxBdwD+9hXfLdMubpup3JvsYXBDXm5J8qmxT2 9b4kaznTTkjdqHVdrZwpistpnMBw1B57v2CFoZf+dJ3peOIGAwwB/+8GIUGFBimWiH0PwX hhuYq4g+0v1WUGU6zOR/0XLRruP+tlI= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="BoB6ftT/"; spf=pass (imf29.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.166.53 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-io1-f53.google.com with SMTP id ca18e2360f4ac-7b6f19f3af9so35258239f.3 for ; Thu, 07 Dec 2023 10:15:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701972944; x=1702577744; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=d4OjwNy8mbYcl7XPkSSgPHOa+dg/ygA00wlk4vTz/nQ=; b=BoB6ftT/6PGJ6bbzOMdgCnTOLjjZ9Qq4cyuO2u5bnovIipxUsdDhbhkMq2NnNHXrGO MOerXhHwGjVvF8vC4Wf+P1gUs06DzGKHhNJHO3ybJVPOEvyxuorToMY/t2Bn79oGnUFS XrzgaGnE+xvahefMAZmVAHrs8gQjyV0ulpbW/4TMgVoOZ50eGbQbmGivt5s6gXbxTEl+ 1syuTfk3HaNX6O4NK0rjC9VwEngVnFVgj9ynRi/QQA9V9kC1x/AVlIXjSkx1RuVl10sA L9rO/qVsFN4hFrsz13dM+fJt3j2qId9DvBcSKj5QHDuBwbtHAncdX4Em0/8JsYlqAHRP HJ+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701972944; x=1702577744; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=d4OjwNy8mbYcl7XPkSSgPHOa+dg/ygA00wlk4vTz/nQ=; b=Ii99DsVuAynZLsrY0FVVnAOO6nhqUVP/CL+rCubT9oBorbkicWKz9Fg/I43Xx2O//7 wTeAwgL6cOiqFywuGuZtJ4KU6cTr32KlwzETAGBLrcC6ygBXRy3PkLoHX5PAa35OwJkL RFOoHLPbbvUKhpb8lGd5h9CPyOegNXRoGMCPTVX9HGIeO2LIeeOIu62Q5bvEk5/jRC90 ikxGnwC0K+hD9Q5cKnJCpgzXnebvWXxbMb/6Wxpqb1v/KRxcXOER8zCeTmnGfFAKy9KY 8gQTNbfrtKWkFy/PGiidyURT/c/GHwNhqI+8ULZuG8ikGWVUVc21bLvsTI+mmblFa0gj 6+Dg== X-Gm-Message-State: AOJu0Yxb/HTsvUkagSyTZeYXXGXlvf3yaQ9fbjOYEH6/4AZnIGJyuwZp p2YcByR34Xbh9gscxSfHo4yxGRP0oZivn/+P2F8= X-Google-Smtp-Source: AGHT+IHvOAiE2ju1GBZ/CFHTHGpNGr5Vi2yEZRVmfREFCMD3Vl7ucKNaRzMspe0VPKbiMSg0BdH4+eOvpvJeIDGnMOU= X-Received: by 2002:a05:6602:3981:b0:79f:da3d:c47f with SMTP id bw1-20020a056602398100b0079fda3dc47fmr4069859iob.13.1701972943812; Thu, 07 Dec 2023 10:15:43 -0800 (PST) MIME-Version: 1.0 References: <20231206-zswap-lock-optimize-v1-0-e25b059f9c3a@bytedance.com> <6bf18314-7749-476e-aece-064ecfb0dffc@bytedance.com> In-Reply-To: <6bf18314-7749-476e-aece-064ecfb0dffc@bytedance.com> From: Nhat Pham Date: Thu, 7 Dec 2023 10:15:32 -0800 Message-ID: Subject: Re: [PATCH 0/7] mm/zswap: optimize the scalability of zswap rb-tree To: Chengming Zhou Cc: Vitaly Wool , Johannes Weiner , Michal Hocko , Seth Jennings , Dan Streetman , Andrew Morton , Yosry Ahmed , linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: E3A9A12002D X-Rspam-User: X-Stat-Signature: 77u884cwgxmeffwxnwd8hyerz4ygickc X-Rspamd-Server: rspam03 X-HE-Tag: 1701972944-479106 X-HE-Meta: U2FsdGVkX1+yQ5T0tKGAZzbS7gWdCcR4HnhhSdar82xTXy2PmQ8twzKnix50hwcuhjD39Ma9303PTZDbDRQhCssojvR4DWXd6MyA4bCW1QpLKkChm3Y4ZL70F/uxyPP/ZpYaHi/Fa6W0+zj2/nLoXM26kMxccdnih4NTxa94ZCSJXqAHHgEwkuGBuVWhzWGVCJSvtQG3KC2Zc95yQ2YG5gk10rbVf9kUMovr9uqLBf579j7qHlDDMOY3hmXndbPn2NOkpxDiyFsHkheLqRqLN3bNtSfT6Ra54RB/Fj6jwMaS9fXvKIMFzjqlBCYV+pfO46zk2jFRwxS91+Ug88IlqodK2nOHEu/cL8DT+4IhOjQJjXf5y1NPWbcYkiVju0noo4DZoNHI0g7aYK/MwGGXWyL7SWHjxv+fx1dk333uxqcOm23LpiZBmmrbx7KZaHa8GlPF7OUoqSJj4eluSWVABbdp7zhdo4cmY1+pvY4lzZJ+oFY1jddOunemY9VvBqBquTO7gdDgSaf2bXYlBCizRhziy8HzKNgnBa3peRAmRtYPd17TcmaRvhR3YIg208JanUuJzvB1hqwDSmmARShDhkEgDgUOjvHAM/EqCg3iHLIhb03dkU3kGOVBl6l9ZbXFAOSGKDGlYCXj1tylyEjPV1Dc6L59yTqCZpwWPg2RMdLK6q1i6nwuHtCWVY23w9JdLauxs89BAxqsWWmjTFGb0Gsx8KnjAwkLQ9bw9kd6eXbrnTyvsL10wjpCd4Xo9PuGe+MnUPkLFxrukqUpePs08anwXx5CzZEnBOXwkc2KxDc19XR+F0V2VbkvXXPKfDZuzLKIyGGKZeliTf9Y7JdToBZJXJ/F1pL5fW8hEjlL1gwm4892+A44rkSZzYx8x4lKGpNYqxo8iIm50PbZcNb+RgDKfJBU8M5myWTTpWHyeUSNmwDHWUfToVDBBkc9vXhnEp4YS0VaFyhMPpMH6+u RRuwhA/a OMDvDoAmodigKL/iwTXCq1MqrCsw4TbYI1UOkXapVH+YE4wIkNKALTzIWMFJ1lGZGb6uQBfoD6IKGDWlttBAOx29VsaoxL5i9dPVSZMvvvopVOIagztwD2V53QuJ76nJvMee0FLkaFN2jZUvcvgCdztTY5l4Wt9ok4hgmPH2bYZyo52PdaOOADlIhe6PVOSlHNjByecAphTNr4CELzQ9JpxBDbt5+cpK/sE9ebqziy4AiopyDfYFno7mxnhLVKoJMatbasQoNxNIfPXSYpwC1ya/ntGp8LJdjHItceTzpt4zHMQlBoaGF2RF0efyKdz4NAB+T0bwo+a915utb1JElQ7P5UX9AD1zTUclSrJWDgnZ+h7QmtGElGaJi+9PTnQwjP3/HylYR5WcuS7iUdJFNYZ/z5Ng7uT4mILVF X-Bogosity: Ham, tests=bogofilter, spamicity=0.000018, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Dec 7, 2023 at 7:18=E2=80=AFAM Chengming Zhou wrote: > > On 2023/12/7 11:13, Chengming Zhou wrote: > > On 2023/12/7 04:08, Nhat Pham wrote: > >> On Wed, Dec 6, 2023 at 1:46=E2=80=AFAM Chengming Zhou > >> wrote: > >>> When testing the zswap performance by using kernel build -j32 in a tm= pfs > >>> directory, I found the scalability of zswap rb-tree is not good, whic= h > >>> is protected by the only spinlock. That would cause heavy lock conten= tion > >>> if multiple tasks zswap_store/load concurrently. > >>> > >>> So a simple solution is to split the only one zswap rb-tree into mult= iple > >>> rb-trees, each corresponds to SWAP_ADDRESS_SPACE_PAGES (64M). This id= ea is > >>> from the commit 4b3ef9daa4fc ("mm/swap: split swap cache into 64MB tr= unks"). > >>> > >>> Although this method can't solve the spinlock contention completely, = it > >>> can mitigate much of that contention. > >> > >> By how much? Do you have any stats to estimate the amount of > >> contention and the reduction by this patch? > > > > Actually, I did some test using the linux-next 20231205 yesterday. > > > > Testcase: memory.max =3D 2G, zswap enabled, make -j32 in tmpfs. > > > > 20231205 +patchset > > 1. !shrinker_enabled: 156s 126s > > 2. shrinker_enabled: 79s 70s > > > > I think your zswap shrinker fix patch can solve !shrinker_enabled case. > > > > So will test again today using the new mm-unstable branch. > > > > Updated test data based on today's mm-unstable branch: > > mm-unstable +patchset > 1. !shrinker_enabled: 86s 74s > 2. shrinker_enabled: 63s 61s > > Shows much less optimization for the shrinker_enabled case, but still > much optimization for the !shrinker_enabled case. > > Thanks! I'm gonna assume this is build time since it makes the zswap shrinker look pretty good :) I think this just means some of the gains between this patchset and the zswap shrinker overlaps. But on the positive note: a) Both are complementary, i.e enable both (bottom right corner) gives us the best result. b) Each individual change improves the runtime. If you disable the shrinker, then this patch helps tremendously, so we're onto something. c) The !shrinker_enabled is no longer *too* bad - once again, thanks for noticing the regression and help me fix it! In fact, every cell improves compared to the last run. Woohoo!