From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4F9A1C48260 for ; Tue, 13 Feb 2024 14:31:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A93A16B0082; Tue, 13 Feb 2024 09:31:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A43C96B0083; Tue, 13 Feb 2024 09:31:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8E52C8D000E; Tue, 13 Feb 2024 09:31:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 7BFB06B0082 for ; Tue, 13 Feb 2024 09:31:26 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 4BDE180C12 for ; Tue, 13 Feb 2024 14:31:26 +0000 (UTC) X-FDA: 81787018572.16.8D1CDB2 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) by imf11.hostedemail.com (Postfix) with ESMTP id C808D4000F for ; Tue, 13 Feb 2024 14:31:22 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=a27GFt3Q; spf=pass (imf11.hostedemail.com: domain of zhouchengming@bytedance.com designates 209.85.214.175 as permitted sender) smtp.mailfrom=zhouchengming@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707834684; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=V99/mc0VYUGKp45C9Pun5G/w1J409z65OdhG0HwAmcs=; b=ohYmPdBtPjkFF+B7B0+sxYkALQcYXPTP0QX+2RqcSgePQfYo1zzNU33Zkx3x47XM9BK8qP bplQC/bG1D6r9uPaAC4RM4LQ/JN0vKP0AWDljq0crML3Ig3sbfKRzbr5D10k2QqzT0EfxB B0NHqSwptX1cU+GyO97WrgdYuhl+Dxo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707834684; a=rsa-sha256; cv=none; b=vvH4GsCGW/bnF8xsmnnSikXv7ofnNDAkVRCDfR2DOx7LpGcBoH9KEA2bB2yhcenXDngQzu 7Ku2XaWjg//NCQ0mTFADJBQImzCSh5yk2hd9H+ykWu9qSLqwSxH0siicF3FQuyvHe0n9qh idjWeqNslzBmM4Y7u8jetoNUX1PYd2c= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=a27GFt3Q; spf=pass (imf11.hostedemail.com: domain of zhouchengming@bytedance.com designates 209.85.214.175 as permitted sender) smtp.mailfrom=zhouchengming@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-1d51ba18e1bso43139845ad.0 for ; Tue, 13 Feb 2024 06:31:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1707834682; x=1708439482; darn=kvack.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=V99/mc0VYUGKp45C9Pun5G/w1J409z65OdhG0HwAmcs=; b=a27GFt3Qd/dAgwImBRfIvZuaOvZPzYyMAv0naurb3zsLRlw0Ubx1sTsoXhbwkPOYOo nHtMVSzW0sWTBqSAzTLkBJRu9MoZZsyO1L1bkd7xy1iCW3e66ZTrmuXdQH/GRFL0q/sa qyJKoVa6rXiBQSFq06mq4Pklq/1ZOr8pD63Xjz0rHlMWlXNMmyAEL8x1K1MZGa8HSvff AOCleNiVDX5MGDpC6ldlq3rcPT3c1p5M/0K2WKMuX5U0BNOTITb26W3ULdqph4Kg1ZCU ukZ0LU1a5SlA9ztUjhic3+d7WT6HQMsdTao+tbKVnFWwTMtumNHFqffd7ce5VyUCbZ5W UpHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707834682; x=1708439482; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=V99/mc0VYUGKp45C9Pun5G/w1J409z65OdhG0HwAmcs=; b=lRa4JXkOi9ap3d4qliI05SINH4EgoQaYB50nEPdmCUBR4CaJI+FtKjh3epUWcAAFnC weNqu+LiEAJfor5c2/UFmwoW2rAzQ4YSs4GUbTLy9X9Vp6idakBabv7ZjBTnHWrFVxEr Shfc4JiC3jA96UCmz2eD5oDTLJm31nY0T565zpQ2InMfbNbidm37YiMIPbcwTDX/Ka2w kjX6uD+kFPrtz04cXlJ60CLoOplu4Uggr2JkCgbQiyRBarca2QWax/ubByJ+HHUwasgr pT1VpT9+xGEIg7gHBueAGjEbmXRBaWlwlXN+1t2607m0cBlogJB9fpJDF0JsKsmJVuDN CuVw== X-Forwarded-Encrypted: i=1; AJvYcCXfL7SN5t7+lIAKWRNakjT9hVZuoDLN2JQWYvQQHUI/RuV6qRLOUfsZbsCRA+oo/f4r+aBK1YorUvmyyPG1mNfRK4o= X-Gm-Message-State: AOJu0YwTKcRlRrBNys7MVAs58QznC3Q48SuNSOC3QkyIfvcSmPpc68/I BB58Hbz3lmUoJ/AHy4KBBWXOK4xIhAsITOYpLte7SyJc/hISpz7SR7fOH3OzPi8= X-Google-Smtp-Source: AGHT+IGk1iguApZyRzMzCweyH30bvxFnhDWdmR2Rf3m7srwhrLrvHPd6ix3AaZNbU5t7d3JL7g25kA== X-Received: by 2002:a17:902:f809:b0:1d9:7729:2a6f with SMTP id ix9-20020a170902f80900b001d977292a6fmr9763587plb.61.1707834681660; Tue, 13 Feb 2024 06:31:21 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCV7rdMD6LftNZV6R/QH86Kr+89QTN4sVGMtpNwTroxeOtZggX2k/6hDDuLVDEcrjTr+mH1+NCHJsHTW9ZNWy+3RINXjtFbgG/0dFz7/DS1VaMoL8aBfwNJ5G2K89JDXNAvSDcojg3L4oV7F4ORkmwU7vDf9vPguXGTfP/CFRh8Pt1Ep01YW8Ng= Received: from [10.254.125.113] ([139.177.225.244]) by smtp.gmail.com with ESMTPSA id d9-20020a170902cec900b001da1fae8a73sm2166963plg.12.2024.02.13.06.31.18 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 13 Feb 2024 06:31:21 -0800 (PST) Message-ID: <1e5ffefa-8c80-44b8-986f-ee574c3b3349@bytedance.com> Date: Tue, 13 Feb 2024 22:31:16 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 2/2] mm/zswap: change zswap_pool kref to percpu_ref Content-Language: en-US To: Yosry Ahmed Cc: Andrew Morton , Johannes Weiner , Nhat Pham , linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <20240210-zswap-global-lru-v1-0-853473d7b0da@bytedance.com> <20240210-zswap-global-lru-v1-2-853473d7b0da@bytedance.com> From: Chengming Zhou In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Stat-Signature: fkjzjx3o7f7gr1rtk584p49bax6hsx3s X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: C808D4000F X-Rspam-User: X-HE-Tag: 1707834682-228471 X-HE-Meta: U2FsdGVkX19LfhDN+p90brLPSEXarvRh8eNXaNZ3cMCxXKkiWYPczKl6b6KgYZoI+wyuD7iy7Rf9Yhy6yfxQjq0NhosZqvgOawwHCiAcnyoAr9L37BffHhleRkrTOdagbuOYiiwkz14s43wSWzIt4H6Q0w0yZ74qN3i2f4hxCNLxWX3p6L6NeYH1NNGzpH/AhwHYfhGZ3CnJQK1BFV9kzgGCL4e/u9sfe1ZzXBw1FwWi7N0K9oBaPZDQt2jPgGTM49J/2Jq+4CjQrOZNpjTOJbdwC5rB7Wf38TdDYsybSf1Vv2MDFgRWFASLVy6FMElTvgKyZWpapEDEw/ILfNsL8+j4TMtO/JNVqH9YFtdLQA22cR7crTL9dAp3jaq2wdjrJYBu8DyVpRb+mDBWGdWg7b7v53E7Ux4IHmqcRL8qHYyIvdEe87CxQpLvQ1xMJ0SmDWc85mpi1rFivbg4dfZ+1eo3v028AV0T5MLIKgk4CP0c/OV/oCabcg9yVWR+v02Ydl19zIvMNC1IN5UtqOY7oXBPpy2otdXMiq5axhOfhBWC5f6VQ5wj7hscdbCk0fadTkPBvdsEoaPysDFUUHyi5bYDrjztuhI17ziBpPd89XiNYBW5Nb7WgHBWwYkhj6thAgdFsDc9KqENc4NmQdV3qSlXr+h7WDpnISIrHh+ZZLp+boLk52aFPs6lYcfMYSxRVNQQWl1YzwDcyjtPtuvGfiAiTLDbmbYJzO3RBQnjFftwnXwPdrEARFhPvoQmQsJZ5mdtkOWIPnEtmvUYdtw4gENqE9/UM4yaW2a+biS1/aECp4u/931lalvv6Dt748t7/jiZw9czQdMfQE5SZ7gYdhYI/LjT2hECo3EN4qGGHtW/OtM8AQBvWbfYGGyvcXm7FvCvaMy+ZM3koVXg6h2TkDlP5UfiKfRsi7Hr0tBfFSPbTwB8sAui4t1IjLGsqBS82nqYVHQRqroghKYgUz5 nvOGOvb7 6GXJut5Vn3I4NpxBJdCa3BWDYj/3cBGvTtgfP3Lthvp0R5VbP1wAMjEB+vw+Cf/Co8brWvAXEe2vyxYF2sp7liU13+ehuEWp85Fal7kt1SR50qqhJlyxZnTqblpyPDu2ur2gpIvZsYFBYMcwd3A01xX5SfOetwO78tnxPX61/rt0AHm0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024/2/13 06:42, Yosry Ahmed wrote: > On Sun, Feb 11, 2024 at 5:58 AM Chengming Zhou > wrote: >> >> All zswap entries will take a reference of zswap_pool when >> zswap_store(), and drop it when free. Change it to use the >> percpu_ref is better for scalability performance. >> >> Testing kernel build in tmpfs with memory.max=2GB >> (zswap shrinker and writeback enabled with one 50GB swapfile). >> >> mm-unstable zswap-global-lru >> real 63.20 63.12 >> user 1061.75 1062.95 >> sys 268.74 264.44 > > Are these numbers from a single run or the average of multiple runs? The average of 5 runs. And I just checked/compared each run result, the improvement is stable. So yes, it should be a real performance gain. > It just seems that the improvement is small, and percpu refcnt is > slightly less intuitive (and uses a bit more memory), so let's make > sure there is a real performance gain first. Right, percpu_ref use a bit more memory which should be ok for our use case, since we almost have only one zswap_pool to be using. The performance gain is for zswap_store/load hotpath. > > It would also be useful to mention how many threads/CPUs are being used here. My bad, the testing uses 32 threads on a 128 CPUs x86-64 machine. Thanks.