From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7A126C4828F for ; Fri, 2 Feb 2024 22:31:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1C0896B00B2; Fri, 2 Feb 2024 17:31:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 171466B00B4; Fri, 2 Feb 2024 17:31:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 038FC6B00B5; Fri, 2 Feb 2024 17:31:44 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id E8BAB6B00B2 for ; Fri, 2 Feb 2024 17:31:44 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id B925EA26FB for ; Fri, 2 Feb 2024 22:31:44 +0000 (UTC) X-FDA: 81748312128.16.085B5CD Received: from mail-io1-f47.google.com (mail-io1-f47.google.com [209.85.166.47]) by imf21.hostedemail.com (Postfix) with ESMTP id 059831C0016 for ; Fri, 2 Feb 2024 22:31:42 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=LJVBKqus; spf=pass (imf21.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.166.47 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706913103; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UIG1a/joWmOzqMF2AOTCZUCUHJV5S4v0T7lXHSi+5hw=; b=tyiw93+7WV+cgM/vrWIE6J3sruUBVwRpl5fCkRHdN3qZwwsAyt97vrCDWYq75BsPXG0yVB 2FqOVjOm7eroL0w7Xsu/a6ypaOgZ9+g/zLnY9uO1AEawzArVI6YTXoXDoZdn+FUVdE7LeB hLHcCkE97M68+3Qh1Vuzy0p0l/R/Fec= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706913103; a=rsa-sha256; cv=none; b=yEuAoOpujx1jVqqQkyihrE9BMFqudX3x5OMz47/5CJ/FfSNsjIVY3DjL8mBWFWAeJMv9TT OMaFkjPPXcrkBWD3Wk2zTy+i4ilEJ7soZXSu4j6K+/hLjbIhe0YUC5Qg4ntaoRrNNmuht/ JrY6luTB/SqwRR2panDE2wGZpbSQ/B0= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=LJVBKqus; spf=pass (imf21.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.166.47 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-io1-f47.google.com with SMTP id ca18e2360f4ac-7bed8fee278so124304539f.2 for ; Fri, 02 Feb 2024 14:31:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706913102; x=1707517902; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=UIG1a/joWmOzqMF2AOTCZUCUHJV5S4v0T7lXHSi+5hw=; b=LJVBKqusRKnCXZJ/bZwVjfyE+OAwo4J9NmKekoW657OSXi2xBVqIb0Y5DOUgse1cTF AkKUCnYiGGRT2LyEypAc2sx/g+6XcJEQHpnCliQwTFLTuN3qC+opjRRvi8lYSAnGlL3p /mG/LcfDl7sOwmPfoQB4AdnhWtTMZc/+w4hlLyWvEtCoINhtoUPPp5BbuL6qU++RM5Ks ogWSNWZsXO+UnUvcEVEi6Vjll1y0RhpmOAaG4y89Xlgw8zZ8mTgPjOo8V63AlT10x6mL yOIM8HGSVI/Fe7ScW5tzMG2U2zHfrg5aiq7oCqwYeXr3pU+qtGt/pp4iER7DsKBpXAMs NB6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706913102; x=1707517902; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UIG1a/joWmOzqMF2AOTCZUCUHJV5S4v0T7lXHSi+5hw=; b=YRjvWt2J9AnqA+NNmK8amltzOZxARqYzmv4W27Al0BBTpE24VDXI7aVBPZlGPWK+1W +9s4t6XUG8J5ApVfA9ztNejS1A7w/fOl/jsFVvPstj8jGdFxsgZSJHN4xLIZtKvf6RwM ByQk6PVMTEOwv7mbeXt8eHEoyNYG7Gf+i4B38TYeZr+szaeJv+9FX8ys3YYiBhx0j6Kw vX4sPvO0Skoh/VdZa/AyyMxz3hEcDpYR+nFkZRBlGI7WFc/pVsW1DZgbw/jliURt66Nu fX/FPL5d4MTRQAZAFoW47h5VzzqGKZBh+wKYlNDMx2rj9JHCzPhJrYidgTCME38WsqON Zj+Q== X-Gm-Message-State: AOJu0YxSBJF7SEzYobQvKw7ilQLPVsGDAUE0/8+J4hPXWhMCdjf+Eb+m ubTh1DzwGmmu8rqABXElFEK9MK0XHG1xIsgpaa3IU47HbW/TIQ4I3XbRnZKARmkHc5tBuvIu+Pt M9HXTwk0cin8XQq6Go4KHEdRa6ZuEgKPnn/I= X-Google-Smtp-Source: AGHT+IHvT5NUDHpd8oCLZ/al8SEZ5XnhRxs3msvI799Rlu1Ndk2dQd2H1kH0r2JL/ZFf/WgVJt4nbg6f7xIzwvgaosY= X-Received: by 2002:a5e:da0a:0:b0:7c0:a42:9ce2 with SMTP id x10-20020a5eda0a000000b007c00a429ce2mr4003612ioj.8.1706913102094; Fri, 02 Feb 2024 14:31:42 -0800 (PST) MIME-Version: 1.0 References: <20240201-b4-zswap-invalidate-entry-v1-0-56ed496b6e55@bytedance.com> <20240201-b4-zswap-invalidate-entry-v1-5-56ed496b6e55@bytedance.com> <20240201181240.GE321148@cmpxchg.org> In-Reply-To: From: Nhat Pham Date: Fri, 2 Feb 2024 14:31:31 -0800 Message-ID: Subject: Re: [PATCH 5/6] mm/zswap: only support zswap_exclusive_loads_enabled To: Chengming Zhou Cc: Johannes Weiner , Andrew Morton , Yosry Ahmed , linux-kernel@vger.kernel.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 059831C0016 X-Rspam-User: X-Stat-Signature: qqccuq7477oc3ogwg7snmd4gbgfxefem X-Rspamd-Server: rspam03 X-HE-Tag: 1706913102-208946 X-HE-Meta: U2FsdGVkX1/qditK7crQgtrgrv03BsXNyameOJAkt1Yn9SmZpHLD5oZUIeDfPhri0NxoKLHgVYuMOPofEzLYvPtzrSlsmTTfnrsAiJnjzOHIbkFFilZl4MS5qxQaEv7llxyr6wk7yAGJTB6qCalP2aQsgHIQPcoWT1K6ieWNz185KWQYHRCGol9sKQjSSo2DFc9dbkbvd5GcNt14U0WjeHly6+gP5KJg4X1BRSSX9ZGxnIsm/rt52+wG+bsfkFAA0SXAFJsSX1BgOg39eCagBIAyU08Q0FmcwqyEiomsYHppUK8vbRJ/lBKuhzoBEG8wPu6y0sJMpgd9/oem1hXDFRN2qG2CyjMONiM/Z+UI1rK0uGAZXE3eEW/vLiVBRn7NrU4IIaevU109sX6hk6G5tkmv8kiDJYQqwJHBpNUO2+66sBgLh+vr34WVWcGdv5Kgczi/POlYV0u2N6xd3o+w5nsZK1Pdc5rU7PapTLZO7CcYHZZNYpS4u+ztilAjDwHe/pHASPe3z/PToF5eHvaopPHeWRoKzqxJm3rHeaLekdHJ8XHaBJKgA8x37B4KGtx2sVsb/X+4pupSXV0Zf0M9fivp5Zy5jYQ2eNdey+KnBZxG0PnsDw+ssIUFBq98NLbM1EY/FXMZWclV6oqO14pL83ybVWiAcoycRzOCkjW6UrvXNLm++/HfQVa8fursquSwhFK3Odym9uAvcuRpsXFtcCQIfZxi6GsChlvuKBVumvNdFyJybqJaEohffmKFTGVHYJFweocz40KAIUb355KL5n1ZiCdeYx74FDtBlQ8a8W+ytNpod8Vj3Qx+0zmSCa1D7OqN3nkGKfucJl2CjD+S1PXKXFVF65Pnad7iyHXRbIB1irQ8F3BiP/HuAq/o/pAf/07llP3xrAd62//t2I3gjHsfNjFrdd6OpU/sxLmJNYMXvdIVSzZolhYJIfYHuFuAz3HEdrmhcpZlDZMMIte xNB+Gnaq 06m+/3IXtz1ZwC/r5h0cxFVmMtV1tGTqNsxDJwcr/8JByBqL8a4tpp/1Ly/4nXxtGg9P+kYiHQBq4p+E/MeOhN/9O4va3AVpmvn3vkhAVNGkO89lBRzN+oNcU2GBbFhwQIfjsreYZf9uPm0YJNVW9Gccp2YtKxsYqpqtivpUQC2fO4BuRpaLWCBqYj1vry3Wn4NYoYxKE5VIuxMz0GTR8vR5/wpXYjvwDprfBBYbhwCZhtmMWHEnvoLd5cMCYu7pLqUabXB57d/tNWLNWpQLdwbN7VR7VAu8fyJP0w6Ss3oGT/60Ij8OQ5IJCkBbydxfZuBPEuY5edF93olMbsdwvSIZojTeaJp9zbA81xL9xe6UvvGmN253xfw7fKpn+RfSPzbUQR7eGLc0HY4c= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000046, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Feb 2, 2024 at 4:57=E2=80=AFAM Chengming Zhou wrote: > > On 2024/2/2 02:12, Johannes Weiner wrote: > > On Thu, Feb 01, 2024 at 03:49:05PM +0000, Chengming Zhou wrote: > >> The !zswap_exclusive_loads_enabled mode will leave compressed copy in > >> the zswap tree and lru list after the folio swapin. > >> > >> There are some disadvantages in this mode: > >> 1. It's a waste of memory since there are two copies of data, one is > >> folio, the other one is compressed data in zswap. And it's unlikely > >> the compressed data is useful in the near future. > >> > >> 2. If that folio is dirtied, the compressed data must be not useful, > >> but we don't know and don't invalidate the trashy memory in zswap. > >> > >> 3. It's not reclaimable from zswap shrinker since zswap_writeback_entr= y() > >> will always return -EEXIST and terminate the shrinking process. > >> > >> On the other hand, the only downside of zswap_exclusive_loads_enabled > >> is a little more cpu usage/latency when compression, and the same if > >> the folio is removed from swapcache or dirtied. > >> > >> Not sure if we should accept the above disadvantages in the case of > >> !zswap_exclusive_loads_enabled, so send this out for disscusion. > >> > >> Signed-off-by: Chengming Zhou > > > > This is interesting. > > > > First, I will say that I never liked this config option, because it's > > nearly impossible for a user to answer this question. Much better to > > just pick a reasonable default. > > Agree. > > > > > What should the default be? > > > > Caching "swapout work" is helpful when the system is thrashing. Then > > recently swapped in pages might get swapped out again very soon. It > > certainly makes sense with conventional swap, because keeping a clean > > copy on the disk saves IO work and doesn't cost any additional memory. > > > > But with zswap, it's different. It saves some compression work on a > > thrashing page. But the act of keeping compressed memory contributes > > to a higher rate of thrashing. And that can cause IO in other places > > like zswap writeback and file memory. > > > > It would be useful to have an A/B test to confirm that not caching is > > better. Can you run your test with and without keeping the cache, and > > in addition to the timings also compare the deltas for pgscan_anon, > > pgscan_file, workingset_refault_anon, workingset_refault_file? > > I just A/B test kernel building in tmpfs directory, memory.max=3D2GB. > (zswap writeback enabled and shrinker_enabled, one 50GB swapfile) > > From the below results, exclusive mode has fewer scan and refault. > > zswap-invalidate-entry zswap-invalid= ate-entry-exclusive > real 63.80 63.01 > user 1063.83 1061.32 > sys 290.31 266.15 > zswap-invalidate-entry zswap-invalid= ate-entry-exclusive This is one of those cases where something might make sense conceptually, but does not pan out in practice. Removing non-invalidate seems to simplify the code a bit, and that's one less thing to worry about for users, so I like this :) Reviewed-by: Nhat Pham > workingset_refault_anon 2383084.40 1976397.40 > workingset_refault_file 44134.00 45689.40 > workingset_activate_anon 837878.00 728441.20 > workingset_activate_file 4710.00 4085.20 > workingset_restore_anon 732622.60 639428.40 > workingset_restore_file 1007.00 926.80 > workingset_nodereclaim 0.00 0.00 > pgscan 14343003.40 12409570.20 > pgscan_kswapd 0.00 0.00 > pgscan_direct 14343003.40 12409570.20 > pgscan_khugepaged 0.00 0.00