From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01326C4828F for ; Fri, 2 Feb 2024 12:57:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 86B826B0083; Fri, 2 Feb 2024 07:57:48 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 81BFE6B0087; Fri, 2 Feb 2024 07:57:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6E42C6B0088; Fri, 2 Feb 2024 07:57:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 5DDA46B0083 for ; Fri, 2 Feb 2024 07:57:48 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 17A6C120EDE for ; Fri, 2 Feb 2024 12:57:48 +0000 (UTC) X-FDA: 81746865816.19.675E6AB Received: from mail-pl1-f176.google.com (mail-pl1-f176.google.com [209.85.214.176]) by imf12.hostedemail.com (Postfix) with ESMTP id 3AF614001B for ; Fri, 2 Feb 2024 12:57:45 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=PHlGEYdc; spf=pass (imf12.hostedemail.com: domain of zhouchengming@bytedance.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=zhouchengming@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706878666; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dsW8Dk1o9bsbV3Vb7IbQbusPN3ry+IgXo47UpIKTbrA=; b=cze5cm1DfY2W+rVLdFWasQvIp+CO/c9FhzzhVTaEoKtwvIU9TnpXkLgkkksr02bAbpVQhU YXd0LVWSzEhKQUmXckZqOo7N1Xu916biUwwHRtJY78+QRUgHk/nkUGmD+E+eOGMTEiw8my L0svahQLo81yKh/5F1KA3EEPCWsBWdA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706878666; a=rsa-sha256; cv=none; b=0Ica1X9Qo5O3YQ++Du6Xl7a1y19xKOkcbjoOWrnyOWJ6bixlR73bADFFfCW4GUfknq8wwK goLuc4n4Nfgph13wXPMl5QzSxz/VW4XRqoJRK0xu9NrIDVYf7j2p+JqZp/ObEloXcTHI/T j9w8eCNd8b1Ebnx2q13RVcORhhKDZKs= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=PHlGEYdc; spf=pass (imf12.hostedemail.com: domain of zhouchengming@bytedance.com designates 209.85.214.176 as permitted sender) smtp.mailfrom=zhouchengming@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com Received: by mail-pl1-f176.google.com with SMTP id d9443c01a7336-1d918008b99so16328975ad.3 for ; Fri, 02 Feb 2024 04:57:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1706878664; x=1707483464; darn=kvack.org; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=dsW8Dk1o9bsbV3Vb7IbQbusPN3ry+IgXo47UpIKTbrA=; b=PHlGEYdcLXSq6B2Zv6PckOpMn/EfIl6M21DYSkDGqY2yVqG/IXuxPftgfOmeEzTQul tG+gs6XcUKf9Lxx/13Pf8C28cP4i4J49wZNXLpYfZtYx8XClqktcjL/h/9nbkFBTmZ/c QTHG+74OAwg15mOoF/S1wmN2ECMkmk0IxuTS3WaNQuuYJwARJuZsMv2VGzmlqlhAB8A9 obRwtwfHQPVXMJVlfUtxmOfBq1qBhirznKd6R6E3h/K8dkSyRSb8qgFp5gSFHLx15B/P mPin/By4UI7Z3Wp59ErwR6gZn721UYsWLiiQ8W63UJHoXys8v+T2ciSBv1los1RNJ7tX ZwHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706878664; x=1707483464; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=dsW8Dk1o9bsbV3Vb7IbQbusPN3ry+IgXo47UpIKTbrA=; b=CD1TwBAQnQRyf/swpvoPCQuZgrzwhK1DY+V+T1qb/uNgI5QpMgdJKmNvLY9Pkt7ppY hkeDSrO4bGV6EQKAE3KTipWejr0UFQaGZXmpByJvvDOuP11J1heYQVrKs3MZkbhppom1 u84cm4z9LNxVAPwbeqTSoMyvAv6TAdNtu1mtyKcH7jc9sb6S8VmKrtLB0uGdn4I5HsfT qPNLa2G0P4wpR6+1MaOEfuhWWF2quKAciMzCB7pXHdandq1J+NYG7HY+ZMeCtVdSsBXR P7Q5mk4lLdMkUYXBexf4aFtpfDClSUARUY9/RlIEjdHPKB+yFr+rOshe/FcF8st2h9Mt /Gcg== X-Gm-Message-State: AOJu0YxVNTUpv92RZ+PItiEM4AZtw4fPKPZ/zXl4VM2Lx28QwWKLT8F2 hx88q3Rya3o7rmnBS0k8WfS9sxiS9OrBgHG9rrsHKwpSyh6Nn7ShAdmY7VWCwPE= X-Google-Smtp-Source: AGHT+IGWW9Hoi/4/LJTWVSRQ7rIcOqbmVxoPE3T2xJYK4L/lSWtt5qNm62jLDFpkJ/zS5VzSbD1G3Q== X-Received: by 2002:a17:902:ce8c:b0:1d9:7412:834 with SMTP id f12-20020a170902ce8c00b001d974120834mr1734009plg.8.1706878663788; Fri, 02 Feb 2024 04:57:43 -0800 (PST) X-Forwarded-Encrypted: i=0; AJvYcCULSxMhAwPJO3U6ooCD+Fkc+VZxPIeIMjdrABIKtoDN7IyTpqM3RTVnOErBifu88v9zNK2fR++p9I6yyydVhl0EqpDcP57FzHJXoz1LB0dgkx+FHJIepfxqsiSrhF0+FKBa1HabX19FKoxeK41C1Uk4ZAVGRA4eWCljXbBkh0UMpvRS3vU0RoJTHhPQ6he0eZ37nw== Received: from [10.255.208.99] ([139.177.225.251]) by smtp.gmail.com with ESMTPSA id kg13-20020a170903060d00b001d97fe26d47sm296367plb.34.2024.02.02.04.57.41 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 02 Feb 2024 04:57:43 -0800 (PST) Message-ID: Date: Fri, 2 Feb 2024 20:57:38 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 5/6] mm/zswap: only support zswap_exclusive_loads_enabled Content-Language: en-US To: Johannes Weiner Cc: Nhat Pham , Andrew Morton , Yosry Ahmed , linux-kernel@vger.kernel.org, linux-mm@kvack.org References: <20240201-b4-zswap-invalidate-entry-v1-0-56ed496b6e55@bytedance.com> <20240201-b4-zswap-invalidate-entry-v1-5-56ed496b6e55@bytedance.com> <20240201181240.GE321148@cmpxchg.org> From: Chengming Zhou In-Reply-To: <20240201181240.GE321148@cmpxchg.org> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 3AF614001B X-Rspam-User: X-Stat-Signature: bpz199kx9e9eu7qfqrsdxcwgzbidffbj X-Rspamd-Server: rspam03 X-HE-Tag: 1706878665-191453 X-HE-Meta: U2FsdGVkX1/HjecX5t+bid0NlpjHR6mj/KHNcJBq4J6Xdoj8ldO6tg5taYsznuTjAV+oXOOTbTtShId0q9yFBpEAtMlpWEQnlC1sUChTVDoEk+Va5X/JlqAYdFEMeSRkjpmOmaekYNeEviCWtMsadJEAI6FueCierSRXhQQMbKLKV/VO3KKsqE8riz9LIbvR6Np8ygKjPo2g7QIwTDdiQMiV9x8LQtrMtRMozWgmjCqeIwPG667uOKEMxNcslpmeZImeMJ0U4tJhiGnGGuHYXA+gM4vnidGeu3KS3jR+HlvzNm1Br+Dm8AyT/GJrA0MQxarGgFvAG0UbNBsVAf3i9CsfA/u7JmtzNd/XsPt18yKLk1/qbG2WSDvd3Z5CHSDhMxr06Iik5EgVR7y4aW5RnHtABkM64TDgBpF6fbX6azsV+3L/pVM2qoiTxdXMKTkeikKTl6r/DMNqq9xNHCj3YiNXfAjr+QriRrGuT12Xt0/dISJ2VJ7/nt7SletcG+OrN/LaSfqt20XeotvQe2XYwrHaixCWZNd99ckw0wwdWrMT+v/DaP3HKXi39mTNdHluNNJpk3c4rpK64HJJjwxbPfwHvv72l2VPhr0nGGJEsmmS5QrCFKmvgBZfABqm/cwmJUnvJ5TDzIm/WSWD23d3DobdmQqxjq7eLeuDjQi0+bM5/KWHIxBZLGyJGe7KBYISWC+tQMRza1QyUF+FFefmB3A4FqLVrtxgMLU2QYrYaN9hxmUZWTiPWc0UEraICT28QkHGZJOwDJDbOPJ368tA/qUIQBxE3QE61S78zEQk9hMQ9czRb4TTJbOzi6ihz+cbTneY4Q4AFU0+E0mUaLqrHO7Pb+h60xvZJHlGkX4SGvR8pTg2/Nj4vrzge1Sxvdr4IaEeFt2A92K1xDjJRV7A3aEbxKg/fLGaVwR1nWk9IinmkICuZKVfCvVuhRjAHyfm11Ik48q0EojRGRQhFWI GvcrHdMj Bpg/Buge+A84b4+mKS0hh4Bhkfsth8C0xvRS2j9mNQxYASS5CuhXbNtpOw79IilIdjRF48bo5yIR20aMGavO8gUC6HHhm9pCk/JTISsMG9f9MXdRMfDCCFjCtlLxw0zC0y9zwu3GeKGLVrDKBs7/6nRdL86j7bkjuh/LixLUkGI4nelGRGw3tLclDRpajEKY5xcBJnk2YLICIe9tsWkwUjoBWDe5DkyjIld+r2UGisPq1Hu34fXEj1qva5d1Vj7ysdQcFUvrWnxwd0cH2EkLNNATanWuwcSYWRIwzUaXi4U2YD9CXs2zspg+PCqLVynyIjqjclobjKfmi8XDmrqETEpfbI+8cFqEFhnqhRMdf7LzKnbeDDzl6xgfCbwu2RJpC9BkHkGjg68Y84HpB69/sLktu5Xvo1t5o+9xfOVH362BdnWQxF5gE6Xis5b75+J6npy4KvVJkRDOrJDUKHKNy/6Kw9424x614VE/PR7szbus8jKn5COFa4USQmZG1XMUx6vVP X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024/2/2 02:12, Johannes Weiner wrote: > On Thu, Feb 01, 2024 at 03:49:05PM +0000, Chengming Zhou wrote: >> The !zswap_exclusive_loads_enabled mode will leave compressed copy in >> the zswap tree and lru list after the folio swapin. >> >> There are some disadvantages in this mode: >> 1. It's a waste of memory since there are two copies of data, one is >> folio, the other one is compressed data in zswap. And it's unlikely >> the compressed data is useful in the near future. >> >> 2. If that folio is dirtied, the compressed data must be not useful, >> but we don't know and don't invalidate the trashy memory in zswap. >> >> 3. It's not reclaimable from zswap shrinker since zswap_writeback_entry() >> will always return -EEXIST and terminate the shrinking process. >> >> On the other hand, the only downside of zswap_exclusive_loads_enabled >> is a little more cpu usage/latency when compression, and the same if >> the folio is removed from swapcache or dirtied. >> >> Not sure if we should accept the above disadvantages in the case of >> !zswap_exclusive_loads_enabled, so send this out for disscusion. >> >> Signed-off-by: Chengming Zhou > > This is interesting. > > First, I will say that I never liked this config option, because it's > nearly impossible for a user to answer this question. Much better to > just pick a reasonable default. Agree. > > What should the default be? > > Caching "swapout work" is helpful when the system is thrashing. Then > recently swapped in pages might get swapped out again very soon. It > certainly makes sense with conventional swap, because keeping a clean > copy on the disk saves IO work and doesn't cost any additional memory. > > But with zswap, it's different. It saves some compression work on a > thrashing page. But the act of keeping compressed memory contributes > to a higher rate of thrashing. And that can cause IO in other places > like zswap writeback and file memory. > > It would be useful to have an A/B test to confirm that not caching is > better. Can you run your test with and without keeping the cache, and > in addition to the timings also compare the deltas for pgscan_anon, > pgscan_file, workingset_refault_anon, workingset_refault_file? I just A/B test kernel building in tmpfs directory, memory.max=2GB. (zswap writeback enabled and shrinker_enabled, one 50GB swapfile) >From the below results, exclusive mode has fewer scan and refault. zswap-invalidate-entry zswap-invalidate-entry-exclusive real 63.80 63.01 user 1063.83 1061.32 sys 290.31 266.15 zswap-invalidate-entry zswap-invalidate-entry-exclusive workingset_refault_anon 2383084.40 1976397.40 workingset_refault_file 44134.00 45689.40 workingset_activate_anon 837878.00 728441.20 workingset_activate_file 4710.00 4085.20 workingset_restore_anon 732622.60 639428.40 workingset_restore_file 1007.00 926.80 workingset_nodereclaim 0.00 0.00 pgscan 14343003.40 12409570.20 pgscan_kswapd 0.00 0.00 pgscan_direct 14343003.40 12409570.20 pgscan_khugepaged 0.00 0.00