From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6CDBC282EC for ; Thu, 13 Mar 2025 16:58:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 96CE1280006; Thu, 13 Mar 2025 12:58:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 91C88280001; Thu, 13 Mar 2025 12:58:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7C01D280006; Thu, 13 Mar 2025 12:58:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 5B0CB280001 for ; Thu, 13 Mar 2025 12:58:13 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id A6F0151FAE for ; Thu, 13 Mar 2025 16:58:14 +0000 (UTC) X-FDA: 83217135708.24.66B62E2 Received: from mail-vk1-f169.google.com (mail-vk1-f169.google.com [209.85.221.169]) by imf24.hostedemail.com (Postfix) with ESMTP id CB74A180010 for ; Thu, 13 Mar 2025 16:58:12 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="LvH2L/SM"; spf=pass (imf24.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.221.169 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741885092; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ADIdlKa1fA4/imYaZXygiUs98rCXU1xydtVTW+Hgst4=; b=7lETJ0D1gZdTs/KevDGmB0DPPeszQBtLpdpCDuxLSFtl5+B4ttb8sOm+Ng1d/6Huf74gPX MZ8rtIVtt9eCHk2ROPyG/NkS/PR57vGfKy3ytpGwNmH8n7izrIcHZYiwHBqwBH2HfsXI2v Ph3CJx92WoirM25dtRMpI9ZJpPM2yfc= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="LvH2L/SM"; spf=pass (imf24.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.221.169 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741885092; a=rsa-sha256; cv=none; b=yNPR/17i/0dcoeneC0Bqs4vA9vAC6Z70EhWtaSoLRgo74MPI1P6WVOIwHCp9t7qr0+oco7 7f0DYC5CCceflHSJLik4moM7bu2GKwJy+dBStNjaOUE4+RriuYew6YhSSs4wdGSseP7J7m 0UM8y1InTML61lZpmyfdhtBsWUqHFXw= Received: by mail-vk1-f169.google.com with SMTP id 71dfb90a1353d-51eb1823a8eso552813e0c.3 for ; Thu, 13 Mar 2025 09:58:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1741885092; x=1742489892; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=ADIdlKa1fA4/imYaZXygiUs98rCXU1xydtVTW+Hgst4=; b=LvH2L/SM4s6OAYUsWfRcxc2tWmwSFzY1yec0Yixc3jd47JEX38rNw2Zrjv7Ck26Erj jWWuniM/09noCrrfpxM6rkFtMXWlTF+EXTCsRgrzN8Nm21zTmQfVkXCs75ksdJSYquY+ gpu9OSvNpL285EuQurBsVLTgeCvrAYCjGWkgh7RZKGQNSk2QTEgQ1dakAF56NJ9Fntl0 3n13Es3gArkfoTkDLws1+UdyAuhWq/BiNTwX5S2Bau1ysSIHMDWQcp0tGS1V4/vlW92W TJF419dAELh1WNfa6ci3Pji+rnkxrrdyAalG6/dZfqpkHgnSfGRbzyQdjWdAO+9yX5y/ UwkQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741885092; x=1742489892; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ADIdlKa1fA4/imYaZXygiUs98rCXU1xydtVTW+Hgst4=; b=e95lRsrrq8ZHYZN5bf7FNWC54Fs1gM8FQZmKyNvWvxolHvfNjQB6qB9IcpRoeFiq9p dcuCdnJzZxV5mjtH4jiRkW9ioilGSgLIIUHEbrZgTsl3t5I+WEn/PFdmN/WPOehDnR7K eKUDiwJc6+TBtwN44nO9x0Cwd9OiqA5xqlErJHn5f3udiXMHBNM/P1UbEGpzBnNCDNIW OoZqdDoSA8fiPLk/fjkI2EXPeykwhEq/BQ8PcW8d3/d/HFxrQPatWpBhTbFLd3rwMoON ExuTc51AB2gilj2Eh4Vc9kxXKbZXnNxcokoPLroqGiP4gQCAx+Df+atUUmvWWpL/h3JM /cOw== X-Forwarded-Encrypted: i=1; AJvYcCUMLUdhBUHCaK9qo2XFMhPMUuHdiRlG8dpEzRiys8gbG9ku1YZmOQzjr2S1MkLngGd+RsnvEUWoSA==@kvack.org X-Gm-Message-State: AOJu0Yy45dFbcNE2xB9g2vvsyrf5MElq9YFdyxXHjKSyOn70N0hg5M+a xtLLAf9GudwLaxdboEFfobOd2s9ftKqtexIExRjYA9phNeg9qN4ZGUo5P68JhzDKT1+hFDlF9Vr IFC6TADZ/PAwmoeHB9rk4r04snww= X-Gm-Gg: ASbGncvIlETtbyjz6vTgIzigfXKk7NJp95Gv5mninpMkpnN2IgqWzMnrKF8nPTPXv5q JWV2t/VuCyrFz0qUk2eA+IKaEs97IbiQuOmuF20++9viMquC+TKPOPJvqZt/2XjVz3FBEUDkbY3 AuP9VijdPMioRz5rKZTRbTXuXn1Q== X-Google-Smtp-Source: AGHT+IGqd4mPuYQMTlhSCpe62t8tLvAKCPyLobikqm+RE8kQK34379HRDecEtPPNH8Li/NEYmxYtyykiKOTPdHHJXWU= X-Received: by 2002:a05:6122:2001:b0:51f:3eee:89f4 with SMTP id 71dfb90a1353d-524462587bbmr1339574e0c.9.1741885091858; Thu, 13 Mar 2025 09:58:11 -0700 (PDT) MIME-Version: 1.0 References: <20250307120141.1566673-1-qun-wei.lin@mediatek.com> <5gqqbq67th4xiufiw6j3ewih6htdepa4u5lfirdeffrui7hcdn@ly3re3vgez2g> In-Reply-To: From: Barry Song <21cnbao@gmail.com> Date: Fri, 14 Mar 2025 05:58:00 +1300 X-Gm-Features: AQ5f1JoQPpin_yzy1xmzTa5CZUM1Y8yc8ZWtftl1w5WEvizQdxmWfzWVnzMz6I0 Message-ID: Subject: Re: [PATCH 0/2] Improve Zram by separating compression context from kswapd To: Minchan Kim Cc: Sergey Senozhatsky , Qun-Wei Lin , Jens Axboe , Vishal Verma , Dan Williams , Dave Jiang , Ira Weiny , Andrew Morton , Matthias Brugger , AngeloGioacchino Del Regno , Chris Li , Ryan Roberts , "Huang, Ying" , Kairui Song , Dan Schatzberg , Al Viro , linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, nvdimm@lists.linux.dev, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-mediatek@lists.infradead.org, Casper Li , Chinwen Chang , Andrew Yang , James Hsu Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: CB74A180010 X-Stat-Signature: ki5yejnwujcagpbbpjiui1ffg77uwgp6 X-HE-Tag: 1741885092-234388 X-HE-Meta: U2FsdGVkX1+n+jTI6j3kO/rXRq3JZtNL/7x7zlEtN1NRNJAC/GTXyQ3UVwsuyfPvH4vnt6QHfxKhbAAhIUawphhvKBJys2r7R/2tfV62va88TG0X8dfFoH9fV+/qtO0T67EfhKyehEgtiCa0zCWZkokknNsI0ygJKqkPoYYbSkXZZw6fqc28UDwaFMB98MWI+VdPx67mTp9Zt1PhLvk8tK3csP2WdlvWQ4RYseRh038iEtZoX7oeuEIEXX/EuSKDZSBWinG880x9sNj57WbkH+7dNaMviRD2I/QjtjLRjfvzjM1KWGlnpGAGWVd2vDKpuAAXyf/2aa3rQiL9Xoxv69L78w66qnHRbiT7Bg3WtVfTR1vDrolMznEm74LAwJRIAS43xqQ3fq+CgXlYxTSnUWrSX6Yyiv50zI7L3K/vwh3f+oQn747hLmJOLBiKwa/ImhiDnPyRojjcO7wmd6DjPQP+5W6Ygh3uZzpXOVs4jBwfjiCieMk/wqvliAv//6kOkBOFUIWP+16s+tSjv9mLXVraGuAg2WH3eQtUejV+huw5f/UdFpnGRxGYZQPkEnZxAW7+G68w/I4BDFhS8Xsoe2JC2xiymadEmOVDv2TkZHgOifwr5L9XxYQD2d1Bgl9Jx2xSnAK3V1sKMrlvlD+t2VfdDx8sgzOcuZ8Uura9+DZrU8kjHJ0rlyJZKnximwmaN8B9GH8JemaNBcF5luQ1IpMnCw1wE/80YnAHcSmNa05LvtV8RXhZi8TDpZapiE0YhN1NaQg2qcgLbm13lXgsL2YQf/I/H00j/fkENlcYRDzyY67kFtLBshYGtUB2g1UeG1f9JIbLara2GmoR/KT5p2Q1418wyUF1C9r9z0CZWssrqf5SKifMEIqu1sSdBFSG3ZYl10HwEeu4TBWO2hfwE5tD7dxtIn1p7T3HQbWzy1Ia0wRxF+Ns4EpsKiwdAtdffH46G0c/coIlb/u7Y+r w2m059th pAscIJMUnYS/N7AEZhLpAa1VRj8yaUZQlQ26/ifAk2jtWCcSARpkv/QnlDgtktQ9Kp4LSE9PuThW0KSO/DguBn6nvG1rrRrT6p/dPstVvvuA7FGwI1L3IIbEAs2urgbIo7nzBMl9g+3aCe2vMmwHHFtQIkp6F5uwpnO2oflJRDgfFRcIKp6SLdTlWevegZ8z5Kjz6g8Tz+1RZEm8Lby3kxrIp25i58ZEQido7/gNpfN/s6bkwX9PIgGf4f+B13Cm3VN1VYoY/5dJnrA1LGAu9NLPG9aOJ18y1JbobqzGuXg+0M2jmXrluxxSktE6bh98SBq8vNdbYB7jHA/Y7Wviys3Sz1nDDIauxl6HsMCbaq1uwDlpnkFMvAN3HsAu8pq7HJjCkOAkAbW+9AmUhMTIzpgBqINNfA3Di+ZwAiGp0Ezs/FyRNQh94Rhp0ww== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Mar 14, 2025 at 5:07=E2=80=AFAM Minchan Kim wr= ote: > > On Thu, Mar 13, 2025 at 04:45:54PM +1300, Barry Song wrote: > > On Thu, Mar 13, 2025 at 4:09=E2=80=AFPM Sergey Senozhatsky > > wrote: > > > > > > On (25/03/12 11:11), Minchan Kim wrote: > > > > On Fri, Mar 07, 2025 at 08:01:02PM +0800, Qun-Wei Lin wrote: > > > > > This patch series introduces a new mechanism called kcompressd to > > > > > improve the efficiency of memory reclaiming in the operating syst= em. The > > > > > main goal is to separate the tasks of page scanning and page comp= ression > > > > > into distinct processes or threads, thereby reducing the load on = the > > > > > kswapd thread and enhancing overall system performance under high= memory > > > > > pressure conditions. > > > > > > > > > > Problem: > > > > > In the current system, the kswapd thread is responsible for both > > > > > scanning the LRU pages and compressing pages into the ZRAM. This > > > > > combined responsibility can lead to significant performance bott= lenecks, > > > > > especially under high memory pressure. The kswapd thread becomes= a > > > > > single point of contention, causing delays in memory reclaiming = and > > > > > overall system performance degradation. > > > > > > > > Isn't it general problem if backend for swap is slow(but synchronou= s)? > > > > I think zram need to support asynchrnous IO(can do introduce multip= le > > > > threads to compress batched pages) and doesn't declare it's > > > > synchrnous device for the case. > > > > > > The current conclusion is that kcompressd will sit above zram, > > > because zram is not the only compressing swap backend we have. > > Then, how handles the file IO case? I didn't quite catch your question :-) > > > > > also. it is not good to hack zram to be aware of if it is kswapd > > , direct reclaim , proactive reclaim and block device with > > mounted filesystem. > > Why shouldn't zram be aware of that instead of just introducing > queues in the zram with multiple compression threads? > My view is the opposite of yours :-) Integrating kswapd, direct reclaim, etc., into the zram driver would violate layering principles. zram is purely a block device driver, and how it is used should be handled separately. Callers have greater flexibility to determine its usage, similar to how different I/O models exist in user space. Currently, Qun-Wei's patch checks whether the current thread is kswapd. If it is, compression is performed asynchronously by threads; otherwise, it is done in the current thread. In the future, we may have additional reclaim threads, such as for damon or madv_pageout, etc. > > > > so i am thinking sth as below > > > > page_io.c > > > > if (sync_device or zswap_enabled()) > > schedule swap_writepage to a separate per-node thread > > I am not sure that's a good idea to mix a feature to solve different > layers. That wouldn't be only swap problem. Such an parallelism under > device is common technique these days and it would help file IO cases. > zswap and zram share the same needs, and handling this in page_io can benefit both through common code. It is up to the callers to decide the I/O model. I agree that "parallelism under the device" is a common technique, but our case is different=E2=80=94the device achieves parallelism with offload hardware, whereas we rely on CPUs, which can be scarce. These threads may also preempt CPUs that are critically needed by other non-compression tasks, and burst power consumption can sometimes be difficult to control. > Furthermore, it would open the chance for zram to try compress > multiple pages at once. We are already in this situation when multiple callers use zram simultaneou= sly, such as during direct reclaim or with a mounted filesystem. Of course, this allows multiple pages to be compressed simultaneously, even if the user is single-threaded. However, determining when to enable these threads and whether they will be effective is challenging, as it depends on system load. For example, Qun-Wei's patch chose not to use threads for direct reclaim as, I guess, it might be harmful. Thanks Barry