From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9DAF8C369DC for ; Wed, 30 Apr 2025 22:50:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 315926B009E; Wed, 30 Apr 2025 18:50:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2C1806B00A5; Wed, 30 Apr 2025 18:50:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 13FCA6B00A9; Wed, 30 Apr 2025 18:50:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id E7A636B009E for ; Wed, 30 Apr 2025 18:50:02 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 2EA645DCCC for ; Wed, 30 Apr 2025 22:50:04 +0000 (UTC) X-FDA: 83392204728.21.ED4A810 Received: from mail-ua1-f43.google.com (mail-ua1-f43.google.com [209.85.222.43]) by imf05.hostedemail.com (Postfix) with ESMTP id 55F70100003 for ; Wed, 30 Apr 2025 22:50:02 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="Bk/BTKhD"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf05.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.222.43 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1746053402; a=rsa-sha256; cv=none; b=Tkg67jEW97M6eWu/OJreS2vEVLxw5D582jRtUUpN+gK8k17iLv5KO5qb71E1zQw88J4Wmu /WiJFgrTMZEBgyntnvYXLMuUbPTk2JC++7NH39c/jGr53EiXBGD+ziMgG0UZ56QQDruWuc HJm3l/myWnqFpIF+Bw32GILiEmkkciI= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="Bk/BTKhD"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf05.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.222.43 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1746053402; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=nbpbINSrxfUumV9lwyjlqLA8g7akUpbSjiCjBJP9b4Q=; b=WE6CAFepKMOqFDZEKqYVq788Ky8bHUdTQRbR5F2DEO0SiDhDajywuvVEMlOKbD6oMQssFQ Gw+WsD5ArDpl1x9K1bjcZDl5ChHqNWIAA7NLV9reM0Jpr1HoCowWrKjdL44b7yvNReJ2O5 lINqmgq6fDk4SlONzLXoCpzuW09Ty68= Received: by mail-ua1-f43.google.com with SMTP id a1e0cc1a2514c-86715793b1fso115112241.0 for ; Wed, 30 Apr 2025 15:50:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1746053401; x=1746658201; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=nbpbINSrxfUumV9lwyjlqLA8g7akUpbSjiCjBJP9b4Q=; b=Bk/BTKhDCpsf+cJhcJvkVjEus9iMrjbAe1+pCJ4cgDewvkj1S98KaLZDz1Fo5I4xLn T/OuCUL90IKfQd+1rZD28xrewIqPr4l4iqOGegG2BmZnbyS3CxAafjiMY3N5QfBdsKkR 1HCaNfwioKc2S8f4/jUqvo9jx3/wz/GbFX6cwJkXbYS361SzBua3syHB+fpzzSYDEiik TRtoBZDCn98Vh3rRa9dfooKmPMCHlzf96UNpTv8uMWZCg21zRg4/k14v/EgsMgCYQDb7 CQe6a4is7+4V/ymRp97Eqe4haK2hDuIxStwvgflWI4GBZtNf40Z9hziTuqA9iMWGQM3T Pj6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746053401; x=1746658201; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nbpbINSrxfUumV9lwyjlqLA8g7akUpbSjiCjBJP9b4Q=; b=a8Ft6CemZ6jftc3EJdUq4rvHyjilzCsma8u9jOQzVMW4pyzw8o3SVO2bhFi8ztlvpj QdE9KPWu40UqfqABOEXmOyMIp1fqezlcHbIDVEQQN8payxnZlT/u/Gc6Iu+sEjqov03V /h4EhqKupd0Z+C4rnAIhJ4pJ44JOxmLc5Zvza+yGjSkgfR0JpR3ocE9jcGdg0AmUOyeg +CU7s5D6UsaPVr/J8yzFkb+P0fldraSEvoH63TK3UYkxySpbgvlcEAfNeO26rQGKNjtW 5+2615nrjwjB2ZDYGHIN4DYdr26qIwZ5zMZ6efSAYkIKZUif3Cc4TDxX6gXCwp/2Tkae uH/A== X-Forwarded-Encrypted: i=1; AJvYcCX4Mow9BWKmFZbS7yRi2u7X54pHQHwxLJpjTRVx1i/0Yn1lD8E1JasSlfwQqUUMwqyAdoGcs1sfpg==@kvack.org X-Gm-Message-State: AOJu0YxnUuhvzMm2cvdAaz1/2OYZoInLQ4eHZzSwLh99gwyr8brk2LbE wzaPq1dFnMoG3pWf7K94hbO6FFs3Vo7GqF1JAMMiu3vWXfJHtMWas4ph2yigaDO2Z7NTur0pxjK wQLY2a+FxBjElJhCZ7ydO/sseIB8= X-Gm-Gg: ASbGnctQUG7mQGJMI3VpOFkh/lHnNUZwx+3TCWfAfsmzxGK5wLeRdRBSagdyUYCHsP1 FZ2rCm5hwmUqRFXp0Ar03U430KcMTad5xY7l6fvocwI0JRiPZLl0fsoH76LGyNz7ppoBeFpR77r 9xDIFd+KtsCUHtoBaYSk2/XY2hwltKDkqP X-Google-Smtp-Source: AGHT+IGzkaGVgrKTVWAg6uYcHiRtwbA0R/supqUwGLiy3GtgazMGGDJF+F5gy1Ww2dKzbjTsV1APhcaThlpXeBvRPg8= X-Received: by 2002:a05:6102:3e8d:b0:4c4:e018:326f with SMTP id ada2fe7eead31-4dae53bb727mr916415137.10.1746053401317; Wed, 30 Apr 2025 15:50:01 -0700 (PDT) MIME-Version: 1.0 References: <20250430082651.3152444-1-qun-wei.lin@mediatek.com> <20250430145106.8ce79a05d35cec72aa02baa6@linux-foundation.org> In-Reply-To: <20250430145106.8ce79a05d35cec72aa02baa6@linux-foundation.org> From: Barry Song <21cnbao@gmail.com> Date: Thu, 1 May 2025 10:49:50 +1200 X-Gm-Features: ATxdqUGvtOD0hW5rJBr5f2DF6ZxtN6Be6JTfSzIEhz5_WOzsDzAADbdK5XApyCE Message-ID: Subject: Re: [PATCH] mm: Add Kcompressd for accelerated memory compression To: Andrew Morton Cc: Qun-Wei Lin , Mike Rapoport , Matthias Brugger , AngeloGioacchino Del Regno , Nhat Pham , Sergey Senozhatsky , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mediatek@lists.infradead.org, Casper Li , Chinwen Chang , Andrew Yang , James Hsu Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 55F70100003 X-Stat-Signature: twgbgu1qo6kyy68ny3edwga9sj335o1n X-Rspam-User: X-HE-Tag: 1746053402-644195 X-HE-Meta: U2FsdGVkX18oryoOIdFilXWRH+cLXsak/szEwJAxALnxVc+ziGAALolnKYt9989vEk1CRFIZEPup1DpBmddTxe8MFTqBKjdBX4vu0bNOzQjd7kKqI5l/sim88bnhe+Vv7ix2pbZeQyAaPTIOIAiCjQgZSju7EQpIZ0OZQQHfrqqysBiU7Xff/lNJd3PmQt1O45GPojIHeVI0QfvuwlIxQCbos3riaUNPd0jdLFFvOd1L490rs0lWj/KRgC8HSCPRIQRr6WN9d0nKlbkqTXqT5BKtvpKTU/YTOS3qyyhdPadWOE7KnyLbpUbnc27xDRORKYd5WzLKHnfGrlFXytdq/s2ZsUwGRRQzTi18qyxY6pL4BV1NmTG5BYEJUiyyLYkOoH0KNYtHHJegd+OQlH1KQT/MoGHV5qtrqU7GxHkmAiraCGX+l7wc0fTZWa8JAHgXyeiVZ71Tc+NRkjNydc7QaUX2h62VRQhOR704C4XPwo/Dn4gcXcy4KKeOz/xNFLP2ud/GoKwVLDgYGj/bdhFOzP114I+SffZhl0A6F935DTYXBlPOtEsWThCABUmdHzAXJjtlUZJh6Gn+LD+nXrX4GApmQeJXom+c8/Kuq4ui2aCePfNXq+VoYVjT5WjFS7kUZv06at9Q6E230CCG5MJyzQEK7GNLJ0XROr27qtTpbHKRTAv4qPj+PKUzgmMUxwTAufOhHHZVZ4lsPv9y5K8niRUMkOQ2aiGxwn/DUUpqeRJZo44qDgb4EjBLCBrbiqjfqgYdOxBsTv1PMx7jhcg3MGIHvxvIJvNHmkVWjseNu+Y6OaTIOH/UPrtXMnBwFCT4ngNpOjtceugVfhWvFgRjRHxl6rSpr17XopjB2Sl89TFJPfdUeMLMBk/5euCGOvP2t1GsqahaLVuZgMpAFfhMlVigjQSggTSdC48jv8pJG6CD/ONzzJKe9yEO6INjxh5nbXrSK+IVI711pnuxpzK baEUcRpz 1+bFTcDV0Y3jEdm77kVKyXJgn8j5/Vv5h27/spurRJNCGjhM5O8wy/VJrJ5AUvnHhT5Y1civoZ6qGV2Bzv78zAqy7owfMHNAnUD0tRlQEHofq5rumQCyEg4qgA3bbMjypGdxL9qUKORYWUpEP17ngThSbfsX7du19pZaYm9r3nENrqQkuGbjWJGgluU86BMHGHkDCY4uVafIGju+ZWgN0QT4Mua/zKHCOOukIL0WBs53RKnh6HrwIE/2zgmInDTfNJWI4RgNCfl+424wDTg5mEojzczOHAFQR2HD0Zv2gXNSXtzbq2aPJKgUJO0hHHhZqL76yjzF7rhdl9qLGBAVzvuZzzqeJA0WDSHJbOeL7P5Sv7xXJ5sPsD7u02GOJDsM94+k3ZmShqq2GYzPTA7zJx45dzTCVrO9sk4sw9af577m3ns0+JRa4EvFEMp9Y06lUZLCNHYCSWyxQUX3sJSUSKltKQA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, May 1, 2025 at 9:51=E2=80=AFAM Andrew Morton wrote: > > On Wed, 30 Apr 2025 16:26:41 +0800 Qun-Wei Lin = wrote: > > > This patch series introduces a new mechanism called kcompressd to > > improve the efficiency of memory reclaiming in the operating system. > > > > Problem: > > In the current system, the kswapd thread is responsible for both scan= ning > > the LRU pages and handling memory compression tasks (such as those > > involving ZSWAP/ZRAM, if enabled). This combined responsibility can l= ead > > to significant performance bottlenecks, especially under high memory > > pressure. The kswapd thread becomes a single point of contention, cau= sing > > delays in memory reclaiming and overall system performance degradatio= n. > > > > Solution: > > Introduced kcompressd to handle asynchronous compression during memor= y > > reclaim, improving efficiency by offloading compression tasks from > > kswapd. This allows kswapd to focus on its primary task of page recla= im > > without being burdened by the additional overhead of compression. > > > > In our handheld devices, we found that applying this mechanism under hi= gh > > memory pressure scenarios can increase the rate of pgsteal_anon per sec= ond > > by over 260% compared to the situation with only kswapd. Additionally, = we > > observed a reduction of over 50% in page allocation stall occurrences, > > further demonstrating the effectiveness of kcompressd in alleviating me= mory > > pressure and improving system responsiveness. > > It's a significant change and I'm thinking that broader performance > testing across a broader range of machines is needed before we can > confidently upstream such a change. We ran the same test on our phones and saw the same results as Qun-Wei. The async compression significantly reduces allocation stalls and improves reclamation speed. However, I agree that broader testing is needed, and we=E2=80=99ll also need the zswap team=E2=80=99s help with testing zswap ca= ses. > > Also, it's presumably a small net loss on single-CPU machines (do these > exist any more?). Is it hard to disable this feature on such machines? A net loss is possible, but kswapd can sometimes enter sleep contexts, allowing the parallel kcompressd thread to continue compression. This could actually be a win. But I agree that additional testing on single-CPU machines may be necessary. It could be disabled by the following if we discover any regression on single-CPU machines? if (num_online_cpus() =3D=3D 1) return false; > > > > > +static bool swap_sched_async_compress(struct folio *folio) > > +{ > > + struct swap_info_struct *sis =3D swp_swap_info(folio->swap); > > + int nid =3D numa_node_id(); > > + pg_data_t *pgdat =3D NODE_DATA(nid); > > + > > + if (unlikely(!pgdat->kcompressd)) > > + return false; > > + > > + if (!current_is_kswapd()) > > + return false; > > + > > + if (!folio_test_anon(folio)) > > + return false; > > Are you sure the above three tests are really needed? Currently, it runs as a per-node thread mainly to accelerate asynchronous reclamation, which effectively reduces direct reclamation. Since direct reclamation already follows the slow path, asynchronous compression offers limited additional benefit in that context. Moreover, it's difficult to determine the optimal number of threads for direct reclamation, whereas the compress= ion in the current direct reclamation allows it to utilize all CPUs. The first condition checks whether kcompressd is present. The second ensures that we're in kswapd asynchronous reclamation, not direct reclamation. The third condition might be optimized or dropped, at least fo= r swap-backed shmem, and similar cases. Thanks Barry