From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A74FC369DC for ; Thu, 1 May 2025 14:02:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6468D6B0088; Thu, 1 May 2025 10:02:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5F54F6B0089; Thu, 1 May 2025 10:02:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4BC296B008A; Thu, 1 May 2025 10:02:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 311406B0088 for ; Thu, 1 May 2025 10:02:33 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 9F1EE1CD4A1 for ; Thu, 1 May 2025 14:02:34 +0000 (UTC) X-FDA: 83394504228.16.8F44E8B Received: from mail-qv1-f67.google.com (mail-qv1-f67.google.com [209.85.219.67]) by imf12.hostedemail.com (Postfix) with ESMTP id 6E43D4001A for ; Thu, 1 May 2025 14:02:32 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=PUHK5jsH; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf12.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.67 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1746108152; a=rsa-sha256; cv=none; b=413wHz0dUAu8mUTswu5XHdrkLDyBbeh55AYK8ZLzcyQuYq3o8WuEjqlAVpr2vryxheDIn8 7dfmMmaExzPsBxmhsouxduGr28olS57rEaupRULhZ54XrAEw2MDNLdX++SdanG8WL6OfB7 x+H8ShtOFK7sC0wsJcylzsOBJCA42zg= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=PUHK5jsH; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf12.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.67 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1746108152; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wsPTvIUlonTz6aoFURvjc48QUmCzpcROb/H1/P6AxH4=; b=XyQHRyIAU7ZdAY9EdSIfzevSSHlraBC5+T4N1q+6UTvRvKE30D4eKMcH5XBj7Vmr8w+/81 /wzEVph/zeqkCVmqapkOSbXq5+JusfLJnD0PxaS8fBOZ7KQe95/7631PW0Msq7c14l9n47 Q4rnHirJPR3bHNH8pKQg2lEpIAw7hro= Received: by mail-qv1-f67.google.com with SMTP id 6a1803df08f44-6e8f43c1fa0so15560546d6.3 for ; Thu, 01 May 2025 07:02:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1746108151; x=1746712951; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=wsPTvIUlonTz6aoFURvjc48QUmCzpcROb/H1/P6AxH4=; b=PUHK5jsH3WNB1FuWt191VB5n9LQUSLmaIXBNvAvzU8S/+tM5w0H0pLHykNOb8nsPip nTW/rY2X+6fEpFdDvS8lrHPUMONcMGE6tYL3+ianpeOplVs66cl9S5S5bkA0KvP4ZQk8 WAiIuitvzq5jx0dKzUFUN7UqL24GaNSLuuuvfu3JSWfZYdJ57nGeR8uoqRlZBaYvNL34 Ft4ijpQ9IYRrlTXSwwjpwNdxhrfkZqDtGvMOCCR4WoXchTDHTWJugDf9aHaWykIgs5wc YXLEhlt7MVQNDdhgQoUQW7Z9VTzoWX0BBHofoVofmdz+rE5sEa4q5UN8r6BCwVYJO7K1 EVSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746108151; x=1746712951; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=wsPTvIUlonTz6aoFURvjc48QUmCzpcROb/H1/P6AxH4=; b=B3sF3EyfRT2gOJlGmkpehUspYOfMJm7lGzPU+mspSobA19YMYCRmqSpNVNb29FcGxn KkpGmNXu22gtFOY73xu1UehLdU8LPftJhTtwEzuZL8OnCl5MoGyK4mejcDXaaPSmYXBb oBrgLV7TDkZCB3sQ7Kg4hmUe6zj4A+OzF+Fuok91NYz+1RK1ctiCsq68uMuuLdGEhNwd OnExyQijxZrHBriP5iPERjP9mwODKYrbBby72jZq06VBOD1KGHUihpRygcvRPlZVl1Tt x2t7WeY/5HDj7RbB9Ik2wEm7aR/mMKBJHpdHMSn90+j7ziQb/Q9KePdE4TpmBCifQaFy TkIQ== X-Forwarded-Encrypted: i=1; AJvYcCXGsS6A4Z2QbEIqRzgZ/zGPJ5XHtkTvPda+udrZEYeZH3GizKQB+vJ4WHZm84vsVU/5+oguvExVZw==@kvack.org X-Gm-Message-State: AOJu0Ywk9ukfge3dE/TihRUokvadXZJ2Q4aRFRCquyr/euWw6E6kQOrI K8bTfMU/CWVTI1c51P745KnRTo/ZyJ7pfaXJdX4KfBUG0wfAKNkcA2c35xjB40M= X-Gm-Gg: ASbGnct8C3MFbL7MzuM83D7RJlOpcPG27hd/BJzm0nRXAwKWDf3CM2aWTJ1VoF7IvmN pDOXhQoQw+IWzJerHuVYmSNcP2Jd3HdwQ2MAQMPPtsMEyrrQ2aK10qPA00q2LwvzU62s4EZ9mCe wyaYbNeearbsU44nqvfcPD7s25rrFMnUnw2/GgE568okLaymUX3h1/I1t3NdohB9M0a57cq1OHC F4WjFgSO/KgM3gYfjmibkjGLzIUEsVv8iGEwGJjfEv2C23zkCFSX9SeDZnbB3TWJtPwJ1V0PkRu 2TG2xLpr6fhfElKMP3FWo6kdUdKQXYKLGHMn9tDXL9L2cWNXdw== X-Google-Smtp-Source: AGHT+IHMMzdAha+mSe5FENeDnuJbBSEZwl1rV0kTSW+UDo/woOhAYywdrFjkvAnvanulduGVihb2gQ== X-Received: by 2002:a05:6214:1306:b0:6e4:3eb1:2bdb with SMTP id 6a1803df08f44-6f4fe1357c5mr100135346d6.43.1746108151086; Thu, 01 May 2025 07:02:31 -0700 (PDT) Received: from localhost ([2603:7000:c01:2716:365a:60ff:fe62:ff29]) by smtp.gmail.com with UTF8SMTPSA id 6a1803df08f44-6f50f3b058dsm4858996d6.4.2025.05.01.07.02.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 01 May 2025 07:02:30 -0700 (PDT) Date: Thu, 1 May 2025 10:02:26 -0400 From: Johannes Weiner To: Qun-Wei Lin Cc: Andrew Morton , Mike Rapoport , Matthias Brugger , AngeloGioacchino Del Regno , Nhat Pham , Sergey Senozhatsky , Minchan Kim , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mediatek@lists.infradead.org, Casper Li , Chinwen Chang , Andrew Yang , James Hsu , Barry Song <21cnbao@gmail.com> Subject: Re: [PATCH] mm: Add Kcompressd for accelerated memory compression Message-ID: <20250501140226.GE2020@cmpxchg.org> References: <20250430082651.3152444-1-qun-wei.lin@mediatek.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20250430082651.3152444-1-qun-wei.lin@mediatek.com> X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 6E43D4001A X-Stat-Signature: qecsiube8qe4y5ey6ee74jz16us6o4ce X-Rspam-User: X-HE-Tag: 1746108152-287926 X-HE-Meta: U2FsdGVkX19TQRfLb2MToPkyJAwRmi//Q1xlSsFxLXSY80YFV+pIhJTmaNb0P51uhF8tVGNg21MRiUeJZB6EJRTvLqldgFauF6TV84loygf8aHInRKk7nIlDybFW1QEFa7X+IMEOU0BoTIMCr+VV36r7FlYfFPgaNEL4qOFUYLLW+X2OvowRcnT9UOiSgUu4rtB40+wTZFJkdZ7rPKNIb9Kt4Of732L9ON7dreP4Un8RmEKq/416q9aG8H3AjkT8tGADb2WGXhHiCSvBwqgDXD4ZmsvhWXp0b/+Jw0OOZvPiIL4SGAq8z78RTKxyA8P6T6F1bmjdoChCTt7aSeVEbmuJ2ksGDc+QM3iJDcnIHCfuRwF7FI7IjlThhfS05Q3+knNm8FdcFG9sJTYAva2ZNMSMvCt0rSmRKumQDiKKLPpLAMol3WA+M+U6FA5RRY0Wfm40RDn0EXCHycGgc9yfC/sV/gGvrKzvmQ1YJF8hXmOzG0R13lOuK2LUzgyi/pNBWzTkCuZ6dzS34DT5CRH+q4+QBAHseZPV4tf9DKUA3ZMrKNnGDkCc13d8DOaeMV8JWLD658C1qM93RVwpJZhyF5bSVkxJfTF515ao+llc1QQfqMZFNfsSJXv1xKXXWl7TA3h/d17eANkDcvhV+dEQSs4sbFBnBChnhvps3waQCaoAeXQPFEUpqJa24Bz0Xd6bFHmu/iTXVC/Cu9KHE10Vwuv/Lq2com42swkLRkfQ0o6skimRJORC70aVtkRUeHn3LrHL3aSdUV2v0GXsmjEych0HcDFHvSCr7X85YLIAZ4EsYBcNa7ipq2ipTWnmZcoFALayLS9gklA1sqbkWT33G8qNq/oHVnAOw5aio2js3UnAVZVtiiEIBXJZqe0BQHTYtzCGetFWiXd9D6ND4h15vm3oqEicutULrkiJHEB563+qIy/lyCQfFWUsYHU1b9zjzV4IAqBZemQUSbOeTqX sbQYqWku 8KJKoyKN8Gu3rIDcbteSYuHk2vsE1dpEMxbg2LrrXwWnUUaKnO8ZaWnx5cvkSTwUke0ucJVjAtn7AIvQsLkqk+ZSWljIbae9cIdtO+L1qoEZVI0X0PstuCExR5ZWr2e0uGkWiJp3TGdMaFneHZY4cBLXCsm8so0lyFtUb4n2znyPkTVcBq6Afb6qcbzxBWtIsCeSH9EeoLE3YMcrekmBarmj+Sly3h7R4flONRp/441yEX9K9DfkXkOgcjG5VvNWJ6zn7z0PRAK0Eb9PiZfdBzl7hRg0LZgyPgmJtoE0Hf89IdSeNS3DQ4EpYmGKNrdfqhwbZwIxQ/S0O2AmwwS9c2/iooRCysb/sZ243gcZ5rUzErcVYoj6PPz42f6dur9jVz4qRIafXqImDAdiOHNjNzzeV/N8E2HY5V7yMw2nhhDrd0c90QgviP48bNI4wrYuKw2+plR1/rISaiCfId3pBTt/bDzlFvZ0yIVWKLecg1EqLFZAgr7QhtBUrYNEW4tDu+1krHY/uONL8rm084h13Y0wNF1JnpOG7QuXgScOz1J+tWn/7tSH4KDFBnipO/SwuVJG6WrAZAh5cTvsaBh7BiDHo8oBncYkqYuuk X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Apr 30, 2025 at 04:26:41PM +0800, Qun-Wei Lin wrote: > This patch series introduces a new mechanism called kcompressd to > improve the efficiency of memory reclaiming in the operating system. > > Problem: > In the current system, the kswapd thread is responsible for both scanning > the LRU pages and handling memory compression tasks (such as those > involving ZSWAP/ZRAM, if enabled). This combined responsibility can lead > to significant performance bottlenecks, especially under high memory > pressure. The kswapd thread becomes a single point of contention, causing > delays in memory reclaiming and overall system performance degradation. > > Solution: > Introduced kcompressd to handle asynchronous compression during memory > reclaim, improving efficiency by offloading compression tasks from > kswapd. This allows kswapd to focus on its primary task of page reclaim > without being burdened by the additional overhead of compression. > > In our handheld devices, we found that applying this mechanism under high > memory pressure scenarios can increase the rate of pgsteal_anon per second > by over 260% compared to the situation with only kswapd. Additionally, we > observed a reduction of over 50% in page allocation stall occurrences, > further demonstrating the effectiveness of kcompressd in alleviating memory > pressure and improving system responsiveness. Yes, I think parallelizing this work makes a lot of sense. > Co-developed-by: Barry Song <21cnbao@gmail.com> > Signed-off-by: Barry Song <21cnbao@gmail.com> > Signed-off-by: Qun-Wei Lin > Reference: Re: [PATCH 0/2] Improve Zram by separating compression context from kswapd - Barry Song > https://lore.kernel.org/lkml/20250313093005.13998-1-21cnbao@gmail.com/ > --- > include/linux/mmzone.h | 6 ++++ > mm/mm_init.c | 1 + > mm/page_io.c | 71 ++++++++++++++++++++++++++++++++++++++++++ > mm/swap.h | 6 ++++ > mm/vmscan.c | 25 +++++++++++++++ > 5 files changed, 109 insertions(+) > > diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h > index 6ccec1bf2896..93c9195a54ae 100644 > --- a/include/linux/mmzone.h > +++ b/include/linux/mmzone.h > @@ -23,6 +23,7 @@ > #include > #include > #include > +#include > #include > > /* Free memory management - zoned buddy allocator. */ > @@ -1398,6 +1399,11 @@ typedef struct pglist_data { > > int kswapd_failures; /* Number of 'reclaimed == 0' runs */ > > +#define KCOMPRESS_FIFO_SIZE 256 > + wait_queue_head_t kcompressd_wait; > + struct task_struct *kcompressd; > + struct kfifo kcompress_fifo; The way you implemented this adds time-and-space overhead even on systems that don't have any sort of swap compression enabled. That seems unnecessary. There is an existing method for asynchronous writeback, and pageout() is naturally fully set up to handle this. IMO the better way to do this is to make zswap_store() (and zram_bio_write()?) asynchronous. Make those functions queue the work and wake the compression daemon, and then have the daemon call folio_end_writeback() / bio_endio() when it's done with it.