From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4048DC7115B for ; Mon, 23 Jun 2025 05:16:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C4A796B007B; Mon, 23 Jun 2025 01:16:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BFAF36B0093; Mon, 23 Jun 2025 01:16:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AE9A96B0089; Mon, 23 Jun 2025 01:16:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 930408D0001 for ; Mon, 23 Jun 2025 01:16:56 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 3474A81A74 for ; Mon, 23 Jun 2025 05:16:56 +0000 (UTC) X-FDA: 83585506032.26.62ED6D6 Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) by imf12.hostedemail.com (Postfix) with ESMTP id 40E4B40006 for ; Mon, 23 Jun 2025 05:16:54 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=b5RStwnx; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf12.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.216.53 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1750655814; a=rsa-sha256; cv=none; b=0VBHRdic012OyERR44hNFg4scHRpxRYWEAHyB2yWiVyw4DJ77SLmvW0+aiYYuo/sq4Biuy Mm3yvI6VgXQYCKLVual7Fg2C3KTVaIWe29WkPDfOofnm+D/WPiClNQSaJeUmC/hcaWvaNb ibM8n2cz5UzVWstgRbr9AzwXwEnZsHg= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=b5RStwnx; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf12.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.216.53 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1750655814; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=En0pq+jCzMYs3olwKBVBf+cE3fdHEMn7c2+JHjmC2Bc=; b=g6KjdnflQNl6uqz06CS+l9gsSIZbsn1KJQcj9MnQtwk+To7XKRlEAjw9dcIHA1Zi4bKrb+ HFN/5J8AQpJWQ0pvyw3zB6Ox57kOFzCAN0GfAhG4wcRNc3FM/jXJ8GFdNxJTnS4icVrDtw fs4mD7my1X16duHyDpTn1PqDRLpN+YI= Received: by mail-pj1-f53.google.com with SMTP id 98e67ed59e1d1-311e2cc157bso2886096a91.2 for ; Sun, 22 Jun 2025 22:16:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1750655813; x=1751260613; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=En0pq+jCzMYs3olwKBVBf+cE3fdHEMn7c2+JHjmC2Bc=; b=b5RStwnxccjuB0hA5XYF8ECYZiDwy1uL1OsCEjAFrNms4ogwBbZUEB/7Ng7pAzNh+q CjRgQP8JEzpvv4IzoJ/201/Qm+5RirH01+/ZT7nj5Tbnj7GoNwtBQ6xy2MUDN7S1W1Wp DnEBPZhnfvjix1Nqt5eaAEkXdSqlZHGNrjQn93cib3zyuVhDmJmHDjP/6jVemfZDi6wo AIzipytLUZoUEtlOpAXTGkDpwu0/8l9rsfcaNefAq8jc4IlOIA8QIkwfk9JoYsSdhcLi FNMjmvFSFNXE7w9fEPVPBLfmKxG7LGYlylN+pY5rm8LQVsAlBrXYES9VG3ZLbvvxIwcR dRoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750655813; x=1751260613; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=En0pq+jCzMYs3olwKBVBf+cE3fdHEMn7c2+JHjmC2Bc=; b=lFrhMVGA+VU+S2Ydfd3xGO32zUNe1sVYns9mbglenFKEE2a5PRU6nzee5NGXtmDwY5 sX9AK+3r5owtIciNCgkT57+8dPWJFGInHCfYzUJDNps9zkZ7t0dBQbsAwFQiiSe19RFP 9zImNtrHk6bHV7x6OKRONC52TMggzG/T+BHmtJ6/Ft6lAB2K/7qmXfIHnFo6QZq4m4mA tcUJYvAUH/vhytmIgJ4M7fKF5/z54U4PyPFS1PiHppsRgk0eZup5VLo9i/Yabl2WNKf3 ePGt5UxcIr2fR81q8ovcT5U5iCVEG/gIsuaHQSIvl+eGqg7GaFphFC5QeqPFtCv4Jj9W al2w== X-Forwarded-Encrypted: i=1; AJvYcCXR9th/A/iAI/4PyOFLK+UJzY4+bLCRe/lIqYdJieXl+xFVZPD9gATzfyD6PZKSeFuaD1xHyzZoWg==@kvack.org X-Gm-Message-State: AOJu0YxYD+93pSW/LSASZvsvDyRo2+ll6/rdtDihiWoh53DIK7RzPik6 aW82sXedjZiVytFgIm/z4C39cxj5uXmH0upPwsPgXKmluYOLY6KJ1rCX X-Gm-Gg: ASbGncupX7FUIQc/WJc2L08w99mjdsmsibYvnmkoZ8mlGw3Fb7rl5IbaCPYUvVXlHpV fPy+CmZKzgbkrpd5PVbHRCzTviKUADpH5p4sk/CQAPrjiLd2zW8uWTzxHMVwkR3coNL9YuKETst B2UCH19xQejbonuleIfp0smtS+YShnlL+5cc1UxdcazuIHTVhUR48NxsFhRukwnMyzm/CVMQ+I2 tWmARuMGdg48NMPA7Ov/xdPDb7oYkUEmkNijLdLLlr2/5j0T6zIUehL29nhKARFvzWKHZZvrbMM zaC5+3qkzpAYTwAAEi6Ye11P/epxxUeJAAbTKfzRtGChpY3qkEGbVoY0rfHU/x4+/RWLAQlC X-Google-Smtp-Source: AGHT+IG9TUKXoeyvlp4X4lyFMSjBOFbruQUMqCiCZF+ohvgS9TLgNer/JkiWcjcRvc8wDgqad/VFHw== X-Received: by 2002:a17:90b:390f:b0:311:e8cc:425d with SMTP id 98e67ed59e1d1-3159d6470dbmr17802629a91.10.1750655812825; Sun, 22 Jun 2025 22:16:52 -0700 (PDT) Received: from Barrys-MBP.hub ([118.92.21.225]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3159e06ce4bsm7136997a91.38.2025.06.22.22.16.46 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 22 Jun 2025 22:16:52 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: nphamcs@gmail.com Cc: 21cnbao@gmail.com, akpm@linux-foundation.org, andrew.yang@mediatek.com, angelogioacchino.delregno@collabora.com, casper.li@mediatek.com, chinwen.chang@mediatek.com, hannes@cmpxchg.org, james.hsu@mediatek.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mediatek@lists.infradead.org, linux-mm@kvack.org, matthias.bgg@gmail.com, minchan@kernel.org, qun-wei.lin@mediatek.com, rppt@kernel.org, senozhatsky@chromium.org, sj@kernel.org Subject: Re: [PATCH] mm: Add Kcompressd for accelerated memory compression Date: Mon, 23 Jun 2025 17:16:42 +1200 Message-Id: <20250623051642.3645-1-21cnbao@gmail.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspam-User: X-Rspamd-Queue-Id: 40E4B40006 X-Rspamd-Server: rspam10 X-Stat-Signature: f5n4ra3meb9dcwugomkm859r8jbud3r5 X-HE-Tag: 1750655814-339879 X-HE-Meta: U2FsdGVkX1+CZ/l11qpu6+uT1C+AucYkICkdHTp4G3esX0aH6k8ddAdwB0XwwNw9M6sF2c4xwv4admSFyVj0L0N5fVCiXeMZk/LxNrOg4O8PGX90uVrkdFslJchJcGo9u8DufWj0iHN8DnKsKbOp3L9VAR1V1WCIHu6gsioWBqgd+pWX9bu0EWjAFCe+jUdfudYLYDqCMurNhqwf6BzbJ3coXzXVafzVHFLvWx85WwLB8yw7qSJp0nfnZG9/05+sWql1fRAHQedG1FtivrQ1GiBhD0RI+BWVSmJMhtgk/jAAAYFDWH79TXUvvRQmbtH3mZLKvTWihj1/63LLAxS8kx6zjFOOdN4BuZs3b3fNYNTd027q++aAx2LHIma+zKG5LsmsRS9AWB1CDGVbzcfskQbD9PHGW/rhYoRUw4Yzie7WFcgckQsaEcGpD6VxgJWAVtVC3KNY9iLnu5V9lM7vz0FGjgwY7BJ5RzJ/aI4CmSctsQ7ABzdzyJsmPM8DK+NGjn4FTTFGQbRpA4PNkUd5NxYyRrhHAc4PltfkbTJTTrj4yyFi1+kCpN37GBa8VmGmeed6d9fPTQZ/Vm18qAggy0rgYg2BAbFj0NWL4VT/QOoTgArEBsUvgDH5aXi7xmjxMzQTMHa1IgSfPlyokUexP1XtJoGgssnIkWT7gjN1FVBDcJRMycI71AHq0qjZ+V8ZN86ITZ8i7GKpt2octElWE/dfYQR8gJdI3pIRWilTmELYpShJGDlFPV38Jc2kjqlS9BonbKvW2MZ+Uqp1oXurS4B115opPgVBGrt1FCjbBZbuuj6M4NhPBmCQ0IgxcnfmLF5DHqkP6GySJNgZ2Jc/ArdAQMhBHmyu96BNzzlUhJhCubzIDE9oKF1eCS7ZA6l4FRceF3F9HoaH1ixGtuK9ZKM1uL5ThhCfdhDTiHF6SOnNxL1Whn4/Vznv4gnrBGLiBK3dtlUMC4aLemVSI9I T00IUhNy y1Cufh4Sx2NGftcHh23tll8bNI4RG7bmsxp1CyXM5UXJlYTRZRy+pdZp9CR9BuRs3qA2QC+745i+047Yl2uBddQaHbf0u0hg9WeP2Zm70ocCpseIbZxrJC/wqQhw/N8G57lN+VSVbib25nEZDUk9QancVqPzbWOKHszN9+2CAaBm/1BPqy7JVkCkYwrDDUweHna4DA1Jp/sn7uxpaqLDinWQsdS36+6htLMeu4hlg6j2rFes3MFZooeU7G0qNJCbmBzX/dHLpy90i9JVS+B0xF2Iss7uxCjf7DOvv+kLX48dpHFQyavrwqaep2GIsCQTtxzW25m40UiFHh/Is/UKqgfOB3XT+3nueZHIGApCnl84kQZp8kHMU1IWU+zLQeMN+VComimakazAJ2JGVb6CcCT9sV0JxBlVeDKgMN4D6QHCSVe3tdOkF7JgUxl4QI0k9PXQMDM3+1IiRAC5grx8qEKvJHV45jLh0+53/ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi Nhat, On Wed, Jun 18, 2025 at 2:21 AM Nhat Pham wrote: > > On Sun, Jun 15, 2025 at 8:41 PM Barry Song <21cnbao@gmail.com> wrote: > > >> > > >> That seems unnecessary. There is an existing method for asynchronous > > >> writeback, and pageout() is naturally fully set up to handle this. > > >> > > >> IMO the better way to do this is to make zswap_store() (and > > >> zram_bio_write()?) asynchronous. Make those functions queue the work > > >> and wake the compression daemon, and then have the daemon call > > >> folio_end_writeback() / bio_endio() when it's done with it. > > > > > +1. > > > > > > But, > > How could this be possible for zswap? zswap_store() is only a frontend — > > we still need its return value to determine whether __swap_writepage() > > is required. Waiting for the result of zswap_store() is inherently a > > synchronous step. > > Hmm, I might be misunderstanding either of you, but it sounds like > what you're describing here does not contradict what Johannes is > proposing? It seems contradictory: Johannes proposes that zswap could behave like zRAM by invoking `folio_end_writeback()` or `bio_endio()`, but this doesn’t align with actual behavior since zswap_store might not end `swap_writeout()`—it may still proceed to `__swap_writeback()` to complete the final steps. Meanwhile, Qun-wei’s RFC has already explored using `folio_end_writeback()` and `bio_endio()` at the end of `__swap_writepage()` for zRAM, though that approach also has its own issues. > > > > > My point is that folio_end_writeback() and bio_endio() can only be > > called after the entire zswap_store() → __swap_writepage() sequence is > > completed. That’s why both are placed in the new kcompressed. > > Hmm, how about: > > 1. Inside zswap_store(), we first obtain the obj_cgroup reference, > check cgroup and pool limit, and grab a zswap pool reference (in > effect, determining the slot allocator and compressor). > > 2. Next, we try to queue the work to kcompressd, saving the folio and > the zswap pool (and whatever else we need for the continuation). If > this fails, we can proceed with the old synchronous path. > > 3. In kcompressed daemon, we perform the continuation of > zswap_store(): compression, slot allocation, storing, zswap's LRU > modification, etc. If this fails, we check if the mem_cgroup enables > writeback. If it's enabled, we can call __swap_writepage(). Ideally, > if writeback is disabled, we should activate the page, but it might > not be possible since shrink_folio_list() might already re-add the > page to the inactive lru. Maybe some modification of pageout() and > shrink_folio_list() can make this work, but I haven't thought too > deeply about it :) If it's impossible, we can perform async > compression only for cgroups that enable writeback for now. Once we > fix zswap's handling of incompressible pages, we can revisit this > decision (+ SJ). > > TLDR: move the work-queueing step forward a bit, into the middle of > zswap_store(). > > One benefit of this is we skip pages of cgroups that disable zswap, or > when zswap pool is full. I assume you meant something like the following: bool try_to_sched_async_zswap_store() { get_obj_cgroup_from_folio() if (err) goto xxx; zswap_check_limits(); if (err) goto xxx; zswap_pool_current_get() if (err) goto xxx;     queue_folio_to_kcompressd(folio); return true; xxx: error handler things; return false; } If this function returns true, it suggests that compression requests   have been queued to kcompressd. Following that, in kcompressd(): int __zswap_store(folio) { for(i=0;i