From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49A96C19F32 for ; Fri, 7 Mar 2025 19:34:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4E1FD6B0083; Fri, 7 Mar 2025 14:34:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 491CE6B0085; Fri, 7 Mar 2025 14:34:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 35AA66B0088; Fri, 7 Mar 2025 14:34:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 1961B6B0083 for ; Fri, 7 Mar 2025 14:34:42 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 4D2B91A0C0C for ; Fri, 7 Mar 2025 19:34:43 +0000 (UTC) X-FDA: 83195757246.28.8877246 Received: from mail-ua1-f52.google.com (mail-ua1-f52.google.com [209.85.222.52]) by imf01.hostedemail.com (Postfix) with ESMTP id 6BE794000C for ; Fri, 7 Mar 2025 19:34:41 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=SwX2GnD2; spf=pass (imf01.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.222.52 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741376081; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fsLIF0KuSa0+9gIStTKUSkGySCuhFSqgeXthFQ3bTsE=; b=BTMn59w+Sv8I/LLFE5qlEBOtGquZKiBjnNpGXee1A8UsH+tTv9aKvQErlDW532edS9mA0L rsb3543Kqp+88l2RHrxTHdoN+yxVINfg1q39+F5oXlsM8UMI/4P4gwOopp47TdWnn/wQ77 BGFVJuwnLbtQCbR8nOjsMA/kjHm/UMY= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=SwX2GnD2; spf=pass (imf01.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.222.52 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1741376081; a=rsa-sha256; cv=none; b=sOWvKSDjPA+LncIY2Eb8Kop4tyuCBOL6PDzdIk/xHVJ87uecwe/CkbPn5skmmnZjMx/w0a xjPBGQCJw/Y36euBy1yyrH6sliAl1bMQ/SYS8Znh0+lsWbQ2j0uBFwxOpNU4Ejo+C5M3eg y8iMvXD+nxkrYZfc018AQfhOAvRSUwA= Received: by mail-ua1-f52.google.com with SMTP id a1e0cc1a2514c-86b9b1def28so1904123241.3 for ; Fri, 07 Mar 2025 11:34:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1741376080; x=1741980880; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=fsLIF0KuSa0+9gIStTKUSkGySCuhFSqgeXthFQ3bTsE=; b=SwX2GnD23J3vP72VtSNRGw1Do3n+XXNlUg4UUxxZARrjk4WCwVv6EBuvpduplI2NI/ cP8bOv/NvBCNxfYlI41zgunyNjM1pXEBtZWugyVQ9CoQUvdKHr2shsU1HQY1AekVd2h/ OmmzJ8OX1F41j00rLjA53tZ4bq2DK9CusSEjY4Ts1nykA7JgbYABCK53+oXu8EN374a8 HxVZCXG8ZsR19WbT24srHecJHQXcKh8qz01K0pTNThppMB/TPtm4/QvgmN1GD9l72wWL g9SfK8Q6BDNrqz+9oS5OocI/yUTsjU+WuYUlNguIegnPw33p6e9kzm1fYqVIF7FsF1Qj 00Uw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1741376080; x=1741980880; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fsLIF0KuSa0+9gIStTKUSkGySCuhFSqgeXthFQ3bTsE=; b=o5MtfshyBZ5qRCEyQtBcFamH7ITK+xwCM6qs4tu4mlQpP9A/9T+QK1x+xZszLUQmoU gENe+FuAlD5VEYnZRtGjavuUDXOqFlfWWZeeCGrDfk16ZFz/X/DVBEm/9cMNX2WqnKVa 5Y9lfcamLK9NXyGyC41pnkMk36mLufjJt+GaccRK0SEovwUcAYxhou6GUZPpIGgJYXsw xpUMlGn3A5EzrFcP4iShMXq3H7yzBoLFIwUUnwwUpP8j9Z74UQk8UyXR/p6q6GIVXTuJ BAAyP+4b+k0qp+Ji1gpoeAks5eVuT79l/q6ttndNyeddEy+IHvZepgxeX/AhlYTaBmPc v09Q== X-Forwarded-Encrypted: i=1; AJvYcCULwrMQH68wyiKFHPeI7GwUEzsd5wzmWeGX2RALJ8DTP8Jq8TxYaAd8kj4FYS0pHixHuHM0fDG0ew==@kvack.org X-Gm-Message-State: AOJu0YyjxgJT42jvnUZ1xW4DcHn6jmhEVjMbV6DJqPzDYPgWJbdADSmH AHAyxCbbgFqGaLrBN/drBrN5O9pCBNXXqk/ZqVJb9FNlTNbvry0LdKT+XHeQR9Opmet7X2+i7fR t147gLZ7XVrY0sZBCyDonK+1rli9mzw30ZNI= X-Gm-Gg: ASbGnctZwby66t5SwY7K5b01QBr7MNml00sT6sGvkPZxCU3nvfhZjuaiC5gk646mWrj 11Pm1ExUSB4PqxKB05b8YTlGiUu0uKli3l5saPfiyap2+pOj6YQaDQEKFtaSFUjHDU20VHm4JUN W2rZNbpi2iUuIHJ9b8hApjoKGZfg== X-Google-Smtp-Source: AGHT+IFgqqDqQofofV/JVROxQO86oLW8odJwAWvtAGFI3OGLPlYXWCYP7AEGD+cKOelEyOiLNufTRKzO83KiF2eNeL0= X-Received: by 2002:a05:6102:cd3:b0:4bb:e1c9:80b4 with SMTP id ada2fe7eead31-4c30a36790dmr4084956137.0.1741376069586; Fri, 07 Mar 2025 11:34:29 -0800 (PST) MIME-Version: 1.0 References: <20250307120141.1566673-1-qun-wei.lin@mediatek.com> In-Reply-To: <20250307120141.1566673-1-qun-wei.lin@mediatek.com> From: Barry Song <21cnbao@gmail.com> Date: Sat, 8 Mar 2025 08:34:17 +1300 X-Gm-Features: AQ5f1JpJm8Pz4qzzGj-4qBmXe9E99kwhewd_E_rbXmfvrZ8eGN5IKtbpRd0ER-s Message-ID: Subject: Re: [PATCH 0/2] Improve Zram by separating compression context from kswapd To: Qun-Wei Lin Cc: Jens Axboe , Minchan Kim , Sergey Senozhatsky , Vishal Verma , Dan Williams , Dave Jiang , Ira Weiny , Andrew Morton , Matthias Brugger , AngeloGioacchino Del Regno , Chris Li , Ryan Roberts , "Huang, Ying" , Kairui Song , Dan Schatzberg , Al Viro , linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, nvdimm@lists.linux.dev, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linux-mediatek@lists.infradead.org, Casper Li , Chinwen Chang , Andrew Yang , James Hsu Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Queue-Id: 6BE794000C X-Rspamd-Server: rspam03 X-Stat-Signature: gjcot713zdfuf1yyjssga5qn7hx66wy1 X-HE-Tag: 1741376081-276849 X-HE-Meta: U2FsdGVkX19ehsuXtfORGQmAnzkcQOVphUcceXtxM3oJSyX8kA70io/5m4gKj/zfXkm0F8aXFiVy6mbizGpGeHp1PI5I0wym2/jbXpXmJD/DLWl2GTbBMbKKy9YSn9hP06AG7cfQYkWK3+DaTb5RMALBxK/YPMXITAMls4H6Ldk4FdBPCt+J4fn+Z6ssUov+iOlV1oBkm9Zni+ujI2gxJIRHz7xcB4rZWWT2GgIz3su1uL9TGmFf68LK0wVzP/JDFQIcRLzu5vT3afB5OMzh1R//IAM/+9ALiu2RYruq9QhTYGL5lQnAgSFAI7TW6IhhRXaJAuJWAWDpCuFhQF5ynNDEA47x3hL1cnFrWVDrnoSkDuMRxB/qNvjIvPt4ZJCVBix4AhFg4IumvqnzQZQB0w+3fqFgcvqIs0Go1ee9lAlVcYxaAVtmk6HQ6jg448hVnH0m5rAB8MZdosQ3a3PR7a1fXl93ttfHmZnAogSddi/TQoa4iOlsvNu+feY43rqxmMfrCWo/cJf+tJhMohjGUgH9cgkl0Xzynau8T3jPeG37Ouyk5ZQblkA8Mc03yQiPdGjs2VOc5k0lJ1bwtlECle+sUOT3OQAvxFC4phq4GSquARupUv8dABUT6inUCV9mwXhZQnVXI8OG0Z7KIMyChBSt/47PM9aWMudoKH/xCG2giVp11MdHEnGOet+VNVqWPo/eETjusZXJa0Jo3rZmTUpArGqUvY6PI8roVwvrd07uhhm1vj7qnXzU7EEBYckpxSRqhmg0aPk2r49v0psdypHYvVaLBzlDg3uYX9E9k/sKr13Kw16HeC37nY1rKDO69nCFfgabawwSr3rZedwzlIP/h+mqI3/c+jJIoipfvMHcbBOyU18zbNeBalAlwoOujTh0U55Zzzb0SVPKOsPSWBwxrYrYgToOoJD4mOepV3hPKEncRX0n42lxiyT/zbDgDgu6nUO3Y2f9jP7F8hh +bdNZhru LgOBrGFRFwQFc0yWRpIWXMsB5rJmYusbM4zemhQVedtnMVS00N5+HviYvPT2myvrqkoezjqzmb0QZRbJN4CGtpx7hgJK0zU/4ovYoGiL72q94pQr5S/i8bC+8KQpF+RIKXAVjwrp/e+HvkAGVE5VeAVYTxzvnXoC7zre7LzKFaSSpNxpiCZ24Qov+Jjp35Xdic/fKtqyLBrxI3qqo0kTuDrpCTJqtHhAkkdjFjypva6Q6tuwxL1vPmxk9sg2RvhIZasyKBeVsb2PqVCfrMpdywi0aRJBU7GrsbBX5oPTMsn6H0JV5OKT996ASAvfb94SXXnpfXdpVQrCmK/l21sfBNrDPrw9gJLOuu4fwoB9s3R+cY5bsVpXQEKJ9XB4+We9xJ2jQgOryVabIG2mWmuEBQf5stJl3ItPSO5NQ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Sat, Mar 8, 2025 at 1:02=E2=80=AFAM Qun-Wei Lin wrote: > > This patch series introduces a new mechanism called kcompressd to > improve the efficiency of memory reclaiming in the operating system. The > main goal is to separate the tasks of page scanning and page compression > into distinct processes or threads, thereby reducing the load on the > kswapd thread and enhancing overall system performance under high memory > pressure conditions. > > Problem: > In the current system, the kswapd thread is responsible for both > scanning the LRU pages and compressing pages into the ZRAM. This > combined responsibility can lead to significant performance bottlenecks, > especially under high memory pressure. The kswapd thread becomes a > single point of contention, causing delays in memory reclaiming and > overall system performance degradation. > > Target: > The target of this invention is to improve the efficiency of memory > reclaiming. By separating the tasks of page scanning and page > compression into distinct processes or threads, the system can handle > memory pressure more effectively. Sounds great. However, we also have a time window where folios under writeback are kept, whereas previously, writeback was done synchronously without your patch. This may temporarily increase memory usage until the kept folios are re-scanned. So, you=E2=80=99ve observed that folio_rotate_reclaimable() runs shortly wh= ile the async thread completes compression? Then the kept folios are shortly re-scanned? > > Patch 1: > - Introduces 2 new feature flags, BLK_FEAT_READ_SYNCHRONOUS and > SWP_READ_SYNCHRONOUS_IO. > > Patch 2: > - Implemented the core functionality of Kcompressd and made necessary > modifications to the zram driver to support it. > > In our handheld devices, we found that applying this mechanism under high > memory pressure scenarios can increase the rate of pgsteal_anon per secon= d > by over 260% compared to the situation with only kswapd. Sounds really great. What compression algorithm is being used? I assume that after switching to = a different compression algorithms, the benefits will change significantly. F= or example, Zstd might not show as much improvement. How was the CPU usage ratio between page scan/unmap and compression observed before applying this patch? > > Qun-Wei Lin (2): > mm: Split BLK_FEAT_SYNCHRONOUS and SWP_SYNCHRONOUS_IO into separate > read and write flags > kcompressd: Add Kcompressd for accelerated zram compression > > drivers/block/brd.c | 3 +- > drivers/block/zram/Kconfig | 11 ++ > drivers/block/zram/Makefile | 3 +- > drivers/block/zram/kcompressd.c | 340 ++++++++++++++++++++++++++++++++ > drivers/block/zram/kcompressd.h | 25 +++ > drivers/block/zram/zram_drv.c | 21 +- > drivers/nvdimm/btt.c | 3 +- > drivers/nvdimm/pmem.c | 5 +- > include/linux/blkdev.h | 24 ++- > include/linux/swap.h | 31 +-- > mm/memory.c | 4 +- > mm/page_io.c | 6 +- > mm/swapfile.c | 7 +- > 13 files changed, 446 insertions(+), 37 deletions(-) > create mode 100644 drivers/block/zram/kcompressd.c > create mode 100644 drivers/block/zram/kcompressd.h > > -- > 2.45.2 > Thanks Barry