From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85545D5E141 for ; Fri, 8 Nov 2024 06:51:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C9BA76B00A3; Fri, 8 Nov 2024 01:51:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C4C2D6B00A9; Fri, 8 Nov 2024 01:51:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AC4B36B00AA; Fri, 8 Nov 2024 01:51:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 8C8606B00A3 for ; Fri, 8 Nov 2024 01:51:26 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 0E90EC09FF for ; Fri, 8 Nov 2024 06:51:26 +0000 (UTC) X-FDA: 82762005438.23.C05D2A8 Received: from mail-vk1-f171.google.com (mail-vk1-f171.google.com [209.85.221.171]) by imf02.hostedemail.com (Postfix) with ESMTP id 583C38001B for ; Fri, 8 Nov 2024 06:50:15 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=I9Q11Lsn; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.221.171 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1731048559; a=rsa-sha256; cv=none; b=rOCTnI3FU69h/bqVN/K8fQjBPfT77C8DPgeDfwxhq28JHWqJYx3CobEhm3CE9C5SE+7FOe OL1Oh7K89sHrPFRxeIsVzTHo0YuTEHPICyvbZd1saUSEqhuJmHQO82vvY67OR7VgYuhXMR bRQAfymoyE5Xw+Mrx1qwojvynk0JNoY= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=I9Q11Lsn; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.221.171 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1731048559; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=u/x8OS/dbVQbN/39ZA54E6EOHTTw1AI3bxvOtuzpiWo=; b=nIDs0r3eE1V2B6ReIxiyRMjuEk85ZdLVxDbFQ5+e0HkjxLSGl3ygpJczLdUbom8J1MRAFt HCq0EYsHz32D6dJ2zEHdDAFjDD0HfGaXX5GDBENKChL+qY0Tbm7dvzri42KOdWP3ho5Btk HBoq022J4AJsRxv/jqaraHFL1p2Fxec= Received: by mail-vk1-f171.google.com with SMTP id 71dfb90a1353d-50d3365c29cso2036925e0c.0 for ; Thu, 07 Nov 2024 22:51:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731048683; x=1731653483; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=u/x8OS/dbVQbN/39ZA54E6EOHTTw1AI3bxvOtuzpiWo=; b=I9Q11LsnoMiq/wGOU0m+uE4ZUcpCFD2ktrBkRYcMVjivlf7s2fJSPhlZe91Kz1zX4m SMXFUy2etrnTmhVfNtwUClVrD//SJ/3KnLH352Hau+iJw3hP/bXAPpJ918p8i/kguowi 45oSBfa0Zve+E+hTGGrat78FBTCCLJxTaFhKT77nIfX5Z1wkC00BaER1wpZdKm095V6Y MbDLoV4PZUaDSSToJXeZMzcLE9nOCZYHT6PopJ2XitvFA7eDYU1OP0E70JiO/z7hK8Tz NxgVSAAlJonBrE5ciHYoatuWv5EBVxyugnRUphqPY6aM1OdKjC9T6G3Psv/2JmSKaXEI DOAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731048683; x=1731653483; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=u/x8OS/dbVQbN/39ZA54E6EOHTTw1AI3bxvOtuzpiWo=; b=TEfPrv+im1F3hnwRS/6wtS0fxTLtXmz5JC3Rhi4uafT2IYExyFOEbCEBHBsEQ61k5i VB8kXcDLN3LngCjvgiTGM4RNJEkuVR3r0g41uhqglypUFvOKyaMXBIC+40cj/CC+H1Kj YKlIo2qgEanvmyij6hsC2Calk8soX6HJoJXn3MlV8kStsR/VQdDUC0QAG7uNSvid0lpZ jQEGCUJsAPLn5DVBEkYrs21SW3ssIviiLWO+lsgVC47fFbdpEbiQb3V2pnpe2dsOFh9I GupMf9/LAY/G9/t0KuI0QfxnKN6Gybt2dJX3kUvL1OlRS9aKWom5yAd0gpOMZpD3kJwX Ekyw== X-Gm-Message-State: AOJu0YzP75U+ii04+ksCWUnB4NN87sMnKJLCSYX7YXodgoa9EOkfEPCt HjUR2F7hZtqA7TI33C64ChXRippHPig/S2RVdHn4g9s16+epsvDyriZo1bXJje5PnjeztkRJjtA wdA52YvSt59DgSWjImg2+u00+8no= X-Google-Smtp-Source: AGHT+IH7kUuIZa2vfKwPYPTg9nq9/C2sMTAeKq8r3tJMucaNqEigVgTz4bKbe03UFM9znsRNLTpYPaK+o9Pzpot+Wqw= X-Received: by 2002:a05:6122:1d56:b0:50d:6cfc:ac4d with SMTP id 71dfb90a1353d-514026152dcmr1169824e0c.5.1731048683318; Thu, 07 Nov 2024 22:51:23 -0800 (PST) MIME-Version: 1.0 References: <20241107101005.69121-1-21cnbao@gmail.com> <87iksy5mkh.fsf@yhuang6-desk2.ccr.corp.intel.com> In-Reply-To: <87iksy5mkh.fsf@yhuang6-desk2.ccr.corp.intel.com> From: Barry Song <21cnbao@gmail.com> Date: Fri, 8 Nov 2024 19:51:12 +1300 Message-ID: Subject: Re: [PATCH RFC v2 0/2] mTHP-friendly compression in zsmalloc and zram based on multi-pages To: "Huang, Ying" Cc: linux-mm@kvack.org, akpm@linux-foundation.org, axboe@kernel.dk, bala.seshasayee@linux.intel.com, chrisl@kernel.org, david@redhat.com, hannes@cmpxchg.org, kanchana.p.sridhar@intel.com, kasong@tencent.com, linux-block@vger.kernel.org, minchan@kernel.org, nphamcs@gmail.com, senozhatsky@chromium.org, surenb@google.com, terrelln@fb.com, v-songbaohua@oppo.com, wajdi.k.feghali@intel.com, willy@infradead.org, yosryahmed@google.com, yuzhao@google.com, zhengtangquan@oppo.com, zhouchengming@bytedance.com, usamaarif642@gmail.com, ryan.roberts@arm.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 583C38001B X-Stat-Signature: czdec37myp5aferk5urm4nb1xsf6fr84 X-Rspam-User: X-Rspamd-Server: rspam05 X-HE-Tag: 1731048615-940682 X-HE-Meta: U2FsdGVkX19dfg5sa1KrEmx/hKeHvEmTf6F4XPkqSuwXYthLNBUqe2ieD9Po+o1qKtBM8Vz/j8ilLI6YlmAnJ5iCjtLyS07CWKtSZh/+IiMmwEuJiEwxoe78fM2tRTgGIsZoLVr2Z234b2UvJAhyE+oI1D3UOnh36QDrMmjIx04MkfX0CPirK3tnUf5urcSjW5w7+dNIgpvPFZW9b9C3Cf1E5sanKSWy1o2U/gDWeP1ZA8Ayqv+iuvJesl8eOVK5ntesaXNsmHrLkmVvUpxh4aZ0izfOmlKDKQSmpCsPbxpPKjpYIWY6tumn5P1SL9CtQ/cOuo2LN/TJGfupRZKMAQ5sNP1nFt1vHQ8KLHppSUNF+IvNt23gZ7R1rizVGe1zFPAzTwXE3ZtMdgOpRo9QK1N50baIrJ5DhcZFd5LK88m9t4jq7dHQxKlHuBUIPxFaEnfWUjhB+mnooj+tlqqC5JT0uNhBpCKBF9M/p+HKRODjH1x33DMSRgX+eVhnyjNZ0GZ5rWvJupJmnmEKHsznwrWCm0FdP0HuS752iv3GiOTrnrINBpEvCD5fskpjFOfSE52Pj9rgpDPncbsCrhHKyboSV+6umnz/Zw0sPcdWflT8olaVJhgtmDhjW3gb5xtyQmjaIJnokXFJxt5a1fEH+mDd9vAsHQrt8PyUjYoEMHEsOBw424WZkMEoycQ7koq92F0dcB0y/mrT2ccvMxYo2KjsDTvW5YJgaLtnkxBNiGVBywO0i+d694SKiuyJGa/wBTS5lAOLX0NXTT5X0wPYOz27XQuZKXNEsFJ7jH3Nj8yQ9Fx6UY4dTi5Kp2B8+mw5+FM43MTCTvu4Qt/8vwvOLI72n6hfp8AUyV0fEAXdmSH6aROMNLdIh0VULLsEaAY4JgX3MGStD1c73duMG6FkPAAPjB1rvlmURul90LHDqiGq5ktmLRKgbTdshcVRZUv7gtVP0pzYT2CnKTLVLXx zYvAvsND Y64MdcsThSCbhFs/gODGrvnhmYQPLxAJY5WZZ/Vx4TIpXqCRPhKOLRvbyk0zD8FIV2AV9TZTHnxGpG5ubaxQrQwgCwIt+J3SQSqSKh5RWl637YGdqE/LWkBkl/LN7tmnnpTGS2z8KqAoiSTQbRh6UGWl8Kr5bhNa7WJGXxCzRHEGBH5BBWzpbE0t+L4Gc1tLQ6bBEuiDPGIyaEY5/iJcgULP1YcRsOttGFZcL/Vjt0C45HD84nNs3xZ0zU04km5N+mfyXtP73RkJVf0Fc8CcDDbnLMLveeKXGyVA1LShKbTozBu11o70/UeOSJLJhT4o6hdooHy0P5Qzumow3fdlAYXrqKqQI3QSNiqgL1tY0PJ87Su04n1CSEYFwHRPHQFU0wxNXKdva0AwX5m4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Nov 8, 2024 at 6:23=E2=80=AFPM Huang, Ying w= rote: > > Hi, Barry, > > Barry Song <21cnbao@gmail.com> writes: > > > From: Barry Song > > > > When large folios are compressed at a larger granularity, we observe > > a notable reduction in CPU usage and a significant improvement in > > compression ratios. > > > > mTHP's ability to be swapped out without splitting and swapped back in > > as a whole allows compression and decompression at larger granularities= . > > > > This patchset enhances zsmalloc and zram by adding support for dividing > > large folios into multi-page blocks, typically configured with a > > 2-order granularity. Without this patchset, a large folio is always > > divided into `nr_pages` 4KiB blocks. > > > > The granularity can be set using the `ZSMALLOC_MULTI_PAGES_ORDER` > > setting, where the default of 2 allows all anonymous THP to benefit. > > > > Examples include: > > * A 16KiB large folio will be compressed and stored as a single 16KiB > > block. > > * A 64KiB large folio will be compressed and stored as four 16KiB > > blocks. > > > > For example, swapping out and swapping in 100MiB of typical anonymous > > data 100 times (with 16KB mTHP enabled) using zstd yields the following > > results: > > > > w/o patches w/ patches > > swap-out time(ms) 68711 49908 > > swap-in time(ms) 30687 20685 > > compression ratio 20.49% 16.9% > > The data looks good. Thanks! > > Have you considered the situation that the large folio fails to be > allocated during swap-in? It's possible because the memory may be very > fragmented. That's correct, good question. On phones, we use a large folio pool to main= tain a relatively high allocation success rate. When mTHP allocation fails, we h= ave a workaround to allocate nr_pages of small folios and map them together to avoid partial reads. This ensures that the benefits of larger block compre= ssion and decompression are consistently maintained. That was the code running on production phones. We also previously experimented with maintaining multiple buffers for decompressed large blocks in zRAM, allowing upcoming do_swap_page() calls to use them wh= en falling back to small folios. In this setup, the buffers achieved a high hit rate, though I don=E2=80=99t recall the exact number. I'm concerned that this fault-around-like fallback to nr_pages small folios may not gain traction upstream. Do you have any suggestions for improvement? > > > -v2: > > While it is not mature yet, I know some people are waiting for > > an update :-) > > * Fixed some stability issues. > > * rebase againest the latest mm-unstable. > > * Set default order to 2 which benefits all anon mTHP. > > * multipages ZsPageMovable is not supported yet. > > > > Tangquan Zheng (2): > > mm: zsmalloc: support objects compressed based on multiple pages > > zram: support compression at the granularity of multi-pages > > > > drivers/block/zram/Kconfig | 9 + > > drivers/block/zram/zcomp.c | 17 +- > > drivers/block/zram/zcomp.h | 12 +- > > drivers/block/zram/zram_drv.c | 450 +++++++++++++++++++++++++++++++--- > > drivers/block/zram/zram_drv.h | 45 ++++ > > include/linux/zsmalloc.h | 10 +- > > mm/Kconfig | 18 ++ > > mm/zsmalloc.c | 232 +++++++++++++----- > > 8 files changed, 699 insertions(+), 94 deletions(-) > > -- > Best Regards, > Huang, Ying Thanks barry