From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5598AD5A6DF for ; Tue, 26 Nov 2024 05:09:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AC4966B0083; Tue, 26 Nov 2024 00:09:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A74C56B0085; Tue, 26 Nov 2024 00:09:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9153A6B0088; Tue, 26 Nov 2024 00:09:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 704736B0083 for ; Tue, 26 Nov 2024 00:09:28 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 0307F12032A for ; Tue, 26 Nov 2024 05:09:27 +0000 (UTC) X-FDA: 82827067890.22.7B9C68C Received: from mail-pg1-f181.google.com (mail-pg1-f181.google.com [209.85.215.181]) by imf17.hostedemail.com (Postfix) with ESMTP id EE01740011 for ; Tue, 26 Nov 2024 05:09:22 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=BhXasDWI; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf17.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.215.181 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732597762; a=rsa-sha256; cv=none; b=Gy8vIS7pjfWmwJ/WaN1aTbndVXcsI0Zg9vjS00prkRExH0moDNIjd9FRXGfLlaDXGWyWZy KW5LTCaBpEHqsQxDOf31fvGKDNdLkuXpolVz5bqSCwuaSEH93nEmjrmVvk3jiQ4r/jnsTM Wjmx3xH8GZOpvKM33BCYuBSK6LQrih0= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=BhXasDWI; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf17.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.215.181 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732597762; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=A+OG1QCsOSf4XvgOD+GJSm1qpPZxTwxxd1aIUTNljBQ=; b=sWoSLcOsVBwE4/6w/4FJCGJehTD1Zh/1PDw/tmL+PEqbRqGWwNGo10+WnC/guttfldAPbx kFH+9yIoOdrC0ynY+//pIg0/2varQDHipYQiRt+dLuOO0jpGAlq2H8jMupSghWp0zYRHwz YIwzVee45SxA7hqmxnmjkM1c3hZ4Bec= Received: by mail-pg1-f181.google.com with SMTP id 41be03b00d2f7-7ee51d9ae30so4021294a12.1 for ; Mon, 25 Nov 2024 21:09:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1732597765; x=1733202565; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=A+OG1QCsOSf4XvgOD+GJSm1qpPZxTwxxd1aIUTNljBQ=; b=BhXasDWIQtCFzi9PJA6mu5ae6+dGnsDHZxZ8GvXvmr9fYQ8i/tXoMzRT6+Fjnmpz9d wLSNuUje7RovbJAuFlkNYERyXQxowTVZM5m6RU8nfaSpwn7glY6P9HmGhhFk939DmQTb UizNbv1IdGFdHcuN1Xd3JDVYxZG/YpUToT950= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732597765; x=1733202565; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=A+OG1QCsOSf4XvgOD+GJSm1qpPZxTwxxd1aIUTNljBQ=; b=fTIIIfu6+JPT/Ni0fCYV6l2MKGK2PgCrGp9gy75XDbtaXA7u+7YhJ2iIy6iJM8WIuL RLn7qxfMcVs2F4G//61QLPZ91wu1CX8OzWq2Imqt1f+spb0C6TGu++kCQvZa2OCbNGSi KzFVH+Ld7DuEAA5C1UmclFJryb4M5GL+e13ShXB3mq1SUuT1ikSzXZ5DgGb30oW/ZnFB cqJLCQBYZ0PK6ypG6fJ/C4JSpVghzA44PYqaYTnqekixrnKzDoQkLqoXE14dSnqa3lTq +Xh4/la3vOVwWHE7mCZ2inxuU6AwEE2x6uh/clQE7mgWVBqVHbVIlw/oJVNWNaS7P8KZ yeLg== X-Forwarded-Encrypted: i=1; AJvYcCXE/IhIbcZcxIg4x4fapYpH99G+BBx7smSjylTrfYBTVKE9KDhrgopqX7sD031YHbVfWYG3X4380w==@kvack.org X-Gm-Message-State: AOJu0YwEE7Z/QdnqY0CqFP7F9OBYyZAh2pzoGqBn8l6qwXQeFORsgWo/ fFLInKuwZs9/dgK1FsC4VWZ0REYzksVSFXuHV8VGcfaFRiuHKp18ThuzldGhmg== X-Gm-Gg: ASbGncvLqeE8iyskrnCeoPSpEkRt/V+qcS3+unPyhVxL921OIKuyecluTjx96Hmy1cx 5/xiPKV35ZbPa+TptMJXaK9/S0IncQNeNMd5YL2I/SL1NzxOGdTLH8cYjsHd6bxzqOmPciFNcSe 5eB8So4IMBGRhOML417RSFZmIWDTgRQbwX65yfvFPEtFYtKNDJIRC5n0JqB0FecuwzIAh6ENObS Vy8cMv9wCTc9L61qXN0fhJ63MM7SodpNuKCrseVAoR7uGsI301p X-Google-Smtp-Source: AGHT+IGETkofIRaSA+TbiDN7k3Ev3Htnf+AZCTMoSGNyrFrov3FuRj6a4IWrq/2IHSdGeem5NKIjgg== X-Received: by 2002:a05:6a20:3d89:b0:1e0:d9e9:a2c7 with SMTP id adf61e73a8af0-1e0d9e9a633mr652313637.15.1732597764637; Mon, 25 Nov 2024 21:09:24 -0800 (PST) Received: from google.com ([2401:fa00:8f:203:7631:203f:1b91:cbb]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2129dc207bbsm74562855ad.228.2024.11.25.21.09.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 25 Nov 2024 21:09:24 -0800 (PST) Date: Tue, 26 Nov 2024 14:09:17 +0900 From: Sergey Senozhatsky To: Barry Song <21cnbao@gmail.com> Cc: akpm@linux-foundation.org, linux-mm@kvack.org, axboe@kernel.dk, bala.seshasayee@linux.intel.com, chrisl@kernel.org, david@redhat.com, hannes@cmpxchg.org, kanchana.p.sridhar@intel.com, kasong@tencent.com, linux-block@vger.kernel.org, minchan@kernel.org, nphamcs@gmail.com, ryan.roberts@arm.com, senozhatsky@chromium.org, surenb@google.com, terrelln@fb.com, usamaarif642@gmail.com, v-songbaohua@oppo.com, wajdi.k.feghali@intel.com, willy@infradead.org, ying.huang@intel.com, yosryahmed@google.com, yuzhao@google.com, zhengtangquan@oppo.com, zhouchengming@bytedance.com Subject: Re: [PATCH RFC v3 0/4] mTHP-friendly compression in zsmalloc and zram based on multi-pages Message-ID: <20241126050917.GC440697@google.com> References: <20241121222521.83458-1-21cnbao@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20241121222521.83458-1-21cnbao@gmail.com> X-Stat-Signature: ypoa987doj6eshxay1w7867y56x9cmc9 X-Rspamd-Queue-Id: EE01740011 X-Rspamd-Server: rspam08 X-Rspam-User: X-HE-Tag: 1732597762-768645 X-HE-Meta: U2FsdGVkX1/riiGSQhJi1HCEoV0hInY+y8u1QEvnTrT3cHWPFrv9oRzC1pxct0MqLPTyOT8Sul3nRsUPoAg4RP5nWmeS40IleQN8FF7CCrWeTZowVSYv5aUtrKE03l8HwXcRIoMd/DYt/24mQMVMIkx02Og5ZbyLO9yTs96y4g9mbBxsgXwT8OXSInEmflZCslC79AlkVbWCQYRrm6opR3ZTUCDD0MlppDjlKR3iHefXsJ6a+ZVMSrHwWHfQEFB4gORs9tcd7kO+Aabu6KFHeR09i99LipmTo3kns1DTYjaNxWyFbgYr/NEikVTCQxEg+5lA5+a/wajheRDz0zCipioOwusGmaD5kwDTprxMzHeHNy/s4Gq+jRvCtUnJ6HPhy5dnOMsuCHS+a/iKQZDwM1g/Dy2dnEX1OOysHYURr8+JsFQaC1tqE3b5qUD/POCdXPWLYjbGC1WS05wAWtdszx9Ps4MTphYiPzSv/+QcmGKUyPqEXj9y5xepaDxX79GxUYjdS5pQ7+Efx9FA8X0oNsxspuGvcZ8zn2IEUkbzTvQjqV9xakPBr2LbaEixCIM609ACdRJkTz7le/RJnFonUCS3zCbCWNK1O6OC91WC18LxjRVL57w8rxtbZrQmRwbbRRKYFLzeeO0yfYiW+FnWKmAJmybYFJFGLnbLJ9xB8FWxiOe6CyzYP+KDGJmJ3o6t67yNI0yiVBmXMraq0wjqdL9WvmJj822jrbTHrj3eBC5rtnO1fnwatm6uPwZKCcB/mqOiVBnyunf5V5eDWxsNiZn7c1pZFBfoJe4Cl86+IyNexM+j5t10B+XeXLQYma02LYP0L16HsCaWgByg1MdoLrpLDTcs7i8M6GYf7+VZllWORhNrCSgqMax/XYlsl0TewSq1hyucFzQnS8DHSZG6SgIafr1a+TKDjvGK3g5quhNrVZKPcU/kymxZ4qqfUCXT1o8S7rHg7Ees+hpqRuP anIIUGOp YIhoyw1fXg7rhjhEQ/NeXhEDMEu5ClUT9ph9uAzldw30qMXVxf6V9PMRaOguhWHlZ3e0m5r+XJ/LMDNCcrIuMdP3L3BfehGWlGOqlUPxQXEJNO9KeZ+kQ6o/KNpsNkwQIyt4YV+vbDPN8/mtOaKZqbSGXrsAFfZelY7eM8nbKYYgT7fBBq2Q9fh1OvOAlgeh2lRvboqrcrr/V1OMjHOgwq05I5NhOKaFfrsIKXqkiweoto4j98kI/N8OKs4Wi/XzalCuPBIFBzPcAmT6XXOb0fzm7go21d77rPHorcN401Vq1U49VjMnmlR4RqT4XXeDieZZiBMQem8Go8GfEuG4vWfKehcuV1cckiuCJdNZLQFT4fzA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000188, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On (24/11/22 11:25), Barry Song wrote: > When large folios are compressed at a larger granularity, we observe > a notable reduction in CPU usage and a significant improvement in > compression ratios. > > This patchset enhances zsmalloc and zram by adding support for dividing > large folios into multi-page blocks, typically configured with a > 2-order granularity. Without this patchset, a large folio is always > divided into `nr_pages` 4KiB blocks. > > The granularity can be set using the `ZSMALLOC_MULTI_PAGES_ORDER` > setting, where the default of 2 allows all anonymous THP to benefit. I can't say that I'm in love with this part. Looking at zsmalloc stats, your new size-classes are significantly further apart from each other than our tradition size classes. For example, with ZSMALLOC_CHAIN_SIZE of 10, some size-classes are more than 400 (that's almost 10% of PAGE_SIZE) bytes apart // stripped 344 9792 348 10048 351 10240 353 10368 355 10496 361 10880 368 11328 370 11456 373 11648 377 11904 383 12288 387 12544 390 12736 395 13056 400 13376 404 13632 410 14016 415 14336 Which means that every objects of size, let's say, 10881 will go into 11328 size class and have 447 bytes of padding between each object. And with ZSMALLOC_CHAIN_SIZE of 8, it seems, we have even larger padding gaps: // stripped 348 10048 351 10240 353 10368 361 10880 370 11456 373 11648 377 11904 383 12288 390 12736 395 13056 404 13632 410 14016 415 14336 418 14528 447 16384 E.g. 13632 and 13056 are more than 500 bytes apart. > swap-out time(ms) 68711 49908 > swap-in time(ms) 30687 20685 > compression ratio 20.49% 16.9% These are not the only numbers to focus on, really important metrics are: zsmalloc pages-used and zsmalloc max-pages-used. Then we can calculate the pool memory usage ratio (the size of compressed data vs the number of pages zsmalloc pool allocated to keep them). More importantly, dealing with internal fragmentation in a size-class, let's say, of 14528 will be a little painful, as we'll need to move around 14K objects. As, for the speed part, well, it's a little unusual to see that you are focusing on zstd. zstd is slower than any from the lzX family, sort of a fact, zstsd sports better compression ratio, but is slower. Do you use zstd in your smartphones? If speed is your main metrics, another option might be to just use a faster algorithm and then utilize post-processing (re-compression with zstd or writeback) for memory savings? Do you happen to have some data (pool memory usage ratio, etc.) for lzo or lzo-rle, or lz4?