From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D18DEC36010 for ; Tue, 8 Apr 2025 23:12:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E37FE280035; Tue, 8 Apr 2025 19:12:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DC0FA280036; Tue, 8 Apr 2025 19:12:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C6A4D280035; Tue, 8 Apr 2025 19:12:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id A48D5280035 for ; Tue, 8 Apr 2025 19:12:44 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 4BC031216C7 for ; Tue, 8 Apr 2025 23:12:45 +0000 (UTC) X-FDA: 83312428290.10.EB2D0E9 Received: from mailrelay4-3.pub.mailoutpod3-cph3.one.com (mailrelay4-3.pub.mailoutpod3-cph3.one.com [46.30.212.3]) by imf29.hostedemail.com (Postfix) with ESMTP id DEEC912000F for ; Tue, 8 Apr 2025 23:12:42 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=konsulko.se header.s=rsa1 header.b=vvSWBgNZ; dkim=pass header.d=konsulko.se header.s=ed1 header.b=Aoke+BTH; spf=none (imf29.hostedemail.com: domain of vitaly.wool@konsulko.se has no SPF policy when checking 46.30.212.3) smtp.mailfrom=vitaly.wool@konsulko.se; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1744153963; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Y+YiStRvb9f3Iob4ax1/jWCFHRVxG9qjdqNePl5lmL4=; b=7P1erngF+TIKcKNur/aDGYawTBP6AXuKMYJoE5oFpW7ehUN2v9lf2l9jU3SEGRcuTNO+r+ hmia8wmnELOleOV679XVpdqmONn+uAiOva/KXPTIy6JhkseBvc0CwG+Oupduol3dxx8dmk 6NSNCX/DOx4lkq4flCKUj95FJonZvFM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1744153963; a=rsa-sha256; cv=none; b=KicDG3E8S1u9IjeUx89aL8YLdcc1iN3F9WUmcXUw9bo+OCv/djzQNWEiy/DUtzdQU2jR8F LA6fydc57Uod0okKVbkk9bikO/hYjZbAZmLFIXU2M2hS5pX/JSF0BS+PaI1sYXuCZxjD/S 8aBQygbuQNrBMCDhKAbmZsngNemZpJE= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=konsulko.se header.s=rsa1 header.b=vvSWBgNZ; dkim=pass header.d=konsulko.se header.s=ed1 header.b=Aoke+BTH; spf=none (imf29.hostedemail.com: domain of vitaly.wool@konsulko.se has no SPF policy when checking 46.30.212.3) smtp.mailfrom=vitaly.wool@konsulko.se; dmarc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1744153959; x=1744758759; d=konsulko.se; s=rsa1; h=content-transfer-encoding:content-type:in-reply-to:from:references:cc:to: subject:mime-version:date:message-id:from; bh=Y+YiStRvb9f3Iob4ax1/jWCFHRVxG9qjdqNePl5lmL4=; b=vvSWBgNZUyj0z4H+0cV1bylSZnOy0qTLeMI4htnAI8WeJcQJ1IhntC6jXkuCQgrjfd/50uKAf3KyY eFA9rK65tJjlR6N/6LcAafhjboFdEn7AcDYLPYK1+UVLHwfuMS600T59UD0p7CBBbY2j1BaanLPdTY n8kC2sWM6/eFLM8OcSZLGvDz7Wsp53+LPQ4x9koUuB+cPJfW1dNhuEg2gHPkPTl6gBKWRafha74W0O sNIr4xbsjXGrsXElg74DzRTEb1pQ3f3JSMCnNjkNuqYmwlhfhPeXCadczUgYa+TkG0ApsW7cKhzdHX 1oBH3VzsBJ8SDBCH063e9t2ltLP0LsA== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; t=1744153959; x=1744758759; d=konsulko.se; s=ed1; h=content-transfer-encoding:content-type:in-reply-to:from:references:cc:to: subject:mime-version:date:message-id:from; bh=Y+YiStRvb9f3Iob4ax1/jWCFHRVxG9qjdqNePl5lmL4=; b=Aoke+BTH8IDPVT4OO+RoxOsvU2gYpEEwASCH+iny4aVBwfUpBtEKu+pgQEbnRfyYQJWlX6/MNonaQ /nGW4fPBg== X-HalOne-ID: f5ff7cc9-14ce-11f0-94d3-29b2d794c87d Received: from [192.168.10.245] (host-90-233-218-222.mobileonline.telia.com [90.233.218.222]) by mailrelay4.pub.mailoutpod3-cph3.one.com (Halon) with ESMTPSA id f5ff7cc9-14ce-11f0-94d3-29b2d794c87d; Tue, 08 Apr 2025 23:12:39 +0000 (UTC) Message-ID: Date: Wed, 9 Apr 2025 01:12:38 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2] mm: add zblock allocator To: Nhat Pham Cc: Johannes Weiner , Igor Belousov , linux-mm@kvack.org, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, Shakeel Butt , Yosry Ahmed References: <1743810988579.7.125720@webmail-backend-production-7b88b644bb-5mmj8> <0dbbbe9d17ed489d4a7dbe12026fc6fd@beldev.am> <3f013184c80e254585b56c5f16b7e778@beldev.am> <20250408195533.GA99052@cmpxchg.org> <24e77aad-08ca-41c4-8e64-301fcc9370b1@konsulko.se> Content-Language: en-US From: Vitaly Wool In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Stat-Signature: qp9zx7f6yhb15ca7p5qbru1eho3tn5ed X-Rspam-User: X-Rspamd-Queue-Id: DEEC912000F X-Rspamd-Server: rspam08 X-HE-Tag: 1744153962-89110 X-HE-Meta: U2FsdGVkX1/Ib9XTxiI3kPjpZS4tih770VLAnLLMu97tFWy544QorEb6rzHioeQ1R9PwaE3dEbPdePBv6PXbixnzy06mwSdbrR5NdkrYNMBpuOqC0XD1/aOtAUuW9phr9ZjOSRuWgbFQYjTWzGAw+V0wEMykG+elcBNQXgxrcBrihkp5KM6GsLXdnotaQzD9BDgKLls/frSkyMqp3GUV3NcKIJYgb+AsayP8p6H3XEctmU9QwvfXPobFnxdwwQqB7WAh1U6PEMyr5NSQEB31ZyT9qYqlhHen0iAUymMPOd0y1HpPhA+2ChT45d8d20cFxFA3IMkqfiIVp3dxDqRJJ54x+ThETSOD8IU+DMXsd/J/mILHkShyfHmKIt8Vuleq4BYyhR/NNv5aMF6OrTGEv/uzYtMo/0Q4WDTrvIZAl+Wun6XkZAC73ZqA8e2sFmJtgn7S5RmVtJT2tS+pH/soLZp7KT63vNApwmuiJUZBXGA5e3gV1R6Mw/4TD8QpNL9Oiy2xYyOrl/EXGc65IRnCLoKAYFmyjV+4EgDi1YwiKW+CIKFDeHgkC9AGPNOH+z44hrZR5OSYmX9QE6Z0KtsC5uX3/wW9ofcQCOUBvN4YBZHtukyPBY9BDQggupd6Os2HTE/25GTJGXWN1DkuDOwcjFyq8fqoekUr+5Zuh7rqw1rwUUW694DwHQdhkwaMibOuX0pt172l6Esu/GHnxu+5vVpxkGss2wy7tEDUjFcZsBwqX2GOoC5B2+pErgMz0oVlqfukY1c8u4GeYmjXK8buRIbpTG2yl9J9A+rW89q4betcLtbnqJ8SE0wA/o/0ZexNm5aWdG7IOT8zDlhrRQDTxxJpnKrLwDU9f/YWigHehHT6CbCydAGCvRdDiWnBykvTR0ItNPWxFPTlfIZiV3S5M7fOWZLcBWOYVCsWKiQEzZc5Jc6qayczTOV2DoYCApxaPMrM5iAk5CPP7G0H0+2 C9yndn7c lBYpqW9gA02EcU9+HjE7LLH3papGlJl39MWHe+wRC5lawBswvLxfvatuAKt38C8q5bXDs9cnhpncUD4UnlorRjA0ANBknNmIYQ2d8Hu8SBM7KIg4YW8dr2Ehl6Y2LuuaSTw3UKnOjaSXRlfWK/IkoV75FYQBxwWS9VmDnjfwioznMqNhSuLvg1sb7dh6SpezPybsQukLFo+vvlNmfE8rLUkj6U4tMXAapKYVmpDQXxTo+3ug= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: >>> So zstd results in nearly double the compression ratio, which in turn >>> cuts total execution time *almost in half*. >>> >>> The numbers speak for themselves. Compression efficiency >>> allocator >>> speed, because compression efficiency ultimately drives the continuous >>> *rate* at which allocations need to occur. You're trying to optimize a >>> constant coefficient at the expense of a higher-order one, which is a >>> losing proposition. >> >> Well, not really. This is an isolated use case with >> a. significant computing power under the hood >> b. relatively few cores >> c. relatively short test >> d. 4K pages >> >> If any of these isn't true, zblock dominates. >> !a => zstd is too slow >> !b => parallelization gives more effect >> !c => zsmalloc starts losing due to having to deal with internal >> fragmentation >> !d => compression efficiency of zblock is better. >> >> Even !d alone makes zblock a better choice for ARM64 based servers. >> >> ~Vitaly > > Could you expand on each point? And do you have data to show this? > > For b, we run zswap + zsmalloc on hosts with hundreds of cores, and > have not found zsmalloc to be a noticeable bottleneck yet, FWIW. I don't have the numbers at hand, I think Igor will be able to provide those tomorrow. > For c - in longer runs, how does zblock perform better than zsmalloc? > In fact, my understanding is that zsmalloc does compaction, which > should help with internal fragmentation over time. zblock doesn't seem > to do this, or maybe I missed it? The thing is, zblock doesn't have to. Imagine a street with cars parked at side. If you have cars of different lengths which drive in and out, you'll end up with spaces in between that longer cars won't be able to squeeze in to. This is why zsmalloc does compaction. Now for zblock you can say that only same length cars are allowed to park on one street and therefore that street is either full or you will have a place. > For d too. I see that you hard code special configurations for zblock > blocks in the case of 0x4000 page size, but how does that help with > compression efficiency? Well, to be able to answer that I need to dig more into zsmalloc operation, but i would guess that zsmalloc's chunks are just multiplied by 4 in case of 16K page and thus you lose all the granularity you used to have, but I'm not completely certain. Meanwhile I did a quick measurement run with zblock and zsmalloc on a Raspberry Pi 5 (native kernel build test) with zstd as the compression backend and the results are the following: 1. zsmalloc *** The build was OOM killed *** real 26m58.876s user 95m32.425s sys 4m39.017s Zswap: 250944 kB Zswapped: 871536 kB zswpin 108 zswpout 54473 663296 /mnt/tmp/build/ 2. zblock real 27m31.579s user 96m42.845s sys 4m40.464s Zswap: 66592 kB Zswapped: 563168 kB zswpin 243 zswpout 35262 1423200 /mnt/tmp/build/ You can see by the size of the build folder that the first run was terminated prematurely not at all close to the end of it. So, I can re-run the tests on 8-core high performance ARM64 with 16K pages tomorrow, but so far everything we have seen points in one direction: zblock is clearly superior to zsmalloc in 16K page configuration. Besides, zblock can do even better if we extend that very hardcoded table you mentioned (and BTW, it can be automatically generated at init but I don't see the point in that). ~Vitaly