From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F4C2C64ED6 for ; Sun, 26 Feb 2023 03:50:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A00536B0074; Sat, 25 Feb 2023 22:50:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 98BC06B0075; Sat, 25 Feb 2023 22:50:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8513E6B0078; Sat, 25 Feb 2023 22:50:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 733A56B0074 for ; Sat, 25 Feb 2023 22:50:53 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 4A9CDAACC4 for ; Sun, 26 Feb 2023 03:50:53 +0000 (UTC) X-FDA: 80508066786.23.52C8D07 Received: from mail-pj1-f41.google.com (mail-pj1-f41.google.com [209.85.216.41]) by imf12.hostedemail.com (Postfix) with ESMTP id 6E25140009 for ; Sun, 26 Feb 2023 03:50:51 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=OiTnRUDp; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf12.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.41 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1677383451; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=w4y311XvzcL4qcpLEKERp48i6Icgj5mKMLUZWjnpXrI=; b=ZNJojEKOh5nHespwdOGghvXBqqcAiuVWKZHTHp5frB6eJgpp4Xf/ClXtV0+Rc9zfn9MEgD YqubUSd5QzeIEAGnZyu9kz83Gv+SKIu+ay4Nef2uPabvl0yoev5i4jLIcLfRksoZMUgCl3 CMV7M42TniaEpuHMAEKOpM/Pci934s0= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=OiTnRUDp; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf12.hostedemail.com: domain of senozhatsky@chromium.org designates 209.85.216.41 as permitted sender) smtp.mailfrom=senozhatsky@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1677383451; a=rsa-sha256; cv=none; b=fW+dAANVdNTMl9AjHGD8o5Vf17SqKZCahZb9LZMaHevEFagmz50O4wtiH2/WamPFAha8pG d2Dj6WprW7+/hfpwdC76Ydwj1Q5px6I1pab/GtJCo0b2iSfY5KKNnbgy0zhCnYeh7YH6C0 liXJroxWO2WCczlMOixCcaKpFqRYvr8= Received: by mail-pj1-f41.google.com with SMTP id l1so2918620pjt.2 for ; Sat, 25 Feb 2023 19:50:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=w4y311XvzcL4qcpLEKERp48i6Icgj5mKMLUZWjnpXrI=; b=OiTnRUDpZ4YIjM2/sq915dvhW8mbZNbYCM3fJcj4eY6ey/xyl0tjd8h8eP2IlN06Vz sGVNoH3/7AQxUFUlXxAVRYyBLocEm4uI2nu1aXCYgB7qye8lsY8qxBY3KMoIl1JP9iqb fGcJmXFt9XP65T3Kbuw+ClqJAwZ2GjHrWebSU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=w4y311XvzcL4qcpLEKERp48i6Icgj5mKMLUZWjnpXrI=; b=c0Me6Vc5VHmt+iIGzxDHBzrgNyIKEd7MrJnVqVjbAgn1qp6U1W6o9xtjPG5jwd20Yn a6K4ZWNeyO0iKgWkg58qGBFMAN54Y+d+7CXcJV68sr8M81h/dOCbbcJykJxjtdRc+kiu o2yJdW8ugmyHGjsANy9s8lYq90scU1BD3DDrcxdsAzythZYYCnLqAxfsLkCltVKrik6p Hx/GEExnwtTzOCvoGQ/lZhlT/ySvhmn3kc++Xu+wFD4Z1Do0lWjw/qX2DZ+PIkkeDMl+ 9tA4W3bLv4aoBoRLG1gRISBzvOMQkM8C1/vFRP8h7zd/q4djNqOrjQD+FG6GBTPT+OvS YkIg== X-Gm-Message-State: AO0yUKXPTm8odpdEpgPxy1oD1/h3DIUUdD1c9rZP4Nq4WYU3CZAefHtr 4O2cTZP9X2/R0TJcYPwrUwyeoA== X-Google-Smtp-Source: AK7set/i11X7lphfd5+MPslv0zxE4QFrWYN6+0mbt2yItzGiX93i/bOA95HL6K2M022D8wc9hw8W2w== X-Received: by 2002:a17:902:dac6:b0:19b:c491:21be with SMTP id q6-20020a170902dac600b0019bc49121bemr24607680plx.64.1677383450004; Sat, 25 Feb 2023 19:50:50 -0800 (PST) Received: from google.com (KD124209188001.ppp-bb.dion.ne.jp. [124.209.188.1]) by smtp.gmail.com with ESMTPSA id y21-20020a63b515000000b00502e48db9aesm1662256pge.53.2023.02.25.19.50.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 25 Feb 2023 19:50:49 -0800 (PST) Date: Sun, 26 Feb 2023 12:50:45 +0900 From: Sergey Senozhatsky To: Minchan Kim Cc: Sergey Senozhatsky , Andrew Morton , Yosry Ahmed , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCHv2 0/6] zsmalloc: fine-grained fullness and new compaction algorithm Message-ID: References: <20230223030451.543162-1-senozhatsky@chromium.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 6E25140009 X-Stat-Signature: 9jag93d47x11wb7dszja7c1jcojb1rgg X-HE-Tag: 1677383451-821271 X-HE-Meta: U2FsdGVkX1+4CR8IWytzRg4rCbVFwWJ3t5+gjggnLYdDsy3wEWtYMYyTTqeZpQ23XLsONhMCZ4MK0J6Ajtbdd9Xw1+o1W2sqB5uBKTDjb3r0E+mTndhSE7vC6NbAx/Ero5fstyFy491yy9T3m9HUL0OmBG2Tz+/B280AOnLGW4f3wuiSNklDmN/YmezMMEdZrh1/0A/5brKjAIriPQUjUrECMrKJP9ThZMdyri4udMcvOyuhO+JPvCpIdwGt8+Q/JE3rpUpJL6X8gfS19l9O68bwnO9Z1TD9OZ1+ZSt8dKnyhGhIleeAKQrCsVaRr4Nq5SPbEXIk6+tbAvgRqybrmrRjoQoxRN4/xSR9Q5U3bjvA4Ecy66KDKY5wMkklWVfkGZbnGCPxcB58BgXW3Fy2jH3LvAkeYIm9JQWVPhETuQ1P2LlPt1kcD4D+ljsF5uihbPzHYPalrmo2Z3WBxFMNLUBI94mAddv84GuilppsffNv/kY6VyHTdvQd5jfXD1z4n406ZujtkgVElBBXx1k11z5YNTKA/u4+WlrlTAadgnAJp1DFZmkClb9Kz2l0kFuFbsl97q16AKCEsM+01IOcmHSiLeu9Ovt0MzMRHABIVJtmb0gyzOwBvDhbTrtBYeeudhjbjw8jxXxl4sq5iD8owoLSzCvPI7PNnDMyayLJ5SB/wPOHxAra3OVHSI/hTTxeRIrpvzVLoZ0zNJXlTeFCwvIbLlPJysE3SM1hoyPP6rSquruiSsFgz0NIcOlT7TLyLiBgDa316rMD9aD40ks9cB2pLkDuRPCZY9mXpj7XnbX8O8RcCnHhDLulz4PpP9oJ9kvS2154dl2G11XwS12lQyTSHUtOnrcIYAD2kxJi0pJyV5Ta5/gTzdJOuI7c6rK2k4ymqOJJ750uFu2bUPfpprqp3iJsxg7K/Hs0hCAuI/rbTxjteuUllyB1D+cIR1kJz+TledKr/KjWIkq029K utZEnvg1 sD7yNuLBC1Hu+g1wBIjcDkoblE4CG+d+jZV+SxazqWG7kFNQqY9K55ben7bYeorwS8j8Ed0Er3SDeNhI575iHt5tj2aG4jjrbmfyqKg/V4bDU9fgPjiWFXHChv/qteDt2XHG9hR1SxFKtob3KFcj7KR+q1b/QVLNNFusAi05vvKTUa+VEVEuCagEfwmz33WFMJ3/7tcOcuLBBGdXjXG7G0hYtVopzxsLbzJ/sq2D193Y71c2ICRPW86xHFt18zNMj7SPy2FPHtVmcxcfTyfpMqKWEyjdJZVOI/YN9TXC2pyvH+rMMfSCsng+yeNC8xRlRn5XzIvx223tghpQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.017466, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On (23/02/23 15:53), Minchan Kim wrote: > > TEST > > ==== > > > > It's very challenging to reliably test this series. I ended up > > developing my own synthetic test that has 100% reproducibility. > > The test generates significan fragmentation (for each size class) > > and then performs compaction for each class individually and tracks > > the number of memcpy() in zs_object_copy(), so that we can compare > > the amount work compaction does on per-class basis. > > > > Total amount of work (zram mm_stat objs_moved) > > ---------------------------------------------- > > > > Old fullness grouping, old compaction algorithm: > > 323977 memcpy() in zs_object_copy(). > > > > Old fullness grouping, new compaction algorithm: > > 262944 memcpy() in zs_object_copy(). > > > > New fullness grouping, new compaction algorithm: > > 213978 memcpy() in zs_object_copy(). > > > > > > Per-class compaction memcpy() comparison (T-test) > > Just curiosity: What's the T-test? T-test is a statistical method used to compare the means of two independent groups or samples and determine if the difference between them is statistically significant. > > x Old fullness grouping, old compaction algorithm > > + Old fullness grouping, new compaction algorithm > > > > N Min Max Median Avg Stddev > > x 140 349 3513 2461 2314.1214 806.03271 > > + 140 289 2778 2006 1878.1714 641.02073 > > Difference at 95.0% confidence > > -435.95 +/- 170.595 > > -18.8387% +/- 7.37193% > > (Student's t, pooled s = 728.216) > > > > > > x Old fullness grouping, old compaction algorithm > > + New fullness grouping, new compaction algorithm > > > > N Min Max Median Avg Stddev > > x 140 349 3513 2461 2314.1214 806.03271 > > + 140 226 2279 1644 1528.4143 524.85268 > > Difference at 95.0% confidence > > -785.707 +/- 159.331 > > -33.9527% +/- 6.88516% > > (Student's t, pooled s = 680.132) > > What's the different with result above? Did you just run two times and > shows they are consistent or this is new result based on different > testing? The test is exactly the same, it is designed to have 0 variability, it creates exactly same fragmentation during each run, so we always compare apples to apples. What is being changed (and hence tested) are fullness grouping and compaction algorithm. The first one tests the effect of new compaction algorithm alone: old fullness grouping and old compaction algorithm VS old fullness grouping and new compaction algorithm. The data show that with sufficient level of confidence (95%) we can claim that new compaction does make a statstically significant improvement and reduce the number of memcpy() calls (by 18.3% in this particular case). The second one tests the effect of new fullness grouping and new compaction algorithm. The data show that with sufficient level of confidence we can claim that new fullness grouping and new compaction do make a statstically significant improvement and reduce the number of memcpy() calls (by 33.9% in this particular case).