From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 36C16C52D7C for ; Mon, 19 Aug 2024 03:20:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A45936B0085; Sun, 18 Aug 2024 23:20:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9CED46B0088; Sun, 18 Aug 2024 23:20:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 848466B0092; Sun, 18 Aug 2024 23:20:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 5DF416B0085 for ; Sun, 18 Aug 2024 23:20:39 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id D6DB6A8420 for ; Mon, 19 Aug 2024 03:20:38 +0000 (UTC) X-FDA: 82467542556.09.FD976CE Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.15]) by imf08.hostedemail.com (Postfix) with ESMTP id 46C7316000F for ; Mon, 19 Aug 2024 03:20:36 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=jBl9rree; spf=pass (imf08.hostedemail.com: domain of ying.huang@intel.com designates 192.198.163.15 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724037559; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OUcmvDOC+mifBRfRpyEX0THdy54PFkL56m1zUohaiy8=; b=8SsGP1pV6TNH7PYiJQv4huN0O/pVk2Gnl+tXOJy53lFpYis3bw92pETFDXQVPevYlp48sj btX7S2J/Wqh6iFfiOEAkO0C7tEuK93YoxJ7xWUWRxDBo0gqEyQ7pjniLtVRimydRUHCCVX S2Rt1kaJ5+2Nq7O+AvEzQ0Z6alxdZLw= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724037559; a=rsa-sha256; cv=none; b=KZVIyArJ651NkAPUm7Bpmh/jEmUdI+U/5iqaAiHAF7iLPOKY8Ff4TnGGNEvhEjfpMfhOuT 60mreBFuaxN5jO57KYvEW91i+Pvh9Ig/5nHLHsA7/V8JwGy1HRNUy6opRQVuTYRFGwtxxj /Fi9st7Wf7Y0b3peB7m7CA30kyMooPQ= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=jBl9rree; spf=pass (imf08.hostedemail.com: domain of ying.huang@intel.com designates 192.198.163.15 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1724037636; x=1755573636; h=from:to:cc:subject:in-reply-to:references:date: message-id:mime-version; bh=iMSdCfe90roZCySYWjTtZqKWjVx15CpcGY5MMofjvRY=; b=jBl9rreeXa7PXpS7iT9F5p5TJUI4y7QAwp53NmyDyfxyn5tKmgFrovLt OfGLq7CjgdT6BW+YOugyGnmb1of4bqJ3HoASLvH8Tmwwf1Uu3g3TedgYz J3ciY6Az9Wh9WIEB3tRD749q0OO1OBNTAqdhJ+oxsyX8WVdYHWh7Zr1Xi 1o/HCZp7vPcvxXyNIx6C5XYdHq2++Npw8RaJ4KwQ4t1bn1EuRjDImEhNH 0bO265dgqqVSABb4qnN4zL86QhgU70oom3Gl1bxDX9YM2Y5l03brMiW9P M4IATJmcBR3vgY4NtQZs3aeB33m8jcw3DwEpqnT17galTaJAWR/tL1sLF Q==; X-CSE-ConnectionGUID: 3ZW4kAy+Qkqm4L7HZ9EM4Q== X-CSE-MsgGUID: AbwS5UpKRsC1BxgPtyNW6A== X-IronPort-AV: E=McAfee;i="6700,10204,11168"; a="22417295" X-IronPort-AV: E=Sophos;i="6.10,158,1719903600"; d="scan'208";a="22417295" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by fmvoesa109.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Aug 2024 20:20:35 -0700 X-CSE-ConnectionGUID: 8TXaih5eQ8WqL/3reE8QsQ== X-CSE-MsgGUID: ZcS1nmkfQcOwH7kesMDSiA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,158,1719903600"; d="scan'208";a="60519339" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orviesa006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 18 Aug 2024 20:20:32 -0700 From: "Huang, Ying" To: Kanchana P Sridhar Cc: , , , , , , <21cnbao@gmail.com>, , , , Subject: Re: [PATCH v4 0/4] mm: ZSWAP swap-out of mTHP folios In-Reply-To: <20240819021621.29125-1-kanchana.p.sridhar@intel.com> (Kanchana P. Sridhar's message of "Sun, 18 Aug 2024 19:16:17 -0700") References: <20240819021621.29125-1-kanchana.p.sridhar@intel.com> Date: Mon, 19 Aug 2024 11:16:59 +0800 Message-ID: <87msl9i4lw.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 46C7316000F X-Stat-Signature: mrh1esem7ytqedhg7nahozba5jzk1asm X-HE-Tag: 1724037636-219854 X-HE-Meta: U2FsdGVkX1//wWyqKtVp2CO2Qlsgovsmq8Sv9HFjHII1Z1CoxMA7ljpWGM1gDV7ch+O6nW1VrPfaHQ3ivgNVrAQGGlWTdUJAK6i/LI1MyYek+vMROdH5CaU7wHcsAypzOOh0GNMg8xmBi/X3MI+jXXnFhTDD1X/yR8Yj0mpnUI83BkmDXjecfg5JKWjMOewC14CwGEp81jX2FuiBnfLqIYzHWMn7kbWrejmTalEdoU4QSDZz+fzu1iaEutxCmIvrUbk97ONNe2BJva/ssih5il4HJEBUQZtPi6uNrIFXUiUVo7iY6CyfrvSSAOVsMpusAAtNsVrerONWymlNiopOJgngJFk16RSm1LZng/4j+/QFhG3zHgOTdqlby0mJabtAHp3v4CH6GO5cQJtayr8vUEjvQ2jAwmg5Bg+jGESfOPtarpVyVVn6YR5uSdm8FwWhqwYMJoDixZaMMu0KD/rdiDh7C6Ed22gsu40+oJr97lyy0TkPBTVhys83nyaQALEyZZhWaoHHDsmybT5wSGaXy7wDpYWI1RjHNJ/xCYu9K7yvqrKEkQ58pZ77KoQmKnWxpQ/ioPFqD84eMuPkmyh2kaMmNas229gTviAMfcd5Ac3jBsbIOkGwU2/6J4AHeCk7w7/nnRsVb2digDwqz8+FAYkj7bHugpCVOkUBtwIeQSQTa0Nbc4WlFGeTEjPpPkRGD4JqnHsZPf/EZ9eZv9ORsf2XR1MbGkiUud10nsG/rSjpFsoy3F4YQuRGj9sGCh1+YzioB42MbKGz27GmKZcyYOK8IxsJ6HyhZumRxtoxv1Su1MIt6dhsYPLovWaBVr4pPyDjmkPjoHco70xpX8pGeokZxpnDWmwdD0munE9uZb01ytqxjK71lCEZRc+CNUFuacVfZBSiYWnC88GMz3v+FCr4pDWNUbMnLc1QN5MOQkgewle1KGW4zB+MUU8mfNp5boat4Eckj5Jk30/EUFk 6akTxpU3 cF+IBGs7Pm0iFOw3ALEJ7j0rBBH2CifeJbVjMrDZnfJmPXPRfbsOX+yd67myB3LFamXMFlE0yktJziwzQJTs136ihcE+ylMjSRGt3gT5JV+7VuMFqTEmdNn3jqJxTT0JnVaSIeoHB8yWY0jRiZzv+8313A66nvKWUrGxMwL+vakFcC/1E9KsjN1xQTB7YLEzmPiFKEy7P3qz6U1o= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Kanchana P Sridhar writes: [snip] > > Performance Testing: > ==================== > Testing of this patch-series was done with the v6.11-rc3 mainline, without > and with this patch-series, on an Intel Sapphire Rapids server, > dual-socket 56 cores per socket, 4 IAA devices per socket. > > The system has 503 GiB RAM, with a 4G SSD as the backing swap device for > ZSWAP. Core frequency was fixed at 2500MHz. > > The vm-scalability "usemem" test was run in a cgroup whose memory.high > was fixed. Following a similar methodology as in Ryan Roberts' > "Swap-out mTHP without splitting" series [2], 70 usemem processes were > run, each allocating and writing 1G of memory: > > usemem --init-time -w -O -n 70 1g > > Since I was constrained to get the 70 usemem processes to generate > swapout activity with the 4G SSD, I ended up using different cgroup > memory.high fixed limits for the experiments with 64K mTHP and 2M THP: > > 64K mTHP experiments: cgroup memory fixed at 60G > 2M THP experiments : cgroup memory fixed at 55G > > The vm/sysfs stats included after the performance data provide details > on the swapout activity to SSD/ZSWAP. > > Other kernel configuration parameters: > > ZSWAP Compressor : LZ4, DEFLATE-IAA > ZSWAP Allocator : ZSMALLOC > SWAP page-cluster : 2 > > In the experiments where "deflate-iaa" is used as the ZSWAP compressor, > IAA "compression verification" is enabled. Hence each IAA compression > will be decompressed internally by the "iaa_crypto" driver, the crc-s > returned by the hardware will be compared and errors reported in case of > mismatches. Thus "deflate-iaa" helps ensure better data integrity as > compared to the software compressors. > > Throughput reported by usemem and perf sys time for running the test > are as follows, averaged across 3 runs: > > 64KB mTHP (cgroup memory.high set to 60G): > ========================================== > ------------------------------------------------------------------ > | | | | | > |Kernel | mTHP SWAP-OUT | Throughput | Improvement| > | | | KB/s | | > |--------------------|-------------------|------------|------------| > |v6.11-rc3 mainline | SSD | 335,346 | Baseline | > |zswap-mTHP-Store | ZSWAP lz4 | 271,558 | -19% | zswap throughput is worse than ssd swap? This doesn't look right. > |zswap-mTHP-Store | ZSWAP deflate-iaa | 388,154 | 16% | > |------------------------------------------------------------------| > | | | | | > |Kernel | mTHP SWAP-OUT | Sys time | Improvement| > | | | sec | | > |--------------------|-------------------|------------|------------| > |v6.11-rc3 mainline | SSD | 91.37 | Baseline | > |zswap-mTHP=Store | ZSWAP lz4 | 265.43 | -191% | > |zswap-mTHP-Store | ZSWAP deflate-iaa | 235.60 | -158% | > ------------------------------------------------------------------ > > ----------------------------------------------------------------------- > | VMSTATS, mTHP ZSWAP/SSD stats| v6.11-rc3 | zswap-mTHP | zswap-mTHP | > | | mainline | Store | Store | > | | | lz4 | deflate-iaa | > |-----------------------------------------------------------------------| > | pswpin | 0 | 0 | 0 | > | pswpout | 174,432 | 0 | 0 | > | zswpin | 703 | 534 | 721 | > | zswpout | 1,501 | 1,491,654 | 1,398,805 | It appears that the number of swapped pages for zswap is much larger than that of SSD swap. Why? I guess this is why zswap throughput is worse. > |-----------------------------------------------------------------------| > | thp_swpout | 0 | 0 | 0 | > | thp_swpout_fallback | 0 | 0 | 0 | > | pgmajfault | 3,364 | 3,650 | 3,431 | > |-----------------------------------------------------------------------| > | hugepages-64kB/stats/zswpout | | 63,200 | 63,244 | > |-----------------------------------------------------------------------| > | hugepages-64kB/stats/swpout | 10,902 | 0 | 0 | > ----------------------------------------------------------------------- > [snip] -- Best Regards, Huang, Ying