From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DBEE4C7115B for ; Thu, 29 Aug 2024 01:02:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6FFB96B00C2; Wed, 28 Aug 2024 21:02:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6B0676B00C6; Wed, 28 Aug 2024 21:02:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 551656B00C8; Wed, 28 Aug 2024 21:02:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 34EE96B00C2 for ; Wed, 28 Aug 2024 21:02:26 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id A92F116086C for ; Thu, 29 Aug 2024 01:02:25 +0000 (UTC) X-FDA: 82503482250.22.345D266 Received: from mail-ej1-f54.google.com (mail-ej1-f54.google.com [209.85.218.54]) by imf12.hostedemail.com (Postfix) with ESMTP id CD4A640002 for ; Thu, 29 Aug 2024 01:02:23 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="G5jUX9/H"; spf=pass (imf12.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.54 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724893274; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2/VB8quUYt2uT5KgpHetU9IRNh5nnyUMgsIX+0fJAHQ=; b=bSNPfGuippKU3bA/Mqdurt8xPXZfVkgTBu+HWi5t0piZ03cCEh9W5OV5UjY9Bm41wS8jD8 fy35E/Q+GKyU1DBjuv/j0nL8ZTAN4+19JBd/bkyrBgOA3wPrGCl8OBYyT84JfbXcvm9OHe 9D8sJ+UFiaMzA9M+2rbDH+iMCM+Iks4= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="G5jUX9/H"; spf=pass (imf12.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.54 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724893274; a=rsa-sha256; cv=none; b=GwLkYUtF715SaTGnhhIDW/+XwYdy1vMmwcIVBBDa1mtlZDmWr+lL0m05AT+C9RpJbbtG/z AeMpILiofWhcKw01upEFSNxDLgDZEWzfmJ9HpbMHQER9hXJ1XCtOqX6VoaF2x3taa+Qcu3 aUra1hvDI6FMogAdVziclgaf3QLHC8Q= Received: by mail-ej1-f54.google.com with SMTP id a640c23a62f3a-a86c476f679so7477266b.1 for ; Wed, 28 Aug 2024 18:02:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1724893342; x=1725498142; darn=kvack.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=2/VB8quUYt2uT5KgpHetU9IRNh5nnyUMgsIX+0fJAHQ=; b=G5jUX9/H5he5n5SK/92DykLYmqi1jTDgbevV/wN4rGysgdv3I7NRaqG+TO0HIZKx54 SNrGE5GbKeoL/UYnBPUbS0/fYSKlTZLVG8FT/WzNXEME26ChEENzgC6BdRzVZxVlZgn8 680h3rUSpaK6+dy/TD6+zUQBjtLk+ZM9fJS+HMR6foB5h23NepCK7/0nDZg3IH8PGVp2 O6YOUE8ePCZlC2FJL5w/je1HaICBjJ80nTgOk0MS8yOD6K59OJ3rLC1SSgNIVu5Sy3Is FiGkvArSL5YyBVT9Z11ppLonuBPILmidKO0Ji3Pdn/zuzgx5TRE881v9RdCWf/FopB+F QSgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724893342; x=1725498142; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=2/VB8quUYt2uT5KgpHetU9IRNh5nnyUMgsIX+0fJAHQ=; b=IJJ8RrbpNPQerNVIo8lxZ9oIAh3dbhY8KK/QrNLHWSG05hxb8Ai92DoHog6grUMBGA r8WP6Z08n/p8fc80yyudyJOV70QIXMJQS8a91Bce5I2nYcIZr1QwBKE+ptyKqhHFe/JI S6bwVkgiWAR2gPw0KcOEYYpvnwzKzS13WSS8Ci/lhBQBamQwCA3NVpc9h6ilOhsgchwk hObnmKU/MJMU0gvEuj8nDugkB84I21zQK+jhm6TTk3DzfmKyESdMC6oqpdiwGyTlhcj+ 31eSbO70HMm6rS/S0JgHQKvLcouegcAa8XhumflZFiuA1jUZEFi2L/I8oatbYtb9I13L IRMg== X-Forwarded-Encrypted: i=1; AJvYcCXeojDHunb+R0hxtFp3QwWeOEi/lZ3UQKsiySsdVAqdW24nhhv+ulKnsIMQavDSnvIvXe7SOk+bnQ==@kvack.org X-Gm-Message-State: AOJu0YzqJYQJbvF+M0qs7G1EGuRKP7+wX81WXKYnR2/yyl+DobHZwDwB tIYyK79hXxVO3rc0tnar9x59cvJNvuYlE0YNPtqT4o3TscNzTtoXaVlVJoaOpeSQCiot2Ui8Uzq SfhbIuSIfQeGs5H5qdQLkbXH5N70IazyKgAlR X-Google-Smtp-Source: AGHT+IF1bPH1PJC2EWwNS8u4FKF7LLaeLWheuE1hROdXMI1FL11h/Ib+V0/gJKSPIa+Fca/UI49+n4gkTyPgfOdAs/M= X-Received: by 2002:a17:906:f5a3:b0:a86:9258:aeed with SMTP id a640c23a62f3a-a897fad655fmr70664666b.61.1724893341386; Wed, 28 Aug 2024 18:02:21 -0700 (PDT) MIME-Version: 1.0 References: <20240828093516.30228-1-kanchana.p.sridhar@intel.com> In-Reply-To: From: Yosry Ahmed Date: Wed, 28 Aug 2024 18:01:45 -0700 Message-ID: Subject: Re: [PATCH v5 0/3] mm: ZSWAP swap-out of mTHP folios To: "Sridhar, Kanchana P" Cc: "linux-kernel@vger.kernel.org" , "linux-mm@kvack.org" , "hannes@cmpxchg.org" , "nphamcs@gmail.com" , "ryan.roberts@arm.com" , "Huang, Ying" , "21cnbao@gmail.com" <21cnbao@gmail.com>, "akpm@linux-foundation.org" , "Zou, Nanhai" , "Feghali, Wajdi K" , "Gopal, Vinodh" , Chris Li Content-Type: text/plain; charset="UTF-8" X-Stat-Signature: hzmsbnuqybmcsspzdg9rg1oahanh8dak X-Rspam-User: X-Rspamd-Queue-Id: CD4A640002 X-Rspamd-Server: rspam02 X-HE-Tag: 1724893343-329906 X-HE-Meta: U2FsdGVkX199w9vNzQ4u8BSqA66mgvwPveg0rb4kL7aTG3DeWJRa2o0XUhxedv8CZGkw/VdAkSPW+Vjk5MzTV5j30rlgAXQLOj4+xMJJ/OxMECMQalPPVaWLnbCOTww6LH80dtG8bEBUUheLlB8tx9ERONpsxJAN7XSJx4Yjhf0KjTfFPx0xAug9sdO8cc1kw9hx/XYLHuxEqQyBvo3WH/YKeAjXNJtcWLHyy32V1moyN9zkRjK3yp2zG1sQ0URR/tzEd+Ee2cPm0mhfaU4TNbaRrLR5/aola82ou7nSTaB736nGloIfV8asxh7tZh4d5ZsONvPgWfFtSVujrxGs7yozfWbE5HVZ36TkyRt7GjOXHchcMyLnG+Es9QhcKX1Hg86jN9/mifrIyr0tvD5LC0PrIR7IRdWsHCE4tcIrUcMSlbOe5DeN/rPjcQNFxYuPMZlaxfwP55QW5vb6N7fDP0KRHtHMbiVda6Jw66yEnctUEZZAV9q91EY7eib3g3jhvLjXh8pTf5V39CkYfC7vcviQkiaFyqt8sTw6RY/AeCAqSf7Nm6bSiL2vLS5GhNOQchdMd5MlXuznXOnU7RoYxBayo3gARwLMbQjZdoTdRi9nshvQ+FB+wO09h7/j2BtGnP5jCF03Tn0I6U2oVCFCKw6Y4fnzbQwCBkqLaW/gFBBG2+FgO0/iuJqOCe5MRfF25E8l24IIfsaBnx9bODjpyvlFvJdPXTzBdAtz4woHMOAEC6ji9bINoseq0QSrY7sRm2zvJ/yuHYlvT5gAZ7ueCAGgwaOcWqiFRA2POxJr3mpwOHkS0bC4Y0f+252OMCw4xR5NKh6X3yrYbKp3VVieVyCnDZSaBrcefkQfuPDkKicX6rpMZUOHGq0fIvTnoKpIcMfXQhFjV6BDYoNz6j67x13rcGnKHlfN7d+JK68z2xuJ8OlnyqFR5vIxiLjbsTnwRVEjoi8TmBVoL5uaf3u w87Xs6Z8 3kPRijzE59Imp/FBAUVTbMDryqkxDvnheYIz5Nb2dWOSrIZCmdvcYape3j2nezcl41Q2rw2glyjJgkbCFnWRiE5Jj0sdZhnMfEYFCpJ/HP0tdGmEqbH8XGubtcMksl+nLPzJfnK/4otTZLN0h1hOfuu0vDhR03r97gRC5uS9eE7+xtKuc7Imq0aDBZiHEhjP4Cw9UYMIDycAOkiiDIEC+tG396A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000419, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: [..] > > > In the "Before" scenario, when zswap does not store mTHP, only allocations > > > count towards the cgroup memory limit. However, in the "After" scenario, > > > with the introduction of zswap_store() mTHP, both, allocations as well as > > > the zswap compressed pool usage from all 70 processes are counted > > towards > > > the memory limit. As a result, we see higher swapout activity in the > > > "After" data. Hence, more time is spent doing reclaim as the zswap cgroup > > > charge leads to more frequent memory.high breaches. > > > > > > This causes degradation in throughput and sys time with zswap mTHP, more > > so > > > in case of zstd than deflate-iaa. Compress latency could play a part in > > > this - when there is more swapout activity happening, a slower compressor > > > would cause allocations to stall for any/all of the 70 processes. > > > > > > In my opinion, even though the test set up does not provide an accurate > > > way for a direct before/after comparison (because of zswap usage being > > > counted in cgroup, hence towards the memory.high), it still seems > > > reasonable for zswap_store to support (m)THP, so that further performance > > > improvements can be implemented. > > > > Are you saying that in the "Before" data we end up skipping zswap > > completely because of using mTHPs? > > That's right, Yosry. > > > > > Does it make more sense to turn CONFIG_THP_SWAP in the "Before" data > > We could do this, however I am not sure if turning off CONFIG_THP_SWAP > will have other side-effects in terms of disabling mm code paths outside of > zswap that are intended to be mTHP optimizations that could again skew > the before/after comparisons. Yeah that's possible, but right now we are testing mTHP swapout that does not go through zswap at all vs. mTHP swapout going through zswap. I think what we really want to measure is 4K swapout going through zswap vs. mTHP swapout going through zswap. This assumes that current zswap setups disable CONFIG_THP_SWAP, so we would be measuring the benefit of allowing them to enable CONFIG_THP_SWAP by supporting it properly in zswap. If some setups with zswap have CONFIG_THP_SWAP enabled then that's a different story, but we already have the data for this case as well right now in case this is a legitimate setup. Adding Chris Li here from Google. We have CONFIG_THP_SWAP disabled with zswap, so for us we would want to know the benefit of supporting CONFIG_THP_SWAP properly in zswap. At least I think so :) > > Will wait for Nhat's comments as well. > > Thanks, > Kanchana > > > to force the mTHPs to be split and for the data to be stored in zswap? > > This would be a more fair Before/After comparison where the memory > > goes to zswap in both cases, but "Before" has to be split because of > > zswap's lack of support for mTHP. I assume most setups relying on > > zswap will be turning CONFIG_THP_SWAP off today anyway, but maybe not. > > Nhat, is this something you can share?