From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92117C5474A for ; Wed, 28 Aug 2024 07:44:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 111646B00B1; Wed, 28 Aug 2024 03:44:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0C1BD6B00B2; Wed, 28 Aug 2024 03:44:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EA46B6B00B3; Wed, 28 Aug 2024 03:44:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id CAB736B00B1 for ; Wed, 28 Aug 2024 03:44:26 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 79D2FAB262 for ; Wed, 28 Aug 2024 07:44:26 +0000 (UTC) X-FDA: 82500866532.16.0756491 Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) by imf02.hostedemail.com (Postfix) with ESMTP id B0DB380012 for ; Wed, 28 Aug 2024 07:44:24 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="ZLtYA/Lo"; spf=pass (imf02.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.50 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724830948; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=P1SGtxe3LJWnrEFmfRwmJX3rq26Smm/57691EhDFOtw=; b=yz+U0NCZTJ5s1CfRSl9JraunPLTR8tyOY9G22ANpl3AfviO8ALJdUQyxBzuKD4RZUXkGh9 MFt3EopMUpqhQbTqz9A6dnKaReFkAeftxZY1AzCoy26HY5GmBPKCvkiIitg6dBOsqwnWhH 8ZXfsRV4oSPyRAQYqM2vp5c+3/jf3yw= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="ZLtYA/Lo"; spf=pass (imf02.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.50 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724830948; a=rsa-sha256; cv=none; b=cAFSZ2NuzCvaj5JcUUaBBAAYyCjCEBKYyLUs4cRgrcPOpI+bQTGA9ciDDshneYCSHiLj6A EIMRMueKSwz/m/QV9fr88JlvkvGYvLJqDQOuRQVm9UdKIQHROZ0ioYPP0oVrtmDY2JEgeR BdSwIYF+cuoxirCF9kfHpmQtWsImOho= Received: by mail-ej1-f50.google.com with SMTP id a640c23a62f3a-a86933829dcso725938566b.3 for ; Wed, 28 Aug 2024 00:44:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1724831063; x=1725435863; darn=kvack.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=P1SGtxe3LJWnrEFmfRwmJX3rq26Smm/57691EhDFOtw=; b=ZLtYA/Loym0ubcNheHnfDvQf6n5cKTX0SSGuujCLFY569sETd6/KLogkmsqJaEL46H ZMhQDCXmG8NAZEn5K2rTzBG79q3+h8UyZttnnuUc6nIB3InF5KywBe1ozZpebm9icqBN K84twuOeE8qcP8bZLfHT31Sb3GEoL2Cczm6A/a2meaBPQ7dWWEblzHkKDvxqb4nQ/gkC xuK7VFeVJMXD2F2RfeasWCeevjG2a8kNoBiYmQVKyflBBAgFTqbJKYIHGMgYVZRrFRh2 gfeCL3/+oH+aaEWb+nqXVTsKNVIwNF7ejYyJ4dHbyOVLrP7n2RmiTHtOovAxOmzX3/84 rxLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724831063; x=1725435863; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=P1SGtxe3LJWnrEFmfRwmJX3rq26Smm/57691EhDFOtw=; b=oUKeoRVBX5JJIw2Z9aI90bWnty5B1JnLwvIRx05zKOGsGH4Q9nOlFr2kjlebE1dblL jT/a+Xa1gox505tbEjO/yTK5dsgbz1odwdhaODsUGESe7F0Nd0XFrHEiP5FGFx4w4CQK d3ghJ+2AERujEXWVLzF0mtuEO9gAg6o0FNBbTOcf41p7aFxOr2ZQIDuNErcrFMFsPlri zR94QPl+KUE2N9ZSOPUIe09RNmmMVQ4VBHA/fIFqvEfPanLslnUvvWV/Qu7sSOGOfoy7 KS/RQ4u2v1MvBhJ5ORgO5iNNQkgAIk8ZlpuA0xF0OV8tLtcFB41zToxp/wJtRJAj8CAq SozQ== X-Forwarded-Encrypted: i=1; AJvYcCWGf6C0Qm1p9msbyxD6XZhQBBr+kxH9xrA5St9qQCMBo4cjEwiqugLL5kUxDJjhFmiZvfLbC7j7bA==@kvack.org X-Gm-Message-State: AOJu0Yw6pKZw88Yb38ymBPSbNuBqCwU229YR+roEspPkoY1rPW+/MHJN y6wUdrlHs2JR7IGiAXc7EylA/8lPjlW3szZWOq8njH3lWrEav0indYhHDTtibcUCGdRGXBVj4y5 8+xgY4lySXwSug/kOr72yrUrxZe9pFIrE53zM X-Google-Smtp-Source: AGHT+IEt3auLBz+8ctUX4K+GsmZnMbeCZLriBqlM/+qFiYssZzYH6PGYYK8mpK4GN058/6bgrZuqGw3a8/d3MyhfMCA= X-Received: by 2002:a17:907:86a4:b0:a86:9fc2:8395 with SMTP id a640c23a62f3a-a86a518b2ebmr1217425466b.11.1724831062592; Wed, 28 Aug 2024 00:44:22 -0700 (PDT) MIME-Version: 1.0 References: <20240819021621.29125-1-kanchana.p.sridhar@intel.com> In-Reply-To: From: Yosry Ahmed Date: Wed, 28 Aug 2024 00:43:45 -0700 Message-ID: Subject: Re: [PATCH v4 0/4] mm: ZSWAP swap-out of mTHP folios To: "Sridhar, Kanchana P" Cc: Nhat Pham , "linux-kernel@vger.kernel.org" , "linux-mm@kvack.org" , "hannes@cmpxchg.org" , "ryan.roberts@arm.com" , "Huang, Ying" , "21cnbao@gmail.com" <21cnbao@gmail.com>, "akpm@linux-foundation.org" , "Zou, Nanhai" , "Feghali, Wajdi K" , "Gopal, Vinodh" Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: B0DB380012 X-Stat-Signature: tdf6kab4fztz8irnp5w1aqd1wka8z6zt X-Rspam-User: X-HE-Tag: 1724831064-310219 X-HE-Meta: U2FsdGVkX1+8nojh8rdMQpIfdD0JqXyr5stLZbXq6e59CffZdZrKs3XI8o14yXw06vUfgrxMYofwR0ScXlKk9Em0ykdbmLvvaeWsXX8yBDbH1lWnTXhZGyzwfLPD7j841CaaNG+zBe7qH3gwbmVIRxsn12dB9H2VudVND2F10pqL4HQRMM8wPpTKQ8yHAT0XjJ/h0Lugg52/A6lkR9bsRi+IE+G6piM1LaYhriLNhdbl9Tv6XV15UV+Zvl5vA497iy1P3XILzsqhS2Oj/mtu7Wfrd7ZazBkfHsxFUfC9dLU3e4AWhF8AVhPx2ckcU180+L/+cUMdMqXHTtv308oMElGHRjJCRlQcK60G0HgbqZTzBE3fnXrNO/MYQGKNjdtzXkDFb005zp11aMqxbnno1dcXuB2uIPUzf/S1TQzq+5DoRAj4pomxQ0ucZnh/B3zuCmUXUbl3eZXBenrs4K3svmE2AJb0+10tEROkN3jUCdYaNF9KdDr9xcUaUlup7T1h6Rmu2Y/dRtw3GR3m6zHbUZqvNPQnMERCAxb38uE9PBD9LlYIEj8E1hQYekkYagEXreyiVWAcBONIVht/2KqtF3A9qBjbtpVyFveDf/DFEYuJaMbEsK7X9vLfY+NV/QiUxztH9WHJ/57PG1DUI4BW2sSXYSr5rCCnyRWpikz2uilrRAXos2E+U5SjFatIvXE1Qk31C5X5iHmwGPDSMqrLM7bhkivW4uZLDuAMJud2jZg40L+bOZuHS8yjKBANl8ohDYRJTBiMR10lUus9HJkBwmjR4xL4VJaZeMkDs0DLxp27Fn2TdUQmMreOf79JSVYBpE2LrZhB+hCYJpFEwCxZhNbYIg645FpU5IYycSQC/sW0xLh3gLuu6mphr/7rnRy/kJv9y4s5PrtZsMj0AT3hG5h41tct7B2KL/FhHEyUEPLHJ5Ds6PLGcpfHHH7yIxkeNSC+zqoOd7zjmsvBBDM s48KpHEf p/V549wO91KTmlrxw+B1NE/2R0IqHJnX6pwKvgy61/uyEexyBSvR1KngXH+WLgMy0/Xux6AvfWRMT1CkqsQWecR5y7Mg0CMFN8CxBGLIjITMDXgsdS5+Jy69ZPbMq/2TGi3PF0TvSalxJhfxW8XqUvXmSLQAMYqFDtvExN6jlo9XHjK+X0kiMWjRuDseY7AxkNCdPZSgW2DnwRTffv5Oa8Ct2qDwGa3tqqQCgzLjUgFlnmDo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000082, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: [..] > > This shows that in all cases, reclaim_high() is called only from the return > path to user mode after handling a page-fault. I am sorry I haven't been keeping up with this thread, I don't have a lot of capacity right now. If my understanding is correct, the summary of the problem we are observing here is that with high concurrency (70 processes), we observe worse system time, worse throughput, and higher memory_high events with zswap than SSD swap. This is true (with varying degrees) for 4K or mTHP, and with or without charging zswap compressed memory. Did I get that right? I saw you also mentioned that reclaim latency is directly correlated to higher memory_high events. Is it possible that with SSD swap, because we wait for IO during reclaim, this gives a chance for other processes to allocate and free the memory they need. While with zswap because everything is synchronous, all processes are trying to allocate their memory at the same time resulting in higher reclaim rates? IOW, maybe with zswap all the processes try to allocate their memory at the same time, so the total amount of memory needed at any given instance is much higher than memory.high, so we keep producing memory_high events and reclaiming. If 70 processes all require 1G at the same time, then we need 70G of memory at once, we will keep thrashing pages in/out of zswap. While with SSD swap, due to the waits imposed by IO, the allocations are more spread out and more serialized, and the amount of memory needed at any given instance is lower; resulting in less reclaim activity and ultimately faster overall execution? Could you please describe what the processes are doing? Are they allocating memory and holding on to it, or immediately freeing it? Do you have visibility into when each process allocates and frees memory?