From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06A07C27C53 for ; Fri, 7 Jun 2024 05:50:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3A0CD6B009B; Fri, 7 Jun 2024 01:50:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 34F4E6B009D; Fri, 7 Jun 2024 01:50:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2170E6B009E; Fri, 7 Jun 2024 01:50:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 033906B009B for ; Fri, 7 Jun 2024 01:50:24 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 99DE0C01F5 for ; Fri, 7 Jun 2024 05:50:24 +0000 (UTC) X-FDA: 82203017568.06.C6F7AF8 Received: from out-187.mta0.migadu.com (out-187.mta0.migadu.com [91.218.175.187]) by imf15.hostedemail.com (Postfix) with ESMTP id 39E02A0004 for ; Fri, 7 Jun 2024 05:50:21 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=V+RV9tSV; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf15.hostedemail.com: domain of chengming.zhou@linux.dev designates 91.218.175.187 as permitted sender) smtp.mailfrom=chengming.zhou@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717739422; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+yyJ/LKuKfn3MmPsJ34/SLspsUgQV15PcfdP8ap5KjA=; b=H56f/QgHAnqwC9Prams+6/GCDkuTZOBNxcqKxkJuDFbgejEO3wQZpgqg6u5caRBd22gvuw ZPtb/1fszmcyKrhzrmQth73DSItmvvvAySmlEZR4Txja9+pIr71zvs1UPsKfjTng31e5cS cy1pmtxtKy+DoY1xoUZpzOI2aDuNaJI= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=V+RV9tSV; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf15.hostedemail.com: domain of chengming.zhou@linux.dev designates 91.218.175.187 as permitted sender) smtp.mailfrom=chengming.zhou@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717739422; a=rsa-sha256; cv=none; b=MXtSwmlMiq7o2/HjlMJE+ZLW0k2rjioj+sJyjhr8v6qejtiB/cFoY9g7yZ3/Au1OLrb+ZE JlHddQtSrnfWhN+UBikYuGnhc/AEV6RS/wIuReCiR45Ulv+5pMNAOLaQImfU/7qtD2otcq AdBehMm440DhxOw8onbfdZRq8hF2hFU= X-Envelope-To: yosryahmed@google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1717739419; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+yyJ/LKuKfn3MmPsJ34/SLspsUgQV15PcfdP8ap5KjA=; b=V+RV9tSVZpFiZoFYBh7eTuWzhsUlmLUbyyiF4B+E8s8VHkoVTkTs6e0IhbtU4U31z9Kv4Z hvLkysSqH5MLEC0Awv+AgB7wpwNeVEx46zv0CCAnshGj6Ptz6/0VDbrMR7j1A4kmuuiGwO JyD1/55d+99OuHwT1Rdb9TWCiiXJ3DU= X-Envelope-To: flintglass@gmail.com X-Envelope-To: hannes@cmpxchg.org X-Envelope-To: nphamcs@gmail.com X-Envelope-To: akpm@linux-foundation.org X-Envelope-To: linux-mm@kvack.org X-Envelope-To: linux-kernel@vger.kernel.org Message-ID: Date: Fri, 7 Jun 2024 13:49:51 +0800 MIME-Version: 1.0 Subject: Re: [PATCH] mm: zswap: limit number of zpools based on CPU and RAM Content-Language: en-US To: Yosry Ahmed , Takero Funaki Cc: Johannes Weiner , Nhat Pham , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org References: <20240606165303.431215-1-flintglass@gmail.com> X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Chengming Zhou In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT X-Rspamd-Queue-Id: 39E02A0004 X-Stat-Signature: 8aweigtspq6hdoch3f4dogj8u3w6azxj X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1717739421-3918 X-HE-Meta: U2FsdGVkX1/ff8SS57VDqf46HPbiHRvEr6NZe/VQnBK7FCZaSpJI0ALnJbk45T/sES05mo9PcVJyAxaZ7uSEJS+9evzuraY1UfnDfmHTIN1gBFpStigHDbRXw4E9xozlVSONA3xkA/RIZ/07ejtIck19ns4+7gnaVsasEkzqhSc4SoTt5VEVfLFPbP2v/nBrq2jBYClWKsmAN2YIlYqFNIUCYf+Lwyemk/5A0mxwPSfdJzXBeaHsDVnIDXfx1fc+/YIr8fcFgvxwHxRxR79F0vytlDvwvuNT24axYjgMcJNbp1pDsRHw7N/RJcolk0B3e6viF+dD+tKQYfEYLbjrJ/LueaWwTwg8qVPnCflIjIaKtl/rvf89AlWqi1vZSceYUw0Ir4hNCt44NGtz/ncGx1z+7KTff2gBimi2+LDU4/ZkODSIW9vRZTGaDzZYqpMJ8+3uyW3EGQp6ptFyXVaZ64hHU54uH2p71klDCzqWdzdwPz9C7GlmUuux38CF/xuGgX8KPdbKznaHKnebDzXriQQ6QGNUY3oVIdEhnpf9mOnXyvqOsJGGzZRKrLw8YT8qycgRZJf+ypdS8eXOlXSP0xQhMHiz/Y/S+eJdYwmpJooCGomy3MaaXumjQ000mNTs00tqbW0MOtm26CEmNaCIJ8rjeDWv1psbGieamkUwieHOOarZ21LUdZ4FO3NpXBMcXN2LmPMu5FaoCHUFAAYSEE25NwZTJoMLnh7mX3Mo+7cqE+oJcWOHmjNu+WzklpDiAXcDyPKCW6XKklMJ4bqM9ez8vmXgYeIyoyUxibZXAfJ56YI4mAwnRlD7LkImtFKEHsN7/7I1olJrcaSYskhgU+W6oouv1L3WIVnO16ezuS6uX+Ibnu8gNVLyJiWQg0uTt8Y0gV1PLqkh8Lts5tYk58nJWVhr9un1BbQKMnDlBR6dhZuZFcJFP0QzYOtZu4DFurad7tKlFhkDKiUy4iW 7HtjQB01 ZBEPtmLAJN2YJZPfXEqBB/pNY+XhzbXJ3zaD9XB1I8nqwIHwfZfTkP7uc8BcyAcUGPDwROh4Z3LiM/u12mDt30EcAuwKiyOSx+LG1dsbdxwpNq5OF3yTmnvcwA3y/0JQsZIvHk/dNz24TbEHI+oGilkFko+kdT9IbppMT X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024/6/7 12:58, Yosry Ahmed wrote: > On Thu, Jun 6, 2024 at 6:01 PM Takero Funaki wrote: >> >> 2024年6月7日(金) 2:46 Yosry Ahmed : >> >>> >>> There are a lot of magic numbers in this patch, and it seems like it's >>> all based on theory. I don't object to making the number of zpools >>> dynamic in some way, but unless we do it in a data-driven way where we >>> understand the implications, I think the added complexity and >>> inconsistency is not justified. >>> >>> For example, 2*CPU zpools is an overkill and will cause a lot of >>> fragmentation. We use 32 zpools right now for machines with 100s of >>> CPUs. I know that you are keeping 32 as the limit, but why 2*CPUs if >>> nr_cpus <= 16? >>> >>> Also, the limitation based on memory size assumes that zsmalloc is the >>> only allocator used by zswap, which is unfortunately not the case. >>> >>> The current implementation using 32 zpools all the time is not >>> perfect, and I did write a patch to make it at least be min(nr_cpus, >>> 32), but it is simple and it works. Complexity should be justified. >>> >> >> Thanks for your comments. >> I agree the 2*cpu is too much. it was conservatively chosen assuming >> 1/2 contention while all cores are accessing zswap. Much smaller >> factor or non-linear scale as your comments in the main thread would >> be better. > > Chengming is currently experimenting with fixing the lock contention > problem in zsmalloc by making the lock more granular. Based on the > data he finds, we may be able to just drop the multiple zpools patch > from zswap. Hope so, not sure now, will test and compare one pool with multiple pools. > > I'd wait for his findings before investing more into improving this. Ok, I will get back with code and some testing data a few days later. Thanks. > >> >> I found your patch from the main thread. >> One point I'm afraid, this hashing will fail if nr_zswap_zpools is 1 >> or is not rounded to order of 2. hash_ptr crashes when bit is 0. > > Yeah that patch was just for experimenting, I did not test it well. > Thanks for taking a look.