From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4CC69C27C53 for ; Fri, 7 Jun 2024 04:58:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D0EDC6B009C; Fri, 7 Jun 2024 00:58:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CBEAA6B009D; Fri, 7 Jun 2024 00:58:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BAD6D6B009E; Fri, 7 Jun 2024 00:58:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 9E42C6B009C for ; Fri, 7 Jun 2024 00:58:51 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 1938D1417EA for ; Fri, 7 Jun 2024 04:58:51 +0000 (UTC) X-FDA: 82202887662.28.1991DEC Received: from mail-ej1-f44.google.com (mail-ej1-f44.google.com [209.85.218.44]) by imf30.hostedemail.com (Postfix) with ESMTP id 4777680005 for ; Fri, 7 Jun 2024 04:58:49 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=iwvTan05; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf30.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.44 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717736329; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=09oLwl2Fi7q3ZOGUqyDhPiw36+NyQKSV2deDo6izUjk=; b=mXa+0NcVCLd+3prESfUHFziq9eY3QxeS7FWUS7UYqnWEpxH5Aa0NMv8pdWtUq9ePc1Lzgs lyOY53abdfrDJuZy0N8ymEK1he4qlwc6P1j8MOG4nDNidbRVd502oejhHX2S5UcxUxGZ0g ImxTHQ4iBKV7oCfboqbbA6sidBxSwQw= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=iwvTan05; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf30.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.44 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717736329; a=rsa-sha256; cv=none; b=4GNfeynNFvlgNzESautRfnZg6A7th8R+t02naS1uLJmbV3pUJChXEAQPGs9ec9hV/f50NR JI5ROcOvvYcnYucY65wQocRpoDOrzltfFwvIVs3CEJczgdWS1bYGCIJHIw6AkEQSbZ4BO/ A8pRMbFXMO8Qg05FL0Y/zxrRlmwXJD0= Received: by mail-ej1-f44.google.com with SMTP id a640c23a62f3a-a68b41ef3f6so170211766b.1 for ; Thu, 06 Jun 2024 21:58:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1717736328; x=1718341128; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=09oLwl2Fi7q3ZOGUqyDhPiw36+NyQKSV2deDo6izUjk=; b=iwvTan05InowufGOs1ezsQTmEgvHUyET3Wrd8+hwUaSlMg//96a6c4YKyXso5lfZSP InVpWye0jx54YN99aqdCmwEWQf8VIaemC4Qesot3t/9X2KguB5jLlfLkAOqtqU96xBj8 1MrIKM5HcPjvpL5iVrsN8CzTwJkEBuRCUAS+Yu36bZvIH9jaKUxVJlJXTcrdBUaaEfl3 Ywzm6HBqErYFnfwi7bZsJc2eWu9wdWSHKGaW8sFQl/MEdJTn7fTxB7eXrRKlCRL5DtqP onBgrf3lDau/gQtpclIEoOd4ebzBZGVtpfllFq1/Y6mD42CJzzNiZOOy5WpHhLAa3yU4 M13A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717736328; x=1718341128; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=09oLwl2Fi7q3ZOGUqyDhPiw36+NyQKSV2deDo6izUjk=; b=uYF+MYhDv3wFhV7JLY+xJ4nApZpJgdxfWAKJ0yZZJM5JGh8cllFohcPOkYGYVdilL6 I87vlgtlDTYhOiaNWWw5DMdux9zGC7+UQVqK3KPbunpEP8BIVNBXjmXQar9gwK3WU3na 3crs/2M4N3XwYjLOeq41+LykXvmymsk45T15N5t7CEBo0pyY9SmRfzgngmfG4C4BUQpL 5XtRpmrUNDRpbGWqvC5ohM3FiungbsyJTRXlTbFzcSGeheFwY9VWJfjdvkzT/XJF2EF8 KNTD4aiKF7XcaaIDjFNQET/Z+gQ9BOGzLbg8UnkGTMg3Ka3+CTAYklT6DUpf2jRsmOZh AIDg== X-Forwarded-Encrypted: i=1; AJvYcCX3SssYo43DFZkryRuUgZFn55yTWFHWvv6o+pMmGpN0vU9ou7a9Rs47Ti4qgOmyIC7jbypdtlJDizQaq7Z46mUmXJk= X-Gm-Message-State: AOJu0YyR31ZbpxuZhEkjX1dzyl1OBwk+tB3Ac95vmqShBxEnuWJO0/UT CHtjgkR8ArgMQVjU8zBjjQBZuEZaXLXpWMXCotFL9xVK1sRxhjFqplBueIQRjbg0C7HITE5TUx+ GB8r1WKyumj5LHaCxnno6ByeyMixq49HQ5pre X-Google-Smtp-Source: AGHT+IEQDEkW/UIMBTN67S6TtKaGz+0SsoOGahvJjV/bvIcWOW58RFnZgGEnkj/89Q52sY1SCqwuAn43LLmE3J0H/0s= X-Received: by 2002:a17:907:766f:b0:a68:f1ca:db46 with SMTP id a640c23a62f3a-a6cdc0e077cmr81577266b.64.1717736327464; Thu, 06 Jun 2024 21:58:47 -0700 (PDT) MIME-Version: 1.0 References: <20240606165303.431215-1-flintglass@gmail.com> In-Reply-To: From: Yosry Ahmed Date: Thu, 6 Jun 2024 21:58:11 -0700 Message-ID: Subject: Re: [PATCH] mm: zswap: limit number of zpools based on CPU and RAM To: Takero Funaki Cc: Johannes Weiner , Nhat Pham , Chengming Zhou , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 4777680005 X-Stat-Signature: yrrizwjrrkh4ogswc99joysddzsop87k X-HE-Tag: 1717736329-351273 X-HE-Meta: U2FsdGVkX1+LZC64Mgpe2ZwlEQu/bJRC0BAFgt25werc+K/mm9bl8qvRIBrDV4U17yWAeDfvGCZhooOAEIUU4wCMquQdVJvP6nsWhx0SSDqTkgiyaw+3o7baWGKugA9JdugTOqN0G/O7afL8PYclA2lt6MdTBGH3Epx8Xq6SSSj9d0qfS/RrIaLKouA3k/D3Cj88IWDuPeS2AEDWy+YkRh5ZiUmvGaf7y8IShQvQ91gaOj9t6QTzDYe7EkuTEOnCGOe+wMHm6uw1RWN6w/LMIGGHsFPfY1sPrBCvXCBfBNaamBr5RBpQMBdXAAyM9lnpxDUhkL4f0M3lsPtAkvQCRBy9dNauVQL5EkQWT0j2QkHCEg/5KrajTZma0qvbeVycuPnyzT3B0SfO8Tx3ghJYxGy5GCSd3HlIyFV/uH3HOTE6cEwYDvGXCY6OZl0jkLBr8MuXvWBJzeU6I5yw6wkcC0DhdvLc249KHMIjDBkLX9k19yjR8QJAsqrYF7pZUGgqxPw5CKe9qcv1Qgfd9N3IHXhnzYqKHo1SGCcw41wNbL7bWyYmhQEWw0eHydicVxHOcOPMrXxxfEsRYyKVF/2UBsMmkvCooRCR5ef9vKrZ5YAKZwnB2w9nyc8OFSb/c1vaRx4evfCSRwg7uJOY2JHp2nTRbFxPZJL8V1rYwZBhr8BiuAOKaGLE9V5dftbOUu8kzc9UPIEu8ujVrMeMwLtbkJCjp5UkeqerSgBU3s5RRAezREJM+B3l/9G685k5SrGuLEas02qT2kY92W3uvUzv/AbfTQUR43YbS9BPvCaHBupfy2d7zxTaLJoH7N+ym69hg0Fona9sDifKaipg13nbWyfyE+91gNnlDXExjlq/TeGmV6O84DJIPkbQN0YEdDyzZcgf4FfO66o5PCt6FxDuw0mRL7Yuwprcu1pfnElsFHa6qroZSE/j4SzdUbHPAWWQTw2EcLhtapQdiFZ/zME yZ4nzMT1 dUU5EA7Dc3500HI6Hzideqsnd4GrkoXqkfNxDlWkNQrgLWo24QoVnWGo7evgQxSt/+Jf6naIDPBWD9TzERB2DHrlprBrh7hPLutiWpt4ga3v5zJavZ0iqvWorFiu+J71d77etERjlCez47Jl3/eiNCLJwLsSeuuF4DNNuk9fLSahvyMZgnPpp0T5BvkmfMcmDSiJZ07eEoYlTmMktO8dWXP0m7jbZx8FijXGRjs2NXd1df+h/FULgxt3EpZCQCCJ/mOTKeVmiIdRgLoA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.001854, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Jun 6, 2024 at 6:01=E2=80=AFPM Takero Funaki = wrote: > > 2024=E5=B9=B46=E6=9C=887=E6=97=A5(=E9=87=91) 2:46 Yosry Ahmed : > > > > > There are a lot of magic numbers in this patch, and it seems like it's > > all based on theory. I don't object to making the number of zpools > > dynamic in some way, but unless we do it in a data-driven way where we > > understand the implications, I think the added complexity and > > inconsistency is not justified. > > > > For example, 2*CPU zpools is an overkill and will cause a lot of > > fragmentation. We use 32 zpools right now for machines with 100s of > > CPUs. I know that you are keeping 32 as the limit, but why 2*CPUs if > > nr_cpus <=3D 16? > > > > Also, the limitation based on memory size assumes that zsmalloc is the > > only allocator used by zswap, which is unfortunately not the case. > > > > The current implementation using 32 zpools all the time is not > > perfect, and I did write a patch to make it at least be min(nr_cpus, > > 32), but it is simple and it works. Complexity should be justified. > > > > Thanks for your comments. > I agree the 2*cpu is too much. it was conservatively chosen assuming > 1/2 contention while all cores are accessing zswap. Much smaller > factor or non-linear scale as your comments in the main thread would > be better. Chengming is currently experimenting with fixing the lock contention problem in zsmalloc by making the lock more granular. Based on the data he finds, we may be able to just drop the multiple zpools patch from zswap. I'd wait for his findings before investing more into improving this. > > I found your patch from the main thread. > One point I'm afraid, this hashing will fail if nr_zswap_zpools is 1 > or is not rounded to order of 2. hash_ptr crashes when bit is 0. Yeah that patch was just for experimenting, I did not test it well. Thanks for taking a look.