From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A746AEB64D8 for ; Wed, 14 Jun 2023 15:00:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2E1036B0075; Wed, 14 Jun 2023 11:00:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 290416B0078; Wed, 14 Jun 2023 11:00:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 01E2D6B007B; Wed, 14 Jun 2023 10:59:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id E85236B0075 for ; Wed, 14 Jun 2023 10:59:59 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 80BE81406DA for ; Wed, 14 Jun 2023 14:59:59 +0000 (UTC) X-FDA: 80901663318.24.D9B41F6 Received: from mail-ot1-f52.google.com (mail-ot1-f52.google.com [209.85.210.52]) by imf28.hostedemail.com (Postfix) with ESMTP id EEF8FC0029 for ; Wed, 14 Jun 2023 14:59:55 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=cmpxchg-org.20221208.gappssmtp.com header.s=20221208 header.b=aSGtMLAR; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf28.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.210.52 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686754796; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dfTAd0/uVK+gWwVM3TF/SLIQlNoUyF/c3rvI5XRmuAQ=; b=FajFev9hucfAHTjgltJ/hWTW4MVs9KEFkwUbZ9bEvynw8av9R1dXDlvVU5eSsTxfLg8q2p 0ep3NgkE12Gt6lhGf0emlD176KmCEE10lxn8bWQybfrh8ErqFp3o8rxVGPcBC8SWfPrP06 0z6xQPoybVI+6kJVoOFyTSyTKbTGpBI= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=cmpxchg-org.20221208.gappssmtp.com header.s=20221208 header.b=aSGtMLAR; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf28.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.210.52 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686754796; a=rsa-sha256; cv=none; b=IQ05BXHV+o4hZNXy88R1AKbglVKeyinASn8To2MaTpoQOTWw+SEWEpA+AsT3qFxq1ax5gI wrd03MusbmwV5My/CNHqYqf7xu74e0XIty9qBUXrVYZtGQDNLVVY+6C2QowsxCXMGkS3cm cHgC1R71PZxhA8h9kKK2Z3H4zeKVws8= Received: by mail-ot1-f52.google.com with SMTP id 46e09a7af769-6b2c6238cf3so3922115a34.3 for ; Wed, 14 Jun 2023 07:59:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20221208.gappssmtp.com; s=20221208; t=1686754795; x=1689346795; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=dfTAd0/uVK+gWwVM3TF/SLIQlNoUyF/c3rvI5XRmuAQ=; b=aSGtMLARzci2di/1Rpdv8xh5c1HJsKdm2H4SR6/LQjfYaNAJxKiz09SlynAzHB0I9h 66ICKopEL4NhcLH1nLIvsatgho0jHLzitzSn0o2zat7OzwfKAWFwwAWP1mWEVZ06YCNO nGPYmmnl5OxWbuteMziKGLn0ZLpxNAkJcxUv7Uizr6qgFw9Xs4VvFAjLR+msqlKyRcKB I3U19BEov3D0Km7Zhqg/PdZ7raeuwWY+nuqgsVUgYezn7iV5JXZTe866AtL5/iz/CHOL NOTYaKgfoe4Yhc/OcPearfE7+6ld5wXxQrZ9WCV+5hx5+7AybxwaBp4XZ/0rlGFfxX5z UwqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686754795; x=1689346795; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=dfTAd0/uVK+gWwVM3TF/SLIQlNoUyF/c3rvI5XRmuAQ=; b=UVpUen3Oh0uavLEmckifDplTDirq1X4ptaxYKhKTAzTMuCgz14r3dXXNFOgqGMe4Uf mwoykwlrNPru1IeVVuTpGJV5+BcfLZmu5DmN2VE1/bbKRVb2w4FiYeGsSQPqi2T1O05H ByIZR/ItCMx2q0atH09pyAeUAda8kK1ttXxnMaQafHUPA697ztinxr7J8t28gMBu4jB2 2tKWDXaZ9yYSMbV4Q25XvvhCFhFJtILIV9SGHZ6klLxhFa0BA/0Ql+hqA8lVBRLlryXF eJwSfhAVJqO6wDvAj+LcYR73awMwLaZ/c0L9vV/QsBIl/Izp1lmxdAaRg7stc3xWgCN4 Kr7A== X-Gm-Message-State: AC+VfDwzNZ9dxsB0ooLGrKd7kx4yXRZd0FSPGtNCUWQShQ7mA4FWyz9i B/53VbvSmoFoDLo04XOAReKVbw== X-Google-Smtp-Source: ACHHUZ7zIGFCJJU8MQqg8L9cSdY5qrNalqmf/z+VT8WF0tStfZz8Mq0+DJDKzoSCYzC6GAuN2Tasqw== X-Received: by 2002:a05:6359:609:b0:12b:da3f:d0e3 with SMTP id eh9-20020a056359060900b0012bda3fd0e3mr6919711rwb.10.1686754794703; Wed, 14 Jun 2023 07:59:54 -0700 (PDT) Received: from localhost ([2601:586:4c7f:f985:f03e:5c2f:42af:384a]) by smtp.gmail.com with ESMTPSA id p131-20020a817489000000b00560beb1c97bsm3906768ywc.97.2023.06.14.07.59.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 14 Jun 2023 07:59:54 -0700 (PDT) Date: Wed, 14 Jun 2023 10:59:53 -0400 From: Johannes Weiner To: Yosry Ahmed Cc: Sergey Senozhatsky , Minchan Kim , Konrad Rzeszutek Wilk , Andrew Morton , Seth Jennings , Dan Streetman , Vitaly Wool , Nhat Pham , Domenico Cerasuolo , Yu Zhao , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH] mm: zswap: multiple zpool support Message-ID: References: <20230531022911.1168524-1-yosryahmed@google.com> <20230601155825.GF102494@cmpxchg.org> <20230602164942.GA215355@cmpxchg.org> <20230602183410.GB215355@cmpxchg.org> <20230602202453.GA218605@cmpxchg.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspamd-Queue-Id: EEF8FC0029 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: jtzpweq9pbgd7b5n39iu1qkww4a5y7dk X-HE-Tag: 1686754795-454195 X-HE-Meta: U2FsdGVkX18NayCqdwius9SFjuDnieTsJA30OTrDX2i5LpIX3amw8W37lL6JVIeQ0rEMDSpO46aluSOdz2WTQEGcjmjou5IokpXbJ+SRjRQYlymsTqXbnmcpmh5Kg5uRTQ/ygjfLhqLxwXAcMGiL1kWToOOWUarMsxkLzViX/j5Y9VasOW5U9QPNuQzuimYlk7I11905eRJNPPUUg8YrZslfJW5s26E5eDTyoiixZBZojLtbaF2+0BwwsYesYAfxqq00uxDYIVK8YfdH5ORSOXDQmduNywXsdBssIgY+VDH2jlgq96II9ZcgCw7seIgt9b6wyvC34IgCqwzH9Mo0zYndFrUQG9dq7CsMtKMPqsDGN1SNxAvak0n3XyJjizeGIdVz8fcjiPhH8R2utA6WAKKo3T+KEbmc7qzHuKNtsLp1fNV/XdrkJhje8irIUh9w15hdV0+Mw/1qCOHLFZydLI50tG/zVFqxSX7fUkBCXaGksRAsPALjCQSWQJTJeOIiTDHQoPqnRH/DRRhP42EDE1/rDzI8IYnq3FN4EM4NG46/MEDwlYz+Sii2kLxGF+U2klvbHComnRMpDVeEbWCCEDPkCLiW8OX5GT1prlJEJ7C1mkTxikjZw0nLKwAfHHtiK2FzIAyktnSmWb2c1JEEvhPKriUUfzw3p41fJbf+g3fp1tEun9+QmCd7ko6T0/20So/M5a+BwwzgYl80mYb7bQmeBCrX77f5r5M6lS+z2vXcsYlURj+sQlYpR5G15EjQYAOPwvK3++3BcnGVJn8S2qmgLwZeUl88In/P90ylwKRkHF8Raci8+5Ks26y5JWXIR7E28K2JU2fGwBB7vXkAiUgpeeIkORHmPTaPQMAW+aY47i8l4/JfHl9dbbX1rtm+5YjPGXJrANJ3vdY8aRDMxoymOTIeBzISTjWCFVMamXWDU3eYCffUl5Y4sMPRkXQrfvhlT2G08VBpR/XX8Ir xAQQ8gBr wKU81WKRJ53ZWtSOijwny7k3/daqqdTEMkMmy1gVziEPfEp0p9RkWQNbi5LxwhGpd7h/iLqDklFVyJKzNpLVqcFTDVUJYn6mBFPKlPBkwbRU2FKI6Cv0vA8rqWQ+fkRx25f7x4fc75NYa4EVd1GBM+iDsKZra2xfAqSOnirP7EVsbXJIu49UGmMYdJ6z0e7+gorjJ/uaVcwBuEK342hxo/eh5jNRw3LXYC7nd4BpSfVyFZliPSX4/9wAYSCjQvjqvF8xm8F+4R/HfVjz8TBb9+MuwsLVXLJPyR9aSRP0FtcSEPYdnSiQbn+CaO9AFs832RvY/TmlgO1H0XFlQeaeKd8coI2tA8YBbUhRMMvQPsG/7BiWKpdM87H+x//pA5FZBfqYtGtW8VqND6eCiF8XL0Zhptyf8Kksj6OC+V9W70hsOt1qNjIvTCzfZg8LjpV1meCkw12I1b9JJvnqCxmNsvsc3/4wRknmyYC8NJayqPd9/XUwmuJAbyVwcl/tBwT6KRvIB X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Jun 13, 2023 at 01:13:59PM -0700, Yosry Ahmed wrote: > On Mon, Jun 5, 2023 at 6:56 PM Yosry Ahmed wrote: > > > > On Fri, Jun 2, 2023 at 1:24 PM Johannes Weiner wrote: > > > Sorry, I should have been more precise. > > > > > > I'm saying that using NR_CPUS pools, and replacing the hash with > > > smp_processor_id(), would accomplish your goal of pool concurrency. > > > But it would do so with a broadly-used, well-understood scaling > > > factor. We might not need a config option at all. > > > > > > The lock would still be there, but contention would be reduced fairly > > > optimally (barring preemption) for store concurrency at least. Not > > > fully eliminated due to frees and compaction, though, yes. > > I thought about this again and had some internal discussions, and I am > more unsure about it. Beyond the memory overhead, having too many > zpools might result in higher fragmentation within the zpools. For > zsmalloc, we do not compact across multiple zpools for example. > > We have been using a specific number of zpools in our production for > years now, we know it can be tuned to achieve performance gains. OTOH, > percpu zpools (or NR_CPUS pools) seems like too big of a hammer, > probably too many zpools in a lot of cases, and we wouldn't know how > many zpools actually fits our workloads. Is it the same number across your entire fleet and all workloads? How large *is* the number in relation to CPUs? > I see value in allowing the number of zpools to be directly > configurable (it can always be left as 1), and am worried that with > percpu we would be throwing away years of production testing for an > unknown. > > I am obviously biased, but I don't think this adds significant > complexity to the zswap code as-is (or as v2 is to be precise). I had typed out this long list of reasons why I hate this change, and then deleted it to suggest the per-cpu scaling factor. But to summarize my POV, I think a user-facing config option for this is completely inappropriate. There are no limits, no guidance, no sane default. And it's very selective about micro-optimizing this one lock when there are several locks and datastructures of the same scope in the swap path. This isn't a reasonable question to ask people building kernels. It's writing code through the Kconfig file. Data structure scalability should be solved in code, not with config options. My vote on the patch as proposed is NAK.