From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66B28C27C52 for ; Thu, 6 Jun 2024 17:42:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B6E246B009F; Thu, 6 Jun 2024 13:42:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AF4F76B00A3; Thu, 6 Jun 2024 13:42:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 96E6C6B00B3; Thu, 6 Jun 2024 13:42:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 72B156B009F for ; Thu, 6 Jun 2024 13:42:35 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 29AA6A055A for ; Thu, 6 Jun 2024 17:42:35 +0000 (UTC) X-FDA: 82201183470.18.95A6D13 Received: from mail-ej1-f43.google.com (mail-ej1-f43.google.com [209.85.218.43]) by imf12.hostedemail.com (Postfix) with ESMTP id 45B0D4001C for ; Thu, 6 Jun 2024 17:42:33 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=MowjI9wk; spf=pass (imf12.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.43 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1717695753; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=v1Ut3Dkh+dW6ZjfW0SAbH75TKykJw59snMxcOhxXPyw=; b=TVW2IoSkABsBiCX1Fmn5Cc+gyHJqWdO6El1BKqX8U+em2nv91tSRvtZnrk46D8l0CGjv5j 4Toktw81QjKfGjhbRxUEo0ssG1Fjbqv68UsPrBS57qlwqEF2lWOEIsHMhQRm8q21W+HbSS owr9yrCSFhiCFV2fAcfeOWmuAQi4Qho= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=MowjI9wk; spf=pass (imf12.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.43 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1717695753; a=rsa-sha256; cv=none; b=zoiLLQFciByUrMr6YmbbJdGaDKG/Fm+pNW0ddhgNA9hrOVEQdx5PpHbsnZOaLOfNJ78Qwt wyXUiTJHrr/NSUJ51BFaEPjERMuUu30uAt3BiRfQR+FpkXS+7BDIEkPcsf3JgOM5IalDy8 htnPEBWXvowq78P4+xUDPaDJq5iL+3w= Received: by mail-ej1-f43.google.com with SMTP id a640c23a62f3a-a68f1017170so158198566b.0 for ; Thu, 06 Jun 2024 10:42:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1717695751; x=1718300551; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=v1Ut3Dkh+dW6ZjfW0SAbH75TKykJw59snMxcOhxXPyw=; b=MowjI9wkisMaUEs6hpV7U9MwV04X9ano4u7Dm+F7WloWvs3q7Tnkyp6zcAd+lapq0v G8vijKOBPSprQZnvGZ2AVyjysqsrBT5rZokUkuuHyRdkEDGsvuKADX7Zv0nYUH+7R6/h rF2JvKcfIHBrEOu8IxQmJ6H58G2Z0ihcB159UMPitUUMcFOjiRENCFwh59y9xOPQl6TC iJBTZnxPJD2Iq8R2q7W0VRop6+avvaEeovx77vCkP369HyhvHyT3EkRIyZxgjPMPzXZ4 EuRLcQBJWelhV6g/wEDO5ZE/QjPLXb6voxHETTrDJwwJDl8T0oufk6gyY3PrP80VGIPq Yq/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717695751; x=1718300551; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=v1Ut3Dkh+dW6ZjfW0SAbH75TKykJw59snMxcOhxXPyw=; b=U0h2bvvW8qLk0/Lbw8McspDp5h5vfyzpMDewj1bNtFDUc4xjHEhvp4ZypuvLjqIjd2 oDpTki5mPcVvFddiojnSaQxSa0cNc7p69hE8urfZyh5QzbvJdiB4AaQvoyXk0tXXGkqn Qq/bJTHwV4gdFMqC2NgRVzJt3FCNyINkAO2A6wEgCKF534cEzLXb/EokV4Fz/t+dSwWR Axn+2I7YC8DibPelMEgjvKTCzHZrq9lKxF3GzIdS+NIoaOxjtmsEUMwqhm1JSUa85Hxz wdggErdF3BWAIvPeoNzU+9GwpunelhwacX+YI1JqhNfqvcyOioOikaEbNvq/Jr6bVz2V Gdpg== X-Forwarded-Encrypted: i=1; AJvYcCU6FQF93nQZdVjq47CTjaEzHcxJslVbfMHOaCtaDChOUm0vOCRptSOB4Pne8W3CU6bYlajN8aLVqxKLuZQD+NjQOwk= X-Gm-Message-State: AOJu0Yzh6s73HzOyAxXkDFYox2upN/8lgYg43+nKPcabracADCy/OoFL k+YLSKbgbwx48QqtDfXFI6V3SWqQnExUrG4cwo4c/D1WWJB7akhYn3561n7aKaFsXlkJktacHc2 xZOXRgbPe5+4MJ9yN4XaC5FHM9pg/6e+lGZ/M X-Google-Smtp-Source: AGHT+IEGBdWWUZC+WrQXxIBv4KOZPlFFbVSfkIEVyzAaxtZJ4s8YMZ9cpHiZkAazV4knO5rL7j2QTzt4B0g4tFKtZN8= X-Received: by 2002:a17:906:fb11:b0:a68:c5b0:9890 with SMTP id a640c23a62f3a-a6cd7a80f9dmr20626266b.42.1717695751357; Thu, 06 Jun 2024 10:42:31 -0700 (PDT) MIME-Version: 1.0 References: <20240508202111.768b7a4d@yea> <20240515224524.1c8befbe@yea> <20240602200332.3e531ff1@yea> <20240604001304.5420284f@yea> <20240604134458.3ae4396a@yea> <20240604231019.18e2f373@yea> <20240606010431.2b33318c@yea> In-Reply-To: From: Yosry Ahmed Date: Thu, 6 Jun 2024 10:41:52 -0700 Message-ID: Subject: Re: kswapd0: page allocation failure: order:0, mode:0x820(GFP_ATOMIC), nodemask=(null),cpuset=/,mems_allowed=0 (Kernel v6.5.9, 32bit ppc) To: Takero Funaki Cc: Erhard Furtner , Yu Zhao , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, Johannes Weiner , Nhat Pham , Chengming Zhou , Sergey Senozhatsky , Minchan Kim , "Vlastimil Babka (SUSE)" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 45B0D4001C X-Stat-Signature: 44fgfmnsb43jwbgpizmaw1731mdtxxd5 X-HE-Tag: 1717695753-936061 X-HE-Meta: U2FsdGVkX1/OnPY7TrLnRw+qxC8sGDseHHpalM4r4L8pr0oYHN50K8Tec0V131VQCIhhCAgNcq41WBUtOkOv9JC8n60ZWROqr36TLu1aAEg5LqYn4JiLEAaljpQM2eaHHzdU6oHLmjpTTvBuebg3ktUSI9ux5HirzoJuD9uYZjLlau7FpKga/tPNox/ANYv9eFv4q5k4bU4khRiBPubmPfESyoiZlcprVw1GzwloM+Lp3CluZbdxbD6BDybEs8hyIwFRxM+MKOL4jJ9lj0GIaPLllr87YkoneFSpLxwWHgjuwle7LxTyul3MkDPtxWDE6yepKz0rmGI/vr2a8xCK0fFm8YvHdfxfQaCdzEXMsXkuWqXDqw17zZZg8h7IXAmvLeTCxK5bP6Maa1RWixyglGJHdPKV62JvQQ46AfeWNGrRY1yTknAhfA6sd8BQGNhk6fROKq37DFOWmN9eA8jQbRJ1BvcKUjGSQ9WDTWbkNdsMbDGXp8z3a0DFoeHc12+IsOzVrBEhfKODSfYBzsfG6hyfS6lmHJjr65rTw1mDV9i5XZYgnuJLjTcEARG3+ZuXAgy3cjzlXMUTQj0qdGNGxefABx0EMnlTGdvW8Uuz9cU6oFjZJT66kTqifjblNH6W/zoZ5aXLHw00+RbQU8h1P9A2/ANXxK2CBjmfGIc0rbIr3J7defnYMf1pyq70PctFOnntKrN0sf4rqRK/F48mkI8CV3Ndk5LDTrzdbsVKfmnUoBSyYWnCVkSn/KPRbPpJ/VYCUW+b0YZQgqhnZ9zbybHK2YtzzCH9fdAZYeBoyGp+LpbtGMT6hQ0XCCgCUTYJ0P8LXga5OcWfeK6l/up4Gsu7gfqXh3b0DYq/OzpAQqkIu9BxN67yxGTprkgk3bP9Y5rNIe6UNBQaIFpxH1LFZPZdzQm9UabEgk69VmtQZRfBkr7IbHJvp2xylasecHTeUIkkEYTSyLEcDSXoN/k V3FY+94k Am4odtBsXGrsQKL8DzjoAp5u1btmUcIHreLejlF73nYSigUgkdwGx8O4XxAFyTrLbkTs6JUIHq8oFGaN9THcktr/o6oxQ1fHYb4STfNXUsh5o3EYrQo/LIomFGT1BUvo3chUsRqJf4wFn/xgxcrZs9pWW0VHbZpnGWUro596qlmmqoD5ykJ27gb/WWoyTmmWGpDbo02zWh65ccZkvRZtQJtZXxxUAFXJRBGs7NaAHPso3BhzzaiaaWLN2jxJt9GFoIkB6B7S/NTVPWY2mT5/MkNgQphHkHAf+A28UJEVteDiNQcimA9r+rVDRa455Lb/s5nR4 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000008, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Jun 6, 2024 at 10:14=E2=80=AFAM Takero Funaki wrote: > > 2024=E5=B9=B46=E6=9C=886=E6=97=A5(=E6=9C=A8) 8:42 Yosry Ahmed : > > > I think there are multiple ways to go forward here: > > (a) Make the number of zpools a config option, leave the default as > > 32, but allow special use cases to set it to 1 or similar. This is > > probably not preferable because it is not clear to users how to set > > it, but the idea is that no one will have to set it except special use > > cases such as Erhard's (who will want to set it to 1 in this case). > > > > (b) Make the number of zpools scale linearly with the number of CPUs. > > Maybe something like nr_cpus/4 or nr_cpus/8. The problem with this > > approach is that with a large number of CPUs, too many zpools will > > start having diminishing returns. Fragmentation will keep increasing, > > while the scalability/concurrency gains will diminish. > > > > (c) Make the number of zpools scale logarithmically with the number of > > CPUs. Maybe something like 4log2(nr_cpus). This will keep the number > > of zpools from increasing too much and close to the status quo. The > > problem is that at a small number of CPUs (e.g. 2), 4log2(nr_cpus) > > will actually give a nr_zpools > nr_cpus. So we will need to come up > > with a more fancy magic equation (e.g. 4log2(nr_cpus/4)). > > > > I just posted a patch to limit the number of zpools, with some > theoretical background explained in the code comments. I believe that > 2 * CPU linearly is sufficient to reduce contention, but the scale can > be reduced further. All CPUs are trying to allocate/free zswap is > unlikely to happen. > How many concurrent accesses were the original 32 zpools supposed to > handle? I think it was for 16 cpu or more. or nr_cpus/4 would be > enough? We use 32 zpools on machines with 100s of CPUs. Two zpools per CPU is an overkill imo. I have further comments that I will leave on the patch, but I mainly think this should be driven by real data, not theoretical possibility of lock contention. > > -- > >