From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 58C8FC021AA for ; Thu, 20 Feb 2025 02:48:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DB26E28028E; Wed, 19 Feb 2025 21:48:48 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D62B528028C; Wed, 19 Feb 2025 21:48:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C2A2828028E; Wed, 19 Feb 2025 21:48:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id A5AF228028C for ; Wed, 19 Feb 2025 21:48:48 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 36FF74E6B0 for ; Thu, 20 Feb 2025 02:48:48 +0000 (UTC) X-FDA: 83138790336.03.24C6EB1 Received: from mail-lj1-f170.google.com (mail-lj1-f170.google.com [209.85.208.170]) by imf13.hostedemail.com (Postfix) with ESMTP id 5724720002 for ; Thu, 20 Feb 2025 02:48:46 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=EuUeSHhD; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf13.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.208.170 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740019726; a=rsa-sha256; cv=none; b=8cd27cqt5G9lrB0kFKEdIFOYV78+t1aK2Xuiwokyp5NvKbhvd+BQeBz7K9/dgS95AAGRpP NIbK1MVV8v4BSLVzm43f/qI8RuGqpfNt0lhygfLNTek7iz3WBr3YeSOS1yIphHG3/8Y2+q VOpYTodq6SFj/a+LjVF7/ewH+1rNoaA= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=EuUeSHhD; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf13.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.208.170 as permitted sender) smtp.mailfrom=ryncsn@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740019726; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HzMxoSg2tOMNdSdKmknVvWlQeKQe3RfI+4wHzR7T3b4=; b=FwSMhH+6ZOtxK7ZKAWRJsQ1HC9y6Io0UZhU1zHeeWbdtDHtSNHkUfPinuNZzDBoQo3JtzS JnUptb/YqSF0+A/nu6SNKpx/H4v/IsVPq8bcoi+0hV6SyqlY/f/lYhjw3A/ZAixe0B/iSB /lAfoKp4F/9DoDNUwmEBMQLb2BfwMig= Received: by mail-lj1-f170.google.com with SMTP id 38308e7fff4ca-3072f8dc069so4038751fa.3 for ; Wed, 19 Feb 2025 18:48:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1740019724; x=1740624524; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=HzMxoSg2tOMNdSdKmknVvWlQeKQe3RfI+4wHzR7T3b4=; b=EuUeSHhDBZmqVXb+U5sk+GKnL3ig21lpcQw19ru9hMveS1qb/pKd7lAcsUPWQvclRr /kRoic8yvadUSCwkT45f0TEy+zxcX+Fr6q97crcL3BnA0ydeUhvbcDA9BvbqxZqiC6rz 70DYQ4hniPqSLXo+4ZSSdUKPDDMVq5KEQ3mBpYqGJJScjqoIoM7RbYpE7WVAeaWO7LK7 MfCzVcbspj/xHvrraV7J3mIh4UNRV3EkzqZPPoaQCtz2deAmOpASAdS9jXkY532D6/Zm aU/z7mAFoXyxYaeFoMOzOO6BYAjG1kL1JJP2BPh391+yu5TscoI7ezC2BumBFlZSAX6A deJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1740019724; x=1740624524; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HzMxoSg2tOMNdSdKmknVvWlQeKQe3RfI+4wHzR7T3b4=; b=gyWygbI5Kjr0YnD08lGBYE49w7Jqofda7qmFRojjFRG0qfh5I8bMoPMfs5EopyLVPp WFp72mDfJj0zj9F48WZ1La2lWT0KvK+l46/iqnDHqEbp2JuT9mChQEb7KnYjBgZHXoSy BGTRGHPZMbutDAqWpDxlCk7X72LOZ1YHEpbwBCUwTrgpK7F7r/IMAlOu2l2c4qge/XAI UH3rKD/7uJaSubnV1b4MK3O2UfawwGy3DdMbDdz1GPhlKA5hhlHAFe/3Rn5Oirike4I4 W9tS+4UcUQobKAKx741muTB0LjmkY+eXDz7Y7VlgIJ5JV3yIIjBLxROxTp3xd/glrISL VJ9w== X-Gm-Message-State: AOJu0YxK7fCKm5rllLLBOtXmPmNYg4N0e29CsFqYbUr+BHSJS46N9qwr jSWpfD0NfQshCvvnyaTsalMEIYlnN95R62AcWdl2VdH4TAEzItntvRBz3iOmt9HPsbwtbML3Qxc r1WEMV3iS1ttR5UJ1PI05RJHB3uk= X-Gm-Gg: ASbGncshRtPbbKXW5/lQ6XekJtl08z7m24t2BiKs4PUFPYrBoTy01jeM8NPTyvwrnYi Q84KVMUUGMupXXPKZ5CxYABQLIt2jHrp6S+1TdseB71UZcWLFmTuDUK2Mp1aRhbAgxj6YsYMy X-Google-Smtp-Source: AGHT+IF3/3fgIH0kPsdjO0751ew9g49kVMTVwbqnmwVARgLHdLLEf65naAudyXxcqmXlKBL0A2SGzFY3OSjHOVv7Fdw= X-Received: by 2002:a2e:7d17:0:b0:307:db5f:c44f with SMTP id 38308e7fff4ca-30a44dd63bemr19360061fa.19.1740019724073; Wed, 19 Feb 2025 18:48:44 -0800 (PST) MIME-Version: 1.0 References: <20250214175709.76029-1-ryncsn@gmail.com> <20250214175709.76029-6-ryncsn@gmail.com> In-Reply-To: From: Kairui Song Date: Thu, 20 Feb 2025 10:48:27 +0800 X-Gm-Features: AWEUYZkH97aGhO_riBep9RjdBLFBO16QqiFUnpPMB-Nduf69I6IlMOZZDA4acNA Message-ID: Subject: Re: [PATCH 5/7] mm, swap: use percpu cluster as allocation fast path To: Baoquan He Cc: linux-mm@kvack.org, Andrew Morton , Chris Li , Barry Song , Hugh Dickins , Yosry Ahmed , "Huang, Ying" , Nhat Pham , Johannes Weiner , Kalesh Singh , linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Queue-Id: 5724720002 X-Rspamd-Server: rspam12 X-Stat-Signature: 9jkuet87sgjy6336nxxq8jtr7k8pnao5 X-HE-Tag: 1740019726-863927 X-HE-Meta: U2FsdGVkX1+x130kqaoIvalrcpdCAsStb2NeV50B8fsBFiZXPPCn3hIVWBb8ZW1jBJ7sWe6GikKdmFV7ZkRj9vPtef02LYquWCtN/g5hGeyQksS/yMgyDAuECDph3MLmfsBwFXJOGLyxA1gZF+3UL5gJgfP8Oy4FPMoEasZJeMfUvu7YmeCki+D+uU78WI6+G8Hm3ZBsbME0USEQ5uON5dwXlwHA4AlpG7xvcOb87OgHcIvKG0TTbdz2Sl0pZwHbGF1StXtcuonQVUNG1g7mbtMylgyYJFh73YeJoeewS82eT4zvotqGqKZqJW/jKhaAxb8ZSstlLs3f0gzGVa+DvFiMBq2ZyXziei85K+9e/0hq+WUixoMwT2U8KDNOBur4rgQMX0Tz2DqziP99IdYTJ4vV8/4+7Mr9biFgyqJ/RwQyMBiGqNE4jBlu0vNLy537UfPebnCkdoAevcIkkN4snQ1bz0wFHj0HjJvXb8u+xvOwwzkkeUJwg9kshXaB4hJjm+FrZLKqc2qTJOJ1j92fVYe7G28ULR6W4MtOwtIBqczXKTqZKP7OdrxUslCxmWFrmhps1NJl/uMGFnEYd/ze3bVy2jBgXacSKqi8DKx9LqEhkoM4N2/R/yhx3/pAbxW5IFm2h4rcZ6XD3RnGJFBSjRD5n4d0lZqcsMeDbEeaBRNhC76AkGD1U+aLd5RZPpnFfiB4ZnEiJjyn1XCNKHDEDVW6InDssAIKHdgrunXgwqX0oH/fe7CGxDRKSi6Kx5yL3lXrdg4NfXcqlumEzJSPaEE5V4K36tkgL98Mi0nX4b4uKvpkPpzfnA1DIlWvwuOPtdVT4ijQTp0DLk5Stl5V9WWwL+DCf3RpRsZYqKeN+qG7l0jL34jU2ly7+ivbazS30/UU3r1AdzqvxNrZZDX5ofwsgdDPipZjctd8mxe5UaP+oZhftwxlfTx5wU2bcr6avMqxQUHeKuJoAs5t1sp HKhVL/tD ECoy1TDngQdb42XIfEhWTQhimA+WZl9oJNWJKVQSEbie2BIao6DT5pECkzUgOAnkwRZ6bo953VSftUwbBHu61uL+AORnJoQIlVlEcovk/z0KELMJz0uyw1WpQwOMjeDH5SgJSm2SMMmgnv0VRSppiehJQvWEYm9Q1fp8bHgK2nSEtYrNYZ/jNWrD7jmHa0cNaQeY9WTxLF5We9DG97YpkmfyhJErIR+xpcFKk6CUwtafJJLl4CtgHNKimRh6v3s8r4TjC/6P/ne+lAsdTI16rdGDATPPe6CWU5SMME0szvGVtsGld6HY+90bU/wxWw1OJhjuPzuV0Z/XHtXJpp+HbIiloUw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.011905, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Feb 20, 2025 at 10:35=E2=80=AFAM Baoquan He wrote: > > On 02/19/25 at 07:12pm, Kairui Song wrote: > > > > > n reality it may be very difficult to achieve the 'each 2M space has = been consumed for each order', > > > > Very true, but notice for order >=3D 1, slot cache never worked before. > > And for order =3D=3D 0, it's very likely that a cluster will have more > > than 64 slots usable. The test result I posted should be a good > > example, and device is very full during the test, and performance is > > basically identical to before. My only concern was about the device > > My worry is the global percpu cluster may impact performance among > multiple swap devices. Before, per si percpu cluster will cache the > valid offset in one cluster for each order. For multiple swap devices, > this consumes a little bit more percpu memory. While the new global > percpu cluster could be updated to a different swap device easily only > of one order is available, then the whole array is invalid. That looks a > little drastic cmpared with before. Ah, now I got what you mean. That's seems could be a problem indeed. I think I can change the +struct percpu_swap_cluster { + struct swap_info_struct *si; to +struct percpu_swap_cluster { + struct swap_info_struct *si[SWAP_NR_ORDERS]; Or embed the swp type in the offset, this way each order won't affect each other. How do you think? Previously high order allocation will bypass slot cache so allocation could happen on different same priority devices too. So the behaviour that each order using different device should be acceptable. > > Yeah, the example you shown looks good. Wonder how many swap devices are > simulated in your example. > > > rotating, as slot cache never worked for order >=3D 1, so the device > > rotates was very frequently. But still seems no one really cared about > > it, mthp swapout is a new thing and the previous rotation rule seems > > even more confusing than this new idea. > > I never contact a real product environment with multiple tier and > many swap devices. In reality, with my shallow knowledge, usually only > one swap device is deployed. If that's true in most of time, the old > code or new code is fine, otherwise, seems we may need consider the > impact.