From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AEF08E69E86 for ; Mon, 2 Dec 2024 19:38:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3D4B46B0089; Mon, 2 Dec 2024 14:38:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 384846B008A; Mon, 2 Dec 2024 14:38:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 24BDC6B008C; Mon, 2 Dec 2024 14:38:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 072C16B0089 for ; Mon, 2 Dec 2024 14:38:13 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 86B251A02AC for ; Mon, 2 Dec 2024 19:38:12 +0000 (UTC) X-FDA: 82851029772.18.BA6D335 Received: from mail-qt1-f178.google.com (mail-qt1-f178.google.com [209.85.160.178]) by imf22.hostedemail.com (Postfix) with ESMTP id 3EC6BC0017 for ; Mon, 2 Dec 2024 19:37:57 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=CiF4en1L; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf22.hostedemail.com: domain of yosryahmed@google.com designates 209.85.160.178 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733168285; a=rsa-sha256; cv=none; b=a1mU+Ukl3FlPcw/pvwXBZ4YMkP/04LcCGLBZQHXVGGmknGsv/cst9UF9e3cZWPUa1pJxot f51ESMvAPcNuJYisRBq+QEjEmRlwPehwwH+0+/6IAVPCPhmKietFBHtcgr/J4g0w+8dhEu MDJUwQ+EnhCONfWHHMoq9mV5L4GMf0I= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=CiF4en1L; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf22.hostedemail.com: domain of yosryahmed@google.com designates 209.85.160.178 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733168285; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IxzZa/FeaHenizjIU+OA48psYXufFL/hW5pMalhLMKg=; b=afbMuBG+bf72XkSBZOJmgKIzu9JTHXYqGzr8kaDdKXS4TorzvylpzJzgbyb02/0lW1SnQ0 n5VQc/GUnomZHNt64E3BWGmvognvsblUj7b/locp7ai7BGMtwOn9wyM1Rr5Y8tyAIKnbU0 TwI0HB9IhADcimd8VAIyfhGyfVVJZ+E= Received: by mail-qt1-f178.google.com with SMTP id d75a77b69052e-46677ef6910so30065351cf.2 for ; Mon, 02 Dec 2024 11:38:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1733168290; x=1733773090; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=IxzZa/FeaHenizjIU+OA48psYXufFL/hW5pMalhLMKg=; b=CiF4en1LvCgtpD5qKEg8K3gBVXo4xE70NUCtf1Z8XbhmyenuxiA+avkqBVxv3bcjne eWMopbQFq521+wQLjGRLAWHb6MlJ9aKvW1MZvfM+BC+WGOEEU/vfLiMBATtwWRBSjdEU igPqi55HPT0Q2aheUVNl9yoJgScuoT6ZxbwIQajHWxlXyhH/MEBKsC859fu/g4vHv08Y X7uECmoyqJTw4PvPjc06fasyYfU5RUNNhWPW4DEj2JMAGK3h91WLjq+f0aJ8BUklFZlv U1kvZx4AGRcoT0Vvdm9OrWZEGfHNpjv745l0j68t+0WmaLc9vxZ31tfW3WgmSQG0LrXS x7tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733168290; x=1733773090; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IxzZa/FeaHenizjIU+OA48psYXufFL/hW5pMalhLMKg=; b=FH/R/JfgaeVMsCAGQqlXYu+B7KoYMEAiy/eWoubmvDO0L/ZFHJbk6jV+eFrB7vKedT cbyR4K6mHjWRYA4SySysP71z3J/OS4dBcrNkU9atgOXK5VOpg6yDC2wIbLm4W1Yt9nQK fUugAYF23e5UzDGvg+2nm0SLrrNKMQ0oYmRWF4pc7Q5pkHcxKWfQlkD2foKJAiq/Tz3h hzIvDz3iDlmaUux+lPSmXRhwlKgpuRUI2ktf2HYbSO44Vv/SkDXr3mxNnBs7XQUxMyR3 09SJhNZBMLmZE9HasK304mKBrC4pd9vRsq+ic2XzJaTdgSfEt7mSRNdBhTJR1gtUamyi es2Q== X-Gm-Message-State: AOJu0YxnZPvnlgEPpatH4MJBt3FZTdORpJR2KOlLqOSYGDHpO6unV76P 02rKHTHeC6xE/tt8JXYwQkZWlmhdNz548o+e4JAR2t0JSHxL/4dxyQz0tzAf/FEVxWFkBDYDXOK qjuhUA5rFYWrp7L1Aodd9QhgROVsXcMEWr9Lz X-Gm-Gg: ASbGncvSWsNeL3uoMngvfmQenJtf5XlOwK4J/e3tgxindnu9xFt4I8JPz1l3qVKViw4 VJ4Ojbi9SDOAKX1jabbN0N7zNTU4X X-Google-Smtp-Source: AGHT+IHxJdu5xIM6BIgrsiHPcVu7o/uvaOAXgt3dUjkQOdUQsKeM/9i7bc65ZyYHF2Tld/jcYUd4DpRB8Av3RKj6IhU= X-Received: by 2002:a05:6214:c6a:b0:6d8:b371:6a0f with SMTP id 6a1803df08f44-6d8b3716e20mr18594046d6.31.1733168289587; Mon, 02 Dec 2024 11:38:09 -0800 (PST) MIME-Version: 1.0 References: <20241202184154.19321-1-ryncsn@gmail.com> <20241202184154.19321-5-ryncsn@gmail.com> In-Reply-To: <20241202184154.19321-5-ryncsn@gmail.com> From: Yosry Ahmed Date: Mon, 2 Dec 2024 11:37:32 -0800 Message-ID: Subject: Re: [PATCH 4/4] mm, swap_cgroup: remove global swap cgroup lock To: Kairui Song Cc: linux-mm@kvack.org, Andrew Morton , Chris Li , Hugh Dickins , Roman Gushchin , Shakeel Butt , Johannes Weiner , Barry Song , Michal Hocko , linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 3EC6BC0017 X-Stat-Signature: gnmbbe639athbchkfb7rpm64w3n9a8t5 X-HE-Tag: 1733168277-140415 X-HE-Meta: U2FsdGVkX19/chAYIj+jcRpa4sEWw36RwK97x1/mjyeeAkYpGYLSJCG7KNNWGGwXqvO4c7e+yudtSsRcBWx0BPcDv8IpH68AmiRjhuVja29c26ipQC0axfopNbmygdRJqVdTI+vz5gytf31fyEOoH0CTsNolEQaEE9Hh4B9uuJbaCjesic41IQxbwsdAxBnyAPUsFAesVaEktEAk2Bh0u7VPwJfQHBl9+MRsy4cBXzgmzjUUewgTL0BdLvWPjcM1bOBqNaUhKEhZxx7MY+ihq3VJDeUfM1dWIVnXoXbsTt1Sz/L6TjipoOAlqLyuNRvnGEv9K+ro11IchVNDf3M/wHC56rmNqar50opY3IjIhsbJZHpP3CllNaYEpqiRtAwc1H5VPUjFL2L56XTZm2s+/xbxeBZTRZVl/qB79Z6v3diINbx4+W8Q9j60vCWKFR47dK5vCgwEEmsejjNu89BtSep2W5uus31ZBCbgWYbRDvSG2HSTP343adGAqZOk9tmWjNWtxS669V8R4FLBluvDeSLVl/BNpeE9ToHATxAI0B4iFwWK0VaVmQGDNIu4M5+63SCph9zup9JTu72eLsZ1DehuBXAponPC5vSoXfhMlqnjRPTIAYswGbS63ioJkU+nA8Rb+LOfyseupYBUfVlxgAfxY9v+k2XWP6vSrQCjaKlhYB0QXlxJbW8DannPOMi0Ls9yk2Oa8py0DKfYTlHoco/tmmJzzX0hmzs6rcuJEmarL1rvBicYLpEqAlXDcjnY7DPTkOmol7H+7ABBBC8hP/qkw6yie1wFr4kJZQ3/osX8/xOffIGAlPnFTh85tMF4kBLcs31NNMarTTqspDgJAaEWECNWo/qPijYpYG85h8Z2m8Vc3XEwoEgkFMFbjx6tSrmZ3TpBGX6sEHbOowIqEKUuCOllEcVXk/1K9apMamDllU+XfLb5Q1Zfp37nrFQvcepjUBkIPEddY9lZDle EKHNFj7D RwFifTUlhWmKy/aRZGdsZfZLjJVaTlnHa7TugFwsB6wobPX2RWjtuOcnmpT7P6ho5Fj0e6kO+KxG/60vFR4MiHHpyod2FAumBgqlc4POmNaFAFJd/Kk3VZD4NeCtxRL1ckhiX7YMys3daZpPj+AYS91sFKX/3EZyJZdPz5kml//u/A6At9eCanVXwVsSSK+r0gFY/XeV2/WiewmdRgPGjFyJeZZBKm5u5KIEN455tl/l/x3aH7jXouMcdLh3t+sxFz0+O+U8blgIwtpMphnIgDfs5ZeYeC0hO6FyE X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Dec 2, 2024 at 10:42=E2=80=AFAM Kairui Song wrot= e: > > From: Kairui Song > > commit e9e58a4ec3b1 ("memcg: avoid use cmpxchg in swap cgroup maintainanc= e") > replaced the cmpxchg/xchg with a global irq spinlock because some archs > doesn't support 2 bytes cmpxchg/xchg. Clearly this won't scale well. > > And as commented in swap_cgroup.c, this lock is not needed for map > synchronization. > > Emulation of 2 bytes cmpxchg/xchg with atomic isn't hard, so implement > it to get rid of this lock. > > Testing using 64G brd and build with build kernel with make -j96 in 1.5G > memory cgroup using 4k folios showed below improvement (10 test run): > > Before this series: > Sys time: 10730.08 (stdev 49.030728) > Real time: 171.03 (stdev 0.850355) > > After this commit: > Sys time: 9612.24 (stdev 66.310789), -10.42% > Real time: 159.78 (stdev 0.577193), -6.57% > > With 64k folios and 2G memcg: > Before this series: > Sys time: 7626.77 (stdev 43.545517) > Real time: 136.22 (stdev 1.265544) > > After this commit: > Sys time: 6936.03 (stdev 39.996280), -9.06% > Real time: 129.65 (stdev 0.880039), -4.82% > > Sequential swapout of 8G 4k zero folios (24 test run): > Before this series: > 5461409.12 us (stdev 183957.827084) > > After this commit: > 5420447.26 us (stdev 196419.240317) > > Sequential swapin of 8G 4k zero folios (24 test run): > Before this series: > 19736958.916667 us (stdev 189027.246676) > > After this commit: > 19662182.629630 us (stdev 172717.640614) > > Performance is better or at least not worse for all tests above. > > Signed-off-by: Kairui Song > --- > mm/swap_cgroup.c | 56 +++++++++++++++++++++++++++++++++++------------- > 1 file changed, 41 insertions(+), 15 deletions(-) > > diff --git a/mm/swap_cgroup.c b/mm/swap_cgroup.c > index a76afdc3666a..028f5e6be3f0 100644 > --- a/mm/swap_cgroup.c > +++ b/mm/swap_cgroup.c > @@ -5,6 +5,15 @@ > > #include /* depends on mm.h include */ > > +#define ID_PER_UNIT (sizeof(atomic_t) / sizeof(unsigned short)) > +struct swap_cgroup_unit { > + union { > + int raw; > + atomic_t val; > + unsigned short __id[ID_PER_UNIT]; > + }; > +}; > + > static DEFINE_MUTEX(swap_cgroup_mutex); > > struct swap_cgroup { > @@ -12,8 +21,10 @@ struct swap_cgroup { > }; > > struct swap_cgroup_ctrl { > - unsigned short *map; > - spinlock_t lock; > + union { > + struct swap_cgroup_unit *units; > + unsigned short *map; > + }; > }; > > static struct swap_cgroup_ctrl swap_cgroup_ctrl[MAX_SWAPFILES]; > @@ -31,6 +42,24 @@ static struct swap_cgroup_ctrl swap_cgroup_ctrl[MAX_SW= APFILES]; > * > * TODO: we can push these buffers out to HIGHMEM. > */ While you're at it, I think the comment above is quite outdated :)