From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2EA76C77B7A for ; Mon, 17 Apr 2023 18:33:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7DCCA8E0001; Mon, 17 Apr 2023 14:33:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 78C856B0072; Mon, 17 Apr 2023 14:33:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6543D8E0001; Mon, 17 Apr 2023 14:33:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 53AC86B0071 for ; Mon, 17 Apr 2023 14:33:04 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 2C610120550 for ; Mon, 17 Apr 2023 18:33:04 +0000 (UTC) X-FDA: 80691729888.07.D630D12 Received: from mail-ej1-f44.google.com (mail-ej1-f44.google.com [209.85.218.44]) by imf13.hostedemail.com (Postfix) with ESMTP id 2D1AC20007 for ; Mon, 17 Apr 2023 18:33:00 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=WnJy3uru; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf13.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.44 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681756381; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bzpHOK90fOGsax3+uctbvy5okdN/PImGhFs2yhf2bQ8=; b=EqAoFo+m95iRN9ZmQ+jEgsMnLkA3utT+Xuc+fUAna6dgnAFFJMPrgPcYGC5lJAjdyhVIKO HFrcc0AFXcDE7K0O5oMRpxuCpH6s32YVwcpMlyBKYo3/aIDYeEwMSDRVrEv89Xr2HePS+Q nGI0b9j05AfltcgCsiGeehlwMWoF9vY= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=WnJy3uru; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf13.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.44 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1681756381; a=rsa-sha256; cv=none; b=QG4PjS69zfanx309WTpFVgs11tTH6BKL8lXMbQysb+rwdnk799xUSCRh5sehC60MxI+oc2 uEwRZp6iVNv8Ib0ZhYbSAPr0Jsn7kDEKdJXQomKD/r2cX0Ia5kLV7VaSrPNu9FmC7n/NMP Sc+eq3G9pGA1x32i2DrGQdBPevVmyFk= Received: by mail-ej1-f44.google.com with SMTP id a640c23a62f3a-94a342f3ebcso607014366b.0 for ; Mon, 17 Apr 2023 11:33:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1681756379; x=1684348379; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=bzpHOK90fOGsax3+uctbvy5okdN/PImGhFs2yhf2bQ8=; b=WnJy3uruy1xNvNJQEv6CEwVBKBSKSxtsyLl7lWNpE2T6NotgnKmEPc1FHhuzOT67fg UM7fexdqCt5BKSz5vjQHGsGJaA9YnU4bust+HIB/TcfoIAWA+Yw1yRksm1jhP8TpBzES JPWjpwE7rrdvClsIhGYYsoZZFjG6Tpy1LFJngI5obMA7yrmIOd+9t4Ny4NWr8uSImZuh ClFfHRe/oPE4CaktDzkaBFxn0T2WosNKnUh4fm6w5l6zv3vhlzt7V/xCcaqBt6/JPsiu LAG4ZLyjXvO7eo8dXkgaei3mnOqLv1ZY/C0PuLJY3DFLvibiCWZXL0dPHHkJyqwgUwag 0wHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681756379; x=1684348379; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bzpHOK90fOGsax3+uctbvy5okdN/PImGhFs2yhf2bQ8=; b=PUb5XktlvqS4VhL34msSpeVm19zJDvHFKs1PWaiSLrtoFHu0Bzix+xCk9x7BJedToL dkYAE3rq7cqeq6f+MQII3psmgRYoYcdluBohup7I8U7NZakVb68XMXXKGRH8CcklqVNr 3IxBuUueuTVvRkDQyyjC3mvOU4ng+r4uKfKmCrqaRpNlego5fQpjla4Tm8fkHZ1tbnep P4g7j4PI6G0wb/FW2kBSrnnmFhbfHaWdqjO5ocx6uPB6NXZmZvF2/po9Pi9TNzqt8Q1d WlQpvlXBvtfbqfZba2Za54B7b2tEy1oABvj6pogQ57GWu/FLM15BBZw/TWCpDRtg6Pkv 1A7g== X-Gm-Message-State: AAQBX9eq6uCL2YTxxXFslCSX8wAA5nLaCrnLLNuG68atvFrAQ4Lq5KF/ gaKmZh81SZcdJG32GS+3ojyQ1l7mXEBVhRJCFgN8rw== X-Google-Smtp-Source: AKy350Y4Y151kI2stQPw0OPxfnrCNgp/vXvL7SOXx5UOeFUsRfMXKZPL48wlBo92r+gGNeZdNlbg9Huh4E+PVBi+34c= X-Received: by 2002:a50:9319:0:b0:502:1d1c:7d37 with SMTP id m25-20020a509319000000b005021d1c7d37mr7725076eda.8.1681756379333; Mon, 17 Apr 2023 11:32:59 -0700 (PDT) MIME-Version: 1.0 References: <20230417135420.1836741-1-senozhatsky@chromium.org> In-Reply-To: <20230417135420.1836741-1-senozhatsky@chromium.org> From: Yosry Ahmed Date: Mon, 17 Apr 2023 11:32:22 -0700 Message-ID: Subject: Re: [PATCHv2] zsmalloc: allow only one active pool compaction context To: Sergey Senozhatsky Cc: Minchan Kim , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 2D1AC20007 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: cs8tuzjutwg3tkjab5qikju7iw9bgzxx X-HE-Tag: 1681756380-330606 X-HE-Meta: U2FsdGVkX18w/BTMwIJhGxzJ3ih7KByC9K7ekItBOW5+6/FD/jqgVcXXcmeneB9hnF7eP0c8VenILjYufAE93Fn5NGVKHBC6l1CUE8FsHL9NekeY3InMpLCyfhGW7V1LtONgG+KczB2DRvj/+lTbt6hCJ5GsqXxZUikEDWv0/9sEgRkh96StHrqppKoHJXnnlhnR5WpOQVTPHQSJPNLGrK0nVotoWmlssrPru6sVCzydOCvYjPDhbq8NVI6KuI6AXcHPb3D7BtHgUNhhLN+bAkzTD4DRN4ujy3mw7TPvppzOb/2kvrsyGiRFKfQ2syYfVyM/eMgW9Vzyc59Aw42V9KeZRYAf5x296p3MEjhWinzkwz5pN/dSSFA9tZEBkXJkjrB4+vBZI6OM0t3SxAOiMuzjRdD+KX9uid9n6DtWQ/agkc+ZLe+ofoCWE594sGjxW/Roc3FLB3ir0COxrHmOMVoD2Ctip0By56amTID6UApgNcFfwtU+KGCJZErYmG+E/N+33NHD5/zrr5w2DT/T2111ysEPHSqe9uUEmikw+qKSOhcb62O9AGzZXPyF1HiA05PZPdWSvkd+hEQlnYujMq4FK/MJ0daZ2ZnRwmTFHMp6LZ7k0n4Wi2E+pIXKMpVU/FIIidwkyoBPxpRBz8I3nS+U4uRED2a058QsetOYGAUVeXM6JXRSnDp55f6mB7aIYm1p7ubBTjciD5WVKdfcIFzOt3raS22xpulDb2fufydTb6SyHMkLFW/tYcBqQTEawIY8EiIRQxixhnJbU44tgvp5ON8ZkIlVlZhC0KhCmUUmiVFObvNLJW0n0GdU3hCsPACVgpgaiJD8UQVTfW+RriJ1G3FZOHfLkvFB5ED1gL83JOXCwDrfxz7gzOqA6k2vlnIGotMHDoBuHT5JW+qej+cVZ9SvfPeSsL3y7/qZe2FdRaEuuMSrKRzK5g5hAnUNp3005MwhT9GsmJuvDJx DhWjN8hu C/WXk5rQAt3rantqWmsz8XO9XnWNWHG/+sg39+dpAcRtaLUYuKZ0tJoopgHQt3HEoX3E93xMupcpGoRuOcs0LtuIdWeI3K0bxgoCx81ufEHy4fFO5F5Asjm9D8lY8ZqfcGP4QdAH8K96M9TAokLzjsss3Rc4U06x9lcISJywt+U1IRJqOh46RWFRWcFzyBuNCMBMjpuOCCal/RhGGj9RICldPnJULr60XucK9BY8d5GnAgnOkEdTS7E61hJlP79jlW2i5SfvAdCmmC4wXtw6BDvvJzAUR5iVY+eIr3iHAf8lGDz6+cMQQgxlUqAhTFwnEjsxg X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Apr 17, 2023 at 6:54=E2=80=AFAM Sergey Senozhatsky wrote: > > zsmalloc pool can be compacted concurrently by many contexts, > e.g. > > cc1 handle_mm_fault() > do_anonymous_page() > __alloc_pages_slowpath() > try_to_free_pages() > do_try_to_free_pages( > lru_gen_shrink_node() > shrink_slab() > do_shrink_slab() > zs_shrinker_scan() > zs_compact() > > This creates unnecessary contention as all those processes > compete for access to the same classes. A single compaction > process is enough. Moreover contention that is created by > multiple compaction processes impact other zsmalloc functions, > e.g. zs_malloc(), since zsmalloc uses "global" pool->lock to > synchronize access to pool. > > Introduce pool compaction mutex and permit only one compaction > context at a time. This reduces overall pool->lock contention. > > /proc/lock-stat after make -j$((`nproc`+1)) linux kernel for > &pool->lock#3: > > Base Patched > ------------------------------------------ > con-bounces 2035730 1540066 > contentions 2343871 1774348 > waittime-min 0.10 0.10 > waittime-max 4004216.24 2745.22 > waittime-total 101334168.29 67865414.91 > waittime-avg 43.23 38.25 > acq-bounces 2895765 2186745 > acquisitions 6247686 5136943 > holdtime-min 0.07 0.07 > holdtime-max 2605507.97 482439.16 > holdtime-total 9998599.59 5107151.01 > holdtime-avg 1.60 0.99 The numbers seem to be better when using an atomic vs. a mutex, is this just noise or significant difference? (I am not familiar with lock-stat). > > Test run time: > Base > 2775.15user 1709.13system 2:13.82elapsed 3350%CPU > > Patched > 2608.25user 1439.03system 2:03.63elapsed 3273%CPU > > Signed-off-by: Sergey Senozhatsky FWIW, Reviewed-by: Yosry Ahmed > --- > mm/zsmalloc.c | 6 ++++++ > 1 file changed, 6 insertions(+) > > diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c > index cc81dfba05a0..dfec2fc6a30f 100644 > --- a/mm/zsmalloc.c > +++ b/mm/zsmalloc.c > @@ -264,6 +264,7 @@ struct zs_pool { > struct work_struct free_work; > #endif > spinlock_t lock; > + atomic_t compaction_in_progress; > }; > > struct zspage { > @@ -2274,6 +2275,9 @@ unsigned long zs_compact(struct zs_pool *pool) > struct size_class *class; > unsigned long pages_freed =3D 0; > > + if (atomic_xchg(&pool->compaction_in_progress, 1)) > + return 0; > + > for (i =3D ZS_SIZE_CLASSES - 1; i >=3D 0; i--) { > class =3D pool->size_class[i]; > if (class->index !=3D i) > @@ -2281,6 +2285,7 @@ unsigned long zs_compact(struct zs_pool *pool) > pages_freed +=3D __zs_compact(pool, class); > } > atomic_long_add(pages_freed, &pool->stats.pages_compacted); > + atomic_set(&pool->compaction_in_progress, 0); > > return pages_freed; > } > @@ -2388,6 +2393,7 @@ struct zs_pool *zs_create_pool(const char *name) > > init_deferred_free(pool); > spin_lock_init(&pool->lock); > + atomic_set(&pool->compaction_in_progress, 0); > > pool->name =3D kstrdup(name, GFP_KERNEL); > if (!pool->name) > -- > 2.40.0.634.g4ca3ef3211-goog >