From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91B3CC54EE9 for ; Wed, 7 Sep 2022 21:34:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 010C28D0002; Wed, 7 Sep 2022 17:34:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ED97B6B0073; Wed, 7 Sep 2022 17:34:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA19F8D0002; Wed, 7 Sep 2022 17:34:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 88DA66B0072 for ; Wed, 7 Sep 2022 17:34:30 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 645DAA08A0 for ; Wed, 7 Sep 2022 21:34:30 +0000 (UTC) X-FDA: 79886593500.19.3A7E543 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf01.hostedemail.com (Postfix) with ESMTP id EDC8D40073 for ; Wed, 7 Sep 2022 21:34:29 +0000 (UTC) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D89C661AA5; Wed, 7 Sep 2022 21:34:28 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B7E25C433D6; Wed, 7 Sep 2022 21:34:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1662586468; bh=UgoD/NLQ0MqihZ87VFspXN0wI9uOU/Esj7gHe+uAoqo=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=lWhoM2fwKkQ0+S3jLkEBCouMBZXU7I4Tg04RzM5+IrYRYOobXKSOVdFVr4tcHdWjO EVP9+WRSEW+9l1GV5Qoh9gWaWoXbY3LYcfo1RxrZrWB2jqQx06huNTu0MnL/0ynzEG zKHib011BQJcyBAO1tyDumbf/8nTVs+l35cI4WvM= Date: Wed, 7 Sep 2022 14:34:27 -0700 From: Andrew Morton To: Tim Chen Cc: Jiebin Sun , vasily.averin@linux.dev, shakeelb@google.com, dennis@kernel.org, tj@kernel.org, cl@linux.com, ebiederm@xmission.com, legion@kernel.org, manfred@colorfullife.com, alexander.mikhalitsyn@virtuozzo.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, tim.c.chen@intel.com, feng.tang@intel.com, ying.huang@intel.com, tianyou.li@intel.com, wangyang.guo@intel.com Subject: Re: [PATCH v4] ipc/msg: mitigate the lock contention with percpu counter Message-Id: <20220907143427.0ce54bbf096943ffca197fee@linux-foundation.org> In-Reply-To: References: <20220907172516.1210842-1-jiebin.sun@intel.com> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1662586470; a=rsa-sha256; cv=none; b=hrMB5WLNme7ROCTNWZIIGZ80DcPxVqwkrif/UvbOh4ueg+Zg/t7uEVY8LnRXggjXpTrwen 2T+x5hmV11DqfK3xBWhTGGkIzUH8cZLhv3vNRpOLgmRMaaKafDZlZr64qV6Ond/JaMiCGs uduR0n2ieR+jGpEH5f/wFbDYOGEhkvM= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=lWhoM2fw; spf=pass (imf01.hostedemail.com: domain of akpm@linux-foundation.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1662586470; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=27K80smNAv5aUF9gk8jWKD6E9xQPMeNUJzT3V7IQsCw=; b=642cHRS5zNLBnfzkTynwcPNnJZIeooXGvyEVafzizz3CR9KmNnmHvxTbAarrTfVz5ducRz m4lHI/F3aOyR1R9koLzAup5P18FvOtXIbCJgvjSrCv0h62G1ztHtxJZTT51bIGGf9Fv4Vy 9HJfD9e3Fq7xaqG4Sl7zUXZnfWIOjeo= X-Rspam-User: Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b=lWhoM2fw; spf=pass (imf01.hostedemail.com: domain of akpm@linux-foundation.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none X-Rspamd-Server: rspam04 X-Stat-Signature: jkzxfon7hpjffcsfnnxfws4khjkktxoc X-Rspamd-Queue-Id: EDC8D40073 X-HE-Tag: 1662586469-327611 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, 07 Sep 2022 09:01:53 -0700 Tim Chen wrote: > On Thu, 2022-09-08 at 01:25 +0800, Jiebin Sun wrote: > > The msg_bytes and msg_hdrs atomic counters are frequently > > updated when IPC msg queue is in heavy use, causing heavy > > cache bounce and overhead. Change them to percpu_counter > > greatly improve the performance. Since there is one percpu > > struct per namespace, additional memory cost is minimal. > > Reading of the count done in msgctl call, which is infrequent. > > So the need to sum up the counts in each CPU is infrequent. > > > > > > Apply the patch and test the pts/stress-ng-1.4.0 > > -- system v message passing (160 threads). > > > > Score gain: 3.17x > > > > > ... > > > > +/* large batch size could reduce the times to sum up percpu counter */ > > +#define MSG_PERCPU_COUNTER_BATCH 1024 > > + > > Jiebin, > > 1024 is a small size (1/4 page). > The local per cpu counter could overflow to the gloabal count quickly > if it is limited to this size, since our count tracks msg size. > > I'll suggest something larger, say 8*1024*1024, about > 8MB to accommodate about 2 large page worth of data. Maybe that > will further improve throughput on stress-ng by reducing contention > on adding to the global count. > I think this concept of a percpu_counter_add() which is massively biased to the write side and with very rare reading is a legitimate use-case. Perhaps it should become an addition to the formal interface. Something like /* * comment goes here */ static inline void percpu_counter_add_local(struct percpu_counter *fbc, s64 amount) { percpu_counter_add_batch(fbc, amount, INT_MAX); } and percpu_counter_sub_local(), I guess. The only instance I can see is block/blk-cgroup-rwstat.h:blkg_rwstat_add() which is using INT_MAX/2 because it always uses percpu_counter_sum_positive() on the read side. But that makes two!