From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0ADDFC433FE for ; Tue, 14 Sep 2021 02:13:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9F97861151 for ; Tue, 14 Sep 2021 02:13:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 9F97861151 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 35CC9900002; Mon, 13 Sep 2021 22:13:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2E48F6B0072; Mon, 13 Sep 2021 22:13:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 15E50900002; Mon, 13 Sep 2021 22:13:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0045.hostedemail.com [216.40.44.45]) by kanga.kvack.org (Postfix) with ESMTP id 0211D6B0071 for ; Mon, 13 Sep 2021 22:13:31 -0400 (EDT) Received: from smtpin31.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 9B0252C5A0 for ; Tue, 14 Sep 2021 02:13:31 +0000 (UTC) X-FDA: 78584557422.31.DD5BE96 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by imf24.hostedemail.com (Postfix) with ESMTP id B338CB0000A1 for ; Tue, 14 Sep 2021 02:13:30 +0000 (UTC) X-IronPort-AV: E=McAfee;i="6200,9189,10106"; a="244178044" X-IronPort-AV: E=Sophos;i="5.85,291,1624345200"; d="scan'208";a="244178044" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Sep 2021 19:13:28 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.85,291,1624345200"; d="scan'208";a="543584748" Received: from shbuild999.sh.intel.com (HELO localhost) ([10.239.146.151]) by FMSMGA003.fm.intel.com with ESMTP; 13 Sep 2021 19:13:23 -0700 Date: Tue, 14 Sep 2021 10:13:22 +0800 From: Feng Tang To: Shakeel Butt Cc: Tejun Heo , Hillf Danton , LKML , Xing Zhengjun , Linux MM Subject: Re: [memcg] 45208c9105: aim7.jobs-per-min -14.0% regression Message-ID: <20210914021322.GE56674@shbuild999.sh.intel.com> References: <20210907033000.GA88160@shbuild999.sh.intel.com> <20210912111756.4158-1-hdanton@sina.com> <20210912132914.GA56674@shbuild999.sh.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-Stat-Signature: ripp9a5okucx9tucy9tfhz6kcafuq8ob Authentication-Results: imf24.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=intel.com (policy=none); spf=none (imf24.hostedemail.com: domain of feng.tang@intel.com has no SPF policy when checking 192.55.52.88) smtp.mailfrom=feng.tang@intel.com X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: B338CB0000A1 X-HE-Tag: 1631585610-462189 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000198, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi Shakeel, On Mon, Sep 13, 2021 at 01:13:57PM -0700, Shakeel Butt wrote: > On Mon, Sep 13, 2021 at 1:10 PM Tejun Heo wrote: > > > > On Mon, Sep 13, 2021 at 01:09:11PM -0700, Shakeel Butt wrote: > > > Thanks a lot for the explanation. Are there any concerns to call > > > cgroup_rstat_flush_irqsafe(root_mem_cgroup->css.cgroup) in system_w= q? > > > This will be called every 2 seconds, so, we can assume the updated > > > tree would be small most of the time. > > > > I can't think of a reason why this would be problematic. > > >=20 > Thanks again. >=20 > Feng, is it possible to re-run these benchmarks with > queue_work(system_wq) instead of queue_work(system_unbound_wq)? I just run the patch twice, and there was no obvious change, the hotspot is still the spinlock. Thanks, Feng $git-diff aa48e47e diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 4d8c9af..fa9cae9 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -683,7 +683,7 @@ void __mod_memcg_lruvec_state(struct lruvec *lruvec, = enum node_stat_item idx, /* Update lruvec */ __this_cpu_add(pn->lruvec_stats_percpu->state[idx], val); if (!(__this_cpu_inc_return(stats_flush_threshold) % MEMCG_CHARGE= _BATCH)) - queue_work(system_unbound_wq, &stats_flush_work); + queue_work(system_wq, &stats_flush_work); } 7e1c0d6f58207e7e aa48e47e3906c332eaf1e5d7b58 1638eee6432c1a5175685a7945a=20 ---------------- --------------------------- ---------------------------=20 \ | \ | \ =20 648.62 +243.0% 2224 =C2=B1 3% +246.1% 2244 = aim7.cpu 588139 -13.4% 509421 -13.5% 508738 = aim7.jobs-per-min 196.05 -13.4% 169.81 -13.5% 169.58 = aim7.jobs-per-min-per-task 3.93 =C2=B1 3% +62.8 66.70 +63.4 67.37 = pp.child.native_queued_spin_lock_slowpath 3.96 =C2=B1 4% +62.8 66.76 +63.5 67.46 = pp.child._raw_spin_lock_irqsave 3.66 =C2=B1 5% +62.9 66.54 +63.6 67.22 = pp.child.lock_page_lruvec_irqsave 0.00 +0.1 0.10 =C2=B1 5% +0.1 0.09 = pp.self.queue_work_on 0.00 +0.2 0.18 =C2=B1 5% +0.2 0.22 = pp.self.cgroup_rstat_flush_locked 0.00 +0.6 0.60 =C2=B1 7% +0.7 0.72 =C2= =B1 3% pp.self.mem_cgroup_css_rstat_flush 0.19 +0.7 0.86 =C2=B1 5% +0.7 0.84 =C2= =B1 4% pp.self.cgroup_rstat_updated 3.92 =C2=B1 3% +62.8 66.70 +63.4 67.37 = pp.self.native_queued_spin_lock_slowpath