From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.1 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5614C12002 for ; Fri, 16 Jul 2021 15:03:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5CCC16128C for ; Fri, 16 Jul 2021 15:03:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5CCC16128C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=samsung.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7D11A8D00F4; Fri, 16 Jul 2021 11:03:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7A92C8D00EC; Fri, 16 Jul 2021 11:03:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5D3AF8D00F4; Fri, 16 Jul 2021 11:03:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0251.hostedemail.com [216.40.44.251]) by kanga.kvack.org (Postfix) with ESMTP id 3AC7D8D00EC for ; Fri, 16 Jul 2021 11:03:59 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 04E58181CC1C1 for ; Fri, 16 Jul 2021 15:03:58 +0000 (UTC) X-FDA: 78368770956.26.C70C913 Received: from mailout1.w1.samsung.com (mailout1.w1.samsung.com [210.118.77.11]) by imf03.hostedemail.com (Postfix) with ESMTP id 14C0630001A8 for ; Fri, 16 Jul 2021 15:03:56 +0000 (UTC) Received: from eucas1p1.samsung.com (unknown [182.198.249.206]) by mailout1.w1.samsung.com (KnoxPortal) with ESMTP id 20210716150354euoutp01900068429f1408ad8eb84f8d5e911113~STbBG9aMo0443204432euoutp01Z for ; Fri, 16 Jul 2021 15:03:54 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout1.w1.samsung.com 20210716150354euoutp01900068429f1408ad8eb84f8d5e911113~STbBG9aMo0443204432euoutp01Z DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1626447834; bh=LOCs/2mBF1xQdHZWGjG7nJ493Pn7vzTndVuGZjYhqEE=; h=Subject:To:Cc:From:Date:In-Reply-To:References:From; b=mc5j3zWYYjis0C49ZdMoEo8FM51UtDZ3HBop1C2LaiZ2xbrpO1ZpFgdOOxvtKZFDJ p6pPbz5DZ98fiVoxIOjwGfxbwUS3nZ2JYfCvAqNhYerYjKVr7TqUClauL9J38WIHQK byV+/10nyolNh9Wg765pB7uAb+p0idP4Vpj10RAA= Received: from eusmges1new.samsung.com (unknown [203.254.199.242]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20210716150354eucas1p143ecd48e7f567ff2d5a5678496f96620~STbAymKlX1352413524eucas1p1a; Fri, 16 Jul 2021 15:03:54 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges1new.samsung.com (EUCPMTA) with SMTP id 7E.B1.45756.ADF91F06; Fri, 16 Jul 2021 16:03:54 +0100 (BST) Received: from eusmtrp2.samsung.com (unknown [182.198.249.139]) by eucas1p2.samsung.com (KnoxPortal) with ESMTPA id 20210716150353eucas1p2c9ad1d1021ee584de587e5ec10b8467b~STbAKCjuL2293522935eucas1p2y; Fri, 16 Jul 2021 15:03:53 +0000 (GMT) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eusmtrp2.samsung.com (KnoxPortal) with ESMTP id 20210716150353eusmtrp22df49664c581a0d500ea80ea68f80bdb~STbAFExxt0241302413eusmtrp24; Fri, 16 Jul 2021 15:03:53 +0000 (GMT) X-AuditID: cbfec7f2-7d5ff7000002b2bc-f1-60f19fda5773 Received: from eusmtip1.samsung.com ( [203.254.199.221]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id 6A.CE.20981.9DF91F06; Fri, 16 Jul 2021 16:03:53 +0100 (BST) Received: from [106.210.134.192] (unknown [106.210.134.192]) by eusmtip1.samsung.com (KnoxPortal) with ESMTPA id 20210716150352eusmtip1bf0249c52e5b8bed50796605b10cead5~STa-Qn6uW0172601726eusmtip1g; Fri, 16 Jul 2021 15:03:52 +0000 (GMT) Subject: Re: [PATCH v4 2/2] memcg: infrastructure to flush memcg stats To: Shakeel Butt , Tejun Heo , Johannes Weiner , Muchun Song Cc: Michal Hocko , Roman Gushchin , =?UTF-8?Q?Michal_Koutn=c3=bd?= , Huang Ying , Hillf Danton , Andrew Morton , cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org From: Marek Szyprowski Message-ID: <78005c4c-9233-7bc8-d50e-e3fe11f30b5d@samsung.com> Date: Fri, 16 Jul 2021 17:03:53 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.12.0 MIME-Version: 1.0 In-Reply-To: <20210714013948.270662-2-shakeelb@google.com> Content-Language: en-US X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrBKsWRmVeSWpSXmKPExsWy7djP87q35n9MMDi9S8Fizvo1bBY3ls9g sfg66QKbxepNvhYHfj5nsbi8aw6bxb01/1ktXn9bxmwxd9Z+FouD114zWhy8f47Z4tfyo4wW J2dNZnHg9fh3Yg2bx+E375k9Jja/Y/dYsKnUY/Gel0wem1Z1snls+jSJ3ePEjN8sHpNeuHus 33KVxePzJrkA7igum5TUnMyy1CJ9uwSujJvNE9kKXlhWNG7ZytrAOE+/i5GTQ0LARKL1933W LkYuDiGBFYwSd2b9g3K+MEq0zFjEBFIlJPCZUeL5BEOYjt9XXrJDFC1nlLg/eQ4LhPORUWJ5 018WkCphATeJg1e2giVEBCYySlz7vQKshVlgO5PEvW9vWEGq2AQMJbredrGB2LwCdhJr3zWw g9gsAqoSj3avALNFBZIl7px+D1UjKHFy5hOwDZwCVhIt55Ywg9jMAvISzVtnQ9niEreezGcC WSYhsJ1T4siWB0DNHECOi8Tk5VIQPwhLvDq+hR3ClpH4vxOmvplR4uG5tewQTg+jxOWmGYwQ VdYSd879AhvELKApsX4XNPgcJe7fP88EMZ9P4sZbQYgb+CQmbZvODBHmlehoE4KoVpOYdXwd 3NqDFy4xT2BUmoXks1lIvpmF5JtZCHsXMLKsYhRPLS3OTU8tNsxLLdcrTswtLs1L10vOz93E CEx8p/8d/7SDce6rj3qHGJk4GA8xSnAwK4nwfqv5mCDEm5JYWZValB9fVJqTWnyIUZqDRUmc d9XsNfFCAumJJanZqakFqUUwWSYOTqkGpu6TDO3fA+I3GLzNZP8wVbSTLT++9J1M08uCagFj jUXq0tnafNUtHc+K7flsdIQOth3QkrOre3Ry27Leh49OLxBddWKBwPQ/H9q9tOT1MhanbIzw ELoorhB1nnfBrsOCCc0FTb0+R1dfyvvAYu9wWujB/RWvGnyWr2U1K4h0VOVMZKthnrAgpVB8 fdqp2Y3795dfPqHseJ/R3PH4vThFu0+LRXa871wS+1AuNSiq5UL8rrpp1kv+716b+Sf9goXh BAUDfinJgCd3fpVlOW/kPC3ySSzMSP/FRL2tKwwuTfuzcbeuXybD/8/rzm3ZInHg1brKh+u2 bRAReG3o1rLT3dBc+ebLF/EuNx8UTvbZrcRSnJFoqMVcVJwIAEkgUATrAwAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrNIsWRmVeSWpSXmKPExsVy+t/xu7o3539MMNh/WMdizvo1bBY3ls9g sfg66QKbxepNvhYHfj5nsbi8aw6bxb01/1ktXn9bxmwxd9Z+FouD114zWhy8f47Z4tfyo4wW J2dNZnHg9fh3Yg2bx+E375k9Jja/Y/dYsKnUY/Gel0wem1Z1snls+jSJ3ePEjN8sHpNeuHus 33KVxePzJrkA7ig9m6L80pJUhYz84hJbpWhDCyM9Q0sLPSMTSz1DY/NYKyNTJX07m5TUnMyy 1CJ9uwS9jJvNE9kKXlhWNG7ZytrAOE+/i5GTQ0LAROL3lZfsXYxcHEICSxklrv95zwKRkJE4 Oa2BFcIWlvhzrYsNoug9o8S+vn+MIAlhATeJg1e2soAkRAQmMkrs6twGVsUssJ1J4v+x+0wg VUICmRK31vSyg9hsAoYSXW9BRnFy8ArYSax91wAWZxFQlXi0ewWYLSqQLNH3ZQIjRI2gxMmZ T8BO4hSwkmg5t4QZxGYWMJOYt/khlC0v0bx1NpQtLnHryXymCYxCs5C0z0LSMgtJyywkLQsY WVYxiqSWFuem5xYb6RUn5haX5qXrJefnbmIExvq2Yz+37GBc+eqj3iFGJg7GQ4wSHMxKIrzf aj4mCPGmJFZWpRblxxeV5qQWH2I0BfpnIrOUaHI+MNnklcQbmhmYGpqYWRqYWpoZK4nzmhxZ Ey8kkJ5YkpqdmlqQWgTTx8TBKdXA5B8snvbvqpX0HPWbe/Q7Yj5w/khavTtfPnDhTc/kD6ca OT+HuTZ3yy/lvtrEdaTPVyB/e1K5QdXkm/dkj52x0Vj4bFnzIm1Le+kr80qtz8aXfvb4eNJH lelu77SZ6v9XRFTZfmsU3DNz9s5TDxbUCBsvEAnZqb57r+8Si+ttaTYiv4NETUN9At/cmDPh Ysjno/vy3+Vc0znuOz/rxMTfEYcneD7POfcj4OUt4ZMTQ+5lxK6RYv99uHzpn5sHKu/kdlg8 /fJ6ha3u/EnLP0n/CzLete7tyk6mB/0ruyJL63k42zicjW3LN52Wup2Z5ZxZ9c0xL/uor1bv dQPD4x+fTDm999iceRcqJ2oU1NzZ/kGJpTgj0VCLuag4EQDRmWd5fgMAAA== X-CMS-MailID: 20210716150353eucas1p2c9ad1d1021ee584de587e5ec10b8467b X-Msg-Generator: CA Content-Type: text/plain; charset="utf-8" X-RootMTR: 20210716150353eucas1p2c9ad1d1021ee584de587e5ec10b8467b X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20210716150353eucas1p2c9ad1d1021ee584de587e5ec10b8467b References: <20210714013948.270662-1-shakeelb@google.com> <20210714013948.270662-2-shakeelb@google.com> X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 14C0630001A8 X-Stat-Signature: d5sj3didze6fzm5sqarax1o768fnsqh7 Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=samsung.com header.s=mail20170921 header.b=mc5j3zWY; dmarc=pass (policy=none) header.from=samsung.com; spf=pass (imf03.hostedemail.com: domain of m.szyprowski@samsung.com designates 210.118.77.11 as permitted sender) smtp.mailfrom=m.szyprowski@samsung.com X-HE-Tag: 1626447836-369478 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi, On 14.07.2021 03:39, Shakeel Butt wrote: > At the moment memcg stats are read in four contexts: > > 1. memcg stat user interfaces > 2. dirty throttling > 3. page fault > 4. memory reclaim > > Currently the kernel flushes the stats for first two cases. Flushing th= e > stats for remaining two casese may have performance impact. Always > flushing the memcg stats on the page fault code path may negatively > impacts the performance of the applications. In addition flushing in th= e > memory reclaim code path, though treated as slowpath, can become the > source of contention for the global lock taken for stat flushing becaus= e > when system or memcg is under memory pressure, many tasks may enter the > reclaim path. > > This patch uses following mechanisms to solve these challenges: > > 1. Periodically flush the stats from root memcg every 2 seconds. This > will time limit the out of sync stats. > > 2. Asynchronously flush the stats after fixed number of stat updates. > In the worst case the stat can be out of sync by O(nr_cpus * BATCH) for > 2 seconds. > > 3. For avoiding thundering herd to flush the stats particularly from th= e > memory reclaim context, introduce memcg local spinlock and let only one > flusher active at a time. This could have been done through > cgroup_rstat_lock lock but that lock is used by other subsystem and for > userspace reading memcg stats. So, it is better to keep flushers > introduced by this patch decoupled from cgroup_rstat_lock. > > Signed-off-by: Shakeel Butt This patch landed in today's linux-next (next-20210716) as commit=20 42265e014ac7 ("memcg: infrastructure to flush memcg stats"). On my test=20 system's I found that it triggers a kernel BUG on all ARM64 boards: =C2=A0BUG: sleeping function called from invalid context at=20 kernel/cgroup/rstat.c:200 =C2=A0in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 7, name:=20 kworker/u8:0 =C2=A03 locks held by kworker/u8:0/7: =C2=A0 #0: ffff00004000c938 ((wq_completion)events_unbound){+.+.}-{0:0},= at:=20 process_one_work+0x200/0x718 =C2=A0 #1: ffff80001334bdd0 ((stats_flush_dwork).work){+.+.}-{0:0}, at:=20 process_one_work+0x200/0x718 =C2=A0 #2: ffff8000124f6d40 (stats_flush_lock){+.+.}-{2:2}, at:=20 mem_cgroup_flush_stats+0x20/0x48 =C2=A0CPU: 2 PID: 7 Comm: kworker/u8:0 Tainted: G=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 W 5.14.0-rc1+ #3713 =C2=A0Hardware name: Raspberry Pi 4 Model B (DT) =C2=A0Workqueue: events_unbound flush_memcg_stats_dwork =C2=A0Call trace: =C2=A0 dump_backtrace+0x0/0x1d0 =C2=A0 show_stack+0x14/0x20 =C2=A0 dump_stack_lvl+0x88/0xb0 =C2=A0 dump_stack+0x14/0x2c =C2=A0 ___might_sleep+0x1dc/0x200 =C2=A0 __might_sleep+0x4c/0x88 =C2=A0 cgroup_rstat_flush+0x2c/0x58 =C2=A0 mem_cgroup_flush_stats+0x34/0x48 =C2=A0 flush_memcg_stats_dwork+0xc/0x38 =C2=A0 process_one_work+0x2a8/0x718 =C2=A0 worker_thread+0x48/0x460 =C2=A0 kthread+0x12c/0x160 =C2=A0 ret_from_fork+0x10/0x18 This can be also reproduced with QEmu. Please let me know if I can help=20 fixing this issue. > --- > Changes since v3: > - Add back the sigoff > > Changes since v2: > - Changed the subject of the patch > - Added mechanism to bound errors to nr_cpus instead of nr_cgroups > - memcg local lock to let one active flusher > > Changes since v1: > - use system_unbound_wq for flushing the memcg stats > > include/linux/memcontrol.h | 6 ++++++ > mm/memcontrol.c | 34 ++++++++++++++++++++++++++++++++++ > mm/vmscan.c | 6 ++++++ > 3 files changed, 46 insertions(+) > > diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h > index 0bfa0409af22..fa095a94ae56 100644 > --- a/include/linux/memcontrol.h > +++ b/include/linux/memcontrol.h > @@ -991,6 +991,8 @@ static inline unsigned long lruvec_page_state_local= (struct lruvec *lruvec, > return x; > } > =20 > +void mem_cgroup_flush_stats(void); > + > void __mod_memcg_lruvec_state(struct lruvec *lruvec, enum node_stat_i= tem idx, > int val); > void __mod_lruvec_kmem_state(void *p, enum node_stat_item idx, int va= l); > @@ -1400,6 +1402,10 @@ static inline unsigned long lruvec_page_state_lo= cal(struct lruvec *lruvec, > return node_page_state(lruvec_pgdat(lruvec), idx); > } > =20 > +static inline void mem_cgroup_flush_stats(void) > +{ > +} > + > static inline void __mod_memcg_lruvec_state(struct lruvec *lruvec, > enum node_stat_item idx, int val) > { > diff --git a/mm/memcontrol.c b/mm/memcontrol.c > index 848d711bf576..39a00991fc80 100644 > --- a/mm/memcontrol.c > +++ b/mm/memcontrol.c > @@ -103,6 +103,14 @@ static bool do_memsw_account(void) > return !cgroup_subsys_on_dfl(memory_cgrp_subsys) && !cgroup_memory_n= oswap; > } > =20 > +/* memcg and lruvec stats flushing */ > +static void flush_memcg_stats_dwork(struct work_struct *w); > +static DECLARE_DEFERRABLE_WORK(stats_flush_dwork, flush_memcg_stats_dw= ork); > +static void flush_memcg_stats_work(struct work_struct *w); > +static DECLARE_WORK(stats_flush_work, flush_memcg_stats_work); > +static DEFINE_PER_CPU(unsigned int, stats_flush_threshold); > +static DEFINE_SPINLOCK(stats_flush_lock); > + > #define THRESHOLDS_EVENTS_TARGET 128 > #define SOFTLIMIT_EVENTS_TARGET 1024 > =20 > @@ -685,6 +693,8 @@ void __mod_memcg_lruvec_state(struct lruvec *lruvec= , enum node_stat_item idx, > =20 > /* Update lruvec */ > __this_cpu_add(pn->lruvec_stats_percpu->state[idx], val); > + if (!(__this_cpu_inc_return(stats_flush_threshold) % MEMCG_CHARGE_BAT= CH)) > + queue_work(system_unbound_wq, &stats_flush_work); > } > =20 > /** > @@ -5248,6 +5258,10 @@ static int mem_cgroup_css_online(struct cgroup_s= ubsys_state *css) > /* Online state pins memcg ID, memcg ID pins CSS */ > refcount_set(&memcg->id.ref, 1); > css_get(css); > + > + if (unlikely(mem_cgroup_is_root(memcg))) > + queue_delayed_work(system_unbound_wq, &stats_flush_dwork, > + 2UL*HZ); > return 0; > } > =20 > @@ -5339,6 +5353,26 @@ static void mem_cgroup_css_reset(struct cgroup_s= ubsys_state *css) > memcg_wb_domain_size_changed(memcg); > } > =20 > +void mem_cgroup_flush_stats(void) > +{ > + if (!spin_trylock(&stats_flush_lock)) > + return; > + > + cgroup_rstat_flush(root_mem_cgroup->css.cgroup); > + spin_unlock(&stats_flush_lock); > +} > + > +static void flush_memcg_stats_dwork(struct work_struct *w) > +{ > + mem_cgroup_flush_stats(); > + queue_delayed_work(system_unbound_wq, &stats_flush_dwork, 2UL*HZ); > +} > + > +static void flush_memcg_stats_work(struct work_struct *w) > +{ > + mem_cgroup_flush_stats(); > +} > + > static void mem_cgroup_css_rstat_flush(struct cgroup_subsys_state *cs= s, int cpu) > { > struct mem_cgroup *memcg =3D mem_cgroup_from_css(css); > diff --git a/mm/vmscan.c b/mm/vmscan.c > index a7602f71ec04..1cc05ab8ca15 100644 > --- a/mm/vmscan.c > +++ b/mm/vmscan.c > @@ -2893,6 +2893,12 @@ static void shrink_node(pg_data_t *pgdat, struct= scan_control *sc) > target_lruvec =3D mem_cgroup_lruvec(sc->target_mem_cgroup, pgdat); > =20 > again: > + /* > + * Flush the memory cgroup stats, so that we read accurate per-memcg > + * lruvec stats for heuristics. > + */ > + mem_cgroup_flush_stats(); > + > memset(&sc->nr, 0, sizeof(sc->nr)); > =20 > nr_reclaimed =3D sc->nr_reclaimed; Best regards --=20 Marek Szyprowski, PhD Samsung R&D Institute Poland