From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E96C6C433ED for ; Mon, 12 Apr 2021 22:55:30 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 942D36124C for ; Mon, 12 Apr 2021 22:55:30 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 942D36124C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2AAB46B006E; Mon, 12 Apr 2021 18:55:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2818F6B0070; Mon, 12 Apr 2021 18:55:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 14AAE6B0071; Mon, 12 Apr 2021 18:55:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0020.hostedemail.com [216.40.44.20]) by kanga.kvack.org (Postfix) with ESMTP id EE4D06B006E for ; Mon, 12 Apr 2021 18:55:29 -0400 (EDT) Received: from smtpin17.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id A74B08248D51 for ; Mon, 12 Apr 2021 22:55:29 +0000 (UTC) X-FDA: 78025223178.17.282E681 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf12.hostedemail.com (Postfix) with ESMTP id 572E2D2 for ; Mon, 12 Apr 2021 22:55:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1618268128; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:in-reply-to:in-reply-to:references:references; bh=80OiZdRF0zG+/9/xLJu7PaNs7xvMkYou02sp8RHghi4=; b=O52Q174tXu9wscSdAZqDgX5ezzhbzq93PShBNYx+fyPECIWvVwF+PHy3FKTHZuWzqT1VOn YWLoK3RH71bWLHMUVg6l420lPgQGhALlHc2hHQFGuh/D2gcBHrdiw/Yu5JhFlT1RhrOqCu m5pF4jBGhmVPYBQCJx+TZqrvwx5yRqY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-413-nLRY1qLLPVefw4I5x2ltEw-1; Mon, 12 Apr 2021 18:55:27 -0400 X-MC-Unique: nLRY1qLLPVefw4I5x2ltEw-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4EE52801814; Mon, 12 Apr 2021 22:55:23 +0000 (UTC) Received: from llong.com (ovpn-114-18.rdu2.redhat.com [10.10.114.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 177255D71B; Mon, 12 Apr 2021 22:55:21 +0000 (UTC) From: Waiman Long To: Johannes Weiner , Michal Hocko , Vladimir Davydov , Andrew Morton , Tejun Heo , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Vlastimil Babka , Roman Gushchin Cc: linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Shakeel Butt , Muchun Song , Alex Shi , Chris Down , Yafang Shao , Wei Yang , Masayoshi Mizuma , Xing Zhengjun , Waiman Long Subject: [PATCH v2 1/5] mm/memcg: Pass both memcg and lruvec to mod_memcg_lruvec_state() Date: Mon, 12 Apr 2021 18:54:59 -0400 Message-Id: <20210412225503.15119-2-longman@redhat.com> In-Reply-To: <20210412225503.15119-1-longman@redhat.com> References: <20210412225503.15119-1-longman@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 572E2D2 X-Stat-Signature: ni534cieo7f6khhh8ndg16owucbbz7h8 Received-SPF: none (redhat.com>: No applicable sender policy available) receiver=imf12; identity=mailfrom; envelope-from=""; helo=us-smtp-delivery-124.mimecast.com; client-ip=216.205.24.124 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618268124-589764 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The caller of mod_memcg_lruvec_state() has both memcg and lruvec readily available. So both of them are now passed to mod_memcg_lruvec_state() and __mod_memcg_lruvec_state(). The __mod_memcg_lruvec_state() is updated to allow either of the two parameters to be set to null. This makes mod_memcg_lruvec_state() equivalent to mod_memcg_state() if lruvec is null. The new __mod_memcg_lruvec_state() function will be used in the next patch as a replacement of mod_memcg_state() in mm/percpu.c for the consolidation of the memory uncharge and vmstat update functions in the kmem_cache_free() path. Signed-off-by: Waiman Long Acked-by: Roman Gushchin Reviewed-by: Shakeel Butt --- include/linux/memcontrol.h | 12 +++++++----- mm/memcontrol.c | 19 +++++++++++++------ mm/slab.h | 2 +- 3 files changed, 21 insertions(+), 12 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 0c04d39a7967..95f12996e66c 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -955,8 +955,8 @@ static inline unsigned long lruvec_page_state_local(struct lruvec *lruvec, return x; } -void __mod_memcg_lruvec_state(struct lruvec *lruvec, enum node_stat_item idx, - int val); +void __mod_memcg_lruvec_state(struct mem_cgroup *memcg, struct lruvec *lruvec, + enum node_stat_item idx, int val); void __mod_lruvec_kmem_state(void *p, enum node_stat_item idx, int val); static inline void mod_lruvec_kmem_state(void *p, enum node_stat_item idx, @@ -969,13 +969,14 @@ static inline void mod_lruvec_kmem_state(void *p, enum node_stat_item idx, local_irq_restore(flags); } -static inline void mod_memcg_lruvec_state(struct lruvec *lruvec, +static inline void mod_memcg_lruvec_state(struct mem_cgroup *memcg, + struct lruvec *lruvec, enum node_stat_item idx, int val) { unsigned long flags; local_irq_save(flags); - __mod_memcg_lruvec_state(lruvec, idx, val); + __mod_memcg_lruvec_state(memcg, lruvec, idx, val); local_irq_restore(flags); } @@ -1369,7 +1370,8 @@ static inline unsigned long lruvec_page_state_local(struct lruvec *lruvec, return node_page_state(lruvec_pgdat(lruvec), idx); } -static inline void __mod_memcg_lruvec_state(struct lruvec *lruvec, +static inline void __mod_memcg_lruvec_state(struct mem_cgroup *memcg, + struct lruvec *lruvec, enum node_stat_item idx, int val) { } diff --git a/mm/memcontrol.c b/mm/memcontrol.c index e064ac0d850a..d66e1e38f8ac 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -799,20 +799,27 @@ parent_nodeinfo(struct mem_cgroup_per_node *pn, int nid) return mem_cgroup_nodeinfo(parent, nid); } -void __mod_memcg_lruvec_state(struct lruvec *lruvec, enum node_stat_item idx, - int val) +/* + * Either one of memcg or lruvec can be NULL, but not both. + */ +void __mod_memcg_lruvec_state(struct mem_cgroup *memcg, struct lruvec *lruvec, + enum node_stat_item idx, int val) { struct mem_cgroup_per_node *pn; - struct mem_cgroup *memcg; long x, threshold = MEMCG_CHARGE_BATCH; + /* Update lruvec */ pn = container_of(lruvec, struct mem_cgroup_per_node, lruvec); - memcg = pn->memcg; + + if (!memcg) + memcg = pn->memcg; /* Update memcg */ __mod_memcg_state(memcg, idx, val); - /* Update lruvec */ + if (!lruvec) + return; + __this_cpu_add(pn->lruvec_stat_local->count[idx], val); if (vmstat_item_in_bytes(idx)) @@ -848,7 +855,7 @@ void __mod_lruvec_state(struct lruvec *lruvec, enum node_stat_item idx, /* Update memcg and lruvec */ if (!mem_cgroup_disabled()) - __mod_memcg_lruvec_state(lruvec, idx, val); + __mod_memcg_lruvec_state(NULL, lruvec, idx, val); } void __mod_lruvec_page_state(struct page *page, enum node_stat_item idx, diff --git a/mm/slab.h b/mm/slab.h index 076582f58f68..bc6c7545e487 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -293,7 +293,7 @@ static inline void mod_objcg_state(struct obj_cgroup *objcg, rcu_read_lock(); memcg = obj_cgroup_memcg(objcg); lruvec = mem_cgroup_lruvec(memcg, pgdat); - mod_memcg_lruvec_state(lruvec, idx, nr); + mod_memcg_lruvec_state(memcg, lruvec, idx, nr); rcu_read_unlock(); } -- 2.18.1