From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E871C43461 for ; Tue, 15 Sep 2020 21:50:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 2703F20809 for ; Tue, 15 Sep 2020 21:50:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20150623.gappssmtp.com header.i=@cmpxchg-org.20150623.gappssmtp.com header.b="uOWcBox3" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2703F20809 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=cmpxchg.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id C188890009A; Tue, 15 Sep 2020 17:50:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BC88690006B; Tue, 15 Sep 2020 17:50:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A920390009A; Tue, 15 Sep 2020 17:50:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0060.hostedemail.com [216.40.44.60]) by kanga.kvack.org (Postfix) with ESMTP id 8EF4690006B for ; Tue, 15 Sep 2020 17:50:11 -0400 (EDT) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 4F15F181AEF1A for ; Tue, 15 Sep 2020 21:50:11 +0000 (UTC) X-FDA: 77266639422.14.owl59_3e01cb427114 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin14.hostedemail.com (Postfix) with ESMTP id 2545E18229818 for ; Tue, 15 Sep 2020 21:50:11 +0000 (UTC) X-HE-Tag: owl59_3e01cb427114 X-Filterd-Recvd-Size: 9305 Received: from mail-qt1-f195.google.com (mail-qt1-f195.google.com [209.85.160.195]) by imf11.hostedemail.com (Postfix) with ESMTP for ; Tue, 15 Sep 2020 21:50:10 +0000 (UTC) Received: by mail-qt1-f195.google.com with SMTP id 19so4528474qtp.1 for ; Tue, 15 Sep 2020 14:50:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=gwAGdDXVM4r4OL55XSo1884lNeDAwaOm7bwctRtLrys=; b=uOWcBox38dGodh/iQrPqVF2GLIJyW9L4Y3ANaEZXp2ypNq6/uKrX+x43BZP8K1yuwB TjBbkfHme3SAZjHyCDwS27BnDVQcAodcI29aOajz9qZKP6YTdfgiHiG7ERgW0xkoMkZc vnnYhnh1hdS2aYSiqfXF918Ttjkg/AZluGilhI1SM4HJ2oSeNm8sbJrtEH/4xheskIPM Fn349hTQnPUj1MoyZwZEIsaKO2GhGt3GRpPEws4hszG3mSahAm4PmzMdhBav2zpg/aGG GkAk3u/IFwWEN8bb5pmjgdXCg+UJyIAZ1cTAk/PMXcWapHOK0hyjAleZE0fB0RVgd8O4 IhMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=gwAGdDXVM4r4OL55XSo1884lNeDAwaOm7bwctRtLrys=; b=Dye0HYJIrOaL6QYCNXNJmxy2q8jTiiI4RrzgGGlc5ne0UDu5gntWxn17rRGvALaiJN F8uVcGTYsCGZ4CZa/nJJxKT64nw260+jMtMjwGyjBSJF8uls6OTNs5itiNb2Xxq5RNWt 0ho5aeZVCECeFOS96aev77ApKQcKTtbCPd/aiPorFGgwze5vTJ3TN3+Z1jHSE5vwZBKu HqkpT+LS9WDQzpZaOkWrWitqLzDFQ9H+OX7m+0l/vZERrejkWODPuV0elWcOdrScHYyO VUpfkKMCjocqD9AAmJGq3w9nv5x/LVFSi8u+uZF1zi+MNIjea/nQOpztqwzD/8e1+gdv NrZA== X-Gm-Message-State: AOAM531F0uJZygL7ef9gzcJ3+W437KOtoYTR2AqbHc0a2Vu1u4SmvIyD mU9Dje/lKGhZbwjQoHhdkle18Q== X-Google-Smtp-Source: ABdhPJzQh75yII3N5KV/1tjYmS/zxa6QaCbe44anS0rhYQ2i1j9XbUmXAcwDIwvYDSQHA5IH+rQ8SQ== X-Received: by 2002:ac8:863:: with SMTP id x32mr7523555qth.27.1600206609589; Tue, 15 Sep 2020 14:50:09 -0700 (PDT) Received: from localhost ([2620:10d:c091:480::1:124c]) by smtp.gmail.com with ESMTPSA id b79sm18586859qkg.10.2020.09.15.14.50.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Sep 2020 14:50:08 -0700 (PDT) Date: Tue, 15 Sep 2020 17:48:45 -0400 From: Johannes Weiner To: Muchun Song Cc: tj@kernel.org, lizefan@huawei.com, corbet@lwn.net, mhocko@kernel.org, vdavydov.dev@gmail.com, akpm@linux-foundation.org, shakeelb@google.com, guro@fb.com, rdunlap@infradead.org, cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH v5] mm: memcontrol: Add the missing numa_stat interface for cgroup v2 Message-ID: <20200915214845.GB189808@cmpxchg.org> References: <20200915171801.39761-1-songmuchun@bytedance.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200915171801.39761-1-songmuchun@bytedance.com> X-Rspamd-Queue-Id: 2545E18229818 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam03 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Sep 16, 2020 at 01:18:01AM +0800, Muchun Song wrote: > In the cgroup v1, we have a numa_stat interface. This is useful for > providing visibility into the numa locality information within an > memcg since the pages are allowed to be allocated from any physical > node. One of the use cases is evaluating application performance by > combining this information with the application's CPU allocation. > But the cgroup v2 does not. So this patch adds the missing information. > > Signed-off-by: Muchun Song > Suggested-by: Shakeel Butt > Reviewed-by: Shakeel Butt Yup, that would be useful information to have. Just a few comments on the patch below: > @@ -1368,6 +1368,78 @@ PAGE_SIZE multiple when read back. > collapsing an existing range of pages. This counter is not > present when CONFIG_TRANSPARENT_HUGEPAGE is not set. > > + memory.numa_stat > + A read-only flat-keyed file which exists on non-root cgroups. It's a nested key file, not flat. > + This breaks down the cgroup's memory footprint into different > + types of memory, type-specific details, and other information > + per node on the state of the memory management system. > + > + This is useful for providing visibility into the NUMA locality > + information within an memcg since the pages are allowed to be > + allocated from any physical node. One of the use case is evaluating > + application performance by combining this information with the > + application's CPU allocation. > + > + All memory amounts are in bytes. > + > + The output format of memory.numa_stat is:: > + > + type N0= N1= ... > + > + The entries are ordered to be human readable, and new entries > + can show up in the middle. Don't rely on items remaining in a > + fixed position; use the keys to look up specific values! > + > + anon > + Amount of memory per node used in anonymous mappings such > + as brk(), sbrk(), and mmap(MAP_ANONYMOUS). > + > + file > + Amount of memory per node used to cache filesystem data, > + including tmpfs and shared memory. > + > + kernel_stack > + Amount of memory per node allocated to kernel stacks. > + > + shmem > + Amount of cached filesystem data per node that is swap-backed, > + such as tmpfs, shm segments, shared anonymous mmap()s. > + > + file_mapped > + Amount of cached filesystem data per node mapped with mmap(). > + > + file_dirty > + Amount of cached filesystem data per node that was modified but > + not yet written back to disk. > + > + file_writeback > + Amount of cached filesystem data per node that was modified and > + is currently being written back to disk. > + > + anon_thp > + Amount of memory per node used in anonymous mappings backed by > + transparent hugepages. > + > + inactive_anon, active_anon, inactive_file, active_file, unevictable > + Amount of memory, swap-backed and filesystem-backed, > + per node on the internal memory management lists used > + by the page reclaim algorithm. > + > + As these represent internal list state (e.g. shmem pages are on > + anon memory management lists), inactive_foo + active_foo may not > + be equal to the value for the foo counter, since the foo counter > + is type-based, not list-based. > + > + slab_reclaimable > + Amount of memory per node used for storing in-kernel data > + structures which might be reclaimed, such as dentries and > + inodes. > + > + slab_unreclaimable > + Amount of memory per node used for storing in-kernel data > + structures which cannot be reclaimed on memory pressure. > + > memory.swap.current > A read-only single value file which exists on non-root > cgroups. > diff --git a/mm/memcontrol.c b/mm/memcontrol.c > index 75cd1a1e66c8..ff919ef3b57b 100644 > --- a/mm/memcontrol.c > +++ b/mm/memcontrol.c > @@ -6425,6 +6425,86 @@ static int memory_stat_show(struct seq_file *m, void *v) > return 0; > } > > +#ifdef CONFIG_NUMA > +struct numa_stat { > + const char *name; > + unsigned int ratio; > + enum node_stat_item idx; > +}; > + > +static struct numa_stat numa_stats[] = { > + { "anon", PAGE_SIZE, NR_ANON_MAPPED }, > + { "file", PAGE_SIZE, NR_FILE_PAGES }, > + { "kernel_stack", 1024, NR_KERNEL_STACK_KB }, > + { "shmem", PAGE_SIZE, NR_SHMEM }, > + { "file_mapped", PAGE_SIZE, NR_FILE_MAPPED }, > + { "file_dirty", PAGE_SIZE, NR_FILE_DIRTY }, > + { "file_writeback", PAGE_SIZE, NR_WRITEBACK }, > +#ifdef CONFIG_TRANSPARENT_HUGEPAGE > + /* > + * The ratio will be initialized in numa_stats_init(). Because > + * on some architectures, the macro of HPAGE_PMD_SIZE is not > + * constant(e.g. powerpc). > + */ > + { "anon_thp", 0, NR_ANON_THPS }, > +#endif > + { "inactive_anon", PAGE_SIZE, NR_INACTIVE_ANON }, > + { "active_anon", PAGE_SIZE, NR_ACTIVE_ANON }, > + { "inactive_file", PAGE_SIZE, NR_INACTIVE_FILE }, > + { "active_file", PAGE_SIZE, NR_ACTIVE_FILE }, > + { "unevictable", PAGE_SIZE, NR_UNEVICTABLE }, > + { "slab_reclaimable", 1, NR_SLAB_RECLAIMABLE_B }, > + { "slab_unreclaimable", 1, NR_SLAB_UNRECLAIMABLE_B }, > +}; This is a bit duplicative with memory_stat_format(), and the collections will easily go out of sync as we add/change stat items. Can you please convert memory_stat_format() to use the same shared table? You may have to add another flag for the MEMCG_* items for which we don't have per-node counters. The same applies to the documentation. Please don't duplicate the list of items, but have memory.numa_stat refer to the list for memory.stat. You can add (not in memory.numa_stat) or something to percpu and sock. > +static unsigned long memcg_node_page_state(struct mem_cgroup *memcg, > + unsigned int nid, > + enum node_stat_item idx) > +{ > + VM_BUG_ON(nid >= nr_node_ids); > + return lruvec_page_state(mem_cgroup_lruvec(memcg, NODE_DATA(nid)), idx); > +} Please drop this wrapper and use lruvec_page_state directly below. Otherwise, this looks reasonable to me.