From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85AC6EB64D8 for ; Wed, 21 Jun 2023 18:17:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1FE228D0008; Wed, 21 Jun 2023 14:17:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 188A28D0002; Wed, 21 Jun 2023 14:17:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ECE818D0008; Wed, 21 Jun 2023 14:17:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id DD73F8D0002 for ; Wed, 21 Jun 2023 14:17:13 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id A91F7140311 for ; Wed, 21 Jun 2023 18:17:13 +0000 (UTC) X-FDA: 80927561946.16.96ED86F Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) by imf21.hostedemail.com (Postfix) with ESMTP id A611D1C0017 for ; Wed, 21 Jun 2023 18:17:11 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=TwWoi+pZ; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf21.hostedemail.com: domain of 3pj6TZAcKCJsTP5I7CPBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--yuanchu.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3pj6TZAcKCJsTP5I7CPBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--yuanchu.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1687371431; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=P8vYzFHxLWdhm/YYBuM4cir+l3MkexuRd1yB7xK/qZo=; b=qVWekJmYxhvv5ut79oy0LTU3mfwskQj1/GkHEydfzpMCO2sakBXpXer+zr2apCDh3h7itz 5QL6D0c5vxvB8JLhtxkzZjElhRKI2usP3pDQMSgZPwsUBUHdGqcFWKw6PDz3njPJy+AreJ kqoP/KWGMDxbz4Nt2LsihQQ7jx2xlkk= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=TwWoi+pZ; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf21.hostedemail.com: domain of 3pj6TZAcKCJsTP5I7CPBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--yuanchu.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3pj6TZAcKCJsTP5I7CPBJJBG9.7JHGDIPS-HHFQ57F.JMB@flex--yuanchu.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1687371431; a=rsa-sha256; cv=none; b=BxEUXxkmYru+r2MDnxC69H5mnEgcN1+FiMKVA/VucOZP67sNulD3kU6GVJ4vWsytTH+6j5 dm737BnaTFspp9W8rB/XuMCQCf/CGXmJqrGWD833NzWrz3oqIkNxiwL+uSr/MhdGwLgTXY 8ELdBN7C/tuw6Yh22ABl9HxTe++Aya4= Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-25eac66423dso4648637a91.2 for ; Wed, 21 Jun 2023 11:17:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1687371430; x=1689963430; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=P8vYzFHxLWdhm/YYBuM4cir+l3MkexuRd1yB7xK/qZo=; b=TwWoi+pZl5v8X1NfXYAXDFTjFQRH0gRf8kcFXat6SAjcaa5QAAlB1haHO0q77RAinF D7xdtDa76dKs0GBJ34FBnPIsbh4xyimY5wrW1f6snuskuEcZJCOzknkqQ824BwEme3MM E6lxh0I4tvcYJRHOS1/KATWGHFctdscLKfObeEhRzcu1IiMQFS9ynJo1hdgECo0P8Gpt Xor2kFxaCjpsZ3henaaUXtkntEKa7iivsPZyOrHfOsxsId1bV6FY8Ohj1/LpkWsG+GdH d7oCOnaC7hgluf1VkKFS52Jq6dvfFstI19Xby8uEeRklXtHAMo72yh/opONH6YPZCAbt StMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687371430; x=1689963430; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=P8vYzFHxLWdhm/YYBuM4cir+l3MkexuRd1yB7xK/qZo=; b=c6RHqfTej8ZkAzmIPAIydbpW1XZW03gmz3BBq8Th8XRSVg58EAnzPQ4sjtzJAiOKGw WN+1+w7xlLuaIN72B9Yqk0+VlOcQfHWZBdmSaGsgEB0MeW9O/9zwsrxFJlD5c8wlthak Wf31AccIGt7kBAreHj6Fn2e5WexxRAmK3FXlaKIMf17PRbA0nVJE99g7NVrSDy3e6+MW /AiRPmGSYngdYEssw1s10hTdwMf8FaoHHKDCC/jqKIzR5O/vMyMRYBqhbu+DJkSZbFJj tLwO2+ecFVeFKl/IEqxGCOe0S1DsIXjYClHB4dniXiiC4iLUvbnT9F+mOLVI4ni0fn6l m+fw== X-Gm-Message-State: AC+VfDxLQ3hrdMZLqLzryDAkWyW4/iCyzMHA3JhfFng88cc+iEhKxjJG Cd4psxSByRy4C86dQSTlpDPOcKtQeaLq X-Google-Smtp-Source: ACHHUZ5ChKgXaGOZiUgiMpHUxjfmbgkoCy3pGADXdgSNrd1s/teZwOa9M7BVnVWQRd1xAjPY/MRtw95ar8Wv X-Received: from yuanchu.c.googlers.com ([fda3:e722:ac3:cc00:24:72f4:c0a8:4024]) (user=yuanchu job=sendgmr) by 2002:a17:90a:cb06:b0:25b:f973:f944 with SMTP id z6-20020a17090acb0600b0025bf973f944mr2138645pjt.3.1687371430482; Wed, 21 Jun 2023 11:17:10 -0700 (PDT) Date: Wed, 21 Jun 2023 18:04:52 +0000 In-Reply-To: <20230621180454.973862-1-yuanchu@google.com> Mime-Version: 1.0 References: <20230621180454.973862-1-yuanchu@google.com> X-Mailer: git-send-email 2.41.0.162.gfafddb0af9-goog Message-ID: <20230621180454.973862-5-yuanchu@google.com> Subject: [RFC PATCH v2 4/6] mm: extend working set reporting to memcgs From: Yuanchu Xie To: Greg Kroah-Hartman , "Rafael J . Wysocki" , "Michael S . Tsirkin" , David Hildenbrand , Jason Wang , Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Yu Zhao , Kefeng Wang , Kairui Song , Yosry Ahmed , Yuanchu Xie , "T . J . Alumbaugh" Cc: Wei Xu , SeongJae Park , Sudarshan Rajagopalan , kai.huang@intel.com, hch@lst.de, jon@nutanix.com, Aneesh Kumar K V , Matthew Wilcox , Vasily Averin , linux-kernel@vger.kernel.org, virtualization@lists.linux-foundation.org, linux-mm@kvack.org, cgroups@vger.kernel.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: A611D1C0017 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 4ku4x83xpkg6ocnf1r8ykoctdaiuukk3 X-HE-Tag: 1687371431-187874 X-HE-Meta: U2FsdGVkX1+EU0gPIoYNZq5zIxKPd//vDmn53ZKF+/+JxPeziv6LX0H3SFTdpvBdTgzCxcxtNSTU7iGP10r5gW/kNyBELuKKFt41wVPhnQWzOSQopIm6GcY8hTjua3pC5qfg5WN5qjQVXGz+SLQ3k0eQuyVYKF1Q9VJKcBPCXPnwE66FoTYcFigLcVuulJ1DkxZzeDdCCvswxAX33sPvXgW+WY3DTUFEqvIY5Qyxv5LmLe5DJdQdQrKrdfMGFohgtDTgSMJcm9vcMwLbAaaDpuVcp2RFNNu7XpcaSJ6CvdXDQiXcAnaq57W6dYh8qll0szLJSxgtN7lPfaQf4RLLtwvVeqC6uPqBNPpnOkJysb34Lh5jxOvzeg6g2MobwNOw+ETaJBKol9erSL0uy2pk+LczwN3x3d67ZRFVyBQJ3DoGq+THNVxROSIekcXMh97vSiDLjlANp2G7zc0elEW5KuGTHph/n45EOjrnUUnV9AsENnzUTyvm7niBQkiFgAKBOFzPbfUaXOBN2iMXZAaoFF1oUEXJGxncu7dfyyNJNw6jlH3rmSnv3h6K+wWaPtgEZUz2w8wQAEi+0HiJG/w5DcJaWFbd4Cihg5GjiLv56N/K6SHPARcVE/2MNK8OnAmK4xia+LR2+wU7S4AZf5ho7VFVAnNALqRZhlXS39yfpqCOj0a3QrgrzqtwpQeY9KMkkeVDV57dk5yIaPISbKfWIjjM/KSEAtOzo01Ki6/O2fjNlvz2IYO3IJKfbHHCbFdHFleIqPYUoZ08iLkT+lfcNqBT9fPK2h/1QU+FzWXQBti2UJh2vHZb+7/oA+D5t9bcHEBjB46dO/CZ8wSS2lsYagdILDxjvOWPcztygKbt3IiIKlP0PmEPQFPv5D8EW1C2qK4/xwSTU5/N26zkZqUwyIDSh9pcP119syqU0T1bVK67VHffgr/JVjPUxkBHpiSPrlncRQEp7y/7X5d3xmT wEB6rttT 3WWysoihteuaApbajwppExxJl/CW18hhIF2o5T0D9uKq/aWQjm1vn4RGt9U3gnM19jlrcofHGm7QiKTjZvvBBFYfWX15k9Su1KfCge6ZTM4VBFgmY5C+a6iwoaU5Q6mYWf4WdzuqCSu705NOGN70BFvT/rAhZs1IhhdquWsxpH6COkJN1JajavoVQs2bRYWDirZHOtDXNCWR33slrhBEr/JrEhp4NKa2tldgS2eYevu7GDDOIvsWtjxwvWfGs52qeBU3uQwzqnOGwbLZFLAMAKpVK2t5HKcWhNb+QcO3l158f6lxI1pfB1fGK8FylNvba5cAuXKJqHbmrinvxroNJBeKSEu24K5yeqpdmaVbpWnWINoZZ6H2du/b57isvAZ6DedqIlrFnlLa6cLpFI6iVyGNuO2z/nvvqIsyuN7eCYzKJhhOx5LhCPzWvcJ1CfD2qJtQUU1BF/roSNcOZWMBhLIO+fB1vOWgbuNaiTqx1sKESZAhtWH34Kc3EE8pagFTq99ROcVysw3zJn6zAPBZX8mMHdgcuW2y/jzFA+uqnjp0CxPGc8kA/xqQPfYHfV5RIzGxdlTFC6w6RcCjxYXLLG2OSxrUvBjrw9+H8XLHh+CmLLL8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Break down the system-wide working set reporting into per-memcg reports, which aggregages its children hierarchically. The per-node working set reporting histograms and refresh/report threshold files are presented as memcg files, showing a report containing all the nodes. Signed-off-by: T.J. Alumbaugh Signed-off-by: Yuanchu Xie --- include/linux/memcontrol.h | 6 + include/linux/wsr.h | 4 + mm/memcontrol.c | 262 ++++++++++++++++++++++++++++++++++++- mm/vmscan.c | 9 +- 4 files changed, 277 insertions(+), 4 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 85dc9b88ea379..96971aa6a48cd 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -10,6 +10,7 @@ #ifndef _LINUX_MEMCONTROL_H #define _LINUX_MEMCONTROL_H +#include #include #include #include @@ -325,6 +326,11 @@ struct mem_cgroup { struct lru_gen_mm_list mm_list; #endif +#ifdef CONFIG_WSR + int wsr_event; + wait_queue_head_t wsr_wait_queue; +#endif + struct mem_cgroup_per_node *nodeinfo[]; }; diff --git a/include/linux/wsr.h b/include/linux/wsr.h index 85c901ce026b9..d45f7cc0672ac 100644 --- a/include/linux/wsr.h +++ b/include/linux/wsr.h @@ -48,6 +48,7 @@ ssize_t wsr_intervals_ms_parse(char *src, struct ws_bin *bins); */ void wsr_refresh(struct wsr *wsr, struct mem_cgroup *root, struct pglist_data *pgdat); +void report_ws(struct pglist_data *pgdat, struct scan_control *sc); #else struct ws_bin; struct wsr; @@ -73,6 +74,9 @@ static inline void wsr_refresh(struct wsr *wsr, struct mem_cgroup *root, struct pglist_data *pgdat) { } +static inline void report_ws(struct pglist_data *pgdat, struct scan_control *sc) +{ +} #endif /* CONFIG_WSR */ #endif /* _LINUX_WSR_H */ diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 2eee092f8f119..edf5bb31bb19c 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -25,6 +25,7 @@ * Copyright (C) 2020 Alibaba, Inc, Alex Shi */ +#include #include #include #include @@ -65,6 +66,7 @@ #include #include "internal.h" #include +#include #include #include "slab.h" #include "swap.h" @@ -5233,6 +5235,7 @@ static void free_mem_cgroup_per_node_info(struct mem_cgroup *memcg, int node) if (!pn) return; + wsr_destroy(&pn->lruvec); free_percpu(pn->lruvec_stats_percpu); kfree(pn); } @@ -5311,6 +5314,10 @@ static struct mem_cgroup *mem_cgroup_alloc(void) spin_lock_init(&memcg->deferred_split_queue.split_queue_lock); INIT_LIST_HEAD(&memcg->deferred_split_queue.split_queue); memcg->deferred_split_queue.split_queue_len = 0; +#endif +#ifdef CONFIG_WSR + memcg->wsr_event = 0; + init_waitqueue_head(&memcg->wsr_wait_queue); #endif idr_replace(&mem_cgroup_idr, memcg, memcg->id.id); lru_gen_init_memcg(memcg); @@ -5411,6 +5418,11 @@ static void mem_cgroup_css_offline(struct cgroup_subsys_state *css) } spin_unlock_irq(&memcg->event_list_lock); +#ifdef CONFIG_WSR + wake_up_pollfree(&memcg->wsr_wait_queue); + synchronize_rcu(); +#endif + page_counter_set_min(&memcg->memory, 0); page_counter_set_low(&memcg->memory, 0); @@ -6642,6 +6654,228 @@ static ssize_t memory_reclaim(struct kernfs_open_file *of, char *buf, return nbytes; } +#ifdef CONFIG_WSR +static int memory_wsr_intervals_ms_show(struct seq_file *m, void *v) +{ + int nid; + struct mem_cgroup *memcg = mem_cgroup_from_seq(m); + + for_each_node_state(nid, N_MEMORY) { + struct wsr *wsr; + struct ws_bin *bin; + + wsr = lruvec_wsr(mem_cgroup_lruvec(memcg, NODE_DATA(nid))); + mutex_lock(&wsr->bins_lock); + seq_printf(m, "N%d=", nid); + for (bin = wsr->bins; bin->idle_age != -1; bin++) + seq_printf(m, "%u,", jiffies_to_msecs(bin->idle_age)); + mutex_unlock(&wsr->bins_lock); + + seq_printf(m, "%lld ", LLONG_MAX); + } + seq_putc(m, '\n'); + + return 0; +} + +static ssize_t memory_wsr_intervals_ms_parse(struct kernfs_open_file *of, + char *buf, size_t nbytes, + unsigned int *nid_out, + struct ws_bin *bins) +{ + char *node, *intervals; + unsigned int nid; + int err; + + buf = strstrip(buf); + intervals = buf; + node = strsep(&intervals, "="); + + if (*node != 'N') + return -EINVAL; + + err = kstrtouint(node + 1, 0, &nid); + if (err) + return err; + + if (nid >= nr_node_ids || !node_state(nid, N_MEMORY)) + return -EINVAL; + + err = wsr_intervals_ms_parse(intervals, bins); + if (err) + return err; + + *nid_out = nid; + return 0; +} + +static ssize_t memory_wsr_intervals_ms_write(struct kernfs_open_file *of, + char *buf, size_t nbytes, + loff_t off) +{ + unsigned int nid; + int err; + struct wsr *wsr; + struct ws_bin *bins; + struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of)); + + bins = kzalloc(sizeof(wsr->bins), GFP_KERNEL); + if (!bins) + return -ENOMEM; + + err = memory_wsr_intervals_ms_parse(of, buf, nbytes, &nid, bins); + if (err) + goto failed; + + wsr = lruvec_wsr(mem_cgroup_lruvec(memcg, NODE_DATA(nid))); + mutex_lock(&wsr->bins_lock); + memcpy(wsr->bins, bins, sizeof(wsr->bins)); + mutex_unlock(&wsr->bins_lock); +failed: + kfree(bins); + return err ?: nbytes; +} + +static int memory_wsr_refresh_ms_show(struct seq_file *m, void *v) +{ + int nid; + struct mem_cgroup *memcg = mem_cgroup_from_seq(m); + + for_each_node_state(nid, N_MEMORY) { + struct wsr *wsr = + lruvec_wsr(mem_cgroup_lruvec(memcg, NODE_DATA(nid))); + + seq_printf(m, "N%d=%u ", nid, + jiffies_to_msecs(READ_ONCE(wsr->refresh_threshold))); + } + seq_putc(m, '\n'); + + return 0; +} + +static ssize_t memory_wsr_threshold_parse(char *buf, size_t nbytes, + unsigned int *nid_out, + unsigned int *msecs) +{ + char *node, *threshold; + unsigned int nid; + int err; + + buf = strstrip(buf); + threshold = buf; + node = strsep(&threshold, "="); + + if (*node != 'N') + return -EINVAL; + + err = kstrtouint(node + 1, 0, &nid); + if (err) + return err; + + if (nid >= nr_node_ids || !node_state(nid, N_MEMORY)) + return -EINVAL; + + err = kstrtouint(threshold, 0, msecs); + if (err) + return err; + + *nid_out = nid; + + return nbytes; +} + +static ssize_t memory_wsr_refresh_ms_write(struct kernfs_open_file *of, + char *buf, size_t nbytes, loff_t off) +{ + unsigned int nid, msecs; + struct wsr *wsr; + struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of)); + ssize_t ret = memory_wsr_threshold_parse(buf, nbytes, &nid, &msecs); + + if (ret < 0) + return ret; + + wsr = lruvec_wsr(mem_cgroup_lruvec(memcg, NODE_DATA(nid))); + WRITE_ONCE(wsr->refresh_threshold, msecs_to_jiffies(msecs)); + return ret; +} + +static int memory_wsr_report_ms_show(struct seq_file *m, void *v) +{ + int nid; + struct mem_cgroup *memcg = mem_cgroup_from_seq(m); + + for_each_node_state(nid, N_MEMORY) { + struct wsr *wsr = + lruvec_wsr(mem_cgroup_lruvec(memcg, NODE_DATA(nid))); + + seq_printf(m, "N%d=%u ", nid, + jiffies_to_msecs(READ_ONCE(wsr->report_threshold))); + } + seq_putc(m, '\n'); + + return 0; +} + +static ssize_t memory_wsr_report_ms_write(struct kernfs_open_file *of, + char *buf, size_t nbytes, loff_t off) +{ + unsigned int nid, msecs; + struct wsr *wsr; + struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of)); + ssize_t ret = memory_wsr_threshold_parse(buf, nbytes, &nid, &msecs); + + if (ret < 0) + return ret; + + wsr = lruvec_wsr(mem_cgroup_lruvec(memcg, NODE_DATA(nid))); + WRITE_ONCE(wsr->report_threshold, msecs_to_jiffies(msecs)); + return ret; +} + +static int memory_wsr_histogram_show(struct seq_file *m, void *v) +{ + int nid; + struct mem_cgroup *memcg = mem_cgroup_from_seq(m); + + for_each_node_state(nid, N_MEMORY) { + struct wsr *wsr = + lruvec_wsr(mem_cgroup_lruvec(memcg, NODE_DATA(nid))); + struct ws_bin *bin; + + seq_printf(m, "N%d\n", nid); + + mutex_lock(&wsr->bins_lock); + wsr_refresh(wsr, memcg, NODE_DATA(nid)); + for (bin = wsr->bins; bin->idle_age != -1; bin++) + seq_printf(m, "%u anon=%lu file=%lu\n", + jiffies_to_msecs(bin->idle_age), + bin->nr_pages[0], bin->nr_pages[1]); + + seq_printf(m, "%lld anon=%lu file=%lu\n", LLONG_MAX, + bin->nr_pages[0], bin->nr_pages[1]); + + mutex_unlock(&wsr->bins_lock); + } + + return 0; +} + +__poll_t memory_wsr_histogram_poll(struct kernfs_open_file *of, + struct poll_table_struct *pt) +{ + struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of)); + + if (memcg->css.flags & CSS_DYING) + return DEFAULT_POLLMASK; + + poll_wait(of->file, &memcg->wsr_wait_queue, pt); + if (cmpxchg(&memcg->wsr_event, 1, 0) == 1) + return DEFAULT_POLLMASK | EPOLLPRI; + return DEFAULT_POLLMASK; +} +#endif + static struct cftype memory_files[] = { { .name = "current", @@ -6710,7 +6944,33 @@ static struct cftype memory_files[] = { .flags = CFTYPE_NS_DELEGATABLE, .write = memory_reclaim, }, - { } /* terminate */ +#ifdef CONFIG_WSR + { + .name = "wsr.intervals_ms", + .flags = CFTYPE_NOT_ON_ROOT | CFTYPE_NS_DELEGATABLE, + .seq_show = memory_wsr_intervals_ms_show, + .write = memory_wsr_intervals_ms_write, + }, + { + .name = "wsr.refresh_ms", + .flags = CFTYPE_NOT_ON_ROOT | CFTYPE_NS_DELEGATABLE, + .seq_show = memory_wsr_refresh_ms_show, + .write = memory_wsr_refresh_ms_write, + }, + { + .name = "wsr.report_ms", + .flags = CFTYPE_NOT_ON_ROOT | CFTYPE_NS_DELEGATABLE, + .seq_show = memory_wsr_report_ms_show, + .write = memory_wsr_report_ms_write, + }, + { + .name = "wsr.histogram", + .flags = CFTYPE_NOT_ON_ROOT | CFTYPE_NS_DELEGATABLE, + .seq_show = memory_wsr_histogram_show, + .poll = memory_wsr_histogram_poll, + }, +#endif + {} /* terminate */ }; struct cgroup_subsys memory_cgrp_subsys = { diff --git a/mm/vmscan.c b/mm/vmscan.c index c56fddcec88fb..ba254b6e91e19 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -4559,8 +4559,6 @@ static bool age_lruvec(struct lruvec *lruvec, struct scan_control *sc, unsigned return true; } -static void report_ws(struct pglist_data *pgdat, struct scan_control *sc); - /* to protect the working set of the last N jiffies */ static unsigned long lru_gen_min_ttl __read_mostly; @@ -5937,7 +5935,7 @@ void wsr_refresh(struct wsr *wsr, struct mem_cgroup *root, } } -static void report_ws(struct pglist_data *pgdat, struct scan_control *sc) +void report_ws(struct pglist_data *pgdat, struct scan_control *sc) { static DEFINE_RATELIMIT_STATE(rate, HZ, 3); @@ -5969,6 +5967,8 @@ static void report_ws(struct pglist_data *pgdat, struct scan_control *sc) if (wsr->notifier) kernfs_notify(wsr->notifier); + if (memcg && cmpxchg(&memcg->wsr_event, 0, 1) == 0) + wake_up_interruptible(&memcg->wsr_wait_queue); } #endif /* CONFIG_WSR */ @@ -6486,6 +6486,9 @@ static void shrink_zones(struct zonelist *zonelist, struct scan_control *sc) if (zone->zone_pgdat == last_pgdat) continue; last_pgdat = zone->zone_pgdat; + + if (!sc->proactive) + report_ws(zone->zone_pgdat, sc); shrink_node(zone->zone_pgdat, sc); } -- 2.41.0.162.gfafddb0af9-goog