From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF4E0CD1284 for ; Wed, 27 Mar 2024 21:31:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9094F6B008A; Wed, 27 Mar 2024 17:31:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8BA756B008C; Wed, 27 Mar 2024 17:31:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 70CB86B0092; Wed, 27 Mar 2024 17:31:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 5121F6B008A for ; Wed, 27 Mar 2024 17:31:37 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id CBD1A801E1 for ; Wed, 27 Mar 2024 21:31:36 +0000 (UTC) X-FDA: 81944115792.04.D91159E Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by imf11.hostedemail.com (Postfix) with ESMTP id F379A40016 for ; Wed, 27 Mar 2024 21:31:34 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=NQkUDlar; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf11.hostedemail.com: domain of 3NpAEZgcKCJ8XT9MBGTFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--yuanchu.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3NpAEZgcKCJ8XT9MBGTFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--yuanchu.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711575095; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=CyJDixqg00pkxMZu+nlTfhoUxwjhkmzgxYkGofwS92E=; b=tIb9myFbzrFCzVeVSp4xmoHIse3E5Yo+Jopqm4k3Z7XjFY7Zq3VZ801ILIfOnaZnVbZtvG WY8lxpcbw7SMHq6FwJ49nJDAymQ90vvC0WZgZp9HyDySJe992bugrza3g6n7iBnJz0UoAG w84DpIkymCKb9dXilliabIbJWZtClxU= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=NQkUDlar; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf11.hostedemail.com: domain of 3NpAEZgcKCJ8XT9MBGTFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--yuanchu.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3NpAEZgcKCJ8XT9MBGTFNNFKD.BNLKHMTW-LLJU9BJ.NQF@flex--yuanchu.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711575095; a=rsa-sha256; cv=none; b=xee83UKABHbW9s+cVm1mGQxWc+WF++zR2KhzeitFQW9wqYmVu/Knm9BT8Mflylx3tNwh2D CQRAe7DFfYmGIG+V9iOqkjLvBQa3UDuNf70dnj6Kt2xcGWjIwfJsyv65H8jvALqoRin3Pi t3YNEjI/O7dTn7Weu+IJeqIjgt2My9s= Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-60a61b31993so5660107b3.1 for ; Wed, 27 Mar 2024 14:31:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1711575094; x=1712179894; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=CyJDixqg00pkxMZu+nlTfhoUxwjhkmzgxYkGofwS92E=; b=NQkUDlarx3h8QvQ2w3ETsg70nrrygtiQ5wUfgdJ6M7GYeqpJhxHKNff+T4Xhg9OJyy +dvrzOYog/EW1HvQp8bi+Mf0m1iqVDHhH2lukWHiRg7H5e+LlVFmuzRnZaGx+/JnsCbu Cp8lhTMXMsXadXqsOL7GFj7hUYSRFhzYK/VBztDrXn8dllFJU/bPe9JFfi94Wutgi8TC rXmVw0G5qjldM0F8523OEvFpDgTCKZrFe4GMM41Qji8PEnqt6s5UF3+JZvd1ju5vTwDq vP/UxcIbCcpgA6JVt+3KH6DcuT0i/Y1M/RnUBO7wCXtnxZg64o6kE+dGYZiQsasHaadV r+Hg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711575094; x=1712179894; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=CyJDixqg00pkxMZu+nlTfhoUxwjhkmzgxYkGofwS92E=; b=o59ASd8mMA0Uyd42II35Hkrq3t3B5L391El9jV8W4+AVbVAueuriOZpBCwyqma6yvJ 6ymuTy92W9AG86cq75q+CgNZUi4iA+DbhbFKXDrelKAaCtJb1vS0g0BPMiwZPRnwNTUk WYZUWdDaG9NHXW0JCvvBdQG/3kEVHOSRYKCdFX5Z24PsqTfutH9BDeNssJnrxAOHmTB4 W8ufTRWXTC4sbmO9mdEqhDn7Ux+5ngylbl8rpZZwvxQ6jcZFj7ltYEIySrEZ4UrdQqJU 7huknnZ8BS/pgE6gJaK9Ad6I6hu+a5HM32umPv64NinAEYK3g14eVRvB/Xc6wms+EFtU gcaQ== X-Forwarded-Encrypted: i=1; AJvYcCXGkyuqoz7f4tbgu1dNHlf/TkvS3WAhEcdIKoa+ZgVH/ft1sksEzNgE0UX6qnXUGiMMljc1peoZHNIOiK3KGznwedc= X-Gm-Message-State: AOJu0YxhC5rf3V+s2hm5oSTtb9UiaBbjahTCarGA2vaVr0nTcH0It+U/ UDhoSTD6YJLBp4mAbCMu0C/yhymCWkef6/gnqqJjd9hnsUXmGOBL9ePTyLcC/RNYQ6SrZ701cjZ CRJvcXQ== X-Google-Smtp-Source: AGHT+IGSGqv5KUSJmM8bk0cJQQF7BQEKotxJoozxsxA3rktWM2RtKyv04koLrGenDZXEe0nH0BWimglXW+HF X-Received: from yuanchu-desktop.svl.corp.google.com ([2620:15c:2a3:200:6df3:ef42:a58e:a6b1]) (user=yuanchu job=sendgmr) by 2002:a0d:ca8b:0:b0:60a:e67:2ed0 with SMTP id m133-20020a0dca8b000000b0060a0e672ed0mr212398ywd.9.1711575094066; Wed, 27 Mar 2024 14:31:34 -0700 (PDT) Date: Wed, 27 Mar 2024 14:31:02 -0700 In-Reply-To: <20240327213108.2384666-1-yuanchu@google.com> Mime-Version: 1.0 References: <20240327213108.2384666-1-yuanchu@google.com> X-Mailer: git-send-email 2.44.0.396.g6e790dbe36-goog Message-ID: <20240327213108.2384666-4-yuanchu@google.com> Subject: [RFC PATCH v3 3/8] mm: use refresh interval to rate-limit workingset report aggregation From: Yuanchu Xie To: David Hildenbrand , "Aneesh Kumar K.V" , Khalid Aziz , Henry Huang , Yu Zhao , Dan Williams , Gregory Price , Huang Ying Cc: Wei Xu , David Rientjes , Greg Kroah-Hartman , "Rafael J. Wysocki" , Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Shuah Khan , Yosry Ahmed , Matthew Wilcox , Sudarshan Rajagopalan , Kairui Song , "Michael S. Tsirkin" , Vasily Averin , Nhat Pham , Miaohe Lin , Qi Zheng , Abel Wu , "Vishal Moola (Oracle)" , Kefeng Wang , Yuanchu Xie , linux-kernel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kselftest@vger.kernel.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: F379A40016 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: q1b55urh13pmm1j75mwyjnh9tbfamkmx X-HE-Tag: 1711575094-219755 X-HE-Meta: U2FsdGVkX19YJisw+2/0cny/9Qv3r1uw3CU8xXXunlaLNoFn435lmyR7Zn7SzLkE6Vq6/fqLDXCMAAF0fL+Tj+JUxOOe1hlCw4cHW95pzPhizXoa/Mo9cXUliCixZ0tQGso+Ym/1HDFcr6tBAhZi7t4IEesB1GhK/AzonBuB8vb7rNPOlCht2yJParU2uhRDR3grSEXgW2km0V+5Wxc0HwX39ffbrXIALWT0S8qE3jX3jN7bdIXyE1CBMgN2T938SUKSfmsoJhYa6NFO5xHYU75xhgdLOaz3ysqgqTecG3vp8cF9Km8p1CoAYfkIX0qvOjCLaFc16RHBI24cdcaDsxiCWV1nCHhXj+/ZqYeVJKIHxdY+my+BPUKTbcx+qqulum6XALIWwsaXvXUYrrxR7AG3FlmperXyLtJdDVQdc8wsP76PEl/SNmCeuC3EpH6ymj2AiQjaYz7c/7meW7YtVUuOnajQCD52p850/fAgQkXBm3PYRgluX5tApzUIrUpB5SxBFLyVL80mFYHO0x5xCx8JakA9XDUFNfuqRkI/qbueDz0SgWdyGQiKh+eFdwZS139Crl/rbVySdObj7AUY6mq3JNrg4DD3rLbQoY1njAClkxNsS6EMTu/H31VtZZMQvulLBezjiui27reHY+dVy7xHJxIfKinV2w789kdVxwmx6MZiSyCT1Ttma/QjnPwrlt0SFJCFOcARf9kJSLoSc9K0V8yviLYbwW6PsIwN8SfxJjPW3UaAIJ8p1kYna8dYyHOujBDJConzk+ds6491IvvcCED8kKhFq7QiuyRhu3jEthq26nlSf3zY/tFNsuJno5OL6PboS6YdwmRCsaYF91czWSket8t5RUsf+4RJQkXjj5Vq0LbJL0qiDxEWcUI/3xYBCM7n6pNpWbWiTc6luIvZ1fV9wmXDBEdSsAyPVFSeQWx1U6iCkTUdYygZJLonPKC5+OeurQaM5pMayKo fvWTqmZy wGmRkGUK1/UYFN4p1sxxFxQSZ14jDsSkfOgqitjzrFLOUFkzX4WJq1kf4YdVzm73TfSMTUmYnYlJiQutdhMaWUtrg3wgKBikV75A6x8zYKxh3W+co6vOZWbfQxJTpSnhVXbCQOiudn3TixbVBJYGntp9uJIVVlP8eFmDd8IMpIi8ASgVxIVejy9Fn+Obq29ipTYl1IfYKh4Ji92sk1QpnjKN3YCTgRv8N0cF459QE4TbUPgL1kRa9JO3c6ZR0Y02YPzVRWVGgOHUq+9XnHIyWaOzuw+sQEpLMSKynNPErfiwkj5vv8PWjdoXkx1RctfWSAJ1nxoHAbbaFY9V4mtGg/dQC6Dj6VaiNiN3HS/X2knMo0IS7uLCIkta7SETRYdSTF1Rz43KjsXGwkNO0HlP4Mu4QKnnvEeWztqQdpKl/eivw8nKTk9x2IBMkSrXs+oa7vqrJidk3wD1Xc5n6jEwxwkpDq+yOx6vzekkQeCVEYtlrkb8AezkIbokS09XH51pUpGV0Nfh0S1/PXOWQOYThDNzg4eb/7n/xFZSDTww3t4Az3G+q1PV3Fn6urBg2ms8wbk//20CJD4YD7i63S93Jl/VJF1sXFbwAhUnLX1jAQD4cveYGaek+C8Gva4sRyDE5Ch3i6Hb2jIKrRNo1TB7eJfhQPcsKUNNhCeSZA3ew6RoJtgE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The refresh interval is a rate limiting factor to workingset page age histogram reads. When a workingset report is generated, a timestamp is noted, and the same report will be read until it expires beyond the refresh interval, at which point a new report is generated. Sysfs interface /sys/devices/system/node/nodeX/workingset_report/refresh_interval time in milliseconds specifying how long the report is valid for Signed-off-by: Yuanchu Xie --- include/linux/workingset_report.h | 1 + mm/internal.h | 2 +- mm/vmscan.c | 27 ++++++++------ mm/workingset_report.c | 58 ++++++++++++++++++++++++++----- 4 files changed, 69 insertions(+), 19 deletions(-) diff --git a/include/linux/workingset_report.h b/include/linux/workingset_report.h index 0de640cb1ef0..23d2ae747a31 100644 --- a/include/linux/workingset_report.h +++ b/include/linux/workingset_report.h @@ -35,6 +35,7 @@ struct wsr_page_age_histo { }; struct wsr_state { + unsigned long refresh_interval; /* breakdown of workingset by page age */ struct mutex page_age_lock; struct wsr_page_age_histo *page_age; diff --git a/mm/internal.h b/mm/internal.h index 5e0caba64ee4..151f09c6983e 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -210,7 +210,7 @@ extern void reclaim_throttle(pg_data_t *pgdat, enum vmscan_throttle_state reason * in mm/wsr.c */ /* Requires wsr->page_age_lock held */ -void wsr_refresh_scan(struct lruvec *lruvec); +void wsr_refresh_scan(struct lruvec *lruvec, unsigned long refresh_interval); #endif /* diff --git a/mm/vmscan.c b/mm/vmscan.c index b694d80ab2d1..5f04a04f5261 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -5620,7 +5620,7 @@ late_initcall(init_lru_gen); * workingset reporting ******************************************************************************/ #ifdef CONFIG_WORKINGSET_REPORT -void wsr_refresh_scan(struct lruvec *lruvec) +void wsr_refresh_scan(struct lruvec *lruvec, unsigned long refresh_interval) { DEFINE_MAX_SEQ(lruvec); struct scan_control sc = { @@ -5633,15 +5633,22 @@ void wsr_refresh_scan(struct lruvec *lruvec) }; unsigned int flags; - set_task_reclaim_state(current, &sc.reclaim_state); - flags = memalloc_noreclaim_save(); - /* - * setting can_swap=true and force_scan=true ensures - * proper workingset stats when the system cannot swap. - */ - try_to_inc_max_seq(lruvec, max_seq, &sc, true, true); - memalloc_noreclaim_restore(flags); - set_task_reclaim_state(current, NULL); + if (refresh_interval) { + int gen = lru_gen_from_seq(max_seq); + unsigned long birth = READ_ONCE(lruvec->lrugen.timestamps[gen]); + + if (time_is_before_jiffies(birth + refresh_interval)) { + set_task_reclaim_state(current, &sc.reclaim_state); + flags = memalloc_noreclaim_save(); + /* + * setting can_swap=true and force_scan=true ensures + * proper workingset stats when the system cannot swap. + */ + try_to_inc_max_seq(lruvec, max_seq, &sc, true, true); + memalloc_noreclaim_restore(flags); + set_task_reclaim_state(current, NULL); + } + } } #endif /* CONFIG_WORKINGSET_REPORT */ diff --git a/mm/workingset_report.c b/mm/workingset_report.c index 98cdaffcb6b4..370e7d355604 100644 --- a/mm/workingset_report.c +++ b/mm/workingset_report.c @@ -181,7 +181,8 @@ static void collect_page_age(struct wsr_page_age_histo *page_age, /* First step: hierarchically scan child memcgs. */ static void refresh_scan(struct wsr_state *wsr, struct mem_cgroup *root, - struct pglist_data *pgdat) + struct pglist_data *pgdat, + unsigned long refresh_interval) { struct mem_cgroup *memcg; @@ -189,7 +190,7 @@ static void refresh_scan(struct wsr_state *wsr, struct mem_cgroup *root, do { struct lruvec *lruvec = mem_cgroup_lruvec(memcg, pgdat); - wsr_refresh_scan(lruvec); + wsr_refresh_scan(lruvec, refresh_interval); cond_resched(); } while ((memcg = mem_cgroup_iter(root, memcg, NULL))); } @@ -231,16 +232,25 @@ static void refresh_aggregate(struct wsr_page_age_histo *page_age, bool wsr_refresh_report(struct wsr_state *wsr, struct mem_cgroup *root, struct pglist_data *pgdat) { - struct wsr_page_age_histo *page_age; + struct wsr_page_age_histo *page_age = NULL; + unsigned long refresh_interval = READ_ONCE(wsr->refresh_interval); if (!READ_ONCE(wsr->page_age)) return false; - refresh_scan(wsr, root, pgdat); + if (!refresh_interval) + return false; + mutex_lock(&wsr->page_age_lock); page_age = READ_ONCE(wsr->page_age); - if (page_age) - refresh_aggregate(page_age, root, pgdat); + if (!page_age) + goto unlock; + if (time_is_after_jiffies(page_age->timestamp + refresh_interval)) + goto unlock; + refresh_scan(wsr, root, pgdat, refresh_interval); + refresh_aggregate(page_age, root, pgdat); + +unlock: mutex_unlock(&wsr->page_age_lock); return !!page_age; } @@ -259,6 +269,35 @@ static struct wsr_state *kobj_to_wsr(struct kobject *kobj) return &mem_cgroup_lruvec(NULL, kobj_to_pgdat(kobj))->wsr; } +static ssize_t refresh_interval_show(struct kobject *kobj, + struct kobj_attribute *attr, char *buf) +{ + struct wsr_state *wsr = kobj_to_wsr(kobj); + unsigned int interval = READ_ONCE(wsr->refresh_interval); + + return sysfs_emit(buf, "%u\n", jiffies_to_msecs(interval)); +} + +static ssize_t refresh_interval_store(struct kobject *kobj, + struct kobj_attribute *attr, + const char *buf, size_t len) +{ + unsigned int interval; + int err; + struct wsr_state *wsr = kobj_to_wsr(kobj); + + err = kstrtouint(buf, 0, &interval); + if (err) + return err; + + WRITE_ONCE(wsr->refresh_interval, msecs_to_jiffies(interval)); + + return len; +} + +static struct kobj_attribute refresh_interval_attr = + __ATTR_RW(refresh_interval); + static ssize_t page_age_intervals_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) { @@ -267,7 +306,7 @@ static ssize_t page_age_intervals_show(struct kobject *kobj, mutex_lock(&wsr->page_age_lock); - if (!!wsr->page_age) { + if (wsr->page_age) { int i; int nr_bins = wsr->page_age->bins.nr_bins; @@ -373,7 +412,10 @@ static ssize_t page_age_show(struct kobject *kobj, struct kobj_attribute *attr, static struct kobj_attribute page_age_attr = __ATTR_RO(page_age); static struct attribute *workingset_report_attrs[] = { - &page_age_intervals_attr.attr, &page_age_attr.attr, NULL + &refresh_interval_attr.attr, + &page_age_intervals_attr.attr, + &page_age_attr.attr, + NULL }; static const struct attribute_group workingset_report_attr_group = { -- 2.44.0.396.g6e790dbe36-goog