From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8678EE95A8E for ; Sun, 8 Oct 2023 10:00:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 22ED86B0299; Sun, 8 Oct 2023 06:00:13 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1DE4B6B029A; Sun, 8 Oct 2023 06:00:13 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 07FE26B029B; Sun, 8 Oct 2023 06:00:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id E66DF6B0299 for ; Sun, 8 Oct 2023 06:00:12 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id B74E3A0177 for ; Sun, 8 Oct 2023 10:00:12 +0000 (UTC) X-FDA: 81321848664.19.CE732F5 Received: from mail-oo1-f49.google.com (mail-oo1-f49.google.com [209.85.161.49]) by imf06.hostedemail.com (Postfix) with ESMTP id D6907180008 for ; Sun, 8 Oct 2023 10:00:10 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=transsion-corp-partner-google-com.20230601.gappssmtp.com header.s=20230601 header.b=FdV6pvBP; spf=pass (imf06.hostedemail.com: domain of lincheng.yang@transsion.corp-partner.google.com designates 209.85.161.49 as permitted sender) smtp.mailfrom=lincheng.yang@transsion.corp-partner.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696759210; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=uhruBmH/7vx7sF4+aMlxk61hZnogrYd+YWcZxSg4QF4=; b=7Tp4oMjtjxy+bTaBRQdGgw4A3onKxSfyxfZ3WIfdXfKjL1Dxw7pUudpclDgWAY+HR2o/xa 7GS+gfBaBBetZ2MgLuidWPl+tcsXR+oChhsbzhHtmoz6iAUpfoyeE0GtJ6l6C5H52L7ZK1 lGpG3l/ZBrIn1Q4ceiZ00bxWUEBPNQA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696759210; a=rsa-sha256; cv=none; b=A5l2z8csKf0up9K2wmAG2VWQW7qkSa7pOa5M9/H80ZQBx+/wXZzJO0p7uHTTY90yyEk4CP KP/5ffMa4ylbkcOo++wNHleEHr780bBoqrkcoEw292ZxsqUtKjLZ7PE/v9yp5jfw7hZWTH 4MbKH4WRdXvdx7TrrwRBPkC+4GEPuhA= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=transsion-corp-partner-google-com.20230601.gappssmtp.com header.s=20230601 header.b=FdV6pvBP; spf=pass (imf06.hostedemail.com: domain of lincheng.yang@transsion.corp-partner.google.com designates 209.85.161.49 as permitted sender) smtp.mailfrom=lincheng.yang@transsion.corp-partner.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-oo1-f49.google.com with SMTP id 006d021491bc7-57b6c7d0cabso2189160eaf.1 for ; Sun, 08 Oct 2023 03:00:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=transsion-corp-partner-google-com.20230601.gappssmtp.com; s=20230601; t=1696759210; x=1697364010; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uhruBmH/7vx7sF4+aMlxk61hZnogrYd+YWcZxSg4QF4=; b=FdV6pvBPfDjehLfRpimNb32OXl80ke4iiehpRm1MolMwU/thcn3/c2nQVQJUKc9P/2 N6PrkGt9xoL1J+V13tM44yahfDqel6EiddenYBhr+f5b2ZT0RFjdvynpZhgHO2sPH33g pb0qR/qTRgP0ygs3ldLcVxC/4Qa+jdJgGO3GNrsSnK940JLEr2oj5myxMUuQTMziu+JZ zk3Evmimgx9AS19k0LqcCOHuzVrAd75UAVHIcHYw3oR+bNMxttz6PvgVFRdAZvLUgtSF 9jF1KhnM1S0tcHf7X82o5PQcCRSQt0TkDUuJbppL+fGUDX5sKtriaZl0MR3rNnGiN4q0 wP8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696759210; x=1697364010; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uhruBmH/7vx7sF4+aMlxk61hZnogrYd+YWcZxSg4QF4=; b=tsjvvIRIfmBbGcYP0oKMYhQZeB1GBnSBrDXxCctiJ/n/CIiPDg7EAqsQA6IKxXtNwn ZKtcgR8Ls06NYQ4E3t9DuL/ChpABB8AcW3faWp35aixAeiZFi+go8gS5Gwe2Oqmo9Y0i EMoGTEg8ebsvo+7cwlA5+K83ERV7e2MqGiRKkQE5/AKZyTPKvgqSJvmAIwdyoQGR0aSz cPVVHasSeherYgEUwmE62S7b6TrpdfHUBWsT4dEw0JoznOUkLyI7C83yqSEHHaNngit9 n7z/ztlXkvJ05XYV/ijTdNX8KPN5E6pJq6VnbYVxMc07Z66nGvHS8eeWkKmg50qRFTfF cX5g== X-Gm-Message-State: AOJu0YwqR7/9PFY9Jd2UC0XMLkIRe02RMjkuPUnDPb+rM5abF6ZN7+h5 k/neEGbpf3guE5vrzehcTuaD/w== X-Google-Smtp-Source: AGHT+IHtkKnrm3jMctPq5VJC/qCzlOyj5CuPQEncbsTHJCX7WO3gKj247iDJ9X2uxpVhIZ21mNbn0w== X-Received: by 2002:a05:6870:6488:b0:1d1:39d8:da1a with SMTP id cz8-20020a056870648800b001d139d8da1amr16056418oab.15.1696759209770; Sun, 08 Oct 2023 03:00:09 -0700 (PDT) Received: from linchengyang-VB.. ([103.40.76.4]) by smtp.gmail.com with ESMTPSA id u9-20020a17090282c900b001c72c07c9d9sm7126343plz.308.2023.10.08.03.00.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 08 Oct 2023 03:00:09 -0700 (PDT) From: Lincheng Yang X-Google-Original-From: Lincheng Yang To: akpm@linux-foundation.org, rostedt@goodmis.org, mhiramat@kernel.org, willy@infradead.org, hughd@google.com, peterx@redhat.com, mike.kravetz@oracle.com, jgg@ziepe.ca, surenb@google.com, steven.price@arm.com, pasha.tatashin@soleen.com, kirill.shutemov@linux.intel.com, yuanchu@google.com, david@redhat.com, mathieu.desnoyers@efficios.com, dhowells@redhat.com, shakeelb@google.com, pcc@google.com, tytso@mit.edu, 42.hyeyoo@gmail.com, vbabka@suse.cz, catalin.marinas@arm.com, lrh2000@pku.edu.cn, ying.huang@intel.com, mhocko@suse.com, vishal.moola@gmail.com, yosryahmed@google.com, findns94@gmail.com, neilb@suse.de Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, wanbin.wang@transsion.com, chunlei.zhuang@transsion.com, jinsheng.zhao@transsion.com, jiajun.ling@transsion.com, dongyun.liu@transsion.com, Lincheng Yang Subject: [RFC PATCH 4/5] mm: add page implyreclaim flag Date: Sun, 8 Oct 2023 17:59:23 +0800 Message-Id: <20231008095924.1165106-5-lincheng.yang@transsion.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231008095924.1165106-1-lincheng.yang@transsion.com> References: <20231008095924.1165106-1-lincheng.yang@transsion.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Stat-Signature: nsmcbrq88gpt768twzgr1n33asw4edrd X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: D6907180008 X-Rspam-User: X-HE-Tag: 1696759210-910171 X-HE-Meta: U2FsdGVkX19oQIUIvYlSIWJsnrkI3LLWcBkmBCuKFYOo2iAXQrLEbS3yddvCfut1wHxDImNlmCEHlQkmcHEfmSEl3/IewGeMTGXgV0wNsyoE3OhA0A1cqeprgY/zVR1ViBCa0SABuTPn2FzlQlY1wJFSR9gR2BfeegStCty2radYNNfVLAyJ1d9/upJmttwY0Cjw8BhrG9YyqWfSBHu3K8OZZn3vd23QqOsiMUprIXZTxX/MvNqu+owNdLTOR55fW7yxtOFSTpTDM12E4PvMmyZKTC+Xs3GWw/DHmgbEdmt37rzcJrzOwzuOyk9Afef3+bTiBnFnAqdrNHkNxtbXRvSAs4Hvd+cMJf0uwR6civn1QyPXKr4xG4DtyUJTA/5vpWuJncBb4A7H3CDMYecrLjRSpSwSyOnFOTZi0vDPMJfQUuB58reliknGROL9dHC8iLUgF/CEd2hRQ6dDMV6wgWOl8X92Wm21ClMsCTQgcMID/0w0XeTtLmqUZT+vIig51LVVoamzS8OUBhsBdo3nEWm/3FUd6RZlWdHY+HG3LYiSyuPYMQ65yZILK+O3Dj+BStZW46a+9pEcP9YcMDLkXvfONfx318jjbCUtEEqYi26oyyVqu6emc2kV+vxy57UhQaqOcJBr10KzlU6dJFgw1DeM01WPziFq9GOEEvycxHTXwnbzYAbdQdT2euNOaWborawAr+h2h7TZf/2qVHOwOWmfUO/H4IM2QPPIw51MWSob66C86hwVN64PiG+s50l3AovHO5CgErGnFooh4b8joOn9y/r1wzmIXm0XnwWS/dFF5yPXIkZ0V83cRJERJANzGV/Ud69xHGnqlXucsSJVmVJ4VXPt/LWZgkPBgsdQqxufrNa/WNt1cmPVUpyaAy88UVfulVG9O3exmTv314SD8Ln3o76nN+cESR+2ECcy4SR9GJKPAiFxbSfyOFpogjrsELulcb8pyySwJ5e37Pw Lg16SmZb WjlDmKl5d3o7mNolSqfp4XuN1zJZ3QczVlBO4ZDmgDjRNAbuOKYDbswp8NlGOw6FKc6VQRBm3C4UFuXlGKiLqY8LpLIUN0B1OeMsudMzTsezfrj+nClxTbNZlBvtholJdv/+8nypZWj8Az1dsNjz586EK1ILwFaQ57XBA8z/DuBbLR04RLtgRKn+rSU6W7lA90smYU3O92qd/fnbd9hZCEh5Zfbq+nl7+RJbY2hbdovHqu1PiQMfcYICAEMoeo9+6dB/DbYULoW4xWyBgvsxQhnr+bYj06wNyRZM70R75pE4+hW8Q6Bk1dohNSmHlaiVXC7r6GbuevDUK8ViuXqdS+s5goPY2kC0a8g0Txt0MT9+e+cJVVUfuQTSELtVB0/5J+CYwFx9ZRLqGpQuCDsG8I6mEZbp0Znh/j6nQgM/ZqMoJdXKe1cb7ZJWRsuUXTlg6zG8N0rqUzhw5rMLpsESa/ScdhcO11bbEo0rq5tN2D3870fIQ6wQEP7yT7Mxs6ZrZhlvoi3xnZti4YdGY3OBF9F2zg/iIIIr7S1RWfTUPCEpKM4w= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add implyrecalim flag means that the page is reclaim from the user advise. If the number of restore times for these implyrecalim pages exceeds workingset_restore_limit, it means they are frequently used and are hot pages. Otherwise, continue to determine whether it has been restored. If so, it means that it is frequently used also and belongs to the hot page. Signed-off-by: Lincheng Yang --- include/linux/mmzone.h | 1 + include/linux/page-flags.h | 3 ++ include/trace/events/mmflags.h | 3 +- mm/madvise.c | 1 + mm/migrate.c | 2 ++ mm/swapfile.c | 64 +++++++++++++++++++++++++++++++++- mm/vmscan.c | 3 ++ 7 files changed, 75 insertions(+), 2 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 5e50b78d58ea..b280e6b0015a 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -626,6 +626,7 @@ struct lruvec { atomic_long_t nonresident_age; /* Refaults at the time of last reclaim cycle */ unsigned long refaults[ANON_AND_FILE]; + unsigned long restores[ANON_AND_FILE]; /* Various lruvec state flags (enum lruvec_flags) */ unsigned long flags; #ifdef CONFIG_LRU_GEN diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index a2c83c0100aa..4a1278851d4b 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -138,6 +138,7 @@ enum pageflags { #endif PG_hot, PG_cold, + PG_implyreclaim, __NR_PAGEFLAGS, PG_readahead = PG_reclaim, @@ -482,6 +483,8 @@ PAGEFLAG(Hot, hot, PF_HEAD) TESTCLEARFLAG(Hot, hot, PF_HEAD) PAGEFLAG(Cold, cold, PF_HEAD) TESTCLEARFLAG(Cold, cold, PF_HEAD) +PAGEFLAG(Implyreclaim, implyreclaim, PF_HEAD) + TESTCLEARFLAG(Implyreclaim, implyreclaim, PF_HEAD) __PAGEFLAG(Slab, slab, PF_NO_TAIL) PAGEFLAG(Checked, checked, PF_NO_COMPOUND) /* Used by some filesystems */ diff --git a/include/trace/events/mmflags.h b/include/trace/events/mmflags.h index f266f92c41c6..ee014f955aef 100644 --- a/include/trace/events/mmflags.h +++ b/include/trace/events/mmflags.h @@ -120,7 +120,8 @@ DEF_PAGEFLAG_NAME(swapbacked), \ DEF_PAGEFLAG_NAME(unevictable), \ DEF_PAGEFLAG_NAME(hot), \ - DEF_PAGEFLAG_NAME(cold) \ + DEF_PAGEFLAG_NAME(cold), \ + DEF_PAGEFLAG_NAME(implyreclaim) \ IF_HAVE_PG_MLOCK(mlocked) \ IF_HAVE_PG_UNCACHED(uncached) \ IF_HAVE_PG_HWPOISON(hwpoison) \ diff --git a/mm/madvise.c b/mm/madvise.c index a5c19bb3f392..199b48dfa8c5 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -518,6 +518,7 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd, } else { list_add(&folio->lru, &folio_list); folio_set_cold(folio); + folio_set_implyreclaim(folio); } } } else diff --git a/mm/migrate.c b/mm/migrate.c index 9f97744bb0a8..691b4f7bf1ae 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -565,6 +565,8 @@ void folio_migrate_flags(struct folio *newfolio, struct folio *folio) folio_set_hot(newfolio); if (folio_test_cold(folio)) folio_set_cold(newfolio); + if (folio_test_implyreclaim(folio)) + folio_set_implyreclaim(newfolio); if (folio_test_checked(folio)) folio_set_checked(newfolio); /* diff --git a/mm/swapfile.c b/mm/swapfile.c index 5378f70d330d..629e6a291e9b 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -105,6 +105,8 @@ static atomic_t proc_poll_event = ATOMIC_INIT(0); atomic_t nr_rotate_swap = ATOMIC_INIT(0); +static unsigned int workingset_restore_limit; + static struct swap_info_struct *swap_type_to_swap_info(int type) { if (type >= MAX_SWAPFILES) @@ -120,11 +122,33 @@ static inline bool swap_info_hot(struct swap_info_struct *si) bool swap_folio_hot(struct folio *folio, bool hotness) { + struct lruvec *lruvec; + struct mem_cgroup *memcg; + unsigned long restores; + int delta; + if (hotness) return true; - if (folio_test_swapbacked(folio) && folio_test_hot(folio)) + if (folio_test_swapbacked(folio) && folio_test_hot(folio)) { + folio_clear_implyreclaim(folio); + return true; + } + + rcu_read_lock(); // prevent writing from delaying reading + memcg = folio_memcg_rcu(folio); + rcu_read_unlock(); + + lruvec = mem_cgroup_lruvec(memcg, folio_pgdat(folio)); + restores = lruvec_page_state(lruvec, WORKINGSET_RESTORE_ANON); + delta = restores - lruvec->restores[WORKINGSET_ANON]; + + if (folio_test_clear_implyreclaim(folio)) { + if (delta > workingset_restore_limit) + return true; + } else if (delta) { return true; + } if (folio_test_cold(folio)) folio_clear_cold(folio); @@ -2715,9 +2739,47 @@ static const struct proc_ops swaps_proc_ops = { .proc_poll = swaps_poll, }; +static ssize_t workingset_restore_limit_write(struct file *file, + const char __user *ubuf, + size_t count, loff_t *pos) +{ + unsigned int val; + int ret; + + ret = kstrtouint_from_user(ubuf, count, 10, &val); + if (ret) + return ret; + + workingset_restore_limit = val; + + return count; +} + +static int workingset_restore_limit_show(struct seq_file *m, void *v) +{ + seq_printf(m, "%d\n", workingset_restore_limit); + + return 0; +} + +static int workingset_restore_limit_open(struct inode *inode, struct file *file) +{ + return single_open(file, workingset_restore_limit_show, NULL); +} + +const struct proc_ops workingset_restore_limit_fops = { + .proc_open = workingset_restore_limit_open, + .proc_read = seq_read, + .proc_lseek = seq_lseek, + .proc_release = seq_release, + .proc_write = workingset_restore_limit_write, +}; + static int __init procswaps_init(void) { proc_create("swaps", 0, NULL, &swaps_proc_ops); + proc_create("workingset_restore_limit", S_IALLUGO, NULL, &workingset_restore_limit_fops); + return 0; } __initcall(procswaps_init); diff --git a/mm/vmscan.c b/mm/vmscan.c index 11d175d9fe0c..8107f8d86d7f 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -6805,6 +6805,9 @@ static void snapshot_refaults(struct mem_cgroup *target_memcg, pg_data_t *pgdat) target_lruvec->refaults[WORKINGSET_ANON] = refaults; refaults = lruvec_page_state(target_lruvec, WORKINGSET_ACTIVATE_FILE); target_lruvec->refaults[WORKINGSET_FILE] = refaults; + + refaults = lruvec_page_state(target_lruvec, WORKINGSET_RESTORE_ANON); + target_lruvec->restores[WORKINGSET_ANON] = refaults; } /* -- 2.34.1