From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E435C54E94 for ; Thu, 26 Jan 2023 17:08:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D4F846B0072; Thu, 26 Jan 2023 12:08:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D01BC8E0002; Thu, 26 Jan 2023 12:08:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BC80E8E0001; Thu, 26 Jan 2023 12:08:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id ADABA6B0072 for ; Thu, 26 Jan 2023 12:08:45 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 86BDD120DC2 for ; Thu, 26 Jan 2023 17:08:45 +0000 (UTC) X-FDA: 80397584610.22.E64E0E1 Received: from mail-qv1-f46.google.com (mail-qv1-f46.google.com [209.85.219.46]) by imf21.hostedemail.com (Postfix) with ESMTP id CA3031C001D for ; Thu, 26 Jan 2023 17:08:43 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=KfXi+HE0; spf=pass (imf21.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.219.46 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674752923; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2Xa2N7Ivh7dt7l8G2gaP7xy0D45O0idREEQejp58kyY=; b=sULOOwdKNN7hnqigTRnOz8urUQhq0Fb3wXqoD7e/36nqROrZiSoRK62aUrrlf3zVfAxi5w wArF3hSDc8CvZFFzkO8TgClZJ9YJuVZCtAB9tojZbe1b6APEDXqrIG+sAwGwNjnYWAa4Pg kloRmMxj4/5opEsYEF9tz5+pgxIVioA= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=KfXi+HE0; spf=pass (imf21.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.219.46 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674752923; a=rsa-sha256; cv=none; b=NhkwgmoYQlY+o5wcqO1HSed9DL5rZJFg1VqB1yUN1viDUl2fHChO8pwiPW+nZk1sshGTcm oMC7gUrhSWX4KvAOUDxB/i6Ys++2EfWzzKwBHSOuh78T8w0RGGYVxy32pJ2z+oBwreZwhy uo1E/GJ4+F+NL5MZM3urv5mRVibw1oE= Received: by mail-qv1-f46.google.com with SMTP id k12so1909400qvj.5 for ; Thu, 26 Jan 2023 09:08:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=2Xa2N7Ivh7dt7l8G2gaP7xy0D45O0idREEQejp58kyY=; b=KfXi+HE0YksbS0l42GwHifDlm23mRKMFrrSURr01IMR9ArF9XRysIljCM8DlydjnNB jWk451dIU3wOuikcys5ESTKA0b9trbFzqZ8ctw9/LAiHXC2ovf+bgxGnrJvyMCkVo7pm p+m0eAO09K8UUADuOnT3LNe1N3tV+kbNhTVU9oAEJSoTLrvrqaqQkPlUvX4L23iG6mL7 WCwSs8869xpjn7aU7i48PXtb2vJiex7Eq8YHQuj6DpvbR+76r/ufI1zqDAiB+vkoiNGm /O5VpClS83TEoxvHMI13G+enmGJzGQWvBo7YAlKzljWsmHvy3SLeBldTrTv2WFPn9qbR ZHIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=2Xa2N7Ivh7dt7l8G2gaP7xy0D45O0idREEQejp58kyY=; b=lwwX5B1R2YUUuzkePecK7n7NevFLIEctyafgtZdaGXyeC4UR/56Zakiq8XRcmisltd swu8TpHEOJBPWRYskWwUdSXBU1PiisTeI8daWIDpaj5uJvA3/WdgbGp2XjPNUho6Tlxv J/Hk4udFLLCA17efKnSzH3EK9kthUIS5aZMzGTNFqT8flTFFzj2C0vxzmMI7KVt6dT8R LCYKqiCfzJIQU/a/qwNSqflmUq/iXZXpIqasckpwCu9Z1lVf2ZmzyIXRit1ImMQP+fwm Fk4XfRvJmuD0DETdIdGiM6y9JNEzH9RlUdadoeGSSmuywCtOZ+vtpdQj7Cs76J/DPuQG D2iQ== X-Gm-Message-State: AFqh2kok4cHxfT9XPnUVPIDgl530ujqWmYwsIp5ZUJhNcL9Zs56w6/nq YyKGcOLsrCVoeO/mi03eo3csXISpXld1k9E8yus= X-Google-Smtp-Source: AMrXdXt5QJSbsB4oqgWPTVg7geEsV9QGTheG7aGVF0OlUCn9aTsqx0JJmb888T1MXJzJIEX1JUvM3Fq0Lbajb0G3lrw= X-Received: by 2002:a05:6214:5f82:b0:534:252f:b091 with SMTP id ls2-20020a0562145f8200b00534252fb091mr1660515qvb.130.1674752922880; Thu, 26 Jan 2023 09:08:42 -0800 (PST) MIME-Version: 1.0 References: <20230124021118.154078-1-nphamcs@gmail.com> <20230124021118.154078-2-nphamcs@gmail.com> In-Reply-To: From: Nhat Pham Date: Thu, 26 Jan 2023 09:08:32 -0800 Message-ID: Subject: Re: [PATCH v7 1/3] workingset: refactor LRU refault to expose refault recency check To: Yu Zhao Cc: akpm@linux-foundation.org, hannes@cmpxchg.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, bfoster@redhat.com, willy@infradead.org, linux-api@vger.kernel.org, kernel-team@meta.com Content-Type: text/plain; charset="UTF-8" X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: fiy95m7gwdmpxmt8ushfkhyhxbjkhgrf X-Rspamd-Queue-Id: CA3031C001D X-HE-Tag: 1674752923-691141 X-HE-Meta: U2FsdGVkX19PFleu3t+VrsXe5YXqHoo5Fgycs3aYO9Zo4IUo2C/+6fjRq1fKfgYnKKVXz4WF2HeS1uWSlpOa2yOVg773Yo6cRp7DyDt2qC2D2PwuCbJ0ZESxoiGd8HITcm4bEl3PGd5Zh03YRUx0S1HibtLbXHo2kk6rXS9VHcVoF7M4jP6Q8PtadaseTSqJpKxkHaDC7ZaOZooDFfM1Tb8JGD6QhHqt9Jk1LnMF5PBdgiiE+1huHdCEjKCo3t1wC10qAyw1K+sGaedfacbvLeXMeuKI4qp3He1i82Gy0wgyuymRbxFdxMkzezPvkk0MyfPbyFyVvaCLHBKzRjLjosdHEiehFH3rLX7pkbMpB3lSLt0JVzVNsBy4Tf0j/VffvduOXlyTbE0IIoz52F+dC61mc31GnqUBE3YuUzPfxBQtgKxJWCa6cxA5v/AJpHXGvnvDUxrn+8FeyB8Ap2TIa4dQt+hMERn6UOQwcomoOX+VZEAoMOBA/x3v3jJkSSdsgFfiHQ1mOnM4nQLlAQGWrrUSuPwcf6ZF6yDcPOkaoLlgAoGsgTj8chE8PSo2cT6/xr080agwqmmopQz3viF84vrtQldsbtpaAFKpWwfzsEYALJJvvQs1L2otRqqZ8H5iR50HAr7w72RvhljZhkJuSNvqXvVrp/dAu2mTK9hyiOB30lUza1oa0xlvWAlYlawdBkFKat7u86INRgDP/1pqFtakoWTs0mSzUVT3hjur9N+tZ08RjMnFjcayCuouxyKWiaqeH+q8fdehM2xPcaAP7D+rV9/VSVKZKrj5dMq6W8dWUeK5+W3J8BtQXoTO/MgRlZJDocqYH/9jgsY2Y+IXvJw9Z1mHokMIPyIUDlH9Z/S5xa4qES1NKIir+mQbw+X6Mcd+mUFthD0o/ZldnhTA8TREU8aZKzOqrLRiiEFO4D9znkSgj1F7kXqiPjPEkopv5mWJuc0bDGZoAfZ/drD SNnnQB/0 Yx5/4LyJo9yGU6EnyCTxYbBLz8/CwAJQ2Z8TzuwBboEaunucqdQHF8uZ9lIMwjW9M4Iz1GOh4bIAfsX94Ggfb/G9784MjuPvVtvQYTVlxRHgF4FpqNQZVTPKsxwzLw5l4YorpBayDML9GuHQ4XYLvfQG6VPZXGmyP380Gd4LxYZX7RRpECCXUn1dGfJIL6jIRvnoZ8VUmrJT3cgCwYjHistWAtoEm737kJi3A2LrSAYFih6RUQn9cUzJIpU8ugWjFifO4g02pCn2P6vjVxtMHg4Rk2O5k/7e0VxgR3mIZL5A7Z2gZkDX7KO2zb8us384BDkKzhhNjDlDWaFYu3CFI9qKRREhB0l0U5zPhRI+kDiCbMRwsX5z8NBYMyb4YuPfvKvNiOek7jGlB5ufrkBJhjWi8SCaO2A4YJl9t X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Jan 25, 2023 at 1:13 PM Yu Zhao wrote: > > On Mon, Jan 23, 2023 at 7:11 PM Nhat Pham wrote: > > > > In preparation for computing recently evicted pages in cachestat, > > refactor workingset_refault and lru_gen_refault to expose a helper > > function that would test if an evicted page is recently evicted. > > > > Signed-off-by: Nhat Pham > > --- > > include/linux/swap.h | 1 + > > mm/workingset.c | 142 +++++++++++++++++++++++++++++-------------- > > 2 files changed, 97 insertions(+), 46 deletions(-) > > > > diff --git a/include/linux/swap.h b/include/linux/swap.h > > index a18cf4b7c724..dae6f6f955eb 100644 > > --- a/include/linux/swap.h > > +++ b/include/linux/swap.h > > @@ -361,6 +361,7 @@ static inline void folio_set_swap_entry(struct folio *folio, swp_entry_t entry) > > } > > > > /* linux/mm/workingset.c */ > > +bool workingset_test_recent(void *shadow, bool file, bool *workingset); > > void workingset_age_nonresident(struct lruvec *lruvec, unsigned long nr_pages); > > void *workingset_eviction(struct folio *folio, struct mem_cgroup *target_memcg); > > void workingset_refault(struct folio *folio, void *shadow); > > diff --git a/mm/workingset.c b/mm/workingset.c > > index 79585d55c45d..2f2d94867366 100644 > > --- a/mm/workingset.c > > +++ b/mm/workingset.c > > @@ -244,6 +244,30 @@ static void *lru_gen_eviction(struct folio *folio) > > return pack_shadow(mem_cgroup_id(memcg), pgdat, token, refs); > > } > > > > +/* > > + * Test if the folio is recently evicted. > > + * > > + * As a side effect, also populates the references with > > + * values unpacked from the shadow of the evicted folio. > > + */ > > +static bool lru_gen_test_recent(void *shadow, bool file, int *memcgid, > > + struct pglist_data **pgdat, unsigned long *token, bool *workingset) > > +{ > > + struct mem_cgroup *eviction_memcg; > > + struct lruvec *lruvec; > > + struct lru_gen_struct *lrugen; > > + unsigned long min_seq; > > + > > + unpack_shadow(shadow, memcgid, pgdat, token, workingset); > > + eviction_memcg = mem_cgroup_from_id(*memcgid); > > + > > + lruvec = mem_cgroup_lruvec(eviction_memcg, *pgdat); > > + lrugen = &lruvec->lrugen; > > + > > + min_seq = READ_ONCE(lrugen->min_seq[file]); > > + return (*token >> LRU_REFS_WIDTH) == (min_seq & (EVICTION_MASK >> LRU_REFS_WIDTH)); > > +} > > + > > static void lru_gen_refault(struct folio *folio, void *shadow) > > { > > int hist, tier, refs; > > @@ -258,23 +282,22 @@ static void lru_gen_refault(struct folio *folio, void *shadow) > > int type = folio_is_file_lru(folio); > > int delta = folio_nr_pages(folio); > > > > - unpack_shadow(shadow, &memcg_id, &pgdat, &token, &workingset); > > - > > - if (pgdat != folio_pgdat(folio)) > > - return; > > - > > rcu_read_lock(); > > > > + if (!lru_gen_test_recent(shadow, type, &memcg_id, &pgdat, &token, > > + &workingset)) > > + goto unlock; > > + > > memcg = folio_memcg_rcu(folio); > > if (memcg_id != mem_cgroup_id(memcg)) > > goto unlock; > > > > + if (pgdat != folio_pgdat(folio)) > > + return; > > + > > lruvec = mem_cgroup_lruvec(memcg, pgdat); > > lrugen = &lruvec->lrugen; > > - > > min_seq = READ_ONCE(lrugen->min_seq[type]); > > - if ((token >> LRU_REFS_WIDTH) != (min_seq & (EVICTION_MASK >> LRU_REFS_WIDTH))) > > - goto unlock; > > > > hist = lru_hist_from_seq(min_seq); > > /* see the comment in folio_lru_refs() */ > > @@ -306,6 +329,12 @@ static void *lru_gen_eviction(struct folio *folio) > > return NULL; > > } > > > > +static bool lru_gen_test_recent(void *shadow, bool file, int *memcgid, > > + struct pglist_data **pgdat, unsigned long *token, bool *workingset) > > +{ > > + return false; > > +} > > + > > static void lru_gen_refault(struct folio *folio, void *shadow) > > { > > } > > @@ -373,40 +402,31 @@ void *workingset_eviction(struct folio *folio, struct mem_cgroup *target_memcg) > > folio_test_workingset(folio)); > > } > > > > -/** > > - * workingset_refault - Evaluate the refault of a previously evicted folio. > > - * @folio: The freshly allocated replacement folio. > > - * @shadow: Shadow entry of the evicted folio. > > +/* > > + * Test if the folio is recently evicted by checking if > > + * refault distance of shadow exceeds workingset size. > > * > > - * Calculates and evaluates the refault distance of the previously > > - * evicted folio in the context of the node and the memcg whose memory > > - * pressure caused the eviction. > > + * As a side effect, populate workingset with the value > > + * unpacked from shadow. > > */ > > -void workingset_refault(struct folio *folio, void *shadow) > > +bool workingset_test_recent(void *shadow, bool file, bool *workingset) > > { > > - bool file = folio_is_file_lru(folio); > > struct mem_cgroup *eviction_memcg; > > struct lruvec *eviction_lruvec; > > unsigned long refault_distance; > > unsigned long workingset_size; > > - struct pglist_data *pgdat; > > - struct mem_cgroup *memcg; > > - unsigned long eviction; > > - struct lruvec *lruvec; > > unsigned long refault; > > - bool workingset; > > int memcgid; > > - long nr; > > + struct pglist_data *pgdat; > > + unsigned long eviction; > > > > - if (lru_gen_enabled()) { > > - lru_gen_refault(folio, shadow); > > - return; > > - } > > + if (lru_gen_enabled()) > > + lru_gen_test_recent(shadow, file, &memcgid, &pgdat, &eviction, > > + workingset); > > Missing "return", which was correctly handled in your v2, btw. Oops copy-paste hazard. I'll fix this...