From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 58761C54E94 for ; Wed, 25 Jan 2023 21:13:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E85C46B0074; Wed, 25 Jan 2023 16:13:39 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DE8506B0075; Wed, 25 Jan 2023 16:13:39 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C612E6B0078; Wed, 25 Jan 2023 16:13:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B0B276B0074 for ; Wed, 25 Jan 2023 16:13:39 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 9000C408B8 for ; Wed, 25 Jan 2023 21:13:39 +0000 (UTC) X-FDA: 80394572958.10.254997B Received: from mail-vs1-f44.google.com (mail-vs1-f44.google.com [209.85.217.44]) by imf30.hostedemail.com (Postfix) with ESMTP id D620580015 for ; Wed, 25 Jan 2023 21:13:37 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=CcWiG7BH; spf=pass (imf30.hostedemail.com: domain of yuzhao@google.com designates 209.85.217.44 as permitted sender) smtp.mailfrom=yuzhao@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674681217; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=b1nnGEuKKkDGSq1WmxosoaK0D1FohFPeB383P/IXUAg=; b=TnmJYasnSr17yz2/whZSmvEIrKz6WcTEU+tx1IzlT7V1zJP+TnHXEupFILSXINaB5VtLXk 5b0r+CcBA8TUj0F3T3MhI6TkO+Rm4uPM+D+JbgBs/sgZECn7sK2Pw+Kpk7c2cgDBYzx1UF oQuTCe+ABhohFiewdSi34V7rO+T0Agc= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=CcWiG7BH; spf=pass (imf30.hostedemail.com: domain of yuzhao@google.com designates 209.85.217.44 as permitted sender) smtp.mailfrom=yuzhao@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674681217; a=rsa-sha256; cv=none; b=CjwO2WqGhztPcMviQWL18jTxpzNPibzGEqHxXRXieLpP4y88CVSvP/9X5Akwsprvl72dW2 UnhG32Ksad663f1p15ekXgQGNRF/uBu9rlNP+Le5HSM3Cb1YQVOads6KcFbzq6miXvoCj9 nUhANNP2zMZZ+eUXg5MEmZo5gqYrm+A= Received: by mail-vs1-f44.google.com with SMTP id t10so14910vsr.3 for ; Wed, 25 Jan 2023 13:13:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=b1nnGEuKKkDGSq1WmxosoaK0D1FohFPeB383P/IXUAg=; b=CcWiG7BHedadfBKBk+HAhiO4vDglOmcdQXzynTfuKA6qPgTYgRfEzB+kFHcgrMLZJP J5VIwbS5mADkLR0BjZW6WDKfB6ljKeObUJ/i8bbl5yRYRSCGsUZrErumXTHHnTCImwQ3 LT1qV/A3G2uA1lD5P2tnY6sejYKH4ivzstIxeCNXVPCORrZPpuziYCqvZ8y/wBcyIFjr FMev4qyhkip6P4DtwoqrigQEqBZ+PECcR6jhNVzYBe4DiIfyBI1e0JfD72pvwSKZIZTV IMEQU+5zriiPIeoUo2A3XiRbV6vNeQCa7JPWiu70dvBubF9oej5dobizU386rnv8o7CG TzJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=b1nnGEuKKkDGSq1WmxosoaK0D1FohFPeB383P/IXUAg=; b=G3IfbvshMazMIO1SUgazIZXjWWcsdGx53ONBeP/ePKvbJe2idPoUZ7bQbsQNXZLBE6 +hfxg8lUZ4sSOQx/koH0SSdr9RIJzBrJbRA9145RpPlzGsnxdV41/LLQtJSx1+JS1LkG 4ixI9q8nJs1qSmo70oH1bqHNSbsJaDzsWLEtBdP3I/Tf4bVUJlNZZhO8bHvOfCni4M5h Y/RcExOYvM04eAFgVTR4Td/O9SU9IdgQxb50e8WHkl106EuPWeNN3TzLPKAq1ZU1xtYB ioNNBrXFNXvszUR43lMo+CsECrx9EISJKbWs6iTiJN6DkUkyU5g1EAWGTnJCHCeWRg1G Wwxg== X-Gm-Message-State: AFqh2koCdTQcBAQsKIRnihbaef+uCRYpyZuav2JfLy+5EGXZws3PfgmU qnLOQkEz11fuooA1yPBGqQlJBXsXRl+8FhNRea7Ta1LMyLwzFd6T X-Google-Smtp-Source: AMrXdXt6YoKcTFxrHffLK6kUMgFVjJX89HAI9O5bzW7f5KR45XY2fbmgFm0W1W4KWBZbSO1iEqG3pUI+WItBVOKIKaA= X-Received: by 2002:a67:f650:0:b0:3d3:db6b:e761 with SMTP id u16-20020a67f650000000b003d3db6be761mr4775930vso.46.1674681216795; Wed, 25 Jan 2023 13:13:36 -0800 (PST) MIME-Version: 1.0 References: <20230124021118.154078-1-nphamcs@gmail.com> <20230124021118.154078-2-nphamcs@gmail.com> In-Reply-To: <20230124021118.154078-2-nphamcs@gmail.com> From: Yu Zhao Date: Wed, 25 Jan 2023 14:13:00 -0700 Message-ID: Subject: Re: [PATCH v7 1/3] workingset: refactor LRU refault to expose refault recency check To: Nhat Pham Cc: akpm@linux-foundation.org, hannes@cmpxchg.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, bfoster@redhat.com, willy@infradead.org, linux-api@vger.kernel.org, kernel-team@meta.com Content-Type: text/plain; charset="UTF-8" X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: ym1pejpyo5e4pueuw9tkc4ubdrxcg9so X-Rspamd-Queue-Id: D620580015 X-HE-Tag: 1674681217-747048 X-HE-Meta: U2FsdGVkX1+yUgRJ3JGUkOGtl1/K0UX56IVNR/Z0S65Ts5y9GWTRMlTkAToWcbCCzj4PQVOmFIiYxOKGUGzqBl/GzvfW5i5Ggy1D7J7akvFanFtSkzEQdX5H9I2FcicGN7XpEYrGueyWtAdteBkuPt02CuDEPHRy1smAn0yP38YCicrw9HkNpb38g1DDC2hSngpJ7dzU5l60cAWb/OYm8iYGXaBd4DRMO7SdoDrARZjN/XvHsrG5u6NyItdHbkUJB11uge9c7zs3FEdM6tL54wbFlaViblhC5gwlbewb/5YOvFYfAGUvs9FjnfRrRdlTCudhZbuKHe0hgR8NBAzPJHRhAm/roWbju0rDlV23KIKxpZDIS81Vb3z37kh7jLfkyHKUz7LjdyPFsX/SFUSnsAkSsxm+Yvj5RitdxSJt9qro3K+kbPz6UJniIXBHdX47OjbMHDfcwePUzz2lYAlHRdh4W0ozVnk8etrkVBFKSoD6Cb/uZNu0zmgg+GgAYLf4VAoJ8GHTpqM9AsJE730UX6aY1NtnKkYQvGx9Ttqz5zD+xtmNIVNuKQrDxedV1r4wf/bncTL2ZqYaJ+hJewXk+1a0/mmVRf6gXbBJ6fTPaFthykj0u5QhEm1jzP/WzhJNJKnG/fkaSFgBJP05toRI9GdPbodttk6x01flhXJNMVorofQ0vQ0l32RAqhtvbDCq5srqDUebs4N9I838UW/Zda9cxY7lJc5tDRR8waoV1I1z+krDa51v1Mo2a9lxS2n7mpd1bXBswP7Q613qGLPUuGzyy6055YUj0vuyQIzQY4OIxiPJAIYEumxE2MP1gUCh0hMTTgy0nHuGVXTtbsRtgRDn6VcY++yl7Vfl/5S7dSPCshMu0pmdlCrE4TG1CpMq1yEOv+fuTo2H9+RibTEpFPoVX7VFhu8Y2GcIGzS9JYw8l1O7eatF5rCto3m54l2lhSl5VAMSkYaoQsbU4Mp +Ux6rWcd 5eOtQe7V6C40BgG1gndvDADngocFW9vFrDpUL0Brg7Bk+K5A2qllp9KO+ufOKJWALy6J+Pj3JfCIT/5aPYOpQW9hzVBDPPzXmsjdZSpXpETUDHDRNSTCxDKbIbGlxZD3INWw9WSR024XHGzWv2aYvZ1pvdIsZT3xY2SObn/MWa8QDyj9wfpUnkofCbGidlNq/1fjT9Yio2GAlUM86LV9U7o90DNvzbdzW5rCPYDn2QT7MeXyuLHxw9RgBdMPGssj4aTrrtk/EcGFvAmU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Jan 23, 2023 at 7:11 PM Nhat Pham wrote: > > In preparation for computing recently evicted pages in cachestat, > refactor workingset_refault and lru_gen_refault to expose a helper > function that would test if an evicted page is recently evicted. > > Signed-off-by: Nhat Pham > --- > include/linux/swap.h | 1 + > mm/workingset.c | 142 +++++++++++++++++++++++++++++-------------- > 2 files changed, 97 insertions(+), 46 deletions(-) > > diff --git a/include/linux/swap.h b/include/linux/swap.h > index a18cf4b7c724..dae6f6f955eb 100644 > --- a/include/linux/swap.h > +++ b/include/linux/swap.h > @@ -361,6 +361,7 @@ static inline void folio_set_swap_entry(struct folio *folio, swp_entry_t entry) > } > > /* linux/mm/workingset.c */ > +bool workingset_test_recent(void *shadow, bool file, bool *workingset); > void workingset_age_nonresident(struct lruvec *lruvec, unsigned long nr_pages); > void *workingset_eviction(struct folio *folio, struct mem_cgroup *target_memcg); > void workingset_refault(struct folio *folio, void *shadow); > diff --git a/mm/workingset.c b/mm/workingset.c > index 79585d55c45d..2f2d94867366 100644 > --- a/mm/workingset.c > +++ b/mm/workingset.c > @@ -244,6 +244,30 @@ static void *lru_gen_eviction(struct folio *folio) > return pack_shadow(mem_cgroup_id(memcg), pgdat, token, refs); > } > > +/* > + * Test if the folio is recently evicted. > + * > + * As a side effect, also populates the references with > + * values unpacked from the shadow of the evicted folio. > + */ > +static bool lru_gen_test_recent(void *shadow, bool file, int *memcgid, > + struct pglist_data **pgdat, unsigned long *token, bool *workingset) > +{ > + struct mem_cgroup *eviction_memcg; > + struct lruvec *lruvec; > + struct lru_gen_struct *lrugen; > + unsigned long min_seq; > + > + unpack_shadow(shadow, memcgid, pgdat, token, workingset); > + eviction_memcg = mem_cgroup_from_id(*memcgid); > + > + lruvec = mem_cgroup_lruvec(eviction_memcg, *pgdat); > + lrugen = &lruvec->lrugen; > + > + min_seq = READ_ONCE(lrugen->min_seq[file]); > + return (*token >> LRU_REFS_WIDTH) == (min_seq & (EVICTION_MASK >> LRU_REFS_WIDTH)); > +} > + > static void lru_gen_refault(struct folio *folio, void *shadow) > { > int hist, tier, refs; > @@ -258,23 +282,22 @@ static void lru_gen_refault(struct folio *folio, void *shadow) > int type = folio_is_file_lru(folio); > int delta = folio_nr_pages(folio); > > - unpack_shadow(shadow, &memcg_id, &pgdat, &token, &workingset); > - > - if (pgdat != folio_pgdat(folio)) > - return; > - > rcu_read_lock(); > > + if (!lru_gen_test_recent(shadow, type, &memcg_id, &pgdat, &token, > + &workingset)) > + goto unlock; > + > memcg = folio_memcg_rcu(folio); > if (memcg_id != mem_cgroup_id(memcg)) > goto unlock; > > + if (pgdat != folio_pgdat(folio)) > + return; > + > lruvec = mem_cgroup_lruvec(memcg, pgdat); > lrugen = &lruvec->lrugen; > - > min_seq = READ_ONCE(lrugen->min_seq[type]); > - if ((token >> LRU_REFS_WIDTH) != (min_seq & (EVICTION_MASK >> LRU_REFS_WIDTH))) > - goto unlock; > > hist = lru_hist_from_seq(min_seq); > /* see the comment in folio_lru_refs() */ > @@ -306,6 +329,12 @@ static void *lru_gen_eviction(struct folio *folio) > return NULL; > } > > +static bool lru_gen_test_recent(void *shadow, bool file, int *memcgid, > + struct pglist_data **pgdat, unsigned long *token, bool *workingset) > +{ > + return false; > +} > + > static void lru_gen_refault(struct folio *folio, void *shadow) > { > } > @@ -373,40 +402,31 @@ void *workingset_eviction(struct folio *folio, struct mem_cgroup *target_memcg) > folio_test_workingset(folio)); > } > > -/** > - * workingset_refault - Evaluate the refault of a previously evicted folio. > - * @folio: The freshly allocated replacement folio. > - * @shadow: Shadow entry of the evicted folio. > +/* > + * Test if the folio is recently evicted by checking if > + * refault distance of shadow exceeds workingset size. > * > - * Calculates and evaluates the refault distance of the previously > - * evicted folio in the context of the node and the memcg whose memory > - * pressure caused the eviction. > + * As a side effect, populate workingset with the value > + * unpacked from shadow. > */ > -void workingset_refault(struct folio *folio, void *shadow) > +bool workingset_test_recent(void *shadow, bool file, bool *workingset) > { > - bool file = folio_is_file_lru(folio); > struct mem_cgroup *eviction_memcg; > struct lruvec *eviction_lruvec; > unsigned long refault_distance; > unsigned long workingset_size; > - struct pglist_data *pgdat; > - struct mem_cgroup *memcg; > - unsigned long eviction; > - struct lruvec *lruvec; > unsigned long refault; > - bool workingset; > int memcgid; > - long nr; > + struct pglist_data *pgdat; > + unsigned long eviction; > > - if (lru_gen_enabled()) { > - lru_gen_refault(folio, shadow); > - return; > - } > + if (lru_gen_enabled()) > + lru_gen_test_recent(shadow, file, &memcgid, &pgdat, &eviction, > + workingset); Missing "return", which was correctly handled in your v2, btw.