From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB34BE7717F for ; Tue, 10 Dec 2024 04:55:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 64FFE6B0110; Mon, 9 Dec 2024 23:55:07 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5FFAA6B0112; Mon, 9 Dec 2024 23:55:07 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4A1116B0113; Mon, 9 Dec 2024 23:55:07 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 2517B6B0110 for ; Mon, 9 Dec 2024 23:55:07 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 92F6C1602BD for ; Tue, 10 Dec 2024 04:55:06 +0000 (UTC) X-FDA: 82877834802.29.C8AEEB1 Received: from mail-vk1-f175.google.com (mail-vk1-f175.google.com [209.85.221.175]) by imf26.hostedemail.com (Postfix) with ESMTP id 6F4EF140003 for ; Tue, 10 Dec 2024 04:54:48 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=iIqX7PQ4; spf=pass (imf26.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.221.175 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733806490; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=mDFbbwwOVomYolQdyT9bOHykr5RTMJRxIyRn7hebbDs=; b=tEL/TMx4ZQUbW+ANsId7e/uukAmzpxgvJJCcKS9k0hFwK/gnV1EWOc6dnEyJMN2kklPrBH IZHxxBf0zo95aEWQ28qe3iNtpNumCtsF7WYMXRc3+9EDaENgtW17YNFFq7TdSnSP7TT6Vg XhhRb7fzyiIRm/wtv53AVapRfFs7kPA= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=iIqX7PQ4; spf=pass (imf26.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.221.175 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733806490; a=rsa-sha256; cv=none; b=VcvD0kj2tFMNaThIfjtyedO4FMZzWOEqH6SVH3+gp43UexixngvCVPr3u6hwYSqUc+XDLy 763lmGXdAYrtNuSh8nnkV7nHP2qk/1GXtJZ+5BmdsOXQwRI2nhrYGr24rf5cX39x0Ymav6 MKUTz3d3huexUd48awLuGwuf4w7gYwE= Received: by mail-vk1-f175.google.com with SMTP id 71dfb90a1353d-5187f0b893dso425709e0c.3 for ; Mon, 09 Dec 2024 20:55:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733806504; x=1734411304; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=mDFbbwwOVomYolQdyT9bOHykr5RTMJRxIyRn7hebbDs=; b=iIqX7PQ4RZQ+f9jla8QwePZUJQC8FEYNHSdaOoMnCgTq1SdZcfnqVWUI2jw0FeStZw jHrjB7lnQmreEjK3XUgN7cz7O269gcQdsceRJBiiK4FpC+GfObXPV4gMCLQpr+nh+NH2 C55aDowtw2kT93x0p5dzi1yLEJU84/S2m/ZmoT6/KRTNUJlnXy+jZrcVFB5Bhh87uq+Q blJrBuaRDXioqSE/T9/r0RWrsGG59HY6FUwThLdlsBHXbm9xFU4ZKh8VzdwIA84sa8mb 72FGSeCeRRn31jKjwD9DB/nh4NWb5OAekJUHATFE7K8Ld0PN3VeNs6NhKw1d0rRE51F4 PrZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733806504; x=1734411304; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mDFbbwwOVomYolQdyT9bOHykr5RTMJRxIyRn7hebbDs=; b=bLku/gJooQbdpWc867AijAIV1M7XFOp1W3qs+LSttIEqs/gcMjxLSH5UIhwxjnyKDi tZ3BGR/usdjlFOm+XHnT39rAryGT/tpPlKJKsUusr/lUmlJzxipomS5ikHXZVb+Q+4CB wrzYjxQL/zps4FyIujbP/uVPRwvzTOeZOIh9F3ol9Fgda9Yu19a/yDj9t9Q858Rti62i L3l4mmv4f/rTmnpViXAFnBXAet3PzIhVkWm1odftI70TS3tkj5pGOdWB2uHt8DsTxvZx 32p3GLcfDSSxsl+yA3wBTmZCGPmYy9GwfrK96TVsK7tgDLwCD4rQ8MczAJ0zOpDJV4xJ bgaw== X-Forwarded-Encrypted: i=1; AJvYcCU12/qTD24Gh2SgQT6UgiyMn1YnV+tUAA+aHY0ZQ3EF3CPJ1ccvf6qPRk4CHJnyDCNJiN7qzoABEg==@kvack.org X-Gm-Message-State: AOJu0Yz/ZljRbpCjWp5jjahwFWT8R8HVCLPfRRw/18jTeDuj2MOWzfRI 6DVEXGr8ytFluFlJg3NBLGTK5/hdC60pQVZ5zZ4LnbK2hIAc/bNH5nshQc32OWUP0MUfcqUN9je eVZDnPSApOuFxChKzHKJDGR/g9B4= X-Gm-Gg: ASbGnctztUMuNBspQccmyqJX0FVvWswF7KKL3YKT9vYFpxEudqSGZKjXgVh1t90RLi+ 6VxeMPK52/rexfzrtXQgb+jcezBunroPuzoUQo+xbeasrc+Y96kfpWgMNBn91d702Cz9dJQ== X-Google-Smtp-Source: AGHT+IHN2+Hf+Os/pMGS099DB15tkL5YmQXUHjxOaDv7QInNXh2kZ0vi196IPZ9fbMU6pb8yYQfGLbAYtlehBosBFIk= X-Received: by 2002:a05:6122:2213:b0:515:4b68:c37f with SMTP id 71dfb90a1353d-5188834be77mr3512181e0c.6.1733806503561; Mon, 09 Dec 2024 20:55:03 -0800 (PST) MIME-Version: 1.0 References: <20241209083618.2889145-1-chenridong@huaweicloud.com> <20241209083618.2889145-2-chenridong@huaweicloud.com> In-Reply-To: <20241209083618.2889145-2-chenridong@huaweicloud.com> From: Barry Song <21cnbao@gmail.com> Date: Tue, 10 Dec 2024 12:54:52 +0800 Message-ID: Subject: Re: [PATCH v4 1/1] mm: vmascan: retry folios written back while isolated for traditional LRU To: Chen Ridong Cc: akpm@linux-foundation.org, mhocko@suse.com, hannes@cmpxchg.org, yosryahmed@google.com, yuzhao@google.com, david@redhat.com, willy@infradead.org, ryan.roberts@arm.com, wangkefeng.wang@huawei.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, chenridong@huawei.com, wangweiyang2@huawei.com, xieym_ict@hotmail.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam05 X-Stat-Signature: cinhk7ecztkzt4spxwomg1fja96uite4 X-Rspamd-Queue-Id: 6F4EF140003 X-Rspam-User: X-HE-Tag: 1733806488-915673 X-HE-Meta: U2FsdGVkX18g8lcz8nU7gySJklYMa03fgowLgbqhK30NhU8xB9hjPYiu6a590Z+XVyJuKd/TQ296VaSwyRfd4Ls2DnIlq0asnrrHj8ZQFo0u7OSTXWJqW1K9BCtHC79AT66qBWzU7TAPIpzAQP9OYcgUovbnWW++AcRWEuDZSxppiEghapn+9+L/Jpozv6+itB2NqjKrLUQKbXPQBpsfqdJHessLGkRcCOpqQjktrzDFeY9sHUDf+z1oYCbfCcCQzL/1k9e4F5w2bvwc4CUfqHF1w7Tna94A+7OdMQt4bLm8f6dWF0Ke1x9yJs4MQwCCaRHUOSlIuDwEsiuaR4JVi4+MfidWtRP6GOyHhJUbIpFWFlsuQkPSylu/+YGYaxgQRTgfGt4yUgmWlEbRd4SsZBO3HG9NueQPFrs1aXAa+4W+jqCsRO4sx/Ekq7IWrwED1vgguzYEAg4wEH84ecw8pPRb2o+7ZYgA+v2O/5I2zoKr9BcfUw1vCx7amGEs88jfMu5Bj864kY6GFsIf6nY3CctthcoOvtXNkqU1uY+zUNwagKkPVu38u1CTGX1/d9a+tn/stAqG8zvGL3+lNVttz6iChfFbx3RI20rfitH/9njeGa1AZaA2kurji0aY91R1GXUp6mozvSwearF8OVswbv6jUkunhV3XA+X3FP0kl2TdJB/B5lSsge+G7ybD3ES2uNNejLJv9LHZ328hcvU+5eoa/g9YUVQc+6wDRVLuNAAKqxC4t8xP9HIKu0Ps2pG/EX8g1yrnh49jvbhr5wTZcuY580HSwb6D0JS80snKGTafBRYWRTA9/DBg/7kL5i/KJuzJrlnKSRByXA+69YcWUvGWwu6jq6R5/ddA+0VRwOnIJaiHR+VJm5NbEzfz2sEl2dx3CM9S2YPqbMBVhL/ZllSfvahn5lxjJ3l8lByPttHWsZntpUWJ61KWZ7lNULqy411ypHg6K4NR6MRsAT5 cmU9tUnr tbdjmi4s56aaeWYrFqgYBYX2dt8K/bwQ0Z6hu+pnmj5v+2+Gk4Hcd5CHedaEYLMQy8LqV4eQjduFyoapjZRaVFIrvMTqD5jOWp3KKiP8/BTkGk2M914RYtgwQsCL7KNEw51fd+V/zY34Hro7ZNtPIYAO2+A2RMS+e/3V79rEgtayeSgeKNs9bOOmLkF8obHpJZNwoRArFKkgav1mB9vqIFblPdUsgcnDWArHHFmMKcTZZzO5ZwFDGjQt3HYRfTBm8fsfDW7TszqS8GzTr3y2Qoc6wyWzWvY7cyCjAo0O+uvjU9C+i8HV/3zWP0Uo8pj45NJ/KWi5lGTB56fZcMOyYCRMqSFft24Ve/wCpeycOU/c3xDgIhpPAaGDe5w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.137602, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Dec 9, 2024 at 4:46=E2=80=AFPM Chen Ridong wrote: > > From: Chen Ridong > > The commit 359a5e1416ca ("mm: multi-gen LRU: retry folios written back > while isolated") only fixed the issue for mglru. However, this issue > also exists in the traditional active/inactive LRU. This issue will be > worse if THP is split, which makes the list longer and needs longer time > to finish a batch of folios reclaim. > > This issue should be fixed in the same way for the traditional LRU. > Therefore, the common logic was extracted to the 'find_folios_written_bac= k' > function firstly, which is then reused in the 'shrink_inactive_list' > function. Finally, retry reclaiming those folios that may have missed the > rotation for traditional LRU. let's drop the cover-letter and refine the changelog. > > Signed-off-by: Chen Ridong > --- > include/linux/mmzone.h | 3 +- > mm/vmscan.c | 108 +++++++++++++++++++++++++++++------------ > 2 files changed, 77 insertions(+), 34 deletions(-) > > diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h > index b36124145a16..47c6e8c43dcd 100644 > --- a/include/linux/mmzone.h > +++ b/include/linux/mmzone.h > @@ -391,6 +391,7 @@ struct page_vma_mapped_walk; > > #define LRU_GEN_MASK ((BIT(LRU_GEN_WIDTH) - 1) << LRU_GEN_PGOF= F) > #define LRU_REFS_MASK ((BIT(LRU_REFS_WIDTH) - 1) << LRU_REFS_PG= OFF) > +#define LRU_REFS_FLAGS (BIT(PG_referenced) | BIT(PG_workingset)) > > #ifdef CONFIG_LRU_GEN > > @@ -406,8 +407,6 @@ enum { > NR_LRU_GEN_CAPS > }; > > -#define LRU_REFS_FLAGS (BIT(PG_referenced) | BIT(PG_workingset)) > - > #define MIN_LRU_BATCH BITS_PER_LONG > #define MAX_LRU_BATCH (MIN_LRU_BATCH * 64) > > diff --git a/mm/vmscan.c b/mm/vmscan.c > index 76378bc257e3..1f0d194f8b2f 100644 > --- a/mm/vmscan.c > +++ b/mm/vmscan.c > @@ -283,6 +283,48 @@ static void set_task_reclaim_state(struct task_struc= t *task, > task->reclaim_state =3D rs; > } > > +/** > + * find_folios_written_back - Find and move the written back folios to a= new list. > + * @list: filios list > + * @clean: the written back folios list > + * @skip: whether skip to move the written back folios to clean list. > + */ > +static inline void find_folios_written_back(struct list_head *list, > + struct list_head *clean, bool skip) > +{ > + struct folio *folio; > + struct folio *next; > + > + list_for_each_entry_safe_reverse(folio, next, list, lru) { > + if (!folio_evictable(folio)) { > + list_del(&folio->lru); > + folio_putback_lru(folio); > + continue; > + } > + > + if (folio_test_reclaim(folio) && > + (folio_test_dirty(folio) || folio_test_writeback(foli= o))) { > + /* restore LRU_REFS_FLAGS cleared by isolate_foli= o() */ > + if (lru_gen_enabled() && folio_test_workingset(fo= lio)) > + folio_set_referenced(folio); > + continue; > + } > + > + if (skip || folio_test_active(folio) || folio_test_refere= nced(folio) || > + folio_mapped(folio) || folio_test_locked(folio) || > + folio_test_dirty(folio) || folio_test_writeback(folio= )) { > + /* don't add rejected folios to the oldest genera= tion */ > + if (lru_gen_enabled()) > + set_mask_bits(&folio->flags, LRU_REFS_MAS= K | LRU_REFS_FLAGS, > + BIT(PG_active)); > + continue; > + } > + > + /* retry folios that may have missed folio_rotate_reclaim= able() */ > + list_move(&folio->lru, clean); > + } > +} > + > /* > * flush_reclaim_state(): add pages reclaimed outside of LRU-based recla= im to > * scan_control->nr_reclaimed. > @@ -1907,6 +1949,25 @@ static int current_may_throttle(void) > return !(current->flags & PF_LOCAL_THROTTLE); > } > > +static inline void acc_reclaimed_stat(struct reclaim_stat *stat, > + struct reclaim_stat *curr) > +{ > + int i; > + > + stat->nr_dirty +=3D curr->nr_dirty; > + stat->nr_unqueued_dirty +=3D curr->nr_unqueued_dirty; > + stat->nr_congested +=3D curr->nr_congested; > + stat->nr_writeback +=3D curr->nr_writeback; > + stat->nr_immediate +=3D curr->nr_immediate; > + stat->nr_pageout +=3D curr->nr_pageout; > + stat->nr_ref_keep +=3D curr->nr_ref_keep; > + stat->nr_unmap_fail +=3D curr->nr_unmap_fail; > + stat->nr_lazyfree_fail +=3D curr->nr_lazyfree_fail; > + stat->nr_demoted +=3D curr->nr_demoted; > + for (i =3D 0; i < ANON_AND_FILE; i++) > + stat->nr_activate[i] =3D curr->nr_activate[i]; > +} you had no this before, what's the purpose of this=EF=BC=9F > + > /* > * shrink_inactive_list() is a helper for shrink_node(). It returns the= number > * of reclaimed pages > @@ -1916,14 +1977,16 @@ static unsigned long shrink_inactive_list(unsigne= d long nr_to_scan, > enum lru_list lru) > { > LIST_HEAD(folio_list); > + LIST_HEAD(clean_list); > unsigned long nr_scanned; > unsigned int nr_reclaimed =3D 0; > unsigned long nr_taken; > - struct reclaim_stat stat; > + struct reclaim_stat stat, curr; > bool file =3D is_file_lru(lru); > enum vm_event_item item; > struct pglist_data *pgdat =3D lruvec_pgdat(lruvec); > bool stalled =3D false; > + bool skip_retry =3D false; > > while (unlikely(too_many_isolated(pgdat, file, sc))) { > if (stalled) > @@ -1957,10 +2020,20 @@ static unsigned long shrink_inactive_list(unsigne= d long nr_to_scan, > if (nr_taken =3D=3D 0) > return 0; > > - nr_reclaimed =3D shrink_folio_list(&folio_list, pgdat, sc, &stat,= false); > + memset(&stat, 0, sizeof(stat)); > +retry: > + nr_reclaimed +=3D shrink_folio_list(&folio_list, pgdat, sc, &curr= , false); > + find_folios_written_back(&folio_list, &clean_list, skip_retry); > + acc_reclaimed_stat(&stat, &curr); > > spin_lock_irq(&lruvec->lru_lock); > move_folios_to_lru(lruvec, &folio_list); > + if (!list_empty(&clean_list)) { > + list_splice_init(&clean_list, &folio_list); > + skip_retry =3D true; > + spin_unlock_irq(&lruvec->lru_lock); > + goto retry; > + } > > __mod_lruvec_state(lruvec, PGDEMOTE_KSWAPD + reclaimer_offset(), > stat.nr_demoted); > @@ -4567,8 +4640,6 @@ static int evict_folios(struct lruvec *lruvec, stru= ct scan_control *sc, int swap > int reclaimed; > LIST_HEAD(list); > LIST_HEAD(clean); > - struct folio *folio; > - struct folio *next; > enum vm_event_item item; > struct reclaim_stat stat; > struct lru_gen_mm_walk *walk; > @@ -4597,34 +4668,7 @@ static int evict_folios(struct lruvec *lruvec, str= uct scan_control *sc, int swap > scanned, reclaimed, &stat, sc->priority, > type ? LRU_INACTIVE_FILE : LRU_INACTIVE_ANON); > > - list_for_each_entry_safe_reverse(folio, next, &list, lru) { > - if (!folio_evictable(folio)) { > - list_del(&folio->lru); > - folio_putback_lru(folio); > - continue; > - } > - > - if (folio_test_reclaim(folio) && > - (folio_test_dirty(folio) || folio_test_writeback(foli= o))) { > - /* restore LRU_REFS_FLAGS cleared by isolate_foli= o() */ > - if (folio_test_workingset(folio)) > - folio_set_referenced(folio); > - continue; > - } > - > - if (skip_retry || folio_test_active(folio) || folio_test_= referenced(folio) || > - folio_mapped(folio) || folio_test_locked(folio) || > - folio_test_dirty(folio) || folio_test_writeback(folio= )) { > - /* don't add rejected folios to the oldest genera= tion */ > - set_mask_bits(&folio->flags, LRU_REFS_MASK | LRU_= REFS_FLAGS, > - BIT(PG_active)); > - continue; > - } > - > - /* retry folios that may have missed folio_rotate_reclaim= able() */ > - list_move(&folio->lru, &clean); > - } > - > + find_folios_written_back(&list, &clean, skip_retry); > spin_lock_irq(&lruvec->lru_lock); > > move_folios_to_lru(lruvec, &list); > -- > 2.34.1 > Thanks Barry >