From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88502D3C52B for ; Thu, 17 Oct 2024 18:18:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F39DC6B008A; Thu, 17 Oct 2024 14:18:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EE9C26B008C; Thu, 17 Oct 2024 14:18:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D8A626B0092; Thu, 17 Oct 2024 14:18:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id B4E846B008A for ; Thu, 17 Oct 2024 14:18:54 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id A6D1EA02B8 for ; Thu, 17 Oct 2024 18:18:33 +0000 (UTC) X-FDA: 82683905262.14.976FFA0 Received: from mail-oi1-f170.google.com (mail-oi1-f170.google.com [209.85.167.170]) by imf09.hostedemail.com (Postfix) with ESMTP id 22246140012 for ; Thu, 17 Oct 2024 18:18:44 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=jOxrIZM3; spf=pass (imf09.hostedemail.com: domain of weixugc@google.com designates 209.85.167.170 as permitted sender) smtp.mailfrom=weixugc@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729189058; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Vh+Fe+JVI07ZHvub7dZIuFupJMAI8JV+zQomFL2+VuI=; b=kzmtEuvYaKpyOp1CollRW444ub53Z6stRYP91p1O8aotIeRZ5H8fJ1/N19+pWNctZX79K/ LfT8HHq5sycqptwBbthXD26WhjsUl8TKES49dpuUeeuzH20Ft8AzSfLLXHobpPipa/ejNs GFpHbf2tTc2uKYOChSUNZt0E0eMbChg= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=jOxrIZM3; spf=pass (imf09.hostedemail.com: domain of weixugc@google.com designates 209.85.167.170 as permitted sender) smtp.mailfrom=weixugc@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729189058; a=rsa-sha256; cv=none; b=hidkpOgDeNFpL5myVnZBYmTf87HtltKuMPTw1Gl/vmQmSjwFrWN6FxuXp7BeBq8y2t+e77 HYkrjjLF25/AZ1i3HEzbC+0eg6DiqMdw2nmS/I5ZMS3PRNOz5h19mjt7AyUMxXuXuDaoUY FOUfndlGK9kvFneO7LHkgOuiZTaQ9Ro= Received: by mail-oi1-f170.google.com with SMTP id 5614622812f47-3e4d597d6f0so637904b6e.1 for ; Thu, 17 Oct 2024 11:18:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1729189131; x=1729793931; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=Vh+Fe+JVI07ZHvub7dZIuFupJMAI8JV+zQomFL2+VuI=; b=jOxrIZM3rKpfqVzncAsYhqa63vMvF55vzrqaDySrkDbsHCcsTDoNZ2C1wi6aejFQC6 s2OuSQg7Z0bYLh/5/hT+1A0GiMp1EqLupzhj0adsMFpzK9TZQcBQnRLpGRJ9LW6kWSIV Z1XXPyhpnINfHo4It3ArHAGZ4pnKJuqHtZVhNKo45DlIsynV850lCks9wuQfokamkOrN oyaUEaCx5LOk81f4XHTnEVcD0vLSNmR6EGTyKwBMcfAwTKKksYuiezm6cHHTmgGO4L0l PdmoIJne7YZJKljdg4pYF8nL7RYkkTpNoATZClSYPm56OXZczO37OUOuNXHFOJ7htRYx Y9JA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729189131; x=1729793931; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Vh+Fe+JVI07ZHvub7dZIuFupJMAI8JV+zQomFL2+VuI=; b=PxxwdxIAOqMzXfTNtOuK244Q/THMfFkBHFGVYxK3BEtbxKcahecE1N0SIWsJM+7nXM IToQxq2l5ZyLqL6Iq+H7a22P7a7nTssme6nv+EgAWnIHDWHFVapGmjCGXVLEAcXWhisM mAkVrB9xrI6ewkUojAVEeX7iKDzR/j3wp2uYOPxqAjmE8DKGbVi9gJ6GnoavTM+QeLeG LSEASlWrj9KjWCxfKr09jVMD4t9m7S505R0DDuD+xxghRWIouOK8vn+9T5+j9RXj59mQ O7jJeaGB9j+6cRSHNDJMap4id1X7+0ypKQ5QVvIE8jYA9TLB6NzMMpPcn61SpGUec+Yb NJOg== X-Forwarded-Encrypted: i=1; AJvYcCV/8NiB0koL9ydC+I2FEU8nnI/t0BYigjMEXYDdT5YyLSylhzHj3F+nmWWdA351pHw8fXkA+kn3YQ==@kvack.org X-Gm-Message-State: AOJu0Yz4wJRMhJrc71dXCT4hoSaJrskd/PGPQ7cmKhjTQM5jTCCB5qi+ yiLkKGpO58GEL54S6Cv028lpNjPRN7ZeIE72u4u7j3RkZnzaior7fsr3ncnZV5zhAiJRALlhIv8 6vH1Livgv3be2ntK90RqULkbrRgBnqAMO0biO X-Google-Smtp-Source: AGHT+IFBixTSBteXjwgBE0GYuX+NzgPSES2v1B6Ij68JD+0kTXTGGXPYuvL0IrejXDh8IRXrjdun0Idt0xKpZbj2+ug= X-Received: by 2002:a05:6808:1887:b0:3e5:e347:55c7 with SMTP id 5614622812f47-3e5f0237482mr7872066b6e.11.1729189131234; Thu, 17 Oct 2024 11:18:51 -0700 (PDT) MIME-Version: 1.0 References: <20241017181528.3358821-1-weixugc@google.com> In-Reply-To: <20241017181528.3358821-1-weixugc@google.com> From: Wei Xu Date: Thu, 17 Oct 2024 11:18:38 -0700 Message-ID: Subject: Re: [PATCH v2] mm/mglru: reset page lru tier bits when activating To: Yu Zhao Cc: Andrew Morton , Axel Rasmussen , Brian Geffon , Jan Alexander Steffens , Suleiman Souhlal , linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam03 X-Rspam-User: X-Rspamd-Queue-Id: 22246140012 X-Stat-Signature: pjcebu8b71uonc95b51wmu9ir1o9jb6b X-HE-Tag: 1729189124-173698 X-HE-Meta: U2FsdGVkX19ix9cDKYlUeIU5TuPLZU0lMj7IGI0OiTSzW0+F0Kg+uL2cbDSPPgthgnw682o7FK3ohTgkjwsJf7xII0CDxwMfjJwwoIaRio0LRxvz0HLTT7c+B9hdBwUA5JDw3up8cyAXCZn/0xWwpft6AyJNreLDLXdOZyXQwedQm+xfE1hrSQtlhdQiuNkniULe/FDSYeZq3/9GrWssYfBBNLEzAzqviJzUWl5fgeoEgRtbw7+Cv3xEli2HVf0wfK2u01sNgizJAsVvh9XGVs/rRmC1JfJtS2CGspE+dO7OvYPHdut2oHcg+ePMuTI1gYWv2WlmCIigQpfdb8FrzR0CVCO2ce48x0FApPQlik3s2QS6p2xN1Xh7jzsjOOnyJoDlQ/Zg+aFFnm2sWyZEm1X4FjfV7jEXFH/As+pVQ7MRq3gU9uwE42IDODsnKi1rcovehAZdfauZ+iqu7rg9M+fLM/uQjTBgtGY3gOzYsVJxLVydEClSzolPYzKsPtpujONTaLC/ZPzZ4wY7C9iazmyX8SV5vfEt758USvLKTSDp93FTD4N5P1vgVaRxfFFJqsyJDLHorfThhnl8dWLKO2IbxyWjhkeTZi/dmXfjgeKNd/BfUFAX7O+Dpd1MFEzFSOYVcaeZsjfHbAigyC+gYo4OAEQ5btPkC1ZkY+KcjEjWL+kPUodK7rH5hUTdD776f6qPa8zJ52RStng0e8fGcEhjZvNvoOwGv4iK2QTSp9trDofpnpW77qKZRkkiRZafy4sPbZxb0R08wQxoZxJDiWjVTx+pvAwDcpVcfnBlrb3D+FHRxcvETPLz6/yrji0CVwAUp26kewShV8DfxfKT8eZtIetWWwHlXx1OaEQl9ps7TrCqMz2nQkwQtiyYB99xXvLGTOniTRHb2rLuxclxaXAx3T7wcSCZj1w2qUEXbzmEdXPzsKMqFH2iGABdd4WcV5GUncMc/V7st3ahBhY Nx4wY7Xz WYbvqHBGFtOVDbQQLaW8xBZ5Ajip8kZxsgVcXrIN8NLXjfqIeSqbPzPVQUHLO48aBaoq4LEqK9Xuk45Pc8ZnSQFN4euK3hONNHNbZFo/XRZMd8v4X+6zHgQl4R2vGoMP1HoythbzEF6yBZUHsOnKWoMjDpCaRmprxbquLZbGggSRBqFZ3ljpgOou00JCn7F1bZ+ymgJliL+mPLPgCnfnrDqxMiA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Changelog since v1: - Addressed Yu Zhao's comments on not clearing PG_workingset. - Updated the commit message. On Thu, Oct 17, 2024 at 11:15=E2=80=AFAM Wei Xu wrote: > > When a folio is activated, lru_gen_add_folio() moves the folio to the > youngest generation. But unlike folio_update_gen()/folio_inc_gen(), > lru_gen_add_folio() doesn't reset the folio lru tier bits > (LRU_REFS_MASK | LRU_REFS_FLAGS). This inconsistency can affect how > pages are aged via folio_mark_accessed() (e.g. fd accesses), though > no user visible impact related to this has been detected yet. > > Note that lru_gen_add_folio() cannot clear PG_workingset if the > activation is due to workingset refault, otherwise PSI accounting > will be skipped. So fix lru_gen_add_folio() to clear the lru tier > bits other than PG_workingset when activating a folio, and also > clear all the lru tier bits when a folio is activated via > folio_activate() in lru_gen_look_around(). > > Fixes: 018ee47f1489 ("mm: multi-gen LRU: exploit locality in rmap") > Signed-off-by: Wei Xu > --- > include/linux/mm_inline.h | 15 ++++++++++++++- > include/linux/mmzone.h | 2 ++ > mm/vmscan.c | 8 ++++---- > 3 files changed, 20 insertions(+), 5 deletions(-) > > diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h > index 6f801c7b36e2..355cf46a01a6 100644 > --- a/include/linux/mm_inline.h > +++ b/include/linux/mm_inline.h > @@ -155,6 +155,11 @@ static inline int folio_lru_refs(struct folio *folio= ) > return ((flags & LRU_REFS_MASK) >> LRU_REFS_PGOFF) + workingset; > } > > +static inline void folio_clear_lru_refs(struct folio *folio) > +{ > + set_mask_bits(&folio->flags, LRU_REFS_MASK | LRU_REFS_FLAGS, 0); > +} > + > static inline int folio_lru_gen(struct folio *folio) > { > unsigned long flags =3D READ_ONCE(folio->flags); > @@ -222,6 +227,7 @@ static inline bool lru_gen_add_folio(struct lruvec *l= ruvec, struct folio *folio, > { > unsigned long seq; > unsigned long flags; > + unsigned long mask; > int gen =3D folio_lru_gen(folio); > int type =3D folio_is_file_lru(folio); > int zone =3D folio_zonenum(folio); > @@ -257,7 +263,14 @@ static inline bool lru_gen_add_folio(struct lruvec *= lruvec, struct folio *folio, > gen =3D lru_gen_from_seq(seq); > flags =3D (gen + 1UL) << LRU_GEN_PGOFF; > /* see the comment on MIN_NR_GENS about PG_active */ > - set_mask_bits(&folio->flags, LRU_GEN_MASK | BIT(PG_active), flags= ); > + mask =3D LRU_GEN_MASK; > + /* > + * Don't clear PG_workingset here because it can affect PSI accou= nting > + * if the activation is due to workingset refault. > + */ > + if (folio_test_active(folio)) > + mask |=3D LRU_REFS_MASK | BIT(PG_referenced) | BIT(PG_act= ive); > + set_mask_bits(&folio->flags, mask, flags); > > lru_gen_update_size(lruvec, folio, -1, gen); > /* for folio_rotate_reclaimable() */ > diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h > index 17506e4a2835..96dea31fb211 100644 > --- a/include/linux/mmzone.h > +++ b/include/linux/mmzone.h > @@ -403,6 +403,8 @@ enum { > NR_LRU_GEN_CAPS > }; > > +#define LRU_REFS_FLAGS (BIT(PG_referenced) | BIT(PG_workingset)) > + > #define MIN_LRU_BATCH BITS_PER_LONG > #define MAX_LRU_BATCH (MIN_LRU_BATCH * 64) > > diff --git a/mm/vmscan.c b/mm/vmscan.c > index c12f78b042f3..2d0486189804 100644 > --- a/mm/vmscan.c > +++ b/mm/vmscan.c > @@ -2602,8 +2602,6 @@ static bool should_clear_pmd_young(void) > * shorthand helpers > ***********************************************************************= *******/ > > -#define LRU_REFS_FLAGS (BIT(PG_referenced) | BIT(PG_workingset)) > - > #define DEFINE_MAX_SEQ(lruvec) \ > unsigned long max_seq =3D READ_ONCE((lruvec)->lrugen.max_seq) > > @@ -4138,8 +4136,10 @@ void lru_gen_look_around(struct page_vma_mapped_wa= lk *pvmw) > old_gen =3D folio_lru_gen(folio); > if (old_gen < 0) > folio_set_referenced(folio); > - else if (old_gen !=3D new_gen) > + else if (old_gen !=3D new_gen) { > + folio_clear_lru_refs(folio); > folio_activate(folio); > + } > } > > arch_leave_lazy_mmu_mode(); > @@ -4370,7 +4370,7 @@ static bool isolate_folio(struct lruvec *lruvec, st= ruct folio *folio, struct sca > > /* see the comment on MAX_NR_TIERS */ > if (!folio_test_referenced(folio)) > - set_mask_bits(&folio->flags, LRU_REFS_MASK | LRU_REFS_FLA= GS, 0); > + folio_clear_lru_refs(folio); > > /* for shrink_folio_list() */ > folio_clear_reclaim(folio); > -- > 2.47.0.rc1.288.g06298d1525-goog >