From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9D04BC47258 for ; Fri, 26 Jan 2024 00:56:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1A5098D0006; Thu, 25 Jan 2024 19:56:33 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1554C8D0001; Thu, 25 Jan 2024 19:56:33 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 043C68D0006; Thu, 25 Jan 2024 19:56:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id E7FFF8D0001 for ; Thu, 25 Jan 2024 19:56:32 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 87A31140DF4 for ; Fri, 26 Jan 2024 00:56:32 +0000 (UTC) X-FDA: 81719646624.03.B35FDFC Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf28.hostedemail.com (Postfix) with ESMTP id DEE49C0017 for ; Fri, 26 Jan 2024 00:56:30 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=NbAonHmC; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf28.hostedemail.com: domain of chrisl@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=chrisl@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706230591; a=rsa-sha256; cv=none; b=8SxGvBzd5jPeZxt1ahF+R+VWI2tNSqfJ6+655rW46imGJPBVHvEnfQlhikfbWkTdVk2kds CJPPJvbPP2fjKfwcnB2jifrXrXXhtHkLJusPfovE72v8o4sAGZWbu94RMs6DlScRq+vcFY CqZbd7Sv2PUWJzN6SHZ3yKKSLJqnIL0= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=NbAonHmC; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf28.hostedemail.com: domain of chrisl@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=chrisl@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706230591; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5Xe+E5u69NNQ3hulZWxHlw7v4qfKV7up1CnXnj5Y2JU=; b=S5EUUw/1AGQYVCWJ8U6xgS0bprJHLx9PwD2+cZkndQVg5wVInA9bHwScvLDFkOiIO8Rrin Xegyju18lYIBqQY0cE94QLt8+V1bDBz/C19dW3f7ZaugXJ6NcJnMdAtDd58636tDIIDB35 x9H4p3n8yHb65AoLbMWH8nt2tFkB9uI= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 10A6762354; Fri, 26 Jan 2024 00:56:30 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 74933C433F1; Fri, 26 Jan 2024 00:56:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1706230589; bh=WbcgTvp8TfQS/w6R/mlYgDgTg9ShuwbJ49WG68DhQG4=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=NbAonHmCb1uEHhagz9V59i37+JdZmwW/mX2o1B9LYr5vWW2oUobKAKLPSx62sJC55 2eMNxfkmknCfnuaU3MXbBidqvfz7Uh5ImMh9iqAGvZkRiOP7o6actc3E6AYd4t3LfU nfQYnXKQ8Sk0c7VEZdTB4NUZofcK2em+UCmHhA1sG9zYVkmwGvZiM+yezDb8+i2i+1 mS5UHjzl9WzUMXtBTb667aBfu/TIGklkHPY+fi9e34/Bwh0gQNiqnVn0M0rYF4Leiq qOpGRn+/B5qquUOJOoMLIXHnBNOZ0twi7gHHa3TZfIaGipRLe6lWQKmgPvEjkgppzI Sn9OKxRhVmaUQ== Date: Thu, 25 Jan 2024 16:56:27 -0800 From: Chris Li To: Kairui Song Cc: linux-mm@kvack.org, Andrew Morton , Yu Zhao , Wei Xu , Matthew Wilcox , linux-kernel@vger.kernel.org Subject: Re: [PATCH v3 1/3] mm, lru_gen: try to prefetch next page when scanning LRU Message-ID: References: <20240123184552.59758-1-ryncsn@gmail.com> <20240123184552.59758-2-ryncsn@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: DEE49C0017 X-Stat-Signature: he53mmjjae9k9zdwm5mbm41a3bb4hpzy X-HE-Tag: 1706230590-848959 X-HE-Meta: U2FsdGVkX18EM1axpRoAHdCNNlffipIVdpmO6BMB22cMB84DAzHH63B+/nQlCRxDoWS1Wv5p2ydBnf2gMMmal1+F9rlHG3zbn2jwSb5OSlUw18Jb4Ca3Y2aiuHbkQCtaRVxXOS8Klo/nCDiJKCtOJoUOaUg50Fk0OQEtEFiojgFd0xKaoCAdEVcQ3emU8pHbQuiWx2UTPtvtqmcVT5ZoZXUML3o5Ydsx9wsW5nmjpRe/brOemRNTKNZfRc60ntHY9QNPalwE8enjE3npeGlz4S7mlA+TJ+YcmVAGx8L5nO8gN5qAI26MAXcshLvrVz4FLcHFJNybCaXbVl9MK9d8W9x38t6+p3r9Oz+WIwSLRJ2nwzR3RMIZT3BB8eI5DHShTkl83mEp874aiJuaMbTN1Ca+yGlIjEB8q3pUoFTZHh9WMCgqmLyWT5PladPXgVepm8THrv6+IBsYrcTXA7V/97Qsb0IMxVjHjfinmHbomeedChQLOGeCECtLGB2gQvq435zloD71Ww5y0TJsyOH0Vyu0gPu1KYYgjtbWt5okh5/u6hrQs+gMyPk4k8ER9FQ6sgc+EB1c17HXz0S8dvZFj9+IvaXpaiZVZCyIuiZ9Vqwx5R+AxvRQpyQgWKG2dQbIeQCD/VWY2LFB/Qybxupv1YCLZC1FmGhk/ORPP3npew4aogN/VkuyFT+pLUlFSySUjz+zfzkDoMQfjrOnn9NrAltlRFEyFi0mPejQzhRhb4B8uitvOjOIXhqLmAIUORGhGRmTmZU3Uy5JfiAnoiyNt8Q6dVWGp0NEFmSgBa+3rZogLOBiYpVwfIXVJmEY13X+Fd9FA45BbGWKwFgEE3EFeYhyY5pyvHVsUALduWP3mYaapVyv3mowTGN8bu8nB/mMGbPPRiobzP+agTScd7XWSTqEF0SfHKpG7ill4ul7nk9Ib6e/vAA8qjzoM9wCcqkY8pSSsn9EKYphJBIPX9j TOGbYuCA p0q3DORRIlwPF7HHHeIOucHbLsLpnEnQtzppEls71WpmQbkASeFMtB7QqIBnPmE2ocu/7j0k/8jlZu536uWdOxfuITtVTHPLdhci7KlDT3l1a5l9ro94i3+naVE63wSsZ/2Jw9UpSjjpyCyGJ/bzhh1o3CybNEaf5iieAM1G+7trPnCl+ArTf/ohEAFfdEsuLPm6B X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Jan 26, 2024 at 01:51:44AM +0800, Kairui Song wrote: > > > mm/vmscan.c | 30 ++++++++++++++++++++++++++---- > > > 1 file changed, 26 insertions(+), 4 deletions(-) > > > > > > diff --git a/mm/vmscan.c b/mm/vmscan.c > > > index 4f9c854ce6cc..03631cedb3ab 100644 > > > --- a/mm/vmscan.c > > > +++ b/mm/vmscan.c > > > @@ -3681,15 +3681,26 @@ static bool inc_min_seq(struct lruvec *lruvec, int type, bool can_swap) > > > /* prevent cold/hot inversion if force_scan is true */ > > > for (zone = 0; zone < MAX_NR_ZONES; zone++) { > > > struct list_head *head = &lrugen->folios[old_gen][type][zone]; > > > + struct folio *prev = NULL; > > > > > > - while (!list_empty(head)) { > > > - struct folio *folio = lru_to_folio(head); > > > + if (!list_empty(head)) > > > + prev = lru_to_folio(head); > > > + > > > + while (prev) { > > > + struct folio *folio = prev; > > > > > > VM_WARN_ON_ONCE_FOLIO(folio_test_unevictable(folio), folio); > > > VM_WARN_ON_ONCE_FOLIO(folio_test_active(folio), folio); > > > VM_WARN_ON_ONCE_FOLIO(folio_is_file_lru(folio) != type, folio); > > > VM_WARN_ON_ONCE_FOLIO(folio_zonenum(folio) != zone, folio); > > > > > > + if (unlikely(list_is_first(&folio->lru, head))) { > > > + prev = NULL; > > > + } else { > > > + prev = lru_to_folio(&folio->lru); > > > + prefetchw(&prev->flags); > > > + } > > > > This makes the code flow much harder to follow. Also for architecture > > that does not support prefetch, this will be a net loss. > > > > Can you use refetchw_prev_lru_folio() instead? It will make the code > > much easier to follow. It also turns into no-op when prefetch is not > > supported. > > > > Chris > > > > Hi Chris, > > Thanks for the suggestion. > > Yes, that's doable, I made it this way because in previous series (V1 > & V2) I applied the bulk move patch first which needed and introduced > the `prev` variable here, so the prefetch logic just used it. > For V3 I did a rebase and moved the prefetch commit to be the first > one, since it seems to be the most effective one, and just kept the Maybe something like this? Totally not tested. Feel free to use it any way you want. Chris diff --git a/mm/vmscan.c b/mm/vmscan.c index 4f9c854ce6cc..2100e786ccc6 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -3684,6 +3684,7 @@ static bool inc_min_seq(struct lruvec *lruvec, int type, bool can_swap) while (!list_empty(head)) { struct folio *folio = lru_to_folio(head); + prefetchw_prev_lru_folio(folio, head, flags); VM_WARN_ON_ONCE_FOLIO(folio_test_unevictable(folio), folio); VM_WARN_ON_ONCE_FOLIO(folio_test_active(folio), folio); @@ -4346,7 +4347,10 @@ static int scan_folios(struct lruvec *lruvec, struct scan_control *sc, while (!list_empty(head)) { struct folio *folio = lru_to_folio(head); - int delta = folio_nr_pages(folio); + int delta; + + prefetchw_prev_lru_folio(folio, head, flags); + delta = folio_nr_pages(folio); VM_WARN_ON_ONCE_FOLIO(folio_test_unevictable(folio), folio); VM_WARN_ON_ONCE_FOLIO(folio_test_active(folio), folio);