From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83D76C4332F for ; Wed, 16 Nov 2022 03:56:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AE2216B0071; Tue, 15 Nov 2022 22:56:05 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A927D6B0072; Tue, 15 Nov 2022 22:56:05 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 959C96B0073; Tue, 15 Nov 2022 22:56:05 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 859A16B0071 for ; Tue, 15 Nov 2022 22:56:05 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 44F5316052F for ; Wed, 16 Nov 2022 03:56:05 +0000 (UTC) X-FDA: 80137942290.09.B8060F5 Received: from mail-vk1-f174.google.com (mail-vk1-f174.google.com [209.85.221.174]) by imf13.hostedemail.com (Postfix) with ESMTP id EBB8820003 for ; Wed, 16 Nov 2022 03:56:04 +0000 (UTC) Received: by mail-vk1-f174.google.com with SMTP id j24so5413291vkk.0 for ; Tue, 15 Nov 2022 19:56:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=MJUPigI3sWELfUfSnKk0gvyS2tshqqZLPZ8WD6VvUzs=; b=KpDa8nzEGsWuyIMqRf1EIXyoFvK2iD4o2CDtzQzFftNyhuVZ4LRVliV5u1xHUt/MIa DdfMYxV30zfe0YGgh7UrjkjFdA9vCQMAXk8JTF68BrCyyhnA9DlM1M+m80nt5oZbJC15 RxAWILSQ06rpaujH2J6ygK3EAER4qwHpVuj5uYvcXZpR9jX/AyLcz+0YbP/Kqfy3W+JY DiqSYDyXtPLIgS7Bjpln8nD+jFYS8LHnKnC6eh0m883G56GVkZr+vv0J2Pg+h5OzUPxi VRv+Wzp/MvH9nbNf/4WKwz66m/fDe3MWG1C/ARc5CzosQGPIiOcvy7uoFA93TkKk3Jum 79/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=MJUPigI3sWELfUfSnKk0gvyS2tshqqZLPZ8WD6VvUzs=; b=YemlHdtJNng0g2UZT61Iz4IFHTaL5osHvp6RyWIhdJDrr1dZbB7UoQ6pc28tqmaZ4H kJAf7xANp47f5U0HkIVg+zAqayuuT+aIoVbT72YDw3WBUdxD80yXgeOUeV+4dhzHdeu3 FxImpxgeDlZKV9isBp0dDY9zmHCTnGge29Q0VF6UDD9rtJDQ5zCJboNp7n6rq59ci0qG hqSD0R1KkD/S1Qhe4Bk8H0GnjIkhOPSQltISixcP/LL885+X7jJjFj4/Ov1mTdfA8XuR D7reWasbmZhwWaQhj29Q7dDYqD8kNlBkeAB3/0DTYJ43E9j4mAvU0EyjsfbwMs7kCNkz N5kQ== X-Gm-Message-State: ANoB5pmHW/XQqf8DiLnlKsf/XW0QKOpsPOHi/RzM0c8aOvOiasU44cXq I688oZJwWphrvbxqTf73wiOYEbG0FDp08OKs2DEwIA== X-Google-Smtp-Source: AA0mqf4bauHPqSnUcBpk/zUwgx2W3DIhVClfvruICYR5r5/RduXZixeW+D9vc+6W2AqeMxJq6yllUM6Yj2iV9fcUk8A= X-Received: by 2002:a1f:2d2:0:b0:3b7:dc80:1333 with SMTP id 201-20020a1f02d2000000b003b7dc801333mr11777765vkc.30.1668570963975; Tue, 15 Nov 2022 19:56:03 -0800 (PST) MIME-Version: 1.0 References: <20221116013808.3995280-1-yuzhao@google.com> In-Reply-To: From: Yu Zhao Date: Tue, 15 Nov 2022 20:55:27 -0700 Message-ID: Subject: Re: [PATCH 1/2] mm: multi-gen LRU: retry folios written back while isolated To: "Yin, Fengwei" Cc: Andrew Morton , linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1668570965; a=rsa-sha256; cv=none; b=AnP3Ous5BqSO3QFdpPhoigL/0B4DRxyaVQ0WG4oVZfr3L6ovaasjhCbqJXupYl1mI/YsjY 6HHoUPDxc1Mq5EOqcYo3FuvZfZuC9kx3ziMq1s1eoFpdhowPL4To4hEZQug2oUkPyjIgYD T7Rk/byWppseDJyKm3BP6z0aa5HjFHs= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=KpDa8nzE; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf13.hostedemail.com: domain of yuzhao@google.com designates 209.85.221.174 as permitted sender) smtp.mailfrom=yuzhao@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1668570965; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MJUPigI3sWELfUfSnKk0gvyS2tshqqZLPZ8WD6VvUzs=; b=dL84fSzUmuMRRjb35y0alLq1Dbb2A0fOKYX2Mjd7DxspGghAKuLZFseYbZYLplLSS/AHVC mHuKMHJfrwkeIQ/i5RZRUme80RcqdWKYU8a+toIBpeNwq9XXFKaKjfv3TjkxkK54m9oJSw 1/JYC1JwfSBOOMIyYp+1MYaVuxrQxCw= X-Stat-Signature: zrrj65j36o4tyaqosomrjxepkkif9at4 X-Rspamd-Queue-Id: EBB8820003 X-Rspam-User: Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=KpDa8nzE; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf13.hostedemail.com: domain of yuzhao@google.com designates 209.85.221.174 as permitted sender) smtp.mailfrom=yuzhao@google.com X-Rspamd-Server: rspam09 X-HE-Tag: 1668570964-331689 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Nov 15, 2022 at 8:07 PM Yin, Fengwei wrote: > > On 11/16/2022 9:38 AM, Yu Zhao wrote: > > The page reclaim isolates a batch of folios from the tail of one of > > the LRU lists and works on those folios one by one. For a suitable > > swap-backed folio, if the swap device is async, it queues that folio > > for writeback. After the page reclaim finishes an entire batch, it > > puts back the folios it queued for writeback to the head of the > > original LRU list. > > > > In the meantime, the page writeback flushes the queued folios also by > > batches. Its batching logic is independent from that of the page > > reclaim. For each of the folios it writes back, the page writeback > > calls folio_rotate_reclaimable() which tries to rotate a folio to the > > tail. > > > > folio_rotate_reclaimable() only works for a folio after the page > > reclaim has put it back. If an async swap device is fast enough, the > > page writeback can finish with that folio while the page reclaim is > > still working on the rest of the batch containing it. In this case, > > that folio will remain at the head and the page reclaim will not retry > > it before reaching there. > > > > This patch adds a retry to evict_folios(). After evict_folios() has > > finished an entire batch and before it puts back folios it cannot free > > immediately, it retries those that may have missed the rotation. > > > > Before this patch, ~60% of folios swapped to an Intel Optane missed > > folio_rotate_reclaimable(). After this patch, ~99% of missed folios > > were reclaimed upon retry. > > > > This problem affects relatively slow async swap devices like Samsung > > 980 Pro much less and does not affect sync swap devices like zram or > > zswap at all. > > > > Fixes: ac35a4902374 ("mm: multi-gen LRU: minimal implementation") > > Signed-off-by: Yu Zhao > > --- > > mm/vmscan.c | 48 +++++++++++++++++++++++++++++++++++++----------- > > 1 file changed, 37 insertions(+), 11 deletions(-) > > > > diff --git a/mm/vmscan.c b/mm/vmscan.c > > index 04d8b88e5216..dc6ebafa0a37 100644 > > --- a/mm/vmscan.c > > +++ b/mm/vmscan.c > > @@ -4971,10 +4971,13 @@ static int evict_folios(struct lruvec *lruvec, struct scan_control *sc, int swap > > int scanned; > > int reclaimed; > > LIST_HEAD(list); > > + LIST_HEAD(clean); > > struct folio *folio; > > + struct folio *next; > > enum vm_event_item item; > > struct reclaim_stat stat; > > struct lru_gen_mm_walk *walk; > > + bool skip_retry = false; > > struct mem_cgroup *memcg = lruvec_memcg(lruvec); > > struct pglist_data *pgdat = lruvec_pgdat(lruvec); > > > > @@ -4991,20 +4994,37 @@ static int evict_folios(struct lruvec *lruvec, struct scan_control *sc, int swap > > > > if (list_empty(&list)) > > return scanned; > > - > > +retry: > > reclaimed = shrink_folio_list(&list, pgdat, sc, &stat, false); > > + sc->nr_reclaimed += reclaimed; > > > > - list_for_each_entry(folio, &list, lru) { > > - /* restore LRU_REFS_FLAGS cleared by isolate_folio() */ > > - if (folio_test_workingset(folio)) > > - folio_set_referenced(folio); > > + list_for_each_entry_safe_reverse(folio, next, &list, lru) { > > + if (!folio_evictable(folio)) { > > + list_del(&folio->lru); > > + folio_putback_lru(folio); > > + continue; > > + } > dump question: > My understanding: unevictable folios were filtered out in sort_folios. > So this is because folio could become unevictable during retry? Thanks. If a folio is mlocked, it's unevictable. A different thread can call mlock_folio() anytime, so we can see unevictable folios in sort_folios(), here before and after retry, and later in move_folios_to_lru(). In all these cases, we move mlocked folios to the (imaginary) unevictable LRU and __mlock_page() bails out early.