From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B92EAC4332F for ; Fri, 18 Nov 2022 01:40:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 37CBB6B0073; Thu, 17 Nov 2022 20:40:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 305BC6B0075; Thu, 17 Nov 2022 20:40:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 17F338E0001; Thu, 17 Nov 2022 20:40:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 05F536B0073 for ; Thu, 17 Nov 2022 20:40:43 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id CE994C0CB6 for ; Fri, 18 Nov 2022 01:40:42 +0000 (UTC) X-FDA: 80144858724.03.6C7DA83 Received: from mail-vk1-f177.google.com (mail-vk1-f177.google.com [209.85.221.177]) by imf08.hostedemail.com (Postfix) with ESMTP id 808D2160006 for ; Fri, 18 Nov 2022 01:40:42 +0000 (UTC) Received: by mail-vk1-f177.google.com with SMTP id g26so1745394vkm.12 for ; Thu, 17 Nov 2022 17:40:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=Jtt6xlcB/4nLL/J9r8uxn9Q2UOqB3vHffWh2AWYj+TQ=; b=F0TGg3HaUqW2iuCxf6/fVMrsKmLzOhFbhDqetI+cRg4NxKEJ9FA+yCcJsFwBiCZPuM bxa6iBFIr528nZR4Z5ob+Mh3zZ4asHk6PmqvCiAL09bd0D4SerisINcEVjpmUXTyS7XF +W44xm94CxZgF5Oe0qRlrcu46FdHYe3YR10/d794kuCZ7qhl8wqlSVrlfyUXXMtwl/+Z 8ge72K41exAFRDZG13Lf+YSzr9DxBLxXp3gBuEiI86k17SNDO6mbx8cuJa5VWOuE2YIS JbJQ2PFCLYsbI+efGLM2Dqi9sekBl27uRz9U7eSBLdu0cNR6+P72Zb+MiKr12SWbTMRC 4ADw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Jtt6xlcB/4nLL/J9r8uxn9Q2UOqB3vHffWh2AWYj+TQ=; b=SD7vXNuWmlRQB7kK/pFI61Bk2JlpE9SxYkAwFkmDaEjwXPmhT9dK+E8LhOzydp2Onm MX0LPKp/xa63BzCmp5NlcYKuaHnYIBzHFicguJHGK19nSRrtcxZz2Lld8vmAAkYLuOzA oV0eSCSeFBZnMq4qqflLUDRqNJid+FLB2cSTK9GC8C3LuwAC3J0KK4HjdJcbOd9iOyvZ VBKrVbpN2gXr31Egzfa9w29bE3xmRCS2zhcHjJGAGtlZ0rGUHgqVunTSEBeBvEOD0dwX FNezWL1P+wy86yTMWTWO9W7aazdv1WLrMzca7LyO5vCVI71nRmf6FIuHFSDC9JESAv6m cNgg== X-Gm-Message-State: ANoB5pl1H4Y2jj/VaQQ2Ms6uSKoAwolUt+RK+h/a25LAEZqUQZ8IZ2Ap YNnfirfnGSB3mxjUpJekg3Mq64I8e7zjZev5Pwu0HQ88pTzNtw== X-Google-Smtp-Source: AA0mqf6xdzZCLKGGcl2le7TIYRj/WKRn1hKnaXETsyMeSstT2jNF1Voxc/sE2o2IOvYe3yJZDRCmon43PUGAP0xFPOE= X-Received: by 2002:a05:6122:118a:b0:3b7:93d0:aec5 with SMTP id x10-20020a056122118a00b003b793d0aec5mr3057153vkn.24.1668735641547; Thu, 17 Nov 2022 17:40:41 -0800 (PST) MIME-Version: 1.0 References: <20221116013808.3995280-1-yuzhao@google.com> In-Reply-To: From: Yu Zhao Date: Thu, 17 Nov 2022 18:40:05 -0700 Message-ID: Subject: Re: [PATCH 1/2] mm: multi-gen LRU: retry folios written back while isolated To: Minchan Kim Cc: Andrew Morton , linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1668735642; a=rsa-sha256; cv=none; b=JT99l4HZf3s3DhWvf9Y1WP02Bbu4EHI5Rzjh+I+52fWNRpJj9ol85sQ2bzBX6drisHgLZl PK52Ch3KUB/4a0n++7V4cDg63WORiGSHIu1Zv5zUgntg1y50e+6H+BgzTvOpm3HYcXX0Gy PiFfi7xtU9AnbBHXqXgYD6SBMPlPVCQ= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=F0TGg3Ha; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf08.hostedemail.com: domain of yuzhao@google.com designates 209.85.221.177 as permitted sender) smtp.mailfrom=yuzhao@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1668735642; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Jtt6xlcB/4nLL/J9r8uxn9Q2UOqB3vHffWh2AWYj+TQ=; b=VFPQZ89aPjGmLmMWOufffts2RmA0pDDuldCnlI/bQobaXsXN3mTBt1O4XJ9nc5TY/V99mf 0YvyJXr3t0AZ9126J+eya7dtDLONYZq+ERxakPFkpCgLHr3Oxb2uPw3kj0tZkX8o8wZMNy 2x2dO5xAnW7ZLpPtLeVXGbmlyRkKP/A= Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=F0TGg3Ha; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf08.hostedemail.com: domain of yuzhao@google.com designates 209.85.221.177 as permitted sender) smtp.mailfrom=yuzhao@google.com X-Rspam-User: X-Stat-Signature: gfgjaanmfbk8e7t1fjuxx8cns3dsobmr X-Rspamd-Queue-Id: 808D2160006 X-Rspamd-Server: rspam11 X-HE-Tag: 1668735642-467544 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Nov 17, 2022 at 6:26 PM Minchan Kim wrote: > > On Thu, Nov 17, 2022 at 03:22:42PM -0700, Yu Zhao wrote: > > On Thu, Nov 17, 2022 at 12:47 AM Minchan Kim wrote: > > > > > > On Tue, Nov 15, 2022 at 06:38:07PM -0700, Yu Zhao wrote: > > > > The page reclaim isolates a batch of folios from the tail of one of > > > > the LRU lists and works on those folios one by one. For a suitable > > > > swap-backed folio, if the swap device is async, it queues that folio > > > > for writeback. After the page reclaim finishes an entire batch, it > > > > puts back the folios it queued for writeback to the head of the > > > > original LRU list. > > > > > > > > In the meantime, the page writeback flushes the queued folios also by > > > > batches. Its batching logic is independent from that of the page > > > > reclaim. For each of the folios it writes back, the page writeback > > > > calls folio_rotate_reclaimable() which tries to rotate a folio to the > > > > tail. > > > > > > > > folio_rotate_reclaimable() only works for a folio after the page > > > > reclaim has put it back. If an async swap device is fast enough, the > > > > page writeback can finish with that folio while the page reclaim is > > > > still working on the rest of the batch containing it. In this case, > > > > that folio will remain at the head and the page reclaim will not retry > > > > it before reaching there. > > > > > > > > This patch adds a retry to evict_folios(). After evict_folios() has > > > > finished an entire batch and before it puts back folios it cannot free > > > > immediately, it retries those that may have missed the rotation. > > > > > > Can we make something like this? > > > > This works for both the active/inactive LRU and MGLRU. > > I hope we fix both altogether. > > > > > But it's not my prefered way because of these two subtle differences: > > 1. Folios eligible for retry take an unnecessary round trip below -- > > they are first added to the LRU list and then removed from there for > > retry. For high speed swap devices, the LRU lock contention is already > > quite high (>10% in CPU profile under heavy memory pressure). So I'm > > hoping we can avoid this round trip. > > 2. The number of retries of a folio on folio_wb_list is unlimited, > > whereas this patch limits the retry to one. So in theory, we can spin > > on a bunch of folios that keep failing. > > > > The most ideal solution would be to have the one-off retry logic in > > shrink_folio_list(). But right now, that function is very cluttered. I > > plan to refactor it (low priority at the moment), and probably after > > that, we can add a generic retry for both the active/inactive LRU and > > MGLRU. I'll raise its priority if you strongly prefer this. Please > > feel free to let me know. > > Well, my preference for *ideal solution* is writeback completion drops > page immediately without LRU rotating. IIRC, concern was softirq latency > and locking relevant in the context at that time when I tried it. Are we good for now or are there other ideas we want to try while we are at it?