From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F5EEC4829E for ; Sun, 18 Feb 2024 08:09:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A4BA06B009B; Sun, 18 Feb 2024 03:09:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9FC956B009C; Sun, 18 Feb 2024 03:09:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 89DB46B009D; Sun, 18 Feb 2024 03:09:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 762106B009B for ; Sun, 18 Feb 2024 03:09:09 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id E32BFC012B for ; Sun, 18 Feb 2024 08:09:08 +0000 (UTC) X-FDA: 81804199176.01.EE1B8FF Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com [209.85.128.43]) by imf06.hostedemail.com (Postfix) with ESMTP id 3C68A180009 for ; Sun, 18 Feb 2024 08:09:06 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=X1XZEBoZ; spf=pass (imf06.hostedemail.com: domain of yuzhao@google.com designates 209.85.128.43 as permitted sender) smtp.mailfrom=yuzhao@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1708243747; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OfqijOxZRqV8G3+K8nq9Fq3ADRHflW4wUyZr5KDyQS4=; b=PwGkPoG98NCgW3zlmHra/i7bkf58TjChI8mmRX9XlcO2hYspoihJ20a1baKRwplDu3f2F/ Mhxk87vsnWp7WKPKWeAy6a6vFf2EwuBAadE+0LjB4pCFGVp78Y7y9v7GVXtZVKLfwnX5iF GpRCIl2vLLlXNlLc8/OqxSMk0pqFVfo= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1708243747; a=rsa-sha256; cv=none; b=pHQ6qDhWkAL8C5fwL4mHGIPP1rRo3akyKcdkkr5QdNmVfNlj+lt7cnoxtkRvVS5ggqyPOs IpNCuzTQfnuBt+t/4zmwk5Vg899EP4l9NQZp2MO1bl+/4Lpmwvm+2jJv8J74xH9FNhKsk9 kQfPG6IdUZKTkbVzoqok7QdXzRiFcTM= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=X1XZEBoZ; spf=pass (imf06.hostedemail.com: domain of yuzhao@google.com designates 209.85.128.43 as permitted sender) smtp.mailfrom=yuzhao@google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-wm1-f43.google.com with SMTP id 5b1f17b1804b1-410acf9e776so39725e9.1 for ; Sun, 18 Feb 2024 00:09:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1708243745; x=1708848545; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=OfqijOxZRqV8G3+K8nq9Fq3ADRHflW4wUyZr5KDyQS4=; b=X1XZEBoZX6eVFuBybc1uVZzPiS4MXlvE7qiaPMHk0KcLu9Hik8P0w0Fd6I+Yrzchd1 GiQ9VdkN1gStdIKM2YDirEkcjQF2cuyeS0r7j+pyKDUx1t9A7glt9sdmHJ4O7Dp7NGGB J20oYgdsfTE8ovkWCe/7P97wTi0HXP83TH/hhMIjGFlpB18q66RcWRtu4CttKTqqqeaa kTdFZDmHKXiC6oBYXVtXprpqDokkVRn6KFwYTZ7/ypXjjowQCmju1EAHMPsUyfBMFayi Dq+LqMXcEgmIwoKlbVdkKLHVWZi/fwNHAL9suoO3ZJn0OtZ0X/gs82VfQQT8p+NTCelg 8dGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708243745; x=1708848545; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OfqijOxZRqV8G3+K8nq9Fq3ADRHflW4wUyZr5KDyQS4=; b=LgTq0nk4FARfrUIeOp5CYVnR3jq6sWlwwWIxG8ZczKznw3B0zlEfDAkad8Q2vFMgZ/ m2FoAchFFhUGc2Sm9ZJ2dYTl0f7XYVId6UVFP+nIXKtv0x67OUF+NbnhQ63TZrbEkfaX FuqF5rWY4DRin97SqdxMHx9dAa6l4DOUxxep2XS/yC/cy1M0Hbpamd6gDW8QhF2BpOnv MQ8EWsjO9tEKafred7qDWOjztHXltYkkb3NcdfueT+cgQiA267YeZv3A3VzTO+KEAlKo JnrnrrlkMywEu2g4wQRcE+KJPxHH5o9rQQw0jBASNrs6g6KmR5sawOL/RZGA9LUStui9 PYbQ== X-Forwarded-Encrypted: i=1; AJvYcCUap1I+yhwfXmswlk9nZ/sup7Z4GpLW8ivyR/CdneMbosb/LBxhAdUdoHwUM1/rL4vl4K0SrSUeRW1bCz8mcrB46SM= X-Gm-Message-State: AOJu0Yzp00CeHc3weZ/xTHSW94MdSfVWNCJMYRjHGDcR84bEx4kTPRLz Xs2phfBDypSBKYMmA8yyYoYQJAr2DZYDztkIl6nTr98xmc0SFPyFLAFu4xQwxBNburCuiKOCKhO 8N6iGzufdHJPxM04tcTDzLJyYzmPitkPTAScg X-Google-Smtp-Source: AGHT+IEvcmX6w18+RZ4VvQtXTZFiX72nSc5+q6s3K5JVP0z008xlrObQ3vJyH+wszldvDNrIN7BDsh7Xu1LTB9IbjaI= X-Received: by 2002:a05:600c:4e49:b0:412:6101:912e with SMTP id e9-20020a05600c4e4900b004126101912emr55157wmq.5.1708243745467; Sun, 18 Feb 2024 00:09:05 -0800 (PST) MIME-Version: 1.0 References: <20240209115950.3885183-1-chengming.zhou@linux.dev> <20240209115950.3885183-2-chengming.zhou@linux.dev> <8123c4be-d696-4e9e-884f-aa12f6099ddb@linux.dev> In-Reply-To: From: Yu Zhao Date: Sun, 18 Feb 2024 03:08:27 -0500 Message-ID: Subject: Re: [PATCH RFC 1/1] mm/swap: queue reclaimable folio to local rotate batch when !folio_test_lru() To: Chengming Zhou Cc: willy@infradead.org, hannes@cmpxchg.org, yosryahmed@google.com, nphamcs@gmail.com, akpm@linux-foundation.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Chengming Zhou Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: 76wx3z5desebh3xyyejg5nuknnuhqxa8 X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 3C68A180009 X-Rspam-User: X-HE-Tag: 1708243746-598287 X-HE-Meta: U2FsdGVkX1+ixRnwg/5il/SAFgomTXJAMG0ecs/4Ds3fPDr3z1kKtn5E3DucHI4M52fFtv1ZPVm5GKNLhp95ljrDq4tNvTAhkkqz7JCdneOxmZZaOSz9hoYsa+9Qw6PfJKce989zQdq27zg2hgwwyHadXXafirm0rWOO9Nb1Wq4/iurEjJaU0/xkW+Zifpu47S7OcIEDXpaDUwsrNQeMXrzdDv00rzsUTKfWIz5yJXyu1r8FPv8TQjGiUVrA23X5FwZhou+ItnUmgZRtSBnCrCMmKtiW6BxkeC6AWiVTRy8OEltRZml0R9eY7I9yFXJEkAbgU/j/QeDTelMwgJIclVQOuDBvpmADCD2ylFoGAlnLLOVWUzF176KrvbHOjCSxhvq2RaHDCd84lpd31N0+xwaWH+q5UlvDdxW6xfSQSJfHf/1toD/0naby9HYCs5NfWPA1E4MvG3tD4b7YCpYvK3AYnwUZZmCHF9iHtrEvwOIZj04SG0mjwwvbHiuJcTBTWPBS84CGpLDp84+OJqKorL4zm2D9MvaBeylu8tqbLft9Pi666Yili75gpIVqmwvXXhof50lnmvth4pRByGKfs94n8MwDaTnL5iHFrupXmCxEtisHgmGWhu/LyIurKKHIO8GaWJPZ6hxD1etvsFyLXoiKkmNub9GOl6vBNUV2PNlQCrALFovo1UMNh1IjI1UEvAcOk/oq55aPRbf8XuJgR4amNDAXpN8JJYvtprJOzvKTENlnryGTVGJyzQEDgDxpczmDVQKSEPAFdt7P0JhgKQ5fJLm4pQ3TXaChMelV2C1gzU1hxBaOigZAE6yBWnXkI/gY9zZQL3GiIcfcB6V5Y1Lh2z5fBfA5DDSljgmHi54UQile1AfLDDiOJFNFDNJ+hbyOdFUuWSrfboXj8HMTAftquMrkZeHZmr4bZ4kF6HOI1kejVbyz30XoyTJRw/81NISMvKA4NQyyeSOcDoD bCglzoRe +gIg0RwHRbdPF7lpYuTzwaeskjFPsrCUN4d3m9twz1xGfrQdwgh7wlnjnRpae8ta02Sn2L4Mkqxhtqd4tcX3kFkp49Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Sat, Feb 17, 2024 at 9:52=E2=80=AFPM Chengming Zhou wrote: > > On 2024/2/15 15:06, Yu Zhao wrote: > > On Wed, Feb 14, 2024 at 4:18=E2=80=AFAM Chengming Zhou wrote: > >> > >> On 2024/2/14 15:13, Yu Zhao wrote: > >>> On Fri, Feb 9, 2024 at 6:00=E2=80=AFAM wro= te: > >>>> > >>>> From: Chengming Zhou > >>>> > >>>> All LRU move interfaces have a problem that it has no effect if the > >>>> folio is isolated from LRU (in cpu batch or isolated by shrinker). > >>>> Since it can't move/change folio LRU status when it's isolated, most= ly > >>>> just clear the folio flag and do nothing in this case. > >>>> > >>>> In our case, a written back and reclaimable folio won't be rotated t= o > >>>> the tail of inactive list, since it's still in cpu lru_add batch. It > >>>> may cause the delayed reclaim of this folio and evict other folios. > >>>> > >>>> This patch changes to queue the reclaimable folio to cpu rotate batc= h > >>>> even when !folio_test_lru(), hoping it will likely be handled after > >>>> the lru_add batch which will put folio on the LRU list first, so > >>>> will be rotated to the tail successfully when handle rotate batch. > >>>> > >>>> Signed-off-by: Chengming Zhou > >>> > >>> I don't think the analysis is correct. IIRC, writeback from non > >>> reclaim paths doesn't require isolation and the reclaim path doesn't > >>> use struct folio_batch lru_add. > >> > >> Ah, my bad, I forgot to mention the important context in the message: > >> > >> This is not from the normal reclaim context, it's from zswap writeback > >> reclaim context, which will first set PG_reclaim flag, then submit the > >> async writeback io. > >> > >> If the writeback io complete fast enough, folio_rotate_reclaimable() > >> will be called before that folio put on LRU list (it still in the loca= l > >> lru_add batch, so it's somewhat like isolated too) > >> > >>> > >>> Did you see any performance improvements with this patch? In general, > >>> this kind of patches should have performance numbers to show it reall= y > >>> helps (not just in theory). > >> > >> Right, there are some improvements, the numbers are put in cover lette= r. > >> But this solution is not good enough, just RFC for discussion. :) > >> > >> mm-unstable-hot zswap-lru-reclaim > >> real 63.34 62.72 > >> user 1063.20 1060.30 > >> sys 272.04 256.14 > >> workingset_refault_anon 2103297.00 1788155.80 > >> workingset_refault_file 28638.20 39249.40 > >> workingset_activate_anon 746134.00 695435.40 > >> workingset_activate_file 4344.60 4255.80 > >> workingset_restore_anon 653163.80 605315.60 > >> workingset_restore_file 1079.00 883.00 > >> workingset_nodereclaim 0.00 0.00 > >> pgscan 12971305.60 12730331.20 > >> pgscan_kswapd 0.00 0.00 > >> pgscan_direct 12971305.60 12730331.20 > >> pgscan_khugepaged 0.00 0.00 > >> > >>> > >>> My guess is that you are hitting this problem [1]. > >>> > >>> [1] https://lore.kernel.org/linux-mm/20221116013808.3995280-1-yuzhao@= google.com/ > >> > >> Right, I just see it, it's the same problem. The only difference is th= at > >> in your case the folio is isolated by shrinker, in my case, the folio = is > >> in cpu lru_add batch. Anyway, the result is the same, that folio can't= be > >> rotated successfully when writeback complete. > > > > In that case, a better solution would be to make lru_add add > > (_reclaim() && !_dirty() && !_writeback()) folios at the tail. > > (_rotate() needs to leave _reclaim() set if it fails to rotate.) > > Right, this is a solution. But PG_readahead is alias of PG_reclaim, > I'm afraid this would rotate readahead folio to the inactive tail. Then drain before setting readahead, since readahead isn't set on every fol= io.