From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C775C4829E for ; Thu, 15 Feb 2024 07:07:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AF47C6B00AB; Thu, 15 Feb 2024 02:07:27 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AA2666B00AC; Thu, 15 Feb 2024 02:07:27 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 943016B00AD; Thu, 15 Feb 2024 02:07:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 801406B00AB for ; Thu, 15 Feb 2024 02:07:27 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 6292CA26B4 for ; Thu, 15 Feb 2024 07:07:27 +0000 (UTC) X-FDA: 81793157334.09.6FE4C27 Received: from mail-wm1-f42.google.com (mail-wm1-f42.google.com [209.85.128.42]) by imf15.hostedemail.com (Postfix) with ESMTP id B8474A0009 for ; Thu, 15 Feb 2024 07:07:24 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=rP4pWGAL; spf=pass (imf15.hostedemail.com: domain of yuzhao@google.com designates 209.85.128.42 as permitted sender) smtp.mailfrom=yuzhao@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707980844; a=rsa-sha256; cv=none; b=mLeRBeUfqNMDLImDPUeWDtTXrnEdI45Jpt3bHh0BeWtip2sJ6tz0qxPo2KEhU1ezFlvk4g 1B8Dy6wOZ0JTflFXFJBB+Z0pF7YqN/SJnuTuZg05SusDPzSZvxnebODmT5omTOWph/kedj rq796Rtl1MwTfwHLfxi7l+12pzd7Fq4= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=rP4pWGAL; spf=pass (imf15.hostedemail.com: domain of yuzhao@google.com designates 209.85.128.42 as permitted sender) smtp.mailfrom=yuzhao@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707980844; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=nddqiVq/3r6Pac3WeXAXpKp40eZ00zZsFZh1wnDRavA=; b=GJC5wHSooMNyVf+I+v67Low43lVH/2OzVVJtJYwrW+B51LmchlVgsoff7LXUVP/lcGdInq w4gByuK4AFJOpGuVECaU5hRGzsJPDCJ6k8E05Kk2PcY3z15uNHJaBUJhr+inoLLO06gVsl ngkgrjpQtT3DrkXsAoyIBuYLEKzsKSM= Received: by mail-wm1-f42.google.com with SMTP id 5b1f17b1804b1-410acf9e776so34215e9.1 for ; Wed, 14 Feb 2024 23:07:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1707980843; x=1708585643; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=nddqiVq/3r6Pac3WeXAXpKp40eZ00zZsFZh1wnDRavA=; b=rP4pWGALG+WpJlmlZH/J6H5fMn5xHGo2Z23OgAm+3NHT79KKS3hkqsyYVDzj7ZNrQs fQ9oqb3rsIR+Sq4ok4ZIq4au4xscZjRccQsyqFN3tiMd3EC4dQkRfCWrYUkps6iytA26 36xMC7Dxe6CPhLSpQ49esUXcLX5XMYTPJNiUR7eFMDbSdFVj9GQYBjIFAv+qfWTFdSHF dipGZHqdWn4FClrV2dOGvUXWYHKWYOHzUkFLowDHE3UrJWgPdQL/w3FfG0Q24NYXT/VP eyseGv+W8Z0qp2S9KyWRxIkrK9PZIX+jmDusRnmMwpxDZmtYOgMNpiUva6upZnRi6rsI 9aJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1707980843; x=1708585643; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nddqiVq/3r6Pac3WeXAXpKp40eZ00zZsFZh1wnDRavA=; b=BD7TORCcNKL/+Tj8rvfijR8hsDJA5NW+qQ+E252pNyVNt2x0cCVgpmDqQQrrPtbvGk qDtY/1GPtXYpSaYGKuB7jEAr/Af74tS3RgcQ3nAhu4HMZuhlyNnxu3/hcvqfyddKoRXQ ub9CXpv9Orc5AthBcLe1ViqA+teIU945+A8EVa6tEvEYysLG/GUbqu7k7MNxIAZsm+lS KHX9l02VO5yVKPIQmMXREmZCD5q7cOLgPYm3ztQNPH5vjANd63NN4Zvwm0ysXoPJu1to d1MqV8rncTeaCEnFG9CRCUwDWeao6pE+VCYdSi8pyiIE1VXNXrLBrIC4ZNbVcgLex9pA YoYg== X-Forwarded-Encrypted: i=1; AJvYcCXo/n66mGlWrQg2M0/Acj/jHkWxrI0pIYXIOUtMYtK7zkziDWVv5xzk1bw8YWDuKHcP3vBPfY/gsCoOuI9qxxFc0nM= X-Gm-Message-State: AOJu0YyMcrIOYEcAWZ/cdP+a08v4fJGNZdPgJ4HEZQ/g4ogyPrR2mRsN x1Wmv76esQB4Eni7v4Dm/p6rzLoOP0p5yHnCpZ0EjyqeLflLQncddqQVdGGVyeRC+oMXehyumzi OPFgf7l9mXslaTrTxgwlvOkMuhtCpU7lAUP4o X-Google-Smtp-Source: AGHT+IEZGK7rs/84Y0QnbGxrQrX9DEMxASQW0pm5voFFt9sqVtOtLAZsKqN9qO601ExmYB592dcSsKBKGPOQoW+IK6I= X-Received: by 2002:a05:600c:4e12:b0:411:f6b6:faf5 with SMTP id b18-20020a05600c4e1200b00411f6b6faf5mr249572wmq.3.1707980843006; Wed, 14 Feb 2024 23:07:23 -0800 (PST) MIME-Version: 1.0 References: <20240209115950.3885183-1-chengming.zhou@linux.dev> <20240209115950.3885183-2-chengming.zhou@linux.dev> <8123c4be-d696-4e9e-884f-aa12f6099ddb@linux.dev> In-Reply-To: <8123c4be-d696-4e9e-884f-aa12f6099ddb@linux.dev> From: Yu Zhao Date: Thu, 15 Feb 2024 02:06:46 -0500 Message-ID: Subject: Re: [PATCH RFC 1/1] mm/swap: queue reclaimable folio to local rotate batch when !folio_test_lru() To: Chengming Zhou Cc: willy@infradead.org, hannes@cmpxchg.org, yosryahmed@google.com, nphamcs@gmail.com, akpm@linux-foundation.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Chengming Zhou Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: B8474A0009 X-Stat-Signature: tu9skpnjgx5xgjsee33jx9n7youkegy8 X-Rspam-User: X-HE-Tag: 1707980844-831001 X-HE-Meta: U2FsdGVkX1+wsUQS0CyhkdefvXA+ZjCX68ptt35aDFB8EZ3KL6JBU1KoHbrcChIKPYVZwGfdGxfZjYXF9fO8/SUFRS8D/IOjCIJHfIhc9A4pEMhNxbOE8QV9N7wQ9RfY7PeffaXOBWXkn64L+/R23ZQHW3aQaDfRb0mDDaDP09VNNdrQkJZjYJwEzUDQA72yyJ+UIZQuEMFbUb6U3qaYG2xOsa6ci1iXJJ+2BfAuleoWYhzJ6MoEVMg0tKxsspifD/0WDgskiR+KkZuGBXnoU1s1I8T8SXLzhfyFdavBjUns8jXjYIq3Onyhri1mXxLMy19m/I+pepfpAD6INiDi7iQcEAi5ev6hS+sopeEHO3eXm04N9dImvf5UbUTKC+JDRirb3QlQCosCZPpPUysJZA2wdN4SK+tqZMleclnqPbBWhfib0RH5Om91mEwbEadyYgmVfWLIKfzRhVYjSDo7p1jEQ5F1DA7J6h/9FbN8+QepoSG32XGdqwdcPf9B++wQ3nvDTgusI4ofzFSptf0WY0lwg4WrguOcoO5AoDRCKuZcqwFkeT82LP5GIC7iJ67eggI/JXxJaeUsDy9vZIvO31iYqQt0hbWC4YZnbPxy3JE3d6qrwwYlR/qwsym2BT/2u9N2kMTcINqaHK3L2sIxJg9FVk4DNHlscRyIGGj/Mr8vk+0zUKDdtNYgvdi/vIvGBsCyE+Eagdx/QkzmpyGhZ2ok1SjPaSve3mR01cUKRqAFw7eUZG8KbPjml+2g9VWYK1Uj2OGzSKZbT1sySoFcNQtyff9pnKd0Ncq/yl2VVJ5V66OS629D22Qx3GYmtaJWUCnSrRwMCL99t2Mz5E1wQCvEKz2wjlBN8WNpraHe4SM17m7LN4D9g/9yV3xJCH5GdMl9UMRX1B39NKHUxotPWsz7eaYc/akFwvWk/qhwNQqziU8Kl5fpC3j3JDc2g+HJnLKYkJ85Bwn/Khq7fuH 24p9CMvV H6qssHE6uUcfBQ5kqtKgqQBuGsCTaUU31iSdwrTccYzMRPQQUs6v0U2trmvD4Koa2MI9/pz3QJGR0J0mL03RZmdZrpQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Feb 14, 2024 at 4:18=E2=80=AFAM Chengming Zhou wrote: > > On 2024/2/14 15:13, Yu Zhao wrote: > > On Fri, Feb 9, 2024 at 6:00=E2=80=AFAM wrote= : > >> > >> From: Chengming Zhou > >> > >> All LRU move interfaces have a problem that it has no effect if the > >> folio is isolated from LRU (in cpu batch or isolated by shrinker). > >> Since it can't move/change folio LRU status when it's isolated, mostly > >> just clear the folio flag and do nothing in this case. > >> > >> In our case, a written back and reclaimable folio won't be rotated to > >> the tail of inactive list, since it's still in cpu lru_add batch. It > >> may cause the delayed reclaim of this folio and evict other folios. > >> > >> This patch changes to queue the reclaimable folio to cpu rotate batch > >> even when !folio_test_lru(), hoping it will likely be handled after > >> the lru_add batch which will put folio on the LRU list first, so > >> will be rotated to the tail successfully when handle rotate batch. > >> > >> Signed-off-by: Chengming Zhou > > > > I don't think the analysis is correct. IIRC, writeback from non > > reclaim paths doesn't require isolation and the reclaim path doesn't > > use struct folio_batch lru_add. > > Ah, my bad, I forgot to mention the important context in the message: > > This is not from the normal reclaim context, it's from zswap writeback > reclaim context, which will first set PG_reclaim flag, then submit the > async writeback io. > > If the writeback io complete fast enough, folio_rotate_reclaimable() > will be called before that folio put on LRU list (it still in the local > lru_add batch, so it's somewhat like isolated too) > > > > > Did you see any performance improvements with this patch? In general, > > this kind of patches should have performance numbers to show it really > > helps (not just in theory). > > Right, there are some improvements, the numbers are put in cover letter. > But this solution is not good enough, just RFC for discussion. :) > > mm-unstable-hot zswap-lru-reclaim > real 63.34 62.72 > user 1063.20 1060.30 > sys 272.04 256.14 > workingset_refault_anon 2103297.00 1788155.80 > workingset_refault_file 28638.20 39249.40 > workingset_activate_anon 746134.00 695435.40 > workingset_activate_file 4344.60 4255.80 > workingset_restore_anon 653163.80 605315.60 > workingset_restore_file 1079.00 883.00 > workingset_nodereclaim 0.00 0.00 > pgscan 12971305.60 12730331.20 > pgscan_kswapd 0.00 0.00 > pgscan_direct 12971305.60 12730331.20 > pgscan_khugepaged 0.00 0.00 > > > > > My guess is that you are hitting this problem [1]. > > > > [1] https://lore.kernel.org/linux-mm/20221116013808.3995280-1-yuzhao@go= ogle.com/ > > Right, I just see it, it's the same problem. The only difference is that > in your case the folio is isolated by shrinker, in my case, the folio is > in cpu lru_add batch. Anyway, the result is the same, that folio can't be > rotated successfully when writeback complete. In that case, a better solution would be to make lru_add add (_reclaim() && !_dirty() && !_writeback()) folios at the tail. (_rotate() needs to leave _reclaim() set if it fails to rotate.)