From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 473A1C48BEB for ; Wed, 14 Feb 2024 09:18:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AD1026B0087; Wed, 14 Feb 2024 04:18:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A80F26B008A; Wed, 14 Feb 2024 04:18:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 949866B008C; Wed, 14 Feb 2024 04:18:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 803DE6B0087 for ; Wed, 14 Feb 2024 04:18:34 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 32D3B12064D for ; Wed, 14 Feb 2024 09:18:34 +0000 (UTC) X-FDA: 81789858948.23.C071235 Received: from out-186.mta1.migadu.com (out-186.mta1.migadu.com [95.215.58.186]) by imf01.hostedemail.com (Postfix) with ESMTP id 3249140026 for ; Wed, 14 Feb 2024 09:18:31 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=qqAG8HYO; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf01.hostedemail.com: domain of chengming.zhou@linux.dev designates 95.215.58.186 as permitted sender) smtp.mailfrom=chengming.zhou@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707902312; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ZZXNf1eaZwAP1QgbwSl8cYxTPoxlUmBtCQWVKIvjUxc=; b=UDC+rOj7GTbSNyomKSGZ4dOFONfPelMF5NU4/JU3W0IKj3Y2TfDA5bICVIE2UId3y9R283 naWMTBWpHSTOilicETsPTFfHiLdJp0SH5tsJCWwKhkSLwrSoodiRGqWaZKImzMLmXYb8fc DhLUxOPQ1IMxzbcvouqGA4XtyziPUts= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=qqAG8HYO; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf01.hostedemail.com: domain of chengming.zhou@linux.dev designates 95.215.58.186 as permitted sender) smtp.mailfrom=chengming.zhou@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707902312; a=rsa-sha256; cv=none; b=EKOHApZc8SB7O7Idd+5hFsysJQ2m2yalgoIW9RcHNub+MEGUG7UirdozSJBX5dbj6Uwv7c iVf5MRCL++u8LsUbDEUnIAA0mvHYLpwkPrDeyFFJamHDl3SH+uwfww1xed8ZZCuoHTpo2U VpeVDx6dxn/rSuirNJLegYtJX3iyqWw= Message-ID: <8123c4be-d696-4e9e-884f-aa12f6099ddb@linux.dev> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1707902309; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZZXNf1eaZwAP1QgbwSl8cYxTPoxlUmBtCQWVKIvjUxc=; b=qqAG8HYOUOy4RLm4Prej9qUYtE3O0ivc3/5+pwaaRgfdRe97TVW1ePltlSF9cWJp89oWeo qQ8qdc734NOXoVX3jdETm8NM+tlh6iuhDZ7oCbSVprwZGUGgTgeA5pvEuZtaMl2WBGW/DZ rGCLWATf3VkoXu46CBLk1BOgzOUYPwA= Date: Wed, 14 Feb 2024 17:18:22 +0800 MIME-Version: 1.0 Subject: Re: [PATCH RFC 1/1] mm/swap: queue reclaimable folio to local rotate batch when !folio_test_lru() Content-Language: en-US To: Yu Zhao Cc: willy@infradead.org, hannes@cmpxchg.org, yosryahmed@google.com, nphamcs@gmail.com, akpm@linux-foundation.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Chengming Zhou References: <20240209115950.3885183-1-chengming.zhou@linux.dev> <20240209115950.3885183-2-chengming.zhou@linux.dev> X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Chengming Zhou In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 3249140026 X-Stat-Signature: 9zkfq98hg7ft7k3ux14pm8mwfxymgis4 X-HE-Tag: 1707902311-431259 X-HE-Meta: U2FsdGVkX19zre5BX2vf5BbTtGsAIYY5+WGGlWDLAGhN7KERuRHMzzHvDVnGLpfPXT/Om24Ctmm1IHQyHae8UzOSKq0J4e8uvce6fdKVd0rbNjWLqIOmvcRr8xRIT7QILTO05kaYeQTb1eMsOdcDOaBF3Yk5gJPhCcQXJQDdm+cAp6IKvfC5YieFjpQpCVNYg8MIITBrrloxbsUxlhN/UusJqDUrCH+SeeRCXK+G2vmgmdHjp6dAjI2/b1PK0oQhGHw5WpfGNXCEAh50v7o722rpc53ZXa843FPU4JHFc6RuflrZ0I+0DWrVAlcrLQcUI6bIuf1l6mTW5EAkI9fm9DuteijpUh2+DjOXlP8DdTcvV8jdAIGmpeixxI9xUdyvLuuwhxbroUWx8hXXzlL7UzVsMr3HBLxTvB23w1ghLsy+Jv3EvbQqpSoRkOOiD0vQecMSDz+mJ+lnVb7PBxAEhIlwE59tcgqXY4osZy7cGa1G42LyFjg/ThM/rbyM/X404xTi9pqQw9LE/5rHD7jwIaDlhcNe4ndfbJQSxSg9InTbD1XViKER/guCLBvT+rhP47P7OL3GQv8FK7aSt0xiwBhuJRHGm/urQf5iwmTi7zG5YRJRzkcMVARwQ36/YOUSZJtx7Fl2wJfG52Lh7x35Iou3jFQtS7DlPkaykXRrZTwSfHqkx+phJsVbx9jEX7SxLewvkaD+nmkUJ4EhMBXZCXyndDFcad3oBPAtz9oDdL4cO41WgsTrAQVBf6ZWUAfVAtuF8lawJ7VaZVITj00K2+oUbZpclFqooOa2Pud3eSX+Pt26pnb7IATQdGWaG1ul4+WX9hbivbSvo3UekbEJtPWa/ptaaP6MEjgFuIcd6/928YZmS8EzCrzKzDPhjig7e+GcHsiNS9vCG3vADJJDBM4LxmX7J8QDggafRLYI+TROR7BAlBxHMyKyz6zmOGEMoky7W2aNBgXZwDCrwiU KlEggbvG gOeF8PfHiaiQ8Cz99WS1B23Kc24Lq5UMTuwU2EdlI1dXrRhYThqQmA4uKg4Xvqcw4fqds4KoK4Phy732yYIGi6ayz+uJEFo7KsN2w47ovMTN6xc+tnzvbGjDiwhW+PlXztS4jqwPn3qrpWSqYxq89GImMZkFqM0DiBmblokQmNrgj3MsAXnY3UEPDWgwxx+Dq+zZ8NjdECv5dmkKCGGJXYpYMBuBm9zKEwkxa0SopR5nuij2Dp3+DO5A7KemdECaY8pIV+4obg/xNc6UGCbz8hZmsfrsPUEqa/8GIEoo9Q/8JOULAIhbRWmXKInAvPSw/0cNjO+yxWSYRnQL8hCvHZiQE4w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024/2/14 15:13, Yu Zhao wrote: > On Fri, Feb 9, 2024 at 6:00 AM wrote: >> >> From: Chengming Zhou >> >> All LRU move interfaces have a problem that it has no effect if the >> folio is isolated from LRU (in cpu batch or isolated by shrinker). >> Since it can't move/change folio LRU status when it's isolated, mostly >> just clear the folio flag and do nothing in this case. >> >> In our case, a written back and reclaimable folio won't be rotated to >> the tail of inactive list, since it's still in cpu lru_add batch. It >> may cause the delayed reclaim of this folio and evict other folios. >> >> This patch changes to queue the reclaimable folio to cpu rotate batch >> even when !folio_test_lru(), hoping it will likely be handled after >> the lru_add batch which will put folio on the LRU list first, so >> will be rotated to the tail successfully when handle rotate batch. >> >> Signed-off-by: Chengming Zhou > > I don't think the analysis is correct. IIRC, writeback from non > reclaim paths doesn't require isolation and the reclaim path doesn't > use struct folio_batch lru_add. Ah, my bad, I forgot to mention the important context in the message: This is not from the normal reclaim context, it's from zswap writeback reclaim context, which will first set PG_reclaim flag, then submit the async writeback io. If the writeback io complete fast enough, folio_rotate_reclaimable() will be called before that folio put on LRU list (it still in the local lru_add batch, so it's somewhat like isolated too) > > Did you see any performance improvements with this patch? In general, > this kind of patches should have performance numbers to show it really > helps (not just in theory). Right, there are some improvements, the numbers are put in cover letter. But this solution is not good enough, just RFC for discussion. :) mm-unstable-hot zswap-lru-reclaim real 63.34 62.72 user 1063.20 1060.30 sys 272.04 256.14 workingset_refault_anon 2103297.00 1788155.80 workingset_refault_file 28638.20 39249.40 workingset_activate_anon 746134.00 695435.40 workingset_activate_file 4344.60 4255.80 workingset_restore_anon 653163.80 605315.60 workingset_restore_file 1079.00 883.00 workingset_nodereclaim 0.00 0.00 pgscan 12971305.60 12730331.20 pgscan_kswapd 0.00 0.00 pgscan_direct 12971305.60 12730331.20 pgscan_khugepaged 0.00 0.00 > > My guess is that you are hitting this problem [1]. > > [1] https://lore.kernel.org/linux-mm/20221116013808.3995280-1-yuzhao@google.com/ Right, I just see it, it's the same problem. The only difference is that in your case the folio is isolated by shrinker, in my case, the folio is in cpu lru_add batch. Anyway, the result is the same, that folio can't be rotated successfully when writeback complete. Thanks.