From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A798C3DA6E for ; Fri, 5 Jan 2024 14:10:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D10BC6B02D9; Fri, 5 Jan 2024 09:10:22 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CC0CB6B02DE; Fri, 5 Jan 2024 09:10:22 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B87D96B02E1; Fri, 5 Jan 2024 09:10:22 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id A863A6B02D9 for ; Fri, 5 Jan 2024 09:10:22 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 6ADD040AE5 for ; Fri, 5 Jan 2024 14:10:22 +0000 (UTC) X-FDA: 81645442284.13.7A898EA Received: from mail-lf1-f50.google.com (mail-lf1-f50.google.com [209.85.167.50]) by imf19.hostedemail.com (Postfix) with ESMTP id 683AD1A0022 for ; Fri, 5 Jan 2024 14:10:19 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=Cb4lAK38; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf19.hostedemail.com: domain of hezhongkun.hzk@bytedance.com designates 209.85.167.50 as permitted sender) smtp.mailfrom=hezhongkun.hzk@bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1704463820; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/dfmz42MdXvMCi7ectWieUKdzDvevksoN5fHYgqMM0k=; b=4yrqJD6lLMNO52F4xMOOUQKvM/rUSnTLu22aCNzoLgvwrJAAsuca5d5q+ExRo/m8DDdFSq P1+fC81IBfdnRQGLDbeI00yLzM02v+EO7dML8naCTv7EL7HNJEzc5TQ0J5mOYA5ujgzKg5 B3m/Rjg7EPRlHwQdZHzMFrYcnJU6EWY= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=Cb4lAK38; dmarc=pass (policy=quarantine) header.from=bytedance.com; spf=pass (imf19.hostedemail.com: domain of hezhongkun.hzk@bytedance.com designates 209.85.167.50 as permitted sender) smtp.mailfrom=hezhongkun.hzk@bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1704463820; a=rsa-sha256; cv=none; b=6h/LV/w5P0S5sIJvSPJBBQidPoxj+ffj2e1S3krItzWXCSQBb0w483pNyzC4NOH/+CZ8n+ f5MX2DxN6AA7Yd6e5R01NfiEb4SsZ04vjK3slA7mXdEiunP9iFGe0cEztKdpDaG1ouwRQd 3YrrE3D3UCglhjkqXuTU5VxcIMl6STs= Received: by mail-lf1-f50.google.com with SMTP id 2adb3069b0e04-50eabbc3dccso1762769e87.2 for ; Fri, 05 Jan 2024 06:10:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1704463817; x=1705068617; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=/dfmz42MdXvMCi7ectWieUKdzDvevksoN5fHYgqMM0k=; b=Cb4lAK38+ywyjq7tghTOjBGmH72LZmNCvNZXHx738kf0rQ9/dse1lXIuErxtNF5zNJ qrESAPRrYYFS7xZ++PuE0WX0x80eRdeZeFOyM2nzlD9Oq1oxHjxelalJ8NQdrKh8gmXi obPYGxs9hUpaJWca2LdMIHyrbmwElut470z6o6I1sMMV1s/M5V8fhmlcf3dag6CIWVsK q6KH2Rr/TSocDZkoq33OQIStdXx+jBoRdiHXP980Ckzd2WZ68a/jh9mkMFI9kj//ftbE bMj4p1wQs/wjh/DfCA/WeubeeKuxOGpwSLcQmhADm50p9M1I+i6HfgFkLMKUCNOBot03 y0Sw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704463817; x=1705068617; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/dfmz42MdXvMCi7ectWieUKdzDvevksoN5fHYgqMM0k=; b=rlxlhczn+PH/mQanGBvOII2+5lGFepGtBO/50RPpG6vBqfGTyYv9XgEpb3Z5agBjzD /fN+BNjT1HxVn7meTHKxFYGHHftNUrQ03K2lwmPcY170EHWqGL8s+U2BNb5LDe7eKYbU ISEtgifqv5RfphxH5MzHtA1ZSpITBNTYOCXt/xQc8nrxLp2IOtQWsVKZe0mreJrZhl5z r5d5cnXeSLuxOkSppOy/i68bPcmtHmO6T67GO0iGKtZjt6S1ZORu1XNgi7CzwPsgbtcB wOp2FSd8O/Yt8Cqv2im02ze3qqWdZAv+wiF8lIUgsImY2MfffI3kNPOE6PrRvoFavQDx kwHA== X-Gm-Message-State: AOJu0Yxtm2BqYOmkzgaWrR4SD5v+1W5YAVnx38VhXgBKfj74NurZTXuW rRYer0OTo90uUAz/KM9JsbzsjoQje/mdpZKYbrnEJisF6RO7Og== X-Google-Smtp-Source: AGHT+IFnymXKLPL9NqlZMDlVvqkcLHoX1m1Jq3xz0o1WWGb7AE9j+EWaQNVEHYc4jqZ5eI9x4ye0XqJhmm97eSoGhw0= X-Received: by 2002:a05:6512:48cd:b0:50e:3774:9db6 with SMTP id er13-20020a05651248cd00b0050e37749db6mr518256lfb.196.1704463817498; Fri, 05 Jan 2024 06:10:17 -0800 (PST) MIME-Version: 1.0 References: <20231024142706.195517-1-hezhongkun.hzk@bytedance.com> In-Reply-To: From: Zhongkun He Date: Fri, 5 Jan 2024 22:10:06 +0800 Message-ID: Subject: Re: [External] Re: [PATCH] mm: zswap: fix the lack of page lru flag in zswap_writeback_entry To: Nhat Pham Cc: akpm@linux-foundation.org, hannes@cmpxchg.org, yosryahmed@google.com, sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Chris Li Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 683AD1A0022 X-Stat-Signature: g8ni6ykg9mikyauwzbhjy6otnm43saz5 X-HE-Tag: 1704463819-138730 X-HE-Meta: U2FsdGVkX18zBBcUQ/W8SVHQKBo0qL+c8ZHsBk1/ENAWGCSnEWKkDIP2cj14lhgBe/tqxdvVFxBdi9/nOAghcInoz6m/oIyZs77RkUC63LDdc+xeolHJOBDfSRN2P+VSKHhGU/rHixytWwhqz3x5CLSr2Ctp3QcfIhAuDRDYGLDbVp/93u949DKe+OPJB7kERbV9+jxOo5MfHF4uD/0qtNYN0Vs62FcoTyX8LMsXjIcKaV6TV3gplkVOKF2FxGK8btBPS3u8OlH8LSIsxPM9MG6hD2axxqfQF0TRT/lrzHq2HxNYUDg4ps10ttymVbMkCPBJlAqAvwQKrbtji9QnbP3bRKXdxBv8moJtK4o9bIyKmA6SqBva3FF2x9twLvdl6Fn522kfPNiw7Ak+Z9CEKaTynZKVTPQz6mSV2KQBA+ewo4MANim9VzyV88niWH1yIDLyzPegA+TXPn3tx5lIJj0ThYf1IN/ArSJk0zq0vuJRmt9SIxjdoA3+5YKdg7s1zyecBuBnUuNKx1Q9CgOXBbtibAbiJv4cpYR6tOAbl4z23vcpEuWPgFgmB+QIhWFXPa8o8BAUUS3+qEBvD3aYPGL6jyAvKqCeheqb+0xZSiG0HA2wHWmdTH4Q83OXzNjrK4ySo+n0I0bJJARDn/lBVqlmnKwsFU409EzkGWa9FDVKU02uVw/4RTVmDzw379Qdgsh92MaSX2iO4HxGD6+1AsxsNs91/VuOxEiOmpfNXpkm8HFADfEXrL8OAZm+OM+qOK+3pmRLvAvA4p7X1BWaGn0lUcM33a0ULvAoFGdSd+rZ9sr5R7ZQ/M6gX0MxiDrwpa8vEy3AwIzMd/mmlgSne0taN+pkdOhvCnWk3BfQsS9c4ESSl0IoHx/qFlBEsbIgxtojjxynti3d/UkgbkTJvy8TGg6nQSvNg2U9WSwA3y1zsN0iwfJWs0HnIvdtD/3bKIiiyZDiAMNQ3YwC530 ltCyAF7z fBRN63clZUr72kTaqrhKrTYIAWS+RHbL8t38B7yPHmWq/R9szW4NpMBf9gyREfAd76OORIE/orpMs/nv0lcfQ8OpEm2vSeBXVzp3IoKUVVnqNsbw9l11Htz5nzaV8wFfKPCKnjfXKBh5CC05havPlp+8uqwhcFEM1Ws59eUXlzQ5t3v2P8R5nhYwfXSANmqgPYm5hMYi+yWNQk92aoxYP3xCD1xFbKv/XIrLDNFy1M7DT06NuNz1nEBIOBWTXM79S+jarWwwKjmQmqxE20QiRHOgC9aLEkt9rY71mkNkfL3cDPDWJsy7dEiqVe5rIaTpiz3KmzsJkFh0NKVyTDJie3Xsgw9lSAjuXSffu X-Bogosity: Ham, tests=bogofilter, spamicity=0.000003, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: > > Hmm originally I was thinking of doing an (unconditional) > lru_add_drain() outside of zswap_writeback_entry() - once in > shrink_worker() and/or zswap_shrinker_scan(), before we write back any > of the entries. Not sure if it would work/help here tho - haven't > tested that idea yet. > The pages are allocated by __read_swap_cache_async() in zswap_writeback_entry() and it must be newly allocated=EF=BC=8C not cached in swap. Please see the code below in zswap_writeback_entry() page =3D __read_swap_cache_async(swpentry, GFP_KERNEL, mpol, NO_INTERLEAVE_INDEX, &page_was_allocated); if (!page) { goto fail;} /* Found an existing page, we raced with load/swapin */ if (!page_was_allocated) { put_page(page); ret =3D -EEXIST; goto fail; } So when it comes to SetPageReclaim(page), The page has just been allocated and is still in the percpu batch, which has not been added to the LRU. Therefore=EF=BC=8Clru_add_drain() did not work outside the zswap_writeback_entry(=EF=BC=89 > > > > New test: > > This patch will add the execution of folio_rotate_reclaimable(not execu= ted > > without this patch) and lru_add_drain,including percpu lock competition= . > > I bind a new task to allocate memory and use the same batch lock to com= pete > > with the target process, on the same CPU. > > context: > > 1:stress --vm 1 --vm-bytes 1g (bind to cpu0) > > 2:stress --vm 1 --vm-bytes 5g --vm-hang 0=EF=BC=88bind to cpu0=EF=BC=89 > > 3:reclaim pages, and writeback 5G zswap_entry in cpu0 and node 0. > > > > Average time of five tests > > > > Base patch patch + compete > > 4.947 5.0676 5.1336 > > +2.4% +3.7% > > compete means: a new stress run in cpu0 to compete with the writeback p= rocess. > > PID USER %CPU %MEM TIME+ COMMAND P > > 1367 root 49.5 0.0 1:09.17 bash =EF=BC=88writeb= ack=EF=BC=89 0 > > 1737 root 49.5 2.2 0:27.46 stress (use percpu > > lock) 0 > > > > around 2.4% increase in real time,including the execution of > > folio_rotate_reclaimable(not executed without this patch) and lru_add_d= rain,but > > no lock contentions. > > Hmm looks like the regression is still there, no? Yes, it cannot be eliminated. > > > > > around 1.3% additional increase in real time with lock contentions on = the same > > cpu. > > > > There is another option here, which is not to move the page to the > > tail of the inactive > > list after end_writeback and delete the following code in > > zswap_writeback_entry(), > > which did not work properly. But the pages will not be released first. > > > > /* move it to the tail of the inactive list after end_writeback */ > > SetPageReclaim(page); > > Or only SetPageReclaim on pages on LRU? No, all the pages are newly allocted and not on LRU. This patch should add lru_add_drain() directly, remove the if statement. The purpose of writing back data is to release the page, so I think it is necessary to fix it. Thanks for your time, Nhat.