From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C9535C4167B for ; Fri, 8 Dec 2023 07:12:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B55C16B0075; Fri, 8 Dec 2023 02:12:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B069B6B0078; Fri, 8 Dec 2023 02:12:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9F3DB6B0080; Fri, 8 Dec 2023 02:12:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 8CCA96B0075 for ; Fri, 8 Dec 2023 02:12:45 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 5D04D1C0A0E for ; Fri, 8 Dec 2023 07:12:45 +0000 (UTC) X-FDA: 81542783490.20.5BFF721 Received: from out0-219.mail.aliyun.com (out0-219.mail.aliyun.com [140.205.0.219]) by imf20.hostedemail.com (Postfix) with ESMTP id 8668C1C001E for ; Fri, 8 Dec 2023 07:12:42 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=antgroup.com; spf=pass (imf20.hostedemail.com: domain of henry.hj@antgroup.com designates 140.205.0.219 as permitted sender) smtp.mailfrom=henry.hj@antgroup.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702019563; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0jCKjINSeCT77PAEDxhYqvu8YaWjxfjdg/6ob432Tw4=; b=LLy+4IJBSlUtw6gmOo8Jd3ISQANI0u6bQdkmFl5hFnwl9EypF05Qz2gPCe4mlSLXsr2jMa Wgu9tYFuCdk9A5wLbor5woPcEge6MJc0q4FMRp56z25Um9l4EwFCnDYPUTfJzOlaAUVYfE OdvLbxm2UdQwH8m+tXTxi8hMPadBibE= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=antgroup.com; spf=pass (imf20.hostedemail.com: domain of henry.hj@antgroup.com designates 140.205.0.219 as permitted sender) smtp.mailfrom=henry.hj@antgroup.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1702019563; a=rsa-sha256; cv=none; b=zr6NLjzCPaLb6DV9jXs6xSOQqo56f0ACQ2gc/8fEjN/nLq4QPz9yjW+6JiNqBSVjjOoNhv jaK6QXsOixG8kj2qMS9QKlmS0hAChZYKKoauEgzQ4HY3BCaS/ZrJmgd8EoPkXim9efqGmm Z3ClhvOreQ/24hsE4W8hd+6zdg4aMMY= X-Alimail-AntiSpam:AC=PASS;BC=-1|-1;BR=01201311R131e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018047190;MF=henry.hj@antgroup.com;NM=1;PH=DS;RN=6;SR=0;TI=SMTPD_---.VfcfN-q_1702019557; Received: from localhost(mailfrom:henry.hj@antgroup.com fp:SMTPD_---.VfcfN-q_1702019557) by smtp.aliyun-inc.com; Fri, 08 Dec 2023 15:12:37 +0800 From: "Henry Huang" To: yuzhao@google.com Cc: , , "=?UTF-8?B?6LCI6Ym06ZSL?=" , "=?UTF-8?B?5pyx6L6JKOiMtuawtCk=?=" , Subject: Re: [RFC v2] mm: Multi-Gen LRU: fix use mm/page_idle/bitmap Date: Fri, 08 Dec 2023 15:12:35 +0800 Message-ID: <20231208071235.17812-1-henry.hj@antgroup.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 8668C1C001E X-Stat-Signature: m6b66e5t8r547xpk8kxaaj38k4hb63jx X-Rspam-User: X-HE-Tag: 1702019562-242127 X-HE-Meta: U2FsdGVkX1/gjMxiUz03LUzq+0d4vyfPvvRMPgxJHtaoQimVvb73RlqgAvbRRTwA7I/PyvoHJ4KAjs5sYRyNRFuxGrTVNDxwgN1XdKKrkWUI1KgRmm2m3UMgMIaFEQXbhG07U9xnRkr232SR7A4NrhM4TGCYagH9saxoQbxCtCerV+nduTLW8TYlsrGI2vk5/ugrpRMSYg2sTCYKCBf6AWmqaV3WpvFoNdiDY7j4Q2xokypCQLpeQmvQ8l5QcIZV3HDuhgmpn3k8e0uVvJHvD/MdLglCgvFhi/tsdI4sv9Spw5U+xiyhRal5yWNL1fldVQmej7s4Lx3ItXxtTIRPlkLBcva7oTuIUHg1TVRNBd3GMl6aNsWEQm+8+VS86qdbU4MPiQIHh1tF+5WTPiVKXeqie0osMPAaBtWCfPhZLBM8GkokX3NjN6jRj/31TuwRPQY1w8wh5OijPn69ewPmzUxCMYQan089HEHVakTOcsvhhk2nWcm41m9Ej+jD2ePsClsZDGj3fumqxLtWdxDbX3fJF6A1qzZ7h1aFj46kWpsHebgLgg2rcQcnhZfaHQJxffFUz1+6Y7f1UjHG6PC0VcALZAZtlQYeGNyaUQ14lw0mxB0MDSpxvwD92a4vy7fr9rOZGiMwAsMhvD3DnnbGHH4Ch7fO3EhRNMcpiK6JXUmRuvs4JNt0nxPBhttShBsGBh3j7//ik0ldqkvpxbG18Qrju2CVibw6K7llPna1wWCS8s/4j6LvarZ37UAuMZc+aPPXiyBE8o7YbnmSI4TtohR/Lpr3guyacHtPu30t2PNe2wUokkNtftKrw+UX+HBKQyzP+auoBN5JlMLgwrm9yqIPdRSq0kiWRRzTraaqXYb39bfPFVeNIgZOCGikV+I/ECvWbd8nU7FTyHSCYSAumDt8v+yXJA4wyXVMHADNw4k/6aQMuV1BWl4N+wKJz940g0G2jltxjI/zTIXZTLe h7BIBPJ+ +egh+P2qTGfvkj4KARZPYLPovTYrKqaaiWNqrLZKW93z1xXnTEXkuMnymgpB1kqgriHPgIvVjOTTCtNL7Syz/rQzI0/3Vle5/Cp4ZSythPZKNGaw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Thanks for replying this RFC. > 1. page_idle/bitmap isn't a capable interface at all -- yes, Google > proposed the idea [1], but we don't really use it anymore because of > its poor scalability. In our environment, we use /sys/kernel/mm/page_idle/bitmap to check pages whether were accessed during a peroid of time. We manage all pages idle time in userspace. Then use a prediction algorithm to select pages to reclaim. These pages would more likely be idled for a long time. We only need kernel to tell use whether a page is accessed, a boolean value in kernel is enough for our case. > 2. PG_idle/young, being a boolean value, has poor granularity. If > anyone must use page_idle/bitmap for some specific reason, I'd > recommend exporting generation numbers instead. Yes, at first time, we try using multi-gen LRU proactvie scan and exporting generation&refs number to do the same thing. But there are serveral problems: 1. multi-gen LRU only care about self-memcg pages. In our environment, it's likely to see that different memcg's process share pages. multi-gen LRU only update gen of pages in current memcg. It's hard to judge a page whether is accessed depends on gen update. We still have no ideas how to solve this problem. 2. We set swappiness 0, and use proactive scan to select cold pages & proactive reclaim to swap anon pages. But we can't control passive scan(can_swap = false), which would make anon pages cold/hot inversion in inc_min_seq. Is it a good idea to include a interface to config passive scan option? -- 2.43.0