From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32364C4706C for ; Fri, 12 Jan 2024 04:40:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 97F076B0085; Thu, 11 Jan 2024 23:40:37 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 92F466B0089; Thu, 11 Jan 2024 23:40:37 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 81FA16B008A; Thu, 11 Jan 2024 23:40:37 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 702746B0085 for ; Thu, 11 Jan 2024 23:40:37 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 31C26A1F54 for ; Fri, 12 Jan 2024 04:40:37 +0000 (UTC) X-FDA: 81669408114.02.FBA0F92 Received: from out0-209.mail.aliyun.com (out0-209.mail.aliyun.com [140.205.0.209]) by imf24.hostedemail.com (Postfix) with ESMTP id 1E399180011 for ; Fri, 12 Jan 2024 04:40:33 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=none; spf=pass (imf24.hostedemail.com: domain of henry.hj@antgroup.com designates 140.205.0.209 as permitted sender) smtp.mailfrom=henry.hj@antgroup.com; dmarc=pass (policy=quarantine) header.from=antgroup.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1705034435; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xfOxzzXYXmHCZ083e+lA7aFwvPpCcgTSJ2++Tqb/A8g=; b=mRcPnUqxJumSycjukLJzJzJm/oqGCX1W33qMW//YJtQh+9xHB5M1IJ9pgLbOzWytuSkElW GItg93u/GvBEVF3M4DbbL5vzsD+jaq8vC6DIGGSK9Ef4YmXCkoIhXNNeaOETHS2DYtwhYp +ZxQ0wqU3mb4ZqJCgCDJXf0vSspc76A= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1705034435; a=rsa-sha256; cv=none; b=VP0jS7UB4SG38LVd1dh/c3Ksf29lOTE0MdiuLqfsvU11VEI9PEQpOKcRSFZdrCzurZqwJX 2UbGEXp6+dj1743x32RTE5W7gocBDzRjOB5BRh0otJbM1RczGP3yO6cNrwzfnFYrfYJQAQ pIwWd5Pyv4qTdmgWqSgJM4M7fg8q6VQ= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=none; spf=pass (imf24.hostedemail.com: domain of henry.hj@antgroup.com designates 140.205.0.209 as permitted sender) smtp.mailfrom=henry.hj@antgroup.com; dmarc=pass (policy=quarantine) header.from=antgroup.com X-Alimail-AntiSpam:AC=PASS;BC=-1|-1;BR=01201311R661e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018047194;MF=henry.hj@antgroup.com;NM=1;PH=DS;RN=9;SR=0;TI=SMTPD_---.W4EnNzt_1705034428; Received: from localhost(mailfrom:henry.hj@antgroup.com fp:SMTPD_---.W4EnNzt_1705034428) by smtp.aliyun-inc.com; Fri, 12 Jan 2024 12:40:29 +0800 From: "Henry Huang" To: yuanchu@google.com Cc: , "Henry Huang" , "=?UTF-8?B?6LCI6Ym06ZSL?=" , , , , "=?UTF-8?B?5pyx6L6JKOiMtuawtCk=?=" , Subject: Re: [RFC v2] mm: Multi-Gen LRU: fix use mm/page_idle/bitmap Date: Fri, 12 Jan 2024 12:40:22 +0800 Message-ID: <20240112044026.58580-1-henry.hj@antgroup.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 1E399180011 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: rrsy419hsjktfoym5ezrc9s6bk3udfaw X-HE-Tag: 1705034433-375141 X-HE-Meta: U2FsdGVkX1/PeOqGiTB75zvC4rqKaEAo21JDkoNTz8x86TAF9Bx3a2AaintEAR2JEhO9xxy4DQ36h3r0KnKGNlD4UUVsjmGieCMhNx2bCXwulIY3hGJA9HyvesaajpBLbAOtJlaq79CRgziWBckFuYB44VCcoybsdBTpSOtBcnC28Y/4KpSUzVnOhZA2hisySPK3SsSybFQJN8DQkEK+Wy1OkhZgi9i6q6zY5NYKMyME2z5817JdTOg68O9M/hFrqIukqS2peK7Ua+zcDpgR7ZVSKf429tEbUmZ8WXZ7LYwgwxRw3PaBnfKkyywQfPin2Xv335hrth0/LgaPKVjtBp4xS8hm0TTY1/lKprhwg8CVhnd/OVmy3kIk/v/oYbpgkK+r3XSmAjZpuUAB1y1bnZpH+j8jgySeFDqWpE/CtAwGFscidTdCSmxgtREk+qpFzk84fsvXipQMpAe+TdrF+7slngJpo4AtuV7cIyXHPwnviD1VhR6dlU4SXRsXTy1vKsgy7TaRTFg6hJtMl01gV+pFqmiB+zyUjqWGp9pzR1F27mEWAaOvAxSPbthENqUncDyuuEDY7V2V72YPgsWHb9nljVYDAfxR0wb6c3Qmwr0lxcQwiX88lIulFWl4sKyXWJuC3wT/WOicusn41y83pmv3jgWhcXoUce9tB1o/ZLNeXs9xY7SkQdkN9y29IfY2qrlb5Vaa/lD3H22G8d55TEak4icQIfEG8peWxxw8bAMC/5wb6OvPIpvO9dwN2SVRCq1Wc/Uqld6M432FXqrntRbd4QuUHwJ9LuVzSIUsyaAclyCDJ5j/5ey7scJ1nDkOz0ozS+aBcm9giL6tMm0CMZSKubOOtPhfygRXxMmvTWBq5z3pw7EAhWzmlnFsPdgbuz6ArsSH0VYQS4s7ihVv8OhbfHmQhj7/IvHOvRt8c8ChHucI0QGSKuqxhAhcw8UeP0ZgCsoIEMYqgHrLZKg 2h++theU /dpeCxrI+hIjSYSzXhKd4wUt4I7cGJav97o8TiPYSEsxZh/kyjXVU+O6ibnFgB9vIDAMheVJbgShv4CWMlnCwfuC9lbGo9yjmx3gkAzWjnPLaSTT2u2syeqtLBYH1a/EodEtKZi89Rsj0wN+pyhzgHjXiYxiBW5Wgjan8CXNoLa6ezJWDrT1JTI5L/6iPRhxdxcRRuI7gqTUhoY5aB/bLgm9j04M+bgEHxYzOqexdxE4RoBepPvGtMtPHUQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Thanks for replying. On Thu, Jan 11, 2024 at 03:24 AM Yuanchu Xie wrote: > Does this present a problem with setting memcg limits or OOMs? It > seems like deterministically charging shared pages would be highly > desirable. Mina Almasry previously proposed a memcg= mount option to > implement deterministic charging[1], but it wasn't a generic sharing > mechanism. Nonetheless, the problem remains, and it would be > interesting to learn if this presents any issues for you. > > [1] https://lore.kernel.org/linux-mm/20211120045011.3074840-1-almasrymina@google.com/ In this case, total size of shared memory usually is small(< 100MB). What's more, almost shared pages were file cache. So it doesn't present any problems. > I'm working on a kernel driver/per-memcg interface to perform aging > with MGLRU, including configuration for the MGLRU page scanning > optimizations. I suspect scanning the PTE accessed bits for pages > charged to a foreign memcg ad-hoc has some performance implications, > and the more general solution is to charge in a predetermined way, > which makes the scanning on behalf of the foreign memcg a bit cleaner. > This is possible nonetheless, but a bit hacky. Let me know you have > any ideas. Wow! per-memcg interface is also what we need. It's hardly to control user's behaviors in our envrionment. We can't promise that all process who share memory would be in same memcg. Maybe kernel should provide new interface to make shared page charge more predictable. I think it would take some overhead to do this. The key problem of us is that we can't check whether page is accesed(no gen or ref changed) in this case. page belongs to A, but maybe process in B read/write this page more frequently. we may treat this page as cold page but accutly hot page. Maybe just call folio_mark_accessed instead of folio_update_gen(should hold memcg lru lock) for those remote memcg pages? -- 2.43.0