From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E965BC54798 for ; Tue, 27 Feb 2024 09:44:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7974394002D; Tue, 27 Feb 2024 04:44:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 74617940008; Tue, 27 Feb 2024 04:44:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 60FB294002D; Tue, 27 Feb 2024 04:44:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 504E4940008 for ; Tue, 27 Feb 2024 04:44:10 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 23C0C80B87 for ; Tue, 27 Feb 2024 09:44:10 +0000 (UTC) X-FDA: 81837097860.21.16C4DD0 Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) by imf08.hostedemail.com (Postfix) with ESMTP id 7307D16000B for ; Tue, 27 Feb 2024 09:44:08 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=lyuRdrhl; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf08.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.50 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709027048; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XKqylisLkKWj1tKsSnvFCiCRPsvTil4uIEHJgJNYUJI=; b=I1kKk43VsFLHl7ZgiyGoNRsrrbZYbgugwyDitJWZWb9mGyZjzt7n4SntOt7wnj2NL86Uhf fJ8ZoFWkPZROxqOemc4jAh9KW2puMPOz59OrqSKKuSYBTBxMmlmQBQEjXdJ/Apczs9mSxS U8q5pZ5dxpeh0lcPJ0IEsiPUG5CaK+M= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=lyuRdrhl; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf08.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.50 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709027048; a=rsa-sha256; cv=none; b=Y6ycGW/7wvFyry5q7GlbB+SvivSfc5pk6S2h1yEz0l5BdabtItly550ZJomoOqGxVYkA+6 vOIPXdtVoDllz26UMi/zwCMZ4dbSJ0t9qysrJD9A5GT5CZbPzzoVGEjjfeoRt5jHWxMdE1 gVZWzZXA5B0V7C/PMe3PzffmDC3upqk= Received: by mail-ej1-f50.google.com with SMTP id a640c23a62f3a-a3f829cde6dso467752866b.0 for ; Tue, 27 Feb 2024 01:44:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1709027047; x=1709631847; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=XKqylisLkKWj1tKsSnvFCiCRPsvTil4uIEHJgJNYUJI=; b=lyuRdrhlr3BHgpyIidbHXqVqSaZ7GSNtpMiJO8KIQWEX77iSAnpCtWvC0nvgYwF6NC v0WRmhwDXVj1q4jUi1yDqSzaN1z28g6kZlzPzaA6G+fy8B+6yQd8C+6S8xR5bJ/D3zWq Yvqape1w5ulYhC5IOLc6P5DM79votJVMAit/i6oKRIptJUaHZsKaU69HLEpADG4DsXIG 16x8fYccXsZHpOab1DK8UbfHZpVOiKwFiVMN2yeagNgDTGBrphNGPJodl3shjWwEca0k tlhLuJ/1WN6Ikk6RRQdA2JrXvscXVpAm0Nnj6GC6FwCiAQeyJf5A25QBTUl4a6DwRoEH PnBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709027047; x=1709631847; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XKqylisLkKWj1tKsSnvFCiCRPsvTil4uIEHJgJNYUJI=; b=H2Letayk2MyZ/YpbJ6jfj4gDMGXEhJcK8/QNHGqA93UnKZIGNcVhU+2LbZAagzJwFJ hsor6ddnJVnLOj3vrnyS9YmCApFtG5of1GJS2NTrTg/GT9agtCXntu9sO0KoDXag7KIJ 3Bv+RHrOKNP0JG9JrqVDKE1tYM6wQbkanBkRQcZ0R1IlL/Ro0FEA3B/VcFzdfXCQXn3H TncGdhnaixNuLhKPsm5s4yCiTuAGLdc+l4yh6Pk4Wdh+m5KkBoYSXZTQRAhE5GYz2g1o 5nkAMCNAOY6O8L7lt+RELhR21RYUU5VKwJnyW7u8ZY3Nsd6nGeo3GQNdrE5iC+vFwKgs vJVw== X-Forwarded-Encrypted: i=1; AJvYcCXCLvBHk6/Nt8jF0taThaVvf2gCQrVEmw3FO9GVgubB5sn2hKKRfpDxlN5tTE1sqAJ4aSb9LCA4ogxdWiK6A2iXkQ4= X-Gm-Message-State: AOJu0Yw23rd26eXfJIdyhhIJ2PyFKx9rG0PD+IXkgKWoL1cELiKUgKdn X8/4NENqWRt3kUxO2oL5YVs6bJInStm3XcI/LOe3zDezYvI4BYt5lquu01ggB9lJKdLMbUK4nbj yh7CzlstieCQepv3I95yTF1/25tgVbMaShrUz X-Google-Smtp-Source: AGHT+IEWfdyuyqqzy8p+A54ygWe23mzaEB0kXzk/C5osxUahYut7nlnbjOyWXz5zA7neIniXHK4K5PFB9Aewcr4kiI4= X-Received: by 2002:a17:906:4a8e:b0:a43:552a:9572 with SMTP id x14-20020a1709064a8e00b00a43552a9572mr3048171eju.30.1709027046746; Tue, 27 Feb 2024 01:44:06 -0800 (PST) MIME-Version: 1.0 References: <20240225114204.50459-1-laoar.shao@gmail.com> In-Reply-To: From: Yosry Ahmed Date: Tue, 27 Feb 2024 01:43:30 -0800 Message-ID: Subject: Re: [RFC PATCH] mm: Add reclaim type to memory.reclaim To: Yafang Shao Cc: Michal Hocko , akpm@linux-foundation.org, hannes@cmpxchg.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 7307D16000B X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: yx4m7pxq1miytwz8sa33se5gjp8kpjqi X-HE-Tag: 1709027048-731210 X-HE-Meta: U2FsdGVkX1+pYfFj5j/0IoqluzrP7XUpAndMcPdzNWjptIBJtrOrV7MvkX4rsAkRvwl2XPxc1K5p8GV3eo95wKY7tOuu+D6olrWyK0i6DEl4HYLXZ+jFX18tDpIGsiK4xs9wj8QLnxtWJuYFFqYzEEJNG1tMAcOD2FMmtkrAL+SuEkvIq+B0sx/aYcFGlNI2WN2RrmGH2w4OuKGydlbWGWR2qQwJIaWjlgxivYJ9/xT59DoXxUf7LRSio0snJ4+oz3bhtmUbo0Eg0TNa0uLLC6TSvWWkOFyO4eHYL5yKXfSqOBVhAtGDhnqlC9VfWCGAYZzbaCiYtsy2NPU73LMMeE1x/GM+qtjTlT7J/dM+F2XW6Ferbnsg6oQNSV+rN3ueXYEFVd+oRML+9/hd6KiXOVydaY0ekJSiz3Pdo4k8B4cFjOdnQ05Cf8G6QTJJYE8oDO3i75uocIuhw9yMA3fo3SOhRR1w+GphQrlsSBeEYtk0J6PWvOMZcto2gfXyPzYjZvKT4+3+Aqx8d7DgKOfoIOK5NoIJkaGVkThyOnWDsyh50hr5HmxgxsYwBTigUqUEcjDLB0z4DP+FLByuh8QBYINa0pAcb84AA6I2nzOo8qGCaiEO+fS0SljPGUni2J89xMkdFjAWZrsdy5Hna9ye/At9tghuOY5SUlmdKhDYmufiGppP3+xQW9V+dwVsjOTZfkDhy/gamNTlOs8Qg8Tz5B/JKYfFQ4FPdAXaoAHKkI234gTknF0YvYBDalwu9hrE+UbpFSyzibfImorm/K7k0lvS9fTf1BPBHTvcR799PdtCJq6TBlWunkMJX9ftRGpWhzzuLkQLmsbizYw6VOx8OIUddcnkFXMc1a/A9cIBY7+0yFAnq6vN+vD55A4d6wYEAEMKyRyR3CfKsKZIEnXdzQmF4V7Em4Z7UZUTnFqxJX5mNyl696eL9QWb/7kIG4Nw3PiRaxuv1cwdquALxRi 9iLR0xaN U0GxM5nyEsYOj9leugQC+bmUiQVFSmNI3S2nilmmiZvqualYKwM57+Lcsjv4VyZ5DPo8y X-Bogosity: Ham, tests=bogofilter, spamicity=0.009113, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Feb 27, 2024 at 1:39=E2=80=AFAM Yafang Shao = wrote: > > On Tue, Feb 27, 2024 at 5:05=E2=80=AFPM Michal Hocko wr= ote: > > > > On Tue 27-02-24 13:48:31, Yafang Shao wrote: > > > On Mon, Feb 26, 2024 at 10:05=E2=80=AFPM Michal Hocko wrote: > > [...] > > > > > To manage disk > > > > > storage efficiently, we employ an agent that identifies container= images > > > > > eligible for destruction once all instances of that image exit. > > > > > > > > > > However, during destruction, dealing with directories containing = numerous > > > > > negative dentries can significantly impact performance. > > > > > > > > Performance of what. I have to say I am kind of lost here. We are > > > > talking about memory or a disk storage? > > > > > > Removing an empty directory with numerous dentries can significantly > > > prolong the process of freeing associated dentries, leading to high > > > system CPU usage that adversely affects overall system performance. > > > > Is there anything that prevents you from reclaiming the memcg you are > > about to remove? We do have interfaces for that. > > Reclaiming numerous dentries through force_empty can also lead to > potential issues, which is why we attempt to shrink the slab gradually > to mitigate them. However, it's important to note that the underlying > causes of the issues in force_empty and rmdir are not identical, as > they involve different locks. > > > > > > > > To mitigate this > > > > > issue, we aim to proactively reclaim these dentries using a user = agent. > > > > > Extending the memory.reclaim functionality to specifically target= slabs > > > > > aligns with our requirements. > > > > > > > > Matthew has already pointed out that this has been proposed several > > > > times already and rejected. > > > > > > With that being said, we haven't come up with any superior solutions > > > compared to the proposals mentioned. > > > > > > > Dedicated slab shrinking interface is > > > > especially tricky because you would need a way to tell which shrink= ers > > > > to invoke and that would be very kernel version specific. > > > > > > The persistence of this issue over several years without any > > > discernible progress suggests that we might be heading in the wrong > > > direction. Perhaps we could consider providing a kernel interface to > > > users, allowing them to tailor the reclamation process based on their > > > workload requirements. > > > > There are clear problems identified with type specific reclaim and thos= e > > might easily strike back with future changes. Once we put an interface > > in place we won't be able remove it and that could lead to problems wit= h > > future changes in the memory reclaim. > > That shouldn't deter us from actively seeking a resolution to an issue > that has persisted for tens of years. > As observed, numerous memcg interfaces have been deprecated in recent yea= rs. There has been recent work to add a swapiness=3D argument to memory.reclaim to balance between anon and file pages. Adding a type=3D argument together with that is a recipe for eternal confusion. *If* we want to support this, we need to have a way to combine these two into something more user-friendly.