From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 861D3C54798 for ; Tue, 27 Feb 2024 12:09:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1746A280011; Tue, 27 Feb 2024 07:09:58 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0FDCC940008; Tue, 27 Feb 2024 07:09:58 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EBA3F280011; Tue, 27 Feb 2024 07:09:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id D510E940008 for ; Tue, 27 Feb 2024 07:09:57 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id A29E2A0467 for ; Tue, 27 Feb 2024 12:09:57 +0000 (UTC) X-FDA: 81837465234.22.4DADBC9 Received: from mail-oi1-f177.google.com (mail-oi1-f177.google.com [209.85.167.177]) by imf28.hostedemail.com (Postfix) with ESMTP id 035BEC0023 for ; Tue, 27 Feb 2024 12:09:55 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=l7KBROCJ; spf=pass (imf28.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.167.177 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709035796; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=y79N6YZMWNJriYzAiGJj6eaLxZi+oOXPJzIejSWq+Jg=; b=J9/pPdcLP/lY8hJym3JUitUip6uzte19Vu5ThdTnydWjUO4JLUQKTh3T7ig5Y4w3cvYhjk c9q3uIkHRDgtVFUGqOkLCSqodXM+/FTnvcb/MZkD9uHTUxGq48awitOZLOSD6gbBU1HaxO GITRJgmttsC+QKqLfYYRNMkmah31Zp0= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=l7KBROCJ; spf=pass (imf28.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.167.177 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709035796; a=rsa-sha256; cv=none; b=um1zfmIoXA2np4HbL/FNUZIHWC/TlgeVDZQiYSF3WE8kQROLZIOvcr/LoxGpPYHAVY1Zw1 ziidN7L/z03SFjXhPpB3nbn3er05ZRKeeZEep9qm0j6o0ohyLgifh6iXxBhQ105HluMhe5 V4V9qW6eRRlnEpTgwHlaVKrM+y7EW8A= Received: by mail-oi1-f177.google.com with SMTP id 5614622812f47-3bbbc6bcc78so3094481b6e.1 for ; Tue, 27 Feb 2024 04:09:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709035795; x=1709640595; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=y79N6YZMWNJriYzAiGJj6eaLxZi+oOXPJzIejSWq+Jg=; b=l7KBROCJUtqFNe0FR5ePH3hp0/ozkMEZ/Hb60B/LBql/lKqRcPUhbGwNeUE+kpxvVb SZuaDcpxaZAMPoI8MfhEnfkt/lgv9+dz+Lmi+qyCHrChrJgZDtBku7auHV/vRnTOrFOW aCJOppDBDX4SradX/o+uzJpR5Y6U7gdkLeKyPA7bsuLjQFK3uzngXjG08CjpfmTg88VM ZlFydOWXv03bgQ1zCUeSAj09MweWVqvkeopSsGGfDKtwGtvIuLzbdvNklpDRJuwu9cQd GXmKXmTfnYDyxlo+stjWAgIx7jWm5xNsUrA0Ha5UWrtdOHVOntE3nIRzbaH5MirZfZyu GM7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709035795; x=1709640595; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=y79N6YZMWNJriYzAiGJj6eaLxZi+oOXPJzIejSWq+Jg=; b=ra07nGjIFoCu38KpH2b1mU5e3JyvKkIGB2Vpkw/b7npy5r0BAgKFmNSiOT43zZLqsc d2rziufScZ0csquNy2rhFdao/L2S6ubAEnmRrHtqr3BUyl37JszZv9s1WjaSehVtCQzo 5EATOUx/CrPMEvSRTSsyK2hH2cI26Js9mSNhc4MgfzdCXnem7CLG+1f2uwlQa0/kkQo2 M8QYBPFFJOPQlpJrKvmAuPEIuCrRmtw59Mz4AVSTO3cpHzqmfkpkjYNmXw9XcG1HA4h3 IrzP81LybINunwDOMiliCnZOTTkTd+A06zP8sAYwpfMHLAA5okqTj/+3Q8ucCJuLK2SF Ut0w== X-Forwarded-Encrypted: i=1; AJvYcCUSnLGAs3F7l0FqDDFbH5GmD2/iJJSsMzpmAg4FV2HBDypMqIKoL8AYTeRTsvco34QuanW1Kb+w8XdlxX4G2JDixe4= X-Gm-Message-State: AOJu0YziNw1VUuEVVd+V+HkbrSAcj4W624Oz+cG6mZKMnNgptEwBtK7B hCuSBkoLp0AT+SjGG114FymxMH0UyhCitwByUHI3QTcut6g2mcuCCCmc40RDAM49aCelr44nvDd /RKQgnyB2hoXpvS4viLZgMX5M/Qg= X-Google-Smtp-Source: AGHT+IG7tq6E4rFSegVDt3lRtjWhw0KVH1aCrQ/6GNSQ82vV33TGYkRSTbJ+IeIDgboVs45IXUZChUv0feSzEGzacf8= X-Received: by 2002:a05:6358:76a4:b0:17b:5e34:7186 with SMTP id e36-20020a05635876a400b0017b5e347186mr11866624rwg.11.1709035794925; Tue, 27 Feb 2024 04:09:54 -0800 (PST) MIME-Version: 1.0 References: <20240225114204.50459-1-laoar.shao@gmail.com> In-Reply-To: From: Yafang Shao Date: Tue, 27 Feb 2024 20:09:18 +0800 Message-ID: Subject: Re: [RFC PATCH] mm: Add reclaim type to memory.reclaim To: Yosry Ahmed Cc: Michal Hocko , akpm@linux-foundation.org, hannes@cmpxchg.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 035BEC0023 X-Rspam-User: X-Stat-Signature: ys5tmcskx49xnohgpic34q71bmex5tw6 X-Rspamd-Server: rspam01 X-HE-Tag: 1709035795-95090 X-HE-Meta: U2FsdGVkX18Wim6jpGcCYKXl8XHHv53rgmyT0Zhv5gsGtFPY669Yg/vOZz3e6s5n54inCNZGqkwWZSD5pyVG2blDKyzbFg/9xNop6rZaj5SROfLmUZF4AH6Mh/X6vqrd82CLbOcYxhJtAMdm3icqcd6tlx0Y6K8FrCjYQXn896FWUh0nS39y0X8O3pQuMF/QK/+Yr/cnoAWslhzWyfSFkjInmgsOvaTj2saYaB58+uldVeCvj0PxQ9+bn22dd/72J7YYBmh9uAZiE7cMBCURmRs3awpvfH2PvHfyGWJK7s1oiWgvTGncrglH8521O+BI1ixjB/h2/lai3FydhMKuJ+9yXBQTvnLVQaedW38a7yT+a6YNtY5/pkXylR4D3AxcvWL9Q5Duqf1XQW+TZ/rlBTr4icejKd8DovoTxtfEfv7/P3xaAT+N0lhCEJI9eE8SYI6RB/lR0quPqzyQg2S+Fr6tcZB10GioEsOoYkBRw+Pnf7/YX1j8gJ7G39P6cdsoPpddlL55mi7wCI9pvazUrfOn4ZKUGdnm/jWGBoh7DVS09aME5eu085rRf9D5+NcXKTA+CzwfxMFlyYti85xPQIkj18HNJQrteXn+bXGdaKx24AggQ8W7YWIe8HEFYHCMPd+7cV49zod3RF8La7XZM2DXj1aqbsMOj3tvdZYDNUJa/zZuiEGJLRAAv9bjtGCzX9dSQrDZE04qunrSwKpwL2Z3f6fEjmr6ThW6i/C4e6/CMkfShkb3UimCjwgEbd4zclXA7pbXJGAtXk3aMteLBH6wRuwRwbLZngEuxIo7iut7t0dhKO9v++yNJw5VlFLp9F2nNNrjsQdDOejmIwsO18UcKjq5JP6iqUBdUV1s5VOllDGzdaJdIHN+3uquK92sBSQwYVYYoQyCKnuZoLq0T6lpjC9EIg1g5O0ffz6NAUKvoIoCRORFolz4Sm7LVSY5f38JCasQyKnDOJi3wuH HYPNbHvd ekBb4YVY8xNdIEVOhOWW9gLWWxbhvKcj2J6v5cjJJ2xVoqRJoDG8l0wpNSRrrY3JVdoze X-Bogosity: Ham, tests=bogofilter, spamicity=0.000388, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Feb 27, 2024 at 7:58=E2=80=AFPM Yafang Shao = wrote: > > On Tue, Feb 27, 2024 at 7:40=E2=80=AFPM Yosry Ahmed wrote: > > > > On Tue, Feb 27, 2024 at 3:37=E2=80=AFAM Yafang Shao wrote: > > > > > > On Tue, Feb 27, 2024 at 5:44=E2=80=AFPM Yosry Ahmed wrote: > > > > > > > > On Tue, Feb 27, 2024 at 1:39=E2=80=AFAM Yafang Shao wrote: > > > > > > > > > > On Tue, Feb 27, 2024 at 5:05=E2=80=AFPM Michal Hocko wrote: > > > > > > > > > > > > On Tue 27-02-24 13:48:31, Yafang Shao wrote: > > > > > > > On Mon, Feb 26, 2024 at 10:05=E2=80=AFPM Michal Hocko wrote: > > > > > > [...] > > > > > > > > > To manage disk > > > > > > > > > storage efficiently, we employ an agent that identifies c= ontainer images > > > > > > > > > eligible for destruction once all instances of that image= exit. > > > > > > > > > > > > > > > > > > However, during destruction, dealing with directories con= taining numerous > > > > > > > > > negative dentries can significantly impact performance. > > > > > > > > > > > > > > > > Performance of what. I have to say I am kind of lost here. = We are > > > > > > > > talking about memory or a disk storage? > > > > > > > > > > > > > > Removing an empty directory with numerous dentries can signif= icantly > > > > > > > prolong the process of freeing associated dentries, leading t= o high > > > > > > > system CPU usage that adversely affects overall system perfor= mance. > > > > > > > > > > > > Is there anything that prevents you from reclaiming the memcg y= ou are > > > > > > about to remove? We do have interfaces for that. > > > > > > > > > > Reclaiming numerous dentries through force_empty can also lead to > > > > > potential issues, which is why we attempt to shrink the slab grad= ually > > > > > to mitigate them. However, it's important to note that the underl= ying > > > > > causes of the issues in force_empty and rmdir are not identical, = as > > > > > they involve different locks. > > > > > > > > > > > > > > > > > > > > To mitigate this > > > > > > > > > issue, we aim to proactively reclaim these dentries using= a user agent. > > > > > > > > > Extending the memory.reclaim functionality to specificall= y target slabs > > > > > > > > > aligns with our requirements. > > > > > > > > > > > > > > > > Matthew has already pointed out that this has been proposed= several > > > > > > > > times already and rejected. > > > > > > > > > > > > > > With that being said, we haven't come up with any superior so= lutions > > > > > > > compared to the proposals mentioned. > > > > > > > > > > > > > > > Dedicated slab shrinking interface is > > > > > > > > especially tricky because you would need a way to tell whic= h shrinkers > > > > > > > > to invoke and that would be very kernel version specific. > > > > > > > > > > > > > > The persistence of this issue over several years without any > > > > > > > discernible progress suggests that we might be heading in the= wrong > > > > > > > direction. Perhaps we could consider providing a kernel inter= face to > > > > > > > users, allowing them to tailor the reclamation process based = on their > > > > > > > workload requirements. > > > > > > > > > > > > There are clear problems identified with type specific reclaim = and those > > > > > > might easily strike back with future changes. Once we put an in= terface > > > > > > in place we won't be able remove it and that could lead to prob= lems with > > > > > > future changes in the memory reclaim. > > > > > > > > > > That shouldn't deter us from actively seeking a resolution to an = issue > > > > > that has persisted for tens of years. > > > > > As observed, numerous memcg interfaces have been deprecated in re= cent years. > > > > > > > > There has been recent work to add a swapiness=3D argument to > > > > memory.reclaim to balance between anon and file pages. Adding a typ= e=3D > > > > argument together with that is a recipe for eternal confusion. *If*= we > > > > want to support this, we need to have a way to combine these two in= to > > > > something more user-friendly. > > > > > > What if we introduce a new file, like memory.shrink? This could serve > > > as a foundation for potential future extensions, allowing us to shrin= k > > > specific slabs with specific counts. > > > > Shrinking specific slabs is something that shouldn't be exposed as an > > interface, as this is a kernel implementation detail. Also, > > If that's the case, why was slabs info initially exposed through > /proc/slabinfo? Isn't that level of detail considered a kernel > implementation detail? Currently, users can identify which slab is > consuming the most memory but lack the ability to take action based on > that information. This suggests a flaw in the kernel implementation. BTW, we even expose more detailed kernel implementation details through /sys/kernel/slab. That is really confusing... --=20 Regards Yafang