From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2FE4C54798 for ; Tue, 27 Feb 2024 11:58:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2354D280004; Tue, 27 Feb 2024 06:58:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1E54C940008; Tue, 27 Feb 2024 06:58:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 05E91280004; Tue, 27 Feb 2024 06:58:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id E4C05940008 for ; Tue, 27 Feb 2024 06:58:52 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id A9274A0751 for ; Tue, 27 Feb 2024 11:58:52 +0000 (UTC) X-FDA: 81837437304.24.8F9E431 Received: from mail-qv1-f50.google.com (mail-qv1-f50.google.com [209.85.219.50]) by imf24.hostedemail.com (Postfix) with ESMTP id 1317E180002 for ; Tue, 27 Feb 2024 11:58:50 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Q2UDl5IZ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf24.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.219.50 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709035131; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=967KWMxz7JTDPNz65NDDjapj/hsrUU1LNFWSSbDuY7M=; b=HLIRIL8o3mGbrLtl1mmkC1EkqqvQtrbtU+FIRBDAq9pvLGatLwaZllOL3lYW+ic+pnSQtj gvrbB4vpMBHXHv4lzytnfKLaEsOtinPXHgjciIBifzTHATZINoR3CtwaBf26/1rwAL7Cuo pqjG9IWkAAzddTz9UC3CL462KmEETpE= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=Q2UDl5IZ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf24.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.219.50 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709035131; a=rsa-sha256; cv=none; b=lfmFlex8hVkbBrzY44NxvvuyG8cle9npc6curGH3u4TCdTR8c7VYQB1YPusSwmpDko5tfn LIMJntbgbgsYR39FvKTj2wRb2uTo0Z6ybHLhgqpPwfpiJDFZRLYQBTfrREgfrgQVLF4PNs hP/7jCmDrM34T++UH/VDmV1RfBqQlTM= Received: by mail-qv1-f50.google.com with SMTP id 6a1803df08f44-690105304b4so5761776d6.3 for ; Tue, 27 Feb 2024 03:58:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709035130; x=1709639930; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=967KWMxz7JTDPNz65NDDjapj/hsrUU1LNFWSSbDuY7M=; b=Q2UDl5IZ6H/W6kVjSSXXr91yr5gaCNkON4RPe9WLX/QHBgQfG63O9PMJ+v/vudmy4J RJ8alWRCSg7WNS9BcyWYStYtQhttCoszTKaiauDobvQc9ac+qWZOdmllrspgdhNgRexB 3p37OLyp7rrymgexzbIs6RHC4afuH0aPhEyOfZNX4DKzJuZuRLE1KI74DCktAbixLvjv JAQiY79gVYefHj+k1mz3JG+fROHIZyarB0fS8JJKsi8SA1Cq61pyu83dLLQHlYdirIiR cslG5aVKqY3t6mSDwZyJI3nWObFCOD52q1I3QAIC14vSBA8Eu3g90u/lc7fcoqe++nvI UYKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709035130; x=1709639930; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=967KWMxz7JTDPNz65NDDjapj/hsrUU1LNFWSSbDuY7M=; b=SijeThI9v6t8XqhD1ANAd21fwqNx+XTW80oXiL1xjNEJy1/2DqgEw0Pc/wsbyCrFIN 4IQw7K2OSylCWcT6doEtUc/ELejveF6RqqafJrwQYELGq9C9sQzxOjEv7eoo4OOO8Lrg /tGTKMVYyWa6ScMblkyW9+zsIsvlngJC4ZaBZk9oIAe9r/has4nLWIBCZO6vk1z7iYBT Ad3CpmIvJFUxN9Q61LxG6yEkgK0ccQl4CI99G0b7H1aBgcEbbb/mrmHromuSFig5+I/Q HStte84901Efj2rFJWNtffjJVbaQ5nLPBGVulWSZJwp+AwPSi2XiRw05lg+aJFAEOejf nxCQ== X-Forwarded-Encrypted: i=1; AJvYcCUjhWGzRf8wQ/mIYVxgsoKBiJdfDHdffti2T/hhW69Cz80YM6ktlq5LO6TcwkkIEifRX4MwuSjGC3AsaKnN0CzvQQ0= X-Gm-Message-State: AOJu0Yxe6rfvgG2ZtdFNXT8ClnU3LglONtwBteSyVDfH8IAnFCL04Air jXA2uvAIV+LfGEtZeu2vJhof4gq9AQbBiN8BVYt1vfo8oiMo8kOxhNGsb3LVhjMG/WhPyz9l7K8 IzRCadGYwcJrQxWd+aPmhqtGOOIM= X-Google-Smtp-Source: AGHT+IGbZdw/0DW6WHZV9+/K41VsAeYhsVymujKRI7367YJLH1ZWSs2XsMp2hy91XBgX3FwH8PrbCsCJJlps3tJIocM= X-Received: by 2002:a05:6214:2aa4:b0:68f:cbdd:aaf2 with SMTP id js4-20020a0562142aa400b0068fcbddaaf2mr1729096qvb.33.1709035130259; Tue, 27 Feb 2024 03:58:50 -0800 (PST) MIME-Version: 1.0 References: <20240225114204.50459-1-laoar.shao@gmail.com> In-Reply-To: From: Yafang Shao Date: Tue, 27 Feb 2024 19:58:13 +0800 Message-ID: Subject: Re: [RFC PATCH] mm: Add reclaim type to memory.reclaim To: Yosry Ahmed Cc: Michal Hocko , akpm@linux-foundation.org, hannes@cmpxchg.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 1317E180002 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: x4uxoefangwtxpucdnahw4tcnbme45ux X-HE-Tag: 1709035130-82238 X-HE-Meta: U2FsdGVkX18b9YpeUTGbOdYyZqMs1kKpEmNk/7+wcV0SmAhjgYOA88xdEU+aKmrqQcwzvwUgGnx9t8ZBQiwb3q7lJz3T3rKshSdwK/B5Z4jfpvu5BK58eABy8MnOY2QsMQa+qr9Bj2HWY2ylINbb5afW17f+TJfcJHwz8+c7cHktWtTJHS+y88Oj4pRmARqxM+AoaJruiROEaIQt5LT7smq5uSaKMCjRu1rZUSTOr4Fib1QE0Z6TUnFtHVb0TTira1axpF9tTyih90YLQxjIfm3a3J81luhl4bgkkfVXCXfhDh/YqmcN6fSs+JIV79jRYGpm4sjoJ3uOM/Wap8q73RyItvswaf/N3BrtzvT2AL4YXrDgG5ZyCrnD/lgEe96nWw1neE7z7KmghZaiSqzcXAANLYbjFZ8eM4Xp1SDi61/QyqlNJUIvR5sVIYkBiIblrUdWGWUIoAjYXH8aUp9V1txt3LaWgdcFLM2CwpBqyYuQm5zbr8iNlTK7siOl5Ua/T8AYqEal2xUWasKmfHqqsLBQ0AL9LWwyXaIWwE/fYjQysb59UZOFjCSJ5JgWepDiuFwc4B2XwI+GZFTL0X6Nxpc8LHnWnH2U6HiFEa31f0WsVEOdDOxCTE8lBGvCEhJj+BkvS5bIB5gqF5tWc3sRz9crAiOL9jq6uL1yQyMg3jQyXudh4qR0bOL9r1ZLIBJbvwknk51W0DVLr97eHQAPBCWdbeyUErh36BJSj4Nap4wG0bYnwCSHcmwDJWC/t6U06UlI6QY/13kBCrIaneqrmDz6cOLaHzJn2oRm3Oyu/UCfJEAGKW0ygnoOqi3x8qqly3LxMPFaHrCEcXYUWpCnQzb0J1rQ2DY9h/xWk2zTOSwrfRthgw/HYyxX8aZmtTOARWoWvbgCfDpZsv711n4k0kYVKd3XG4f78iwFWoGD4A0aO/piKWzp6Lkr4KhRuzOysiZM796qoB0ke7ubJFT IKPAylgp 0dDpofF382ZZC3dh92R6S/5jOGu3Ber/qou/RzMdcK5ZLg/O2aLAELg6TVk+0l31kV+f2 X-Bogosity: Ham, tests=bogofilter, spamicity=0.020090, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Feb 27, 2024 at 7:40=E2=80=AFPM Yosry Ahmed = wrote: > > On Tue, Feb 27, 2024 at 3:37=E2=80=AFAM Yafang Shao wrote: > > > > On Tue, Feb 27, 2024 at 5:44=E2=80=AFPM Yosry Ahmed wrote: > > > > > > On Tue, Feb 27, 2024 at 1:39=E2=80=AFAM Yafang Shao wrote: > > > > > > > > On Tue, Feb 27, 2024 at 5:05=E2=80=AFPM Michal Hocko wrote: > > > > > > > > > > On Tue 27-02-24 13:48:31, Yafang Shao wrote: > > > > > > On Mon, Feb 26, 2024 at 10:05=E2=80=AFPM Michal Hocko wrote: > > > > > [...] > > > > > > > > To manage disk > > > > > > > > storage efficiently, we employ an agent that identifies con= tainer images > > > > > > > > eligible for destruction once all instances of that image e= xit. > > > > > > > > > > > > > > > > However, during destruction, dealing with directories conta= ining numerous > > > > > > > > negative dentries can significantly impact performance. > > > > > > > > > > > > > > Performance of what. I have to say I am kind of lost here. We= are > > > > > > > talking about memory or a disk storage? > > > > > > > > > > > > Removing an empty directory with numerous dentries can signific= antly > > > > > > prolong the process of freeing associated dentries, leading to = high > > > > > > system CPU usage that adversely affects overall system performa= nce. > > > > > > > > > > Is there anything that prevents you from reclaiming the memcg you= are > > > > > about to remove? We do have interfaces for that. > > > > > > > > Reclaiming numerous dentries through force_empty can also lead to > > > > potential issues, which is why we attempt to shrink the slab gradua= lly > > > > to mitigate them. However, it's important to note that the underlyi= ng > > > > causes of the issues in force_empty and rmdir are not identical, as > > > > they involve different locks. > > > > > > > > > > > > > > > > > To mitigate this > > > > > > > > issue, we aim to proactively reclaim these dentries using a= user agent. > > > > > > > > Extending the memory.reclaim functionality to specifically = target slabs > > > > > > > > aligns with our requirements. > > > > > > > > > > > > > > Matthew has already pointed out that this has been proposed s= everal > > > > > > > times already and rejected. > > > > > > > > > > > > With that being said, we haven't come up with any superior solu= tions > > > > > > compared to the proposals mentioned. > > > > > > > > > > > > > Dedicated slab shrinking interface is > > > > > > > especially tricky because you would need a way to tell which = shrinkers > > > > > > > to invoke and that would be very kernel version specific. > > > > > > > > > > > > The persistence of this issue over several years without any > > > > > > discernible progress suggests that we might be heading in the w= rong > > > > > > direction. Perhaps we could consider providing a kernel interfa= ce to > > > > > > users, allowing them to tailor the reclamation process based on= their > > > > > > workload requirements. > > > > > > > > > > There are clear problems identified with type specific reclaim an= d those > > > > > might easily strike back with future changes. Once we put an inte= rface > > > > > in place we won't be able remove it and that could lead to proble= ms with > > > > > future changes in the memory reclaim. > > > > > > > > That shouldn't deter us from actively seeking a resolution to an is= sue > > > > that has persisted for tens of years. > > > > As observed, numerous memcg interfaces have been deprecated in rece= nt years. > > > > > > There has been recent work to add a swapiness=3D argument to > > > memory.reclaim to balance between anon and file pages. Adding a type= =3D > > > argument together with that is a recipe for eternal confusion. *If* w= e > > > want to support this, we need to have a way to combine these two into > > > something more user-friendly. > > > > What if we introduce a new file, like memory.shrink? This could serve > > as a foundation for potential future extensions, allowing us to shrink > > specific slabs with specific counts. > > Shrinking specific slabs is something that shouldn't be exposed as an > interface, as this is a kernel implementation detail. Also, If that's the case, why was slabs info initially exposed through /proc/slabinfo? Isn't that level of detail considered a kernel implementation detail? Currently, users can identify which slab is consuming the most memory but lack the ability to take action based on that information. This suggests a flaw in the kernel implementation. > memory.reclaim and memory.shrink would still have overlapping > functionalities. --=20 Regards Yafang