From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CBE0DC5478C for ; Tue, 27 Feb 2024 09:45:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 54F6F80007; Tue, 27 Feb 2024 04:45:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4AE68940008; Tue, 27 Feb 2024 04:45:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 317D880007; Tue, 27 Feb 2024 04:45:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 15C7F940008 for ; Tue, 27 Feb 2024 04:45:10 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id DE790C0C2B for ; Tue, 27 Feb 2024 09:45:09 +0000 (UTC) X-FDA: 81837100338.14.2F23A31 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf10.hostedemail.com (Postfix) with ESMTP id A21E8C0005 for ; Tue, 27 Feb 2024 09:45:07 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=j2syM1kB; dkim=pass header.d=suse.com header.s=susede1 header.b=j2syM1kB; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf10.hostedemail.com: domain of mhocko@suse.com designates 195.135.223.131 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709027108; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yTTnAGHypHRQNmnUZC9wqwmtMHAZ7HrxFSW0vWVtRjQ=; b=xY3otOoiLlnsYjgdIfsEd2WVvID8FTcpAK6TMjHRSPuGJ/5ignIoEOb5x09TpWUoSZjOQ4 jdSWYLVcQzbLOwdbsQdATh8iTgBlbizIaJF8YnfAAIIbSiC0nnAskXDqksk78cAoKhE5/+ u2kblmFywGZX6JpADo/v2EWl1u/gvZs= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=j2syM1kB; dkim=pass header.d=suse.com header.s=susede1 header.b=j2syM1kB; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf10.hostedemail.com: domain of mhocko@suse.com designates 195.135.223.131 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709027108; a=rsa-sha256; cv=none; b=mqeUj231xNOoGZQXXFPYPwC8PBKj4GSQrAwseub7hCuTaBKMuB472g5Nqc4pnGOqiOjbX0 JuOa4QzXe43RD3SBYWHOdDi7hjJg6AgUf2uD7sIdq8UinwfKGnO81z9UTiSTx4PVNEoSCT botDmTlkoSJlEQyB0Ww2wGoP6A8bT84= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 058801FB94; Tue, 27 Feb 2024 09:45:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1709027106; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yTTnAGHypHRQNmnUZC9wqwmtMHAZ7HrxFSW0vWVtRjQ=; b=j2syM1kBmYDcnU9iAdo4Hq/XlnO+qqx1HRLugE8/9BCopEuwyfpCZx+OZJUY26DyjxhlmF vynlVDc3uuXhdEaQ/AhxU8dVhMO23CRXaajsJIusNhhTYm+CAIKhRxz/BO4WMupjR4Wtkf lMTKriE43RGT9Wt/CJW2q72sWAnrDic= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1709027106; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yTTnAGHypHRQNmnUZC9wqwmtMHAZ7HrxFSW0vWVtRjQ=; b=j2syM1kBmYDcnU9iAdo4Hq/XlnO+qqx1HRLugE8/9BCopEuwyfpCZx+OZJUY26DyjxhlmF vynlVDc3uuXhdEaQ/AhxU8dVhMO23CRXaajsJIusNhhTYm+CAIKhRxz/BO4WMupjR4Wtkf lMTKriE43RGT9Wt/CJW2q72sWAnrDic= Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id F1AB113A58; Tue, 27 Feb 2024 09:45:05 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id mDn8OiGv3WUmdgAAD6G6ig (envelope-from ); Tue, 27 Feb 2024 09:45:05 +0000 Date: Tue, 27 Feb 2024 10:45:05 +0100 From: Michal Hocko To: Yafang Shao Cc: akpm@linux-foundation.org, hannes@cmpxchg.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, linux-mm@kvack.org Subject: Re: [RFC PATCH] mm: Add reclaim type to memory.reclaim Message-ID: References: <20240225114204.50459-1-laoar.shao@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspam-User: X-Stat-Signature: z1q1x8rc34q39ygti5xw99iuxjh7o6cc X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: A21E8C0005 X-HE-Tag: 1709027107-642308 X-HE-Meta: U2FsdGVkX1/T6RfwozDfDbcyJFDJQWa/UTtdsw1bJaJz0bLx+xsd5KUvup5eJmdveNNHJM91/XzPiCJ5vh1l+xCT2So5c+kst9B1TRI/AUqwB3UnzrWIpaMB8qgNWoWrpdJrgmlrzzaLxDB0IjwhdiH9GM5B/99MclwKbHRRbBwNjv5GZMg72RtZ7yghlRsgFzImo0P5bCRlTSj4kCdOLJKWAFeJ2BiNJ1GTCQ2mDfKXV9zbGILq2h4MnRmnMm2NdJnKuMVvYDOpb/NW/M+uiITJcBjdq8LzuCbz/HCDOReju8WfBEejkxqmP9jBkvfcuuMeqHe8JgHVErGi8c6/NcQKAoeCkfv/f9gi+nGU+gy8QTRq6yi84ELxpCUC/ZGs0f5JB401Cuf257gjgh3o/XPlfiZ4y59E0sRsoQF86wQx1Ja1TbSc/JTnsfF2mTfTP0DOGXjXHkfKhcV3thVuya8lhwq9KpnejfmN+MmLVbBX13/o6ARbWYK/32QvzA6tviuaxLtUzmCsdB3zbUT9cEuCo4IRp0DGXufcR9ONUo36mZk1wYbpF4cLNSN7u8NZm08c4mTiRKWkp/ML4s6IonjjjwKD2Sfnb3L0BAkMnGtZ67JncEgpCdBQ1wk44kWHohAb5eWvny+hFYyWCyqM83MbLvn5wmY/TF+U4dzHUg7jdAbS0hgSOK2WDCfCKwOSNyi3BgRASvfXKf4l+1mLifdxvf5JMOcLgJlqqIcNJY+1o9RKbb0WQw0NVJ6Iu7phFdop7UPEAbc8zIPF+gPiK87CEEvmw3UuioLzXFT5RLFxY+G/fBoKC5VXrfIKulZMQ/d12ArrT3CcfB9y5hN/czOfdWpxCYGVDwmBGmtWsVl2ZkDoNiSlpJKDMFfiXRu90M21HbNth2Di1cOyIb/q6a93oogCd/fCcNUOhXFD4156fkHGD6rVsJfc+8Ujjd7KB329yu03gFZus4WNfKT tqoaZNzf YOxPEkv8DMp037NrZMfq0xtQTGJRXlpZdnIUqbsyoj6fFpfsbXcSoJujmCg4851lBzJwgsX9S6hdq71TfAfQKkBlLzdvbXrAGj6uFf79hRyiX/BB26EH4KreHOWLWOvwL/4U9Wp8uvkqNQYIoBlNYDJ0th+3PXfGZ00CHjaLQF5/PUOIojHhEpH+/AU3VJvqGto7MZQCeGNQ4BWdEEy+FHvFUcQMrXbAiLDYpfPt/V+wALvPNCY+x/a9nAkXRRLsRY1BM2h3x0JxKuGM/sLAWg+P2rpM99mIpUMgt6AWOl3rMNHsS6PGLVMyFjV5KGH0Q0kv2P0U2LRhP7V3ZkzyQ++c4eVrLlLsVIVF3eTHe8ITr7JGZiOMJuNO+/g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.001089, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue 27-02-24 17:39:01, Yafang Shao wrote: > On Tue, Feb 27, 2024 at 5:05 PM Michal Hocko wrote: > > > > On Tue 27-02-24 13:48:31, Yafang Shao wrote: > > > On Mon, Feb 26, 2024 at 10:05 PM Michal Hocko wrote: > > [...] > > > > > To manage disk > > > > > storage efficiently, we employ an agent that identifies container images > > > > > eligible for destruction once all instances of that image exit. > > > > > > > > > > However, during destruction, dealing with directories containing numerous > > > > > negative dentries can significantly impact performance. > > > > > > > > Performance of what. I have to say I am kind of lost here. We are > > > > talking about memory or a disk storage? > > > > > > Removing an empty directory with numerous dentries can significantly > > > prolong the process of freeing associated dentries, leading to high > > > system CPU usage that adversely affects overall system performance. > > > > Is there anything that prevents you from reclaiming the memcg you are > > about to remove? We do have interfaces for that. > > Reclaiming numerous dentries through force_empty can also lead to > potential issues, which is why we attempt to shrink the slab gradually > to mitigate them. However, it's important to note that the underlying > causes of the issues in force_empty and rmdir are not identical, as > they involve different locks. Please be more specific about those issues. > > > > > To mitigate this > > > > > issue, we aim to proactively reclaim these dentries using a user agent. > > > > > Extending the memory.reclaim functionality to specifically target slabs > > > > > aligns with our requirements. > > > > > > > > Matthew has already pointed out that this has been proposed several > > > > times already and rejected. > > > > > > With that being said, we haven't come up with any superior solutions > > > compared to the proposals mentioned. > > > > > > > Dedicated slab shrinking interface is > > > > especially tricky because you would need a way to tell which shrinkers > > > > to invoke and that would be very kernel version specific. > > > > > > The persistence of this issue over several years without any > > > discernible progress suggests that we might be heading in the wrong > > > direction. Perhaps we could consider providing a kernel interface to > > > users, allowing them to tailor the reclamation process based on their > > > workload requirements. > > > > There are clear problems identified with type specific reclaim and those > > might easily strike back with future changes. Once we put an interface > > in place we won't be able remove it and that could lead to problems with > > future changes in the memory reclaim. > > That shouldn't deter us from actively seeking a resolution to an issue > that has persisted for tens of years. Right, I do not believe we would deter anybody from doing that. This is just not an easy problem to tackle. So either you find solid arguments that previous conclusions do not hold anymore or you need to look into options which haven't been discussed so far. I do realize that chasing previous discussions in email archives is not fun but maybe a good (re)start would be documenting those problems somewhere under Documentation/. > As observed, numerous memcg interfaces have been deprecated in recent years. yes, for very good reasons -- Michal Hocko SUSE Labs