From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B033C5478C for ; Tue, 27 Feb 2024 05:49:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0D33B4401EA; Tue, 27 Feb 2024 00:49:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 034494401E8; Tue, 27 Feb 2024 00:49:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DCBC24401EA; Tue, 27 Feb 2024 00:49:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id C33C04401E8 for ; Tue, 27 Feb 2024 00:49:10 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 67C4A40AD0 for ; Tue, 27 Feb 2024 05:49:10 +0000 (UTC) X-FDA: 81836505660.29.49B71D8 Received: from mail-qk1-f179.google.com (mail-qk1-f179.google.com [209.85.222.179]) by imf23.hostedemail.com (Postfix) with ESMTP id CFC98140005 for ; Tue, 27 Feb 2024 05:49:08 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=XBYRPqF2; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf23.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.222.179 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709012948; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4IvR6eUnyzagUmtq1F7/yzqcXXBospm2EDylUCr1GRM=; b=5yyvpEzd9zpAKKblqpTFcjX8wuqEgYuq2IjTGcnrFvxcMHuFVpVJLrGB+MraJ5RMPeJ6me 8O4/5TRxoesxMfbQ6/SzKzkEDoZcuMunp9bxJgt3ZibmLMf9TCRXpK/x7fp9XYylgUxAnu /0n3K2UBPlFz3C74pUhYSzWPjttDDWU= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=XBYRPqF2; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf23.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.222.179 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709012948; a=rsa-sha256; cv=none; b=xuPE23bx9UAeqhg7QZD6Bgbv8RQTr8MnXCRzbqAFia1q79gLjlnSk7eZ8aY5mOD1SEyT7X 5Bx6KYAtvQ/LmRf4PUcYJCbIoEYX2FHY8/sWq5ricHOjCPaJNCf9MQowv5cB2uEtOvcjwR b/7B5rKJkXznjSUfxko6vXZou+FNlVY= Received: by mail-qk1-f179.google.com with SMTP id af79cd13be357-787df45e513so51647685a.1 for ; Mon, 26 Feb 2024 21:49:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709012948; x=1709617748; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=4IvR6eUnyzagUmtq1F7/yzqcXXBospm2EDylUCr1GRM=; b=XBYRPqF29pbPevA7aO7lf6rKxeADFB1xvILpc9bdeTXs7xyvFokwMmGZDHvDz4oaXC xKrwoCcUz5uDd3S+i0+wCzS0t4PRrRWugjxbSI9LXPG4ggAUKmRrgdJuDbK4anfeEIdk MrL7pnV3E4nSIH2MQ1L//3GfmFyvWoyvI3F8beE3hRlNbFMseJ8S6AjAWEAC05iFmDVr k3tuSbjmOe9iWkpXNsq98oK3f4S0w3Z0P+sI/wJ7JrlEeo7O1EoyhDDzX445GGE6ZXkL xBfjMSNpQYjAbMYM6F95j8OjD7XNbhh/ZQu0QAtpIxuY8/wZWmWhPoooTeiGVqK2sht3 yHEw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709012948; x=1709617748; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4IvR6eUnyzagUmtq1F7/yzqcXXBospm2EDylUCr1GRM=; b=PCS5gOGubAQBUOGR6ulmE1G8xNhQiBsAT81MkyvdaYkV31uzELs8zUtrZkVNmpgqJ2 IGtJZiPQBnie6vRoPdEZBC0xOnYtmokLohzCtoz/7uCPhdPF/hUaOjeCfBj+bRz2ZiE/ DxlbzH1PH5nrgNrJVo9/BlpVR4Ne4aSoBzgmRLJEqKD4sPjsDZIQM0rEs/Pmhvt4wXm6 G72DswM6nOTNm6AW60uUWK/83kbdjSXEHYeuBy+RVXZ9gJUASXa7eVNhRiuDAMVC7+Sd OuluMCRqBwnv3M1obY0GKGVXWXoYRNN1uGHIRh0sbDwjzGYDjNEHo7QG+hKu3otoDCYT yFmg== X-Forwarded-Encrypted: i=1; AJvYcCXBISCVgnKG1YkyjBYhmxCVGEko3WLd7OKLb918DrI2Lufrsj0FzMoVOANJ7QIEs53qArGNmi+5iUW/M+S+FyIPvsA= X-Gm-Message-State: AOJu0YwGdDAezJpUPN+CoPEa1BzbUWTrUzxeL1VqKmopT4kbnmGN4HAl 0hTENenJhIBPyj94A9p7PG05k4WSzUIGMylLUl4peyJgWupWVeNwqjjc+Tp4UynGosWJaZt1vwU xCp9NlIKTssSmzk6P3iTaKkttS2Y= X-Google-Smtp-Source: AGHT+IGFrO1PvO4RWverL4Jm4W+peT7rIyQkiGxO84waztcVecbpYAWCejpF8wCXAbpeNRiJ3/kG71j+fa+4cjBcI4s= X-Received: by 2002:a0c:f5d2:0:b0:68f:331e:a0e1 with SMTP id q18-20020a0cf5d2000000b0068f331ea0e1mr960917qvm.40.1709012947942; Mon, 26 Feb 2024 21:49:07 -0800 (PST) MIME-Version: 1.0 References: <20240225114204.50459-1-laoar.shao@gmail.com> In-Reply-To: From: Yafang Shao Date: Tue, 27 Feb 2024 13:48:31 +0800 Message-ID: Subject: Re: [RFC PATCH] mm: Add reclaim type to memory.reclaim To: Michal Hocko Cc: akpm@linux-foundation.org, hannes@cmpxchg.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: CFC98140005 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: jo1q8nsc1er1i6pkwejqx6r7ya4y7t9k X-HE-Tag: 1709012948-87480 X-HE-Meta: U2FsdGVkX18tfs1MmUMBk2Ql0TfeWNK+9aZVfalLpCPNLlatNzR9NDL2QNCkcKhKSp/onudWlgJYyeOPzENtWntJeugpbZKqSOsDpUodU5p4ag2ub7L0n577oV2dkdG9nLP7WI5cG4vyIgfED3AESllUpbGso+Fmep23dqMg912NukYu9/1hAV80dQ32gTVWmmgEa5qmK9a5QfzAiOiqu+oIa3RCj6SSAguZO/rOiNU6Ac4aOyxWlXNYE7d6n9SIuo7t7aT37ByHfQbSeiNuAuiHn96LNa6l2+26WtsZWedZo1d7QVbGWC/xb+Lgpg9wwurXB8CeX75YOZDBQS2kip9F//Y7tT4PBBxFeP54fUF95taY1B+A6es5hMC7WNywIXOZ3RNTCkE7hGuQPvH/ST652qNuF0xCDE0Nm14r2mUw+ynudmNsJcwqmtdR0rsVBhShbUmhVzNIq41GElr47PBQiyOTcfd7HLevI8iFSW51iNQmcrHLbsZr210GHiCZtVS1OIunnom07npnUkSQGoPWHqy4n5plrI+GCIReCKfsQ0xXiEqoFbPuzQFpiZSoJPT0RElzFA0ZABWIdDbHjeIzYAu4mYrNt30kyuXUjSZPD9Fv09Oyei53CbwH+h3l6hXgnvECnTZoq+F5dJSfbLm3O22x0MMNmX43fZ+hN2sN7k1QSbJ/mWLHEKcyGUKMMVnX6NM26PDAfSzDeWoCjBeG/+tQX35dBRlf7/rTMDIHZ308k2D0KGU6OlY9M4wGUBIbZX6jvj/wkhOD1Y74BOfK0/WrrvspAzp8t+jS0AgwpCuY962+jirg+acq8OuLzGk89TI0qlNcsjIxJ2AXvD6MRolVBZi2UqPmnJX/JiS8i9zH285JWgaZ+PnuCEMt6hWutfo4vhyQKSLRR1/EGzInP61orXPiCWOV6sLYlzIfdFnHJXc9MOALogbvru4bsGMli/WXSr6SVWTo1W+ yPkaFFn7 RyZq0IHMJLGMMjD/scwnE9xpvX0gk3neeeRvO8GiVB1V1TFIOl99Ux1F8ptcqa5raqOTW X-Bogosity: Ham, tests=bogofilter, spamicity=0.000569, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Feb 26, 2024 at 10:05=E2=80=AFPM Michal Hocko wro= te: > > On Sun 25-02-24 19:42:04, Yafang Shao wrote: > > In our container environment, we've observed that certain containers ma= y > > accumulate more than 40GB of slabs, predominantly negative dentries. Th= ese > > negative dentries remain unreclaimed unless there is memory pressure. E= ven > > after the containers exit, these negative dentries persist. > > Have you considered using memory.high limit to trigger that memory > reclaim? That is surely not focused on the neg. dentry cache but it > should keep the overal memory consumption under control. Upon reaching the high watermark, both pagecache and slabs are reclaimed. However, it tends to prioritize reclaiming more pagecache pages over slab pages, leading to persistent memory consumption by slabs. Setting memory.high too low may negatively impact the working set. > > > To manage disk > > storage efficiently, we employ an agent that identifies container image= s > > eligible for destruction once all instances of that image exit. > > > > However, during destruction, dealing with directories containing numero= us > > negative dentries can significantly impact performance. > > Performance of what. I have to say I am kind of lost here. We are > talking about memory or a disk storage? Removing an empty directory with numerous dentries can significantly prolong the process of freeing associated dentries, leading to high system CPU usage that adversely affects overall system performance. > > > To mitigate this > > issue, we aim to proactively reclaim these dentries using a user agent. > > Extending the memory.reclaim functionality to specifically target slabs > > aligns with our requirements. > > Matthew has already pointed out that this has been proposed several > times already and rejected. With that being said, we haven't come up with any superior solutions compared to the proposals mentioned. > Dedicated slab shrinking interface is > especially tricky because you would need a way to tell which shrinkers > to invoke and that would be very kernel version specific. The persistence of this issue over several years without any discernible progress suggests that we might be heading in the wrong direction. Perhaps we could consider providing a kernel interface to users, allowing them to tailor the reclamation process based on their workload requirements. -- Regards Yafang