From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99454C54E4A for ; Tue, 27 Feb 2024 11:37:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E8039280003; Tue, 27 Feb 2024 06:37:42 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E2FEE940008; Tue, 27 Feb 2024 06:37:42 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CD187280003; Tue, 27 Feb 2024 06:37:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id BA95B940008 for ; Tue, 27 Feb 2024 06:37:42 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 8768A80C19 for ; Tue, 27 Feb 2024 11:37:42 +0000 (UTC) X-FDA: 81837383964.30.2BB7C1E Received: from mail-qv1-f50.google.com (mail-qv1-f50.google.com [209.85.219.50]) by imf02.hostedemail.com (Postfix) with ESMTP id DDCB880004 for ; Tue, 27 Feb 2024 11:37:40 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=MpkxNfoZ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.219.50 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709033860; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Jp3I53DEKbYBOWpVcGgFVvNiPCsJ/kOBbceJ2jQyIEA=; b=O70lANYdx64Lq0+NF4lE+yQWSergmXIsEMSnZb9BhOLLHgv4nWNoEmFm02uCOho0on6s9Q qBNgFNuFsouZeBpKWRJY/tt/+DwDqy/3XpKiqRTXqEjl3y/CkGkqoy2+LDTTcrxvR4JNVX otKREwtBsk0VLrrEjc6z+NP1nWLdctU= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=MpkxNfoZ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.219.50 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709033860; a=rsa-sha256; cv=none; b=1TnEJbs0CBjJOwD0HkKHmJHmpVtMY3f5y5o1SP9OCd1E4owlAZ/UWSB2sFdIndaF000QEi BGiew5Z6Px7jRRrhHZ4tTPzZwytiLuqW1A5ErrA67hQ2p2GarFqdKpQUWhBTjY2zLLgaan Wj/y0UNcpgB/wJx3StDfynaVnJQLTh8= Received: by mail-qv1-f50.google.com with SMTP id 6a1803df08f44-68f51ba7043so26647526d6.3 for ; Tue, 27 Feb 2024 03:37:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1709033860; x=1709638660; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=Jp3I53DEKbYBOWpVcGgFVvNiPCsJ/kOBbceJ2jQyIEA=; b=MpkxNfoZzt6hyeMXuYzKTl2crPjlIghF6ONRV1UqKBZcQ4WsN1DhVdBEw/IX9imEXl e5JyDSHwp/hpAyUn5MMqO5zPfd8ROjeRpwl7ryrvkfu8PhFP1tvHM8cBtOU4kYz/jk0z m+eDetb4CQAoU4CnWXSKe1bTmLn+TVT+NSOQTmwPfAkKMy+8YNT277qeClNsXDhb1bFy JFcPTcqVbyySQV7MTf9xXHExNxI/qohBqVzZ5x5p7pv/S0DJ3NYF5ANRRsj4hyhy5pWP VogFAYGim2JsyN2nEHaXpU2r/rqjm4amxQCEOaHB0bYoN7i2DMUcUrsXUABkpi01tg8w X15w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709033860; x=1709638660; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Jp3I53DEKbYBOWpVcGgFVvNiPCsJ/kOBbceJ2jQyIEA=; b=PgpAIhkb11KNIY2BF16AjQIUvhxKVv0P4TpHpv2MoMLw3pWMIzEcK7H8UCjkUoZODv tb6MoFdWnHZz55jQutLniip4Emr+ySlekpstQLjldnKxLuwjqy0TXD8bdzhkVWuKIpDW HC8GdzwR6YeFbunCxbU03HTcUn3hVeLpUIlBKQxjLYKYkIYh8TbtrQjcu/yuc36thz+L Vyt4xq2Z9Jo0lhBL6o3OdoplEA5HoVZltGMjg3iA0KjeSVSFXZwzhrDrRpRX/D46EVXl Fb/egXN0DY0wCB+pbkja6Zw+/Ao1sB0JcUrnIk0F11cAIkLb06MZ076f+EDrvHI8mLBL SwnA== X-Forwarded-Encrypted: i=1; AJvYcCV0BFx/NasVkkLaws31ousQ+/SAp6q/6+zrouIPwEWqtJqvBIRUrxF/U3PI6KZJPoKEb6nWsz7PXOOtU5/8lD2D7jA= X-Gm-Message-State: AOJu0YwRdodh42TBkNl95kY0IF1rhutKotNGUbNPrmFKQ0V1Zhtq2xsG PEXzhDi9qH5E0weQR4HvEqgpWxkthPVnecphbN/05xSvvlEbynLlszT9GkWG+TYvILx3LJaSfkv u2X7Y/VrlYl0614NBsgRnoL5l+Uo/RoVcOVvi+fUbFoM= X-Google-Smtp-Source: AGHT+IEeHdktQalok7nMJMYpQh7nbhWbf+Dfw/Mj9U0IonvlldyXTTnVXP0QYPrxSTBIYfIiLVU1NDW8ZXjVvVmAILc= X-Received: by 2002:a05:6214:29ce:b0:68f:3ecd:f73 with SMTP id gh14-20020a05621429ce00b0068f3ecd0f73mr1936885qvb.47.1709033860060; Tue, 27 Feb 2024 03:37:40 -0800 (PST) MIME-Version: 1.0 References: <20240225114204.50459-1-laoar.shao@gmail.com> In-Reply-To: From: Yafang Shao Date: Tue, 27 Feb 2024 19:37:02 +0800 Message-ID: Subject: Re: [RFC PATCH] mm: Add reclaim type to memory.reclaim To: Yosry Ahmed Cc: Michal Hocko , akpm@linux-foundation.org, hannes@cmpxchg.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: DDCB880004 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 4gft3cyuhaeqr5wyb4b3isy3xtasykjo X-HE-Tag: 1709033860-6976 X-HE-Meta: U2FsdGVkX1/0Kkz5VuAW8Enb+yuJXNRWgN7vdXADNziJSb4F0lOrNlR45voRv3xfutemlG2oNGjRVqEruxg90FTfnGuifozRGm0AKK8JY6T+Vvoc/gLjG1tcJttGHVA6i3am431PjjiHM21xP1N5O2m2fiRL+nUqWjGD81v00I1fVd3gtBFwUHR1/sE5CtHdC6+uzYy/525AS5M/ZW30sivfizTWvjOLKS5QRvXFj5nq546IxRTLChcDNuMfP+hDpGNnRVopFtNs0g0XtfgeVw61j0RHPEVbAk73SAgG+X5R13HpqIuV2DR65QBxWmSPMJ10HgOSG7aiwJfLSkJbtTgbkXCpQpLiLnljP3rdoo15Ocw6HJyrsutHnr0vUes5gHAsC+MJO7l4EvsaEwt1qAoNMHw4T537RSNuRIKbDyitOe03+HTIqG6/dVLAKjv4/GQ50tHcI5IZja5lvgv3NTvKTYhtDQwonxUu4KOr5+55pVxNo6DO94RL2u4ey5J0Ad2Q/1WfkPLPAfegYJFg+Q99SM3EduRzse7AWmSmr1z3B5Ho6eD3rkcSsO5NUq5FUTTKnjE+cyfMFGMUwUflSBwU3nLG7ecUuvYa0X7bpTi8sMeUARVQ+7BRYm49PviIqeJdVe7F0sqHsG/eDYVOZSN5NePwH5Lkyvk4c5yfZACJpW0xtrf+3kqpSqFccVBD9L0d+bzzB7QHoPUZthcSy+ps3YKa+BY+37XVVRTdCUxTi1KujbPLe27/pjSCA2E4YtYPZfHo7FdchXucmH+WcLMyDUlz78Pi89JALI0NemBr5AgHgfnvJLqgwhwu5q/gxbT+gqcbFgBwMZrxVtYJjRsJPE8QwM21pwFEQhSTUQRCWXGMUGxOarnYx4GMDozILMA3Fu5J9CiQfEzkUoZUi9vaZuO80q4pI+Cx7J/qrVxu/B8wEjswMCr2eeG/2Tlbc4Lz73VRsGln60WfudQ qiV3ZWNU 02nKBDog6Kp5fzcfCGNfR7A8V+sf5gn8t6k+Q9cy6xyDXkpdYB0SUuwIWVfr8Olwey6B+ X-Bogosity: Ham, tests=bogofilter, spamicity=0.003374, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Feb 27, 2024 at 5:44=E2=80=AFPM Yosry Ahmed = wrote: > > On Tue, Feb 27, 2024 at 1:39=E2=80=AFAM Yafang Shao wrote: > > > > On Tue, Feb 27, 2024 at 5:05=E2=80=AFPM Michal Hocko = wrote: > > > > > > On Tue 27-02-24 13:48:31, Yafang Shao wrote: > > > > On Mon, Feb 26, 2024 at 10:05=E2=80=AFPM Michal Hocko wrote: > > > [...] > > > > > > To manage disk > > > > > > storage efficiently, we employ an agent that identifies contain= er images > > > > > > eligible for destruction once all instances of that image exit. > > > > > > > > > > > > However, during destruction, dealing with directories containin= g numerous > > > > > > negative dentries can significantly impact performance. > > > > > > > > > > Performance of what. I have to say I am kind of lost here. We are > > > > > talking about memory or a disk storage? > > > > > > > > Removing an empty directory with numerous dentries can significantl= y > > > > prolong the process of freeing associated dentries, leading to high > > > > system CPU usage that adversely affects overall system performance. > > > > > > Is there anything that prevents you from reclaiming the memcg you are > > > about to remove? We do have interfaces for that. > > > > Reclaiming numerous dentries through force_empty can also lead to > > potential issues, which is why we attempt to shrink the slab gradually > > to mitigate them. However, it's important to note that the underlying > > causes of the issues in force_empty and rmdir are not identical, as > > they involve different locks. > > > > > > > > > > > To mitigate this > > > > > > issue, we aim to proactively reclaim these dentries using a use= r agent. > > > > > > Extending the memory.reclaim functionality to specifically targ= et slabs > > > > > > aligns with our requirements. > > > > > > > > > > Matthew has already pointed out that this has been proposed sever= al > > > > > times already and rejected. > > > > > > > > With that being said, we haven't come up with any superior solution= s > > > > compared to the proposals mentioned. > > > > > > > > > Dedicated slab shrinking interface is > > > > > especially tricky because you would need a way to tell which shri= nkers > > > > > to invoke and that would be very kernel version specific. > > > > > > > > The persistence of this issue over several years without any > > > > discernible progress suggests that we might be heading in the wrong > > > > direction. Perhaps we could consider providing a kernel interface t= o > > > > users, allowing them to tailor the reclamation process based on the= ir > > > > workload requirements. > > > > > > There are clear problems identified with type specific reclaim and th= ose > > > might easily strike back with future changes. Once we put an interfac= e > > > in place we won't be able remove it and that could lead to problems w= ith > > > future changes in the memory reclaim. > > > > That shouldn't deter us from actively seeking a resolution to an issue > > that has persisted for tens of years. > > As observed, numerous memcg interfaces have been deprecated in recent y= ears. > > There has been recent work to add a swapiness=3D argument to > memory.reclaim to balance between anon and file pages. Adding a type=3D > argument together with that is a recipe for eternal confusion. *If* we > want to support this, we need to have a way to combine these two into > something more user-friendly. What if we introduce a new file, like memory.shrink? This could serve as a foundation for potential future extensions, allowing us to shrink specific slabs with specific counts. --=20 Regards Yafang