From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E06DCC021A9 for ; Mon, 17 Feb 2025 17:57:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5A7A56B009E; Mon, 17 Feb 2025 12:57:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4E2816B00B0; Mon, 17 Feb 2025 12:57:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 35D336B00A4; Mon, 17 Feb 2025 12:57:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 151DB6B00B0 for ; Mon, 17 Feb 2025 12:57:53 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 79CD04BB58 for ; Mon, 17 Feb 2025 17:57:52 +0000 (UTC) X-FDA: 83130194784.28.6340DDE Received: from mail-ed1-f65.google.com (mail-ed1-f65.google.com [209.85.208.65]) by imf20.hostedemail.com (Postfix) with ESMTP id 857241C0018 for ; Mon, 17 Feb 2025 17:57:50 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=suse.com header.s=google header.b=gTjhQ05M; spf=pass (imf20.hostedemail.com: domain of mkoutny@suse.com designates 209.85.208.65 as permitted sender) smtp.mailfrom=mkoutny@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739815070; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BBNDe4VDZ7B1rKJS4SghuqNUBX7l/JBij6Kgv1XI5D0=; b=psooirC1eD8iI0ZnDZDZyDWlYKjDz0tjUkowlzh4UqMedBQp2OHJDu5OEfP/rzFY2ehYer oizmzzpJU3p890lOaiMY0qhDhBKk+cfNXleQ7dlrOuxUYy0m52S7QVznmbgN/tYY4iV9p4 RfonLqLFMm77S2mw7H2+4+HF3245m2Q= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=suse.com header.s=google header.b=gTjhQ05M; spf=pass (imf20.hostedemail.com: domain of mkoutny@suse.com designates 209.85.208.65 as permitted sender) smtp.mailfrom=mkoutny@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739815070; a=rsa-sha256; cv=none; b=rBHddXxOFZ3TwC46X3TmmgTEDhgETBmL1LoFsWk52GYniEZyrU7ZHbjLIPmVq7HOA/PEtI 3ytHjgUBsxTYf98gMA18AOzpFv06eWcGh2q3kXlJj26l9v7yF1CQsx1HzinLiadps/P8yg GxuhXKFkTSZf/Hv2YSI4oPwK/y25jU8= Received: by mail-ed1-f65.google.com with SMTP id 4fb4d7f45d1cf-5e0505275b7so2509097a12.3 for ; Mon, 17 Feb 2025 09:57:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=google; t=1739815069; x=1740419869; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=BBNDe4VDZ7B1rKJS4SghuqNUBX7l/JBij6Kgv1XI5D0=; b=gTjhQ05MFcTbILchNp17tRMoibM0xER2CE0vthOaU+0n4MR/9WcfnB0eZrb3ahzWhS sMq9TQ5xqYii6/yVdZy87XLPhyDmLe1a2fgxCPXxAhyZjgPesW+hGvJNhEXVTh5U3W4+ 29eVF4cUSyYk7nrMY46zFdBLcd1Dr8RnImP/J0ShgkL6FCwnEZ+uTJ1Rj78nkZTfYJwy spzZq1eakCA3GAdFGDpr/lBxpCI5Knq8FPZbpxrBSlaRQ8eTWAIPHFYBsiRuQNpGiVX9 05cZbnhBPtnt2zQZApO2QUSS+m0oM6wdBHlkmOGsgql3DkxnorllGxtqC4vQDnmpkx0f lzTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739815069; x=1740419869; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=BBNDe4VDZ7B1rKJS4SghuqNUBX7l/JBij6Kgv1XI5D0=; b=wzMe5QroprdB5fB6KELhMwh8JSVFvGuNHNlNmtni/k7dWl1LSfry4QiVVE92xTKYEv k9nGP8+lTGiw2WvKL2+GUvehgCnNbPW1fOF2SfF0THVUbYDj23y4tPOXnwjPHTGi9bxV U6NKHdPu5gPzdLyYrS6xMxS9ftSpGcmk+VsIBidK/6vD50DJ7vhkxWmqP0D7Wq7XkmoU 6s3T/L2+zxSZ2vqv+aKdGCYMfhWLZc3r7v2y+bS8Gvn6GrqVddTTJcZ9NURZM1WTBgSR yTtdhQUyzdqK3G13VVT1cjN+rE2r+83vPrCmMWJRXjsfoh1OPbp8hHm+IOf/vqqqz9H7 EFBg== X-Forwarded-Encrypted: i=1; AJvYcCXmsUNYBJ1Ph7vbhV8mXlPtwUAfcqIlTtFNYVo4Xha7ch/McTBDMdEPre3h8fjN98A5wcHsAzrYIw==@kvack.org X-Gm-Message-State: AOJu0YxrnsDkhmJXbL+9KkHU5SPObgkfpZqcmK68N3uUZeUu9LZV9GF+ CVVPbl7+Mvn0B0BzQPqNJZ/DRASOBfIqdRI5yUjMc3Fzv04YxoRFH44j5cLyPe8= X-Gm-Gg: ASbGncvsMzLjKeZ94gHuz5fhrVNSvCzSWNjEuFEr8oRrlpVGFI9p3bIZtOrlwknmDlb SToOxBH1XzZbQPwrHyHlV39jhW+j+Jj38eswLWL0QgM0p4c9NQssVFFgN+26WieKX0Pge1CtVxi OkbcaiAmw1vZ0ywGZ+h1nXaK0JzS0kvpjRXb7Shz/s/rQto4wOJf1g+u4K8WhPq0FBg/xoYf80b 25tZVkKtSOYJrIcwM9l/EGQaaLDxBtEtL94yMtdQN4tf20avFXeaBiOqZ/LFSDWo8oOrVbLM8/L B8aEjZraq1ZJkxQc6w== X-Google-Smtp-Source: AGHT+IHhLdwZj2jv74Mbs98LUBVEWhEdTq/6oUDnDWYmDroBwSv6ozcHSeV/0qNpWeUox6P6nvdeuQ== X-Received: by 2002:a05:6402:3495:b0:5dc:c9ce:b01b with SMTP id 4fb4d7f45d1cf-5e0360f9622mr9042695a12.8.1739815068694; Mon, 17 Feb 2025 09:57:48 -0800 (PST) Received: from blackdock.suse.cz ([193.86.92.181]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5dece28808fsm7407349a12.75.2025.02.17.09.57.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Feb 2025 09:57:48 -0800 (PST) Date: Mon, 17 Feb 2025 18:57:46 +0100 From: Michal =?utf-8?Q?Koutn=C3=BD?= To: Shakeel Butt Cc: Roman Gushchin , Johannes Weiner , "T.J. Mercier" , Tejun Heo , Michal Hocko , Muchun Song , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Meta kernel team Subject: Re: [PATCH] memcg: add hierarchical effective limits for v2 Message-ID: References: <20250205222029.2979048-1-shakeel.butt@linux.dev> <5jwdklebrnbym6c7ynd5y53t3wq453lg2iup6rj4yux5i72own@ay52cqthg3hy> <20250210225234.GB2484@cmpxchg.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="nesrzqfldixxuyvr" Content-Disposition: inline In-Reply-To: X-Rspam-User: X-Stat-Signature: jeto9o7k7bsimxgdt3h3gr9hjp9hbcb6 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 857241C0018 X-HE-Tag: 1739815070-510090 X-HE-Meta: U2FsdGVkX19Z2QrAvmRKXR2E2ZWOHtYR7RSbwQJ2GbBb8MHUMpPX6BZZLnU0yS/bINaNFe8vZzx9Bbc/9fLQBh9V16SGlssL5NkepbF8gFqbXPIUhEa/gLvvBbYlGUQqZAXsBPEzF8LJh+wwLq3WJyxKHKLtIzTKHMuDhZE5kRLZ7/8fIcjkFvnq7W0tkhhwsQRqhlyYx0QcP57/uvAxIw2SEm4sSUcM2dX31e+TvCROcx0NnoqXFmVsM1cNzwQiGgQNW/siB2/+AyeEtrrItJHZWIx3YfuEFMNfXno5jbB5Z2Bn30/OaktfPh0dGNAsAVGd0CqeXyjXg39/pyB57FF3NaZksEemSzfbhoJG6pvXeSpnWscCaW067MhDk/9BpZAlNO/Q2IkfVCO5Sheu6MkeNn6l1VfpcA6fZiMOaWWZf6siOvwtlpUHCgEyKHXQM574lNjSkXpykLxaJjM0YaOEEnT+w3JAeOLZIUOt/5oAjEmTY+C/VDEQ/fxZ6mUsofCqXU0dCp0A+NSdJP95qofriy9CMp5SCEhkZwFQO+BqEZtVrhBJCG7rDGKEdUlhIZCChv6O5GhyxYWzYf+2nlbSeyZq9TUXWCtiBLIcepkihmBdz1x0Bp53LtBXhdG+sgI7P63Q7yAI0sNyhmdngSH91nbUaaLCoWKZlGU80pZjpT1MA2w377xthvjV+MEZAZnnnpKRc/ckmM7hilaoyEAPPj1fcU6k36F8jeafa1+z1xHC+7ngVGGUstmztijq5bl8JuB9xLGHAxfFglhVG1NNEbVe8ScQNfo7QyzZA7H9+531aE76nSXdFb88LC1jB2HxZ9haICuwtdNyZ2WEUYt5dwLF49pF40jpt6iEj4fM5GDDGMIHS4nkOuLBkcb2B9WAK48FHEXvUd5QH7VWTyOKtsL6OXoUNBMZ74kIvGenZpdNL195mRbnRc/l6ruZ+3OYvpUM0XsiW+5B1oY KGaXHmQ7 pnawhOS6UpAL8yUEpe09iYBntDKkPVDKxukr/nPm0Z1ThVw+vo5ug9Y8EsO0hCd/UfXsYKRbfzUAA7J58oE2Ll0tItNEpx0DOsBs2O6Y0to3OOV5XBhNB7b61GLRSurABgNzDrriJqxz46UemiVEjmYCTZIwW4DVdGF/FsS1ljRNMvb8G0zxpEmngZITHy//xLu+Ki7Um1FExozFPAjfGWMylZhVYmFLHOXZqGDIrXNzzHbgiPvkaqo2KI54Qx+mqKOXXKRQgttludv/aOwE4zFtQPFVfWiNW6OpS6rjW9Wpe3LPePCSDuIwZvkW3fHp50aMTjN6UqRdjcPLtuaTicLVCeWJvcsq2grVnuv07tHlFwilMNRgrrHgZOcrNiGkRXygvtLkTGCCvHeg2PspptWYUSq7KZGniGa79qsmqVRyN0QGBBUQp7SYDVEZua5fQBfgP0kgKascpwMHswyAx1Ni1sjU+evq0bmVkkNQZ1rciAR+K0F1s089Cm2d/dLM4tZJefGIEPDxz5Ssnxp7zM9Vi4oSCG7FPm5DcgotKA9qj+/p9nQdKwAzLolF6r1Y8uTEOAcON8OuxuktWge9spiuJTaEu3wiTWbUpOqXVxviMvEDNIswCqQeigMUt0pBeCPo17qIHIFjLELhjY6Lho3AILQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000002, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: --nesrzqfldixxuyvr Content-Type: text/plain; protected-headers=v1; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Subject: Re: [PATCH] memcg: add hierarchical effective limits for v2 MIME-Version: 1.0 Hello. On Tue, Feb 11, 2025 at 05:08:03PM -0800, Shakeel Butt wrote: > > So maybe someone can come up with a better explanation of a specific pr= oblem > > we're trying to solve here? =20 In my experience, another factor is the switch from v1 to v2 (which propagates slower to downstreams) and applications that rely on memory.stat:hierarchical_memory_limit. (Funnily, enough the commit fee7b548e6f2b ("memcg: show real limit under hierarchy mode") introduces it primarily for debugging purposes (not sizing). An application being killed with no apparent (immediate) limit breach.) Roman, you may also remember that it had already popped ~year ago [1]. > The most simple explanation is visibility. Workloads that used to run > solo are being moved to a multi-tenant but non-overcommited environment > and they need to know their capacity which they used to get from system > metrics. > Now they have to get from cgroup limit files but usage of > cgroup namespace limits those workloads to extract the needed > information. I remember Shakeel said the limit may be set higher in the hierarchy for container + siblings but then it's potentially overcommitted, no? I.e. namespace visibility alone is not the problem. The cgns root's memory.max is the shared medium between host and guest through which the memory allowance can be passed -- that actually sounds to me like Johannes' option b). (Which leads me to an idea of memory.max.effective that'd only present the value iff there's no sibling between tightest ancestor..self. If one looks at nr_tasks, it's partial but correct memory available. Not that useful due to the partiality.) Since I was originally fan of the idea, I'm not a strong opponent of plain memory.max.effective, especially when Johannes considers the option of kernel stepping back here and it may help some users. But I'd like to see the original incarnations [2] somehow linked (and maybe start only with memory.max as that has some usecases). Thanks, Michal [1] https://lore.kernel.org/all/ZcY7NmjkJMhGz8fP@host1.jankratochvil.net/ [2] https://lore.kernel.org/all/20240606152232.20253-1-mkoutny@suse.com/ --nesrzqfldixxuyvr Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iHUEABYKAB0WIQTd6mfF2PbEZnpdoAkt3Wney77BSQUCZ7N4lwAKCRAt3Wney77B SatUAQDZaNdWHwjzXd9poJ9nHN5lCECypBk//h7v52pBxjbhGgEAzcXby8B//Pga UrXQF9ErO8OmlXS3SMQq9MXR1nAU8Ak= =C5m/ -----END PGP SIGNATURE----- --nesrzqfldixxuyvr--