From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 239B1D30000 for ; Fri, 18 Oct 2024 12:31:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 746296B0085; Fri, 18 Oct 2024 08:31:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6F6476B0088; Fri, 18 Oct 2024 08:31:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5BC806B0089; Fri, 18 Oct 2024 08:31:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 3DD366B0085 for ; Fri, 18 Oct 2024 08:31:31 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id E8D02A07A5 for ; Fri, 18 Oct 2024 12:31:08 +0000 (UTC) X-FDA: 82686658278.13.1FA87AC Received: from mail-qk1-f178.google.com (mail-qk1-f178.google.com [209.85.222.178]) by imf22.hostedemail.com (Postfix) with ESMTP id 86196C0017 for ; Fri, 18 Oct 2024 12:31:15 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b="r/Bsl3ZL"; spf=pass (imf22.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.178 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729254543; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Ljwrlb/z0YLznPxkxUIrF1QefOIilHj7zulCESswlBA=; b=iIElfiHrMMyPw6w0RTIYo+GAKJXnmsM3xLpym61WYaierpMHCt6uwPtWzUiditWvqnBmq5 UdFFdRsvtwtdisWDKjHP9mbrbE0letBUQy/eTbS+UkV1NehQNDgAXW4bc3TbAj18PY6xPc ZmHXzKCmMnGzEkXJA5lqlFW1RTfOrnI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729254543; a=rsa-sha256; cv=none; b=Fl721jWcNCitTIVBc8/zinxE/9AE+Yzpn1df5xQGh3pLjhwULrlGG5DaaLOu2CcNTn3Uxz W1Qw3NMNyOX+fENKGfk2bSzMxFXA5DfZZKbvh6TjdH/Z9QGVMJQsyeTjudd9zAB40V4LqU GsPq80KN2p1e2zRDGJougPQYeQLNvr4= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b="r/Bsl3ZL"; spf=pass (imf22.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.178 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org Received: by mail-qk1-f178.google.com with SMTP id af79cd13be357-7b1488fde46so152682685a.2 for ; Fri, 18 Oct 2024 05:31:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1729254687; x=1729859487; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Ljwrlb/z0YLznPxkxUIrF1QefOIilHj7zulCESswlBA=; b=r/Bsl3ZLPywfsaNipY6Llkfu8W3pUBlglLHRVpWs6gFZCvvX7Gf/Amyogan3UhLf/J eZYebtWY9axryrsH7E1IJsICoEU7ndLXScd/6YJAQZJ+y2+WJOy1u+bKgUDP64a6/epc WWRVbECIkrNJsp5o8wn7ijJZkF/wYP3YvMzRUoHjk+rnnKD8OjE3G4DIBiEjrpg4Vkyq T35aPib61Nm+u5WkMmy20SGIarg7mgVkylBI9eSsk4fIRqpj9JNOgt1J0Ibsrf0F+0ha rtp8/x1XhBc96IfOVyfKAS8B/2QviJxweEtaNlm3fDrhsoUV7egjbprW6EDaolQ4OMVg Xnpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729254687; x=1729859487; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Ljwrlb/z0YLznPxkxUIrF1QefOIilHj7zulCESswlBA=; b=Vt72X4AAQMPd14loId1hGqRrwLOB7ZrVRs0RNrCyU9NmEW87VVZLs1ILhWjUzQU3QI j7wYtS5HRo5/0z6GZJ/jVLOO/RHN+QHx4VrlKNWq/D5fHHBtHjM4k2RERXoAULJN4Sq9 +yXl5MPW4g1tqpVyvtZ90uTKtmoRXkNQV4aAWVo2/NWMr81x4qpfl+c2UXwhCA+YUbbs PubasqqUe3kEe3nI/F086kIl+h5XtmuJ70vwdXz7o6Kpl9OzT1TuGBsTmkKUYpoQ7zzm wvDspxG4Q1wsBsFgYUPk+uRpFffLvJm78NpF0q5rKITgGLx7t8zLPkORVwO4k9nLFPFo hviw== X-Forwarded-Encrypted: i=1; AJvYcCUXQjyMKdn4/3cKHegPAO8DZtVETT8Vbi+3FUSg84XZlA2y9tlqBomx8dJTpKw1cu3VF54exoU99Q==@kvack.org X-Gm-Message-State: AOJu0YyU1ChsEzIlwbIsq7GsOFgzl8TLdLKFC4EG6zsmu00wKq1hapMr bT3RXzKfz9R9Y7gRubwhwEk0tt36enopwvfUMkzxssnEBJcU65xbSC7xt+Gjk1E= X-Google-Smtp-Source: AGHT+IEpkqPl93VHak2DLYK90RodDlnBgH2nXjIf3o6uxKklx+2vSWQ80oe+0osSbgMlMBVXosCFnQ== X-Received: by 2002:a05:620a:468f:b0:7b1:3bf8:b3cc with SMTP id af79cd13be357-7b157b5a96amr270233785a.17.1729254687596; Fri, 18 Oct 2024 05:31:27 -0700 (PDT) Received: from localhost ([2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id af79cd13be357-7b15700b3f3sm63750785a.136.2024.10.18.05.31.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Oct 2024 05:31:26 -0700 (PDT) Date: Fri, 18 Oct 2024 08:31:22 -0400 From: Johannes Weiner To: Michal Hocko Cc: Joshua Hahn , nphamcs@gmail.com, roman.gushchin@linux.dev, shakeel.butt@linux.dev, muchun.song@linux.dev, akpm@linux-foundation.org, cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, lnyng@meta.com Subject: Re: [PATCH 0/1] memcg/hugetlb: Adding hugeTLB counters to memory controller Message-ID: <20241018123122.GB71939@cmpxchg.org> References: <20241017160438.3893293-1-joshua.hahnjy@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 86196C0017 X-Stat-Signature: myhpezn53zchzxaec31ucpn1zi8ddapz X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1729254675-117156 X-HE-Meta: U2FsdGVkX18i8vg1iU5oi13fC8Hj+PVGY3pIutoY+/ePeP6m+ngW7nHPawA47XNfVoN+pyDXSP37X1n4VZ5jmVN6AlANMwqCIgAE23qhYGR2TGJrXB5d2a5+eVH83FQCr1BHqiIgfUt20AcOvz95zemLXUpIAURZtB2J0acRmPo7dq1TnG6iXU79C4nwL0259/9mzh2NBKGHV44Z7OdvL0CH+g/d9HTN/GUKfCHTMgeJdUM5if0VgeLCegG2igXRO3YLtAcxtjakmNxwQlxrtmbZ7vh0/PwKmxmmB3vZkQ2lEJMuopcOC24y/lPRhUqvTttvyjmLoMjHVwlPD7oGjDktfUd7HrIHdk13JiwtISbVnm77XZZWqIF15F0CsG8pVx86Y/3lVaiEcbQzviUY1JDS4hsDCq8iTRzWAbEtqQxTYBB/xEoE+34YgO4BaUPvEpjh9esUiRLEDjYHOxLKx+dVdywuLn/8zU9UNWmL0TTPCyRdsAoRgSEGj51HeO80cIzaRVetmE60zdRuXpJJnBs6iwIyotfqjgA/m+dt+3neW3lreAzOVR6zAySpYzp9p9M4RPCHdRlGb9kZ+xYKunOKw6COfyorK1pZZwXr7XErHKHKJfRwP3eSGYP1SAsQW7YM1h9d1tk6Ibe4l9aLQcaSzWERpTx618JF7PewbuOloOgB4bQJZh9O2U1xz3CR6/7eiUUYh2iX5rMkyRxkeAuZTbUFxCqk2DQDKcedKFqs7O5rraTEllbF+yPZrLsb8scZKu47F2RsuFZONfZ+I3mkvKX/q5uHuOINJAM6FnsKwIQl+OQ42qHEDYxRGDzqZG1IFesoHs5AVwdg7MrSjn6HBsWxd/SDWUwQSuZONmrmn9QRDMoGGHh1MMKKmdcby9luRXVyoKn908tGfQ0JAVIbBo57DdS3pHbP0oEjaVnahl+B5vGux5Kv5nPW9zP8jFdS0f+fHeQSJDlAGl9 fwCL06O4 pZDo8q8IRs1MJ4lQPW0nhocSEw3u/24UU197AgmgzacN7Nrxf0AvK63Z8YV+J6cdl27T66adAC47DabqiYMa+TI+cUlcJpOHN0Db2dOGg7i+9k8BUValUUF0i2STdYyof0TkDMniyIwyh6Gzp6KSlrstqViMYcm0Nh58o3/MyGloXcSjlZpmAjsZI1swmO8mrGWbdaxHJk311SGSqinMnKsB6EcV2PNLk87WzHDaZJZ75q3fdBtQGURzdiA3o2X8u2joJQTgpZbMHBCy/bSkEOhnOzpUO9/l2RSBrqszBkpBZ1GWinNK8qpReqWrnwnfJuCXBckps6yNPSevFfxNn7mITu9Xq7PYAZtXDpDcvCo9sq+3stuVT/VYvqoodxnCRwA/mic+8nyi9NsxCmMoaywji5rzAXT1cdntWkIsjMFLBanMP59ypYH3vLWfodXmG+KjZzofHt5+GVoyd0a+gZvCi4C2tysjGsTYTGGNjZupT7D5ktMUbuXm60acSnU8NfE7O7l2cbPQbtzvasguWSun1BA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Oct 18, 2024 at 12:12:00PM +0200, Michal Hocko wrote: > On Thu 17-10-24 09:04:37, Joshua Hahn wrote: > > HugeTLB usage is a metric that can provide utility for monitors hoping > > to get more insight into the memory usage patterns in cgroups. It also > > helps identify if large folios are being distributed efficiently across > > workloads, so that tasks that can take most advantage of reduced TLB > > misses are prioritized. > > > > While cgroupv2's hugeTLB controller does report this value, some users > > who wish to track hugeTLB usage might not want to take on the additional > > overhead or the features of the controller just to use the metric. > > This patch introduces hugeTLB usage in the memcg stats, mirroring the > > value in the hugeTLB controller and offering a more fine-grained > > cgroup-level breakdown of the value in /proc/meminfo. > > This seems really confusing because memcg controller is not responsible > for the hugetlb memory. Could you be more specific why enabling hugetlb > controller is not really desirable when the actual per-group tracking is > needed? We have competition over memory, but not specifically over hugetlb. The maximum hugetlb footprint of jobs is known in advance, and we configure hugetlb_cma= accordingly. There are no static boot time hugetlb reservations, and there is no opportunistic use of hugetlb from jobs or other parts of the system. So we don't need control over the hugetlb pool, and no limit enforcement on hugetlb specifically. However, memory overall is overcommitted between job and system management. If the main job is using hugetlb, we need that to show up in memory.current and be taken into account for memory.high and memory.low enforcement. It's the old memory fungibility argument: if you use hugetlb, it should reduce the budget for cache/anon. Nhat's recent patch to charge hugetlb to memcg accomplishes that. However, we now have potentially a sizable portion of memory in memory.current that doesn't show up in memory.stat. Joshua's patch addresses that, so userspace can understand its memory footprint. I hope that makes sense.