From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73D60E75458 for ; Tue, 3 Oct 2023 12:59:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D9EE38D006C; Tue, 3 Oct 2023 08:59:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D27FD8D0003; Tue, 3 Oct 2023 08:59:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BC8A48D006C; Tue, 3 Oct 2023 08:59:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id A89B88D0003 for ; Tue, 3 Oct 2023 08:59:03 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 75A98160307 for ; Tue, 3 Oct 2023 12:59:03 +0000 (UTC) X-FDA: 81304155366.02.4A597CD Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf04.hostedemail.com (Postfix) with ESMTP id 73E214002B for ; Tue, 3 Oct 2023 12:59:01 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=fy0ZykLJ; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf04.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696337941; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=FYg5ibJBXkGWVzVR22ngBY7niQUGDtz5Ol63vTS001g=; b=lRQKt4s97Vg/7b4sB438Y7In2MQobas04FYU7+rEKhlwqgWHIAkRvTEw4awbRUCykBDcDs NBSU0gSO1y1y5E6XB5s/ckICMC0lm1KNrqgPeyfW75PkJIYRA3WPxzPoH8roHdPfgQAWja bqVJ6d1pGyjo6H5mb43ATXv93pE3ATE= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=fy0ZykLJ; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf04.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696337941; a=rsa-sha256; cv=none; b=m1nCmYr4Cxz8ZwFik92EYD3+ZPFASzBzLhNuurXhWfseGtXCAuX3I8+kOXyR/sCF/B4/RB cR2vA8Vse605sTvDIgwwYrnmxGV7JTaoumcNHB8m5CZ8q4dBGloRGEelQJdBx15iASBvnp xzwYg88eRdpwL5iYA0Vf9UlXqFs/DdI= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 3766721892; Tue, 3 Oct 2023 12:58:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1696337939; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=FYg5ibJBXkGWVzVR22ngBY7niQUGDtz5Ol63vTS001g=; b=fy0ZykLJX2w2QY0lOH5c4Q8/XyQN7db0uLKnnd1wdWWbz+3/0xJXFEPpzAd5GK+nFTPqFC pAR5aEAkOEobNFSgo7D66JNwPHP+vG1Dir3iiSe433DMWGdYuDn/2NGfoIuKh2V4VL5rnj fuCnJ/PaeS6nHDtq8LLxdMuFJp+sNhs= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 252B5132D4; Tue, 3 Oct 2023 12:58:59 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 0okxCBMQHGXKKgAAMHmgww (envelope-from ); Tue, 03 Oct 2023 12:58:59 +0000 Date: Tue, 3 Oct 2023 14:58:58 +0200 From: Michal Hocko To: Nhat Pham Cc: akpm@linux-foundation.org, riel@surriel.com, hannes@cmpxchg.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, tj@kernel.org, lizefan.x@bytedance.com, shuah@kernel.org, mike.kravetz@oracle.com, yosryahmed@google.com, fvdl@google.com, linux-mm@kvack.org, kernel-team@meta.com, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org Subject: Re: [PATCH v3 2/3] hugetlb: memcg: account hugetlb-backed memory in memory controller Message-ID: References: <20231003001828.2554080-1-nphamcs@gmail.com> <20231003001828.2554080-3-nphamcs@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20231003001828.2554080-3-nphamcs@gmail.com> X-Rspam-User: X-Stat-Signature: sht34qgsynr66ztpkdg8xd4akmo9dmqd X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 73E214002B X-HE-Tag: 1696337941-447729 X-HE-Meta: U2FsdGVkX18qKNK60jevQyopYfZK1RZaIPAkWNZ6BGzRDhuM219YMS7MTR901/fGVyDoM+UblBffNd2zKlL87qpbOsUeguM2Ea0X/ZNITnAIVSUOnpfpSmSMEmD9ChKrCVxLDOd5/GCM66MbLLZwJMlGkze/50iirET2zPX1O8p+DUxYxExiWS0YYUKy/pmASyINwwA+C9VisVMnoEJEYX0wnR4EZelvA7vvZ/mtv5uG2OqWTloVRdjwy3OpT6kDcZDvSwsFqVMC5RLhLW4VfwrtapJhUmaigSlXaH5e6aS48iFBkWLV9VxbESUrXt2WX2gv7iFaw71rl9fACpl9uWoH7ovYZvNK7L9Z7cZo2Xrssam0YFYBvZZ7sb8SU784XAdn9OxQwITWzj+fNr9QpEW3jELw2+SvDYS0y/dtV3L9ECeI3naaj4er50yFzr4NbdtGMNDXZ+BshjLtGD65E9bjBLlC5TDRrji3MA8lQF3iGOHYVEEPqkTY06UDsawDwDTQVoGUWw1y4km2BI72JofwU3r3gz4Yaun4YMNI3DRhBUTb06syGZggEYZzxCMmO/jG/vb9wAoVj4c4AoYJ/CDJBP46WvnFIy7Z7bD2s0V8rgm3J0ReNwacm1KiL5ovWgtQAS83Fi/1areC360vSMPagTjs2PuvtibUwF1InfvVmC6NndiW7sgB41g7nJDUGrsa5+1Wwadm6gfWBsxbDHABmzIqsex6rGVu3BSmxBLMvCpoGdOOLKkXEw5YEAqtlX+iVFJKqhYvNRsQ7r7am++K8Nwgru3BXw9krygxhILE5H7D9eXyrjwJY3cBfluQzaUkOYDmxbX+gEPnSMNr5LndKvhXGpAhemjNOm1wiHCjXQ43CMNptguni5eZy6B3+RLLh5MqmZbWJm7MEuX8XNfV37xn5i9rLYqATuNH2lDj3HYVNVjAKainjbpxf451fEgkOgaStOtbnxo3I3v 1SWZB9Wj wMxVX0M7xJIjdoB4eC0wobTt+UaD7+MH4W7lMHxp5P3kOLY3gpnecJyXxoza3Le143y+TJGjAy8rUVf1QP/lUuZVYD2wXWSgB2mFzkdUn8sT5MhhEJOdNuUlA5bmHB6dM6bR0kodiWfV2vrLLeO9WbL8tgw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000006, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon 02-10-23 17:18:27, Nhat Pham wrote: > Currently, hugetlb memory usage is not acounted for in the memory > controller, which could lead to memory overprotection for cgroups with > hugetlb-backed memory. This has been observed in our production system. > > For instance, here is one of our usecases: suppose there are two 32G > containers. The machine is booted with hugetlb_cma=6G, and each > container may or may not use up to 3 gigantic page, depending on the > workload within it. The rest is anon, cache, slab, etc. We can set the > hugetlb cgroup limit of each cgroup to 3G to enforce hugetlb fairness. > But it is very difficult to configure memory.max to keep overall > consumption, including anon, cache, slab etc. fair. > > What we have had to resort to is to constantly poll hugetlb usage and > readjust memory.max. Similar procedure is done to other memory limits > (memory.low for e.g). However, this is rather cumbersome and buggy. Could you expand some more on how this _helps_ memory.low? The hugetlb memory is not reclaimable so whatever portion of its memcg consumption will be "protected from the reclaim". Consider this parent / \ A B low=50% low=0 current=40% current=60% We have an external memory pressure and the reclaim should prefer B as A is under its low limit, correct? But now consider that the predominant consumption of B is hugetlb which would mean the memory reclaim cannot do much for B and so the A's protection might be breached. As an admin (or a tool) you need to know about hugetlb as a potential contributor to this behavior (sure mlocked memory would behave the same but mlock rarely consumes huge amount of memory in my experience). Without the accounting there might not be any external pressure in the first place. All that being said, I do not see how adding hugetlb into accounting makes low, min limits management any easier. -- Michal Hocko SUSE Labs