From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AF380C25B75 for ; Thu, 23 May 2024 19:55:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2843A6B0093; Thu, 23 May 2024 15:55:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 20D216B0095; Thu, 23 May 2024 15:55:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0AEB76B0096; Thu, 23 May 2024 15:55:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id DE4276B0093 for ; Thu, 23 May 2024 15:55:57 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 685CFA0423 for ; Thu, 23 May 2024 19:55:57 +0000 (UTC) X-FDA: 82150716354.30.8955896 Received: from out-187.mta0.migadu.com (out-187.mta0.migadu.com [91.218.175.187]) by imf13.hostedemail.com (Postfix) with ESMTP id DF39220019 for ; Thu, 23 May 2024 19:55:54 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=R+CIzOY+; spf=pass (imf13.hostedemail.com: domain of roman.gushchin@linux.dev designates 91.218.175.187 as permitted sender) smtp.mailfrom=roman.gushchin@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1716494155; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=EfhkEzWabknfi/ShuVBS0FWHdosprJqhhLCoPMo+WAI=; b=z81CEnRy5MM1TAS6Q24Cp9EK55dWWjyle1HNeD/JoWVXdWy+uoC0Ru49RiiiqR8/jpDrmy TvMNDFyd8X+gsxG0Qf8xZ8SZPUKtJkyoe4AO+gG84+u9TfbMCgU/wFd2JzhtASFyN8eMvr aKL1kh+tyAVjsOGBpXNidOCKKOpqU7U= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=R+CIzOY+; spf=pass (imf13.hostedemail.com: domain of roman.gushchin@linux.dev designates 91.218.175.187 as permitted sender) smtp.mailfrom=roman.gushchin@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1716494155; a=rsa-sha256; cv=none; b=PA1NUxggMVWQidYeNXlvynQ7Jo1CZ4d2cDDET67J9VYKJYOS7yKd6rOxbalFdNneIEUHKv y5liAb/5YomSyQhE9vYM9gHEFZchm+gNdY8adiStPFAfr10itE/0IgvPURKqEJ5zHfrTJm Tsu4QuKl0QoI1/WDXXoR3QIhrBwUZ/E= X-Envelope-To: ryncsn@gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1716494152; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=EfhkEzWabknfi/ShuVBS0FWHdosprJqhhLCoPMo+WAI=; b=R+CIzOY+P1dVjCcgGIclYcx9Ar6XmcGgx2OC/vPyjecvjUv3j810uDfQAczcRgk9EjKJYP F8lOyXOcRGA/m0Is3BcHwegpdQxx9ua8T0QScHaLDm1DKtcVipSIOS3dHlkM6Mif/2EIF5 aGcVkB9dUi6xyXqQ+XSuiJ5mqq+SL04= X-Envelope-To: shakeel.butt@linux.dev X-Envelope-To: akpm@linux-foundation.org X-Envelope-To: muchun.song@linux.dev X-Envelope-To: hannes@cmpxchg.org X-Envelope-To: mhocko@kernel.org X-Envelope-To: willy@infradead.org X-Envelope-To: linux-mm@kvack.org X-Envelope-To: linux-kernel@vger.kernel.org X-Envelope-To: gthelen@google.coma X-Envelope-To: rientjes@google.com X-Envelope-To: chrisl@kernel.org Date: Thu, 23 May 2024 12:55:46 -0700 X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Roman Gushchin To: Kairui Song Cc: Shakeel Butt , Andrew Morton , Muchun Song , Johannes Weiner , Michal Hocko , Matthew Wilcox , linux-mm@kvack.org, linux-kernel@vger.kernel.org, gthelen@google.coma, rientjes@google.com, Chris Li Subject: Re: [PATCH rfc 0/9] mm: memcg: separate legacy cgroup v1 code and put under config option Message-ID: References: <20240509034138.2207186-1-roman.gushchin@linux.dev> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Migadu-Flow: FLOW_OUT X-Rspam-User: X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: DF39220019 X-Stat-Signature: tmet55ewdtwybs47z6frj9czjtqrkwct X-HE-Tag: 1716494154-671796 X-HE-Meta: U2FsdGVkX1/fVwevfsl0gSXIBOt/BiNJVdkCWZOy1P8cJGvdFiSRSD4C1t1dIfTxmSc55I0+wbgUpbJNBetUkxk06xBXOJjZM3N1D2Wuk1i7Shq4GrSwv3Auj+IP+UarV9zcqUJNtHcrUQkVXc543YNTHmEeFW7wV02K+6/BDZvajkdA42mxFuPoQm8txLTBNcl7cE0DhRHKNCKMlnwFKJBJHCTGw0G7cr7o1PWv3HZaZJXXu+GrwL1q7eKjNf2EgLZzgij6SazcqAegyUT/ER2FXHgTiVh0fo/wRTbkKhpX4NScxBhBqkeoabjjrDD6vxMCXIozcoPX7ZKbn/lc2GxD09olBUMQImgKB3dZF9sYDE7lx+d05IUb6MdkS6e1ipmIE3mRdq4HR/pCuGxKCGxUQXV48xT+9IrSaolklsRhY0cpOtljGGy0PKz/goeWxfwJNV+18FK+hzdRG1GH5ajPHwrkviiUIjBlyltIvsNdrCTIhv+XG3q3Gf2Qfbwn9CAHxgPIQYOkX+sSmUrr3YX+Q4jDgM+l2IkfM0ovNvy8zOokGVFu7wnBi4zavnW3BcDmXIgk/9jR1sD6kyDXpnM4mG9FASOkAWDn/mqkyPIT50x0aZqj8s07AzCyp6aq2BU1ACo1D9P4UyELNHSCS8ejrTQ4TgDDbiZB77RI7MTlfTRBKUT++1wydqjglzVF4SQexNR8XwE7ML7MuBdI5IXd6ln+ksFoN+7TjjWxXGah7fHWJlBHDsWcexZyAFh/hNP3io7FTWUn8c3vv8sYT2wJmgcw1Re0nfM8HaDkNToyk8asoFPIRHmkzPkzuXvyE3ERP8FuSBwFMahytwp5MlzncVooYjjZ2fTbzp+7sTUGGkiq8lR5h+7zplh0O2dUCW3dcupR9p4K2xj3+BswC35UoTEzqvd1UEuTWw075Fgeleoi9ezsy1ekiTZ/Uw6dxj4ASn+G3N+d/w1V6+6 mrWj0SdR lxxercTRRuqgw7OWLaJYOkgK534j/tAopL3AO3szBKpWT7UEWCZEqAv7RFvrTCmE37qaOH5l5/xSEkuAc+0qOYFCV6eLsK5Sd0ZC8Xs7Vsg4oQRUDy9ymWBwSsRNwkDf5DPaWZopfdJ9XpJ9awVIDb8G9YiFJ6Le8plHHwoyyEeePvB4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, May 23, 2024 at 01:58:49AM +0800, Kairui Song wrote: > On Thu, May 9, 2024 at 2:33 PM Shakeel Butt wrote: > > > > On Wed, May 08, 2024 at 08:41:29PM -0700, Roman Gushchin wrote: > > > Cgroups v2 have been around for a while and many users have fully adopted them, > > > so they never use cgroups v1 features and functionality. Yet they have to "pay" > > > for the cgroup v1 support anyway: > > > 1) the kernel binary contains useless cgroup v1 code, > > > 2) some common structures like task_struct and mem_cgroup have never used > > > cgroup v1-specific members, > > > 3) some code paths have additional checks which are not needed. > > > > > > Cgroup v1's memory controller has a number of features that are not supported > > > by cgroup v2 and their implementation is pretty much self contained. > > > Most notably, these features are: soft limit reclaim, oom handling in userspace, > > > complicated event notification system, charge migration. > > > > > > Cgroup v1-specific code in memcontrol.c is close to 4k lines in size and it's > > > intervened with generic and cgroup v2-specific code. It's a burden on > > > developers and maintainers. > > > > > > This patchset aims to solve these problems by: > > > 1) moving cgroup v1-specific memcg code to the new mm/memcontrol-v1.c file, > > > 2) putting definitions shared by memcontrol.c and memcontrol-v1.c into the > > > mm/internal.h header > > > 3) introducing the CONFIG_MEMCG_V1 config option, turned on by default > > > 4) making memcontrol-v1.c to compile only if CONFIG_MEMCG_V1 is set > > > 5) putting unused struct memory_cgroup and task_struct members under > > > CONFIG_MEMCG_V1 as well. > > > > > > This is an RFC version, which is not 100% polished yet, so but it would be great > > > to discuss and agree on the overall approach. > > > > > > Some open questions, opinions are appreciated: > > > 1) I consider renaming non-static functions in memcontrol-v1.c to have > > > mem_cgroup_v1_ prefix. Is this a good idea? > > > 2) Do we want to extend it beyond the memory controller? Should > > > 3) Is it better to use a new include/linux/memcontrol-v1.h instead of > > > mm/internal.h? Or mm/memcontrol-v1.h. > > > > > > > Hi Roman, > > > > A very timely and important topic and we should definitely talk about it > > during LSFMM as well. I have been thinking about this problem for quite > > sometime and I am getting more and more convinced that we should aim to > > completely deprecate memcg-v1. > > > > More specifically: > > > > 1. What are the memcg-v1 features which have no alternative in memcg-v2 > > and are blocker for memcg-v1 users? (setting aside the cgroup v2 > > structual restrictions) > > > > 2. What are unused memcg-v1 features which we should start deprecating? > > > > IMO we should systematically start deprecating memcg-v1 features and > > start unblocking the users stuck on memcg-v1. > > > > Now regarding the proposal in this series, I think it can be a first > > step but should not give an impression that we are done. The only > > concern I have is the potential of "out of sight, out of mind" situation > > with this change but if we keep the momentum of deprecation of memcg-v1 > > it should be fine. > > > > I have CCed Greg and David from Google to get their opinion on what > > memcg-v1 features are blocker for their memcg-v2 migration and if they > > have concern in deprecation of memcg-v1 features. > > > > Anyone else still on memcg-v1, please do provide your input. > > > > Hi, > > Sorry for joining the discussion late, but I'd like to add some info > here: We are using the "memsw" feature a lot. It's a very useful knob > for container memory overcommitting: It's a great abstraction of the > "expected total memory usage" of a container, so containers can't > allocate too much memory using SWAP, but still be able to SWAP out. > > For a simple example, with memsw.limit == memory.limit, containers > can't exceed their original memory limit, even with SWAP enabled, they > get OOM killed as how they used to, but the host is now able to > offload cold pages. > > Similar ability seems absent with V2: With memory.swap.max == 0, the > host can't use SWAP to reclaim container memory at all. But with a > value larger than that, containers are able to overuse memory, causing > delayed OOM kill, thrashing, CPU/Memory usage ratio could be heavily > out of balance, especially with compress SWAP backends. > > Cgroup accounting of ZSWAP/ZRAM doesn't really help, we want to > account for the total raw usage, not the compressed usage. One example > is that if a container uses tons of duplicated pages, then it can > allocate much more memory than it is limited, that could cause > trouble. So you don't need separate swap knobs, only combined, right? > I saw Chris also mentioned Google has a workaround internally for it > for Cgroup V2. This will be a blocker for us and a similar workaround > might be needed. It will be great so see an upstream support for this. I think that _at least_ we should refactor the code so that it would be a minimal patch (e.g. one #define) to switch to the old mode. I don't think it's reasonable to add a new interface, but having a patch/config option or even a mount option which changes the semantics of memory.swap.max to the v1-like behavior should be ok. I'll try to do the first part (refactoring this code), and we can have a discussion from there. Thanks!