From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 634B6EB64DD for ; Fri, 21 Jul 2023 20:44:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D45378D0002; Fri, 21 Jul 2023 16:44:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CF5AB8D0001; Fri, 21 Jul 2023 16:44:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B96228D0002; Fri, 21 Jul 2023 16:44:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id A9C048D0001 for ; Fri, 21 Jul 2023 16:44:14 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 6ADCE1403D7 for ; Fri, 21 Jul 2023 20:44:14 +0000 (UTC) X-FDA: 81036796428.30.684C3E2 Received: from mail-oi1-f180.google.com (mail-oi1-f180.google.com [209.85.167.180]) by imf18.hostedemail.com (Postfix) with ESMTP id BB4C31C0008 for ; Fri, 21 Jul 2023 20:44:11 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=cmpxchg-org.20221208.gappssmtp.com header.s=20221208 header.b=5hAajZX2; spf=pass (imf18.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.167.180 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689972252; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tEshZgxJu/wBZMZ7+AtLkJHqS7609jEJDv2p0gloftc=; b=jO1yGMtnq2nDZK2n/DKh/yB7AGb3S9EP+HSPfFEnruoi8q2Y3PmZ1cH2oQMBGeeQXbOs9x 9BLtxRJowf2pUA2BFus2GiTKHzV8YIuFhsadjmb1Vd1QOIRspxN1vxNE+wAv/FQTM34b/G BQ6agmYmCtDlYGncEt3kkcS6FrKg1xA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689972252; a=rsa-sha256; cv=none; b=FFt1YmH72O45H74XhiK9cmDntKvr1jkDvo14CAPOoeScyMG7DDuhb9BBAnfE6hl8yLH/FW chlR/pca6PiwrWRSu4z3LKd0fxDgwzgul0H2WO0S5Bj8TL2lBq0VlqlBvWYHIBYEonVez6 1+Z8CYKSNVlQ1mJ2ghWNuPTUY2BKdGU= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=cmpxchg-org.20221208.gappssmtp.com header.s=20221208 header.b=5hAajZX2; spf=pass (imf18.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.167.180 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org Received: by mail-oi1-f180.google.com with SMTP id 5614622812f47-3a1c162cdfeso1633891b6e.2 for ; Fri, 21 Jul 2023 13:44:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20221208.gappssmtp.com; s=20221208; t=1689972250; x=1690577050; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=tEshZgxJu/wBZMZ7+AtLkJHqS7609jEJDv2p0gloftc=; b=5hAajZX29mBiOPuUXN0gJdiJQTDXp7oyiK0PE9JoornRTaNyZJhhteqbrpVFU4f4bZ y8xH5CKnTiuIM+YS5OgrMUAx3smz9SHvL3Rv1rnrG/LRq6Z2II71YIH/nCzn5j93rMkn yhgVintKzHo7qPI4NNFEw8rtKqkoGY/AhPnczh5A9ORL9epMzWr8anD9S3h/9cNfPTUM Zem1aR1Xb4PsxWQKl54wYJyP63CTDJCfWZRcmPf98zGrIn1LAQp0Dk/qExa5bdvCnve/ 7o+H9RD2hsD+yb/3hwpSOSsg9CGV3bE7ORPPUWBaCqgs0YbD7db5MS1AJ9FT6jy6lLOS ktjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689972250; x=1690577050; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=tEshZgxJu/wBZMZ7+AtLkJHqS7609jEJDv2p0gloftc=; b=XIfo/WJGD9t1M2jjCJyZ1SHclYOVTxpNs1WKvekKm3Q7aR1YaDGPTzybenz8VZ/FVa ifywj0By9tI+5jsjjxQ7oubBqv+WTURPvqHRbZA0XB54fgCAJuS/KPfF9xKJhf6IRkKM Ya6gkwr9Xpupk7e99hI84XXDJACGMTnlyp6xNFX5H4cdqMMs6W+AiafDZYzH/uUK/1zC d4NkaxlAUo3toCZ77yEmNKnXT6uyAEg/JyI5K+rt6nNjRzcfKNZbxZBJisXpEZTPuDru 1mo79XBq7m05yoOcDZmCLKukZmqeF+NRMJZ/07OzEqUO97gM5XnwpE+/WSR8G2qjxZVA lJ7w== X-Gm-Message-State: ABy/qLaBtWUDzfQAVoipeH0XfEOsnX5nFWE2MypkTrqssYGSCPLnQyAB joiPndEb/Kbuohh0wQSUko4u2A== X-Google-Smtp-Source: APBJJlG4GXig7mMHEXbirSicRuSY4RYWbqogZZ+blZKS+lSMdFHQr1pfJPo47Q8YnRrqHa7PZ2cGxQ== X-Received: by 2002:a54:4d06:0:b0:3a3:6ae3:bd9 with SMTP id v6-20020a544d06000000b003a36ae30bd9mr3284589oix.55.1689972250682; Fri, 21 Jul 2023 13:44:10 -0700 (PDT) Received: from localhost (2603-7000-0c01-2716-8f57-5681-ccd3-4a2e.res6.spectrum.com. [2603:7000:c01:2716:8f57:5681:ccd3:4a2e]) by smtp.gmail.com with ESMTPSA id h15-20020a0cab0f000000b0063707f03e2bsm1564056qvb.19.2023.07.21.13.44.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Jul 2023 13:44:10 -0700 (PDT) Date: Fri, 21 Jul 2023 16:44:08 -0400 From: Johannes Weiner To: Yosry Ahmed Cc: Tejun Heo , Andrew Morton , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , "Matthew Wilcox (Oracle)" , Zefan Li , Yu Zhao , Luis Chamberlain , Kees Cook , Iurii Zaikin , "T.J. Mercier" , Greg Thelen , linux-kernel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org Subject: Re: [RFC PATCH 0/8] memory recharging for offline memcgs Message-ID: <20230721204408.GA1033322@cmpxchg.org> References: <20230720070825.992023-1-yosryahmed@google.com> <20230720153515.GA1003248@cmpxchg.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Stat-Signature: tee8gubktacykaogddkobste7dt196wt X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: BB4C31C0008 X-Rspam-User: X-HE-Tag: 1689972251-348933 X-HE-Meta: U2FsdGVkX19mdNF6beOR3I9fUyxHXO+BEAMX1zBD5Y7OsbOBy9cF9nnd0k9bfnlf1mBJ3ddufXSTj/ny2/4Kvlq5wEAiYgVpdtbF+8nc7stfBXU8hz+CEwrcS4M1SaVa+dPVjwxfyo7rghLiTenHPiT+QsqZC6itqPxb6ywks33N2YLY6ykAtS8fTAZSuV3sX7CJSmlYfOeVEC82NhBkAGizOJ980LNvbg+PeMvjRJgG+jz/JipdGw07d090Lvs0CwX/vnDv2fdLhh+f3dTmlsFF36oQHp2P0gnYgRrfOEA6+IS0uZtrYKXLg1UAcwPZ9zjDv+Cw14LOf4DiZXED4dDkhdvpRVn8r45ptDh0R2clhHc9EStuGuDNFcYYQf+tv1KJJ8wdHD5jiGOq+9g8fSVvCsw1quS8CnDVUTBKF4Uk9dWVUsfQ15Ov6p+FhwiGO/FgYr+J/NqS5GsZDPsRF4Ei5xdr+RJ6aEMQqG8cHt+AwNUBqWCcqy4wSnpmQ81nYP/JZLwP3KJ4iB4BnTfNuYTifyXmyn3y6z6zxY1WCKmcFsLCVte0COFNPiGRuv19gRWkdxmS9F6uEJ0EXASWNtjzBJ8/LpiEICe/0vzwBnyhPPzVTuoTACY0Zsm5ut/L2mqBocsfdA/HxFHi7eYtVmGDUOP6fJ5pdm21vA7GVKb25cztSJG/s3I+NLlSG+cBwFSGZVHgWIenMdhBAyagG6PgOs/2HB0Iu2SAjxtCkXs/rUCEFhHAoLXOhwhKXSPgXUYSZa0t4lIqNC5p925D6ZuN+NImsoaECZuEaIY3H/7fA1Vokom66oL9Zenc22jMGbYU25bUxhgXRBBNZ3t6O676SQMt0yHqVQOWh38FzbJsHGyOtSbY/fq4CfjQ8QYYNEvnmc0cBs9upAFjVmXlLZjrNph4Y9GaXptgumQbye+Vh1jZt0ZBsTuPnYZTEwXY2qD8SUt3uC/vK3G5QQD k9URZeYG IMbrbD5FHDuklkDRPhzdDDxyK3J4xBeEURcyfcU1TYyDjRMl2sUHScRGadhrz0a94vOlTvSpPvnhHJSf3i1VMS7PH7hqNNhP9H9OaIM9NXckkYyYsmb7WiZSJIE81RYL7zecTXWeLsiU2zpQrBRWennajDew7RKVZtwWkEDmYjHITAaTvgZzynqTIqX7kIHkpZWJM0Z1GJSog9XaQcw3o3tNdYuaf3J9GhIMpHrgJX3oeIx4LgJnhfnBt4o7RiO22h4g3mRhXECDHP+MbHmvHnuyoXyI1WSpxy8eiIVDvCsCOeHLAzN5wFlHyhFThSgWnIt0w0a0X466f9RuvsdZcGuc53wbOEImoDrzkLgjcLATtF3pjdq3XtGx1kH/6Jyo6j2E5DJCJXhAOqMTlBCDt1u9UOOc8xt1rjKzQYDS0g7jJGuKX8UozcGIfscTRcCOC5BpO9SNYDhw3PgnCbgzzKu+KqWm20XXLTzsNAcSPFmmohTJCi2hwBQVH9g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Jul 21, 2023 at 11:47:49AM -0700, Yosry Ahmed wrote: > On Fri, Jul 21, 2023 at 11:26 AM Tejun Heo wrote: > > > > Hello, > > > > On Fri, Jul 21, 2023 at 11:15:21AM -0700, Yosry Ahmed wrote: > > > On Thu, Jul 20, 2023 at 3:31 PM Tejun Heo wrote: > > > > memory at least in our case. The sharing across them comes down to things > > > > like some common library pages which don't really account for much these > > > > days. > > > > > > Keep in mind that even a single page charged to a memcg and used by > > > another memcg is sufficient to result in a zombie memcg. > > > > I mean, yeah, that's a separate issue or rather a subset which isn't all > > that controversial. That can be deterministically solved by reparenting to > > the parent like how slab is handled. I think the "deterministic" part is > > important here. As you said, even a single page can pin a dying cgroup. > > There are serious flaws with reparenting that I mentioned above. We do > it for kernel memory, but that's because we really have no other > choice. Oftentimes the memory is not reclaimable and we cannot find an > owner for it. This doesn't mean it's the right answer for user memory. > > The semantics are new compared to normal charging (as opposed to > recharging, as I explain below). There is an extra layer of > indirection that we did not (as far as I know) measure the impact of. > Parents end up with pages that they never used and we have no > observability into where it came from. Most importantly, over time > user memory will keep accumulating at the root, reducing the accuracy > and usefulness of accounting, effectively an accounting leak and > reduction of capacity. Memory that is not attributed to any user, aka > system overhead. Reparenting has been the behavior since the first iteration of cgroups in the kernel. The initial implementation would loop over the LRUs and reparent pages synchronously during rmdir. This had some locking issues, so we switched to the current implementation of just leaving the zombie memcg behind but neutralizing its controls. Thanks to Roman's objcg abstraction, we can now go back to the old implementation of directly moving pages up to avoid the zombies. However, these were pure implementation changes. The user-visible semantics never varied: when you delete a cgroup, any leftover resources are subject to control by the remaining parent cgroups. Don't remove control domains if you still need to control resources. But none of this is new or would change in any way! Neutralizing controls of a zombie cgroup results in the same behavior and accounting as linking the pages to the parent cgroup's LRU! The only thing that's new is the zombie cgroups. We can fix that by effectively going back to the earlier implementation, but thanks to objcg without the locking problems. I just wanted to address this, because your description/framing of reparenting strikes me as quite wrong.