From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C2F5C43334 for ; Tue, 19 Jul 2022 11:30:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7D89C6B0071; Tue, 19 Jul 2022 07:30:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7882C6B0073; Tue, 19 Jul 2022 07:30:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 64FED6B0074; Tue, 19 Jul 2022 07:30:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 526A06B0071 for ; Tue, 19 Jul 2022 07:30:55 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 0DD8B115F for ; Tue, 19 Jul 2022 11:30:55 +0000 (UTC) X-FDA: 79703632470.03.07C1FBC Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf17.hostedemail.com (Postfix) with ESMTP id 4D7CF40091 for ; Tue, 19 Jul 2022 11:30:53 +0000 (UTC) Received: from relay2.suse.de (relay2.suse.de [149.44.160.134]) by smtp-out1.suse.de (Postfix) with ESMTP id D0D3A3510A; Tue, 19 Jul 2022 11:30:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1658230251; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=gEeCCLSta3HcRHaPk6iGTZCTxTMRWXyTHqEYQ05Ux9w=; b=CVeCxGYYUGLBzabGk0Qenp6GKA/EIm671QkK9E8AKYbh4yLJY9CWEkKLyii4XtALul/0JP Z1qQmhICaixrcdJzX727B+/FUBmszeOirlXsIWPYhbiiltq6C2XA2FB+LqaqjaoTC/YfrD iy8wPkjA0tjzeXUDx0MateWjxmj73vI= Received: from suse.cz (unknown [10.100.201.86]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by relay2.suse.de (Postfix) with ESMTPS id 81FEA2C141; Tue, 19 Jul 2022 11:30:50 +0000 (UTC) Date: Tue, 19 Jul 2022 13:30:49 +0200 From: Michal Hocko To: Yosry Ahmed Cc: Roman Gushchin , Yafang Shao , Alexei Starovoitov , Shakeel Butt , Matthew Wilcox , Christoph Hellwig , "David S. Miller" , Daniel Borkmann , Andrii Nakryiko , Tejun Heo , Martin KaFai Lau , bpf , Kernel Team , linux-mm , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Andrew Morton , Vlastimil Babka Subject: cgroup specific sticky resources (was: Re: [PATCH bpf-next 0/5] bpf: BPF specific memory allocator.) Message-ID: References: <20220708215536.pqclxdqvtrfll2y4@google.com> <20220710073213.bkkdweiqrlnr35sv@google.com> <20220712043914.pxmbm7vockuvpmmh@macbook-pro-3.dhcp.thefacebook.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1658230253; a=rsa-sha256; cv=none; b=nY6S3blrwVv2IrtT6oIS3Pau0VnZZxa7ZKY75OUKAO6mxBfjG4pgomhu3AOuWKtfZY93kh lMG5Fln6Gi8KIrF2VcIV1fV50SJRMjmuaG33iJMCuYZQoVfkss4SMKH75qEOcYFMexnJxe E1yfbqUb/5+Pk3Za7fNQsH5mDXm8ek8= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=CVeCxGYY; spf=pass (imf17.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1658230253; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=gEeCCLSta3HcRHaPk6iGTZCTxTMRWXyTHqEYQ05Ux9w=; b=rFIFiuNpAF5RQn0OfQYPyM0E4uTC/Ui+JdOLEDDZqnQbJ8kj74OjODV2CdirLUuX4FwX1E MxzQeMDycS8k+ZEbGUnEv8rePRtC2VnDnszCYlf3p38z7bVBarj3CJQ+8lJGFGnrxMoytE +SOysnWv2N2Xokm7pIonR3IMmUQ1rGw= X-Stat-Signature: 4uepz8mi56oj7rso1ppxqpsz7o3w41mi X-Rspamd-Queue-Id: 4D7CF40091 Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=CVeCxGYY; spf=pass (imf17.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1658230253-83349 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon 18-07-22 10:55:59, Yosry Ahmed wrote: > On Tue, Jul 12, 2022 at 7:39 PM Roman Gushchin wrote: > > > > On Tue, Jul 12, 2022 at 11:52:11AM +0200, Michal Hocko wrote: > > > On Tue 12-07-22 16:39:48, Yafang Shao wrote: > > > > On Tue, Jul 12, 2022 at 3:40 PM Michal Hocko wrote: > > > [...] > > > > > > Roman already sent reparenting fix: > > > > > > https://patchwork.kernel.org/project/netdevbpf/patch/20220711162827.184743-1-roman.gushchin@linux.dev/ > > > > > > > > > > Reparenting is nice but not a silver bullet. Consider a shallow > > > > > hierarchy where the charging happens in the first level under the root > > > > > memcg. Reparenting to the root is just pushing everything under the > > > > > system resources category. > > > > > > > > > > > > > Agreed. That's why I don't like reparenting. > > > > Reparenting just reparent the charged pages and then redirect the new > > > > charge, but can't reparents the 'limit' of the original memcg. > > > > So it is a risk if the original memcg is still being charged. We have > > > > to forbid the destruction of the original memcg. > > > > I agree, I also don't like reparenting for !kmem case. For kmem (and *maybe* > > bpf maps is an exception), I don't think there is a better choice. > > > > > yes, I was toying with an idea like that. I guess we really want a > > > measure to keep cgroups around if they are bound to a resource which is > > > sticky itself. I am not sure how many other resources like BPF (aka > > > module like) we already do charge for memcg but considering the > > > potential memory consumption just reparenting will not help in general > > > case I am afraid. > > > > Well, then we have to make these objects a first-class citizens in cgroup API, > > like processes. E.g. introduce cgroup.bpf.maps, cgroup.mounts.tmpfs etc. > > I easily can see some value here, but it's a big API change. > > > > With the current approach when a bpf map pins a memory cgroup of the creator > > process (which I think is completely transparent for most bpf users), I don't > > think preventing the deletion of a such cgroup is possible. It will break too > > many things. > > > > But honestly I don't see why userspace can't handle it. If there is a cgroup which > > contains shared bpf maps, why would it delete it? It's a weird use case, I don't > > think we have to optimize for it. Also, we do a ton of optimizations for live > > cgroups (e.g. css refcounting being percpu) which are not working for a deleted > > cgroup. So noone really should expect any properties from dying cgroups. > > > > Just a random thought here, and I can easily be wrong (and this can > easily be the wrong thread for this), but if we introduce a more > generic concept to generally tie a resource explicitly to a cgroup > (tmpfs, bpf maps, etc) using cgroupfs interfaces, and then prevent the > cgroup from being deleted unless the resource is freed or moved to a > different cgroup? My understanding is that Tejun would prefer a user space defined policy by a proper layering. And I would tend to agree that this is less prone to corner cases. Anyway, how would you envision such an interface? > This would be optional, so the current status quo is maintainable, but > also gives flexibility to admins to assign resources to cgroups to > make sure nothing is ( unaccounted / accounted to a zombie memcg / > reparented to an unrelated parent ). This might be too fine-grained to > be practical but I just thought it might be useful. We will also need > to define an OOM behavior for such resources. Things like bpf maps > will be unreclaimable, but tmpfs memory can be swapped out. Keep in mind that the swap is a shared resource in itself. So tmpfs is essentially a sticky resource as well. A tmpfs file is not bound to any proces life time the same way BPF program is. You might need less priviledges to remove a file but in principle they are consuming resources without any explicit owner. -- Michal Hocko SUSE Labs