From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F263C433EF for ; Tue, 1 Mar 2022 18:28:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 12B0E8D0002; Tue, 1 Mar 2022 13:28:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0D98A8D0001; Tue, 1 Mar 2022 13:28:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EBD0D8D0002; Tue, 1 Mar 2022 13:28:42 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.a.hostedemail.com [64.99.140.24]) by kanga.kvack.org (Postfix) with ESMTP id DA19D8D0001 for ; Tue, 1 Mar 2022 13:28:42 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id 9738F61B05 for ; Tue, 1 Mar 2022 18:28:42 +0000 (UTC) X-FDA: 79196653284.05.0B71775 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf17.hostedemail.com (Postfix) with ESMTP id 7C68940005 for ; Tue, 1 Mar 2022 18:28:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=4qQmMjIAiTQvnNnaC1WHCchcXF9E9dDy4G6npTea56M=; b=0NJSVp1GVv2yfmwiqsdwZ5jiXj RrtALBzQC6KEMXVBXGQozfl49Ir1/XaN60DX6Qh1+8DnpoSnKsBRdbr2yhsnk0dloMgw/viPciUI2 +wdAAUwjwKKQA7aowWnivpD7Y49RzNtoT+oM/H/ClT/rgcVYrASHNoK7X6lEcms8+IQcbSIDxJ9fA YM0cnyFiaFxkF+MIsXpnjF2hIRfReWQTXUpZtFVwVhA9+LQhlnsel1S74Hyw8rCeVot2xGLlPYTuI FO2bY3CNIdvaI0H/8MBNP1fM1b3HOxTNh2VhbRuf2jqKB/m4alTTlCpMGY61KePrTEMinzrNxjRbK WfCOGLEQ==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1nP7EY-000C19-Vb; Tue, 01 Mar 2022 18:28:26 +0000 Date: Tue, 1 Mar 2022 10:28:26 -0800 From: Luis Chamberlain To: Shakeel Butt , Colin Ian King , NeilBrown , "Eric W. Biederman" Cc: Vasily Averin , Vlastimil Babka , Michal Hocko , Roman Gushchin , Linux MM , netdev@vger.kernel.org, "David S. Miller" , Jakub Kicinski , Tejun Heo , Greg Kroah-Hartman , Eric Dumazet , Kees Cook , Hideaki YOSHIFUJI , David Ahern , linux-kernel@vger.kernel.org, kernel@openvz.org Subject: Re: [PATCH RFC] net: memcg accounting for veth devices Message-ID: References: <20220301180917.tkibx7zpcz2faoxy@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20220301180917.tkibx7zpcz2faoxy@google.com> X-Rspamd-Queue-Id: 7C68940005 X-Stat-Signature: 6354xj93chrady7uat5ggt68kqchsyix Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=0NJSVp1G; spf=none (imf17.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=none) X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1646159321-868771 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Mar 01, 2022 at 10:09:17AM -0800, Shakeel Butt wrote: > On Mon, Feb 28, 2022 at 06:36:58AM -0800, Luis Chamberlain wrote: > > On Mon, Feb 28, 2022 at 10:17:16AM +0300, Vasily Averin wrote: > > > Following one-liner running inside memcg-limited container consumes > > > huge number of host memory and can trigger global OOM. > > > > > > for i in `seq 1 xxx` ; do ip l a v$i type veth peer name vp$i ; done > > > > > > Patch accounts most part of these allocations and can protect host. > > > ---[cut]--- > > > It is not polished, and perhaps should be splitted. > > > obviously it affects other kind of netdevices too. > > > Unfortunately I'm not sure that I will have enough time to handle it > > properly > > > and decided to publish current patch version as is. > > > OpenVz workaround it by using per-container limit for number of > > > available netdevices, but upstream does not have any kind of > > > per-container configuration. > > > ------ > > > Should this just be a new ucount limit on kernel/ucount.c and have veth > > use something like inc_ucount(current_user_ns(), current_euid(), > > UCOUNT_VETH)? > > > This might be abusing ucounts though, not sure, Eric? > > > For admins of systems running multiple workloads, there is no easy way > to set such limits for each workload. That's why defaults would exist. Today's ulimits IMHO are insane and some are arbitrarily large. > Some may genuinely need more veth > than others. So why not make it a high sensible but not enough to OOM a typical system? But again, I'd like to hear whether or not a ulimit for veth is a mi-use of ulimits or if its the right place. If it's not then perhaps the driver can just have its own atomic max definition. > From admin's perspective it is preferred to have minimal > knobs to set and if these objects are charged to memcg then the memcg > limits would limit them. There was similar situation for inotify > instances where fs sysctl inotify/max_user_instances already limits the > inotify instances but we memcg charged them to not worry about setting > such limits. See ac7b79fd190b ("inotify, memcg: account inotify > instances to kmemcg"). Yes but we want sensible defaults out of the box. What those should be IMHO might be work which needs to be figured out well. IMHO today's ulimits are a bit over the top today. This is off slightly off topic but for instance play with: git clone https://github.com/ColinIanKing/stress-ng cd stress-ng make -j 8 echo 0 > /proc/sys/vm/oom_dump_tasks i=1; while true; do echo "RUNNING TEST $i"; ./stress-ng --unshare 8192 --unshare-ops 10000; sleep 1; let i=$i+1; done If you see: [ 217.798124] cgroup: fork rejected by pids controller in /user.slice/user-1000.slice/session-1.scope Edit /usr/lib/systemd/system/user-.slice.d/10-defaults.conf to be: [Slice] TasksMax=MAX_TASKS|infinity Even though we have max_threads set to 61343, things ulimits have a different limit set, and what this means is the above can end up easily creating over 1048576 (17 times max_threads) threads all eagerly doing nothing to just exit, essentially allowing a sort of fork bomb on exit. Your system may or not fall to its knees. Luis