From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 174ECC54E94 for ; Wed, 25 Jan 2023 23:15:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 21CF86B0071; Wed, 25 Jan 2023 18:15:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1CD426B0073; Wed, 25 Jan 2023 18:15:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0BC326B0074; Wed, 25 Jan 2023 18:15:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id F38276B0071 for ; Wed, 25 Jan 2023 18:15:08 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id AD4F3AB52F for ; Wed, 25 Jan 2023 23:15:08 +0000 (UTC) X-FDA: 80394879096.04.78A612A Received: from out-253.mta0.migadu.com (out-253.mta0.migadu.com [91.218.175.253]) by imf15.hostedemail.com (Postfix) with ESMTP id 5728CA0022 for ; Wed, 25 Jan 2023 23:15:05 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=L++sVVFK; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf15.hostedemail.com: domain of roman.gushchin@linux.dev designates 91.218.175.253 as permitted sender) smtp.mailfrom=roman.gushchin@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674688506; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GQas9hJPH93Zr57s1dcc7T+g336s2ZkXNkxopiNZy5Y=; b=o/rbTaJQ76EeAppRtcYRb6FobySZs3ccYRmP+/hJkoDcjXzE/WixBpNutVw6mbbguPM/vY WqJUwRMn0LNPbL0DcuFiQiifOYEXX+/sQpLfGFjBFlfTa7l1FS6YLEuJJSbN3hPzxA/qct eW+0haKu/gtcinoO48bOlqfcZGbEprM= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=L++sVVFK; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf15.hostedemail.com: domain of roman.gushchin@linux.dev designates 91.218.175.253 as permitted sender) smtp.mailfrom=roman.gushchin@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674688506; a=rsa-sha256; cv=none; b=4FIAC0fQpvQukA96FmbpbqKuCNC0wBfBEdvGA1RB5GkebFmEPQrh12Zu55Ue5lQNv/cyOf 7xlipH7nHKEX3jD07ELfq6M2NYosGa7OQps3EULNc2/JU6NAqZlUwSG3zm/h86dToSHI45 vI/RQKbiXDouPqAa2tttO0SUr+7QRno= Date: Wed, 25 Jan 2023 15:14:48 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1674688503; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GQas9hJPH93Zr57s1dcc7T+g336s2ZkXNkxopiNZy5Y=; b=L++sVVFKgB2QgSfUln8N4pW6K99/uC71K7MHO0eTNd2gOjxQp1AktmI+V60pLnLZKq6KFC hACk9uz3DsoIitBsOXZt7BIu60/uMI/JLjE4PEEUb0PdbUDFsVn25UofpY3G6aCVyuS6ma 52wkIVgDt+qNHn0YlWdzI9qQ/UE4CgA= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Roman Gushchin To: Marcelo Tosatti Cc: Leonardo =?iso-8859-1?Q?Br=E1s?= , Michal Hocko , Johannes Weiner , Shakeel Butt , Muchun Song , Andrew Morton , cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH v2 0/5] Introduce memcg_stock_pcp remote draining Message-ID: References: <20230125073502.743446-1-leobras@redhat.com> <9e61ab53e1419a144f774b95230b789244895424.camel@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Migadu-Flow: FLOW_OUT X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 5728CA0022 X-Stat-Signature: 93q8x1qa9pn34g6tssdsf15zdq9wbwhg X-HE-Tag: 1674688505-780123 X-HE-Meta: U2FsdGVkX18BPO8Ndf4UVkFP1HMyKmk0NcyHMy8PxJ9XrJLzrO3cSRxt0iO/rUu7TXWMkjLy6XXdLsUI2JSV7H4Gwv8lc3sFnmdzJ0TfAIliiCYrlIeqtjiaNeqnY30eq7i6CLGQ2ZHRktzfNpscEfpkL/nc0wOQgSLOtdVvzFsLzqvplSs95d+ybMp5ZAg3ZWveiH/8b6aD1m1Q6r9TFkvZtdvFOwpzoONPfr5nqWzwrAUP1zoLsciMVLPDwVBy/iDso8q1v7palefTMrpLchB7kKbY+vHMmupOEze34Q/dtemDlC/pqDXJm4q17zK6A5aIXncxdi4RNtbFaMeWnJ0dKltLRDC5zVfoUBctmhNVKUOJDLbWKut3/vw7KdOsk57JvFiIoYvz3It54dWTsk2K9lXALI/iqQhhyqS86Zvms+I1stxaP4zCNiVrlDGGb5pgZxcZtLCLbAQmX3QX+tnbeCk7PXi/authT4amWck+5/7Mu9vVPTJU/gv/w9RDQAop91s3l9gdA1s18fUIZqfSSrt49uKwGmpMBMc5dfQatg8rMgQeTaa71nvxFcwM6lXFr9JX+lL0iAoUma2UI4iviVYVWowvAEotm5A0DAlNkCEtoFUUT8pLYhNte7Gm0POMJbFq1nK3w0DzRWm8IdYlqj47nSD3mpmOqHNnhNJOne5sfOQRqpUiZVKFJWAASUTMQnxXpjxQXAeTz0gwF3tH9o3GFm2oTBbA1/biiBmtytPoESZtZJpX/HVpI/D6Tkf0RDeBmdrunYVXA7E/Z3MvuJuxWzOdmJZw3MuLdcZvIox/u5FOB+2adBkiZ2V3LIwZsI+L1tkAu4sv1fBTMm5e3+fysGXY7zf1OmxK21lky2vXiKDCOz0McLkE0q472Q/XszIOxHlH2Pky7LsIaCW5+Aa7Lh+z6wAqMoga14DEqHPA1CFmQ7WNap571bR4ElR1Npo6EUS7ggs6Aym wqDWaaGp VOJMXOglEaVtgYQIanylQUw4qBrRHrFnREYTH544PJN59pfGQOt1LqVArCh2UjXg3IHxm2kLt8y46rST5WYuRKOMlN57SgJEX1W8anI7YdSp5jv6uNaHCZlxtJ1EKwUwwpWPd/976gb5QrFCqNLN0TZ9OHZwKUhvOeyKgRtC9GxKOyjWZzFxOjx+ArnqjyYQmtCey X-Bogosity: Ham, tests=bogofilter, spamicity=0.000485, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Jan 25, 2023 at 03:22:00PM -0300, Marcelo Tosatti wrote: > On Wed, Jan 25, 2023 at 08:06:46AM -0300, Leonardo Brás wrote: > > On Wed, 2023-01-25 at 09:33 +0100, Michal Hocko wrote: > > > On Wed 25-01-23 04:34:57, Leonardo Bras wrote: > > > > Disclaimer: > > > > a - The cover letter got bigger than expected, so I had to split it in > > > > sections to better organize myself. I am not very confortable with it. > > > > b - Performance numbers below did not include patch 5/5 (Remove flags > > > > from memcg_stock_pcp), which could further improve performance for > > > > drain_all_stock(), but I could only notice the optimization at the > > > > last minute. > > > > > > > > > > > > 0 - Motivation: > > > > On current codebase, when drain_all_stock() is ran, it will schedule a > > > > drain_local_stock() for each cpu that has a percpu stock associated with a > > > > descendant of a given root_memcg. Do you know what caused those drain_all_stock() calls? I wonder if we should look into why we have many of them and whether we really need them? It's either some user's actions (e.g. reducing memory.max), either some memcg is entering pre-oom conditions. In the latter case a lot of drain calls can be scheduled without a good reason (assuming the cgroup contain multiple tasks running on multiple cpus). Essentially each cpu will try to grab the remains of the memory quota and move it locally. I wonder in such circumstances if we need to disable the pcp-caching on per-cgroup basis. Generally speaking, draining of pcpu stocks is useful only if an idle cpu is holding some charges/memcg references (it might be not completely idle, but running some very special workload which is not doing any kernel allocations or a process belonging to the root memcg). In all other cases pcpu stock will be either drained naturally by an allocation from another memcg or an allocation from the same memcg will "restore" it, making draining useless. We also can into drain_all_pages() opportunistically, without waiting for the result. On a busy system it's most likely useless, we might oom before scheduled works will be executed. I admit I planned to do some work around and even started, but then never had enough time to finish it. Overall I'm somewhat resistant to an idea of making generic allocation & free paths slower for an improvement of stock draining. It's not a strong objection, but IMO we should avoid doing this without a really strong reason. Thanks!