From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E055C4167B for ; Wed, 29 Nov 2023 17:35:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 113386B03E0; Wed, 29 Nov 2023 12:35:47 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0C4AC6B03E1; Wed, 29 Nov 2023 12:35:47 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EA73A6B03E2; Wed, 29 Nov 2023 12:35:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id DC03D6B03E0 for ; Wed, 29 Nov 2023 12:35:46 -0500 (EST) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id A88FB1205CB for ; Wed, 29 Nov 2023 17:35:46 +0000 (UTC) X-FDA: 81511694292.23.479B515 Received: from mx1.sberdevices.ru (mx1.sberdevices.ru [37.18.73.165]) by imf14.hostedemail.com (Postfix) with ESMTP id 5EBC2100029 for ; Wed, 29 Nov 2023 17:35:43 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=salutedevices.com header.s=mail header.b=UGlogolA; spf=pass (imf14.hostedemail.com: domain of ddrokosov@salutedevices.com designates 37.18.73.165 as permitted sender) smtp.mailfrom=ddrokosov@salutedevices.com; dmarc=pass (policy=quarantine) header.from=salutedevices.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701279344; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=o5aURXIRpu1BR0yiHfEtpAHoFraXQHCwoLT8rRosZhw=; b=cpqwS5e3MVkCin/Ic+S/v9uRpb1gH/tgHEz1t+5I8V/Nw6kVtBhnDkPDWYpEqEbTKbvWqi dD8I53ZuNm5nc+7ME3nFkuEi0uL6F+UxHkv5Jr9Vz1hFyrXWbpmjv7k3VMgdm4igpHGSiq Hs/BesSQ8++PpkQYSUvjGOj2gRkDmGI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701279344; a=rsa-sha256; cv=none; b=OQzndXAawVbV9paLTdgDevTWkLtNPK7QQ0L/Eq87bqz3NGu4nS+mazAkiw4ft4XSCzBSqL 3aYy7UJT2zapEA/8HuVgvFOAOhMM0xnCYwKm3lbZP6ro820/Ao5WyC5TMMl9seFATOgd92 0Id0GqsMkD2YFm5xfDFKrB/dO1QyE7s= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=salutedevices.com header.s=mail header.b=UGlogolA; spf=pass (imf14.hostedemail.com: domain of ddrokosov@salutedevices.com designates 37.18.73.165 as permitted sender) smtp.mailfrom=ddrokosov@salutedevices.com; dmarc=pass (policy=quarantine) header.from=salutedevices.com Received: from p-infra-ksmg-sc-msk01 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id 191CD100019; Wed, 29 Nov 2023 20:35:41 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru 191CD100019 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=salutedevices.com; s=mail; t=1701279341; bh=o5aURXIRpu1BR0yiHfEtpAHoFraXQHCwoLT8rRosZhw=; h=Date:From:To:Subject:Message-ID:MIME-Version:Content-Type:From; b=UGlogolADou1QOUgsNC1t11B9ahIDxCTpBFfbrJFu2a1er/NYLPbtkI50EXZLJ1M9 r/lQbeIEtZYstKx8E5vxCHxnVqEBcEbjYRR+rWnXdk8MP33+d7oStH6WVTDSn0WKH9 hX+Twj7gK6pyEQXwafxmFCPiHn+Wuw1upudk64ZeUBXFs60pHabdDaz2h2TezVO1xE ShoLtETH7VYwM3DnM414Qpi1T6JE/VVoVlc2iA8dgZoVbu17DYc8Aooi38yDDx8LKX Ecztmf+OOB4I4nJiXYXlVGfF7N0uoru7CiVSdYo3hyNUEY7POAFPQQbQAgrk0hzLeq egy/qE4FxnC8g== Received: from p-i-exch-sc-m01.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Wed, 29 Nov 2023 20:35:40 +0300 (MSK) Received: from localhost (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Wed, 29 Nov 2023 20:35:40 +0300 Date: Wed, 29 Nov 2023 20:35:40 +0300 From: Dmitry Rokosov To: Michal Hocko CC: , , , , , , , , , , , , Subject: Re: [PATCH v3 2/2] mm: memcg: introduce new event to trace shrink_memcg Message-ID: <20231129173540.gl2pufeo6ciubcny@CAB-WSD-L081021> References: <20231123193937.11628-3-ddrokosov@salutedevices.com> <20231127113644.btg2xrcpjhq4cdgu@CAB-WSD-L081021> <20231127161637.5eqxk7xjhhyr5tj4@CAB-WSD-L081021> <20231129152057.x7fhbcvwtsmkbdpb@CAB-WSD-L081021> <20231129165752.7r4o3jylbxrj7inb@CAB-WSD-L081021> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: User-Agent: NeoMutt/20220415 X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 181708 [Nov 29 2023] X-KSMG-AntiSpam-Version: 6.0.0.2 X-KSMG-AntiSpam-Envelope-From: ddrokosov@salutedevices.com X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 5 0.3.5 98d108ddd984cca1d7e65e595eac546a62b0144b, {Track_E25351}, {Tracking_from_domain_doesnt_match_to}, d41d8cd98f00b204e9800998ecf8427e.com:7.1.1;100.64.160.123:7.1.2;127.0.0.199:7.1.2;p-i-exch-sc-m01.sberdevices.ru:5.0.1,7.1.1;salutedevices.com:7.1.1, FromAlignment: s, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/11/29 16:31:00 #22572963 X-KSMG-AntiVirus-Status: Clean, skipped X-Rspamd-Queue-Id: 5EBC2100029 X-Rspam-User: X-Stat-Signature: y9dhsqrgmsn1qnk8m9jiyj4q8dtszd4n X-Rspamd-Server: rspam03 X-HE-Tag: 1701279343-708744 X-HE-Meta: U2FsdGVkX1/+OThU/t+BQR3Wbr2gYeLK5OZoLDDTcLrrVvPmc3VmiwhYrCyfPHT0R9BGlOPA6fppn/5P3pSPhsmuZrb6H9UTWDqqDVpO3/RTgF7szzdveFdXAV/+mNPVRt3UQDorgNle3cCwK5T9i2DtIY4/5A7/hnkvNoaJmFysOUv+EqwUwzSfhb4T8wqBNoaBWjhX8SZopzB5m91J8+xmsg6+8G6gxRx8kxhOrJMKBO69FTqZQjMDzAjcD3F2Q4c+ezSPh0/zm/rf/BVrpV+J14dtt1gSmSo2uXqoKpVpw+xWY80g7CZCj2lM2WacjbKZ380Lkr/ZdB+YcSn7hCo54Tma13BIIUfsZCxuy6ZpG9KEF2iFWTwX9Qksq1CFrQkzDgetzqwVZk91cZlfwJTdyilGk6gf+ZU3xpSy5YnsrGFr1txNwVB2ISlpK7StQIbPW+kxKT8usOE+2qPjbW8WybrxxIaTjJT9dsXn0zZPUsBNoC62juWffNMHSGbTYwJJD1Qbrzbv8xWxyFQVf6CzpQ5W4DCeU1YjSpzAo31auswpfnjQF1E0J0a+GpTwKFy51vCUiHNgljMULecQ8XVh+BlaDd2m2eJ40kZC4vPfBSaK5Zh/s8Z++3Al21RhP8bWVkR3UDYA6SxBx2LRI5+0CX4wpihTxAFxx5A7QmmRWzdRPy1KqPIwNSwlB8Vc3Sz9K/b4bXkPh7VB/xoZog1HSWu7ja+xsqKOjG9bFJ54TGI2khflGlCCGOJ+AW7LZneBFuGe4bfAcaV7FjTiqOc7diwFk3g4z1g/kFj9DEuR8bvkhzAaQ99PdY+R3XV1K9Cc9Va+5uCuDbkk9FCgsrOzcfyxNYF5rneKQZYku4vcaa98eiSRsVoNHYd3dQziSn/CQqObrj5VI/lugPH7eTwB1/tmNW0sm+XhskeLYzSaiZ4IS5cdoSiYf1MpFhZAqgISFgvaaRF8WwzSyfm bXN6YVad wxZIKyPYCifRYri7VLeN78AtgJTxiMyfjG8PB7mf/WYP1gbpLFnxZ9aqff+GnwbLb4qX1QRCpG9FfSIBQrmpLWVgeqU+2wH8B1X9/8rzMxTpS3ChLOMPTJ2LrJLJ3IfvtXpvD54H5LQINqGzg6wQ7gvTqOqZrocWa3yo7Xz3FX+TB/CaN3LmikNAdli0K+0JtzXYEsxps+/SD/ucnQRMoyd6JJnP2tMLBVZ4GFKo9m11+AlM6n5aSj8cdf1kDy3C5+ot/iYmhutBjKjFL4K1BQ/IRGzYRtkkWvNKCh9JkHuqKeX56g+XEypEQF9equPx8hi9syge7XlPghnO5pggqRMWQAVxe6Zmy/IDiS7eRHYGA/ITa8ZyZKh0LG4PU4RPpgC28/r0bywoX660= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000033, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Nov 29, 2023 at 06:10:33PM +0100, Michal Hocko wrote: > On Wed 29-11-23 19:57:52, Dmitry Rokosov wrote: > > On Wed, Nov 29, 2023 at 05:06:37PM +0100, Michal Hocko wrote: > > > On Wed 29-11-23 18:20:57, Dmitry Rokosov wrote: > > > > On Tue, Nov 28, 2023 at 10:32:50AM +0100, Michal Hocko wrote: > > > > > On Mon 27-11-23 19:16:37, Dmitry Rokosov wrote: > > > [...] > > > > > > 2) With this approach, we will not have the ability to trace a situation > > > > > > where the kernel is requesting reclaim for a specific memcg, but due to > > > > > > limits issues, we are unable to run it. > > > > > > > > > > I do not follow. Could you be more specific please? > > > > > > > > > > > > > I'm referring to a situation where kswapd() or another kernel mm code > > > > requests some reclaim pages from memcg, but memcg rejects it due to > > > > limits checkers. This occurs in the shrink_node_memcgs() function. > > > > > > Ohh, you mean reclaim protection > > > > > > > === > > > > mem_cgroup_calculate_protection(target_memcg, memcg); > > > > > > > > if (mem_cgroup_below_min(target_memcg, memcg)) { > > > > /* > > > > * Hard protection. > > > > * If there is no reclaimable memory, OOM. > > > > */ > > > > continue; > > > > } else if (mem_cgroup_below_low(target_memcg, memcg)) { > > > > /* > > > > * Soft protection. > > > > * Respect the protection only as long as > > > > * there is an unprotected supply > > > > * of reclaimable memory from other cgroups. > > > > */ > > > > if (!sc->memcg_low_reclaim) { > > > > sc->memcg_low_skipped = 1; > > > > continue; > > > > } > > > > memcg_memory_event(memcg, MEMCG_LOW); > > > > } > > > > === > > > > > > > > With separate shrink begin()/end() tracepoints we can detect such > > > > problem. > > > > > > How? You are only reporting the number of reclaimed pages and no > > > reclaimed pages could be not just because of low/min limits but > > > generally because of other reasons. You would need to report also the > > > number of scanned/isolated pages. > > > > > > > From my perspective, if memory control group (memcg) protection > > restrictions occur, we can identify them by the absence of the end() > > pair of begin(). Other reasons will have both tracepoints raised. > > That is not really great way to detect that TBH. Trace events could be > lost and then you simply do not know what has happened. I see, thank you very much for the detailed review! I will prepare a new patchset with memcg names in the lruvec and slab paths, will back soon. -- Thank you, Dmitry