From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1B19C10DAA for ; Wed, 29 Nov 2023 16:57:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 62AEC6B03D5; Wed, 29 Nov 2023 11:57:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5DABF6B03D7; Wed, 29 Nov 2023 11:57:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4A34F6B03D9; Wed, 29 Nov 2023 11:57:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 3B0106B03D5 for ; Wed, 29 Nov 2023 11:57:59 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id CC252A05BB for ; Wed, 29 Nov 2023 16:57:58 +0000 (UTC) X-FDA: 81511599036.07.7D226AE Received: from mx1.sberdevices.ru (mx2.sberdevices.ru [45.89.224.132]) by imf29.hostedemail.com (Postfix) with ESMTP id 4003D12000C for ; Wed, 29 Nov 2023 16:57:54 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=salutedevices.com header.s=mail header.b=QyEeJHDy; dmarc=pass (policy=quarantine) header.from=salutedevices.com; spf=pass (imf29.hostedemail.com: domain of ddrokosov@salutedevices.com designates 45.89.224.132 as permitted sender) smtp.mailfrom=ddrokosov@salutedevices.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701277076; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=upFUqpfrDquVThehPQDeK2Tu8Ir/v12aprC4JpKU8g8=; b=ISjtgCDn1bmmPcWK5aGsCbq61kQHu8qKQmJKgdTm+cMUF43ilQcHa68G29TtJoGkeFShWQ KOE7k6Wbqmds/DEQiI+IVAlFg6uoEZmuax0pw/pvnoXlg3jL6JGBXyUofZOIoIIrDaqu7B IQm3gjJ1nE7VTU94iqd+l4ZlJO45SIw= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=salutedevices.com header.s=mail header.b=QyEeJHDy; dmarc=pass (policy=quarantine) header.from=salutedevices.com; spf=pass (imf29.hostedemail.com: domain of ddrokosov@salutedevices.com designates 45.89.224.132 as permitted sender) smtp.mailfrom=ddrokosov@salutedevices.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701277076; a=rsa-sha256; cv=none; b=CAdeo6Feog+VnWdJR7HI1dPTvbxwCEIy7tyN3FLXOI33vRzBGnVAtCMRObJUNQYhbTcH2S 4MBSOElUc1l0/htKn60YgO1gUt+0EV8LgfBIV/kQP+fkaRYXMNIsj16tYhAtF0Svi6p1iE gpjRTV9SuwCpyD27JrBjqIClD8nE00Y= Received: from p-infra-ksmg-sc-msk02 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id D921512000B; Wed, 29 Nov 2023 19:57:52 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru D921512000B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=salutedevices.com; s=mail; t=1701277072; bh=upFUqpfrDquVThehPQDeK2Tu8Ir/v12aprC4JpKU8g8=; h=Date:From:To:Subject:Message-ID:MIME-Version:Content-Type:From; b=QyEeJHDyMXu3Q+rlfWn4C2wnOcfgs5R9+/8V5/w7GE8JbjSqUchhlkwfgYa8X7vt0 QU5j3vJciChTqH9yqdDdTyhBzZhWl2sfX4f3UNBrSLnt0woaXc/i9ISrlj5i7Yk1gu VuB+0bZjvyEowbIpCyi8Ghq8X+I1diR+JfYzzv5Xaz4BUBMehQZbDR6iFPfUwRK2U2 5xYjvU7x/If4jFf7xqkvQ8ROrC8Z61gdDhTskFDJhEt2zuztgu4xto9WND+WOSqwPP qoaRD5b22jczH3m5D0MqzXw8IB196L7wEx/ia6A8QM1OqfOGEMvAjZ065aZOHBEKXd YGaRAbUe7ZaGQ== Received: from p-i-exch-sc-m01.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Wed, 29 Nov 2023 19:57:52 +0300 (MSK) Received: from localhost (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Wed, 29 Nov 2023 19:57:52 +0300 Date: Wed, 29 Nov 2023 19:57:52 +0300 From: Dmitry Rokosov To: Michal Hocko , CC: , , , , , , , , , , , , Subject: Re: [PATCH v3 2/2] mm: memcg: introduce new event to trace shrink_memcg Message-ID: <20231129165752.7r4o3jylbxrj7inb@CAB-WSD-L081021> References: <20231123193937.11628-1-ddrokosov@salutedevices.com> <20231123193937.11628-3-ddrokosov@salutedevices.com> <20231127113644.btg2xrcpjhq4cdgu@CAB-WSD-L081021> <20231127161637.5eqxk7xjhhyr5tj4@CAB-WSD-L081021> <20231129152057.x7fhbcvwtsmkbdpb@CAB-WSD-L081021> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: User-Agent: NeoMutt/20220415 X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m02.sberdevices.ru (172.16.192.103) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 181706 [Nov 29 2023] X-KSMG-AntiSpam-Version: 6.0.0.2 X-KSMG-AntiSpam-Envelope-From: ddrokosov@salutedevices.com X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 5 0.3.5 98d108ddd984cca1d7e65e595eac546a62b0144b, {Track_E25351}, {Tracking_from_domain_doesnt_match_to}, d41d8cd98f00b204e9800998ecf8427e.com:7.1.1;p-i-exch-sc-m01.sberdevices.ru:5.0.1,7.1.1;salutedevices.com:7.1.1;100.64.160.123:7.1.2;127.0.0.199:7.1.2, FromAlignment: s, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/11/29 12:04:00 #22572143 X-KSMG-AntiVirus-Status: Clean, skipped X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 4003D12000C X-Stat-Signature: 4yyz7bo43utyenqgak5y913k3zhamoos X-Rspam-User: X-HE-Tag: 1701277074-839685 X-HE-Meta: U2FsdGVkX18mwSFgZcIO/2OQj+Tq6ghPs/5ufe5PIG156/y8e/p1Z16AnY0BewG7+LXaCFKDhwhHdRD/hiLog10YvWdZs5tLFIVJ44xD8/MbktD2KY2KU8G5XTBpjEQRU+qPqLilqJV3dVU/Whwtfsm3kCeoqV2tIU9my6wXm5ZbxkcDuqDySjjb+KXXs2VxKumQiqz/jCIhHk1hjs77f39jZxP/iLts93WoJqGtXM/UpyPlB1gZPtaRt/BBpnR/R6P92/w+pBL4KFoQih1COxfstYJ1HFLKSEwNy0XxheOEifW0z5rRXUYNLAmlWEjAg/gexzMhMJhA/SEEKb7yP/ZV5E4mz2PUtHUKYwFJzuRRqbbucFfeBH7QWa0d5rSJvXalo+nCl73DWwHb/RyAT/lHIOjdPyitMdv38snucS0KgNr+X1QFmDI2oXQfmcL0GP/3HUVAKcw95dOict6qBXGErGwWRggJann0uSdjLX5CUs6A74+ROCgcgOTIDALRMjUZfh/BObIgmZ82W4KeXvtJYqlkGHCmyw9XRh9kJd5ou3omgMRTft7h09uK5tNivLz1jS2zFQuOAaXQjyP86ICK6+LYwr71P6DLE4cEZDXq38rDP0eTOZd2cd1OrEcZMcyR6OfhHmqr4YBpV4q1cPBz4qnddPL688oyN9zRxqO8pNUC4/QSzfcDjlRGyZJE3TtUniS83DgpNdX5RfTX/fIeAU35Xm7wX8A47ZVBQK7/xMrCc1lhPFFWB+Cax0HUIfEbhHz6g+7dq5FbwK46wzVwt0jfogZfsUndmHSAq1XgRc2GavBQpBJNDtAO8aN6SCEQfSq99itn67gp3yOGoJGgwfXpHpgZ4FhYUhNZDghP4Yzy2IBth9Pl5BnzkIqYCGU6Lu+cCLu6Q9ygVx/S8Rya+HOdVMlDPzAmgST0ZXnf/bBPIeg60H67YVZJXLt+F6+WSF4DvJV4NsgCKp6 4wUiiLjU NjEE90ch0R2I0zMXVMS9jldl9ZccLUNZ3rU2Z+c0AgxQAqMrIajdwXXsFQ+JwcyonhrBKtc3qdkmhCGJySdkTLx1JZCp2u+3AuhfVUH0kPYz2gufNX80wUIls5eE+wHEkxq3KkyieHTIv2mrgebkBueGcjn0Z+YX8PglXTN+d6Yng3ssK7QPZcLFV8ClKkI70SXku3AXm+V9teYpXdifAKGaQf2iJ/HhZP7N2ypyndu6gGE2WibYJmIOkxF6EphXKISjxFHYR2Tr95xtwGDeRSSK7uDlD6JwOZIzoOZAMDot7e/wAm33C6sndNGmIy6ByOV3GJlDUcaOMDmING6/IfYceAOfnEyHB7xN6RKN+UHup5BErFerxPKzCVx1cumoCwdKxJ03BGO/GT4s= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Nov 29, 2023 at 05:06:37PM +0100, Michal Hocko wrote: > On Wed 29-11-23 18:20:57, Dmitry Rokosov wrote: > > On Tue, Nov 28, 2023 at 10:32:50AM +0100, Michal Hocko wrote: > > > On Mon 27-11-23 19:16:37, Dmitry Rokosov wrote: > [...] > > > > 2) With this approach, we will not have the ability to trace a situation > > > > where the kernel is requesting reclaim for a specific memcg, but due to > > > > limits issues, we are unable to run it. > > > > > > I do not follow. Could you be more specific please? > > > > > > > I'm referring to a situation where kswapd() or another kernel mm code > > requests some reclaim pages from memcg, but memcg rejects it due to > > limits checkers. This occurs in the shrink_node_memcgs() function. > > Ohh, you mean reclaim protection > > > === > > mem_cgroup_calculate_protection(target_memcg, memcg); > > > > if (mem_cgroup_below_min(target_memcg, memcg)) { > > /* > > * Hard protection. > > * If there is no reclaimable memory, OOM. > > */ > > continue; > > } else if (mem_cgroup_below_low(target_memcg, memcg)) { > > /* > > * Soft protection. > > * Respect the protection only as long as > > * there is an unprotected supply > > * of reclaimable memory from other cgroups. > > */ > > if (!sc->memcg_low_reclaim) { > > sc->memcg_low_skipped = 1; > > continue; > > } > > memcg_memory_event(memcg, MEMCG_LOW); > > } > > === > > > > With separate shrink begin()/end() tracepoints we can detect such > > problem. > > How? You are only reporting the number of reclaimed pages and no > reclaimed pages could be not just because of low/min limits but > generally because of other reasons. You would need to report also the > number of scanned/isolated pages. > >From my perspective, if memory control group (memcg) protection restrictions occur, we can identify them by the absence of the end() pair of begin(). Other reasons will have both tracepoints raised. > > > > 3) LRU and SLAB shrinkers are too common places to handle memcg-related > > > > tasks. Additionally, memcg can be disabled in the kernel configuration. > > > > > > Right. This could be all hidden in the tracing code. You simply do not > > > print memcg id when the controller is disabled. Or just simply print 0. > > > I do not really see any major problems with that. > > > > > > I would really prefer to focus on that direction rather than adding > > > another begin/end tracepoint which overalaps with existing begin/end > > > traces and provides much more limited information because I would bet we > > > will have somebody complaining that mere nr_reclaimed is not sufficient. > > > > Okay, I will try to prepare a new patch version with memcg printing from > > lruvec and slab tracepoints. > > > > Then Andrew should drop the previous patchsets, I suppose. Please advise > > on the correct workflow steps here. > > Andrew usually just drops the patch from his tree and it will disappaer > from the linux-next as well. Okay, I understand, thank you! Andrew, could you please take a look? I am planning to prepare a new patch version based on Michal's suggestion, so previous one should be dropped. -- Thank you, Dmitry