From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3468C4167B for ; Mon, 27 Nov 2023 16:16:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2D8276B02EC; Mon, 27 Nov 2023 11:16:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 260866B02EE; Mon, 27 Nov 2023 11:16:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0D97D6B02EF; Mon, 27 Nov 2023 11:16:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id E90EF6B02EC for ; Mon, 27 Nov 2023 11:16:45 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id BC365160451 for ; Mon, 27 Nov 2023 16:16:45 +0000 (UTC) X-FDA: 81504237570.10.02EF0E9 Received: from mx1.sberdevices.ru (mx1.sberdevices.ru [37.18.73.165]) by imf01.hostedemail.com (Postfix) with ESMTP id 5FCB34003C for ; Mon, 27 Nov 2023 16:16:39 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=salutedevices.com header.s=mail header.b=YaLSKgFh; spf=pass (imf01.hostedemail.com: domain of ddrokosov@salutedevices.com designates 37.18.73.165 as permitted sender) smtp.mailfrom=ddrokosov@salutedevices.com; dmarc=pass (policy=quarantine) header.from=salutedevices.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701101801; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hdl5H6z9ObTOtv6adXMqE2wmW+A7MydWPmmY48GMjKg=; b=D1eGZ2iWh+8la6yuKGB2SzVHzigdXIto+DQeWfZvtifaVZ3KjNXjSoHIRujCtID6iTpzPK SZ8K1yZuEKKtUczq2cHJexwY2Ad8Z4/Ppl0l9YC24QNDMX5UNGCvcE9bDa07dV29k1aWI8 Y0d/0rLY6cyOzmEifRW1MVzK+tvK1zE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701101801; a=rsa-sha256; cv=none; b=CmLwGE+CnGF6St/Ho1Plrrk2jSiqGWUdNFcQBYG46OYuN6G4KYN79YY2c3vvt/bsrbqGOZ y3HA/vT7D7PZ9taapkbDgzhvYC0+k/SYx2JK1LcrzTXmkJZyPkr/8Qb0O5LuldG8Wbu506 GdytNOIsdUyyuS8JXDWRRxo7Caa3KWs= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=salutedevices.com header.s=mail header.b=YaLSKgFh; spf=pass (imf01.hostedemail.com: domain of ddrokosov@salutedevices.com designates 37.18.73.165 as permitted sender) smtp.mailfrom=ddrokosov@salutedevices.com; dmarc=pass (policy=quarantine) header.from=salutedevices.com Received: from p-infra-ksmg-sc-msk01 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id D178A100014; Mon, 27 Nov 2023 19:16:37 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru D178A100014 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=salutedevices.com; s=mail; t=1701101797; bh=hdl5H6z9ObTOtv6adXMqE2wmW+A7MydWPmmY48GMjKg=; h=Date:From:To:Subject:Message-ID:MIME-Version:Content-Type:From; b=YaLSKgFh70HTTDN8NeRo+4ZcBTysjy16nbLfVWs6hStfEBf3QgnLeyE+gx4k8YczG 3iS+TzD1QVw+N9w4M6AdABXgwCSowtWY28R40pSLfMCieeNzAOWVnFu8xHP2GF3ne/ 4g48OhJa0r3kU0zvzr2DaFBtKRYO7LXE2Wo8+ncRcYcIrJg1xmi9z2+WknQDXmuJMB v6GnDGqQBCm1NLT8eXmE58kJlZSP9WL3HkiDOZ0eDUlCuhJfmRgIJooU3o8N7bGTg7 K7lpkd2BtK6WByWC7/MbZATP6DuFYQD+NlbWN+VE5K0X11ZfmgTmw9K4YJoOHjiEsa s91gS9u+EFEMg== Received: from p-i-exch-sc-m01.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Mon, 27 Nov 2023 19:16:37 +0300 (MSK) Received: from localhost (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Mon, 27 Nov 2023 19:16:37 +0300 Date: Mon, 27 Nov 2023 19:16:37 +0300 From: Dmitry Rokosov To: Michal Hocko CC: , , , , , , , , , , , , Subject: Re: [PATCH v3 2/2] mm: memcg: introduce new event to trace shrink_memcg Message-ID: <20231127161637.5eqxk7xjhhyr5tj4@CAB-WSD-L081021> References: <20231123193937.11628-1-ddrokosov@salutedevices.com> <20231123193937.11628-3-ddrokosov@salutedevices.com> <20231127113644.btg2xrcpjhq4cdgu@CAB-WSD-L081021> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: User-Agent: NeoMutt/20220415 X-Originating-IP: [100.64.160.123] X-ClientProxiedBy: p-i-exch-sc-m02.sberdevices.ru (172.16.192.103) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 181624 [Nov 27 2023] X-KSMG-AntiSpam-Version: 6.0.0.2 X-KSMG-AntiSpam-Envelope-From: ddrokosov@salutedevices.com X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 5 0.3.5 98d108ddd984cca1d7e65e595eac546a62b0144b, {Track_E25351}, {Tracking_from_domain_doesnt_match_to}, salutedevices.com:7.1.1;d41d8cd98f00b204e9800998ecf8427e.com:7.1.1;127.0.0.199:7.1.2;p-i-exch-sc-m01.sberdevices.ru:5.0.1,7.1.1;100.64.160.123:7.1.2, FromAlignment: s, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2023/11/27 13:23:00 #22553759 X-KSMG-AntiVirus-Status: Clean, skipped X-Rspamd-Queue-Id: 5FCB34003C X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: xzpm71noyrp3mufky6h6wkfxrd5z4eek X-HE-Tag: 1701101799-85694 X-HE-Meta: U2FsdGVkX1+Ux39Sb563HQ7P3X2u5bORP4I+5uX5zigqlsbdFrIbJAO8d4W5/Fms5U69LbXpZUmFLmmAheZMfm6t28X2opN96tlWpuexwxs0emABY9aBu6mpDp7Bm8J4w94RlPVrDuhgtniSLlN1KPI/K4ROpG64fJ+F75NtE1nqAUEJl46oGRUsvDwKq0wWnrJ1GdbXUZ/UuyOIOpAzOvJzCwD9eCpyclvfmsL976YyEoWlr8phF462SL3AukW3Hhr7s+RGJK6spM1HqrRgtyomWqzlluJ6KK974OHkU+3j3ziO563wuInDO1mcOH2F+JRgRwg5r5RWDPQ6elUskqRFQeHm+JPHp29abDhrTotJx6EkmLIS4zbRLQHRjFzBCh06NreXWE/tnylsmUqtJicXeH3aEIqeNUqLjATXcy0x9cR96DUmYYcHqCBHjPBLzbeJQpl8tLfTLbF5+YrW1FprLxLCJJ9GkxNqQJh1OPEmP2YuqU7ioDcqOiRzcN7RQFNErrLMK7bnVGQ+wZ782ekgM1OtgdTLVG9258Apsw2ozvaavv/XLlhNHzNvj+5E++15QREYbmV4p6ql9wPl3RONgzvu2gIRYTsMdCihTx6vRhoYJ0zBtATu5X/I53Cp0YNj9kWGoTkpZKU2K8eVuBve4F9mOKlFpcRkDUCVOUiG9g9xfp3EPw+2QMlqzSw2M11wSDZdrPONqNP6cmllcF82Qs42YPJlfyNH6YCF9elB3515n8na7jCru7iwKh5x62dxiD0SoZbxfJMr/thmVKhkQH7gYt9tvXL03zmRXB58dalsJcVSw+VtRUdX4UUiGvBeLkHRrQ5u25ffyi0y1AaqyD9OuCcn2JLBNCNHPs7RFZYt5DJVo1+B8xGsE8EwrpeJ+zoBSxGsui07YbZ2sZgsM0KpPogQ/qBVnxi/NwATndhAaw0CNVZadOV4IBmm4fYcXuf12KKsWmWwu2I odFotV4Q 9uJQbTyVJaBdsSjz37PljykLY6CpiCaF3pJh+kOgZRLs1Q0VFYc8KMIaHI9GGhvP8sZBNsjO6hbE7oL7FBMDOja0szXy3GOG27cJlRuSlLBhncsKyVqNs7PoKO9z2F4FEaXD0gRWcCLI/yPlecDNMRvytitueqUF9Sj3XeUQyxkDJMJ9L+DO8S1TN454R+Qc6DjMXGnOg8WCM1nncWEIkMRkXtQ+FvqfP1r8LoI7Q2le2vxLSON9iahDSVT+vH2Z2j/p9FqFbKTZvIGNKMIQ0ljxOTMBrXxiSKcGloofy66GlUS2zSWYqJHmddDT4vAW/pDxPTgn4K3TEKOTcMdqgSFIDDiYUXhSs8jHwHjB3XQjZWg8jSsZ69hhRvrFZ0ch4XtmBulJ3Jp1MWO0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Nov 27, 2023 at 01:50:22PM +0100, Michal Hocko wrote: > On Mon 27-11-23 14:36:44, Dmitry Rokosov wrote: > > On Mon, Nov 27, 2023 at 10:33:49AM +0100, Michal Hocko wrote: > > > On Thu 23-11-23 22:39:37, Dmitry Rokosov wrote: > > > > The shrink_memcg flow plays a crucial role in memcg reclamation. > > > > Currently, it is not possible to trace this point from non-direct > > > > reclaim paths. However, direct reclaim has its own tracepoint, so there > > > > is no issue there. In certain cases, when debugging memcg pressure, > > > > developers may need to identify all potential requests for memcg > > > > reclamation including kswapd(). The patchset introduces the tracepoints > > > > mm_vmscan_memcg_shrink_{begin|end}() to address this problem. > > > > > > > > Example of output in the kswapd context (non-direct reclaim): > > > > kswapd0-39 [001] ..... 240.356378: mm_vmscan_memcg_shrink_begin: order=0 gfp_flags=GFP_KERNEL memcg=16 > > > > kswapd0-39 [001] ..... 240.356396: mm_vmscan_memcg_shrink_end: nr_reclaimed=0 memcg=16 > > > > kswapd0-39 [001] ..... 240.356420: mm_vmscan_memcg_shrink_begin: order=0 gfp_flags=GFP_KERNEL memcg=16 > > > > kswapd0-39 [001] ..... 240.356454: mm_vmscan_memcg_shrink_end: nr_reclaimed=1 memcg=16 > > > > kswapd0-39 [001] ..... 240.356479: mm_vmscan_memcg_shrink_begin: order=0 gfp_flags=GFP_KERNEL memcg=16 > > > > kswapd0-39 [001] ..... 240.356506: mm_vmscan_memcg_shrink_end: nr_reclaimed=4 memcg=16 > > > > kswapd0-39 [001] ..... 240.356525: mm_vmscan_memcg_shrink_begin: order=0 gfp_flags=GFP_KERNEL memcg=16 > > > > kswapd0-39 [001] ..... 240.356593: mm_vmscan_memcg_shrink_end: nr_reclaimed=11 memcg=16 > > > > kswapd0-39 [001] ..... 240.356614: mm_vmscan_memcg_shrink_begin: order=0 gfp_flags=GFP_KERNEL memcg=16 > > > > kswapd0-39 [001] ..... 240.356738: mm_vmscan_memcg_shrink_end: nr_reclaimed=25 memcg=16 > > > > kswapd0-39 [001] ..... 240.356790: mm_vmscan_memcg_shrink_begin: order=0 gfp_flags=GFP_KERNEL memcg=16 > > > > kswapd0-39 [001] ..... 240.357125: mm_vmscan_memcg_shrink_end: nr_reclaimed=53 memcg=16 > > > > > > In the previous version I have asked why do we need this specific > > > tracepoint when we already do have trace_mm_vmscan_lru_shrink_{in}active > > > which already give you a very good insight. That includes the number of > > > reclaimed pages but also more. I do see that we do not include memcg id > > > of the reclaimed LRU, but that shouldn't be a big problem to add, no? > > > > >From my point of view, memcg reclaim includes two points: LRU shrink and > > slab shrink, as mentioned in the vmscan.c file. > > > > > > static void shrink_node_memcgs(pg_data_t *pgdat, struct scan_control *sc) > > ... > > reclaimed = sc->nr_reclaimed; > > scanned = sc->nr_scanned; > > > > shrink_lruvec(lruvec, sc); > > > > shrink_slab(sc->gfp_mask, pgdat->node_id, memcg, > > sc->priority); > > ... > > > > So, both of these operations are important for understanding whether > > memcg reclaiming was successful or not, as well as its effectiveness. I > > believe it would be beneficial to summarize them, which is why I have > > created new tracepoints. > > This sounds like nice to have rather than must. Put it differently. If > you make existing reclaim trace points memcg aware (print memcg id) then > what prevents you from making analysis you need? You are right, nothing prevents me from making this analysis... but... This approach does have some disadvantages: 1) It requires more changes to vmscan. At the very least, the memcg object should be forwarded to all subfunctions for LRU and SLAB shrinkers. 2) With this approach, we will not have the ability to trace a situation where the kernel is requesting reclaim for a specific memcg, but due to limits issues, we are unable to run it. 3) LRU and SLAB shrinkers are too common places to handle memcg-related tasks. Additionally, memcg can be disabled in the kernel configuration. -- Thank you, Dmitry