From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 18768C4167B for ; Wed, 29 Nov 2023 16:07:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 98CA96B03B4; Wed, 29 Nov 2023 11:07:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 93C996B03B6; Wed, 29 Nov 2023 11:07:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 852D46B03BA; Wed, 29 Nov 2023 11:07:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 767BB6B03B4 for ; Wed, 29 Nov 2023 11:07:15 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 3E3A9C055C for ; Wed, 29 Nov 2023 16:07:14 +0000 (UTC) X-FDA: 81511471188.12.3A16F7A Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) by imf05.hostedemail.com (Postfix) with ESMTP id 8A91210008B for ; Wed, 29 Nov 2023 16:06:55 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of mhocko@suse.com designates 195.135.223.131 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701274015; a=rsa-sha256; cv=none; b=iZPrj9Acxfb5Kb0mbuglr6XWwOSo6QTjbTjY9E6LjwTzWbFtSD2Y0Egwq7onOFMqRx6N7z ixhV/xmOL/B0bbTu1bQGL+G8nM23RImka4Bi7gLU14DvkyjwPub6jc6F2Okn+WCne7I4Vi c0m/P6JowZDGATjWjqyr5kQ9+n0d7NI= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of mhocko@suse.com designates 195.135.223.131 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701274015; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qHp746NuxHnA2irxqrli6B5cKV7C4dVHQY9GGTXk504=; b=7ZE+Ezq4QMjhkPdGFmjyleeadJP2U/C2k7kA0/s375Av1PuzAuWfikVyRn4ZoBgyjMgCob 2X4kI1PTZHSXc4sHTenBlcWVwYrO9kmRJvCcp+SonfXVvyGYzhL/bRUahHZJWyUmLfMYsT 7v+R1gojviR6A4mzwbW2wn9l+hGoLJM= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id AB71D1FB40; Wed, 29 Nov 2023 16:06:53 +0000 (UTC) Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 86CD61388B; Wed, 29 Nov 2023 16:06:53 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id P5KmHZ1hZ2VffAAAD6G6ig (envelope-from ); Wed, 29 Nov 2023 16:06:53 +0000 Date: Wed, 29 Nov 2023 17:06:37 +0100 From: Michal Hocko To: Dmitry Rokosov Cc: rostedt@goodmis.org, mhiramat@kernel.org, hannes@cmpxchg.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, akpm@linux-foundation.org, kernel@sberdevices.ru, rockosov@gmail.com, cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: Re: [PATCH v3 2/2] mm: memcg: introduce new event to trace shrink_memcg Message-ID: References: <20231123193937.11628-1-ddrokosov@salutedevices.com> <20231123193937.11628-3-ddrokosov@salutedevices.com> <20231127113644.btg2xrcpjhq4cdgu@CAB-WSD-L081021> <20231127161637.5eqxk7xjhhyr5tj4@CAB-WSD-L081021> <20231129152057.x7fhbcvwtsmkbdpb@CAB-WSD-L081021> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20231129152057.x7fhbcvwtsmkbdpb@CAB-WSD-L081021> X-Spamd-Bar: +++++++++++++++ X-Spam: Yes X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 8A91210008B X-Stat-Signature: w4uxotnpozbjhxnyoem4axmuytpcn4t1 X-HE-Tag: 1701274015-570046 X-HE-Meta: U2FsdGVkX19i4zGD6qGu9CNncbuW48QJjnk+61cy+ss9bwSjjFKcQHGDadBPtEXiIRpyAoBp8T0YQxreiX9bZw0AeRz/k+oEtCmkhvnEC5mKe5kpCthXLOeBiwziC87kgUfPFiljtJqbKIFZC5UPAEfROsYuzBP4zHs1NatCUYT6QqRtg1YRZgU5gutdf6nS6NPd2vS14rsTJfS6nk2Yu7BERRQisQDW8AgP2X+6bHZejGns1PTzq6eRBwUqBS5kHDgu60PJGdv3b7BN2LXAu5BbFmHZ53I8WvonC4syPsaxSjXFmoZe3pUw0Ujp1nKRPpS0FO5s+eNnQRhanoCKNJNpORF8swjABHaxYSPNA8bEBGJkFKNU3zTAXBQzS+zwy6qtOAmJDQ2NRs7nutGrILOFn2TRNTLSiLGga02Qh3a9+BuS+20L0405QsfUa1NUs7SwvpyZcTq9i7J470GbjeuATW/khSHRt0eWkNEmMOYoPb7cIo/3dXzczzjLoCfikPOmHVZBpOodv8sdo/xwJjs66Cuq7UNVH+aYE3SWJFKW7xqBobgKiE5jnn80+CEXD9wx2Fp2I6RBTZNageYW2LszxQmgo+JSQSEzuuhcEwoAAOK/TnakknkaxrgvSHvO9fv6RBrLJG7ny3LcDYrIB5XEheV75MgQtI58wSavyP5H5YRH/PsPj/lLfXPcOMVilZK+BLhIsRtdt3bBB83rGT4rXBHOXFMIUB0sVXGFTogsQghWpbVodzrzpCsItdmmuvmqN0IYK92PBalmFBpz1HVGAem2wPuTXb9rSjamWLLsoiI55MSLNv1iajKylxgnkt/pmmK7fqH6w+QPWHqhQwcpHnJ9CGzB5iA7w13sCOTQsYJZEejjfzYJZW73dg/qFLpD4y9AIokyU07hoQTQkUXA05ZZnyCiIjXmDBGL2faR4vBqkmSYj8AzcipHm7Utcm/owpJSeM4rYv1ijz8 NDpSngxG xhc/k++31lDTCuB4T6ojsCfTo2u/4MlBX8MGc3WUuGQRJ7b5UrchOxB5lf8vEJiWnY3X3l7vKR+MhzFerPkPhQInMntrQboyyGafy7mlBQerct1XZ/tAOC+H7FszopgIg//d4ps8EErwNO/ByftNh7DhvSaPqJOWEHXLOnhbhoAhTCRtdGBqpQsCXOG/nXUDmiqN9ufbz/hMd14VXD9wYrv7OhdbN0PsAGCfw X-Bogosity: Ham, tests=bogofilter, spamicity=0.001772, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed 29-11-23 18:20:57, Dmitry Rokosov wrote: > On Tue, Nov 28, 2023 at 10:32:50AM +0100, Michal Hocko wrote: > > On Mon 27-11-23 19:16:37, Dmitry Rokosov wrote: [...] > > > 2) With this approach, we will not have the ability to trace a situation > > > where the kernel is requesting reclaim for a specific memcg, but due to > > > limits issues, we are unable to run it. > > > > I do not follow. Could you be more specific please? > > > > I'm referring to a situation where kswapd() or another kernel mm code > requests some reclaim pages from memcg, but memcg rejects it due to > limits checkers. This occurs in the shrink_node_memcgs() function. Ohh, you mean reclaim protection > === > mem_cgroup_calculate_protection(target_memcg, memcg); > > if (mem_cgroup_below_min(target_memcg, memcg)) { > /* > * Hard protection. > * If there is no reclaimable memory, OOM. > */ > continue; > } else if (mem_cgroup_below_low(target_memcg, memcg)) { > /* > * Soft protection. > * Respect the protection only as long as > * there is an unprotected supply > * of reclaimable memory from other cgroups. > */ > if (!sc->memcg_low_reclaim) { > sc->memcg_low_skipped = 1; > continue; > } > memcg_memory_event(memcg, MEMCG_LOW); > } > === > > With separate shrink begin()/end() tracepoints we can detect such > problem. How? You are only reporting the number of reclaimed pages and no reclaimed pages could be not just because of low/min limits but generally because of other reasons. You would need to report also the number of scanned/isolated pages. > > > 3) LRU and SLAB shrinkers are too common places to handle memcg-related > > > tasks. Additionally, memcg can be disabled in the kernel configuration. > > > > Right. This could be all hidden in the tracing code. You simply do not > > print memcg id when the controller is disabled. Or just simply print 0. > > I do not really see any major problems with that. > > > > I would really prefer to focus on that direction rather than adding > > another begin/end tracepoint which overalaps with existing begin/end > > traces and provides much more limited information because I would bet we > > will have somebody complaining that mere nr_reclaimed is not sufficient. > > Okay, I will try to prepare a new patch version with memcg printing from > lruvec and slab tracepoints. > > Then Andrew should drop the previous patchsets, I suppose. Please advise > on the correct workflow steps here. Andrew usually just drops the patch from his tree and it will disappaer from the linux-next as well. -- Michal Hocko SUSE Labs