From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E6D9C4167B for ; Wed, 29 Nov 2023 17:10:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E4BB66B0344; Wed, 29 Nov 2023 12:10:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DF92C6B0349; Wed, 29 Nov 2023 12:10:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CE97B6B034B; Wed, 29 Nov 2023 12:10:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id BD59E6B0344 for ; Wed, 29 Nov 2023 12:10:38 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 8D511405F8 for ; Wed, 29 Nov 2023 17:10:38 +0000 (UTC) X-FDA: 81511630956.18.D17707C Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf24.hostedemail.com (Postfix) with ESMTP id 26C0E18002D for ; Wed, 29 Nov 2023 17:10:35 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=c5lp+V55; spf=pass (imf24.hostedemail.com: domain of mhocko@suse.com designates 195.135.223.130 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701277836; a=rsa-sha256; cv=none; b=8WhOW8M+fue4vW85XNu8WyTSzXfGl/DJy/QOWDuG41gPBlrSPniqbfZQFB1mCjCZwYhJPL pRQiVpQn+i5lllIZpwW63T905xqEPNYB1A/Lu16JwARW7v+cOZP02eLfOidzgm2aFXdfdv BiFMZ/vfk9aax7kUa7QgdS2EW3A/974= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=c5lp+V55; spf=pass (imf24.hostedemail.com: domain of mhocko@suse.com designates 195.135.223.130 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701277836; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vzwYMvZH+lLs7nIcf9848QxwWYfgK9k6V/VFPR/CPRo=; b=dCYyq4MEDBkvkRmGkmnEmWhNzJVPHuJr3BEWDoN3VfJrU2OrELlNKDC0OP+0Nk6NKi1n/L L5CeYTm3tEyXGGPQkNtOmK6B42odC0FT/8NzoCw/O2WlkRQYee1XrjphFbGMi73Gu7OpIN J4RwngztjNNBmLthSfH58AI/fjphiPQ= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 486F7219B5; Wed, 29 Nov 2023 17:10:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1701277834; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=vzwYMvZH+lLs7nIcf9848QxwWYfgK9k6V/VFPR/CPRo=; b=c5lp+V55CaoT+Xdq7Wwd+8OE9dGrFAn2IzXhIrl90H2aFxKRO6Ii6NJEzhC7kZQmNiA1RG cd9t4hK0Yg4/h5Z+6oIfZSu4CPnTItNmi9PW9op1hC60azk4mNv3lNiAKZZPZP4eAAUZ8j NputPfcch4/rlsKJdpGm3lxHYX0cirI= Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 1B94213637; Wed, 29 Nov 2023 17:10:34 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id AycXA4pwZ2V1DgAAD6G6ig (envelope-from ); Wed, 29 Nov 2023 17:10:34 +0000 Date: Wed, 29 Nov 2023 18:10:33 +0100 From: Michal Hocko To: Dmitry Rokosov Cc: akpm@linux-foundation.org, rostedt@goodmis.org, mhiramat@kernel.org, hannes@cmpxchg.org, roman.gushchin@linux.dev, shakeelb@google.com, muchun.song@linux.dev, kernel@sberdevices.ru, rockosov@gmail.com, cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: Re: [PATCH v3 2/2] mm: memcg: introduce new event to trace shrink_memcg Message-ID: References: <20231123193937.11628-1-ddrokosov@salutedevices.com> <20231123193937.11628-3-ddrokosov@salutedevices.com> <20231127113644.btg2xrcpjhq4cdgu@CAB-WSD-L081021> <20231127161637.5eqxk7xjhhyr5tj4@CAB-WSD-L081021> <20231129152057.x7fhbcvwtsmkbdpb@CAB-WSD-L081021> <20231129165752.7r4o3jylbxrj7inb@CAB-WSD-L081021> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20231129165752.7r4o3jylbxrj7inb@CAB-WSD-L081021> X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 26C0E18002D X-Stat-Signature: 74cuyn8mdxk4mwrdomkxzda9k1tuqgcm X-Rspam-User: X-HE-Tag: 1701277835-313057 X-HE-Meta: U2FsdGVkX1+4KaDAojHU1Hc4uRnSMMbhwMuIGPvO5BqLVBFzWigAWagacVNKZphre2SaZOzCRuj49kqCZtaDPDu97RcpFYTqLVDcdVbxGRdnCfHfAR022iVcmhIm5eZpkXkjBDjKzDFb2QcOM7prDu10SUKnx4lZts1qe95oysSn5KUbVnlhZ8RWz1T2zn8boHh7LrHOdXgPGf46kdrHGmrb06FUYxp+eSbUbZahSwVaaJz8uvMCeM8Lxx1gd6wZJQdG5RTm9j9RoZ60VlucD6R7ij9nZj6WrTv9P7cUfCwhzX9NsG+793oQb8zmrkUySHTDKNA6H3KA5JIL45DK/tURc8XHIalHuhzj19aU/x/CJK+9YgoLLzF9e1pMgaflWMdmvCVpGsth2ipXtx0S+bXVDmJojv+dCRqPlqOlgRX7djtYgcrPqHxeJUJ6PTOVthsukkV0w9rf4POtotEC646T7I9aS7wLsozQFfx6rhLyN50gCnecI1nIVe+33ghuX9egNWhieaRb04Qu5zvC18JOZUgU0U2o0ZoYolijFfppsLWwasmshrKlLEYTsyMLaqgJnP/GNpGalKs7Zlhl8f/kFm6vnP9QkHBVHNmlM8T4TbxJBRxGYllEL/6SMdAOPKw5ZgCL6jeQY0tocEkWrb7IFIlKFefSv06/5UsispLuNd0zoMBnOZGmVbPOsWYjcZQHzsNswJzRVyRdIe9PcgwynuT7McPk8F73p4KOM5eN7QLFtFLv3WY9cWY8FL5lMr8c+i35wz6Qv16BP1BTSsWRhJBganGsj1DrmVtRxwkz7c/AWlUqal2SBq6AjjTe0bZDW5FsuCX4a1KW+YP/q7czyiDDTJU9NnQjRsuXAW87kWZSlY+7oiuj3aQ1TTJ7lDKgKqLBoNYl3spslnzByGLf+fuqFXY4MPuF2ViQW6ynBInFj8NekH+JCRvj1L7NRgF6Y8gVPCygb4262w8 TXr4mMA4 Go+ot7qJySPMOn4aoGjYpFIByMSP+DX/6c3dqOqb2NT+fQzhnfyrene5YMI/idgHWPf2iivUUd+jO8v8PFzGGUq4NiruLUCbpr8EZip7DLC+sUv1tlkdOFXthlRqBSDCp5zBLmW20zBmxObA6XaQsO3CLhO4rl+w1J3A0u3X65Q8yhaVyZnnYkap68VjKnq//FTrzvhSSiYHet8uDzUnfVzsUwvDKhAJxbNnew1NyZfBBvWV3jekQX4izijw14OP0zdGbO7kKe5Kk6P8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000232, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed 29-11-23 19:57:52, Dmitry Rokosov wrote: > On Wed, Nov 29, 2023 at 05:06:37PM +0100, Michal Hocko wrote: > > On Wed 29-11-23 18:20:57, Dmitry Rokosov wrote: > > > On Tue, Nov 28, 2023 at 10:32:50AM +0100, Michal Hocko wrote: > > > > On Mon 27-11-23 19:16:37, Dmitry Rokosov wrote: > > [...] > > > > > 2) With this approach, we will not have the ability to trace a situation > > > > > where the kernel is requesting reclaim for a specific memcg, but due to > > > > > limits issues, we are unable to run it. > > > > > > > > I do not follow. Could you be more specific please? > > > > > > > > > > I'm referring to a situation where kswapd() or another kernel mm code > > > requests some reclaim pages from memcg, but memcg rejects it due to > > > limits checkers. This occurs in the shrink_node_memcgs() function. > > > > Ohh, you mean reclaim protection > > > > > === > > > mem_cgroup_calculate_protection(target_memcg, memcg); > > > > > > if (mem_cgroup_below_min(target_memcg, memcg)) { > > > /* > > > * Hard protection. > > > * If there is no reclaimable memory, OOM. > > > */ > > > continue; > > > } else if (mem_cgroup_below_low(target_memcg, memcg)) { > > > /* > > > * Soft protection. > > > * Respect the protection only as long as > > > * there is an unprotected supply > > > * of reclaimable memory from other cgroups. > > > */ > > > if (!sc->memcg_low_reclaim) { > > > sc->memcg_low_skipped = 1; > > > continue; > > > } > > > memcg_memory_event(memcg, MEMCG_LOW); > > > } > > > === > > > > > > With separate shrink begin()/end() tracepoints we can detect such > > > problem. > > > > How? You are only reporting the number of reclaimed pages and no > > reclaimed pages could be not just because of low/min limits but > > generally because of other reasons. You would need to report also the > > number of scanned/isolated pages. > > > > From my perspective, if memory control group (memcg) protection > restrictions occur, we can identify them by the absence of the end() > pair of begin(). Other reasons will have both tracepoints raised. That is not really great way to detect that TBH. Trace events could be lost and then you simply do not know what has happened. -- Michal Hocko SUSE Labs