From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 581B1C369D9 for ; Wed, 30 Apr 2025 10:05:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ADE806B00C3; Wed, 30 Apr 2025 06:05:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A3CC46B00C4; Wed, 30 Apr 2025 06:05:54 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8DC9C6B00C5; Wed, 30 Apr 2025 06:05:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 6D7106B00C3 for ; Wed, 30 Apr 2025 06:05:54 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id E6C025ECBC for ; Wed, 30 Apr 2025 10:05:54 +0000 (UTC) X-FDA: 83390279028.12.4EB55B7 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf15.hostedemail.com (Postfix) with ESMTP id 45F59A000F for ; Wed, 30 Apr 2025 10:05:52 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b="mZ5Fv/Wi"; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=AXUgg6dA; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b="mZ5Fv/Wi"; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=AXUgg6dA; dmarc=none; spf=pass (imf15.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1746007552; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=aXUil+ET8/TFI2RImwQgr+2V8ec+IC8TtX59DMlILv8=; b=iFp1+u4gjxcKSICfJRHnSoM+XPzEgpwf9uZCJ5G/c3dJY9Yunskxdwz+9DNtV5MVeS3Ehr wrCfHthQyodRI1iOtQVDKKHxjFpPd84LSrBQkLETzks1mLAGgRByq8rC8OmaV5kf1hIavd gElG0JMfGhSbfQfr4FBk9drBaB/G0G4= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b="mZ5Fv/Wi"; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=AXUgg6dA; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b="mZ5Fv/Wi"; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=AXUgg6dA; dmarc=none; spf=pass (imf15.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1746007552; a=rsa-sha256; cv=none; b=JuT7yk+bl6yt3RUo+SGpCaHUAPWOpEiVTwT/Cae6yuPXveRk7B1TcA7U7gIPTS5kW5espU VzM+aGKT7Q5z2YydKOTAMyoiT2Ktl4CD6b9KFSTVVvauNUQHTx9bb+9jmWiDd7nELZijY9 dg5kD+Zy15WcYXrQhqVYEOGi65Qg3bE= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 4A35421297; Wed, 30 Apr 2025 10:05:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1746007549; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aXUil+ET8/TFI2RImwQgr+2V8ec+IC8TtX59DMlILv8=; b=mZ5Fv/WizkLn5NQrcxZu/FLmX8MdFpKMB2/SZ/wtUbFUGK/2tfr+ejbshEi0VS25qAVrhw Jr/aJcKXiDqdhCtW7FazUfoq6X2F2L6CO+6ZxSOoWlMzQwW9hKl98zAfdbe+XafoLd4gGI 8FMRNOssevnX6TOFPmOa44mL8ZXzvqo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1746007549; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aXUil+ET8/TFI2RImwQgr+2V8ec+IC8TtX59DMlILv8=; b=AXUgg6dA2eZI/VUxqCn6CCv2P4mL/A5vhBIQhfCeFvmiADbiR/+r5X8swdJIyEa66qO2pI WilQZ+8X0ltfl7CA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1746007549; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aXUil+ET8/TFI2RImwQgr+2V8ec+IC8TtX59DMlILv8=; b=mZ5Fv/WizkLn5NQrcxZu/FLmX8MdFpKMB2/SZ/wtUbFUGK/2tfr+ejbshEi0VS25qAVrhw Jr/aJcKXiDqdhCtW7FazUfoq6X2F2L6CO+6ZxSOoWlMzQwW9hKl98zAfdbe+XafoLd4gGI 8FMRNOssevnX6TOFPmOa44mL8ZXzvqo= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1746007549; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aXUil+ET8/TFI2RImwQgr+2V8ec+IC8TtX59DMlILv8=; b=AXUgg6dA2eZI/VUxqCn6CCv2P4mL/A5vhBIQhfCeFvmiADbiR/+r5X8swdJIyEa66qO2pI WilQZ+8X0ltfl7CA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 1D3C8139E7; Wed, 30 Apr 2025 10:05:49 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id IBLQBv31EWhxMAAAD6G6ig (envelope-from ); Wed, 30 Apr 2025 10:05:49 +0000 Message-ID: Date: Wed, 30 Apr 2025 12:05:48 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH] memcg: multi-memcg percpu charge cache Content-Language: en-US To: Shakeel Butt , Andrew Morton Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Jakub Kicinski , Eric Dumazet , Soheil Hassas Yeganeh , linux-mm@kvack.org, cgroups@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Meta kernel team , Hugh Dickins References: <20250416180229.2902751-1-shakeel.butt@linux.dev> From: Vlastimil Babka In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Action: no action X-Stat-Signature: xdw1ej7qp46w3tsu8i1g5hsdijn3o1fa X-Rspamd-Queue-Id: 45F59A000F X-Rspam-User: X-Rspamd-Server: rspam05 X-HE-Tag: 1746007552-18097 X-HE-Meta: U2FsdGVkX1+899g/Oh+lt6jWjXS6BeS1tTcjXWw9GXNqoqi/Ep/TXGyO5MTMXXDC7kVQ+sA6cBSnuB4eAFx/RErco2fhqoXY1+WUfM03hrm0+jzSxb9wSQ9nb9iji/OvbqASnPspCO1G1XmCi/wUFVIfanuFMOVTRJu8owbiArBEu5aTdGskXG6ikQDxLabUpfykPPClNnjl0DJiXyIKsc5ABL3frMW4wU7sLdqgYYNMPw8AxCxr1tvORSUkI9JHMt+M/Xwp5kXiJBrjJ9jJtxvFbUSctzrCg12hbKcXLCFAnq9BoywguaVgZSzvN8FvvYwuN22XlMxyCXS7ilIgKLjzZue0Q4kUumwepFO0EuoTh61R+hDo2WELWFkWE2eX4WjOPlcMrEO/CFn2l+cuBw6HGVI1mItz7q2jODIIxkrCmzUYiZ62NgDNLgAl6B8wH5pvVubXWFUzSXnLjBZR9E1ABttiH22t4NI+sIbKoKuWXkaYIsyhdFHgNZBHozkRuzkaulKw0kPqIxpdUSwax7Zsrp59LMeP89nlfxeC3wArw3vQCVlyg0HLJ6p6m0GmvNTcy1XqeaISIKQgOMPe7EUaLaUv8cp1/2u70OthZv+3P+WFwLHn6aEWGUBjbsvKXi34OHKESllRGDZFpvkbTAQA2/PpugcZ1OYsab7E0nJ+yB3DU9QmYbB/fkWI5KzCE6xAdZeAll9472AhtY22osT/s1B1e3I9CX7LF/zt7lFpMBnEzNI3n4/w6xwDf/igEnnZ/ZPFbAbzy1H4fVYTHS8SmSihX3a2Hi57Xy+0b4URd57gKR3ApQwLXIS1JbNZ7nUSschbwezCdzNFnlXySbifNyH3vOsHhZdBRHVfQ2GfGO/fK1NyzUvpxknKk7gWNyNvwWZzWYeWf5BBNL9AcUU+KNk+z2f5Bqd8zVOhqBN9u0Z9B1/AgyzEgr9EyujAj4htCfF3qNOi6e0krQG FsT4X5JR kI8UGwvJE83rr/GvMnMZ3zBQbGS95GYMFNtZnun1U0F3TuU/xuAIkWpLaIWSvbKav+Dhg5yjwdJ5kFG11eT5HDGz9Ab5Kp0Pwyl3/cw325KOhoDjh96nXRDtfJSruyjku/KdEhMlmfPpDi9wY4iyhSKolIJJftf8ErtzZ554cdCf730aP8jVvBKCva2idHtUVS33K2fSVTBIUIG7zW6JzqvGdoNtFXVbNKFCJgyjB1iz6ZYYkr6Jq+LqlEsurumNz0zv0zgzU/spL3bfccVQmF7bBQxwUkC6nOwtf X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 4/25/25 22:18, Shakeel Butt wrote: > Hi Andrew, > > Another fix for this patch. Basically simplification of refill_stock and > avoiding multiple cached entries of a memcg. > > From 6f6f7736799ad8ca5fee48eca7b7038f6c9bb5b9 Mon Sep 17 00:00:00 2001 > From: Shakeel Butt > Date: Fri, 25 Apr 2025 13:10:43 -0700 > Subject: [PATCH] memcg: multi-memcg percpu charge cache - fix 2 > > Simplify refill_stock by avoiding goto and doing the operations inline > and make sure the given memcg is not cached multiple times. > > Signed-off-by: Shakeel Butt It seems to me you could simplify further based on how cached/nr_pages arrays are filled from 0 to higher index and thus if you see a NULL it means all higher indices are also NULL. At least I don't think there's ever a drain_stock() that would "punch a NULL" in the middle? When it's done in refill_stock() for the random index, it's immediately reused. Of course if that invariant was made official and relied upon, it would need to be documented and care taken not to break it. But then I think: - refill_stock() could be further simplified - loops in consume_stop() and is_drain_needed() could stop on first NULL cached[i] encountered. WDYT? > --- > mm/memcontrol.c | 27 +++++++++++++++------------ > 1 file changed, 15 insertions(+), 12 deletions(-) > > diff --git a/mm/memcontrol.c b/mm/memcontrol.c > index 997e2da5d2ca..9dfdbb2fcccc 100644 > --- a/mm/memcontrol.c > +++ b/mm/memcontrol.c > @@ -1907,7 +1907,8 @@ static void refill_stock(struct mem_cgroup *memcg, unsigned int nr_pages) > struct mem_cgroup *cached; > uint8_t stock_pages; > unsigned long flags; > - bool evict = true; > + bool success = false; > + int empty_slot = -1; > int i; > > /* > @@ -1931,26 +1932,28 @@ static void refill_stock(struct mem_cgroup *memcg, unsigned int nr_pages) > > stock = this_cpu_ptr(&memcg_stock); > for (i = 0; i < NR_MEMCG_STOCK; ++i) { > -again: > cached = READ_ONCE(stock->cached[i]); > - if (!cached) { > - css_get(&memcg->css); > - WRITE_ONCE(stock->cached[i], memcg); > - } > - if (!cached || memcg == READ_ONCE(stock->cached[i])) { > + if (!cached && empty_slot == -1) > + empty_slot = i; > + if (memcg == READ_ONCE(stock->cached[i])) { > stock_pages = READ_ONCE(stock->nr_pages[i]) + nr_pages; > WRITE_ONCE(stock->nr_pages[i], stock_pages); > if (stock_pages > MEMCG_CHARGE_BATCH) > drain_stock(stock, i); > - evict = false; > + success = true; > break; > } > } > > - if (evict) { > - i = get_random_u32_below(NR_MEMCG_STOCK); > - drain_stock(stock, i); > - goto again; > + if (!success) { > + i = empty_slot; > + if (i == -1) { > + i = get_random_u32_below(NR_MEMCG_STOCK); > + drain_stock(stock, i); > + } > + css_get(&memcg->css); > + WRITE_ONCE(stock->cached[i], memcg); > + WRITE_ONCE(stock->nr_pages[i], stock_pages); > } > > local_unlock_irqrestore(&memcg_stock.stock_lock, flags);