From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70A20C3ABDA for ; Wed, 14 May 2025 18:42:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 03CCD6B009C; Wed, 14 May 2025 14:42:54 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F073B6B009D; Wed, 14 May 2025 14:42:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DADEF6B009E; Wed, 14 May 2025 14:42:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id BAB356B009C for ; Wed, 14 May 2025 14:42:53 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 6971D161096 for ; Wed, 14 May 2025 18:42:54 +0000 (UTC) X-FDA: 83442385068.15.4FEDF3D Received: from out-182.mta1.migadu.com (out-182.mta1.migadu.com [95.215.58.182]) by imf03.hostedemail.com (Postfix) with ESMTP id B39402000E for ; Wed, 14 May 2025 18:42:52 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b="Z5+Yysz/"; spf=pass (imf03.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.182 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1747248172; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=t4sog0H8PQnyhxh0qyypANz3qKaJTBcH3eV85qDti8s=; b=XTAB5kt4NfHu395gHW/t97oLpT+6f9NqHdGfAV6a3f/FlkJmSG48J3iujtjkbodlHSbVTo nVupi63rUopHGOJ/sHa3VlM6oC3SDsjSSTMpc+Cog/d8UB2GiBmRZ1R69lRa/Tp0A/6Nem yj+L2spiInBlQQBePQGZfti0rzfUczQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1747248172; a=rsa-sha256; cv=none; b=CGvAtMB/sMTcAWF7EiCKiZTibxu0Q7VfPXfIxD4Zsf2HWkzHAXcl5CCzj4PWZvz7P9HJAM 0CU28qULw0B+vq1WHQ4xY5vfLuq+EkYsC3kMOMqYO4hVSjNJZjY9ozPUby09CjlOv0/JKe 1E27BLExtaSIKRvF4Jvvy56nuC7JSpk= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b="Z5+Yysz/"; spf=pass (imf03.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.182 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1747248171; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=t4sog0H8PQnyhxh0qyypANz3qKaJTBcH3eV85qDti8s=; b=Z5+Yysz/a0FdntBwdRVd8vs8HsgqhcPMP+VKoaKIvnPkLTDX4LkzhvGK1PfZ7VZPIKZMun JBNzghWN4lIUeiS04Y8Rcd9V0puEcW7IGz7xPLlH0EmzJODv/2n91JroIJvlIakoJCqSkL VAjiedXqXbQZ+nC5FqwmFn+0vYrU1DA= From: Shakeel Butt To: Andrew Morton Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Vlastimil Babka , Alexei Starovoitov , Sebastian Andrzej Siewior , Harry Yoo , Yosry Ahmed , bpf@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Meta kernel team Subject: [PATCH v2 7/7] memcg: objcg stock trylock without irq disabling Date: Wed, 14 May 2025 11:41:58 -0700 Message-ID: <20250514184158.3471331-8-shakeel.butt@linux.dev> In-Reply-To: <20250514184158.3471331-1-shakeel.butt@linux.dev> References: <20250514184158.3471331-1-shakeel.butt@linux.dev> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: B39402000E X-Stat-Signature: 8eipb7uq1r7ykgwcg3pqyznjwuub14bq X-Rspam-User: X-HE-Tag: 1747248172-774025 X-HE-Meta: U2FsdGVkX18dGkr0W+VRH1KV98HAP9uYKCA2nDi3cBKKdNaJOpOghwGTn76QU87VllZOic58088fedfNQ0wwefZ8xUniDyG/M5Ij71VFycjwtv/E9p+uafCToGbxJLyj9tOpe1VzdQNfaaGqWNglBNHZvgf16GCXqPiaRZZj9kxd1eQk9FndP2NyGFkhbXE6sh0Z+LAi+7W5UJarCTHAMzDFMzhAdfnqbR5OFTr1sIrue7j7+EatLMOLzHMK3xYuiKlkbnN9iEXGr6ij4i+KYM5UBiprE1mR99kDVYXJOFQvxNFMDEFwOF9/HoHoY1JOgT1Hl8hDN3FDrM9W2vqdGLHyINgdLtkmP2eOjclbzLSC1Tcfi0R5EO5Ez+zFFJTdSn76gCtXE4XMLSGoVnbFf5HfLfaeSdi/mejWZHH4kC0FsQJMTWL5GZILOtRPplHcHx/145EKVRXwtNYKuj262ZGfnnu9hZ6ICBRjzFNweYT+nnYDusEdvlOcY0zbcR7pZ3RnsYz14wy0NDvdXLBmv/ypvuM+9DpvO9LmV0XMwzQOBlQiOxFEUVk2JZMC1J2ekz/VbKDg2Bpwf8sheq0nXlN//tuzb4av45s4qodGN7mK1LC0nJTAZQ/gHJHBPR68hrOeBPcEDU/lJdufFaj3b94H/QH/NagdtW9eIEbVDfgKOrB4Hvcz2NjL1YXr1ulskm1gRrun7T8lzl07+X6HIWr/ZoG2QzUM7UQrD2frcO02Wb0UVTDPdpQUwUBviUZbQQJ4jqVXNSdqjF6hJY7M2/XECM0WaEaglQP4/9qMAeeWz5UaNee/sHKxaXunAueh9yLKW61Elo5oFmqKJ069mEM18B7/QmaRGo/rU1WKlm27r6qwd88Di9KRJ5UFFai5xA6G6Pv4vV+aoxYoz24DM41gVoft8uaPEC7bFt+oTflrQP7CwcEGBa1+3MKqABZj9pZBDxT7eqCewQ4Fa5R 4Mah9IzR VC4McDGJN4RyDV20NVicai+HoQN4CapekxpU1nxJV6f9N25T+NfgjFQ1t39/raV9vwMQNz32BAMaS6CnMlcB9Yb6Y/1W7BJ1q3A+3S78ixiUKQQWah9vM+AD71v88rTusNrO7kpifik+Vm8aKGU6vwto0osqXWmTHV1NjhoTGkvr97Ik= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There is no need to disable irqs to use objcg per-cpu stock, so let's just not do that but consume_obj_stock() and refill_obj_stock() will need to use trylock instead to avoid deadlock against irq. One consequence of this change is that the charge request from irq context may take slowpath more often but it should be rare. Signed-off-by: Shakeel Butt Acked-by: Vlastimil Babka --- mm/memcontrol.c | 25 +++++++++++++++---------- 1 file changed, 15 insertions(+), 10 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 04d756be708b..e17b698f6243 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1882,18 +1882,17 @@ static void drain_local_memcg_stock(struct work_struct *dummy) static void drain_local_obj_stock(struct work_struct *dummy) { struct obj_stock_pcp *stock; - unsigned long flags; if (WARN_ONCE(!in_task(), "drain in non-task context")) return; - local_lock_irqsave(&obj_stock.lock, flags); + local_lock(&obj_stock.lock); stock = this_cpu_ptr(&obj_stock); drain_obj_stock(stock); clear_bit(FLUSHING_CACHED_CHARGE, &stock->flags); - local_unlock_irqrestore(&obj_stock.lock, flags); + local_unlock(&obj_stock.lock); } static void refill_stock(struct mem_cgroup *memcg, unsigned int nr_pages) @@ -2876,10 +2875,10 @@ static bool consume_obj_stock(struct obj_cgroup *objcg, unsigned int nr_bytes, struct pglist_data *pgdat, enum node_stat_item idx) { struct obj_stock_pcp *stock; - unsigned long flags; bool ret = false; - local_lock_irqsave(&obj_stock.lock, flags); + if (!local_trylock(&obj_stock.lock)) + return ret; stock = this_cpu_ptr(&obj_stock); if (objcg == READ_ONCE(stock->cached_objcg) && stock->nr_bytes >= nr_bytes) { @@ -2890,7 +2889,7 @@ static bool consume_obj_stock(struct obj_cgroup *objcg, unsigned int nr_bytes, __account_obj_stock(objcg, stock, nr_bytes, pgdat, idx); } - local_unlock_irqrestore(&obj_stock.lock, flags); + local_unlock(&obj_stock.lock); return ret; } @@ -2979,10 +2978,16 @@ static void refill_obj_stock(struct obj_cgroup *objcg, unsigned int nr_bytes, enum node_stat_item idx) { struct obj_stock_pcp *stock; - unsigned long flags; unsigned int nr_pages = 0; - local_lock_irqsave(&obj_stock.lock, flags); + if (!local_trylock(&obj_stock.lock)) { + if (pgdat) + mod_objcg_mlstate(objcg, pgdat, idx, nr_bytes); + nr_pages = nr_bytes >> PAGE_SHIFT; + nr_bytes = nr_bytes & (PAGE_SIZE - 1); + atomic_add(nr_bytes, &objcg->nr_charged_bytes); + goto out; + } stock = this_cpu_ptr(&obj_stock); if (READ_ONCE(stock->cached_objcg) != objcg) { /* reset if necessary */ @@ -3004,8 +3009,8 @@ static void refill_obj_stock(struct obj_cgroup *objcg, unsigned int nr_bytes, stock->nr_bytes &= (PAGE_SIZE - 1); } - local_unlock_irqrestore(&obj_stock.lock, flags); - + local_unlock(&obj_stock.lock); +out: if (nr_pages) obj_cgroup_uncharge_pages(objcg, nr_pages); } -- 2.47.1