From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 061B0C3ABC5 for ; Wed, 14 May 2025 05:09:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 09F186B00CE; Wed, 14 May 2025 01:09:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 050CE6B00CF; Wed, 14 May 2025 01:09:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E598F6B00D1; Wed, 14 May 2025 01:09:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id C812E6B00CE for ; Wed, 14 May 2025 01:09:10 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 17A14E0793 for ; Wed, 14 May 2025 05:09:11 +0000 (UTC) X-FDA: 83440334502.05.87B00B8 Received: from out-176.mta1.migadu.com (out-176.mta1.migadu.com [95.215.58.176]) by imf21.hostedemail.com (Postfix) with ESMTP id C6A081C0002 for ; Wed, 14 May 2025 05:09:07 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=BB6zKS6f; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf21.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.176 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1747199348; a=rsa-sha256; cv=none; b=LafTBTt5ZbP1yKSarBtVMehWQpOtOBJa2txsZ+DfEz5I2mTpVjTP2JLo4NbFkn7E9lv3Qr 5ozM6Njdz+FJQ1iNNtl+JezP305a7hclRPDQnm2zT/XSWOa5+FJ7+obSvJM00aLXU5+YK5 HWlyNHv/Q+CP6A3OFXol0XO19YXbiAM= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=BB6zKS6f; dmarc=pass (policy=none) header.from=linux.dev; spf=pass (imf21.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.176 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1747199348; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NuiqYU5xQUJzlD4OF0DY6DzP++awncv0NldqTEFbrbk=; b=QUmNvAZVPbRDsdbWeZ3uplWzjB+oEmm3osYtTBwVFXnRciY0aT8ThZVGZ0XCKHR2kxBv4U 1pnbkWd1yHVCJtM/XTR4DtiJKAmUmBvSRaP1jzgptuTls3lmhJs9wzLKoplY/Z18ASPcgi vexzIl/6qO1CYDitoxdsGlIFtmgJ+S8= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1747199346; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NuiqYU5xQUJzlD4OF0DY6DzP++awncv0NldqTEFbrbk=; b=BB6zKS6fZt3IpK/PR2CCdNoGUYOkfDyeW0jJMjAtx0JRxkzb3wR1CwWeIyeUXF6F8CVPyQ RIxNVBrlQPoCKw9bPu6nsPBuDBXncIYIHIJ/NBJu2AQ/6VwHqfxddK4NDXmwlv6OuYPqAu 3X6ncWadCJ7SIUgeaoUy8vmHHxLLaVg= From: Shakeel Butt To: Andrew Morton Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Vlastimil Babka , Alexei Starovoitov , Sebastian Andrzej Siewior , Harry Yoo , Yosry Ahmed , bpf@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Meta kernel team Subject: [PATCH 7/7] memcg: objcg stock trylock without irq disabling Date: Tue, 13 May 2025 22:08:13 -0700 Message-ID: <20250514050813.2526843-8-shakeel.butt@linux.dev> In-Reply-To: <20250514050813.2526843-1-shakeel.butt@linux.dev> References: <20250514050813.2526843-1-shakeel.butt@linux.dev> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT X-Rspam-User: X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: C6A081C0002 X-Stat-Signature: up9ngz6sscxq7kpnwh4knzni5amwn1ok X-HE-Tag: 1747199347-199916 X-HE-Meta: U2FsdGVkX1+c5Epnw4bHl3/exgcwS/AFbi5THrODujDg6Bx2+snaplzRh/5Aci96WwBQEHFawiIyJUWFbkEUGL6TIo4Oh4qgT/XckjkUoI5O88ZbvlkQKUCPEAe2+G2sBqE75dozc0EDZXgvq4cnTAiuQXbByXKs2iQBDDqAfc0LIlXLa4iqWquaH1VP1XqCLcHUhEu/ApvuRbc8PhkvOsZh9ucyT5Q1ARZrAEHn746R9hP1Wv6nWB42VL6bEQN5teO+NOCQPuVr1MMDpnASYf+34k5CC7boarRL6AMaKS9hvigH74jzaLsFN4ko5oSZaBcIhhdN0IHlUw9lMOAb8qKHW3pX0A+fL/sCz0KRFqeDj0v2wqk9eZL0w3cUleWh87oPdjz+4hqeVw8Evixwa/V9R/2yNN5RncMXTjXUFVKcZeZiBhuDlP8RNFsiOyr60KFQ1cNbDriOB64Hdec6mmWPyylx5bis/8fd0jzp5n91vJ0DNy7q95TSJOAwMgg3XctyvN+Sy4sWH/JKR5ZPTsL6LglvgXpBbsOfnzvXjR3XjgvjWMMCxlAgG+JRLvHYer+rb+1RkwdPtX5hmHbTR1yZUpI0/Y4UbQBkHv3Mr2rcxt8zu+PQCcznL4RpLVJgcTLSGXMuCaFs3mw0SUysVYZCs8DRzDE//m/6Uxfu07la5jvqOyF7yKbe33jeMVrh5TS9LreHQcT5HHJXZSdes8rtHkkPfusgxSdniYqdPDuqTgE4VXp9gph7Gm6C0Rrh+xffw38A1EDGRs5cBz0+evIVYUbOvdXIRm5aHRYWGJ5/EPrsyP3Wnn1uUVDRc+Fh2YtZQVWkbneHrSJV+oVW4NpppYKvYD28QTQuMX41FuhkOfxEXaychSJKm+d9WcuQqQvb/EyygLmSvI87c73pb56qk9JgqyIkg2MhPxhA1SS9LRU6PeGliyt0N98b4zg5+2ozhNteD7PaUoUEaoi F/HyRH9S z/yzYHH7UmOMXFX2lgCiGanUjBa4kL1MrcC0+ThBeYQ3jnCt0lC0nggm3CfhiWCXPBwjveLGjDvvsDPsY0BrkK/UUEJt7vNqAZIrfGJ/gdInUtG72nmDyQTGwkTBjxqbAEEyRMzGLkKYDHqUBlri9KVsHYwM5qoycUTZXgNEMWWtdgiI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: There is no need to disable irqs to use objcg per-cpu stock, so let's just not do that but consume_obj_stock() and refill_obj_stock() will need to use trylock instead to avoid deadlock against irq. One consequence of this change is that the charge request from irq context may take slowpath more often but it should be rare. Signed-off-by: Shakeel Butt Acked-by: Vlastimil Babka --- mm/memcontrol.c | 25 +++++++++++++++---------- 1 file changed, 15 insertions(+), 10 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 78a41378b8f3..73b19137901a 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -1907,18 +1907,17 @@ static void drain_local_memcg_stock(struct work_struct *dummy) static void drain_local_obj_stock(struct work_struct *dummy) { struct obj_stock_pcp *stock; - unsigned long flags; if (WARN_ONCE(!in_task(), "drain in non-task context")) return; - local_lock_irqsave(&obj_stock.lock, flags); + local_lock(&obj_stock.lock); stock = this_cpu_ptr(&obj_stock); drain_obj_stock(stock); clear_bit(FLUSHING_CACHED_CHARGE, &stock->flags); - local_unlock_irqrestore(&obj_stock.lock, flags); + local_unlock(&obj_stock.lock); } static void refill_stock(struct mem_cgroup *memcg, unsigned int nr_pages) @@ -2901,10 +2900,10 @@ static bool consume_obj_stock(struct obj_cgroup *objcg, unsigned int nr_bytes, struct pglist_data *pgdat, enum node_stat_item idx) { struct obj_stock_pcp *stock; - unsigned long flags; bool ret = false; - local_lock_irqsave(&obj_stock.lock, flags); + if (!local_trylock(&obj_stock.lock)) + return ret; stock = this_cpu_ptr(&obj_stock); if (objcg == READ_ONCE(stock->cached_objcg) && stock->nr_bytes >= nr_bytes) { @@ -2915,7 +2914,7 @@ static bool consume_obj_stock(struct obj_cgroup *objcg, unsigned int nr_bytes, __account_obj_stock(objcg, stock, nr_bytes, pgdat, idx); } - local_unlock_irqrestore(&obj_stock.lock, flags); + local_unlock(&obj_stock.lock); return ret; } @@ -3004,10 +3003,16 @@ static void refill_obj_stock(struct obj_cgroup *objcg, unsigned int nr_bytes, enum node_stat_item idx) { struct obj_stock_pcp *stock; - unsigned long flags; unsigned int nr_pages = 0; - local_lock_irqsave(&obj_stock.lock, flags); + if (!local_trylock(&obj_stock.lock)) { + if (pgdat) + mod_objcg_mlstate(objcg, pgdat, idx, nr_bytes); + nr_pages = nr_bytes >> PAGE_SHIFT; + nr_bytes = nr_bytes & (PAGE_SIZE - 1); + atomic_add(nr_bytes, &objcg->nr_charged_bytes); + goto out; + } stock = this_cpu_ptr(&obj_stock); if (READ_ONCE(stock->cached_objcg) != objcg) { /* reset if necessary */ @@ -3029,8 +3034,8 @@ static void refill_obj_stock(struct obj_cgroup *objcg, unsigned int nr_bytes, stock->nr_bytes &= (PAGE_SIZE - 1); } - local_unlock_irqrestore(&obj_stock.lock, flags); - + local_unlock(&obj_stock.lock); +out: if (nr_pages) obj_cgroup_uncharge_pages(objcg, nr_pages); } -- 2.47.1