From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E3A85C3ABC9 for ; Fri, 9 May 2025 23:29:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 346728E0015; Fri, 9 May 2025 19:29:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2801A8E000E; Fri, 9 May 2025 19:29:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 120048E0015; Fri, 9 May 2025 19:29:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id DEE208E000E for ; Fri, 9 May 2025 19:29:55 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id C40B85D0AE for ; Fri, 9 May 2025 23:29:57 +0000 (UTC) X-FDA: 83424964434.04.117FD6C Received: from out-180.mta1.migadu.com (out-180.mta1.migadu.com [95.215.58.180]) by imf14.hostedemail.com (Postfix) with ESMTP id 247D7100003 for ; Fri, 9 May 2025 23:29:55 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=jO54i0gB; spf=pass (imf14.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.180 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1746833396; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kOY56VtkuEsaEOIjh23t6VcD+q32Hzr2b1NcpUgpvDE=; b=eY3pVtB2O2jXljGUoN2sFRqHCCDUSIC0tZYTU7Yl90hkouui93cHlN3eQia7kCInSSUdxu +OWomqK5zKbHFgXRPegg8SR8LUY0SgrqrnxGWWQ71J9IDvTw63IBAg0OBhQNt5RtgIsdGt tGG8HDDnKyGMX9s86L+3VE0notz/RXM= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=jO54i0gB; spf=pass (imf14.hostedemail.com: domain of shakeel.butt@linux.dev designates 95.215.58.180 as permitted sender) smtp.mailfrom=shakeel.butt@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1746833396; a=rsa-sha256; cv=none; b=m5due9L6pn8ZkiobWTXXhw2iE7v8h9ESMO07zYVsCR+ieB2v1pZY+uSIrgONl6TyEkQ5BA eUR+wo7VzFSb25dz9lb12HCrpapDAIhHRCkDVF6yZkPAZ6zcopwsVItqLWgoh00NU54/P4 6o0RV4IADfQRhPb83ruS6gbrEhwxMzc= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1746833394; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kOY56VtkuEsaEOIjh23t6VcD+q32Hzr2b1NcpUgpvDE=; b=jO54i0gBEBaFktQDWRNMuv4pLq+uKpVvOSG8rTGoHARaxg8BKTT6XbD+GIS15P62AwsIgo GN6rjfqV9p4hIv4kuKltdlIFtazFX1Sdk5xLUX4DldmlmVbQcCnTMwi4oc6iq5souJ0Adb 19j4Qj8OvDs/mAh60jbhxHW5ENDLVBw= From: Shakeel Butt To: Andrew Morton Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Vlastimil Babka , Alexei Starovoitov , Sebastian Andrzej Siewior , bpf@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Meta kernel team Subject: [PATCH 4/4] memcg: make objcg charging nmi safe Date: Fri, 9 May 2025 16:28:59 -0700 Message-ID: <20250509232859.657525-5-shakeel.butt@linux.dev> In-Reply-To: <20250509232859.657525-1-shakeel.butt@linux.dev> References: <20250509232859.657525-1-shakeel.butt@linux.dev> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Migadu-Flow: FLOW_OUT X-Stat-Signature: ka8uehzseqge3th9kgfepquegsescr7b X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 247D7100003 X-Rspam-User: X-HE-Tag: 1746833395-975545 X-HE-Meta: U2FsdGVkX18rH2biuLMqyg8+SeXl/+Rf9xj7kQQ9kKpkV/mb6tDh90DLx4Seu3j73R9kBUfz9MrrNt/zQarhOijCAHec5XOOFJN8AZ+srM5qBEofHlg0ki5qo0b06qM4nh/HBkJJZrg8kPb5idhBF299v/x3AWHltssI08t+Cj8GR8GPdcvfzAahfxcBQjEi5wZz43tq3vRGvkfik9qP8nShHvOFXjgDWloUicpnMysxRTibChML419Pv/IE7lz9FB42K5BCr3Ffy2LzHrd++cONnt/TpB660n7UYQZdiAtC7Zgw3z41Q8yLBGo992owLrmyPFIglklls4f+5joChqNoDtU6FK8ZZGyC4Oln7ySTZbUR+4r8tgOZ6j9SZKOYgZhPMxUKH5PJzu+nzNJ3oOEtnvR3midQY1fzze2BeMqe1yRe0Wd3usJviHAm6toLSvuAa5sQViFyGu+k1v1kaOltbcduix+WZdZIqKuFVyoxDbWah6vURxsWmdqW7TUxIfZofWIGbTWstoV03Dxv2P38I+nzqNCGdiB/qQvaC7Lv1m8Rcf8+2YUFg59xiTTnLLu6ywgK6Y2yhlxLmS1nuUjoBg//dNSvwmOBgswV905uuEQVst9elcK96q+WcFId8SFZFZdRsw+GcLB39NwgDOzt2b4zNMPf6WD79qUcjcAWgkPK6jLwBGMyo0nAf2N4EsrQi0mu71UWLDuhj2qgySLU3wsM+fgCc1FM2BhzFc80DzLTzT4pmAWzzXUxAG0FJPPNxEtt9LpBBajpVROOVp9NSOmpbpE9KguAcX5XIvuGc9kByh9jaNGjixQH2r0XELic2m5pf6vR116cIysCL7xUYj1Qoi0lqt7TH//wTSUFOEZ0VX8fmVCc5wfLJxynx7s6AGnFyCbsJj+hwUcwsCQdFAtwEkjXu8J0+1wliZh9y7YQ3at+Q1umeQ498C44eS6QFCTbA6G2qNgFBYk 53rAZhHD tj4Ju/6LyBlliT+Z4BztyjFxYO35kDXIyZTKvPJLGFpQqCN+ezOHfGs4YeQxbi9b775U7FC/gkSYwsxIV0TXB3P8/uSBxPoVFNa7cZzZbgHP2MY5uPlwCx2i0ExEmmIEEScxeJooy2xjr9xWsTouP199//H/VHZ5gNBjI7ludtanMfGE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: To enable memcg charged kernel memory allocations from nmi context, consume_obj_stock() and refill_obj_stock() needs to be nmi safe. With the simple in_nmi() check, take the slow path of the objcg charging which handles the charging and memcg stats updates correctly for the nmi context. Signed-off-by: Shakeel Butt --- mm/memcontrol.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index bba549c1f18c..6cfa3550f300 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2965,6 +2965,9 @@ static bool consume_obj_stock(struct obj_cgroup *objcg, unsigned int nr_bytes, unsigned long flags; bool ret = false; + if (unlikely(in_nmi())) + return ret; + local_lock_irqsave(&obj_stock.lock, flags); stock = this_cpu_ptr(&obj_stock); @@ -3068,6 +3071,15 @@ static void refill_obj_stock(struct obj_cgroup *objcg, unsigned int nr_bytes, unsigned long flags; unsigned int nr_pages = 0; + if (unlikely(in_nmi())) { + if (pgdat) + __mod_objcg_mlstate(objcg, pgdat, idx, nr_bytes); + nr_pages = nr_bytes >> PAGE_SHIFT; + nr_bytes = nr_bytes & (PAGE_SIZE - 1); + atomic_add(nr_bytes, &objcg->nr_charged_bytes); + goto out; + } + local_lock_irqsave(&obj_stock.lock, flags); stock = this_cpu_ptr(&obj_stock); @@ -3091,7 +3103,7 @@ static void refill_obj_stock(struct obj_cgroup *objcg, unsigned int nr_bytes, } local_unlock_irqrestore(&obj_stock.lock, flags); - +out: if (nr_pages) obj_cgroup_uncharge_pages(objcg, nr_pages); } -- 2.47.1