From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25A71C3ABC3 for ; Tue, 13 May 2025 22:25:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 621666B00D0; Tue, 13 May 2025 18:25:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5A7AF6B00D7; Tue, 13 May 2025 18:25:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4205C6B00D8; Tue, 13 May 2025 18:25:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 1B4316B00D0 for ; Tue, 13 May 2025 18:25:44 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 96C7612184B for ; Tue, 13 May 2025 22:25:45 +0000 (UTC) X-FDA: 83439317850.30.4EAF53D Received: from mail-wm1-f50.google.com (mail-wm1-f50.google.com [209.85.128.50]) by imf11.hostedemail.com (Postfix) with ESMTP id 9EA7440004 for ; Tue, 13 May 2025 22:25:43 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=RINvQz6k; spf=pass (imf11.hostedemail.com: domain of alexei.starovoitov@gmail.com designates 209.85.128.50 as permitted sender) smtp.mailfrom=alexei.starovoitov@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1747175143; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wAwr1O0koPzIlKh8epOygSIaJtugVEfkRrBmy7awSbw=; b=ahD4cS8U2qx95DIqEpzXaW1ANs+6hLaVTpDlPTMZ2pPbvIuql6GmvI6VYFDFnpS9RM7mwO mDTAiO+cYB1o2KaRuKcBYVoeeRLNyaw5ULUGTxR9uVpqTtcb7RMxAmsDRTF96ZxI/Kw9ra uCuYJvFMgQZtxNmMZD1JrGHP8IEnnn4= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=RINvQz6k; spf=pass (imf11.hostedemail.com: domain of alexei.starovoitov@gmail.com designates 209.85.128.50 as permitted sender) smtp.mailfrom=alexei.starovoitov@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1747175143; a=rsa-sha256; cv=none; b=cwwCKn0li3OFbcqT4FWsABXYw+9WVOaPdOHddIo9SwWPYGErO+xnP4mdV+kBLq88dt0XAc 5LzFYZHMYaJm1dpeOkPvGjHRmgGe8muX1KDcSHZOWPqsOfS4ifqSOvLYX8mfnsItghGrhb pCZKh+ou0tdIaW+ir9Gcmt9ViBOzdVc= Received: by mail-wm1-f50.google.com with SMTP id 5b1f17b1804b1-442ea95f738so12453845e9.3 for ; Tue, 13 May 2025 15:25:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1747175142; x=1747779942; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=wAwr1O0koPzIlKh8epOygSIaJtugVEfkRrBmy7awSbw=; b=RINvQz6kM0mHmA7LJPNpYpqxTP0AD7js3Mk5TwVZmsft2Z6GthRvFUPRsYyl9XE6Db NunxfIBrRhye9qUNx7zGbvonpEcQVV7ZG3h/CtpZbkh7W6eQQFmFs2oj6qb216IQ+TGJ QLm8QNAdi36PLGrze/qUZzjw9NDadI7/YIMfR1fzfFu7ZVAULFCcWwPM42N5CASHHTWx nC8wOxv+3EtAOY41FS/yBAr0WM26ARm1vGt+qeBm9xBYXhQ1gZOtAu7CjDFdrr4GH5Uv TS+xK9XYmEguheRr+uKxDMmdYCqcHDJI7/8tL/rAxEwgAuMDjicO/mJXSwQ85s9XPfzV EAEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747175142; x=1747779942; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wAwr1O0koPzIlKh8epOygSIaJtugVEfkRrBmy7awSbw=; b=VF8xsuSvfu6uvDvtDXU+JzVQuX0Q9D+YdgDwc+jAeEZtf0GLNOP8qwm5QfNXV4HXva uKpkZlMmk0U/8MS56sriaxzCfngCr6XIaK9NYvvmy78GeVaGKRwncYxpAWBR8dKA9zTE 728kW10CfpAWNnbnfN1FAiKovY17GYRhnGixqxqIvHpR0xyTSH7DRJA07y9DgrnA+YZS ZAGfeYHs2WwQWVBV/2S9Pf+6xh1jy9YmFMczMuSnMORx1OYR2579QZ9js8+eJc7dvDew ggsRizgSAmZU3riJ4DKGcoFawyWBoTW/VR1lbQ4FXPvCuBMUkTl+ARt9P2fCaYwmeL48 bQEg== X-Forwarded-Encrypted: i=1; AJvYcCUmvmkqwFcUEa/qppfb3jJ3BZdNv7cmPQtnwcmSyvAt7BbRTz0bejK54C0Qo1MW+mOOGCxfeqczTA==@kvack.org X-Gm-Message-State: AOJu0Yzo1bXu3raLBeY6aJvpksn4UcICOHMb+EpZJgsTjtJ74EyP2nCA Gul6IgL7/TEj0Ps8Er2YFUsulKjnAWJYF53Gp486OaXcYblaHbkpwLVddoEmjPT2/Laz6FAoG8U OKQ6qiQC68fgvTWFtb5Fv/nnCvhA= X-Gm-Gg: ASbGncvi8kTtjaMmxquD42o8slxcZ4IQqNVDGe7nK6HQ17Q4g3/9H6jYJuniDBydw+0 NbSqE3EIyOb3W4BL8w40QP1KA6P5nscuBDS1/mJXvoRYyM7mUHykVIvmvwnSldFcfO+D7Vs2mZC wllZcjUvCf4iTrkg0t30XURRPX/fg5CBu5Na68irOVNDY+KYshKqTCKaAKDoe04A== X-Google-Smtp-Source: AGHT+IGCSV1CW2V8g3e0buvKmCKp4+U6DS5ChJ8qy9B6coY/qGEWuanukGv1ONA7/0rEN2Xci8nFgq7f5PO+enTp454= X-Received: by 2002:a05:6000:430d:b0:3a0:b4f1:8bd1 with SMTP id ffacd0b85a97d-3a3499532b6mr669366f8f.52.1747175141983; Tue, 13 May 2025 15:25:41 -0700 (PDT) MIME-Version: 1.0 References: <20250509232859.657525-1-shakeel.butt@linux.dev> <20250509232859.657525-5-shakeel.butt@linux.dev> In-Reply-To: <20250509232859.657525-5-shakeel.butt@linux.dev> From: Alexei Starovoitov Date: Tue, 13 May 2025 15:25:31 -0700 X-Gm-Features: AX0GCFu-xAuvef9BfHGE2CGCgnqITmOEvaYA7dWvW7qUfVnFWb0NhsjFDSFfKcA Message-ID: Subject: Re: [PATCH 4/4] memcg: make objcg charging nmi safe To: Shakeel Butt Cc: Andrew Morton , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Vlastimil Babka , Alexei Starovoitov , Sebastian Andrzej Siewior , bpf , linux-mm , "open list:CONTROL GROUP (CGROUP)" , LKML , Meta kernel team Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 9EA7440004 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: z9xq5ind9aipn8o5ikunyubfm6z4ahr9 X-HE-Tag: 1747175143-873099 X-HE-Meta: U2FsdGVkX18DZqv4ocmquKanT5StZVeAzinJo247k2WNzRxlF3/NC9jW6RcPCca3e+zMwxymeNPbLIlwHMwMK89hTXl/BBuRSQNwRAwNEHnNJKxbbNMrMmHlvX9YNCabd36uNBnTa3HCOV06emTXRVpJphXZyH7p4O16fFIGB0g7yb1704Em0cLuyf+MujJDqDofea5Y8hPk7XAz1I7ij56U8kKAmcC2z3QEex7KaoNhgRI0NriyrYLZTHgKDVas19ZH0usOBGHl1cJetVAEwuRL8FVuve91uRRV5QjOpntJWjAmnmYEw4L98sVWbvazCPeHse9A5zVfOGnhgs6lVb+MAiwbRPowf+OJi8A92IQ73QsW/h4Xu/N0ACfBFmxjV+4+/Yv3n9+65+8CIUyEIHV7C+62miW7b+RwRi1BQjCRXcWZUGmWoesi2gBgSTch/PlVbprUGsYFdsZR61x3Ev5BthH2UACnwOs3Dt7oG3weugQNSW4JB7l0jJcrMg5Ec/T/SASxwVgr/uQ7TNOkPveXEEONbnfOjm2aABySl1v7T1nQqdu127UEJ71dWkRMWfLkYScGBScn4fzA5poztnfCupIAw9pYayTx6YXIIS2K3F3KFGhHAPUl9imbqLALu29HEapi4a7eBv2iQQ2RavBkmvKPAHxmoxobW85jEvdyivNPQSKnQ20o4U8hZecFMVTC16ZxQboiX4jhL81ykpyGOVkHehFUxYOx3hlVJatWph8dEhK9sqDnCoo5QfDH9cYGOCdjzqWAJWzTEL2+s3jAPMK1OFuoxWtGKE2koMk/ycoC+DdSaGuOMsZE5gMbaYhSjIuJuNhVIzADXeGDZKdzBnkPohR+RiS5daR7NeB1DYcK4PhNXDNX2SlrZs7bQQzDmJUjseOayyf9NjSqN1VOevviIUAyO0fGIvQgWRvek43pJ9/QRHT4uCcbMtByU7t2e2Pj+36NqklasHo edLJMqkq 5qVK8U6O3yGbr3gZEFH7LGSL4QUN+qKPCseh0jnoooZL0laxmWHdC44rj8DyS51X0WfZewZRbkgoeW47fZ59SCG2csO9PIB0sJfGLicldeF7cU/eQmlpSsVLnfq7t+RNu1JooEeTi+YP5TkevxC+SrgNPBe2nwcFKaKdd2ggkYSOS6fuyO/fwRnUpOWL8/XLLtngwU9oKdT2rUfCno8Dag9L6KIjPecXbhfBD+sMfDiO4lNeQpfb6H5KlBeWggHRPT1Z937pakLzy7VIq6yak6dOASygeHJxnk6EVcl9wjNWtVmJ5TCNGG+As7AItxs40L8jkbZJQ/uxbCFn4kkWB0DPCVJDRWKc3yt0b X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, May 9, 2025 at 4:29=E2=80=AFPM Shakeel Butt wrote: > > To enable memcg charged kernel memory allocations from nmi context, > consume_obj_stock() and refill_obj_stock() needs to be nmi safe. With > the simple in_nmi() check, take the slow path of the objcg charging > which handles the charging and memcg stats updates correctly for the nmi > context. > > Signed-off-by: Shakeel Butt > --- > mm/memcontrol.c | 14 +++++++++++++- > 1 file changed, 13 insertions(+), 1 deletion(-) > > diff --git a/mm/memcontrol.c b/mm/memcontrol.c > index bba549c1f18c..6cfa3550f300 100644 > --- a/mm/memcontrol.c > +++ b/mm/memcontrol.c > @@ -2965,6 +2965,9 @@ static bool consume_obj_stock(struct obj_cgroup *ob= jcg, unsigned int nr_bytes, > unsigned long flags; > bool ret =3D false; > > + if (unlikely(in_nmi())) > + return ret; > + > local_lock_irqsave(&obj_stock.lock, flags); > > stock =3D this_cpu_ptr(&obj_stock); > @@ -3068,6 +3071,15 @@ static void refill_obj_stock(struct obj_cgroup *ob= jcg, unsigned int nr_bytes, > unsigned long flags; > unsigned int nr_pages =3D 0; > > + if (unlikely(in_nmi())) { > + if (pgdat) > + __mod_objcg_mlstate(objcg, pgdat, idx, nr_bytes); > + nr_pages =3D nr_bytes >> PAGE_SHIFT; > + nr_bytes =3D nr_bytes & (PAGE_SIZE - 1); > + atomic_add(nr_bytes, &objcg->nr_charged_bytes); > + goto out; > + } Now I see what I did incorrectly in my series and how this patch 4 combined with patch 3 is doing accounting properly. The only issue here and in other patches is that in_nmi() is an incomplete condition to check for. The reentrance is possible through kprobe or tracepoint. In PREEMP_RT we will be fully preemptible, but obj_stock.lock will be already taken by the current task. To fix it you need to use local_lock_is_locked(&obj_stock.lock) instead of in_nmi() or use local_trylock_irqsave(&obj_stock.lock). local_trylock_irqsave() is cleaner and works today, while local_lock_is_locked() hasn't landed yet, but if we go is_locked route we can decouple reentrant obj_stock operation vs normal. Like the if (!local_lock_is_locked(&obj_stock.lock)) can be done much higher up the stack from __memcg_slab_post_alloc_hook() the way I did in my series, and if locked it can do atomic_add()-style charging. So refill_obj_stock() and friends won't need to change.