From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51864E77188 for ; Tue, 14 Jan 2025 10:39:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D83B46B0085; Tue, 14 Jan 2025 05:39:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D2F896B0088; Tue, 14 Jan 2025 05:39:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BF7B8280001; Tue, 14 Jan 2025 05:39:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id A054F6B0085 for ; Tue, 14 Jan 2025 05:39:35 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 6194240A6D for ; Tue, 14 Jan 2025 10:39:35 +0000 (UTC) X-FDA: 83005711110.27.76B329C Received: from mail-wr1-f42.google.com (mail-wr1-f42.google.com [209.85.221.42]) by imf14.hostedemail.com (Postfix) with ESMTP id 59238100019 for ; Tue, 14 Jan 2025 10:39:33 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=suse.com header.s=google header.b=eFqFVxGn; spf=pass (imf14.hostedemail.com: domain of mhocko@suse.com designates 209.85.221.42 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736851173; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=FsRTQIRkFQb4leXv1P1v2eNnz9iWaV1jp8hC6C+H31Q=; b=Tc7LTQHv7gAa8yt7TxFo3vMOmdi/9Bsqv9n80qb4+T9Xo1Brgk5PvoCA2HZv1YrA/T+uV7 zqsU7Gr5t69cVm36jOx2uz0h4598ykU7rK+nvRpwZnFopxxMBAXYFuD4SNGiAX0r6eUOvw b2xk31LOb/kKhZQbfzW1maOv+nYv51U= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=suse.com header.s=google header.b=eFqFVxGn; spf=pass (imf14.hostedemail.com: domain of mhocko@suse.com designates 209.85.221.42 as permitted sender) smtp.mailfrom=mhocko@suse.com; dmarc=pass (policy=quarantine) header.from=suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736851173; a=rsa-sha256; cv=none; b=31elmyHpTVhuPx4hxkIMU34QiysALuOTVouEB4NC51lT8IFeyqO20koYwjDMK2u2JZk8xK LpBq5PaKvdPeo8tai9T0bjmEW7cAUwJqE3yzCd2rjFLi0hqd1V2Fqfm5jQJF4wjGW8u4O/ fEqT6qgdOvIc6nguP61Cm6IvG/qzPPE= Received: by mail-wr1-f42.google.com with SMTP id ffacd0b85a97d-385dece873cso2600307f8f.0 for ; Tue, 14 Jan 2025 02:39:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=google; t=1736851172; x=1737455972; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=FsRTQIRkFQb4leXv1P1v2eNnz9iWaV1jp8hC6C+H31Q=; b=eFqFVxGnp8kvoMCv91KtoENhdgX46RKwu1hLc6oZmf3Cllo9BLXfkZEaC5ZmbgTSRq Xdl485EgMA7DGKSn9NxyIasVO5jQ1A+WVJ7/Ex23rxKyCZUtFdeHqfOLg00a46l+y4B+ rwp5sUDwnCH4ICt/rmpccHGBPT1r8wMhar1oWmflmSgUO+CMaO/xeIcnw8ZA2Egf+RV9 TZk24lK2hLs/E9Ig1HAghMh2+eYybN9ujhJkymesAzkaT3tfPLvBul38/XI6H+EEdyzM +oKt+ru9NY1HkEaDtdx+54WWGNq0gcxhLG62ZsaDGWtuknyxlROEjfrneq5keXPYUNPC hGNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736851172; x=1737455972; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=FsRTQIRkFQb4leXv1P1v2eNnz9iWaV1jp8hC6C+H31Q=; b=HOTYk3fnR+1CFoN7ofSF6D97tc/Qvmoym2wzcIrSch3PAPP31L9wj0gzAM4h25ia22 NKhUFNtpLBMAWURrAUspyycW7KQAqusYD1xYoCcM76Vo5QlYb71mEZWPQoHWk7VWjjtA E9M1dpTKsfuYCkMLsEhO8TY+9c9T4thrlokulWVfsZlJjcNHg1bRI22kxwljNUiVHw9y TlYf4KVtDOcjDggU05guGQtbBFX53/0kdmmJRu+CLSxpg33QFNrhfYWSEva5ZWsxYpJo avMLHp6gh3kixDe4A7Oo+RAQTJl43YHs6+2I7hSjj6s8nAbNIPiqvw6eh2EqdSJiD7no vOXg== X-Forwarded-Encrypted: i=1; AJvYcCWxFCaxMRdCO4ehKokjfdTU2SBBbBGO7eQBeDQdfXJxrR/xfat0OCzczbhzoFEFi6oBbb237/aKPw==@kvack.org X-Gm-Message-State: AOJu0Yz5ecsoHE0JGebZQ2QQIPVz7WeyxkIhkgomRrZ2D3MK8wqWbVyr RjOyDmZB8h00joPQDLlc3iwGngj03SREsQSLwlkTapagXNckle7PFWKO8fekpS4= X-Gm-Gg: ASbGncvJdTiLlbs6QeL9184hvmF/caz57nGsGPCrHqe0V1iNX6FWteW9hE18ThNpU8J cYnverGDyuLML2Ky9xEpIMjizY7h55N9nz1xTkYQDlISI31WgtqBiwwrQf6oKAfv67ba7NuM2Ab 22QYuZajZUuDgjZJLbPhBH8WugGQm2C5FAZOA+AOhR+bXZW473xhP2IFnSOa2IbizI+2es13+/T IxtDiRO5M8TiOq+7aKIggyeZ0JXxCu5UDm06My7mgQqFZwVqfUihoiMZg+Ag3PCRYYVeA== X-Google-Smtp-Source: AGHT+IFAVa5tyrLidBCYPwKeJgLed+u3W2u71vh3DJJzVgf1eg7sePiqTjsCTdbp5GPDSJ3Dar9buw== X-Received: by 2002:a05:6000:709:b0:386:3803:bbd5 with SMTP id ffacd0b85a97d-38a8733a1f9mr23210306f8f.45.1736851170333; Tue, 14 Jan 2025 02:39:30 -0800 (PST) Received: from localhost (109-81-90-202.rct.o2.cz. [109.81.90.202]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-38a8e4c1d13sm14618544f8f.91.2025.01.14.02.39.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 Jan 2025 02:39:30 -0800 (PST) Date: Tue, 14 Jan 2025 11:39:29 +0100 From: Michal Hocko To: Alexei Starovoitov Cc: bpf@vger.kernel.org, andrii@kernel.org, memxor@gmail.com, akpm@linux-foundation.org, peterz@infradead.org, vbabka@suse.cz, bigeasy@linutronix.de, rostedt@goodmis.org, houtao1@huawei.com, hannes@cmpxchg.org, shakeel.butt@linux.dev, willy@infradead.org, tglx@linutronix.de, jannh@google.com, tj@kernel.org, linux-mm@kvack.org, kernel-team@fb.com Subject: Re: [PATCH bpf-next v4 4/6] memcg: Use trylock to access memcg stock_lock. Message-ID: References: <20250114021922.92609-1-alexei.starovoitov@gmail.com> <20250114021922.92609-5-alexei.starovoitov@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20250114021922.92609-5-alexei.starovoitov@gmail.com> X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 59238100019 X-Rspam-User: X-Stat-Signature: 4uez5wwbyd6wsfzz7efjnf7brqfoqpku X-HE-Tag: 1736851173-338639 X-HE-Meta: U2FsdGVkX1/rNHFDlW7hr8GnGphF7Es6b2gC4aNO39+JbrvL5eTNlF/hnUgO4/gE9WjU3FYYTnwVzMuP5GSHnp0Zg0wxfr/eFF62XNTyXrIqbpqjvc6opVXA1/lAbcpKW2xbj5FreiV9tFwo6NyK33AYP7O31mwlnjsYJZePKK2UnH93d7K4ujLBcdkHZHBSdPwAOU0uj6L8+0GMEt2jYa42g7u97nyWlD9Gzu8ZP1x9QQQP7JU5pDJL+aKrYO+ICvIUabjMxz8hbWKRA96HHYxut0t5BUDxGGAsfmRTrtg1SdbzDZ5dIBa7nM5X8DfxCgTE3+pKFLMbU3v9Nl2b2a+yBM4XMd+XCtVIPO3wAHLiW2BNKGnawFu4klK9tuYx0AH6RWl6XEjuiUAk4xGKRkmBMpp7gLsOEq2VdO8h3cLNeA68ZlrOCR4rCtTL1pLBKtmUmq/aE/SYEdBNKKSoVKlfcMZnlWedpWfRuEB8FzaFxjPFJiAAk7267PBdT1r9AXLLXstExqB2ritzmnSoXZ5BqnHlVVp/BEGkTDVAlMteyp1GjNs3iq6SfFL14njZxDMRoKqmxe65lOP1horz3pc7VcYrudTQqRB3RWX7Gz+wW68mBwee6TmA3BZm7BWHGn5oa/RU/+iKs+dAfbOAcGf8Ita06o5llb9lbQCPlEYBAPqZoUz45/7knYAcoMC90Rfu1IPFUt5dexlb0edx9jnso3sZPvvYozwQDompPrEZc8WIcoItoHeGDOD7rM8CcPedsms6f7gGsNtwxmd4K4gtqQ8W5EgUPKKOquNcVF7JpcB/ZumvufJ2ku5K8Xqu0EptCXIWHrVjTQgZ76486tvcguMmJc86PSlwCcTW3vFPJGdS8TpuYEG7GKdGAu9W5cRjNCdLgis98qQxI+vVWauKiT3zAbgeeljLUDYy4HEw9SGqnB9JdvCZvjFoyyYPKWWdX8lJ949fnHwBOu2 6DiABrc+ igcOZrhkEK8hWtdjLLWnpeCShjWQQmWI8gSRkmFPX1ebodI4+xi8ILP1F0Z8M+ZvtMptg+OLzPyu28bIB5W24vgaWmYCuSfPERPGCcLrCiVsTQSxf/9hDk6QNhhA4gO11uEs6qgx+E605JDXlf4J6RTJsMPwiXWA929QqegafOL6rbvKqmK3toSTmUsfSRWYiIJyzmAlmf3jYT7ZPqQtwileZKyh29/j3wUByjhtZ7G1tg/vn7ZqiOO2dnDSn440BDDSQeLHDrtAJXxpe8CUVpsHQMmi9twqKBxPu+DaI3fhxsPBjBhSWOAukt0fZdf8TBQSPRyf64jUUJTkaAVz3++eaZU28Y7oMyXoMJF9RgKTyF3raJOUfgXVt3zoQojlKkKJBriv/B9Kk+Tjd0r6RYck+VoRYN4blgLsBfztPhEv8EsRlS2IOVFIGhG2m0SDLo9TIr9N4CBuEXdVZeg8mzUMr8BDxEl4N2qFKUPggU/o1lks5FV4pmYv8fMZrdLSrmszg X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon 13-01-25 18:19:20, Alexei Starovoitov wrote: > From: Alexei Starovoitov > > Teach memcg to operate under trylock conditions when > spinning locks cannot be used. > The end result is __memcg_kmem_charge_page() and > __memcg_kmem_uncharge_page() are safe to use from > any context in RT and !RT. > In !RT the NMI handler may fail to trylock stock_lock. > In RT hard IRQ and NMI handlers will not attempt to trylock. I believe this is local_trylock_irqsave specific thing that is not that interesting for the particular code path. It is more useful to mention consequences. I would phrase it this way. local_trylock might fail and this would lead to charge cache bypass if the calling context doesn't allow spinning (gfpflags_allow_spinning). In those cases we try to charge the memcg counter directly and fail early if that is not possible. This might cause a pre-mature charge failing but it will allow an opportunistic charging that is safe from try_alloc_pages path. > Signed-off-by: Alexei Starovoitov Acked-by: Michal Hocko > --- > mm/memcontrol.c | 24 ++++++++++++++++++++---- > 1 file changed, 20 insertions(+), 4 deletions(-) > > diff --git a/mm/memcontrol.c b/mm/memcontrol.c > index 7b3503d12aaf..e4c7049465e0 100644 > --- a/mm/memcontrol.c > +++ b/mm/memcontrol.c > @@ -1756,7 +1756,8 @@ static bool obj_stock_flush_required(struct memcg_stock_pcp *stock, > * > * returns true if successful, false otherwise. > */ > -static bool consume_stock(struct mem_cgroup *memcg, unsigned int nr_pages) > +static bool consume_stock(struct mem_cgroup *memcg, unsigned int nr_pages, > + gfp_t gfp_mask) > { > struct memcg_stock_pcp *stock; > unsigned int stock_pages; > @@ -1766,7 +1767,11 @@ static bool consume_stock(struct mem_cgroup *memcg, unsigned int nr_pages) > if (nr_pages > MEMCG_CHARGE_BATCH) > return ret; > > - local_lock_irqsave(&memcg_stock.stock_lock, flags); > + if (!local_trylock_irqsave(&memcg_stock.stock_lock, flags)) { > + if (!gfpflags_allow_spinning(gfp_mask)) > + return ret; > + local_lock_irqsave(&memcg_stock.stock_lock, flags); > + } > > stock = this_cpu_ptr(&memcg_stock); > stock_pages = READ_ONCE(stock->nr_pages); > @@ -1851,7 +1856,14 @@ static void refill_stock(struct mem_cgroup *memcg, unsigned int nr_pages) > { > unsigned long flags; > > - local_lock_irqsave(&memcg_stock.stock_lock, flags); > + if (!local_trylock_irqsave(&memcg_stock.stock_lock, flags)) { > + /* > + * In case of unlikely failure to lock percpu stock_lock > + * uncharge memcg directly. > + */ > + mem_cgroup_cancel_charge(memcg, nr_pages); > + return; > + } > __refill_stock(memcg, nr_pages); > local_unlock_irqrestore(&memcg_stock.stock_lock, flags); > } > @@ -2196,9 +2208,13 @@ int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask, > unsigned long pflags; > > retry: > - if (consume_stock(memcg, nr_pages)) > + if (consume_stock(memcg, nr_pages, gfp_mask)) > return 0; > > + if (!gfpflags_allow_spinning(gfp_mask)) > + /* Avoid the refill and flush of the older stock */ > + batch = nr_pages; > + > if (!do_memsw_account() || > page_counter_try_charge(&memcg->memsw, batch, &counter)) { > if (page_counter_try_charge(&memcg->memory, batch, &counter)) > -- > 2.43.5 -- Michal Hocko SUSE Labs