From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3FD4DE77188 for ; Tue, 14 Jan 2025 18:42:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CCB2E28001E; Tue, 14 Jan 2025 13:42:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C7B53280017; Tue, 14 Jan 2025 13:42:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B42A628001E; Tue, 14 Jan 2025 13:42:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 980AE280017 for ; Tue, 14 Jan 2025 13:42:34 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 1F45A160933 for ; Tue, 14 Jan 2025 18:42:34 +0000 (UTC) X-FDA: 83006928228.18.13CEF55 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf12.hostedemail.com (Postfix) with ESMTP id 5CE0640004 for ; Tue, 14 Jan 2025 18:42:32 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=rhLsueKz; spf=pass (imf12.hostedemail.com: domain of "SRS0=OKOt=UG=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org" designates 139.178.84.217 as permitted sender) smtp.mailfrom="SRS0=OKOt=UG=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org"; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736880152; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=SN/u1z34jJ0edNnQ1/0RtJk+YL/N/pihlipLvHOWYHI=; b=YI+jk/6Nli7DOBPp8P/q9Lbc0GcL+EYtcIWOMlbzmYnybvGuFFqDL0IfxK0QuZCsvY2+kD 7wbU2sjncRKi35u4xdORQrX9D+s7OIdhwVAkf4uk2Ds53DNgOaXKrkt7A9FWesTV0D7y1X xLoNhJqZJQhNZ5WCXMybibDOrhbyTcM= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=rhLsueKz; spf=pass (imf12.hostedemail.com: domain of "SRS0=OKOt=UG=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org" designates 139.178.84.217 as permitted sender) smtp.mailfrom="SRS0=OKOt=UG=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org"; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736880152; a=rsa-sha256; cv=none; b=yI7S4KGKRm0HaxYhEWLIKdsQ6h9JJUIcIxYjtU0K4VlKF9H85Kt/nUYLKhOxX+hStqAY8I M3rcQZpjrXRO3e11ybtxEO5Qjg1plkFBUWod/ckm2GZhkZkcVBocORqvZaYfOHuGigZGRc pwW69/u6WR3zugplqIzNESv+N9vl2uw= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 72DDF5C59CE; Tue, 14 Jan 2025 18:41:50 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BE2AEC4CEE3; Tue, 14 Jan 2025 18:42:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1736880150; bh=s5XHy7GS3WGeMJh5qpqB6ZPE3LuyXt5cAZB+qzn1hCw=; h=Date:From:To:Cc:Subject:Reply-To:References:In-Reply-To:From; b=rhLsueKzKvYN0LLxLWREYv8Rgsq0gwFjmgK5FLCOnEbnDhMDogsnQZ3l6h6d4fXyO W+AHCQTBgeWnK9xx5UxzOuDJ/AyPw0sSv4XgMTJBfDvrBPG+4CFSruSCUKLu6FkXQp pPt8pM9p5EvhqfuNrks96oSsGaoTXzyp0MG2Jnv+o598VuEgCnkc/8jfyRdqFCOUZe VFljliT1O36nLeriD7azVvYb/wDsjq2r0KEgUg9G/pCvwFd/U9WSs+njpD0f2hL6dR ezxzzHRCI6HOzJiK4/+C8yJ9nrM3KC6ApkgGFwNrs3BcGJNxl7iuoiDNEF/5xlJe05 s0JWOkVbjMytQ== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id 6B1B9CE13CE; Tue, 14 Jan 2025 10:42:30 -0800 (PST) Date: Tue, 14 Jan 2025 10:42:30 -0800 From: "Paul E. McKenney" To: Chen Ridong Cc: Vlastimil Babka , Michal Hocko , Andrew Morton , hannes@cmpxchg.org, yosryahmed@google.com, roman.gushchin@linux.dev, shakeel.butt@linux.dev, muchun.song@linux.dev, davidf@vimeo.com, handai.szj@taobao.com, rientjes@google.com, kamezawa.hiroyu@jp.fujitsu.com, RCU , linux-mm@kvack.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, chenridong@huawei.com, wangweiyang2@huawei.com Subject: Re: [PATCH v3] memcg: fix soft lockup in the OOM process Message-ID: <81294730-1cd9-4793-b886-9ababe29d071@paulmck-laptop> Reply-To: paulmck@kernel.org References: <20241224025238.3768787-1-chenridong@huaweicloud.com> <1ea309c1-d0f8-4209-b0b0-e69ad4e986ae@suse.cz> <58caaa4f-cf78-4d0f-af31-8a9277b6ebf5@huaweicloud.com> <20250113194546.3de1af46fa7a668111909b63@linux-foundation.org> <0b6a3935-8b6c-4d11-bacc-31c1ba15b349@huaweicloud.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <0b6a3935-8b6c-4d11-bacc-31c1ba15b349@huaweicloud.com> X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 5CE0640004 X-Rspam-User: X-Stat-Signature: krixjxw4sgqwhcxdxeyyw4b49tz1d9ax X-HE-Tag: 1736880152-958691 X-HE-Meta: U2FsdGVkX1/F8u5M4fXQTsfQ7kj2/3U7qMDY9BON8OLSEFU79x1L+VPy2tnQyd4B2D9BHMvabCzCKefD1+LnlceUIZJVBLjhjEgro/8wsFdN9s5Xs23f/jYC60snILb3IWfXM/nkonq6Li0aHWpe6IhN4Q5fVefH1gL6iXHgD2ELXDPxxeyGvTU0ukisjSVkN2wtp0Qaq8aBKGa6ewb7N5l6kqTp4l3m1CluPcwrMHg8w+xb4clOi67sZh60N5VqXHo8YKyqY4eQWKzmyZvCHD6uZMjZN5yzdAbt/+TY7y68eCSlA+nbV5vngIwORlGNoog4Pk+T1n7yUAaINjGl8moQQcIzbgAqIOsa2IdMfi8WgZU6mkv0a6RbrsTnGTEbRq6kwEwb4qJTAscdcz2xRrKspxuWu0GozY1oxoRXuLUB11w2UmXqQ5ZYv22mYxboDSzSnJjTB2dKIUncp7h1S+hTn/7hdojhmqO2xRECf3B0e8CA9NyV6HNqg9CDOlhtxTSQX9cjrNaBAwomITy92fK523EKKMpgThNZbby2FuuSy9aKTfH1E4MH5DtCY1XHs5qPb2tVmHSkZrZDV3s36ZlvbX3Myrwh8Mm2ya3CzgUolKiZZZqKIOvDhZtan5yJFKUIbqwQIKZI8vUKSu/e7BvrZ2EAJU9tx8T+IbcVGtd0vZ1mcuSp691vjz3r07sK6M87wVpBFHjRP9LuZujzrv460YQTcphq0R2iCNAMhxtqvJeZWagp1y59wBf01KVB+OolwcXF1LDFPusRqvap2uu2zM3CVCyFLcFgtX8TwfIWUlEcfBT8lULG59Y0Rjdx0h6xRHx+B1i2sA5xDSHVFe7GrD+QOIXlzdk/SuM/9kJZarq6RyNKvrEwUnHb036MzqaUdl6ulnm7J4Gb4KnDk29Mo60svWGowhxsWUrRyF3rn4MDDWbIU3HQlXrY7a0V3oC8/RviJI6VCK5x9SU z2ct0MzE P6CFF4W/ZLx8rBjaBFv0ZSZy+3FtZGXCYjbKUUJd0RKvLf3i4bE81Riho8Mom6zjzmj1FM1KXqKzHiWderW/L13SaEG8+N6OQMi5zAWeAFOH80HRMV+dO9D415EAQ019haPxODxu+kso+JE1UC2fB9YGFcUFnn0Hy1UIPZBM1pi69wL4vlmRyncMS2MXb7kReUi3pUhF+Ue7d1e2ya5hHDB/xZ4qOa3Sv14TnCYfQdutjr8/FxlvH6Ylqfzc4JM6bxFjlH6LB6p7uHpeLecpkbW0UwPkSXuQBdBcTbBdCru1JIILD06h+50iz+SEkmkM1bJh9t5f7PGiwD/ySaCydyiSd1cRksGUx3qwMXa+Pk17tTA5wyKYmGcPXuNMaIxszmCh/oXDI5bU+MxUWPKH0KwMF5na/7gfM27zuDtB0WD813F025Z61nadZKL7w5Vkn0Y1d1F4h5OU6x718MWJ6WzRaxEqfgdyQkMJhI0bSjjxBg1mtTy45UuXD5Wsz0owI+THqW3dlMJjXlFhSxx5jel859de3sm2xnF9j0p9puYbrad9DUOTuLbfPMDq+00NWfKlHNcyyDONdgGfunpU54qZySW5CRNmyU78x9XCFAeVvS1IuOHD+39i2IRfDEuEO6Psx X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, Jan 14, 2025 at 08:13:37PM +0800, Chen Ridong wrote: > > > On 2025/1/14 17:20, Vlastimil Babka wrote: > > On 1/14/25 09:40, Michal Hocko wrote: > >> On Mon 13-01-25 19:45:46, Andrew Morton wrote: > >>> On Mon, 13 Jan 2025 14:51:55 +0800 Chen Ridong wrote: > >>> > >>>>>> @@ -430,10 +431,15 @@ static void dump_tasks(struct oom_control *oc) > >>>>>> mem_cgroup_scan_tasks(oc->memcg, dump_task, oc); > >>>>>> else { > >>>>>> struct task_struct *p; > >>>>>> + int i = 0; > >>>>>> > >>>>>> rcu_read_lock(); > >>>>>> - for_each_process(p) > >>>>>> + for_each_process(p) { > >>>>>> + /* Avoid potential softlockup warning */ > >>>>>> + if ((++i & 1023) == 0) > >>>>>> + touch_softlockup_watchdog(); > >>>>> > >>>>> This might suppress the soft lockup, but won't a rcu stall still be detected? > >>>> > >>>> Yes, rcu stall was still detected. > > > > "was" or "would be"? I thought only the memcg case was observed, or was that > > some deliberate stress test of the global case? (or the pr_info() console > > stress test mentioned earlier, but created outside of the oom code?) > > > > It's not easy to reproduce for global OOM. Because the pr_info() console > stress test can also lead to other softlockups or RCU warnings(not > causeed by OOM process) because the whole system is struggling.However, > if I add mdelay(1) in the dump_task() function (just to slow down > dump_task, assuming this is slowed by pr_info()) and trigger a global > OOM, RCU warnings can be observed. > > I think this can verify that global OOM can trigger RCU warnings in the > specific scenarios. We do have a recently upstreamed rcutree.csd_lock_suppress_rcu_stall kernel boot parameter that causes RCU CPU stall warnings to suppress most of the output when there is an ongoing CSD-lock stall. Would it make sense to do something similar when the system is in OOM, give or take the traditional difficulty of determining exactly when OOM starts and ends? 1dd01c06506c ("rcu: Summarize RCU CPU stall warnings during CSD-lock stalls") Thanx, Paul > >>>> For global OOM, system is likely to struggle, do we have to do some > >>>> works to suppress RCU detete? > >>> > >>> rcu_cpu_stall_reset()? > >> > >> Do we really care about those? The code to iterate over all processes > >> under RCU is there (basically) since ever and yet we do not seem to have > >> many reports of stalls? Chen's situation is specific to memcg OOM and > >> touching the global case was mostly for consistency reasons. > > > > Then I'd rather not touch the global case then if it's theoretical? It's not > > even exactly consistent, given it's a cond_resched() in the memcg code (that > > can be eventually automatically removed once/if lazy preempt becomes the > > sole implementation), but the touch_softlockup_watchdog() would remain, > > while doing only half of the job? > >