From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E5CFC43334 for ; Fri, 22 Jul 2022 07:19:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8EE4A6B0072; Fri, 22 Jul 2022 03:19:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 89E678E0001; Fri, 22 Jul 2022 03:19:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 767586B0075; Fri, 22 Jul 2022 03:19:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 6612B6B0072 for ; Fri, 22 Jul 2022 03:19:29 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 42D3F1A0BEA for ; Fri, 22 Jul 2022 07:19:29 +0000 (UTC) X-FDA: 79713885258.25.417AFE5 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by imf05.hostedemail.com (Postfix) with ESMTP id B6837100077 for ; Fri, 22 Jul 2022 07:19:27 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 4A4471FA6F; Fri, 22 Jul 2022 07:19:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1658474366; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=t3VBXpPoYCKHzFWqve75hDICbONfr+sYnc6fZg2ES9M=; b=PKdyTGXKEJZmz1b9UajtM5gwwsIKjWlmG7PvzgtT16/wslzCpbIAZ2VstVPRecVVZjmBNR d07RELSVBFRxPo/RtG/PpaTrYSE9O7lMYK5Oj99xRbn1Wx2iK4JeYnJTz0WP6GY2CW0UiC 7802qem+YzNBVkC+4Ac/32ld9IcP9Mc= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 380DE134A9; Fri, 22 Jul 2022 07:19:26 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id RSRfDH5P2mIJKwAAMHmgww (envelope-from ); Fri, 22 Jul 2022 07:19:26 +0000 Date: Fri, 22 Jul 2022 09:19:25 +0200 From: Michal Hocko To: Tetsuo Handa Cc: Johannes Weiner , Andrew Morton , linux-mm Subject: Re: [PATCH v2] mm: memcontrol: fix potential oom_lock recursion deadlock Message-ID: References: <000000000000471c2905e3c2c2c2@google.com> <20220714141813.yi5p4o2tiyvkao6b@quack3> <534fa596-0c29-0f1e-b292-53ad9c3dbbe3@I-love.SAKURA.ne.jp> <20220715013908.ayyimue5yhfwonho@google.com> <03304bf8-d153-698f-0376-9e9a0ec1048e@I-love.SAKURA.ne.jp> <834b896d-68fb-caeb-4316-2e0a2190e3eb@I-love.SAKURA.ne.jp> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <834b896d-68fb-caeb-4316-2e0a2190e3eb@I-love.SAKURA.ne.jp> ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1658474367; a=rsa-sha256; cv=none; b=o5xRFERJJwLMxBbU+rDO3apdguPJ+95hDb2oSWGlO4Pkz4KczSr2HMwCYX3ZC/WRNL+KOw CONKjxRkShiRXHbbqutaEH2WAGyNclKq4rSUw1G8rHDIqJWfgzcddAD5zIq42DugfW6Tes L9Qkx3a+b0GDxlP90IsT4xpDx3a4AxE= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=PKdyTGXK; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf05.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.29 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1658474367; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=t3VBXpPoYCKHzFWqve75hDICbONfr+sYnc6fZg2ES9M=; b=3pnNGvCvHF5wM6O2Dn4D4kpcCPc9pBW95mrW2wnIBMJPs2R0y36iIbGNNCjyUxc0uwZWkr ljDCsz3tjccoCcgScgLh+kI59hkcnJASK0xsMo5Bpiv0qTk+dGWuZTReCXNNV4oCfzYbRc H4Lwy8mUz4lwseDgXdOQ6Pa51YzGxJQ= Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=PKdyTGXK; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf05.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.29 as permitted sender) smtp.mailfrom=mhocko@suse.com X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: B6837100077 X-Stat-Signature: o9kkupity6himm9g9q1ebqxrwjepiu4b X-Rspam-User: X-HE-Tag: 1658474367-56732 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri 22-07-22 09:46:27, Tetsuo Handa wrote: > syzbot is reporting GFP_KERNEL allocation with oom_lock held [1]. We > must make sure that such allocation won't hit __alloc_pages_may_oom() > path which will retry forever if oom_lock is already held. Use static > buffer when oom_lock is already held. The changelog is rather cryptic. Your previous one was more readable. I would go with: " syzbot is reporting GFP_KERNEL allocation with oom_lock held [1] when reporting memcg oom. This is problematic because this creates a dependency between GFP_NOFS and GFP_KERNEL over oom_lock which could dead lock the system. Fix the problem by removing the allocation from memory_stat_format completely. Use a statically preallocated buffer instead for this path. OOM dumping is synchronized by the oom_lock so there is no exclusion required here. memory_stat_show can use GFP_KERNEL allocation. " > Link: https://syzkaller.appspot.com/bug?extid=2d2aeadc6ce1e1f11d45 [1] > Reported-by: syzbot > Suggested-by: Michal Hocko > Fixes: c8713d0b23123759 ("mm: memcontrol: dump memory.stat during cgroup OOM") > Signed-off-by: Tetsuo Handa Acked-by: Michal Hocko Thanks! > --- > Changes in v2: > Use static buffer for OOM reporting, suggested by Michal Hocko . > > mm/memcontrol.c | 22 +++++++++------------- > 1 file changed, 9 insertions(+), 13 deletions(-) > > diff --git a/mm/memcontrol.c b/mm/memcontrol.c > index 618c366a2f07..8092be2fbb7c 100644 > --- a/mm/memcontrol.c > +++ b/mm/memcontrol.c > @@ -1460,14 +1460,12 @@ static inline unsigned long memcg_page_state_output(struct mem_cgroup *memcg, > return memcg_page_state(memcg, item) * memcg_page_state_unit(item); > } > > -static char *memory_stat_format(struct mem_cgroup *memcg) > +static void memory_stat_format(struct mem_cgroup *memcg, char *buf, int bufsize) > { > struct seq_buf s; > int i; > > - seq_buf_init(&s, kmalloc(PAGE_SIZE, GFP_KERNEL), PAGE_SIZE); > - if (!s.buffer) > - return NULL; > + seq_buf_init(&s, buf, bufsize); > > /* > * Provide statistics on the state of the memory subsystem as > @@ -1533,8 +1531,6 @@ static char *memory_stat_format(struct mem_cgroup *memcg) > > /* The above should easily fit into one page */ > WARN_ON_ONCE(seq_buf_has_overflowed(&s)); > - > - return s.buffer; > } > > #define K(x) ((x) << (PAGE_SHIFT-10)) > @@ -1570,7 +1566,10 @@ void mem_cgroup_print_oom_context(struct mem_cgroup *memcg, struct task_struct * > */ > void mem_cgroup_print_oom_meminfo(struct mem_cgroup *memcg) > { > - char *buf; > + /* Use static buffer, for the caller is holding oom_lock. */ > + static char buf[PAGE_SIZE]; > + > + lockdep_assert_held(&oom_lock); > > pr_info("memory: usage %llukB, limit %llukB, failcnt %lu\n", > K((u64)page_counter_read(&memcg->memory)), > @@ -1591,11 +1590,8 @@ void mem_cgroup_print_oom_meminfo(struct mem_cgroup *memcg) > pr_info("Memory cgroup stats for "); > pr_cont_cgroup_path(memcg->css.cgroup); > pr_cont(":"); > - buf = memory_stat_format(memcg); > - if (!buf) > - return; > + memory_stat_format(memcg, buf, sizeof(buf)); > pr_info("%s", buf); > - kfree(buf); > } > > /* > @@ -6335,11 +6331,11 @@ static int memory_events_local_show(struct seq_file *m, void *v) > static int memory_stat_show(struct seq_file *m, void *v) > { > struct mem_cgroup *memcg = mem_cgroup_from_seq(m); > - char *buf; > + char *buf = kmalloc(PAGE_SIZE, GFP_KERNEL); > > - buf = memory_stat_format(memcg); > if (!buf) > return -ENOMEM; > + memory_stat_format(memcg, buf, PAGE_SIZE); > seq_puts(m, buf); > kfree(buf); > return 0; > -- > 2.18.4 > -- Michal Hocko SUSE Labs