From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42AF5C433DF for ; Wed, 26 Aug 2020 08:11:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0E16520825 for ; Wed, 26 Aug 2020 08:11:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0E16520825 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 92CEA6B0003; Wed, 26 Aug 2020 04:11:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8DE0F6B0005; Wed, 26 Aug 2020 04:11:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7F3616B0006; Wed, 26 Aug 2020 04:11:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0021.hostedemail.com [216.40.44.21]) by kanga.kvack.org (Postfix) with ESMTP id 692686B0003 for ; Wed, 26 Aug 2020 04:11:05 -0400 (EDT) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 2194C3630 for ; Wed, 26 Aug 2020 08:11:05 +0000 (UTC) X-FDA: 77191999290.30.day16_1102c6d27062 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin30.hostedemail.com (Postfix) with ESMTP id E67DF180B3AA7 for ; Wed, 26 Aug 2020 08:11:04 +0000 (UTC) X-HE-Tag: day16_1102c6d27062 X-Filterd-Recvd-Size: 3047 Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by imf48.hostedemail.com (Postfix) with ESMTP for ; Wed, 26 Aug 2020 08:11:04 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.221.27]) by mx2.suse.de (Postfix) with ESMTP id 795CDAD36; Wed, 26 Aug 2020 08:11:34 +0000 (UTC) Date: Wed, 26 Aug 2020 10:11:02 +0200 From: Michal Hocko To: Xunlei Pang Cc: Johannes Weiner , Andrew Morton , Vladimir Davydov , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH] mm: memcg: Fix memcg reclaim soft lockup Message-ID: <20200826081102.GM22869@dhcp22.suse.cz> References: <1598426822-93737-1-git-send-email-xlpang@linux.alibaba.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1598426822-93737-1-git-send-email-xlpang@linux.alibaba.com> X-Rspamd-Queue-Id: E67DF180B3AA7 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam02 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed 26-08-20 15:27:02, Xunlei Pang wrote: > We've met softlockup with "CONFIG_PREEMPT_NONE=y", when > the target memcg doesn't have any reclaimable memory. Do you have any scenario when this happens or is this some sort of a test case? > It can be easily reproduced as below: > watchdog: BUG: soft lockup - CPU#0 stuck for 111s![memcg_test:2204] > CPU: 0 PID: 2204 Comm: memcg_test Not tainted 5.9.0-rc2+ #12 > Call Trace: > shrink_lruvec+0x49f/0x640 > shrink_node+0x2a6/0x6f0 > do_try_to_free_pages+0xe9/0x3e0 > try_to_free_mem_cgroup_pages+0xef/0x1f0 > try_charge+0x2c1/0x750 > mem_cgroup_charge+0xd7/0x240 > __add_to_page_cache_locked+0x2fd/0x370 > add_to_page_cache_lru+0x4a/0xc0 > pagecache_get_page+0x10b/0x2f0 > filemap_fault+0x661/0xad0 > ext4_filemap_fault+0x2c/0x40 > __do_fault+0x4d/0xf9 > handle_mm_fault+0x1080/0x1790 > > It only happens on our 1-vcpu instances, because there's no chance > for oom reaper to run to reclaim the to-be-killed process. > > Add cond_resched() in such cases at the beginning of shrink_lruvec() > to give up the cpu to others. I do agree that we need a cond_resched but I cannot say I would like this patch. The primary reason is that it doesn't catch all cases when the memcg is not reclaimable. For example it wouldn't reschedule if the memcg is protected by low/min. What do you think about this instead? diff --git a/mm/vmscan.c b/mm/vmscan.c index 99e1796eb833..bbdc38b58cc5 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -2617,6 +2617,8 @@ static void shrink_node_memcgs(pg_data_t *pgdat, struct scan_control *sc) mem_cgroup_calculate_protection(target_memcg, memcg); + cond_resched(); + if (mem_cgroup_below_min(memcg)) { /* * Hard protection. This should catch both cases. I even have a vague recollection that somebody has proposed something in that direction but I cannot remember what has happened with that patch. -- Michal Hocko SUSE Labs