From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2AFCC46466 for ; Mon, 21 Sep 2020 18:06:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3B1872071A for ; Mon, 21 Sep 2020 18:06:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=cisco.com header.i=@cisco.com header.b="VOnsLu89" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3B1872071A Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=cisco.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id A0F289000A4; Mon, 21 Sep 2020 14:06:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9C07F90008B; Mon, 21 Sep 2020 14:06:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8FE4A9000A4; Mon, 21 Sep 2020 14:06:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0249.hostedemail.com [216.40.44.249]) by kanga.kvack.org (Postfix) with ESMTP id 79FD790008B for ; Mon, 21 Sep 2020 14:06:16 -0400 (EDT) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 436E8180AD815 for ; Mon, 21 Sep 2020 18:06:16 +0000 (UTC) X-FDA: 77287847952.23.tax09_04180f227147 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin23.hostedemail.com (Postfix) with ESMTP id 1EE0C37604 for ; Mon, 21 Sep 2020 18:06:16 +0000 (UTC) X-HE-Tag: tax09_04180f227147 X-Filterd-Recvd-Size: 4842 Received: from alln-iport-1.cisco.com (alln-iport-1.cisco.com [173.37.142.88]) by imf37.hostedemail.com (Postfix) with ESMTP for ; Mon, 21 Sep 2020 18:06:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=2781; q=dns/txt; s=iport; t=1600711575; x=1601921175; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=GFYSntlMbW/qItAAXl9Nwbk/9D2EBJzUVKqIAKTYdWw=; b=VOnsLu89PEkr+BQl1nHFyTl/MG3z3n1uC7AtwippNg0YdtGL5wuYDJ1m vtqu2oBu1qnmhYOBKAzAX9mVz/1Msr2704sWF1t3f09VWNRrTzfu/iwCb R75uB4CRsXHcNsdbVZfgz2sadvAgADNJ9rbQumCuCOPhiTRNso5cy1tw3 Y=; X-IronPort-AV: E=Sophos;i="5.77,287,1596499200"; d="scan'208";a="546077107" Received: from alln-core-11.cisco.com ([173.36.13.133]) by alln-iport-1.cisco.com with ESMTP/TLS/DHE-RSA-SEED-SHA; 21 Sep 2020 18:06:13 +0000 Received: from sjc-ads-9087.cisco.com (sjc-ads-9087.cisco.com [10.30.208.97]) by alln-core-11.cisco.com (8.15.2/8.15.2) with ESMTP id 08LI6CwR018277; Mon, 21 Sep 2020 18:06:13 GMT Received: by sjc-ads-9087.cisco.com (Postfix, from userid 396877) id CA57FB66; Mon, 21 Sep 2020 11:06:12 -0700 (PDT) From: Julius Hemanth Pitti To: gregkh@linuxfoundation.org, akpm@linux-foundation.org, xlpang@linux.alibaba.com, mhocko@suse.com, vdavydov.dev@gmail.com, ktkhai@virtuozzo.com, hannes@cmpxchg.org Cc: stable@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, xe-linux-external@cisco.com, Linus Torvalds , Julius Hemanth Pitti Subject: [PATCH stable v4.19] mm: memcg: fix memcg reclaim soft lockup Date: Mon, 21 Sep 2020 11:06:10 -0700 Message-Id: <20200921180610.62242-1-jpitti@cisco.com> X-Mailer: git-send-email 2.19.1 MIME-Version: 1.0 X-Auto-Response-Suppress: DR, OOF, AutoReply X-Outbound-SMTP-Client: 10.30.208.97, sjc-ads-9087.cisco.com X-Outbound-Node: alln-core-11.cisco.com Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Xunlei Pang commit e3336cab2579012b1e72b5265adf98e2d6e244ad upstream We've met softlockup with "CONFIG_PREEMPT_NONE=3Dy", when the target memc= g doesn't have any reclaimable memory. It can be easily reproduced as below: watchdog: BUG: soft lockup - CPU#0 stuck for 111s![memcg_test:2204] CPU: 0 PID: 2204 Comm: memcg_test Not tainted 5.9.0-rc2+ #12 Call Trace: shrink_lruvec+0x49f/0x640 shrink_node+0x2a6/0x6f0 do_try_to_free_pages+0xe9/0x3e0 try_to_free_mem_cgroup_pages+0xef/0x1f0 try_charge+0x2c1/0x750 mem_cgroup_charge+0xd7/0x240 __add_to_page_cache_locked+0x2fd/0x370 add_to_page_cache_lru+0x4a/0xc0 pagecache_get_page+0x10b/0x2f0 filemap_fault+0x661/0xad0 ext4_filemap_fault+0x2c/0x40 __do_fault+0x4d/0xf9 handle_mm_fault+0x1080/0x1790 It only happens on our 1-vcpu instances, because there's no chance for oom reaper to run to reclaim the to-be-killed process. Add a cond_resched() at the upper shrink_node_memcgs() to solve this issue, this will mean that we will get a scheduling point for each memcg in the reclaimed hierarchy without any dependency on the reclaimable memory in that memcg thus making it more predictable. [jpitti@cisco.com: - backported to v4.19.y - Upstream patch applies fix in shrink_node_memcgs(), which is not present to v4.19.y. Appled to shrink_node()] Suggested-by: Michal Hocko Signed-off-by: Xunlei Pang Signed-off-by: Andrew Morton Acked-by: Chris Down Acked-by: Michal Hocko Acked-by: Johannes Weiner Link: http://lkml.kernel.org/r/1598495549-67324-1-git-send-email-xlpang@l= inux.alibaba.com Signed-off-by: Linus Torvalds Fixes: b0dedc49a2da ("mm/vmscan.c: iterate only over charged shrinkers du= ring memcg shrink_slab()") Cc: stable@vger.kernel.org Signed-off-by: Julius Hemanth Pitti --- mm/vmscan.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/mm/vmscan.c b/mm/vmscan.c index bc2ecd43251a..b93dc8fc6007 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -2708,6 +2708,14 @@ static bool shrink_node(pg_data_t *pgdat, struct s= can_control *sc) unsigned long reclaimed; unsigned long scanned; =20 + /* + * This loop can become CPU-bound when target memcgs + * aren't eligible for reclaim - either because they + * don't have any reclaimable pages, or because their + * memory is explicitly protected. Avoid soft lockups. + */ + cond_resched(); + switch (mem_cgroup_protected(root, memcg)) { case MEMCG_PROT_MIN: /* --=20 2.17.1