From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC15FC433B4 for ; Fri, 2 Apr 2021 07:03:50 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 6561261103 for ; Fri, 2 Apr 2021 07:03:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6561261103 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BFBA16B0071; Fri, 2 Apr 2021 03:03:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B83A46B0072; Fri, 2 Apr 2021 03:03:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9D7056B0073; Fri, 2 Apr 2021 03:03:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0197.hostedemail.com [216.40.44.197]) by kanga.kvack.org (Postfix) with ESMTP id 7B46B6B0071 for ; Fri, 2 Apr 2021 03:03:49 -0400 (EDT) Received: from smtpin33.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 3A7BFFB51 for ; Fri, 2 Apr 2021 07:03:49 +0000 (UTC) X-FDA: 77986536978.33.7F9A449 Received: from mail-yb1-f179.google.com (mail-yb1-f179.google.com [209.85.219.179]) by imf13.hostedemail.com (Postfix) with ESMTP id 8E92BE000102 for ; Fri, 2 Apr 2021 07:03:47 +0000 (UTC) Received: by mail-yb1-f179.google.com with SMTP id z1so4263168ybf.6 for ; Fri, 02 Apr 2021 00:03:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc; bh=ChLU3i2ozhkWUPiactWWbhoY/VIPI1ZBVxjZRPuvVi4=; b=jXX/8/fZKiV5ztogepB8iDNp7hDkOpxscSS7MjSd2d/GDYXHFdIQoGeYuibqhUSifZ DUln0fVjOtgydKeRBmlDVqRWKYkN0EjBfAoKRvMjxJJ/AkRKPbFQU3MBuhzrTLdOxmvD t9rO4AUOr5YTpOEv45wa4C5GPBkv3MjouRQSBgkeU+7XmRqRfF/weZ9NkLsm9/t6lhdd wtadKXIUvaHNlNodtPg6E7N4BvtmHoU8+57pCkV5pdqRH9FZSD+/e/XsdNhpXaSI+r62 6hBDYY+Jp1h+/N/OTk6SQLaT+WubUj/xnlcb0U9WMdCv5+rhV7YEYesPB46FZDWbBrAk 0nIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=ChLU3i2ozhkWUPiactWWbhoY/VIPI1ZBVxjZRPuvVi4=; b=MYDxyMuJGGbD228TCas2gdmjubCX51dXEAlT9p5wf2w6Ht1PdJYIouxkciOQrJBYrt U5Vjg8ljfgBKeZ9s4XF8KzA2O7fHOFKE56v0YFpofho0F5HozdOVDTxVCoVuDTCRzoB/ XXSII6On0nrpdo5tE1zOCyyhc7GolnIlXSHpBhCtX75rutTyxhcF4rSt59oaGy8WXXyA AILuaM6dSYW/1QAQa8GoF41m0i5Tt6YEJ5aP2/6uPYXXUqeTSvW81vlbGFfUa9cGch3O eY4jZAvViQEjwFlqO5aGWFeqV/2l8pLVP6+JaezNxDlaMMK/MhiwrkUBMwuARZLZHY0I xMzg== X-Gm-Message-State: AOAM531ay4DwtIO+9ATRDnVkWZQpYSjC+eo+eOF632flpWFwnt/GoG9e 4kllGQFipnkM7JHw4JJp10vbEWkwVfpnLeKzhvY= X-Google-Smtp-Source: ABdhPJyzdoUJA13V2kTCpONkFzmvz6JDuvN8Os/JJYlYpENcjDm3+F5rPlijwmL9H6X08WAg1vqzth3FL6wfUm32at0= X-Received: by 2002:a25:6a88:: with SMTP id f130mr17007762ybc.234.1617347028240; Fri, 02 Apr 2021 00:03:48 -0700 (PDT) MIME-Version: 1.0 From: Stillinux Date: Fri, 2 Apr 2021 15:03:37 +0800 Message-ID: Subject: [RFC PATCH] mm/swap: fix system stuck due to infinite loop To: akpm@linux-foundation.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, liuzhengyuan@kylinos.cn, liuyun01@kylinos.cn Content-Type: multipart/alternative; boundary="00000000000018cb7805bef7f3f6" X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 8E92BE000102 X-Stat-Signature: ftz5p475x6fqpaofywh87tebb5sy7arb Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf13; identity=mailfrom; envelope-from=""; helo=mail-yb1-f179.google.com; client-ip=209.85.219.179 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1617347027-884560 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: --00000000000018cb7805bef7f3f6 Content-Type: text/plain; charset="UTF-8" In the case of high system memory and load pressure, we ran ltp test and found that the system was stuck, the direct memory reclaim was all stuck in io_schedule, the waiting request was stuck in the blk_plug flow of one process, and this process fell into an infinite loop. not do the action of brushing out the request. The call flow of this process is swap_cluster_readahead. Use blk_start/finish_plug for blk_plug operation, flow swap_cluster_readahead->__read_swap_cache_async->swapcache_prepare. When swapcache_prepare return -EEXIST, it will fall into an infinite loop, even if cond_resched is called, but according to the schedule, sched_submit_work will be based on tsk->state, and will not flash out the blk_plug request, so will hang io, causing the overall system hang. For the first time involving the swap part, there is no good way to fix the problem from the fundamental problem. In order to solve the engineering situation, we chose to make swap_cluster_readahead aware of the memory pressure situation as soon as possible, and do io_schedule to flush out the blk_plug request, thereby changing the allocation flag in swap_readpage to GFP_NOIO , No longer do the memory reclaim of flush io. Although system operating normally, but not the most fundamental way. Signed-off-by: huangjinhui --- mm/page_io.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/page_io.c b/mm/page_io.c index c493ce9ebcf5..87392ffabb12 100644 --- a/mm/page_io.c +++ b/mm/page_io.c @@ -403,7 +403,7 @@ int swap_readpage(struct page *page, bool synchronous) } ret = 0; - bio = bio_alloc(GFP_KERNEL, 1); + bio = bio_alloc(GFP_NOIO, 1); bio_set_dev(bio, sis->bdev); bio->bi_opf = REQ_OP_READ; bio->bi_iter.bi_sector = swap_page_sector(page); -- 2.25.1 --00000000000018cb7805bef7f3f6 Content-Type: text/html; charset="UTF-8" 
In the case of high system memory and load pressure, we ran ltp test
and found that the system was stuck, the direct memory reclaim was
all stuck in io_schedule, the waiting request was stuck in the blk_plug
flow of one process, and this process fell into an infinite loop.
not do the action of brushing out the request.

The call flow of this process is swap_cluster_readahead.
Use blk_start/finish_plug for blk_plug operation,
flow swap_cluster_readahead->__read_swap_cache_async->swapcache_prepare.
When swapcache_prepare return -EEXIST, it will fall into an infinite loop,
even if cond_resched is called, but according to the schedule,
sched_submit_work will be based on tsk->state, and will not flash out
the blk_plug request, so will hang io, causing the overall system  hang.

For the first time involving the swap part, there is no good way to fix
the problem from the fundamental problem. In order to solve the
engineering situation, we chose to make swap_cluster_readahead aware of
the memory pressure situation as soon as possible, and do io_schedule to
flush out the blk_plug request, thereby changing the allocation flag in
swap_readpage to GFP_NOIO , No longer do the memory reclaim of flush io.
Although system operating normally, but not the most fundamental way.

Signed-off-by: huangjinhui <huangjinhui@kylinos.cn>
---
 mm/page_io.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/page_io.c b/mm/page_io.c
index c493ce9ebcf5..87392ffabb12 100644
--- a/mm/page_io.c
+++ b/mm/page_io.c
@@ -403,7 +403,7 @@ int swap_readpage(struct page *page, bool synchronous)
 	}
 
 	ret = 0;
-	bio = bio_alloc(GFP_KERNEL, 1);
+	bio = bio_alloc(GFP_NOIO, 1);
 	bio_set_dev(bio, sis->bdev);
 	bio->bi_opf = REQ_OP_READ;
 	bio->bi_iter.bi_sector = swap_page_sector(page);
-- 
2.25.1
--00000000000018cb7805bef7f3f6--