From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ABCD9C433ED for ; Mon, 26 Apr 2021 22:56:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3BF8A61151 for ; Mon, 26 Apr 2021 22:56:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3BF8A61151 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 489C96B0036; Mon, 26 Apr 2021 18:56:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 43A0D6B006E; Mon, 26 Apr 2021 18:56:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2DA306B0070; Mon, 26 Apr 2021 18:56:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0102.hostedemail.com [216.40.44.102]) by kanga.kvack.org (Postfix) with ESMTP id 09AD96B0036 for ; Mon, 26 Apr 2021 18:56:51 -0400 (EDT) Received: from smtpin12.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id A970C362C for ; Mon, 26 Apr 2021 22:56:50 +0000 (UTC) X-FDA: 78076029780.12.A3DC807 Received: from mail-ed1-f44.google.com (mail-ed1-f44.google.com [209.85.208.44]) by imf08.hostedemail.com (Postfix) with ESMTP id 6BA1280192EB for ; Mon, 26 Apr 2021 22:56:28 +0000 (UTC) Received: by mail-ed1-f44.google.com with SMTP id h10so67667034edt.13 for ; Mon, 26 Apr 2021 15:56:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Mlc9M23HPrqoa6cq8YMD/YOr+XHLyqpTtPtD3sQanDc=; b=LqmlRtHkanGDoQJlT3hxiEQ/VvDGDUvI/t+RwhQahj82KYxpFqUaO7Vlt3zJLipOq2 s4SGyECjOCsoVrLYc4R2B5zag5j/6253C40oAvAFsxPCjchVAQbyHOlNM+eZ9qaDLtAr 52rKBvd6m2gBvvOqEE/3JnbdU8MWzzAhXsF4vJ8fbU+gD4swKEYeBxTs0g5bDu9vAVyH PO27hF027Z7AVTYqdEneewGjOWdVrKZMO19+6R7hoAi4ugUIdZWNMRVvMMlz4tW1PD9e 3FlEtz9nWhp6NM4zil9vVqWDDsVDN23NMrZpEvt7PeFd6k3P5kJg3CfnJW+cNc/1S34c W3OQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Mlc9M23HPrqoa6cq8YMD/YOr+XHLyqpTtPtD3sQanDc=; b=mE1j3UPjtiJNVzayhgnBYq7gD/8AJGf5RFqhbFMMVzu666lADLTDLwxXfVfsAZYcRK TNQRMn47H7lk42/shbCNSsCBoyud/gc6TOg6B495GFBlQ/LYoFqh7pp/nDiF8R3OPS9t LEI9mkIAGbwMpeMyQUEnRsJmLfMrCNCHEHVdN8tUxtCxvrnIhHBNquKXPsulVtqGlnKn cNQF5syO1DUovkF0vxEBhXfymHCx992wT8xxj4KpT5peviolLd6yN3KtwM3CjmCG8pCF yXuIZWIcDMYp5iVh9JjvA+lW6Trxjf/Xo/0RZ4UqCrTdQ9SxM073Tk3c2Z/aqtEwttfs MNXw== X-Gm-Message-State: AOAM530kbTdAXIRbthw8hjG23DXmgps76Ei5+3/OcKCXeNqzMx+sLMLz EJ2SMNDtqqQXVuPTGV8OqcE7wwQIbpkwh+k4s+0= X-Google-Smtp-Source: ABdhPJxNQIkNgXziFYpK79X6zWJ67Hc3CwYe24wHReEUFV+0ts5V5NDWE2dp5jOj1TSWEGPXBQ7CU1bx8oudbmecHc0= X-Received: by 2002:a05:6402:348d:: with SMTP id v13mr1030645edc.294.1619477809041; Mon, 26 Apr 2021 15:56:49 -0700 (PDT) MIME-Version: 1.0 References: <20210423160753.6A51.409509F4@e16-tech.com> <20210424132826.89B1.409509F4@e16-tech.com> In-Reply-To: <20210424132826.89B1.409509F4@e16-tech.com> From: Yang Shi Date: Mon, 26 Apr 2021 15:56:36 -0700 Message-ID: Subject: Re: kernel BUG at mm/huge_memory.c:2736(linux 5.10.29) To: Wang Yugui Cc: "Kirill A. Shutemov" , Linux MM Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 6BA1280192EB X-Stat-Signature: 1qkw1u966az66mhbwqekm78hqy734xoe X-Rspamd-Server: rspam02 Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf08; identity=mailfrom; envelope-from=""; helo=mail-ed1-f44.google.com; client-ip=209.85.208.44 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1619477788-66276 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Apr 23, 2021 at 10:28 PM Wang Yugui wrote: > > Hi, > > > On Fri, Apr 23, 2021 at 1:07 AM Wang Yugui wrote: > > > > > > Hi, > > > > > > > With this patch, the problem yet not happen after 4 tests(5.10.x). > > > > > > With this patch , another problem happened at 6th test. > > > > > > kernel BUG at mm/huge_memory.c:2343! > > > static void unmap_page(struct page *page) > > > { > > > enum ttu_flags ttu_flags = TTU_IGNORE_MLOCK | > > > TTU_RMAP_LOCKED | TTU_SPLIT_HUGE_PMD; > > > bool unmap_success; > > > > > > VM_BUG_ON_PAGE(!PageHead(page), page); > > > > > > if (PageAnon(page)) > > > ttu_flags |= TTU_SPLIT_FREEZE; > > > > > > unmap_success = try_to_unmap(page, ttu_flags); > > > L2343:VM_BUG_ON_PAGE(!unmap_success,page); > > > > Thanks for running the test. This is what I expected from the debug > > patch. It means try_to_unmap() didn't unmap the huge page > > successfully. The huge page is PTE-mapped, try_to_unmap() is supposed > > to unmap every mapped subpage. But it seems it didn't unmap any > > subpage at all (the refcount of the huge page is 512 per the log from > > earlier email). > > > > By reading the code, I didn't figure out what went wrong yet. You > > mentioned that the 5.4.x kernel is fine, so may you try to do some > > bisect? > > This maybe happen on some memory reclaim path. Yes, it does. The stack trace already showed so. > > Our application need to process the file about 300G-400G. > > We have 4 servers, two servers have 192G memory, 1 server has 512G > memory, 1 server has 768G memory. > > If the memory(total memory * 10 / 12 - 120G) is enough to process the > files, no temp file is needed. else, we will write the buffer to temp > file, and continue to process another part. > > this problem happened on the server with 192G memory && kernel 5.10.x, > but yet not happen on the server with kernel 5.4.x || > total memory>=512G. > > so this maybe a timing problem too. debug code maybe userful than code bisect? If you want to add some debug code, there would be a lot of places to add. I'd suggest you try to add some debug code in page_vma_mapped_walk() first, particularly in check_pte(). I suspect it didn't find valid PTEs since the unmap itself would be quite simple. (I assumed CONFIG_MIGRATION is enabled). Then you can try to add debug code in try_to_unmap_one(). And I'm not sure if khugepaged may have race condition with split, it sounds unlikely, but collapsing PTE-mapped THP support was added in v5.8, so you may try to reproduce this on v5.7 to narrow it down. > > fedora with new linux kernel configured with CONFIG_TRANSPARENT_HUGEPAGE_MADVISE=y, > so new linux kernel with CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS=y maybe not well > tested? > > Best Regards > Wang Yugui (wangyugui@e16-tech.com) > 2021/04/24 >