From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22754C433ED for ; Wed, 19 May 2021 15:23:24 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 90F60610CB for ; Wed, 19 May 2021 15:23:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 90F60610CB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 162746B006E; Wed, 19 May 2021 11:23:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 14B148E0001; Wed, 19 May 2021 11:23:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0006D8D0001; Wed, 19 May 2021 11:23:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0054.hostedemail.com [216.40.44.54]) by kanga.kvack.org (Postfix) with ESMTP id C56346B006E for ; Wed, 19 May 2021 11:23:22 -0400 (EDT) Received: from smtpin40.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 625EAA8D3 for ; Wed, 19 May 2021 15:23:22 +0000 (UTC) X-FDA: 78158349444.40.9AE379B Received: from mail-pg1-f170.google.com (mail-pg1-f170.google.com [209.85.215.170]) by imf06.hostedemail.com (Postfix) with ESMTP id 7D38AC0042D0 for ; Wed, 19 May 2021 15:23:20 +0000 (UTC) Received: by mail-pg1-f170.google.com with SMTP id f22so8757592pgb.9 for ; Wed, 19 May 2021 08:23:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=qd20th7hRcH2q8Nunhpc+31ZIDOCqw3EIS/TwDw4/5s=; b=hrQ2z1MJedI9hhV2DkzXya0LObavlrHaeahHSCDI5cDEQdt8Z791xlEqqNS9mSsoax jYoTsbeTE6KR0NkjofrHUE2OD6bnD2NKzCZ0qk5AiYtjDwTAAylxNAGVoFPsBSxwHBli vrNhUJUUaF2MxrrzjC9Lu9URErZq07UlYBWMMKWmh6vLwfdK0Y1bWBH6/BOWBb6tgKpw ti9QcZvP8ylE7WhnFchZtWnI7afM+cJHzgKDkoGO4MWiZnMU4TmrmO+Hb1pRUO/SUaz4 Cpd8gu+5lyikno0EJqjhAd8LPOz0b81l7les/DhfhJaEeoIFpeEifRVxX+8/ZrHPjX7g vNgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=qd20th7hRcH2q8Nunhpc+31ZIDOCqw3EIS/TwDw4/5s=; b=O3DfMdl077OMd5BBYerZmQJH1IryXKN0//OAWwUqkasWSHF737GhwKqx+LEzhNsrHU Uv4P5Q/OFVjDkv6sgOu3eFx3WRoQCWGv/YMAp2BhNLmHs+PvivO5ZizDVlfy1LN7k3P/ Be6gRziGHghWUijsAIcTWbEwsWSa37ud7SJocv1Og71ku0vgH/TiJyNwx/euEc5MLRUu d69uRgnfm+GhDyBLyObGn+IdG+1nApe9/AqSYyiJh/OVMXEU4b0fsblG50GcNAIVeBtf FR9XuXY0W4AAhBgW+q4UGcoQvfSsGKUI9XGELUnS9NpuBa/JshoRPY2Bfd3rCW3MajXZ lgUA== X-Gm-Message-State: AOAM532bkbCw5RKnngtgaIwJapDZ9atR7oFVuoOFcYfKs2ydKDXzCvw+ SmO4dzUJ4miiRxfr+yXYRGyqIqVkPUBEtUkU3CDdQQ== X-Google-Smtp-Source: ABdhPJx2oArkMo6klT1vMbR7KAuXL96KcBr/xM1HVB2qpd5pMbiuJ1baIg/WKvqauKfLrSAo71zWY00dLxreNi+tfXA= X-Received: by 2002:a65:5288:: with SMTP id y8mr11305466pgp.31.1621437800210; Wed, 19 May 2021 08:23:20 -0700 (PDT) MIME-Version: 1.0 References: <20210518091826.36937-1-songmuchun@bytedance.com> <5ae7a4be-dfd5-faf6-a75c-a2adf5a344b2@arm.com> In-Reply-To: From: Muchun Song Date: Wed, 19 May 2021 23:22:44 +0800 Message-ID: Subject: Re: [External] Re: [PATCH] arm64: mm: hugetlb: add support for free vmemmap pages of HugeTLB To: Anshuman Khandual Cc: Will Deacon , Andrew Morton , David Hildenbrand , "Bodeddula, Balasubramaniam" , Oscar Salvador , Mike Kravetz , David Rientjes , linux-arm-kernel@lists.infradead.org, LKML , Linux Memory Management List , Xiongchun duan , fam.zheng@bytedance.com, zhengqi.arch@bytedance.com Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 7D38AC0042D0 Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=bytedance-com.20150623.gappssmtp.com header.s=20150623 header.b=hrQ2z1MJ; spf=pass (imf06.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.215.170 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com X-Rspamd-Server: rspam04 X-Stat-Signature: usoxrkgigwpbte8pt9esz3nfi1sww1tw X-HE-Tag: 1621437800-997755 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, May 19, 2021 at 10:43 PM Muchun Song wrote: > > On Wed, May 19, 2021 at 8:35 PM Anshuman Khandual > wrote: > > > > > > On 5/18/21 2:48 PM, Muchun Song wrote: > > > The preparation of supporting freeing vmemmap associated with each > > > HugeTLB page is ready, so we can support this feature for arm64. > > > > > > Signed-off-by: Muchun Song > > > --- > > > arch/arm64/mm/mmu.c | 5 +++++ > > > fs/Kconfig | 2 +- > > > 2 files changed, 6 insertions(+), 1 deletion(-) > > > > > > diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c > > > index 5d37e461c41f..967b01ce468d 100644 > > > --- a/arch/arm64/mm/mmu.c > > > +++ b/arch/arm64/mm/mmu.c > > > @@ -23,6 +23,7 @@ > > > #include > > > #include > > > #include > > > +#include > > > > > > #include > > > #include > > > @@ -1134,6 +1135,10 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, > > > pmd_t *pmdp; > > > > > > WARN_ON((start < VMEMMAP_START) || (end > VMEMMAP_END)); > > > + > > > + if (is_hugetlb_free_vmemmap_enabled() && !altmap) > > > + return vmemmap_populate_basepages(start, end, node, altmap); > > > + > > > do { > > > next = pmd_addr_end(addr, end); > > > > > > diff --git a/fs/Kconfig b/fs/Kconfig > > > index 6ce6fdac00a3..02c2d3bf1cb8 100644 > > > --- a/fs/Kconfig > > > +++ b/fs/Kconfig > > > @@ -242,7 +242,7 @@ config HUGETLB_PAGE > > > > > > config HUGETLB_PAGE_FREE_VMEMMAP > > > def_bool HUGETLB_PAGE > > > - depends on X86_64 > > > + depends on X86_64 || ARM64 > > > depends on SPARSEMEM_VMEMMAP > > > > > > config MEMFD_CREATE > > > > > > > How does this interact with HugeTLB migration as such which might iterate > > over individual constituent struct pages (overriding the same struct page > > for all tail pages when this feature is enabled). A simple test involving > > madvise(ptr, size, MADV_SOFT_OFFLINE) fails on various HugeTLB page sizes, > > with this patch applied. Although I have not debugged this any further. > > It is weird. Actually, I didn't change the behaviour of the page migration. > This feature is default off. If you want to enable this feature, you can pass > "hugetlb_free_vmemmap=on" to the boot cmdline. Do you mean that the > success rate of page migration will decrease when you enable this feature? > The rate will increase if disbale. Right? I have done the test and found the issue. Because unmap_and_move_huge_page always returns -EBUSY. I will look into this issue in depth. Thanks for your report. The return point is as below: if (page_private(hpage) && !page_mapping(hpage)) { rc = -EBUSY; goto out_unlock; } > > Thanks. > > > > > > Soft offlining pfn 0x101c00 at process virtual address 0xffff7fa00000 > > soft offline: 0x101c00: hugepage migration failed 1, type bfffc0000010006 > > (referenced|uptodate|head|node=0|zone=2|lastcpupid=0xffff) > >