From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E790C433DB for ; Wed, 20 Jan 2021 14:23:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 16B4D23329 for ; Wed, 20 Jan 2021 14:23:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 16B4D23329 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 5F40A6B0007; Wed, 20 Jan 2021 09:23:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5A5FB6B0008; Wed, 20 Jan 2021 09:23:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4954D6B000D; Wed, 20 Jan 2021 09:23:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0116.hostedemail.com [216.40.44.116]) by kanga.kvack.org (Postfix) with ESMTP id 2E59C6B0007 for ; Wed, 20 Jan 2021 09:23:32 -0500 (EST) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id D8F313642 for ; Wed, 20 Jan 2021 14:23:31 +0000 (UTC) X-FDA: 77726371422.26.wax81_58012352755b Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin26.hostedemail.com (Postfix) with ESMTP id B4A9C1804B660 for ; Wed, 20 Jan 2021 14:23:31 +0000 (UTC) X-HE-Tag: wax81_58012352755b X-Filterd-Recvd-Size: 9940 Received: from mail-pj1-f46.google.com (mail-pj1-f46.google.com [209.85.216.46]) by imf48.hostedemail.com (Postfix) with ESMTP for ; Wed, 20 Jan 2021 14:23:30 +0000 (UTC) Received: by mail-pj1-f46.google.com with SMTP id md11so2266575pjb.0 for ; Wed, 20 Jan 2021 06:23:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=BhFA3dUSDH5j1SmI6VzHAtoR/C+MRZ1dv46zw/rCVqE=; b=Xwroxzkk6rTNIr4WrzoC7/JLrMCHLCFuw0ZdnWrHl39esBbZsUrT5P4pzx1gR2EE9T HiZ9NbQQuPznMHuFR4t2mPGMZVVdWW8+57YlWsLmi/asHvfgzZEEZYhwpELaHC/WC4Wm cWO2w8eDQ0hnqz1pfLz5quCMfpxJnDZ6MwCWAar9Yg5eCm8/G/ba6AsxIabEFK/jemsX Th/GWrDuwcipv4c2PC9o3psSDozs4rNaU2gyAISZ0PtabXIVflEF9FXGr8Lr94onTr70 A4Ao03ncinN48bcnxNAJ2mozBxoG+maKT/1a2wzSKlKXVS1Fu/FN3C8EYMvrNnett7Ej Kakw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=BhFA3dUSDH5j1SmI6VzHAtoR/C+MRZ1dv46zw/rCVqE=; b=bqyU/BRb9q/SNZpvD7QB4T0EPqfAvQhP6QifpP9RxOE+MZV+2vQAJD1s500QnkUS6h hsN8E7LQrL9Dq0dafy0ZvTNHUiIxpRRN0PeaTE35r4W6NmKfoNsQ5upobAv/R3FRpXjJ hk2IGEbGKPZCLUMd9v0XYPE4pBNMVNa4d4ZUH75FmrUPp+QNFiY2eBaUkcZZW4Ukswjp PhAT3/c8n9ssyAop45l1F0Rewqjva9d3ZTEq0nVqjv8poN7j2xePGoQWCGoUmfJqM2d0 4OY4QHwrgpuYkfdzcjLd3f/kWahU2RlgSCHq/zjjfTheAyg0lWJ7kFe53pJLUAF5wd0U dj/Q== X-Gm-Message-State: AOAM530kvh+FjnauJqryxzHurJxw07HoW+dVYYCbXVzUWFJZIOgpUc2B CSdo6er455DVOkC5JE1tkaFUDkXF99FqbLZ4abI1zg== X-Google-Smtp-Source: ABdhPJzEHyIORYWhQhm5vKluxtTTZmnEz7Q3xViPDBzSCdqJd+EksAg5z5KqzSBPxPVkQ26HV+CWQL7czNr+2fdo6U4= X-Received: by 2002:a17:902:8503:b029:dc:44f:62d8 with SMTP id bj3-20020a1709028503b02900dc044f62d8mr10138409plb.34.1611152609471; Wed, 20 Jan 2021 06:23:29 -0800 (PST) MIME-Version: 1.0 References: <20210117151053.24600-1-songmuchun@bytedance.com> <20210120130959.GA7881@localhost.localdomain> In-Reply-To: <20210120130959.GA7881@localhost.localdomain> From: Muchun Song Date: Wed, 20 Jan 2021 22:22:51 +0800 Message-ID: Subject: Re: [External] Re: [PATCH v13 00/12] Free some vmemmap pages of HugeTLB page To: Oscar Salvador Cc: Mike Kravetz , Xiongchun duan , Jonathan Corbet , Thomas Gleixner , paulmck@kernel.org, dave.hansen@linux.intel.com, anshuman.khandual@arm.com, oneukum@suse.com, bp@alien8.de, hpa@zytor.com, x86@kernel.org, Randy Dunlap , mingo@redhat.com, mchehab+huawei@kernel.org, luto@kernel.org, Andrew Morton , viro@zeniv.linux.org.uk, Peter Zijlstra , David Rientjes , Michal Hocko , jroedel@suse.de, Mina Almasry , pawan.kumar.gupta@linux.intel.com, =?UTF-8?B?SE9SSUdVQ0hJIE5BT1lBKOWggOWPoyDnm7TkuZ8p?= , David Hildenbrand , "Song Bao Hua (Barry Song)" , linux-doc@vger.kernel.org, LKML , Linux Memory Management List , linux-fsdevel , Matthew Wilcox Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Jan 20, 2021 at 9:10 PM Oscar Salvador wrote: > > On Wed, Jan 20, 2021 at 08:52:50PM +0800, Muchun Song wrote: > > Hi Oscar and Mike, > > > > Any suggestions about this version? Looking forward to your > > review. Thanks a lot. > > Hi Muchun, > > I plan to keep reviewing it in the coming days (tomorrow or Friday). > I glanced over patch#3 when you posted the series and nothing sticked out besides > what you have already pointed out, but I will have a further look. OK. Thanks :) > > thanks > > > > > > > > > > > Changelog in v11 -> v12: > > > - Move VM_WARN_ON_PAGE to a separate patch. > > > - Call __free_hugepage() with hugetlb_lock (See patch #5.) to serialize > > > with dissolve_free_huge_page(). It is to prepare for patch #9. > > > - Introduce PageHugeInflight. See patch #9. > > > > > > Changelog in v10 -> v11: > > > - Fix compiler error when !CONFIG_HUGETLB_PAGE_FREE_VMEMMAP. > > > - Rework some comments and commit changes. > > > - Rework vmemmap_remap_free() to 3 parameters. > > > > > > Thanks to Oscar and Mike's suggestions and review. > > > > > > Changelog in v9 -> v10: > > > - Fix a bug in patch #11. Thanks to Oscar for pointing that out. > > > - Rework some commit log or comments. Thanks Mike and Oscar for the suggestions. > > > - Drop VMEMMAP_TAIL_PAGE_REUSE in the patch #3. > > > > > > Thank you very much Mike and Oscar for reviewing the code. > > > > > > Changelog in v8 -> v9: > > > - Rework some code. Very thanks to Oscar. > > > - Put all the non-hugetlb vmemmap functions under sparsemem-vmemmap.c. > > > > > > Changelog in v7 -> v8: > > > - Adjust the order of patches. > > > > > > Very thanks to David and Oscar. Your suggestions are very valuable. > > > > > > Changelog in v6 -> v7: > > > - Rebase to linux-next 20201130 > > > - Do not use basepage mapping for vmemmap when this feature is disabled. > > > - Rework some patchs. > > > [PATCH v6 08/16] mm/hugetlb: Free the vmemmap pages associated with each hugetlb page > > > [PATCH v6 10/16] mm/hugetlb: Allocate the vmemmap pages associated with each hugetlb page > > > > > > Thanks to Oscar and Barry. > > > > > > Changelog in v5 -> v6: > > > - Disable PMD/huge page mapping of vmemmap if this feature was enabled. > > > - Simplify the first version code. > > > > > > Changelog in v4 -> v5: > > > - Rework somme comments and code in the [PATCH v4 04/21] and [PATCH v4 05/21]. > > > > > > Thanks to Mike and Oscar's suggestions. > > > > > > Changelog in v3 -> v4: > > > - Move all the vmemmap functions to hugetlb_vmemmap.c. > > > - Make the CONFIG_HUGETLB_PAGE_FREE_VMEMMAP default to y, if we want to > > > disable this feature, we should disable it by a boot/kernel command line. > > > - Remove vmemmap_pgtable_{init, deposit, withdraw}() helper functions. > > > - Initialize page table lock for vmemmap through core_initcall mechanism. > > > > > > Thanks for Mike and Oscar's suggestions. > > > > > > Changelog in v2 -> v3: > > > - Rename some helps function name. Thanks Mike. > > > - Rework some code. Thanks Mike and Oscar. > > > - Remap the tail vmemmap page with PAGE_KERNEL_RO instead of PAGE_KERNEL. > > > Thanks Matthew. > > > - Add some overhead analysis in the cover letter. > > > - Use vmemap pmd table lock instead of a hugetlb specific global lock. > > > > > > Changelog in v1 -> v2: > > > - Fix do not call dissolve_compound_page in alloc_huge_page_vmemmap(). > > > - Fix some typo and code style problems. > > > - Remove unused handle_vmemmap_fault(). > > > - Merge some commits to one commit suggested by Mike. > > > > > > Muchun Song (12): > > > mm: memory_hotplug: factor out bootmem core functions to > > > bootmem_info.c > > > mm: hugetlb: introduce a new config HUGETLB_PAGE_FREE_VMEMMAP > > > mm: hugetlb: free the vmemmap pages associated with each HugeTLB page > > > mm: hugetlb: defer freeing of HugeTLB pages > > > mm: hugetlb: allocate the vmemmap pages associated with each HugeTLB > > > page > > > mm: hugetlb: set the PageHWPoison to the raw error page > > > mm: hugetlb: flush work when dissolving a HugeTLB page > > > mm: hugetlb: introduce PageHugeInflight > > > mm: hugetlb: add a kernel parameter hugetlb_free_vmemmap > > > mm: hugetlb: introduce nr_free_vmemmap_pages in the struct hstate > > > mm: hugetlb: gather discrete indexes of tail page > > > mm: hugetlb: optimize the code with the help of the compiler > > > > > > Documentation/admin-guide/kernel-parameters.txt | 14 ++ > > > Documentation/admin-guide/mm/hugetlbpage.rst | 3 + > > > arch/x86/mm/init_64.c | 13 +- > > > fs/Kconfig | 18 ++ > > > include/linux/bootmem_info.h | 65 ++++++ > > > include/linux/hugetlb.h | 37 ++++ > > > include/linux/hugetlb_cgroup.h | 15 +- > > > include/linux/memory_hotplug.h | 27 --- > > > include/linux/mm.h | 5 + > > > mm/Makefile | 2 + > > > mm/bootmem_info.c | 124 +++++++++++ > > > mm/hugetlb.c | 218 +++++++++++++++++-- > > > mm/hugetlb_vmemmap.c | 278 ++++++++++++++++++++++++ > > > mm/hugetlb_vmemmap.h | 45 ++++ > > > mm/memory_hotplug.c | 116 ---------- > > > mm/sparse-vmemmap.c | 273 +++++++++++++++++++++++ > > > mm/sparse.c | 1 + > > > 17 files changed, 1082 insertions(+), 172 deletions(-) > > > create mode 100644 include/linux/bootmem_info.h > > > create mode 100644 mm/bootmem_info.c > > > create mode 100644 mm/hugetlb_vmemmap.c > > > create mode 100644 mm/hugetlb_vmemmap.h > > > > > > -- > > > 2.11.0 > > > > > > > -- > Oscar Salvador > SUSE L3