From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63DA6C4361B for ; Wed, 16 Dec 2020 03:53:30 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id B52FB22CB8 for ; Wed, 16 Dec 2020 03:53:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B52FB22CB8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 173096B0036; Tue, 15 Dec 2020 22:53:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1229D6B005D; Tue, 15 Dec 2020 22:53:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 05E796B0068; Tue, 15 Dec 2020 22:53:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0106.hostedemail.com [216.40.44.106]) by kanga.kvack.org (Postfix) with ESMTP id E4D4B6B0036 for ; Tue, 15 Dec 2020 22:53:28 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id B10B4181AEF1E for ; Wed, 16 Dec 2020 03:53:28 +0000 (UTC) X-FDA: 77597775696.16.pot91_2b13bb527429 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin16.hostedemail.com (Postfix) with ESMTP id 910E2100E6903 for ; Wed, 16 Dec 2020 03:53:28 +0000 (UTC) X-HE-Tag: pot91_2b13bb527429 X-Filterd-Recvd-Size: 6835 Received: from mail-pg1-f196.google.com (mail-pg1-f196.google.com [209.85.215.196]) by imf31.hostedemail.com (Postfix) with ESMTP for ; Wed, 16 Dec 2020 03:53:27 +0000 (UTC) Received: by mail-pg1-f196.google.com with SMTP id n7so16702940pgg.2 for ; Tue, 15 Dec 2020 19:53:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=au0l4ODmUzLyBniSivZ0fFaGBE1KkNQntC4U8UGGvjw=; b=HitdVFCAOtsQCu8zvH8qmYCPYzX3PeWE6fktkqoxytKqigp9RNcBtKks+keo8Yodhe 7n2W+zAZoxAU4Tv95vdUIkIlQdX026OWKMsoJ3ppouGncBTXSDspdPZOCIrOikVZDyU8 4jRAVH5o5rPy3FMNBjZjWHRGTxZN/ZhutryDrs7J7hZvQTTx+0V1XZ/YuUyYYG6J5l3Z sBGcqxKFdhI1BPQlkPIA3xbuPw8Ei4eAmwXNRW3dpQrUt/nhnSQVAG49+VgbfX5v0xQp ehKe8JCYXykuOj/3p2DI8pyPmKR7kJdfjaGrJ+tUNMC+pSiHLi+oNmry3uWmDnXCrC9X vjAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=au0l4ODmUzLyBniSivZ0fFaGBE1KkNQntC4U8UGGvjw=; b=uX8bALr4S+xHu9nKiR0Vt23EWrNJV0u0zVmS7xC9f4myfDdzvDu30ivwrWiOirl2PV /MYYgZzxFpEScFNg4MmtxtE4H8DRQ224SZ1LWKgJce40Ob1NL3D3UUFFrmerRNHvQb8S glY4x5AdzrovBFcz8fYnF2NIhlbcENSHZawoOTpHV3Qc480fNH8JGcspiXVN6e0iZkp6 jW41WPRgX0uNWpM7V6dWq5AhM4X9QmO38XKPR5kI1X5lZu3qEvwvZeUx/LqsxfqTbVIh 1LclrShnM/Lbds7DW4XnDvPyIM0mXtyXt74R74wxbXk3XmN1h54jsNMtRAP/XahEpjPI OZPg== X-Gm-Message-State: AOAM531KK2JRm4Xr3Xr5CR4ZoVEGwOthlh4PmXZl0gZaj4nVuEeW/rMK jj9JU66KyU8yhkNzvQU2MdtsVGWrZbIOlIGMsmhHOA== X-Google-Smtp-Source: ABdhPJxhomQrUSLQh8uMb9UuwA9CmbPbyO9SWshT6Q/hiYWWgqBvJj56FQvMJ2tmDKyx4NK5BlBrXhKdVumTZXHNy/0= X-Received: by 2002:aa7:979d:0:b029:1a4:3b76:a559 with SMTP id o29-20020aa7979d0000b02901a43b76a559mr15688095pfp.49.1608090806411; Tue, 15 Dec 2020 19:53:26 -0800 (PST) MIME-Version: 1.0 References: <20201213154534.54826-1-songmuchun@bytedance.com> <20201213154534.54826-3-songmuchun@bytedance.com> <7cfe44aa-3753-82d9-6630-194f1532e186@oracle.com> In-Reply-To: From: Muchun Song Date: Wed, 16 Dec 2020 11:52:50 +0800 Message-ID: Subject: Re: [External] Re: [PATCH v9 02/11] mm/hugetlb: Introduce a new config HUGETLB_PAGE_FREE_VMEMMAP To: Mike Kravetz Cc: Jonathan Corbet , Thomas Gleixner , mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, Peter Zijlstra , viro@zeniv.linux.org.uk, Andrew Morton , paulmck@kernel.org, mchehab+huawei@kernel.org, pawan.kumar.gupta@linux.intel.com, Randy Dunlap , oneukum@suse.com, anshuman.khandual@arm.com, jroedel@suse.de, Mina Almasry , David Rientjes , Matthew Wilcox , Oscar Salvador , Michal Hocko , "Song Bao Hua (Barry Song)" , David Hildenbrand , Xiongchun duan , linux-doc@vger.kernel.org, LKML , Linux Memory Management List , linux-fsdevel Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Dec 16, 2020 at 11:45 AM Mike Kravetz wrote: > > On 12/15/20 5:03 PM, Mike Kravetz wrote: > > On 12/13/20 7:45 AM, Muchun Song wrote: > >> diff --git a/fs/Kconfig b/fs/Kconfig > >> index 976e8b9033c4..4c3a9c614983 100644 > >> --- a/fs/Kconfig > >> +++ b/fs/Kconfig > >> @@ -245,6 +245,21 @@ config HUGETLBFS > >> config HUGETLB_PAGE > >> def_bool HUGETLBFS > >> > >> +config HUGETLB_PAGE_FREE_VMEMMAP > >> + def_bool HUGETLB_PAGE > >> + depends on X86_64 > >> + depends on SPARSEMEM_VMEMMAP > >> + depends on HAVE_BOOTMEM_INFO_NODE > >> + help > >> + When using HUGETLB_PAGE_FREE_VMEMMAP, the system can save up some > >> + memory from pre-allocated HugeTLB pages when they are not used. > >> + 6 pages per HugeTLB page of the pmd level mapping and (PAGE_SIZE - 2) > >> + pages per HugeTLB page of the pud level mapping. > >> + > >> + When the pages are going to be used or freed up, the vmemmap array > >> + representing that range needs to be remapped again and the pages > >> + we discarded earlier need to be rellocated again. > > > > I see the previous discussion with David about wording here. How about > > leaving the functionality description general, and provide a specific > > example for x86_64? As mentioned we can always update when new arch support > > is added. Suggested text? > > > > The option HUGETLB_PAGE_FREE_VMEMMAP allows for the freeing of > > some vmemmap pages associated with pre-allocated HugeTLB pages. > > For example, on X86_64 6 vmemmap pages of size 4KB each can be > > saved for each 2MB HugeTLB page. 4094 vmemmap pages of size 4KB > > each can be saved for each 1GB HugeTLB page. > > > > When a HugeTLB page is allocated or freed, the vmemmap array > > representing the range associated with the page will need to be > > remapped. When a page is allocated, vmemmap pages are freed > > after remapping. When a page is freed, previously discarded > > vmemmap pages must be allocated before before remapping. > > Sorry, I am slowly coming up to speed with discussions when I was away. > > It appears vmemmap is not being mapped with huge pages if the boot option > hugetlb_free_vmemmap is on. Is that correct? Right. > > If that is correct, we should document the trade off of increased page > table pages needed to map vmemmap vs the savings from freeing struct page > pages. If a user/sysadmin only uses a small number of hugetlb pages (as > a percentage of system memory) they could end up using more memory with > hugetlb_free_vmemmap on as opposed to off. Perhaps, it should be part of > the documentation for hugetlb_free_vmemmap? If this is true, and people Right, it is better to document it around hugetlb_free_vmemmap. This should be a part of pathe #8. Thanks. > think this should be documented, I can try to come up with something. > > -- > Mike Kravetz -- Yours, Muchun