From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 24663C433F5 for ; Tue, 5 Apr 2022 03:50:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 73F546B0071; Mon, 4 Apr 2022 23:50:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6EE756B0073; Mon, 4 Apr 2022 23:50:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5B68A6B0074; Mon, 4 Apr 2022 23:50:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.28]) by kanga.kvack.org (Postfix) with ESMTP id 4CFC76B0071 for ; Mon, 4 Apr 2022 23:50:17 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id 18425606A5 for ; Tue, 5 Apr 2022 03:50:07 +0000 (UTC) X-FDA: 79321447254.02.D994467 Received: from mail-yb1-f177.google.com (mail-yb1-f177.google.com [209.85.219.177]) by imf29.hostedemail.com (Postfix) with ESMTP id E445A120024 for ; Tue, 5 Apr 2022 03:50:05 +0000 (UTC) Received: by mail-yb1-f177.google.com with SMTP id y38so21161805ybi.8 for ; Mon, 04 Apr 2022 20:50:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=TeIlPBykaaoOQQ8YagbDwarHCQQ/nrdr/5DasCXPXhU=; b=wneh0BiXc+5llZgked4jWuiU/DBGvp91kCAV1p0+Bk/v0mKdq3eNhoPxFpl4Qr6JNz I2BPx4rs+GpJLmuw0bVzUjLJQmXtkYyVVyTS/+c9WHWC9IwZM5nkJ7+BTsFdriHTFSJr lTBju3CN3ktZXrYOAMvEv4QwM/t+Es5VVfI6r4dfcNnrBJCcCJHJToHHCjsBYgrNJUwg Mrgh27PPtbfCYjkuHgr1wJvJFaTd8eDgNGRqhTs3Ct/Dn91bINafMkD+cpT3JnIT924q 6WemLZYZIET3loPw/ZsP/scr1vVwajweZs82K5JS/dh4C3/pXOmny1DoKiFETFZq2uk7 ZYpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=TeIlPBykaaoOQQ8YagbDwarHCQQ/nrdr/5DasCXPXhU=; b=iTQm/0W5Ewxx3GxuAnHy5+kWikDPeLZQy5KhybzUniaqWHXbut7WcO5B/0gR+D+ngf hrAyDqS2N/vCcNxOuXMM/xWUPjkNozcnnfF0vr714Qd1u4UBvqG4sL2j8y99sSf9lkF1 IcuHqXx7LLiUfiTGe6H+Bvf/lm/Bk7IgOZZfLj7tncAx7Xiq5tQznH+pagx7eRmDjCjl yhn+PHZa51fIa6p1/tYpXVgdkD+QHp8E/PSgLe0v6dH5FP9nGHxmzo/YM8r3zTqXLPWI WE27oKW9BIt9HGy1IRDBaivd+SyZ9ww6sIdRssRMJ6UaJndms2LHMkHGPSFMkh6daR0c Y6cg== X-Gm-Message-State: AOAM533kxQwOQuCgODpTTWY0pM/Krdl45NgEyp8efm8Becbubyo0zyug xp7ox+aVdfAIn5VwSul9gHZhUcZDB0Ctnklsco4o6A== X-Google-Smtp-Source: ABdhPJxmMeHW15bgG0AdZv0n4c1UsCDtlJM8yR802ncfdpyZCmnbtjP0omm3Qf8pNGZwxVi040NP5n/ADpHNAHIw6GE= X-Received: by 2002:a25:cc92:0:b0:63d:b330:e8a4 with SMTP id l140-20020a25cc92000000b0063db330e8a4mr1120705ybf.427.1649130604963; Mon, 04 Apr 2022 20:50:04 -0700 (PDT) MIME-Version: 1.0 References: <20220331065640.5777-1-songmuchun@bytedance.com> <20220331065640.5777-2-songmuchun@bytedance.com> In-Reply-To: From: Muchun Song Date: Tue, 5 Apr 2022 11:49:29 +0800 Message-ID: Subject: Re: [PATCH v4 2/2] arm64: mm: hugetlb: Enable HUGETLB_PAGE_FREE_VMEMMAP for arm64 To: Anshuman Khandual Cc: Will Deacon , Andrew Morton , David Hildenbrand , "Bodeddula, Balasubramaniam" , Oscar Salvador , Mike Kravetz , David Rientjes , Mark Rutland , Catalin Marinas , james.morse@arm.com, Barry Song <21cnbao@gmail.com>, LAK , LKML , Linux Memory Management List , Xiongchun duan , Fam Zheng , Muchun Song Content-Type: text/plain; charset="UTF-8" X-Stat-Signature: txeo38dwfu41a1wcermqy3yp51g99ppq Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=wneh0BiX; spf=pass (imf29.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.219.177 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: E445A120024 X-HE-Tag: 1649130605-435460 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Apr 5, 2022 at 11:34 AM Anshuman Khandual wrote: > > > > On 4/4/22 17:31, Muchun Song wrote: > > On Mon, Apr 4, 2022 at 5:25 PM Anshuman Khandual > > wrote: > >> > >> Hello Muchun, > >> > >> On 3/31/22 12:26, Muchun Song wrote: > >>> The feature of minimizing overhead of struct page associated with each > >>> HugeTLB page aims to free its vmemmap pages (used as struct page) to > >>> save memory, where is ~14GB/16GB per 1TB HugeTLB pages (2MB/1GB type). > >> > >> Enabling this feature saves us around 1.4/1.6 % memory but looking from > >> other way around, unavailability of vmemmap backing pages (~1.4GB) when > >> freeing up a corresponding HugeTLB page, could prevent ~1TB memory from > >> being used as normal page form (requiring their own struct pages), thus > >> forcing the HugeTLB page to remain as such ? Is not this problematic ? > >> > >> These additional 1TB memory in normal pages, from a HugeTLB dissolution > >> could have eased the system's memory pressure without this feature being > >> enabled. > > > > You are right. If the system is already under heavy memory pressure, it could > > prevent the user from freeing HugeTLB pages to the buddy allocator. If the > > HugeTLB page are allocated from non-movable zone, this scenario may be > > not problematic since once a HugeTLB page is freed, then the system will > > But how can even the first HugeTLB page be freed without vmemmmap which is > throttled due to lack of sufficient memory ? It's unfortunate, we're deadlocked and will have to try again later :-( > > > have memory to be allocated to be used as vmemmap pages, subsequent > > freeing of HugeTLB pages may be getting easier. However, if the HUgeTLB > > pages are allocated from the movable zone, then the thing becomes terrible, > > which is documented in Documentation/admin-guide/mm/memory-hotplug.rst. > > > > So there is a cmdline "hugetlb_free_vmemmap" to control if enabling this > > feature. The user should enable/disable this depending on their workload. > > Should there also be a sysfs interface for this knob as well ? Perhaps the > system usage might change on the way, without requiring a reboot. Yep. I'm working on this [1] and will cc you in the next version. [1] https://lore.kernel.org/all/20220330153745.20465-1-songmuchun@bytedance.com/ Thanks.