From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 430DFC433F5 for ; Wed, 16 Feb 2022 08:05:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B96C36B0078; Wed, 16 Feb 2022 03:05:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B46646B007B; Wed, 16 Feb 2022 03:05:46 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A5BE56B007D; Wed, 16 Feb 2022 03:05:46 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0090.hostedemail.com [216.40.44.90]) by kanga.kvack.org (Postfix) with ESMTP id 969D36B0078 for ; Wed, 16 Feb 2022 03:05:46 -0500 (EST) Received: from smtpin18.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 59B588249980 for ; Wed, 16 Feb 2022 08:05:46 +0000 (UTC) X-FDA: 79147909092.18.C7FAA8C Received: from mail-yb1-f181.google.com (mail-yb1-f181.google.com [209.85.219.181]) by imf26.hostedemail.com (Postfix) with ESMTP id 376C7140003 for ; Wed, 16 Feb 2022 08:05:45 +0000 (UTC) Received: by mail-yb1-f181.google.com with SMTP id p19so3630668ybc.6 for ; Wed, 16 Feb 2022 00:05:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=sKNzKBb19Do3NGmL01/2ZoAEl367WiU5/pzUcTTd170=; b=Vs36gc3ftF5yVkEl7PjXZQrnqiSL9oAPR1PmipoQVMOA0w+rD3oBNP6pQzK/rZADH3 Bk0A7vPTVpmr8Y4S9uEOpwGDZq232z+e9v18C7bAy3SCoHzi1r8E2pOBBowACqQN7WpN 0XMs0LATWPNEfJLFqeXftM4qgZuHS7FFtVMrmX2qh1QW8tiTUI44Xss5hoSv7Soh3Lwd /lLqcu/xYrF8Cibm41ZPRqiFjCTo+nTNjDBERw0A9pfK0q7tc0mIoqySOZY0fndicvvg mMWRM/YLGdaGuoNgpfmeYVdC1jDck29WGCmOcojgFIIda8arRXJlNWgBtki73KRdDHCU KkaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=sKNzKBb19Do3NGmL01/2ZoAEl367WiU5/pzUcTTd170=; b=d4dgpE6/ta9YUdd9mPkZfFFYUKHyVlPWC+RcxWAQ/O4Nu1IY55juonUtCovGGvbehP z6uve7UOX8HRuJ/xA+XBYKTScK/RwOPgBVR/u4vITPS8+WT69NcUEwrAydRtBIoENA+P RcGsvqQYO6sPGlzqimQsYZkEYJ/QNvKkzTDquAuOk8PEFgze1W9YFQeMY7aPPkqvGW5s 2kcSZEmXYWrOBIIkx9ojjJgrK9Y1HRe5++1CIMXGKUNQRWj+lNUBQ2Jx+NEi45IiHp/E KkQrVkRWkXe+lXHlZqAhRJjgMSBcceMyIFusdJ7FnnwI1so3FeB9S8i6m1iULvebZ8QN PRRw== X-Gm-Message-State: AOAM5301rH+ikitP9nWfpEiWXVil8cE476NXs7+wIpAKsPwcSGsM0BSm +rzF1snzJpznlXMh+E/06p3t8310bxsM1f98Mpybzg== X-Google-Smtp-Source: ABdhPJzjoYb2AkrlJR467OpvCCj/z0HN/EHTUaZqEyMMZ5g5FOHNWEVK5sQaZOhyqd9wQ93nQtdnJNLkwUGeFyk6AHg= X-Received: by 2002:a25:e0d0:0:b0:613:e5b1:277e with SMTP id x199-20020a25e0d0000000b00613e5b1277emr1198675ybg.132.1644998744242; Wed, 16 Feb 2022 00:05:44 -0800 (PST) MIME-Version: 1.0 References: <20220208054632.66534-1-songmuchun@bytedance.com> <20220208054632.66534-2-songmuchun@bytedance.com> In-Reply-To: <20220208054632.66534-2-songmuchun@bytedance.com> From: Muchun Song Date: Wed, 16 Feb 2022 16:05:07 +0800 Message-ID: Subject: Re: [PATCH v2 2/2] arm64: mm: hugetlb: add support for free vmemmap pages of HugeTLB To: mark.rutland@arm.com, catalin.marinas@arm.com, james.morse@arm.com Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, duanxiongchun@bytedance.com, fam.zheng@bytedance.com, David Hildenbrand , Mike Kravetz , David Rientjes , Oscar Salvador , "Bodeddula, Balasubramaniam" , Andrew Morton , Will Deacon Content-Type: text/plain; charset="UTF-8" X-Rspam-User: X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 376C7140003 X-Stat-Signature: 8ttgqyez7xn7z7nkrk1tkfz9t7kmsp5d Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=Vs36gc3f; spf=pass (imf26.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.219.181 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com X-HE-Tag: 1644998745-407652 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Feb 8, 2022 at 1:46 PM Muchun Song wrote: > > The feature of minimizing overhead of struct page associated with each > HugeTLB page aims to free its vmemmap pages (used as struct page) to > save memory, where is ~14GB/16GB per 1TB HugeTLB pages (2MB/1GB type). > In short, when a HugeTLB page is allocated or freed, the vmemmap array > representing the range associated with the page will need to be remapped. > When a page is allocated, vmemmap pages are freed after remapping. > When a page is freed, previously discarded vmemmap pages must be > allocated before remapping. More implementations and details can be > found here [1]. > > The preparation of freeing vmemmap pages associated with each HugeTLB > page is ready, so we can support this feature for arm64 now. The > flush_dcache_page() need to be adapted to operate on the head page's > flags since the tail vmemmap pages are mapped with read-only after > the feature is enabled (clear operation is not permitted). > > There was some discussions about this in the thread [2], but there was > no conclusion in the end. And I copied the concern proposed by Anshuman > to here. > > 1st concern: > ''' > But what happens when a hot remove section's vmemmap area (which is > being teared down) is nearby another vmemmap area which is either created > or being destroyed for HugeTLB alloc/free purpose. As you mentioned > HugeTLB pages inside the hot remove section might be safe. But what about > other HugeTLB areas whose vmemmap area shares page table entries with > vmemmap entries for a section being hot removed ? Massive HugeTLB alloc > /use/free test cycle using memory just adjacent to a memory hotplug area, > which is always added and removed periodically, should be able to expose > this problem. > ''' > > Answer: At the time memory is removed, all HugeTLB pages either have been > migrated away or dissolved. So there is no race between memory hot remove > and free_huge_page_vmemmap(). Therefore, HugeTLB pages inside the hot > remove section is safe. Let's talk your question "what about other > HugeTLB areas whose vmemmap area shares page table entries with vmemmap > entries for a section being hot removed ?", the question is not > established. The minimal granularity size of hotplug memory 128MB (on > arm64, 4k base page), any HugeTLB smaller than 128MB is within a section, > then, there is no share PTE page tables between HugeTLB in this section > and ones in other sections and a HugeTLB page could not cross two > sections. In this case, the section cannot be freed. Any HugeTLB bigger > than 128MB (section size) whose vmemmap pages is an integer multiple of > 2MB (PMD-mapped). As long as: > > 1) HugeTLBs are naturally aligned, power-of-two sizes > 2) The HugeTLB size >= the section size > 3) The HugeTLB size >= the vmemmap leaf mapping size > > Then a HugeTLB will not share any leaf page table entries with *anything > else*, but will share intermediate entries. In this case, at the time memory > is removed, all HugeTLB pages either have been migrated away or dissolved. > So there is also no race between memory hot remove and > free_huge_page_vmemmap(). > > 2nd concern: > ''' > differently, not sure if ptdump would require any synchronization. > > Dumping an wrong value is probably okay but crashing because a page table > entry is being freed after ptdump acquired the pointer is bad. On arm64, > ptdump() is protected against hotremove via [get|put]_online_mems(). > ''' > > Answer: The ptdump should be fine since vmemmap_remap_free() only exchanges > PTEs or split the PMD entry (which means allocating a PTE page table). Both > operations do not free any page tables (PTE), so ptdump cannot run into a > UAF on any page tables. The wrost case is just dumping an wrong value. > > [1] https://lore.kernel.org/all/20210510030027.56044-1-songmuchun@bytedance.com/ > [2] https://lore.kernel.org/all/20210518091826.36937-1-songmuchun@bytedance.com/ > > Signed-off-by: Muchun Song Hi Mark, I have updated the commit suggested from you in the previous version, do you (or other maintainers) have any comments on this? Thanks.