From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BAE68C433FE for ; Wed, 9 Dec 2020 09:44:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0ECFF23433 for ; Wed, 9 Dec 2020 09:44:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0ECFF23433 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 298416B00CB; Wed, 9 Dec 2020 04:44:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 221DA6B00CD; Wed, 9 Dec 2020 04:44:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 09B5D6B00CE; Wed, 9 Dec 2020 04:44:36 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0042.hostedemail.com [216.40.44.42]) by kanga.kvack.org (Postfix) with ESMTP id E589F6B00CB for ; Wed, 9 Dec 2020 04:44:35 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id B1E7D1EF2 for ; Wed, 9 Dec 2020 09:44:35 +0000 (UTC) X-FDA: 77573258910.16.start83_360192b273ee Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin16.hostedemail.com (Postfix) with ESMTP id 902D0100E7FC3 for ; Wed, 9 Dec 2020 09:44:35 +0000 (UTC) X-HE-Tag: start83_360192b273ee X-Filterd-Recvd-Size: 9418 Received: from mail-pf1-f196.google.com (mail-pf1-f196.google.com [209.85.210.196]) by imf47.hostedemail.com (Postfix) with ESMTP for ; Wed, 9 Dec 2020 09:44:34 +0000 (UTC) Received: by mail-pf1-f196.google.com with SMTP id f9so651625pfc.11 for ; Wed, 09 Dec 2020 01:44:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=nfvm5fXTz5ubcwIn8BzSkwLy3ZTr87nHqelVwd15V9g=; b=MnwiYzHgJ5r0N7rGYnF+LzMQf8/k2y4A4sCmBj7aCWy1sofWibzNsQKk+Vy9z6ZKM9 Pzg04aU0YadFRpXEGjHD/fHQ1c15Z850iXrYYH7ayPNustv2vJR8ekws98hkt51WM/mL cEJB7Vc5LYhB1LS+I77IqqRxhWusDLKzCMp+eGKCeF8rSvdFBlsRjLHPlwlkcsWsPks6 VTmGfr5cFIWytsrrz3eSyRqlgYKRwE7HtzOJlbaIkGHGRXQryOzfncnyExBMCOxP8jmx EKnllaEYSStd11vmyDYuJ5TnE+0Lx7uzn7voJshRSfmrGPIwP5inT9PwctxMyc9Bg02q sZZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=nfvm5fXTz5ubcwIn8BzSkwLy3ZTr87nHqelVwd15V9g=; b=PdOFcALxVl5FTPSSuqQO+He+oOQOfiu/rH/Ny8zf4H6fuB2VzN+Y17ZSdqAwye350u e6ziJQGMU5XdyLPXPw7pf7XjouRsOKVD3eo2az5TIvRuYtoWTaRC2gFjBN2pX80hIfWH yuezA1RK3EocDvkV6lB1TdIejFKx3MdNQ5N9KitZPcPbXmCnktY1fE3Wngf8MQ9twMmA oWz9Lfac8ELck23MUrfexuhvgpT6iAncA8T0pjXZ6QvI1hQRZlKOt2lkvKlAV8zqyTgE XPyraZ+6bMIxAJP4JwuEEvW2cR7rjNekNAfzT7ZzjWvZkzq1a9yWhCwNBKbPZ4Q/ayL7 4lqg== X-Gm-Message-State: AOAM530J5JAi3M3F6FsMCIr1gcTMUlTOSOyvwBd2X6i9OyfJ/OMVceeR GWzQJMs9J3e3QcL8swGf33an2vi4UWkft+oTJ9nK5w== X-Google-Smtp-Source: ABdhPJzcJdhojJpXVfSNU4qNhSr+vBOR6o0JD9ATM/ZWrBV4K8Ymc7R7qMxPWpVfMqyY9XB8ySzlfXfQ+IW1rSVwFzc= X-Received: by 2002:a63:c15:: with SMTP id b21mr1190022pgl.341.1607507073559; Wed, 09 Dec 2020 01:44:33 -0800 (PST) MIME-Version: 1.0 References: <20201130151838.11208-1-songmuchun@bytedance.com> <20201130151838.11208-6-songmuchun@bytedance.com> <17abb7bb-de39-7580-b020-faec58032de9@redhat.com> <096ee806-b371-c22b-9066-8891935fbd5e@redhat.com> <73832edd-13ec-8032-d8d6-4afc53297fdb@redhat.com> In-Reply-To: <73832edd-13ec-8032-d8d6-4afc53297fdb@redhat.com> From: Muchun Song Date: Wed, 9 Dec 2020 17:43:56 +0800 Message-ID: Subject: Re: [External] Re: [PATCH v7 05/15] mm/bootmem_info: Introduce {free,prepare}_vmemmap_page() To: David Hildenbrand Cc: Jonathan Corbet , Mike Kravetz , Thomas Gleixner , mingo@redhat.com, bp@alien8.de, x86@kernel.org, hpa@zytor.com, dave.hansen@linux.intel.com, luto@kernel.org, Peter Zijlstra , viro@zeniv.linux.org.uk, Andrew Morton , paulmck@kernel.org, mchehab+huawei@kernel.org, pawan.kumar.gupta@linux.intel.com, Randy Dunlap , oneukum@suse.com, anshuman.khandual@arm.com, jroedel@suse.de, Mina Almasry , David Rientjes , Matthew Wilcox , Oscar Salvador , Michal Hocko , "Song Bao Hua (Barry Song)" , Xiongchun duan , linux-doc@vger.kernel.org, LKML , Linux Memory Management List , linux-fsdevel Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Dec 9, 2020 at 5:33 PM David Hildenbrand wrote: > > On 09.12.20 10:25, Muchun Song wrote: > > On Wed, Dec 9, 2020 at 4:50 PM David Hildenbrand wrote: > >> > >> On 09.12.20 08:36, Muchun Song wrote: > >>> On Mon, Dec 7, 2020 at 8:39 PM David Hildenbrand wrote: > >>>> > >>>> On 30.11.20 16:18, Muchun Song wrote: > >>>>> In the later patch, we can use the free_vmemmap_page() to free the > >>>>> unused vmemmap pages and initialize a page for vmemmap page using > >>>>> via prepare_vmemmap_page(). > >>>>> > >>>>> Signed-off-by: Muchun Song > >>>>> --- > >>>>> include/linux/bootmem_info.h | 24 ++++++++++++++++++++++++ > >>>>> 1 file changed, 24 insertions(+) > >>>>> > >>>>> diff --git a/include/linux/bootmem_info.h b/include/linux/bootmem_info.h > >>>>> index 4ed6dee1adc9..239e3cc8f86c 100644 > >>>>> --- a/include/linux/bootmem_info.h > >>>>> +++ b/include/linux/bootmem_info.h > >>>>> @@ -3,6 +3,7 @@ > >>>>> #define __LINUX_BOOTMEM_INFO_H > >>>>> > >>>>> #include > >>>>> +#include > >>>>> > >>>>> /* > >>>>> * Types for free bootmem stored in page->lru.next. These have to be in > >>>>> @@ -22,6 +23,29 @@ void __init register_page_bootmem_info_node(struct pglist_data *pgdat); > >>>>> void get_page_bootmem(unsigned long info, struct page *page, > >>>>> unsigned long type); > >>>>> void put_page_bootmem(struct page *page); > >>>>> + > >>>>> +static inline void free_vmemmap_page(struct page *page) > >>>>> +{ > >>>>> + VM_WARN_ON(!PageReserved(page) || page_ref_count(page) != 2); > >>>>> + > >>>>> + /* bootmem page has reserved flag in the reserve_bootmem_region */ > >>>>> + if (PageReserved(page)) { > >>>>> + unsigned long magic = (unsigned long)page->freelist; > >>>>> + > >>>>> + if (magic == SECTION_INFO || magic == MIX_SECTION_INFO) > >>>>> + put_page_bootmem(page); > >>>>> + else > >>>>> + WARN_ON(1); > >>>>> + } > >>>>> +} > >>>>> + > >>>>> +static inline void prepare_vmemmap_page(struct page *page) > >>>>> +{ > >>>>> + unsigned long section_nr = pfn_to_section_nr(page_to_pfn(page)); > >>>>> + > >>>>> + get_page_bootmem(section_nr, page, SECTION_INFO); > >>>>> + mark_page_reserved(page); > >>>>> +} > >>>> > >>>> Can you clarify in the description when exactly these functions are > >>>> called and on which type of pages? > >>>> > >>>> Would indicating "bootmem" in the function names make it clearer what we > >>>> are dealing with? > >>>> > >>>> E.g., any memory allocated via the memblock allocator and not via the > >>>> buddy will be makred reserved already in the memmap. It's unclear to me > >>>> why we need the mark_page_reserved() here - can you enlighten me? :) > >>> > >>> Sorry for ignoring this question. Because the vmemmap pages are allocated > >>> from the bootmem allocator which is marked as PG_reserved. For those bootmem > >>> pages, we should call put_page_bootmem for free. You can see that we > >>> clear the PG_reserved in the put_page_bootmem. In order to be consistent, > >>> the prepare_vmemmap_page also marks the page as PG_reserved. > >> > >> I don't think that really makes sense. > >> > >> After put_page_bootmem() put the last reference, it clears PG_reserved > >> and hands the page over to the buddy via free_reserved_page(). From that > >> point on, further get_page_bootmem() would be completely wrong and > >> dangerous. > >> > >> Both, put_page_bootmem() and get_page_bootmem() rely on the fact that > >> they are dealing with memblock allcoations - marked via PG_reserved. If > >> prepare_vmemmap_page() would be called on something that's *not* coming > >> from the memblock allocator, it would be completely broken - or am I > >> missing something? > >> > >> AFAIKT, there should rather be a BUG_ON(!PageReserved(page)) in > >> prepare_vmemmap_page() - or proper handling to deal with !memblock > >> allocations. > >> > > > > I want to allocate some pages as the vmemmap when > > we free a HugeTLB page to the buddy allocator. So I use > > the prepare_vmemmap_page() to initialize the page (which > > allocated from buddy allocator) and make it as the vmemmap > > of the freed HugeTLB page. > > > > Any suggestions to deal with this case? > > If you obtained pages via the buddy, there shouldn't be anything special > to handle, no? What speaks against > > > prepare_vmemmap_page(): > if (!PageReserved(page)) > return; > > > put_page_bootmem(): > if (!PageReserved(page)) > __free_page(); > Thanks. > > Or if we care about multiple references, get_page() and put_page(). > > > > > I have a solution to address this. When the pages allocated > > from the buddy as vmemmap pages, we do not call > > prepare_vmemmap_page(). > > > > When we free some vmemmap pages of a HugeTLB > > page, if the PG_reserved of the vmemmap page is set, > > we call free_vmemmap_page() to free it to buddy, > > otherwise call free_page(). What is your opinion? > > That would also work. Then, please include "bootmem" as part of the > function name. If you plan on using my suggestion, you can drop > "bootmem" from the name as it works for both types of pages. > Agree. Thanks. > > -- > Thanks, > > David / dhildenb > -- Yours, Muchun