From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE6D1C433EF for ; Thu, 24 Feb 2022 05:55:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4C1548D0002; Thu, 24 Feb 2022 00:55:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4703C8D0001; Thu, 24 Feb 2022 00:55:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 35F898D0002; Thu, 24 Feb 2022 00:55:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.a.hostedemail.com [64.99.140.24]) by kanga.kvack.org (Postfix) with ESMTP id 099E38D0001 for ; Thu, 24 Feb 2022 00:55:34 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 9ED58700 for ; Thu, 24 Feb 2022 05:55:34 +0000 (UTC) X-FDA: 79176611388.14.AC7FB53 Received: from mail-yw1-f170.google.com (mail-yw1-f170.google.com [209.85.128.170]) by imf14.hostedemail.com (Postfix) with ESMTP id 63CF4100002 for ; Thu, 24 Feb 2022 05:55:33 +0000 (UTC) Received: by mail-yw1-f170.google.com with SMTP id 00721157ae682-2d07c4a0d06so12917147b3.13 for ; Wed, 23 Feb 2022 21:55:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=kvhU+1d1xBuiwuTvVQmOP1FBfYrMuzDuvR5ypoAEjro=; b=CEt6WoNPJExnK4pZ3D56UCeA564FCpoElTbHUG7guQrxewH8Ef4bcrKv615NXBRlGg vBtPBeEeoQNZU2567HdocwW4qpALbKec2QFSlYDInWSsnOW9/wYQI+frra60lxHZ7ej6 KA492jINm/YiiL6pOlnXDFLQiP7GcwKeFrkBl20tJ3fcDCto8o94c472V781OracUOuN UBhtq3m3eMFoVLMR7ZohfgfjkR04+UvHT35hf/wePMDEOiF2kkqpub3oDeKVbp51W+Wn BnexbEPlDwQhzil3kFMp4DDSV2lqRwKClXc4VIDKzeyVgImJeXRENSj9cnUzXqXJnxun uY2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=kvhU+1d1xBuiwuTvVQmOP1FBfYrMuzDuvR5ypoAEjro=; b=TGR+bNIp3Y6yIqKFnk1UoK2c9/kK0FfiO2jJFAdeU7Wy/U+OzpFgkoVSsq48vJZX7F 6q4asiBGe8lI3SR+uG0Dly2ZbieMur1ab8tKoWOIVbfNioSFG/L0B1+yzXbjaDryz0dt FAocP6WjO8VT0PrCRsWPOwyTmmvCtcJ0m/bQdk5VFXXEQRxlhxtddOsdS/Euz6t8O3MW sCXBWhtBkD91ZZlaR1bvmF7M6WF14C3mXLr4zRj7E3diPy9uAVveJCjhrqsEEdLPNQi5 78fQeypMZQLfQHfe8bwypdQpmmkYkFykF6PKdtNqMPM3it54TfLKMJMxMV/5g2GHLVWp TFQw== X-Gm-Message-State: AOAM530EgOi/pPh+4Zj7DmPkYHBkJg7UlFWYPkerTdiFJm0qPTvmoxxI Is+0YLljLvGrt+ksic1yj2iAvnjNWJW1p0xwSt2DUw== X-Google-Smtp-Source: ABdhPJxQBX3Dc+MjzYS5ebkRBFbDibF6wIJL7Bn+424LeZUsSwel/NdC9XRIpxfLvof5dstbTtuwpAn4KzhMnTIfxz8= X-Received: by 2002:a81:5dd6:0:b0:2d6:3041:12e0 with SMTP id r205-20020a815dd6000000b002d6304112e0mr976524ywb.331.1645682132153; Wed, 23 Feb 2022 21:55:32 -0800 (PST) MIME-Version: 1.0 References: <20220223194807.12070-1-joao.m.martins@oracle.com> <20220223194807.12070-5-joao.m.martins@oracle.com> In-Reply-To: <20220223194807.12070-5-joao.m.martins@oracle.com> From: Muchun Song Date: Thu, 24 Feb 2022 13:54:54 +0800 Message-ID: Subject: Re: [PATCH v6 4/5] mm/sparse-vmemmap: improve memory savings for compound devmaps To: Joao Martins Cc: Linux Memory Management List , Dan Williams , Vishal Verma , Matthew Wilcox , Jason Gunthorpe , Jane Chu , Mike Kravetz , Andrew Morton , Jonathan Corbet , Christoph Hellwig , nvdimm@lists.linux.dev, Linux Doc Mailing List Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 63CF4100002 X-Stat-Signature: ohpasoz111hiu16i57591w9k8fcmb35d Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=CEt6WoNP; dmarc=pass (policy=none) header.from=bytedance.com; spf=pass (imf14.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.128.170 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1645682133-259051 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Feb 24, 2022 at 3:48 AM Joao Martins wrote: > > A compound devmap is a dev_pagemap with @vmemmap_shift > 0 and it > means that pages are mapped at a given huge page alignment and utilize > uses compound pages as opposed to order-0 pages. > > Take advantage of the fact that most tail pages look the same (except > the first two) to minimize struct page overhead. Allocate a separate > page for the vmemmap area which contains the head page and separate for > the next 64 pages. The rest of the subsections then reuse this tail > vmemmap page to initialize the rest of the tail pages. > > Sections are arch-dependent (e.g. on x86 it's 64M, 128M or 512M) and > when initializing compound devmap with big enough @vmemmap_shift (e.g. > 1G PUD) it may cross multiple sections. The vmemmap code needs to > consult @pgmap so that multiple sections that all map the same tail > data can refer back to the first copy of that data for a given > gigantic page. > > On compound devmaps with 2M align, this mechanism lets 6 pages be > saved out of the 8 necessary PFNs necessary to set the subsection's > 512 struct pages being mapped. On a 1G compound devmap it saves > 4094 pages. > > Altmap isn't supported yet, given various restrictions in altmap pfn > allocator, thus fallback to the already in use vmemmap_populate(). It > is worth noting that altmap for devmap mappings was there to relieve the > pressure of inordinate amounts of memmap space to map terabytes of pmem. > With compound pages the motivation for altmaps for pmem gets reduced. > > Signed-off-by: Joao Martins [...] > diff --git a/include/linux/mm.h b/include/linux/mm.h > index 5f549cf6a4e8..b0798b9c6a6a 100644 > --- a/include/linux/mm.h > +++ b/include/linux/mm.h > @@ -3118,7 +3118,7 @@ p4d_t *vmemmap_p4d_populate(pgd_t *pgd, unsigned long addr, int node); > pud_t *vmemmap_pud_populate(p4d_t *p4d, unsigned long addr, int node); > pmd_t *vmemmap_pmd_populate(pud_t *pud, unsigned long addr, int node); > pte_t *vmemmap_pte_populate(pmd_t *pmd, unsigned long addr, int node, > - struct vmem_altmap *altmap); > + struct vmem_altmap *altmap, struct page *block); Have forgotten to update @block to @reuse here. [...] > + > +static int __meminit vmemmap_populate_range(unsigned long start, > + unsigned long end, > + int node, struct page *page) All of the users are passing a valid parameter of @page. This function will populate the vmemmap with the @page and without memory allocations. So the @node parameter seems to be unnecessary. If you want to make this function more generic like vmemmap_populate_address() to handle memory allocations (the case of @page == NULL). I think vmemmap_populate_range() should add another parameter of `struct vmem_altmap *altmap`. Otherwise, is it better to remove @node and rename @page to @reuse? Thanks.