From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 227A4C433EF for ; Fri, 4 Mar 2022 04:12:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9A05C8D0002; Thu, 3 Mar 2022 23:12:24 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 94EB98D0001; Thu, 3 Mar 2022 23:12:24 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 83D568D0002; Thu, 3 Mar 2022 23:12:24 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.a.hostedemail.com [64.99.140.24]) by kanga.kvack.org (Postfix) with ESMTP id 70CE08D0001 for ; Thu, 3 Mar 2022 23:12:24 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 2B37325407 for ; Fri, 4 Mar 2022 04:12:24 +0000 (UTC) X-FDA: 79205381808.08.E1961C9 Received: from mail-ot1-f44.google.com (mail-ot1-f44.google.com [209.85.210.44]) by imf03.hostedemail.com (Postfix) with ESMTP id 3ABA820003 for ; Fri, 4 Mar 2022 04:12:23 +0000 (UTC) Received: by mail-ot1-f44.google.com with SMTP id j3-20020a9d7683000000b005aeed94f4e9so6440620otl.6 for ; Thu, 03 Mar 2022 20:12:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=6q+ly/u39xYyKacUqKnY0+oHLRpmW1dHHfAGkiJCu1Y=; b=u3huWPeiz+Oe7K8PoY56nQ1fGZIlx5pbI1hIo7DmXRqmm88lM/h9wHq5wPisneW+3a i0lkATj0Tl5U90bQYe/pVgB9s3qNBIbZ5Idd8JZycguC0ViPbTVWGAkt4uXfvY9xdTHU GewXmlMYyrX+Jn38s7VSvnoEE0CU8K1ZcpP7EQDdaTlD2P6rIcAMF2MeCHJQobwlqX5s jIj6tX9ZiYJfiq68xVLTy/F5Wy0d3KKdnDgOiHcoih7x9LRQu18KasD4umDe5Mr5SWuc hIEKFs4kHra0W1WxYTsB3lG77rF080+GqLkMKeIXC/7C14sC4/Qe+oXSb5tqMofwU/Bd xukQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=6q+ly/u39xYyKacUqKnY0+oHLRpmW1dHHfAGkiJCu1Y=; b=R38gplA1ck2D9jVcLLgJcf94PxzpyILASxKEDUa4mY2HIpq4gq/+yQ4AgdBX/bufpb vNhciXTMf+mIZ1c0xEZvi8hiyipgZuKqoIWU23hBG96mVwTMfjnoJevq/elPWebMzZ3S AcpkpL4F1DaYn5OTxiV8tpIXHjrXKyHFdV781Qn+cFC3YeFUIwZ2Zo5m+v6PDh/GgFkt d/3dKBOwHUe7kAKpohsmoVGrY/2c+C7fLuyDy8mRxIZV6ZwnAYy7UONQOf2sqCWgwVyK fEec3C38tJ203qAn8kuCWwVvu7w0q7/Za4YtzrwpTtjmtRRYdYw4mcjP2OIChD9q/5kn VSbw== X-Gm-Message-State: AOAM531TtAKxFA10Jk5gksnAmGgC8aS7a7+HBRMjRWJwFNSP7eEV/Kv3 G5gSwJ/m1dC7O+ScW0KbiKLRLOS+VKvXpIQE7trwAxd/YAeGCzNV X-Google-Smtp-Source: ABdhPJwn1/iKslLrd8kE67UZS3lsvCoC1aQ/in9KlZnkDOoOUsMSdpG+WvlGFvaQ1Sn1SoK7DfyY1+ZgFIwe/+lG8Bg= X-Received: by 2002:a25:3d87:0:b0:61e:170c:aa9 with SMTP id k129-20020a253d87000000b0061e170c0aa9mr35759918yba.89.1646363397800; Thu, 03 Mar 2022 19:09:57 -0800 (PST) MIME-Version: 1.0 References: <20220303213252.28593-1-joao.m.martins@oracle.com> <20220303213252.28593-5-joao.m.martins@oracle.com> In-Reply-To: <20220303213252.28593-5-joao.m.martins@oracle.com> From: Muchun Song Date: Fri, 4 Mar 2022 11:09:10 +0800 Message-ID: Subject: Re: [PATCH v7 4/5] mm/sparse-vmemmap: improve memory savings for compound devmaps To: Joao Martins Cc: Linux Memory Management List , Dan Williams , Vishal Verma , Matthew Wilcox , Jason Gunthorpe , Jane Chu , Mike Kravetz , Andrew Morton , Jonathan Corbet , Christoph Hellwig , nvdimm@lists.linux.dev, Linux Doc Mailing List Content-Type: text/plain; charset="UTF-8" X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 3ABA820003 X-Stat-Signature: up1y8ena1nrfzho5pd7wwkqp1fmdaj13 Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=u3huWPei; dmarc=pass (policy=none) header.from=bytedance.com; spf=pass (imf03.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.210.44 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com X-HE-Tag: 1646367143-375277 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Mar 4, 2022 at 5:33 AM Joao Martins wrote: > > A compound devmap is a dev_pagemap with @vmemmap_shift > 0 and it > means that pages are mapped at a given huge page alignment and utilize > uses compound pages as opposed to order-0 pages. > > Take advantage of the fact that most tail pages look the same (except > the first two) to minimize struct page overhead. Allocate a separate > page for the vmemmap area which contains the head page and separate for > the next 64 pages. The rest of the subsections then reuse this tail > vmemmap page to initialize the rest of the tail pages. > > Sections are arch-dependent (e.g. on x86 it's 64M, 128M or 512M) and > when initializing compound devmap with big enough @vmemmap_shift (e.g. > 1G PUD) it may cross multiple sections. The vmemmap code needs to > consult @pgmap so that multiple sections that all map the same tail > data can refer back to the first copy of that data for a given > gigantic page. > > On compound devmaps with 2M align, this mechanism lets 6 pages be > saved out of the 8 necessary PFNs necessary to set the subsection's > 512 struct pages being mapped. On a 1G compound devmap it saves > 4094 pages. > > Altmap isn't supported yet, given various restrictions in altmap pfn > allocator, thus fallback to the already in use vmemmap_populate(). It > is worth noting that altmap for devmap mappings was there to relieve the > pressure of inordinate amounts of memmap space to map terabytes of pmem. > With compound pages the motivation for altmaps for pmem gets reduced. > > Signed-off-by: Joao Martins Reviewed-by: Muchun Song Thanks.