From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD203C0015E for ; Mon, 24 Jul 2023 15:41:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F00C76B0071; Mon, 24 Jul 2023 11:41:22 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EB1036B0074; Mon, 24 Jul 2023 11:41:22 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D9FA76B0075; Mon, 24 Jul 2023 11:41:22 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id CBB756B0071 for ; Mon, 24 Jul 2023 11:41:22 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 734541409F9 for ; Mon, 24 Jul 2023 15:41:22 +0000 (UTC) X-FDA: 81046919604.27.65B11FA Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf19.hostedemail.com (Postfix) with ESMTP id 261791A0018 for ; Mon, 24 Jul 2023 15:41:19 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="YzpBA/sr"; spf=pass (imf19.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690213280; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=LavDxiiQvJQ9tdstFAk8l+qf/sEnxlhRWrdlKUhoAGA=; b=cYJIKbEGOLG9hIcy2E2VYNGh8NecDFIRzz94OixStnGxTacCOm07wnztoC28jEM2+SiWr0 3l3Z2K180cC4pH8WZ8kwAijpmrx1bobeXCxFrdLpro6dcmuAQZZ6xjB/S5IKoEj/AxJbNe VxyPtkADsa/M/YQLuUUh3h6gTT/SCvU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690213280; a=rsa-sha256; cv=none; b=zSo/V0PwpjauuPt30b9bf12T8SxQasKBcObppNHsSNZpeqfR8TkiNXTPyL///nt+q5pMSS dUiw/nz8dbzOr+CmAZHRqmTvsRJbVMTIWAUJo7mhRcjgpJebnolRnEvSMuHH9aG/l4HjTz OtY8LmMnzAw6v4P/G8oMh0LOv9FDW9k= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="YzpBA/sr"; spf=pass (imf19.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690213279; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LavDxiiQvJQ9tdstFAk8l+qf/sEnxlhRWrdlKUhoAGA=; b=YzpBA/srV1LL4R/vtPqYjxu9AoX/2qMuq5VlC+Atu3sa2NSwD9S+sEfHZk614Cuf9NCNWx gHEdtGh2goS+FfWd4NtK8LTFIzIwZUsG15GWuTJ2eqe52jKgW44wgPORJFxYzW7tcbVzlO BIT0AGWLHjR89LtfhG0KBv53XgNQ/Hk= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-257-BAHRGRZMMUOajHR2EE2aqw-1; Mon, 24 Jul 2023 11:41:17 -0400 X-MC-Unique: BAHRGRZMMUOajHR2EE2aqw-1 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-316f39e3e89so2657480f8f.1 for ; Mon, 24 Jul 2023 08:41:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690213277; x=1690818077; h=content-transfer-encoding:in-reply-to:subject:organization:from :references:cc:to:content-language:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=LavDxiiQvJQ9tdstFAk8l+qf/sEnxlhRWrdlKUhoAGA=; b=JXBqgJCf9yh0OyGmwCddHUjAGa/enNiA1pTcM7Qagrx7bHlQfHSG7rapZZZZNpPckA w5ZAWB0+gdxbFS4/G5ZThaTBTKJF8buIENcCv2HilBBk4MNK3oT9UPsExLLaNoyQdbRL dvCPbAlvnqJx45UMCt1LdKW3hO1X9iIOFvD9clXl8pQmVpP7hAzt6HHs6pI8HSpLyYmc 8fMGwLJMTzUi25eAC9/BRW6NEVGYNgoeLDWG/MYdiht99oF3zVtKYHMQFMlJ8nQ8OR+b e69ykUvt9bK7t0YnBwoZgh8ZRqklKmWaqe0+EGlYVsx5eLHQZP4CoTrA1y53s5jcw7r8 OU8Q== X-Gm-Message-State: ABy/qLYkvwzAPw1fyHSbes7Pk2/F75lc4UuglAF/yW/UpQv+eht5MOOl mxRrS/2CMciJaXcWsmFQkFwre7sVydOMlRbUrAPB8tVcNbmISfQLwKOiv36KfMup9pcU4DB7Kcl sZ2S5+SJLC4I= X-Received: by 2002:a5d:424a:0:b0:317:5c36:4539 with SMTP id s10-20020a5d424a000000b003175c364539mr3129807wrr.25.1690213276739; Mon, 24 Jul 2023 08:41:16 -0700 (PDT) X-Google-Smtp-Source: APBJJlHk+qyFcEVWJ5hfaknxo7d3c0qjdJTrzuYVUjRP+uNCc3JwYeT7v6d8IgFJVisFkqwng2ydOQ== X-Received: by 2002:a5d:424a:0:b0:317:5c36:4539 with SMTP id s10-20020a5d424a000000b003175c364539mr3129786wrr.25.1690213276288; Mon, 24 Jul 2023 08:41:16 -0700 (PDT) Received: from [192.168.3.108] (p4ff23474.dip0.t-ipconnect.de. [79.242.52.116]) by smtp.gmail.com with ESMTPSA id r1-20020adfce81000000b0031128382ed0sm13354006wrn.83.2023.07.24.08.41.15 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 24 Jul 2023 08:41:15 -0700 (PDT) Message-ID: <29eb32f0-fb0b-c8f9-ba23-8295147808ea@redhat.com> Date: Mon, 24 Jul 2023 17:41:06 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 To: Aneesh Kumar K V , linux-mm@kvack.org, akpm@linux-foundation.org, mpe@ellerman.id.au, linuxppc-dev@lists.ozlabs.org, npiggin@gmail.com, christophe.leroy@csgroup.eu Cc: Oscar Salvador , Michal Hocko , Vishal Verma References: <20230718024409.95742-1-aneesh.kumar@linux.ibm.com> <20230718024409.95742-5-aneesh.kumar@linux.ibm.com> From: David Hildenbrand Organization: Red Hat Subject: Re: [PATCH v4 4/6] mm/hotplug: Allow pageblock alignment via altmap reservation In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Stat-Signature: a1nt7rf6ryfbny3ba8ofan1bu3u9x1mr X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 261791A0018 X-Rspam-User: X-HE-Tag: 1690213279-852592 X-HE-Meta: U2FsdGVkX1+ML+fogs1oCmzdfUhN7BooeO8t8Xa5rwNEXVKYL7+NrtNr0g2gcYbllWjWDdjgGAdayrfhykAl+pTCqiOGLZYE4sqFhDHCp/fi5B9yIyrxtl53jWkFnQoIhUVAeIebHCUrnlhMBhlXQZLt9uq1klITTofqU16Qj8Qy7449pkTyha8Xx2SmL07fLboOR8chR3neGXExgO0NJtvV/6f+tMSNTDIrPhfKaDmj61Ir8gk/uc2u2g6nWcP9gQsu6GlB/apxWN90Pjjg4dHQN3EyERW0BKz9qkZkYO5QNR/Fp5IccU7cYN8utu2u2wUswHG73ZiTp8BjihlnbaNuu/LY/tQwskDmWSV7JYjm6q++aN7zWxYtm1LarRJxtPZ8Iz0G8Kuq/R+e0fwTCiA61LZUR8nGY+UMcvkp9jSiin2A+YeyNMC2go8uLtTrYjIbgQXN+VYhIaEqL23PCTCgiSo/qsJ+KcON88+Yz6Q/sr5uTOVUST1Ji1GSOmP2BBQYXVYd7/7qV2EGifIiRDGQDkZTIFpM7Cronei2VyJ36DseIUzCT7hXeQWswyGTKsC8OftMt5ydqvyPUMSbCukz/NjL9kFv/Rooqgw2fHUnzFhoUreXBaawKI0/oPCm++yfylHypsiJyOTEisenBuJQHwgf+9Xltb4js+FGZBOpeGp07VYE7JX2ghKWh1yrqyJwWI2C/J/sNNq7c3oKpWli+jpwf0qMPm5JsqNhsKv2CH6egdy5EgyjBmt5AYT6flOSgD/galSQ0j1VT+dw2WK8Gjal164IiVV3Vl0rQi0/es5M0EHwh6ezWqozen5WqOdLrQRhWD6YJ27KWzRdQkxoQE9piA3rZGaH7NQtDe9SUknNBkexZ9VOIjo97KkD4OXOpn5i1jKgJnNCy906O/E0TRN/J/uLL3fTGGK6/7Pm+gPo1D3ZL+YUF6UQ1Z9unsumSiqw8NKbw1Ke2lP bJ4lN5ok NIUsvZFwpubu86joqk2R9Xx0cpZlBQs/HXAWf7OAyLT+ecszjpnJ4FNiBH3K8AAHD1/C1UrcIA61FBzycLqZoVr3rmZgJErI0JZ2n1q2VYrD9OTAhi5NHtIMFVreLve0Ee73M/bEtI/GgBr5dMRtLHKNVRugiWd5D4lcBdxIn6VwAa0JFhzVIR1W8NzZ6KBSBPve70uABNvZcoQ6vJ7LW+XMwfnSKuu2A1TeTZJ3UOmHb/2fF7Xb8NokzIlRoi9Mci6v1W/kngfJSSlczGPO67ejSwAGbI1yl82Npe1i6UXh/+QR4qCB/Of/Tsb+ZS/r8bmPPRPuG8b8AT76Crz3pwHL1OCDKjA+yq1mDOmz5hOaCAVdywGytdGJ2oyvd2oi3bxb2U5AjWaynmxQr8tv0QVrgrA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 24.07.23 17:16, Aneesh Kumar K V wrote: >> >> /* >>  * In "forced" memmap_on_memory mode, we always align the vmemmap size up to cover >>  * full pageblocks. That way, we can add memory even if the vmemmap size is not properly >>  * aligned, however, we might waste memory. >>  */ > > I am finding that confusing. We do want things to be pageblock_nr_pages aligned both ways. > With MEMMAP_ON_MEMORY_FORCE, we do that by allocating more space for memmap and > in the default case we do that by making sure only memory blocks of specific size supporting > that alignment can use MEMMAP_ON_MEMORY feature. See the usage inm hp_supports_memmap_on_memory(), I guess that makes sense then. But if you have any ideas on how to clarify that (terminology), I'm all ears! [...] >>> +    return arch_supports_memmap_on_memory(size); >>>   } >>>     /* >>> @@ -1311,7 +1391,11 @@ int __ref add_memory_resource(int nid, struct resource *res, mhp_t mhp_flags) >>>   { >>>       struct mhp_params params = { .pgprot = pgprot_mhp(PAGE_KERNEL) }; >>>       enum memblock_flags memblock_flags = MEMBLOCK_NONE; >>> -    struct vmem_altmap mhp_altmap = {}; >>> +    struct vmem_altmap mhp_altmap = { >>> +        .base_pfn =  PHYS_PFN(res->start), >>> +        .end_pfn  =  PHYS_PFN(res->end), >>> +        .reserve  = memory_block_align_base(resource_size(res)), >> >> Can you remind me why we have to set reserve here at all? >> >> IOW, can't we simply set >> >> .free = memory_block_memmap_on_memory_size(); >> >> end then pass >> >> mhp_altmap.alloc + mhp_altmap.free >> >> to create_memory_block_devices() instead? >> > > But with the dax usage of altmap, altmap->reserve is what we use to reserve things to get > the required alignment. One difference is where we allocate the struct page at. For this specific > case it should not matter. > > static unsigned long __meminit vmem_altmap_next_pfn(struct vmem_altmap *altmap) > { > return altmap->base_pfn + altmap->reserve + altmap->alloc > + altmap->align; > } > > And other is where we online a memory block > > We find the start pfn using mem->altmap->alloc + mem->altmap->reserve; > > Considering altmap->reserve is what dax pfn_dev use, is there a reason you want to use altmap->free for this? "Reserve" is all about "reserving that much memory for driver usage". We don't care about that. We simply want vmemmap allocations coming from the pageblock(s) we set aside. Where exactly, we don't care. > I find it confusing to update free when we haven't allocated any altmap blocks yet. " @reserve: pages mapped, but reserved for driver use (relative to @base)" @free: free pages set aside in the mapping for memmap storage @alloc: track pages consumed, private to vmemmap_populate() " To me, that implies that we can ignore "reserve". We set @free to the aligned value and let the vmemmap get allocated from anything in there. free + alloc should always sum up to our set-aside pageblock(s), no? -- Cheers, David / dhildenb