From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4262C433EF for ; Mon, 10 Jan 2022 16:22:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EA31D6B0071; Mon, 10 Jan 2022 11:22:17 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E52516B0073; Mon, 10 Jan 2022 11:22:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CF3B16B0074; Mon, 10 Jan 2022 11:22:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0008.hostedemail.com [216.40.44.8]) by kanga.kvack.org (Postfix) with ESMTP id BD08D6B0071 for ; Mon, 10 Jan 2022 11:22:17 -0500 (EST) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 7895D181EE41A for ; Mon, 10 Jan 2022 16:22:17 +0000 (UTC) X-FDA: 79014894714.09.1A3F82E Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf25.hostedemail.com (Postfix) with ESMTP id F1945A000F for ; Mon, 10 Jan 2022 16:22:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1641831736; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RwAYJqp7dsVrXDL0GKFE+MPM2u9HyGkfJOJ9hF2tEI4=; b=QKaJPunsG3PginBcM603rlWM1UPzbY852hSzxubWyOfmw/x9jNDxLccyVISui9RcU2v4wR gNp1uCDsBlbD7Oy4c3iwIHqT09fC6KjHX6rr2xQncyGVkBk4357/Wgm/mAIduuptnwMo21 0VQ0xFBfgDvE8d/Wc1Uy0GCiU4A+YmI= Received: from mail-ed1-f70.google.com (mail-ed1-f70.google.com [209.85.208.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-7-6Qfu5cLKPgSmZfegALe-ig-1; Mon, 10 Jan 2022 11:22:15 -0500 X-MC-Unique: 6Qfu5cLKPgSmZfegALe-ig-1 Received: by mail-ed1-f70.google.com with SMTP id i9-20020a05640242c900b003fe97faab62so770764edc.9 for ; Mon, 10 Jan 2022 08:22:15 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent :content-language:to:references:from:organization:subject :in-reply-to:content-transfer-encoding; bh=RwAYJqp7dsVrXDL0GKFE+MPM2u9HyGkfJOJ9hF2tEI4=; b=rMSYBmzuO42LbIkP3fehsAhGL39KZNJlGlzF6V5tNhC3x9EBbIozxc1iOt5l5oxVpX B6RzyKCOG5y6bgCNVOAlcqI1hs63cRvK1HARwm+286V+UkP/xtuk8No5yNw4ZlAOg7SB qwvr1IvXkL8Kh7XhstzWcGjfGd9ZR7jBjUEU2IkX/YCvaflXnH5/CDekYcBesg/b1Vaq vvkOxtK/+mbcKGhDTTvwCit/LdDJp95EM33XewVOXqvSsoefzwelBPLnl3T1bhpm50vv 0ULfaHA4OrFfG1t7fodprLqEVMMF+7ptT35nOlMhfDnLyowflfDXhA8oJA7V62lrNi7H G5Ww== X-Gm-Message-State: AOAM532A7XnlKJsCchSM53nxD40BOOUL/YXlGvt/SXyRmR3JLDE1l/05 s/GtabwmcffHRIJC9CGZWLxye6P+/JZxLtfCTzpmsndnT1pBH+TyeXztiGpwtcXAxDnjSGgmq37 RBsKk8zNIOks= X-Received: by 2002:aa7:cb09:: with SMTP id s9mr370760edt.379.1641831734229; Mon, 10 Jan 2022 08:22:14 -0800 (PST) X-Google-Smtp-Source: ABdhPJxBI4DBPeOVf/CtEg1sQpIPazrcibBUeJkAoZN7gRVhUK5Q3C9bW1EoRu+mmmx/+nlQV6qWZQ== X-Received: by 2002:aa7:cb09:: with SMTP id s9mr370742edt.379.1641831734025; Mon, 10 Jan 2022 08:22:14 -0800 (PST) Received: from ?IPV6:2003:cb:c701:cf00:ac25:f2e3:db05:65c3? (p200300cbc701cf00ac25f2e3db0565c3.dip0.t-ipconnect.de. [2003:cb:c701:cf00:ac25:f2e3:db05:65c3]) by smtp.gmail.com with ESMTPSA id e16sm3764524edu.15.2022.01.10.08.22.12 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 10 Jan 2022 08:22:12 -0800 (PST) Message-ID: <59d7f87a-b725-4dd6-4f04-fce8950f8033@redhat.com> Date: Mon, 10 Jan 2022 17:22:12 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.4.0 To: "Ranjan, Vikash" , "linux-mm@kvack.org" References: <14101f99fda548dc960e2041a6807536@HIBDWSMB03.ad.harman.com> <4821e34b-b7b5-a281-588d-fe8242856b6b@redhat.com> <0106b4c9c9e740d5927cf805712ebe3f@HIBDWSMB03.ad.harman.com> From: David Hildenbrand Organization: Red Hat Subject: Re: [EXTERNAL] Re: memory_block_size reduction in memory hot plug, can we reduced it to 1GB, currently it is 4GB In-Reply-To: <0106b4c9c9e740d5927cf805712ebe3f@HIBDWSMB03.ad.harman.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=QKaJPuns; dmarc=pass (policy=none) header.from=redhat.com; spf=none (imf25.hostedemail.com: domain of david@redhat.com has no SPF policy when checking 170.10.129.124) smtp.mailfrom=david@redhat.com X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: F1945A000F X-Stat-Signature: 73bciuau91nsj8wnpupm1yzqrceohxid X-HE-Tag: 1641831736-43896 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 10.01.22 17:07, Ranjan, Vikash wrote: > Hi David, > Hi, > You were right , in my source code > arch/arm64/include/asm/sparsemem.h:#define SECTION_SIZE_BITS 30 arch/arm64/include/asm/sparsemem.h:#define SECTION_SIZE_BITS 27 Ever since f0b13ee23241 ("arm64/sparsemem: reduce SECTION_SIZE_BITS"), it should be 512 MiB / 128 MiB. That one should be easy to backport also to older kernels IIRC. > > I need your input on below. > 1) currently due to code ( if (phys_addr & ((pages_per_block << PAGE_SHIFT) - 1)) return -EINVAL;) as of now we could give start addr in the increasing step of 40000000 such as 940000000,980000000,9C0000000. However we cannot give 95000000,990000000,9d0000000. Can we give start address which is not in step of 40000000 Right, in your kernel it should be 1 GiB-aligned chunks. Upstream it should be 512 MiB / 128 MiB-aligned chunks. Memory blocks always have to be aligned, starting in the middle of one is impossible. So with a section size of 512 MiB, best you can do is probing 512 MiB aligned and sized memory blocks. Similarly, with 128 MiB. > > 2) You were right MIN_MEMORY_BLOCK_SIZE is 1 GB, can we further reduce memory region length to 128 MB or it will be good if we can make it customizable( even smaller then 128 MB). Like in our system I want to hot-plug memory which is starting from 9e0000000 to 9e3000000, As we can see memory region is dependent on MIN_MEMORY_BLOCK_SIZE in below code. Upstream, with 4k pages, 128 MiB should work. Anything smaller than that is impossible to hotplug/probe. There are certain restrictions that impose this limit: for example, the pageblock size of 512 MiB limits on 64k limits the section size on 64k to 512 MiB as well. With 4k, we're using 128 MiB, because it's the smallest possible value that still allows for having huge pages to store the vmemmap easily: the memmap of 128 MiB is exactly 2 MiB, corresponding to one huge page on arm64. We usually don't want smaller memory blocks, because it makes /sys/devices/system/memory/ explode in size, can degrade performance (e.g., vmemmap), and might result in other issues (sub-section hot-add for ZONE_DEVICE) > > ret = __add_memory(nid, phys_addr, > MIN_MEMORY_BLOCK_SIZE * sections_per_block); > > > My ask for above is due to non-availability of non -reserved memory in continuous bigger chunk in our system. If you can use 4k base pages, with f0b13ee23241 ("arm64/sparsemem: reduce SECTION_SIZE_BITS") you'll get 128 MiB sections and consequently 128 MiB MIN_MEMORY_BLOCK_SIZE. With that, you can probe 9e0000000 to 9e3000000 by probing 4 individual 128 MiB-sized memory blocks. -- Thanks, David / dhildenb