From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63F33C433EF for ; Thu, 9 Jun 2022 08:16:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A502C6B0082; Thu, 9 Jun 2022 04:16:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9FFFB6B0083; Thu, 9 Jun 2022 04:16:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8C7166B0085; Thu, 9 Jun 2022 04:16:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 7ED546B0082 for ; Thu, 9 Jun 2022 04:16:06 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 53307208D8 for ; Thu, 9 Jun 2022 08:16:06 +0000 (UTC) X-FDA: 79557989532.12.E6F632A Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by imf20.hostedemail.com (Postfix) with ESMTP id F25731C0072 for ; Thu, 9 Jun 2022 08:16:04 +0000 (UTC) Received: from dggpemm500020.china.huawei.com (unknown [172.30.72.54]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4LJcLl2xRHz1K9H5; Thu, 9 Jun 2022 16:14:07 +0800 (CST) Received: from dggpemm500014.china.huawei.com (7.185.36.153) by dggpemm500020.china.huawei.com (7.185.36.49) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Thu, 9 Jun 2022 16:15:57 +0800 Received: from [10.174.178.120] (10.174.178.120) by dggpemm500014.china.huawei.com (7.185.36.153) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Thu, 9 Jun 2022 16:15:55 +0800 Message-ID: <1e7df7bc-5a18-f76a-4408-0579a60c91e3@huawei.com> Date: Thu, 9 Jun 2022 16:15:55 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.10.0 From: mawupeng Subject: Re: [PATCH v3 5/6] mm: Add mirror flag back on initrd memory To: , CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , References: <20220607093805.1354256-1-mawupeng1@huawei.com> <20220607093805.1354256-6-mawupeng1@huawei.com> <99900b31-2605-2c85-a1b7-9ef2666b58da@redhat.com> <29900b05-ec44-76a2-645a-22a13399d7fd@huawei.com> In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-Originating-IP: [10.174.178.120] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To dggpemm500014.china.huawei.com (7.185.36.153) X-CFilter-Loop: Reflected ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1654762565; a=rsa-sha256; cv=none; b=Ljqvjl4YLPEfDVej4nMI7fO3Cj+SROqeeeA4rkmfOYI9Wbg3cNC35MzG7F250Igi6gHORJ Kta3YGq527yOJkPgxFlpiQX6s//zmbzNeR5Bz8FibLPNMN4/BvU1EteMqBAUznh/wuQl5E nT9Sdb272RHfd3uzAMCxx4c3y+8QI7U= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf20.hostedemail.com: domain of mawupeng1@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=mawupeng1@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1654762565; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Unh3EXbdDm+Hlsri89+IfiLG+SqUfuREx44sSMD1BNU=; b=E7+fHaZ9UoP21ImQd31YNHpac6pFgh7jzvA0gNa6WhxKJNm7cXGeYfjAY/AHDtGIUgI0FV YeuiaPfkbzpFkQS/Xjcw1bPLnShlE7MkLsd3aAXaN0zYTejLPxHk4dg9pdY/dUpThMmKV9 5IyICtvCSGFqsc1U8/83mUqFPySBKb0= X-Stat-Signature: h7wipewru9ddcgfxkg55f7pxau9sttid X-Rspam-User: X-Rspamd-Queue-Id: F25731C0072 X-Rspamd-Server: rspam07 Authentication-Results: imf20.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf20.hostedemail.com: domain of mawupeng1@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=mawupeng1@huawei.com X-HE-Tag: 1654762564-566806 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: 在 2022/6/8 18:12, Ard Biesheuvel 写道: > On Wed, 8 Jun 2022 at 12:08, David Hildenbrand wrote: >> >> On 08.06.22 12:02, Mike Rapoport wrote: >>> On Wed, Jun 08, 2022 at 03:27:09PM +0800, mawupeng wrote: >>>> >>>> 在 2022/6/7 22:49, Ard Biesheuvel 写道: >>>>> On Tue, 7 Jun 2022 at 14:22, David Hildenbrand wrote: >>>>>> >>>>>> On 07.06.22 11:38, Wupeng Ma wrote: >>>>>>> From: Ma Wupeng >>>>>>> >>>>>>> Initrd memory will be removed and then added in arm64_memblock_init() and this >>>>>>> will cause it to lose all of its memblock flags. The lost of MEMBLOCK_MIRROR >>>>>>> flag will lead to error log printed by find_zone_movable_pfns_for_nodes if >>>>>>> the lower 4G range has some non-mirrored memory. >>>>>>> >>>>>>> In order to solve this problem, the lost MEMBLOCK_MIRROR flag will be >>>>>>> reinstalled if the origin memblock has this flag. >>>>>>> >>>>>>> Signed-off-by: Ma Wupeng >>>>>>> --- >>>>>>> arch/arm64/mm/init.c | 9 +++++++++ >>>>>>> include/linux/memblock.h | 1 + >>>>>>> mm/memblock.c | 20 ++++++++++++++++++++ >>>>>>> 3 files changed, 30 insertions(+) >>>>>>> >>>>>>> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c >>>>>>> index 339ee84e5a61..11641f924d08 100644 >>>>>>> --- a/arch/arm64/mm/init.c >>>>>>> +++ b/arch/arm64/mm/init.c >>>>>>> @@ -350,9 +350,18 @@ void __init arm64_memblock_init(void) >>>>>>> "initrd not fully accessible via the linear mapping -- please check your bootloader ...\n")) { >>>>>>> phys_initrd_size = 0; >>>>>>> } else { >>>>>>> + int flags, ret; >>>>>>> + >>>>>>> + ret = memblock_get_flags(base, &flags); >>>>>>> + if (ret) >>>>>>> + flags = 0; >>>>>>> + >>>>>>> memblock_remove(base, size); /* clear MEMBLOCK_ flags */ >>>>>>> memblock_add(base, size); >>>>>>> memblock_reserve(base, size); >>>>>> >>>>>> Can you explain why we're removing+re-adding here exactly? Is it just to >>>>>> clear flags as the comment indicates? >>>>>> >>>>> >>>>> This should only happen if the placement of the initrd conflicts with >>>>> a mem= command line parameter or it is not covered by memblock for >>>>> some other reason. >>>>> >>>>> IOW, this should never happen, and if re-memblock_add'ing this memory >>>>> unconditionally is causing problems, we should fix that instead of >>>>> working around it. >>>> >>>> This will happen if we use initrdmem=3G,100M to reserve initrd memory below >>>> the 4G limit to test this scenario(just for testing, I have trouble to boot >>>> qemu with initrd enabled and memory below 4G are all mirror memory). >>>> >>>> Re-memblock_add'ing this memory unconditionally seems fine but clear all >>>> flags(especially MEMBLOCK_MIRROR) may lead to some error log. >>>> >>>>> >>>>>> If it's really just about clearing flags, I wonder if we rather want to >>>>>> have an interface that does exactly that, and hides the way this is >>>>>> actually implemented (obtain flags, remove, re-add ...), internally. >>>>>> >>>>>> But most probably there is more magic in the code and clearing flags >>>>>> isn't all it ends up doing. >>>>>> >>>>> >>>>> I don't remember exactly why we needed to clear the flags, but I think >>>>> it had to do with some corner case we hit when the initrd was >>>>> partially covered. >>>> If "mem=" is set in command line, memblock_mem_limit_remove_map() will >>>> remove all memory block without MEMBLOCK_NOMAP. Maybe this will bring the >>>> memory back if this initrd mem has the MEMBLOCK_NOMAP flag? >>>> >>>> The rfc version [1] introduce and use memblock_clear_nomap() to clear the >>>> MEMBLOCK_NOMAP of this initrd memblock. >>>> So maybe the usage of memblock_remove() is just to avoid introducing new >>>> function(memblock_clear_nomap)? >>>> >>>> Since commit 4c546b8a3469 ("memblock: add memblock_clear_nomap()") already >>>> introduced memblock_clear_nomap(). Can we use this to remove flag MEMBLOCK_NOMAP >>>> to solve this problem rather than bring flag MEMBLOCK_MIRROR back? >>> >>> AFAICT, there are two corner cases that re-adding initrd memory covers: >>> * initrd memory is not a part of the memory reported to memblock, either >>> because of firmware weirdness or because it was cut out with mem= >>> * initrd memory overlaps a NOMAP region >>> >>> So to make sure initrd memory is mapped properly and retains >>> MEMBLOCK_MIRROR I think the best we can do is >>> >>> memblock_add(); >>> memblock_clear_nomap(); >>> memblock_reserve(); >> >> Would simply detect+rejecting to boot on such setups be an option? The >> replies so far indicate to me that this is rather a corner case than a >> reasonable use case. >> > > The sad reality is that mem= is known to be used in production for > limiting the amount of memory that the kernel takes control of, in > order to allow the remainder to be used in platform specific ways. > > Of course, there are much better ways to achieve that, but given that > we currently support it, I don't think we can easily back that out. > > I do think that there is no need to go out of our way to make this > case work seamlessly with mirrored memory, though. So I'd prefer to > make the remove+re-add conditional on there actually being a need to > do so. That way, we don't break the old use case or mirrored memory, > and whatever happens when the two are combined is DONTCARE. Does that mean that we don't need to care about this scenario with mirror memory? Thanks for reviewing. > .