From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 583F3C001DC for ; Wed, 26 Jul 2023 07:50:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E9B836B0071; Wed, 26 Jul 2023 03:50:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E4AFA6B0074; Wed, 26 Jul 2023 03:50:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CECA98D0001; Wed, 26 Jul 2023 03:50:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id C0D466B0071 for ; Wed, 26 Jul 2023 03:50:26 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 8FE061C9EE3 for ; Wed, 26 Jul 2023 07:50:26 +0000 (UTC) X-FDA: 81052990452.21.9DE0884 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf22.hostedemail.com (Postfix) with ESMTP id 51FDDC0003 for ; Wed, 26 Jul 2023 07:50:24 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=PfLIjnWd; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf22.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1690357824; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=SXtdvsYorh2jLmkQgOQxjjs6N6GWZk4UdbudzqwlXs0=; b=knIBo7e80qC55aRyQCuwWu2JJsVX1cQH6DMRFOn8DDIQlMWubm5hgciXIZL9cVA5FaGIM/ xNXuNHdrSiT0cuRS0w3PgngdQJizzaAmyaOuNPD0W03ZXYNw49JJVW9XIKIZCZFDIF56JK 3WLRbivzgnn4IJMmnPBSjPkITdi6nyo= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=PfLIjnWd; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf22.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1690357824; a=rsa-sha256; cv=none; b=qyRquQJtL5gP9kUtMbemdtKuoYlY/k0mFjvMpzqxrkHNux+FiiUMAPfjKt+6OSQ8R+J8Dh aRO+2jmKAKqxyEDPrnAUYy+JgesaQ5iT6Kt0P6+EyYVCOmZXtlLmJ7MCTh2XC2RSJbDPQl Pbdgr0K/2GE8P4l6JKFQUl04YmHzTng= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690357823; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SXtdvsYorh2jLmkQgOQxjjs6N6GWZk4UdbudzqwlXs0=; b=PfLIjnWdosIpml58Ix2Ma52mQ5q6TgqdKm6FKszT9we/R3/6u0mf/DF6ieAO41+ZeRlI57 ezdI5aRex3D1SuStrIDaoMit3xpykd6er0nCRfZEALw+a6UN5BSg3ZXjXOOi6IH2kd9neM HePZxFLYsvYnixyVBjHYU4h79t6Ndww= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-685-ee61iDJGO2ulUYshAYRyng-1; Wed, 26 Jul 2023 03:50:22 -0400 X-MC-Unique: ee61iDJGO2ulUYshAYRyng-1 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-3fd2dec82a6so24209345e9.3 for ; Wed, 26 Jul 2023 00:50:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690357821; x=1690962621; h=content-transfer-encoding:in-reply-to:subject:organization:from :content-language:references:cc:to:user-agent:mime-version:date :message-id:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=SXtdvsYorh2jLmkQgOQxjjs6N6GWZk4UdbudzqwlXs0=; b=kNdkCACkz73N2mL0vput8KIGf8Za0d/3L/F/pl3tvnDyYrqmzuuDhbNyxgO354IyWV AdDoLQ0qbxxPYOfhGKvrggmHfuLTJmOJaKSFzxyDx/nb0hFGuU/OhbcWxJqmm9FUlq6v mHrWEQjemzm2BY+N5gEGZrBypsZQ+hJaMJdCbBaC28M82tL5yUqU7zqbrJLAQn9ic+L6 OI19fwWynsdmFZmPrvkZyR1I1yXwaraMvNIuOSpz5mxs5OqlZSTZ/lLh+bmE9jMhuZco 5LQrXBjlL5XxPWP1QumVchmY8Xq+hyZMLXK7T+tu7MgG/paQKfzdiQJHQXhbjKYoY3MY D9iw== X-Gm-Message-State: ABy/qLZOoWJ+DGYM/xgPOiFH17hK1ll+6BrG2tBIaJKpZUVI3+G5C7cL SNvH1HQujuwwFQe6cvWqwn4lBoXwmOOjouiRnlaaaGZoMoeZeKDiKzmcrWJ6P0sKv/nKeJL93hI PGKBxF/VU1zk= X-Received: by 2002:a7b:c4d5:0:b0:3fd:2e89:31bd with SMTP id g21-20020a7bc4d5000000b003fd2e8931bdmr799979wmk.14.1690357821345; Wed, 26 Jul 2023 00:50:21 -0700 (PDT) X-Google-Smtp-Source: APBJJlE4S4B0waCmCbeErLjqmp5GKGPjgUCuHGr4eplSbJ3PIIdk1LdvZMPPowpzjgQp6MomB5Vm7A== X-Received: by 2002:a7b:c4d5:0:b0:3fd:2e89:31bd with SMTP id g21-20020a7bc4d5000000b003fd2e8931bdmr799954wmk.14.1690357820938; Wed, 26 Jul 2023 00:50:20 -0700 (PDT) Received: from ?IPV6:2003:cb:c705:f600:a519:c50:799b:f1e3? (p200300cbc705f600a5190c50799bf1e3.dip0.t-ipconnect.de. [2003:cb:c705:f600:a519:c50:799b:f1e3]) by smtp.gmail.com with ESMTPSA id s23-20020a7bc397000000b003fba6a0c881sm1189152wmj.43.2023.07.26.00.50.19 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 26 Jul 2023 00:50:20 -0700 (PDT) Message-ID: Date: Wed, 26 Jul 2023 09:50:19 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 To: mawupeng , anshuman.khandual@arm.com, will@kernel.org Cc: catalin.marinas@arm.com, akpm@linux-foundation.org, sudaraja@codeaurora.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, wangkefeng.wang@huawei.com, linux-arm-kernel@lists.infradead.org, mark.rutland@arm.com References: <20230717115150.1806954-1-mawupeng1@huawei.com> <20230721103628.GA12601@willie-the-truck> <35a0dad6-4f3b-f2c3-f835-b13c1e899f8d@huawei.com> <732e0db0-eb41-6c58-85b7-46257b4ba0b7@redhat.com> <3149f5f8-7878-dfe1-5745-870fddcc1108@huawei.com> From: David Hildenbrand Organization: Red Hat Subject: Re: [RFC PATCH] arm64: mm: Fix kernel page tables incorrectly deleted during memory removal In-Reply-To: <3149f5f8-7878-dfe1-5745-870fddcc1108@huawei.com> X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspam-User: X-Stat-Signature: k1ycush395e5gdom8yssq5zoo8rk91fr X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 51FDDC0003 X-HE-Tag: 1690357824-705237 X-HE-Meta: U2FsdGVkX18C8D0uXyy30YBR7Ja4gG8bQSXU+Jb7jU64qEulUa2JWAadJ8rIODNvUeVPdeQ2PhkgfisFOiWmIgIhBExeCdFTGfawJHCNWKP6se6qB/hG0x/l+VLmK+dJZPt3xZcm3Qy2u2DepIn2kQ/JNk+1ngKIIyoWwE741KHRfEZEUBQrDy4ka6CY8USdN35AmEvvMFfFwfFGwQ2be+ZfxhEtiaLoJFxEz0o9eBDNnhEMaKdADcxe0zi5DpVBPKB5WrAR4iJHg2DviiIRNwlhglTMhJPpzQ+Y2UPOPr9r8mv0Z51Eastd7WFaPC5s2qiWb+j0BF4QMsJVqsdmkSQq6l+SdgbwxV+5cVZvBVmIPVCHBB/VFQn9u9N9TgEXiqgz8V/vTOVWpPfBhVzsMFR9j17WT0l5JEcWLUx4pgWI8yfeyeNN9mU70X8/wSIXz+13OTUGlk9ff6LR8zYsSQkVh2Z9ZIynpXNAZdooyB/x0luXqWPs3XvVkZA9Fdn1srj5UG33cjxJTmHFJMhZgxHxdrDhy82+1Sij/L4f119VrFOTiVatKM3Ue2lIiQ50RR0NgcV+f1MSC3zFoL/BQ9IFgM5yRZsm5CFcGQQkeb0JfZSIWkml8KNtZdfC1CzdyhPdFT+zp35xoVww0U5S90t7iJz5hqRzSlw9Q1oy2wG03iImqRWYnRbbNit4IHaBBebxwApGazIqMNG6V6UQT6A5ezw+bzEpFvsKsJsZGcM0EKevAuN9dCxOIilYEhhUGPhsI2B+rqUCdSFwKHGgRZtVvQAw8W5TV1ZRNaHr7W/2IKCpP5yHwy2B/9KeAasap5NQdW1YxAHf/9XAHLlkMSnB+f1ufv+cHokRhcmMfWXIBsQyoilwEHR4/d9Kl+bas++isktOR5DxpOsoemWNMtADT4G0vK4jeMi6o4C1p4xuxdeWKgI+pic91QGwVKXfMv3yAMb9D+yAagMnUDt BiJscq+K FX34Wr7Gw2mX/4qbNjxPqKSRFnUi5NefhctzMMdPEZ1tNPFVIDCYJphbDM18hbCwTl2xIxcNDRqSRYDdYZaVYyTp4MXkuS/aLHS2U/iJmduDFbHvFh/rwkzZUijUil/qihyF66UWKRVk6JxePvlYAZ3iDHNMljWpa+oZhfCcr+W64+TDZ78tSZsdG7VD7Q1Z+tsvr85tVSbTfiRmHWGwSNtrAg9rqDNBiexAcInai9neJi0nkw5kg9NlULwVmZeIXsSBQvwdjI7lhv8e+oGV1IZzUSAqDaLbsPn79OfYJq9g+xiFNd9gV0sf9wWJDogjy2gboeh1Vqj5OTZ3BD47VCJba6S0PEJL7OzfZ/JSO7UE2ODH5sPMxkc5BPizCjarIGxGJt2AiP6VgOqiAXN0DwsK+j1CBcUq8q5/i X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 26.07.23 08:20, mawupeng wrote: > > > On 2023/7/24 14:11, David Hildenbrand wrote: >> On 24.07.23 07:54, Anshuman Khandual wrote: >>> >>> >>> On 7/24/23 06:55, mawupeng wrote: >>>> >>>> On 2023/7/21 18:36, Will Deacon wrote: >>>>> On Mon, Jul 17, 2023 at 07:51:50PM +0800, Wupeng Ma wrote: >>>>>> From: Ma Wupeng >>>>>> >>>>>> During our test, we found that kernel page table may be unexpectedly >>>>>> cleared with rodata off. The root cause is that the kernel page is >>>>>> initialized with pud size(1G block mapping) while offline is memory >>>>>> block size(MIN_MEMORY_BLOCK_SIZE 128M), eg, if 2G memory is hot-added, >>>>>> when offline a memory block, the call trace is shown below, >> >> Is someone adding memory in 2 GiB granularity and then removing parts of it in 128 MiB granularity? That would be against what we support using the add_memory() / offline_and_remove_memory() API and that driver should be fixed instead. > > Yes, this kind of situation. > > The problem occurs in the following scenarios: > 1. use mem=xxG to reserve memory. > 2. add_momory to online memory. > 3. offline part of the memroy via offline_and_remove_memory. > > During my research, ACPI memory removal use memory_subsys_offline to offline memory section and > this will not delete page table entry which do not trigger this kind of problem. > > So I understand what you are talking about. > 1. 3rd-party driver shouldn't use add_memory/offline_and_remove_memory to online/offline memory. > If it have to use, this can be achieved by driver. > 2. memory_subsys_offline is perfered to do such thing. No, my point is that 1) If you use add_memory() and offline_and_remove_memory() in the *same granularity* it has to be working, otherwise it has to be fixed. 2) If you use add_memory() and offline_and_remove_memory() in different granularity (especially, add_memory() in bigger granularity) , then change your code to do add_memory() in the same granularity. If you run into 1), then we populated a PUD for boot memory that also covers yet unpopulated physical memory ranges that are later populated by add_memory(). If that's the case, then we can either fix it by a) Not doing that. Use PMD tables instead for that piece of memory. b) Detecting that that PUD still covers memory and refusing to remove that PUD. c) Rejecting to hotadd memory in this situation at that location. We have mhp_get_pluggable_range() -> arch_get_mappable_range() to kind- of handle something like that. -- Cheers, David / dhildenb