From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA93EE7718B for ; Wed, 25 Dec 2024 09:59:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2965D6B007B; Wed, 25 Dec 2024 04:59:41 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 245C66B0083; Wed, 25 Dec 2024 04:59:41 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0E6FE6B0085; Wed, 25 Dec 2024 04:59:41 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id E34E56B007B for ; Wed, 25 Dec 2024 04:59:40 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 73B641404CE for ; Wed, 25 Dec 2024 09:59:40 +0000 (UTC) X-FDA: 82933033764.04.569CCBE Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) by imf19.hostedemail.com (Postfix) with ESMTP id 3738A1A0017 for ; Wed, 25 Dec 2024 09:58:56 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=quicinc.com header.s=qcppdkim1 header.b=TDQ4jjiG; dmarc=pass (policy=none) header.from=quicinc.com; spf=pass (imf19.hostedemail.com: domain of quic_zhenhuah@quicinc.com designates 205.220.180.131 as permitted sender) smtp.mailfrom=quic_zhenhuah@quicinc.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735120740; a=rsa-sha256; cv=none; b=0vfyyB27wTOQenbny6MuzqG+UmyFx6/MwJ1qNegJ9YCD3kJIjM/dogGZQNyxrpaVGrL4+q x2t/qFMbSNzAH7gj6yjdZAmhkG/jMIcfvsj6SuBmkteT5LZEAzbhYRcs07vkbdiSQ1Te4D BDr43/IOoPIhFSGeRREtT3hoeFyL5HA= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=quicinc.com header.s=qcppdkim1 header.b=TDQ4jjiG; dmarc=pass (policy=none) header.from=quicinc.com; spf=pass (imf19.hostedemail.com: domain of quic_zhenhuah@quicinc.com designates 205.220.180.131 as permitted sender) smtp.mailfrom=quic_zhenhuah@quicinc.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735120740; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JaVSkWKpP5B4CziKH+iVS73s8XWZ5XSk/s+7bHULCNw=; b=tJQDxZEcHBQ54/OsX6oPRaZnirEA3TCTP5faPpN4RUiJ4B82h0PvbSZ52sZjCS5+7Uv/La n96Txmc+tgljeHOSKN9iAKn8YtfZbgvz4q7m7y6AOkluZSOMXDWQRlFwgdmYnjIVauXfWX FWOy8YktUHP8Z0hxBQxLfVSFRP9L/lY= Received: from pps.filterd (m0279870.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4BP3hkUh031265; Wed, 25 Dec 2024 09:59:32 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= cc:content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=qcppdkim1; bh= JaVSkWKpP5B4CziKH+iVS73s8XWZ5XSk/s+7bHULCNw=; b=TDQ4jjiGxWTigkXf e03BKSacyQGnfRErQ5RmWdTkNdys9nml+jgJT689SvjU/udeR9zbZx4225kRw9DY 1NEQ9FN71L4DubhiMJ/DApvH05+hyE7+U8hiwcbgG1j9QYAMJZ/2pOKaqNrYTIxC kPVMdCOgzZbiKqLRV+gl10Ylw0q9PdaYh6MEGX560rbaFr5yGGl00HvoV0q1ms1c biFhES/P4Qb/gSACjrHgWGtYdSMEU1Q98xGvcEktyrzo9LtwGjQ9nD/+znLPzyK1 3a/SZf1NC87+v7JIs5lc8FbtgZuxCBEfJfacRske6VF1o6DvRdo8sAS6RwVnUBAm gzWUfQ== Received: from nalasppmta01.qualcomm.com (Global_NAT1.qualcomm.com [129.46.96.20]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 43rad81jrk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Dec 2024 09:59:32 +0000 (GMT) Received: from nalasex01a.na.qualcomm.com (nalasex01a.na.qualcomm.com [10.47.209.196]) by NALASPPMTA01.qualcomm.com (8.18.1.2/8.18.1.2) with ESMTPS id 4BP9xUCP024401 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Dec 2024 09:59:30 GMT Received: from [10.239.132.245] (10.80.80.8) by nalasex01a.na.qualcomm.com (10.47.209.196) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.9; Wed, 25 Dec 2024 01:59:27 -0800 Message-ID: <04de1444-b28b-4c16-aafa-2b04f0a59304@quicinc.com> Date: Wed, 25 Dec 2024 17:59:25 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v2 1/2] arm64: mm: vmemmap populate to page level if not section aligned To: Catalin Marinas CC: , , , , , , , , , , , , Anshuman Khandual , Tingwei Zhang References: <20241209094227.1529977-1-quic_zhenhuah@quicinc.com> <20241209094227.1529977-2-quic_zhenhuah@quicinc.com> Content-Language: en-US From: Zhenhua Huang In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-Originating-IP: [10.80.80.8] X-ClientProxiedBy: nasanex01a.na.qualcomm.com (10.52.223.231) To nalasex01a.na.qualcomm.com (10.47.209.196) X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: 3_1_jcqzwtTTdVsDy7yyv22gtFZOm1mn X-Proofpoint-ORIG-GUID: 3_1_jcqzwtTTdVsDy7yyv22gtFZOm1mn X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-06_09,2024-09-06_01,2024-09-02_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 adultscore=0 mlxlogscore=999 suspectscore=0 impostorscore=0 clxscore=1015 mlxscore=0 spamscore=0 phishscore=0 priorityscore=1501 bulkscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2411120000 definitions=main-2412250087 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 3738A1A0017 X-Stat-Signature: znyo4ehacagow3ha1xo7qhs6pch7ucfx X-Rspam-User: X-HE-Tag: 1735120736-270656 X-HE-Meta: U2FsdGVkX1+O6/5ffEyPUTLpQf8S5TeMeI7bcXSlooqrtxeuVlZ0bW46HboqtNXTCl4+/dJZDhaOmxTb2vSrCkNL0k3y3C7v+JczqN+W0yVv25+m+EhYYv+7VTaOWPMeiFHS2qw+Ev/5LRb/KO8aZdJeUnBHKfsXxackyMrJx3mbr4dNg9l1SR4175fNlD1qTRdcLiTFCfNzzbJzGjLxbfS00OPAPcESQ3ujkaMAEJGMLzP+Zh2U/6CMQfCDXifidzIX4GRmNq3YNdDSqZkEPIYGRjg8gVyPe+axM7Zs9QtMnqufZthots3/7uf9Igqgu9BiViFtJOkR4JL65b+f3ZDR2XQuN4NbVkCR3c05uBwUSpjzCEWgnFi32qLvSF9ZBBlqYplXnw6Q4LzrHN4w/SVhwiALs08McqC2hA/Z2r2wa0kjVqoD1CZKDoSTVwqH8Jwpa9AKMmpObeK8xnt1Jy7T4w/vnzZjU9p/OC1eie9lGcBwiOl19g20bjs+qqZKHEBRGShMmTMuUFam4daTalT7ZF7Or0Ecan57vRJOwXUNGIASJUEBQA5oBKA1ejvGa2EDPF2RlEDY86YC1CiLLqlSAc3SnH2RgkFq1jaJEiTZ9UCIiulpyIo06Pru8/oTwcDBVELvXM9oaJFhFcWjlKwHI/vVIl40aeLY42RpHKZGOLAW6T3k5Ovu06Pkdn56AfUbv+d6GBEcpqAIM7sIrG3x2L2xubnrQh3txA8i9cU0ALuOWiPyMUDL5TmiK1bdPuVH1YEs+SxDdxBIghOYniBGHPFIojrNEBNj50CH+73QVylXPwsagsCEvEe4pjSqBjgrYJQW2kdgDZNnULDaogrw6wJR6DFwEB41adNCeZqcfJzmlfcwindYxHJOOoGiWN6Lu9+xntZjWUMBSsQ2Yu1q0M5ciYfy1lobsApO3E0bO4MNAc88FDlxHaPL0GpNb+wCgMe1eSCZdXVziwS 7Tfr+INj EDqiV8ICSnKNM2fJaFuCqtCmsc0/nCzi3EHOLHtf1qoFJUo1dAMppFWfpEj9JLDF0ra0y0ChY+NUdtdSLzVsmGWPbNRgZWcSBk4CVUS/XD1zpdRs2akLqirIxZc5Zl096SzmsTUvL6XA6TzaGVa4MHO7r3Dj0lhr2L66aQ/SouzVOfzpYER0+yzyowkVt5kvBA+P5E0qvKJ/D/AVhXzLNY+xxw7kbSDz3HbN6rk7IeHdLTvKZFoVgvkIejAkJFC+aeAfEB/gaQeo1ptyWuK+0y1E6SvvIoAXX8hRLnefZrFUTEGAOubTlT1YfCpJkuA5bqf5SEUAHXo9Q1wQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2024/12/24 22:09, Catalin Marinas wrote: > On Tue, Dec 24, 2024 at 05:32:06PM +0800, Zhenhua Huang wrote: >> Thanks Catalin for review! >> Merry Christmas. > > Merry Christmas to you too! > >> On 2024/12/21 2:30, Catalin Marinas wrote: >>> On Mon, Dec 09, 2024 at 05:42:26PM +0800, Zhenhua Huang wrote: >>>> Fixes: c1cc1552616d ("arm64: MMU initialisation") >>> >>> I wouldn't add a fix for the first commit adding arm64 support, we did >>> not even have memory hotplug at the time (added later in 5.7 by commit >>> bbd6ec605c0f ("arm64/mm: Enable memory hot remove")). IIUC, this hasn't >>> been a problem until commit ba72b4c8cf60 ("mm/sparsemem: support >>> sub-section hotplug"). That commit broke some arm64 assumptions. >> >> Shall we add ba72b4c8cf60 ("mm/sparsemem: support sub-section hotplug") >> because it broke arm64 assumptions ? > > Yes, I think that would be better. And a cc stable to 5.4 (the above > commit appeared in 5.3). Got it, Thanks. > >>>> diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c >>>> index e2739b69e11b..fd59ee44960e 100644 >>>> --- a/arch/arm64/mm/mmu.c >>>> +++ b/arch/arm64/mm/mmu.c >>>> @@ -1177,7 +1177,9 @@ int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, >>>> { >>>> WARN_ON((start < VMEMMAP_START) || (end > VMEMMAP_END)); >>>> - if (!IS_ENABLED(CONFIG_ARM64_4K_PAGES)) >>>> + if (!IS_ENABLED(CONFIG_ARM64_4K_PAGES) || >>>> + !IS_ALIGNED(page_to_pfn((struct page *)start), PAGES_PER_SECTION) || >>>> + !IS_ALIGNED(page_to_pfn((struct page *)end), PAGES_PER_SECTION)) >>>> return vmemmap_populate_basepages(start, end, node, altmap); >>>> else >>>> return vmemmap_populate_hugepages(start, end, node, altmap); >>> >>> An alternative would be to fix unmap_hotplug_pmd_range() etc. to avoid >>> nuking the whole vmemmap pmd section if it's not empty. Not sure how >>> easy that is, whether we have the necessary information (I haven't >>> looked in detail). >>> >>> A potential issue - can we hotplug 128MB of RAM and only unplug 2MB? If >>> that's possible, the problem isn't solved by this patch. >> >> Indeed, seems there is no guarantee that plug size must be equal to unplug >> size... >> >> I have two ideas: >> 1. Completely disable this PMD mapping optimization since there is no >> guarantee we must align 128M memory for hotplug .. > > I'd be in favour of this, at least if CONFIG_MEMORY_HOTPLUG is enabled. > I think the only advantage here is that we don't allocate a full 2MB > block for vmemmap when only plugging in a sub-section. Yeah.. In my opinion, w/o subsection hotplugging support, it is definitely beneficial. However, w/ subsection hotplugging support, it may lead to memory overhead and necessitate special logic in codes since we always use a full 2MB block.. > >> 2. If we want to take this optimization. >> I propose adding one argument to vmemmap_free to indicate if the entire >> section is freed(based on subsection map). Vmemmap_free is a common function >> and might affect other architectures... The process would be: >> vmemmap_free >> unmap_hotplug_range //In unmap_hotplug_pmd_range() as you mentioned：if >> whole section is freed, proceed as usual. Otherwise, *just clear out struct >> page content but do not free*. >> free_empty_tables // will be called only if entire section is freed >> >> On the populate side, >> else if (vmemmap_check_pmd(pmd, node, addr, next)) //implement this function >> continue; //Buffer still exists, just abort.. >> >> Could you please comment further whether #2 is feasible ? > > vmemmap_free() already gets start/end, so it could at least check the > alignment and avoid freeing if it's not unplugging a full section. It > does leave a 2MB vmemmap block in place when freeing the last subsection > but it's safer than freeing valid struct page entries. In addition, it > could query the memory hotplug state with something like > find_memory_block() and figure out whether the section is empty. > Do you mean that we need not clear struct page entries of subsection until the entire section fully unplugged ? That seems feasible. BTW, You're right, I went through codes again, only export is_subsection_map_empty() for query is another option.. page_to_pfn() to get pfn __nr_to_section() to get mem_section last call is_subsection_map_empty() we can get subsection hotplug status per section w/ this approach, we need not to do changes for func vmemmap_free > Anyway, I'll be off until the new year, maybe I get other ideas by then. > Sure, Happy Holiday! I will prepare both of patches and wait for your further comments :