From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 218F0E95A91 for ; Mon, 9 Oct 2023 10:11:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A60208D0049; Mon, 9 Oct 2023 06:11:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A11058D0031; Mon, 9 Oct 2023 06:11:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8B0EA8D0049; Mon, 9 Oct 2023 06:11:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 7A20D8D0031 for ; Mon, 9 Oct 2023 06:11:49 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 4A9C01601D5 for ; Mon, 9 Oct 2023 10:11:49 +0000 (UTC) X-FDA: 81325506738.05.11ED1F8 Received: from mail-lj1-f177.google.com (mail-lj1-f177.google.com [209.85.208.177]) by imf20.hostedemail.com (Postfix) with ESMTP id 19ADC1C0003 for ; Mon, 9 Oct 2023 10:11:46 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=linaro.org header.s=google header.b=pCSZsUAJ; dmarc=pass (policy=none) header.from=linaro.org; spf=pass (imf20.hostedemail.com: domain of konrad.dybcio@linaro.org designates 209.85.208.177 as permitted sender) smtp.mailfrom=konrad.dybcio@linaro.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696846307; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IGY0FBb36qCXjvr1NWROo1BhBbwPapXrgUBbbWrCwgg=; b=bUWA/96J64dfUNWsFSUoMmVudGP4ddTlvf+BEBxTBUaDs9B7BJDG7EiB2e79h6qyuiGe77 R1mUwDBiknMOYhXOuIMgqKt4PniyOEM19l1mB6FAPGCB/De0573Hfu9WrIPehLFpZ14UfJ J/4k0j8+SdqItCqUO00mRIFkCq8supE= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=linaro.org header.s=google header.b=pCSZsUAJ; dmarc=pass (policy=none) header.from=linaro.org; spf=pass (imf20.hostedemail.com: domain of konrad.dybcio@linaro.org designates 209.85.208.177 as permitted sender) smtp.mailfrom=konrad.dybcio@linaro.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696846307; a=rsa-sha256; cv=none; b=Et5eYJKne669nKa8HZSevOPUjyuX1gHnTtIK9JGHBlFmHIVjjlC8zRE89/L6CD4rubteoT 8PsY5BjCS8476/2aeqLsElozAWH3Kw5cjur1fYlhkHra7npAkS9S58gx015MBqf2XdzJ6Q 1iYLS2bPh0/cRNIT48DrDWVXJ26TqCU= Received: by mail-lj1-f177.google.com with SMTP id 38308e7fff4ca-2bb9a063f26so54155271fa.2 for ; Mon, 09 Oct 2023 03:11:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1696846305; x=1697451105; darn=kvack.org; h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc :to:content-language:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=IGY0FBb36qCXjvr1NWROo1BhBbwPapXrgUBbbWrCwgg=; b=pCSZsUAJvbLuQjGUqRoUwb6YslvZGv27YXpp9T/0agFyxbJrepOVMu9v64d6QSXC2L M/x6xuXxlfj4MLBlGYCGPHaESedp25jCVG3n4venbtkz8vWowb9P4yMzkvKHzv9zqoNm BPN5VsypoSklwYqn3b/16qCOT8lbANOEnqt0DwUtIfsnnuoinQqIvDKpIqWaBk6F7Hmr K0/sXvo5/4bQt2DLf0nFyLM4V1+4wn075ZrRT1Z/6Jdog/WFj2F5hqwWTS9NNwF8hK2K N8ky6F+kEAFSG+dJSmIQG1sG64ur68n4oCDRIP++sV6HV9RF0I4NhX+1xUaAg/uKU6Ov uhoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696846305; x=1697451105; h=content-transfer-encoding:in-reply-to:autocrypt:from:references:cc :to:content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=IGY0FBb36qCXjvr1NWROo1BhBbwPapXrgUBbbWrCwgg=; b=UZuf2sWRXAce1UEu3kZnaRHZyMJRiGbGTs4jlBx0x3NOyMXulp2lpIhOPHGrZXzAwA wqcgF8xWAG93s538pp+YlEBZLy6qynnB9XkJvkhyGrTWBjT9OzPvIGG3+lU2Vbm6ClVO VOx63uX0ZnE3WTbxIfoG6zFC9RpSxz+Ek0H0oO5ReXOuBNc6DoLX6RniokpPLoLhkuhh fqsdaWAfEmk+KEXPL+YShjBywxKsCzKnErcDRVH4L3+nv2ppVxk63083IDEIqiIVdFSu DnvV9gQdVyg7BYSDa4FGqBosjOPOum63ppmMLzxP0jZro05HDNiWCc0Qe1gfxcHkIdBc 25qw== X-Gm-Message-State: AOJu0YwPIeIbYw9uq8+4fr54LvRee1SpUiHIKNZexPs40+mE1Zj1dJqg HCp/Sd2+lbnrXcTEa9ElUukecA== X-Google-Smtp-Source: AGHT+IF06qVhDrgVrjGhmjIW9BJWBQjOdsy3kHUarYO2CVraC19PXLzrv97AeJbJjiTBliTEWl/mgw== X-Received: by 2002:a2e:3803:0:b0:2bc:fa8f:83c4 with SMTP id f3-20020a2e3803000000b002bcfa8f83c4mr12461404lja.39.1696846305151; Mon, 09 Oct 2023 03:11:45 -0700 (PDT) Received: from [192.168.200.173] (178235177147.dynamic-4-waw-k-1-1-0.vectranet.pl. [178.235.177.147]) by smtp.gmail.com with ESMTPSA id d17-20020a2e3311000000b002bcb9956a69sm2001771ljc.41.2023.10.09.03.11.43 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 09 Oct 2023 03:11:44 -0700 (PDT) Message-ID: <070bd916-d4d6-41c2-9f51-af35e80c96b9@linaro.org> Date: Mon, 9 Oct 2023 12:11:42 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v6 2/8] hugetlb: restructure pool allocations Content-Language: en-US To: Mike Kravetz , Anshuman Khandual Cc: Andrew Morton , Xiongchun Duan , Barry Song <21cnbao@gmail.com>, David Rientjes , Miaohe Lin , Matthew Wilcox , linux-mm@kvack.org, Naoya Horiguchi , Joao Martins , David Hildenbrand , Michal Hocko , Oscar Salvador , linux-kernel@vger.kernel.org References: <20230925234837.86786-3-mike.kravetz@oracle.com> <20230929205731.GA10357@monkey> <20231006030836.GC86415@monkey> <5d14b681-a2c9-4210-a0c1-bf38708b7998@linaro.org> <20231006223512.GC3861@monkey> <20231009032926.GA3376@monkey> From: Konrad Dybcio Autocrypt: addr=konrad.dybcio@linaro.org; keydata= xsFNBF9ALYUBEADWAhxdTBWrwAgDQQzc1O/bJ5O7b6cXYxwbBd9xKP7MICh5YA0DcCjJSOum BB/OmIWU6X+LZW6P88ZmHe+KeyABLMP5s1tJNK1j4ntT7mECcWZDzafPWF4F6m4WJOG27kTJ HGWdmtO+RvadOVi6CoUDqALsmfS3MUG5Pj2Ne9+0jRg4hEnB92AyF9rW2G3qisFcwPgvatt7 TXD5E38mLyOPOUyXNj9XpDbt1hNwKQfiidmPh5e7VNAWRnW1iCMMoKqzM1Anzq7e5Afyeifz zRcQPLaqrPjnKqZGL2BKQSZDh6NkI5ZLRhhHQf61fkWcUpTp1oDC6jWVfT7hwRVIQLrrNj9G MpPzrlN4YuAqKeIer1FMt8cq64ifgTzxHzXsMcUdclzq2LTk2RXaPl6Jg/IXWqUClJHbamSk t1bfif3SnmhA6TiNvEpDKPiT3IDs42THU6ygslrBxyROQPWLI9IL1y8S6RtEh8H+NZQWZNzm UQ3imZirlPjxZtvz1BtnnBWS06e7x/UEAguj7VHCuymVgpl2Za17d1jj81YN5Rp5L9GXxkV1 aUEwONM3eCI3qcYm5JNc5X+JthZOWsbIPSC1Rhxz3JmWIwP1udr5E3oNRe9u2LIEq+wH/toH kpPDhTeMkvt4KfE5m5ercid9+ZXAqoaYLUL4HCEw+HW0DXcKDwARAQABzShLb25yYWQgRHli Y2lvIDxrb25yYWQuZHliY2lvQGxpbmFyby5vcmc+wsGOBBMBCAA4FiEEU24if9oCL2zdAAQV R4cBcg5dfFgFAmQ5bqwCGwMFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQR4cBcg5dfFjO BQ//YQV6fkbqQCceYebGg6TiisWCy8LG77zV7DB0VMIWJv7Km7Sz0QQrHQVzhEr3trNenZrf yy+o2tQOF2biICzbLM8oyQPY8B///KJTWI2khoB8IJSJq3kNG68NjPg2vkP6CMltC/X3ohAo xL2UgwN5vj74QnlNneOjc0vGbtA7zURNhTz5P/YuTudCqcAbxJkbqZM4WymjQhe0XgwHLkiH 5LHSZ31MRKp/+4Kqs4DTXMctc7vFhtUdmatAExDKw8oEz5NbskKbW+qHjW1XUcUIrxRr667V GWH6MkVceT9ZBrtLoSzMLYaQXvi3sSAup0qiJiBYszc/VOu3RbIpNLRcXN3KYuxdQAptacTE mA+5+4Y4DfC3rUSun+hWLDeac9z9jjHm5rE998OqZnOU9aztbd6zQG5VL6EKgsVXAZD4D3RP x1NaAjdA3MD06eyvbOWiA5NSzIcC8UIQvgx09xm7dThCuQYJR4Yxjd+9JPJHI6apzNZpDGvQ BBZzvwxV6L1CojUEpnilmMG1ZOTstktWpNzw3G2Gis0XihDUef0MWVsQYJAl0wfiv/0By+XK mm2zRR+l/dnzxnlbgJ5pO0imC2w0TVxLkAp0eo0LHw619finad2u6UPQAkZ4oj++iIGrJkt5 Lkn2XgB+IW8ESflz6nDY3b5KQRF8Z6XLP0+IEdLOOARkOW7yEgorBgEEAZdVAQUBAQdAwmUx xrbSCx2ksDxz7rFFGX1KmTkdRtcgC6F3NfuNYkYDAQgHwsF2BBgBCAAgFiEEU24if9oCL2zd AAQVR4cBcg5dfFgFAmQ5bvICGwwACgkQR4cBcg5dfFju1Q//Xta1ShwL0MLSC1KL1lXGXeRM 8arzfyiB5wJ9tb9U/nZvhhdfilEDLe0jKJY0RJErbdRHsalwQCrtq/1ewQpMpsRxXzAjgfRN jc4tgxRWmI+aVTzSRpywNahzZBT695hMz81cVZJoZzaV0KaMTlSnBkrviPz1nIGHYCHJxF9r cIu0GSIyUjZ/7xslxdvjpLth16H27JCWDzDqIQMtg61063gNyEyWgt1qRSaK14JIH/DoYRfn jfFQSC8bffFjat7BQGFz4ZpRavkMUFuDirn5Tf28oc5ebe2cIHp4/kajTx/7JOxWZ80U70mA cBgEeYSrYYnX+UJsSxpzLc/0sT1eRJDEhI4XIQM4ClIzpsCIN5HnVF76UQXh3a9zpwh3dk8i bhN/URmCOTH+LHNJYN/MxY8wuukq877DWB7k86pBs5IDLAXmW8v3gIDWyIcgYqb2v8QO2Mqx YMqL7UZxVLul4/JbllsQB8F/fNI8AfttmAQL9cwo6C8yDTXKdho920W4WUR9k8NT/OBqWSyk bGqMHex48FVZhexNPYOd58EY9/7mL5u0sJmo+jTeb4JBgIbFPJCFyng4HwbniWgQJZ1WqaUC nas9J77uICis2WH7N8Bs9jy0wQYezNzqS+FxoNXmDQg2jetX8en4bO2Di7Pmx0jXA4TOb9TM izWDgYvmBE8= In-Reply-To: <20231009032926.GA3376@monkey> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 19ADC1C0003 X-Stat-Signature: epsgm8s8awdnpce3pitut33ab7sjbbuw X-Rspam-User: X-HE-Tag: 1696846306-235280 X-HE-Meta: U2FsdGVkX1+hjGgYuGvAaJAt0FJUig2sn7jd2LCkhtvFTcxh/VFhEYlGe/ZOvxLJGTecd0drLxIUwC4IdwDl11PwE3mp3t7/Um2XJlz0DbbAfPgNYjsn2cKh6KYdaPSfB8SZcFzjdmKjNdSNQTvTHwmuxHrISNSs/Y8u8xX7xejCuJ/J1fXWVjBvBkMKJ56qHpBHnBOdOaVcAU/zf/d3PO12FRIx0haFtoPTLlCTv3BlYHnd9ozdgM1XW2RT2a6rAmCu8oY1ObqCMSUPBifGUSZYjGBq1lFsfOxWDgRtmosR7XeBSglPLYhFFkoOrJlEWiUX2MAsOsSQ/S1WSVrDouGwvn+m0ihll7QTKuTcG+HeNDPpAdhZoETVENssE4xKRLLNSvX48mB9cVUJquwGtfzSz8+vb3DSTGlrVb6CGsK5VBYXtnImhCnGNjbSryIagoqqkIguNRl+Shzb3XJ2x61YVq8AJ0iuvpcqCPqvz7NwLw+zzShjy+mOZgXPIGjhs0Lmm1DRK3JvW/U8FYLqrklhqLKrv+Jd4QOuf9sCkHzbtfAVSTiJ6DEJ+BHHMmvWqNWAA0qkBuwZFUXV4wEdJUSbd3tXmBYMqcGSU5C0J6WsmKm6Ax5ja4lzk1j4dUWTdQpY39qTMLkdiXxDM+8PQJY9a+Uj7kyWQgLUcBPypsfsUZGf7LiFZRHiPmmyxO/XKqoyTYC5B8Lvyct53zR9mVo7By9gVkk5y43bVCk2BFPbdRxl58NnYd7bvgDkFj39rEqXUUEo6dC+CZb0/45Qy+NdFIqtlzjoRg12KRpTx4YYCKdJYPcDLlKwkSoKVQnbcmaLUzFsI3PotK3Sm0O3yIXIL7Ga/deI7S8ryYxDrZEOJLaKAmGGqaSMxpnwOS+S5NvAQD2JvmZET55OI1bqNxVpFUZYgyRDC0IndLRUGU5KIFAYm2v75M+ykFQI6JTrwEK1qiiST2f1LVSDvP2 w1meFN8k XO03Nn1cmoKKbDaupQjqhONFg/qwHZX01kX2pQ3pP4jze1jM9MbGUn5Q8r6dX6JR0POpWPdgVJEk3Em3bn2ZfHh2kK3kM2wl89g33RZuhLXCDX/mt+zjTP37xuCQBaVrsJmi2xSwpCHO+MKd7abLms6FNzRDF9h/W8vOiGID+0+/XPXSxsBBiBxN5LM1IzYL141FjiQwdFdmcvs6kCGgElyfD9RcFQrKe/B3JX0X+qnW7E5aV4E3eLAmC2VWQTYD3xGLPLLJGms0ETb/2gPyqVHwjojDqSr+zI6pOJ35l0P+pKm4FNFCuZzFovlfyQE7+aFG0I7y4O/O1nNunklxREivBGPP4QmAzqZWAYDDPuc/zuWqCJ3sxYQu4zTti289G9xutTh78rdOWpa4uY5x9UxWo4EJxBhAIOsl9DIN56YnHzIowiUkNrhv9X2Yt6u86FH084cN4U70YaQ99i1UR2tS1HvvG0XcZFma60CbZPhRoB67n2/rabEvM5PVLCIZSjadlzr/IcusdOn8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 9.10.2023 05:29, Mike Kravetz wrote: > On 10/06/23 15:35, Mike Kravetz wrote: >> On 10/06/23 23:39, Konrad Dybcio wrote: >>> On 6.10.2023 05:08, Mike Kravetz wrote: >>>> On 10/02/23 11:57, Konrad Dybcio wrote: >>>>> >>>>> >>>>> On 9/29/23 22:57, Mike Kravetz wrote: >>>>>> On 09/27/23 13:26, Konrad Dybcio wrote: >>>>>>> >>>>>>> >>>>>>> On 26.09.2023 01:48, Mike Kravetz wrote: >>>>>>>> Allocation of a hugetlb page for the hugetlb pool is done by the routine >>>>>>>> alloc_pool_huge_page. This routine will allocate contiguous pages from >>>>>>>> a low level allocator, prep the pages for usage as a hugetlb page and >>>>>>>> then add the resulting hugetlb page to the pool. >>>>>>>> >>>>>>>> In the 'prep' stage, optional vmemmap optimization is done. For >>>>>>>> performance reasons we want to perform vmemmap optimization on multiple >>>>>>>> hugetlb pages at once. To do this, restructure the hugetlb pool >>>>>>>> allocation code such that vmemmap optimization can be isolated and later >>>>>>>> batched. >>>>>>>> >>>>>>>> The code to allocate hugetlb pages from bootmem was also modified to >>>>>>>> allow batching. >>>>>>>> >>>>>>>> No functional changes, only code restructure. >>>>>>>> >>>>>>>> Signed-off-by: Mike Kravetz >>>>>>>> Reviewed-by: Muchun Song >>>>>>>> --- >>>>>>> Hi, looks like this patch prevents today's next from booting >>>>>>> on at least one Qualcomm ARM64 platform. Reverting it makes >>>>>>> the device boot again. >>>>>> >>>>>> Can you share the config used and any other specific information such as >>>>>> kernel command line. >>>>> Later this week. >>>> >>>> As mentioned, I have been unable to reproduce on arm64 platforms I can >>>> access. I have tried various config and boot options. While doing so, >>>> I came across one issue impacting kernels compiled without >>>> CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP defined. This is not something >>>> that would prevent booting. >>>> >>>> I will send out an updated version series in the hope that any other >>>> issues may be discovered. >>> I'm pushing the "later this week" by answering near end of calendar >>> day, Friday, but it seems like this patch in v7 still prevents the >>> device from booting.. >>> >>> You can find my defconfig at the link below. >>> >>> https://gist.github.com/konradybcio/d865f8dc9b12a98ba3875ec5a9aac42e >>> >> >> Thanks! >> >> I assume there is no further information such as any console output? >> Did any of you other arm64 platforms have this issue? >> >> Just trying to get as much information as possible to get to root cause. > > I have not had success isolating the issue with your config file. > > Since the only code changes in this patch deal with allocating hugetlb > pages, I assume this is what you are doing? Can you let me know how you > are performing the allocations? I assume it is on the kernel command > line as these would be processed earliest in boot. > > If you are not allocating hugetlb pages, then I need to think of what > else may be happening. > > Anshuman, any chance you (or someone else with access to arm64 platforms) > could throw this on any platforms you have access to for a quick test? I managed to get a boot log: https://pastebin.com/GwurpCw9 This is using arch/arm64/boot/dts/qcom/sm8550-mtp.dts for reference Konrad