From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3954EC54E58 for ; Mon, 11 Mar 2024 14:56:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 97D096B00A4; Mon, 11 Mar 2024 10:56:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 92D206B00A5; Mon, 11 Mar 2024 10:56:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7A6AA6B00A6; Mon, 11 Mar 2024 10:56:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 65DDD6B00A4 for ; Mon, 11 Mar 2024 10:56:46 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 0128C140CE6 for ; Mon, 11 Mar 2024 14:56:45 +0000 (UTC) X-FDA: 81885060012.10.18D67A7 Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by imf25.hostedemail.com (Postfix) with ESMTP id 1BE5BA0006 for ; Mon, 11 Mar 2024 14:56:42 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=samsung.com header.s=mail20170921 header.b=RkK6CGw9; dmarc=pass (policy=none) header.from=samsung.com; spf=pass (imf25.hostedemail.com: domain of m.szyprowski@samsung.com designates 210.118.77.12 as permitted sender) smtp.mailfrom=m.szyprowski@samsung.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710169003; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=uOPd2AwFVdGKb+UngFBePmqdQoXqrn4uH0VfBRH90zo=; b=vZuiF4YtkyRHal/sMQxb5lhss7+9MOnnLDLLL2MqGw6b/Ze9HQB4Exe54gkrFnN/gsKAAt rZMH5EkxCLifU2TgKiKXDMnakAy6BtbO1CSPNhFPX8IDI84EHLQZE8F4Wv2OzQc4SdODkW x2I7XhZBRKWDXJN3FhrtfJ6MUQQgAK0= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=samsung.com header.s=mail20170921 header.b=RkK6CGw9; dmarc=pass (policy=none) header.from=samsung.com; spf=pass (imf25.hostedemail.com: domain of m.szyprowski@samsung.com designates 210.118.77.12 as permitted sender) smtp.mailfrom=m.szyprowski@samsung.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710169003; a=rsa-sha256; cv=none; b=urWMKqDdRQnTPyT50Oz/+TnGx9ZqyX1WpRCyWpR5qUZ5MGaxLvdtS9WA/yUejq4m29E5fD Hobzk+AybD+p5bTvVvbdXT92L0qMIHA0XkOG1rStGTbhWz90PHVvyYA1KGY8ijliUNf9zY ndgbNbD5qnZ6wGJgdx4BLiZQ8CwwvD0= Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout2.w1.samsung.com (KnoxPortal) with ESMTP id 20240311145640euoutp0245343b7c572f2776359c0a7f4882ef90~7vaUc_AFh0524305243euoutp02G for ; Mon, 11 Mar 2024 14:56:40 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout2.w1.samsung.com 20240311145640euoutp0245343b7c572f2776359c0a7f4882ef90~7vaUc_AFh0524305243euoutp02G DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1710169000; bh=uOPd2AwFVdGKb+UngFBePmqdQoXqrn4uH0VfBRH90zo=; h=Date:Subject:To:Cc:From:In-Reply-To:References:From; b=RkK6CGw91isXmWz7NCu8mBBKkjgSkwvNssUVCy4UD5IWlcL5wiZ8INxSBdHX/FOt1 dBQJgS7aUF7xggzbGH8OLf/ZfH/g7REWyRbxGoh+i0HRzFBL+d9xbcFJd9ZOMH16Sx ZmwWHl16SWC9r2IbG2f3k+UwJw2z/NswDvDL4A1A= Received: from eusmges1new.samsung.com (unknown [203.254.199.242]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20240311145639eucas1p1c2f9d8b6e5fa1c36088b4babf0c0056d~7vaUBXqwo2427124271eucas1p1k; Mon, 11 Mar 2024 14:56:39 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges1new.samsung.com (EUCPMTA) with SMTP id B8.01.09539.7AB1FE56; Mon, 11 Mar 2024 14:56:39 +0000 (GMT) Received: from eusmtrp2.samsung.com (unknown [182.198.249.139]) by eucas1p2.samsung.com (KnoxPortal) with ESMTPA id 20240311145639eucas1p2dee661847c8c8a29dcba021d800ca73a~7vaTYZ1MU1472714727eucas1p25; Mon, 11 Mar 2024 14:56:39 +0000 (GMT) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eusmtrp2.samsung.com (KnoxPortal) with ESMTP id 20240311145639eusmtrp290064aef32d0f8802d12f1e81dc11274~7vaTXWamJ1058210582eusmtrp2d; Mon, 11 Mar 2024 14:56:39 +0000 (GMT) X-AuditID: cbfec7f2-515ff70000002543-61-65ef1ba733ca Received: from eusmtip1.samsung.com ( [203.254.199.221]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id BC.6F.10702.7AB1FE56; Mon, 11 Mar 2024 14:56:39 +0000 (GMT) Received: from [106.210.134.192] (unknown [106.210.134.192]) by eusmtip1.samsung.com (KnoxPortal) with ESMTPA id 20240311145638eusmtip1ea6702de7106114eb7a6c1426ce239d7~7vaSRcF-G3139931399eusmtip1A; Mon, 11 Mar 2024 14:56:37 +0000 (GMT) Message-ID: <8abb1a69-6cbd-4a36-ab1d-d269cdafa391@samsung.com> Date: Mon, 11 Mar 2024 15:56:37 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v3] ARM64: Dynamically allocate cpumasks and increase supported CPUs to 512 Content-Language: en-US To: Mark Rutland , "Christoph Lameter (Ampere)" Cc: Catalin Marinas , "linux-pm@vger.kernel.org" , "Rafael J. Wysocki" , Viresh Kumar , Will Deacon , Jonathan.Cameron@huawei.com, Matteo.Carlini@arm.com, Valentin.Schneider@arm.com, akpm@linux-foundation.org, anshuman.khandual@arm.com, Eric Mackay , dave.kleikamp@oracle.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux@armlinux.org.uk, robin.murphy@arm.com, vanshikonda@os.amperecomputing.com, yang@os.amperecomputing.com, Nishanth Menon , Stephen Boyd From: Marek Szyprowski In-Reply-To: Content-Transfer-Encoding: 7bit X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFupnk+LIzCtJLcpLzFFi42LZduzned3l0u9TDV7OMraYs34Nm8W+E8kW 75f1MFpc//aG0eLAs0vMFk//LWO3WLXwGpvFpsfXWC0u75rDZnFvzX9Wi8+9RxgtDk3dy2ix 9PpFJovfy74zWbz5cZbJYu6XqcwWBz88YbX4d20ji8X2+T+ZLV7eusNisfnBMTaLljumFr1L TrI7iHusmbeG0ePytYvMHi1H3rJ6bFrVyeax6dMkdo8TM36zeDy5Mp3JY/OSeo+PT2+xeHRf vsHucfzGdiaPz5vkAniiuGxSUnMyy1KL9O0SuDLmN1xiLvjKW/Gs8xlbA+MS7i5GTg4JAROJ rx9WMXUxcnEICaxglNg9cxEbhPOFUWLV7G+sEM5nRonWIwuYYFrOztvKDGILCSxnlPh4nwei 6COjxI5pRxlBErwCdhJfd90As1kEVCVOPVjLDhEXlDg58wkLiC0qIC9x/9YMsLiwQKLEq+0L wIYyC4hL3HoyH2yZiECYxO7zn8HuYxZYyyrR+eYN2FA2AUOJrrddbCA2p4COxJxP/SwQzfIS 29/OYQZpkBBo45LoXfUc6mwXiRsfLjJC2MISr45vYYewZST+75zPBNHQziix4Pd9KGcCo0TD 81tQHdYSd879AlrHAbRCU2L9Ln2IsKPEq7717CBhCQE+iRtvBSGO4JOYtG06M0SYV6KjTQii Wk1i1vF1cGsPXrjEPIFRaRZSuMxC8v8sJO/MQti7gJFlFaN4amlxbnpqsWFearlecWJucWle ul5yfu4mRmCaPf3v+KcdjHNffdQ7xMjEwXiIUYKDWUmE97XO21Qh3pTEyqrUovz4otKc1OJD jNIcLErivKop8qlCAumJJanZqakFqUUwWSYOTqkGJhNdZ4742F8h58z+xYuofzh1Mu3kes49 Gbq8WXe/sirPNao9HR0R3Tppna1kpdp2jt9JH838jb7MMFgnwPsnZNex/4c/+F5O3Tbx6sTG KxLmU0RexGj8ijj1UOrG3E2swoKceoHBXsVLdKbHSXCWLhbvNzy2d0eakXQp3/LWCbezrM5V 26qz84X+nubJu2tJhPPTtIl3ix6dT/di2PlmSvm6rB35J8Wfux5pNQ6pinnw6U1Zofxj5aab Sg6JJ2K/X5z61idQPiQ7dEbUTLUvPy1/aOzaHh1wrlLw4ou+nPqgD8FtmRVZub9VI0KXTy8M 2f3717NzbNFiiosntp2euTC25PzUOv07Fl83KXD5KbEUZyQaajEXFScCAAm8wP4iBAAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrEKsWRmVeSWpSXmKPExsVy+t/xu7rLpd+nGrRMZrOYs34Nm8W+E8kW 75f1MFpc//aG0eLAs0vMFk//LWO3WLXwGpvFpsfXWC0u75rDZnFvzX9Wi8+9RxgtDk3dy2ix 9PpFJovfy74zWbz5cZbJYu6XqcwWBz88YbX4d20ji8X2+T+ZLV7eusNisfnBMTaLljumFr1L TrI7iHusmbeG0ePytYvMHi1H3rJ6bFrVyeax6dMkdo8TM36zeDy5Mp3JY/OSeo+PT2+xeHRf vsHucfzGdiaPz5vkAnii9GyK8ktLUhUy8otLbJWiDS2M9AwtLfSMTCz1DI3NY62MTJX07WxS UnMyy1KL9O0S9DLmN1xiLvjKW/Gs8xlbA+MS7i5GTg4JAROJs/O2MncxcnEICSxllOhetYcR IiEjcXJaAyuELSzx51oXG0TRe0aJvVNusIMkeAXsJL7uugHWwCKgKnHqwVqouKDEyZlPWEBs UQF5ifu3ZoDFhQUSJV5tX8AMYjMLiEvcejKfCcQWEQiTeHmiiwlkAbPAWlaJprPPoE56zSSx +ep3sA1sAoYSXW9BzuDk4BTQkZjzqZ8FYpKZRNfWLkYIW15i+9s5zBMYhWYhOWQWkoWzkLTM QtKygJFlFaNIamlxbnpusZFecWJucWleul5yfu4mRmBi2Xbs55YdjCtffdQ7xMjEwXiIUYKD WUmE97XO21Qh3pTEyqrUovz4otKc1OJDjKbA0JjILCWanA9MbXkl8YZmBqaGJmaWBqaWZsZK 4ryeBR2JQgLpiSWp2ampBalFMH1MHJxSDUydd9+kau6+wWGxWFX3XuaMbiNxn7zSEK5zdxvN pVX9GITnCdrMFTne8Ptl/cMNXvp9gW/dJr7dUJ8Wv79y37nlUbUraySfFvy9/JZ9b9Plnw7v /Cunb7s/d/php6cTIhm+G4UXK84wafsZLz3VMvzu3PPn2KV/XxIpncc3eZPAzGXe/St+188V zH470dbvbvCOn5rHu/gt502/rcSnpSzFenrq5Sd5S3TWK2XnqU77smLJvSU/gxQ6D7Kd59W6 wbkru0D1aLZuyqLVOZ/3vH1x+qjF6wtKny5F3L+afPiis/AdB1fno1yy275tzl13wfSvedub Vzw9pVwFV4q/LHBL2e27ze5xa3PpOvkbrx4kKrEUZyQaajEXFScCAAeoS561AwAA X-CMS-MailID: 20240311145639eucas1p2dee661847c8c8a29dcba021d800ca73a X-Msg-Generator: CA Content-Type: text/plain; charset="utf-8" X-RootMTR: 20240308140130eucas1p1259c805a0b6491ce2f69c6fca0264b1f X-EPHeader: CA CMS-TYPE: 201P X-CMS-RootMailID: 20240308140130eucas1p1259c805a0b6491ce2f69c6fca0264b1f References: <37099a57-b655-3b3a-56d0-5f7fbd49d7db@gentwo.org> X-Rspamd-Queue-Id: 1BE5BA0006 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: 8yud6weif36h46wmeng8iagj1ushmki4 X-HE-Tag: 1710169002-285060 X-HE-Meta: U2FsdGVkX1/27yNBoRvFgLSaFw65B14+3fKWzw9/D/AkenIWMEhOiPoIZnMueNZmO3xyotLaSTUM3jpNoFm6Wj0wOs9H4u5GpSkzeBuxIiQ0+7mVtqPcfvBtd4pOcBGcXaxEgQ+4/s/2jezIpj683MiTOB2KX128R4mfoM59WYZKF/zBQY1Augbd7s2v8/Nm/DAUMWiDvIdk42p8/NC8h+7olkOD3QyK1Ve9rg9/1F6KEqwacPUYA7EzclYwF8rWDTWV1MeO8QbT3Ocz3cJ1N1YHeF2psTiI7i2LsyAGJ3lsYi/p10Izsj1YXlqrswdzsdmc3A113/m1MEUKX4SxTcZKgSCV383fmHp9BA2qAQKUCjj7tBmxP9fiOThKloBSegZWJnA5pS3nm2RJPIFd8EW8/DSkhH1lwqN7Itet0n3OP/GHNQ62VTetXXvx+FyRj4TpNJ+J2A7jQJZFD7K5kpAG1Enu492m0kNJ+89YcJ3NnK8PcmwuzqRYaw6QFtcjQ/mSAuOm6uk5ennqNuZklU/VTpwe2V4ckjXJQZefvAtlnSKl+/3vibgRiYkT1+Qm+MJ1Pb+iNyg6XBK4AAsq7TXJ8QSNuJdLV1JXCOndalffi3PCi16xfvwUq7YXYx6+iGYCG3M7lCIW/rSyu+TiCu1QPTKM68jCkrgsLzenMzoVNO4mU+bpnHgsMD22eNlnxlzxDZsqUGcNG5kfHxdP8ffv/UuCkbGIWQVJFYRey/f/zZwtL7mlBmi4NfB0Ta9NqXXL/TAiiTc2fGTZbI0QuaZtuOThMsqeYQtR1IhJIXzZ4AqNd20p7W02Kl4qNo0fK4o4cpgarosfRFW46oKtTNllZFv1vMPAsEbJ2XFVuRG28oreGY5xKB10FPR/owYsjriNhwbFqwNu5nq7G6//TVe+LkPf2UEXhc3JhXl/N6IpK44Ep8UWyUt3FkqkRmiOh/tcyWXqUT7RBEeS+TQ S0NUJtMB iBcrXsRbH3Zwm3zSff2hBd2AdB4Aqk/KTihxDa4pcSdYrgH4fd7lM4Lk8f1jAcUzRKyJtEGwmdXdQpKQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 11.03.2024 13:12, Mark Rutland wrote: > On Fri, Mar 08, 2024 at 09:08:59AM -0800, Christoph Lameter (Ampere) wrote: >> On Fri, 8 Mar 2024, Marek Szyprowski wrote: >>>>> It looks that cpufreq-dt and/or opp drivers needs some adjustments >>>>> related with this change. >>>> That's strange. Is this with defconfig? I wonder whether NR_CPUS being >>>> larger caused the issue with this specific code. Otherwise >>>> CPUMASK_OFFSTACK may not work that well on arm64. >> cpumask handling must use the accessor functions provided in >> include/linux/cpumask.h for declaring and accessing cpumasks. It is likely >> related to the driver opencoding one of the accessors. > I took a look at both the OPP code and the cpufreq-dt code and it looks like > those are doign the right thing w.r.t. cpumask manipulation (i.e. they only use > the cpumask accessors, and use the cpumask_var_*() functions to dynamically > allocate/free cpumasks). Maybe I've missed something, but superficially those > look right. > > Marek, can you try reverting this commit and trying defconfig + NR_CPUS=512? Yes, with $subject reverted and CONFIG_NR_CPUS=512 everything works fine, so it must be something else broken. > That'll have CPUMASK_OFFSTACK=n, and: > > * If that blows up, we know the problem is independent of CPUMASK_OFFSTACK, and > has something to do with large cpumasks (either a driver bug, or elsewhere). > > * If that doesn't blow up, it suggests the problem is related to > CPUMASK_OFFSTACK rather than with large cpumasks specifically. > > Either way, we probably need to revert this patch for now, as this won't have > enough time to soak in linux-next in time for v6.9. Best regards -- Marek Szyprowski, PhD Samsung R&D Institute Poland