From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 71CF4C2BA18 for ; Fri, 21 Jun 2024 03:30:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D4DDA8D0127; Thu, 20 Jun 2024 23:30:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CD66F8D00DB; Thu, 20 Jun 2024 23:30:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B4FF78D0127; Thu, 20 Jun 2024 23:30:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 961FE8D00DB for ; Thu, 20 Jun 2024 23:30:30 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 2AE6C120804 for ; Fri, 21 Jun 2024 03:30:30 +0000 (UTC) X-FDA: 82253468220.14.AE04268 Received: from APC01-TYZ-obe.outbound.protection.outlook.com (mail-tyzapc01on2074.outbound.protection.outlook.com [40.107.117.74]) by imf29.hostedemail.com (Postfix) with ESMTP id C8345120017 for ; Fri, 21 Jun 2024 03:30:26 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=oppo.com header.s=selector1 header.b=tDTMFMgt; spf=pass (imf29.hostedemail.com: domain of hailong.liu@oppo.com designates 40.107.117.74 as permitted sender) smtp.mailfrom=hailong.liu@oppo.com; dmarc=pass (policy=quarantine) header.from=oppo.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718940620; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yoxSo6n5OyRjUQ1Jz52s4G5ESzFfW2UgwEtOIbPtFZg=; b=lA8QlXcBJpcpmTJqxHO3bPxfEypaua46p93jTHdjgDf0WV6dk+GJkMoWqA/wgecKKqpB7e q0CTVqQXlhTWubLqaIIclARp8zrPeZhT2T3grg8baJgkXB6Y0Sa9phxKPUjQoZhOwicKx8 7OzQlHOCO8ZgxrR6dXY1tATLy0I5E8Q= ARC-Authentication-Results: i=2; imf29.hostedemail.com; dkim=pass header.d=oppo.com header.s=selector1 header.b=tDTMFMgt; spf=pass (imf29.hostedemail.com: domain of hailong.liu@oppo.com designates 40.107.117.74 as permitted sender) smtp.mailfrom=hailong.liu@oppo.com; dmarc=pass (policy=quarantine) header.from=oppo.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1718940620; a=rsa-sha256; cv=pass; b=6kEdz5tsnjSanipkMbbCvPzqArh0Zi9n9Z/GBbhmRgO2n56CWL09//FAIeAfi32WLuoKki U2HhNCVFrQOczyqGAz8mAIZ4XqlRtqYkMqP2rUDkwGRlMy+MFBaQr2TYl50BmmmscubQZP PnboBnzHJgT3kftj0eEuE1FTby8GV4c= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Fpvoe9pC02BlWt2W6ieAJ5dniLHhA3ybK6joaJhhxQaeqwD51VgAfLIH5hDTBDJqKngiL1G1lbtl3NPU9ZHGBuu1GzCDQm98wF6gXZP/1/83MgagptOtoD1lH2WocFOPwt3DB6f4brV8MdMBSrogUdUzZ9vfZqAIveGcQW6z3bEPVIS8j4dzskLffIf1vk5vafS8HPShD6jg1tZ+63/tRW/r8uaK6FxjtCFI8Dy3iUVVaDkcokVCeS4+8vBxdMOQTt0rbXAFjYftmzNyMDjEV3duHG4seITjLtdaykFdhw2H6ExmSeP+ABon5v7RRKVg0qLmI/MZUXClEq+n2HwVuQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=yoxSo6n5OyRjUQ1Jz52s4G5ESzFfW2UgwEtOIbPtFZg=; b=jYxk4rfYb+zG9YdeoNkY0xYYRzWpYTg5ZKMc+aDrLIYjidx54mmgfQKFyNKsNKQqDXA3yJ7Sy0p0G8pdQpEK+SUzDoJ2u78ZRz9fVeVb2X5Z/vI2GLuNOmPKPc8Sl91v3GNwSXtvM60XioxbgW1z3KeBN7Hw6Cphgy6sqXm/j6CALDA/OUSD1O35Np4Hy7aF/cWgjXwBrnVW8WP06FSkG43czufQqynPa1j51vaSgVGRUJY+jAJvh+BzHar2zB/kl+D8+ny+2ELswZ8X8YDq4PmRqsZgWlgyN0HHgNp3Vm8rHEznZdVXrKSaoqwDyVLHI/mCmB7TlXyAQt1mQc+jSw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 58.252.5.68) smtp.rcpttodomain=draconx.ca smtp.mailfrom=oppo.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=oppo.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oppo.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yoxSo6n5OyRjUQ1Jz52s4G5ESzFfW2UgwEtOIbPtFZg=; b=tDTMFMgtk/BKu7m4konqFwpojVJHN1EYw9oZUrxVOn6FRc+VHp+aB7nH3W+g45Rb5i/E2Rn7B7Kd7wiCYzymAsk7YJwGwCmlu4WwTEtOapE0sslgqE1motXUcwmmmc4c1XPFreP+7p/PGR9Q3/Vaq3LEiPyecnuv5TC+yos257k= Received: from KL1PR01CA0099.apcprd01.prod.exchangelabs.com (2603:1096:820:3::15) by TY0PR02MB5924.apcprd02.prod.outlook.com (2603:1096:400:205::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7698.19; Fri, 21 Jun 2024 03:30:22 +0000 Received: from HK3PEPF0000021C.apcprd03.prod.outlook.com (2603:1096:820:3:cafe::9b) by KL1PR01CA0099.outlook.office365.com (2603:1096:820:3::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.33 via Frontend Transport; Fri, 21 Jun 2024 03:30:21 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 58.252.5.68) smtp.mailfrom=oppo.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=oppo.com; Received-SPF: Pass (protection.outlook.com: domain of oppo.com designates 58.252.5.68 as permitted sender) receiver=protection.outlook.com; client-ip=58.252.5.68; helo=mail.oppo.com; pr=C Received: from mail.oppo.com (58.252.5.68) by HK3PEPF0000021C.mail.protection.outlook.com (10.167.8.38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7677.15 via Frontend Transport; Fri, 21 Jun 2024 03:30:20 +0000 Received: from oppo.com (172.16.40.118) by mailappw31.adc.com (172.16.56.198) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Fri, 21 Jun 2024 11:30:05 +0800 Date: Fri, 21 Jun 2024 11:30:05 +0800 From: Hailong Liu To: Nick Bowler CC: , Linux regressions mailing list , , , Baoquan He , "Uladzislau Rezki (Sony)" , Andrew Morton Subject: Re: PROBLEM: kernel crashes when running xfsdump since ~6.4 Message-ID: <20240621033005.6mccm7waduelb4m5@oppo.com> References: <75e17b57-1178-4288-b792-4ae68b19915e@draconx.ca> <00d74f24-c49c-460e-871c-d5af64701306@draconx.ca> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <00d74f24-c49c-460e-871c-d5af64701306@draconx.ca> X-Originating-IP: [172.16.40.118] X-ClientProxiedBy: mailappw31.adc.com (172.16.56.198) To mailappw31.adc.com (172.16.56.198) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: HK3PEPF0000021C:EE_|TY0PR02MB5924:EE_ X-MS-Office365-Filtering-Correlation-Id: f7136f1c-a4d8-40d0-6a17-08dc91a27ad4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230037|1800799021|82310400023|376011|36860700010; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?ue9R0KkAYSnDizo4T7une3UovIr9f1VxhT7g0mZQEkskw6xp39C7klysDpDg?= =?us-ascii?Q?BojaB3RiuPKeT+rZ0JDvhL5xk4pw3oaUS4qk5ML29Eql48mtnVFsa156bRvn?= =?us-ascii?Q?/mtE+wQnwO9e0sLR2cnjqw4eJt2YplRRlF+1qco5S60cUTRu7QS7/+0YpIAp?= =?us-ascii?Q?l1D1+eSr+dtx0KoHKVsSMEHA4sGmwO4Rf8i+m/v9gkXs9Ab5iZ6RCJEeFEso?= =?us-ascii?Q?IhvkjAjuWWjt+hHfQzFc58PEb4jxqAghI6r0ukvGsbTB8pPIbuAJ+5558WyJ?= =?us-ascii?Q?2o/5teWWaoIS8wuOQ/R2sg7wQYur3q96Ksj3J1nbbw3twRu3CnfyLxd3RJmn?= =?us-ascii?Q?SAKVkranrfOcl6Pl6lITTQa7wCtqi1+YB67IXLkJfv00VrPspIa7la2wLGky?= =?us-ascii?Q?VsgWBoUy7zk8hiK+UqyGkKdSUYTE3HtJTCfryaIGFYle4thFiQdaYDSATW/C?= =?us-ascii?Q?aIpLR5eJ6Qia4vLf+ctfVHMff+gsuie8PAmTjqTpm8aGkDt7B5TB8Mbd374q?= =?us-ascii?Q?TJw4Y2KeXWJTpi7dL4qfIHPN7Q4xThBvOKy9u1oMDkVzEmtk3DFfwM6mktRy?= =?us-ascii?Q?3czhkXKodE70db5OdzxjBmE/NU3bgxfKk6RtDRgU/Z/RDUVL0DNv6KHyeKXc?= =?us-ascii?Q?67rePC+Qccnjdf7onAk57RT84H0e2e9+RtTxWxYsvpPCBaH5QHLjvdbg5dLZ?= =?us-ascii?Q?ngmug5Zkl6SxURcrm221FpSpBsL35rWpYKzNSKLWSinyt6tpY8uIbc0SU9KK?= =?us-ascii?Q?+29cL5uU7F7nfCkO4hJcVmmfNt/JDBh/NnrApz2gNmbqaLxnrmmC9fyPf+Hw?= =?us-ascii?Q?b11WtjIxPrAOfUynvbj9m7qXmrvqjoqBV2dQroMqyJstZpXv6y0rPdOV5G7B?= =?us-ascii?Q?UDhAzaqhxUH1xzwRIz9ZKolNeaCtZRnIMFURNAjiis7sxUzL+OzpjanIXZiY?= =?us-ascii?Q?7XSYxbp6QzVhxUMC+NeDkv4kh2DZQOpqltG0qoMSorhq1UnQSYc2pJlrImyQ?= =?us-ascii?Q?oLbhSu+Zs/GP3PuyU6CABVdDIw5WmYG7WADryDg1q2N0xcUwzjnH59PXwZ7r?= =?us-ascii?Q?3x6KZYuGcSiDSA7fPcIjZvLV6pzQG62R1UU++oT/TQpoJj4prFAx8XnWpEhK?= =?us-ascii?Q?l6SNE2cbf2SBTs720cViXsrAK/aIFzTdziqo/EiO9zYVDAvf5mehCScZRH7L?= =?us-ascii?Q?792Bki2bG2a2hfIbkHtk6H+IsFoRY8BtNtp5VYH21SeRb6RE0jcbGE+FThnw?= =?us-ascii?Q?xa/AXUjDbfcxkg4tT58wU08HjbZsn08l6NWQ2Xdw8AdNQPeWzSnWR2Ts6V6b?= =?us-ascii?Q?5Qbb/MUiCt3qAcxMb49uY3sLRIjnnjlrqLpLDx/Q+dMIkudBH+vb4ZcHOe48?= =?us-ascii?Q?njcNPz6HmsiYt94U/zLxID56HM4ZyhHn9nZd/exCwqEoK1GweA=3D=3D?= X-Forefront-Antispam-Report: CIP:58.252.5.68;CTRY:CN;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.oppo.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230037)(1800799021)(82310400023)(376011)(36860700010);DIR:OUT;SFP:1101; X-OriginatorOrg: oppo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jun 2024 03:30:20.9519 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f7136f1c-a4d8-40d0-6a17-08dc91a27ad4 X-MS-Exchange-CrossTenant-Id: f1905eb1-c353-41c5-9516-62b4a54b5ee6 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f1905eb1-c353-41c5-9516-62b4a54b5ee6;Ip=[58.252.5.68];Helo=[mail.oppo.com] X-MS-Exchange-CrossTenant-AuthSource: HK3PEPF0000021C.apcprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: TY0PR02MB5924 X-Stat-Signature: difxn6w1iguzb6nq3haqzx4dw51gikf3 X-Rspam-User: X-Rspamd-Queue-Id: C8345120017 X-Rspamd-Server: rspam02 X-HE-Tag: 1718940626-480591 X-HE-Meta: U2FsdGVkX19a7S6Uf/pvnSuHobu2DFAezhoLFdPeIllrPaQOetiDHlANVXQIe2LG2cQpdLoJ0lEMf3Gk+d6l4UCpjQJmfMj8iT5qivNGvjwymKZqikQxDceinm33ogdh8QKLhH+vSDbqz/RIy7DDy94bIr1ACdz+NVxl8LhNt39ScBWyyl69RuSzIYC58n8yR95DWbIaMnru2FNFtA5lddI9rS/l+mRyUO87T3a02z5W5cM5PafgZIxUB+RsVbAIiXODBm0RYQx0WRh/c/Fup22gy114acj3QeuJWbO8fCA3q3sfIWw5tIY1tAgC8QSzjJawqWkSp5jGo4bmFbLfFHtPUvFGfAjslIo0Ocdw1qJIAHmxP/qTrkE5HV/AQBSxrY72ICppOFwmwxF2pWUOUixxdJlmdeATmzWF6HYE8ha9s3rXNlcPAQGC2A6swx0hp2ACiGgBa4yTmfQHzF2L9e4Ii0G0KG71ySJmPI8S6zr87ksVNw+lytRIkDjVQ2xFNo1vevtlWPao5nYXG9o56uR7H1xOF5R1gPZ7ZgVoHF6/qd1mPyq2a6WC041nnCwVNtfrPA0D5q9EMzRNIK6boT9bDO+3229M72bJDwO/EZnugeDL8RNqfQ/m0gKCEGL65RvjmniO+BRxiT2JPbmeeDdOdhgFL+77lkrEvcrDndKAjgJZIENloEahtJZLItawBaQz8s4F6/Wd9R/nG1qGWNUuQIY0SKlmxjaIgEBJHLUahVgN183HcMKgvyKnImUG8+/ZJaOpF/0UU0ZHhyyFC61Vt0Tmi3a6YQh/aUqVbvYKb+p0RlpPsWr/xC6adz8q7imCZtlWr3Tb9T20mt2uXUIiz7H+56i/BqFfX347hS8Ep+jTX0/9fijxgdRfV57ofvMUq3DY950qGtCh3m0b/R30sXLxYu34qiGY2EpmZVDNOmiMq0KmcIA21kUe8IUHkdjwxodRr7J97cN/Thh qBtz+Diq gr6UlsR0Lcvkx2LXDXflf4vHEJc4umbfXI8jftIrYxMQm6FoLVwqTT5wh8kQoW1E/VuY4/fZQv+A9RJMfzQ4J4LDfs5lAWPLuOY75pgEgsErvAlVdDbR9eNtZn9WB0/Rfeg9SU884oXsozgMOg5BHLu/TTo8URUIuLnpSFbGtwregRCw75hiZr40B+3hJrTAio//6vRGQaAJio172l2ANtUXFxwVClKTHqNxijzM4Hy/V/AXCrPJ1L1erFwxMU4L3A9BZ/cApGSE89061zFIvoSnLM/A3XI/3Jttz2TBrD/pX50asEEErOWOQzO19SR0pfdQZ4nCNSfeXHGWPhnDHDW/EAvnt5tiaJEt1QTIuxEfiB/XGSGXj+2UzC6bNlXS+bkxrQ9NnDKnWlykFdbjJHMPSG12ohr5Jkba2 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, 20. Jun 14:02, Nick Bowler wrote: > On 2024-06-20 02:19, Nick Bowler wrote: > > After upgrading my sparc to 6.9.5 I noticed that attempting to run > > xfsdump instantly (within a couple seconds) and reliably crashes the > > kernel. The same problem is also observed on 6.10-rc4. > [...] > > 062eacf57ad91b5c272f89dc964fd6dd9715ea7d is the first bad commit > > commit 062eacf57ad91b5c272f89dc964fd6dd9715ea7d > > Author: Uladzislau Rezki (Sony) > > Date: Thu Mar 30 21:06:38 2023 +0200 > > > > mm: vmalloc: remove a global vmap_blocks xarray > > I think I might see what is happening here. > > On this machine, there are two CPUs numbered 0 and 2 (there is no CPU1). > +Baoquan Ahh, I thought you are right. addr_to_vb_xa assume that the CPU numbers are contiguous. I don't have knowledge about CPU at all. Technically change the implement addr_to_vb_xa() to return &per_cpu(vmap_block_queue, raw_smp_processor_id()).vmap_blocks; would also work, but it violate the load balance. Wating for experts reply. > The per-cpu variables in mm/vmalloc.c are initialized like this, in > vmalloc_init > > for_each_possible_cpu(i) { > /* ... */ > vbq = &per_cpu(vmap_block_queue, i); > /* initialize stuff in vbq */ > } > > This loops over the set bits of cpu_possible_mask, bits 0 and 2 are set, > so it initializes stuff with i=0 and i=2, skipping i=1 (I added prints to > confirm this). > > Then, in vm_map_ram, with the problematic change it calls the new > function addr_to_vb_xa, which does this: > > int index = (addr / VMAP_BLOCK_SIZE) % num_possible_cpus(); > return &per_cpu(vmap_block_queue, index).vmap_blocks; > > The num_possible_cpus() function counts the number of set bits in > cpu_possible_mask, so it returns 2. Thus, index is either 0 or 1, which > does not correspond to what was initialized (0 or 2). The crash occurs > when the computed index is 1 in this function. In this case, the > returned value appears to be garbage (I added prints to confirm this). > > If I change addr_to_vb_xa function to this: > > int index = ((addr / VMAP_BLOCK_SIZE) & 1) << 1; /* 0 or 2 */ > return &per_cpu(vmap_block_queue, index).vmap_blocks; > > xfsdump is working again. > > Cheers, > Nick > > -- help you, help me, Hailong.