From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F0450C3DA59 for ; Mon, 22 Jul 2024 04:17:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 731376B0082; Mon, 22 Jul 2024 00:17:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6BAD86B0083; Mon, 22 Jul 2024 00:17:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4E5836B0085; Mon, 22 Jul 2024 00:17:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 2D4BE6B0082 for ; Mon, 22 Jul 2024 00:17:19 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 9C7F11C36FB for ; Mon, 22 Jul 2024 04:17:18 +0000 (UTC) X-FDA: 82366078956.14.6C1DA05 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2048.outbound.protection.outlook.com [40.107.223.48]) by imf05.hostedemail.com (Postfix) with ESMTP id C9A2710001F for ; Mon, 22 Jul 2024 04:17:15 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=DLWIGnu2; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=quarantine) header.from=amd.com; spf=pass (imf05.hostedemail.com: domain of bharata@amd.com designates 40.107.223.48 as permitted sender) smtp.mailfrom=bharata@amd.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1721621799; a=rsa-sha256; cv=pass; b=1/SbAaVUtmOxozatVZqQRUXipm6LLD+3eAquxcstZ9jHkiiEdKWhVW4sd1uAvt5exqHK4e xx4k3VilnYuOX7nR9/4XbJUlotcs3fwk1AhVTudu3X8y3f1+gJVvB717t03mnovAcdvME4 OU08OUqbtXgRBol1ohBaOu4kNgUS1Lw= ARC-Authentication-Results: i=2; imf05.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=DLWIGnu2; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=quarantine) header.from=amd.com; spf=pass (imf05.hostedemail.com: domain of bharata@amd.com designates 40.107.223.48 as permitted sender) smtp.mailfrom=bharata@amd.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721621799; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MofPqHVnJ+Wj0SUPDWoIra3Him8GwTrRXZ6U59+mHFI=; b=GewdSrj3e5sQnn/WWAJGzdw8ghNZQ1Oe7K09X/1m2GPE8dAvi8qY/OZywa/crtA8BTpCDA qkdErBoi2yDDMPjo9fSGTgwrHI5x/XfiJwO/dTack0Iuqm0LZVM9F/f0YPxNMOc8lhe9ZP vONwKTH2Baa4b5RCqGV2i6V9pVf7GYg= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=b3I39wF/0Fwl5UlU3lHPj5ngpfERBCVDwwxzBEW/QiKEhjlbwDhlSmq/vY8q7Qs0wLibxfRFiBai3du7SEOwmCPxSTM9b2tE0QBpfaFxn0WOYf2wlUxFjNHYBiKJ7esKP+DdSVw126sy6AyU/KCesLFuC/iLUGabyCOH+jwP0GnJU7uYcU1XrNIktWC0D4TP6hufFxm6HV9ahS4la8Wi8XkD3CPaR5uEFgiP1Gq3S1e8M6sy8OXHOXHyjccU4b98R91ylsX1gFM8hVkz/tlFSpr52Veiq74wnWxXNeiwyfqPLKdEbYpltY8k0+kUCVmlrQaMaWG7AI5gpmmZr0KM9Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=MofPqHVnJ+Wj0SUPDWoIra3Him8GwTrRXZ6U59+mHFI=; b=dCv80gexIPKZmU2k/ImgP3w7QbJR1KJTFtgKRh+aHslitiEZRv9jas7MGNrxen0xcOff1HpqABGmWfRYLml3eJbhwzwH2RchSCLoy8nYd5SMgJ3fuBdIAqEt9YgEIotoNv8gu5tWeiYSrd1X4flvPEprmsxxwgA7dk9ez2NRoiJwa68bEtFQcmgBYlBGpcb6C/INcojC3rP4Gq6JGp6fYgHCnyhAlNZ3pQeQ2sPUgRAHIwEqYXlsfxkHmFIiAXlbfho+J9ezGs3Fx+W/8eRTnv8stJyjePdo4WwZFLaZi6RTOF2D2215z04kukATcwXabWEUeOdLLF552EBK3tKKkg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MofPqHVnJ+Wj0SUPDWoIra3Him8GwTrRXZ6U59+mHFI=; b=DLWIGnu2VxH7WZ3B/28lGlaldsR9zuRhN+ZKIw6/CnkjtVtP7kYMpIaKCEBFqL+gd6ICqArPY9q3GWVEXy1PMxJb/eN0ukkF6mzRK2B3ktt3SzDaOxH3Ps4nufJOYeilIwtt8ZY5thitXX8nqADVN4cqbCTsnSOlHRd/S7V5JVw= Received: from IA1PR12MB6434.namprd12.prod.outlook.com (2603:10b6:208:3ae::10) by CH0PR12MB8532.namprd12.prod.outlook.com (2603:10b6:610:191::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7784.14; Mon, 22 Jul 2024 04:17:11 +0000 Received: from IA1PR12MB6434.namprd12.prod.outlook.com ([fe80::dbf7:e40c:4ae9:8134]) by IA1PR12MB6434.namprd12.prod.outlook.com ([fe80::dbf7:e40c:4ae9:8134%3]) with mapi id 15.20.7784.017; Mon, 22 Jul 2024 04:17:10 +0000 Message-ID: Date: Mon, 22 Jul 2024 09:47:01 +0530 User-Agent: Mozilla Thunderbird Subject: Re: Hard and soft lockups with FIO and LTP runs on a large system To: Mateusz Guzik , Yu Zhao Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, nikunj@amd.com, "Upadhyay, Neeraj" , Andrew Morton , David Hildenbrand , willy@infradead.org, vbabka@suse.cz, kinseyho@google.com, Mel Gorman References: <1998d479-eb1a-4bc8-a11e-59f8dd71aadb@amd.com> <7a06a14e-44d5-450a-bd56-1c348c2951b6@amd.com> <893a263a-0038-4b4b-9031-72567b966f73@amd.com> Content-Language: en-US From: Bharata B Rao In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-ClientProxiedBy: MAXPR01CA0109.INDPRD01.PROD.OUTLOOK.COM (2603:1096:a00:5d::27) To IA1PR12MB6434.namprd12.prod.outlook.com (2603:10b6:208:3ae::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: IA1PR12MB6434:EE_|CH0PR12MB8532:EE_ X-MS-Office365-Filtering-Correlation-Id: 4e5b8cfd-f4a3-494b-fb73-08dcaa052838 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: =?utf-8?B?TlZTcnByRzBZMDFhWHcvWWFYUVhVbG5QQmlDa1pobHlJR09hYzhEM3UyaGp2?= =?utf-8?B?RmdPV1N0Wm0vY2JMK01peUdZM2xiY3BScUlQNUhJTHRKdW92Q0pYNlQxVDVI?= =?utf-8?B?dmhtbXdKS1M5VHNpQVh1UklIMktreFVuUlVuVWppd3ViMHJmV29WdzJGb0hz?= =?utf-8?B?YXprdFQrVXNydWFIVVJnMmlhOGlyTkdPQ0xJQkRXT015OXlycW9YNWRvNzVo?= =?utf-8?B?ZDNZajhmdlRGbGNEOERteU5yVmI2bXRYMlVHM2NZTDNMbFFjMWY1Qzg3WFQ2?= =?utf-8?B?NzlqYlZ6d0MraGtWRDJBdWR1UFU2Y2k0Z3VybkFiWTlJL0RpblZvaDhDOTJi?= =?utf-8?B?WUZNTEpOMjZVemJSTUJwU3daVStKRXY2cG8wNDVLUGsvL0VXd2ROODdZTmlV?= =?utf-8?B?MFh3VXRkNnFNT3pibjdrQmQxQ3FVanhwUm0ydVdhVGNQNEhxQmtGSUd5K21o?= =?utf-8?B?YVVmamdXM1RHd0sya1hzRS92R0ZwRmZoZFdtczdPM1JzWmFkTG1uc1FlRSta?= =?utf-8?B?ZU93NkJlMEZjZlA2R0Jla0RTelpwb1lkQ21zMHlnbG5obkdHZys3YXh0SDZY?= =?utf-8?B?RjlUWHN5TUY2S2RGc2ZGK0lMTjNHa1VBaDhyQXhveGdac2oxQmNMYVQremNH?= =?utf-8?B?b1hsSXJhMmc3T2NnOHJJVVF6Q09xYVovSkJtKzhmUFpqUUVwNzdTZHpQUHB1?= =?utf-8?B?LzMvQ3BWeHVaMDI3Q0IvdjcyYjhoK29WQ1hPT213aDVBZzRiU0FLZCs5ZzBm?= =?utf-8?B?RjlqVDN4N2hKc2tWWEg2SnVoYVYvR0ZhOU5KakZ3bEdIME1OcUYrUFN6Mzc3?= =?utf-8?B?TkExemorZzFSUGg0RTV4OW5YdVVLbURtTGxTa2tRNGNHaFBjR3BOajN6M3pk?= =?utf-8?B?RHRVM2VaVkpHOFQyYmlhQ2c1a2JmbklXbUhRcDh0ZkpBbUdNYzlqZGg1Tkti?= =?utf-8?B?VWhPNlZaV01kT2ttbWM5bGpzWWJPUmNFS3hYRFJnVXpMaFFJZEdWRzJsL1ds?= =?utf-8?B?TE1LUmJoL0VMSFMva1lNQk0xbTVtQmRYbmpzZ2Y1YVMycXRoUzVHYWMwaDQw?= =?utf-8?B?Q0hDVGtIR0lJeXB1K0Nza2ZFMlNqQVZERHRicThRdE5yV2wvV20xNUY0bTJS?= =?utf-8?B?UXBsOUJyMWdzNjBJdjNTeGRXWlBHdHhWaXh2UmtzM0lyNGphZU9SemltTU1G?= =?utf-8?B?NDhsamRWdGQ2cmpwVGIvbUxrSVpTRnVjZ0RoUkV0VWl3L1ptN0lqUXVaeHhL?= =?utf-8?B?ZHJCMTNTMzBKekw4NGVsWkhkT1A4RG9kdSt0WmR6Qm05d1lxRy9Bd3NwVy9o?= =?utf-8?B?OXo3QTdpRVR1S2RSRllPUktKOXZtOERPUnpNNnUyUmozVXRhVWhvdzMxWlFo?= =?utf-8?B?ODh0VFBKaEVRNFBPamUwaTRFNFZ6anUxZU5vTlgwR0piN283TzhKMW1BQzVU?= =?utf-8?B?aEFwanFHUVJFbkpaeTlINjFSdGZlYjBHOFBsSjRIRGx5L2MxU1k5RVAxNmpT?= =?utf-8?B?dzRzcTdBWXVoTlJNR0NTTVB5eXVvR2ZKaDVsTFZlaUVJL0R2dUd0d3N6aU9O?= =?utf-8?B?MTF3aHlKSVlxQWhsMTFOMEI4dFNRVmVEUWlHb1BmaUtLQVowZkhTMW44UVFI?= =?utf-8?B?NWRybzdmTGZwNWtWcGpNTEFrVS9YN2NXUE1kb01YNkxHQ2Y5MGp5VFdyNmxy?= =?utf-8?B?em5ZeVYrcEJVOUEwdEhGN2NjNGJYcXpRR2kvWmx2TEVvNkpFQ1ZQekY1RHpY?= =?utf-8?B?NVhxdllaNnh3THJMemwyNFJJSEFxejhTYWladE5oL0NrdXlaM2hlVk9jR1ZJ?= =?utf-8?B?eWY1SkJOSExpZUIwV3kvQT09?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:IA1PR12MB6434.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?MkwrTmVSeVI1RDJUOEFkS0hFWExhajlWT3cwQ1JsNTdMMVJYaGhZNHdlcnBH?= =?utf-8?B?TkNFV20wMzlMazFEcm5FejM3WDl6dy9RMWR6Tnl4ZkNvQXdzcCtZRTV6SGkv?= =?utf-8?B?Z1k5UlhIYWtwVkZ3TU1ueWxIaUtDekhOYUJYRldsL3czMDh2cTJRUGUrUTJY?= =?utf-8?B?Rm5WTFBPWmJ6aTYxZW5RdEl3SXNabnlBUzFRTzYwUGhPUHBnMktFemhmRHBh?= =?utf-8?B?WW1yWDFSRXRkS09RbVlJNUpSamxmNEZ0eDgrbmVXRFZyUVlaS1FpbFNSaUlL?= =?utf-8?B?WkxkN3Y5dmJoR0ZTUnhxRy9qRllIM1U1bFphRGx0NkZMaXhvaTd5LzdlV2Zp?= =?utf-8?B?SXlmai91OTlNMUc0MTMyTHJwa0w2dE0zNlBqbWVvcFo1Rytkc20vQ2JYdTdO?= =?utf-8?B?Z1JoSFdsL3Z6cy9qc055RjRIc3oyWW41RDJlbldzbWtNWVVrNnp5NnlROXRG?= =?utf-8?B?bTNFVStPNnU4UzQ5T3VUOEUzMWp2UWE5OE1jME54c0tMMnc5ZHlEL052d0tu?= =?utf-8?B?djNiRVhLVDFhWVlSQkpQelE0STQyV2krQ2s5Z0xod3JsVzdNWTJqWGdFK2ly?= =?utf-8?B?Q0xRRFp0d1JZMzRxMFJPNE1DamNvRXRyY016NWVPVzRQbVdqQWRFV3M4N1ZB?= =?utf-8?B?aXU5S25YV2FMRTIyKzQ3Q1pudmFSM2s1eEU5KzA2eHRVWFk3Y2ZpdW5tWi9T?= =?utf-8?B?R3M2L0xsamk5Qm8ySGVrSkllWllwRUxkbE9haWZ1QklHdmRsRDlRbEdpS2Rm?= =?utf-8?B?MkdnNFBGQWtpUjJZekRBeGUxdE02bUFtKzVkMnJVT1VaTGU1bUZoUmlCTHd5?= =?utf-8?B?VlQyK2JPaHRWNERGRzNiZEJxVFNEQkJmY3JsQkM4c1oreFcwU3pqSjFsRmV5?= =?utf-8?B?c01YaHVhSkxPdzRBZUdiT2liRit2cGNqaDhDZGliMEZYTXd1QXoxME14bFps?= =?utf-8?B?Mk02clNmSXVJYjlVTTNSeVozeUtFK3EvczhMaDI4Z0xnQngyOWdmUXJMdTVk?= =?utf-8?B?cEVXaCtMZkhGSGJyRjhTN3RPdUp3d3EzelhNTm5TREtMK3JMZVNPS09kMVBv?= =?utf-8?B?cjhkSGRaMThIRTZmSkVlSGZINVpSbEswRFBaM2VRYThFR01oanI1YWFhRSt4?= =?utf-8?B?aHpwZktYUm4rdG1UUWhRWHZoS2huaFBGWVdIaERmNTNEYUQ0YncwMnV4Q3Rr?= =?utf-8?B?aDg4UmtvMlR4WURtaUcyUzIwcmdWcUd5WFA2L0tJNStHbWpXdSs0Z05OTVY5?= =?utf-8?B?eXhweW80Umk0d2VscFhQL2orb0t1cjlqVkVQbU1PSDZGNmZWci8wZ2xnQklB?= =?utf-8?B?ME9HTmh4bW5KOUxiUGYwbFFyY0NTZUdNSG5PWCtvWlBTWlQ0c0VJUDFOL0xY?= =?utf-8?B?TmI2aHZkNDhhL091RWlyd0c4TFdwUzErTnM5aEVWYnFPcmtWZ0RVMStQWjVY?= =?utf-8?B?K2lOcW1GR0tRT2h0OVpWUms3N1YwYnBzanI4cExDQVEyYVJPYTcxZG4xczR5?= =?utf-8?B?bnRVcllocUY5bndyakNxa0dDZkNpZTRnS0tXdk53eXFRVVNENFNZLytIb1d6?= =?utf-8?B?V3VPR29KeE9nYVlZemtYSmtiaVZUa1RLSHJLbk5xS0s5bkdhYWJQY21VNmlj?= =?utf-8?B?M3V4VFF0T1ZGZ2R4TVlVbFJJYlBPaGh5dHhGZlA2ZmhtNGFva3ZWWjhlekYz?= =?utf-8?B?S051QVpWYisyMFNSTUlvTDFMenQzRGZXQkkzTG5kaFd2em5YTGVUNmRSaE1Z?= =?utf-8?B?NU12NGhYMjI2eTNVOWxWK3Z2VWNaQ2gvRmdCSERIUmRMOFk4Y0tjOWNtQ3JB?= =?utf-8?B?TW81emFjMGo4dUJuZkt2V2ZOU2h2U3RxNHBWMGtpdS9LbFpKNWlUNFBySlJl?= =?utf-8?B?U09yM2l1ZndlbjczWDI0YTdOelV5aFhDQlBvbVkwSmpEVkhKWVFBdmxaQktj?= =?utf-8?B?VFlEdytWSXc2T1AvRlhDU2M2K094QXNiWWMwSVhmSkFYbTFobVE4bXpUV2pO?= =?utf-8?B?MG9VOTdMT1RKREQ5UmMxTlpXeGJzOGNZcSt3VjlzTTQraTZPQkpYQ2xTQXll?= =?utf-8?B?NjNlSjA1YXgxY010US9aY3pEUE5PNHRBUGNMVWJhQU5Va29XVFFoQW5NQmNn?= =?utf-8?Q?R9qroaJ7noMpLdfTusP2/AeMF?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4e5b8cfd-f4a3-494b-fb73-08dcaa052838 X-MS-Exchange-CrossTenant-AuthSource: IA1PR12MB6434.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Jul 2024 04:17:10.8939 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: PfDTaPKRcrNtqFcYfMRuOx6FXyFWNYyXNEqrs/3b+aWREZThpIe2Un6/d2fxDlmdlEYRkUL8HSwxmIycJUWtMg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR12MB8532 X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: C9A2710001F X-Stat-Signature: be97uxh9xxurhucrn1ki4s5au35zqoip X-Rspam-User: X-HE-Tag: 1721621835-883538 X-HE-Meta: U2FsdGVkX19Ys6DgHBIEHVSDv1HO32HMQUey7jR7W2r2oohZMeUlWzMXIQW1hxkvJh3GGU5dQz/tYaMJqmqdEWOnyYlcE6krj43s2aRKlSzIcw+lgiUvDx+CSCCgNl9/vkZk/4zLjXFZz1cxcFN00tsrMb1Q48jQIY/0BuSVRFdDJRlvtot/xkmZkccfa7Ux0W+a4l4m/WgkTPtzVxF3JhZGSg3IDfiEqra7fHa0HoxohNiz10RaH2iJ+s2p2FP4jP9lgSmFZZXWmTAlxGDNdgLK1BKxF5EZC9k8ARoArpQxhRKDq5XAurzWrOHwOEQwjxMaGUBDn6GRobW+3XHknEy5CqLiyzjbJXpZOGZuPUvybdyhjWZrfkwa9PRAyIaLufE6dIJdnb+n89FT3vC+x8hUCOicQvttf+51bm/O9IS2nhRY+m7+rV1aAD5s79PVgNhKxem7AKu6v4z9nEkt67FVNdoMYZ7acvztwfy3Xz8e/xJMjub5TsEtnFpXgvsoSXPtV12+MgMlQdLMkfvkNmobVkpsOQqVCpMkemLtCRIBzwy7O5w/UOrwPsOcDSgwfvdh1hH6PLH6L0UW+vrme9FYNhM8hyYndVu5Bay+Zc4RLZHD953w2zAB4zMNHgn63EXHKVKSjpCaChOcOOCavhjaUGXMR4CtO8pjcFDlNrdNn6g8v4WbfvGMtqi/EVno0Eyx+fvmOo32kjU7eTHwXH75cB3Htt/W6/wd1AOredr4xDPSCtuMqAVgKeSH09YZd/kwYpuDheg2jXcb+TPpUhyXmFwNxudjDfwMxYdJ0imNnmW3esob2i7fK+3pp4FrBgc8YBsh/DRE7rLQLVGVMbQPzi+nAkXSodBuFHWRF0hzA/rH0J9MINyDZT9RBOkchaLjfAjAOc731bormySIC0/uATD6s4vg3hlliB4kzPGLxMGZYeeVrk1wH0gW8a8gbJUuBf9hn+J+sExH3Ir /RMstimQ 9avWjpCfzv1aDlO/xmHWkVNVaNJcuQLiQaNLC/l+P23C6okCZWf84GVsYxOi00xi2lmBKOH30xyUuTHz3Ml5m5qDHsxE3/dVDLEdGJS8KqYeSPcAWlbhzivgrmWqwdb3SW8SCVnb34EbwRpBVQVDC69nq52OpITDsXT5zVzUWUEY1sz20wvxWa7pdudkMkrdFN5iMs99YXwRlGFzXNPzrG1RGnsd3CcfVW1S6jwfi6VpSZRkDKACpMpNMuX/bBF3qr52Thp+Y5f53oHH2qQ+65zvcciQX5Pmm6kU9+8V7oXhWt8/tf7s3HggIyENs/cpkMZGfxqAyTTiBAff70sS0yOOtrw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 20-Jul-24 1:27 PM, Mateusz Guzik wrote: > On Fri, Jul 19, 2024 at 10:21 PM Yu Zhao wrote: >> I can't come up with any reasonable band-aid at this moment, i.e., >> something not too ugly to work around a more fundamental scalability >> problem. >> >> Before I give up: what type of dirty data was written back to the nvme >> device? Was it page cache or swap? >> > > With my corporate employee hat on, I would like to note a couple of > three things. > > 1. there are definitely bugs here and someone(tm) should sort them out(R) > > however.... > > 2. the real goal is presumably to beat the kernel into shape where > production kernels no longer suffer lockups running this workload on > this hardware > 3. the flamegraph (to be found in [1]) shows expensive debug enabled, > notably for preemption count (search for preempt_count_sub to see) > 4. I'm told the lruvec problem is being worked on (but no ETA) and I > don't think the above justifies considering any hacks or otherwise > putting more pressure on it > > It is plausible eliminating the aforementioned debug will be good enough. > > Apart from that I note percpu_counter_add_batch (+ irq debug) accounts > for 5.8% cpu time. This will of course go down if irq tracing is > disabled, but so happens I optimized this routine to be faster > single-threaded (in particular by dodging the interrupt trip). The > patch is hanging out in the mm tree [2] and is trivially applicable > for testing. > > Even if none of the debug opts can get modified, this should drop > percpu_counter_add_batch to 1.5% or so, which may or may not have a > side effect of avoiding the lockup problem. Thanks, A few debug options were turned ON to gather debug data. Will do a full run once with them turned OFF and with the above percpu_counter_add_batch patch. Regards, Bharata.