From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16D6FC83F17 for ; Thu, 10 Jul 2025 11:52:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9ADCC6B0099; Thu, 10 Jul 2025 07:52:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9378A6B009A; Thu, 10 Jul 2025 07:52:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7D7A36B009C; Thu, 10 Jul 2025 07:52:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 6AAD26B0099 for ; Thu, 10 Jul 2025 07:52:19 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 34A2256063 for ; Thu, 10 Jul 2025 11:52:19 +0000 (UTC) X-FDA: 83648191998.21.FEEF660 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2068.outbound.protection.outlook.com [40.107.94.68]) by imf19.hostedemail.com (Postfix) with ESMTP id 3988F1A000E for ; Thu, 10 Jul 2025 11:52:15 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=z18bjKY7; dmarc=pass (policy=quarantine) header.from=amd.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf19.hostedemail.com: domain of Suneeth.D@amd.com designates 40.107.94.68 as permitted sender) smtp.mailfrom=Suneeth.D@amd.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1752148336; a=rsa-sha256; cv=pass; b=JdnkrZCe1yRb7u5F6dV2TrphRByBVsJMgtoO8LP7GZ8OxDqUl17hZvXsvVYOxocxFY3L4b iSpCF1MJ8kyTXl37eEEdtzvGJSmAmsytTK5i5pNqmMLHIvO1AxvokF5FgrQc26hRP4xqCu Y3Wc4x4zLVO6fFmZHt9arydaUebB+bY= ARC-Authentication-Results: i=2; imf19.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=z18bjKY7; dmarc=pass (policy=quarantine) header.from=amd.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf19.hostedemail.com: domain of Suneeth.D@amd.com designates 40.107.94.68 as permitted sender) smtp.mailfrom=Suneeth.D@amd.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1752148336; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=WX5jmr9UleQ5lNKR+oMbW0ejQLXPbIbYF47R1UsotME=; b=u1xkPXZfUbM/kdLEq8e8ZqL9PCpXTSPf8aVTSIepyqk6uZmKsKEC54RLw7YlXvjcg5YjD+ 6avMgFfELC2K1Khtdg3ZSSUAzhyQltXgFgCxHwO0XOXb+WWLHl4K78FgmgJ4VqxPxQbEdK pFyDZgNwgUTJOGGnzdGwTvcy2/RXuwQ= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ebGn6gMQzP7VD/ZcL0cDbyPs0nAP+sv9/i9/EaWuNpg3ZHh5PYh4yREtJ5VK+Dc456hFgjGs2vuasw+Mxyc7A4YW4yVJ2GoJ3hLGJZfhdqanOc/vo/1s2I5eSeLzt2CAPk2k9fd/dsUz3AKuqkoWA3x1ngcJWCFZaQrasSavcZdwJ2VmZEDWmxsyXIqwDVEtugXwvDFZxpcNuBfWex9XHOUWxUUZQEH42Dok9GvFX/dZ5TLrxziuVPWGfadBC+Cqx7NW7Sfvl7afn9nGf18k6uNTuDO7P6UsGwKSkAKMwePk19Yv19qHzVLvhGenMFJJGhtcdgilbsV5KSsQIs+vFQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=WX5jmr9UleQ5lNKR+oMbW0ejQLXPbIbYF47R1UsotME=; b=TgnbMpOboIL8TcCA9cKs+GKxht/ZRby7mSZIwt8lZ7omcMZJS2fNYgUH72AdkN2Va54+CieCx5+7tzKwV6/Da7miJRmiIrl8iR6YjTTOSgGR+eVkXIs9nIQNW6pjSv7/hKHl+4J+ds4qNqCJG/8ZximNC+ZzquJxx8+G+ZEkEF6Gd2ZnEPBmGySSCWW3IL7A3MttQSFbHuwjkXdMGvs33d4/iqvqHw4xrqpJJ+Ex+KZjPyIm6ZtrzOo/LMxr+7AAaBKg4recYLkS5fVu3zyHWlfAhRYUEcuNoE68tnrURMbXrksr9mNu+60Ew6mHy+GS4zLmPnhQmM7N5PkDSKDiEQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WX5jmr9UleQ5lNKR+oMbW0ejQLXPbIbYF47R1UsotME=; b=z18bjKY7VaE5+zDy0r7ynLGzKxN4vb3xBxA4RVUp3JLg1GRMPmS5OJ4G9okieZsYiOru0vqZU0MakHfx387x9NjiSUQ4IW0a/11r+ErdVxZy6SbbzVrLIfvjIxgQlX/adc4kqoANWNK7uhq0whAqdrmK+NAZBeNtwJ9iwQ64MPA= Received: from CH2PR10CA0019.namprd10.prod.outlook.com (2603:10b6:610:4c::29) by IA0PR12MB8745.namprd12.prod.outlook.com (2603:10b6:208:48d::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8880.23; Thu, 10 Jul 2025 11:52:13 +0000 Received: from CH1PEPF0000AD79.namprd04.prod.outlook.com (2603:10b6:610:4c:cafe::93) by CH2PR10CA0019.outlook.office365.com (2603:10b6:610:4c::29) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8922.22 via Frontend Transport; Thu, 10 Jul 2025 11:52:12 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CH1PEPF0000AD79.mail.protection.outlook.com (10.167.244.57) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8922.22 via Frontend Transport; Thu, 10 Jul 2025 11:52:12 +0000 Received: from [10.85.33.20] (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Thu, 10 Jul 2025 06:52:07 -0500 Message-ID: <639f17bf-8d4c-47a3-aebc-8d71597aba3c@amd.com> Date: Thu, 10 Jul 2025 17:21:57 +0530 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US To: , From: "D, Suneeth" Subject: will-it-scale.per_process_ops -15% regression on v6.16-rc1 Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH1PEPF0000AD79:EE_|IA0PR12MB8745:EE_ X-MS-Office365-Filtering-Correlation-Id: f189f92a-3216-4d32-f515-08ddbfa83567 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|36860700013|82310400026; X-Microsoft-Antispam-Message-Info: =?utf-8?B?UXZ0MkMxd2Z0Z3FVRm82UzFLM3NvMTFJSC9aaVQ4OGFmU1VWTW5aMjF5RDBT?= =?utf-8?B?UjU1QjlidDBCaTJ3ZU1UYVdSVWMrbU50dG9zMEQrOVZyb3lQV1E4Zmk5MnJB?= =?utf-8?B?SUVNTjh3dHBqUDlkMkMzT3dRSnVCSGFlTU1jTmRqTk9PR2hUb3FUQ0U0NDEr?= =?utf-8?B?RERsT0FiNjhTT1RJSk96ZC9jQXFPT3JacEhMSmNFeUE5RmtXN3RUVWRVM3N0?= =?utf-8?B?ZDQxMnNWOTl0WmFGNytxWFh6SS8wZjBtajdJQVJCeC82SFJrRHA2N3V5Nk1R?= =?utf-8?B?blNrNUhHTzB2NkVWMFQ1TmYzZWpia24xbGJHbUdLQUVEcG9NaGM3b29Cd3Nr?= =?utf-8?B?WG9RMTFPY3RPVzlFWFJDNUcrUmt5dVRMcWFUZ0trdHhjS2luMDFhcG1rR1Bm?= =?utf-8?B?SE5XOUZESlhGN3g0RnM2T2lpZDdLWlE3a0lwSkMwanA5NW9UeXpyNlJNN3M5?= =?utf-8?B?Nng3bWc5RUtOVXhEa1VYcnBKNGh5clF6b3JjeG5oS1V6YW93amZSKzB5a3dx?= =?utf-8?B?aWpDVU5SQ0VlM0l5TkF0Y0hmZ3ptSjdsK3VFOEduNmt2ZXovb0VldGRCc0ZH?= =?utf-8?B?clNkWis1Ly95cmlzVTM3TGdBUWhIU25NTE1tb0dQT1BoZnV1Ymg0aEVIcXFH?= =?utf-8?B?RXF5d0JtcWZPc01WWEl4Tml2cUhMRW1MWHVZc3JQOVl2MlQyUGJqb1o3U1Np?= =?utf-8?B?ZVhmQytLQzNUVCtIQlJrenU3YXJLbWNsK0szbUtPSVBZdjZvcjlHeGt6ZFBP?= =?utf-8?B?M0MrQkFnR3RDdmtydE1NVlU5WjMvdmJmakRneE9vbDFoYlZyUXJuVGplcmlS?= =?utf-8?B?dnErK3NFRGZVK3F0NkJKTnBSSXpLQll4OHJQY0pLWGNuYjhzbk0veFNwVEpk?= =?utf-8?B?QTBxa2diVmlCYlR6RWZUM3YrTHFEZytLd0FhVEQzOW9MOU5pRS9hb0NJRmdJ?= =?utf-8?B?WlAxSTdKcC9YbFQ4TXRnem5TbUtnbDluZS85ZE1pSWZCUk9hcWtML25WV2xD?= =?utf-8?B?cVBXbW5BTCsyM3dqWmt1eUoxSUhydDZNOGpJeE9pMndLbEZOOXFXUmlDQ1NT?= =?utf-8?B?OHlVRTdwSnZuaG1XM0kxTEVVVWdqWGpzVHVtU2U2REl0YU9XS0Z2OWwwbjl6?= =?utf-8?B?TnA1SE8wUXZ5dytuekdGOXZVcG5NS2U0ZHVmaFVpQjNnN1BIY05VeEtSdXM3?= =?utf-8?B?Qm9MVkhLTmtKTE5zY0lkSWFpTlI0M3o4aCtmVll3MnIrU202SzhuU3o0VURn?= =?utf-8?B?TjVNMDVZQVl5TG9QMEVNNXJPbCtZTC9SdUs4WmxwSWRyTWRSU01MMGlBL0VY?= =?utf-8?B?RVV2OVBhUUdqYlQxSm1YUElHRHp3UkxUWHVxUXBlVzNQdDcvY1FVNzZIVnY3?= =?utf-8?B?NDBOUFBnbEpJSGNEV1pFUnhiOXNLV0VYd1lFOHdKaFFOYjE0ckNoUHpELzN4?= =?utf-8?B?UVI0em5pQTVHdEIwdzRMd09LQVJ0M2pBYk9KSHdQY0k1aVQraVlrdVB0VmE1?= =?utf-8?B?VFI2MFFXMHlteVAxcnYzRUh1RDB4QWdJS0FEcmxObkxEbk1zZE96Q09tT1Zs?= =?utf-8?B?bmkwOW04MDVWU2c3THNpM3BjLy9Jb2k0Z3pUS29vNzZDNS91c3NpdXRBQzRP?= =?utf-8?B?YXRFZ3ZGbWwzeHFhU3paWDhXelI0UXFhTmVvZGJML0hQUkhVUHltaUZRU3hG?= =?utf-8?B?RnJsKzBRRGczeVR2elJOdU9ia0pmRE0wV3pncU9SVU9ISFIrL1FFQThkMFg2?= =?utf-8?B?SWJQcDJ1ekVUWjc5T3dBYXZTNUxhcHVJKzF6QkRiMm1tNFM0R01zZm1jK1BC?= =?utf-8?B?NUhNVElGSkJoa1oybUM4SG9SQUZQZ20xQ3Zab0h2bC8veTBoTHVTbDY5ODA3?= =?utf-8?B?K2IyTWNOWUJkSXJGSStMdlFaRllFNzhWUlpiUkMxVzlSZHRwUFRuZlhwV2lM?= =?utf-8?B?K2R2c1hvM2hrWjBGNjZnM2EzNDZlR3J6Tm96d2R1L2JFbmlIamRqNWlNUi9N?= =?utf-8?B?NnZYa2lOcDJHdDk5SWJkVUx0SG9yTW1xbzdhT1N4RVpiQkZZem9aczU0T3k4?= =?utf-8?B?TVRFV3NqZFZMcHNtc2ovVkJ3Z2hLMUhRSGg3UT09?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(376014)(36860700013)(82310400026);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Jul 2025 11:52:12.7359 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f189f92a-3216-4d32-f515-08ddbfa83567 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CH1PEPF0000AD79.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8745 X-Rspamd-Queue-Id: 3988F1A000E X-Stat-Signature: gjs3x6e95td1ikhkigppxr58bwo8s4ff X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1752148335-209454 X-HE-Meta: U2FsdGVkX1/b15ZgZW2iIGXztRN5Rpm7spfnd2fdSQugKMYk+m2dDNrtTnWFCT+gA6QqOjVR2lKvQ2UCSNEEPqVtapueiiHYXyKv7w87FyQI6FqBJZ63fmJn+787aLpjLqaUZBKinOEIOXEwA4tLVSmSVmM8m+TpT1zDHVcOckw87CbN4lCRgUyESwG/gck55dl/at8gn5wLBklj2r7vpevtQCu6O1McX6Qwyc0PVfxTZqz0K+/AiARH60pj0gZnqlpETntBrHklC/61uoRJUwAlrusd0fX2CS0HjLBvAUnq0zlbe3qUXxyxpLK0WlUeZ0hlbpozKOdTth69uOIXJtC5skJULH6yHcKS6OxA8CMpVQg/NoWYMzF/nzxTphbCc3DrTqyC9Xj7I1AtKUHKSYaJrpluNgr2YgaMOpbo1ZRd7rH4Ok2qaN+dlxlDxu5V8xH9GcabpkI1yCyZDoNvDj82pjcNl+s/zO7luMwWZrVCm52MNQmERhPmejVfgEwdopMmI1a3iOCukULP2STUV8z+Mv4MWGIhsFq/SWzdocKNHCpGGbGmqyZv+rmIIXPs7ATwFh034e/MssFi1KuZ7WtaTZLcWzrV9qUwXb+S67wrolllan68UgwasTUGFV6061FWXDjCBEM4l2Cz4kKZdCU6+CYd6huKSFdyyIqb/8JyUM9bo3nWrIYPHE8hmSpg3ZMRN375+E+gHSQszzSlQFLyuZlWhhmbzxIMqcUbYfIuCkbz9i8i89ccuc5yx+XMc8DIE18D7Bx7B1nmlY+1KE1fmQvmYhN5a15t7iMpGuzu+/6YuUAdUCLmYjvujvahlLNArqitHguUmAzIkihp4ZQVZjD2S6T+VaVcjfbxKMPV5qOCLo/MvFgjfGnSO8ZMTlKpuc0z8MEZTbXA9sWpEja5Lv/HT/9d7vFXXbboaJuHqcW58HmplVhh7T1EuNFrJCnhaVDdfpbg9I/Nf5N 81c2Le0W cB2v7uAvyJ2wHiF0CAvyqgL7D84z2YBG07mbxY8p4OAjBJRINBx2hVHuZqMWQG2uyvXh5GgGMRKLDE67tYIy5XJwLPWtPrZeqVtVqs9slDklX8peYNQAWC2xnhNQA6nr+w++utFwZB00ioZAY6xseK+LzkLvZ/sk6KsmvoFQ6eQPhMbsQBFErv9afxFsw51aBRk8B4/BCVsEn+gM7UUJn/cDCAm7sq+39aU3mWsfO+jy1ZUDVrVsY2+JmDekfKLQGCOk0qtotf4TKj8FOJY1jVKD5cjva5h36wsRAK9OTdEyU8Q8NW7W366vPMlRseYTgfgnlZhaMvwbrkiMQYAGGE/86asBaXd6X5ek1I3svoe8VAWm+23t7uhDPI7BNXnIYnmPGmPUNOFppObkNInguwYD4xafiFIFNMBv0Xtpju7zUQcc5m72p06Fpf+ORIjuO+EvGOBI+5nJRURcxSxMm4XRnSJpsYR5KrlPllqjgLSWXwl4caf6uNL2xXW4F3HLemCe9UidRjxRcmXV8B/GvyruxasdasvVDsG6gyDFlU+ZAHrQ8vpPhkj7rC3I8AO4qC+QGCRR2ZaAianzUjMjCNLG4ccoHcyPn2pOkj6NfcDGa8AvWKqx9qsnD0kmf767YJu+3Hz7sXjJmA77CX5DOfoz6VfKlJasx5BE+tlbY5YDBNy791zYoK6OHaIyiIdk7UcPH9ApCE9KqHiUHaK4R8leWnA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000002, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Greetings, We have observed ~(15-16)% regression on the will-it-scale-page_fault3 variant when the testcase was run in process mode on the kernel v6.16-rc5. Further investigation showed that the regression was sparked in v6.16-rc1 and has propagated all its way through v6.16-rc5. Below are the test parameters and Machine configuration that have been tested with:- testcase: will-it-scale compiler: gcc-13 test machine: 256 threads 1 sockets AMD EPYC 9754 128-Core Processor @ 2.2GHz [Bergamo] with 258G memory Test params: nr_task: [1 8 128 192 256] mode: process test: page_fault3 cpufreq_governor: performance stable version (v6.15) %diff per_process_ops kernel_rc_ver ---------------------- ----- --------------- ------------- 492057 -15% 416927 v6.16-rc1 492057 -16% 414140 v6.16-rc2 492057 -15% 419158 v6.16-rc3 492057 -15% 420476 v6.16-rc4 492057 -15% 416334 v6.16-rc5 Also had a run with latest stable v6.15 %diff v6.15.5 ----- ----- ------- 492057 1% 494990 Recreation steps: 1) git clone https://github.com/antonblanchard/will-it-scale.git 2) git clone https://github.com/intel/lkp-tests.git 3) cd will-it-scale && git apply lkp-tests/programs/will-it-scale/pkg/will-it-scale.patch 4) make 5) python3 ./runtest.py page_fault3 32 process 0 0 1 8 128 192 256 NOTE: [5] is specific to machine's architecture. starting from 1 is the array of no.of tasks that you'd wish to run the testcase which here is no.cores per CCX, per NUMA node/ per Socket, nr_threads. Currently bisection is under progress b/w v6.15 and v6.16-rc1 to spot the culprit commit. Thanks & Regards, Suneeth D