From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 74F8FC30659 for ; Wed, 26 Jun 2024 07:42:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ED18E6B0088; Wed, 26 Jun 2024 03:42:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E81146B008C; Wed, 26 Jun 2024 03:42:53 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CD3596B0092; Wed, 26 Jun 2024 03:42:53 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id B13A56B0088 for ; Wed, 26 Jun 2024 03:42:53 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 5E59CC0A73 for ; Wed, 26 Jun 2024 07:42:53 +0000 (UTC) X-FDA: 82272248226.27.BDB05EF Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2070.outbound.protection.outlook.com [40.107.243.70]) by imf25.hostedemail.com (Postfix) with ESMTP id 50D5DA001B for ; Wed, 26 Jun 2024 07:42:50 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=m7FqvCgG; spf=pass (imf25.hostedemail.com: domain of Raghavendra.KodsaraThimmappa@amd.com designates 40.107.243.70 as permitted sender) smtp.mailfrom=Raghavendra.KodsaraThimmappa@amd.com; dmarc=pass (policy=quarantine) header.from=amd.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1719387762; a=rsa-sha256; cv=pass; b=7/nvKL6h5pLTllunmmgVTidKTyVHKRWkdk5KAcsUt0fi/ZLW/dclB89VEw8rGonfyBB7Aa O6t3+VhfnDZZfV3bt7iyDGLxWGp+rZRRHmakVoJER0H7mgJGTwtGxqiJ7fWEl0p6/A6tFg FtA9ZZxnS4306djheLLouLY1QOjsh7E= ARC-Authentication-Results: i=2; imf25.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=m7FqvCgG; spf=pass (imf25.hostedemail.com: domain of Raghavendra.KodsaraThimmappa@amd.com designates 40.107.243.70 as permitted sender) smtp.mailfrom=Raghavendra.KodsaraThimmappa@amd.com; dmarc=pass (policy=quarantine) header.from=amd.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1719387762; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=oB8ajsve03n/d3txtTSdktyvL9aAeeJ5xUDSrZGshUM=; b=7dzAplY57qbvZIwGuF9H5+VxGZ/4m8qYks44gRAfcfDGr2xtK9P7NUAcq3fJamkuMjjozP HgA62G4Kiqrz1Bi3KiXwDWCkXxPHYibWJeM68LLYOHzq3Y6qWcRRXMCqgz5x7y/cZvvbte kqwhgROao/a4CHxHBkVLwyzhBOMqDx0= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YjZldHx8l5OcJjfMRJJxCZyRx0C4+FSMO+dbJE7hUdqjl2UIQikCG2Velk5klvFlQ7dBzc3snX7FU+HZ6vwWlrnBS+FcbLoktUh7oNH6IQHWfKkakgZW4NlSqzIjtRviZ6ch23A5aC5sjLlQbDhlCTWtWwOd2oBHEcTBAdm9mrhV2Qqrd/keEZ8QfQmvOtL22RqNmCO00YhbLncPPOu45wlGq4b3H117kYhFE3zCrqbsHhH0Kn0ZbejtuWO9cilkQ4hjRW9PfQXjdIlihdcN9YorNbpLTnWOnmUd99fBQ4rzgEEcNN67bj7FIRg2D09J+nKNuMam4jF05rGiPld/3Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=oB8ajsve03n/d3txtTSdktyvL9aAeeJ5xUDSrZGshUM=; b=bCNcoqK1py/uIZIuzr6IWBbxRrM0aDaiXBT7u9umm+tlUnMnnf/whr1hUo3fg3ACu9WVBbsju6oLCORxeaHwCZF4Nh6y+AqRgtrHN5evkJhHUOrXFL/hn04/nsBnaL/wz61EvZM4Pi2O/3fb06zN35I2G5yILrqIwygf4YnUKLtSx45Mi48bcef2J9VwVxsgVCUmos/voxZMVSdMI1q2z98DVosKLKGk+etbBihHeN3Agc6c+eHGOVPSf0gmv4K09VULszoV2pZxNOl+umS+67I3NWjHCeVx53pltYoR9MabUGG2dJpJNCjBo9ydh2AhJGOPPtTZyxcyloTygyqYmQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=intel.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=oB8ajsve03n/d3txtTSdktyvL9aAeeJ5xUDSrZGshUM=; b=m7FqvCgGzsyo4yH8TJqK1LW/Zpd1Tfmc78GKFBSVOE14EvKZdDhK2c3rrHxNuR7PtYkWceEPSoTu6+9KfevsoWNyq03kXbfqa2L84gC0MMI8ZxMT9shKjDj82uLb8F63XRKBxFZ7rVrB5dlUt7NM/8ifxIDjofDXaYCO6dGDr4c= Received: from MW4P221CA0027.NAMP221.PROD.OUTLOOK.COM (2603:10b6:303:8b::32) by DM4PR12MB6037.namprd12.prod.outlook.com (2603:10b6:8:b0::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7698.33; Wed, 26 Jun 2024 07:42:46 +0000 Received: from CO1PEPF000044FC.namprd21.prod.outlook.com (2603:10b6:303:8b:cafe::e3) by MW4P221CA0027.outlook.office365.com (2603:10b6:303:8b::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7719.24 via Frontend Transport; Wed, 26 Jun 2024 07:42:45 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CO1PEPF000044FC.mail.protection.outlook.com (10.167.241.202) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7741.0 via Frontend Transport; Wed, 26 Jun 2024 07:42:45 +0000 Received: from [172.31.179.82] (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Wed, 26 Jun 2024 02:42:40 -0500 Message-ID: <86dee1e0-e2b7-7134-82fb-c501cc42dfa8@amd.com> Date: Wed, 26 Jun 2024 13:12:32 +0530 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.3.0 Subject: Re: [RFC PATCH 1 1/1] sched/numa: Hot VMA and shared VMA optimization To: Chen Yu CC: , , Ingo Molnar , Peter Zijlstra , Mel Gorman , Andrew Morton , "David Hildenbrand" , , Juri Lelli , Vincent Guittot , Bharata B Rao , Johannes Weiner , "kernel test robot" , Yujie Liu References: <88d16815ef4cc2b6c08b4bb713b25421b5589bc7.1710829750.git.raghavendra.kt@amd.com> Content-Language: en-US From: Raghavendra K T In-Reply-To: Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000044FC:EE_|DM4PR12MB6037:EE_ X-MS-Office365-Filtering-Correlation-Id: c63100e8-55a6-4bec-3c46-08dc95b391e5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230038|376012|7416012|36860700011|1800799022|82310400024; X-Microsoft-Antispam-Message-Info: =?utf-8?B?eDIra2V5QUczSDErdWpQTUcySWJjeksvdDMzc253VTdDWHZqbTJVOTFTNmZR?= =?utf-8?B?TWplQXFwSFNuTWgyVjVGWEhSblJXWlpYb3RKL1ROZmN4YnNOQ08ybG1rTUp6?= =?utf-8?B?RkJmc3pabExxRElwZVZxU1ZTT1NNVUxaYWhBb2IxMW5FN2VLOWhGQ0JkV1Nj?= =?utf-8?B?WmJ4SUw2OG9lWFpuMmpoeldHOVlNc1cyb2xrMXUrMEF4Q0JjcEFPc1lMejdZ?= =?utf-8?B?eVZ0bEtlN29ySTh5WHBPVG5FUERBRlo0SExza2dlUEFHR3M3N2h0Mzg0YUZU?= =?utf-8?B?MUFBYlVkT1A4ZGRCamQ1cjhlMHhtaWEwWGwzSmpidUEzUVBNcjNmVnV2TnFj?= =?utf-8?B?U2xBVmcwUm81MTBuai8zWVdMTjJWSE4vOTJOZDVCK2dwRjFoRXZxR010cWlM?= =?utf-8?B?dlRydEdaZUlmdUVxYjJvbHF4SFB6M2liazZXMERxOFAzT001dUxhN25uYXJX?= =?utf-8?B?aHlMTVhKTWx4Q2pPMzZVRVdTWDV4QTZ1NHhkMVR5NnI4L05iVk92VE1uQlUr?= =?utf-8?B?dnlYbGtPa0t0RVFKRi9XazVMZWxHWC9qOTJPQnpzVjY3dmc3Z2NEVVAvSVVH?= =?utf-8?B?aG5najJhUS9KRHBKVFQ0R3A2eWJwSlMxSFp3RGs0eGJ0UERrTENXdlV3RVRO?= =?utf-8?B?eG5Eb1ZPUkM1eXZ2OUlnM05wVjZYL0MySkJ2NTFKcUpWQkp5NU9oWmk4SWE0?= =?utf-8?B?eXprbkZnMnhDdDl6bjBDNVNrMEYrMGU4ay9vREVhVnA3T2JYbzBIa2ZOU2NB?= =?utf-8?B?T2RNeitFaDlUYkZxbjRJV0xCWFoyUkc2VVBZVmV2WUtscGgxZ0laSUZZOW8y?= =?utf-8?B?OFMzaklkTDJkMnhNSnI4ZzNGKzhkdEtycGNmbGZ2dEloVS9PVGcxQlZrVmhL?= =?utf-8?B?NlhKZ25GcFFjR2UvV0I1ZzlQSS9QVmZHVUxyamhEUGx6OVRlQ1hlWFZDNVB0?= =?utf-8?B?NG1DZUJIZGVFaDZ2eWFEcEhhcGlzNVNlbXBDR1VMbGpNektYKzBTTDY2Sk85?= =?utf-8?B?cWFYS0E0cW9sV0xIMW9GcSs1WGdLenlob1ZZa2NTTUlCVnBWbzkvUFgyakhJ?= =?utf-8?B?cER0TVFBZklSUzRDcCszMWJHck1qdUZTcXcwTklVL3o2TnVyS1NrQ1ZJaHFn?= =?utf-8?B?cmV4RHdlbUFTaGtRbEFkazFKdGg1UXNWM3MvbEpuSjAvWnRhTnhMVTJMbFI5?= =?utf-8?B?ZndiM1BxM0tkczRRajhvWW5vL2JiZTNQRDJMNDVkdG1OK1hYcjNzSEZTVlo2?= =?utf-8?B?bnViU1VEUGhxNU5WaDJJNmErOVhJS29mQ3pVMmVpUm4zb0ZxczdyWk1LNkxs?= =?utf-8?B?NWdTTXBCMkQ3ekJrZVRYeTEwWlB1ajk2ZDlkM1RJU05NUm1rbVZjR290MnBt?= =?utf-8?B?UTloOElVbk1GcXQ3dDlVMEVQbVhodHhJNmlwVkN0NFY4Z1RVbzFrK1NnM1Ev?= =?utf-8?B?dS9IdWJzcmF3V3MyNEQ4QXpjZk9mMjdyQkorRUZrU09HWVpMZU01Nk1DcWVC?= =?utf-8?B?emlSOEhvMmsxN1BPVzJQaW5rVVFPSExWbmZRa21kV3JxbHJaZmM3RTYvOXMw?= =?utf-8?B?VktPT3NwbmtFZDAzUUJ5dW8wdW5GSmprOW9PN2tNQTR1SmJaL2NsaWdXUUZy?= =?utf-8?B?RHRsYU02aDR2ZEFqUFAxWnNXSlN6SDJNbVhxQmhHckJ1L1ZyT0ZLSTRQVC9T?= =?utf-8?B?TjlhVEJiZjVOaWhBc0cyQlprelJNYXNobUI5TUdYZTNuVENKeDFrbGFUOWN5?= =?utf-8?B?c0hBTm9JRk1RYmVIcEhKOWFRMnJoTENUOTBRd0Ewa3hUL3FZT1RDbVVSUUZC?= =?utf-8?B?dlNCVWZsQWlmdWYzM2VVWldScWpaU0xZU2xmcGR5aGljY3lzcWY4UmIxTm4w?= =?utf-8?B?V3J4LzlVNVV4SkRlZkwzNnlHeUpxdmIwbEJDNUJxeXFyN2NJOWFKOWNrMWZO?= =?utf-8?Q?yfWiISBoUHtFrYIckjhf6vuqrIzKphPg?= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230038)(376012)(7416012)(36860700011)(1800799022)(82310400024);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Jun 2024 07:42:45.7453 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c63100e8-55a6-4bec-3c46-08dc95b391e5 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000044FC.namprd21.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6037 X-Stat-Signature: qi868sdcgsrxbf4wdank3ausjr6styus X-Rspamd-Queue-Id: 50D5DA001B X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1719387770-428328 X-HE-Meta: U2FsdGVkX19ECgTD1Z5oheYeJBqlir0pccBINIUNzRDx/TOytF20nTw5lqa754ao9QvObRDeJdLY91iadRRniPhrRrRciZ38Zet2ENKvrSO1vXFN1bQiSwfOy1pOvzF3DMyEFO9NZkDrzIPrAd6RekBlHN0wVq0HSEKcqVov26GZDFoFboYAvVlr+lY7GT8L0YJl9IKI+7EFVDESTgXdv6XcqZ980AsNHEwj89Bg/lZym8TLmTVsyFO2DcJrBXi+HlVQmPKbd6T0aJF8AxWzNvxRqw9VcCe9QO1drkIAEyBM3noe7WVbe1h59GeIYOG5BfLN6TrYgwm2YUDXKo7iVsOTB8L8/xEEquOgmes8prn7UoNSwbcBmw7yciUnrJat5R1Ynv/X7/hJRjJDM3o7QAOAP7s8qv40akZ5BXX/26XCWnLiSCzSD12FrQDRZulEcOC4DXSjRakJZvqp6dnCK44ThQodrCn7PB3er9BdP896kcJnurD+JfYPb/WP1XXiiu9OvsdNujWEr5MpTWqD8j3Fl2BR/hT1D3pQ5yMoFwDhUH62Hh1WTYOjpkjCq68Ba4QPI8eJ9u2oWEYajDEIdTVt56BWdbs0PTEFX//TnX9Ma4Ihwgu9wH0b+Nwsc896b+Z7sr7lmWuoG36bkkpwVzegtJN/AJAEABM6g+Kfaz4mt/1FSkMP6b+LkT8Tsf5SxNygVcLXKzgOMr4LXL9UQ/q5z8i7KOT/gssKM4+3dr0+2ToYq7IiQXDa3yCTel9DKvMRhQ8rsKM7IG1PpKCdVFG3zNKqwXq4rQeDUXKuD4ZkI4ruTVyu+/Qlit4G8GQ/0geT1yrzN2cq2uPa08+VkCdtkBvD5M1ylbr5TEuspIP/CAULAeP/Pgqdne2ivhF8OihsjahPorYOEd+V6Trl5X6cvTQn8/1LODWdcRLt1aV/GuLMksZYfk4zu17vgwFd3TmuXb/h9gl2wQSa5tC fOZknPxA 0zeBqL9Zzjb5C6F6y9r7ozKZfzpx2hM8LDSdiE+VwkL4XvIujAAR2Sfnt2xTpRgV9FqSwoHGNjWlNLq8J4vAdnl4lwRjvjgDjN3zzf6mI5kHubOU9nBnYWqA+z6mRhs8fXIq+eZ6y2TJVCFfCeq3XkFjZmLg17J6p/Jp+whdg9Z9CagQ7xY4XL0aAsCq/1v5KueKsVlKnFon+WxmNkCGwYUya5utjohQwu+s1i+KOb8PDEVI3Pq9gHh/zDmyvuSL0HoHO0LDSAaBF+OLW7o2xXfm/OvFb0IZ0HFMSugReVbzGSfjjJVSgAmWYxavAiFQPvZTAc27BJC8X8TiQM80TDSpXgZCUNLcPqTQJgNnFlQeajKsL7Nl+1oTAsptUPTCNYnahnJULRcr0NiCSUJ8v+fUfvkfgSDFe43hWoymt+fY2V8TXmoKZR8UUTD7laHQhh4aACGFn6ougAnkC9AYUOS5L5EznmxSu8Jcq7BGbaxQ7tcbUntKZx/UNmB30hO4ecstE/U/SXDS+fwQ4wson2RMTXcWLw6G9+lMkG3M7Q/nVE/3O263pymqXjpYuxVtRZPa6KDL8rFw+r3aiSVG9buo1ug== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 6/25/2024 7:50 PM, Chen Yu wrote: > Hi Raghavendra, > > On 2024-03-22 at 19:11:12 +0530, Raghavendra K T wrote: >> Optimizations are based on history of PIDs accessing VMA. >> >> - Increase tasks' access history windows (PeterZ) from 2 to 4. >> ( This patch is from Peter Zijlstra ) >> >> Idea: A task is allowed to scan a VMA if: >> - VMA was very recently accessed as indicated by the latest >> access PIDs information (hot VMA). >> - VMA is shared by more than 2 tasks. Here whole history of VMA's >> access PIDs is considered using bitmap_weight(). >> >> Signed-off-by: Raghavendra K T >> --- >> I will split the patset and post if we find this pathset useful >> going further. First patch is from PeterZ. >> > > This is a good direction I think. We did an initial test using autonumabench > THREADLOCAL on a 240 CPUs 2 nodes system. It seems that this patch does not > show obvious difference, but it shows a more stable result(less run-to-run > variance). We'll enable the Sub-Numa-Cluster to see if there is any difference. > My understanding is that, if we can extend the NR_ACCESS_PID_HIST further, > the THREADLOCAL could see more benefits, as each thread has its own VMA. Or maybe > make the length of VMA access history adaptive(rather than a fixed 4) could be > more flexible. > numa_scan_orig numa_scan_4_history > Min syst-NUMA01_THREADLOCAL 388.47 ( 0.00%) 397.43 ( -2.31%) > Min elsp-NUMA01_THREADLOCAL 40.27 ( 0.00%) 38.94 ( 3.30%) > Amean syst-NUMA01_THREADLOCAL 467.62 ( 0.00%) 459.10 ( 1.82%) > Amean elsp-NUMA01_THREADLOCAL 42.20 ( 0.00%) 44.84 ( -6.26%) > Stddev syst-NUMA01_THREADLOCAL 74.11 ( 0.00%) 60.90 ( 17.81%) > CoeffVar syst-NUMA01_THREADLOCAL 15.85 ( 0.00%) 13.27 ( 16.29%) > Max syst-NUMA01_THREADLOCAL 535.36 ( 0.00%) 519.21 ( 3.02%) > Max elsp-NUMA01_THREADLOCAL 43.96 ( 0.00%) 56.33 ( -28.14%) > BAmean-50 syst-NUMA01_THREADLOCAL 388.47 ( 0.00%) 397.43 ( -2.31%) > BAmean-50 elsp-NUMA01_THREADLOCAL 40.27 ( 0.00%) 38.94 ( 3.30%) > BAmean-95 syst-NUMA01_THREADLOCAL 433.75 ( 0.00%) 429.05 ( 1.08%) > BAmean-95 elsp-NUMA01_THREADLOCAL 41.31 ( 0.00%) 39.09 ( 5.39%) > BAmean-99 syst-NUMA01_THREADLOCAL 433.75 ( 0.00%) 429.05 ( 1.08%) > BAmean-99 elsp-NUMA01_THREADLOCAL 41.31 ( 0.00%) 39.09 ( 5.39%) > Thanks for the test and report. I will split the patches and also test for N=6,8. (on top of your patch perhaps to make sure we have benefits further). Making adaptive may be little difficult. How to assess which size is doing better dynamically seems to be little hard to imagine for me. (/me Need to think here) Thanks and Regards - Raghu