From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62373C64EC4 for ; Mon, 6 Mar 2023 15:31:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E98926B0072; Mon, 6 Mar 2023 10:31:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E48C76B0073; Mon, 6 Mar 2023 10:31:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CE934280001; Mon, 6 Mar 2023 10:31:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id BDC056B0072 for ; Mon, 6 Mar 2023 10:31:06 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 975DE40790 for ; Mon, 6 Mar 2023 15:31:06 +0000 (UTC) X-FDA: 80538861732.29.983DC09 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2061.outbound.protection.outlook.com [40.107.243.61]) by imf12.hostedemail.com (Postfix) with ESMTP id 93A8640039 for ; Mon, 6 Mar 2023 15:31:03 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=S5iZacou; dmarc=pass (policy=quarantine) header.from=amd.com; spf=pass (imf12.hostedemail.com: domain of bharata@amd.com designates 40.107.243.61 as permitted sender) smtp.mailfrom=bharata@amd.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678116663; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=AFLIAAi94sGcpIhUzluEJEgP0hJKTMpXcfKQp2IzVQQ=; b=8YMqySwGJEudc+b2dpZtDbuAdxcn/Ew/BCMVZS6h/0MGgNJvv3a1kWBUcFgWhZdOgb89CF LFpg8QQpd2hwv76ZNIAvPJe5dTdpUXhLTpOktnCCZfGFYmxL4HqYOsxWXQhDecj6aeLuw7 rD7/BnT3zc99pxRMdl0O9VowJ8GsU4Y= ARC-Authentication-Results: i=2; imf12.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=S5iZacou; dmarc=pass (policy=quarantine) header.from=amd.com; spf=pass (imf12.hostedemail.com: domain of bharata@amd.com designates 40.107.243.61 as permitted sender) smtp.mailfrom=bharata@amd.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1678116663; a=rsa-sha256; cv=pass; b=77oGUvKNj/Rp487hWIe8Qoj8KMSSlug0kwKUHoa+C0QqgcjzHW4mEXr7kE9U6gyqOBPi1u BiQ3GO1JukJw5GL3svP2k8e5aWwqhPzZLTw+B7QaxBCkS1lc/4aPZNT69OM0+yA0Kr7Zwf W8QfkcqwUXZVkfJRgnnKojTpUd7W4FE= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ALIS5LVXQhiWxsYtapnOVr2IDGVQitUl4DK0CDWfnwnbaOCfV1sXaOynE1iqCfnGzmVuCMu/7gzbwPFgmX6Jq2n8qH3DezqwJr31pC3hvRvHUwA1xRtVwGFO4beftAN92yRVblHP//xHqVmErnCug3Pt4xagH3ekWL+4GYdLObwpiv/jhawnzrh5b4c277PDC617lIIzHWvwqRubPfO0eMdj9QFeCFfHyS3xNjKixk9P74RzquoIkaQjHQxwEATAsROMixSB+8tLnjJRjR/fW39Dk4/WJ0vYdnU2jUFKsK2PwyNR3+8dJCYlioAoZ5d6DqPBLjMA/Vs8iLJpDZJTKw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=AFLIAAi94sGcpIhUzluEJEgP0hJKTMpXcfKQp2IzVQQ=; b=E9vChVgYZ67UOhPsloH2vRY3lfx7iQAuqaQwhgtWuJpC8PAqBT5S+fuHsBh99677EOQVvhk9ZRMq7KAnKj+GtbWgEc2Il5XWmKePtjPIWNH6uD722tusGEuq3MX1gjkRCROnEXcFqKWolG6HkI83xCZpI+h3KrKZCgRsYd3DotpnoNc8JaCws6kzk7nkhE9V/ZzNtuKfS9afmvB6tEyOwCwanJUQBGlXfbNut6kdlfd8JxhIsAsq/NOVLdcqZLqyFV6x+3COPEOVUf8AY8VnU3EC8uPAModDveEd1/YaDqbVrD/rF62OXRUxiIDo4hvubd/qlswLPVCbNUycs0JtGw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AFLIAAi94sGcpIhUzluEJEgP0hJKTMpXcfKQp2IzVQQ=; b=S5iZacous+h7gVgHeQXHdlGSIfy1uqoWPrh/CVpuEqfvrCsqKqOxZ3p8Y7s6uy6JIc/oQ2HItWosOzSKvJjkWCNZBdSze8xa7alDHrq3MoxHRZa5yf6Hnx5pDGz1gRIgZmpf95zmIqNZmK5GeSw79ebTjfGmehWJOxa1XhxLF5o= Received: from IA1PR12MB6434.namprd12.prod.outlook.com (2603:10b6:208:3ae::10) by SJ0PR12MB6829.namprd12.prod.outlook.com (2603:10b6:a03:47b::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6156.28; Mon, 6 Mar 2023 15:31:01 +0000 Received: from IA1PR12MB6434.namprd12.prod.outlook.com ([fe80::fe66:3fca:2cb4:1e7c]) by IA1PR12MB6434.namprd12.prod.outlook.com ([fe80::fe66:3fca:2cb4:1e7c%9]) with mapi id 15.20.6156.023; Mon, 6 Mar 2023 15:31:00 +0000 Message-ID: <55a344fa-4325-e82d-eeaa-1a77611ff513@amd.com> Date: Mon, 6 Mar 2023 21:00:47 +0530 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.6.1 Subject: Re: [RFC PATCH 0/5] Memory access profiler(IBS) driven NUMA balancing Content-Language: en-US To: "Huang, Ying" Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, mgorman@suse.de, peterz@infradead.org, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, akpm@linux-foundation.org, luto@kernel.org, tglx@linutronix.de, yue.li@memverge.com, Ravikumar.Bangoria@amd.com References: <20230208073533.715-1-bharata@amd.com> <878rh2b5zt.fsf@yhuang6-desk2.ccr.corp.intel.com> <72b6ec8b-f141-3807-d7f2-f853b0f0b76c@amd.com> <87zg9i9iw2.fsf@yhuang6-desk2.ccr.corp.intel.com> <1547d291-1512-faae-aba5-0f84c3502be4@amd.com> <87zg9c7rrf.fsf@yhuang6-desk2.ccr.corp.intel.com> <8fea74ec-8feb-1709-14f2-cecb63fdc9ed@amd.com> <87v8jnbl22.fsf@yhuang6-desk2.ccr.corp.intel.com> <87jzzz8tgm.fsf@yhuang6-desk2.ccr.corp.intel.com> <41b47cd7-1ba9-3205-165e-02e8384e7064@amd.com> <87356m8jo1.fsf@yhuang6-desk2.ccr.corp.intel.com> From: Bharata B Rao In-Reply-To: <87356m8jo1.fsf@yhuang6-desk2.ccr.corp.intel.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-ClientProxiedBy: PN3PR01CA0129.INDPRD01.PROD.OUTLOOK.COM (2603:1096:c01:bf::8) To IA1PR12MB6434.namprd12.prod.outlook.com (2603:10b6:208:3ae::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: IA1PR12MB6434:EE_|SJ0PR12MB6829:EE_ X-MS-Office365-Filtering-Correlation-Id: 2d087e7f-e49f-4347-fa5d-08db1e57c9e4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ATUM0CIxLtkhlv8AG0lXFTKF3GV7olhxcXg2PsfBdh/z90PjHo/HQ/FNlsDFJKTVO5hzndVlLh94S2RKha6vBWxQwYq9XMOzf9WdByBOkwVTIlvAqCRoi7HG8Er2dGBMJp5bBNaAZjUuSWsn8qutwdQooDrjmQawIp1AUJM8Q5hjSqQWgpl0oNB/CBmzgBc7ZKQF3OEiE21ZbuzPGeHTiLOaZ2Bb1y8l+r4ngNpbpQxTw1n+tZ+k/C76QCCf2OgfqSeFZ8X5hRBuMtMPXT+YRzznRCbvpLVTG2jnphdy0avRJ1VxWh7VkF+vJqdTGmlouv4cUVC7PvsJgtESt5iKs2soL353Xuvz8aglnIbqNc6pccvNRa5ODNGuIhnRYxDfvTSlCepaOUzgumVm1Zdm4VG2Ayb3heyN9N6rDQRIAXogAs+GZ/+NdPwizJdthZRNNmqQzxyc/P2ZWhYrITTv6pOBmYEUvUcDv7b7UNaVDQxPoQWelNBbqe4z5L3MxwDJzWKRRDZlDxi/GyBTJ4Ot8CSHs7Zbixo1uxARr1yfR1xQejS6vTeIFPbbUdnw1drCAPMYPTKL43+5v/MJTcH0aYpxZIqkTAVjTJwnmBVot4c3lQjc7KSO22XagAKhV3p65g/vDR8uwuz8lB8il5GWPDfoJypr29rc+xmr8uIl8RryvpFHUdoL1y9zK2iI0kHiSHqIxDSxPGAbxwuiduVVpb9flaQIc54l++YcyxwcjeY= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:IA1PR12MB6434.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(366004)(346002)(39860400002)(376002)(396003)(136003)(451199018)(31686004)(2906002)(4744005)(7416002)(38100700002)(86362001)(31696002)(6486002)(6506007)(53546011)(186003)(478600001)(36756003)(6512007)(66556008)(66946007)(4326008)(6916009)(8676002)(66476007)(83380400001)(6666004)(2616005)(26005)(8936002)(5660300002)(316002)(41300700001)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RUJ1dytHT2dPOTNWQytVdkRra0hUNVI5eStwc2FzcGRHUGhzemhwWVBYdGda?= =?utf-8?B?SDR3OVRRYnpWTFBvRVRkMitWV2d0YTM3ZEZ5N2RaQzJYTUczYm5FNUhGeTFG?= =?utf-8?B?STRqZ1hJczZMSkFsMDk4U1JJd05Zemc2ckxYU3lOMzJ0bzVmby91bVZOK1pR?= =?utf-8?B?SnRjV2F0dlhVMnhTZXEvaFdYQndWMEVTeUNkSzZ1aHc1eHg4QXRLWEdjMnp6?= =?utf-8?B?UDVzUjYwOFhMRmdRcm1ZdS84S29ob0dEcFRiZUpLUVdHU3E0OTJVTWdOeTVE?= =?utf-8?B?VWRSMTFNY1kwRDArOS9zREtveUVJTUczcmc1N3JkZ0lMM09kNERmYU05MTlu?= =?utf-8?B?aUhDZTY3YVc0blRqR2tpZ1pOVVdvNlBlblkyZzRBNXJweXp1L2dHZVdxRjly?= =?utf-8?B?eFE0SGdVbDdhamJwQUUzcUhBT3dXWHJ6UGthRjBtQjdJMmM4R0E5NVpKZkJL?= =?utf-8?B?WGVsbjFnNm9DSWRJK3k3dStFUlpFWERLYkQzcVJXK2VpNHcxbEtndU5aQ0ZW?= =?utf-8?B?cHFjdnVTZlRnNTJidGF2TG94UnR6TENJazM4dGxQSDk2SkRFQ01zeFI2TEdO?= =?utf-8?B?UkZjOWlzUWc5bGd2OVgvbWt5Ri8reTRMQ1JzMGRCVUZGTkZsTWpBbEg1VTg4?= =?utf-8?B?ekZHb2d6ZVcrTzhyaFhESXZuMkR4dFFTVzl5djY2eUcyczB2dnRYemg5RllW?= =?utf-8?B?UUcxZVI1QS9HY1FWVC90Uy9KUUdQc0ZKMi9sbXpGK2dlT0pRb0c5bmRNeS83?= =?utf-8?B?Y2lRZ1pRSHVxUUVWVmM5V2kzMzJmcXdDVVZYYkJGYksxTFNjNzE5b3cxV2Z2?= =?utf-8?B?dm1ZSlJ3YzZoQ1NxV0RUNGRjNkYzdVZvRXBvS0IvSkhoVGhDUDFNc3hEcG01?= =?utf-8?B?dmRCZUNxelUrWjQyMGZqZ0Y1eFVwNTFlOEV4VUhzRXRIcks1cndXU0x3Rko1?= =?utf-8?B?cUN3N1BFV2JoUG1ZWmgvSGU5SElCRnpNbnI2aGQySndka2UzT2dpN3V1VU9r?= =?utf-8?B?cUlpNDRzRlJwVEtObjc3TE5BS01kM2pXTFRYQUdwSzVQM2sydmdwMDdWWUFy?= =?utf-8?B?L055L1Q4ampMNTBzdDZGenNmRlgwakhQellzTWp4TFU2SnRPdEMybXN5eWNX?= =?utf-8?B?cnFoSEdjWitlVXZlWXl5M3Fmc0xvc2MvN254OFYrSnNacWtlTyt4T3plb1hT?= =?utf-8?B?UVZsaEVHRHNUMkl2b29hN0xhNHJteUdJZjh0ZXZZMzdWNEtVUTF5VERmTUp3?= =?utf-8?B?eFVNUS9Pekwyd3ZSV0dOeTNMRWV5NVJRczFiM21ja1dRZkJyS1NVYitXSnRU?= =?utf-8?B?emlGK2g1ZEdiVjcrdU1WWjRqWksxcGtVaXRIWW1WQjlxMHZoNC9VWThINHN4?= =?utf-8?B?YVUxbGtnY0tBdVNscDAzNmR2M2tMakMxVnhFRFZVL1BvcFN0ODBqQWw0MFg3?= =?utf-8?B?cUgvdnNXWDR2ZktrcFZmT056bUNqbmQ5anEyWlJoTkdCQkRucm1Yc25HbUR0?= =?utf-8?B?cnJqcXpkU1hZTG1jUEVMWit0NTlIbUtyeFY1Zkp5N3d5N3hJYUk3OHI0NTBN?= =?utf-8?B?c244NHFSN3hnajBXUWFkVE1oV1FJOUd2bkdrYlpFY0huMUlaYVhQaVJ6dCtr?= =?utf-8?B?d3NQTU1yQUJHYW4zSkphZEZXb0g2cExIckgyTk56bFhVbkI3QW9KMy93QVpx?= =?utf-8?B?UWE2dCt3U2NuTzU0azBoSXl5dTdNUUsxN0N6NW1RTmluamQra3FyZ3lpc2sw?= =?utf-8?B?alhJMU1tOHFPUW81K1BRYVh4d2pZZWp5d1lId3J5SmZvV2ljNmJHNndCbzNP?= =?utf-8?B?WlVpR213OTRvUm1QVyt0dDZROWVSR2NDaUVkVWFVR0x0cFdHdHZXN1I3aHhp?= =?utf-8?B?djZxcGdjSFpVcU8vTlNEUURnelJCWVovYWE2S2MxVDl2WTZMNXpMVWI3RS9T?= =?utf-8?B?dWMzS0ZzM3V0dS9rT3JGMmY0OGhnTEhEVFNLMjVBTFczNmpnRXhzdnlVakl2?= =?utf-8?B?MWQrT05wU0hPTFZ1WjJOSWRZUm52VlFZQXJjN3FqZ05XZjVoOFFDRGlicGpL?= =?utf-8?B?Z2x5SXgycWgxN1BldlNOd1RoNTl2SlZ2cURTazdpUFJUeXpLbWRsOXFXMDJQ?= =?utf-8?Q?ke8t94ce88jHeNmXtyLlbaNoL?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2d087e7f-e49f-4347-fa5d-08db1e57c9e4 X-MS-Exchange-CrossTenant-AuthSource: IA1PR12MB6434.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Mar 2023 15:31:00.7989 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ciN8eT+IFUunEwlL+3F4butHGVT0kIXDF49axanTjvjtzmTbf9/FUzLn8d8CJQJ1RCCbfXHnglqeBrTq9k7SWg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB6829 X-Rspamd-Queue-Id: 93A8640039 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: w8ma1tuedjr9id57syexqngapcqrhy4p X-HE-Tag: 1678116663-282622 X-HE-Meta: U2FsdGVkX1/sDjGIMDlr8XPhOFV+ctK7j3EpMxc5G9ipTNCAUZtnmcW8e8XW5zoCbvdEzn8yjXaufB+avBH1aouV+1YU/iCXsB2QLbE1dyQpc/ywm8UEyRz6pW1pXGr4Ns3ppVE4tLfgLbU4LJZcvVqLVaEmBG6jHY7rdGQu9cmX38k7aCxiqMFZ9jUpJ7BCGd4jbMadpUFssrWVuBdLIMInSU9f+uQ5ZqnoPxk5XsRsoxP3ZY0eewiu3EM16y+hlD4VCdldGaZsrxn5TNkX2Z67/ckWIJ4F7ocHqo8XrZVC4YHkjunjjckO6P0LYLTylURLSp9FI+RblQJNG1HTQCxHERu6fY6b+owyZZOjk4+gt9Va92tkLSLdIH/9jd/YCsWsjjZoZtzpO7k8Ocox5rDS47twVMCRdMmXYqqaT2X1ztP/Yo8IwESXLLkTzoD8J9sWyKvQrAU+ymncwE53/vVN7/31sMbqtifyUIlpM6VG7fhq+l8tWjYriv7uQrlJ2iTp5YM8pbAzB5vXIpkhKkm+iTl8fsZFiP0boxeB3nZFW5YGhkWEAS3mTNqH23p5RYcdu5SFnRvw45wGCCyNn+0zlIAiAPuA/4IEy7dMjBX+W+eW/1OXGG4PlQEC+k+bU4Vd/mjG3oAxH8zVTSPzUfGNVyn+/Li2qaKJ+TMfqLskG4RVLXnqSj1StIP9BZdazr0vYS+xlj5Rs0q7jrfPH5lxlU3AgTLwGNBYJBj3ScJIbNek5/MNE1WP81DXcIx/GCD/4OD5SCG5dx0E0ZJsy+WFde8fH9Qm8xxzUYeaVi7xDJTUQxnC8De3ksATG8ThojxpVDlQOZk8lU/DdxkplgVepHB/q0DN8QZJqcyL25PwxERU55tjuja1RcqFKO09GMUDwVYptfWYeAvSEdrb5/4lbumF1ugUNw2vcQZ/sRmWT1tb/OPx7sQG1PARTyCOElgKLVA7pTxDiUDg9xq lQm83krJ MkfEitVJ/yL6v4M5aTSkZu9zn9B7YMDYPWwsOGuiJwRGFEhbybfkuMYrIw2mq9mA8iAReqmNPzHbtR1hjXA20FVSKf2YRnsbwjzomQovvPjojlDFXNuOQE4G/patfVtHGklsRyF06OKfr8STkg0zu4aA0pQbCn1q1qZgW2qwjik1++tBiIKLnGj497NUSSPyaRqn4Mn9KTr6gmDa9td8VDcCOfAzfYDndsSk2 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 03-Mar-23 11:23 AM, Huang, Ying wrote: > > What is the memory accessing pattern of the workload? Uniform random or > something like Gauss distribution? Multiple iterations of uniform access from beginning to end of the memory region. > > Anyway, it may take some time for the original method to scan enough > memory space to trigger enough hint page fault. We can check > numa_pte_updates to check whether enough virtual space has been scanned. I see that numa_hint_faults is way higher (sometimes close to 5 times) than numa_pte_updates. This doesn't make sense. Very rarely I do see saner numbers and when that happens the benchmark score is also much better. Looks like an issue with the default kernel itself. I will debug this further and get back. Regards, Bharata.