From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C989DC0015E for ; Fri, 11 Aug 2023 05:49:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 48BBA6B0071; Fri, 11 Aug 2023 01:49:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 43BB46B0072; Fri, 11 Aug 2023 01:49:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2DBFA6B0074; Fri, 11 Aug 2023 01:49:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 1E01E6B0071 for ; Fri, 11 Aug 2023 01:49:29 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id E0E4A1A03A8 for ; Fri, 11 Aug 2023 05:49:28 +0000 (UTC) X-FDA: 81110746416.19.A6BBAB6 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.93]) by imf26.hostedemail.com (Postfix) with ESMTP id F29BD140017 for ; Fri, 11 Aug 2023 05:49:24 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Z8POYr0F; arc=reject ("signature check failed: fail, {[1] = sig:microsoft.com:reject}"); spf=pass (imf26.hostedemail.com: domain of yan.y.zhao@intel.com designates 192.55.52.93 as permitted sender) smtp.mailfrom=yan.y.zhao@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1691732965; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=E8LB5rOySyBBgdWvBzuhDQibnFtQRwHwEEOYTWbcbwA=; b=6sIs+goAmkgRP7kepvdRIBF1DUlFi0YzQDcMI0EbgreNPdnyRxCcM64pVqqm18Slnvp3Gl 02cAR0yRFLctDTZ/qQTTxopoN27FfXfUjhw9R8VbUbus3TbWXoieTLmSgZqEcG5fdnEbQP t17SjRVbsFVyvI2xcrxESsyT/Jop1A0= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1691732965; a=rsa-sha256; cv=fail; b=G7aloDyOO9lFFq7c1ACxGeLZZKEGiB3d+ciPjHPTM3eU1Klk9WFa5BcuCliTbgm5DeFkeK 5S63SFnbiBnkftcyHdvb78Or6l0zp+uhLG9TSOzF87NqkA2BvipG9M2gV8SJTRITA+Ewys jA/7pyusYNAqU+TgfjQLCYBCk0eOJeg= ARC-Authentication-Results: i=2; imf26.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Z8POYr0F; arc=reject ("signature check failed: fail, {[1] = sig:microsoft.com:reject}"); spf=pass (imf26.hostedemail.com: domain of yan.y.zhao@intel.com designates 192.55.52.93 as permitted sender) smtp.mailfrom=yan.y.zhao@intel.com; dmarc=pass (policy=none) header.from=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691732965; x=1723268965; h=date:from:to:cc:subject:message-id:reply-to:references: in-reply-to:mime-version; bh=F0KyBAsh/yPpfpxlIJ0N7vvCJyOiEO1TPChGiALt3yQ=; b=Z8POYr0FCLxQXPI+KhGI9pzERswvR73P1s31uq+xM6/XW1NHfYjUlWiE tb8StETVPaz2zDckI/kZaazgmx6XBSewKFEbCW8opuYH6qLOMU+4smI3W GMqU4ZqvGotRhfURAxwaDZ6AhMTJsgGVgiH6F64JxMaZosidBgVcvfhuF GW3rsViJyocOdevMgDXl/7mVKXQ8V665jVhbmSc0rWFBgnrvn/K+i7wXJ OgqTmrarwqeNnfyfwtxnIRm3TBPOXZnd5Ync5FkC6ZBW+YgBM+fuR+MKf gox8wcva+veVjJ37HHpSriUlHtO5GgGURWPejO8+QOGJfKbhV7L9/IFBa w==; X-IronPort-AV: E=McAfee;i="6600,9927,10798"; a="369072418" X-IronPort-AV: E=Sophos;i="6.01,164,1684825200"; d="scan'208";a="369072418" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Aug 2023 22:49:23 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.01,202,1684825200"; d="scan'208";a="876037972" Received: from fmsmsx603.amr.corp.intel.com ([10.18.126.83]) by fmsmga001.fm.intel.com with ESMTP; 10 Aug 2023 22:49:26 -0700 Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Thu, 10 Aug 2023 22:49:23 -0700 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27 via Frontend Transport; Thu, 10 Aug 2023 22:49:23 -0700 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (104.47.57.168) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.27; Thu, 10 Aug 2023 22:49:22 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eI48cpOHdKSY5m673bURNTdqHpzuMJDV7QpaU3qq76XkaY/nt9SBmOqLp4AmwhebJbs8yI7/OD/EBX9KdEDgx0x+lvgqLPuYgLyC1yF6aOsWCTQ0jYZDXZUDDBpiFkhmnfLmFpuqPsh2vSFSAYyMa3pjQh32I8SnHiaXxojBWwMijj7RrEUsbllO+7DF0V9/fOfcxQKVEuButYMNfjpSCEBCSLJViQ4ZtYTNX+sMVaJvCsNcUfa3kvMnzNKvpRCkGvar/e+kAnNPZkqAvCJGhtSVH6tl89DX/k+V0ePf8szcps3BRVU4qcDDLsbI5JMM7KTz2pClkRGHAC6Hzf8wqA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=E8LB5rOySyBBgdWvBzuhDQibnFtQRwHwEEOYTWbcbwA=; b=Fyunw8UsZYsyt25V3ct1ZFr8hxmHHRf+CJ8MBQJaVwLZYr42Ic66kX5iiSw9xXSquGolBnvUTcxrEFMeFlq6eAzCZXl7QpuritZu2LvPsbRZ35u0cYFqGODK2j8mX+Cgg2k4sLSxVDMjlXKqiwYXZS0d0jsm05mzTgMCiG1aI1BBucz8CSCbQQHbbbgCgXL4HAvMeW94/wL+clPtBCAG5iynAb93xfdHVP1DPVab0QhSEkPICF3aq9kM86Nt+GfW0Bp5Xcp2Ptz551ivdXrCsnFYVVCkerkN/ljrMNarQoD6OQbUAm2pVD/jEKF1AIZJAl+7LFkm+Zv7HNLMhL++EA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from DS7PR11MB5966.namprd11.prod.outlook.com (2603:10b6:8:71::6) by CO1PR11MB5138.namprd11.prod.outlook.com (2603:10b6:303:94::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6678.19; Fri, 11 Aug 2023 05:49:15 +0000 Received: from DS7PR11MB5966.namprd11.prod.outlook.com ([fe80::1b1a:af8e:7514:6f63]) by DS7PR11MB5966.namprd11.prod.outlook.com ([fe80::1b1a:af8e:7514:6f63%2]) with mapi id 15.20.6652.029; Fri, 11 Aug 2023 05:49:15 +0000 Date: Fri, 11 Aug 2023 13:22:15 +0800 From: Yan Zhao To: Chao Gao CC: , , , , , , , , , , , Subject: Re: [RFC PATCH v2 0/5] Reduce NUMA balance caused TLB-shootdowns in a VM Message-ID: Reply-To: Yan Zhao References: <20230810085636.25914-1-yan.y.zhao@intel.com> Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: SI2PR02CA0027.apcprd02.prod.outlook.com (2603:1096:4:195::14) To DS7PR11MB5966.namprd11.prod.outlook.com (2603:10b6:8:71::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR11MB5966:EE_|CO1PR11MB5138:EE_ X-MS-Office365-Filtering-Correlation-Id: 3a058e89-e44a-4cde-9870-08db9a2eb225 X-LD-Processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: bXUJm66sDudUO7AxN7mLenwigtL+NtfNNBCnsbarviBcV7C+zCQ99YawiKnPr8PpgOkyAV2Z7tj0sbeo4n4KHcK4XgHDDmB5rVEXaID25O3BsqmbCV9q9a7FAeKKE2lZJWrmYSNU6dB8tpv4L41FxmBCuS79tPV3vCWQqpQJVkcoSpw6l2ZJHEn2Hh1fy0aYD/PUXzOXU6PN24Y9yZXzjUYMewwHtlIEYF8jfbaOsZSWT/075gUjYYHIbNGwEh9VReG8wyXcwt8+lS12j394mgIY/hdv8Ct6Z8XUepA/eP1EuSJPkwLOT/p/V91BBxc+GjsaEmW0u18ijbM2h6Mok5JYhjCsRoW0ga9HwGyXOLv7CXyCklo+BD6yPnGb65jpSTKEn6CUAVoSWJ1e2gHYaVuPTNbeAAOOgV68fp9sKWZmBWpnT/CRTcTUd5ujNHSPD2L0PGgRP1rdHkEHRHK8YNk9nsApUkxNO98IK1/fJIUHLWMDCsQIw4zKG1KBitrFJmPDhpOipWh/PTDWX83I7Wev+Hm069w9ZLqIyJNNro6v2+uBFmbO8EVu9teuVNKk X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR11MB5966.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(39860400002)(376002)(396003)(366004)(346002)(136003)(1800799006)(186006)(451199021)(66476007)(66946007)(66556008)(4326008)(6636002)(41300700001)(316002)(6506007)(26005)(83380400001)(86362001)(38100700002)(82960400001)(6486002)(6512007)(478600001)(2906002)(3450700001)(8676002)(8936002)(6862004)(5660300002)(7416002);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?phQigCwxc18JFkSYe6Cj0fgccNiPPsG6p/a18Geew76jd3qd2BQZqvnbdcVo?= =?us-ascii?Q?REagu0SE4O/j/BDUXa3YQG9i0aJuH4o+/DizH60BG+LMHbD4iz/3GcqqZU/4?= =?us-ascii?Q?g7TsnTi29mDH0tpmjpk0eELhfgxsfcrsWDs+O+ZRO6/LF36CKdO91UbuOitU?= =?us-ascii?Q?YZuCDuFUv3NjhswKCFeCBFW8ffnkV89yc20zsv8gmyt4beWP0gJMJib+m4kT?= =?us-ascii?Q?RwcWCjRrCSkc791qgTeeHq0IbiljFP/Eu4hBakEgzzdmY6GVzLs6d4v80Rjn?= =?us-ascii?Q?rBGXMz2hpUGu1v8Tj2SVN2i0jJRVpmQcNbB/nfrEs4WjZkH3ZTsIbeeDuwiW?= =?us-ascii?Q?57a+2jEqZ0CkjQe2tQieuEnA1UinfeLrQ3UqjiTdPOufLGTuty+DJhYGxw3T?= =?us-ascii?Q?oEIdMVUD2iBuxrDIPj6CRCWqIasFt0ox5GyFTMpzu0Y1Fai8w33cn/K/AwWr?= =?us-ascii?Q?lVJvwJUnUjZxjgAjt68xUKyW1WNfPhqDS2q1OgvkJiEqxiVInKudunG8Jg2K?= =?us-ascii?Q?UEFCfsWkeBm0kvDsc9WuT7soXJIQfyMd4GA43uoygoQVym1hnsUPy0NRNson?= =?us-ascii?Q?NI98W1OzKbGmAklre1pXS3mr8cq86TxBXtUk4HflJ5B+YmFw1iAkLr/i+Srh?= =?us-ascii?Q?9JGKRhu4eR1tJUCnyDR5l4ZLVhJdAOc4pRRbBV9U+d5fCSbyZlZUUPaUEC/T?= =?us-ascii?Q?sKEUMkhb886n/s1SxSRRbltdr5sfSQ+9N+KXHKOlLScc2on328apgq3cebks?= =?us-ascii?Q?YzMkRH0EGTMFkyGBmx9R+eVgwe97UVAwu/gTN7Cr3t/HzNmZVXF4tJ65p7Fp?= =?us-ascii?Q?rTfmUU8EYOgkXKpL9TgWzL84oQ6PnqWpZ9YpkI4WjuV1ViS7uJ++2z/iQSr6?= =?us-ascii?Q?k2NfTKkqPxP0r6VGsHuZ3spAyNnYNs2zIQ5JfMQGgaIGj3n0D2Vt9zbV6Xwz?= =?us-ascii?Q?QQro5Ku54j5U6efii7hazods4kUgRS5pVnByc9D3ajWnD/GLwWuFbFu3nzYy?= =?us-ascii?Q?gh0BmuTZPLbbpyk8jH7erq4e3JDBDki5AXNkqM7CQjzG6OkIpIMInINSWqKD?= =?us-ascii?Q?ywv312iWu7dsAza9JlSb66dEcRP8XIxHX62+QwYjUq1GSpXEnINCClEi3mQ/?= =?us-ascii?Q?PcQ5kumyv/jBStXJPcwiv+D4hdlSGv5y1ANp6jtJOVPCJ7YzYTxnGxMwLVfA?= =?us-ascii?Q?nJ6EKNSEWZLVeeyf2OlbSdjlpgeJH1HPuha+Jju2My5PDpsrWaTFJanY6FAa?= =?us-ascii?Q?iTLzBjYvKHxH7Tc3g/ehgUd3JpayUFOP3XA/HQwvCAaVCALlFJDardxz+GMF?= =?us-ascii?Q?pdqFd93dbpw7XTEyhfEuK+2N0a4Cf/eL2huIb+huM1I8OuL7C/wl84M/TswS?= =?us-ascii?Q?4oF6nm3FovCbfUFm7z70+MN0ltFjygqMuMUPvs4AQQpghE/Ms/5J+S1Bo5nE?= =?us-ascii?Q?pmlLOEN4fFcIrM6kTkXIx4fz5sXgcfxV1qjIExG/eulreakyP/ljgQFBmsSa?= =?us-ascii?Q?kjmg7Tz2Gc13vNrMXcl8zDEO8o49UtcAGg/m0fxY0EmMDE6Vs4yOtghXChb0?= =?us-ascii?Q?B6gh210FPhjKpYCG9o4qNbV1t7W4K5YLNYe0MghY?= X-MS-Exchange-CrossTenant-Network-Message-Id: 3a058e89-e44a-4cde-9870-08db9a2eb225 X-MS-Exchange-CrossTenant-AuthSource: DS7PR11MB5966.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Aug 2023 05:49:15.2402 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: UFufpkBpKxSVFoqVZ9UL/ATydQrkn+VIYuJeoYRXR3XVFAdLbxS86wfhSACk5WDm8sOFq+lh4Ojcna2C1SfJQA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR11MB5138 X-OriginatorOrg: intel.com X-Stat-Signature: doseuiypag6r1pq3aycbk55umpufrgxg X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: F29BD140017 X-Rspam-User: X-HE-Tag: 1691732964-721014 X-HE-Meta: U2FsdGVkX18JMK/wiWI7J45VaJReWqxF5DKEZ645HkgRAWKHt5JUMZ9F+hnTI34jjrOkujHvYkLWmY8d/swgmzyfNA7oCLdiOQtkxrj+Nu2XImc5ASgOO8RHRENdJlMSO0eJIwyrn3u9Zq0JVkRomEWA3EOFo6em/BnBF7rmsijZfL9on1bGTgB90YtNWZ7Jupr0H6EO3x+jFlW4WsZdOJplUO3FP8BgFpk+ZkAFVCgFSSx7Tp1qs863lLiUqt2EzyFNheKxJ0xBQ0q6GAYXwjIUyQFz/mWOFTEtn1g3XnvA6oZiOE9dkcNBdROEKUSNr5C0F4vUD4Xhrna/dia4MD0j7W6t6PoE6hBRa9whUIEFdKOH0RCHqfeigZVXi0JKP9jg1w4pv6xzORfR0OfRUHjiP0NLm5JtPL7e9pnKJKakuvZVJwwxI+ZEyHWMJzPl0fuJEpJp2VduNUaTFCRibP6xyetw/oy7NH+Y+SZJCKoVAjJRspBZ0EN+OgAphGJtPY+UxT11oIp1l6f8Kx9Ge/7JAIE+kYqZH3XKpY7qyPhrkmE5HHz+43ZUdi3VyOsgqLdZhCL1VheKuWWjidILiOMV8YZeRQ0h8H/DrWS2RAiIas0a3DJeBoLB+XLSkgTk4zH9B48d0zgemIltPs8meGPz0lWnKxDuN3WhKB9EyPKxhlsRGHkiASW1+dYBQ+UD8x0Lj42M+5f/y7UiA0d6bTVOZA3SvuwZuAu+pIXmxeFdRIhLgJcZcBQQ4447PCmxueHhByNA3DOU70aFQ+WVARVcJ3J/VFqHXR9Fpy6iKtQuNmPB4YxI8z41P0qKaLFQWiDaLwRzEU/4xSeHDYTlSDpaVqGzOvBhrAIi4a/SLul0oI99jg7mYCT4u3GL6BdWPxjSNozKksfmJzNuVVZWGaTbEQZEkh57komrUdoR5cxKfANx0UlRl/iM+qWomfAACyLu2yPdBrRWXTUqBt9 MssGl+TW FR8xXuPdVUZwHuG6yW1+/o8+nnRBgOIVXsgtlFKbkVq3lcAMPT10S/NtJE1XADOxaVMQqkeOKri6jtOXDkv/arApr5HrmDBHa+SGynsTcigzYj2q1EA7YnapFT3t6sWmmGMeqfGzgl/EpWBA7AozLA2uo5aDzpIuEcd6F9f9fU7CNaGYVhhL69pfw/8NQdiJYwN8Ag8Rpw7aUkBiS95jnLDNd1guAtVSUS5I8BYgFd5M/RlFAAaRkDT3fpwMby/fc40/YQAKqxKLf7au56joc4KU/6c5qOXSjzm0NQp5IhBAdykwm03aiZuUp0Cm6yY3YZoBWzbn+C3B0RB59OM1NuKWL705p0eBr5XNQdRjbxsDsEQbDobTvw7HBVc44qr7z53Kc X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Aug 10, 2023 at 09:58:43PM +0800, Chao Gao wrote: > On Thu, Aug 10, 2023 at 04:56:36PM +0800, Yan Zhao wrote: > >This is an RFC series trying to fix the issue of unnecessary NUMA > >protection and TLB-shootdowns found in VMs with assigned devices or VFIO > >mediated devices during NUMA balance. > > > >For VMs with assigned devices or VFIO mediated devices, all or part of > >guest memory are pinned for long-term. > > > >Auto NUMA balancing will periodically selects VMAs of a process and change > >protections to PROT_NONE even though some or all pages in the selected > >ranges are long-term pinned for DMAs, which is true for VMs with assigned > >devices or VFIO mediated devices. > > > >Though this will not cause real problem because NUMA migration will > >ultimately reject migration of those kind of pages and restore those > >PROT_NONE PTEs, it causes KVM's secondary MMU to be zapped periodically > >with equal SPTEs finally faulted back, wasting CPU cycles and generating > >unnecessary TLB-shootdowns. > > In my understanding, NUMA balancing also moves tasks closer to the memory > they are accessing. Can this still work with this series applied? > For pages protected with PROT_NONE in primary MMU in scanning phase, yes; For pages not set to PROT_NONE, no. Because looks this task_numa_migrate() is only triggered in next page fault when PROT_NONE and accessible VMA is found.