From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4396AC2FC04 for ; Thu, 17 Aug 2023 05:32:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 48272280017; Thu, 17 Aug 2023 01:32:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4315E280006; Thu, 17 Aug 2023 01:32:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2AB13280017; Thu, 17 Aug 2023 01:32:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 1E208280006 for ; Thu, 17 Aug 2023 01:32:20 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id D24E91602D2 for ; Thu, 17 Aug 2023 05:32:19 +0000 (UTC) X-FDA: 81132475998.27.41327C8 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.65]) by imf21.hostedemail.com (Postfix) with ESMTP id C96C91C0013 for ; Thu, 17 Aug 2023 05:32:15 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=fb4+kNc1; spf=pass (imf21.hostedemail.com: domain of yan.y.zhao@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=yan.y.zhao@intel.com; dmarc=pass (policy=none) header.from=intel.com; arc=reject ("signature check failed: fail, {[1] = sig:microsoft.com:reject}") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692250336; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=keKgCO5OtbMBMCoT655Vjfq5w6ocqAvV5Jx1EYLc+90=; b=OsS2IHQfsR/U5wEgREfG8SobtKiY9DeK7t5b25791JEm91iTctoLziXTMG8BHv4+ERDSEj 8NPf21QbB1qq9HZxLtGrLOqEJ2MQuqoP7M3TOCsjE24HPCO0jj6s51nk9goFf6t445AIem yhQ4hL29IV1DBZYJ3PeaIHrkASFw1F8= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1692250336; a=rsa-sha256; cv=fail; b=Z30KfNeoHVKIKR4wOJXDJ3CIxjZFDjZ+OwTusprrMqT5XJGtNgdsKAHu/idksHRHceKxXc BWD3ki0atQdXlNWx5RWzIfb2ubqfj4rCXNpFYW9+DuotSQgK0x84wZZUdNVFPx3SQ9cklM iwNfxUOcFS2Giqc/Csm22Vc10rH8oPY= ARC-Authentication-Results: i=2; imf21.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=fb4+kNc1; spf=pass (imf21.hostedemail.com: domain of yan.y.zhao@intel.com designates 134.134.136.65 as permitted sender) smtp.mailfrom=yan.y.zhao@intel.com; dmarc=pass (policy=none) header.from=intel.com; arc=reject ("signature check failed: fail, {[1] = sig:microsoft.com:reject}") DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1692250335; x=1723786335; h=date:from:to:cc:subject:message-id:reply-to:references: in-reply-to:mime-version; bh=TexmDdrWcJNTLmeGnLXsJY9dzgZ1oU6KJkucO4WWx3A=; b=fb4+kNc1WRlwlIOZZH68rfD42cz4x7YCFU4KPTc7jzMCoqd18R8zFDP6 OvK6InplHYTQXHplzvBdGVJjv9q6SY2y+fklgV9uwJDGWEFhfYUmqM8Xg lm1ahyrosz6cfJmQDnzr2fABuRAE/EvvB0iIwqQXAn6PENgmBXUbYimM2 Dfj+eNjJaEB59/FHBqkJ7loaQ2gwGQ4HT9c+1TA4fVdawNd2H9RV6S3Ul GhIhiTAtVs7nSYHvphmglsQ16Fh2WTMTgMRgoPLC84ItVwVcPazpvEClM 2azJSF48skxuIlSsA92b0N27JXHd0pp9OvjXfkrr0dNpUa0ubcwMNPMdd A==; X-IronPort-AV: E=McAfee;i="6600,9927,10803"; a="376452937" X-IronPort-AV: E=Sophos;i="6.01,179,1684825200"; d="scan'208";a="376452937" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Aug 2023 22:32:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10803"; a="804498772" X-IronPort-AV: E=Sophos;i="6.01,179,1684825200"; d="scan'208";a="804498772" Received: from orsmsx601.amr.corp.intel.com ([10.22.229.14]) by fmsmga004.fm.intel.com with ESMTP; 16 Aug 2023 22:32:12 -0700 Received: from orsmsx601.amr.corp.intel.com (10.22.229.14) by ORSMSX601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Wed, 16 Aug 2023 22:32:12 -0700 Received: from ORSEDG601.ED.cps.intel.com (10.7.248.6) by orsmsx601.amr.corp.intel.com (10.22.229.14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27 via Frontend Transport; Wed, 16 Aug 2023 22:32:12 -0700 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (104.47.56.172) by edgegateway.intel.com (134.134.137.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.27; Wed, 16 Aug 2023 22:32:12 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UeLOztkkjq78L0yw6uQLht68oyynw1fFtrrflbgdBD0C247SOzYoFR+eZRQWIlvvAZXADzKWUnXi560MFDVSrJW2JGvjR4NqI+UJZbqdP+37tt5l1W1g+byIckuunqKUYTC1BYXgYQcWSxwtFW4TKXiKV7dZRiGiSZXiz6cPbZwr6nGgDpkAY4knlU4oBZ5sPgcJluWbcaFMlNbGMwanIfuq9BqSqbGGOnFVf9zT8bM/G0zEmypgDDzjXJUlLdP/Ksb3p+P1nXfXlxzcpzRJSdxQeuD+CJzizMHGqbB81+LxtgXIRyQsxeYub7oHQ701LwBqRcDed9DYlPvHyGdY1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=keKgCO5OtbMBMCoT655Vjfq5w6ocqAvV5Jx1EYLc+90=; b=a9ymQNby2vdWcHcMlt3zQ2HBQG67MIw3k6gY2uwnqsULkn63r93kW3y/43Kqp1LovIPrRBwDkt2/7wN0L04snbFevWSCh2LLkLMdc5wRYKr+R3KTomHm6QDY/3eO2eWrED5iG2BtnfzVcuhn/YnJ/8ccdRr5jBgWjQDopq9UPaDgPM8UHI9vY1cWrOcNTkX3czeKru+olibMPAiQYSmbq9W/4oRlrRFyt+vJY6A2WhJ5z8o/EpbEU63E6W86rlqYA1CCCcy700GvlypAfj9p2FogRGerM/V/YIPpCDtgkcZnGQGdmOq+nuyvot29LeUuex0M2wJw9/s1mVvTor1lXg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from DS7PR11MB5966.namprd11.prod.outlook.com (2603:10b6:8:71::6) by SJ0PR11MB5181.namprd11.prod.outlook.com (2603:10b6:a03:2de::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6678.26; Thu, 17 Aug 2023 05:32:10 +0000 Received: from DS7PR11MB5966.namprd11.prod.outlook.com ([fe80::1b1a:af8e:7514:6f63]) by DS7PR11MB5966.namprd11.prod.outlook.com ([fe80::1b1a:af8e:7514:6f63%2]) with mapi id 15.20.6678.029; Thu, 17 Aug 2023 05:32:09 +0000 Date: Thu, 17 Aug 2023 13:05:07 +0800 From: Yan Zhao To: John Hubbard CC: David Hildenbrand , , , , , , , , , , , , Mel Gorman , Subject: Re: [RFC PATCH v2 0/5] Reduce NUMA balance caused TLB-shootdowns in a VM Message-ID: Reply-To: Yan Zhao References: <6b48a161-257b-a02b-c483-87c04b655635@redhat.com> <1ad2c33d-95e1-49ec-acd2-ac02b506974e@nvidia.com> <846e9117-1f79-a5e0-1b14-3dba91ab8033@redhat.com> <4271b91c-90b7-4b48-b761-b4535b2ae9b7@nvidia.com> <37325c27-223d-400d-bd86-34bdbfb92a5f@nvidia.com> Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: <37325c27-223d-400d-bd86-34bdbfb92a5f@nvidia.com> X-ClientProxiedBy: SG2PR03CA0091.apcprd03.prod.outlook.com (2603:1096:4:7c::19) To DS7PR11MB5966.namprd11.prod.outlook.com (2603:10b6:8:71::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR11MB5966:EE_|SJ0PR11MB5181:EE_ X-MS-Office365-Filtering-Correlation-Id: 1b3b1896-4e6c-48ef-92fc-08db9ee34d45 X-LD-Processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: fxW9rwpUV+eEFsVbp3F2WvMAzmjYB6fAjgHL30X9wIoDbq2o2D3aHgT3iNTqDK4qtZ2p84TtzbcEUJ6HniByrbRFltaYISyOcTPISwgDbsiuVBwQEunhOxFXTbj9uL2rlIxuRymb6j7phTVQDa2HYtjcL0pTfafPlTLieQfnYFvplwX7Wz9dFgPT+lo4hterBcvtmXEYAfs+VyOh9ltuFJKkl2lzi629rZCGooCCRL4I49yCHKcIZ/zSCLH8/P343mjAvWTwD/JWAmv3tLT4YqauUKI1F/z6xPpaX0wCbBXXVWjeQ4mJL1mXMzZh8bmTsimAZ9f24QZcIHLTKGsHNksdslS9/Q9mqC3AzWKswTM/sIuvfCnbnHBER/GE9jXCJL+tk17EdgM8zJ+9AKn54SOgEvIlo08JvtNNp5VS6nwaNjyvqLx0QK3WvFDSWdxIscudAWjQJ/Z1X6sc+Gw60KPl8jw7L1xC66VZKZaGdiFGIO4QwyQY6UOf6emwNSt8ABBBEAvoc0eiJXAhKp0XYk4ynwO31Pz9PvO2KH1FPSIaBL0h/XBMZ65LUwQqA/So X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR11MB5966.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366004)(136003)(39860400002)(396003)(346002)(376002)(1800799009)(451199024)(186009)(2906002)(3450700001)(4744005)(26005)(7416002)(86362001)(478600001)(6506007)(6486002)(6512007)(53546011)(5660300002)(41300700001)(54906003)(316002)(66946007)(66556008)(66476007)(6916009)(4326008)(8676002)(8936002)(82960400001)(38100700002);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?lMa3TUzW4rv02GRL8FPdOGvFy/3F/XDWQMrApOsV6KvmInwBGodf256q/ssP?= =?us-ascii?Q?IqnX3NfRXWICqo9c/75cYYqMBpHSjCcDQ/R6i9Sbz5IohO/brCL3Hg/gcMV4?= =?us-ascii?Q?E8zgYmSvQ2cX1XhRrogacLrkC1C4MIpaftnQrM8vHhnY8AjoHiSMdu1k8YUK?= =?us-ascii?Q?20SWbezvEsmlCHwjsGplzVNizdrrucOunyknr0zqIpCUMhxm3aia4dvDPsSh?= =?us-ascii?Q?L+yRmWrNyzQCg1eFIQrP4ol8k1PnausWC/7em/L6h4UtiVoU91/ceyj8sPGo?= =?us-ascii?Q?GaFz3SlTJD3Cr/bZFEHVD9kBnxy2vxqmVN8MhicJNmS/vDuq211I/Hsr7VHI?= =?us-ascii?Q?gNsyAB9tKJuELVJPM+vL0tnHzVwO4XFZCrzL2TpccRL6GOwnE4ZO3qsGPHBN?= =?us-ascii?Q?nfwjw+jA4EfqIRlW8sd1vKx6JUUM+H3kJRl6xmYYt5NPGhLC1d/ImiF5TWq9?= =?us-ascii?Q?pc/r3CE83Pj9sLaM+BkHmwZgt0441ZWaimUlgfzo7UefAiCGtgFdOta6fQQd?= =?us-ascii?Q?gf/f5mh8u09CdOyfuPq7G77rmbR9SvzfYwvZvarXqRJJbZ0P3RTD0eP1DdZP?= =?us-ascii?Q?T4XQSnzqJQYbB7q+sOQVcKMQR8AZyoRDvG1W/VGMqig32z5EyLNdDzGSXJOw?= =?us-ascii?Q?H7+psr4n6UAufZMx/KlKDeqzVNIf1LS3rXPSbI/lkK3sYoA0tWUT3Fp1a9T1?= =?us-ascii?Q?XAIBpnt4yVrVlJWJS2ai2rt5uFmQeKe1lEdiDkCtE03YSZHQoUGKORpqXkBd?= =?us-ascii?Q?SJ1z9pNPTZ0iPGUIcKZxfp7haFwF4aj+SyOIoxVeVtkkez/NPLdtHgzlygrc?= =?us-ascii?Q?r2j04VcQ+4wF/Xwe7dtsrnN/G34f9D/qiEK0uv4YwKNabv2cyGYwT6ZNkecT?= =?us-ascii?Q?ZvQOPbLobjv7EfIMP3LLik50XjFHN9XvgLLwW7aa3KXCEImtqfPI8B+RrvS4?= =?us-ascii?Q?e+fYABYxS0pZm65hmQYQ0PUW1ib6CNTCN8uzGlK3S9VWYXaw0S/V2ZADs4Ie?= =?us-ascii?Q?iNs91zw2JBqcSCnHIBYPHhz8AHb3C+VmNgpzsTH2uvHTBFxu5DT6pGG0kWfe?= =?us-ascii?Q?mvgaaPAnDZXacZns5EtAkYsY4PI9Hcopj2qjJ/YKiBhOnQNOlbVTAxHGxTfy?= =?us-ascii?Q?UGig3L6UpgqIecPUyk+TFgaD2ar/uvQz0zYstvgKYKvzZCfR75Dzjg40HqAs?= =?us-ascii?Q?T0l6NKp7lxE+bpYZSJEXddERAXqGF0O103Qj5fOHEdqG9ZkWSake0Hkz5/TY?= =?us-ascii?Q?xwMESZhfU3SbM42QkCFBaWWu/F9Yr3I4FWBOzZ+bj2FsVK3h6eu9FMUL4TIY?= =?us-ascii?Q?SlSvKxT8Jm7YwLPX1NWqLDwxoZexegMuQlgPu3e/x4xPP9ntOVPkO1FhpK6H?= =?us-ascii?Q?5yHid66gW1GzR0xFvZ4+36RYLWD4J0BMUN5R5O+Ub43fKx0203QJk90fx9F7?= =?us-ascii?Q?SPLNeGKJTUS7N/pn5E+MgQSKewleCcxlJcMFSxIfDsKuI5lFmR0o3ECGmp9c?= =?us-ascii?Q?sM5kYTwxkPcOi63fUguAvu9XCCkNptjw5GrVh++puZ6DDsVRu+Jkvd8ze9c5?= =?us-ascii?Q?do9uql2GTKH+19xTJ1LliVKP/C8yQUePUxCgM3R5?= X-MS-Exchange-CrossTenant-Network-Message-Id: 1b3b1896-4e6c-48ef-92fc-08db9ee34d45 X-MS-Exchange-CrossTenant-AuthSource: DS7PR11MB5966.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Aug 2023 05:32:09.6345 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ma5CCtaRDwuGUAOL0m+lTfXA48GYj93BRH9RgNUb6iRepuFOZlj2oj6WteM/3vwLMQ+y5rlw57G/M8CaoKOTOA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR11MB5181 X-OriginatorOrg: intel.com X-Rspamd-Queue-Id: C96C91C0013 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 8bar4kfn1y686anmy1qd9tayd3azcz4o X-HE-Tag: 1692250335-378354 X-HE-Meta: U2FsdGVkX1/olJjp14QFuJ+2FscIH2wvsq1ZUPW+SST4imb3Aud0o6VBL581DWwGV90bl+sq9WmmNY+oPushRM/FX3qSrpSqsRkNd/n+6oaIudzW+5oPzk8b7ptEj+fLGnqzqJXgl8xLNcGVoBxFw0JXh6wgkTFcIlgbjj5hmVitQMu9l/KP1sYrKbrTCcrOElVuAlyUk/pWZlybGgpl4B9uTqC3eBs4TZlWplrIHqaIHIna/Ls018vl1i/N/ln1txgNkY8Za9PodcT89gjpF3bykyWTQSnLh/ZdctotEh9s/oHtP9S3Ju8nee3JwJRu0Cpp7Jogn7WKXTUHsMHoPskwA7cpJF8GlyUzBP50jzXYCKxGw1W3+wCfli58oTqmEyEGFAqQTAC0tfB986CbQmJeo6n37jZ6d3z9DNa2B6q2Tq9AvWf3yxoJT87CkMZC9P9mF1gyN4yMWHRk4wuNtD+3YXBFtTKjnq9o1UdRY7Y/MPVpArK3JnLq87WsHe8uhzbCZ8KtEIOQ9Ja9US3P3elNC9KsC28OE5kTtf5Xg07KKwrz1Hqps3ftuJ/kdt+raPxTqAjQzZTkjswYFIiAitpeubhY7+pnaBDV3km43toYMT1G4KUP/SRSnUQg0YGd8H7juAGR/qcaf5WyflceySRv9MLB2CRV+RZ7A9JyZUf5l2SMuTHAdazYPbCcDJSozJFrzQ01fteTl6z5yITycdXnMtVIdFxrCmax08uneIgJ9hvfhhbwN0KS+cpEi+LHPx0NAfLn8fNEaR3GDvpHt75hg9PVXnoW9YhXujVMza+k4CIhzz2rA73Od/hw+ZR0AYYSG7IGYUskovxleMmZ4DxQ/1V/0nEgW353Cg/3bMICasah1ayZqc/P/9A57sYJe2j/MKI0nOcWgZAk4XiXGObn+qJwDXTiBUmoSAy41rV7GkPxlcd5T89BEatZdQYiEuNCHtcXlMvVi11uaaD f4+NA88i teM7YEVDXp0dTrl6Rp5gWyWr+ag2ZMgfp94DQd2BOwEsTMlgpsZLfl3P5NRC/DeZPV4Owp1+7RbWEEsl3B4ZZSIYNANRxDFkN7fjHrt1dlECD+iulbnp0m7qpAO1UZLZl0swvwA74Esgbm+wiUCZtJDENxquMu5/o3dJm3nZ/ZddU4xCQzJNuHmSLJwkICD6JbM20mOJOZ1j49XuH1o2V6RcB1UzKnJrmt89q5ugqbBrr9G3YLBpkUZxLuci2wyR+WcyGwNXT6ZgXXRgPEerGndXTS87sEMTiKkfoty5G2ZObTd1XIb/C7IrEM1qh4r+OfZJDVZhLUxamN5iqXFBX5gejRje8hG2G6C8JrYwdRijIEbFYmz56gu3rjLAuxU9Qo8GJYBeucv6etZH6I8kw4b2tMg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Aug 16, 2023 at 11:00:36AM -0700, John Hubbard wrote: > On 8/16/23 02:49, David Hildenbrand wrote: > > But do 32bit architectures even care about NUMA hinting? If not, just > > ignore them ... > > Probably not! > > ... > > > So, do you mean that let kernel provide a per-VMA allow/disallow > > > mechanism, and > > > it's up to the user space to choose between per-VMA and complex way or > > > global and simpler way? > > > > QEMU could do either way. The question would be if a per-vma settings > > makes sense for NUMA hinting. > > From our experience with compute on GPUs, a per-mm setting would suffice. > No need to go all the way to VMA granularity. > After an offline internal discussion, we think a per-mm setting is also enough for device passthrough in VMs. BTW, if we want a per-VMA flag, compared to VM_NO_NUMA_BALANCING, do you think it's of any value to providing a flag like VM_MAYDMA? Auto NUMA balancing or other components can decide how to use it by themselves.