From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90048EB64DD for ; Wed, 9 Aug 2023 03:25:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A30326B0071; Tue, 8 Aug 2023 23:25:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9E0DE8D0002; Tue, 8 Aug 2023 23:25:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 832A18D0001; Tue, 8 Aug 2023 23:25:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 71B3D6B0071 for ; Tue, 8 Aug 2023 23:25:34 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 443C7140811 for ; Wed, 9 Aug 2023 03:25:34 +0000 (UTC) X-FDA: 81103126188.10.3DDC9C8 Received: from mgamail.intel.com (mgamail.intel.com [192.55.52.88]) by imf22.hostedemail.com (Postfix) with ESMTP id 55CA9C0017 for ; Wed, 9 Aug 2023 03:25:30 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=ClTqGVdU; spf=pass (imf22.hostedemail.com: domain of yan.y.zhao@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=yan.y.zhao@intel.com; dmarc=pass (policy=none) header.from=intel.com; arc=reject ("signature check failed: fail, {[1] = sig:microsoft.com:reject}") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1691551531; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=I0A8flsy57uStzc9JUL/wDHLKdvVHpJfdJdQq47SrLM=; b=vJ1oeofuE89aVmQ3XUH0G4Oh3thKL+tsgk1vHDB+ljuXoZO1TSz5yezdmbfITm6whoLpAW uEk4seqbj+kHp8c/CjVHnJB9NWUa4UwRKa5BaiDz8ovKAE4j3CEUXB2iUyFh0B1J8hckZj z6HKAmkGxyMZeSSC0M3NuqJN8F81PUA= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1691551531; a=rsa-sha256; cv=fail; b=WZeiTV4J+TYjludq+56ypbbT3kOjdGL8iGybGbnGoDNamMZSgAMSp/25AuV4vbVmDEUZHL /8EYTZM9IKT1oAXKjuf2uTfKqR8LW17HnWqqVrQeeyeq5cSf3hz0aggUFS4vI5Mj6Nxgjw XXtfj/Jhm6ytzk0FSWVJgqo+hCiK+xM= ARC-Authentication-Results: i=2; imf22.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=ClTqGVdU; spf=pass (imf22.hostedemail.com: domain of yan.y.zhao@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=yan.y.zhao@intel.com; dmarc=pass (policy=none) header.from=intel.com; arc=reject ("signature check failed: fail, {[1] = sig:microsoft.com:reject}") DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1691551530; x=1723087530; h=date:from:to:cc:subject:message-id:reply-to:references: in-reply-to:mime-version; bh=ZZNuRulOgPL4sn6qfKbq7GQEtvqc/iXC/Ygq03ZGC0g=; b=ClTqGVdUmq5u9jg3tmJOSKrvMZbb0E+wMnTt6luHORvDvSZk4uXafekh 79L0mq728bRpG9n1XogEX45S2hDxH4P0gjpRXKQcc5d4+N773TEiTw7gP eJcXvHOvgXvsL1ejFan02UZTtUWdbM5G0uenahPiyJcLGiC+14P7QImeW nLdTHOiLoksvDbSY5ZWRvI0xcFSVaw1tUcEbbfjSJkG/r3dprN23mfYmg ufrfTPhUqpRq03T7uZq3CsXap3yGrTO8FV6xPx7Pr6+eLYo1HAn+iJEiz GeJ9am1V3ho5nWaSqzz+G6taizsGFTmH7+1m5va7TrDw25BE7Fn7KN/1J g==; X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="401975299" X-IronPort-AV: E=Sophos;i="6.01,158,1684825200"; d="scan'208";a="401975299" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Aug 2023 20:25:27 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10795"; a="766643012" X-IronPort-AV: E=Sophos;i="6.01,158,1684825200"; d="scan'208";a="766643012" Received: from fmsmsx602.amr.corp.intel.com ([10.18.126.82]) by orsmga001.jf.intel.com with ESMTP; 08 Aug 2023 20:25:27 -0700 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx602.amr.corp.intel.com (10.18.126.82) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Tue, 8 Aug 2023 20:25:26 -0700 Received: from FMSEDG603.ED.cps.intel.com (10.1.192.133) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27 via Frontend Transport; Tue, 8 Aug 2023 20:25:26 -0700 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (104.47.66.47) by edgegateway.intel.com (192.55.55.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.27; Tue, 8 Aug 2023 20:25:26 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ITGOfHr+9pBOkd9PhUrk6C3thx7N4vcFA+0DAl0ZHHxiO55Jw8CaubA9J9k2WysKPO7LwzO/9PfbRyln5/mum7yLClqnJYQfCd0lvXL7CrT8Ij7YGXEYAg7OAQNHTPCWHsAsM4xQqZEtabv2r9u8KnPX5Z23xqeHmFQJQp/tTCwJSprBtP+YPra/wClEG0FprTcHVX+jEL5PRZd2XsfwBia/OpaRJLXC8tTDM0kxMz5Knx5DZi8UUZy2juVPIU62EciOcFJ3j2D41XTOmxRtSjPLcF51dsNL9/cRikDX9G1/Qh56UC0AsucNlvo/pF/j1c4ROfUAjPkxB5fa6eZQiw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=I0A8flsy57uStzc9JUL/wDHLKdvVHpJfdJdQq47SrLM=; b=chhmLwMR/zCCnOpeHOlSke6iJmkSh8MZwOj/vTRXL1Ylh8PoG4d0eRgyc2+0jCu1KsT054p+g347UdmIXpkg7LmfH4ntIg9vMv7B3dllpJbZalKgtjDcrYu+PoukRcA1q0OqI7nmt3lcbiADX01SL0JpmLqpaPOgzRyfv7KYDw1WaBcD0+C9Y3SBGWIzG1hbhqy7xgDhwBa71JRTqbeSElpGBPRi/AhYe+bKaM4x1IT3pQEfyj3+NiNUzl4GUnUqiWjhuKAOLKdw5SHIIIocH7dPOMoo3JepXBXdtsN5PYqssmwm1XpBNHFe0HGfO5sMFhwVrVYY2H8NOFpJshDDfw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from DS7PR11MB5966.namprd11.prod.outlook.com (2603:10b6:8:71::6) by SJ2PR11MB8421.namprd11.prod.outlook.com (2603:10b6:a03:549::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6652.25; Wed, 9 Aug 2023 03:25:25 +0000 Received: from DS7PR11MB5966.namprd11.prod.outlook.com ([fe80::1b1a:af8e:7514:6f63]) by DS7PR11MB5966.namprd11.prod.outlook.com ([fe80::1b1a:af8e:7514:6f63%2]) with mapi id 15.20.6652.028; Wed, 9 Aug 2023 03:25:25 +0000 Date: Wed, 9 Aug 2023 10:58:28 +0800 From: Yan Zhao To: Jason Gunthorpe CC: Sean Christopherson , , , , , , , , , Subject: Re: [RFC PATCH 3/3] KVM: x86/mmu: skip zap maybe-dma-pinned pages for NUMA migration Message-ID: Reply-To: Yan Zhao References: <20230808071329.19995-1-yan.y.zhao@intel.com> <20230808071702.20269-1-yan.y.zhao@intel.com> Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: SG2P153CA0045.APCP153.PROD.OUTLOOK.COM (2603:1096:4:c6::14) To DS7PR11MB5966.namprd11.prod.outlook.com (2603:10b6:8:71::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR11MB5966:EE_|SJ2PR11MB8421:EE_ X-MS-Office365-Filtering-Correlation-Id: 305e6004-3ea3-49d0-c2f8-08db9888454b X-LD-Processed: 46c98d88-e344-4ed4-8496-4ed7712e255d,ExtAddr X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: rpos7kCngd1FbdTPagJYfTCyWLnMOiXbvjfttxWqD2Qe0wMlTUVjIWcSe9nhGQ1N9oAyTVoCqpjx0gpcGMxiBFC2JPr1pIytkA7p05Xv+K7nlfYLmIewxrXn79ghdgAVlIwDI8ys78Qx26LHHMeXwGch9SLvaZdU/3vXgoXOEaoQwQMPLfxO1cuAKv6eMI0JSRle8BUHoUu6qeKDbVQXhjn3S4LFgNAYSUKec1l5RR6NGfT7H6uFTPk8QbnbEKcXQHjsKQ4fQQSX/6r0EfeHIHtxqHala/gSg/7GjZoNN4VnJhCMYnI4OlQXAceLKVCFWA5SozQAUQniY40ahSPClx3czUcfpCWCWMMEdyiG9QxpBXV32vsEagZ4tPWqieL9MNdoE6OWEGI90gOTt/arXUxLuQEfHuweflQKUNh9gaSQWESqLm8kJZZQw/9Oxuxo0GTGaBmscrISqbiDqswpZGt7OUcFofdZn8A5Ylt3gbDuXL5xq46q4LZQA3xBY0MviS5FAMNrLQ+IxVZL50CQHDjE0NxNl3/VcumKqY0YNvT+jQtHYr6Ej6bVK6pk+BZq X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR11MB5966.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(396003)(346002)(39860400002)(366004)(376002)(136003)(451199021)(1800799006)(186006)(86362001)(6486002)(26005)(82960400001)(6512007)(478600001)(107886003)(6506007)(54906003)(41300700001)(6916009)(3450700001)(5660300002)(316002)(4326008)(8676002)(66556008)(38100700002)(66476007)(66946007)(2906002)(8936002)(7416002)(4744005)(83380400001);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?DkDhLK+TpT5xwAXpcrvter1TAbBlULvs98rSY1ZRPJQlxC0pKkoYsmlPfC7U?= =?us-ascii?Q?xMujRQFrculbNJeoEM23EJRpP0CTWmWlb/YnpC/zcFggd7PUuUlNLy9YKOZJ?= =?us-ascii?Q?r2otGJIcsf5+j8ynK7sno5wByU22eddknC+LMQfIqfqM2AWox+0VkqmJTzJj?= =?us-ascii?Q?vrxNJwIL6jZc0sCEmVcbPBxcBu0I3IaSlsoOSj048CyNYnLc7WLm1e4D/7iR?= =?us-ascii?Q?V7xEcCv/mGb9MiFmNk01Pipo+qdYL8XDNJ8D2ZuDqUpr9SKwr5C81+4kNYI9?= =?us-ascii?Q?6UAd0acXoDYoOYEcEbBxS1vvZmakzRumvLwmiEHkMpFcbmKjTVEnU+sXtgty?= =?us-ascii?Q?xGtWgb+aU++blHjD4BHe+/+qfChWCPd7xfi6a0OXepnG67qfUtZkCpQFSRkc?= =?us-ascii?Q?OE1QBp5vPzkEYWzXUuh13EItgDtgfd/NJGOkbWW/L7mm114tMPQWvweP6zO8?= =?us-ascii?Q?m6QJ5NLGyCsrBZB2WFgqxqs+5Ham3xmKwX6t0JvdhQYUVzsQA7m9+y3p8++E?= =?us-ascii?Q?UOMw8ijytyXG3ssqo0i9nFLNQINeK1d+BlKTVhKru10mdWRHifk7oT3TMpiJ?= =?us-ascii?Q?SUYmaUpUoFXVDHVh7iDoAsp1Z6PkhpQKOOvXxjOMzNKBWFwxPuEi84ZeDrWi?= =?us-ascii?Q?yinzubV3bhJ6+1PC0Br8x6L12QhjJMcmJafP57KHBT/stK7pGWkhIM0j7ILq?= =?us-ascii?Q?CJVfUOT13ps/Qg4FuM2O9sCx5nKmeV+1qypLZlxOiQjhdhiLjxdSoL/LFYZJ?= =?us-ascii?Q?E1rnz5LzS5vt8jpPIH7aOvBxrzeW9+lWVsd2PI3yC8dKdmONcrSLX/pVgyrN?= =?us-ascii?Q?ZT0ZDqMecx5faSC6Yo/us7FYc4yfD/coTXpVwmxJGmAQHQIgzBE6FHibpCoJ?= =?us-ascii?Q?EQ8N/be6N2T8iUYSuFVgJ99vbfAV+4FP1Kll86k3XkKbYBG+7J7Qfh+0AbAA?= =?us-ascii?Q?ACDZNMAvqpWmsU31JVJovbD2iqIs6/nClO8NvHk3UOGbDnOjASiuUJc2xrqa?= =?us-ascii?Q?NdkNRLdz5ibbBnYTw42CsGbHCbRRVA1rWcOhLReSqiS22H/wGyb/8UesmTS7?= =?us-ascii?Q?vVZzKPpno5sanG0QelxArhUTjj5BzemKHwa/PTqznDcS3vePNCb/3bNl9Zwn?= =?us-ascii?Q?yYUUb5C1xD3iZUxDRXn1RO406Gan/hFgnFGRNnzwi/DvtcFl0KZeHgbAZpJf?= =?us-ascii?Q?OSURFMmcAqPai0ajLVci7wP0Fr4MS4Ecxs9+LCmjH5rlJKUAduefPz72snul?= =?us-ascii?Q?yZHHdS+rgu8SPHk2TG/W03JtmEV+V+4PoLxqn50c/VzhhE9YRRDtccvHeSyE?= =?us-ascii?Q?xYgrFiwQIt3daNZlczeggapJfV5bTD7mtd3nH1Sw+gudcxrUySD6o7f8NjbS?= =?us-ascii?Q?eaAT6YiY47YyIgYqiGb1PdOm7bLZTrIhmSxJU7YYC5+pGaXOSz7vkTqT6sEd?= =?us-ascii?Q?uiHTaVyCoCALzCkzp0sQIoZYUNQITTEWGDoThdHwNwv3Svtbsk3ZR5jQUIv3?= =?us-ascii?Q?ep1ZK+vLdPdcKFyGu15AFyTbBFh46dbP5FzHN0B0rbBWuiTljV0sla0D6pYk?= =?us-ascii?Q?D1IAofxOXM0H1wgPeDOfKpCMtJ7139Eqhyp59RSg?= X-MS-Exchange-CrossTenant-Network-Message-Id: 305e6004-3ea3-49d0-c2f8-08db9888454b X-MS-Exchange-CrossTenant-AuthSource: DS7PR11MB5966.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Aug 2023 03:25:25.0064 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 1NHba8+boVlA7FRtSwq9Lih4TB+hQ+JQNa/6HHOXb0uI4m9qmXENi5pJbC72JZtdV1IxZ+ixyM9RIqn2LSLYQw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR11MB8421 X-OriginatorOrg: intel.com X-Rspamd-Queue-Id: 55CA9C0017 X-Rspam-User: X-Stat-Signature: ehu97ojzco8nfn8ik1dr98k3649gnbtr X-Rspamd-Server: rspam03 X-HE-Tag: 1691551530-826782 X-HE-Meta: U2FsdGVkX1/UIAF9gxiCwKCgeLAao4HA8RaBVoYhQA+Q8FMDZx5skqd4zqZIzoGb/Ne3ua1oWD6NQ8zY+yJXdNRKCsFuftti8im+fxV6GDRUueCUIJ4Tf/3caWYokfMroe3JgUZg3iWk3b+pKTv9eD0UdZ2hKb7l7p0e1GcsPK1UPSYjZFKKOasHq47vM2xQznFKxp9/A4FlHmOe5lPI3GaDgWPLVLdpbUnAtZwQBKjGME+Up8CLzTBmY6dgxab+1KFLf6f0yap2nnzkNGNPqKmNZ4ek8atksM3m0ID+myONql8lyLae5dv+7iFZ77jkV9IO53rOwpRpyQJKtplKLFN50fDO704hOAlpceeRfzFztnlC7/IDgoIznxitAEwQmSnbfMRM7AdpXsW4JkTKjrxrYJGgYBNgjlJG1Ui66ufbcpx7NnMcN/+sqHHpUeBLEIOjclKXtnONGKOGvO9V9Fu2ymkRaBbyEpqHz1mZfTNYtQtTiauO1Maa3+R85SpynfvRybFomRbhRI89ms2spMRUBDZ1Ezp+sqVKpIDnYVXxnjXkJCKi26AXe8ACpIAUAwAAxwOPb/8o7UGjzOBBqg7oDA8j0Mvk7zJ4RzopDdwselNQDNL8CX/rIi6V7dhATfnbrZCHDlYdDvkfsOVTEQaYtIF+RNxzy+a+Hz/A07Oxe7B17miLayLNCLf08PKzvkutPL57jPNNe4v7V1O32UKuR2PK3lnY5P7+OjmUH7S377YqzdngVpKE7wFxd8lE86UjR0nZL0B78Kf7jDQOuvZ5ofQVpSlowfOY2rtrl3/p+ykcWrp5T9XYGFgFE/iIgWzkrgLq9rsnFjBlJdLwV36SbJReaXkWBU9eTHoVSsGa39ZTi5a6a1MgXCNozgt4PAyC7OCZvou71/oQ8hqodckfeoi/JnhZZ8f9UdpzMI5I1voyCrr/X+j3WiVtZlSQiZiIs9n9x90KWZotgCD ClCpWcar seS7mLoFYgi5EC+3v8+aFz8/4R2IKLzw4e3e7pxmpM4ZhA3RJwIgXm+NkGWNsW2lXw95GGdY8JUt9mI3nJk47JL2/l/V+J4gL8rTrBOgLwvCV72uLWHUwFzzP/zr4UV4H0SGrsMwNZMNT4xDR9COXOoP+aMjfN6Gy9zQf29wlNVD9GRU2W7MscMTO77u2WisMy19hIKEf9WFKUA3sjM3B6COAQAcwsSuBZWhWAQlsWHYuDJgrv6ttYuobkURswPIbfEZSZP+lQgPS5M/HBORRUZmNl3Ult/1ft4KExiGjEqH9rkstW36frpZMZx9GleJ4e/a+oArKsVb+RRZ4TB0XEqAlGGWGKIwJxz6VKyhdgzLuiIdQ4cOsWq68mg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Aug 08, 2023 at 11:32:37AM -0300, Jason Gunthorpe wrote: .... > > This really needs to be fixed in the primary MMU and not require any direct > > involvement from secondary MMUs, e.g. the mmu_notifier invalidation itself needs > > to be skipped. > > This likely has the same issue you just described, we don't know if it > can be skipped until we iterate over the PTEs and by then it is too > late to invoke the notifier. Maybe some kind of abort and restart The problem is that KVM currently performs the zap in handler of .invalidate_range_start(), so before abort in mm, KVM has done the zap in secondary MMU. Or, could we move the zap in KVM side to handler of .invalidate_range_end() only for MMU_NOTIFY_PROTECTION_VMA and MMU_NOTIFIER_RANGE_NUMA? Then, in mm side, we could do the abort and update the range to contain only successful subrange .invalidate_range_end(). Is that acceptable? > scheme could work? >