From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE0E1C636D7 for ; Thu, 23 Feb 2023 09:12:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0317F6B0072; Thu, 23 Feb 2023 04:12:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F23DD6B0073; Thu, 23 Feb 2023 04:11:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DC4326B0074; Thu, 23 Feb 2023 04:11:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id CCE626B0072 for ; Thu, 23 Feb 2023 04:11:59 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 9A7281A0BEB for ; Thu, 23 Feb 2023 09:11:59 +0000 (UTC) X-FDA: 80497989558.10.F00EF3E Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2084.outbound.protection.outlook.com [40.107.94.84]) by imf22.hostedemail.com (Postfix) with ESMTP id A3D01C0015 for ; Thu, 23 Feb 2023 09:11:56 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=japUusMy; spf=pass (imf22.hostedemail.com: domain of Pankaj.Gupta@amd.com designates 40.107.94.84 as permitted sender) smtp.mailfrom=Pankaj.Gupta@amd.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=quarantine) header.from=amd.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1677143516; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BuDM0JiRTxxHqEXlLSGJsthlAO7dpgmYMAq21v6G4EQ=; b=FrWRhFnBqFf9u1sgf6fghp/7tInhyTevoMB6lM4dkvhdugGVopYtGjhgG3/VVFDUIdWIVN l4Wt+XUDsTHXDJ6KXVozQJl/boYJwVYNUM7CcYPWjthBEk6I1TEsGxPUZHXOppuoCqKtaC qWZ40pire8wJwkYqPC4mCbINooagAxE= ARC-Authentication-Results: i=2; imf22.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=japUusMy; spf=pass (imf22.hostedemail.com: domain of Pankaj.Gupta@amd.com designates 40.107.94.84 as permitted sender) smtp.mailfrom=Pankaj.Gupta@amd.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=quarantine) header.from=amd.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1677143516; a=rsa-sha256; cv=pass; b=SMNYEjf36gua2kLpN+kHgNn5ThTBLE/WmHLcqNnizac0cl/W/c2Ff2MltIj8gm3gPTh4oc zIFERQAfKyPRXO1SBvL3KBo8dYROLZ6D2CPgePSVek18tnk7uf/NqrIieJ5tbVye+ozZgH dsB3fPCmmsuEBFc0n6GHXsrGWJgEYKM= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=J0Ccdnh//qS4rno5hSn7u8BWZmUmm7qJRnYq/bAJ5SOseFIEV1Gr5YYpYnZDwEudiqG7nGeA5RWZz2gOzaLFDNOEue7AD+dH/DWfSjJ78vvldeIinKDRB6U9vV4Yty/ivTvhlz8TMeuqjxQfaK/zpCD7X0NimiOmuBiwhjLvz2G6PnXuw4IH/4Zl/4qxSF6C1lCtv8Pv66LIQsRv6+dg+dQMxsBxnhQl0MkNvkfnbWxeYmgnN5Sbb2xRn+q6GS0WWE9CtvoojsHka6Bqq7erd7LGJXRbhIhoECJehO0QGognDVCfFB7LCuZOvYFFUKkagdT/FoKvBwp/X4YUIWm6Bw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BuDM0JiRTxxHqEXlLSGJsthlAO7dpgmYMAq21v6G4EQ=; b=gmZ8SJxY5dr+ZVF7hthBf/aS0JnQrjMPxzuG7hcOEny4f612g1q1ejOCk3OEAH0DIRkaKfm7LHiG7ShximFH4L/KkbKwjcOCfjIxP0jrruW5IW63aJm49CEI38Aty3+0Ng6EYAJEyDp35Z4VFSH2VplWo0nI8kc5cydQugIqrHFhS+97zpNQUPvaA/X7C2I5ORct++0GbXNEwvhK58NvKKUHE+PAWLS920vCwjQIq+7kbtQMPmSZlfjUfct/Y2rgkU19IHtpc0kqeIcHDR0Gm5XWfYgsCvojlPVa1kEmvHLQ0Rf4Hto7ggF9AjeAkObEZJiYTLzTkBixTh/Y1+DAlw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BuDM0JiRTxxHqEXlLSGJsthlAO7dpgmYMAq21v6G4EQ=; b=japUusMyZc7Xye10L3nnRYi/t/NjvTNXyWRUMIfOe7eK/3GDt04H9rf2ipGuXQOnHmxfpY1neXLErifYw1Zc+0qQ32aek2IyaV+iWzeA6x0Pk6e/49CHnQwRVSDtrFWj9q9UDLdGFFe0PoiqmDYXyLAJrTtSuwRNYiuolcUr7Kw= Received: from DM6PR12MB2810.namprd12.prod.outlook.com (2603:10b6:5:41::21) by SJ2PR12MB7822.namprd12.prod.outlook.com (2603:10b6:a03:4ca::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6111.21; Thu, 23 Feb 2023 09:11:53 +0000 Received: from DM6PR12MB2810.namprd12.prod.outlook.com ([fe80::bad5:8f56:fc07:15cf]) by DM6PR12MB2810.namprd12.prod.outlook.com ([fe80::bad5:8f56:fc07:15cf%3]) with mapi id 15.20.6134.019; Thu, 23 Feb 2023 09:11:53 +0000 Message-ID: <5c6ae46f-63e8-7582-01d2-edac1dd45bc9@amd.com> Date: Thu, 23 Feb 2023 10:11:46 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.6.0 Subject: Re: [LSF/MM/BPF TOPIC] Virtual Machine Memory Passthrough Content-Language: en-US To: Pasha Tatashin Cc: lsf-pc@lists.linux-foundation.org, linux-mm , David Hildenbrand References: <6070e228-17f1-8495-470d-80dd38963266@amd.com> From: "Gupta, Pankaj" In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-ClientProxiedBy: FR0P281CA0011.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:15::16) To DM6PR12MB2810.namprd12.prod.outlook.com (2603:10b6:5:41::21) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB2810:EE_|SJ2PR12MB7822:EE_ X-MS-Office365-Filtering-Correlation-Id: 805dceae-1002-4c07-75df-08db157e00f8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: HSthJqRksfDKBt7qxmANzblkuB5y2w56Ku/HuhfOXYgpvdnx4RYrtSOnwnC9jhOUd086fb0Ki5oSZH00rvSxNc3qqo4sqWR36dscFH1VXbtU0dHaQdSuqTNd0pMwfPsjKVw0ey5ZsVseRFpUVYusMGEGuaMJAe9PsmnGRpPZpCQjUb2d/KUVhGYy0VmML7tGrtfZo5HqwmsJs6clzcs1TrTW6dEpZn9QpfJe2hurvmbVMePzP6MnrKyprumxJqlrhAqv/CvYpoVJi0vlFBS1A/NukeWj+wi4W6jshkHNwAnjghNi5RkIlTkJLKle8FD+x6E4RWQALZlqLm79b6nngzbl6I36tCHhMEnIGW789dW3DEU9vbOz0KMuDfR4+U/9vfuBelT+QSe3YG4dnWvFo9in+Nfxj+ppb99xSY/nDamGYpa3mCgRRNQ6FqGEn3t0rtLlMFmdEsyQ92feLbKuNBLd9hknKG6+R/NGDsRiFuXPw9RaL8Pl701CJR8Y3Xfs6OfML/UUMmpibIyXbrAoP7shNfPgQpQBYbv+YaNqALkSAGxWAsu28XivPWOd6lPSpJRd8RCc14d6sN3dgbjS0kQxkV35EPg3Cb9jd3yO72lZxYz55sg5SNQq3CnDCO6KflZoJT9UWRVXSaKkdCDXHtFOeMYMiqBRpUvhu2WEutkr91TjCqJIvO7YqASTkDo4+2fSUTsdGiJPusWuX7X7WI3I6GTZIhNlI3uCf0o3+MY= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB2810.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(396003)(366004)(346002)(39860400002)(136003)(376002)(451199018)(31686004)(316002)(54906003)(36756003)(38100700002)(8676002)(66556008)(66946007)(66476007)(66899018)(2616005)(186003)(4326008)(6666004)(26005)(6506007)(478600001)(6486002)(6916009)(5660300002)(41300700001)(2906002)(31696002)(8936002)(86362001)(6512007)(43740500002)(45980500001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?V1hXWG5kVUdhYStoRTRuNkJmS20xeWk2VXIvekRSNFZBdnRtYW02ZGpZSFlY?= =?utf-8?B?aTM5a1pjMGY4dDhVSzZ6ODFNYWxLeTNpM01PMkViMWJOWm1pS1U1em5FVElj?= =?utf-8?B?MkpiRStKRm4zZGFzbFpvTnlzNTBFdzBZcGxxREIyTkRZMmNkcGhsY0ZXd2lI?= =?utf-8?B?Q1B5UlBGcDNxZDIrZWZlbGl5N1ZUY2FNNUF2eDF4ZE1HTEVTWVVCSHFjS3JW?= =?utf-8?B?RmFFY1dHckVKR2JUblk3dEZwb2FTdVRPSDRJdHdweUFURmI3cExlMEFXcGFJ?= =?utf-8?B?YSs4SzBLeXVYbXR2c0JwTzc0ZG85WUlkOGJFcVRDV243dkwyRlZpaGZ4aktR?= =?utf-8?B?M1R4TExubHdNNWhqb0dYcW4vSXA2YU9FRldBcmsrODUwZEFnK3RGdWFRYjhq?= =?utf-8?B?Nk1hYWpLcWF1WGk2ejhFM3A0WHF0SW9jd2lZZGgvSUxPNE5SMnAydDRneTBl?= =?utf-8?B?R3MvUmJLQzM3blJJT3F4UUNOUEFwcUczWlJRTmxQeXNmMWdjZVJTVllLS1Z6?= =?utf-8?B?dnlxRXZsN2JHejlLcll1WEtIR2o2R29NTWFmdWo4Rk9MUlNhMlV2cHRDMXdv?= =?utf-8?B?S0V3UG1jeUJZalNYNm4wU0VTNmhCb3hudVF1M1cxZUpibzFzVUtFSER5S0xW?= =?utf-8?B?dEl4RWtOdkx4RmhodkpGakFHZGI4UXgxeEhlQThmais3SkU1S2RDcFJFTzFC?= =?utf-8?B?RDZsSnMyWnZ1elV3cUtrUGFJMXhweTdEdSt5WXhaNFNBSDV0TTFpa2MzNWlS?= =?utf-8?B?Wkp0Zm5qVithVTFBTkRDS3htd0tCRTJUcXl1ZDRUN0JSZkorczVEV0dZS0Ir?= =?utf-8?B?YmIwVkx0QjUveW1CRDB5QVFhZGxDaVlNOXZyeEdsbm1yNHF5SXEyYTFqNitU?= =?utf-8?B?b0k1bGc4bW5wMlR3L2t4cm5HdDBjQ1BaK1I0MFZKcitjc2Y1S0duZHBEVnVU?= =?utf-8?B?M0plV2JNbDZCa2Y5THQveU4vOFQ1ZHR3SmtEVkNQQmhZUUVOdG96T29ML0Vr?= =?utf-8?B?TVdwTjRST1lMSGp5UjBsYkRLbVpMSForUWRBNHp5MXVhcThaVGNQRUEwWjRG?= =?utf-8?B?ZmdFbE5saGdPc2NOcm45UDRVMUFNOUFXMkVONHRmQjFxa3JuSzNkSkJxd0Rs?= =?utf-8?B?T3gyekRvSTNtUlI4QzZHeW5EVGxuMmo2S0I4WnlJM2VXbXlJaE0rVXZSUVRM?= =?utf-8?B?YjNuTmNVSlhPSVJZZXl6bnV6czJKeFhrZU9raDNTM0FYSlVQenJjWWNsWVhU?= =?utf-8?B?TkZLNkwwaGZSc0VSM2ZHcFZieHdLaWd0citjbERVZUlPdTJRcUpSWjBkcklz?= =?utf-8?B?QnhzSHRKcEFwODlxcjlodi9JT000cFh5ZWFPM2p1TnZhenFqdUN3ZUp3RTFm?= =?utf-8?B?eWZ4c2RyMUVCTzIzVVpBb2F3NXAvell3KzNJU0FzendZVCtETThld0UrWmgy?= =?utf-8?B?dU5MRG5PNnhqN0tvRVVMY1JPYWdJdXZ1Q2FZY2pSbGJTTis5aXdHUklhSjFh?= =?utf-8?B?QnZhdmIwWnplbytaRk50NFBhL0xRSzBXOUpBZmxFUGlpekU0WUlZaHFYUWxV?= =?utf-8?B?L294a2JFNXN3SnBjTlJuOE01MGx2THRUMENvTzNPbGtkMmFKV0c0NEV1S2xB?= =?utf-8?B?SXlQYzhBMGxBM0FqV3N5MXRjcERNUGJXQkFqNUZhTTIxcGk2Y0twcHp0bHVL?= =?utf-8?B?TFIrNGphU1NLc2tBYmZCMU50ZzJVVHdWYndwRjVFanBwNVJBdHUwQlZoRWpZ?= =?utf-8?B?QWJKM1p5OUUvVENaZUIzNEFTUmxpVENlUEUxSVNNVURiY0Jud1lqQWlYa2h4?= =?utf-8?B?VDQ2b1lsTEh2NlcwNkNIUkRGLzU4Tk02YkdVYzUyUW9Qc0JUcGdPcVpSemp5?= =?utf-8?B?aWhTOTJxbFdoeWFlVmphYlVQOTdsaE14bU1JMERkc0xySW5TVkVRd3JpeXQx?= =?utf-8?B?RGptTmhxbTRVQzhiK2s0VHRXcjA4ZVJja1lhTWVibTlmSWg2aEExSVhTYU9h?= =?utf-8?B?bzJRcG5DZVh0NXY5T1lZYzhtMzk1ODJPV2pDK0xTQkJ1WSt5UEJNOEpQUWYz?= =?utf-8?B?MlhSWSs2Rk95S2I1WVlLNmdOM3NaZHFNMDlwVGRDbm1OZG9rTG9hTHp3b203?= =?utf-8?Q?3HFh4a4kRbggN4lKswLAyg5v1?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 805dceae-1002-4c07-75df-08db157e00f8 X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB2810.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Feb 2023 09:11:53.1590 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 45YW7FHttzeaqHAtdTgZqafFXDqfV4L1+EHqFUykfTWa3puv+y6MDUpbndb8RqM2LraKzG775uHLJJYDflRMZQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7822 X-Rspamd-Queue-Id: A3D01C0015 X-Stat-Signature: m6zzk3hsfn59k9gd4jxpzjjccnr9iri4 X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1677143516-466707 X-HE-Meta: U2FsdGVkX18Gi2kAO71mLbhPYS3VLk2Muqkyxmv1asCP5G++oonxLtKsO5PdmHgFat5hzfakQ7w8VAE9G89ORJ9tK5FqE3Ljwqkyi8W9rAxqcWqjcM2qBUYJ+4q47Vz6tLIQqr9Y+BNFohY0qmPFePZab96qeVOumy2fVKkzJOf86JK+/kXjiFKhdmSzaWLKF7uY0OB4j/n+zaugzCfh67N/HrIiU2RdG0H1iowhnHboYzqDspGEYBvJERxr2hxVnv3z5v9cxzuflNE5alf3crrbXIBpUdRToiyWIwZNPXeWzYBMG+brA9BCYYYuYlXu62tOGNJXjOAY/EUwNfqM61v4vSqxM0Eejt/C2COnmRi/q8x1Dw+Ff968+dHWdFiHfxkgbQgbGbHobfE8k0Ezyd2OCeKVSDkLbiHxJQ4bEurEDlIwmVj974Be3xz7TFDH63Lft9AVRxB8HQaErhgNmY+wQrfXeB0sss+p8581LxmDfQVL/wXt7U/M5NdWLp1Xt/7rBz4zVGwe3uf0+bK2Lfy2Px16S5Wo2cToP0PZyh7KUk9VGykYTYzKOvUW0Bhc3SdT99iOIK7SnI0ym/zt/i6TDb6AVro8PtDPc1RZkyVJ5QW3eGyv0ijKohiPvhP4bamODVWINQaOJAgpYZ/keLG7g/khNS+mivmzpPV0jydMD6rGI6NhQoVV3H3dcHdARXcREayNVH4amWDAEYD1KPjssie/T+G5syL60UAN0EE5FC5Co1U6iaKiqnstMligaTAbLrIQ/w7CJrvPLgmOQsK22bufVHYQ8n3IbMHScxiCFH5aGqxEnuLkB6AClEQIdewHa0SzLhKlmjEvsi8p9oY7ndPJne3N16D1y8L2xtMbmJZd7+/YHSsFbXHahbTHF6NECk2jrxAvAklRp5BeHJFXvIiJDF5VI4dDCtg9Dl84uXS5cYh7d5LmO/UoC06WRN34KVCi3hnxI+Zy+tP H0Aqn+7U 4zyChHTdA/wlId4BD7WcLppG1P6sJJZa+KnN7xiHHWWtWSfMp8STnI8wkERmlCxjmjHGxFW+iR0yAJi+JiiczEBOT3iN5wODBXNPQRhn9Fif3HOOX3udByniPb2M61kYhZ/kAA/tGwZ0JUXaZYM5ednJMhznCeZVmJXwe/FydiVSr1u+512W4omktownM36dM9gGNMFJJQ/npDK1bxB/yp2y8TOPqutiytyEs X-Bogosity: Ham, tests=bogofilter, spamicity=0.000482, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: >> Hi Pasha, >> >>>> Coming from the virtio-pmem and some free page hinting background, I am >>>> interested in this discussion. I saw your proposal about single owner >>>> memory driver in other thread and could not entirely link the dots about >>>> applicability of the idea with "reducing the memory footprint overhead >>>> for virtual machines". Do we plan to co-ordinate guest memory state with >>>> corresponding host state for efficient memory reclaim decisions? >>>> Or something entirely different we are targeting here? >>> >>> Hi Pankaj, >>> >>> The plan is to have a driver /dev/memctl and corresponding VMM agent >>> that synchronously passes information about how guest would like its >>> memory to be backed on the host. >>> >>> For example the following information can come from guest for a range >>> of physical addresses: >>> MADV_NOHUGEPAGE >>> MADV_HUGEPAGE >>> MADV_DONTNEED >>> PR_SET_VMA_ANON_NAME >>> etc. >>> >>> All together this should help by doing memory management operations >>> only on the host side, and reduce the number of operations that are >>> performed on the guest. >> >> o.k. That sounds like guest will have a *special* interface (paravirt?) >> for some of the memory management operations with the coordination of host. > > That is correct, hence memory passthrough. ya. > >> >> Guest would still allow other regular memory operations? which would get >> full-filled by the guest? Just wondering if this solution only be >> useful for specific workloads which are aware of known MADV calls? > > Depending on the flexibility of the interface, we are currently > working supporting tcmalloc(), and also mmap(MAP_ANONYMOUS), but in > the future can be extended to more types of memory. Not sure if its worth to extend the existing paravirt memory management interfaces like virtio-mem or virtio-balloon or create a new driver altogether? Adding David (in Cc) for his thoughts. Thanks, Pankaj > >> And might not do/require continuous allocation/deallocation of memory? > > Contiguous memory allocation on the host is not required. > > Thanks, > Pasha