From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B24B5C3ABD8 for ; Fri, 16 May 2025 17:51:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 164F16B024C; Fri, 16 May 2025 13:51:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 115476B024E; Fri, 16 May 2025 13:51:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F1F166B024F; Fri, 16 May 2025 13:51:54 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D3B706B024C for ; Fri, 16 May 2025 13:51:54 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id A99BDBE848 for ; Fri, 16 May 2025 17:51:55 +0000 (UTC) X-FDA: 83449514190.24.94CB0EB Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) by imf07.hostedemail.com (Postfix) with ESMTP id E2AE64000F for ; Fri, 16 May 2025 17:51:53 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=mrtBUdgB; spf=pass (imf07.hostedemail.com: domain of 3OHsnaAYKCBUDzv84x19916z.x97638FI-775Gvx5.9C1@flex--seanjc.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=3OHsnaAYKCBUDzv84x19916z.x97638FI-775Gvx5.9C1@flex--seanjc.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1747417914; a=rsa-sha256; cv=none; b=TJSsEVLieQJ0JRC9DqTSBGniqQ23CS0nx8R4gsS2NUbFqtLSjJe5VI66A/M/CbgJd6UArq nwRJk2bhn2fEQbspplqWrgzp1xX/IYy1N7ZQsiYpRrCFcaVx1cAnOR6asFgyWUnvxkrCQt MSl9NK6w5EPf4cAsrdEzP21s1x4kHQo= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=mrtBUdgB; spf=pass (imf07.hostedemail.com: domain of 3OHsnaAYKCBUDzv84x19916z.x97638FI-775Gvx5.9C1@flex--seanjc.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=3OHsnaAYKCBUDzv84x19916z.x97638FI-775Gvx5.9C1@flex--seanjc.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1747417914; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=V5HQp6kq+X9VHvMiTR+cwYmP5qs8g8pvYOQWqNMSHLc=; b=37M26mQkGRAN3UrqtC7i0rC1f8woZhTBjK/FENov34W4HJpPxmjuQn58OPNMngymvKFbFB LGIA67g82KGo4a4bsoH+uryZGKRUHnQFuOCbfudJGOvLauiNIhyOLqgtUAcZYnzVa0SKJ3 qWWq0gJRbKGyuBFXX44H/+W9rzS3/dc= Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-30c50f130d9so2110378a91.1 for ; Fri, 16 May 2025 10:51:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1747417913; x=1748022713; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=V5HQp6kq+X9VHvMiTR+cwYmP5qs8g8pvYOQWqNMSHLc=; b=mrtBUdgByxbatdAIw8Y4PbxAbeYphd5oJPPJmKBfX/C73tp/peyozBFeFN+J8sshk6 ipDoPmLks50ZLn4YTdmAhyk4woAVksuLu6um8FnzClf6D9W2hTU0qWIXXZTa2RoxoG5J quUc6FTmQwMJlv7Z5IIZtMjwUMXsfPxtMp35UfRy7o5iAFWFzm/nwPsh7m2iqYb6bG5X 02hLgSdjiXz/Y/lcLUkoiAIflre6jnz1mjhj3n38ilK3hno7OadhlvW94bzST/Z72Jm/ MjcQKbQrXAAL5as4PEl0pJV/yVkJGddHtmO7Vv4wTHUvRaMaMY1eZcRftXcJqvQhcFzb lCEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1747417913; x=1748022713; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=V5HQp6kq+X9VHvMiTR+cwYmP5qs8g8pvYOQWqNMSHLc=; b=mVzMcBUJO0JXWeGmm3iKWVXnImKsg3bo6zKhwppNJsCfn/ZgI64KrVptWP2Aq0JiLp AqrM2fYM42/kq0gH3dqxA5yaCTDyriM0CdOG7NVw+g/jpG8B+3pn4ZHqs4zac5yzimqU kGvFC5DFEeuQxZrA7gOucbe8BkjZHKCVnP6Lby3cutVNGc3Zt06Pp2MAZDtGKY3mqn5b VZUJiboebOGgE/LasUQjCvXqSrPrVVuKmHewsPLUfpo0dQLiiVWVq2ZmdKBSjO+JhAOc DNduH/8WBykitJFJdkOcU0u2qcBivyWeo1HV+xZ875qBuDgUGnNHC1rYEjkei8TsDAPu 841w== X-Forwarded-Encrypted: i=1; AJvYcCUA/FMNRKQZdd+5TtAZYl8nOP60P0IvZbtDKtgdFtvup6FxI/3MxuZvlKvF88L00gkImuRX/NERwQ==@kvack.org X-Gm-Message-State: AOJu0YwL71RVGFod/ebQBWAiSZH2dzyMhkkDKwV9cbPrxKR40R5Cvy8x ciIWwhISIscZzNLLI8HpE6kPmjNhGyL0SMNfTjvOE45QPkgRtrdyRDHCOBk4ch+6hlrlq2lKMqy YNFnO7Q== X-Google-Smtp-Source: AGHT+IFwtvvhEW33lcn8I0JC2VzVOUh9GPDEpoRuyJJmzbZM79EJn+UNnUU3HQ0UiE4FGqM+5/rqg3sdCXQ= X-Received: from pjyd7.prod.google.com ([2002:a17:90a:dfc7:b0:2ff:5752:a78f]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:2b45:b0:30e:823f:ef21 with SMTP id 98e67ed59e1d1-30e83228e05mr4391414a91.32.1747417912343; Fri, 16 May 2025 10:51:52 -0700 (PDT) Date: Fri, 16 May 2025 10:51:50 -0700 In-Reply-To: Mime-Version: 1.0 References: <24e8ae7483d0fada8d5042f9cd5598573ca8f1c5.camel@intel.com> <7d3b391f3a31396bd9abe641259392fd94b5e72f.camel@intel.com> Message-ID: Subject: Re: [RFC PATCH v2 00/51] 1G page support for guest_memfd From: Sean Christopherson To: Rick P Edgecombe Cc: Vishal Annapurve , "pvorel@suse.cz" , "kvm@vger.kernel.org" , "catalin.marinas@arm.com" , Jun Miao , "palmer@dabbelt.com" , "pdurrant@amazon.co.uk" , "vbabka@suse.cz" , "peterx@redhat.com" , "x86@kernel.org" , "amoorthy@google.com" , "jack@suse.cz" , "maz@kernel.org" , "tabba@google.com" , "vkuznets@redhat.com" , "quic_svaddagi@quicinc.com" , "mail@maciej.szmigiero.name" , "hughd@google.com" , "quic_eberman@quicinc.com" , Wei W Wang , "keirf@google.com" , Maciej Wieczor-Retman , Yan Y Zhao , Dave Hansen , "ajones@ventanamicro.com" , "rppt@kernel.org" , "quic_mnalajal@quicinc.com" , "aik@amd.com" , "usama.arif@bytedance.com" , "fvdl@google.com" , "paul.walmsley@sifive.com" , "quic_cvanscha@quicinc.com" , "nsaenz@amazon.es" , "willy@infradead.org" , Fan Du , "anthony.yznaga@oracle.com" , "linux-kernel@vger.kernel.org" , "thomas.lendacky@amd.com" , "mic@digikod.net" , "oliver.upton@linux.dev" , Kirill Shutemov , "akpm@linux-foundation.org" , "steven.price@arm.com" , "binbin.wu@linux.intel.com" , "muchun.song@linux.dev" , Zhiquan1 Li , "rientjes@google.com" , "mpe@ellerman.id.au" , Erdem Aktas , "david@redhat.com" , "jgg@ziepe.ca" , "bfoster@redhat.com" , "jhubbard@nvidia.com" , Haibo1 Xu , "anup@brainfault.org" , Isaku Yamahata , "jthoughton@google.com" , "will@kernel.org" , "steven.sistare@oracle.com" , "quic_pheragu@quicinc.com" , "jarkko@kernel.org" , "chenhuacai@kernel.org" , Kai Huang , "shuah@kernel.org" , "dwmw@amazon.co.uk" , "pankaj.gupta@amd.com" , Chao P Peng , "nikunj@amd.com" , Alexander Graf , "viro@zeniv.linux.org.uk" , "pbonzini@redhat.com" , "yuzenghui@huawei.com" , "jroedel@suse.de" , "suzuki.poulose@arm.com" , "jgowans@amazon.com" , Yilun Xu , "liam.merwick@oracle.com" , "michael.roth@amd.com" , "quic_tsoni@quicinc.com" , "richard.weiyang@gmail.com" , Ira Weiny , "aou@eecs.berkeley.edu" , Xiaoyao Li , "qperret@google.com" , "kent.overstreet@linux.dev" , "dmatlack@google.com" , "james.morse@arm.com" , "brauner@kernel.org" , "ackerleytng@google.com" , "linux-fsdevel@vger.kernel.org" , "pgonda@google.com" , "quic_pderrin@quicinc.com" , "roypat@amazon.co.uk" , "linux-mm@kvack.org" , "hch@infradead.org" Content-Type: text/plain; charset="us-ascii" X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: E2AE64000F X-Stat-Signature: bdm7ithq8gep9msk9gqfypaj3on4nqtc X-HE-Tag: 1747417913-546781 X-HE-Meta: U2FsdGVkX1/YylOEQoFFc+ibCEj4AshuElpGX62YOrfwrn2To8RUPD4rigkI2oyh5A1NV61+4RJGlkMMbSotCUoJc2mK89XhFiL6Hi4nDrf3ZoNCYhmiFCMPZTts0V3wiNyd5KJxdV7IsdPbaFWdq95/64cMzCxhJ3qu8omtXeHaBZw6t/xMXwKkVttQ3eMJoW+4+L/owrDIMyMh8kdFiZBVBVMlQn2dnAIM4c48x2aAgR9qM5JbkyB5jVMpAYOnU8NnwrtC4+k01CuZ43sgrrRYcsA95dBIJxqKWBSufWr5hX7RV0tR0aexuhd4xr6aFsFxabI7yeuiLitZU4yl5qVELJI0AzwezygpnjQiCYKkbSn+mZ5UwDZULKa7GO9HhQRm20H+TZzm7b28HP+/DlrLWxUd5MuOzBiog+CSg+abbj0+W+m72DOfn+scxspN9WCvUeiHNsz2bLAQbKRrDGrtM1ntSFM9Ck3IsYFpc7IU+wRoVY7f+MbJXunL05xXdeCQ2vucVAbWVbob7lApBgO+PKwj+V3txSxHMfCCCA4IF6GyLJORULw9R8AYCv2CQJqvl57Ql3C1J8dYv0oq/Qkre2iiRhW/tlk27mYj5m7LshSgearH3nmr/5/MpDWu+d1E9rmJcb69mfipFGEcQUK6LzDgFJ7Dp+3pCzQagqmZaI/vFSK++lE85i0FAYrVeToL2cLjdUpHZx6WLnbVs/ba97ApCEMHUgauKlRcoxVA1svCgtBGUBIthzv2H6RGSRirXTil9lQo4tMOojNYWCa9Z29+URUb3oLBuMnUHa0iWrQ8MX91sNz+QnGBmGXJSuTh6clA/wVDjbOWw1tGdPDFz+/sj3I0yer4vDtv3yKSc4UgaOHoAKNrD3NTsVAbe1J4PF/gZuHXhjut/n5rv9rdWZlGKhRPg/hGE0r/yg3f1wqmnyXUrHEjuUdSSJvPRYbuJzymam2Rb736dku LTn1NQab 0Tn50uN54OFBUezuOmHs0j+YGnVYgI8iY/i2sBwdVfje3cSKLwZy8UW9UQD+HxG5VKTvshuygFiEycxnZ+17vNDiBhUMll1/hC/h5quogZ8fAzHId1GCQRfslzn3KauEB3NJ9AR2wGuwLwDkt0RRdJqfDa6aKyLDrXX4g69hlulrxoQTSNdcPpM+0XQeVd3Pv729poFoL6VhpwSTdnUZj+DMgJPzjHeDs4QlCwfCaO4s8A/sn3TBOHDqRwli4h/qkPXhp47hXNptHccLyOrZbi6WRIFa0XCeM6VOKOgjlbbCiazWOBharbsxpSuHo1UpR5ggJAPXg4bRig0JIZI+IyO1aZafGLkBNpL3sSkGfrwiLgw3gSboSYVVdJDBbwiDBamy/fXS6xYvgv/I2oDizkxkBRciewRTVllyRGSwLvuAGrbWcPA3tzTQjJjc7LykEvfXktIkLPyGnOjHz2LIJSXYPy67mfNvKBZTbkKBTfIX6eUtk0z+Wp24v9g+uQIYIYbsf9qNkYYpMQOimmSqnegMKsOLSNcEbOQYnrs9rM9Qd49wLhwU4M1ojDustPqN1tjWvBUfIc1ACWPCUervsZqxGvDHF60KuO2vi2CExPX2WMYFSSbgOCmKsfKa2iBCi48vxtjRcNZUhvW7d++tOzbYogXjaWdy8ChQKKA4cWfFy3Kh4FSh7lgBggI9XGyqtNg2eeYsA/55DKvUUT0Og688Xa5bmJmxxsuviCm4+luxJ/DsPP3a5MDGFVw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, May 16, 2025, Rick P Edgecombe wrote: > On Fri, 2025-05-16 at 06:11 -0700, Vishal Annapurve wrote: > > Google internally uses 1G hugetlb pages to achieve high bandwidth IO, > > lower memory footprint using HVO and lower MMU/IOMMU page table memory > > footprint among other improvements. These percentages carry a > > substantial impact when working at the scale of large fleets of hosts > > each carrying significant memory capacity. > > There must have been a lot of measuring involved in that. But the numbers I was > hoping for were how much does *this* series help upstream. ... > I asked this question assuming there were some measurements for the 1GB part of > this series. It sounds like the reasoning is instead that this is how Google > does things, which is backed by way more benchmarking than kernel patches are > used to getting. So it can just be reasonable assumed to be helpful. > > But for upstream code, I'd expect there to be a bit more concrete than "we > believe" and "substantial impact". It seems like I'm in the minority here > though. So if no one else wants to pressure test the thinking in the usual way, > I guess I'll just have to wonder. >From my perspective, 1GiB hugepage support in guest_memfd isn't about improving CoCo performance, it's about achieving feature parity on guest_memfd with respect to existing backing stores so that it's possible to use guest_memfd to back all VM shapes in a fleet. Let's assume there is significant value in backing non-CoCo VMs with 1GiB pages, unless you want to re-litigate the existence of 1GiB support in HugeTLBFS. If we assume 1GiB support is mandatory for non-CoCo VMs, then it becomes mandatory for CoCo VMs as well, because it's the only realistic way to run CoCo VMs and non-CoCo VMs on a single host. Mixing 1GiB HugeTLBFS with any other backing store for VMs simply isn't tenable due to the nature of 1GiB allocations. E.g. grabbing sub-1GiB chunks of memory for CoCo VMs quickly fragments memory to the point where HugeTLBFS can't allocate memory for non-CoCo VMs. Teaching HugeTLBFS to play nice with TDX and SNP isn't happening, which leaves adding 1GiB support to guest_memfd as the only way forward. Any boost to TDX (or SNP) performance is purely a bonus.