From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BB392C4332F for ; Wed, 1 Nov 2023 13:49:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 13A4B8D0062; Wed, 1 Nov 2023 09:49:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0C2518D0001; Wed, 1 Nov 2023 09:49:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E58548D0062; Wed, 1 Nov 2023 09:49:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id CED988D0001 for ; Wed, 1 Nov 2023 09:49:58 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id AC526120416 for ; Wed, 1 Nov 2023 13:49:58 +0000 (UTC) X-FDA: 81409518876.14.086C08D Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf07.hostedemail.com (Postfix) with ESMTP id 9662540017 for ; Wed, 1 Nov 2023 13:49:56 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=jL3WY9bu; spf=pass (imf07.hostedemail.com: domain of pbonzini@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=pbonzini@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1698846596; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8nWuZfMltD4ivs2KoMTFabKsH4/GDdWsbpvxlt/w2Uk=; b=Afd6Blt7fZA3SyTzfdiEYj633FL0DEA+BM9Nw4jHpUCxp8Q7XVAiUlQ3UUsW57yIKBjZeC 1/ZRbjz/l+LOC3q6Q+5YJD4lYasIA6Yt5kY6GaQLnMPGqhECwowjMAUvu+KZXx9HM7vnjW tpfFrwe4z6dz8+IWibB5TyOPlW31RSY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1698846596; a=rsa-sha256; cv=none; b=FIk8UB4wc8l73xmkN8uaVgOhpJrn1WqWRK+YmFjjXS6G1gX9QINcZCWNjovJuHLyscb5xE rJVwLE16Yf8tfTN2H/wMvt9BdDsSaj9qaZHxcoNCjR5CmHrBJsHmLHazAVhJ+yVYVQBuyy Bqh+/fCh5N95iU+iH6eABmcXXLfyNaM= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=jL3WY9bu; spf=pass (imf07.hostedemail.com: domain of pbonzini@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=pbonzini@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1698846595; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8nWuZfMltD4ivs2KoMTFabKsH4/GDdWsbpvxlt/w2Uk=; b=jL3WY9buMvcmSLWYW8VSR+Yzmz3v/6tr0hdtjTAehz4PwjdPhpur56l4p6kZedMQZd2G92 KoeFoTkAwZARXMScnVviyF+D5ZDVBGmkze5if+aufAK0wO8L5Eq3ynYm01uLBKg+heOri9 ig/m+JUsrNkva4s0E6KzPF5cUxB64uk= Received: from mail-vs1-f72.google.com (mail-vs1-f72.google.com [209.85.217.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-509-6i7acexNO5uM5JIDAdYvyQ-1; Wed, 01 Nov 2023 09:49:54 -0400 X-MC-Unique: 6i7acexNO5uM5JIDAdYvyQ-1 Received: by mail-vs1-f72.google.com with SMTP id ada2fe7eead31-457d4020f0fso2359743137.0 for ; Wed, 01 Nov 2023 06:49:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698846593; x=1699451393; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8nWuZfMltD4ivs2KoMTFabKsH4/GDdWsbpvxlt/w2Uk=; b=gmU23FD1MVSY8o7WH/SgDg6Vb/WRXSwbAqWjI30Jq480CIe5PLH3W1HEKIyz3jvYSM 8PlNWY4RENur6uZbgucwEVHCiyx2HG3HJy5qTB9zyth2SAleiRHhGNPdMrVBh+ZSySid AfEhrjNAwCgE3Vn1AqaPbWoEsyi46psqDIk/JNdTiCQX85ehhOUTi1On4JZs0UAsJE0J qb+jGZQZLVEizFm/T2Q5Cgik97lM9+pR6diIWwdnmnvnP69AwsQa5mGaHZV6E8ogQ2hF +tLeGuzUc7ZxTX4q+vLibHZm/igFG2K94JoxgKYopmPEMMBFrPestYrpffeMEIDia3mB vS8Q== X-Gm-Message-State: AOJu0Yz9UPfKb4pqLtThE5pT9GBnpMfI+ajRhJ4a7hhmy0GoqU6PvIAU csJBcy948pTwW3qVzbPaZ7EAtbypD73QiceFz8lpKM09OFTaieWIBSi6emqKcFxBS3Hs1umb1rM l1el8Mx0gaQbnfR0PapXgsgT566I= X-Received: by 2002:a67:a20b:0:b0:45b:529:cffb with SMTP id l11-20020a67a20b000000b0045b0529cffbmr13944643vse.27.1698846593182; Wed, 01 Nov 2023 06:49:53 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFwXCml3dQ0y1Cr/jFjux1lHGIWg//wB9GtnyXEiXbtExQlCwIylI67p1IY3V8uopiP51EusJ56Du+cgZSOAjo= X-Received: by 2002:a67:a20b:0:b0:45b:529:cffb with SMTP id l11-20020a67a20b000000b0045b0529cffbmr13944598vse.27.1698846592873; Wed, 01 Nov 2023 06:49:52 -0700 (PDT) MIME-Version: 1.0 References: <20231027182217.3615211-1-seanjc@google.com> <20231027182217.3615211-18-seanjc@google.com> <7c0844d8-6f97-4904-a140-abeabeb552c1@intel.com> <92ba7ddd-2bc8-4a8d-bd67-d6614b21914f@intel.com> In-Reply-To: From: Paolo Bonzini Date: Wed, 1 Nov 2023 14:49:40 +0100 Message-ID: Subject: Re: [PATCH v13 17/35] KVM: Add transparent hugepage support for dedicated guest memory To: Sean Christopherson Cc: Xiaoyao Li , Marc Zyngier , Oliver Upton , Huacai Chen , Michael Ellerman , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexander Viro , Christian Brauner , "Matthew Wilcox (Oracle)" , Andrew Morton , kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Xu Yilun , Chao Peng , Fuad Tabba , Jarkko Sakkinen , Anish Moorthy , David Matlack , Yu Zhang , Isaku Yamahata , =?UTF-8?B?TWlja2HDq2wgU2FsYcO8bg==?= , Vlastimil Babka , Vishal Annapurve , Ackerley Tng , Maciej Szmigiero , David Hildenbrand , Quentin Perret , Michael Roth , Wang , Liam Merwick , Isaku Yamahata , "Kirill A . Shutemov" X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 9662540017 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 5b3wqyhce7fdrkt6sbawh3dhr4euehoi X-HE-Tag: 1698846596-132274 X-HE-Meta: U2FsdGVkX19+f7CLSrkozBEruZFg8MuVpcmh5/MbpuQhl29xi36RBBynAtEZqfuYkwDlxJ2skjfZVX9a7jsy8lLHeq3hw1Zqd02Pmu2Yg5yvoWjzMaqrtqdCRZJiFNnxuyGPWa++pTwQO7bJwnkDrX06uLjHoiozxUqhRnIgK+b9nQT4LZGQdKgZkn9h+ypI+kpsZnEW6orFq+jmL345i83TMupzoYRmnK8Sjd4wJdzvI09gUpKqk3RWNPClAzG0AreYp3R1MJXfgK/0mVydacVWvFYAJAfudj7THj0fsgaGdZT0IBrR9ylew8EDrY3zRg9QRIu2q1VMXXz1EOdXorkRWl6vucvGZz7lbtlkUaBAfgyd8leK8ZLe3G9I9ZlNwT9vVcy08FgatP0P6IDLbg9X70K9qYY7oO4tfSPynxpTr0jMXZJORczXuirL+KpknSNmFDP+yu7eXC4TWwj4ugOtRB9xphkhi2TJemmgW4Aqe8fbyKR0IwlhhGk5iq5PVSfCZCP4s1eyhwNW0DcQtQMXBOUvHqj01YWpHl44IEYVLRMsqn2L48tRwtwNXx1ojlE4nVBJgvMYRriCGcC4yk/fxtLw//NlSwRqObsKQT/Cc4ScjlDqGhE4+d8b1cjmPoQx5Q/sEIZP9uJr1GHeKVqkc8xt+8T5tCZtk0d5IQIKZcDAtU7krdSy9zOsXCBgrOyrAVwJkof6yRuQnojgk8UyvrXRYA2kwE1QEZoWfJXng6loO41OFpmnZtJqneRWqvOpTgjD4tHD4y5vVVxO9rGDJYuhfDKdz8khA97fEdfn57aulRcVtTw2XABYKvvM7R9HtmEy2ku3MZ0KLa8zRp+hSdefcfVWn1EqTH2pI7DUqAib9aGvneqJ9TSUOWDSbAPCT+Wnv0BcEppIdcIp/Jy8JjicWl05KBWshYABaVMrxIYuVG9AiVYFXzrQPFqhprIWtCdENYIBe4KdvIO YooHfG18 nJ8HHnVMmecvEclvqQuNDRd+/f/UF2443May3k0Dv1rEU19UsJYv1N2XjynZ6wvSGmdnJK1SM5lsqaVhMTVo9aEWAPNIhbwgnN8FVWQkYX/j0EFBCLTkBF0gS2NoxihALP0gjKzlRu63zSm2Ugm8dIaj5Fyu7SSMbjEGt8qpynUgNIEHMZ20gRV62PPWrkt0mzHWracImR6eWFA3CFo23+0jOplotoVphu/H1nZAQ5rLJ72sfebH99y4ms6Ul46tl7Ep8sZGnAB28cqaLD+k9ZzELiJMSPVN9QnPHRDo6K/QLQhcpu7EnCJT1mkHbrcj/+CD+Lwy4JnuwIngdGEkKGUHHf9CT6UhU132+1CELS8wjOSCCIAm2H+qdis6R3jJdb496scp/7L/uoEQyHowdrICdMGVI+tytGCRac37sc41NFZ4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Nov 1, 2023 at 2:41=E2=80=AFPM Sean Christopherson wrote: > > On Wed, Nov 01, 2023, Xiaoyao Li wrote: > > On 10/31/2023 10:16 PM, Sean Christopherson wrote: > > > On Tue, Oct 31, 2023, Xiaoyao Li wrote: > > > > On 10/28/2023 2:21 AM, Sean Christopherson wrote: > > > > > Extended guest_memfd to allow backing guest memory with transpare= nt > > > > > hugepages. Require userspace to opt-in via a flag even though the= re's no > > > > > known/anticipated use case for forcing small pages as THP is opti= onal, > > > > > i.e. to avoid ending up in a situation where userspace is unaware= that > > > > > KVM can't provide hugepages. > > > > > > > > Personally, it seems not so "transparent" if requiring userspace to= opt-in. > > > > > > > > People need to 1) check if the kernel built with TRANSPARENT_HUGEPA= GE > > > > support, or check is the sysfs of transparent hugepage exists; 2)ge= t the > > > > maximum support hugepage size 3) ensure the size satisfies the alig= nment; > > > > before opt-in it. > > > > > > > > Even simpler, userspace can blindly try to create guest memfd with > > > > transparent hugapage flag. If getting error, fallback to create wit= hout the > > > > transparent hugepage flag. > > > > > > > > However, it doesn't look transparent to me. > > > > > > The "transparent" part is referring to the underlying kernel mechanis= m, it's not > > > saying anything about the API. The "transparent" part of THP is that= the kernel > > > doesn't guarantee hugepages, i.e. whether or not hugepages are actual= ly used is > > > (mostly) transparent to userspace. > > > > > > Paolo also isn't the biggest fan[*], but there are also downsides to = always > > > allowing hugepages, e.g. silent failure due to lack of THP or unalign= ed size, > > > and there's precedent in the form of MADV_HUGEPAGE. > > > > > > [*] https://lore.kernel.org/all/84a908ae-04c7-51c7-c9a8-119e1933a189@= redhat.com > > > > But it's different than MADV_HUGEPAGE, in a way. Per my understanding, = the > > failure of MADV_HUGEPAGE is not fatal, user space can ignore it and > > continue. > > > > However, the failure of KVM_GUEST_MEMFD_ALLOW_HUGEPAGE is fatal, which = leads > > to failure of guest memfd creation. > > Failing KVM_CREATE_GUEST_MEMFD isn't truly fatal, it just requires differ= ent > action from userspace, i.e. instead of ignoring the error, userspace coul= d redo > KVM_CREATE_GUEST_MEMFD with KVM_GUEST_MEMFD_ALLOW_HUGEPAGE=3D0. > > We could make the behavior more like MADV_HUGEPAGE, e.g. theoretically we= could > extend fadvise() with FADV_HUGEPAGE, or add a guest_memfd knob/ioctl() to= let > userspace provide advice/hints after creating a guest_memfd. But I suspe= ct that > guest_memfd would be the only user of FADV_HUGEPAGE, and IMO a post-creat= ion hint > is actually less desirable. > > KVM_GUEST_MEMFD_ALLOW_HUGEPAGE will fail only if userspace didn't provide= a > compatible size or the kernel doesn't support THP. An incompatible size = is likely > a userspace bug, and for most setups that want to utilize guest_memfd, la= ck of THP > support is likely a configuration bug. I.e. many/most uses *want* failur= es due to > KVM_GUEST_MEMFD_ALLOW_HUGEPAGE to be fatal. > > > For current implementation, I think maybe KVM_GUEST_MEMFD_DESIRE_HUGEPA= GE > > fits better than KVM_GUEST_MEMFD_ALLOW_HUGEPAGE? or maybe *PREFER*? > > Why? Verbs like "prefer" and "desire" aren't a good fit IMO because they= suggest > the flag is a hint, and hints are usually best effort only, i.e. are igno= red if > there is a fundamental incompatibility. > > "Allow" isn't perfect, e.g. I would much prefer a straight KVM_GUEST_MEMF= D_USE_HUGEPAGES > or KVM_GUEST_MEMFD_HUGEPAGES flag, but I wanted the name to convey that K= VM doesn't > (yet) guarantee hugepages. I.e. KVM_GUEST_MEMFD_ALLOW_HUGEPAGE is strong= er than > a hint, but weaker than a requirement. And if/when KVM supports a dedica= ted memory > pool of some kind, then we can add KVM_GUEST_MEMFD_REQUIRE_HUGEPAGE. I think that the current patch is fine, but I will adjust it to always allow the flag, and to make the size check even if !CONFIG_TRANSPARENT_HUGEPAGE. If hugepages are not guaranteed, and (theoretically) you could have no hugepage at all in the result, it's okay to get this result even if THP is = not available in the kernel. Paolo