From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6558C0219D for ; Thu, 13 Feb 2025 08:16:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0BEB1280003; Thu, 13 Feb 2025 03:16:58 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 06F44280001; Thu, 13 Feb 2025 03:16:58 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E78C4280003; Thu, 13 Feb 2025 03:16:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id CB8D1280001 for ; Thu, 13 Feb 2025 03:16:57 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 684CB120911 for ; Thu, 13 Feb 2025 08:16:57 +0000 (UTC) X-FDA: 83114215674.25.67ACFE2 Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) by imf24.hostedemail.com (Postfix) with ESMTP id 7B66918000D for ; Thu, 13 Feb 2025 08:16:55 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="fC/0hxtb"; spf=pass (imf24.hostedemail.com: domain of haoxing990@gmail.com designates 209.85.216.53 as permitted sender) smtp.mailfrom=haoxing990@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739434615; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=r6LI7V5OeHXNmUTBUom52jZZT/Bwgs7vZpSfVDrmOH8=; b=2e1u5yz8XrDlkDOJtjt13sgXPi117oQAzcTLlaJPBRo4ibHVNJ+5r6xbBhwsi6+Uqs4I/Q tD5x5IHzzTeWc6meDlPYuKSSHRLUroYa7rLxwEYAASK8DpM0hD3hYq43wuPMn15GbYJGAH luYsOoeQ+hHiUuXrTVp9Nsv6uPUYgmU= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b="fC/0hxtb"; spf=pass (imf24.hostedemail.com: domain of haoxing990@gmail.com designates 209.85.216.53 as permitted sender) smtp.mailfrom=haoxing990@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1739434615; a=rsa-sha256; cv=none; b=wLcLtrA7Nu+Pjj6VmlubaHF1/CF1+aTCFfvXW3vsEYgFrSRPFcazUOVRqaxkdSO0xGqnea pzpCSZHK3r1yuHytDybNGBPVH9q2SIcA9Ilado3VQGE5uy9rtPZkLUo5ThbVjplO6ma/sm qNJxjE2Jdk1EgFIZqraBksVAcHP/+aw= Received: by mail-pj1-f53.google.com with SMTP id 98e67ed59e1d1-2f9f6a2fa8dso943265a91.1 for ; Thu, 13 Feb 2025 00:16:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1739434614; x=1740039414; darn=kvack.org; h=in-reply-to:from:references:cc:to:subject:user-agent:mime-version :date:message-id:from:to:cc:subject:date:message-id:reply-to; bh=r6LI7V5OeHXNmUTBUom52jZZT/Bwgs7vZpSfVDrmOH8=; b=fC/0hxtb9/8tfQNZSLnoXa6IrnrEHIUFh0gS4W49hjUrafQcEDwdzgECrFTcmoSd6k 7atFsoFZ3cowSCtqz8KzdFEwGAsBb7XtLmzmp+jMumYkDRPtIHgaKHjV5sT+hc5BRPCH LVtwbXX4srZ40q3xEG4CaLR6CzRmu/Lp5KnbJAcLNrM0oRe86Le4b9ywoz6B2zo33HFL xQF2/+nm8s0VljDMZEKVaKpBS50lkqA0hXhLKWMOuk64PwRUhPXX6o6XBqIU/Gtc4Gur xe4xxgTO6ET2GOMzu9ql6XT70TQgWrN+NPOm5lyMQSXnVVRRG5HT/HlVl5IJXvATOyC/ uKvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739434614; x=1740039414; h=in-reply-to:from:references:cc:to:subject:user-agent:mime-version :date:message-id:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=r6LI7V5OeHXNmUTBUom52jZZT/Bwgs7vZpSfVDrmOH8=; b=eGkNXBU3ao9+KHrCRlcpVh9t+TZWV2n+HIppfBoj+O9bbuah0f0KEPT42IFLRfCLtF Rn7K0dmihujDyCy0fa2YdP1X1+cj/+15YMv5EoA7XsYy/Qk7ID7eObVc4iCZwi6SXkcg YhxHs+7HrcoQzFHuNy1NxFFLQD+5XrQDwmBtmYYymWwMsusn1+4e3pPsJz0KYi1sVrqU Qn8iG2jA1Q3hhx8/tZrxhfoJd1zL1qoEZ46OchTPXzqtcxr+0TRkL+djPZruOi1K75UT Em9OncKpUZdLGCgiAQhNhcmZk9NtkILgDhvUyQZiQnAb+AOcfjGl8PHmqLv5ZXbBVJ1j MdKQ== X-Forwarded-Encrypted: i=1; AJvYcCWY7mVl0Zh2nuJb5pnt6ZoPPnaeBMihjB1Gnx3SGl6feBor3Liy7Z2tDC5VAXkiHW13kKvKcV2yXg==@kvack.org X-Gm-Message-State: AOJu0YwqhmA9cHrcjTUDg1fq/a1MbHAWW59dCk/zpslNu8crUw+Y1nQd V8LSHwjEIzD4DzJVaXck0HnvhI1p1CWM/s7K1WGbGij4EgnvjQdf08RCQQ== X-Gm-Gg: ASbGncs1QgsUVi6y7J/ajXSXRPsza6BkJ44zK6x6zJ+LaAcZaJLbj4DWmLAq85oFLEe 1RVDirWS3FxYj12FJgKh0NB6bUSSgu1zaEDo9LaFEe8nqZLVG2epqG7RDvuhMDeg060rvexYd5A kPLyfZHJj+mMvLHOlpPv9QPGRaHWkmDcGSvjAteByEoXGMic14q8pT3RhQjeEj1U8+joMp4v9je hN+5VxBKYYcXigwBRmeBKXqzQhstHIE3nb6XVDqo5qJ0niPq0B3iO+0U0J+tEOkj4krj8VOYiKT jF1lK1EvKpRkwn5MZY4SNRuzoeuFwxK6ar6k3f0olA8= X-Google-Smtp-Source: AGHT+IFCoHn2btkihiPEqoDCCwzFtPx20/J7sPbjQsE5GMeOaQryNI2Fgot3eBS9lnVnJPmQQfH02w== X-Received: by 2002:a17:90a:d00e:b0:2ee:d433:7c50 with SMTP id 98e67ed59e1d1-2fbf5c5a06dmr7512158a91.23.1739434612708; Thu, 13 Feb 2025 00:16:52 -0800 (PST) Received: from [192.168.255.10] ([43.132.141.25]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2faa4af2472sm1529602a91.0.2025.02.13.00.16.48 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 13 Feb 2025 00:16:52 -0800 (PST) Content-Type: multipart/alternative; boundary="------------3AwzBeNyjAISWP8LJYT5OKGW" Message-ID: <366195ff-3dd4-4d27-9091-6586d681ae13@gmail.com> Date: Thu, 13 Feb 2025 16:16:47 +0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v9 04/12] x86/mm: get INVLPGB count max from CPUID To: Tom Lendacky , Rik van Riel Cc: x86@kernel.org, linux-kernel@vger.kernel.org, bp@alien8.de, peterz@infradead.org, dave.hansen@linux.intel.com, zhengqi.arch@bytedance.com, nadav.amit@gmail.com, kernel-team@meta.com, linux-mm@kvack.org, akpm@linux-foundation.org, jannh@google.com, mhklinux@outlook.com, andrew.cooper3@citrix.com, Manali Shukla References: <20250206044346.3810242-1-riel@surriel.com> <20250206044346.3810242-5-riel@surriel.com> <281d6fd629a1965ee0065f99ac5d693bace9758b.camel@surriel.com> <4f63d409-89c0-412b-8d69-371261f305ea@gmail.com> <93281f16-4237-b6d6-624e-cc805adb1c37@amd.com> From: Vern Hao In-Reply-To: <93281f16-4237-b6d6-624e-cc805adb1c37@amd.com> X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 7B66918000D X-Stat-Signature: qxtee3sn6dbqor33k7n6foqehjn17qfk X-Rspam-User: X-HE-Tag: 1739434615-183368 X-HE-Meta: U2FsdGVkX19dnTtB0qRWpAZTQoNpby0xlrZx49jIsFTwg0K/hAJevEu76FYWymEqJd9hKYrogP2EvplBR9kwLQ8fU1EYZ5jPh/nZBXrS+plgLm13efC/z+3bLZBq+RNioTQQ7E2S5cwMmmwK62aEBOR+Zc1MBoFncZGI6Q8seMOvzyLz3V6h/z1DDM8EocrjlGIVZ/y0qnZqXC/rA8ZxO42fu2mwVT7L9aPdz9Len8vHDqpDlBDvjHkyYysOL09MhH3YHXuQX4OBIGZkR7OD+KbsQ+C5vzQNdWqqm1orWdXr3dgCWW2Y8eAvdgRs/SEmGnRcc98Y1+zb9EdM32zryAmH+QZxM7Zw0ckXYYnJqQoDEw8Zj0A9yz2e/1jQtF4UrG6hUOxnikmWNKatlbSurzYE8uHFdVJTarvYnC+oqUdbAIE0Z6vVtay8fqH0s9tykha7oOk6vJ3cpYOGagqZIka/PVpnysTluqzFAx2m/lnAh6pCI6b0sq8OzOM3LGUSYo1b3hsnM4Py/HUgV6hV7mTTV6Bui7Gsjt+Wsf2uyxPReDPXnmgOL55aCCfvqYxOBa2t1xNvTeBgct+noe0TFRYgn05kCvXVYDhu1RQHH6EMyeDvKLHxNRl4DW/3LZCpolumtxWkDtFWV5X+5NjCUKEGQFyya7A6lUI+heH539WUGRChMxhqYVnBmHDR0EeO8xDTXGvGybPOYFGdOPwK4LqcxSYQPEdTPDJd7APTLpbKDNVaqRHJh2UNvj2LeYaizLcF3sZ3GoTg5GWhDThAAowTmXhM3/VzuDc345evvKPEk3H0MIz+okPyr/HUH7es4IBrvQVUURMYuYM1bbeQ6v5rOcat8AEnruX3tp260gKBNqrVR+PxArNuw2jLTRIAXPUjC+jOPvhoraK5vNmVrLFxGiV8MfStiXb8GJYPI66S21llHHaZkFT2dr9+9tg8xw/f1A5z7sjpoaLdwUK uRsNJMio ctDoVUk3mKO/FV1JVAydLxY+/PJ/QVu7YoDaj10twKhdxYTfxr9wy6yZhrxagdRAwoyvNZugjPr5HmBrwCFlxntUEJEo8TK+DrXZVJll5KOYguckfNlmjYFNdbkTzpIOIkdxnmS7zqzlHA8ZDwrDhfUvzGDuYSP8C4ARfHCcxF66e08F9QSC4DfEy+oEWj+XPxSFEgKGpA9uzkkZFaPvCSiqYHZDo38LtBIW08LFsNjcQTkwWY5u82uxML++jwfkq+frbtlsJCc3fUvtl18FOkg4ioMcTNaukGZ7segy2yDzAZRheDbD2Wk6uhw/98iI5GNoDExRrEDNlLR+lvpAns1iUDpOYwF7F+0ET2i1rmVERHLdwrs52GdEFmLAEyyePbq/PpF0ivcHNGMdoQpL69GK6PNcuiPlT/ia35m2somhorbYjAMoutmt49Z2rcdifztxvEaxfQzKw12QAwd3/6V9z7YO9p4VStybpicFkmc7Pl9QJBexTpAOWHFfdnSkNSH9DZHJqbeoxczEWm3duo5e/jEMkmO/kftcnrsXiPOU/3pRKxYZOJn4ks86jp3sMS1vJ/xX2uLngREQELpPmECtkzm9qNtOJavWBq5dpmmARXyuLa+EXQbzt9sFngjFh9XWsS0VPK31LBELiekHfnL4oQA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This is a multi-part message in MIME format. --------------3AwzBeNyjAISWP8LJYT5OKGW Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 2025/2/12 23:56, Tom Lendacky wrote: > On 2/11/25 19:57, Vern Hao wrote: >> On 2025/2/11 00:48, Rik van Riel wrote: >>> On Mon, 2025-02-10 at 15:30 +0800, Vern Hao wrote: >>>> I do some test on my  Machine with AMD EPYC 7K83,  these patches work >>>> on my host,  but failed on my guest with qemu. >>>> >>>> in host, use lscpu cmd, you can see  invlpgb in flags,  but in guest >>>> no. >>>> >>>> So are you plan to support it in guest? >>> How exactly did it fail in the guest? >>> >>> Did the guest simply not use INVLPGB because that >>> CPUID flag was not presented in the CPUID that >>> qemu shows to the guest, or did things crash somehow? >> i support these patches in host and guest, and add this patch to support >> cpuid flags in kvm. >> >> diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c >> index db3838667466..fd21d9438137 100644 >> --- a/arch/x86/kvm/cpuid.c >> +++ b/arch/x86/kvm/cpuid.c >> @@ -488,7 +488,7 @@ static inline int __do_cpuid_func(struct >> kvm_cpuid_entry2 *entry, u32 function, >> >>         /* cpuid 0x80000008.ebx */ >>         const u32 kvm_cpuid_8000_0008_ebx_x86_features = >> -               F(CLZERO) | F(XSAVEERPTR) | >> +              F(CLZERO) | F(XSAVEERPTR) | F(INVLPGB) | >>                 F(WBNOINVD) | F(AMD_IBPB) | F(AMD_IBRS) | F(AMD_SSBD) | >> F(VIRT_SSBD) | >>                 F(AMD_SSB_NO) | F(AMD_STIBP) | F(AMD_STIBP_ALWAYS_ON); >> >> But  in guest,  use lscpu cmd, i still can not see  invlpgb,  so  i just >> wonder where is wrong ? > Well, for one, the INVLPGB instruction has to be enabled in the VMCB in > order for it to be used (unless it is an SEV-ES or SEV-SNP guest). Also, > lscpu won't show "invlpgb" since the patches don't define the feature in > the way that it would be visible via lscpu. You need to issue CPUID to > see the bit is set or not. Also, you might need VMM support in order for > that CPUID bit to be set in the guest. > > But, it will take hypervisor support to use INVLPGB in a non-SEV guest, > since non-SEV guests do not use global ASIDs. In this case, the > instruction will need to be intercepted and the hypervisor will need to > determine how to process it. > > If you have an SEV guest, which use global ASIDs and use the same ASID > for all vCPUs within a guest, you can use INVLPGB in the guest without > issue and without needing to intercept the instruction. > > See "Guest Usage of INVLPGB" in AMD APM Vol 3 under the INVLPGB > instruction documentation. OK,  Thanks a lot*. * *Obviously, my 5.10 version of the kernel KVM does not support this CPUID well, i test it on upstream kernel, it works well.* > > Thanks, > Tom > >>> My understanding is that while INVLPGB can work >>> in guests, actually implementing that is a whole >>> other can of worms, and definitely not something >>> we should try to tackle at the same time as bare >>> metal support. >>> >>> A TLB flush hypercall, with IRQ-less flushing on >>> the hypervisor side will probably get us 90% of >>> the way there, potentially with less overall >>> complexity than actually supporting INVLPGB in >>> the guest. >>> --------------3AwzBeNyjAISWP8LJYT5OKGW Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit


On 2025/2/12 23:56, Tom Lendacky wrote:
On 2/11/25 19:57, Vern Hao wrote:

On 2025/2/11 00:48, Rik van Riel wrote:

On Mon, 2025-02-10 at 15:30 +0800, Vern Hao wrote:

I do some test on my  Machine with AMD EPYC 7K83,  these patches work
on my host,  but failed on my guest with qemu.

in host, use lscpu cmd, you can see  invlpgb in flags,  but in guest
no.

So are you plan to support it in guest?

How exactly did it fail in the guest?

Did the guest simply not use INVLPGB because that
CPUID flag was not presented in the CPUID that
qemu shows to the guest, or did things crash somehow?

i support these patches in host and guest, and add this patch to support
cpuid flags in kvm.

diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c
index db3838667466..fd21d9438137 100644
--- a/arch/x86/kvm/cpuid.c
+++ b/arch/x86/kvm/cpuid.c
@@ -488,7 +488,7 @@ static inline int __do_cpuid_func(struct
kvm_cpuid_entry2 *entry, u32 function,

        /* cpuid 0x80000008.ebx */
        const u32 kvm_cpuid_8000_0008_ebx_x86_features =
-               F(CLZERO) | F(XSAVEERPTR) |
+              F(CLZERO) | F(XSAVEERPTR) | F(INVLPGB) |
                F(WBNOINVD) | F(AMD_IBPB) | F(AMD_IBRS) | F(AMD_SSBD) |
F(VIRT_SSBD) |
                F(AMD_SSB_NO) | F(AMD_STIBP) | F(AMD_STIBP_ALWAYS_ON);

But  in guest,  use lscpu cmd, i still can not see  invlpgb,  so  i just
wonder where is wrong ?

Well, for one, the INVLPGB instruction has to be enabled in the VMCB in
order for it to be used (unless it is an SEV-ES or SEV-SNP guest). Also,
lscpu won't show "invlpgb" since the patches don't define the feature in
the way that it would be visible via lscpu. You need to issue CPUID to
see the bit is set or not. Also, you might need VMM support in order for
that CPUID bit to be set in the guest.

But, it will take hypervisor support to use INVLPGB in a non-SEV guest,
since non-SEV guests do not use global ASIDs. In this case, the
instruction will need to be intercepted and the hypervisor will need to
determine how to process it.

If you have an SEV guest, which use global ASIDs and use the same ASID
for all vCPUs within a guest, you can use INVLPGB in the guest without
issue and without needing to intercept the instruction.

See "Guest Usage of INVLPGB" in AMD APM Vol 3 under the INVLPGB
instruction documentation.

OK,  Thanks a lot.

Obviously, my 5.10 version of the kernel KVM does not support this CPUID well, i test it on upstream kernel, it works well.


Thanks,
Tom



        

          
My understanding is that while INVLPGB can work
in guests, actually implementing that is a whole
other can of worms, and definitely not something
we should try to tackle at the same time as bare
metal support.

A TLB flush hypercall, with IRQ-less flushing on
the hypervisor side will probably get us 90% of
the way there, potentially with less overall
complexity than actually supporting INVLPGB in
the guest.



        

      

    

  


--------------3AwzBeNyjAISWP8LJYT5OKGW--