From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 86296C433F5 for ; Thu, 6 Jan 2022 20:51:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 116416B0072; Thu, 6 Jan 2022 15:51:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 09D086B0073; Thu, 6 Jan 2022 15:51:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E0ACD6B0074; Thu, 6 Jan 2022 15:51:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0247.hostedemail.com [216.40.44.247]) by kanga.kvack.org (Postfix) with ESMTP id CFD676B0072 for ; Thu, 6 Jan 2022 15:51:03 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 92A5C8BED9 for ; Thu, 6 Jan 2022 20:51:03 +0000 (UTC) X-FDA: 79001056806.11.FADF4F0 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2073.outbound.protection.outlook.com [40.107.94.73]) by imf12.hostedemail.com (Postfix) with ESMTP id E46B040002 for ; Thu, 6 Jan 2022 20:50:35 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TZ+pLzc8dPkE1Pry2NzOm7EHU9d2u5soMidy7cJO2ZQ7bcN/xEdhQ914LdwejbEnz22xZ7lenxPiuU9Y8KQU3xN7DMjejOTTN/xI76srbZbp7elW1pXdcUp6S4qMWFVHFbBG0wzwFc5nKdwMA63UQXxxCGkv779cc5uLdEABjnXeQyCWPzklPUZyV4LFM/z91kZ3AHHrr6dl77EhixIz0LKlto/H4wmeMBF7MBE31gWzrx5GGMVk8nUyBP/mEa8wMsnsrlqlj4zKLeazmAXzbyMHuSXDqY1BCHwbPsyZL4DKo80GV1IvTXImONJOgfUSbYn1wUpT6bSFpjePqZgw/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=coqJMrBLSVlyYomoj08zUQx+rIVlaKWAofBhboH4tgc=; b=A2nPcTGrcQPzRiEEWFI6ZEpOkj1ITiFF/69MsGUw6ftSE1o0IJtLxz3CCjjV2I1MwIdy2BwkOJlk9SfxHIitqT0cllZJ01Jhr7np2ppg/FUt4iMRteho0bV6RHk/8m8vit6hKVirpzxi7bNW8v6GzpMKMJO6Qe40hLukA3SaAhZyc+iU+BtaxXi/+Yc3z+HOiycmR0oColK88ygy7R4AgP9dwbjD+Gu7t/uWRcMMz/XQ/O9hfLSmKG93AGAGBCQcHNBYCb9cTzjLMtpwawa6pp1DuiYgCWKFSpHn2eg1YTso6Qnadb0ur8RXsIJE/R6s7MQnWdUlue5X+hdXQA5w4g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=coqJMrBLSVlyYomoj08zUQx+rIVlaKWAofBhboH4tgc=; b=rmA1fkE7S467QUwn9YZivkFBmkkefUAI+0JqP+uJ7PeMkUlSp+sm22bv/ztgTpeY7iq7iilchcsgeNtJsIK5p7jje8Ayj8/OH9XDuUcPZdghec+EijtiDrwlUpIP89qiu9RHZtdveM/PFWZLrlwCFXaf58C8VZPVF5/+Ax4z3TI= Received: from DM4PR12MB5229.namprd12.prod.outlook.com (2603:10b6:5:398::12) by DM4PR12MB5247.namprd12.prod.outlook.com (2603:10b6:5:39b::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4844.13; Thu, 6 Jan 2022 20:51:00 +0000 Received: from DM4PR12MB5229.namprd12.prod.outlook.com ([fe80::ccd7:e520:c726:d0b]) by DM4PR12MB5229.namprd12.prod.outlook.com ([fe80::ccd7:e520:c726:d0b%5]) with mapi id 15.20.4867.010; Thu, 6 Jan 2022 20:51:00 +0000 Subject: Re: [PATCH v8 13/40] x86/kernel: Make the bss.decrypted section shared in RMP table To: Venu Busireddy Cc: Brijesh Singh , Dave Hansen , x86@kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, linux-efi@vger.kernel.org, platform-driver-x86@vger.kernel.org, linux-coco@lists.linux.dev, linux-mm@kvack.org, Thomas Gleixner , Ingo Molnar , Joerg Roedel , "H. Peter Anvin" , Ard Biesheuvel , Paolo Bonzini , Sean Christopherson , Vitaly Kuznetsov , Jim Mattson , Andy Lutomirski , Dave Hansen , Sergio Lopez , Peter Gonda , Peter Zijlstra , Srinivas Pandruvada , David Rientjes , Dov Murik , Tobin Feldman-Fitzthum , Borislav Petkov , Michael Roth , Vlastimil Babka , "Kirill A . Shutemov" , Andi Kleen , "Dr . David Alan Gilbert" , tony.luck@intel.com, marcorr@google.com, sathyanarayanan.kuppuswamy@linux.intel.com References: <20211210154332.11526-1-brijesh.singh@amd.com> <20211210154332.11526-14-brijesh.singh@amd.com> <18eadf9d-8e31-0945-ccf4-2cb72b8e0dd4@amd.com> <444b6c78-c4d3-f49c-6579-bd28ae32ca3c@amd.com> From: Tom Lendacky Message-ID: <4c1067c7-a85f-79fe-38a8-2f68a94a358a@amd.com> Date: Thu, 6 Jan 2022 14:50:56 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.14.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US X-ClientProxiedBy: CH0PR03CA0006.namprd03.prod.outlook.com (2603:10b6:610:b0::11) To DM4PR12MB5229.namprd12.prod.outlook.com (2603:10b6:5:398::12) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2232576a-d440-4832-8c11-08d9d1563f0f X-MS-TrafficTypeDiagnostic: DM4PR12MB5247:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8882; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: mO9bNiWihNDXTZqksmJyqityQPTcW9e4xHIta8DetpEUC9zyg0RtfPgeYH6OE1v0wO2no2aXoP9lc2Q6m4Wup3l22QOWNWXVxmdOBYvggwlENpClh9CkWvqQJLl/TT3qtG9Net/NbbuaDdMXOdetmkagPpRr0GOLqNEE0qQ8d85clPblLHqyRGc+1cbgTYTkRFIjP0bJcC3omg+aGQ4DfQf3yPWeElhdTfIHGaN1LlAKJuf2vXoYBCh3T4TlcQbgQOWobtAtCjHhAylJpFgelvCAB/d74RCC6AE9h/kiY3DnoZBQQVRTqqqgWdfm+LEAyZJWZ5no1CYqd955g6OUAEI+2ogNu4xr9VnjpmNUYHwHovQcpwe/5otDU5pkbB8bF4NucFNzgLM+g6fIohs3E6saL+DJ30RUkvwYEmcOAwycOmaUyXSKubQ0a+uabotgtkbiN+dG6/MFi222w9sWrjE/z24eVojKawSTTpZkLMiNjVodq+LREeoPhXt57KJAp8vvQV3WZhMTtafDwRbcrFont+MAXge0NJz+hKCAmneDkAFa8nM3tBJp10zBwMu5YSi96/0nInJaRbPvSSRg7HTICxY4qvnXiG+nJzp0xri03Z713vdIEAHlHygJRPrezcphBkX8X1c7LVNzas+5+xXpwTZBFHtPrXZa43ToDk7HY6eN8YwET6Ak/ycEGKpAvDJe18Si83Tq725wEQCo39TOIeIfD9HMWNp0F0dwjDh6AyY87pPttITJho4Md1se X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM4PR12MB5229.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(366004)(2906002)(38100700002)(8676002)(186003)(4326008)(5660300002)(86362001)(6512007)(2616005)(8936002)(54906003)(66946007)(36756003)(31696002)(83380400001)(6916009)(26005)(53546011)(316002)(31686004)(66556008)(7406005)(7416002)(508600001)(6666004)(66476007)(6486002)(6506007)(45980500001)(43740500002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?eVVybjRlVENtRHMzL1ZzdWIrRVFMRjlBOGpyOVVJNXVQVEZHTFlDbXl0NW9i?= =?utf-8?B?ZnhlL1FlL0lrNUdqWGlqdnVid1VCRFc5aFdFVTJvZmpKenBlTEN0QzdGUUZZ?= =?utf-8?B?YTk2bjAxelBXbS9hK2k2YnE3ZmJiRVZFR2pYVjA3WXF1WksxaDh6SW1BeHlK?= =?utf-8?B?VllDSkZBaUw3YVpBRVJzOW0zdi9jaXBhcE5zNUZRcTd4TXdPd0Q2WnVPclBS?= =?utf-8?B?RXZSU2hiZ2NQMkRldGpkVitZWkVNVk1oWVh5djcvYzBIQ3A3c1RrZkIwUWpy?= =?utf-8?B?MGZSeGdOU0dGbWdpbTdyTEhRdWpzbTRUV25PZ0dCdkczZTBoN2ZRS0YrdGJj?= =?utf-8?B?WmRVV05HeVdOc2FkNi9HbzR6ZkRzdWJndWUrUWJRc1hzd01NUHVMeTBjU2p6?= =?utf-8?B?YmJIUjVhNWV2SW8wSVBqY2VrMEJNaEtpVHYxMTN6RENkRC9mQXUvL2J5UzdK?= =?utf-8?B?N25GaUlqZ21FYi9BOTQyN1B0alNJcXZoK1Fpd05Ma2FZK1ZlSU9lSCs3YkFW?= =?utf-8?B?SFdTYlExYTlhSDhZK3M1NGlENDB6Q0tmcDFtUjVEWDJLbFdqaUk0SjNKMllJ?= =?utf-8?B?cmViM1FpMTdRL0QvM3craFM0d1lLcW4wcXJrQWtOMDVYT2RER3hpOHJxcXhF?= =?utf-8?B?SExIRExkVGZvTExsRCtac2h1YkdSbkZLcHNnY2dhdWlqNGJRTlNwRStaV3F3?= =?utf-8?B?alJJNFZyOUpQdlBzaSt3Slc1aXNOY3JFRzVodlJNdDVJdVJSdTBNNjJhdVNo?= =?utf-8?B?N3grYmNJa2xNblNuYlQvRkI3OHNIMGQ5VzYycXcxcmlNc0IwUVFvNEJSKzRh?= =?utf-8?B?YTcxdXVKR01RS1dwcDdnQVZ2NlM2YzZTM2xRSzZMYnNvcTN3SkhoTHNGbnFs?= =?utf-8?B?Y0tIclBIbzRtZms3SHZldkVVSXM2NEp1Sm9SSzVQL3kvRnRRVExKZ2tidEdw?= =?utf-8?B?UzhsbDhIbUtseDc5NWEvRTRlKy9lRjd0TDV5bzVmRDlvQ3d6NUFNcWpBZjRV?= =?utf-8?B?bm9jMWlXaEZxVVRGR0p1bE5DOEZlWldFaGFBSnRJUnhPSFZzbTYvN0QrdVo4?= =?utf-8?B?b3VyQktpNW1hTmZHU0dNaDRPa3RqM0w0VWU0dVI5THNxd3BHanBJcXNHRGtn?= =?utf-8?B?RTUxVGp4WXJNeUV4UllnbnFTUGhRdkxvUVhBMVJVV1ZKTDUrOXNuRnd4SmdL?= =?utf-8?B?bVRhbWJsbVgwVlV4TWRSL0JSRW4vZTcwKzZUaWV5WTdjR0ZPQnBXL2dTOW5s?= =?utf-8?B?cVAwelFNcFFFVkhCczNPT3NRbUVwaFJXQ0xFY09jVThvR1VRcWhWbzZNamlP?= =?utf-8?B?Mi9OY1AzbWpUSHFMek9US2JHNzNIS3dUeCtiSlMzazVCZDF2WitucEt4d3FR?= =?utf-8?B?bE5KRmI1NFRoYlhDYU9aOURwVk5jcGJWQTJkQXFlWUxhcys5enpPVVYrS1p0?= =?utf-8?B?cUpCRGpnOW1VOGxDczJtVjdUejNETGVPSS9DMjNuQkpkTVpIMisyUTRXSTFm?= =?utf-8?B?ZFIxdGZRR2JYeWlKd0JqNTVZRU9uMHF4R0NZT0YyOVVzSlJOaDVrMnRDekx2?= =?utf-8?B?T2hKSEJIdGxQWGF5SEIyakY5bElLNGdKTmlaczN2akRiYkRZYU8rbi9qKzIr?= =?utf-8?B?bXVBVVVqWWJRd2t6Mm8ySjFYL2dVSG9zbGZHOG5oRDQvL3F1emEyNU1ESkxL?= =?utf-8?B?S0FUWFZ1eVdUenpNVk9ua2VEQ2pvVnl6WW1mejRZUnpwekhxZ1FKTVVJK1pz?= =?utf-8?B?YnRkV2g1RnFBcGI1VTVhK2FySlRTUnFxZDBVbzgvSXlaRk91bVdudkQxTURF?= =?utf-8?B?ZEJGeStBSVRldE11U0FtY3hhU3JNWkpzVUNocnNHQlpSM3F6c05Sd0pNTjZT?= =?utf-8?B?NndxLzlzbE1UR0tRVThac1JJUDg2bWRKYmlCak16M29rYlYyTTllN2E0ODY2?= =?utf-8?B?NlJ6SjcvR1lSalArZWx5N1ZSb3dWaGkxekZySWVwb2JWNWxxcWIwaFlYalVT?= =?utf-8?B?L0JsWTVpcmplSy84QjVLcllKbFpjb1Q5TG9aZjRZQWNaTEdaUXF4YmJGditT?= =?utf-8?B?MzZwampLdFdaQ016cVlZdUV4eU9aUFlvazNhaXBDUjczK2U0bThiTEN3L0Fm?= =?utf-8?B?WDdNbFAveitXRUQxMGcyNGZsRGtGeHdjOU51TVBURzgrbWpEcTRTQlFnTXVY?= =?utf-8?Q?SihKnaCAJULa09ALyTGvpy4=3D?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2232576a-d440-4832-8c11-08d9d1563f0f X-MS-Exchange-CrossTenant-AuthSource: DM4PR12MB5229.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Jan 2022 20:51:00.6730 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 26eYZljRejg1wzq2R1hYQDqrfTLxj89YZ4DWrUINsD7c0lE4KCi8XpCFyB/aI3NyX51yiUXKcWCjYgVAKorbIQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5247 X-Rspamd-Queue-Id: E46B040002 X-Stat-Signature: 8jpu7gds3wxejxp5meq83cz7ucfxbdjw Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=rmA1fkE7; dmarc=pass (policy=quarantine) header.from=amd.com; spf=pass (imf12.hostedemail.com: domain of Thomas.Lendacky@amd.com designates 40.107.94.73 as permitted sender) smtp.mailfrom=Thomas.Lendacky@amd.com X-Rspamd-Server: rspam11 X-HE-Tag: 1641502235-876499 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 1/6/22 2:16 PM, Venu Busireddy wrote: > On 2022-01-06 13:06:13 -0600, Tom Lendacky wrote: >> On 1/6/22 11:40 AM, Venu Busireddy wrote: >>> On 2022-01-05 15:39:22 -0600, Brijesh Singh wrote: >>>> >>>> >>>> On 1/5/22 2:27 PM, Dave Hansen wrote: >>>>> On 1/5/22 11:52, Brijesh Singh wrote: >>>>>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 for (; = vaddr < vaddr_end; vaddr +=3D PMD_SIZE) { >>>>>>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 /* >>>>>>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 * When SEV-SNP is active then transition the >>>>>>>> page to shared in the RMP >>>>>>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 * table so that it is consistent with the page >>>>>>>> table attribute change. >>>>>>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 */ >>>>>>>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 early_snp_set_memory_shared(__pa(vaddr), >>>>>>>> __pa(vaddr), PTRS_PER_PMD); >>>>>>> >>>>>>> Shouldn't the first argument be vaddr as below? >>>>>> >>>>>> Nope, sme_postprocess_startup() is called while we are fixing the >>>>>> initial page table and running with identity mapping (so va =3D=3D= pa). >>>>> >>>>> I'm not sure I've ever seen a line of code that wanted a comment so= badly. >>>> >>>> The early_snp_set_memory_shared() call the PVALIDATE instruction to = clear >>>> the validated bit from the BSS region. The PVALIDATE instruction nee= ds a >>>> virtual address, so we need to use the identity mapped virtual addre= ss so >>>> that PVALIDATE can clear the validated bit. I will add more comments= to >>>> clarify it. >>> >>> Looking forward to see your final comments explaining this. I can't >>> still follow why, when PVALIDATE needs the virtual address, we are do= ing >>> a __pa() on the vaddr. >> >> It's because of the phase of booting that the kernel is in. At this po= int, >> the kernel is running in identity mapped mode (VA =3D=3D PA). The >> __start_bss_decrypted address is a regular kernel address, e.g. for th= e >> kernel I'm on it is 0xffffffffa7600000. Since the PVALIDATE instructio= n >> requires a valid virtual address, the code needs to perform a __pa() a= gainst >> __start_bss_decrypted to get the identity mapped virtual address that = is >> currently in place. >=20 > Perhaps my confusion stems from the fact that __pa(x) is defined eithe= r > as "((unsigned long ) (x))" (for the cases where paddr and vaddr are > same), or as "__phys_addr((unsigned long )(x))", where a vaddr needs to > be converted to a paddr. If the paddr and vaddr are same in our case, > what exactly is the _pa(vaddr) doing to the vaddr? But they are not the same and the head64.c file is compiled without=20 defining a value for __pa(), so __pa() is __phys_addr((unsigned long)(x))= .=20 The virtual address value of __start_bss_decrypted, for me, is=20 0xffffffffa7600000, and that does not equal the physical address (take a=20 look at your /proc/kallsyms). However, since the code is running identity= =20 mapped and with a page table without kernel virtual addresses, it cannot=20 use that value. It needs to convert that value to the identity mapped=20 virtual address and that is done using __pa(). Only after using __pa() on= =20 __start_bss_decrypted, do you get a virtual address that maps to and is=20 equal to the physical address. You may want to step through the boot code using KVM to see what the=20 environment is and why things are done the way they are. Thanks, Tom >=20 > Venu >=20