From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 90C8BEB64DB for ; Wed, 14 Jun 2023 16:09:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1DA828E0002; Wed, 14 Jun 2023 12:09:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 18B998E0001; Wed, 14 Jun 2023 12:09:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 053258E0002; Wed, 14 Jun 2023 12:09:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id E917E8E0001 for ; Wed, 14 Jun 2023 12:09:26 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 938661205D0 for ; Wed, 14 Jun 2023 16:09:26 +0000 (UTC) X-FDA: 80901838332.21.3AB0EDE Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf05.hostedemail.com (Postfix) with ESMTP id A568E100007 for ; Wed, 14 Jun 2023 16:09:24 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=H1w2SMZE; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf05.hostedemail.com: domain of lee@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=lee@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686758964; a=rsa-sha256; cv=none; b=WD5iJWwuWmhl6vhvhZKo2RAndWdd97Xw7hKHtTU+RnDfhM6WoChUEW3M79dq1eoyIukq6B WUQFxLmeRxgrLFNkt9cN0Pv9iCKWOpHpjTbdcXn62NWPzD1Rx1ey0IsHyM4TVnK2r84kuW p/9YEoofb8BsXAIuvr1Qrq9sdAXjXyw= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=H1w2SMZE; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf05.hostedemail.com: domain of lee@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=lee@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686758964; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7DzhYic9znDIyhcNRn2U2jQtP8TF5XbQnahDYPEe1lc=; b=sEkWAAPUWGUjqYl6Ns9WyWpmLVrwJWuEYWvprwf+YYccxfjbHVmBUD5pSQR/EIWY/dy+xr kFQlmEivl/WLXn106OrzL9npkrOJX8Vu6ljE0k+TxxChCzDhI8djRsNTaLJs61jKUlPv7z kpwlZOwQfN1bY0u6gcqVuV16uruyCgc= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id BFC0B64437; Wed, 14 Jun 2023 16:09:23 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AC839C433C0; Wed, 14 Jun 2023 16:09:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1686758962; bh=z4EO55OEov8HV8Wooyvn+q5dcyjiojnBToINyf+P/T8=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=H1w2SMZEqvPgnn1xshZ8ykXfYKbNxrLz/tdiJ3WsRf/xUzq6E3+WaDmKvihJeMIXX 2CNGO1JXYmjZdEZg2JgfbSaeIHsK0I0eTjNUo5N5UoTUUKgJ+b5uA9QVNMMRi1bsvX pGLjXv7MOFt/ygIDdqZDXZlPcERrTbJlgkMCzRwDOS7vPkc1JW+Z97Blxib/WwWeVV rrVdXSP99GmRxaerC7hZ5YrJqLwEZDBLe+AbIuJYkODjMYkjxqhU9TajFMWWKWurHu 9Ruy4U2yX3dD+KZgLBoRtEyWq7NKHgbkZx4lrw4ucCwLPRNkPBPoxB6YDyc/NMqral 5YWXemSe7/cyA== Date: Wed, 14 Jun 2023 17:09:17 +0100 From: Lee Jones To: Dave Hansen Cc: Peter Zijlstra , Dave Hansen , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , x86@kernel.org, "H. Peter Anvin" , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: x86: pgtable / kaslr initialisation (OOB) help Message-ID: <20230614160917.GA3635807@google.com> References: <20230614132339.GS3635807@google.com> <20230614141654.GA1640123@hirez.programming.kicks-ass.net> <20230614143732.GW3635807@google.com> <0cefb67a-6fae-daa2-c871-ae35b96aac08@intel.com> <20230614150615.GX3635807@google.com> <20230614151003.GY3635807@google.com> <20230614152632.GZ3635807@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: A568E100007 X-Stat-Signature: 6uj6ntrmtatjrbwy39g1mstzo8bc6ucn X-HE-Tag: 1686758964-117147 X-HE-Meta: U2FsdGVkX1+YZjGMmX5T7hzTZw228hGo+gMBgaLLtfXTN7f3++sb6srGoCMpK4IHmEp5mZghJawQfnE4ON1HR/TKYdZzzHI9YG8widZ4WwGkg6nsfHHLGxAykGO6PKEgaoCKYbSJq94QfR03DR8YJXMKJlAkT81icBel9LfmmjTa2l+CVgcKwpannULytWpX62+kLiXDNRvJK9L9TZZbVw2eQ5p3eydXwUFIPdLel9hcjZSALcY0LQsIzZSNLeIts5wv8E8GN1GUj2RHUYTBpJJ8kdxcnv0bEKYl4pNUzlVUbC0vPokzjRgwe8ZqH8sio4r7udmA4tRm5KUVmgxG3xr3viIJVPvZVzKERqIQKEMI9SZeMdQQuRCL9rP8OTeCJgWRq+Gv8ItUIVw6zE6GtY5pmXoeFlLsoNFEb/kollnbPyZYyUm/OrpmUXynuyvfRe+xo0Fnxn49xVRKXabUgVW0AraOANAhpTHU9I7pWquyIjfYU23t/68IfOg6WAbNkiAfVBH5MRUpgQiscVvy/PULO9SEtQ9gkO7tRQIOb5w+Dox/w0uabUlzTdBb2UdIMPaGBTPJ4PmyRxdT1VbELtdejLdxiSTJ+7hrRr49XcjL7WMQ3xZQCL1+hz7O45c/zHgz4gGL/4SDx0FwVJTY9EXje99mas1nTK8hIzsUWDqQ1yCNfQZJoKguxRbnwSSTxOkhIvaiEdb3v4w1BDLqmu5skiFOnnGCqZ2R4FaQy70r8oGNMkx3fMGto3TBOA8ct6wE3QhFBgzf0o8xk9/MzFUZKghtZmViQo2VvlFL0SqAWmzeExJ33wEDIh+29ZpKpmcQhU9g8rtuOo75OC/LN0Tx39wJxbOLfN2gRuScWVFAfv+5eubWoFLL2OTJQWf44YDSWLPxjdPyimFxTXNiTk4DhDNtg0YZjQ2nmeua8KnW9PWLD1JaTS4CZJZuLD9j2xE9tOilrV43YbQnYe4 5meA5RD2 GhlV/6yiqA1RjO3wAKMefkNLyORxXlSvpYbYWh3iegc7CFf9F2ugSX9Bq+4oNr5fvtRZHqTpOBC+VOwb6lmxYi0xrDdLblJDPxyJRFPZGDO3gPuDvdYJu8ldR6rFwHYH22T0/EV94q7JsAE8lg9gxMdMdFzxqGHzykvhWbYsKRA9zssRO+Ma8ll6FZXt2mB38CTm7pyW7eh7l29cX1cRTTo8hj+L9PQqBWE8Kn9q58L7AsYUNNomn7+gSYwILSVW9RTgP X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, 14 Jun 2023, Dave Hansen wrote: > On 6/14/23 08:26, Lee Jones wrote: > > On Wed, 14 Jun 2023, Lee Jones wrote: > > > >> On Wed, 14 Jun 2023, Lee Jones wrote: > >> > >>> Thanks for chiming in Dave. I hoped you would. > >>> > >>> On Wed, 14 Jun 2023, Dave Hansen wrote: > >>> > >>>> On 6/14/23 07:37, Lee Jones wrote: > >>>>> Still unsure how we (the kernel) can/should write to an area of memory > >>>>> that does not belong to it. Should we allocate enough memory > >>>>> (2*PAGE_SIZE? rather than 8-Bytes) for trampoline_pgd_entry to consume > >>>>> in a more sane way? > >>>> > >>>> No. > >>>> > >>>> I think this: > >>>> > >>>> set_pgd(&trampoline_pgd_entry, > >>>> __pgd(_KERNPG_TABLE | __pa(p4d_page_tramp))); > >>>> > >>>> is bogus-ish. set_pgd() wants to operate on a pgd_t inside a pgd > >>>> *PAGE*. But it's just being pointed at a single _entry_. The address > >>>> of 'trampoline_pgd_entry' in your case also just (unfortunately) > >>>> happens to pass the: > >>>> > >>>> __pti_set_user_pgtbl -> pgdp_maps_userspace() > >>>> > >>>> test. I _think_ we want these to just be something like: > >>>> > >>>> trampoline_pgd_entry = __pgd(_KERNPG_TABLE | > >>>> __pa(p4d_page_tramp); > >>>> > >>>> That'll keep us away from all of the set_pgd()-induced nastiness. > >>> > >>> Okay. Is this what you're suggesting? > >>> > >>> diff --git a/arch/x86/mm/kaslr.c b/arch/x86/mm/kaslr.c v > >>> index d336bb0cb38b..803595c7dcc8 100644 > >>> --- a/arch/x86/mm/kaslr.c > >>> +++ b/arch/x86/mm/kaslr.c > >>> @@ -176,7 +176,7 @@ void __meminit init_trampoline_kaslr(void) > >>> set_pgd(&trampoline_pgd_entry, > >>> __pgd(_KERNPG_TABLE | __pa(p4d_page_tramp))); > >>> } else { > >>> - set_pgd(&trampoline_pgd_entry, > >>> - __pgd(_KERNPG_TABLE | __pa(pud_page_tramp))); > >>> + trampoline_pgd_entry = > >>> + __pgd(_KERNPG_TABLE | __pa(p4d_page_tramp); > >> > >> Note the change of *.page_tramp here. > >> > >> s/pud/p4d/ > >> > >> I'm assuming that too was intentional? > > > > Never mind. I can see that p4d_page_tramp is local to the if() segment. > > > > While we're at it, does the if() segment look correct to you: > > > > if (pgtable_l5_enabled()) { > > p4d_page_tramp = alloc_low_page(); > > > > p4d_tramp = p4d_page_tramp + p4d_index(paddr); > > > > set_p4d(p4d_tramp, > > __p4d(_KERNPG_TABLE | __pa(pud_page_tramp))); > > > > set_pgd(&trampoline_pgd_entry, > > __pgd(_KERNPG_TABLE | __pa(p4d_page_tramp))); > > } else { > > trampoline_pgd_entry = > > __pgd(_KERNPG_TABLE | __pa(pud_page_tramp)); > > } > > > > - pud_page_tramp is being passed to set_p4d() > > - p4d_page_tramp is being passed to set_pgd() > > > > Should those be the other way around, or am I missing the point? > > You're missing the point. :) Super, thanks for the explanation. > PGDs are always set up to point to the physical address of the thing at > one lower level than them. A page is allocated for that level when > 5-level paging is in play. No page is needed when it is not in play. > > The pattern is _almost_ always > > pgd = ... __pa(p4d); > > In other words, point the PGD at the physical address of a p4d. But > things get funky on systems without p4ds, thus the special casing here. > > Does the (completely untested) attached patch fix your problem? I just submitted a (tested) patch. It doesn't cover the if() segment though. I'll do so and resubmit. > --- > > b/arch/x86/mm/kaslr.c | 8 ++++---- > 1 file changed, 4 insertions(+), 4 deletions(-) > > diff -puN arch/x86/mm/kaslr.c~trampoline_pgd_entry arch/x86/mm/kaslr.c > --- a/arch/x86/mm/kaslr.c~trampoline_pgd_entry 2023-06-14 08:54:08.685554094 -0700 > +++ b/arch/x86/mm/kaslr.c 2023-06-14 08:55:36.077089793 -0700 > @@ -172,10 +172,10 @@ void __meminit init_trampoline_kaslr(voi > set_p4d(p4d_tramp, > __p4d(_KERNPG_TABLE | __pa(pud_page_tramp))); > > - set_pgd(&trampoline_pgd_entry, > - __pgd(_KERNPG_TABLE | __pa(p4d_page_tramp))); > + trampoline_pgd_entry = > + __pgd(_KERNPG_TABLE | __pa(p4d_page_tramp)); > } else { > - set_pgd(&trampoline_pgd_entry, > - __pgd(_KERNPG_TABLE | __pa(pud_page_tramp))); > + trampoline_pgd_entry = > + __pgd(_KERNPG_TABLE | __pa(pud_page_tramp)); > } > } > _ -- Lee Jones [李琼斯]