From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E14E9C433E0 for ; Tue, 2 Jun 2020 23:18:56 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 76D542054F for ; Tue, 2 Jun 2020 23:18:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=shutemov-name.20150623.gappssmtp.com header.i=@shutemov-name.20150623.gappssmtp.com header.b="YxsAHw4o" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 76D542054F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=shutemov.name Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CC1CF80007; Tue, 2 Jun 2020 19:18:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C72338E0006; Tue, 2 Jun 2020 19:18:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B3AF480007; Tue, 2 Jun 2020 19:18:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0053.hostedemail.com [216.40.44.53]) by kanga.kvack.org (Postfix) with ESMTP id 9DBC88E0006 for ; Tue, 2 Jun 2020 19:18:55 -0400 (EDT) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 6271B180AD804 for ; Tue, 2 Jun 2020 23:18:55 +0000 (UTC) X-FDA: 76885839030.23.game95_880c1641a0750 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin23.hostedemail.com (Postfix) with ESMTP id 4623537606 for ; Tue, 2 Jun 2020 23:18:55 +0000 (UTC) X-HE-Tag: game95_880c1641a0750 X-Filterd-Recvd-Size: 6331 Received: from mail-lf1-f66.google.com (mail-lf1-f66.google.com [209.85.167.66]) by imf05.hostedemail.com (Postfix) with ESMTP for ; Tue, 2 Jun 2020 23:18:54 +0000 (UTC) Received: by mail-lf1-f66.google.com with SMTP id x27so96646lfg.9 for ; Tue, 02 Jun 2020 16:18:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=shutemov-name.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=w339YpBFgV7tqQf8M7YphxkuH2MrbkG2i3IWInNVEFw=; b=YxsAHw4owitymwvHIwpqwWvn9/NXfSmSAaW4XnOP3SsDR0q/tu0Kjtftp1ifdfbfTc db7b0ZQb7XugZtiYeszap76BEZuj9SR7sEGdl+KzdbvXUErm382A/PEE9pQKqw2RCHW+ fxt3fniEs00mZZGfE4SRefURbUpvaDWjfMSwtxss/P7Hu5hOCAebKrwvL198wVJLAScc 4x1kUwZY1D43yHWgBhNeQakXS1vaWEODvIRwRn+skTC2lMBjsFPh6jtE5RpKnVCGheKI hxnCOSxuHeJWxmzn1wKje/9Ngwo+exug6CDE81AlefvydqUyh2OSPjhGhkcoCl5Hkqrs Ktbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=w339YpBFgV7tqQf8M7YphxkuH2MrbkG2i3IWInNVEFw=; b=cpfUw39fQ20+0Z+2zDXYXMOLAWF0YE4TBk2x9LHVIk2qkw/1Cfp6BhYuMyIgchKZpf 26NtQSaoWkX3FKWz8EvT12jDogfJJ70ToE2Y3n7WA0n9mZixSAYwaU7wcjKpiNOM7bjP kvC0EeATAcQkT+DJ9MRgbMP8f4rN/QjNl5N1/9z4oNtTSoGXoLae73IXeI0PitUvWV9A HmY9OAAiIieAFCGIa/Tt/PhQG1WEDyBQj9+/MGNTKRgFiKHL/WuvALWZF/2kUxfMMDPR RVZPanPq8LPJcvrcNW714xEgSKZrhEd/yFty4VLFKoRheRQAnEm3Y0FJKrq6ekTzqPDz jsXA== X-Gm-Message-State: AOAM5325HWeTDH/XCPpE8HdGaXB7gx4jm647sTWTv25md34iX9I94+tw notRquuXFdH28yapGreJVti57g== X-Google-Smtp-Source: ABdhPJxwAotMVS+UbqsYp+bbhka8nGqAjJAI+8M3Prj7QP/OP9DS5g/xQ3THv/MFg0EiknvWEapPfQ== X-Received: by 2002:ac2:550a:: with SMTP id j10mr860697lfk.46.1591139933189; Tue, 02 Jun 2020 16:18:53 -0700 (PDT) Received: from box.localdomain ([86.57.175.117]) by smtp.gmail.com with ESMTPSA id 15sm63048ljr.104.2020.06.02.16.18.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jun 2020 16:18:51 -0700 (PDT) Received: by box.localdomain (Postfix, from userid 1000) id 87BFB102780; Wed, 3 Jun 2020 02:18:57 +0300 (+03) Date: Wed, 3 Jun 2020 02:18:57 +0300 From: "Kirill A. Shutemov" To: Dave Hansen Cc: Mike Rapoport , "Kirill A. Shutemov" , Dave Hansen , Andy Lutomirski , Peter Zijlstra , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , Dan Williams , Tony Luck , x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, stable@vger.kernel.org Subject: Re: [PATCH] x86/mm: Fix boot with some memory above MAXMEM Message-ID: <20200602231857.ousba2xiks7myxbt@box> References: <20200511191721.1416-1-kirill.shutemov@linux.intel.com> <20200525044902.rsb46bxu5hdsqglt@box> <20200525145943.GA13247@kernel.org> <20200525150820.zljiamptpzi37ohx@box> <24b51944-bfba-a937-484a-5d9ec54fdf01@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <24b51944-bfba-a937-484a-5d9ec54fdf01@intel.com> X-Rspamd-Queue-Id: 4623537606 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam02 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, May 26, 2020 at 07:27:15AM -0700, Dave Hansen wrote: > On 5/25/20 8:08 AM, Kirill A. Shutemov wrote: > >>>> + if (not_addressable) { > >>>> + pr_err("%lldGB of physical memory is not addressable in the paging mode\n", > >>>> + not_addressable >> 30); > >>>> + if (!pgtable_l5_enabled()) > >>>> + pr_err("Consider enabling 5-level paging\n"); > >> Could this happen at all when l5 is enabled? > >> Does it mean we need kmap() for 64-bit? > > It's future-profing. Who knows what paging modes we would have in the > > future. > > Future-proofing and firmware-proofing. :) > > In any case, are we *really* limited to 52 bits of physical memory with > 5-level paging? Yes. It's architectural. SDM says "MAXPHYADDR is at most 52" (Vol 3A, 4.1.4). I guess it can be extended with an opt-in feature and relevant changes to page table structure. But as of today there's no such thing. > Previously, we said we were limited to 46 bits, and now > we're saying that the limit is 52 with 5-level paging: > > #define MAX_PHYSMEM_BITS (pgtable_l5_enabled() ? 52 : 46) > > The 46 was fine with the 48 bits of address space on 4-level paging > systems since we need 1/2 of the address space for userspace, 1/4 for > the direct map and 1/4 for the vmalloc-and-friends area. At 46 bits of > address space, we fill up the direct map. > > The hardware designers know this and never enumerated a MAXPHYADDR from > CPUID which was higher than what we could cover with 46 bits. It was > nice and convenient that these two separate things matched: > 1. The amount of physical address space addressable in a direct map > consuming 1/4 of the virtual address space. > 2. The CPU-enumerated MAXPHYADDR which among other things dictates how > much physical address space is addressable in a PTE. > > But, with 5-level paging, things are a little different. The limit in > addressable memory because of running out of the direct map actually > happens at 55 bits (57-2=55, analogous to the 4-level 48-2=46). > > So shouldn't it technically be this: > > #define MAX_PHYSMEM_BITS (pgtable_l5_enabled() ? 55 : 46) > > ? Bits above 52 are ignored in the page table entries and accessible to software. Some of them got claimed by HW features (XD-bit, protection keys), but such features require explicit opt-in on software side. Kernel could claim bits 53-55 for the physical address, but it doesn't get us anything: if future HW would provide such feature it would require opt-in. On other hand claiming them now means we cannot use them for other purposes as SW bit. I don't see a point. -- Kirill A. Shutemov