From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ob0-f171.google.com (mail-ob0-f171.google.com [209.85.214.171]) by kanga.kvack.org (Postfix) with ESMTP id A68246B006C for ; Wed, 4 Mar 2015 16:17:02 -0500 (EST) Received: by obbgq1 with SMTP id gq1so9786098obb.2 for ; Wed, 04 Mar 2015 13:17:02 -0800 (PST) Received: from g2t2353.austin.hp.com (g2t2353.austin.hp.com. [15.217.128.52]) by mx.google.com with ESMTPS id z1si2832673obg.51.2015.03.04.13.17.01 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 04 Mar 2015 13:17:02 -0800 (PST) Message-ID: <1425503783.17007.263.camel@misato.fc.hp.com> Subject: Re: [PATCH v3 6/6] x86, mm: Support huge KVA mappings on x86 From: Toshi Kani Date: Wed, 04 Mar 2015 14:16:23 -0700 In-Reply-To: <20150304201748.GA6634@gmail.com> References: <1425404664-19675-1-git-send-email-toshi.kani@hp.com> <1425404664-19675-7-git-send-email-toshi.kani@hp.com> <20150303144414.9f97ef25ad8aed7d112896bf@linux-foundation.org> <1425424472.17007.191.camel@misato.fc.hp.com> <20150303170035.85e94c87.akpm@linux-foundation.org> <1425486216.17007.236.camel@misato.fc.hp.com> <20150304201748.GA6634@gmail.com> Content-Type: text/plain; charset="UTF-8" Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: owner-linux-mm@kvack.org List-ID: To: Ingo Molnar Cc: Andrew Morton , "hpa@zytor.com" , "tglx@linutronix.de" , "mingo@redhat.com" , "arnd@arndb.de" , "linux-mm@kvack.org" , "x86@kernel.org" , "linux-kernel@vger.kernel.org" , "dave.hansen@intel.com" , "Elliott, Robert (Server Storage)" On Wed, 2015-03-04 at 21:17 +0100, Ingo Molnar wrote: > * Toshi Kani wrote: > > > On Wed, 2015-03-04 at 01:00 +0000, Andrew Morton wrote: > > > On Tue, 03 Mar 2015 16:14:32 -0700 Toshi Kani wrote: > > > > > > > On Tue, 2015-03-03 at 14:44 -0800, Andrew Morton wrote: > > > > > On Tue, 3 Mar 2015 10:44:24 -0700 Toshi Kani wrote: > > > > : > > > > > > + > > > > > > +#ifdef CONFIG_HAVE_ARCH_HUGE_VMAP > > > > > > +int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot) > > > > > > +{ > > > > > > + u8 mtrr; > > > > > > + > > > > > > + /* > > > > > > + * Do not use a huge page when the range is covered by non-WB type > > > > > > + * of MTRRs. > > > > > > + */ > > > > > > + mtrr = mtrr_type_lookup(addr, addr + PUD_SIZE); > > > > > > + if ((mtrr != MTRR_TYPE_WRBACK) && (mtrr != 0xFF)) > > > > > > + return 0; > > > > > > > > > > It would be good to notify the operator in some way when this happens. > > > > > Otherwise the kernel will run more slowly and there's no way of knowing > > > > > why. I guess slap a pr_info() in there. Or maybe pr_warn()? > > > > > > > > We only use 4KB mappings today, so this case will not make it run > > > > slowly, i.e. it will be the same as today. > > > > > > Yes, but it would be slower than it would be if the operator fixed the > > > mtrr settings! How do we let the operator know this? > > > > > > > Also, adding a message here > > > > can generate a lot of messages when MTRRs cover a large area. > > > > > > Really? This is only going to happen when a device driver > > > requests a huge io mapping, isn't it? That's rare. We could emit > > > a warning, return an error code and fall all the way back to the > > > top-level ioremap code which can then retry with 4k mappings. Or > > > something similar - somehow record the fact that this warning has > > > been emitted or use printk ratelimiting (bad option). > > > > Yes, an IO device with a huge MMIO space that is covered by MTRRs is > > a rare case. BIOS does not need to specify how MMIO of each card > > needs to be accessed with MTRRs (or BIOS should not do it since an > > MMIO address is configurable on each card). > > > > However, PCIe has the MMCONFIG space, PCIe config space, which is > > also memory mapped and must be accessed with UC. The PCI subsystem > > calls ioremap_nocache() to map the entire MMCONFIG space, which > > covers the PCIe config space of all possible cards. Here are boot > > messages on my test system. > > > > : > > PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xc0000000-0xcf > > ffffff] (base 0xc0000000) > > PCI: MMCONFIG at [mem 0xc0000000-0xcfffffff] reserved in E820 > > : > > > > And MTRRs cover this MMCONFIG space with UC to assure that the range is > > always accessed with UC. > > So the PCI code ioremap()s this 256 MB mmconfig space in its entirety > currently? Yes. > > # cat /proc/mtrr > > reg00: base=0x0c0000000 ( 3072MB), size= 1024MB, count=1: uncachable > > > > So, if we add a message into the code, it will be displayed many > > times in this ioremap_nocache() call from PCI. > > So, in this specific case, when a single MTRR covers it with a single > cache policy, I think we can safely map it UC using hugepmds? Yes. > That will 'shut up' the warning the right way: by making the code > work? I see your point. I will look into mtrr_type_lookup() to see if we can make it work in a manageable way. Thanks, -Toshi -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org