* [PATCH v2 1/4] mm, x86: Document return values of mapping funcs
2015-03-12 17:18 [PATCH v2 0/4] mtrr, mm, x86: Enhance MTRR checks for huge I/O mapping Toshi Kani
@ 2015-03-12 17:18 ` Toshi Kani
2015-03-12 17:18 ` [PATCH v2 2/4] mtrr, x86: Fix MTRR lookup to handle inclusive entry Toshi Kani
` (2 subsequent siblings)
3 siblings, 0 replies; 7+ messages in thread
From: Toshi Kani @ 2015-03-12 17:18 UTC (permalink / raw)
To: akpm, hpa, tglx, mingo
Cc: linux-mm, x86, linux-kernel, dave.hansen, Elliott, pebolle, Toshi Kani
Document the return values of KVA mapping functions,
pud_set_huge(), pmd_set_huge, pud_clear_huge() and
pmd_clear_huge().
Simplify the conditions to select HAVE_ARCH_HUGE_VMAP
in the Kconfig, since X86_PAE depends on X86_32.
There is no functional change in this patch.
Signed-off-by: Toshi Kani <toshi.kani@hp.com>
---
arch/x86/Kconfig | 2 +-
arch/x86/mm/pgtable.c | 36 ++++++++++++++++++++++++++++--------
2 files changed, 29 insertions(+), 9 deletions(-)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 110f6ae..ba5e78e 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -99,7 +99,7 @@ config X86
select IRQ_FORCED_THREADING
select HAVE_BPF_JIT if X86_64
select HAVE_ARCH_TRANSPARENT_HUGEPAGE
- select HAVE_ARCH_HUGE_VMAP if X86_64 || (X86_32 && X86_PAE)
+ select HAVE_ARCH_HUGE_VMAP if X86_64 || X86_PAE
select ARCH_HAS_SG_CHAIN
select CLKEVT_I8253
select ARCH_HAVE_NMI_SAFE_CMPXCHG
diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c
index 0b97d2c..4891fa1 100644
--- a/arch/x86/mm/pgtable.c
+++ b/arch/x86/mm/pgtable.c
@@ -563,14 +563,19 @@ void native_set_fixmap(enum fixed_addresses idx, phys_addr_t phys,
}
#ifdef CONFIG_HAVE_ARCH_HUGE_VMAP
+/**
+ * pud_set_huge - setup kernel PUD mapping
+ *
+ * MTRR can override PAT memory types with 4KB granularity. Therefore,
+ * it does not set up a huge page when the range is covered by a non-WB
+ * type of MTRR. 0xFF indicates that MTRR are disabled.
+ *
+ * Return 1 on success, and 0 when no PUD was set.
+ */
int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot)
{
u8 mtrr;
- /*
- * Do not use a huge page when the range is covered by non-WB type
- * of MTRRs.
- */
mtrr = mtrr_type_lookup(addr, addr + PUD_SIZE);
if ((mtrr != MTRR_TYPE_WRBACK) && (mtrr != 0xFF))
return 0;
@@ -584,14 +589,19 @@ int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot)
return 1;
}
+/**
+ * pmd_set_huge - setup kernel PMD mapping
+ *
+ * MTRR can override PAT memory types with 4KB granularity. Therefore,
+ * it does not set up a huge page when the range is covered by a non-WB
+ * type of MTRR. 0xFF indicates that MTRR are disabled.
+ *
+ * Return 1 on success, and 0 when no PMD was set.
+ */
int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot)
{
u8 mtrr;
- /*
- * Do not use a huge page when the range is covered by non-WB type
- * of MTRRs.
- */
mtrr = mtrr_type_lookup(addr, addr + PMD_SIZE);
if ((mtrr != MTRR_TYPE_WRBACK) && (mtrr != 0xFF))
return 0;
@@ -605,6 +615,11 @@ int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot)
return 1;
}
+/**
+ * pud_clear_huge - clear kernel PUD mapping when it is set
+ *
+ * Return 1 on success, and 0 when no PUD map was found.
+ */
int pud_clear_huge(pud_t *pud)
{
if (pud_large(*pud)) {
@@ -615,6 +630,11 @@ int pud_clear_huge(pud_t *pud)
return 0;
}
+/**
+ * pmd_clear_huge - clear kernel PMD mapping when it is set
+ *
+ * Return 1 on success, and 0 when no PMD map was found.
+ */
int pmd_clear_huge(pmd_t *pmd)
{
if (pmd_large(*pmd)) {
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 7+ messages in thread* [PATCH v2 2/4] mtrr, x86: Fix MTRR lookup to handle inclusive entry
2015-03-12 17:18 [PATCH v2 0/4] mtrr, mm, x86: Enhance MTRR checks for huge I/O mapping Toshi Kani
2015-03-12 17:18 ` [PATCH v2 1/4] mm, x86: Document return values of mapping funcs Toshi Kani
@ 2015-03-12 17:18 ` Toshi Kani
2015-03-12 17:18 ` [PATCH v2 3/4] mtrr, x86: Clean up mtrr_type_lookup() Toshi Kani
2015-03-12 17:18 ` [PATCH v2 4/4] mtrr, mm, x86: Enhance MTRR checks for KVA huge page mapping Toshi Kani
3 siblings, 0 replies; 7+ messages in thread
From: Toshi Kani @ 2015-03-12 17:18 UTC (permalink / raw)
To: akpm, hpa, tglx, mingo
Cc: linux-mm, x86, linux-kernel, dave.hansen, Elliott, pebolle, Toshi Kani
When an MTRR entry is inclusive to a requested range, i.e.
the start and end of the request are not within the MTRR
entry range but the range contains the MTRR entry entirely,
__mtrr_type_lookup() ignores such a case because both
start_state and end_state are set to zero.
This patch fixes the issue by adding a new flag, 'inclusive',
to detect the case. This case is then handled in the same
way as (!start_state && end_state).
Signed-off-by: Toshi Kani <toshi.kani@hp.com>
---
arch/x86/kernel/cpu/mtrr/generic.c | 17 +++++++++--------
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/arch/x86/kernel/cpu/mtrr/generic.c b/arch/x86/kernel/cpu/mtrr/generic.c
index 7d74f7b..a82e370 100644
--- a/arch/x86/kernel/cpu/mtrr/generic.c
+++ b/arch/x86/kernel/cpu/mtrr/generic.c
@@ -154,7 +154,7 @@ static u8 __mtrr_type_lookup(u64 start, u64 end, u64 *partial_end, int *repeat)
prev_match = 0xFF;
for (i = 0; i < num_var_ranges; ++i) {
- unsigned short start_state, end_state;
+ unsigned short start_state, end_state, inclusive;
if (!(mtrr_state.var_ranges[i].mask_lo & (1 << 11)))
continue;
@@ -166,15 +166,16 @@ static u8 __mtrr_type_lookup(u64 start, u64 end, u64 *partial_end, int *repeat)
start_state = ((start & mask) == (base & mask));
end_state = ((end & mask) == (base & mask));
+ inclusive = ((start < base) && (end > base));
- if (start_state != end_state) {
+ if ((start_state != end_state) || inclusive) {
/*
* We have start:end spanning across an MTRR.
- * We split the region into
- * either
- * (start:mtrr_end) (mtrr_end:end)
- * or
- * (start:mtrr_start) (mtrr_start:end)
+ * We split the region into either
+ * - start_state:1
+ * (start:mtrr_end) (mtrr_end:end)
+ * - end_state:1 or inclusive:1
+ * (start:mtrr_start) (mtrr_start:end)
* depending on kind of overlap.
* Return the type for first region and a pointer to
* the start of second region so that caller will
@@ -195,7 +196,7 @@ static u8 __mtrr_type_lookup(u64 start, u64 end, u64 *partial_end, int *repeat)
*repeat = 1;
}
- if ((start & mask) != (base & mask))
+ if (!start_state)
continue;
curr_match = mtrr_state.var_ranges[i].base_lo & 0xff;
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 7+ messages in thread* [PATCH v2 3/4] mtrr, x86: Clean up mtrr_type_lookup()
2015-03-12 17:18 [PATCH v2 0/4] mtrr, mm, x86: Enhance MTRR checks for huge I/O mapping Toshi Kani
2015-03-12 17:18 ` [PATCH v2 1/4] mm, x86: Document return values of mapping funcs Toshi Kani
2015-03-12 17:18 ` [PATCH v2 2/4] mtrr, x86: Fix MTRR lookup to handle inclusive entry Toshi Kani
@ 2015-03-12 17:18 ` Toshi Kani
2015-03-13 12:37 ` Ingo Molnar
2015-03-12 17:18 ` [PATCH v2 4/4] mtrr, mm, x86: Enhance MTRR checks for KVA huge page mapping Toshi Kani
3 siblings, 1 reply; 7+ messages in thread
From: Toshi Kani @ 2015-03-12 17:18 UTC (permalink / raw)
To: akpm, hpa, tglx, mingo
Cc: linux-mm, x86, linux-kernel, dave.hansen, Elliott, pebolle, Toshi Kani
MTRRs contain fixed and variable entries. mtrr_type_lookup()
may repeatedly call __mtrr_type_lookup() to handle a request
that overlaps with variable entries. However,
__mtrr_type_lookup() also handles the fixed entries and other
conditions, which do not have to be repeated. This patch moves
such code from __mtrr_type_lookup() to mtrr_type_lookup().
This patch also changes the 'else if (start < 0x1000000)',
which checks a fixed range but has an extra zero in the address,
to 'else' with no condition.
Lastly, the patch updates the function headers to clarify the
return values and output argument. It also updates comments to
clarify that the repeating is necessary to handle overlaps with
the default type, since overlaps with multiple entries alone
can be handled without such repeating.
There is no functional change in this patch.
Signed-off-by: Toshi Kani <toshi.kani@hp.com>
---
arch/x86/kernel/cpu/mtrr/generic.c | 102 +++++++++++++++++++-----------------
1 file changed, 53 insertions(+), 49 deletions(-)
diff --git a/arch/x86/kernel/cpu/mtrr/generic.c b/arch/x86/kernel/cpu/mtrr/generic.c
index a82e370..ef34a4f 100644
--- a/arch/x86/kernel/cpu/mtrr/generic.c
+++ b/arch/x86/kernel/cpu/mtrr/generic.c
@@ -102,12 +102,16 @@ static int check_type_overlap(u8 *prev, u8 *curr)
return 0;
}
-/*
- * Error/Semi-error returns:
- * 0xFF - when MTRR is not enabled
- * *repeat == 1 implies [start:end] spanned across MTRR range and type returned
- * corresponds only to [start:*partial_end].
- * Caller has to lookup again for [*partial_end:end].
+/**
+ * __mtrr_type_lookup - look up memory type in MTRR variable entries
+ *
+ * Return Value:
+ * memory type - Matched memory type or the default memory type (unmatched)
+ *
+ * Output Argument:
+ * repeat - Set to 1 when [start:end] spanned across MTRR range and type
+ * returned corresponds only to [start:*partial_end]. Caller has
+ * to lookup again for [*partial_end:end].
*/
static u8 __mtrr_type_lookup(u64 start, u64 end, u64 *partial_end, int *repeat)
{
@@ -116,42 +120,10 @@ static u8 __mtrr_type_lookup(u64 start, u64 end, u64 *partial_end, int *repeat)
u8 prev_match, curr_match;
*repeat = 0;
- if (!mtrr_state_set)
- return 0xFF;
-
- if (!mtrr_state.enabled)
- return 0xFF;
/* Make end inclusive end, instead of exclusive */
end--;
- /* Look in fixed ranges. Just return the type as per start */
- if (mtrr_state.have_fixed && (start < 0x100000)) {
- int idx;
-
- if (start < 0x80000) {
- idx = 0;
- idx += (start >> 16);
- return mtrr_state.fixed_ranges[idx];
- } else if (start < 0xC0000) {
- idx = 1 * 8;
- idx += ((start - 0x80000) >> 14);
- return mtrr_state.fixed_ranges[idx];
- } else if (start < 0x1000000) {
- idx = 3 * 8;
- idx += ((start - 0xC0000) >> 12);
- return mtrr_state.fixed_ranges[idx];
- }
- }
-
- /*
- * Look in variable ranges
- * Look of multiple ranges matching this address and pick type
- * as per MTRR precedence
- */
- if (!(mtrr_state.enabled & 2))
- return mtrr_state.def_type;
-
prev_match = 0xFF;
for (i = 0; i < num_var_ranges; ++i) {
unsigned short start_state, end_state, inclusive;
@@ -180,7 +152,8 @@ static u8 __mtrr_type_lookup(u64 start, u64 end, u64 *partial_end, int *repeat)
* Return the type for first region and a pointer to
* the start of second region so that caller will
* lookup again on the second region.
- * Note: This way we handle multiple overlaps as well.
+ * Note: This way we handle overlaps with multiple
+ * entries and the default type properly.
*/
if (start_state)
*partial_end = base + get_mtrr_size(mask);
@@ -209,21 +182,18 @@ static u8 __mtrr_type_lookup(u64 start, u64 end, u64 *partial_end, int *repeat)
return curr_match;
}
- if (mtrr_tom2) {
- if (start >= (1ULL<<32) && (end < mtrr_tom2))
- return MTRR_TYPE_WRBACK;
- }
-
if (prev_match != 0xFF)
return prev_match;
return mtrr_state.def_type;
}
-/*
- * Returns the effective MTRR type for the region
- * Error return:
- * 0xFF - when MTRR is not enabled
+/**
+ * mtrr_type_lookup - look up memory type in MTRR
+ *
+ * Return Values:
+ * memory type - The effective MTRR type for the region
+ * 0xFF - MTRR is disabled
*/
u8 mtrr_type_lookup(u64 start, u64 end)
{
@@ -231,12 +201,43 @@ u8 mtrr_type_lookup(u64 start, u64 end)
int repeat;
u64 partial_end;
+ if (!mtrr_state_set || !mtrr_state.enabled)
+ return 0xFF;
+
+ /* Look in fixed ranges. Just return the type as per start */
+ if (mtrr_state.have_fixed && (start < 0x100000)) {
+ int idx;
+
+ if (start < 0x80000) {
+ idx = 0;
+ idx += (start >> 16);
+ return mtrr_state.fixed_ranges[idx];
+ } else if (start < 0xC0000) {
+ idx = 1 * 8;
+ idx += ((start - 0x80000) >> 14);
+ return mtrr_state.fixed_ranges[idx];
+ } else {
+ idx = 3 * 8;
+ idx += ((start - 0xC0000) >> 12);
+ return mtrr_state.fixed_ranges[idx];
+ }
+ }
+
+ /*
+ * Look in variable ranges
+ * Look of multiple ranges matching this address and pick type
+ * as per MTRR precedence
+ */
+ if (!(mtrr_state.enabled & 2))
+ return mtrr_state.def_type;
+
type = __mtrr_type_lookup(start, end, &partial_end, &repeat);
/*
* Common path is with repeat = 0.
* However, we can have cases where [start:end] spans across some
- * MTRR range. Do repeated lookups for that case here.
+ * MTRR ranges and/or the default type. Do repeated lookups for
+ * that case here.
*/
while (repeat) {
prev_type = type;
@@ -247,6 +248,9 @@ u8 mtrr_type_lookup(u64 start, u64 end)
return type;
}
+ if (mtrr_tom2 && (start >= (1ULL<<32)) && (end < mtrr_tom2))
+ return MTRR_TYPE_WRBACK;
+
return type;
}
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH v2 3/4] mtrr, x86: Clean up mtrr_type_lookup()
2015-03-12 17:18 ` [PATCH v2 3/4] mtrr, x86: Clean up mtrr_type_lookup() Toshi Kani
@ 2015-03-13 12:37 ` Ingo Molnar
2015-03-13 13:53 ` Toshi Kani
0 siblings, 1 reply; 7+ messages in thread
From: Ingo Molnar @ 2015-03-13 12:37 UTC (permalink / raw)
To: Toshi Kani
Cc: akpm, hpa, tglx, mingo, linux-mm, x86, linux-kernel, dave.hansen,
Elliott, pebolle
* Toshi Kani <toshi.kani@hp.com> wrote:
> MTRRs contain fixed and variable entries. mtrr_type_lookup()
> may repeatedly call __mtrr_type_lookup() to handle a request
> that overlaps with variable entries. However,
> __mtrr_type_lookup() also handles the fixed entries and other
> conditions, which do not have to be repeated. This patch moves
> such code from __mtrr_type_lookup() to mtrr_type_lookup().
>
> This patch also changes the 'else if (start < 0x1000000)',
> which checks a fixed range but has an extra zero in the address,
> to 'else' with no condition.
>
> Lastly, the patch updates the function headers to clarify the
> return values and output argument. It also updates comments to
> clarify that the repeating is necessary to handle overlaps with
> the default type, since overlaps with multiple entries alone
> can be handled without such repeating.
>
> There is no functional change in this patch.
>
> Signed-off-by: Toshi Kani <toshi.kani@hp.com>
> ---
> arch/x86/kernel/cpu/mtrr/generic.c | 102 +++++++++++++++++++-----------------
> 1 file changed, 53 insertions(+), 49 deletions(-)
>
> diff --git a/arch/x86/kernel/cpu/mtrr/generic.c b/arch/x86/kernel/cpu/mtrr/generic.c
> index a82e370..ef34a4f 100644
> --- a/arch/x86/kernel/cpu/mtrr/generic.c
> +++ b/arch/x86/kernel/cpu/mtrr/generic.c
> @@ -102,12 +102,16 @@ static int check_type_overlap(u8 *prev, u8 *curr)
> return 0;
> }
>
> -/*
> - * Error/Semi-error returns:
> - * 0xFF - when MTRR is not enabled
> - * *repeat == 1 implies [start:end] spanned across MTRR range and type returned
> - * corresponds only to [start:*partial_end].
> - * Caller has to lookup again for [*partial_end:end].
> +/**
> + * __mtrr_type_lookup - look up memory type in MTRR variable entries
> + *
> + * Return Value:
> + * memory type - Matched memory type or the default memory type (unmatched)
> + *
> + * Output Argument:
> + * repeat - Set to 1 when [start:end] spanned across MTRR range and type
> + * returned corresponds only to [start:*partial_end]. Caller has
> + * to lookup again for [*partial_end:end].
> */
> static u8 __mtrr_type_lookup(u64 start, u64 end, u64 *partial_end, int *repeat)
> {
> @@ -116,42 +120,10 @@ static u8 __mtrr_type_lookup(u64 start, u64 end, u64 *partial_end, int *repeat)
> u8 prev_match, curr_match;
>
> *repeat = 0;
> - if (!mtrr_state_set)
> - return 0xFF;
> -
> - if (!mtrr_state.enabled)
> - return 0xFF;
>
> /* Make end inclusive end, instead of exclusive */
> end--;
>
> - /* Look in fixed ranges. Just return the type as per start */
> - if (mtrr_state.have_fixed && (start < 0x100000)) {
> - int idx;
> -
> - if (start < 0x80000) {
> - idx = 0;
> - idx += (start >> 16);
> - return mtrr_state.fixed_ranges[idx];
> - } else if (start < 0xC0000) {
> - idx = 1 * 8;
> - idx += ((start - 0x80000) >> 14);
> - return mtrr_state.fixed_ranges[idx];
> - } else if (start < 0x1000000) {
> - idx = 3 * 8;
> - idx += ((start - 0xC0000) >> 12);
> - return mtrr_state.fixed_ranges[idx];
> - }
> - }
> -
> - /*
> - * Look in variable ranges
> - * Look of multiple ranges matching this address and pick type
> - * as per MTRR precedence
> - */
> - if (!(mtrr_state.enabled & 2))
> - return mtrr_state.def_type;
> -
> prev_match = 0xFF;
> for (i = 0; i < num_var_ranges; ++i) {
> unsigned short start_state, end_state, inclusive;
> @@ -180,7 +152,8 @@ static u8 __mtrr_type_lookup(u64 start, u64 end, u64 *partial_end, int *repeat)
> * Return the type for first region and a pointer to
> * the start of second region so that caller will
> * lookup again on the second region.
> - * Note: This way we handle multiple overlaps as well.
> + * Note: This way we handle overlaps with multiple
> + * entries and the default type properly.
> */
> if (start_state)
> *partial_end = base + get_mtrr_size(mask);
> @@ -209,21 +182,18 @@ static u8 __mtrr_type_lookup(u64 start, u64 end, u64 *partial_end, int *repeat)
> return curr_match;
> }
>
> - if (mtrr_tom2) {
> - if (start >= (1ULL<<32) && (end < mtrr_tom2))
> - return MTRR_TYPE_WRBACK;
> - }
> -
> if (prev_match != 0xFF)
> return prev_match;
>
> return mtrr_state.def_type;
> }
>
> -/*
> - * Returns the effective MTRR type for the region
> - * Error return:
> - * 0xFF - when MTRR is not enabled
> +/**
> + * mtrr_type_lookup - look up memory type in MTRR
> + *
> + * Return Values:
> + * memory type - The effective MTRR type for the region
> + * 0xFF - MTRR is disabled
> */
> u8 mtrr_type_lookup(u64 start, u64 end)
> {
> @@ -231,12 +201,43 @@ u8 mtrr_type_lookup(u64 start, u64 end)
> int repeat;
> u64 partial_end;
>
> + if (!mtrr_state_set || !mtrr_state.enabled)
> + return 0xFF;
> +
> + /* Look in fixed ranges. Just return the type as per start */
> + if (mtrr_state.have_fixed && (start < 0x100000)) {
> + int idx;
> +
> + if (start < 0x80000) {
> + idx = 0;
> + idx += (start >> 16);
> + return mtrr_state.fixed_ranges[idx];
> + } else if (start < 0xC0000) {
> + idx = 1 * 8;
> + idx += ((start - 0x80000) >> 14);
> + return mtrr_state.fixed_ranges[idx];
> + } else {
> + idx = 3 * 8;
> + idx += ((start - 0xC0000) >> 12);
> + return mtrr_state.fixed_ranges[idx];
> + }
> + }
So why not put this into a separate helper function - named
mtrr_type_lookup_fixed()? It has little relation to variable ranges.
> +
> + /*
> + * Look in variable ranges
> + * Look of multiple ranges matching this address and pick type
> + * as per MTRR precedence
> + */
> + if (!(mtrr_state.enabled & 2))
> + return mtrr_state.def_type;
> +
> type = __mtrr_type_lookup(start, end, &partial_end, &repeat);
And this then should be named mtrr_type_lookup_variable() or so?
Thanks,
Ingo
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH v2 3/4] mtrr, x86: Clean up mtrr_type_lookup()
2015-03-13 12:37 ` Ingo Molnar
@ 2015-03-13 13:53 ` Toshi Kani
0 siblings, 0 replies; 7+ messages in thread
From: Toshi Kani @ 2015-03-13 13:53 UTC (permalink / raw)
To: Ingo Molnar
Cc: akpm, hpa, tglx, mingo, linux-mm, x86, linux-kernel, dave.hansen,
Elliott, Robert (Server Storage),
pebolle
On Fri, 2015-03-13 at 12:37 +0000, Ingo Molnar wrote:
> * Toshi Kani <toshi.kani@hp.com> wrote:
:
> > + /* Look in fixed ranges. Just return the type as per start */
> > + if (mtrr_state.have_fixed && (start < 0x100000)) {
> > + int idx;
> > +
> > + if (start < 0x80000) {
> > + idx = 0;
> > + idx += (start >> 16);
> > + return mtrr_state.fixed_ranges[idx];
> > + } else if (start < 0xC0000) {
> > + idx = 1 * 8;
> > + idx += ((start - 0x80000) >> 14);
> > + return mtrr_state.fixed_ranges[idx];
> > + } else {
> > + idx = 3 * 8;
> > + idx += ((start - 0xC0000) >> 12);
> > + return mtrr_state.fixed_ranges[idx];
> > + }
> > + }
>
> So why not put this into a separate helper function - named
> mtrr_type_lookup_fixed()? It has little relation to variable ranges.
Sounds good. I will update as suggested.
> > +
> > + /*
> > + * Look in variable ranges
> > + * Look of multiple ranges matching this address and pick type
> > + * as per MTRR precedence
> > + */
> > + if (!(mtrr_state.enabled & 2))
> > + return mtrr_state.def_type;
> > +
> > type = __mtrr_type_lookup(start, end, &partial_end, &repeat);
>
> And this then should be named mtrr_type_lookup_variable() or so?
Will do as well.
I will send out a new version today since I won't be able to update the
patchset next week.
Thanks,
-Toshi
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH v2 4/4] mtrr, mm, x86: Enhance MTRR checks for KVA huge page mapping
2015-03-12 17:18 [PATCH v2 0/4] mtrr, mm, x86: Enhance MTRR checks for huge I/O mapping Toshi Kani
` (2 preceding siblings ...)
2015-03-12 17:18 ` [PATCH v2 3/4] mtrr, x86: Clean up mtrr_type_lookup() Toshi Kani
@ 2015-03-12 17:18 ` Toshi Kani
3 siblings, 0 replies; 7+ messages in thread
From: Toshi Kani @ 2015-03-12 17:18 UTC (permalink / raw)
To: akpm, hpa, tglx, mingo
Cc: linux-mm, x86, linux-kernel, dave.hansen, Elliott, pebolle, Toshi Kani
This patch adds an additional argument, 'uniform', to
mtrr_type_lookup(), which returns 1 when a given range is
covered uniformly by MTRRs, i.e. the range is fully convered
by a single MTRR entry or the default type.
pud_set_huge() and pmd_set_huge() are changed to check the
new 'uniform' flag to see if it is safe to create a huge page
mapping to the range. This allows them to create a huge page
mapping to a range covered by a single MTRR entry of any
memory type. It also detects a non-optimal request properly.
They continue to check with the WB type since the WB type has
no effect even if a request spans multiple MTRR entries.
pmd_set_huge() logs a warning message to a non-optimal request
so that driver writers will be aware of such a case. Drivers
should make a mapping request aligned to a single MTRR entry
when the range is covered by MTRRs.
Signed-off-by: Toshi Kani <toshi.kani@hp.com>
---
arch/x86/include/asm/mtrr.h | 5 +++--
arch/x86/kernel/cpu/mtrr/generic.c | 34 +++++++++++++++++++++++++++-------
arch/x86/mm/pat.c | 4 ++--
arch/x86/mm/pgtable.c | 25 +++++++++++++++----------
4 files changed, 47 insertions(+), 21 deletions(-)
diff --git a/arch/x86/include/asm/mtrr.h b/arch/x86/include/asm/mtrr.h
index f768f62..5b4d467 100644
--- a/arch/x86/include/asm/mtrr.h
+++ b/arch/x86/include/asm/mtrr.h
@@ -31,7 +31,7 @@
* arch_phys_wc_add and arch_phys_wc_del.
*/
# ifdef CONFIG_MTRR
-extern u8 mtrr_type_lookup(u64 addr, u64 end);
+extern u8 mtrr_type_lookup(u64 addr, u64 end, u8 *uniform);
extern void mtrr_save_fixed_ranges(void *);
extern void mtrr_save_state(void);
extern int mtrr_add(unsigned long base, unsigned long size,
@@ -50,11 +50,12 @@ extern int mtrr_trim_uncached_memory(unsigned long end_pfn);
extern int amd_special_default_mtrr(void);
extern int phys_wc_to_mtrr_index(int handle);
# else
-static inline u8 mtrr_type_lookup(u64 addr, u64 end)
+static inline u8 mtrr_type_lookup(u64 addr, u64 end, u8 *uniform)
{
/*
* Return no-MTRRs:
*/
+ *uniform = 1;
return 0xff;
}
#define mtrr_save_fixed_ranges(arg) do {} while (0)
diff --git a/arch/x86/kernel/cpu/mtrr/generic.c b/arch/x86/kernel/cpu/mtrr/generic.c
index ef34a4f..56c352c 100644
--- a/arch/x86/kernel/cpu/mtrr/generic.c
+++ b/arch/x86/kernel/cpu/mtrr/generic.c
@@ -108,18 +108,22 @@ static int check_type_overlap(u8 *prev, u8 *curr)
* Return Value:
* memory type - Matched memory type or the default memory type (unmatched)
*
- * Output Argument:
+ * Output Arguments:
* repeat - Set to 1 when [start:end] spanned across MTRR range and type
* returned corresponds only to [start:*partial_end]. Caller has
* to lookup again for [*partial_end:end].
+ * uniform - Set to 1 when MTRR covers the region uniformly, i.e. the region
+ * is fully covered by a single MTRR entry or the default type.
*/
-static u8 __mtrr_type_lookup(u64 start, u64 end, u64 *partial_end, int *repeat)
+static u8 __mtrr_type_lookup(u64 start, u64 end,
+ u64 *partial_end, int *repeat, u8 *uniform)
{
int i;
u64 base, mask;
u8 prev_match, curr_match;
*repeat = 0;
+ *uniform = 1;
/* Make end inclusive end, instead of exclusive */
end--;
@@ -167,6 +171,7 @@ static u8 __mtrr_type_lookup(u64 start, u64 end, u64 *partial_end, int *repeat)
end = *partial_end - 1; /* end is inclusive */
*repeat = 1;
+ *uniform = 0;
}
if (!start_state)
@@ -178,6 +183,7 @@ static u8 __mtrr_type_lookup(u64 start, u64 end, u64 *partial_end, int *repeat)
continue;
}
+ *uniform = 0;
if (check_type_overlap(&prev_match, &curr_match))
return curr_match;
}
@@ -194,19 +200,26 @@ static u8 __mtrr_type_lookup(u64 start, u64 end, u64 *partial_end, int *repeat)
* Return Values:
* memory type - The effective MTRR type for the region
* 0xFF - MTRR is disabled
+ *
+ * Output Argument:
+ * uniform - Set to 1 when MTRR covers the region uniformly, i.e. the region
+ * is fully covered by a single MTRR entry or the default type.
*/
-u8 mtrr_type_lookup(u64 start, u64 end)
+u8 mtrr_type_lookup(u64 start, u64 end, u8 *uniform)
{
- u8 type, prev_type;
+ u8 type, prev_type, is_uniform, dummy;
int repeat;
u64 partial_end;
+ *uniform = 1;
+
if (!mtrr_state_set || !mtrr_state.enabled)
return 0xFF;
/* Look in fixed ranges. Just return the type as per start */
if (mtrr_state.have_fixed && (start < 0x100000)) {
int idx;
+ *uniform = 0;
if (start < 0x80000) {
idx = 0;
@@ -231,7 +244,8 @@ u8 mtrr_type_lookup(u64 start, u64 end)
if (!(mtrr_state.enabled & 2))
return mtrr_state.def_type;
- type = __mtrr_type_lookup(start, end, &partial_end, &repeat);
+ type = __mtrr_type_lookup(start, end,
+ &partial_end, &repeat, &is_uniform);
/*
* Common path is with repeat = 0.
@@ -242,15 +256,21 @@ u8 mtrr_type_lookup(u64 start, u64 end)
while (repeat) {
prev_type = type;
start = partial_end;
- type = __mtrr_type_lookup(start, end, &partial_end, &repeat);
+ is_uniform = 0;
+
+ type = __mtrr_type_lookup(start, end,
+ &partial_end, &repeat, &dummy);
- if (check_type_overlap(&prev_type, &type))
+ if (check_type_overlap(&prev_type, &type)) {
+ *uniform = 0;
return type;
+ }
}
if (mtrr_tom2 && (start >= (1ULL<<32)) && (end < mtrr_tom2))
return MTRR_TYPE_WRBACK;
+ *uniform = is_uniform;
return type;
}
diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c
index 35af677..372ad42 100644
--- a/arch/x86/mm/pat.c
+++ b/arch/x86/mm/pat.c
@@ -267,9 +267,9 @@ static unsigned long pat_x_mtrr_type(u64 start, u64 end,
* request is for WB.
*/
if (req_type == _PAGE_CACHE_MODE_WB) {
- u8 mtrr_type;
+ u8 mtrr_type, uniform;
- mtrr_type = mtrr_type_lookup(start, end);
+ mtrr_type = mtrr_type_lookup(start, end, &uniform);
if (mtrr_type != MTRR_TYPE_WRBACK)
return _PAGE_CACHE_MODE_UC_MINUS;
diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c
index 4891fa1..3d6edea 100644
--- a/arch/x86/mm/pgtable.c
+++ b/arch/x86/mm/pgtable.c
@@ -567,17 +567,18 @@ void native_set_fixmap(enum fixed_addresses idx, phys_addr_t phys,
* pud_set_huge - setup kernel PUD mapping
*
* MTRR can override PAT memory types with 4KB granularity. Therefore,
- * it does not set up a huge page when the range is covered by a non-WB
- * type of MTRR. 0xFF indicates that MTRR are disabled.
+ * it only sets up a huge page when the range is mapped uniformly by MTRR
+ * (i.e. the range is fully covered by a single MTRR entry or the default
+ * type) or the MTRR memory type is WB.
*
* Return 1 on success, and 0 when no PUD was set.
*/
int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot)
{
- u8 mtrr;
+ u8 mtrr, uniform;
- mtrr = mtrr_type_lookup(addr, addr + PUD_SIZE);
- if ((mtrr != MTRR_TYPE_WRBACK) && (mtrr != 0xFF))
+ mtrr = mtrr_type_lookup(addr, addr + PUD_SIZE, &uniform);
+ if ((!uniform) && (mtrr != MTRR_TYPE_WRBACK))
return 0;
prot = pgprot_4k_2_large(prot);
@@ -593,18 +594,22 @@ int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot)
* pmd_set_huge - setup kernel PMD mapping
*
* MTRR can override PAT memory types with 4KB granularity. Therefore,
- * it does not set up a huge page when the range is covered by a non-WB
- * type of MTRR. 0xFF indicates that MTRR are disabled.
+ * it only sets up a huge page when the range is mapped uniformly by MTRR
+ * (i.e. the range is fully covered by a single MTRR entry or the default
+ * type) or the MTRR memory type is WB.
*
* Return 1 on success, and 0 when no PMD was set.
*/
int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot)
{
- u8 mtrr;
+ u8 mtrr, uniform;
- mtrr = mtrr_type_lookup(addr, addr + PMD_SIZE);
- if ((mtrr != MTRR_TYPE_WRBACK) && (mtrr != 0xFF))
+ mtrr = mtrr_type_lookup(addr, addr + PMD_SIZE, &uniform);
+ if ((!uniform) && (mtrr != MTRR_TYPE_WRBACK)) {
+ pr_warn("pmd_set_huge: requesting [mem %#010llx-%#010llx], which spans more than a single MTRR entry\n",
+ addr, addr + PMD_SIZE);
return 0;
+ }
prot = pgprot_4k_2_large(prot);
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 7+ messages in thread