* [PATCH v3 1/5] mm: fix device-dax pud write-faults triggered by get_user_pages()
2017-11-21 19:07 [PATCH v3 0/5] fix device-dax pud crash and fixup {pte, pmd, pud}_write Dan Williams
@ 2017-11-21 19:07 ` Dan Williams
2017-11-21 19:07 ` [PATCH v3 2/5] mm: switch to 'define pmd_write' instead of __HAVE_ARCH_PMD_WRITE Dan Williams
` (3 subsequent siblings)
4 siblings, 0 replies; 7+ messages in thread
From: Dan Williams @ 2017-11-21 19:07 UTC (permalink / raw)
To: akpm
Cc: Stephen Rothwell, Arnd Bergmann, linux-nvdimm, x86, stable,
linux-mm, Dave Hansen, Ingo Molnar, H. Peter Anvin,
Thomas Gleixner, Kirill A. Shutemov
Currently only get_user_pages_fast() can safely handle the writable gup
case due to its use of pud_access_permitted() to check whether the pud
entry is writable. In the gup slow path pud_write() is used instead of
pud_access_permitted() and to date it has been unimplemented, just calls
BUG_ON().
kernel BUG at ./include/linux/hugetlb.h:244!
[..]
RIP: 0010:follow_devmap_pud+0x482/0x490
[..]
Call Trace:
follow_page_mask+0x28c/0x6e0
__get_user_pages+0xe4/0x6c0
get_user_pages_unlocked+0x130/0x1b0
get_user_pages_fast+0x89/0xb0
iov_iter_get_pages_alloc+0x114/0x4a0
nfs_direct_read_schedule_iovec+0xd2/0x350
? nfs_start_io_direct+0x63/0x70
nfs_file_direct_read+0x1e0/0x250
nfs_file_read+0x90/0xc0
For now this just implements a simple check for the _PAGE_RW bit similar
to pmd_write. However, this implies that the gup-slow-path check is
missing the extra checks that the gup-fast-path performs with
pud_access_permitted. Later patches will align all checks to use the
'access_permitted' helper if the architecture provides it. Note that the
generic 'access_permitted' helper fallback is the simple _PAGE_RW check
on architectures that do not define the 'access_permitted' helper(s).
Fixes: a00cc7d9dd93 ("mm, x86: add support for PUD-sized transparent hugepages")
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: <stable@vger.kernel.org>
Cc: <x86@kernel.org>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
arch/x86/include/asm/pgtable.h | 6 ++++++
include/asm-generic/pgtable.h | 8 ++++++++
include/linux/hugetlb.h | 8 --------
3 files changed, 14 insertions(+), 8 deletions(-)
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index 09f9e1e00e3b..dcce76ee4aa7 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -1088,6 +1088,12 @@ static inline void pmdp_set_wrprotect(struct mm_struct *mm,
clear_bit(_PAGE_BIT_RW, (unsigned long *)pmdp);
}
+#define pud_write pud_write
+static inline int pud_write(pud_t pud)
+{
+ return pud_flags(pud) & _PAGE_RW;
+}
+
/*
* clone_pgd_range(pgd_t *dst, pgd_t *src, int count);
*
diff --git a/include/asm-generic/pgtable.h b/include/asm-generic/pgtable.h
index 757dc6ffc7ba..1ac457511f4e 100644
--- a/include/asm-generic/pgtable.h
+++ b/include/asm-generic/pgtable.h
@@ -814,6 +814,14 @@ static inline int pmd_write(pmd_t pmd)
#endif /* __HAVE_ARCH_PMD_WRITE */
#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
+#ifndef pud_write
+static inline int pud_write(pud_t pud)
+{
+ BUG();
+ return 0;
+}
+#endif /* pud_write */
+
#if !defined(CONFIG_TRANSPARENT_HUGEPAGE) || \
(defined(CONFIG_TRANSPARENT_HUGEPAGE) && \
!defined(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD))
diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
index fbf5b31d47ee..82a25880714a 100644
--- a/include/linux/hugetlb.h
+++ b/include/linux/hugetlb.h
@@ -239,14 +239,6 @@ static inline int pgd_write(pgd_t pgd)
}
#endif
-#ifndef pud_write
-static inline int pud_write(pud_t pud)
-{
- BUG();
- return 0;
-}
-#endif
-
#define HUGETLB_ANON_FILE "anon_hugepage"
enum {
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 7+ messages in thread* [PATCH v3 2/5] mm: switch to 'define pmd_write' instead of __HAVE_ARCH_PMD_WRITE
2017-11-21 19:07 [PATCH v3 0/5] fix device-dax pud crash and fixup {pte, pmd, pud}_write Dan Williams
2017-11-21 19:07 ` [PATCH v3 1/5] mm: fix device-dax pud write-faults triggered by get_user_pages() Dan Williams
@ 2017-11-21 19:07 ` Dan Williams
2017-11-21 21:58 ` Benjamin Herrenschmidt
2017-11-21 19:07 ` [PATCH v3 3/5] mm: replace pud_write with pud_access_permitted in fault + gup paths Dan Williams
` (2 subsequent siblings)
4 siblings, 1 reply; 7+ messages in thread
From: Dan Williams @ 2017-11-21 19:07 UTC (permalink / raw)
To: akpm
Cc: Stephen Rothwell, Chris Metcalf, Arnd Bergmann, linux-nvdimm,
Benjamin Herrenschmidt, x86, Russell King, Ralf Baechle,
linux-mm, H. Peter Anvin
In response to compile breakage introduced by a series that added the
pud_write helper to x86, Stephen notes:
did you consider using the other paradigm:
In arch include files:
#define pud_write pud_write
static inline int pud_write(pud_t pud)
.....
Then in include/asm-generic/pgtable.h:
#ifndef pud_write
tatic inline int pud_write(pud_t pud)
{
....
}
#endif
If you had, then the powerpc code would have worked ... ;-) and many
of the other interfaces in include/asm-generic/pgtable.h are
protected that way ...
Given that some architecture already define pmd_write() as a macro,
it's a net reduction to drop the definition of
__HAVE_ARCH_PMD_WRITE.
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: <x86@kernel.org>
Suggested-by: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
arch/arm/include/asm/pgtable-3level.h | 1 -
arch/arm64/include/asm/pgtable.h | 1 -
arch/mips/include/asm/pgtable.h | 2 +-
arch/powerpc/include/asm/book3s/64/pgtable.h | 1 -
arch/s390/include/asm/pgtable.h | 2 +-
arch/sparc/include/asm/pgtable_64.h | 2 +-
arch/tile/include/asm/pgtable.h | 1 -
arch/x86/include/asm/pgtable.h | 2 +-
include/asm-generic/pgtable.h | 4 ++--
9 files changed, 6 insertions(+), 10 deletions(-)
diff --git a/arch/arm/include/asm/pgtable-3level.h b/arch/arm/include/asm/pgtable-3level.h
index 2a029bceaf2f..1a7a17b2a1ba 100644
--- a/arch/arm/include/asm/pgtable-3level.h
+++ b/arch/arm/include/asm/pgtable-3level.h
@@ -221,7 +221,6 @@ static inline pte_t pte_mkspecial(pte_t pte)
}
#define __HAVE_ARCH_PTE_SPECIAL
-#define __HAVE_ARCH_PMD_WRITE
#define pmd_write(pmd) (pmd_isclear((pmd), L_PMD_SECT_RDONLY))
#define pmd_dirty(pmd) (pmd_isset((pmd), L_PMD_SECT_DIRTY))
#define pud_page(pud) pmd_page(__pmd(pud_val(pud)))
diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
index c9530b5b5ca8..149d05fb9421 100644
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -345,7 +345,6 @@ static inline int pmd_protnone(pmd_t pmd)
#define pmd_thp_or_huge(pmd) (pmd_huge(pmd) || pmd_trans_huge(pmd))
-#define __HAVE_ARCH_PMD_WRITE
#define pmd_write(pmd) pte_write(pmd_pte(pmd))
#define pmd_mkhuge(pmd) (__pmd(pmd_val(pmd) & ~PMD_TABLE_BIT))
diff --git a/arch/mips/include/asm/pgtable.h b/arch/mips/include/asm/pgtable.h
index 9e9e94415d08..1a508a74d48d 100644
--- a/arch/mips/include/asm/pgtable.h
+++ b/arch/mips/include/asm/pgtable.h
@@ -552,7 +552,7 @@ static inline pmd_t pmd_mkhuge(pmd_t pmd)
extern void set_pmd_at(struct mm_struct *mm, unsigned long addr,
pmd_t *pmdp, pmd_t pmd);
-#define __HAVE_ARCH_PMD_WRITE
+#define pmd_write pmd_write
static inline int pmd_write(pmd_t pmd)
{
return !!(pmd_val(pmd) & _PAGE_WRITE);
diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h
index 9a677cd5997f..44697817ccc6 100644
--- a/arch/powerpc/include/asm/book3s/64/pgtable.h
+++ b/arch/powerpc/include/asm/book3s/64/pgtable.h
@@ -1005,7 +1005,6 @@ static inline int pmd_protnone(pmd_t pmd)
}
#endif /* CONFIG_NUMA_BALANCING */
-#define __HAVE_ARCH_PMD_WRITE
#define pmd_write(pmd) pte_write(pmd_pte(pmd))
#define __pmd_write(pmd) __pte_write(pmd_pte(pmd))
#define pmd_savedwrite(pmd) pte_savedwrite(pmd_pte(pmd))
diff --git a/arch/s390/include/asm/pgtable.h b/arch/s390/include/asm/pgtable.h
index d7fe9838084d..0a6b0286c32e 100644
--- a/arch/s390/include/asm/pgtable.h
+++ b/arch/s390/include/asm/pgtable.h
@@ -709,7 +709,7 @@ static inline unsigned long pmd_pfn(pmd_t pmd)
return (pmd_val(pmd) & origin_mask) >> PAGE_SHIFT;
}
-#define __HAVE_ARCH_PMD_WRITE
+#define pmd_write pmd_write
static inline int pmd_write(pmd_t pmd)
{
return (pmd_val(pmd) & _SEGMENT_ENTRY_WRITE) != 0;
diff --git a/arch/sparc/include/asm/pgtable_64.h b/arch/sparc/include/asm/pgtable_64.h
index 5a9e96be1665..9937c5ff94a9 100644
--- a/arch/sparc/include/asm/pgtable_64.h
+++ b/arch/sparc/include/asm/pgtable_64.h
@@ -715,7 +715,7 @@ static inline unsigned long pmd_pfn(pmd_t pmd)
return pte_pfn(pte);
}
-#define __HAVE_ARCH_PMD_WRITE
+#define pmd_write pmd_write
static inline unsigned long pmd_write(pmd_t pmd)
{
pte_t pte = __pte(pmd_val(pmd));
diff --git a/arch/tile/include/asm/pgtable.h b/arch/tile/include/asm/pgtable.h
index 2a26cc4fefc2..adfa21b18488 100644
--- a/arch/tile/include/asm/pgtable.h
+++ b/arch/tile/include/asm/pgtable.h
@@ -475,7 +475,6 @@ static inline void pmd_clear(pmd_t *pmdp)
#define pmd_mkdirty(pmd) pte_pmd(pte_mkdirty(pmd_pte(pmd)))
#define pmd_huge_page(pmd) pte_huge(pmd_pte(pmd))
#define pmd_mkhuge(pmd) pte_pmd(pte_mkhuge(pmd_pte(pmd)))
-#define __HAVE_ARCH_PMD_WRITE
#define pfn_pmd(pfn, pgprot) pte_pmd(pfn_pte((pfn), (pgprot)))
#define pmd_pfn(pmd) pte_pfn(pmd_pte(pmd))
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index dcce76ee4aa7..95e2dfd75521 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -1061,7 +1061,7 @@ extern int pmdp_clear_flush_young(struct vm_area_struct *vma,
unsigned long address, pmd_t *pmdp);
-#define __HAVE_ARCH_PMD_WRITE
+#define pmd_write pmd_write
static inline int pmd_write(pmd_t pmd)
{
return pmd_flags(pmd) & _PAGE_RW;
diff --git a/include/asm-generic/pgtable.h b/include/asm-generic/pgtable.h
index 1ac457511f4e..b234d54f2cb6 100644
--- a/include/asm-generic/pgtable.h
+++ b/include/asm-generic/pgtable.h
@@ -805,13 +805,13 @@ static inline int pmd_trans_huge(pmd_t pmd)
{
return 0;
}
-#ifndef __HAVE_ARCH_PMD_WRITE
+#ifndef pmd_write
static inline int pmd_write(pmd_t pmd)
{
BUG();
return 0;
}
-#endif /* __HAVE_ARCH_PMD_WRITE */
+#endif /* pmd_write */
#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
#ifndef pud_write
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: [PATCH v3 2/5] mm: switch to 'define pmd_write' instead of __HAVE_ARCH_PMD_WRITE
2017-11-21 19:07 ` [PATCH v3 2/5] mm: switch to 'define pmd_write' instead of __HAVE_ARCH_PMD_WRITE Dan Williams
@ 2017-11-21 21:58 ` Benjamin Herrenschmidt
0 siblings, 0 replies; 7+ messages in thread
From: Benjamin Herrenschmidt @ 2017-11-21 21:58 UTC (permalink / raw)
To: Dan Williams, akpm
Cc: Stephen Rothwell, Chris Metcalf, Arnd Bergmann, linux-nvdimm,
x86, Russell King, Ralf Baechle, linux-mm, H. Peter Anvin,
Aneesh Kumar K.V, Oliver OHalloran
On Tue, 2017-11-21 at 11:07 -0800, Dan Williams wrote:
> In response to compile breakage introduced by a series that added the
> pud_write helper to x86, Stephen notes:
+Aneesh, +Oliver.
> did you consider using the other paradigm:
>
> In arch include files:
> #define pud_write pud_write
> static inline int pud_write(pud_t pud)
> .....
>
> Then in include/asm-generic/pgtable.h:
>
> #ifndef pud_write
> tatic inline int pud_write(pud_t pud)
> {
> ....
> }
> #endif
>
> If you had, then the powerpc code would have worked ... ;-) and many
> of the other interfaces in include/asm-generic/pgtable.h are
> protected that way ...
>
> Given that some architecture already define pmd_write() as a macro,
> it's a net reduction to drop the definition of
> __HAVE_ARCH_PMD_WRITE.
>
> Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> Cc: Chris Metcalf <cmetcalf@mellanox.com>
> Cc: Russell King <linux@armlinux.org.uk>
> Cc: Ralf Baechle <ralf@linux-mips.org>
> Cc: "H. Peter Anvin" <hpa@zytor.com>
> Cc: Arnd Bergmann <arnd@arndb.de>
> Cc: <x86@kernel.org>
> Suggested-by: Stephen Rothwell <sfr@canb.auug.org.au>
> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
> ---
> arch/arm/include/asm/pgtable-3level.h | 1 -
> arch/arm64/include/asm/pgtable.h | 1 -
> arch/mips/include/asm/pgtable.h | 2 +-
> arch/powerpc/include/asm/book3s/64/pgtable.h | 1 -
> arch/s390/include/asm/pgtable.h | 2 +-
> arch/sparc/include/asm/pgtable_64.h | 2 +-
> arch/tile/include/asm/pgtable.h | 1 -
> arch/x86/include/asm/pgtable.h | 2 +-
> include/asm-generic/pgtable.h | 4 ++--
> 9 files changed, 6 insertions(+), 10 deletions(-)
>
> diff --git a/arch/arm/include/asm/pgtable-3level.h b/arch/arm/include/asm/pgtable-3level.h
> index 2a029bceaf2f..1a7a17b2a1ba 100644
> --- a/arch/arm/include/asm/pgtable-3level.h
> +++ b/arch/arm/include/asm/pgtable-3level.h
> @@ -221,7 +221,6 @@ static inline pte_t pte_mkspecial(pte_t pte)
> }
> #define __HAVE_ARCH_PTE_SPECIAL
>
> -#define __HAVE_ARCH_PMD_WRITE
> #define pmd_write(pmd) (pmd_isclear((pmd), L_PMD_SECT_RDONLY))
> #define pmd_dirty(pmd) (pmd_isset((pmd), L_PMD_SECT_DIRTY))
> #define pud_page(pud) pmd_page(__pmd(pud_val(pud)))
> diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
> index c9530b5b5ca8..149d05fb9421 100644
> --- a/arch/arm64/include/asm/pgtable.h
> +++ b/arch/arm64/include/asm/pgtable.h
> @@ -345,7 +345,6 @@ static inline int pmd_protnone(pmd_t pmd)
>
> #define pmd_thp_or_huge(pmd) (pmd_huge(pmd) || pmd_trans_huge(pmd))
>
> -#define __HAVE_ARCH_PMD_WRITE
> #define pmd_write(pmd) pte_write(pmd_pte(pmd))
>
> #define pmd_mkhuge(pmd) (__pmd(pmd_val(pmd) & ~PMD_TABLE_BIT))
> diff --git a/arch/mips/include/asm/pgtable.h b/arch/mips/include/asm/pgtable.h
> index 9e9e94415d08..1a508a74d48d 100644
> --- a/arch/mips/include/asm/pgtable.h
> +++ b/arch/mips/include/asm/pgtable.h
> @@ -552,7 +552,7 @@ static inline pmd_t pmd_mkhuge(pmd_t pmd)
> extern void set_pmd_at(struct mm_struct *mm, unsigned long addr,
> pmd_t *pmdp, pmd_t pmd);
>
> -#define __HAVE_ARCH_PMD_WRITE
> +#define pmd_write pmd_write
> static inline int pmd_write(pmd_t pmd)
> {
> return !!(pmd_val(pmd) & _PAGE_WRITE);
> diff --git a/arch/powerpc/include/asm/book3s/64/pgtable.h b/arch/powerpc/include/asm/book3s/64/pgtable.h
> index 9a677cd5997f..44697817ccc6 100644
> --- a/arch/powerpc/include/asm/book3s/64/pgtable.h
> +++ b/arch/powerpc/include/asm/book3s/64/pgtable.h
> @@ -1005,7 +1005,6 @@ static inline int pmd_protnone(pmd_t pmd)
> }
> #endif /* CONFIG_NUMA_BALANCING */
>
> -#define __HAVE_ARCH_PMD_WRITE
> #define pmd_write(pmd) pte_write(pmd_pte(pmd))
> #define __pmd_write(pmd) __pte_write(pmd_pte(pmd))
> #define pmd_savedwrite(pmd) pte_savedwrite(pmd_pte(pmd))
> diff --git a/arch/s390/include/asm/pgtable.h b/arch/s390/include/asm/pgtable.h
> index d7fe9838084d..0a6b0286c32e 100644
> --- a/arch/s390/include/asm/pgtable.h
> +++ b/arch/s390/include/asm/pgtable.h
> @@ -709,7 +709,7 @@ static inline unsigned long pmd_pfn(pmd_t pmd)
> return (pmd_val(pmd) & origin_mask) >> PAGE_SHIFT;
> }
>
> -#define __HAVE_ARCH_PMD_WRITE
> +#define pmd_write pmd_write
> static inline int pmd_write(pmd_t pmd)
> {
> return (pmd_val(pmd) & _SEGMENT_ENTRY_WRITE) != 0;
> diff --git a/arch/sparc/include/asm/pgtable_64.h b/arch/sparc/include/asm/pgtable_64.h
> index 5a9e96be1665..9937c5ff94a9 100644
> --- a/arch/sparc/include/asm/pgtable_64.h
> +++ b/arch/sparc/include/asm/pgtable_64.h
> @@ -715,7 +715,7 @@ static inline unsigned long pmd_pfn(pmd_t pmd)
> return pte_pfn(pte);
> }
>
> -#define __HAVE_ARCH_PMD_WRITE
> +#define pmd_write pmd_write
> static inline unsigned long pmd_write(pmd_t pmd)
> {
> pte_t pte = __pte(pmd_val(pmd));
> diff --git a/arch/tile/include/asm/pgtable.h b/arch/tile/include/asm/pgtable.h
> index 2a26cc4fefc2..adfa21b18488 100644
> --- a/arch/tile/include/asm/pgtable.h
> +++ b/arch/tile/include/asm/pgtable.h
> @@ -475,7 +475,6 @@ static inline void pmd_clear(pmd_t *pmdp)
> #define pmd_mkdirty(pmd) pte_pmd(pte_mkdirty(pmd_pte(pmd)))
> #define pmd_huge_page(pmd) pte_huge(pmd_pte(pmd))
> #define pmd_mkhuge(pmd) pte_pmd(pte_mkhuge(pmd_pte(pmd)))
> -#define __HAVE_ARCH_PMD_WRITE
>
> #define pfn_pmd(pfn, pgprot) pte_pmd(pfn_pte((pfn), (pgprot)))
> #define pmd_pfn(pmd) pte_pfn(pmd_pte(pmd))
> diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
> index dcce76ee4aa7..95e2dfd75521 100644
> --- a/arch/x86/include/asm/pgtable.h
> +++ b/arch/x86/include/asm/pgtable.h
> @@ -1061,7 +1061,7 @@ extern int pmdp_clear_flush_young(struct vm_area_struct *vma,
> unsigned long address, pmd_t *pmdp);
>
>
> -#define __HAVE_ARCH_PMD_WRITE
> +#define pmd_write pmd_write
> static inline int pmd_write(pmd_t pmd)
> {
> return pmd_flags(pmd) & _PAGE_RW;
> diff --git a/include/asm-generic/pgtable.h b/include/asm-generic/pgtable.h
> index 1ac457511f4e..b234d54f2cb6 100644
> --- a/include/asm-generic/pgtable.h
> +++ b/include/asm-generic/pgtable.h
> @@ -805,13 +805,13 @@ static inline int pmd_trans_huge(pmd_t pmd)
> {
> return 0;
> }
> -#ifndef __HAVE_ARCH_PMD_WRITE
> +#ifndef pmd_write
> static inline int pmd_write(pmd_t pmd)
> {
> BUG();
> return 0;
> }
> -#endif /* __HAVE_ARCH_PMD_WRITE */
> +#endif /* pmd_write */
> #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
>
> #ifndef pud_write
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH v3 3/5] mm: replace pud_write with pud_access_permitted in fault + gup paths
2017-11-21 19:07 [PATCH v3 0/5] fix device-dax pud crash and fixup {pte, pmd, pud}_write Dan Williams
2017-11-21 19:07 ` [PATCH v3 1/5] mm: fix device-dax pud write-faults triggered by get_user_pages() Dan Williams
2017-11-21 19:07 ` [PATCH v3 2/5] mm: switch to 'define pmd_write' instead of __HAVE_ARCH_PMD_WRITE Dan Williams
@ 2017-11-21 19:07 ` Dan Williams
2017-11-21 19:07 ` [PATCH v3 4/5] mm: replace pmd_write with pmd_access_permitted " Dan Williams
2017-11-21 19:08 ` [PATCH v3 5/5] mm: replace pte_write with pte_access_permitted " Dan Williams
4 siblings, 0 replies; 7+ messages in thread
From: Dan Williams @ 2017-11-21 19:07 UTC (permalink / raw)
To: akpm
Cc: linux-nvdimm, Heiko Carstens, linux-mm, Dave Hansen,
Martin Schwidefsky, David S. Miller, Kirill A. Shutemov
The 'access_permitted' helper is used in the gup-fast path and goes
beyond the simple _PAGE_RW check to also:
* validate that the mapping is writable from a protection keys
standpoint
* validate that the pte has _PAGE_USER set since all fault paths where
pud_write is must be referencing user-memory.
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
arch/s390/include/asm/pgtable.h | 6 ++++++
arch/sparc/mm/gup.c | 2 +-
mm/huge_memory.c | 2 +-
mm/memory.c | 2 +-
4 files changed, 9 insertions(+), 3 deletions(-)
diff --git a/arch/s390/include/asm/pgtable.h b/arch/s390/include/asm/pgtable.h
index 0a6b0286c32e..57d7bc92e0b8 100644
--- a/arch/s390/include/asm/pgtable.h
+++ b/arch/s390/include/asm/pgtable.h
@@ -1264,6 +1264,12 @@ static inline pud_t pud_mkwrite(pud_t pud)
return pud;
}
+#define pud_write pud_write
+static inline int pud_write(pud_t pud)
+{
+ return (pud_val(pud) & _REGION3_ENTRY_WRITE) != 0;
+}
+
static inline pud_t pud_mkclean(pud_t pud)
{
if (pud_large(pud)) {
diff --git a/arch/sparc/mm/gup.c b/arch/sparc/mm/gup.c
index 5335ba3c850e..5ae2d0a01a70 100644
--- a/arch/sparc/mm/gup.c
+++ b/arch/sparc/mm/gup.c
@@ -114,7 +114,7 @@ static int gup_huge_pud(pud_t *pudp, pud_t pud, unsigned long addr,
if (!(pud_val(pud) & _PAGE_VALID))
return 0;
- if (write && !pud_write(pud))
+ if (!pud_access_permitted(pud, write))
return 0;
refs = 0;
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 86fe697e8bfb..9583f035ccb0 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -1022,7 +1022,7 @@ struct page *follow_devmap_pud(struct vm_area_struct *vma, unsigned long addr,
assert_spin_locked(pud_lockptr(mm, pud));
- if (flags & FOLL_WRITE && !pud_write(*pud))
+ if (!pud_access_permitted(*pud, flags & FOLL_WRITE))
return NULL;
if (pud_present(*pud) && pud_devmap(*pud))
diff --git a/mm/memory.c b/mm/memory.c
index 85e7a87da79f..24fbd60eb2b0 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -4013,7 +4013,7 @@ static int __handle_mm_fault(struct vm_area_struct *vma, unsigned long address,
/* NUMA case for anonymous PUDs would go here */
- if (dirty && !pud_write(orig_pud)) {
+ if (dirty && !pud_access_permitted(orig_pud, WRITE)) {
ret = wp_huge_pud(&vmf, orig_pud);
if (!(ret & VM_FAULT_FALLBACK))
return ret;
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 7+ messages in thread* [PATCH v3 4/5] mm: replace pmd_write with pmd_access_permitted in fault + gup paths
2017-11-21 19:07 [PATCH v3 0/5] fix device-dax pud crash and fixup {pte, pmd, pud}_write Dan Williams
` (2 preceding siblings ...)
2017-11-21 19:07 ` [PATCH v3 3/5] mm: replace pud_write with pud_access_permitted in fault + gup paths Dan Williams
@ 2017-11-21 19:07 ` Dan Williams
2017-11-21 19:08 ` [PATCH v3 5/5] mm: replace pte_write with pte_access_permitted " Dan Williams
4 siblings, 0 replies; 7+ messages in thread
From: Dan Williams @ 2017-11-21 19:07 UTC (permalink / raw)
To: akpm
Cc: linux-mm, Dave Hansen, Jérôme Glisse,
Kirill A. Shutemov, linux-nvdimm
The 'access_permitted' helper is used in the gup-fast path and goes
beyond the simple _PAGE_RW check to also:
* validate that the mapping is writable from a protection keys
standpoint
* validate that the pte has _PAGE_USER set since all fault paths where
pmd_write is must be referencing user-memory.
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: "JA(C)rA'me Glisse" <jglisse@redhat.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
arch/sparc/mm/gup.c | 2 +-
fs/dax.c | 3 ++-
mm/hmm.c | 4 ++--
mm/huge_memory.c | 4 ++--
mm/memory.c | 2 +-
5 files changed, 8 insertions(+), 7 deletions(-)
diff --git a/arch/sparc/mm/gup.c b/arch/sparc/mm/gup.c
index 5ae2d0a01a70..33c0f8bb0f33 100644
--- a/arch/sparc/mm/gup.c
+++ b/arch/sparc/mm/gup.c
@@ -75,7 +75,7 @@ static int gup_huge_pmd(pmd_t *pmdp, pmd_t pmd, unsigned long addr,
if (!(pmd_val(pmd) & _PAGE_VALID))
return 0;
- if (write && !pmd_write(pmd))
+ if (!pmd_access_permitted(pmd, write))
return 0;
refs = 0;
diff --git a/fs/dax.c b/fs/dax.c
index 95981591977a..78b72c48374e 100644
--- a/fs/dax.c
+++ b/fs/dax.c
@@ -627,7 +627,8 @@ static void dax_mapping_entry_mkclean(struct address_space *mapping,
if (pfn != pmd_pfn(*pmdp))
goto unlock_pmd;
- if (!pmd_dirty(*pmdp) && !pmd_write(*pmdp))
+ if (!pmd_dirty(*pmdp)
+ && !pmd_access_permitted(*pmdp, WRITE))
goto unlock_pmd;
flush_cache_page(vma, address, pfn);
diff --git a/mm/hmm.c b/mm/hmm.c
index ea19742a5d60..93718a391611 100644
--- a/mm/hmm.c
+++ b/mm/hmm.c
@@ -391,11 +391,11 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp,
if (pmd_protnone(pmd))
return hmm_vma_walk_clear(start, end, walk);
- if (write_fault && !pmd_write(pmd))
+ if (!pmd_access_permitted(pmd, write_fault))
return hmm_vma_walk_clear(start, end, walk);
pfn = pmd_pfn(pmd) + pte_index(addr);
- flag |= pmd_write(pmd) ? HMM_PFN_WRITE : 0;
+ flag |= pmd_access_permitted(pmd, WRITE) ? HMM_PFN_WRITE : 0;
for (; addr < end; addr += PAGE_SIZE, i++, pfn++)
pfns[i] = hmm_pfn_t_from_pfn(pfn) | flag;
return 0;
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 9583f035ccb0..ceeb9c4c5276 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -875,7 +875,7 @@ struct page *follow_devmap_pmd(struct vm_area_struct *vma, unsigned long addr,
*/
WARN_ONCE(flags & FOLL_COW, "mm: In follow_devmap_pmd with FOLL_COW set");
- if (flags & FOLL_WRITE && !pmd_write(*pmd))
+ if (!pmd_access_permitted(*pmd, flags & FOLL_WRITE))
return NULL;
if (pmd_present(*pmd) && pmd_devmap(*pmd))
@@ -1396,7 +1396,7 @@ int do_huge_pmd_wp_page(struct vm_fault *vmf, pmd_t orig_pmd)
*/
static inline bool can_follow_write_pmd(pmd_t pmd, unsigned int flags)
{
- return pmd_write(pmd) ||
+ return pmd_access_permitted(pmd, WRITE) ||
((flags & FOLL_FORCE) && (flags & FOLL_COW) && pmd_dirty(pmd));
}
diff --git a/mm/memory.c b/mm/memory.c
index 24fbd60eb2b0..07eef971dbeb 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -4046,7 +4046,7 @@ static int __handle_mm_fault(struct vm_area_struct *vma, unsigned long address,
if (pmd_protnone(orig_pmd) && vma_is_accessible(vma))
return do_huge_pmd_numa_page(&vmf, orig_pmd);
- if (dirty && !pmd_write(orig_pmd)) {
+ if (dirty && !pmd_access_permitted(orig_pmd, WRITE)) {
ret = wp_huge_pmd(&vmf, orig_pmd);
if (!(ret & VM_FAULT_FALLBACK))
return ret;
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 7+ messages in thread* [PATCH v3 5/5] mm: replace pte_write with pte_access_permitted in fault + gup paths
2017-11-21 19:07 [PATCH v3 0/5] fix device-dax pud crash and fixup {pte, pmd, pud}_write Dan Williams
` (3 preceding siblings ...)
2017-11-21 19:07 ` [PATCH v3 4/5] mm: replace pmd_write with pmd_access_permitted " Dan Williams
@ 2017-11-21 19:08 ` Dan Williams
4 siblings, 0 replies; 7+ messages in thread
From: Dan Williams @ 2017-11-21 19:08 UTC (permalink / raw)
To: akpm
Cc: linux-mm, Dave Hansen, Jérôme Glisse,
Kirill A. Shutemov, linux-nvdimm
The 'access_permitted' helper is used in the gup-fast path and goes
beyond the simple _PAGE_RW check to also:
* validate that the mapping is writable from a protection keys
standpoint
* validate that the pte has _PAGE_USER set since all fault paths where
pte_write is must be referencing user-memory.
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: "JA(C)rA'me Glisse" <jglisse@redhat.com>
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
---
mm/gup.c | 2 +-
mm/hmm.c | 4 ++--
mm/memory.c | 4 ++--
3 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/mm/gup.c b/mm/gup.c
index dfcde13f289a..85cc822fd403 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -66,7 +66,7 @@ static int follow_pfn_pte(struct vm_area_struct *vma, unsigned long address,
*/
static inline bool can_follow_write_pte(pte_t pte, unsigned int flags)
{
- return pte_write(pte) ||
+ return pte_access_permitted(pte, WRITE) ||
((flags & FOLL_FORCE) && (flags & FOLL_COW) && pte_dirty(pte));
}
diff --git a/mm/hmm.c b/mm/hmm.c
index 93718a391611..3a5c172af560 100644
--- a/mm/hmm.c
+++ b/mm/hmm.c
@@ -456,11 +456,11 @@ static int hmm_vma_walk_pmd(pmd_t *pmdp,
continue;
}
- if (write_fault && !pte_write(pte))
+ if (!pte_access_permitted(pte, write_fault))
goto fault;
pfns[i] = hmm_pfn_t_from_pfn(pte_pfn(pte)) | flag;
- pfns[i] |= pte_write(pte) ? HMM_PFN_WRITE : 0;
+ pfns[i] |= pte_access_permitted(pte, WRITE) ? HMM_PFN_WRITE : 0;
continue;
fault:
diff --git a/mm/memory.c b/mm/memory.c
index 07eef971dbeb..5eb3d2524bdc 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -3948,7 +3948,7 @@ static int handle_pte_fault(struct vm_fault *vmf)
if (unlikely(!pte_same(*vmf->pte, entry)))
goto unlock;
if (vmf->flags & FAULT_FLAG_WRITE) {
- if (!pte_write(entry))
+ if (!pte_access_permitted(entry, WRITE))
return do_wp_page(vmf);
entry = pte_mkdirty(entry);
}
@@ -4336,7 +4336,7 @@ int follow_phys(struct vm_area_struct *vma,
goto out;
pte = *ptep;
- if ((flags & FOLL_WRITE) && !pte_write(pte))
+ if (!pte_access_permitted(pte, flags & FOLL_WRITE))
goto unlock;
*prot = pgprot_val(pte_pgprot(pte));
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 7+ messages in thread