From: "Matthew Wilcox (Oracle)" <willy@infradead.org>
To: linux-mm@kvack.org, linux-arch@vger.kernel.org
Cc: "Matthew Wilcox (Oracle)" <willy@infradead.org>,
linux-kernel@vger.kernel.org
Subject: [PATCH v2 00/30] New page table range API
Date: Mon, 27 Feb 2023 17:57:11 +0000 [thread overview]
Message-ID: <20230227175741.71216-1-willy@infradead.org> (raw)
This patchset changes the API used by the MM to set up page table entries.
The four APIs are:
set_ptes(mm, addr, ptep, pte, nr)
update_mmu_cache_range(vma, addr, ptep, nr)
flush_dcache_folio(folio)
flush_icache_pages(vma, page, nr)
flush_dcache_folio() isn't technically new, but no architecture
implemented it, so I've done that for you. The old APIs remain around
but are mostly implemented by calling the new interfaces.
The new APIs are based around setting up N page table entries at once.
The N entries belong to the same PMD, the same folio and the same VMA,
so ptep++ is a legitimate operation, and locking is taken care of for
you. Some architectures can do a better job of it than just a loop,
but I have hesitated to make too deep a change to architectures I don't
understand well.
One thing I have changed in every architecture is that PG_arch_1 is now a
per-folio bit instead of a per-page bit. This was something that would
have to happen eventually, and it makes sense to do it now rather than
iterate over every page involved in a cache flush and figure out if it
needs to happen.
The point of all this is better performance, and Fengwei Yin has
measured improvement on x86. I suspect you'll see improvement on
your architecture too. Try the new will-it-scale test mentioned here:
https://lore.kernel.org/linux-mm/20230206140639.538867-5-fengwei.yin@intel.com/
You'll need to run it on an XFS filesystem and have
CONFIG_TRANSPARENT_HUGEPAGE set.
For testing, I've only run the code on x86. If an x86->foo compiler
exists in Debian, I've built defconfig. I'm relying on the buildbots
to tell me what I missed, and people who actually have the hardware to
tell me if it actually works.
I'd like to get this into the MM tree soon after the current merge window
closes, so quick feedback would be appreciated.
Matthew Wilcox (Oracle) (26):
mm: Convert page_table_check_pte_set() to page_table_check_ptes_set()
mm: Add generic flush_icache_pages() and documentation
mm: Add folio_flush_mapping()
mm: Remove ARCH_IMPLEMENTS_FLUSH_DCACHE_FOLIO
alpha: Implement the new page table range API
arc: Implement the new page table range API
arm64: Implement the new page table range API
csky: Implement the new page table range API
hexagon: Implement the new page table range API
ia64: Implement the new page table range API
loongarch: Implement the new page table range API
m68k: Implement the new page table range API
microblaze: Implement the new page table range API
mips: Implement the new page table range API
nios2: Implement the new page table range API
openrisc: Implement the new page table range API
parisc: Implement the new page table range API
powerpc: Implement the new page table range API
riscv: Implement the new page table range API
s390: Implement the new page table range API
superh: Implement the new page table range API
sparc32: Implement the new page table range API
sparc64: Implement the new page table range API
um: Implement the new page table range API
x86: Implement the new page table range API
xtensa: Implement the new page table range API
Yin Fengwei (4):
filemap: Add filemap_map_folio_range()
rmap: add folio_add_file_rmap_range()
mm: Convert do_set_pte() to set_pte_range()
filemap: Batch PTE mappings
Documentation/core-api/cachetlb.rst | 35 +++----
Documentation/filesystems/locking.rst | 2 +-
arch/alpha/include/asm/cacheflush.h | 10 ++
arch/alpha/include/asm/pgtable.h | 18 +++-
arch/arc/include/asm/cacheflush.h | 7 +-
arch/arc/include/asm/pgtable-bits-arcv2.h | 20 +++-
arch/arc/mm/cache.c | 61 +++++++-----
arch/arc/mm/tlb.c | 18 ++--
arch/arm/include/asm/cacheflush.h | 24 +++--
arch/arm/include/asm/pgtable.h | 5 +-
arch/arm/include/asm/tlbflush.h | 13 ++-
arch/arm/mm/copypage-v4mc.c | 5 +-
arch/arm/mm/copypage-v6.c | 5 +-
arch/arm/mm/copypage-xscale.c | 5 +-
arch/arm/mm/dma-mapping.c | 24 ++---
arch/arm/mm/fault-armv.c | 14 +--
arch/arm/mm/flush.c | 99 +++++++++++--------
arch/arm/mm/mm.h | 2 +-
arch/arm/mm/mmu.c | 14 ++-
arch/arm64/include/asm/cacheflush.h | 4 +-
arch/arm64/include/asm/pgtable.h | 25 +++--
arch/arm64/mm/flush.c | 36 +++----
arch/csky/abiv1/cacheflush.c | 32 ++++---
arch/csky/abiv1/inc/abi/cacheflush.h | 2 +
arch/csky/abiv2/cacheflush.c | 30 +++---
arch/csky/abiv2/inc/abi/cacheflush.h | 10 +-
arch/csky/include/asm/pgtable.h | 21 +++-
arch/hexagon/include/asm/cacheflush.h | 7 +-
arch/hexagon/include/asm/pgtable.h | 16 +++-
arch/ia64/hp/common/sba_iommu.c | 26 ++---
arch/ia64/include/asm/cacheflush.h | 14 ++-
arch/ia64/include/asm/pgtable.h | 14 ++-
arch/ia64/mm/init.c | 29 ++++--
arch/loongarch/include/asm/cacheflush.h | 2 +
arch/loongarch/include/asm/pgtable.h | 30 ++++--
arch/m68k/include/asm/cacheflush_mm.h | 26 +++--
arch/m68k/include/asm/pgtable_mm.h | 21 +++-
arch/m68k/mm/motorola.c | 2 +-
arch/microblaze/include/asm/cacheflush.h | 8 ++
arch/microblaze/include/asm/pgtable.h | 17 +++-
arch/microblaze/include/asm/tlbflush.h | 4 +-
arch/mips/include/asm/cacheflush.h | 32 ++++---
arch/mips/include/asm/pgtable.h | 36 ++++---
arch/mips/mm/c-r4k.c | 5 +-
arch/mips/mm/cache.c | 56 +++++------
arch/mips/mm/init.c | 17 ++--
arch/nios2/include/asm/cacheflush.h | 6 +-
arch/nios2/include/asm/pgtable.h | 27 ++++--
arch/nios2/mm/cacheflush.c | 61 ++++++------
arch/openrisc/include/asm/cacheflush.h | 8 +-
arch/openrisc/include/asm/pgtable.h | 27 +++++-
arch/openrisc/mm/cache.c | 12 ++-
arch/parisc/include/asm/cacheflush.h | 14 ++-
arch/parisc/include/asm/pgtable.h | 28 ++++--
arch/parisc/kernel/cache.c | 101 ++++++++++++++------
arch/powerpc/include/asm/book3s/pgtable.h | 10 +-
arch/powerpc/include/asm/cacheflush.h | 14 ++-
arch/powerpc/include/asm/kvm_ppc.h | 10 +-
arch/powerpc/include/asm/nohash/pgtable.h | 13 +--
arch/powerpc/include/asm/pgtable.h | 6 ++
arch/powerpc/mm/book3s64/hash_utils.c | 11 ++-
arch/powerpc/mm/cacheflush.c | 81 ++--------------
arch/powerpc/mm/nohash/e500_hugetlbpage.c | 3 +-
arch/powerpc/mm/pgtable.c | 51 +++++-----
arch/riscv/include/asm/cacheflush.h | 19 ++--
arch/riscv/include/asm/pgtable.h | 26 +++--
arch/riscv/mm/cacheflush.c | 11 +--
arch/s390/include/asm/pgtable.h | 34 +++++--
arch/sh/include/asm/cacheflush.h | 21 ++--
arch/sh/include/asm/pgtable.h | 6 +-
arch/sh/include/asm/pgtable_32.h | 16 +++-
arch/sh/mm/cache-j2.c | 4 +-
arch/sh/mm/cache-sh4.c | 26 +++--
arch/sh/mm/cache-sh7705.c | 26 +++--
arch/sh/mm/cache.c | 54 ++++++-----
arch/sh/mm/kmap.c | 3 +-
arch/sparc/include/asm/cacheflush_32.h | 9 +-
arch/sparc/include/asm/cacheflush_64.h | 18 ++--
arch/sparc/include/asm/pgtable_32.h | 15 ++-
arch/sparc/include/asm/pgtable_64.h | 25 ++++-
arch/sparc/kernel/smp_64.c | 56 +++++++----
arch/sparc/mm/init_32.c | 13 ++-
arch/sparc/mm/init_64.c | 78 ++++++++-------
arch/sparc/mm/tlb.c | 5 +-
arch/um/include/asm/pgtable.h | 15 ++-
arch/x86/include/asm/pgtable.h | 21 +++-
arch/xtensa/include/asm/cacheflush.h | 9 +-
arch/xtensa/include/asm/pgtable.h | 24 +++--
arch/xtensa/mm/cache.c | 83 +++++++++-------
include/asm-generic/cacheflush.h | 5 +
include/linux/cacheflush.h | 4 +-
include/linux/mm.h | 3 +-
include/linux/page_table_check.h | 14 +--
include/linux/pagemap.h | 26 ++++-
include/linux/rmap.h | 2 +
mm/filemap.c | 111 +++++++++++++---------
mm/memory.c | 27 +++---
mm/page_table_check.c | 14 +--
mm/rmap.c | 60 +++++++++---
mm/util.c | 2 +-
100 files changed, 1433 insertions(+), 838 deletions(-)
--
2.39.1
next reply other threads:[~2023-02-27 17:57 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-02-27 17:57 Matthew Wilcox (Oracle) [this message]
2023-02-27 17:57 ` [PATCH v2 01/30] mm: Convert page_table_check_pte_set() to page_table_check_ptes_set() Matthew Wilcox (Oracle)
2023-02-27 17:57 ` [PATCH v2 02/30] mm: Add generic flush_icache_pages() and documentation Matthew Wilcox (Oracle)
2023-02-27 17:57 ` [PATCH v2 03/30] mm: Add folio_flush_mapping() Matthew Wilcox (Oracle)
2023-02-27 17:57 ` [PATCH v2 04/30] mm: Remove ARCH_IMPLEMENTS_FLUSH_DCACHE_FOLIO Matthew Wilcox (Oracle)
2023-02-27 17:57 ` [PATCH v2 05/30] alpha: Implement the new page table range API Matthew Wilcox (Oracle)
2023-02-27 17:57 ` [PATCH v2 06/30] arc: " Matthew Wilcox (Oracle)
2023-02-28 6:34 ` Vineet Gupta
2023-02-28 16:25 ` Matthew Wilcox
2023-02-27 17:57 ` [PATCH v2 07/30] arm64: " Matthew Wilcox (Oracle)
2023-02-27 17:57 ` [PATCH v2 08/30] csky: " Matthew Wilcox (Oracle)
2023-02-28 3:17 ` Guo Ren
2023-02-27 17:57 ` [PATCH v2 09/30] hexagon: " Matthew Wilcox (Oracle)
2023-02-27 17:57 ` [PATCH v2 10/30] ia64: " Matthew Wilcox (Oracle)
2023-02-27 17:57 ` [PATCH v2 11/30] loongarch: " Matthew Wilcox (Oracle)
2023-02-27 17:57 ` [PATCH v2 12/30] m68k: " Matthew Wilcox (Oracle)
2023-02-27 17:57 ` [PATCH v2 13/30] microblaze: " Matthew Wilcox (Oracle)
2023-02-27 17:57 ` [PATCH v2 14/30] mips: " Matthew Wilcox (Oracle)
2023-02-27 17:57 ` [PATCH v2 15/30] nios2: " Matthew Wilcox (Oracle)
2023-02-27 17:57 ` [PATCH v2 16/30] openrisc: " Matthew Wilcox (Oracle)
2023-02-27 17:57 ` [PATCH v2 17/30] parisc: " Matthew Wilcox (Oracle)
2023-02-27 22:49 ` John David Anglin
2023-02-27 23:50 ` Matthew Wilcox
2023-02-27 17:57 ` [PATCH v2 18/30] powerpc: " Matthew Wilcox (Oracle)
2023-02-27 19:45 ` Christophe Leroy
2023-02-27 20:20 ` Matthew Wilcox
2023-02-28 6:58 ` Christophe Leroy
2023-02-27 17:57 ` [PATCH v2 19/30] riscv: " Matthew Wilcox (Oracle)
2023-02-27 17:57 ` [PATCH v2 20/30] s390: " Matthew Wilcox (Oracle)
2023-02-27 17:57 ` [PATCH v2 21/30] superh: " Matthew Wilcox (Oracle)
2023-02-27 17:57 ` [PATCH v2 22/30] sparc32: " Matthew Wilcox (Oracle)
2023-02-27 17:57 ` [PATCH v2 23/30] sparc64: " Matthew Wilcox (Oracle)
2023-02-27 17:57 ` [PATCH v2 24/30] um: " Matthew Wilcox (Oracle)
2023-02-27 17:57 ` [PATCH v2 25/30] x86: " Matthew Wilcox (Oracle)
2023-02-27 17:57 ` [PATCH v2 26/30] xtensa: " Matthew Wilcox (Oracle)
2023-02-27 17:57 ` [PATCH v2 27/30] filemap: Add filemap_map_folio_range() Matthew Wilcox (Oracle)
2023-02-27 17:57 ` [PATCH v2 28/30] rmap: add folio_add_file_rmap_range() Matthew Wilcox (Oracle)
2023-02-27 17:57 ` [PATCH v2 29/30] mm: Convert do_set_pte() to set_pte_range() Matthew Wilcox (Oracle)
2023-02-27 17:57 ` [PATCH v2 30/30] filemap: Batch PTE mappings Matthew Wilcox (Oracle)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230227175741.71216-1-willy@infradead.org \
--to=willy@infradead.org \
--cc=linux-arch@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox