Re: [PATCH v3] mm/page_alloc: replace kernel_init_pages() with batch page clearing

Andrew Morton
2026-04-23T11:12:54Z

On Wed, 22 Apr 2026 10:26:58 +0000 Hrushikesh Salunke <hsalunke@amd.com> wrote:

> When init_on_alloc is enabled, kernel_init_pages() clears every page
> one at a time via clear_highpage_kasan_tagged(), which incurs per-page
> kmap_local_page()/kunmap_local() overhead and prevents the architecture
> clearing primitive from operating on contiguous ranges.
>
> Introduce clear_highpages_kasan_tagged() in highmem.h, a batch
> clearing helper that calls clear_pages() for the full contiguous range
> on !HIGHMEM systems, bypassing the per-page kmap overhead and allowing
> a single invocation of the arch clearing primitive across the entire
> allocation. The HIGHMEM path falls back to per-page clearing since
> those pages require kmap.
>
> Replace kernel_init_pages() with direct calls to the new helper, as it
> becomes a trivial wrapper.
>
> Allocating 8192 x 2MB HugeTLB pages (16GB) with init_on_alloc=1:
>
> Before: 0.445s
> After: 0.166s (-62.7%, 2.68x faster)

Nice.

> Kernel time (sys) reduction per workload with init_on_alloc=1:
>
> Workload Before After Change
> Graph500 64C128T 30m 41.8s 15m 14.8s -50.3%
> Graph500 16C32T 15m 56.7s 9m 43.7s -39.0%
> Pagerank 32T 1m 58.5s 1m 12.8s -38.5%
> Pagerank 128T 2m 36.3s 1m 40.4s -35.7%
>
> ...
>
> --- a/include/linux/highmem.h
> +++ b/include/linux/highmem.h
> @@ -345,6 +345,21 @@ static inline void clear_highpage_kasan_tagged(struct page *page)
> kunmap_local(kaddr);
> }
>
> +static inline void clear_highpages_kasan_tagged(struct page *page, int numpages)
> +{
> + /* s390's use of memset() could override KASAN redzones. */
> + kasan_disable_current();
> + if (!IS_ENABLED(CONFIG_HIGHMEM)) {
> + clear_pages(kasan_reset_tag(page_address(page)), numpages);
> + } else {
> + int i;
> +
> + for (i = 0; i < numpages; i++)
> + clear_highpage_kasan_tagged(page + i);
> + }
> + kasan_enable_current();
> +}

Why was it globally published and inlined? Is there any expectation
that this will be used outside of page_alloc.c?

Both of the callsites are themselves inlined. The patch adds 330 bytes
to my arm allmodcnfig page_alloc.o - did we gain anything from that?