[PATCH] kasan: call clear_page with a match-all tag instead of changing page tag

[PATCH] kasan: call clear_page with a match-all tag instead of changing page tag

[Peter Collingbourne]

Instead of changing the page's tag solely in order to obtain a pointer
with a match-all tag and then changing it back again, just convert the
pointer that we get from kmap_atomic() into one with a match-all tag
before passing it to clear_page().

On a certain microarchitecture, this has been observed to cause a
measurable improvement in microbenchmark performance, presumably as a
result of being able to avoid the atomic operations on the page tag.

Signed-off-by: Peter Collingbourne <pcc@google.com>
Link: https://linux-review.googlesource.com/id/I0249822cc29097ca7a04ad48e8eb14871f80e711
---
 include/linux/highmem.h | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/include/linux/highmem.h b/include/linux/highmem.h
index 44242268f53b..bbfa546dd602 100644
--- a/include/linux/highmem.h
+++ b/include/linux/highmem.h
@@ -245,12 +245,10 @@ static inline void clear_highpage(struct page *page)
 
 static inline void clear_highpage_kasan_tagged(struct page *page)
 {
-	u8 tag;
+	void *kaddr = kmap_atomic(page);
 
-	tag = page_kasan_tag(page);
-	page_kasan_tag_reset(page);
-	clear_highpage(page);
-	page_kasan_tag_set(page, tag);
+	clear_page(kasan_reset_tag(kaddr));
+	kunmap_atomic(kaddr);
 }
 
 #ifndef __HAVE_ARCH_TAG_CLEAR_HIGHPAGE
-- 
2.39.1.581.gbfd45094c4-goog

Re: [PATCH] kasan: call clear_page with a match-all tag instead of changing page tag

[Catalin Marinas]

On Wed, Feb 15, 2023 at 10:47:26PM -0800, Peter Collingbourne wrote:
> Instead of changing the page's tag solely in order to obtain a pointer
> with a match-all tag and then changing it back again, just convert the
> pointer that we get from kmap_atomic() into one with a match-all tag
> before passing it to clear_page().
> 
> On a certain microarchitecture, this has been observed to cause a
> measurable improvement in microbenchmark performance, presumably as a
> result of being able to avoid the atomic operations on the page tag.

Yeah, this would likely break the write streaming mode on some ARM CPUs.

> Signed-off-by: Peter Collingbourne <pcc@google.com>
> Link: https://linux-review.googlesource.com/id/I0249822cc29097ca7a04ad48e8eb14871f80e711
> ---
>  include/linux/highmem.h | 8 +++-----
>  1 file changed, 3 insertions(+), 5 deletions(-)
> 
> diff --git a/include/linux/highmem.h b/include/linux/highmem.h
> index 44242268f53b..bbfa546dd602 100644
> --- a/include/linux/highmem.h
> +++ b/include/linux/highmem.h
> @@ -245,12 +245,10 @@ static inline void clear_highpage(struct page *page)
>  
>  static inline void clear_highpage_kasan_tagged(struct page *page)
>  {
> -	u8 tag;
> +	void *kaddr = kmap_atomic(page);
>  
> -	tag = page_kasan_tag(page);
> -	page_kasan_tag_reset(page);
> -	clear_highpage(page);
> -	page_kasan_tag_set(page, tag);
> +	clear_page(kasan_reset_tag(kaddr));
> +	kunmap_atomic(kaddr);
>  }

Please don't add kmap_atomic() back. See commit d2c20e51e396
("mm/highmem: remove deprecated kmap_atomic"). I'd duplicate the
clear_highpage() logic in here and call clear_page() directly on the
address with the kasan tag reset.

-- 
Catalin

Re: [PATCH] kasan: call clear_page with a match-all tag instead of changing page tag

[Peter Collingbourne]

On Thu, Feb 16, 2023 at 6:27 AM Catalin Marinas <catalin.marinas@arm.com> wrote:
>
> On Wed, Feb 15, 2023 at 10:47:26PM -0800, Peter Collingbourne wrote:
> > Instead of changing the page's tag solely in order to obtain a pointer
> > with a match-all tag and then changing it back again, just convert the
> > pointer that we get from kmap_atomic() into one with a match-all tag
> > before passing it to clear_page().
> >
> > On a certain microarchitecture, this has been observed to cause a
> > measurable improvement in microbenchmark performance, presumably as a
> > result of being able to avoid the atomic operations on the page tag.
>
> Yeah, this would likely break the write streaming mode on some ARM CPUs.
>
> > Signed-off-by: Peter Collingbourne <pcc@google.com>
> > Link: https://linux-review.googlesource.com/id/I0249822cc29097ca7a04ad48e8eb14871f80e711
> > ---
> >  include/linux/highmem.h | 8 +++-----
> >  1 file changed, 3 insertions(+), 5 deletions(-)
> >
> > diff --git a/include/linux/highmem.h b/include/linux/highmem.h
> > index 44242268f53b..bbfa546dd602 100644
> > --- a/include/linux/highmem.h
> > +++ b/include/linux/highmem.h
> > @@ -245,12 +245,10 @@ static inline void clear_highpage(struct page *page)
> >
> >  static inline void clear_highpage_kasan_tagged(struct page *page)
> >  {
> > -     u8 tag;
> > +     void *kaddr = kmap_atomic(page);
> >
> > -     tag = page_kasan_tag(page);
> > -     page_kasan_tag_reset(page);
> > -     clear_highpage(page);
> > -     page_kasan_tag_set(page, tag);
> > +     clear_page(kasan_reset_tag(kaddr));
> > +     kunmap_atomic(kaddr);
> >  }
>
> Please don't add kmap_atomic() back. See commit d2c20e51e396
> ("mm/highmem: remove deprecated kmap_atomic"). I'd duplicate the
> clear_highpage() logic in here and call clear_page() directly on the
> address with the kasan tag reset.

Right, that's how I originally developed this patch. As you might have
guessed, I was developing against a stable kernel, so I was copying
the old version of clear_highpage(). Done in v2.

Peter