[PATCH 1/2] alloc_tag: avoid current->alloc_tag manipulations when profiling is disabled

[PATCH 1/2] alloc_tag: avoid current->alloc_tag manipulations when profiling is disabled

[Suren Baghdasaryan]

When memory allocation profiling is disabled there is no need to update
current->alloc_tag and these manipulations add unnecessary overhead. Fix
the overhead by skipping these extra updates.

Fixes: b951aaff5035 ("mm: enable page allocation tagging")
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
Cc: stable@vger.kernel.org
---
 include/linux/alloc_tag.h | 11 ++++++++---
 lib/alloc_tag.c           |  2 ++
 2 files changed, 10 insertions(+), 3 deletions(-)

diff --git a/include/linux/alloc_tag.h b/include/linux/alloc_tag.h
index 0bbbe537c5f9..a946e0203e6d 100644
--- a/include/linux/alloc_tag.h
+++ b/include/linux/alloc_tag.h
@@ -224,9 +224,14 @@ static inline void alloc_tag_sub(union codetag_ref *ref, size_t bytes) {}
 
 #define alloc_hooks_tag(_tag, _do_alloc)				\
 ({									\
-	struct alloc_tag * __maybe_unused _old = alloc_tag_save(_tag);	\
-	typeof(_do_alloc) _res = _do_alloc;				\
-	alloc_tag_restore(_tag, _old);					\
+	typeof(_do_alloc) _res;						\
+	if (mem_alloc_profiling_enabled()) {				\
+		struct alloc_tag * __maybe_unused _old;			\
+		_old = alloc_tag_save(_tag);				\
+		_res = _do_alloc;					\
+		alloc_tag_restore(_tag, _old);				\
+	} else								\
+		_res = _do_alloc;					\
 	_res;								\
 })
 
diff --git a/lib/alloc_tag.c b/lib/alloc_tag.c
index 7dcebf118a3e..4c373f444eb1 100644
--- a/lib/alloc_tag.c
+++ b/lib/alloc_tag.c
@@ -29,6 +29,8 @@ EXPORT_SYMBOL(_shared_alloc_tag);
 
 DEFINE_STATIC_KEY_MAYBE(CONFIG_MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT,
 			mem_alloc_profiling_key);
+EXPORT_SYMBOL(mem_alloc_profiling_key);
+
 DEFINE_STATIC_KEY_FALSE(mem_profiling_compressed);
 
 struct alloc_tag_kernel_section kernel_tags = { NULL, 0 };

base-commit: 431614f1580a03c1a653340c55ea76bd12a9403f
-- 
2.47.1.613.gc27f4b7a9f-goog

[PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled

[Suren Baghdasaryan]

When memory allocation profiling is disabled, there is no need to swap
allocation tags during migration. Skip it to avoid unnecessary overhead.

Fixes: e0a955bf7f61 ("mm/codetag: add pgalloc_tag_copy()")
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
Cc: stable@vger.kernel.org
---
 lib/alloc_tag.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/lib/alloc_tag.c b/lib/alloc_tag.c
index 4c373f444eb1..4e5d7af3eaa2 100644
--- a/lib/alloc_tag.c
+++ b/lib/alloc_tag.c
@@ -197,6 +197,9 @@ void pgalloc_tag_swap(struct folio *new, struct folio *old)
 	union codetag_ref ref_old, ref_new;
 	struct alloc_tag *tag_old, *tag_new;
 
+	if (!mem_alloc_profiling_enabled())
+		return;
+
 	tag_old = pgalloc_tag_get(&old->page);
 	if (!tag_old)
 		return;
-- 
2.47.1.613.gc27f4b7a9f-goog

Re: [PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled

[Andrew Morton]

On Thu, 26 Dec 2024 13:16:39 -0800 Suren Baghdasaryan <surenb@google.com> wrote:

> When memory allocation profiling is disabled, there is no need to swap
> allocation tags during migration. Skip it to avoid unnecessary overhead.
> 
> Fixes: e0a955bf7f61 ("mm/codetag: add pgalloc_tag_copy()")
> Signed-off-by: Suren Baghdasaryan <surenb@google.com>
> Cc: stable@vger.kernel.org

Are these changes worth backporting?  Some indication of how much
difference the patches make would help people understand why we're
proposing a backport.

Re: [PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled

[Suren Baghdasaryan]

On Thu, Dec 26, 2024 at 3:01 PM Andrew Morton <akpm@linux-foundation.org> wrote:
>
> On Thu, 26 Dec 2024 13:16:39 -0800 Suren Baghdasaryan <surenb@google.com> wrote:
>
> > When memory allocation profiling is disabled, there is no need to swap
> > allocation tags during migration. Skip it to avoid unnecessary overhead.
> >
> > Fixes: e0a955bf7f61 ("mm/codetag: add pgalloc_tag_copy()")
> > Signed-off-by: Suren Baghdasaryan <surenb@google.com>
> > Cc: stable@vger.kernel.org
>
> Are these changes worth backporting?  Some indication of how much
> difference the patches make would help people understand why we're
> proposing a backport.

The first patch ("alloc_tag: avoid current->alloc_tag manipulations
when profiling is disabled") I think is worth backporting. It
eliminates about half of the regression for slab allocations when
profiling is disabled. The second one I couldn't really measure, so I
think it's not as important. Thanks!

>

Re: [PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled

[Andrew Morton]

On Thu, 26 Dec 2024 15:07:39 -0800 Suren Baghdasaryan <surenb@google.com> wrote:

> On Thu, Dec 26, 2024 at 3:01 PM Andrew Morton <akpm@linux-foundation.org> wrote:
> >
> > On Thu, 26 Dec 2024 13:16:39 -0800 Suren Baghdasaryan <surenb@google.com> wrote:
> >
> > > When memory allocation profiling is disabled, there is no need to swap
> > > allocation tags during migration. Skip it to avoid unnecessary overhead.
> > >
> > > Fixes: e0a955bf7f61 ("mm/codetag: add pgalloc_tag_copy()")
> > > Signed-off-by: Suren Baghdasaryan <surenb@google.com>
> > > Cc: stable@vger.kernel.org
> >
> > Are these changes worth backporting?  Some indication of how much
> > difference the patches make would help people understand why we're
> > proposing a backport.
> 
> The first patch ("alloc_tag: avoid current->alloc_tag manipulations
> when profiling is disabled") I think is worth backporting. It
> eliminates about half of the regression for slab allocations when
> profiling is disabled.

um, what regression?  The changelog makes no mention of this.  Please
send along a suitable Reported-by: and Closes: and a summary of the
benefits so that people can actually see what this patch does, and why.

Re: [PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled

[Suren Baghdasaryan]

On Thu, Dec 26, 2024 at 4:23 PM Andrew Morton <akpm@linux-foundation.org> wrote:
>
> On Thu, 26 Dec 2024 15:07:39 -0800 Suren Baghdasaryan <surenb@google.com> wrote:
>
> > On Thu, Dec 26, 2024 at 3:01 PM Andrew Morton <akpm@linux-foundation.org> wrote:
> > >
> > > On Thu, 26 Dec 2024 13:16:39 -0800 Suren Baghdasaryan <surenb@google.com> wrote:
> > >
> > > > When memory allocation profiling is disabled, there is no need to swap
> > > > allocation tags during migration. Skip it to avoid unnecessary overhead.
> > > >
> > > > Fixes: e0a955bf7f61 ("mm/codetag: add pgalloc_tag_copy()")
> > > > Signed-off-by: Suren Baghdasaryan <surenb@google.com>
> > > > Cc: stable@vger.kernel.org
> > >
> > > Are these changes worth backporting?  Some indication of how much
> > > difference the patches make would help people understand why we're
> > > proposing a backport.
> >
> > The first patch ("alloc_tag: avoid current->alloc_tag manipulations
> > when profiling is disabled") I think is worth backporting. It
> > eliminates about half of the regression for slab allocations when
> > profiling is disabled.
>
> um, what regression?  The changelog makes no mention of this.  Please
> send along a suitable Reported-by: and Closes: and a summary of the
> benefits so that people can actually see what this patch does, and why.

Sorry, I should have used "overhead" instead of "regression".
When one sets CONFIG_MEM_ALLOC_PROFILING=y, the code gets instrumented
and even if profiling is turned off, it still has a small performance
cost minimized by the use of mem_alloc_profiling_key static key. I
found a couple of places which were not protected with
mem_alloc_profiling_key, which means that even when profiling is
turned off, the code is still executed. Once I added these checks, the
overhead of the mode when memory profiling is enabled but turned off
went down by about 50%.

Re: [PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled

[Andrew Morton]

On Thu, 26 Dec 2024 16:56:00 -0800 Suren Baghdasaryan <surenb@google.com> wrote:

> On Thu, Dec 26, 2024 at 4:23 PM Andrew Morton <akpm@linux-foundation.org> wrote:
> >
> > On Thu, 26 Dec 2024 15:07:39 -0800 Suren Baghdasaryan <surenb@google.com> wrote:
> >
> > > On Thu, Dec 26, 2024 at 3:01 PM Andrew Morton <akpm@linux-foundation.org> wrote:
> > > >
> > > > On Thu, 26 Dec 2024 13:16:39 -0800 Suren Baghdasaryan <surenb@google.com> wrote:
> > > >
> > > > > When memory allocation profiling is disabled, there is no need to swap
> > > > > allocation tags during migration. Skip it to avoid unnecessary overhead.
> > > > >
> > > > > Fixes: e0a955bf7f61 ("mm/codetag: add pgalloc_tag_copy()")
> > > > > Signed-off-by: Suren Baghdasaryan <surenb@google.com>
> > > > > Cc: stable@vger.kernel.org
> > > >
> > > > Are these changes worth backporting?  Some indication of how much
> > > > difference the patches make would help people understand why we're
> > > > proposing a backport.
> > >
> > > The first patch ("alloc_tag: avoid current->alloc_tag manipulations
> > > when profiling is disabled") I think is worth backporting. It
> > > eliminates about half of the regression for slab allocations when
> > > profiling is disabled.
> >
> > um, what regression?  The changelog makes no mention of this.  Please
> > send along a suitable Reported-by: and Closes: and a summary of the
> > benefits so that people can actually see what this patch does, and why.
> 
> Sorry, I should have used "overhead" instead of "regression".
> When one sets CONFIG_MEM_ALLOC_PROFILING=y, the code gets instrumented
> and even if profiling is turned off, it still has a small performance
> cost minimized by the use of mem_alloc_profiling_key static key. I
> found a couple of places which were not protected with
> mem_alloc_profiling_key, which means that even when profiling is
> turned off, the code is still executed. Once I added these checks, the
> overhead of the mode when memory profiling is enabled but turned off
> went down by about 50%.

Well, a 50% reduction in a 0.0000000001% overhead ain't much.  But I
added the final sentence to the changelog.

It still doesn't tell us the very simple thing which we're all eager to
know: how much faster did the kernel get??

Re: [PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled

[Suren Baghdasaryan]

On Thu, Dec 26, 2024 at 11:59 PM Andrew Morton
<akpm@linux-foundation.org> wrote:
>
> On Thu, 26 Dec 2024 16:56:00 -0800 Suren Baghdasaryan <surenb@google.com> wrote:
>
> > On Thu, Dec 26, 2024 at 4:23 PM Andrew Morton <akpm@linux-foundation.org> wrote:
> > >
> > > On Thu, 26 Dec 2024 15:07:39 -0800 Suren Baghdasaryan <surenb@google.com> wrote:
> > >
> > > > On Thu, Dec 26, 2024 at 3:01 PM Andrew Morton <akpm@linux-foundation.org> wrote:
> > > > >
> > > > > On Thu, 26 Dec 2024 13:16:39 -0800 Suren Baghdasaryan <surenb@google.com> wrote:
> > > > >
> > > > > > When memory allocation profiling is disabled, there is no need to swap
> > > > > > allocation tags during migration. Skip it to avoid unnecessary overhead.
> > > > > >
> > > > > > Fixes: e0a955bf7f61 ("mm/codetag: add pgalloc_tag_copy()")
> > > > > > Signed-off-by: Suren Baghdasaryan <surenb@google.com>
> > > > > > Cc: stable@vger.kernel.org
> > > > >
> > > > > Are these changes worth backporting?  Some indication of how much
> > > > > difference the patches make would help people understand why we're
> > > > > proposing a backport.
> > > >
> > > > The first patch ("alloc_tag: avoid current->alloc_tag manipulations
> > > > when profiling is disabled") I think is worth backporting. It
> > > > eliminates about half of the regression for slab allocations when
> > > > profiling is disabled.
> > >
> > > um, what regression?  The changelog makes no mention of this.  Please
> > > send along a suitable Reported-by: and Closes: and a summary of the
> > > benefits so that people can actually see what this patch does, and why.
> >
> > Sorry, I should have used "overhead" instead of "regression".
> > When one sets CONFIG_MEM_ALLOC_PROFILING=y, the code gets instrumented
> > and even if profiling is turned off, it still has a small performance
> > cost minimized by the use of mem_alloc_profiling_key static key. I
> > found a couple of places which were not protected with
> > mem_alloc_profiling_key, which means that even when profiling is
> > turned off, the code is still executed. Once I added these checks, the
> > overhead of the mode when memory profiling is enabled but turned off
> > went down by about 50%.
>
> Well, a 50% reduction in a 0.0000000001% overhead ain't much.

I wish the overhead was that low :)

I ran more comprehensive testing on Pixel 6 on Big, Medium and Little cores:

                 Overhead before fixes            Overhead after fixes
                 slab alloc      page alloc          slab alloc      page alloc
Big               6.21%           5.32%                3.31%          4.93%
Medium       4.51%           5.05%                3.79%          4.39%
Little            7.62%           1.82%                6.68%          1.02%


> But I
> added the final sentence to the changelog.
>
> It still doesn't tell us the very simple thing which we're all eager to
> know: how much faster did the kernel get??

Re: [PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled

[Suren Baghdasaryan]

On Fri, Dec 27, 2024 at 9:28 AM Suren Baghdasaryan <surenb@google.com> wrote:
>
> On Thu, Dec 26, 2024 at 11:59 PM Andrew Morton
> <akpm@linux-foundation.org> wrote:
> >
> > On Thu, 26 Dec 2024 16:56:00 -0800 Suren Baghdasaryan <surenb@google.com> wrote:
> >
> > > On Thu, Dec 26, 2024 at 4:23 PM Andrew Morton <akpm@linux-foundation.org> wrote:
> > > >
> > > > On Thu, 26 Dec 2024 15:07:39 -0800 Suren Baghdasaryan <surenb@google.com> wrote:
> > > >
> > > > > On Thu, Dec 26, 2024 at 3:01 PM Andrew Morton <akpm@linux-foundation.org> wrote:
> > > > > >
> > > > > > On Thu, 26 Dec 2024 13:16:39 -0800 Suren Baghdasaryan <surenb@google.com> wrote:
> > > > > >
> > > > > > > When memory allocation profiling is disabled, there is no need to swap
> > > > > > > allocation tags during migration. Skip it to avoid unnecessary overhead.
> > > > > > >
> > > > > > > Fixes: e0a955bf7f61 ("mm/codetag: add pgalloc_tag_copy()")
> > > > > > > Signed-off-by: Suren Baghdasaryan <surenb@google.com>
> > > > > > > Cc: stable@vger.kernel.org
> > > > > >
> > > > > > Are these changes worth backporting?  Some indication of how much
> > > > > > difference the patches make would help people understand why we're
> > > > > > proposing a backport.
> > > > >
> > > > > The first patch ("alloc_tag: avoid current->alloc_tag manipulations
> > > > > when profiling is disabled") I think is worth backporting. It
> > > > > eliminates about half of the regression for slab allocations when
> > > > > profiling is disabled.
> > > >
> > > > um, what regression?  The changelog makes no mention of this.  Please
> > > > send along a suitable Reported-by: and Closes: and a summary of the
> > > > benefits so that people can actually see what this patch does, and why.
> > >
> > > Sorry, I should have used "overhead" instead of "regression".
> > > When one sets CONFIG_MEM_ALLOC_PROFILING=y, the code gets instrumented
> > > and even if profiling is turned off, it still has a small performance
> > > cost minimized by the use of mem_alloc_profiling_key static key. I
> > > found a couple of places which were not protected with
> > > mem_alloc_profiling_key, which means that even when profiling is
> > > turned off, the code is still executed. Once I added these checks, the
> > > overhead of the mode when memory profiling is enabled but turned off
> > > went down by about 50%.
> >
> > Well, a 50% reduction in a 0.0000000001% overhead ain't much.
>
> I wish the overhead was that low :)
>
> I ran more comprehensive testing on Pixel 6 on Big, Medium and Little cores:
>
>                  Overhead before fixes            Overhead after fixes
>                  slab alloc      page alloc          slab alloc      page alloc
> Big               6.21%           5.32%                3.31%          4.93%
> Medium       4.51%           5.05%                3.79%          4.39%
> Little            7.62%           1.82%                6.68%          1.02%

Note, this is an allocation microbenchmark doing allocations in a
tight loop. Not a really realistic scenario and useful only to make
performance comparisons.

>
>
> > But I
> > added the final sentence to the changelog.
> >
> > It still doesn't tell us the very simple thing which we're all eager to
> > know: how much faster did the kernel get??

Re: [PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled

[Suren Baghdasaryan]

On Fri, Dec 27, 2024 at 9:28 AM Suren Baghdasaryan <surenb@google.com> wrote:
>
> On Thu, Dec 26, 2024 at 11:59 PM Andrew Morton
> <akpm@linux-foundation.org> wrote:
> >
> > On Thu, 26 Dec 2024 16:56:00 -0800 Suren Baghdasaryan <surenb@google.com> wrote:
> >
> > > On Thu, Dec 26, 2024 at 4:23 PM Andrew Morton <akpm@linux-foundation.org> wrote:
> > > >
> > > > On Thu, 26 Dec 2024 15:07:39 -0800 Suren Baghdasaryan <surenb@google.com> wrote:
> > > >
> > > > > On Thu, Dec 26, 2024 at 3:01 PM Andrew Morton <akpm@linux-foundation.org> wrote:
> > > > > >
> > > > > > On Thu, 26 Dec 2024 13:16:39 -0800 Suren Baghdasaryan <surenb@google.com> wrote:
> > > > > >
> > > > > > > When memory allocation profiling is disabled, there is no need to swap
> > > > > > > allocation tags during migration. Skip it to avoid unnecessary overhead.
> > > > > > >
> > > > > > > Fixes: e0a955bf7f61 ("mm/codetag: add pgalloc_tag_copy()")
> > > > > > > Signed-off-by: Suren Baghdasaryan <surenb@google.com>
> > > > > > > Cc: stable@vger.kernel.org
> > > > > >
> > > > > > Are these changes worth backporting?  Some indication of how much
> > > > > > difference the patches make would help people understand why we're
> > > > > > proposing a backport.
> > > > >
> > > > > The first patch ("alloc_tag: avoid current->alloc_tag manipulations
> > > > > when profiling is disabled") I think is worth backporting. It
> > > > > eliminates about half of the regression for slab allocations when
> > > > > profiling is disabled.
> > > >
> > > > um, what regression?  The changelog makes no mention of this.  Please
> > > > send along a suitable Reported-by: and Closes: and a summary of the
> > > > benefits so that people can actually see what this patch does, and why.
> > >
> > > Sorry, I should have used "overhead" instead of "regression".
> > > When one sets CONFIG_MEM_ALLOC_PROFILING=y, the code gets instrumented
> > > and even if profiling is turned off, it still has a small performance
> > > cost minimized by the use of mem_alloc_profiling_key static key. I
> > > found a couple of places which were not protected with
> > > mem_alloc_profiling_key, which means that even when profiling is
> > > turned off, the code is still executed. Once I added these checks, the
> > > overhead of the mode when memory profiling is enabled but turned off
> > > went down by about 50%.
> >
> > Well, a 50% reduction in a 0.0000000001% overhead ain't much.
>
> I wish the overhead was that low :)
>
> I ran more comprehensive testing on Pixel 6 on Big, Medium and Little cores:
>
>                  Overhead before fixes            Overhead after fixes
>                  slab alloc      page alloc          slab alloc      page alloc
> Big               6.21%           5.32%                3.31%          4.93%
> Medium       4.51%           5.05%                3.79%          4.39%
> Little            7.62%           1.82%                6.68%          1.02%

Hi Andrew,
I just noticed that you added the above results to the description of
this patch in mm-unstable: 366507569511 ("alloc_tag: skip
pgalloc_tag_swap if profiling is disabled") but this improvement is
mostly caused the the other patch in this series: 80aded2b9492
("alloc_tag: avoid current->alloc_tag manipulations when profiling is
disabled"). If this is not too much trouble, could you please move it
into the description of the latter patch?
Thanks,
Suren.

>
>
> > But I
> > added the final sentence to the changelog.
> >
> > It still doesn't tell us the very simple thing which we're all eager to
> > know: how much faster did the kernel get??

Re: [PATCH 2/2] alloc_tag: skip pgalloc_tag_swap if profiling is disabled

[Andrew Morton]

On Tue, 14 Jan 2025 08:38:37 -0800 Suren Baghdasaryan <surenb@google.com> wrote:

> > > Well, a 50% reduction in a 0.0000000001% overhead ain't much.
> >
> > I wish the overhead was that low :)
> >
> > I ran more comprehensive testing on Pixel 6 on Big, Medium and Little cores:
> >
> >                  Overhead before fixes            Overhead after fixes
> >                  slab alloc      page alloc          slab alloc      page alloc
> > Big               6.21%           5.32%                3.31%          4.93%
> > Medium       4.51%           5.05%                3.79%          4.39%
> > Little            7.62%           1.82%                6.68%          1.02%
> 
> Hi Andrew,
> I just noticed that you added the above results to the description of
> this patch in mm-unstable: 366507569511 ("alloc_tag: skip
> pgalloc_tag_swap if profiling is disabled") but this improvement is
> mostly caused the the other patch in this series: 80aded2b9492
> ("alloc_tag: avoid current->alloc_tag manipulations when profiling is
> disabled"). If this is not too much trouble, could you please move it
> into the description of the latter patch?

No probs, done, thanks.

Re: [PATCH 1/2] alloc_tag: avoid current->alloc_tag manipulations when profiling is disabled

[Kent Overstreet]

On Thu, Dec 26, 2024 at 01:16:38PM -0800, Suren Baghdasaryan wrote:
> When memory allocation profiling is disabled there is no need to update
> current->alloc_tag and these manipulations add unnecessary overhead. Fix
> the overhead by skipping these extra updates.

I did it the other way because I was concerned about the overhead of
adding a huge number of static keys. But on further thought a static key
probably isn't any bigger than an alloc tag, no?

> 
> Fixes: b951aaff5035 ("mm: enable page allocation tagging")
> Signed-off-by: Suren Baghdasaryan <surenb@google.com>
> Cc: stable@vger.kernel.org
> ---
>  include/linux/alloc_tag.h | 11 ++++++++---
>  lib/alloc_tag.c           |  2 ++
>  2 files changed, 10 insertions(+), 3 deletions(-)
> 
> diff --git a/include/linux/alloc_tag.h b/include/linux/alloc_tag.h
> index 0bbbe537c5f9..a946e0203e6d 100644
> --- a/include/linux/alloc_tag.h
> +++ b/include/linux/alloc_tag.h
> @@ -224,9 +224,14 @@ static inline void alloc_tag_sub(union codetag_ref *ref, size_t bytes) {}
>  
>  #define alloc_hooks_tag(_tag, _do_alloc)				\
>  ({									\
> -	struct alloc_tag * __maybe_unused _old = alloc_tag_save(_tag);	\
> -	typeof(_do_alloc) _res = _do_alloc;				\
> -	alloc_tag_restore(_tag, _old);					\
> +	typeof(_do_alloc) _res;						\
> +	if (mem_alloc_profiling_enabled()) {				\
> +		struct alloc_tag * __maybe_unused _old;			\
> +		_old = alloc_tag_save(_tag);				\
> +		_res = _do_alloc;					\
> +		alloc_tag_restore(_tag, _old);				\
> +	} else								\
> +		_res = _do_alloc;					\
>  	_res;								\
>  })
>  
> diff --git a/lib/alloc_tag.c b/lib/alloc_tag.c
> index 7dcebf118a3e..4c373f444eb1 100644
> --- a/lib/alloc_tag.c
> +++ b/lib/alloc_tag.c
> @@ -29,6 +29,8 @@ EXPORT_SYMBOL(_shared_alloc_tag);
>  
>  DEFINE_STATIC_KEY_MAYBE(CONFIG_MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT,
>  			mem_alloc_profiling_key);
> +EXPORT_SYMBOL(mem_alloc_profiling_key);
> +
>  DEFINE_STATIC_KEY_FALSE(mem_profiling_compressed);
>  
>  struct alloc_tag_kernel_section kernel_tags = { NULL, 0 };
> 
> base-commit: 431614f1580a03c1a653340c55ea76bd12a9403f
> -- 
> 2.47.1.613.gc27f4b7a9f-goog
> 

Re: [PATCH 1/2] alloc_tag: avoid current->alloc_tag manipulations when profiling is disabled

[Suren Baghdasaryan]

On Thu, Dec 26, 2024 at 4:43 PM Kent Overstreet
<kent.overstreet@linux.dev> wrote:
>
> On Thu, Dec 26, 2024 at 01:16:38PM -0800, Suren Baghdasaryan wrote:
> > When memory allocation profiling is disabled there is no need to update
> > current->alloc_tag and these manipulations add unnecessary overhead. Fix
> > the overhead by skipping these extra updates.
>
> I did it the other way because I was concerned about the overhead of
> adding a huge number of static keys. But on further thought a static key
> probably isn't any bigger than an alloc tag, no?

Hmm but a call

>
> >
> > Fixes: b951aaff5035 ("mm: enable page allocation tagging")
> > Signed-off-by: Suren Baghdasaryan <surenb@google.com>
> > Cc: stable@vger.kernel.org
> > ---
> >  include/linux/alloc_tag.h | 11 ++++++++---
> >  lib/alloc_tag.c           |  2 ++
> >  2 files changed, 10 insertions(+), 3 deletions(-)
> >
> > diff --git a/include/linux/alloc_tag.h b/include/linux/alloc_tag.h
> > index 0bbbe537c5f9..a946e0203e6d 100644
> > --- a/include/linux/alloc_tag.h
> > +++ b/include/linux/alloc_tag.h
> > @@ -224,9 +224,14 @@ static inline void alloc_tag_sub(union codetag_ref *ref, size_t bytes) {}
> >
> >  #define alloc_hooks_tag(_tag, _do_alloc)                             \
> >  ({                                                                   \
> > -     struct alloc_tag * __maybe_unused _old = alloc_tag_save(_tag);  \
> > -     typeof(_do_alloc) _res = _do_alloc;                             \
> > -     alloc_tag_restore(_tag, _old);                                  \
> > +     typeof(_do_alloc) _res;                                         \
> > +     if (mem_alloc_profiling_enabled()) {                            \
> > +             struct alloc_tag * __maybe_unused _old;                 \
> > +             _old = alloc_tag_save(_tag);                            \
> > +             _res = _do_alloc;                                       \
> > +             alloc_tag_restore(_tag, _old);                          \
> > +     } else                                                          \
> > +             _res = _do_alloc;                                       \
> >       _res;                                                           \
> >  })
> >
> > diff --git a/lib/alloc_tag.c b/lib/alloc_tag.c
> > index 7dcebf118a3e..4c373f444eb1 100644
> > --- a/lib/alloc_tag.c
> > +++ b/lib/alloc_tag.c
> > @@ -29,6 +29,8 @@ EXPORT_SYMBOL(_shared_alloc_tag);
> >
> >  DEFINE_STATIC_KEY_MAYBE(CONFIG_MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT,
> >                       mem_alloc_profiling_key);
> > +EXPORT_SYMBOL(mem_alloc_profiling_key);
> > +
> >  DEFINE_STATIC_KEY_FALSE(mem_profiling_compressed);
> >
> >  struct alloc_tag_kernel_section kernel_tags = { NULL, 0 };
> >
> > base-commit: 431614f1580a03c1a653340c55ea76bd12a9403f
> > --
> > 2.47.1.613.gc27f4b7a9f-goog
> >

Re: [PATCH 1/2] alloc_tag: avoid current->alloc_tag manipulations when profiling is disabled

[Suren Baghdasaryan]

On Thu, Dec 26, 2024 at 5:07 PM Suren Baghdasaryan <surenb@google.com> wrote:
>
> On Thu, Dec 26, 2024 at 4:43 PM Kent Overstreet
> <kent.overstreet@linux.dev> wrote:
> >
> > On Thu, Dec 26, 2024 at 01:16:38PM -0800, Suren Baghdasaryan wrote:
> > > When memory allocation profiling is disabled there is no need to update
> > > current->alloc_tag and these manipulations add unnecessary overhead. Fix
> > > the overhead by skipping these extra updates.
> >
> > I did it the other way because I was concerned about the overhead of
> > adding a huge number of static keys. But on further thought a static key
> > probably isn't any bigger than an alloc tag, no?
>
> Hmm but a call

Sorry, I pressed enter before finishing typing.

Is a call to static_branch_maybe() generate a new static key? I
thought mem_alloc_profiling_key would be used everywhere but maybe I'm
missing something?

>
> >
> > >
> > > Fixes: b951aaff5035 ("mm: enable page allocation tagging")
> > > Signed-off-by: Suren Baghdasaryan <surenb@google.com>
> > > Cc: stable@vger.kernel.org
> > > ---
> > >  include/linux/alloc_tag.h | 11 ++++++++---
> > >  lib/alloc_tag.c           |  2 ++
> > >  2 files changed, 10 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/include/linux/alloc_tag.h b/include/linux/alloc_tag.h
> > > index 0bbbe537c5f9..a946e0203e6d 100644
> > > --- a/include/linux/alloc_tag.h
> > > +++ b/include/linux/alloc_tag.h
> > > @@ -224,9 +224,14 @@ static inline void alloc_tag_sub(union codetag_ref *ref, size_t bytes) {}
> > >
> > >  #define alloc_hooks_tag(_tag, _do_alloc)                             \
> > >  ({                                                                   \
> > > -     struct alloc_tag * __maybe_unused _old = alloc_tag_save(_tag);  \
> > > -     typeof(_do_alloc) _res = _do_alloc;                             \
> > > -     alloc_tag_restore(_tag, _old);                                  \
> > > +     typeof(_do_alloc) _res;                                         \
> > > +     if (mem_alloc_profiling_enabled()) {                            \
> > > +             struct alloc_tag * __maybe_unused _old;                 \
> > > +             _old = alloc_tag_save(_tag);                            \
> > > +             _res = _do_alloc;                                       \
> > > +             alloc_tag_restore(_tag, _old);                          \
> > > +     } else                                                          \
> > > +             _res = _do_alloc;                                       \
> > >       _res;                                                           \
> > >  })
> > >
> > > diff --git a/lib/alloc_tag.c b/lib/alloc_tag.c
> > > index 7dcebf118a3e..4c373f444eb1 100644
> > > --- a/lib/alloc_tag.c
> > > +++ b/lib/alloc_tag.c
> > > @@ -29,6 +29,8 @@ EXPORT_SYMBOL(_shared_alloc_tag);
> > >
> > >  DEFINE_STATIC_KEY_MAYBE(CONFIG_MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT,
> > >                       mem_alloc_profiling_key);
> > > +EXPORT_SYMBOL(mem_alloc_profiling_key);
> > > +
> > >  DEFINE_STATIC_KEY_FALSE(mem_profiling_compressed);
> > >
> > >  struct alloc_tag_kernel_section kernel_tags = { NULL, 0 };
> > >
> > > base-commit: 431614f1580a03c1a653340c55ea76bd12a9403f
> > > --
> > > 2.47.1.613.gc27f4b7a9f-goog
> > >

Re: [PATCH 1/2] alloc_tag: avoid current->alloc_tag manipulations when profiling is disabled

[Suren Baghdasaryan]

On Thu, Dec 26, 2024 at 5:09 PM Suren Baghdasaryan <surenb@google.com> wrote:
>
> On Thu, Dec 26, 2024 at 5:07 PM Suren Baghdasaryan <surenb@google.com> wrote:
> >
> > On Thu, Dec 26, 2024 at 4:43 PM Kent Overstreet
> > <kent.overstreet@linux.dev> wrote:
> > >
> > > On Thu, Dec 26, 2024 at 01:16:38PM -0800, Suren Baghdasaryan wrote:
> > > > When memory allocation profiling is disabled there is no need to update
> > > > current->alloc_tag and these manipulations add unnecessary overhead. Fix
> > > > the overhead by skipping these extra updates.
> > >
> > > I did it the other way because I was concerned about the overhead of
> > > adding a huge number of static keys. But on further thought a static key
> > > probably isn't any bigger than an alloc tag, no?
> >
> > Hmm but a call
>
> Sorry, I pressed enter before finishing typing.
>
> Is a call to static_branch_maybe() generate a new static key? I
> thought mem_alloc_profiling_key would be used everywhere but maybe I'm
> missing something?

Or maybe you meant the NoOp/Jump generated for a static_branch ?

>
> >
> > >
> > > >
> > > > Fixes: b951aaff5035 ("mm: enable page allocation tagging")
> > > > Signed-off-by: Suren Baghdasaryan <surenb@google.com>
> > > > Cc: stable@vger.kernel.org
> > > > ---
> > > >  include/linux/alloc_tag.h | 11 ++++++++---
> > > >  lib/alloc_tag.c           |  2 ++
> > > >  2 files changed, 10 insertions(+), 3 deletions(-)
> > > >
> > > > diff --git a/include/linux/alloc_tag.h b/include/linux/alloc_tag.h
> > > > index 0bbbe537c5f9..a946e0203e6d 100644
> > > > --- a/include/linux/alloc_tag.h
> > > > +++ b/include/linux/alloc_tag.h
> > > > @@ -224,9 +224,14 @@ static inline void alloc_tag_sub(union codetag_ref *ref, size_t bytes) {}
> > > >
> > > >  #define alloc_hooks_tag(_tag, _do_alloc)                             \
> > > >  ({                                                                   \
> > > > -     struct alloc_tag * __maybe_unused _old = alloc_tag_save(_tag);  \
> > > > -     typeof(_do_alloc) _res = _do_alloc;                             \
> > > > -     alloc_tag_restore(_tag, _old);                                  \
> > > > +     typeof(_do_alloc) _res;                                         \
> > > > +     if (mem_alloc_profiling_enabled()) {                            \
> > > > +             struct alloc_tag * __maybe_unused _old;                 \
> > > > +             _old = alloc_tag_save(_tag);                            \
> > > > +             _res = _do_alloc;                                       \
> > > > +             alloc_tag_restore(_tag, _old);                          \
> > > > +     } else                                                          \
> > > > +             _res = _do_alloc;                                       \
> > > >       _res;                                                           \
> > > >  })
> > > >
> > > > diff --git a/lib/alloc_tag.c b/lib/alloc_tag.c
> > > > index 7dcebf118a3e..4c373f444eb1 100644
> > > > --- a/lib/alloc_tag.c
> > > > +++ b/lib/alloc_tag.c
> > > > @@ -29,6 +29,8 @@ EXPORT_SYMBOL(_shared_alloc_tag);
> > > >
> > > >  DEFINE_STATIC_KEY_MAYBE(CONFIG_MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT,
> > > >                       mem_alloc_profiling_key);
> > > > +EXPORT_SYMBOL(mem_alloc_profiling_key);
> > > > +
> > > >  DEFINE_STATIC_KEY_FALSE(mem_profiling_compressed);
> > > >
> > > >  struct alloc_tag_kernel_section kernel_tags = { NULL, 0 };
> > > >
> > > > base-commit: 431614f1580a03c1a653340c55ea76bd12a9403f
> > > > --
> > > > 2.47.1.613.gc27f4b7a9f-goog
> > > >