linux-mm.kvack.org archive mirror
 help / color / mirror / Atom feed
* [PATCH] mm/slub: Make __ksize() faster
@ 2024-03-08  3:13 Kent Overstreet
  2024-03-08  4:48 ` Matthew Wilcox
  0 siblings, 1 reply; 10+ messages in thread
From: Kent Overstreet @ 2024-03-08  3:13 UTC (permalink / raw)
  Cc: Kent Overstreet, linux-mm, Vlastimil Babka, Andrew Morton,
	Christoph Lameter, Pekka Enberg, David Rientjes, Roman Gushchin,
	Hyeonggon Yoo, Matthew Wilcox

this is part of a larger in-progress patch series to add tracking of
amount of memory stranded via RCU.

I want to base this on willy's series that rearranges folio batch
freeing - that's got __folio_put() cleanups I want. Once he's got that
up in a git tree I'll do that and finish the rest. 

-- >8 --

with slab gone, we now have a free u32 in struct slab.

This steals it to make __ksize() faster; it's now a single dependent
load, instead of two. This is going to be important for tracking the
amount of memory stranded by RCU, which we want to be able to do if
we're going to be freeing all pagecache folios (and perhaps all folios)
via RCU.

Signed-off-by: Kent Overstreet <kent.overstreet@linux.dev>
Cc: linux-mm@kvack.org
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Christoph Lameter <cl@linux.com>
Cc: Pekka Enberg <penberg@kernel.org>
Cc: David Rientjes <rientjes@google.com>
Cc: Roman Gushchin <roman.gushchin@linux.dev>
Cc: Hyeonggon Yoo <42.hyeyoo@gmail.com>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
---
 mm/slab.h        | 2 +-
 mm/slab_common.c | 9 ++++-----
 mm/slub.c        | 1 +
 3 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/mm/slab.h b/mm/slab.h
index 54deeb0428c6..64f06431cc97 100644
--- a/mm/slab.h
+++ b/mm/slab.h
@@ -84,7 +84,7 @@ struct slab {
 		};
 		struct rcu_head rcu_head;
 	};
-	unsigned int __unused;
+	unsigned int object_size;
 
 	atomic_t __page_refcount;
 #ifdef CONFIG_MEMCG
diff --git a/mm/slab_common.c b/mm/slab_common.c
index 6ec0f6543f34..f209b8cf4965 100644
--- a/mm/slab_common.c
+++ b/mm/slab_common.c
@@ -963,13 +963,12 @@ size_t __ksize(const void *object)
 		if (WARN_ON(object != folio_address(folio)))
 			return 0;
 		return folio_size(folio);
-	}
-
+	} else {
 #ifdef CONFIG_SLUB_DEBUG
-	skip_orig_size_check(folio_slab(folio)->slab_cache, object);
+		skip_orig_size_check(folio_slab(folio)->slab_cache, object);
 #endif
-
-	return slab_ksize(folio_slab(folio)->slab_cache);
+		return folio_slab(folio)->object_size;
+	}
 }
 
 /**
diff --git a/mm/slub.c b/mm/slub.c
index 2ef88bbf56a3..37fe5774c110 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -2366,6 +2366,7 @@ static struct slab *allocate_slab(struct kmem_cache *s, gfp_t flags, int node)
 	}
 
 	slab->objects = oo_objects(oo);
+	slab->object_size = slab_ksize(slab->slab_cache);
 	slab->inuse = 0;
 	slab->frozen = 0;
 
-- 
2.43.0



^ permalink raw reply	[flat|nested] 10+ messages in thread
end of thread, other threads:[~2024-03-08 21:28 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-03-08  3:13 [PATCH] mm/slub: Make __ksize() faster Kent Overstreet
2024-03-08  4:48 ` Matthew Wilcox
2024-03-08  5:16   ` Kent Overstreet
2024-03-08 14:58     ` Matthew Wilcox
2024-03-08 16:27       ` Kent Overstreet
2024-03-08 17:06         ` Matthew Wilcox
2024-03-08 17:12           ` Christoph Lameter (Ampere)
2024-03-08 18:26             ` Matthew Wilcox
2024-03-08 20:58               ` Kent Overstreet
2024-03-08 21:28               ` Christoph Lameter (Ampere)

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox