[PATCH v4 00/16] Prepare slab for memdescs

[PATCH v4 00/16] Prepare slab for memdescs

[Matthew Wilcox (Oracle)]

When we separate struct folio, struct page and struct slab from each
other, converting to folios then to slabs will be nonsense.  It made
sense under the 'folio is just a head page' interpretation, but with
full separation, page_folio() will return NULL for a page which belongs
to a slab.

This patch series removes almost all mentions of folio from slab.
There are a few folio_test_slab() invocations left around the tree that
I haven't decided how to handle yet.  We're not yet quite at the point
of separately allocating struct slab, but that's what I'll be working
on next.

v4:
 - Rebased on next-20251110 + a couple of fixes
 - Fixed compilation warnings when CONFIG_MEMCG not set
 - Picked up R-b tags from Harry (thanks!)
 - Restored the order of slab vs large_kmalloc tests (Vlastimil)
 - Removed setting of LargeKmalloc on tail pages (Vlastimil)
 - Restored handling of non-slab, non-kmalloc checks in
   check_heap_object() (me)

Matthew Wilcox (Oracle) (16):
  slab: Reimplement page_slab()
  slab: Remove folio references from __ksize()
  slab: Remove folio references in memcg_slab_post_charge()
  slab: Remove folio references in slab alloc/free
  slab: Remove folio references from ___kmalloc_large_node()
  slab: Remove folio references from free_large_kmalloc()
  slab: Remove folio references from kvfree_rcu_cb()
  slab: Remove folio references from kfree()
  slab: Remove folio references from __do_krealloc()
  slab: Remove folio references from build_detached_freelist()
  slab: Remove folio references from kfree_rcu_sheaf()
  slab: Remove folio references from kfree_nolock()
  usercopy: Remove folio references from check_heap_object()
  memcg: Convert mem_cgroup_from_obj_folio() to
    mem_cgroup_from_obj_slab()
  kasan: Remove references to folio in __kasan_mempool_poison_object()
  slab: Remove references to folios from virt_to_slab()

 include/linux/page-flags.h |  16 +----
 mm/kasan/common.c          |  12 ++--
 mm/kfence/core.c           |  14 ++--
 mm/memcontrol.c            |  36 ++++------
 mm/slab.h                  |  58 ++++++++--------
 mm/slab_common.c           |  29 ++++----
 mm/slub.c                  | 137 +++++++++++++++++++------------------
 mm/usercopy.c              |  24 ++++---
 8 files changed, 153 insertions(+), 173 deletions(-)

-- 
2.47.2

[PATCH v4 01/16] slab: Reimplement page_slab()

[Matthew Wilcox (Oracle)]

In order to separate slabs from folios, we need to convert from any page
in a slab to the slab directly without going through a page to folio
conversion first.

Up to this point, page_slab() has followed the example of other memdesc
converters (page_folio(), page_ptdesc() etc) and just cast the pointer
to the requested type, regardless of whether the pointer is actually a
pointer to the correct type or not.

That changes with this commit; we check that the page actually belongs
to a slab and return NULL if it does not.  Other memdesc converters will
adopt this convention in future.

kfence was the only user of page_slab(), so adjust it to the new way
of working.  It will need to be touched again when we separate slab
from page.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Alexander Potapenko <glider@google.com>
Cc: Marco Elver <elver@google.com>
Cc: kasan-dev@googlegroups.com
---
 include/linux/page-flags.h | 14 +-------------
 mm/kfence/core.c           | 14 ++++++++------
 mm/slab.h                  | 28 ++++++++++++++++------------
 3 files changed, 25 insertions(+), 31 deletions(-)

diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
index 0091ad1986bf..6d5e44968eab 100644
--- a/include/linux/page-flags.h
+++ b/include/linux/page-flags.h
@@ -1048,19 +1048,7 @@ PAGE_TYPE_OPS(Table, table, pgtable)
  */
 PAGE_TYPE_OPS(Guard, guard, guard)
 
-FOLIO_TYPE_OPS(slab, slab)
-
-/**
- * PageSlab - Determine if the page belongs to the slab allocator
- * @page: The page to test.
- *
- * Context: Any context.
- * Return: True for slab pages, false for any other kind of page.
- */
-static inline bool PageSlab(const struct page *page)
-{
-	return folio_test_slab(page_folio(page));
-}
+PAGE_TYPE_OPS(Slab, slab, slab)
 
 #ifdef CONFIG_HUGETLB_PAGE
 FOLIO_TYPE_OPS(hugetlb, hugetlb)
diff --git a/mm/kfence/core.c b/mm/kfence/core.c
index 727c20c94ac5..e62b5516bf48 100644
--- a/mm/kfence/core.c
+++ b/mm/kfence/core.c
@@ -612,14 +612,15 @@ static unsigned long kfence_init_pool(void)
 	 * enters __slab_free() slow-path.
 	 */
 	for (i = 0; i < KFENCE_POOL_SIZE / PAGE_SIZE; i++) {
-		struct slab *slab;
+		struct page *page;
 
 		if (!i || (i % 2))
 			continue;
 
-		slab = page_slab(pfn_to_page(start_pfn + i));
-		__folio_set_slab(slab_folio(slab));
+		page = pfn_to_page(start_pfn + i);
+		__SetPageSlab(page);
 #ifdef CONFIG_MEMCG
+		struct slab *slab = page_slab(page);
 		slab->obj_exts = (unsigned long)&kfence_metadata_init[i / 2 - 1].obj_exts |
 				 MEMCG_DATA_OBJEXTS;
 #endif
@@ -665,16 +666,17 @@ static unsigned long kfence_init_pool(void)
 
 reset_slab:
 	for (i = 0; i < KFENCE_POOL_SIZE / PAGE_SIZE; i++) {
-		struct slab *slab;
+		struct page *page;
 
 		if (!i || (i % 2))
 			continue;
 
-		slab = page_slab(pfn_to_page(start_pfn + i));
+		page = pfn_to_page(start_pfn + i);
 #ifdef CONFIG_MEMCG
+		struct slab *slab = page_slab(page);
 		slab->obj_exts = 0;
 #endif
-		__folio_clear_slab(slab_folio(slab));
+		__ClearPageSlab(page);
 	}
 
 	return addr;
diff --git a/mm/slab.h b/mm/slab.h
index f7b8df56727d..18cdb8e85273 100644
--- a/mm/slab.h
+++ b/mm/slab.h
@@ -146,20 +146,24 @@ static_assert(IS_ALIGNED(offsetof(struct slab, freelist), sizeof(freelist_aba_t)
 	struct slab *:		(struct folio *)s))
 
 /**
- * page_slab - Converts from first struct page to slab.
- * @p: The first (either head of compound or single) page of slab.
+ * page_slab - Converts from struct page to its slab.
+ * @page: A page which may or may not belong to a slab.
  *
- * A temporary wrapper to convert struct page to struct slab in situations where
- * we know the page is the compound head, or single order-0 page.
- *
- * Long-term ideally everything would work with struct slab directly or go
- * through folio to struct slab.
- *
- * Return: The slab which contains this page
+ * Return: The slab which contains this page or NULL if the page does
+ * not belong to a slab.  This includes pages returned from large kmalloc.
  */
-#define page_slab(p)		(_Generic((p),				\
-	const struct page *:	(const struct slab *)(p),		\
-	struct page *:		(struct slab *)(p)))
+static inline struct slab *page_slab(const struct page *page)
+{
+	unsigned long head;
+
+	head = READ_ONCE(page->compound_head);
+	if (head & 1)
+		page = (struct page *)(head - 1);
+	if (data_race(page->page_type >> 24) != PGTY_slab)
+		page = NULL;
+
+	return (struct slab *)page;
+}
 
 /**
  * slab_page - The first struct page allocated for a slab
-- 
2.47.2

[PATCH v4 02/16] slab: Remove folio references from __ksize()

[Matthew Wilcox (Oracle)]

In the future, we will separate slab, folio and page from each other
and calling virt_to_folio() on an address allocated from slab will
return NULL.  Delay the conversion from struct page to struct slab
until we know we're not dealing with a large kmalloc allocation.
There's a minor win for large kmalloc allocations as we avoid the
compound_head() hidden in virt_to_folio().

This deprecates calling ksize() on memory allocated by alloc_pages().
Today it becomes a warning and support will be removed entirely in
the future.

Introduce large_kmalloc_size() to abstract how we represent the size
of a large kmalloc allocation.  For now, this is the same as
page_size(), but it will change with separately allocated memdescs.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
 include/linux/page-flags.h |  2 +-
 mm/slab.h                  | 10 ++++++++++
 mm/slab_common.c           | 23 ++++++++++++-----------
 3 files changed, 23 insertions(+), 12 deletions(-)

diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
index 6d5e44968eab..f7a0e4af0c73 100644
--- a/include/linux/page-flags.h
+++ b/include/linux/page-flags.h
@@ -1064,7 +1064,7 @@ PAGE_TYPE_OPS(Zsmalloc, zsmalloc, zsmalloc)
  * Serialized with zone lock.
  */
 PAGE_TYPE_OPS(Unaccepted, unaccepted, unaccepted)
-FOLIO_TYPE_OPS(large_kmalloc, large_kmalloc)
+PAGE_TYPE_OPS(LargeKmalloc, large_kmalloc, large_kmalloc)
 
 /**
  * PageHuge - Determine if the page belongs to hugetlbfs
diff --git a/mm/slab.h b/mm/slab.h
index 18cdb8e85273..0422f2acf8c6 100644
--- a/mm/slab.h
+++ b/mm/slab.h
@@ -603,6 +603,16 @@ static inline size_t slab_ksize(const struct kmem_cache *s)
 	return s->size;
 }
 
+static inline unsigned int large_kmalloc_order(const struct page *page)
+{
+	return page[1].flags.f & 0xff;
+}
+
+static inline size_t large_kmalloc_size(const struct page *page)
+{
+	return PAGE_SIZE << large_kmalloc_order(page);
+}
+
 #ifdef CONFIG_SLUB_DEBUG
 void dump_unreclaimable_slab(void);
 #else
diff --git a/mm/slab_common.c b/mm/slab_common.c
index d2824daa98cf..236b4e25fce0 100644
--- a/mm/slab_common.c
+++ b/mm/slab_common.c
@@ -997,26 +997,27 @@ void __init create_kmalloc_caches(void)
  */
 size_t __ksize(const void *object)
 {
-	struct folio *folio;
+	const struct page *page;
+	const struct slab *slab;
 
 	if (unlikely(object == ZERO_SIZE_PTR))
 		return 0;
 
-	folio = virt_to_folio(object);
+	page = virt_to_page(object);
 
-	if (unlikely(!folio_test_slab(folio))) {
-		if (WARN_ON(folio_size(folio) <= KMALLOC_MAX_CACHE_SIZE))
-			return 0;
-		if (WARN_ON(object != folio_address(folio)))
-			return 0;
-		return folio_size(folio);
-	}
+	if (unlikely(PageLargeKmalloc(page)))
+		return large_kmalloc_size(page);
+
+	slab = page_slab(page);
+	/* Delete this after we're sure there are no users */
+	if (WARN_ON(!slab))
+		return page_size(page);
 
 #ifdef CONFIG_SLUB_DEBUG
-	skip_orig_size_check(folio_slab(folio)->slab_cache, object);
+	skip_orig_size_check(slab->slab_cache, object);
 #endif
 
-	return slab_ksize(folio_slab(folio)->slab_cache);
+	return slab_ksize(slab->slab_cache);
 }
 
 gfp_t kmalloc_fix_flags(gfp_t flags)
-- 
2.47.2

[PATCH v4 03/16] slab: Remove folio references in memcg_slab_post_charge()

[Matthew Wilcox (Oracle)]

This allows us to skip the compound_head() call for large kmalloc
objects as the virt_to_page() call will always give us the head page
for the large kmalloc case.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>
---
 mm/slub.c | 23 ++++++++++++-----------
 1 file changed, 12 insertions(+), 11 deletions(-)

diff --git a/mm/slub.c b/mm/slub.c
index 7715c40672a7..ddfc9db4c6c1 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -2376,33 +2376,34 @@ bool memcg_slab_post_charge(void *p, gfp_t flags)
 {
 	struct slabobj_ext *slab_exts;
 	struct kmem_cache *s;
-	struct folio *folio;
+	struct page *page;
 	struct slab *slab;
 	unsigned long off;
 
-	folio = virt_to_folio(p);
-	if (!folio_test_slab(folio)) {
+	page = virt_to_page(p);
+	if (PageLargeKmalloc(page)) {
+		unsigned int order;
 		int size;
 
-		if (folio_memcg_kmem(folio))
+		if (PageMemcgKmem(page))
 			return true;
 
-		if (__memcg_kmem_charge_page(folio_page(folio, 0), flags,
-					     folio_order(folio)))
+		order = large_kmalloc_order(page);
+		if (__memcg_kmem_charge_page(page, flags, order))
 			return false;
 
 		/*
-		 * This folio has already been accounted in the global stats but
+		 * This page has already been accounted in the global stats but
 		 * not in the memcg stats. So, subtract from the global and use
 		 * the interface which adds to both global and memcg stats.
 		 */
-		size = folio_size(folio);
-		node_stat_mod_folio(folio, NR_SLAB_UNRECLAIMABLE_B, -size);
-		lruvec_stat_mod_folio(folio, NR_SLAB_UNRECLAIMABLE_B, size);
+		size = PAGE_SIZE << order;
+		mod_node_page_state(page_pgdat(page), NR_SLAB_UNRECLAIMABLE_B, -size);
+		mod_lruvec_page_state(page, NR_SLAB_UNRECLAIMABLE_B, size);
 		return true;
 	}
 
-	slab = folio_slab(folio);
+	slab = page_slab(page);
 	s = slab->slab_cache;
 
 	/*
-- 
2.47.2

[PATCH v4 04/16] slab: Remove folio references in slab alloc/free

[Matthew Wilcox (Oracle)]

Use pages directly to further the split between slab and folio.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>
---
 mm/slub.c | 26 +++++++++++++-------------
 1 file changed, 13 insertions(+), 13 deletions(-)

diff --git a/mm/slub.c b/mm/slub.c
index ddfc9db4c6c1..4a093140f985 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -3074,24 +3074,24 @@ static inline struct slab *alloc_slab_page(gfp_t flags, int node,
 					   struct kmem_cache_order_objects oo,
 					   bool allow_spin)
 {
-	struct folio *folio;
+	struct page *page;
 	struct slab *slab;
 	unsigned int order = oo_order(oo);
 
 	if (unlikely(!allow_spin))
-		folio = (struct folio *)alloc_frozen_pages_nolock(0/* __GFP_COMP is implied */,
+		page = alloc_frozen_pages_nolock(0/* __GFP_COMP is implied */,
 								  node, order);
 	else if (node == NUMA_NO_NODE)
-		folio = (struct folio *)alloc_frozen_pages(flags, order);
+		page = alloc_frozen_pages(flags, order);
 	else
-		folio = (struct folio *)__alloc_frozen_pages(flags, order, node, NULL);
+		page = __alloc_frozen_pages(flags, order, node, NULL);
 
-	if (!folio)
+	if (!page)
 		return NULL;
 
-	slab = folio_slab(folio);
-	__folio_set_slab(folio);
-	if (folio_is_pfmemalloc(folio))
+	__SetPageSlab(page);
+	slab = page_slab(page);
+	if (page_is_pfmemalloc(page))
 		slab_set_pfmemalloc(slab);
 
 	return slab;
@@ -3315,16 +3315,16 @@ static struct slab *new_slab(struct kmem_cache *s, gfp_t flags, int node)
 
 static void __free_slab(struct kmem_cache *s, struct slab *slab)
 {
-	struct folio *folio = slab_folio(slab);
-	int order = folio_order(folio);
+	struct page *page = slab_page(slab);
+	int order = compound_order(page);
 	int pages = 1 << order;
 
 	__slab_clear_pfmemalloc(slab);
-	folio->mapping = NULL;
-	__folio_clear_slab(folio);
+	page->mapping = NULL;
+	__ClearPageSlab(page);
 	mm_account_reclaimed_pages(pages);
 	unaccount_slab(slab, order, s);
-	free_frozen_pages(&folio->page, order);
+	free_frozen_pages(page, order);
 }
 
 static void rcu_free_slab(struct rcu_head *h)
-- 
2.47.2

[PATCH v4 05/16] slab: Remove folio references from ___kmalloc_large_node()

[Matthew Wilcox (Oracle)]

There's no need to use folio APIs here; just use a page directly.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>
---
 mm/slub.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/mm/slub.c b/mm/slub.c
index 4a093140f985..90c5663f2b71 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -5601,7 +5601,7 @@ unsigned int kmem_cache_sheaf_size(struct slab_sheaf *sheaf)
  */
 static void *___kmalloc_large_node(size_t size, gfp_t flags, int node)
 {
-	struct folio *folio;
+	struct page *page;
 	void *ptr = NULL;
 	unsigned int order = get_order(size);
 
@@ -5611,15 +5611,15 @@ static void *___kmalloc_large_node(size_t size, gfp_t flags, int node)
 	flags |= __GFP_COMP;
 
 	if (node == NUMA_NO_NODE)
-		folio = (struct folio *)alloc_frozen_pages_noprof(flags, order);
+		page = alloc_frozen_pages_noprof(flags, order);
 	else
-		folio = (struct folio *)__alloc_frozen_pages_noprof(flags, order, node, NULL);
+		page = __alloc_frozen_pages_noprof(flags, order, node, NULL);
 
-	if (folio) {
-		ptr = folio_address(folio);
-		lruvec_stat_mod_folio(folio, NR_SLAB_UNRECLAIMABLE_B,
+	if (page) {
+		ptr = page_address(page);
+		mod_lruvec_page_state(page, NR_SLAB_UNRECLAIMABLE_B,
 				      PAGE_SIZE << order);
-		__folio_set_large_kmalloc(folio);
+		__SetPageLargeKmalloc(page);
 	}
 
 	ptr = kasan_kmalloc_large(ptr, size, flags);
-- 
2.47.2

[PATCH v4 06/16] slab: Remove folio references from free_large_kmalloc()

[Matthew Wilcox (Oracle)]

There's no need to use folio APIs here; just use a page directly.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>
---
 mm/slub.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/mm/slub.c b/mm/slub.c
index 90c5663f2b71..05cb7ec48f09 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -6791,12 +6791,12 @@ void kmem_cache_free(struct kmem_cache *s, void *x)
 }
 EXPORT_SYMBOL(kmem_cache_free);
 
-static void free_large_kmalloc(struct folio *folio, void *object)
+static void free_large_kmalloc(struct page *page, void *object)
 {
-	unsigned int order = folio_order(folio);
+	unsigned int order = compound_order(page);
 
-	if (WARN_ON_ONCE(!folio_test_large_kmalloc(folio))) {
-		dump_page(&folio->page, "Not a kmalloc allocation");
+	if (WARN_ON_ONCE(!PageLargeKmalloc(page))) {
+		dump_page(page, "Not a kmalloc allocation");
 		return;
 	}
 
@@ -6807,10 +6807,10 @@ static void free_large_kmalloc(struct folio *folio, void *object)
 	kasan_kfree_large(object);
 	kmsan_kfree_large(object);
 
-	lruvec_stat_mod_folio(folio, NR_SLAB_UNRECLAIMABLE_B,
+	mod_lruvec_page_state(page, NR_SLAB_UNRECLAIMABLE_B,
 			      -(PAGE_SIZE << order));
-	__folio_clear_large_kmalloc(folio);
-	free_frozen_pages(&folio->page, order);
+	__ClearPageLargeKmalloc(page);
+	free_frozen_pages(page, order);
 }
 
 /*
@@ -6838,7 +6838,7 @@ void kvfree_rcu_cb(struct rcu_head *head)
 		 * consider folio order
 		 */
 		obj = (void *) PAGE_ALIGN_DOWN((unsigned long)obj);
-		free_large_kmalloc(folio, obj);
+		free_large_kmalloc(&folio->page, obj);
 		return;
 	}
 
@@ -6878,7 +6878,7 @@ void kfree(const void *object)
 
 	folio = virt_to_folio(object);
 	if (unlikely(!folio_test_slab(folio))) {
-		free_large_kmalloc(folio, (void *)object);
+		free_large_kmalloc(&folio->page, (void *)object);
 		return;
 	}
 
@@ -7305,7 +7305,7 @@ int build_detached_freelist(struct kmem_cache *s, size_t size,
 	if (!s) {
 		/* Handle kalloc'ed objects */
 		if (unlikely(!folio_test_slab(folio))) {
-			free_large_kmalloc(folio, object);
+			free_large_kmalloc(&folio->page, object);
 			df->slab = NULL;
 			return size;
 		}
-- 
2.47.2

[PATCH v4 07/16] slab: Remove folio references from kvfree_rcu_cb()

[Matthew Wilcox (Oracle)]

Remove conversions from folio to page and folio to slab.  This is
preparation for separately allocated struct slab from struct page.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
 mm/slub.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/mm/slub.c b/mm/slub.c
index 05cb7ec48f09..976e116580a6 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -6820,7 +6820,7 @@ static void free_large_kmalloc(struct page *page, void *object)
 void kvfree_rcu_cb(struct rcu_head *head)
 {
 	void *obj = head;
-	struct folio *folio;
+	struct page *page;
 	struct slab *slab;
 	struct kmem_cache *s;
 	void *slab_addr;
@@ -6831,20 +6831,20 @@ void kvfree_rcu_cb(struct rcu_head *head)
 		return;
 	}
 
-	folio = virt_to_folio(obj);
-	if (!folio_test_slab(folio)) {
+	page = virt_to_page(obj);
+	slab = page_slab(page);
+	if (!slab) {
 		/*
 		 * rcu_head offset can be only less than page size so no need to
-		 * consider folio order
+		 * consider allocation order
 		 */
 		obj = (void *) PAGE_ALIGN_DOWN((unsigned long)obj);
-		free_large_kmalloc(&folio->page, obj);
+		free_large_kmalloc(page, obj);
 		return;
 	}
 
-	slab = folio_slab(folio);
 	s = slab->slab_cache;
-	slab_addr = folio_address(folio);
+	slab_addr = slab_address(slab);
 
 	if (is_kfence_address(obj)) {
 		obj = kfence_object_start(obj);
-- 
2.47.2

[PATCH v4 08/16] slab: Remove folio references from kfree()

[Matthew Wilcox (Oracle)]

This should generate identical code to the previous version, but
without any dependency on how folios work.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
 mm/slub.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/mm/slub.c b/mm/slub.c
index 976e116580a6..184bab91e0af 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -6866,7 +6866,7 @@ void kvfree_rcu_cb(struct rcu_head *head)
  */
 void kfree(const void *object)
 {
-	struct folio *folio;
+	struct page *page;
 	struct slab *slab;
 	struct kmem_cache *s;
 	void *x = (void *)object;
@@ -6876,13 +6876,13 @@ void kfree(const void *object)
 	if (unlikely(ZERO_OR_NULL_PTR(object)))
 		return;
 
-	folio = virt_to_folio(object);
-	if (unlikely(!folio_test_slab(folio))) {
-		free_large_kmalloc(&folio->page, (void *)object);
+	page = virt_to_page(object);
+	slab = page_slab(page);
+	if (!slab) {
+		free_large_kmalloc(page, (void *)object);
 		return;
 	}
 
-	slab = folio_slab(folio);
 	s = slab->slab_cache;
 	slab_free(s, slab, x, _RET_IP_);
 }
-- 
2.47.2

[PATCH v4 09/16] slab: Remove folio references from __do_krealloc()

[Matthew Wilcox (Oracle)]

One slight tweak I made is to calculate 'ks' earlier, which means we
can reuse it in the warning rather than calculating the object size twice.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
 mm/slub.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/mm/slub.c b/mm/slub.c
index 184bab91e0af..4570d1dce271 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -6977,16 +6977,16 @@ __do_krealloc(const void *p, size_t new_size, unsigned long align, gfp_t flags,
 	if (is_kfence_address(p)) {
 		ks = orig_size = kfence_ksize(p);
 	} else {
-		struct folio *folio;
+		struct page *page = virt_to_page(p);
+		struct slab *slab = page_slab(page);
 
-		folio = virt_to_folio(p);
-		if (unlikely(!folio_test_slab(folio))) {
+		if (!slab) {
 			/* Big kmalloc object */
-			WARN_ON(folio_size(folio) <= KMALLOC_MAX_CACHE_SIZE);
-			WARN_ON(p != folio_address(folio));
-			ks = folio_size(folio);
+			ks = page_size(page);
+			WARN_ON(ks <= KMALLOC_MAX_CACHE_SIZE);
+			WARN_ON(p != page_address(page));
 		} else {
-			s = folio_slab(folio)->slab_cache;
+			s = slab->slab_cache;
 			orig_size = get_orig_size(s, (void *)p);
 			ks = s->object_size;
 		}
-- 
2.47.2

[PATCH v4 10/16] slab: Remove folio references from build_detached_freelist()

[Matthew Wilcox (Oracle)]

Use pages and slabs directly instead of converting to folios.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
 mm/slub.c | 16 +++++++++-------
 1 file changed, 9 insertions(+), 7 deletions(-)

diff --git a/mm/slub.c b/mm/slub.c
index 4570d1dce271..17b456f6fd4e 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -7297,23 +7297,25 @@ int build_detached_freelist(struct kmem_cache *s, size_t size,
 {
 	int lookahead = 3;
 	void *object;
-	struct folio *folio;
+	struct page *page;
+	struct slab *slab;
 	size_t same;
 
 	object = p[--size];
-	folio = virt_to_folio(object);
+	page = virt_to_page(object);
+	slab = page_slab(page);
 	if (!s) {
 		/* Handle kalloc'ed objects */
-		if (unlikely(!folio_test_slab(folio))) {
-			free_large_kmalloc(&folio->page, object);
+		if (!slab) {
+			free_large_kmalloc(page, object);
 			df->slab = NULL;
 			return size;
 		}
 		/* Derive kmem_cache from object */
-		df->slab = folio_slab(folio);
-		df->s = df->slab->slab_cache;
+		df->slab = slab;
+		df->s = slab->slab_cache;
 	} else {
-		df->slab = folio_slab(folio);
+		df->slab = slab;
 		df->s = cache_from_obj(s, object); /* Support for memcg */
 	}
 
-- 
2.47.2

[PATCH v4 11/16] slab: Remove folio references from kfree_rcu_sheaf()

[Matthew Wilcox (Oracle)]

In preparation for splitting struct slab from struct page and struct
folio, remove mentions of struct folio from this function.  Since
we don't need to handle large kmalloc objects specially here, we
can just use virt_to_slab().

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
 mm/slab_common.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/mm/slab_common.c b/mm/slab_common.c
index 236b4e25fce0..b613533b29e7 100644
--- a/mm/slab_common.c
+++ b/mm/slab_common.c
@@ -1615,17 +1615,15 @@ static void kfree_rcu_work(struct work_struct *work)
 static bool kfree_rcu_sheaf(void *obj)
 {
 	struct kmem_cache *s;
-	struct folio *folio;
 	struct slab *slab;
 
 	if (is_vmalloc_addr(obj))
 		return false;
 
-	folio = virt_to_folio(obj);
-	if (unlikely(!folio_test_slab(folio)))
+	slab = virt_to_slab(obj);
+	if (unlikely(!slab))
 		return false;
 
-	slab = folio_slab(folio);
 	s = slab->slab_cache;
 	if (s->cpu_sheaves) {
 		if (likely(!IS_ENABLED(CONFIG_NUMA) ||
-- 
2.47.2

[PATCH v4 12/16] slab: Remove folio references from kfree_nolock()

[Matthew Wilcox (Oracle)]

In preparation for splitting struct slab from struct page and struct
folio, remove mentions of struct folio from this function.  Since large
kmalloc objects are not supported here, we can just use virt_to_slab().

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
 mm/slub.c | 6 ++----
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/mm/slub.c b/mm/slub.c
index 17b456f6fd4e..d79aa826d3e5 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -6899,7 +6899,6 @@ EXPORT_SYMBOL(kfree);
  */
 void kfree_nolock(const void *object)
 {
-	struct folio *folio;
 	struct slab *slab;
 	struct kmem_cache *s;
 	void *x = (void *)object;
@@ -6907,13 +6906,12 @@ void kfree_nolock(const void *object)
 	if (unlikely(ZERO_OR_NULL_PTR(object)))
 		return;
 
-	folio = virt_to_folio(object);
-	if (unlikely(!folio_test_slab(folio))) {
+	slab = virt_to_slab(object);
+	if (unlikely(!slab)) {
 		WARN_ONCE(1, "large_kmalloc is not supported by kfree_nolock()");
 		return;
 	}
 
-	slab = folio_slab(folio);
 	s = slab->slab_cache;
 
 	memcg_slab_free_hook(s, slab, &x, 1);
-- 
2.47.2

[PATCH v4 13/16] usercopy: Remove folio references from check_heap_object()

[Matthew Wilcox (Oracle)]

Use page_slab() instead of virt_to_folio() followed by folio_slab().
We do end up calling compound_head() twice for non-slab copies, but that
will not be a problem once we allocate memdescs separately.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Kees Cook <kees@kernel.org>
Cc: "Gustavo A. R. Silva" <gustavoars@kernel.org>
Cc: linux-hardening@vger.kernel.org
---
 mm/usercopy.c | 24 ++++++++++++++++--------
 1 file changed, 16 insertions(+), 8 deletions(-)

diff --git a/mm/usercopy.c b/mm/usercopy.c
index dbdcc43964fb..5de7a518b1b1 100644
--- a/mm/usercopy.c
+++ b/mm/usercopy.c
@@ -164,7 +164,8 @@ static inline void check_heap_object(const void *ptr, unsigned long n,
 {
 	unsigned long addr = (unsigned long)ptr;
 	unsigned long offset;
-	struct folio *folio;
+	struct page *page;
+	struct slab *slab;
 
 	if (is_kmap_addr(ptr)) {
 		offset = offset_in_page(ptr);
@@ -189,16 +190,23 @@ static inline void check_heap_object(const void *ptr, unsigned long n,
 	if (!virt_addr_valid(ptr))
 		return;
 
-	folio = virt_to_folio(ptr);
-
-	if (folio_test_slab(folio)) {
+	page = virt_to_page(ptr);
+	slab = page_slab(page);
+	if (slab) {
 		/* Check slab allocator for flags and size. */
-		__check_heap_object(ptr, n, folio_slab(folio), to_user);
-	} else if (folio_test_large(folio)) {
-		offset = ptr - folio_address(folio);
-		if (n > folio_size(folio) - offset)
+		__check_heap_object(ptr, n, slab, to_user);
+	} else if (PageCompound(page)) {
+		page = compound_head(page);
+		offset = ptr - page_address(page);
+		if (n > page_size(page) - offset)
 			usercopy_abort("page alloc", NULL, to_user, offset, n);
 	}
+
+	/*
+	 * We cannot check non-compound pages.  They might be part of
+	 * a large allocation, in which case crossing a page boundary
+	 * is fine.
+	 */
 }
 
 DEFINE_STATIC_KEY_MAYBE_RO(CONFIG_HARDENED_USERCOPY_DEFAULT_ON,
-- 
2.47.2

[PATCH v4 14/16] memcg: Convert mem_cgroup_from_obj_folio() to mem_cgroup_from_obj_slab()

[Matthew Wilcox (Oracle)]

In preparation for splitting struct slab from struct page and struct
folio, convert the pointer to a slab rather than a folio.  This means
we can end up passing a NULL slab pointer to mem_cgroup_from_obj_slab()
if the pointer is not to a page allocated to slab, and we handle that
appropriately by returning NULL.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Michal Hocko <mhocko@kernel.org>
Cc: Roman Gushchin <roman.gushchin@linux.dev>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Muchun Song <muchun.song@linux.dev>
Cc: cgroups@vger.kernel.org
---
 mm/memcontrol.c | 36 +++++++++++++-----------------------
 1 file changed, 13 insertions(+), 23 deletions(-)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 025da46d9959..b239d8ad511a 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -2589,38 +2589,28 @@ static inline void mod_objcg_mlstate(struct obj_cgroup *objcg,
 }
 
 static __always_inline
-struct mem_cgroup *mem_cgroup_from_obj_folio(struct folio *folio, void *p)
+struct mem_cgroup *mem_cgroup_from_obj_slab(struct slab *slab, void *p)
 {
 	/*
 	 * Slab objects are accounted individually, not per-page.
 	 * Memcg membership data for each individual object is saved in
 	 * slab->obj_exts.
 	 */
-	if (folio_test_slab(folio)) {
-		struct slabobj_ext *obj_exts;
-		struct slab *slab;
-		unsigned int off;
-
-		slab = folio_slab(folio);
-		obj_exts = slab_obj_exts(slab);
-		if (!obj_exts)
-			return NULL;
+	struct slabobj_ext *obj_exts;
+	unsigned int off;
 
-		off = obj_to_index(slab->slab_cache, slab, p);
-		if (obj_exts[off].objcg)
-			return obj_cgroup_memcg(obj_exts[off].objcg);
+	if (!slab)
+		return NULL;
 
+	obj_exts = slab_obj_exts(slab);
+	if (!obj_exts)
 		return NULL;
-	}
 
-	/*
-	 * folio_memcg_check() is used here, because in theory we can encounter
-	 * a folio where the slab flag has been cleared already, but
-	 * slab->obj_exts has not been freed yet
-	 * folio_memcg_check() will guarantee that a proper memory
-	 * cgroup pointer or NULL will be returned.
-	 */
-	return folio_memcg_check(folio);
+	off = obj_to_index(slab->slab_cache, slab, p);
+	if (obj_exts[off].objcg)
+		return obj_cgroup_memcg(obj_exts[off].objcg);
+
+	return NULL;
 }
 
 /*
@@ -2637,7 +2627,7 @@ struct mem_cgroup *mem_cgroup_from_slab_obj(void *p)
 	if (mem_cgroup_disabled())
 		return NULL;
 
-	return mem_cgroup_from_obj_folio(virt_to_folio(p), p);
+	return mem_cgroup_from_obj_slab(virt_to_slab(p), p);
 }
 
 static struct obj_cgroup *__get_obj_cgroup_from_memcg(struct mem_cgroup *memcg)
-- 
2.47.2

[PATCH v4 15/16] kasan: Remove references to folio in __kasan_mempool_poison_object()

[Matthew Wilcox (Oracle)]

In preparation for splitting struct slab from struct page and struct
folio, remove mentions of struct folio from this function.  There is a
mild improvement for large kmalloc objects as we will avoid calling
compound_head() for them.  We can discard the comment as using
PageLargeKmalloc() rather than !folio_test_slab() makes it obvious.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Acked-by: David Hildenbrand <david@redhat.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Konovalov <andreyknvl@gmail.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: kasan-dev <kasan-dev@googlegroups.com>
---
 mm/kasan/common.c | 12 ++++--------
 1 file changed, 4 insertions(+), 8 deletions(-)

diff --git a/mm/kasan/common.c b/mm/kasan/common.c
index 22e5d67ff064..1d27f1bd260b 100644
--- a/mm/kasan/common.c
+++ b/mm/kasan/common.c
@@ -517,24 +517,20 @@ void __kasan_mempool_unpoison_pages(struct page *page, unsigned int order,
 
 bool __kasan_mempool_poison_object(void *ptr, unsigned long ip)
 {
-	struct folio *folio = virt_to_folio(ptr);
+	struct page *page = virt_to_page(ptr);
 	struct slab *slab;
 
-	/*
-	 * This function can be called for large kmalloc allocation that get
-	 * their memory from page_alloc. Thus, the folio might not be a slab.
-	 */
-	if (unlikely(!folio_test_slab(folio))) {
+	if (unlikely(PageLargeKmalloc(page))) {
 		if (check_page_allocation(ptr, ip))
 			return false;
-		kasan_poison(ptr, folio_size(folio), KASAN_PAGE_FREE, false);
+		kasan_poison(ptr, page_size(page), KASAN_PAGE_FREE, false);
 		return true;
 	}
 
 	if (is_kfence_address(ptr))
 		return true;
 
-	slab = folio_slab(folio);
+	slab = page_slab(page);
 
 	if (check_slab_allocation(slab->slab_cache, ptr, ip))
 		return false;
-- 
2.47.2

[PATCH v4 16/16] slab: Remove references to folios from virt_to_slab()

[Matthew Wilcox (Oracle)]

Use page_slab() instead of virt_to_folio() which will work
perfectly when struct slab is separated from struct folio.

This was the last user of folio_slab(), so delete it.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
 mm/slab.h | 20 +-------------------
 1 file changed, 1 insertion(+), 19 deletions(-)

diff --git a/mm/slab.h b/mm/slab.h
index 0422f2acf8c6..8430e24bba3b 100644
--- a/mm/slab.h
+++ b/mm/slab.h
@@ -117,19 +117,6 @@ static_assert(sizeof(struct slab) <= sizeof(struct page));
 static_assert(IS_ALIGNED(offsetof(struct slab, freelist), sizeof(freelist_aba_t)));
 #endif
 
-/**
- * folio_slab - Converts from folio to slab.
- * @folio: The folio.
- *
- * Currently struct slab is a different representation of a folio where
- * folio_test_slab() is true.
- *
- * Return: The slab which contains this folio.
- */
-#define folio_slab(folio)	(_Generic((folio),			\
-	const struct folio *:	(const struct slab *)(folio),		\
-	struct folio *:		(struct slab *)(folio)))
-
 /**
  * slab_folio - The folio allocated for a slab
  * @s: The slab.
@@ -192,12 +179,7 @@ static inline pg_data_t *slab_pgdat(const struct slab *slab)
 
 static inline struct slab *virt_to_slab(const void *addr)
 {
-	struct folio *folio = virt_to_folio(addr);
-
-	if (!folio_test_slab(folio))
-		return NULL;
-
-	return folio_slab(folio);
+	return page_slab(virt_to_page(addr));
 }
 
 static inline int slab_order(const struct slab *slab)
-- 
2.47.2

Re: [PATCH v4 00/16] Prepare slab for memdescs

[Vlastimil Babka]

On 11/13/25 01:09, Matthew Wilcox (Oracle) wrote:
> When we separate struct folio, struct page and struct slab from each
> other, converting to folios then to slabs will be nonsense.  It made
> sense under the 'folio is just a head page' interpretation, but with
> full separation, page_folio() will return NULL for a page which belongs
> to a slab.
> 
> This patch series removes almost all mentions of folio from slab.
> There are a few folio_test_slab() invocations left around the tree that
> I haven't decided how to handle yet.  We're not yet quite at the point
> of separately allocating struct slab, but that's what I'll be working
> on next.
> 
> v4:
>  - Rebased on next-20251110 + a couple of fixes
>  - Fixed compilation warnings when CONFIG_MEMCG not set
>  - Picked up R-b tags from Harry (thanks!)
>  - Restored the order of slab vs large_kmalloc tests (Vlastimil)
>  - Removed setting of LargeKmalloc on tail pages (Vlastimil)
>  - Restored handling of non-slab, non-kmalloc checks in
>    check_heap_object() (me)

All LGTM now. Added to slab/for-next, thanks!

Vlastimil

> Matthew Wilcox (Oracle) (16):
>   slab: Reimplement page_slab()
>   slab: Remove folio references from __ksize()
>   slab: Remove folio references in memcg_slab_post_charge()
>   slab: Remove folio references in slab alloc/free
>   slab: Remove folio references from ___kmalloc_large_node()
>   slab: Remove folio references from free_large_kmalloc()
>   slab: Remove folio references from kvfree_rcu_cb()
>   slab: Remove folio references from kfree()
>   slab: Remove folio references from __do_krealloc()
>   slab: Remove folio references from build_detached_freelist()
>   slab: Remove folio references from kfree_rcu_sheaf()
>   slab: Remove folio references from kfree_nolock()
>   usercopy: Remove folio references from check_heap_object()
>   memcg: Convert mem_cgroup_from_obj_folio() to
>     mem_cgroup_from_obj_slab()
>   kasan: Remove references to folio in __kasan_mempool_poison_object()
>   slab: Remove references to folios from virt_to_slab()
> 
>  include/linux/page-flags.h |  16 +----
>  mm/kasan/common.c          |  12 ++--
>  mm/kfence/core.c           |  14 ++--
>  mm/memcontrol.c            |  36 ++++------
>  mm/slab.h                  |  58 ++++++++--------
>  mm/slab_common.c           |  29 ++++----
>  mm/slub.c                  | 137 +++++++++++++++++++------------------
>  mm/usercopy.c              |  24 ++++---
>  8 files changed, 153 insertions(+), 173 deletions(-)
> 

Re: [PATCH v4 01/16] slab: Reimplement page_slab()

[David Hildenbrand (Red Hat)]

On 13.11.25 01:09, Matthew Wilcox (Oracle) wrote:
> In order to separate slabs from folios, we need to convert from any page
> in a slab to the slab directly without going through a page to folio
> conversion first.
> 
> Up to this point, page_slab() has followed the example of other memdesc
> converters (page_folio(), page_ptdesc() etc) and just cast the pointer
> to the requested type, regardless of whether the pointer is actually a
> pointer to the correct type or not.
> 
> That changes with this commit; we check that the page actually belongs
> to a slab and return NULL if it does not.  Other memdesc converters will
> adopt this convention in future.
> 
> kfence was the only user of page_slab(), so adjust it to the new way
> of working.  It will need to be touched again when we separate slab
> from page.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> Cc: Alexander Potapenko <glider@google.com>
> Cc: Marco Elver <elver@google.com>
> Cc: kasan-dev@googlegroups.com
> ---

Acked-by: David Hildenbrand (Red Hat) <david@kernel.org>

-- 
Cheers

David

Re: [PATCH v4 02/16] slab: Remove folio references from __ksize()

[David Hildenbrand (Red Hat)]

On 13.11.25 01:09, Matthew Wilcox (Oracle) wrote:
> In the future, we will separate slab, folio and page from each other
> and calling virt_to_folio() on an address allocated from slab will
> return NULL.  Delay the conversion from struct page to struct slab
> until we know we're not dealing with a large kmalloc allocation.
> There's a minor win for large kmalloc allocations as we avoid the
> compound_head() hidden in virt_to_folio().
> 
> This deprecates calling ksize() on memory allocated by alloc_pages().
> Today it becomes a warning and support will be removed entirely in
> the future.
> 
> Introduce large_kmalloc_size() to abstract how we represent the size
> of a large kmalloc allocation.  For now, this is the same as
> page_size(), but it will change with separately allocated memdescs.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> ---

Acked-by: David Hildenbrand (Red Hat) <david@kernel.org>

-- 
Cheers

David

Re: [PATCH v4 01/16] slab: Reimplement page_slab()

[Marco Elver]

On Thu, 13 Nov 2025 at 01:09, Matthew Wilcox (Oracle)
<willy@infradead.org> wrote:
>
> In order to separate slabs from folios, we need to convert from any page
> in a slab to the slab directly without going through a page to folio
> conversion first.
>
> Up to this point, page_slab() has followed the example of other memdesc
> converters (page_folio(), page_ptdesc() etc) and just cast the pointer
> to the requested type, regardless of whether the pointer is actually a
> pointer to the correct type or not.
>
> That changes with this commit; we check that the page actually belongs
> to a slab and return NULL if it does not.  Other memdesc converters will
> adopt this convention in future.
>
> kfence was the only user of page_slab(), so adjust it to the new way
> of working.  It will need to be touched again when we separate slab
> from page.
>
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> Cc: Alexander Potapenko <glider@google.com>
> Cc: Marco Elver <elver@google.com>
> Cc: kasan-dev@googlegroups.com

Ran kfence_test with different test configs:

Tested-by: Marco Elver <elver@google.com>

> ---
>  include/linux/page-flags.h | 14 +-------------
>  mm/kfence/core.c           | 14 ++++++++------
>  mm/slab.h                  | 28 ++++++++++++++++------------
>  3 files changed, 25 insertions(+), 31 deletions(-)
>
> diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
> index 0091ad1986bf..6d5e44968eab 100644
> --- a/include/linux/page-flags.h
> +++ b/include/linux/page-flags.h
> @@ -1048,19 +1048,7 @@ PAGE_TYPE_OPS(Table, table, pgtable)
>   */
>  PAGE_TYPE_OPS(Guard, guard, guard)
>
> -FOLIO_TYPE_OPS(slab, slab)
> -
> -/**
> - * PageSlab - Determine if the page belongs to the slab allocator
> - * @page: The page to test.
> - *
> - * Context: Any context.
> - * Return: True for slab pages, false for any other kind of page.
> - */
> -static inline bool PageSlab(const struct page *page)
> -{
> -       return folio_test_slab(page_folio(page));
> -}
> +PAGE_TYPE_OPS(Slab, slab, slab)
>
>  #ifdef CONFIG_HUGETLB_PAGE
>  FOLIO_TYPE_OPS(hugetlb, hugetlb)
> diff --git a/mm/kfence/core.c b/mm/kfence/core.c
> index 727c20c94ac5..e62b5516bf48 100644
> --- a/mm/kfence/core.c
> +++ b/mm/kfence/core.c
> @@ -612,14 +612,15 @@ static unsigned long kfence_init_pool(void)
>          * enters __slab_free() slow-path.
>          */
>         for (i = 0; i < KFENCE_POOL_SIZE / PAGE_SIZE; i++) {
> -               struct slab *slab;
> +               struct page *page;
>
>                 if (!i || (i % 2))
>                         continue;
>
> -               slab = page_slab(pfn_to_page(start_pfn + i));
> -               __folio_set_slab(slab_folio(slab));
> +               page = pfn_to_page(start_pfn + i);
> +               __SetPageSlab(page);
>  #ifdef CONFIG_MEMCG
> +               struct slab *slab = page_slab(page);
>                 slab->obj_exts = (unsigned long)&kfence_metadata_init[i / 2 - 1].obj_exts |
>                                  MEMCG_DATA_OBJEXTS;
>  #endif
> @@ -665,16 +666,17 @@ static unsigned long kfence_init_pool(void)
>
>  reset_slab:
>         for (i = 0; i < KFENCE_POOL_SIZE / PAGE_SIZE; i++) {
> -               struct slab *slab;
> +               struct page *page;
>
>                 if (!i || (i % 2))
>                         continue;
>
> -               slab = page_slab(pfn_to_page(start_pfn + i));
> +               page = pfn_to_page(start_pfn + i);
>  #ifdef CONFIG_MEMCG
> +               struct slab *slab = page_slab(page);
>                 slab->obj_exts = 0;
>  #endif
> -               __folio_clear_slab(slab_folio(slab));
> +               __ClearPageSlab(page);
>         }
>
>         return addr;
> diff --git a/mm/slab.h b/mm/slab.h
> index f7b8df56727d..18cdb8e85273 100644
> --- a/mm/slab.h
> +++ b/mm/slab.h
> @@ -146,20 +146,24 @@ static_assert(IS_ALIGNED(offsetof(struct slab, freelist), sizeof(freelist_aba_t)
>         struct slab *:          (struct folio *)s))
>
>  /**
> - * page_slab - Converts from first struct page to slab.
> - * @p: The first (either head of compound or single) page of slab.
> + * page_slab - Converts from struct page to its slab.
> + * @page: A page which may or may not belong to a slab.
>   *
> - * A temporary wrapper to convert struct page to struct slab in situations where
> - * we know the page is the compound head, or single order-0 page.
> - *
> - * Long-term ideally everything would work with struct slab directly or go
> - * through folio to struct slab.
> - *
> - * Return: The slab which contains this page
> + * Return: The slab which contains this page or NULL if the page does
> + * not belong to a slab.  This includes pages returned from large kmalloc.
>   */
> -#define page_slab(p)           (_Generic((p),                          \
> -       const struct page *:    (const struct slab *)(p),               \
> -       struct page *:          (struct slab *)(p)))
> +static inline struct slab *page_slab(const struct page *page)
> +{
> +       unsigned long head;
> +
> +       head = READ_ONCE(page->compound_head);
> +       if (head & 1)
> +               page = (struct page *)(head - 1);
> +       if (data_race(page->page_type >> 24) != PGTY_slab)
> +               page = NULL;
> +
> +       return (struct slab *)page;
> +}
>
>  /**
>   * slab_page - The first struct page allocated for a slab
> --
> 2.47.2
>

Re: [PATCH v4 14/16] memcg: Convert mem_cgroup_from_obj_folio() to mem_cgroup_from_obj_slab()

[Johannes Weiner]

On Thu, Nov 13, 2025 at 12:09:28AM +0000, Matthew Wilcox (Oracle) wrote:
> -	/*
> -	 * folio_memcg_check() is used here, because in theory we can encounter
> -	 * a folio where the slab flag has been cleared already, but
> -	 * slab->obj_exts has not been freed yet
> -	 * folio_memcg_check() will guarantee that a proper memory
> -	 * cgroup pointer or NULL will be returned.
> -	 */
> -	return folio_memcg_check(folio);
> +	off = obj_to_index(slab->slab_cache, slab, p);
> +	if (obj_exts[off].objcg)
> +		return obj_cgroup_memcg(obj_exts[off].objcg);
> +
> +	return NULL;
>  }
>  
>  /*
> @@ -2637,7 +2627,7 @@ struct mem_cgroup *mem_cgroup_from_slab_obj(void *p)
>  	if (mem_cgroup_disabled())
>  		return NULL;
>  
> -	return mem_cgroup_from_obj_folio(virt_to_folio(p), p);
> +	return mem_cgroup_from_obj_slab(virt_to_slab(p), p);

The name undoubtedly sucks, but there is a comment above this function
that this can be used on non-slab kernel pages as well.

E.g. !vmap kernel stack pages -> mod_lruvec_kmem_state -> mem_cgroup_from_obj_slab

How about:

	if ((slab = virt_to_slap(p)))
		return mem_cgroup_from_obj_slab(slab, p);
	return folio_memcg_check(virt_to_folio(p), p);

Re: [PATCH v4 14/16] memcg: Convert mem_cgroup_from_obj_folio() to mem_cgroup_from_obj_slab()

[Vlastimil Babka]

On 11/13/25 17:14, Johannes Weiner wrote:
> On Thu, Nov 13, 2025 at 12:09:28AM +0000, Matthew Wilcox (Oracle) wrote:
>> -	/*
>> -	 * folio_memcg_check() is used here, because in theory we can encounter
>> -	 * a folio where the slab flag has been cleared already, but
>> -	 * slab->obj_exts has not been freed yet
>> -	 * folio_memcg_check() will guarantee that a proper memory
>> -	 * cgroup pointer or NULL will be returned.
>> -	 */

I've only noticed this comment and thought it describes something impossible
to happen (maybe it was possible in the past) so seemed fine to delete it.

>> -	return folio_memcg_check(folio);
>> +	off = obj_to_index(slab->slab_cache, slab, p);
>> +	if (obj_exts[off].objcg)
>> +		return obj_cgroup_memcg(obj_exts[off].objcg);
>> +
>> +	return NULL;
>>  }
>>  
>>  /*
>> @@ -2637,7 +2627,7 @@ struct mem_cgroup *mem_cgroup_from_slab_obj(void *p)
>>  	if (mem_cgroup_disabled())
>>  		return NULL;
>>  
>> -	return mem_cgroup_from_obj_folio(virt_to_folio(p), p);
>> +	return mem_cgroup_from_obj_slab(virt_to_slab(p), p);
> 
> The name undoubtedly sucks, but there is a comment above this function
> that this can be used on non-slab kernel pages as well.

Didn't notice this one.

> E.g. !vmap kernel stack pages -> mod_lruvec_kmem_state -> mem_cgroup_from_obj_slab
> 
> How about:
> 
> 	if ((slab = virt_to_slap(p)))
> 		return mem_cgroup_from_obj_slab(slab, p);
> 	return folio_memcg_check(virt_to_folio(p), p);

page_memcg_check() maybe instead? we shouldn't get a tail page here, no?

Re: [PATCH v4 14/16] memcg: Convert mem_cgroup_from_obj_folio() to mem_cgroup_from_obj_slab()

[Matthew Wilcox]

On Thu, Nov 13, 2025 at 11:14:24AM -0500, Johannes Weiner wrote:
> On Thu, Nov 13, 2025 at 12:09:28AM +0000, Matthew Wilcox (Oracle) wrote:
> > -	/*
> > -	 * folio_memcg_check() is used here, because in theory we can encounter
> > -	 * a folio where the slab flag has been cleared already, but
> > -	 * slab->obj_exts has not been freed yet
> > -	 * folio_memcg_check() will guarantee that a proper memory
> > -	 * cgroup pointer or NULL will be returned.
> > -	 */
> > -	return folio_memcg_check(folio);
> > +	off = obj_to_index(slab->slab_cache, slab, p);
> > +	if (obj_exts[off].objcg)
> > +		return obj_cgroup_memcg(obj_exts[off].objcg);
> > +
> > +	return NULL;
> >  }
> >  
> >  /*
> > @@ -2637,7 +2627,7 @@ struct mem_cgroup *mem_cgroup_from_slab_obj(void *p)
> >  	if (mem_cgroup_disabled())
> >  		return NULL;
> >  
> > -	return mem_cgroup_from_obj_folio(virt_to_folio(p), p);
> > +	return mem_cgroup_from_obj_slab(virt_to_slab(p), p);
> 
> The name undoubtedly sucks, but there is a comment above this function
> that this can be used on non-slab kernel pages as well.

Oh, I see.  Usercopy calls this kind of thing 'heap object', so
perhaps eventually rename this function to mem_cgroup_from_heap_obj()?

> E.g. !vmap kernel stack pages -> mod_lruvec_kmem_state -> mem_cgroup_from_obj_slab

That actually seems to be the only user ... and I wasn't testing on a
!VMAP_STACK build, so I wouldn't've caught it.

> How about:
> 
> 	if ((slab = virt_to_slap(p)))
> 		return mem_cgroup_from_obj_slab(slab, p);
> 	return folio_memcg_check(virt_to_folio(p), p);

Mild updates, here's my counteroffer:

commit 6ca8243530e4
Author: Matthew Wilcox (Oracle) <willy@infradead.org>
Date:   Thu Nov 13 11:30:59 2025 -0500

    fix-memcg-slab

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 597673e272a9..a2e6f409c5e8 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -2599,9 +2599,6 @@ struct mem_cgroup *mem_cgroup_from_obj_slab(struct slab *slab, void *p)
 	struct slabobj_ext *obj_exts;
 	unsigned int off;
 
-	if (!slab)
-		return NULL;
-
 	obj_exts = slab_obj_exts(slab);
 	if (!obj_exts)
 		return NULL;
@@ -2624,10 +2621,15 @@ struct mem_cgroup *mem_cgroup_from_obj_slab(struct slab *slab, void *p)
  */
 struct mem_cgroup *mem_cgroup_from_slab_obj(void *p)
 {
+	struct slab *slab;
+
 	if (mem_cgroup_disabled())
 		return NULL;
 
-	return mem_cgroup_from_obj_slab(virt_to_slab(p), p);
+	slab = virt_to_slab(p);
+	if (slab)
+		return mem_cgroup_from_obj_slab(slab, p);
+	return folio_memcg_check(virt_to_folio(p));
 }
 
 static struct obj_cgroup *__get_obj_cgroup_from_memcg(struct mem_cgroup *memcg)

Re: [PATCH v4 14/16] memcg: Convert mem_cgroup_from_obj_folio() to mem_cgroup_from_obj_slab()

[Johannes Weiner]

On Thu, Nov 13, 2025 at 04:39:41PM +0000, Matthew Wilcox wrote:
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -2599,9 +2599,6 @@ struct mem_cgroup *mem_cgroup_from_obj_slab(struct slab *slab, void *p)
>  	struct slabobj_ext *obj_exts;
>  	unsigned int off;
>  
> -	if (!slab)
> -		return NULL;
> -
>  	obj_exts = slab_obj_exts(slab);
>  	if (!obj_exts)
>  		return NULL;
> @@ -2624,10 +2621,15 @@ struct mem_cgroup *mem_cgroup_from_obj_slab(struct slab *slab, void *p)
>   */
>  struct mem_cgroup *mem_cgroup_from_slab_obj(void *p)
>  {
> +	struct slab *slab;
> +
>  	if (mem_cgroup_disabled())
>  		return NULL;
>  
> -	return mem_cgroup_from_obj_slab(virt_to_slab(p), p);
> +	slab = virt_to_slab(p);
> +	if (slab)
> +		return mem_cgroup_from_obj_slab(slab, p);
> +	return folio_memcg_check(virt_to_folio(p));

Looks good to me, thanks!

With that folded in, for the combined patch:

Acked-by: Johannes Weiner <hannes@cmpxchg.org>

Re: [PATCH v4 14/16] memcg: Convert mem_cgroup_from_obj_folio() to mem_cgroup_from_obj_slab()

[Vlastimil Babka]

On 11/13/25 20:16, Johannes Weiner wrote:
> On Thu, Nov 13, 2025 at 04:39:41PM +0000, Matthew Wilcox wrote:
>> --- a/mm/memcontrol.c
>> +++ b/mm/memcontrol.c
>> @@ -2599,9 +2599,6 @@ struct mem_cgroup *mem_cgroup_from_obj_slab(struct slab *slab, void *p)
>>  	struct slabobj_ext *obj_exts;
>>  	unsigned int off;
>>  
>> -	if (!slab)
>> -		return NULL;
>> -
>>  	obj_exts = slab_obj_exts(slab);
>>  	if (!obj_exts)
>>  		return NULL;
>> @@ -2624,10 +2621,15 @@ struct mem_cgroup *mem_cgroup_from_obj_slab(struct slab *slab, void *p)
>>   */
>>  struct mem_cgroup *mem_cgroup_from_slab_obj(void *p)
>>  {
>> +	struct slab *slab;
>> +
>>  	if (mem_cgroup_disabled())
>>  		return NULL;
>>  
>> -	return mem_cgroup_from_obj_slab(virt_to_slab(p), p);
>> +	slab = virt_to_slab(p);
>> +	if (slab)
>> +		return mem_cgroup_from_obj_slab(slab, p);
>> +	return folio_memcg_check(virt_to_folio(p));
> 
> Looks good to me, thanks!
> 
> With that folded in, for the combined patch:
> 
> Acked-by: Johannes Weiner <hannes@cmpxchg.org>

Thanks, folded.

Re: [PATCH v4 14/16] memcg: Convert mem_cgroup_from_obj_folio() to mem_cgroup_from_obj_slab()

[Shakeel Butt]

On Thu, Nov 13, 2025 at 05:28:59PM +0100, Vlastimil Babka wrote:
> > E.g. !vmap kernel stack pages -> mod_lruvec_kmem_state -> mem_cgroup_from_obj_slab
> > 
> > How about:
> > 
> > 	if ((slab = virt_to_slap(p)))
> > 		return mem_cgroup_from_obj_slab(slab, p);
> > 	return folio_memcg_check(virt_to_folio(p), p);
> 
> page_memcg_check() maybe instead? we shouldn't get a tail page here, no?

Do you mean page_memcg_check(virt_to_page(p), p)? But virt_to_page(p)
can return tail page, right?

Re: [PATCH v4 14/16] memcg: Convert mem_cgroup_from_obj_folio() to mem_cgroup_from_obj_slab()

[Matthew Wilcox]

On Thu, Nov 13, 2025 at 11:42:01AM -0800, Shakeel Butt wrote:
> On Thu, Nov 13, 2025 at 05:28:59PM +0100, Vlastimil Babka wrote:
> > > E.g. !vmap kernel stack pages -> mod_lruvec_kmem_state -> mem_cgroup_from_obj_slab
> > > 
> > > How about:
> > > 
> > > 	if ((slab = virt_to_slap(p)))
> > > 		return mem_cgroup_from_obj_slab(slab, p);
> > > 	return folio_memcg_check(virt_to_folio(p), p);
> > 
> > page_memcg_check() maybe instead? we shouldn't get a tail page here, no?
> 
> Do you mean page_memcg_check(virt_to_page(p), p)? But virt_to_page(p)
> can return tail page, right?

Only if it's legitimate to call mod_lruvec_kmem_state() with "a pointer
to somewhere inside the object" rather than "a pointer to the object".

For example, it's legitimate to call copy_to_user() with a pointer
somewhere inside the object, so the usercopy code has to handle that
case.  But it's only legitimate to call kfree() with a pointer that's
at the start of an object, so kfree() can cheerfully BUG_ON()
PageTail(virt_to_page(p)).

Re: [PATCH v4 14/16] memcg: Convert mem_cgroup_from_obj_folio() to mem_cgroup_from_obj_slab()

[Shakeel Butt]

On Thu, Nov 13, 2025 at 08:33:50PM +0000, Matthew Wilcox wrote:
> On Thu, Nov 13, 2025 at 11:42:01AM -0800, Shakeel Butt wrote:
> > On Thu, Nov 13, 2025 at 05:28:59PM +0100, Vlastimil Babka wrote:
> > > > E.g. !vmap kernel stack pages -> mod_lruvec_kmem_state -> mem_cgroup_from_obj_slab
> > > > 
> > > > How about:
> > > > 
> > > > 	if ((slab = virt_to_slap(p)))
> > > > 		return mem_cgroup_from_obj_slab(slab, p);
> > > > 	return folio_memcg_check(virt_to_folio(p), p);
> > > 
> > > page_memcg_check() maybe instead? we shouldn't get a tail page here, no?
> > 
> > Do you mean page_memcg_check(virt_to_page(p), p)? But virt_to_page(p)
> > can return tail page, right?
> 
> Only if it's legitimate to call mod_lruvec_kmem_state() with "a pointer
> to somewhere inside the object" rather than "a pointer to the object".
> 
> For example, it's legitimate to call copy_to_user() with a pointer
> somewhere inside the object, so the usercopy code has to handle that
> case.  But it's only legitimate to call kfree() with a pointer that's
> at the start of an object, so kfree() can cheerfully BUG_ON()
> PageTail(virt_to_page(p)).

Yes it makes sense. Though for slab objects we are allowing to pass any
address within the object (address of list_head for list_lru) to
mem_cgroup_from_slab_obj() (and thus mod_lruvec_kmem_state()) but for
normal kernel memory, if we use virt_to_page() then we will restrict to
the starting address. Anyways just noticed this pecularity, not
suggesting to do anything differently.

Re: [PATCH v4 01/16] slab: Reimplement page_slab()

[Harry Yoo]

On Thu, Nov 13, 2025 at 12:09:15AM +0000, Matthew Wilcox (Oracle) wrote:
> In order to separate slabs from folios, we need to convert from any page
> in a slab to the slab directly without going through a page to folio
> conversion first.
> 
> Up to this point, page_slab() has followed the example of other memdesc
> converters (page_folio(), page_ptdesc() etc) and just cast the pointer
> to the requested type, regardless of whether the pointer is actually a
> pointer to the correct type or not.
> 
> That changes with this commit; we check that the page actually belongs
> to a slab and return NULL if it does not.  Other memdesc converters will
> adopt this convention in future.
> 
> kfence was the only user of page_slab(), so adjust it to the new way
> of working.  It will need to be touched again when we separate slab
> from page.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> Cc: Alexander Potapenko <glider@google.com>
> Cc: Marco Elver <elver@google.com>
> Cc: kasan-dev@googlegroups.com
> ---

Looks good to me,
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>

-- 
Cheers,
Harry / Hyeonggon

Re: [PATCH v4 02/16] slab: Remove folio references from __ksize()

[Harry Yoo]

On Thu, Nov 13, 2025 at 12:09:16AM +0000, Matthew Wilcox (Oracle) wrote:
> In the future, we will separate slab, folio and page from each other
> and calling virt_to_folio() on an address allocated from slab will
> return NULL.  Delay the conversion from struct page to struct slab
> until we know we're not dealing with a large kmalloc allocation.
> There's a minor win for large kmalloc allocations as we avoid the
> compound_head() hidden in virt_to_folio().
> 
> This deprecates calling ksize() on memory allocated by alloc_pages().
> Today it becomes a warning and support will be removed entirely in
> the future.
> 
> Introduce large_kmalloc_size() to abstract how we represent the size
> of a large kmalloc allocation.  For now, this is the same as
> page_size(), but it will change with separately allocated memdescs.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> ---

Looks good to me,
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>

With a minor comment below.

>  include/linux/page-flags.h |  2 +-
>  mm/slab.h                  | 10 ++++++++++
>  mm/slab_common.c           | 23 ++++++++++++-----------
>  3 files changed, 23 insertions(+), 12 deletions(-)
> 
> diff --git a/mm/slab_common.c b/mm/slab_common.c
> index d2824daa98cf..236b4e25fce0 100644
> --- a/mm/slab_common.c
> +++ b/mm/slab_common.c
> @@ -997,26 +997,27 @@ void __init create_kmalloc_caches(void)
>   */
>  size_t __ksize(const void *object)
>  {
> -	struct folio *folio;
> +	const struct page *page;
> +	const struct slab *slab;
>  
>  	if (unlikely(object == ZERO_SIZE_PTR))
>  		return 0;
>  
> -	folio = virt_to_folio(object);
> +	page = virt_to_page(object);
>  
> -	if (unlikely(!folio_test_slab(folio))) {
> -		if (WARN_ON(folio_size(folio) <= KMALLOC_MAX_CACHE_SIZE))
> -			return 0;
> -		if (WARN_ON(object != folio_address(folio)))
> -			return 0;

nit: we can still have these sanity checks even when we're not converting it
to folio?

size_t size = large_kmalloc_size(page);
if (WARN_ON(size <= KMALLOC_MAX_CACHE_SIZE))
    return 0;
if (WARN_ON(object & (size - 1))
    return 0;
return size;

-- 
Cheers,
Harry / Hyeonggon

Re: [PATCH v4 02/16] slab: Remove folio references from __ksize()

[Matthew Wilcox]

On Mon, Nov 24, 2025 at 11:31:06AM +0900, Harry Yoo wrote:
> >  size_t __ksize(const void *object)
> >  {
> > -	struct folio *folio;
> > +	const struct page *page;
> > +	const struct slab *slab;
> >  
> >  	if (unlikely(object == ZERO_SIZE_PTR))
> >  		return 0;
> >  
> > -	folio = virt_to_folio(object);
> > +	page = virt_to_page(object);
> >  
> > -	if (unlikely(!folio_test_slab(folio))) {
> > -		if (WARN_ON(folio_size(folio) <= KMALLOC_MAX_CACHE_SIZE))
> > -			return 0;
> > -		if (WARN_ON(object != folio_address(folio)))
> > -			return 0;
> 
> nit: we can still have these sanity checks even when we're not converting it
> to folio?
> 
> size_t size = large_kmalloc_size(page);
> if (WARN_ON(size <= KMALLOC_MAX_CACHE_SIZE))
>     return 0;
> if (WARN_ON(object & (size - 1))
>     return 0;
> return size;

We could!  However, I decided that we had these extra warnings to catch
some uses of ksize() on non-kmalloc-large objects, and so they were
really caught by the new:

+       if (WARN_ON(!slab))
+               return page_size(page);

Now, you could argue that we're now not warning for ksize(kmalloc() + N)
for N in the range 1..4095.  But I'm not sure it's particularly useful
to be warning for those cases.

I suppose size <= KMALLOC_MAX_CACHE_SIZE would also catch an internal
bug in kmalloc where we set PageLargeKmalloc on a small allocation,
but given how few places actually call ksize(), I doubt this would be
a useful self-check either.

Re: [PATCH v4 02/16] slab: Remove folio references from __ksize()

[Harry Yoo]

On Mon, Nov 24, 2025 at 04:28:20AM +0000, Matthew Wilcox wrote:
> On Mon, Nov 24, 2025 at 11:31:06AM +0900, Harry Yoo wrote:
> > >  size_t __ksize(const void *object)
> > >  {
> > > -	struct folio *folio;
> > > +	const struct page *page;
> > > +	const struct slab *slab;
> > >  
> > >  	if (unlikely(object == ZERO_SIZE_PTR))
> > >  		return 0;
> > >  
> > > -	folio = virt_to_folio(object);
> > > +	page = virt_to_page(object);
> > >  
> > > -	if (unlikely(!folio_test_slab(folio))) {
> > > -		if (WARN_ON(folio_size(folio) <= KMALLOC_MAX_CACHE_SIZE))
> > > -			return 0;
> > > -		if (WARN_ON(object != folio_address(folio)))
> > > -			return 0;
> > 
> > nit: we can still have these sanity checks even when we're not converting it
> > to folio?
> > 
> > size_t size = large_kmalloc_size(page);
> > if (WARN_ON(size <= KMALLOC_MAX_CACHE_SIZE))
> >     return 0;
> > if (WARN_ON(object & (size - 1))
> >     return 0;
> > return size;
> 
> We could!  However, I decided that we had these extra warnings to catch
> some uses of ksize() on non-kmalloc-large objects, and so they were
> really caught by the new:
> 
> +       if (WARN_ON(!slab))
> +               return page_size(page);

That makes sense!

> Now, you could argue that we're now not warning for ksize(kmalloc() + N)
> for N in the range 1..4095.  But I'm not sure it's particularly useful
> to be warning for those cases.
>
> I suppose size <= KMALLOC_MAX_CACHE_SIZE would also catch an internal
> bug in kmalloc where we set PageLargeKmalloc on a small allocation,
> but given how few places actually call ksize(), I doubt this would be
> a useful self-check either.

Yeah they are a bit paranoid and I don't have enough evidence to argue
that they'll be particularly useful..

Looks like my nitpicking failed ;)
Thanks!

-- 
Cheers,
Harry / Hyeonggon

Re: [PATCH v4 07/16] slab: Remove folio references from kvfree_rcu_cb()

[Harry Yoo]

On Thu, Nov 13, 2025 at 12:09:21AM +0000, Matthew Wilcox (Oracle) wrote:
> Remove conversions from folio to page and folio to slab.  This is
> preparation for separately allocated struct slab from struct page.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> ---

Looks good to me,
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>

-- 
Cheers,
Harry / Hyeonggon

Re: [PATCH v4 08/16] slab: Remove folio references from kfree()

[Harry Yoo]

On Thu, Nov 13, 2025 at 12:09:22AM +0000, Matthew Wilcox (Oracle) wrote:
> This should generate identical code to the previous version, but
> without any dependency on how folios work.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> ---

Looks good to me,
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>

with a nit below.

>  mm/slub.c | 10 +++++-----
>  1 file changed, 5 insertions(+), 5 deletions(-)
> 
> diff --git a/mm/slub.c b/mm/slub.c
> index 976e116580a6..184bab91e0af 100644
> --- a/mm/slub.c
> +++ b/mm/slub.c
> @@ -6866,7 +6866,7 @@ void kvfree_rcu_cb(struct rcu_head *head)
>   */
>  void kfree(const void *object)
>  {
> -	struct folio *folio;
> +	struct page *page;
>  	struct slab *slab;
>  	struct kmem_cache *s;
>  	void *x = (void *)object;
> @@ -6876,13 +6876,13 @@ void kfree(const void *object)
>  	if (unlikely(ZERO_OR_NULL_PTR(object)))
>  		return;
>  
> -	folio = virt_to_folio(object);
> -	if (unlikely(!folio_test_slab(folio))) {
> -		free_large_kmalloc(&folio->page, (void *)object);
> +	page = virt_to_page(object);
> +	slab = page_slab(page);
> +	if (!slab) {

nit: is it intentional to remove unlikely() hint?

> +		free_large_kmalloc(page, (void *)object);
>  		return;
>  	}
>  
> -	slab = folio_slab(folio);
>  	s = slab->slab_cache;
>  	slab_free(s, slab, x, _RET_IP_);
>  }
> -- 
> 2.47.2

-- 
Cheers,
Harry / Hyeonggon

Re: [PATCH v4 09/16] slab: Remove folio references from __do_krealloc()

[Harry Yoo]

On Thu, Nov 13, 2025 at 12:09:23AM +0000, Matthew Wilcox (Oracle) wrote:
> One slight tweak I made is to calculate 'ks' earlier, which means we
> can reuse it in the warning rather than calculating the object size twice.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> ---

Looks good to me,
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>

>  mm/slub.c | 14 +++++++-------
>  1 file changed, 7 insertions(+), 7 deletions(-)
> 
> diff --git a/mm/slub.c b/mm/slub.c
> index 184bab91e0af..4570d1dce271 100644
> --- a/mm/slub.c
> +++ b/mm/slub.c
> @@ -6977,16 +6977,16 @@ __do_krealloc(const void *p, size_t new_size, unsigned long align, gfp_t flags,
>  	if (is_kfence_address(p)) {
>  		ks = orig_size = kfence_ksize(p);
>  	} else {
> -		struct folio *folio;
> +		struct page *page = virt_to_page(p);
> +		struct slab *slab = page_slab(page);
>  
> -		folio = virt_to_folio(p);
> -		if (unlikely(!folio_test_slab(folio))) {
> +		if (!slab) {

nit: unlikely() is gone.

>  			/* Big kmalloc object */
> -			WARN_ON(folio_size(folio) <= KMALLOC_MAX_CACHE_SIZE);
> -			WARN_ON(p != folio_address(folio));
> -			ks = folio_size(folio);
> +			ks = page_size(page);
> +			WARN_ON(ks <= KMALLOC_MAX_CACHE_SIZE);
> +			WARN_ON(p != page_address(page));
>  		} else {
> -			s = folio_slab(folio)->slab_cache;
> +			s = slab->slab_cache;
>  			orig_size = get_orig_size(s, (void *)p);
>  			ks = s->object_size;
>  		}
> -- 
> 2.47.2

-- 
Cheers,
Harry / Hyeonggon

Re: [PATCH v4 10/16] slab: Remove folio references from build_detached_freelist()

[Harry Yoo]

On Thu, Nov 13, 2025 at 12:09:24AM +0000, Matthew Wilcox (Oracle) wrote:
> Use pages and slabs directly instead of converting to folios.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> ---

Looks good to me,
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>

>  mm/slub.c | 16 +++++++++-------
>  1 file changed, 9 insertions(+), 7 deletions(-)
> 
> diff --git a/mm/slub.c b/mm/slub.c
> index 4570d1dce271..17b456f6fd4e 100644
> --- a/mm/slub.c
> +++ b/mm/slub.c
> @@ -7297,23 +7297,25 @@ int build_detached_freelist(struct kmem_cache *s, size_t size,
>  {
>  	int lookahead = 3;
>  	void *object;
> -	struct folio *folio;
> +	struct page *page;
> +	struct slab *slab;
>  	size_t same;
>  
>  	object = p[--size];
> -	folio = virt_to_folio(object);
> +	page = virt_to_page(object);
> +	slab = page_slab(page);
>  	if (!s) {
>  		/* Handle kalloc'ed objects */
> -		if (unlikely(!folio_test_slab(folio))) {
> -			free_large_kmalloc(&folio->page, object);
> +		if (!slab) {

nit: unlikely() is gone.

> +			free_large_kmalloc(page, object);
>  			df->slab = NULL;
>  			return size;
>  		}
>  		/* Derive kmem_cache from object */
> -		df->slab = folio_slab(folio);
> -		df->s = df->slab->slab_cache;
> +		df->slab = slab;
> +		df->s = slab->slab_cache;
>  	} else {
> -		df->slab = folio_slab(folio);
> +		df->slab = slab;
>  		df->s = cache_from_obj(s, object); /* Support for memcg */
>  	}
>  
> -- 
> 2.47.2
> 

-- 
Cheers,
Harry / Hyeonggon

Re: [PATCH v4 11/16] slab: Remove folio references from kfree_rcu_sheaf()

[Harry Yoo]

On Thu, Nov 13, 2025 at 12:09:25AM +0000, Matthew Wilcox (Oracle) wrote:
> In preparation for splitting struct slab from struct page and struct
> folio, remove mentions of struct folio from this function.  Since
> we don't need to handle large kmalloc objects specially here, we
> can just use virt_to_slab().
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> ---

Looks good to me,
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>

-- 
Cheers,
Harry / Hyeonggon

Re: [PATCH v4 12/16] slab: Remove folio references from kfree_nolock()

[Harry Yoo]

On Thu, Nov 13, 2025 at 12:09:26AM +0000, Matthew Wilcox (Oracle) wrote:
> In preparation for splitting struct slab from struct page and struct
> folio, remove mentions of struct folio from this function.  Since large
> kmalloc objects are not supported here, we can just use virt_to_slab().
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> ---

Looks good to me,
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>

-- 
Cheers,
Harry / Hyeonggon

Re: [PATCH v4 13/16] usercopy: Remove folio references from check_heap_object()

[Harry Yoo]

On Thu, Nov 13, 2025 at 12:09:27AM +0000, Matthew Wilcox (Oracle) wrote:
> Use page_slab() instead of virt_to_folio() followed by folio_slab().
> We do end up calling compound_head() twice for non-slab copies, but that
> will not be a problem once we allocate memdescs separately.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> Cc: Kees Cook <kees@kernel.org>
> Cc: "Gustavo A. R. Silva" <gustavoars@kernel.org>
> Cc: linux-hardening@vger.kernel.org
> ---

Looks good to me,
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>

-- 
Cheers,
Harry / Hyeonggon

Re: [PATCH v4 14/16] memcg: Convert mem_cgroup_from_obj_folio() to mem_cgroup_from_obj_slab()

[Harry Yoo]

On Thu, Nov 13, 2025 at 04:39:41PM +0000, Matthew Wilcox wrote:
> On Thu, Nov 13, 2025 at 11:14:24AM -0500, Johannes Weiner wrote:
> > On Thu, Nov 13, 2025 at 12:09:28AM +0000, Matthew Wilcox (Oracle) wrote:
> > > -	/*
> > > -	 * folio_memcg_check() is used here, because in theory we can encounter
> > > -	 * a folio where the slab flag has been cleared already, but
> > > -	 * slab->obj_exts has not been freed yet
> > > -	 * folio_memcg_check() will guarantee that a proper memory
> > > -	 * cgroup pointer or NULL will be returned.
> > > -	 */
> > > -	return folio_memcg_check(folio);
> > > +	off = obj_to_index(slab->slab_cache, slab, p);
> > > +	if (obj_exts[off].objcg)
> > > +		return obj_cgroup_memcg(obj_exts[off].objcg);
> > > +
> > > +	return NULL;
> > >  }
> > >  
> > >  /*
> > > @@ -2637,7 +2627,7 @@ struct mem_cgroup *mem_cgroup_from_slab_obj(void *p)
> > >  	if (mem_cgroup_disabled())
> > >  		return NULL;
> > >  
> > > -	return mem_cgroup_from_obj_folio(virt_to_folio(p), p);
> > > +	return mem_cgroup_from_obj_slab(virt_to_slab(p), p);
> > 
> > The name undoubtedly sucks, but there is a comment above this function
> > that this can be used on non-slab kernel pages as well.
> 
> Oh, I see.  Usercopy calls this kind of thing 'heap object', so
> perhaps eventually rename this function to mem_cgroup_from_heap_obj()?
> 
> > E.g. !vmap kernel stack pages -> mod_lruvec_kmem_state -> mem_cgroup_from_obj_slab
> 
> That actually seems to be the only user ... and I wasn't testing on a
> !VMAP_STACK build, so I wouldn't've caught it.
> 
> > How about:
> > 
> > 	if ((slab = virt_to_slap(p)))
> > 		return mem_cgroup_from_obj_slab(slab, p);
> > 	return folio_memcg_check(virt_to_folio(p), p);
> 
> Mild updates, here's my counteroffer:
> 
> commit 6ca8243530e4
> Author: Matthew Wilcox (Oracle) <willy@infradead.org>
> Date:   Thu Nov 13 11:30:59 2025 -0500
> 
>     fix-memcg-slab

With this folded,
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>

-- 
Cheers,
Harry / Hyeonggon

> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index 597673e272a9..a2e6f409c5e8 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -2599,9 +2599,6 @@ struct mem_cgroup *mem_cgroup_from_obj_slab(struct slab *slab, void *p)
>  	struct slabobj_ext *obj_exts;
>  	unsigned int off;
>  
> -	if (!slab)
> -		return NULL;
> -
>  	obj_exts = slab_obj_exts(slab);
>  	if (!obj_exts)
>  		return NULL;
> @@ -2624,10 +2621,15 @@ struct mem_cgroup *mem_cgroup_from_obj_slab(struct slab *slab, void *p)
>   */
>  struct mem_cgroup *mem_cgroup_from_slab_obj(void *p)
>  {
> +	struct slab *slab;
> +
>  	if (mem_cgroup_disabled())
>  		return NULL;
>  
> -	return mem_cgroup_from_obj_slab(virt_to_slab(p), p);
> +	slab = virt_to_slab(p);
> +	if (slab)
> +		return mem_cgroup_from_obj_slab(slab, p);
> +	return folio_memcg_check(virt_to_folio(p));
>  }
>  
>  static struct obj_cgroup *__get_obj_cgroup_from_memcg(struct mem_cgroup *memcg)

Re: [PATCH v4 15/16] kasan: Remove references to folio in __kasan_mempool_poison_object()

[Harry Yoo]

On Thu, Nov 13, 2025 at 12:09:29AM +0000, Matthew Wilcox (Oracle) wrote:
> In preparation for splitting struct slab from struct page and struct
> folio, remove mentions of struct folio from this function.  There is a
> mild improvement for large kmalloc objects as we will avoid calling
> compound_head() for them.  We can discard the comment as using
> PageLargeKmalloc() rather than !folio_test_slab() makes it obvious.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> Acked-by: David Hildenbrand <david@redhat.com>
> Cc: Alexander Potapenko <glider@google.com>
> Cc: Andrey Konovalov <andreyknvl@gmail.com>
> Cc: Dmitry Vyukov <dvyukov@google.com>
> Cc: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Cc: kasan-dev <kasan-dev@googlegroups.com>
> ---

Acked-by: Harry Yoo <harry.yoo@oracle.com>

>  mm/kasan/common.c | 12 ++++--------
>  1 file changed, 4 insertions(+), 8 deletions(-)
> 
> diff --git a/mm/kasan/common.c b/mm/kasan/common.c
> index 22e5d67ff064..1d27f1bd260b 100644
> --- a/mm/kasan/common.c
> +++ b/mm/kasan/common.c
> @@ -517,24 +517,20 @@ void __kasan_mempool_unpoison_pages(struct page *page, unsigned int order,
>  
>  bool __kasan_mempool_poison_object(void *ptr, unsigned long ip)
>  {
> -	struct folio *folio = virt_to_folio(ptr);
> +	struct page *page = virt_to_page(ptr);
>  	struct slab *slab;
>  
> -	/*
> -	 * This function can be called for large kmalloc allocation that get
> -	 * their memory from page_alloc. Thus, the folio might not be a slab.
> -	 */
> -	if (unlikely(!folio_test_slab(folio))) {
> +	if (unlikely(PageLargeKmalloc(page))) {

nit: no strong opinion from me, but maybe KASAN folks still want to catch
!PageLargeKmalloc() && !slab case gracefully, as they care more about
detecting invalid frees than performance.


>  		if (check_page_allocation(ptr, ip))
>  			return false;
> -		kasan_poison(ptr, folio_size(folio), KASAN_PAGE_FREE, false);
> +		kasan_poison(ptr, page_size(page), KASAN_PAGE_FREE, false);
>  		return true;
>  	}
>  
>  	if (is_kfence_address(ptr))
>  		return true;
>  
> -	slab = folio_slab(folio);
> +	slab = page_slab(page);
>  
>  	if (check_slab_allocation(slab->slab_cache, ptr, ip))
>  		return false;
> -- 
> 2.47.2

-- 
Cheers,
Harry / Hyeonggon

Re: [PATCH v4 16/16] slab: Remove references to folios from virt_to_slab()

[Harry Yoo]

On Thu, Nov 13, 2025 at 12:09:30AM +0000, Matthew Wilcox (Oracle) wrote:
> Use page_slab() instead of virt_to_folio() which will work
> perfectly when struct slab is separated from struct folio.
> 
> This was the last user of folio_slab(), so delete it.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> ---

Looks good to me,
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>

Woo-hoo, finally the last patch!

By the way, there is one remaining code snippet that references folio in
alloc_from_pcs():

  if (folio_nid(virt_to_folio(object)) != node)

which won't work after separating struct slab from struct folio :)

-- 
Cheers,
Harry / Hyeonggon

Re: [PATCH v4 08/16] slab: Remove folio references from kfree()

[Matthew Wilcox]

On Mon, Nov 24, 2025 at 02:39:13PM +0900, Harry Yoo wrote:
> > -	folio = virt_to_folio(object);
> > -	if (unlikely(!folio_test_slab(folio))) {
> > -		free_large_kmalloc(&folio->page, (void *)object);
> > +	page = virt_to_page(object);
> > +	slab = page_slab(page);
> > +	if (!slab) {
> 
> nit: is it intentional to remove unlikely() hint?

It is my understanding that the compiler treats NULL pointeer checks as
unlikely by default so we don't need explicit annotations.

I could have an incorrect understanding though!

Re: [PATCH v4 16/16] slab: Remove references to folios from virt_to_slab()

[Matthew Wilcox]

On Mon, Nov 24, 2025 at 04:09:32PM +0900, Harry Yoo wrote:
> By the way, there is one remaining code snippet that references folio in
> alloc_from_pcs():
> 
>   if (folio_nid(virt_to_folio(object)) != node)
> 
> which won't work after separating struct slab from struct folio :)

Oh, thank you!  This is a fairly silly thing to do today as there's
an unnecessary use of compound_head() to go from page to folio.
Each page knows which node it belongs to (and that's going to continue
to be true until we get to an 8-byte struct page) so there's no need
to do page_to_memdesc().  It'll be worth it once we do get to an 8 byte
struct page as we'll know what kind of memdesc this is rather than
having page_to_nid() figure it out.

Patch soon.

Re: [PATCH v4 13/16] usercopy: Remove folio references from check_heap_object()

[Kees Cook]

On Thu, Nov 13, 2025 at 12:09:27AM +0000, Matthew Wilcox (Oracle) wrote:
> Use page_slab() instead of virt_to_folio() followed by folio_slab().
> We do end up calling compound_head() twice for non-slab copies, but that
> will not be a problem once we allocate memdescs separately.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>

Reviewed-by: Kees Cook <kees@kernel.org>

-- 
Kees Cook