From: Vlastimil Babka <vbabka@suse.cz>
To: Matthew Wilcox <willy@infradead.org>,
Christoph Lameter <cl@linux.com>,
David Rientjes <rientjes@google.com>,
Joonsoo Kim <iamjoonsoo.kim@lge.com>,
Pekka Enberg <penberg@kernel.org>
Cc: linux-mm@kvack.org, Andrew Morton <akpm@linux-foundation.org>,
patches@lists.linux.dev, Vlastimil Babka <vbabka@suse.cz>
Subject: [PATCH v2 14/33] mm/slub: Convert alloc_slab_page() to return a struct slab
Date: Wed, 1 Dec 2021 19:14:51 +0100 [thread overview]
Message-ID: <20211201181510.18784-15-vbabka@suse.cz> (raw)
In-Reply-To: <20211201181510.18784-1-vbabka@suse.cz>
Preparatory, callers convert back to struct page for now.
Also move setting page flags to alloc_slab_page() where we still operate on a
struct page. This means the page->slab_cache pointer is now set later than the
PageSlab flag, which could theoretically confuse some pfn walker assuming
PageSlab means there would be a valid cache pointer. But as the code had no
barriers and used __set_bit() anyway, it could have happened already, so there
shouldn't be such a walker.
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
---
mm/slub.c | 26 ++++++++++++++++----------
1 file changed, 16 insertions(+), 10 deletions(-)
diff --git a/mm/slub.c b/mm/slub.c
index a4f7dafe0610..3d82e8e0fc2b 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -1788,18 +1788,27 @@ static void *setup_object(struct kmem_cache *s, struct page *page,
/*
* Slab allocation and freeing
*/
-static inline struct page *alloc_slab_page(struct kmem_cache *s,
+static inline struct slab *alloc_slab_page(struct kmem_cache *s,
gfp_t flags, int node, struct kmem_cache_order_objects oo)
{
- struct page *page;
+ struct folio *folio;
+ struct slab *slab;
unsigned int order = oo_order(oo);
if (node == NUMA_NO_NODE)
- page = alloc_pages(flags, order);
+ folio = (struct folio *)alloc_pages(flags, order);
else
- page = __alloc_pages_node(node, flags, order);
+ folio = (struct folio *)__alloc_pages_node(node, flags, order);
- return page;
+ if (!folio)
+ return NULL;
+
+ slab = folio_slab(folio);
+ __folio_set_slab(folio);
+ if (page_is_pfmemalloc(folio_page(folio, 0)))
+ slab_set_pfmemalloc(slab);
+
+ return slab;
}
#ifdef CONFIG_SLAB_FREELIST_RANDOM
@@ -1932,7 +1941,7 @@ static struct page *allocate_slab(struct kmem_cache *s, gfp_t flags, int node)
if ((alloc_gfp & __GFP_DIRECT_RECLAIM) && oo_order(oo) > oo_order(s->min))
alloc_gfp = (alloc_gfp | __GFP_NOMEMALLOC) & ~(__GFP_RECLAIM|__GFP_NOFAIL);
- page = alloc_slab_page(s, alloc_gfp, node, oo);
+ page = slab_page(alloc_slab_page(s, alloc_gfp, node, oo));
if (unlikely(!page)) {
oo = s->min;
alloc_gfp = flags;
@@ -1940,7 +1949,7 @@ static struct page *allocate_slab(struct kmem_cache *s, gfp_t flags, int node)
* Allocation may have failed due to fragmentation.
* Try a lower order alloc if possible
*/
- page = alloc_slab_page(s, alloc_gfp, node, oo);
+ page = slab_page(alloc_slab_page(s, alloc_gfp, node, oo));
if (unlikely(!page))
goto out;
stat(s, ORDER_FALLBACK);
@@ -1951,9 +1960,6 @@ static struct page *allocate_slab(struct kmem_cache *s, gfp_t flags, int node)
account_slab(page_slab(page), oo_order(oo), s, flags);
page->slab_cache = s;
- __SetPageSlab(page);
- if (page_is_pfmemalloc(page))
- SetPageSlabPfmemalloc(page);
kasan_poison_slab(page);
--
2.33.1
next prev parent reply other threads:[~2021-12-01 18:22 UTC|newest]
Thread overview: 89+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-12-01 18:14 [PATCH v2 00/33] Separate struct slab from struct page Vlastimil Babka
2021-12-01 18:14 ` [PATCH v2 01/33] mm: add virt_to_folio() and folio_address() Vlastimil Babka
2021-12-14 14:20 ` Johannes Weiner
2021-12-14 14:27 ` Vlastimil Babka
2021-12-01 18:14 ` [PATCH v2 02/33] mm/slab: Dissolve slab_map_pages() in its caller Vlastimil Babka
2021-12-01 18:14 ` [PATCH v2 03/33] mm/slub: Make object_err() static Vlastimil Babka
2021-12-01 18:14 ` [PATCH v2 04/33] mm: Split slab into its own type Vlastimil Babka
2021-12-14 14:24 ` Johannes Weiner
2021-12-01 18:14 ` [PATCH v2 05/33] mm: Add account_slab() and unaccount_slab() Vlastimil Babka
2021-12-14 14:25 ` Johannes Weiner
2021-12-01 18:14 ` [PATCH v2 06/33] mm: Convert virt_to_cache() to use struct slab Vlastimil Babka
2021-12-14 14:26 ` Johannes Weiner
2021-12-01 18:14 ` [PATCH v2 07/33] mm: Convert __ksize() to " Vlastimil Babka
2021-12-14 14:28 ` Johannes Weiner
2021-12-01 18:14 ` [PATCH v2 08/33] mm: Use struct slab in kmem_obj_info() Vlastimil Babka
2021-12-01 18:14 ` [PATCH v2 09/33] mm: Convert check_heap_object() to use struct slab Vlastimil Babka
2021-12-01 18:14 ` [PATCH v2 10/33] mm/slub: Convert detached_freelist to use a " Vlastimil Babka
2021-12-01 18:14 ` [PATCH v2 11/33] mm/slub: Convert kfree() " Vlastimil Babka
2021-12-01 18:14 ` [PATCH v2 12/33] mm/slub: Convert __slab_lock() and __slab_unlock() to " Vlastimil Babka
2021-12-01 18:14 ` [PATCH v2 13/33] mm/slub: Convert print_page_info() to print_slab_info() Vlastimil Babka
2021-12-01 18:14 ` Vlastimil Babka [this message]
2021-12-01 18:14 ` [PATCH v2 15/33] mm/slub: Convert __free_slab() to use struct slab Vlastimil Babka
2021-12-01 18:14 ` [PATCH v2 16/33] mm/slub: Convert pfmemalloc_match() to take a " Vlastimil Babka
2021-12-01 18:14 ` [PATCH v2 17/33] mm/slub: Convert most struct page to struct slab by spatch Vlastimil Babka
2021-12-01 18:14 ` [PATCH v2 18/33] mm/slub: Finish struct page to struct slab conversion Vlastimil Babka
2021-12-01 18:14 ` [PATCH v2 19/33] mm/slab: Convert kmem_getpages() and kmem_freepages() to struct slab Vlastimil Babka
2021-12-01 18:14 ` [PATCH v2 20/33] mm/slab: Convert most struct page to struct slab by spatch Vlastimil Babka
2021-12-01 18:14 ` [PATCH v2 21/33] mm/slab: Finish struct page to struct slab conversion Vlastimil Babka
2021-12-01 18:14 ` [PATCH v2 22/33] mm: Convert struct page to struct slab in functions used by other subsystems Vlastimil Babka
2021-12-02 17:16 ` Andrey Konovalov
2021-12-14 14:31 ` Johannes Weiner
2021-12-01 18:15 ` [PATCH v2 23/33] mm/memcg: Convert slab objcgs from struct page to struct slab Vlastimil Babka
2021-12-14 14:43 ` Johannes Weiner
2021-12-20 23:31 ` Vlastimil Babka
2021-12-01 18:15 ` [PATCH v2 24/33] mm/slob: Convert SLOB to use " Vlastimil Babka
2021-12-10 10:44 ` Hyeonggon Yoo
2021-12-10 11:44 ` Vlastimil Babka
2021-12-10 15:29 ` Hyeonggon Yoo
2021-12-10 18:09 ` Vlastimil Babka
2021-12-11 10:54 ` Hyeonggon Yoo
2021-12-01 18:15 ` [PATCH v2 25/33] mm/kasan: Convert to struct folio and " Vlastimil Babka
2021-12-02 17:16 ` Andrey Konovalov
2021-12-01 18:15 ` [PATCH v2 26/33] mm/kfence: Convert kfence_guarded_alloc() to " Vlastimil Babka
2021-12-01 18:15 ` [PATCH v2 27/33] zsmalloc: Stop using slab fields in struct page Vlastimil Babka
2021-12-01 23:34 ` Minchan Kim
2021-12-14 14:58 ` Johannes Weiner
2021-12-01 18:15 ` [PATCH v2 28/33] bootmem: Use page->index instead of page->freelist Vlastimil Babka
2021-12-14 14:59 ` Johannes Weiner
2021-12-01 18:15 ` [PATCH v2 29/33] iommu: Use put_pages_list Vlastimil Babka
2021-12-01 19:07 ` Matthew Wilcox
2021-12-01 19:45 ` Robin Murphy
2021-12-01 18:15 ` [PATCH v2 30/33] mm: Remove slab from struct page Vlastimil Babka
2021-12-14 14:46 ` Johannes Weiner
2021-12-01 18:15 ` [PATCH v2 31/33] mm/sl*b: Differentiate struct slab fields by sl*b implementations Vlastimil Babka
[not found] ` <20211210163757.GA717823@odroid>
2021-12-10 18:26 ` Vlastimil Babka
2021-12-11 16:23 ` Matthew Wilcox
[not found] ` <20211211115527.GA822127@odroid>
2021-12-11 16:52 ` Matthew Wilcox
2021-12-01 18:15 ` [PATCH v2 32/33] mm/slub: Simplify struct slab slabs field definition Vlastimil Babka
2021-12-14 15:06 ` Johannes Weiner
2021-12-01 18:15 ` [PATCH v2 33/33] mm/slub: Define struct slab fields for CONFIG_SLUB_CPU_PARTIAL only when enabled Vlastimil Babka
2021-12-01 18:39 ` slab tree for next Vlastimil Babka
2021-12-01 20:34 ` Vlastimil Babka
2021-12-02 16:36 ` Vlastimil Babka
2021-12-02 20:39 ` Stephen Rothwell
2022-01-04 0:21 ` Vlastimil Babka
2022-01-04 8:44 ` Stephen Rothwell
2023-08-29 9:55 ` Vlastimil Babka
2023-08-29 21:33 ` Stephen Rothwell
2024-11-05 16:33 ` Vlastimil Babka
2024-11-05 21:08 ` Stephen Rothwell
2021-12-02 12:25 ` [PATCH v2 00/33] Separate struct slab from struct page Vlastimil Babka
2021-12-14 12:57 ` Vlastimil Babka
[not found] ` <20211214143822.GA1063445@odroid>
2021-12-14 14:43 ` Vlastimil Babka
2021-12-15 1:03 ` Roman Gushchin
2021-12-15 23:38 ` Roman Gushchin
2021-12-16 9:19 ` Vlastimil Babka
2021-12-20 0:47 ` Vlastimil Babka
2021-12-20 1:42 ` Matthew Wilcox
2021-12-20 0:24 ` Vlastimil Babka
2021-12-16 15:00 ` Hyeonggon Yoo
2021-12-20 23:58 ` Vlastimil Babka
2021-12-21 17:25 ` Robin Murphy
2021-12-22 7:36 ` Hyeonggon Yoo
2021-12-22 16:56 ` Vlastimil Babka
2021-12-25 9:16 ` Hyeonggon Yoo
2021-12-25 17:53 ` Matthew Wilcox
2021-12-27 2:43 ` Hyeonggon Yoo
2021-12-29 11:22 ` Hyeonggon Yoo
2022-01-03 17:56 ` Vlastimil Babka
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20211201181510.18784-15-vbabka@suse.cz \
--to=vbabka@suse.cz \
--cc=akpm@linux-foundation.org \
--cc=cl@linux.com \
--cc=iamjoonsoo.kim@lge.com \
--cc=linux-mm@kvack.org \
--cc=patches@lists.linux.dev \
--cc=penberg@kernel.org \
--cc=rientjes@google.com \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox