From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E1E9C433FE for ; Tue, 16 Nov 2021 00:17:38 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F0D1E614C8 for ; Tue, 16 Nov 2021 00:17:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org F0D1E614C8 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 605D46B00A2; Mon, 15 Nov 2021 19:16:46 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C27A36B00A3; Mon, 15 Nov 2021 19:16:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 740A26B00A1; Mon, 15 Nov 2021 19:16:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0119.hostedemail.com [216.40.44.119]) by kanga.kvack.org (Postfix) with ESMTP id D34F36B00A7 for ; Mon, 15 Nov 2021 19:16:43 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 94CAB1815BBFC for ; Tue, 16 Nov 2021 00:16:43 +0000 (UTC) X-FDA: 78812877486.24.CE8F30F Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by imf25.hostedemail.com (Postfix) with ESMTP id D7EEEB000193 for ; Tue, 16 Nov 2021 00:16:29 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 1CED61FD73; Tue, 16 Nov 2021 00:16:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1637021802; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4HEc0a+5kUJeYeUUKFV/Jr2mozOhZtMjtJch9JnNS5g=; b=RFFv6xjO4bZgywTvtmKvMDcoNMdUf32NsSKwkpGOyGTTT3Wy1lKfpLMWp5WGoF7/iOM7hy 3CE4C6s8vnKmlST8nZ1LyIjR4brvXigRHoaau4RuNorRtva4aJ8Knz72gEWHMGgD8MQrv6 kwJ82B9TkJLCh2jl6gDCmvQ4yQ4cemE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1637021802; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4HEc0a+5kUJeYeUUKFV/Jr2mozOhZtMjtJch9JnNS5g=; b=PnwVtA/S1JTi/D3xnxvp2NY92JvAMHA0uXxUOI0hCJol0508QkHg2x71i7xFx3peLtIHmP lHb8vv5pbc5hiFBg== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id DDC0113F72; Tue, 16 Nov 2021 00:16:41 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id 0AONNWn4kmFjXAAAMHmgww (envelope-from ); Tue, 16 Nov 2021 00:16:41 +0000 From: Vlastimil Babka To: Matthew Wilcox , linux-mm@kvack.org, Christoph Lameter , David Rientjes , Joonsoo Kim , Pekka Enberg Cc: Vlastimil Babka , Alexander Potapenko , Marco Elver , Dmitry Vyukov , kasan-dev@googlegroups.com Subject: [RFC PATCH 30/32] mm/sl*b: Differentiate struct slab fields by sl*b implementations Date: Tue, 16 Nov 2021 01:16:26 +0100 Message-Id: <20211116001628.24216-31-vbabka@suse.cz> X-Mailer: git-send-email 2.33.1 In-Reply-To: <20211116001628.24216-1-vbabka@suse.cz> References: <20211116001628.24216-1-vbabka@suse.cz> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=4583; h=from:subject; bh=HpvxF6blYd9Gxl8D0B/0lpH3ep8bdHVJ8W1r0M/ydds=; b=owEBbQGS/pANAwAIAeAhynPxiakQAcsmYgBhkvhXodE76eu6sNsRldmo9f4ZC/NuS3j9IsK5x5/Q lJgFbd+JATMEAAEIAB0WIQSNS5MBqTXjGL5IXszgIcpz8YmpEAUCYZL4VwAKCRDgIcpz8YmpEMxSCA Ck/Bodg7xN+SrNWGfwBc0nMLXkVJZUg7QYlOMYrbg14Wb7jdiq4aWhvJ3MkwpN4FUj6qBfxkfNf30X f73MrCXMmKfS63JbXwd0aPcM9ANzTYT31Cys3tiZC8cdObSIq+5iJhxzxlwe24Vh7BW3QbX2+fiTF5 NKXYxdR21dc057mZ/MRTmxIalYYM+5rcFEmUss/Liq8uVAaMMQzuIlSAc4/4DAUl2/y5Nyzp0gIK6s tAwWcynBsk07XBA8OwaI530/l903irks6siXta4ruUOV8hdAq2CZPFAjzpKqchwsegJnJRAmvta9P2 GJQZpz1QcgMKz19hJf3O5SHJeS0zzx X-Developer-Key: i=vbabka@suse.cz; a=openpgp; fpr=A940D434992C2E8E99103D50224FA7E7CC82A664 X-Stat-Signature: xsco67bchg1gtkb3jfo8ok9pprzztfmp Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=RFFv6xjO; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b="PnwVtA/S"; spf=pass (imf25.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.29 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: D7EEEB000193 X-HE-Tag: 1637021789-223695 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: With a struct slab definition separate from struct page, we can go furthe= r and define only fields that the chosen sl*b implementation uses. This means everything between __page_flags and __page_refcount placeholders now depe= nds on the chosen CONFIG_SL*B. Some fields exist in all implementations (slab_li= st) but can be part of a union in some, so it's simpler to repeat them than complicate the definition with ifdefs even more. The patch doesn't change physical offsets of the fields, although it coul= d be done later - for example it's now clear that tighter packing in SLOB coul= d be possible. This should also prevent accidental use of fields that don't exist in giv= en implementation. Before this patch virt_to_cache() and and cache_from_obj(= ) was visible for SLOB (albeit not used), although it relies on the slab_cache = field that isn't set by SLOB. With this patch it's now a compile error, so thes= e functions are now hidden behind #ifndef CONFIG_SLOB. Signed-off-by: Vlastimil Babka Cc: Alexander Potapenko (maintainer:KFENCE) Cc: Marco Elver (maintainer:KFENCE) Cc: Dmitry Vyukov (reviewer:KFENCE) Cc: --- mm/kfence/core.c | 9 +++++---- mm/slab.h | 46 ++++++++++++++++++++++++++++++++++++---------- 2 files changed, 41 insertions(+), 14 deletions(-) diff --git a/mm/kfence/core.c b/mm/kfence/core.c index 4eb60cf5ff8b..46103a7628a6 100644 --- a/mm/kfence/core.c +++ b/mm/kfence/core.c @@ -427,10 +427,11 @@ static void *kfence_guarded_alloc(struct kmem_cache= *cache, size_t size, gfp_t g /* Set required slab fields. */ slab =3D virt_to_slab((void *)meta->addr); slab->slab_cache =3D cache; - if (IS_ENABLED(CONFIG_SLUB)) - slab->objects =3D 1; - if (IS_ENABLED(CONFIG_SLAB)) - slab->s_mem =3D addr; +#if defined(CONFIG_SLUB) + slab->objects =3D 1; +#elif defined (CONFIG_SLAB) + slab->s_mem =3D addr; +#endif =20 /* Memory initialization. */ for_each_canary(meta, set_canary_byte); diff --git a/mm/slab.h b/mm/slab.h index 58b65e5e5d49..10a9ee195249 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -8,9 +8,24 @@ /* Reuses the bits in struct page */ struct slab { unsigned long __page_flags; + +#if defined(CONFIG_SLAB) + + union { + struct list_head slab_list; + struct rcu_head rcu_head; + }; + struct kmem_cache *slab_cache; + void *freelist; /* array of free object indexes */ + void * s_mem; /* first object */ + unsigned int active; + +#elif defined(CONFIG_SLUB) + union { struct list_head slab_list; - struct { /* Partial pages */ + struct rcu_head rcu_head; + struct { struct slab *next; #ifdef CONFIG_64BIT int slabs; /* Nr of slabs left */ @@ -18,25 +33,32 @@ struct slab { short int slabs; #endif }; - struct rcu_head rcu_head; }; - struct kmem_cache *slab_cache; /* not slob */ + struct kmem_cache *slab_cache; /* Double-word boundary */ void *freelist; /* first free object */ union { - void *s_mem; /* slab: first object */ - unsigned long counters; /* SLUB */ - struct { /* SLUB */ + unsigned long counters; + struct { unsigned inuse:16; unsigned objects:15; unsigned frozen:1; }; }; + unsigned int __unused; + +#elif defined(CONFIG_SLOB) + + struct list_head slab_list; + void * __unused_1; + void *freelist; /* first free block */ + void * __unused_2; + int units; + +#else +#error "Unexpected slab allocator configured" +#endif =20 - union { - unsigned int active; /* SLAB */ - int units; /* SLOB */ - }; atomic_t __page_refcount; #ifdef CONFIG_MEMCG unsigned long memcg_data; @@ -47,7 +69,9 @@ struct slab { static_assert(offsetof(struct page, pg) =3D=3D offsetof(struct slab, sl= )) SLAB_MATCH(flags, __page_flags); SLAB_MATCH(compound_head, slab_list); /* Ensure bit 0 is clear */ +#ifndef CONFIG_SLOB SLAB_MATCH(rcu_head, rcu_head); +#endif SLAB_MATCH(_refcount, __page_refcount); #ifdef CONFIG_MEMCG SLAB_MATCH(memcg_data, memcg_data); @@ -623,6 +647,7 @@ static inline void memcg_slab_free_hook(struct kmem_c= ache *s, } #endif /* CONFIG_MEMCG_KMEM */ =20 +#ifndef CONFIG_SLOB static inline struct kmem_cache *virt_to_cache(const void *obj) { struct slab *slab; @@ -669,6 +694,7 @@ static inline struct kmem_cache *cache_from_obj(struc= t kmem_cache *s, void *x) print_tracking(cachep, x); return cachep; } +#endif /* CONFIG_SLOB */ =20 static inline size_t slab_ksize(const struct kmem_cache *s) { --=20 2.33.1