From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 923C8C001B0 for ; Mon, 3 Jul 2023 20:10:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 13F0A280036; Mon, 3 Jul 2023 16:10:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0EF5C280030; Mon, 3 Jul 2023 16:10:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EF904280036; Mon, 3 Jul 2023 16:10:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id E0030280030 for ; Mon, 3 Jul 2023 16:10:50 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id A94841A02D1 for ; Mon, 3 Jul 2023 20:10:50 +0000 (UTC) X-FDA: 80971393860.06.78A9C1C Received: from mail-il1-f182.google.com (mail-il1-f182.google.com [209.85.166.182]) by imf05.hostedemail.com (Postfix) with ESMTP id 5070510000B for ; Mon, 3 Jul 2023 20:10:48 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=W6kvFMDO; spf=pass (imf05.hostedemail.com: domain of rientjes@google.com designates 209.85.166.182 as permitted sender) smtp.mailfrom=rientjes@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688415048; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RUC3FVpBPWS4ikSEsOA9mN51Dh3oGNvpuz1E7SS7pW8=; b=Z0DkkcHk0p4Pw16QRoEBUXvhxaB9NCwZZ2tCZNZzQoOVJKVnzKmBg5chMHxNwH0FwXc9yH +aE8eZOeYYhtD6Fuu4isHy4pum6ZBId1YCGML8bxfDPu6B3HT02ximG18g8hu8Q13wMpA/ /Ya0AyoA2FEg8qUVB0p5nffT3IiRkGo= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=W6kvFMDO; spf=pass (imf05.hostedemail.com: domain of rientjes@google.com designates 209.85.166.182 as permitted sender) smtp.mailfrom=rientjes@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688415048; a=rsa-sha256; cv=none; b=uEXgbb630hX2hm22A7IY+8D+9fX2eW41LUEzy0oZG1OYCdR4OLvCj5jKZb3x0CD+c0vOKr rBG6coPla93n9ThCW8cyJ6ZCxFG4UagicHNwgnO5nfWO2Alp+XhneZ9uyA82SnlScaHqaR Ia79HpEsZ+XF0BlCGoVSYB8luWVo3Qk= Received: by mail-il1-f182.google.com with SMTP id e9e14a558f8ab-346099c6f43so187425ab.0 for ; Mon, 03 Jul 2023 13:10:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1688415048; x=1691007048; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=RUC3FVpBPWS4ikSEsOA9mN51Dh3oGNvpuz1E7SS7pW8=; b=W6kvFMDOO6Pl9UTrtrK/jSx6v+0H/Opp73681u2iMUCVFXuT7Xje1tIpNGfz1OaDyI VdpgH4fgCf20sjpsoiuAFEmhPPa/1fgffTYdj3vIWsjrtTf4ZIMPAEEyEqn+TESCrxYC jVL2jjdqW4g19xqp7Vk9iXwlYXhOWERqW+jJ3fewh0u57Q7HtZ97zNAqbygOMecYkgKy NccrUgeF/Q7ybFa8mpZs8LHdguO00gPjBTZKJWSCFvpT5ELfqEGN2UYpwWiGp9MgkG1C nAq90GyIebUccUhPv31VP1GCX2PPuXAyDPp29hvB62v4WAQLF1I6bkjPIgquivizEeAX DoYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688415048; x=1691007048; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=RUC3FVpBPWS4ikSEsOA9mN51Dh3oGNvpuz1E7SS7pW8=; b=keXjo4T/5IGUIucEeQ8EG5idgUFUidGjPPloAsY/qWcGq+ipziHmaC4IhKq1XMOovk jDg72Y+OUBdmr+Zq9ngybaagRxYNNuqnq+dIp+9iv3hADdGi9HqbtK4olyF1TF/mvJCL XGxa3Qsr7W1wiBFYnkHuIKCRGxlTZ8zhwCXFzmqa9Rp5SiVLUqgIlYQcCVkzTYlYoo4Z 1HPfrl/xMJ+DdjYj5lkxBhyV1q5kBHhQQ5BoB1gCLdkab08gi3whEVMzR0t491bBhqsa t1rGrKmpTmbaHMhBp5RfHo+cc3hqH5Wwftwo0CwADaPgyVayAn+q8eDRPGJknnSxFqeo 5SkA== X-Gm-Message-State: ABy/qLaF0UbbjVQVDfmo6DKvPFkEsCbsAqWornpnUnftvJ7w3EuuQvTC BY4rYTM2a8qmVsXuoQrPL8QVAA== X-Google-Smtp-Source: APBJJlFfY6YmIHiIaFwzFZfADRfTPdrEmECH4yvf25yM58GDY9iJ7h2R/a1TW56egdogUK+tAz/sNA== X-Received: by 2002:a05:6e02:1d03:b0:33b:71e6:d6b4 with SMTP id i3-20020a056e021d0300b0033b71e6d6b4mr18168ila.14.1688415047707; Mon, 03 Jul 2023 13:10:47 -0700 (PDT) Received: from [2620:0:1008:15:17fc:1b56:71df:89b7] ([2620:0:1008:15:17fc:1b56:71df:89b7]) by smtp.gmail.com with ESMTPSA id f4-20020aa78b04000000b0066884d4efdbsm14887991pfd.12.2023.07.03.13.10.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Jul 2023 13:10:47 -0700 (PDT) Date: Mon, 3 Jul 2023 13:10:46 -0700 (PDT) From: David Rientjes To: Matteo Rizzo cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, cl@linux.com, penberg@kernel.org, iamjoonsoo.kim@lge.com, akpm@linux-foundation.org, vbabka@suse.cz, roman.gushchin@linux.dev, 42.hyeyoo@gmail.com, jannh@google.com Subject: Re: [PATCH] mm/slub: refactor freelist to use custom type In-Reply-To: <20230703143820.152479-1-matteorizzo@google.com> Message-ID: References: <20230703143820.152479-1-matteorizzo@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 5070510000B X-Rspam-User: X-Stat-Signature: oa5ooy3xdj1mcs4e54d1ezssx4mnmpdi X-Rspamd-Server: rspam01 X-HE-Tag: 1688415048-63983 X-HE-Meta: U2FsdGVkX19mesBDaTJWNTyOLBl/vmttHzTb0N0dcLu0E+ANCDOTwif9zzbTTXpK7B7u0+2YbByIrQhwQwHJKp+GQBWr3cN9xyuuRfWJX5SsGBYy5cqMm1S7qPCO0KQJo8VbzPuHvg7aJbcbV2kd8tk1lFKAaKfo5y2rcT9EABhdYVDNBz/0P5YdOc5venGAlHHwTej5IThWltsm520Yr2RKfBS9ef0azYCFQFInwpbhzQbT2/ytITJ65ROgPpaIUhwp8HdXHYDmgJNYgAkQ44D7LyDgfk2AFTmQ58FQ8S0vy3nchCaDQ6HgJwh7lF6z2ol1Z8m4rCB3zBm9sp20Me37vpgDfqo2KM0etCHb4GPR9Sy1DqxGuKvDYkTqg16LJlA1XuDF1M4bwhlmjoOvTuTfWMiZIzKGf83k0wincP9GkPXDSQY0s44HpQ/NMkhxjPfsj4Skrip6HmSnfJTpw3tMzQkta5+QxtZvPZA4cESY/SjA5varvWH0u3Rv300ZyViYJctbckydwi3nNM+Da11ohigZb4V0M8zY2l+65klfkxdixspGPv/NPy6Atdd4gMyjHIYvM3zzuoZonrtjVqIC+bco63KTeVUe9ijuU0QBR4qJOV501cX7dUBEQXd6L/Tc2pHeZCCzuuwg/7PkAy+eXm7+OuHuzIL6bfQfLFDuer5dgD+WCMSFizofVT5LtvufjVs47oIOgTzeNtfwr6nY0o9SNQhHvre2NfUn3BiOHO+h0M7zVcboifW0pJaG9VIX9nzB/3jximjO9uO5HEXGhdsGihvX6knhi+WW+dK/ZDB+vLvpTB8yOwiziAe/EMXFEtwTtuRCIj7hL+qbBdPKaEYSlGMqUroSdI8c9t0+2dSEhB3DBteRQqddLL0kiKSUjYEeR9mKIEh4oISdhI8I/mMFIPIZ5n7EMnnmSqKXSxAgQRCbZprB3lpvq4eeSpTvi1fWBZ0GCw2VVfW MzlF0Q1r X5DFMFITl8dGoURPs3Gep9Sdt4vdrBP7+oakMmitEDXiFb14ZFlQA5oTLhjilaiXmzMn7KdpQ+ROHUlIo8hf29V4pqpyfryRz8tiZwcm3Knb1nw1NGnlyIthUCWB/hZCN8W8nC+Dix87f+DKMt46Cda/ALEBN2MPbZjEaVIqBsXCcSpMICTnEn1bmL1GHz4SjVkQrh60ctcyTHOLL5Qjilm6m6Te1cOhhSTCnaqBHNqwea3C95UwaMMv4A3vaxTZS0Z7VoVFs1T3bKa/ssTLYVtN+lz+pwbfH3Kf0q7RxPwxnsXO0P5iGZDzS+HtC7cF4z8783a8DdQzWKqZIRoXPGcpn1af46ko1c2NzuMTau6wXKeEs02gYdSQwP6ilnfmk/IN/A/EzS+GlC7fHEcCtK5yNKs7P42SgW9oyDWEars9Rg/rFL+N1sMFPbfsauBLQeB9ZPLRDmzkztCm0WWSued7C0g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, 3 Jul 2023, Matteo Rizzo wrote: > From: Jann Horn > > Currently the SLUB code represents encoded freelist entries as "void*". > That's misleading, those things are encoded under > CONFIG_SLAB_FREELIST_HARDENED so that they're not actually dereferencable. > > Give them their own type, and split freelist_ptr() into one function per > direction (one for encoding, one for decoding). > I don't feel strongly about this. > Signed-off-by: Jann Horn > Co-developed-by: Matteo Rizzo > Signed-off-by: Matteo Rizzo > --- > include/linux/slub_def.h | 6 ++++++ > mm/slub.c | 37 ++++++++++++++++++++++++++----------- > 2 files changed, 32 insertions(+), 11 deletions(-) > > diff --git a/include/linux/slub_def.h b/include/linux/slub_def.h > index deb90cf4bffb..c747820a55b4 100644 > --- a/include/linux/slub_def.h > +++ b/include/linux/slub_def.h > @@ -43,6 +43,12 @@ enum stat_item { > }; > > #ifndef CONFIG_SLUB_TINY > +/* > + * freeptr_t represents a SLUB freelist pointer, which might be encoded > + * and not dereferenceable if CONFIG_SLAB_FREELIST_HARDENED is enabled. > + */ > +typedef struct { unsigned long v; } freeptr_t; Seems strange this would only appear for configs without CONFIG_SLUB_TINY. Since lots of files include linux/slab.h I think this may start to be used in non-slab code. Not sure why it needs to be added to the header file? > + > /* > * When changing the layout, make sure freelist and tid are still compatible > * with this_cpu_cmpxchg_double() alignment requirements. > diff --git a/mm/slub.c b/mm/slub.c > index e3b5d5c0eb3a..26d0ca02b61d 100644 > --- a/mm/slub.c > +++ b/mm/slub.c > @@ -365,8 +365,8 @@ static struct workqueue_struct *flushwq; > * with an XOR of the address where the pointer is held and a per-cache > * random number. > */ > -static inline void *freelist_ptr(const struct kmem_cache *s, void *ptr, > - unsigned long ptr_addr) > +static inline freeptr_t freelist_ptr_encode(const struct kmem_cache *s, > + void *ptr, unsigned long ptr_addr) > { > #ifdef CONFIG_SLAB_FREELIST_HARDENED > /* > @@ -379,25 +379,40 @@ static inline void *freelist_ptr(const struct kmem_cache *s, void *ptr, > * calls get_freepointer() with an untagged pointer, which causes the > * freepointer to be restored incorrectly. > */ > - return (void *)((unsigned long)ptr ^ s->random ^ > - swab((unsigned long)kasan_reset_tag((void *)ptr_addr))); > + return (freeptr_t){.v = (unsigned long)ptr ^ s->random ^ > + swab((unsigned long)kasan_reset_tag((void *)ptr_addr))}; > #else > - return ptr; > + return (freeptr_t){.v = (unsigned long)ptr}; > #endif > } > > +static inline void *freelist_ptr_decode(const struct kmem_cache *s, > + freeptr_t ptr, unsigned long ptr_addr) > +{ > + void *decoded; > + > +#ifdef CONFIG_SLAB_FREELIST_HARDENED > + /* See the comment in freelist_ptr_encode */ > + decoded = (void *)(ptr.v ^ s->random ^ > + swab((unsigned long)kasan_reset_tag((void *)ptr_addr))); > +#else > + decoded = (void *)ptr.v; > +#endif > + return decoded; > +} > + > /* Returns the freelist pointer recorded at location ptr_addr. */ > static inline void *freelist_dereference(const struct kmem_cache *s, > void *ptr_addr) > { > - return freelist_ptr(s, (void *)*(unsigned long *)(ptr_addr), > + return freelist_ptr_decode(s, *(freeptr_t *)(ptr_addr), > (unsigned long)ptr_addr); > } > > static inline void *get_freepointer(struct kmem_cache *s, void *object) > { > object = kasan_reset_tag(object); > - return freelist_dereference(s, object + s->offset); > + return freelist_dereference(s, (freeptr_t *)(object + s->offset)); > } > > #ifndef CONFIG_SLUB_TINY > @@ -421,15 +436,15 @@ __no_kmsan_checks > static inline void *get_freepointer_safe(struct kmem_cache *s, void *object) > { > unsigned long freepointer_addr; > - void *p; > + freeptr_t p; > > if (!debug_pagealloc_enabled_static()) > return get_freepointer(s, object); > > object = kasan_reset_tag(object); > freepointer_addr = (unsigned long)object + s->offset; > - copy_from_kernel_nofault(&p, (void **)freepointer_addr, sizeof(p)); > - return freelist_ptr(s, p, freepointer_addr); > + copy_from_kernel_nofault(&p, (freeptr_t *)freepointer_addr, sizeof(p)); > + return freelist_ptr_decode(s, p, freepointer_addr); > } > > static inline void set_freepointer(struct kmem_cache *s, void *object, void *fp) > @@ -441,7 +456,7 @@ static inline void set_freepointer(struct kmem_cache *s, void *object, void *fp) > #endif > > freeptr_addr = (unsigned long)kasan_reset_tag((void *)freeptr_addr); > - *(void **)freeptr_addr = freelist_ptr(s, fp, freeptr_addr); > + *(freeptr_t *)freeptr_addr = freelist_ptr_encode(s, fp, freeptr_addr); > } > > /* Loop over all objects in a slab */ > > base-commit: a901a3568fd26ca9c4a82d8bc5ed5b3ed844d451 > -- > 2.41.0.255.g8b1d071c50-goog > >