From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87969CD37AE for ; Fri, 15 Sep 2023 21:27:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 024178D0031; Fri, 15 Sep 2023 17:27:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F16E28D0005; Fri, 15 Sep 2023 17:26:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DB7428D0031; Fri, 15 Sep 2023 17:26:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id CB5AA8D0005 for ; Fri, 15 Sep 2023 17:26:59 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id A4D87160A50 for ; Fri, 15 Sep 2023 21:26:59 +0000 (UTC) X-FDA: 81240116958.03.EEA71BC Received: from mail-ot1-f45.google.com (mail-ot1-f45.google.com [209.85.210.45]) by imf22.hostedemail.com (Postfix) with ESMTP id D3EE0C0019 for ; Fri, 15 Sep 2023 21:26:57 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=VGJ00BF3; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf22.hostedemail.com: domain of keescook@chromium.org designates 209.85.210.45 as permitted sender) smtp.mailfrom=keescook@chromium.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694813217; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=N73xDGDYMf25E5+wcFhTff+1YHHWKUrOFYtp7lI/Z2E=; b=gkm6OmSuFHMksEob7rx6OaxK8bh7oqCh331QWIJkE2bwwg0FDoCzXRXmg2XWuepTakKxb9 1xez0H2VAFoeiJoDxa4izoMfXBuY9RkRisvZsd15piL7wxn+3VcWVPNqz/vRedjRWDH6td gQAjyHPuyis3K3qhBg6rDOxtd7NPwRM= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=chromium.org header.s=google header.b=VGJ00BF3; dmarc=pass (policy=none) header.from=chromium.org; spf=pass (imf22.hostedemail.com: domain of keescook@chromium.org designates 209.85.210.45 as permitted sender) smtp.mailfrom=keescook@chromium.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694813217; a=rsa-sha256; cv=none; b=ca4KERFZDkTKAZAkuqtkEhUMy68C/ZQ49vxVr3aYHaEU1JwmLI1BSlOJTuva0E/8NfhVLm FwyYqG2H4yjWnwXbuolj58U7XParSgpIB30Ndkkflb2vEdXPvC2b607Zk59R1E4GIvDow5 7gAKvEPCqFyxxwKobHOQf+sAFH8D9MI= Received: by mail-ot1-f45.google.com with SMTP id 46e09a7af769-6c0bbbbad81so1529366a34.3 for ; Fri, 15 Sep 2023 14:26:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1694813217; x=1695418017; darn=kvack.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=N73xDGDYMf25E5+wcFhTff+1YHHWKUrOFYtp7lI/Z2E=; b=VGJ00BF3lNPldAfS+7Ps7+zX0gcMqewl4OOUDxHJxN366ime7OFCUXeokZCpHJ202g yBHwDD14HsnqvjeRbn2OW4/g5tiXd+7SUYs1pMQu9xoBhwysAuyDJNZ8wJPIu7zh3lLa cyYDanQPCPs2h2i/6I+fYP4ngrcjAKXqqGla4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694813217; x=1695418017; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=N73xDGDYMf25E5+wcFhTff+1YHHWKUrOFYtp7lI/Z2E=; b=cmN09iIlQPN3bmtXwOwaPT9Yflv9Gq/4ZZbfZ9+llu6cBgosu/3j6rFQjFwfxAUuBU EEyDOxVlfaP5X8X60DL/U/ugv+oPheCfC9pGbdb+ssW8vl7o/dSjThO7lteXgNHhOa4J nEj8jDtR1n/cH2ZkHvRsILbTj81latpbVzXoVmHN5FkEOiQEiQd+97pexaC5CLZfqOMZ BmPEiqWeEaqkqiaf/bJVCkBQGfoGozwMt7ZkHT2jNwnFj2yooHiU++svA2yyH5IEB4CK /XXS2Xso8gGn5Ge6JSIRFsH0Kp0CxknoKwLLMELqQTpfY9dx3VN5HQwR5K97CsGiwJiW s+iw== X-Gm-Message-State: AOJu0YwXEpoknRMtjWh3ZXo2reaR0LlBX+PH/Onkh3aVVby5SP6/xVfL Ua0IE71VmmBqp2dDqupYYswCHw== X-Google-Smtp-Source: AGHT+IHGfEL6UEpI1fhXq1GH6s3XjXanXM4RZV6BwTnJBBNVpqZXF/AagkdBc6hDr45b/FAuDOtwSA== X-Received: by 2002:a05:6358:3412:b0:135:a10e:1ed0 with SMTP id h18-20020a056358341200b00135a10e1ed0mr2576032rwd.23.1694813216900; Fri, 15 Sep 2023 14:26:56 -0700 (PDT) Received: from www.outflux.net (198-0-35-241-static.hfc.comcastbusiness.net. [198.0.35.241]) by smtp.gmail.com with ESMTPSA id p17-20020a639511000000b005740aa41237sm1889280pgd.74.2023.09.15.14.26.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 14:26:56 -0700 (PDT) Date: Fri, 15 Sep 2023 14:26:55 -0700 From: Kees Cook To: Matteo Rizzo Cc: cl@linux.com, penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, akpm@linux-foundation.org, vbabka@suse.cz, roman.gushchin@linux.dev, 42.hyeyoo@gmail.com, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-hardening@vger.kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, corbet@lwn.net, luto@kernel.org, peterz@infradead.org, jannh@google.com, evn@google.com, poprdi@google.com, jordyzomer@google.com Subject: Re: [RFC PATCH 13/14] mm/slub: sanity-check freepointers Message-ID: <202309151425.2BE59091@keescook> References: <20230915105933.495735-1-matteorizzo@google.com> <20230915105933.495735-14-matteorizzo@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230915105933.495735-14-matteorizzo@google.com> X-Rspamd-Queue-Id: D3EE0C0019 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: f1e66uk3cbjhsirixhdczkq6bwenajar X-HE-Tag: 1694813217-747528 X-HE-Meta: U2FsdGVkX19orgockvN1QgGZX1DL/9WFOrRxNGxGxI4AR+gc/+BptR8bU3gCqzF4FDj+uPChS6zT5nQGW55Pw3jcl2MYerAdqg4Z7ePI7kBCZdyhfFENi3eLpczCn4Cg9xGnGUSDOI5zg5c5aRt1+EVYTEPAph6Xy8E4UenH3MqNj6eoqUz0CMO5GtAYQAcGlL+lpEb3aAf0+41y1hmltorFCFE/eofAuaJ5+Wi/xzHHVQr1tO7/jb1+S21O0A7feUagrFCH6tvkwvIBkg/53pqVESEuBRTA6mNg/EXQuRtE3THI2QRJf6dDebnY4xKMYN/phLgjA/4YeuNW0u2Ypb+ZFq6VHCc5xqEkgxYGdXmrbwWrwNVF0gAJMNl84aCyNSUDatAAvKTZbUDsL7sFC0rxTn0oI6nzTC0cAskbdOiI+KGD/qus0vvQMny4vf3rekLO4t7V6C47t6cWxHqtMyMt1vqwz9sDqvyb6aTND3l0JEO8rslfFkt4TrwUQwVT+i4fCwjqDPIZbIDIcMCPJCZGKZTxVVmy3e6c7yoKxe6wsMLdglW/CzuA3enqTLxdat5z54tL5TGoiOzIU42xIK1jiGBBLJZ0OftLbbVOxjO4x5EJ4cX8giogpm3KianwADia7gWyHAc7IG0eWm2JkScrODDzuxTN9LWTk4kjCbC74Ar+AcGHWRYYZBQ7c7I1upV+CEL9xCW1s5SR3dwlFgUAYg7VIs3v/FvP5HTCw1qUIsTSfOPlbVJMA23pZnlgfEG7Uvvt2PgLCQlzlUh0PYB86rbT52eEZ5AOoSftKRIO8sd5WhoODoyzBQg4yHlt/nBOWpseYGYzT7E/v6OUmQ9bSHy+qKplOwRU3ZRhicbH+mo7sSHNu3AfxNawsCSzy4akAOFJ1XP/GJw5b3xH1+O4RYEpZ1Pe7kE9kgncTL6ZTn+/Avvvq8r+qdWySYn3uLj2+aRLUcpnObbpsXS bH7w/ktn Q06F6BvLtXyoSjxS7Jn3xpQmZ9PjwdAfWfmL6S++Ybt3SS6EzQzvYZucgHKkUeWZQ2hGqu8QJOBGn+E8ogHxV1a1AGtrgMnKHEY0IfhgYzMqqWemBCUW2gaPp17j0qqZWjzMfFs0JTo+Mq7stAz9+RWqndYs6GdqjrvkeffVEzvJ6K2zza65WgYp7+A4mYe0OTNK34yjoNa2yFQJdTYwgCSiLyrzgI8Wqyyd6I2f5Ai78WCr9Ve7oAU4ZHTcELNFzqeGcpRfym0UM5Cp69D3hd6CxkIx668QBt3a+vwv/sc9x+l/E2SSHBZdQV6w80KSbDvSfrJoVwVLTCrrLiD9LblTVA0QvXZFj9/R2WkBpY+EP4tZFzH9zhSKjRrGBTUZhEB4TEsUCEdPZC/oRKQ0veITrEWVshyje7IMx9lTyZXXCHbQm1pn1wtIZScPY6ZyPdOZ40BwKyibWX7YRcbbUD0G/PiyyHhSf6TA+1ZDikEBCAhal+U5bo/LWPJhyKGzKedvq X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Sep 15, 2023 at 10:59:32AM +0000, Matteo Rizzo wrote: > From: Jann Horn > > Sanity-check that: > - non-NULL freepointers point into the slab > - freepointers look plausibly aligned > > Signed-off-by: Jann Horn > Co-developed-by: Matteo Rizzo > Signed-off-by: Matteo Rizzo > --- > lib/slub_kunit.c | 4 ++++ > mm/slab.h | 8 +++++++ > mm/slub.c | 57 ++++++++++++++++++++++++++++++++++++++++++++++++ > 3 files changed, 69 insertions(+) > > diff --git a/lib/slub_kunit.c b/lib/slub_kunit.c > index d4a3730b08fa..acf8600bd1fd 100644 > --- a/lib/slub_kunit.c > +++ b/lib/slub_kunit.c > @@ -45,6 +45,10 @@ static void test_clobber_zone(struct kunit *test) > #ifndef CONFIG_KASAN > static void test_next_pointer(struct kunit *test) > { > + if (IS_ENABLED(CONFIG_SLAB_VIRTUAL)) > + kunit_skip(test, > + "incompatible with freepointer corruption detection in CONFIG_SLAB_VIRTUAL"); > + > struct kmem_cache *s = test_kmem_cache_create("TestSlub_next_ptr_free", > 64, SLAB_POISON); > u8 *p = kmem_cache_alloc(s, GFP_KERNEL); > diff --git a/mm/slab.h b/mm/slab.h > index 460c802924bd..8d10a011bdf0 100644 > --- a/mm/slab.h > +++ b/mm/slab.h > @@ -79,6 +79,14 @@ struct slab { > > struct list_head flush_list_elem; > > + /* > + * Not in kmem_cache because it depends on whether the allocation is > + * normal order or fallback order. > + * an alternative might be to over-allocate virtual memory for > + * fallback-order pages. > + */ > + unsigned long align_mask; > + > /* Replaces the page lock */ > spinlock_t slab_lock; > > diff --git a/mm/slub.c b/mm/slub.c > index 0f7f5bf0b174..57474c8a6569 100644 > --- a/mm/slub.c > +++ b/mm/slub.c > @@ -392,6 +392,44 @@ static inline freeptr_t freelist_ptr_encode(const struct kmem_cache *s, > return (freeptr_t){.v = encoded}; > } > > +/* > + * Does some validation of freelist pointers. Without SLAB_VIRTUAL this is > + * currently a no-op. > + */ > +static inline bool freelist_pointer_corrupted(struct slab *slab, freeptr_t ptr, > + void *decoded) > +{ > +#ifdef CONFIG_SLAB_VIRTUAL > + /* > + * If the freepointer decodes to 0, use 0 as the slab_base so that > + * the check below always passes (0 & slab->align_mask == 0). > + */ > + unsigned long slab_base = decoded ? (unsigned long)slab_to_virt(slab) > + : 0; > + > + /* > + * This verifies that the SLUB freepointer does not point outside the > + * slab. Since at that point we can basically do it for free, it also > + * checks that the pointer alignment looks vaguely sane. > + * However, we probably don't want the cost of a proper division here, > + * so instead we just do a cheap check whether the bottom bits that are > + * clear in the size are also clear in the pointer. > + * So for kmalloc-32, it does a perfect alignment check, but for > + * kmalloc-192, it just checks that the pointer is a multiple of 32. > + * This should probably be reconsidered - is this a good tradeoff, or > + * should that part be thrown out, or do we want a proper accurate > + * alignment check (and can we make it work with acceptable performance > + * cost compared to the security improvement - probably not)? Is it really that much more expensive to check the alignment exactly? > + */ > + return CHECK_DATA_CORRUPTION( > + ((unsigned long)decoded & slab->align_mask) != slab_base, > + "bad freeptr (encoded %lx, ptr %p, base %lx, mask %lx", > + ptr.v, decoded, slab_base, slab->align_mask); > +#else > + return false; > +#endif > +} > + > static inline void *freelist_ptr_decode(const struct kmem_cache *s, > freeptr_t ptr, unsigned long ptr_addr, > struct slab *slab) > @@ -403,6 +441,10 @@ static inline void *freelist_ptr_decode(const struct kmem_cache *s, > #else > decoded = (void *)ptr.v; > #endif > + > + if (unlikely(freelist_pointer_corrupted(slab, ptr, decoded))) > + return NULL; > + > return decoded; > } > > @@ -2122,6 +2164,21 @@ static struct slab *get_free_slab(struct kmem_cache *s, > if (slab == NULL) > return NULL; > > + /* > + * Bits that must be equal to start-of-slab address for all > + * objects inside the slab. > + * For compatibility with pointer tagging (like in HWASAN), this would > + * need to clear the pointer tag bits from the mask. > + */ > + slab->align_mask = ~((PAGE_SIZE << oo_order(oo)) - 1); > + > + /* > + * Object alignment bits (must be zero, which is equal to the bits in > + * the start-of-slab address) > + */ > + if (s->red_left_pad == 0) > + slab->align_mask |= (1 << (ffs(s->size) - 1)) - 1; > + > return slab; > } > > -- > 2.42.0.459.ge4e396fd5e-goog > We can improve the sanity checking in the future, so as-is, sure: Reviewed-by: Kees Cook -- Kees Cook