From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E2BEEC4167B for ; Wed, 6 Dec 2023 09:58:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 823696B008A; Wed, 6 Dec 2023 04:58:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 7D3C36B0092; Wed, 6 Dec 2023 04:58:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6752E6B0098; Wed, 6 Dec 2023 04:58:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 58FF16B008A for ; Wed, 6 Dec 2023 04:58:10 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 38F731A0158 for ; Wed, 6 Dec 2023 09:58:10 +0000 (UTC) X-FDA: 81535942740.07.3F18FAB Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) by imf23.hostedemail.com (Postfix) with ESMTP id BB6BA140010 for ; Wed, 6 Dec 2023 09:58:07 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=TyDveYqJ; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=bZz9l9D8; dmarc=none; spf=pass (imf23.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701856688; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8sXZ1VnaFvD+d6tTkWgPtcPa6GYEwUd43mPhrV5ynPM=; b=LHIE3wX7gwDPEpWC13AR77Xx/6jskgAKNCdMQgxlbho4oLkwJ4W7GKOdBLS3EBm9YLLv+4 3j/9ayCUMVwXyi+WnGtqxzr0mHnTRMVyJfszrwRqNivW5EiuDBxB4CD+SSoVjtN/zO9j8C zFWWCCNJufvE1IfnwiRm2rkj0PZ8iBA= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=TyDveYqJ; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=bZz9l9D8; dmarc=none; spf=pass (imf23.hostedemail.com: domain of vbabka@suse.cz designates 195.135.223.130 as permitted sender) smtp.mailfrom=vbabka@suse.cz ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701856688; a=rsa-sha256; cv=none; b=GTIwNoIftD0Ildw7oUB5dmgbE1JdrC5qRmqiYxwI/NsVg5RxSwEtR5W46W2OgtWMWPio+j vtBCls3QGlAZOhaxVieHtxIcPgIToZf4D3a1Uw4sTt6AjceUENaDqWbWDpw2tikKbfi3Ub 2TGUqrXV3AClGGwBE6FYripGUKhrgi4= Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id D07F221F4E; Wed, 6 Dec 2023 09:58:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1701856685; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8sXZ1VnaFvD+d6tTkWgPtcPa6GYEwUd43mPhrV5ynPM=; b=TyDveYqJ1PPmEvcJTebmhW9HnPauTWSbgf+eLVxRdHf3pg4SNbYaA1hyHNV79UkTke54Vu HcGCcODHKXIilovKNSDc1twgp8NfUkmteiiHPMaV4lP2JfqGHMjCvk72rgtDop6Jh2NxkC RJeqDMT5NbOZyWNlVg4NUOemsGljBgU= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1701856685; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8sXZ1VnaFvD+d6tTkWgPtcPa6GYEwUd43mPhrV5ynPM=; b=bZz9l9D8qNs3n1neL4kfIUS8f12gb0EXQQkmz0qyf/LK7Jv4UR09vRg/Wh2Az9fxsTKo4i jJFI7UkUk5oe1SDw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id AFFE413408; Wed, 6 Dec 2023 09:58:05 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id HN+WKq1FcGVxTgAAD6G6ig (envelope-from ); Wed, 06 Dec 2023 09:58:05 +0000 Message-ID: <79e29576-12a2-a423-92f3-d8a7bcd2f0ce@suse.cz> Date: Wed, 6 Dec 2023 10:58:05 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.15.1 Subject: Re: [PATCH 4/4] mm/slub: free KFENCE objects in slab_free_hook() Content-Language: en-US To: Chengming Zhou , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim Cc: Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Alexander Potapenko , Marco Elver , Dmitry Vyukov , linux-mm@kvack.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com References: <20231204-slub-cleanup-hooks-v1-0-88b65f7cd9d5@suse.cz> <20231204-slub-cleanup-hooks-v1-4-88b65f7cd9d5@suse.cz> <44421a37-4343-46d0-9e5c-17c2cd038cf2@linux.dev> From: Vlastimil Babka In-Reply-To: <44421a37-4343-46d0-9e5c-17c2cd038cf2@linux.dev> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: BB6BA140010 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: gwxfk4z6grsd8pm4jkwhsfnn7c5ik1hn X-HE-Tag: 1701856687-506181 X-HE-Meta: U2FsdGVkX18fkTJPLARG8TStH1hYufW8/gKso+Rsicl9qrLL2SE2OYR4umFB2N/O/MNlU1gRTSdG3TvhExDrFS7GJF8FLL3LbYAJcAgKvbeg3eRS3jUXasbFtCaZ22O77lMaKNNRIAuoGXCZtLOyTaU6Ef+u9ad6f1VffDgDG+akUJg7ekQD8fnPQGofUybwc2sGrlg+ZglywVGnofwWvP64fvz7ACIyzJ2AivTg7vd1F1GAmlVUCNRK58pjzlSehjvfM/aL6/JZC3XTeU6kwgr6b1MVZnZjo3LMkEXOKae/huYraq0ueQBQO8NCh2FazyPjyqo8J0SKNYPOC52mrH3N4M6vbBpKzbxEBIc7lEGepVK2iTalU2ZZpiCEIz95kq+YZbBlaMYhanqkMwhSOk+0kdrRngegO5c6u2nRTHruJjRNrCidB3f/pmCx7t9bRInRuMbq1eizEgvlkB8mdQJfjbHHKpJK6L3GfmfWEM1zKl2jxl+q0A+rRbXYpIuFEeQaLId9u29DjTCiyzurete4XBNInhWgXdbduFBC4OmhLoQkdVurmaAOqsBchGCQEi2cbIJ/IJDydUUbMWj+1iaSBzSHe5nd1evTFtDTiDl1kE4/SYcVZr+sCQCdvCI/lYRbXC4JUpKo1P4MLuLQSky3eEvjK4OODXQPHWFj4LJZciw/2gLnMpNbTp/bKDDlfMGBHM19OFIXrgsWiuvpwYvLRGAhRzxd5eEmdC2K4ZlGnHq/Ncj+TuZQr4h/dh0pH+pyTHI3+Wei6LotJ2uijOKT1uPWNwiDi9YEkvEwDTn3geyDSjjZwqZNckLE1j22db5YEMxX6ryfFmXLYpR5oV4KQgEghWZ2/7qi+w2y/vhYupO0j3PnSGkxZWujBcpU7m9eTjVSgy4CiBEdCLhaT4RE8DbWJWxBzTpYpXpdh6a9/ymaHHLcPdOy0nHBdKpLpJm6OCrxI2OPZ7cVJ5Q 8ziN44j6 CUw5xCNc4IPdSr+QpxSQ4C7DJKxr83Z/U56OWfYvXsipacgJxtIz7UpGV+F2qCr2nk+JqiIx2sgrcq/8uc84yGBENLOSo48CZ5THuI2mTaqVbA96L4uJ9T9gENeyYZ80iAEeVtrelvbM7iY2zsUgkbXH3o7PCkQl46364dVC60rGsEfoh5R/TXInmk87SnSjGManCw4jFt0HM4ugAyJ8AOMPffeqjRI0d2H6aeqmfUkAylcBhP0+zGwCxXs57A1sJ3ZYsbG9QpMDtL8UVfCfogltptJkg8msKpyKIRsFl8lp9XaEgjBpTOXl1uMBul82COVPMkuytn8cWRbebQnEqehcnxuPECMMZjrly3qhSvZFTBQfr6or85wKbLB/WF06raIk3YIGLad/d1VfAG2bMWKFjOYG9ocb5E72R4uiVYjwE+nc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 12/5/23 14:27, Chengming Zhou wrote: > On 2023/12/5 03:34, Vlastimil Babka wrote: >> When freeing an object that was allocated from KFENCE, we do that in the >> slowpath __slab_free(), relying on the fact that KFENCE "slab" cannot be >> the cpu slab, so the fastpath has to fallback to the slowpath. >> >> This optimization doesn't help much though, because is_kfence_address() >> is checked earlier anyway during the free hook processing or detached >> freelist building. Thus we can simplify the code by making the >> slab_free_hook() free the KFENCE object immediately, similarly to KASAN >> quarantine. >> >> In slab_free_hook() we can place kfence_free() above init processing, as >> callers have been making sure to set init to false for KFENCE objects. >> This simplifies slab_free(). This places it also above kasan_slab_free() >> which is ok as that skips KFENCE objects anyway. >> >> While at it also determine the init value in slab_free_freelist_hook() >> outside of the loop. >> >> This change will also make introducing per cpu array caches easier. >> >> Tested-by: Marco Elver >> Signed-off-by: Vlastimil Babka >> --- >> mm/slub.c | 22 ++++++++++------------ >> 1 file changed, 10 insertions(+), 12 deletions(-) >> >> diff --git a/mm/slub.c b/mm/slub.c >> index ed2fa92e914c..e38c2b712f6c 100644 >> --- a/mm/slub.c >> +++ b/mm/slub.c >> @@ -2039,7 +2039,7 @@ static inline void memcg_slab_free_hook(struct kmem_cache *s, struct slab *slab, >> * production configuration these hooks all should produce no code at all. >> * >> * Returns true if freeing of the object can proceed, false if its reuse >> - * was delayed by KASAN quarantine. >> + * was delayed by KASAN quarantine, or it was returned to KFENCE. >> */ >> static __always_inline >> bool slab_free_hook(struct kmem_cache *s, void *x, bool init) >> @@ -2057,6 +2057,9 @@ bool slab_free_hook(struct kmem_cache *s, void *x, bool init) >> __kcsan_check_access(x, s->object_size, >> KCSAN_ACCESS_WRITE | KCSAN_ACCESS_ASSERT); >> >> + if (kfence_free(kasan_reset_tag(x))) > > I'm wondering if "kasan_reset_tag()" is needed here? I think so, because AFAICS the is_kfence_address() check in kfence_free() could be a false negative otherwise. In fact now I even question some of the other is_kfence_address() checks in mm/slub.c, mainly build_detached_freelist() which starts from pointers coming directly from slab users. Insight from KASAN/KFENCE folks appreciated :) > The patch looks good to me! > > Reviewed-by: Chengming Zhou Thanks! > Thanks. > >> + return false; >> + >> /* >> * As memory initialization might be integrated into KASAN, >> * kasan_slab_free and initialization memset's must be >> @@ -2086,23 +2089,25 @@ static inline bool slab_free_freelist_hook(struct kmem_cache *s, >> void *object; >> void *next = *head; >> void *old_tail = *tail; >> + bool init; >> >> if (is_kfence_address(next)) { >> slab_free_hook(s, next, false); >> - return true; >> + return false; >> } >> >> /* Head and tail of the reconstructed freelist */ >> *head = NULL; >> *tail = NULL; >> >> + init = slab_want_init_on_free(s); >> + >> do { >> object = next; >> next = get_freepointer(s, object); >> >> /* If object's reuse doesn't have to be delayed */ >> - if (likely(slab_free_hook(s, object, >> - slab_want_init_on_free(s)))) { >> + if (likely(slab_free_hook(s, object, init))) { >> /* Move object to the new freelist */ >> set_freepointer(s, object, *head); >> *head = object; >> @@ -4103,9 +4108,6 @@ static void __slab_free(struct kmem_cache *s, struct slab *slab, >> >> stat(s, FREE_SLOWPATH); >> >> - if (kfence_free(head)) >> - return; >> - >> if (IS_ENABLED(CONFIG_SLUB_TINY) || kmem_cache_debug(s)) { >> free_to_partial_list(s, slab, head, tail, cnt, addr); >> return; >> @@ -4290,13 +4292,9 @@ static __fastpath_inline >> void slab_free(struct kmem_cache *s, struct slab *slab, void *object, >> unsigned long addr) >> { >> - bool init; >> - >> memcg_slab_free_hook(s, slab, &object, 1); >> >> - init = !is_kfence_address(object) && slab_want_init_on_free(s); >> - >> - if (likely(slab_free_hook(s, object, init))) >> + if (likely(slab_free_hook(s, object, slab_want_init_on_free(s)))) >> do_slab_free(s, slab, object, object, 1, addr); >> } >> >>