From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C6E8C4167B for ; Tue, 5 Dec 2023 13:27:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B3EAD6B008C; Tue, 5 Dec 2023 08:27:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id AC7C56B0092; Tue, 5 Dec 2023 08:27:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 941426B0093; Tue, 5 Dec 2023 08:27:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 7D69F6B008C for ; Tue, 5 Dec 2023 08:27:38 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 3D86A80190 for ; Tue, 5 Dec 2023 13:27:38 +0000 (UTC) X-FDA: 81532841796.05.E156233 Received: from out-173.mta1.migadu.com (out-173.mta1.migadu.com [95.215.58.173]) by imf05.hostedemail.com (Postfix) with ESMTP id 21BB0100010 for ; Tue, 5 Dec 2023 13:27:35 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=FQUkcXNP; spf=pass (imf05.hostedemail.com: domain of chengming.zhou@linux.dev designates 95.215.58.173 as permitted sender) smtp.mailfrom=chengming.zhou@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701782856; a=rsa-sha256; cv=none; b=GC2gqSc1+cZIC3NJsTzESZ9KtFIjXY1qOkXR1CQQyNGNfyO6WoyCuFl2fQj7pG0sRVRw1J 2+6xDjf9SugA+bo3dLtRlwwu1dbZ5BmqwX0MNUl70Cw7QyrpqzodPO/Dpgvih4rnaT2lJD 32XhmgyCd2/juzfBckepNG96gBd7Zr0= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=linux.dev header.s=key1 header.b=FQUkcXNP; spf=pass (imf05.hostedemail.com: domain of chengming.zhou@linux.dev designates 95.215.58.173 as permitted sender) smtp.mailfrom=chengming.zhou@linux.dev; dmarc=pass (policy=none) header.from=linux.dev ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701782856; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dSQSwpmn7M/CrcHDOooBYuyEVhVwNmvfJ9+Pn01yWSY=; b=OBPfK7+pB1abZePAeeuQZkc88BXXwcj6AvuNUowrNNzoikhXAkPrrp8WjF0tf81aET7ueQ gcIfKyCkRJlunWYzLka9zx6BW+IWlCEuzKTsQiLdrO6+PqjISXDOLk4q/m2XClCxTR9Bxe aLGJqSYlUYIeS93m4dFa3aYoqwd5TyQ= Message-ID: <44421a37-4343-46d0-9e5c-17c2cd038cf2@linux.dev> DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1701782853; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dSQSwpmn7M/CrcHDOooBYuyEVhVwNmvfJ9+Pn01yWSY=; b=FQUkcXNPzenZDQFP3uDNRiJagjcqB6wRxMxCMUudY3xEcp7t0F5y0sq8ygb37HMilKHsT7 wCokJIR0Qe5WyJaR7p1ixr/kPXSlKnsUHrdNFITfIFuj5+OidhdweTLzaoECxwtrgSIkYm E+c7Xi5sOv4EUj01jKQJ94iF9id1whs= Date: Tue, 5 Dec 2023 21:27:27 +0800 MIME-Version: 1.0 Subject: Re: [PATCH 4/4] mm/slub: free KFENCE objects in slab_free_hook() Content-Language: en-US To: Vlastimil Babka , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim Cc: Andrew Morton , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Alexander Potapenko , Marco Elver , Dmitry Vyukov , linux-mm@kvack.org, linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com References: <20231204-slub-cleanup-hooks-v1-0-88b65f7cd9d5@suse.cz> <20231204-slub-cleanup-hooks-v1-4-88b65f7cd9d5@suse.cz> X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Chengming Zhou In-Reply-To: <20231204-slub-cleanup-hooks-v1-4-88b65f7cd9d5@suse.cz> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Migadu-Flow: FLOW_OUT X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 21BB0100010 X-Stat-Signature: 1jutmay3w5nfbaf8eumu3o3ko9z4dtju X-Rspam-User: X-HE-Tag: 1701782855-535824 X-HE-Meta: U2FsdGVkX18e9rAb0eN9V3fHQxJ9kq/ANn7JYkp1knOSEaiU9V3UUQ+1slfhq/9SqE/PXJjVXd1nf1iskCH1SnEZsQcLCTlfr3dnVeoC4ykDsKqzLqH0bRhJqGqf92vJ/sjGo5/MBrFBCLhN5bVZIUlSgGHJfS7PqnnBkY/EN9VqfDHjRgC9UKBkHdjJYAYLMPeGLD8DQ8ZnHAHHFnNjw7Ewz+DiwkjUAvj7qKntODTL+2BLnKfXrorC96mVhed7Ktas/FmDhceMRpP8IIKKfb+LFQbGIYBly6NcUafRJTsKY+lcvyjWGE4NM8dwy7CKkKRVd1qbtSIxbBLQ0S3ZshaIS8EwarWRr5ibkeagABKP+uZr1e9SA6jm1u3rDUL0mc/sGXAGFFvz+XSy0i8RCJ5NwMooNR2fJQPYtaGnfG4Vgc0YHXnphRu7iMKnxyBIEeDIICSRFtXSd9Lc/y1vCpusv0zPRnEwFFNsNGwo4VSKmPwbLO0Jb70FDI+qaFmOMVedK9zGiLBKqOPuMZiXbeElb3/1CgmzK1s2wM6f8zyLrKXAb1rd+JqrzGGKCPJq+ONhSo8SpH8R9H1dbEptDX5N7nFPNzgxGJRkNdJpl6u422q4kLrUfGNnCatV6JphcDDvBeg0P+YNiWzNsN+Dd1/1kqiNqTgc6nJHp1DZf4sNKVVIHX3mX3/XWc3XVpyGwRC15Adzz6MgqH/SMBpgg8fmkLF/TYmOEejHUKKK05nxZSnymOUKLX2QtU23jeln7q7uq+y/hBucWugKcu+6JwZl6137BrKa4H9Ay3mNgdvUdpyAx/ByIexIxNhzj0q7ismjmnPyjAuBcUHHBY2XgPOCQR7qrpDOHb5IB2HZFxRWPSNQ5dnGd/xV1u3BanIjDLfV5jJiPo7Dshmk1JUyiXmBkGOxnADnIYajxmJABYfiIgEVGszneL9LEizh5O1jIWT9yb507MjYstNPSKl HNBUWVCe 0ypmGX+ly8My61WQpYTOOl0sGWOp6q+mCsk/0qwY13yDBuxvNUVAaQb2D2X49EGEU4TZkUQd67iAgE9KoWjcMvlS8flaOtQiR4H8YWFoASEBBscygCNfu9SjpGKxieNU+MgOCUTTitxZZuQRPtjRrGezMIlUOYeQ8XEw/FfaWkp2ERwxCTiVfbdTJEthDMh6H+G03T5WhbS9sHk/6b4FWW/iYjBhWVjZVNZC1g5p1mndLdni6xJ1SRBChhvqu1EtH1JiNqTELXBG42hUamcjuI2ZIldhGXaFxT82H X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 2023/12/5 03:34, Vlastimil Babka wrote: > When freeing an object that was allocated from KFENCE, we do that in the > slowpath __slab_free(), relying on the fact that KFENCE "slab" cannot be > the cpu slab, so the fastpath has to fallback to the slowpath. > > This optimization doesn't help much though, because is_kfence_address() > is checked earlier anyway during the free hook processing or detached > freelist building. Thus we can simplify the code by making the > slab_free_hook() free the KFENCE object immediately, similarly to KASAN > quarantine. > > In slab_free_hook() we can place kfence_free() above init processing, as > callers have been making sure to set init to false for KFENCE objects. > This simplifies slab_free(). This places it also above kasan_slab_free() > which is ok as that skips KFENCE objects anyway. > > While at it also determine the init value in slab_free_freelist_hook() > outside of the loop. > > This change will also make introducing per cpu array caches easier. > > Tested-by: Marco Elver > Signed-off-by: Vlastimil Babka > --- > mm/slub.c | 22 ++++++++++------------ > 1 file changed, 10 insertions(+), 12 deletions(-) > > diff --git a/mm/slub.c b/mm/slub.c > index ed2fa92e914c..e38c2b712f6c 100644 > --- a/mm/slub.c > +++ b/mm/slub.c > @@ -2039,7 +2039,7 @@ static inline void memcg_slab_free_hook(struct kmem_cache *s, struct slab *slab, > * production configuration these hooks all should produce no code at all. > * > * Returns true if freeing of the object can proceed, false if its reuse > - * was delayed by KASAN quarantine. > + * was delayed by KASAN quarantine, or it was returned to KFENCE. > */ > static __always_inline > bool slab_free_hook(struct kmem_cache *s, void *x, bool init) > @@ -2057,6 +2057,9 @@ bool slab_free_hook(struct kmem_cache *s, void *x, bool init) > __kcsan_check_access(x, s->object_size, > KCSAN_ACCESS_WRITE | KCSAN_ACCESS_ASSERT); > > + if (kfence_free(kasan_reset_tag(x))) I'm wondering if "kasan_reset_tag()" is needed here? The patch looks good to me! Reviewed-by: Chengming Zhou Thanks. > + return false; > + > /* > * As memory initialization might be integrated into KASAN, > * kasan_slab_free and initialization memset's must be > @@ -2086,23 +2089,25 @@ static inline bool slab_free_freelist_hook(struct kmem_cache *s, > void *object; > void *next = *head; > void *old_tail = *tail; > + bool init; > > if (is_kfence_address(next)) { > slab_free_hook(s, next, false); > - return true; > + return false; > } > > /* Head and tail of the reconstructed freelist */ > *head = NULL; > *tail = NULL; > > + init = slab_want_init_on_free(s); > + > do { > object = next; > next = get_freepointer(s, object); > > /* If object's reuse doesn't have to be delayed */ > - if (likely(slab_free_hook(s, object, > - slab_want_init_on_free(s)))) { > + if (likely(slab_free_hook(s, object, init))) { > /* Move object to the new freelist */ > set_freepointer(s, object, *head); > *head = object; > @@ -4103,9 +4108,6 @@ static void __slab_free(struct kmem_cache *s, struct slab *slab, > > stat(s, FREE_SLOWPATH); > > - if (kfence_free(head)) > - return; > - > if (IS_ENABLED(CONFIG_SLUB_TINY) || kmem_cache_debug(s)) { > free_to_partial_list(s, slab, head, tail, cnt, addr); > return; > @@ -4290,13 +4292,9 @@ static __fastpath_inline > void slab_free(struct kmem_cache *s, struct slab *slab, void *object, > unsigned long addr) > { > - bool init; > - > memcg_slab_free_hook(s, slab, &object, 1); > > - init = !is_kfence_address(object) && slab_want_init_on_free(s); > - > - if (likely(slab_free_hook(s, object, init))) > + if (likely(slab_free_hook(s, object, slab_want_init_on_free(s)))) > do_slab_free(s, slab, object, object, 1, addr); > } > >