From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BFF3C47258 for ; Tue, 23 Jan 2024 15:56:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C0EBB6B0085; Tue, 23 Jan 2024 10:56:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BC1716B0088; Tue, 23 Jan 2024 10:56:11 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A880E8D0001; Tue, 23 Jan 2024 10:56:11 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 992026B0085 for ; Tue, 23 Jan 2024 10:56:11 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 10B691C14EE for ; Tue, 23 Jan 2024 15:56:11 +0000 (UTC) X-FDA: 81711027342.21.B83EBA5 Received: from mail-lf1-f48.google.com (mail-lf1-f48.google.com [209.85.167.48]) by imf30.hostedemail.com (Postfix) with ESMTP id 3597A80013 for ; Tue, 23 Jan 2024 15:56:08 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=dKJ15PEQ; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf30.hostedemail.com: domain of dvyukov@google.com designates 209.85.167.48 as permitted sender) smtp.mailfrom=dvyukov@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706025369; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=EBv30KMy0zvXmwC7oxK4cZSZ9HSjQvy5SlIH3Zj4wIY=; b=qXQeOI5NRSm4zESKckg1kFyyrOuHmGHwymidHTScn51O5fCtfhk99vH4k1pp33Yamgx/0F 3NaqQ6VxLogPVOPzs2rQ1WB2B4HrGLe6NV402Br2+UHk5TtW/Gc46QDdaHO/hPTZ6ee84G 32gShxHqzJ/1ZkKRvPbajp10r/teBbM= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=dKJ15PEQ; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf30.hostedemail.com: domain of dvyukov@google.com designates 209.85.167.48 as permitted sender) smtp.mailfrom=dvyukov@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706025369; a=rsa-sha256; cv=none; b=3ehPvbPu+rYbn8t83+IrBa5zQFpx9P4lNPvvAo7ATfkdIK9ZHhJytZ+n/yVHMno9eAPwgK BqrNglnWAkEKXzHLRgd2v3r38WHs+fJLKFgJCjt/jlhw7Xn4gH52AyVc35cHPPjakPVWL5 6+VeMFbfO0/cTc4tLDTR4N7l3feinaU= Received: by mail-lf1-f48.google.com with SMTP id 2adb3069b0e04-50eb9d41d57so2495e87.1 for ; Tue, 23 Jan 2024 07:56:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706025367; x=1706630167; darn=kvack.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=EBv30KMy0zvXmwC7oxK4cZSZ9HSjQvy5SlIH3Zj4wIY=; b=dKJ15PEQ0amHlhbr6Z+MrPKN1x82e8jECg1qyUZX6OYm/DHMUkJi/uyjjiSXoqfeuG 6D1Mb9N8BYg/+kfG4GV0JbHx4zQpb5znaAWVbiKrSz0wzoCW6uJLdG2vhNmJ+WLYa2Cm ij1VcR4hFeAgxFn1bhvHStzKKaObLLuGUG1RlDzDtm7zzzg5Av61atpI9OSG2oPO2yZ0 TVYnn3Ar8juFP+mBVKPvu6M+eC9sHLESTLzwmL80IqIuuWkuZCjgUoeR+ze1HxEIKhpI cVMjS29UAudKO4TWKeCuSAx5Xz/JR41fUAHDwZw2bCGNsdLSfFhvLlKtSmTD2XizbEVB Fklg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706025367; x=1706630167; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=EBv30KMy0zvXmwC7oxK4cZSZ9HSjQvy5SlIH3Zj4wIY=; b=W+myrT96BpyDLLgBbyBJhTXWPlDs42E8GMh+Txi6L1JRVlVKApW2Jlkdul2JyaiaK7 xx7yIcscHcSIoo6ungkYeQzBWtegW0wLQktncj0nkMq544BovdtrDK16OgI6Dq7NhL1L CpHRDs9frbDUxrrHYmCUG1Wos+DYEwXj2ObBVv181JyzMx4Zv97XCKXxU1Oqw4FPcel8 sIcJkW4yNThCHrczOaZ9lH3JVlXbBKeMKRig8Pcvg31p65VChSFAQY1DGI+4VsXYTN5w J8XNXulfvksknfxsJ0kOoYaRwwFX7jvx83OAVunLhu3ngMxYqOEiWFpgmLPG8o8wAqRv hhHw== X-Gm-Message-State: AOJu0YwuArrtdYHq1BWx9tSTc8ZvEBwzGKl/GCVkvQa6/+0fxXLA9Zh0 GtyPvLGI894ksRfNOFE1f5h/C5wrI0iXZLdrkxjETvO3K9+5cVs6+Yq8OcLsPo/UvORs+MLj+Wd c5Z7QBbGvRE5sFeG71NY0iVpVQWoLnSGNfMRJ X-Google-Smtp-Source: AGHT+IGX7ORTrUQQYto8V+zR0lNs54Y7uli6s5LaFoyrmcVGLPjD8aENG1Gjzi2/hwj9N+EQj1O9ExjUZAXgyclAJFg= X-Received: by 2002:ac2:4d85:0:b0:50e:7be0:3ac1 with SMTP id g5-20020ac24d85000000b0050e7be03ac1mr149756lfe.6.1706025367012; Tue, 23 Jan 2024 07:56:07 -0800 (PST) MIME-Version: 1.0 References: <20240122171215.319440-2-elver@google.com> In-Reply-To: From: Dmitry Vyukov Date: Tue, 23 Jan 2024 16:55:54 +0100 Message-ID: Subject: Re: [RFC PATCH] stackdepot: use variable size records for non-evictable entries To: Marco Elver Cc: Andrey Konovalov , Andrew Morton , Alexander Potapenko , Vlastimil Babka , linux-kernel@vger.kernel.org, linux-mm@kvack.org, kasan-dev@googlegroups.com Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 3597A80013 X-Stat-Signature: zimhfnzwpat38m56x3h1emnoqup4mr8u X-Rspam-User: X-HE-Tag: 1706025368-407519 X-HE-Meta: U2FsdGVkX1+Ft1gS1QiecDkO/9nctZj/KeiGGx4JONqoblofFRZQUp+RDeY/RQ2Z24p1NzK0BUKl9MMd9m3rzNkCMZw8EYUH2rjXUvDkIZ2/CCLvZeb3aO1P9W0lyYvVGE8URArjogywUi+0JX5lsIqoGPOhNL8oQPNp1Ez8vZwj2QSfm50Gq3dlIpxyOzKEK+NeGUKJuvi76PtGacJffn2N2HixJfoaaSHTDmdBIye6OhjqfMFpVwx2wFWoDn0t3Pt7oTBlhwf5N2Tg0Hz7Ua6PUBn9GUw7aL7scU2iRMgfXu7rfpO0Bo/ZhXrTGx0T1Q5HqbdRikIdbvXBlD61+zZWY/8s7WSp38IiTfW7G9sH8ABkvylWLUifjvKAqkHzmoBzVkRjcpdQIRIDougsp+0WGkJcBRBCEWoaxOuW/LM7gwFG8TVAlLeHwEatuqszutmqjlNRmK97qQlcZ8tLmUVjmUGrR4qJu2yncYHD2dgvvQYTvSEzviniO070dL7LXP80p0C+oW1ZZzwwvPM5Muls8OEKbF5jop4CZn5ys/jhnuQIZfpEw3pizRWOwV0Xlk3Env+0zc/PoRWaXKSG3CFKtqmYlzOBGcmOGzxz6EAWyTZerxYFLz3MHIMZYqtA9RnStA+z/CL+Oq4Hf6IXW9tTlkKnDeWxUnm/cMfDNVDnVtusvbbEwPh/y1nBSsaLIOuADcttzhtWUIknlBnI5l6UKIQ/ZwIMfIEAbordVNeFADw819LSo+gLqa3Lrkj/nw3ugbJQfTN9Dnb46vAHRX3P9DIhP0k9wqVPjXQyw/fEnAF1Dnqa9sH0bQcKrHqbQ7x8Qrg6Td+HnAG06TKco7WULJ5tQGAMmnlkcrBhB6GSS1P/emm+zSRutatJkQ6ip1qaArmMy13b3l2fHX58kD+jDIbGjrs2+ldsZYCmTvAmwKdIjlnXLNwzDhZ6PSMPnKnbaI2c0Y7YShSLY7T 7L3c9ubd NfmtHGp4SS9CRq71pK/Ns671KKnDqF0JP8O7jGZ+K/8Y4tM3wPbNSJQ6JUfFLk5xb5FHl/ulfoto9fkzwBvf/0vd1XiX2gKIJXcg4AtaoZX2uFRLqU96k3IPTP9Ts2WEn1Jgf71onLKVX2OD8N+Ctq3K4IkYc7e8Tkx6EDuJzKPkOtqeSUQwoNINGcWTTwTxaNVm28d6OB4QTxJMs15ozf36g70cfSXXWk325XnPCm6VT/rnxcj9pEkgJpItQa1cGbVaV87DgNzH+iUjuo5QzIEHujA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Tue, 23 Jan 2024 at 16:53, Marco Elver wrote: > > And on top of this we can make KASAN generic happier again: > > Objections? Not doing refcounting/aux locking for generic KASAN makes sense to me. > ------ >8 ------ > > From: Marco Elver > Date: Tue, 23 Jan 2024 12:11:36 +0100 > Subject: [PATCH RFC] kasan: revert eviction of stack traces in generic mode > > This partially reverts commits cc478e0b6bdf, 63b85ac56a64, 08d7c94d9635, > a414d4286f34, and 773688a6cb24 to make use of variable-sized stack depot > records, since eviction of stack entries from stack depot forces fixed- > sized stack records. Care was taken to retain the code cleanups by the > above commits. > > Eviction was added to generic KASAN as a response to alleviating the > additional memory usage from fixed-sized stack records, but this still > uses more memory than previously. > > With the re-introduction of variable-sized records for stack depot, we > can just switch back to non-evictable stack records again, and return > back to the previous performance and memory usage baseline. > > Before (observed after a KASAN kernel boot): > > pools: 597 > allocations: 29657 > frees: 6425 > in_use: 23232 > freelist_size: 3493 > > After: > > pools: 315 > allocations: 28964 > frees: 0 > in_use: 28964 > freelist_size: 0 > > As can be seen from the number of "frees", with a generic KASAN config, > evictions are no longer used but due to using variable-sized records, I > observe a reduction of 282 stack depot pools (saving 4512 KiB). > > Fixes: cc478e0b6bdf ("kasan: avoid resetting aux_lock") > Fixes: 63b85ac56a64 ("kasan: stop leaking stack trace handles") > Fixes: 08d7c94d9635 ("kasan: memset free track in qlink_free") > Fixes: a414d4286f34 ("kasan: handle concurrent kasan_record_aux_stack calls") > Fixes: 773688a6cb24 ("kasan: use stack_depot_put for Generic mode") > Signed-off-by: Marco Elver > Cc: Alexander Potapenko > Cc: Andrey Konovalov > Cc: Dmitry Vyukov > --- > mm/kasan/common.c | 3 +-- > mm/kasan/generic.c | 54 ++++++---------------------------------------- > mm/kasan/kasan.h | 8 ------- > 3 files changed, 8 insertions(+), 57 deletions(-) > > diff --git a/mm/kasan/common.c b/mm/kasan/common.c > index 610efae91220..ad32803e34e9 100644 > --- a/mm/kasan/common.c > +++ b/mm/kasan/common.c > @@ -65,8 +65,7 @@ void kasan_save_track(struct kasan_track *track, gfp_t flags) > { > depot_stack_handle_t stack; > > - stack = kasan_save_stack(flags, > - STACK_DEPOT_FLAG_CAN_ALLOC | STACK_DEPOT_FLAG_GET); > + stack = kasan_save_stack(flags, STACK_DEPOT_FLAG_CAN_ALLOC); > kasan_set_track(track, stack); > } > > diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c > index df6627f62402..8bfb52b28c22 100644 > --- a/mm/kasan/generic.c > +++ b/mm/kasan/generic.c > @@ -485,16 +485,6 @@ void kasan_init_object_meta(struct kmem_cache *cache, const void *object) > if (alloc_meta) { > /* Zero out alloc meta to mark it as invalid. */ > __memset(alloc_meta, 0, sizeof(*alloc_meta)); > - > - /* > - * Prepare the lock for saving auxiliary stack traces. > - * Temporarily disable KASAN bug reporting to allow instrumented > - * raw_spin_lock_init to access aux_lock, which resides inside > - * of a redzone. > - */ > - kasan_disable_current(); > - raw_spin_lock_init(&alloc_meta->aux_lock); > - kasan_enable_current(); > } > > /* > @@ -506,18 +496,8 @@ void kasan_init_object_meta(struct kmem_cache *cache, const void *object) > > static void release_alloc_meta(struct kasan_alloc_meta *meta) > { > - /* Evict the stack traces from stack depot. */ > - stack_depot_put(meta->alloc_track.stack); > - stack_depot_put(meta->aux_stack[0]); > - stack_depot_put(meta->aux_stack[1]); > - > - /* > - * Zero out alloc meta to mark it as invalid but keep aux_lock > - * initialized to avoid having to reinitialize it when another object > - * is allocated in the same slot. > - */ > - __memset(&meta->alloc_track, 0, sizeof(meta->alloc_track)); > - __memset(meta->aux_stack, 0, sizeof(meta->aux_stack)); > + /* Zero out alloc meta to mark it as invalid. */ > + __memset(meta, 0, sizeof(*meta)); > } > > static void release_free_meta(const void *object, struct kasan_free_meta *meta) > @@ -526,9 +506,6 @@ static void release_free_meta(const void *object, struct kasan_free_meta *meta) > if (*(u8 *)kasan_mem_to_shadow(object) != KASAN_SLAB_FREE_META) > return; > > - /* Evict the stack trace from the stack depot. */ > - stack_depot_put(meta->free_track.stack); > - > /* Mark free meta as invalid. */ > *(u8 *)kasan_mem_to_shadow(object) = KASAN_SLAB_FREE; > } > @@ -571,8 +548,6 @@ static void __kasan_record_aux_stack(void *addr, depot_flags_t depot_flags) > struct kmem_cache *cache; > struct kasan_alloc_meta *alloc_meta; > void *object; > - depot_stack_handle_t new_handle, old_handle; > - unsigned long flags; > > if (is_kfence_address(addr) || !slab) > return; > @@ -583,33 +558,18 @@ static void __kasan_record_aux_stack(void *addr, depot_flags_t depot_flags) > if (!alloc_meta) > return; > > - new_handle = kasan_save_stack(0, depot_flags); > - > - /* > - * Temporarily disable KASAN bug reporting to allow instrumented > - * spinlock functions to access aux_lock, which resides inside of a > - * redzone. > - */ > - kasan_disable_current(); > - raw_spin_lock_irqsave(&alloc_meta->aux_lock, flags); > - old_handle = alloc_meta->aux_stack[1]; > alloc_meta->aux_stack[1] = alloc_meta->aux_stack[0]; > - alloc_meta->aux_stack[0] = new_handle; > - raw_spin_unlock_irqrestore(&alloc_meta->aux_lock, flags); > - kasan_enable_current(); > - > - stack_depot_put(old_handle); > + alloc_meta->aux_stack[0] = kasan_save_stack(0, depot_flags); > } > > void kasan_record_aux_stack(void *addr) > { > - return __kasan_record_aux_stack(addr, > - STACK_DEPOT_FLAG_CAN_ALLOC | STACK_DEPOT_FLAG_GET); > + return __kasan_record_aux_stack(addr, STACK_DEPOT_FLAG_CAN_ALLOC); > } > > void kasan_record_aux_stack_noalloc(void *addr) > { > - return __kasan_record_aux_stack(addr, STACK_DEPOT_FLAG_GET); > + return __kasan_record_aux_stack(addr, 0); > } > > void kasan_save_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags) > @@ -620,7 +580,7 @@ void kasan_save_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags) > if (!alloc_meta) > return; > > - /* Evict previous stack traces (might exist for krealloc or mempool). */ > + /* Invalidate previous stack traces (might exist for krealloc or mempool). */ > release_alloc_meta(alloc_meta); > > kasan_save_track(&alloc_meta->alloc_track, flags); > @@ -634,7 +594,7 @@ void kasan_save_free_info(struct kmem_cache *cache, void *object) > if (!free_meta) > return; > > - /* Evict previous stack trace (might exist for mempool). */ > + /* Invalidate previous stack trace (might exist for mempool). */ > release_free_meta(object, free_meta); > > kasan_save_track(&free_meta->free_track, 0); > diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h > index d0f172f2b978..216ae0ef1e4b 100644 > --- a/mm/kasan/kasan.h > +++ b/mm/kasan/kasan.h > @@ -6,7 +6,6 @@ > #include > #include > #include > -#include > #include > > #if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) > @@ -265,13 +264,6 @@ struct kasan_global { > struct kasan_alloc_meta { > struct kasan_track alloc_track; > /* Free track is stored in kasan_free_meta. */ > - /* > - * aux_lock protects aux_stack from accesses from concurrent > - * kasan_record_aux_stack calls. It is a raw spinlock to avoid sleeping > - * on RT kernels, as kasan_record_aux_stack_noalloc can be called from > - * non-sleepable contexts. > - */ > - raw_spinlock_t aux_lock; > depot_stack_handle_t aux_stack[2]; > }; > > -- > 2.43.0.429.g432eaa2c6b-goog >