From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A47C2C47258 for ; Thu, 25 Jan 2024 22:35:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3FBC66B009C; Thu, 25 Jan 2024 17:35:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3AAB96B009B; Thu, 25 Jan 2024 17:35:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 272F46B009C; Thu, 25 Jan 2024 17:35:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 174666B009A for ; Thu, 25 Jan 2024 17:35:57 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id EAC08A1BC8 for ; Thu, 25 Jan 2024 22:35:56 +0000 (UTC) X-FDA: 81719292312.02.85995C2 Received: from mail-wm1-f41.google.com (mail-wm1-f41.google.com [209.85.128.41]) by imf05.hostedemail.com (Postfix) with ESMTP id 22001100020 for ; Thu, 25 Jan 2024 22:35:54 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=cR8Y34pT; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf05.hostedemail.com: domain of andreyknvl@gmail.com designates 209.85.128.41 as permitted sender) smtp.mailfrom=andreyknvl@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1706222155; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ASPyASwMfpiYxl1C+amLASNi2699ENzeTUD4Y5JDI5E=; b=0IF6MgCh04pHn30SCsLgr4dZXTafQLu9GD48Lvm8VAvLmMCDw0sAgfjN4aorJksOMm3maC zoybG9VLIaPKxY/Lx/qnc5MNyyTwv5iaJ3qruU6ZfU0Pe0hj/n1AcgcbMqsK8Nvykfgluw R3Eyq1qEruvSFES+sWIr9/9iV5vnRUI= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=cR8Y34pT; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf05.hostedemail.com: domain of andreyknvl@gmail.com designates 209.85.128.41 as permitted sender) smtp.mailfrom=andreyknvl@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1706222155; a=rsa-sha256; cv=none; b=4IAqhDxTH8TF+UiK4XvrUkeZFl1CnmDdKEeRDt8QT8deQUw3IocrRK6eUUshfTca5Li0j4 CHQGo8O5eavjMrZsUsqoPJYdfaPsiWA/0mrjjbCKkOeSZaTKlMOkEUkU2enbtkYH3/jQy2 BXKgyItVT6pjrACBt5j7XS2xbXZZFBs= Received: by mail-wm1-f41.google.com with SMTP id 5b1f17b1804b1-40e76109cdeso1008785e9.0 for ; Thu, 25 Jan 2024 14:35:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1706222154; x=1706826954; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=ASPyASwMfpiYxl1C+amLASNi2699ENzeTUD4Y5JDI5E=; b=cR8Y34pTTM2XO2bZwRgZQqqnXpUiWEbnZ3aXKLC98G125enK9OBWkyq/m8gkmSmFK6 TYbQOzU6OhO6q370J5y2fFBA8fEEM5/DKHSTBYO4uBPmzY/nT9jUZrBzluRJU1zkVouZ Vr0nw85Y9unnD5W+paFfIiwpLEczYJN5nvsBzpkWn9vBOxHXz5FroC04++41pABg9yBv mB3zC8yYduRzCFuIHSUoe4bLGz/IdhNO7ksIwKVS6uU5Wx/meX2xDyQCB9/ITsyHA8q7 9hlutSMDiNjee45nLSP9ER0dSuc55n6BwBQbgQj+KujPP02DuSW3e+9nLEiTker4d7bg jIUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706222154; x=1706826954; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ASPyASwMfpiYxl1C+amLASNi2699ENzeTUD4Y5JDI5E=; b=ATFO66xQpNc0V7dBjFglO1QpkDVjvF29+wK5myygBhNb1ot1TItwZJ6ofzHk3l++pq GCWQ85aYyKJVWC9w50OvyIMJer8GPKCYz5pezvfxKU6ChU19v1L1qRdTaBMczNHbXVfU zlEFa6vDo9XsbAowrXlIkOCKOk72i64uFYrJtOlA2NqDD1h7vBdvhOJeGzPxTze7fezg NY0f3d90fCoSDoajEpKkUwY/waXlVJ8Bpqe2bmAKmBTudkoEodKeD8Ul2MurwQGQBZrJ DxnMPkJfNTdsbhCPOWnjd8jtlVJ69iOUSdsa9Wwm28027n1vm/QcQ6tcUn8jVWoaKmKf N0hQ== X-Gm-Message-State: AOJu0YwUgLcTNH0mmorEfnPGH7M0b3REETf9BcsasdcV8c0H8Oq9ZNbe H4nPcIptyIOU40utXA8bEv4/X1YmLq5lmEfnxeC2ptc5NF1u3OEZARWMAxrILv8no5NDVgii7eu s8poNY4e/pAV4H6dBL0dvAsLgYUA= X-Google-Smtp-Source: AGHT+IFwmOOp9s+BFvZENhiMNGpPqpcTB/utne4HbV5mgq41bPwx2/HqyS2+WV1YI1nnCt2qf1uLySUXSKDC0XtJVL0= X-Received: by 2002:a5d:6a88:0:b0:337:bf7f:a2c7 with SMTP id s8-20020a5d6a88000000b00337bf7fa2c7mr319823wru.23.1706222153744; Thu, 25 Jan 2024 14:35:53 -0800 (PST) MIME-Version: 1.0 References: <20240125094815.2041933-1-elver@google.com> In-Reply-To: <20240125094815.2041933-1-elver@google.com> From: Andrey Konovalov Date: Thu, 25 Jan 2024 23:35:42 +0100 Message-ID: Subject: Re: [PATCH 1/2] stackdepot: use variable size records for non-evictable entries To: Marco Elver Cc: Andrew Morton , Alexander Potapenko , Dmitry Vyukov , Vlastimil Babka , Andrey Ryabinin , Vincenzo Frascino , linux-kernel@vger.kernel.org, kasan-dev@googlegroups.com, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Stat-Signature: ixqwpbusmbjndzkp9ygg9mo93irm67un X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 22001100020 X-HE-Tag: 1706222154-429893 X-HE-Meta: U2FsdGVkX1/8VE0pUpAoKwQ2GyTcKySGlNvPcnR5Xl3fjR+5GfM996ejW9pxb/yS6xZGAcUUYZuFHh0Fm1ckc9DXRt4+vSyy/2ufzqDn7flbPV5hVWFa/fL6cczkue+axhPP/xCgHtOyMnUuWNs93zyoaZn9RKR7qHwsWnbqeovLn4QV4Y6nluc/H7Ww3uqByhMqhtcj98D3ZHYVCqcS3/YuzRCmCWwOUyHxuvAEA31xSlps+jqstQbxPqb0BX7+PO//+DlDuaGikJvNRLajjVHsIUg+MGSftVJi7AyW/hgDURx5H1gTrs5iuPInzqAqUmMzDWUrUwpoeBk3SRZhoHMu0rbsxNWRdbqn+lASHqJLA6CS1EkNkQ/l2gCu7/NZOtjHQxoUhUFTysnnOEXqocwCGbr9csTFSBPcxZ/MjnCYyVNCTDdBAzt1JECgHjAQRxupx1dtv7l7cPCRD3cz5R0XA7b/fUqQisG+ZhE9qPyzDrH4TqSd7tj3/dRp4sJhV+r67SQN8oSnaGlDMlF0faOKRVKl+sND8ceG2eEViqXczTNW4DSfD5KO5L4rs1HY3NJgWYUWHnn8X3wGGy9b+kIbK2A2qxIt1X7cBbtLOoa2hPzIWHucctVaM3M+aisOH88qH3ixH0fHdb+PnjUzR/r/ubyEWrCQQugiXkVbl0VbZ8XHMF83VLskN/boTHJgU3VWBO8G/8lMkA+DuJGj9KZuPkQRcVq/S6xKL0L4t7poVEIKMlC4M6oj40hhYllezjdEJCfeU9b6gKKoRr9ughUaNDYfo35AmDJJ8TX+5Wzgol8TzumAgvQEG+LSLTkVch+4j7x4iNvpRfz+B5b+c68hEIbyDrQqFdINUChuZEVk+22OMXhwUfhGH3WXEktODEKMPopCXJElXoozE+rNPxjjrHoIKfwpIF60D/wTBLKL3asURkyjqaBFRmAqSt3ylzgJqyrtWX/oJLEeOIg a9wv8a1H Tp6XYbiufUhkCfO6GjHvQTjIjnRYYU/T8zP/jxFq7kugYAR4OLOVTIVY/mvRlWMgvzzm/b9DXnEFqnpcUpy0qc1Mu4RsZdI5vvHdLvLV58WIFfB1mEtEkuORkcjar5oWcE4xqi6NfpUBafEmiaz5WWfWxM4KzT/WjGnjBaS7rrwS8r87n8Xj5VhZ7xoNJ1+SQHC7MG/0dclbVfgzs8Wn7mGJg1S0xBWPzM6a0/i4JT/6wagVyKyd3cPZnyNbQRp1kqrj2Ezw8k5RFxcvHfJg8SnWwcte/TaipPHbRFmN2TziazThmUrwD7ud7gCTqooORZkOVkmeIdx3kwwY2FL6RWQqZObkgT3LQnZJSAKf+4+79rAlUEAuNCGfGR8efXPlT1Q0aw0NOcTrYJxywI0pvdUTnb+1RG+FECEzWHFtalmgnRay8ZdgGTxvwCoq+EG2o2IsyLoq0TF1IPAQ7flImtdBNCw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Jan 25, 2024 at 10:48=E2=80=AFAM Marco Elver wro= te: > > With the introduction of stack depot evictions, each stack record is now > fixed size, so that future reuse after an eviction can safely store > differently sized stack traces. In all cases that do not make use of > evictions, this wastes lots of space. > > Fix it by re-introducing variable size stack records (up to the max > allowed size) for entries that will never be evicted. We know if an > entry will never be evicted if the flag STACK_DEPOT_FLAG_GET is not > provided, since a later stack_depot_put() attempt is undefined behavior. > > With my current kernel config that enables KASAN and also SLUB owner trac= king, > I observe (after a kernel boot) a whopping reduction of 296 stack depot p= ools, > which translates into 4736 KiB saved. The savings here are from SLUB owne= r > tracking only, because KASAN generic mode still uses refcounting. > > Before: > > pools: 893 > allocations: 29841 > frees: 6524 > in_use: 23317 > freelist_size: 3454 > > After: > > pools: 597 > allocations: 29657 > frees: 6425 > in_use: 23232 > freelist_size: 3493 > > Fixes: 108be8def46e ("lib/stackdepot: allow users to evict stack traces") > Signed-off-by: Marco Elver > Cc: Alexander Potapenko > Cc: Andrey Konovalov > Cc: Dmitry Vyukov > --- > v1 (since RFC): > * Get rid of new_pool_required to simplify the code. > * Warn on attempts to switch a non-refcounted entry to refcounting. > * Typos. > --- > include/linux/poison.h | 3 + > lib/stackdepot.c | 212 +++++++++++++++++++++-------------------- > 2 files changed, 113 insertions(+), 102 deletions(-) > > diff --git a/include/linux/poison.h b/include/linux/poison.h > index 27a7dad17eef..1f0ee2459f2a 100644 > --- a/include/linux/poison.h > +++ b/include/linux/poison.h > @@ -92,4 +92,7 @@ > /********** VFS **********/ > #define VFS_PTR_POISON ((void *)(0xF5 + POISON_POINTER_DELTA)) > > +/********** lib/stackdepot.c **********/ > +#define STACK_DEPOT_POISON ((void *)(0xD390 + POISON_POINTER_DELTA)) > + > #endif > diff --git a/lib/stackdepot.c b/lib/stackdepot.c > index 5caa1f566553..1b0d948a053c 100644 > --- a/lib/stackdepot.c > +++ b/lib/stackdepot.c > @@ -22,6 +22,7 @@ > #include > #include > #include > +#include > #include > #include > #include > @@ -93,9 +94,6 @@ struct stack_record { > }; > }; > > -#define DEPOT_STACK_RECORD_SIZE \ > - ALIGN(sizeof(struct stack_record), 1 << DEPOT_STACK_ALIGN) > - > static bool stack_depot_disabled; > static bool __stack_depot_early_init_requested __initdata =3D IS_ENABLED= (CONFIG_STACKDEPOT_ALWAYS_INIT); > static bool __stack_depot_early_init_passed __initdata; > @@ -121,15 +119,10 @@ static void *stack_pools[DEPOT_MAX_POOLS]; > static void *new_pool; > /* Number of pools in stack_pools. */ > static int pools_num; > +/* Offset to the unused space in the currently used pool. */ > +static size_t pool_offset =3D DEPOT_POOL_SIZE; > /* Freelist of stack records within stack_pools. */ > static LIST_HEAD(free_stacks); > -/* > - * Stack depot tries to keep an extra pool allocated even before it runs= out > - * of space in the currently used pool. This flag marks whether this ext= ra pool > - * needs to be allocated. It has the value 0 when either an extra pool i= s not > - * yet allocated or if the limit on the number of pools is reached. > - */ > -static bool new_pool_required =3D true; > /* The lock must be held when performing pool or freelist modifications.= */ > static DEFINE_RAW_SPINLOCK(pool_lock); > > @@ -294,48 +287,52 @@ int stack_depot_init(void) > EXPORT_SYMBOL_GPL(stack_depot_init); > > /* > - * Initializes new stack depot @pool, release all its entries to the fre= elist, > - * and update the list of pools. > + * Initializes new stack pool, and updates the list of pools. > */ > -static void depot_init_pool(void *pool) > +static bool depot_init_pool(void **prealloc) > { > - int offset; > - > lockdep_assert_held(&pool_lock); > > - /* Initialize handles and link stack records into the freelist. *= / > - for (offset =3D 0; offset <=3D DEPOT_POOL_SIZE - DEPOT_STACK_RECO= RD_SIZE; > - offset +=3D DEPOT_STACK_RECORD_SIZE) { > - struct stack_record *stack =3D pool + offset; > - > - stack->handle.pool_index =3D pools_num; > - stack->handle.offset =3D offset >> DEPOT_STACK_ALIGN; > - stack->handle.extra =3D 0; > - > - /* > - * Stack traces of size 0 are never saved, and we can sim= ply use > - * the size field as an indicator if this is a new unused= stack > - * record in the freelist. > - */ > - stack->size =3D 0; > + if (unlikely(pools_num >=3D DEPOT_MAX_POOLS)) { > + /* Bail out if we reached the pool limit. */ > + WARN_ON_ONCE(pools_num > DEPOT_MAX_POOLS); /* should neve= r happen */ > + WARN_ON_ONCE(!new_pool); /* to avoid unnecessary pre-allo= cation */ > + WARN_ONCE(1, "Stack depot reached limit capacity"); > + return false; > + } > > - INIT_LIST_HEAD(&stack->hash_list); > - /* > - * Add to the freelist front to prioritize never-used ent= ries: > - * required in case there are entries in the freelist, bu= t their > - * RCU cookie still belongs to the current RCU grace peri= od > - * (there can still be concurrent readers). > - */ > - list_add(&stack->free_list, &free_stacks); > - counters[DEPOT_COUNTER_FREELIST_SIZE]++; > + if (!new_pool && *prealloc) { > + /* We have preallocated memory, use it. */ > + WRITE_ONCE(new_pool, *prealloc); > + *prealloc =3D NULL; > } > > + if (!new_pool) > + return false; /* new_pool and *prealloc are NULL */ > + > /* Save reference to the pool to be used by depot_fetch_stack(). = */ > - stack_pools[pools_num] =3D pool; > + stack_pools[pools_num] =3D new_pool; > + > + /* > + * Stack depot tries to keep an extra pool allocated even before = it runs > + * out of space in the currently used pool. > + * > + * To indicate that a new preallocation is needed new_pool is res= et to > + * NULL; do not reset to NULL if we have reached the maximum numb= er of > + * pools. > + */ > + if (pools_num < DEPOT_MAX_POOLS) > + WRITE_ONCE(new_pool, NULL); > + else > + WRITE_ONCE(new_pool, STACK_DEPOT_POISON); > > /* Pairs with concurrent READ_ONCE() in depot_fetch_stack(). */ > WRITE_ONCE(pools_num, pools_num + 1); > ASSERT_EXCLUSIVE_WRITER(pools_num); > + > + pool_offset =3D 0; > + > + return true; > } > > /* Keeps the preallocated memory to be used for a new stack depot pool. = */ > @@ -347,60 +344,48 @@ static void depot_keep_new_pool(void **prealloc) > * If a new pool is already saved or the maximum number of > * pools is reached, do not use the preallocated memory. > */ > - if (!new_pool_required) > + if (new_pool) > return; > > - /* > - * Use the preallocated memory for the new pool > - * as long as we do not exceed the maximum number of pools. > - */ > - if (pools_num < DEPOT_MAX_POOLS) { > - new_pool =3D *prealloc; > - *prealloc =3D NULL; > - } > - > - /* > - * At this point, either a new pool is kept or the maximum > - * number of pools is reached. In either case, take note that > - * keeping another pool is not required. > - */ > - WRITE_ONCE(new_pool_required, false); > + WRITE_ONCE(new_pool, *prealloc); > + *prealloc =3D NULL; > } > > /* > - * Try to initialize a new stack depot pool from either a previous or th= e > - * current pre-allocation, and release all its entries to the freelist. > + * Try to initialize a new stack record from the current pool, a cached = pool, or > + * the current pre-allocation. > */ > -static bool depot_try_init_pool(void **prealloc) > +static struct stack_record *depot_pop_free_pool(void **prealloc, size_t = size) > { > + struct stack_record *stack; > + void *current_pool; > + u32 pool_index; > + > lockdep_assert_held(&pool_lock); > > - /* Check if we have a new pool saved and use it. */ > - if (new_pool) { > - depot_init_pool(new_pool); > - new_pool =3D NULL; > + if (pool_offset + size > DEPOT_POOL_SIZE) { > + if (!depot_init_pool(prealloc)) > + return NULL; > + } > > - /* Take note that we might need a new new_pool. */ > - if (pools_num < DEPOT_MAX_POOLS) > - WRITE_ONCE(new_pool_required, true); > + if (WARN_ON_ONCE(pools_num < 1)) > + return NULL; > + pool_index =3D pools_num - 1; > + current_pool =3D stack_pools[pool_index]; > + if (WARN_ON_ONCE(!current_pool)) > + return NULL; > > - return true; > - } > + stack =3D current_pool + pool_offset; > > - /* Bail out if we reached the pool limit. */ > - if (unlikely(pools_num >=3D DEPOT_MAX_POOLS)) { > - WARN_ONCE(1, "Stack depot reached limit capacity"); > - return false; > - } > + /* Pre-initialize handle once. */ > + stack->handle.pool_index =3D pool_index; > + stack->handle.offset =3D pool_offset >> DEPOT_STACK_ALIGN; > + stack->handle.extra =3D 0; > + INIT_LIST_HEAD(&stack->hash_list); > > - /* Check if we have preallocated memory and use it. */ > - if (*prealloc) { > - depot_init_pool(*prealloc); > - *prealloc =3D NULL; > - return true; > - } > + pool_offset +=3D size; > > - return false; > + return stack; > } > > /* Try to find next free usable entry. */ Please update this to specifically mention the freelist. Otherwise, it's hard to understand what's the difference compared to depot_pop_free_pool without reading into the code. > @@ -420,7 +405,7 @@ static struct stack_record *depot_pop_free(void) > * check the first entry. > */ > stack =3D list_first_entry(&free_stacks, struct stack_record, fre= e_list); > - if (stack->size && !poll_state_synchronize_rcu(stack->rcu_state)) > + if (!poll_state_synchronize_rcu(stack->rcu_state)) > return NULL; > > list_del(&stack->free_list); > @@ -429,45 +414,68 @@ static struct stack_record *depot_pop_free(void) > return stack; > } > > +static inline size_t depot_stack_record_size(struct stack_record *s, uns= igned int nr_entries) > +{ > + const size_t used =3D flex_array_size(s, entries, nr_entries); > + const size_t unused =3D sizeof(s->entries) - used; > + > + WARN_ON_ONCE(sizeof(s->entries) < used); > + > + return ALIGN(sizeof(struct stack_record) - unused, 1 << DEPOT_STA= CK_ALIGN); > +} > + > /* Allocates a new stack in a stack depot pool. */ > static struct stack_record * > -depot_alloc_stack(unsigned long *entries, int size, u32 hash, void **pre= alloc) > +depot_alloc_stack(unsigned long *entries, int nr_entries, u32 hash, depo= t_flags_t flags, void **prealloc) > { > - struct stack_record *stack; > + struct stack_record *stack =3D NULL; > + size_t record_size; > > lockdep_assert_held(&pool_lock); > > /* This should already be checked by public API entry points. */ > - if (WARN_ON_ONCE(!size)) > + if (WARN_ON_ONCE(!nr_entries)) > return NULL; > > - /* Check if we have a stack record to save the stack trace. */ > - stack =3D depot_pop_free(); > - if (!stack) { > - /* No usable entries on the freelist - try to refill the = freelist. */ > - if (!depot_try_init_pool(prealloc)) > - return NULL; > + /* Limit number of saved frames to CONFIG_STACKDEPOT_MAX_FRAMES. = */ > + if (nr_entries > CONFIG_STACKDEPOT_MAX_FRAMES) > + nr_entries =3D CONFIG_STACKDEPOT_MAX_FRAMES; > + > + if (flags & STACK_DEPOT_FLAG_GET) { > + /* > + * Evictable entries have to allocate the max. size so th= ey may > + * safely be re-used by differently sized allocations. > + */ > + record_size =3D depot_stack_record_size(stack, CONFIG_STA= CKDEPOT_MAX_FRAMES); > stack =3D depot_pop_free(); > - if (WARN_ON(!stack)) > - return NULL; > + } else { > + record_size =3D depot_stack_record_size(stack, nr_entries= ); > } > > - /* Limit number of saved frames to CONFIG_STACKDEPOT_MAX_FRAMES. = */ > - if (size > CONFIG_STACKDEPOT_MAX_FRAMES) > - size =3D CONFIG_STACKDEPOT_MAX_FRAMES; > + if (!stack) { > + stack =3D depot_pop_free_pool(prealloc, record_size); > + if (!stack) > + return NULL; > + } > > /* Save the stack trace. */ > stack->hash =3D hash; > - stack->size =3D size; > - /* stack->handle is already filled in by depot_init_pool(). */ > - refcount_set(&stack->count, 1); > - memcpy(stack->entries, entries, flex_array_size(stack, entries, s= ize)); > + stack->size =3D nr_entries; > + /* stack->handle is already filled in by depot_pop_free_pool(). *= / > + memcpy(stack->entries, entries, flex_array_size(stack, entries, n= r_entries)); > + > + if (flags & STACK_DEPOT_FLAG_GET) { > + refcount_set(&stack->count, 1); > + } else { > + /* Warn on attempts to switch to refcounting this entry. = */ > + refcount_set(&stack->count, REFCOUNT_SATURATED); > + } > > /* > * Let KMSAN know the stored stack record is initialized. This sh= all > * prevent false positive reports if instrumented code accesses i= t. > */ > - kmsan_unpoison_memory(stack, DEPOT_STACK_RECORD_SIZE); > + kmsan_unpoison_memory(stack, record_size); > > counters[DEPOT_COUNTER_ALLOCS]++; > counters[DEPOT_COUNTER_INUSE]++; I wonder if we should separate the stat counters for evictable/non-evictable cases. For non-evictable, we could count the amount of consumed memory. > @@ -660,7 +668,7 @@ depot_stack_handle_t stack_depot_save_flags(unsigned = long *entries, > * Allocate memory for a new pool if required now: > * we won't be able to do that under the lock. > */ > - if (unlikely(can_alloc && READ_ONCE(new_pool_required))) { > + if (unlikely(can_alloc && !READ_ONCE(new_pool))) { > /* > * Zero out zone modifiers, as we don't have specific zon= e > * requirements. Keep the flags related to allocation in = atomic > @@ -681,7 +689,7 @@ depot_stack_handle_t stack_depot_save_flags(unsigned = long *entries, > found =3D find_stack(bucket, entries, nr_entries, hash, depot_fla= gs); > if (!found) { > struct stack_record *new =3D > - depot_alloc_stack(entries, nr_entries, hash, &pre= alloc); > + depot_alloc_stack(entries, nr_entries, hash, depo= t_flags, &prealloc); > > if (new) { > /* > -- > 2.43.0.429.g432eaa2c6b-goog > We can also now drop the special case for DEPOT_POOLS_CAP for KMSAN. Otherwise, looks good to me. Reviewed-by: Andrey Konovalov Thank you for cleaning this up!