From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88DABC43334 for ; Mon, 20 Jun 2022 12:02:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2BB428E0002; Mon, 20 Jun 2022 08:02:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 244BC6B0074; Mon, 20 Jun 2022 08:02:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 10B598E0002; Mon, 20 Jun 2022 08:02:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 0130E6B0071 for ; Mon, 20 Jun 2022 08:02:14 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id C43A880D15 for ; Mon, 20 Jun 2022 12:02:14 +0000 (UTC) X-FDA: 79598476188.22.9118A9D Received: from mail-lj1-f169.google.com (mail-lj1-f169.google.com [209.85.208.169]) by imf18.hostedemail.com (Postfix) with ESMTP id 634141C00A3 for ; Mon, 20 Jun 2022 12:02:14 +0000 (UTC) Received: by mail-lj1-f169.google.com with SMTP id j22so5021642ljg.0 for ; Mon, 20 Jun 2022 05:02:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=BLIpOQctpgnjuRGoLORv+IdWppOjSjpJpZ0mv6ogIJE=; b=sXqeUBSqOL43Vh8bHlb9rMEuFx7rlgS62oKqDWhZLaubCkID368lLVMDzc/zJiZ2FL voUeNxWRn/PgWnkQpB+dg3eE1Ee20BYjRrTnOun/bP2s6Tf1i1LWNSYAeJ69wcTBi38s MC8sl2u+S9TsCBRrPA+y6jCcbleXwPCUsxW0bP5Hx5lww+ct6el9W7+tdfNTsuwLX4+g 8dPmAvGZQn7gAUBZy9pKvO8amGfDkenAgdDm4avBdJw0nxiFUPPVSFxEi0VZiuqbovXs jwCD1voo1i4hV6frJPLJO59aAZo69S5mvcXTjj2YYGAjWbmVddHCD0d7qazot+7rMuay FtOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=BLIpOQctpgnjuRGoLORv+IdWppOjSjpJpZ0mv6ogIJE=; b=WtXfKAc7RoRIUCuaRc/YRPDQA6U/899Ua9LloDY29sIrSoJc1gG13Yer7XTPF1dlUq QNWZ6FdYJs041iAWkS7+D6xiebv2R6HtxLqAuGLhxVc2ZrL/bxwEzy73tsN9/UuRF5NH 5IyvP1Os288651xdTb29M9otxoRm66TgvHMKAFRVQPPkTA+Z+MgVhQie4c5wY1KlzP/1 REXaC+0OPJyQHZjTANtUPs/u7u00O1tDFSqQmwGHmJx1xYVapvwtn5X0/1jxAO46uBrs e3O9luGRRJZn8gL56Jj664n6jvzrWCX0+NiXTr4AGr/aT+hucDnIBlu8YOxOgk6PzrmL Ezjg== X-Gm-Message-State: AJIora+8RDBsgYEh9vPZnZeisQ/BINwnKeO4ZcF//lZLSkBms8k+UZlp naGMYYuVP3wP42nKihbEsZ9zF/jZegIgq3pN09brxA== X-Google-Smtp-Source: AGRyM1tpR/fkd1caq35uMR9/85+YZunSAOI/2g68DexLZBss9W59Y+WfG/aG/kjjiNwlTTCtM0B9GAN5MGA6phRY04Q= X-Received: by 2002:a2e:808e:0:b0:255:be23:1372 with SMTP id i14-20020a2e808e000000b00255be231372mr11347409ljg.4.1655726532388; Mon, 20 Jun 2022 05:02:12 -0700 (PDT) MIME-Version: 1.0 References: <20220527113706.24870-1-vbabka@suse.cz> <20220527113706.24870-2-vbabka@suse.cz> <93bf8148-ecc1-75fb-423b-2a76c7252c4e@suse.cz> In-Reply-To: <93bf8148-ecc1-75fb-423b-2a76c7252c4e@suse.cz> From: Dmitry Vyukov Date: Mon, 20 Jun 2022 14:02:00 +0200 Message-ID: Subject: Re: [RFC PATCH 1/1] lib/stackdepot: replace CONFIG_STACK_HASH_ORDER with automatic sizing To: Vlastimil Babka Cc: Marco Elver , Alexander Potapenko , Linus Torvalds , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Andrey Konovalov , kasan-dev@googlegroups.com Content-Type: text/plain; charset="UTF-8" ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1655726534; a=rsa-sha256; cv=none; b=jP7beG7VsMhzhkVfYRTFvGvRf++P+vUXaKiqurkZuVw06SNcKPGSuA+soz8Vs8Z9st7lXE p79DfWEzZ4n1GIwWlEOeUOfH9QfCOXm8s57pZ2IoCe6D/Q1X1AKcB/fpEGeCfudQvrcHdp hUjTEWETdsBavsol4N3m4+eqP0wgILo= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=sXqeUBSq; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf18.hostedemail.com: domain of dvyukov@google.com designates 209.85.208.169 as permitted sender) smtp.mailfrom=dvyukov@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1655726534; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BLIpOQctpgnjuRGoLORv+IdWppOjSjpJpZ0mv6ogIJE=; b=Dm6lOT+MXvuq4jNPlAy0U5Yx34qqtSye5/+3XFgaXHsHOSyzpzvK6jISetWBzXNB+XY97i 9pjxdzmuztvSYffs+bo6qpUHbsSTyozva0wmB2h3At32sIfswJ2hR7rbMXdNEg57rwnDT2 gwqCDxxtx1kBTDm2G+S6AoPF0uppYK8= X-Rspamd-Queue-Id: 634141C00A3 X-Rspam-User: Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=sXqeUBSq; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf18.hostedemail.com: domain of dvyukov@google.com designates 209.85.208.169 as permitted sender) smtp.mailfrom=dvyukov@google.com X-Rspamd-Server: rspam06 X-Stat-Signature: odrbspynccrz9mshstxcn6j81hawpztw X-HE-Tag: 1655726534-424786 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, 20 Jun 2022 at 14:00, Vlastimil Babka wrote: > > On 5/27/22 14:02, Dmitry Vyukov wrote: > > On Fri, 27 May 2022 at 13:37, Vlastimil Babka wrote: > >> > >> As Linus explained [1], setting the stackdepot hash table size as a > >> config option is suboptimal, especially as stackdepot becomes a > >> dependency of less specialized subsystems than initially (e.g. DRM, > >> networking, SLUB_DEBUG): > >> > >> : (a) it introduces a new compile-time question that isn't sane to ask > >> : a regular user, but is now exposed to regular users. > >> > >> : (b) this by default uses 1MB of memory for a feature that didn't in > >> : the past, so now if you have small machines you need to make sure you > >> : make a special kernel config for them. > >> > >> Ideally we would employ rhashtable for fully automatic resizing, which > >> should be feasible for many of the new users, but problematic for the > >> original users with restricted context that call __stack_depot_save() > >> with can_alloc == false, i.e. KASAN. > >> > >> However we can easily remove the config option and scale the hash table > >> automatically with system memory. The STACK_HASH_MASK constant becomes > >> stack_hash_mask variable and is used only in one mask operation, so the > >> overhead should be negligible to none. For early allocation we can > >> employ the existing alloc_large_system_hash() function and perform > >> similar scaling for the late allocation. > >> > >> The existing limits of the config option (between 4k and 1M buckets) > >> are preserved, and scaling factor is set to one bucket per 16kB memory > >> so on 64bit the max 1M buckets (8MB memory) is achieved with 16GB > >> system, while a 1GB system will use 512kB. > > > > Hi Vlastimil, > > > > We use KASAN with VMs with 2GB of memory. > > If I did the math correctly this will result in 128K entries, while > > currently we have CONFIG_STACK_HASH_ORDER=20 even for arm32. > > I am actually not sure how full the table gets, but we can fuzz a > > large kernel for up to an hour, so we can get lots of stacks (we were > > the only known users who routinely overflowed default LOCKDEP tables > > :)). > > Aha, good to know the order of 20 has some real use case then :) > > > I am not opposed to this in general. And I understand that KASAN Is > > different from the other users. > > What do you think re allowing CONFIG_STACK_HASH_ORDER=0/is not set > > which will mean auto-size, but keeping ability to set exact size as > > well? > > Or alternatively auto-size if KASAN is not enabled and use a large > > table otherwise? But I am not sure if anybody used > > CONFIG_STACK_HASH_ORDER to reduce the default size with KASAN... > > Well if you're unsure and nobody else requested it so far, we could try > setting it to 20 when KASAN is enabled, and autosize otherwise. If somebody > comes up with a use-case for the boot-time parameter override (instead of > CONFIG_), we can add it then? > >> If needed, the automatic scaling could be complemented with a boot-time > >> kernel parameter, but it feels pointless to add it without a specific > >> use case. Works for me.