From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61333EB64DA for ; Mon, 3 Jul 2023 00:09:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C78DC8E0087; Sun, 2 Jul 2023 20:09:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C28878E007C; Sun, 2 Jul 2023 20:09:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AF0158E0087; Sun, 2 Jul 2023 20:09:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 9E8F08E007C for ; Sun, 2 Jul 2023 20:09:17 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 42058AFCF4 for ; Mon, 3 Jul 2023 00:09:17 +0000 (UTC) X-FDA: 80968365954.18.79769F2 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by imf18.hostedemail.com (Postfix) with ESMTP id 6B8591C0005 for ; Mon, 3 Jul 2023 00:09:14 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=LJzCZsVl; spf=pass (imf18.hostedemail.com: domain of rientjes@google.com designates 209.85.214.169 as permitted sender) smtp.mailfrom=rientjes@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688342954; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jb4GeIrc9P/pQH3ByeVCMb+DRHFR85yXe4TsMAD4d3Q=; b=om1KfiURP4qcLSVfXV8Z3x1RFeifBAL8Rhup0jB6HesmQi2dGcYS6f/tLBN26VAPdIx+rw Se7ZFFUxXPFMAw76uIGivkXa6gqaMBV9UFhFCADoFL6pu4Y1ndZAaXDlZHHYNQQ1LKm07a s/AUrX/WwuT9kSufQ+racTjtLmwj/xQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688342954; a=rsa-sha256; cv=none; b=VGzQRD9gohY0/7AXRuzIDKvX6+CofzdoXx0gS0Xk0aFnujdIa+I2hSf80qzY7wCGlzWyGE h2h3TOepWultHRjH9mDQHaE7nFmm+BodyWBVqtg2Dz1n83c4JJ5N9w8dX16fIWobS/ocig z+fKsGaVDSRqb04vMSOZ2v/54Cuergk= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=LJzCZsVl; spf=pass (imf18.hostedemail.com: domain of rientjes@google.com designates 209.85.214.169 as permitted sender) smtp.mailfrom=rientjes@google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-pl1-f169.google.com with SMTP id d9443c01a7336-1b52875b8d9so272165ad.0 for ; Sun, 02 Jul 2023 17:09:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1688342953; x=1690934953; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=jb4GeIrc9P/pQH3ByeVCMb+DRHFR85yXe4TsMAD4d3Q=; b=LJzCZsVl+EExK5eipuc9pcTlYtaOz/i9LD/+MjFdKqmWqQe6VAuwSEF4/oLsgbeqju eSYgtvU5XdoBMgID2GYvrRFwsWFbQ/6qwYJyYnXU8aF2ExRDbabXa/rwF2mEgNyLgUah QdJzxH7yCAQqWL+s8ntnRNQ5wB6UtdiD6N6zgkjJ6OhS4NvbAnYMED9VD4Mu4VQlMS+i HdWpc+w6i9FfJ1lLZcy4oGIrhO4tZsoprkkc0ZKaPS23aFoeo6phmExzXoacByLUbsCo 2tJx/V428/q45ZxL/NJnvNxohz++6zRqGvzP9Wdq94xguI3H7BbBDaEx1/6IN02gOW5X 8F2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688342953; x=1690934953; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=jb4GeIrc9P/pQH3ByeVCMb+DRHFR85yXe4TsMAD4d3Q=; b=SWzobpfACuvOzDmpTPjBf6ZDEOvXD4EM4USa6tc5dnfyzd7E2c/6f7TqCPFGN0ng+y 5KCuCm11ec3coPKwWaBCCbp5cHrnLDF4o0xvXiB414alg4F9cyFboqX9aQdckuCfq0WW uNKhobSMgqBN5PprIfY3GOZ+/5cMfGhDyQkdO0BYpL7dQN3T6S0aBrcjjAXcx02WtzGx a4srDuK8N+E7RZAVqCwvB8dkW2ALPsTuEhIWp+mn0oWl41SJ/4unTdBCsXaF4+FNcyLV ut2G7keN02BFm7twGNH8zcHkGzxK6nm3/NjIIBf7vSDdi8Egimvp94QZklMj17wrPsVW uIIA== X-Gm-Message-State: ABy/qLYftCkzbfVXwnfC8d4W/uNmiK4v+wun8FngJJoKkY7MRz8bssXn BjkUV6HiiN8uLIJW6Lhh/r2cgg== X-Google-Smtp-Source: APBJJlH23TcE4Z/nu6BSjuwfnnS+VU9MtqskrHpVVc3C77p3DlfwT/7nBIf1hrTcH1RevkMg/h8/aQ== X-Received: by 2002:a17:902:da8f:b0:1b8:9044:91d3 with SMTP id j15-20020a170902da8f00b001b8904491d3mr134115plx.18.1688342953045; Sun, 02 Jul 2023 17:09:13 -0700 (PDT) Received: from [2620:0:1008:15:f9d4:d371:9d50:fcd6] ([2620:0:1008:15:f9d4:d371:9d50:fcd6]) by smtp.gmail.com with ESMTPSA id p15-20020a63e64f000000b0051b0e564963sm13911327pgj.49.2023.07.02.17.09.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 02 Jul 2023 17:09:12 -0700 (PDT) Date: Sun, 2 Jul 2023 17:09:11 -0700 (PDT) From: David Rientjes To: Julian Pidancet cc: Christoph Lameter , "Lameter, Christopher" , Pekka Enberg , Joonsoo Kim , Andrew Morton , Vlastimil Babka , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org, Jonathan Corbet , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, Matthew Wilcox , Kees Cook , Rafael Aquini Subject: Re: [PATCH v2] mm/slub: disable slab merging in the default configuration In-Reply-To: <20230629221910.359711-1-julian.pidancet@oracle.com> Message-ID: <38083ed2-333b-e245-44e4-2f355e4f9249@google.com> References: <20230629221910.359711-1-julian.pidancet@oracle.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="2003067076-267791279-1688342952=:3716371" X-Rspamd-Queue-Id: 6B8591C0005 X-Rspam-User: X-Stat-Signature: u1sk1kmzmu1ndcwiprcakoxeyseqyadu X-Rspamd-Server: rspam03 X-HE-Tag: 1688342954-37313 X-HE-Meta: U2FsdGVkX19wsKwVqFvXwSJZSt6LOdHfRFq/1MpmfI4QDUJ4zZakVrj/N6xZ1QbrU3dPifdSBKvNbmPYk/tI+ol+eEroWlNdsVvoKuTtOcOS8Bp6yO/QKeo/aPczpMxdSySlDTNRu67nZwxqWJrff7aa8gVH49dtJjUelz+oL3W2dRKFDyGnqb82fMzdFrmAbuXX4JZEBZ9NICTQcOuWhdxPk6kyMKR5kdA+8gulyAkSxfynq5WHjatuEkIk9amvDfFO3NCsnc1llYriNsHjDT12S/iBGgaPVVNV6bvG0ruaUVhrqR8cGrjoWkjfdTFxUDpfmSCFYv0y2+7mO6nz1/QpnWBZn5Oz9J2NCv6w4roOiCvkuG1ohcmEa2DzLqYOi9CqU30aGzZWY23nK6Dv6fzXSRvUJpbhDFP5bffcLQGutcopdOLTVNTOkiNFq1RUZTn0abZRHS1ZIKBhfXzg/HNjULvrdumYLIQZcboGDO5S8RzQhduyWPP1H2VOymz0AnLrYJotWkI+/YyCUCWMj8j0SycQogVdojsbpLettgx/RW9CjnkYdX69lJjz0z7cZqasLWFtL5gLAy+Y/odo6ho9Wty7tPmohGx2xSytITqTucxP4w4+lCsCCHOw2RuTRVW0JZDx7TEKjQhWLzmqbJDjJ1S+ljGgojier/T0L0lVdtikh/8t45doNYqIODsyVc3kLzLsit6mlBAt7zqtKx67NLVnkqfgCvxwJhwlsdzof28nyHdxKea/GP2RscYMxIDbcVbGAMzNPKB22s3M6S5Xe5P/CDc+m8E0vX4B/DpEL5ggIQAKCI02SpnWHSHe+k0szoOui89DcrbxoEDyEN18rlLHs2r0Sqyh6tXTI8FmL46czaBie2Hp9uyr/ZeoJlRtGFm3rRfg0TpvadHwfSdsrBXElw5VOGRelZIhomLETeNgOvS5h7C4yma3OKF5OokxOaRrYGS76yD9uvQ lqKsFtdv VzaCpmC6/zBKkFooQyzOTfd+7MLY0CKizesF8GTeUGcvDL41cYTAr0oCQwYBopa2TNiOn06xX+6w7E4Qv2be8fugQUYwpGrS2dUkEiNMRAPjhShczFvHSV26kTH39ZTaacDaIZ4DELGdFPotBDtXTIKvJHuMbqmq60063PWxMfstRbunPcVn4+quLxsp463HscnHJXjjtjXNLZAJS1QFr5lNoGLHjOTwvYSLFE82gJ/JjhMPZekmYAsQ1XRTSn42QG2X8CLupTEd1DKfIBILG6x6CeMs/Z5GZUA4B9AQj+enLN8JGfY5G1DRi7Mco8RRukwwRwJZlmcdRW/saTi8EsZ2F0+e1rewaWHLXtz/+Sry1RlQiAt258S7HQwh7r9CsE0A9NQ6NFnfdNG3fHPZebBGCS2UDUOHkY30PUR1+L5mDVMmrJxdmKwqwFzGXl/oRr5A2lfKRsA1YS9muKPh8fuNypHUf9A9ffIWJGvGe+Itn6pTUuXDO7EO0oYcklGNPfbX+xDQqZhELD3ARuJ6erCANtlUuWcWD208XvoAOU+uyaQTxnzaYleGOKWV7g6rahON3cREHw8Ro135G84rDFzo6P/wT3ovRMp0jDm/KW+Rc4PbaMmlEk6q3jyyNdrua/ZiZAvWjWYUdysHBMDAiEbFGqfAdDwbB687X X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --2003067076-267791279-1688342952=:3716371 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8BIT On Fri, 30 Jun 2023, Julian Pidancet wrote: > Make CONFIG_SLAB_MERGE_DEFAULT default to n unless CONFIG_SLUB_TINY is > enabled. Benefits of slab merging is limited on systems that are not > memory constrained: the memory overhead is low and evidence of its > effect on cache hotness is hard to come by. > > On the other hand, distinguishing allocations into different slabs will > make attacks that rely on "heap spraying" more difficult to carry out > with success. > > Take sides with security in the default kernel configuration over > questionnable performance benefits/memory efficiency. > > A timed kernel compilation test, on x86 with 4K pages, conducted 10 > times with slab_merge, and the same test then conducted with > slab_nomerge on the same hardware in a similar state do not show any > sign of performance hit one way or another: > > | slab_merge | slab_nomerge | > ------+------------------+------------------| > Time | 588.080 ± 0.799 | 587.308 ± 1.411 | > Min | 586.267 | 584.640 | > Max | 589.248 | 590.091 | > > Peaks in slab usage during the test workload reveal a memory overhead > of 2.2 MiB when using slab_nomerge. Slab usage overhead after a fresh boot > amounts to 2.3 MiB: > > Slab Usage | slab_merge | slab_nomerge | > -------------------+------------+--------------| > After fresh boot | 79908 kB | 82284 kB | > During test (peak) | 127940 kB | 130204 kB | > > Signed-off-by: Julian Pidancet > Reviewed-by: Kees Cook Thanks for continuing to work on this. I think we need more data beyond just kernbench. Christoph's point about different page sizes is interesting. In the above results, I don't know the page orders for the various slab caches that this workload will stress. I think the memory overhead data may be different depending on how slab_max_order is being used, if at all. We should be able to run this through a variety of different benchmarks and measure peak slab usage at the same time for due diligence. I support the change in the default, I would just prefer to know what the implications of it is. Is it possible to collect data for other microbenchmarks and real-world workloads? And perhaps also with different page sizes where this will impact memory overhead more? I can help running more workloads once we have the next set of data. > --- > > v2: > - Re-run benchmark to minimize variance in results due to CPU > frequency scaling. > - Record slab usage after boot and peaks during tests workload. > - Include benchmark results in commit message. > - Fix typo: s/MEGE/MERGE/. > - Specify that "overhead" refers to memory overhead in SLUB doc. > > v1: > - Link: https://lore.kernel.org/linux-mm/20230627132131.214475-1-julian.pidancet@oracle.com/ > > .../admin-guide/kernel-parameters.txt | 29 ++++++++++--------- > Documentation/mm/slub.rst | 7 +++-- > mm/Kconfig | 6 ++-- > 3 files changed, 22 insertions(+), 20 deletions(-) > > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt > index c5e7bb4babf0..7e78471a96b7 100644 > --- a/Documentation/admin-guide/kernel-parameters.txt > +++ b/Documentation/admin-guide/kernel-parameters.txt > @@ -5652,21 +5652,22 @@ > > slram= [HW,MTD] > > - slab_merge [MM] > - Enable merging of slabs with similar size when the > - kernel is built without CONFIG_SLAB_MERGE_DEFAULT. > - > slab_nomerge [MM] > - Disable merging of slabs with similar size. May be > - necessary if there is some reason to distinguish > - allocs to different slabs, especially in hardened > - environments where the risk of heap overflows and > - layout control by attackers can usually be > - frustrated by disabling merging. This will reduce > - most of the exposure of a heap attack to a single > - cache (risks via metadata attacks are mostly > - unchanged). Debug options disable merging on their > - own. > + Disable merging of slabs with similar size when > + the kernel is built with CONFIG_SLAB_MERGE_DEFAULT. > + Allocations of the same size made in distinct > + caches will be placed in separate slabs. In > + hardened environment, the risk of heap overflows > + and layout control by attackers can usually be > + frustrated by disabling merging. > + > + slab_merge [MM] > + Enable merging of slabs with similar size. May be > + necessary to reduce overhead or increase cache > + hotness of objects, at the cost of increased > + exposure in case of a heap attack to a single > + cache. (risks via metadata attacks are mostly > + unchanged). > For more information see Documentation/mm/slub.rst. > > slab_max_order= [MM, SLAB] > diff --git a/Documentation/mm/slub.rst b/Documentation/mm/slub.rst > index be75971532f5..0e2ce82177c0 100644 > --- a/Documentation/mm/slub.rst > +++ b/Documentation/mm/slub.rst > @@ -122,9 +122,10 @@ used on the wrong slab. > Slab merging > ============ > > -If no debug options are specified then SLUB may merge similar slabs together > -in order to reduce overhead and increase cache hotness of objects. > -``slabinfo -a`` displays which slabs were merged together. > +If the kernel is built with ``CONFIG_SLAB_MERGE_DEFAULT`` or if ``slab_merge`` > +is specified on the kernel command line, then SLUB may merge similar slabs > +together in order to reduce memory overhead and increase cache hotness of > +objects. ``slabinfo -a`` displays which slabs were merged together. > Suggest mentioning that one of the primary goals of slab cache merging is to reduce cache footprint. > Slab validation > =============== --2003067076-267791279-1688342952=:3716371--