From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8340AC761A6 for ; Wed, 22 Mar 2023 13:02:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F286A6B0075; Wed, 22 Mar 2023 09:02:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id ED7876B0078; Wed, 22 Mar 2023 09:02:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D9F166B007B; Wed, 22 Mar 2023 09:02:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id CC0096B0075 for ; Wed, 22 Mar 2023 09:02:48 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 127731202DC for ; Wed, 22 Mar 2023 13:02:47 +0000 (UTC) X-FDA: 80596548816.05.1B6DAD7 Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by imf20.hostedemail.com (Postfix) with ESMTP id B962D1C001E for ; Wed, 22 Mar 2023 13:02:45 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=ARhhmfUo; spf=pass (imf20.hostedemail.com: domain of 42.hyeyoo@gmail.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=42.hyeyoo@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1679490165; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ViRmkYI1T/q9AleFDZxnzz9BK+OfJL44ohwRmj/wb8E=; b=KWvzNFQx+DUc9QeWO4F2BM2BUXH6YucPPISpvMRLxDvVCMd+H+yMI0zjPoS5BehKcXSGK8 gskQ2X3NWLybtnxcJ/0Ftl1Mel+QPW2PIys2PF4sOrejrYuyUYxqafIwyIV2jkIJWMxnuH cYRVR3HObjND5VqexoKi4TNSTMqgSY0= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=ARhhmfUo; spf=pass (imf20.hostedemail.com: domain of 42.hyeyoo@gmail.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=42.hyeyoo@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1679490165; a=rsa-sha256; cv=none; b=xBCLddgzAFoupRlQenKF8PYzX/Fj/2kLp5Dq9z/8/v7y6NLfV+e0zk3GY3Zl49UkHVqx75 U2afwfjL01geVyjDteJ5YN8Z7HGpycEffVmVE6iKOWi24Nml8QWq6yns0RK07NZ/oLN1fs EVLO5wWIGnrK65HL7NdZOv4VCkwnhD4= Received: by mail-pl1-f180.google.com with SMTP id c18so19172132ple.11 for ; Wed, 22 Mar 2023 06:02:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1679490164; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date:from:to :cc:subject:date:message-id:reply-to; bh=ViRmkYI1T/q9AleFDZxnzz9BK+OfJL44ohwRmj/wb8E=; b=ARhhmfUoXSena1pM+E1Grp6YR3kCOnLV3kdIDePvpVJUQ2dg3kLrsYVgolxTx2noL5 sog/oqz4IL9fJBM1nyglLa7MRO/EhEoJ1Jeh3J0ytt7mkrOCD8UwWdeQT1ipF6sruQqO 8HoSJW0OcaX4k2Oua8aRQVmQ9Ik3tWCM1ggl+QRNTG9Rtwm/d5rYvl1QKbtiP5tOw0xQ P6vJtMlVTlAXKOnPaESrHNQ3pWA9mdU82lW7FArwmXwdj4ZIm4g9EY9Pt7V/jGiyOJMK LzAC9eu83kinbzMQrsKvkoxtmr7rP0u1E5vHvuMSB5TpJW7X0GsZC5srFqvlKwVD38kZ gN8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679490164; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ViRmkYI1T/q9AleFDZxnzz9BK+OfJL44ohwRmj/wb8E=; b=t6HZC2rdvEKWa9nOf+xXC3WcWTSwn8v4qhhn6GEEZ+zZ58OZ2YfUt/LprydhxqUE5W llE4gtyC0/zcL3f6XXea+39l3DBr5UY3CE1HENqETAJk8ECMp9LVxv/2naUvNm0dHcNX UnmUZQAej+vtzynt67DP9t2kIhqN/XvsWfp5kCtHrJIojhAf0Ql34zgD+RMDD4L3J+hP W1Cy+VE21ujsYBaVbuUQzhauiB3r81LwGGjpNNJYmo0LQyGbaiMTJMrhSLgnGEZ34L13 nAIAoNowrYfRF6l3vO2A18gMm1TTugKd1/M3jhlTJqSMz77UL1FGOWrjkxJOVAWurAXD ciQQ== X-Gm-Message-State: AO0yUKVfq48r3whtapGGndL/fhGlvS3n1FqJ63CuH5iUB8xXNj3UBlpd yLoRWHKf8yBL8zWsevq6ODY= X-Google-Smtp-Source: AK7set8i0yrjEu/hxv3QebJCfgsiDSAkrFFCBeuESthSVvyA2rynV3aw+/+Dw2F3mR4N9qvZR/IeUQ== X-Received: by 2002:a17:90b:4d0d:b0:23a:87d1:9586 with SMTP id mw13-20020a17090b4d0d00b0023a87d19586mr3318684pjb.23.1679490164008; Wed, 22 Mar 2023 06:02:44 -0700 (PDT) Received: from hyeyoo ([210.205.188.148]) by smtp.gmail.com with ESMTPSA id w5-20020a1709029a8500b001a064cff3c5sm10502457plp.43.2023.03.22.06.02.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Mar 2023 06:02:42 -0700 (PDT) Date: Wed, 22 Mar 2023 22:02:28 +0900 From: Hyeonggon Yoo <42.hyeyoo@gmail.com> To: Binder Makin Cc: Vlastimil Babka , lsf-pc@lists.linux-foundation.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-block@vger.kernel.org, bpf@vger.kernel.org, linux-xfs@vger.kernel.org, David Rientjes , Christoph Lameter , Pekka Enberg , Joonsoo Kim , Roman Gushchin Subject: Re: [LSF/MM/BPF TOPIC] SLOB+SLAB allocators removal and future SLUB improvements Message-ID: References: <4b9fc9c6-b48c-198f-5f80-811a44737e5f@suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: B962D1C001E X-Stat-Signature: ifc1ua33qgcccef7ikxsbgkr6rg4ghwe X-HE-Tag: 1679490165-49701 X-HE-Meta: U2FsdGVkX1+zE+QvfmWpUscSexHekAX5qwBYcU0qemBUJWoG6Zs2xWGJiG1JAPhnE4vRudYchszw5lYfm0IXCz3L1iAJWI1mIBSw4CH5xJn5CnQQ0JdqW4ijAwzlm3nhttvOmZPMMMTRF0UxIqejS20UclrAOaEhP8rGz/fdRMlihrG9YPt2kiVIa3ENXRkBDP8Omkau7s6p+fH+LEhbCZL1UMPtIHoFAVku+BlJ9c6+D7QcV8Jn2tAqSoJ8Sfo8zUPedSzaNs1HGQYs1HDH1vmAn6cp/AvmXMMFqtcCfNE9o77b2+Ugh59A0jufCSBu7ZWpPyr8OHqJcBRnaYbNeJ/ktGA0p1Dk1MdQUW07Ck93gGToSnQyqQ9SXKAwGrv+aA5liW28UMs1AqFZ/RCatvMDwaV6WMCApWr8SVJpC2SUzn2ifCCyp77LI7IuTk998TnPjh9PQgxlqzqhqGPgzV9SDHfnljJapPmvMIUmflCaV/T4TPYIumwYUFsC8ygXX3G2AFnTgYKtvBzxcj5D2RFMHUWtwL8+95PytYg8+kTnvSpgYltcRbt+C318wr4YVEVqPu59tBbViJdZZ3PJ2LWiT3I/RLdbHSDdY1ZCXp5g53+AFfB1lIGyBWbf1KWFnfxwdis1Fux0Fj0OIhKszKOXKUuS1EjQZGyir0tnUKU8aXvL6Lo4CCY8+tRHI8UMm0Pg5HCzBE39X+T1qwHEbJRx08Fg2ibpbPocqT7eYaOjLXXBZ9d+dlHV3/MmTCYBWMAGP6iLSpcIQc7gR1F6YEVu2dhHyK+pNEsPngs8ESlQtDhQx+NNWLjPUm0f5ibL6QBLkBn0nmIv4AM9/I1hsyi8asunaiMbVinT/AnmLqb8MH/iLV7+QRoEOjkV6Moijn7fbiJIFIbeYVm9MSrsLN/EVy8vRO++/Xmo1jcPdZVooxNd8SxbBFUtvYteBN61ZD+JJEsmwNCs4HAgGFo biqKWWLC 6/VouGSSABH4M3sknQLRxlkImc/fsRjXmT7ex2VymsZcNj7wAvCMGksn1hSHedPx5u2jBk3aJlemM2/ECCHsV5jYYe0iHK1nk3LCYw1V3L1+HNATHoEZVBjh3orXOQAftkKrIAskR5Uis9DZAJIoeMaUVNDHeetAqslirFUQrvfiBuA9u7Qe6KM/K+3qRNa8j08oNjheomix77P4A01DSbGoWT2L1YuxbcocQwQKNXd0lqigFs7zYUV9jwFNmkk465tcm7bhJFocWb5mINUjK7oyfwrVMpf0FBlGP0/Sq25sVN1jtDO2PgteCx2X+goj/JvFdB1XZiRwxbnVTSvRi3Z6UIL41jYChVc0gIyN2Ix07LotVkVGjZ+FUxzEvPU8B2wRnPjllugx31HZIp59/DGsbqND2BrXqJvWIVgO1axdxmFazbYARYuua43p4kyFSH0fMDd7IiOwS09Ykh9gNNLw3ucRJRA9zhbV25xOdnd3oI0U/kaItUMsXogimyRWR0E99GCQgkuWqf3wnS+pVsz/mlyRHAI9ANtOqZs7o+hj3LyCFruDnGCUYAg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Mar 22, 2023 at 08:15:28AM -0400, Binder Makin wrote: > Was looking at SLAB removal and started by running A/B tests of SLAB vs > SLUB. Please note these are only preliminary results. > > These were run using 6.1.13 configured for SLAB/SLUB. > Machines were standard datacenter servers. > > Hackbench shows completion time, so smaller is better. > On all others larger is better. > https://docs.google.com/spreadsheets/d/e/2PACX-1vQ47Mekl8BOp3ekCefwL6wL8SQiv6Qvp5avkU2ssQSh41gntjivE-aKM4PkwzkC4N_s_MxUdcsokhhz/pubhtml > > Some notes: > SUnreclaim and SReclaimable shows unreclaimable and reclaimable memory. > Substantially higher with SLUB, but I believe that is to be expected. > > Various results showing a 5-10% degradation with SLUB. That feels > concerning to me, but I'm not sure what others' tolerance would be. Hello Binder, Thank you for sharing the data on which workloads SLUB performs worse than SLAB. This information is critical for improving SLUB and deprecating SLAB. By the way, it appears that the spreadsheet is currently set to private. Could you make it public for me to access? I am really interested in performing similar experiments on my machines to obtain comparable data that can be utilized to enhance SLUB. Thanks, Hyeonggon > redis results on AMD show some pretty bad degredations. 10-20% range > netpipe on Intel also has issues.. 10-17% > > On Tue, Mar 14, 2023 at 4:05 AM Vlastimil Babka wrote: > > > As you're probably aware, my plan is to get rid of SLOB and SLAB, leaving > > only SLUB going forward. The removal of SLOB seems to be going well, there > > were no objections to the deprecation and I've posted v1 of the removal > > itself [1] so it could be in -next soon. > > > > The immediate benefit of that is that we can allow kfree() (and > > kfree_rcu()) > > to free objects from kmem_cache_alloc() - something that IIRC at least xfs > > people wanted in the past, and SLOB was incompatible with that. > > > > For SLAB removal I haven't yet heard any objections (but also didn't > > deprecate it yet) but if there are any users due to particular workloads > > doing better with SLAB than SLUB, we can discuss why those would regress > > and > > what can be done about that in SLUB. > > > > Once we have just one slab allocator in the kernel, we can take a closer > > look at what the users are missing from it that forces them to create own > > allocators (e.g. BPF), and could be considered to be added as a generic > > implementation to SLUB. > > > > Thanks, > > Vlastimil > > > > [1] https://lore.kernel.org/all/20230310103210.22372-1-vbabka@suse.cz/