From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8F288C77B62 for ; Tue, 4 Apr 2023 16:03:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1E5DB6B0072; Tue, 4 Apr 2023 12:03:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 195C46B0074; Tue, 4 Apr 2023 12:03:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0851E6B0075; Tue, 4 Apr 2023 12:03:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id EA97E6B0072 for ; Tue, 4 Apr 2023 12:03:13 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 8AF56A09C4 for ; Tue, 4 Apr 2023 16:03:13 +0000 (UTC) X-FDA: 80644177866.10.53F2A98 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by imf15.hostedemail.com (Postfix) with ESMTP id C079CA0020 for ; Tue, 4 Apr 2023 16:03:09 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=0H4UafI3; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=NaRIieVz; spf=pass (imf15.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.29 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680624190; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BuCGsnYMiMa4eeDwQP7y5gx95Bxfi3LY8COha+SD3xw=; b=FiMgjOHFEf4gNkMvUGJ8jKkKnguu5dwO9awxAXebg9Ew/uwz3b4qHIkhB4OD/ZbaP/iiU6 DrPyKZIFspaM/SQkDFcF9yWURIgChrygEnHV1k2Ow2LwVpz52NvVe49PKQjdKaUSexcVWe WPMZqz0hgAiiPLB1YGQP2QYdMx5uePo= ARC-Authentication-Results: i=1; imf15.hostedemail.com; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=0H4UafI3; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=NaRIieVz; spf=pass (imf15.hostedemail.com: domain of vbabka@suse.cz designates 195.135.220.29 as permitted sender) smtp.mailfrom=vbabka@suse.cz; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680624190; a=rsa-sha256; cv=none; b=YnwR9/kXgZoEsqS/C78Xw4BKvx1el+muyNMixO7hgxk+U9pAyIZCvu+joXcGaJo4kOti7X h51LMzEdpJqCiJqQXMTK/ei7+JDkIYf3TwQbt4/z6qrYGYVi1OUjVdCxv5HUTXXrHTKWlf 6PY2i0sxQtgpx287uUu64Q3zNd8XCN4= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 4A50D20258; Tue, 4 Apr 2023 16:03:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1680624188; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BuCGsnYMiMa4eeDwQP7y5gx95Bxfi3LY8COha+SD3xw=; b=0H4UafI3sTZDD0fneaXzmmqJL8Hd/BQ3DOq2WpWAQ7CqD+OpS2cyaSZreiZ1ahbk5DbTEt vWcqZhp3TOo09KDe/1DGZvWihR0PGeEiHt7KLd4vRKwnoy6ArjbZFhzIIB2bFQXlnaKiXq 1efxl2DGlkD7AbYBdwBcQTAK+W9wfBI= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1680624188; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BuCGsnYMiMa4eeDwQP7y5gx95Bxfi3LY8COha+SD3xw=; b=NaRIieVzI5/NC18H0JkKim/Ia/h2tLZHNBp0WFaN6z3RAek6dfYtHu8XUa6xmfgPXwsTU7 CK144MupDkdFcdBA== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 1963B1391A; Tue, 4 Apr 2023 16:03:08 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id arpDBTxKLGQfGQAAMHmgww (envelope-from ); Tue, 04 Apr 2023 16:03:08 +0000 Message-ID: <951d364a-05c0-b290-8abe-7cbfcaeb2df7@suse.cz> Date: Tue, 4 Apr 2023 18:03:07 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.9.0 Subject: Re: [LSF/MM/BPF TOPIC] SLOB+SLAB allocators removal and future SLUB improvements Content-Language: en-US To: Binder Makin Cc: lsf-pc@lists.linux-foundation.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-block@vger.kernel.org, bpf@vger.kernel.org, linux-xfs@vger.kernel.org, David Rientjes , Christoph Lameter , Pekka Enberg , Joonsoo Kim , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Roman Gushchin References: <4b9fc9c6-b48c-198f-5f80-811a44737e5f@suse.cz> From: Vlastimil Babka In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: C079CA0020 X-Stat-Signature: madnz4ggysuj5a4b7qgzkbd1ne8dpemx X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1680624189-240837 X-HE-Meta: U2FsdGVkX19bD1vu/Ua/tX6xmnVHTRsN1MBinUtMtQm6DW7J6P/XCDk+jiCet4tPALCqkRlVavV88W0MVwAXBhYjqpTt8WhfTVWcsjT2vQQVGtK6aba3CdWRXcOFAUqzEyWrhtUNKIooSUocgI8ve0eCEcJmH2BQO/OjZQYy1fqxtjiV/oXWT3YE6uYKxdSykAmcZ9KR0jAKTKFYhsPyl4FPUv8/ecyzSHmG3Wml9u8hH1hOC9ZRyCNxmlZOJAQz+13qukVDTd04wRMB6YDOC8+KBme88cJl52uEs5nShsxbMMB1JfTUkrkPSqC1vKlQo3PYmMOEU+S6IN8whvngLXt1eSNMHXe9By8UctIABy6vrdJuq3A4nLoJB7TG4nTOD1mkp62l81N+sB9P3PlVYZR5aku3FwSOLuwN1Lx3TqYcV2eSYYeNRfZrhE1nbhGcDp/rA+C0PQqif58jSOIMTId2nrL4zD/ctZQRirA2aa3I1UH8UdwkhAt47r7OFJpXXNulHJ433W6tt4emth8ZjHykbvq+PmA7UbnZE1KC/8VkV/bX+ZAaHY0YdbfimftwZHBauq2ARQeFzzM/825Dx0JwqAIja4HvC6mMojlxKjfhZszfniv5A7IChAOqvact8ELxTSESg7iKocYvpL8L69NkV5zzwvBs7xSmUHCXHRBXV9lh/BZquvyw6cyadeFKSAIsZVPwcd53Zzkpk2uwlJU5nsGLGgjggj9mbDRU3iJC+jcvJ4xS8SQ8mTcfWYi6mNmCC/vhqqQWJm5Ebz+F9oZcDPtf6JkyliOk7vnXKuQE0hlW6t7ejbOwlI6pzCRvLIJhmkJxMzuhQApaV0qOTOGGqYQU2+NPP7LoX+lP2bth0GvY9K5itmPoVSnBMxo/rGlkjXL1bEiuTyrGYmZ3Qr2MmNvvVU77+xLe063Jg2Lrne92U2dMAyjuDI0keUAeyDkzbtWoC3uNlZsjhWZ pS5KeF71 evrwMGEmpnGkjpohWoPJVXnwWHUdEdHpvLtA8fx1iYjpX+zUTqY7Z0Tm0rVlTF/R6/j6Gj8bkR5dESH0Ehhne8veT21k6EWodzvGoOiCFGeKcItcgawa/t2FEQ+hNlIZnuJl5fCnA8sCd4firvMr/o+u9I7N5KqiTpdMMKuWto5UXpBwCkkIS1laEHZSKJLQdCjNZxVK4QawMUOtbVKyvh0MnEgHpMABeecxvvIB0uBkWLN+qD0ASBX0596j7FSYfcaibYUnUcGPeKl3G4PoeORI+mklrLweQjCRIQetjNxTkgY8ebW2eZc8JT65KxVM53kW9MOa3ex0QPEQcIa4N3YVEO4NHNqgPRhZ6UxYZF85+FizhCMAJjj+9Dw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 3/22/23 13:30, Binder Makin wrote: > Was looking at SLAB removal and started by running A/B tests of SLAB > vs SLUB. Please note these are only preliminary results. Thanks, that's very useful. > These were run using 6.1.13 configured for SLAB/SLUB. > Machines were standard datacenter servers. > > Hackbench shows completion time, so smaller is better. > On all others larger is better. > https://docs.google.com/spreadsheets/d/e/2PACX-1vQ47Mekl8BOp3ekCefwL6wL8SQiv6Qvp5avkU2ssQSh41gntjivE-aKM4PkwzkC4N_s_MxUdcsokhhz/pubhtml > > Some notes: > SUnreclaim and SReclaimable shows unreclaimable and reclaimable memory. > Substantially higher with SLUB, but I believe that is to be expected. > > Various results showing a 5-10% degradation with SLUB. That feels > concerning to me, but I'm not sure what others' tolerance would be. > > redis results on AMD show some pretty bad degredations. 10-20% range > netpipe on Intel also has issues.. 10-17% I guess one question is which ones are genuine SLAB/SLUB differences and not e.g. some artifact of different cache layout or something. For example it seems suspicious if results are widely different between architectures. E.g. will-it-scale writeseek3_scalability regresses on arm64 and amd, but improves on intel? Or is something wrong with the data, all columns for that whole benchmark suite are identical. hackbench ("smaller is better") seems drastically better on arm64 (30% median time reduction?) and amd (80% reduction?!?), but 10% slower intel? redis seems a bit improved on arm64, slightly worse on intel but much worse on amd. specjbb similar story, also I thought it was a java focused benchmark, should it really be exercising kernel slab allocators in such notable way? I guess netpipe is the least surprising as networking was always mentioned in SLAB vs SLUB discussions. > On Tue, Mar 14, 2023 at 4:05 AM Vlastimil Babka wrote: >> >> As you're probably aware, my plan is to get rid of SLOB and SLAB, leaving >> only SLUB going forward. The removal of SLOB seems to be going well, there >> were no objections to the deprecation and I've posted v1 of the removal >> itself [1] so it could be in -next soon. >> >> The immediate benefit of that is that we can allow kfree() (and kfree_rcu()) >> to free objects from kmem_cache_alloc() - something that IIRC at least xfs >> people wanted in the past, and SLOB was incompatible with that. >> >> For SLAB removal I haven't yet heard any objections (but also didn't >> deprecate it yet) but if there are any users due to particular workloads >> doing better with SLAB than SLUB, we can discuss why those would regress and >> what can be done about that in SLUB. >> >> Once we have just one slab allocator in the kernel, we can take a closer >> look at what the users are missing from it that forces them to create own >> allocators (e.g. BPF), and could be considered to be added as a generic >> implementation to SLUB. >> >> Thanks, >> Vlastimil >> >> [1] https://lore.kernel.org/all/20230310103210.22372-1-vbabka@suse.cz/ >>