From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A37CC77B7C for ; Fri, 5 May 2023 19:44:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 46CB16B007D; Fri, 5 May 2023 15:44:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 41D506B007E; Fri, 5 May 2023 15:44:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2EDDE6B0080; Fri, 5 May 2023 15:44:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from mail-yb1-f180.google.com (mail-yb1-f180.google.com [209.85.219.180]) by kanga.kvack.org (Postfix) with ESMTP id 041E96B007D for ; Fri, 5 May 2023 15:44:12 -0400 (EDT) Received: by mail-yb1-f180.google.com with SMTP id 3f1490d57ef6-b9a7e639656so3345918276.0 for ; Fri, 05 May 2023 12:44:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1683315851; x=1685907851; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=RvDT8zkuC4gVi0fnTc8i/Bkqd4YLxVEG0sioBwkCD+U=; b=CIPVHcOLQdhTXogN62Mw/fCOLjlOwNM45thZp55/G0Y+4E4NexnUvB3vOtM2suLjdz T1rh3EKQF9Q7DLBvkrQA96HMuJKK6ljz6sUoadVwmvME6+Lo2jBkjnaE4Kk2rVIFv44B GTEHr7cll/pqo289odi2h8zMjdKiN7hEFK7aVA5CoC+HI1oO5ICufh8JDIXrvRDAevnP hUDV1hbROSBKB8pVoWmLqfwYKVjSxXq1DDuUre5iez9QOh1dhd88hO/po3v9f9cizadN 32jf30WJaDhnMzH+nura2GGWCByQoFHbfZBMyldKQhm6BHH1vM5Us1DoF2dmrL50knMa moaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683315851; x=1685907851; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RvDT8zkuC4gVi0fnTc8i/Bkqd4YLxVEG0sioBwkCD+U=; b=C9ncQs3KeDDr05g4G9qcmU813iNDoyW2xeu1x3gHFW5GwBBaO/6pzw62QUIDmFFKqE 6JdCYmPi9VzNfAT1o9qRswYawqcvI0WVvj33Q707sd8KQWMBCiyAZcYui6BpVeabG1xP if9VByrVwqXjUhbOCxhQrw94D99lnrKEPcVYCI+Sk4T+EYGrMctn12Gd+5bFbq8t9PJL XOVU1MeOc1KqB60nOQ70u4+6h6iNoFVIf/sSYCsgHirn+8agRHHdSL13r2sEQ3sxrXkN TNJSE4a7URe5XpYeXiVebbCZva9Y9tCvNf0dO4w92artisId9C+QWamQCn+7R5MliJhA CnUg== X-Gm-Message-State: AC+VfDz9jEgVn5g8Ach/w6NkzGgFF4zO6s1ZkhNZ32n1v51c70nRUzVj j2EZW0d0L9Jl4WxRM4TaBz7TeNJCc46hN1AWfZKiTg== X-Google-Smtp-Source: ACHHUZ47iup2wUFCWjaUVQFxudnhr3YZeYNUuAbCUUGTy47k/MsRgf5289v6Uo7/8hYFN/Owo0us95zjRkpUK8gmZes= X-Received: by 2002:a05:6902:100e:b0:b9e:5008:1773 with SMTP id w14-20020a056902100e00b00b9e50081773mr3418662ybt.15.1683315851287; Fri, 05 May 2023 12:44:11 -0700 (PDT) MIME-Version: 1.0 References: <4b9fc9c6-b48c-198f-5f80-811a44737e5f@suse.cz> <951d364a-05c0-b290-8abe-7cbfcaeb2df7@suse.cz> <19acbdbb-fc2f-e198-3d31-850ef53f544e@suse.cz> In-Reply-To: <19acbdbb-fc2f-e198-3d31-850ef53f544e@suse.cz> From: Binder Makin Date: Fri, 5 May 2023 15:44:00 -0400 Message-ID: Subject: Re: [LSF/MM/BPF TOPIC] SLOB+SLAB allocators removal and future SLUB improvements To: Vlastimil Babka Cc: lsf-pc@lists.linux-foundation.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, linux-block@vger.kernel.org, bpf@vger.kernel.org, linux-xfs@vger.kernel.org, David Rientjes , Christoph Lameter , Pekka Enberg , Joonsoo Kim , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Roman Gushchin Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Here are the results of my research. One doc is an overview fo the data and the other is a pdf of the raw data. https://drive.google.com/file/d/1DE8QMri1Rsr7L27fORHFCmwgrMtdfPfu/view?usp= =3Dshare_link https://drive.google.com/file/d/1UwnTeqsKB0jgpnZodJ0_cM2bOHx5aR_v/view?usp= =3Dshare_link On Thu, Apr 27, 2023 at 4:29=E2=80=AFAM Vlastimil Babka wr= ote: > > On 4/5/23 21:54, Binder Makin wrote: > > I'm still running tests to explore some of these questions. > > The machines I am using are roughly as follows. > > > > Intel dual socket 56 total cores > > 192-384GB ram > > LEVEL1_ICACHE_SIZE 32768 > > LEVEL1_DCACHE_SIZE 32768 > > LEVEL2_CACHE_SIZE 1048576 > > LEVEL3_CACHE_SIZE 40370176 > > > > Amd dual socket 128 total cores > > 1TB ram > > LEVEL1_ICACHE_SIZE 32768 > > LEVEL1_DCACHE_SIZE 32768 > > LEVEL2_CACHE_SIZE 524288 > > LEVEL3_CACHE_SIZE 268435456 > > > > Arm single socket 64 total cores > > 256GB rma > > LEVEL1_ICACHE_SIZE 65536 > > LEVEL1_DCACHE_SIZE 65536 > > LEVEL2_CACHE_SIZE 1048576 > > LEVEL3_CACHE_SIZE 33554432 > > So with "some artifact of different cache layout" I didn't mean the > different cache sizes of the processors, but possible differences how > objects end up placed in memory by SLAB vs SLUB causing them to collide i= n > the cache of cause false sharing less or more. This kind of interference = can > make interpreting (micro)benchmark results hard. > > Anyway, how I'd hope to approach this topic would be that SLAB removal is > proposed, and anyone who opposes that because they can't switch from SLAB= to > SLUB would describe why they can't. I'd hope the "why" to be based on > testing with actual workloads, not just benchmarks. Benchmarks are then o= f > course useful if they can indeed distill the reason why the actual worklo= ad > regresses, as then anyone can reproduce that locally and develop/test fix= es > etc. My hope is that if some kind of regression is found (e.g. due to lac= k > of percpu array in SLUB), it can be dealt with by improving SLUB. > > Historically I recall that we (SUSE) objected somwhat to SLAB removal as = our > distro kernels were using it, but we have switched since. Then networking > had concerns (possibly related to the lack percpu array) but seems bulk > allocations helped and they use SLUB these days [1]. And IIRC Google was > also sticking to SLAB, which led to some attempts to augment SLUB for tho= se > workloads years ago, but those were never finished. So I'd be curious if = we > should restart those effors or can just remove SLAB now. > > [1] https://lore.kernel.org/all/93665604-5420-be5d-2104-17850288b955@redh= at.com/ > >