From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5530C433E0 for ; Tue, 26 Jan 2021 13:38:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8BEFC2255F for ; Tue, 26 Jan 2021 13:38:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8BEFC2255F Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id CAE438D00CD; Tue, 26 Jan 2021 08:38:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C84A08D00B0; Tue, 26 Jan 2021 08:38:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B727C8D00CD; Tue, 26 Jan 2021 08:38:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0098.hostedemail.com [216.40.44.98]) by kanga.kvack.org (Postfix) with ESMTP id A07AF8D00B0 for ; Tue, 26 Jan 2021 08:38:28 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 6EFDB8249980 for ; Tue, 26 Jan 2021 13:38:28 +0000 (UTC) X-FDA: 77748030696.05.flag57_3e028a02758e Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin05.hostedemail.com (Postfix) with ESMTP id 4F6661801708D for ; Tue, 26 Jan 2021 13:38:28 +0000 (UTC) X-HE-Tag: flag57_3e028a02758e X-Filterd-Recvd-Size: 5712 Received: from mail-lj1-f172.google.com (mail-lj1-f172.google.com [209.85.208.172]) by imf11.hostedemail.com (Postfix) with ESMTP for ; Tue, 26 Jan 2021 13:38:27 +0000 (UTC) Received: by mail-lj1-f172.google.com with SMTP id s18so6430344ljg.7 for ; Tue, 26 Jan 2021 05:38:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=oZ1J3xOVFtmJXTfZDgdAErA6KmvVsDRz9fNO4jgxHtI=; b=SMgFuNiKEiPhgaDPUI1/GdrArkuGqBjLB3McvDo7jbt34eJ2jgnjX6f9mSvR28tX6E S3n2pwarMwadMM76RXTY7ocxNTiwpwU+6q90aq30CIKjLApsgEURWwrHy39IMez6H+0t 8YsGmK3jPcbZbzVR5hcoqYvngNAyZTiNRI0T6nWv1kqYd0yr4l2JVIjvL2h7nq6P2WHG ak371TlfXR3u3bVbzRGSsYrgkqdz5YahDQzHUSd9R7N9p2sAAuxeEt6Z5D8GuH4jwi61 a4VhTxiXpiFaM9QNf3I0Uo5zp1UEBkIz0NW0JMu68F9Qw03rlXsDHjsmMYDioG8qjSAm c8ig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=oZ1J3xOVFtmJXTfZDgdAErA6KmvVsDRz9fNO4jgxHtI=; b=qyJ4ZAImtVrGLeyVeMM6NqjZeeMJd9JKjOPaw/pFQCB30glJBYsq10ztiOqDYpebwG jIDKwQ3a7iNPFATKvCDCuw6l7VHhfp9gHIMBxiUpnWUtSCC3QAr5qsKTfc48ujwdFBNf 0Ei3c7HtlK7tKhoXSmXFH3fAM9OQyUk2PCAixgeWBioIrQVouc+KmkpCaDeUrVpzsJi+ ubAlL0CxoKyQ+YfFd5txkPaSVZEUCLRD3cERbe9L1gnqBfAo9KXhuNAI2mVHNS3UxrTC OrlNZu4QJI1Mkms6mHCfS8Zc7NNu6i5+QNXhXCYnZGJI/YErXrF6WJI3F/oPI5cBWrDs yejw== X-Gm-Message-State: AOAM530SxLT7GyEx6ATKw/lm+qB3+4+eQgIqW1UciMfaJHxGCttkfdIe TOzXyqRrqxIfkPqKfIuvR72oOa7jqAB61lZR5Elr0g== X-Google-Smtp-Source: ABdhPJwBzZJw5LXtXQwYj+NiFhtZ3u2pojQ4UVxtFwwJkpquZoeXf8pRlfAY2qhHOM3HnqUJjHZI3HjQdaPTmeame2Y= X-Received: by 2002:a2e:5408:: with SMTP id i8mr2910097ljb.221.1611668306093; Tue, 26 Jan 2021 05:38:26 -0800 (PST) MIME-Version: 1.0 References: <20201118082759.1413056-1-bharata@linux.ibm.com> <20210121053003.GB2587010@in.ibm.com> <20210126085243.GE827@dhcp22.suse.cz> In-Reply-To: <20210126085243.GE827@dhcp22.suse.cz> From: Vincent Guittot Date: Tue, 26 Jan 2021 14:38:14 +0100 Message-ID: Subject: Re: [RFC PATCH v0] mm/slub: Let number of online CPUs determine the slub page order To: Michal Hocko Cc: Vlastimil Babka , Christoph Lameter , Bharata B Rao , linux-kernel , linux-mm@kvack.org, David Rientjes , Joonsoo Kim , Andrew Morton , guro@fb.com, Shakeel Butt , Johannes Weiner , aneesh.kumar@linux.ibm.com, Jann Horn Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, 26 Jan 2021 at 09:52, Michal Hocko wrote: > > On Thu 21-01-21 19:19:21, Vlastimil Babka wrote: > [...] > > We could also start questioning the very assumption that number of cpus should > > affect slab page size in the first place. Should it? After all, each CPU will > > have one or more slab pages privately cached, as we discuss in the other > > thread... So why make the slab pages also larger? > > I do agree. What is the acutal justification for this scaling? > /* > * Attempt to find best configuration for a slab. This > * works by first attempting to generate a layout with > * the best configuration and backing off gradually. > * > * First we increase the acceptable waste in a slab. Then > * we reduce the minimum objects required in a slab. > */ > > doesn't speak about CPUs. 9b2cd506e5f2 ("slub: Calculate min_objects > based on number of processors.") does talk about hackbench "This has > been shown to address the performance issues in hackbench on 16p etc." > but it doesn't give any more details to tell actually _why_ that works. > > This thread shows that this is still somehow related to performance but > the real reason is not clear. I believe we should be focusing on the > actual reasons for the performance impact than playing with some fancy > math and tuning for a benchmark on a particular machine which doesn't > work for others due to subtle initialization timing issues. > > Fundamentally why should higher number of CPUs imply the size of slab in > the first place? A 1st answer is that the activity and the number of threads involved scales with the number of CPUs. Regarding the hackbench benchmark as an example, the number of group/threads raise to a higher level on the server than on the small system which doesn't seem unreasonable. On 8 CPUs, I run hackbench with up to 16 groups which means 16*40 threads. But I raise up to 256 groups, which means 256*40 threads, on the 224 CPUs system. In fact, hackbench -g 1 (with 1 group) doesn't regress on the 224 CPUs system. The next test with 4 groups starts to regress by -7%. But the next one: hackbench -g 16 regresses by 187% (duration is almost 3 times longer). It seems reasonable to assume that the number of running threads and resources scale with the number of CPUs because we want to run more stuff. > -- > Michal Hocko > SUSE Labs