From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D32FFC433DB for ; Fri, 22 Jan 2021 13:06:17 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7C8DB23437 for ; Fri, 22 Jan 2021 13:06:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7C8DB23437 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id F083A6B0008; Fri, 22 Jan 2021 08:06:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EB8076B000A; Fri, 22 Jan 2021 08:06:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DA8FD6B000C; Fri, 22 Jan 2021 08:06:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0104.hostedemail.com [216.40.44.104]) by kanga.kvack.org (Postfix) with ESMTP id C5E2E6B0008 for ; Fri, 22 Jan 2021 08:06:16 -0500 (EST) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 88A0F181AF5F7 for ; Fri, 22 Jan 2021 13:06:16 +0000 (UTC) X-FDA: 77733434352.24.horse37_03179bd2756c Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin24.hostedemail.com (Postfix) with ESMTP id 6C0651A4A7 for ; Fri, 22 Jan 2021 13:06:16 +0000 (UTC) X-HE-Tag: horse37_03179bd2756c X-Filterd-Recvd-Size: 4963 Received: from mail-lf1-f51.google.com (mail-lf1-f51.google.com [209.85.167.51]) by imf48.hostedemail.com (Postfix) with ESMTP for ; Fri, 22 Jan 2021 13:06:15 +0000 (UTC) Received: by mail-lf1-f51.google.com with SMTP id o13so7451643lfr.3 for ; Fri, 22 Jan 2021 05:06:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=hdjcX1r0hcP0ziPjwGYoacie9+kzbK27+8zHrMy8Ek0=; b=uy7LbRH2a+1REAcuKTsefeoWzLSAk8lD9cg8t0qFzoBoRZ8P9cscIE2Ez8S826Ssbj oMgNkvLij+BJWNVphXcP2grz7sz5MmhvWX8s8QapQ7Zp7THp4qoDs9RX7XZJsf2PcsY4 uW508nwG14h2e9L+j5tziHYVrWo/G3+6vzgcxNylnWYTLG5kK/XaXj1ogJam7olKIa6w hJh/17mcPKTgLSeAog8TjZgFNuH1x1j0SVpt2z5DnL94M6kimZfip4lMPLYWsWKr+ZtP OhaJjeHvl0Ka0pUYtjLcUAAZ00B+wAXwK28VR9FJ+prxtNRVwZLA/fbeaJWV4sBp6Jup LlNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=hdjcX1r0hcP0ziPjwGYoacie9+kzbK27+8zHrMy8Ek0=; b=lTE8rSiamfIXhwYh+0aFCV8KkPnTz5eSXU7SKXECb//UhYSJvHf+RPc776VYPhgOU+ fGoeerSFX/uwztivtqbpCBGjIERLc5xeFuYfVEv43HPwE2FMcO59w8JYLEje+/UDnWvf RBMAuf3aGq9KJfP8FyiPH4hNSx0XVDuWIBCn6ocqkp7nKiKAIcSDJlW4IZcFx9d/skvI H8DzMH5kqyKFY/npQjDdJZW4G5MrlUE3n1VYBapqT/VvI7S1qFxhwrZGg83AwqH+l5Ws RJ9RbFBn+VDY+TtUXQBcn+4/iruT8SGDUUSRDSgVguXiPAGQNF3xHlmfixOisqWtjK4y veQg== X-Gm-Message-State: AOAM530+oqE0c3I3/GJz3h+otAzC5EdS6t+h11ClfIrtkdtwT65OVDMx GZkitTuJTgqqm5qufwAvGH25YSs01Y3R5PW2LgB78g== X-Google-Smtp-Source: ABdhPJxdr9C9WIbRzCa7KUwKZRMgZQMYRCaAHXUCYpMxlePER4o0QdbU0TKuP4lquVxmc2VkzNuMH6ETuRC6l6GBGLY= X-Received: by 2002:a19:197:: with SMTP id 145mr123836lfb.352.1611320774090; Fri, 22 Jan 2021 05:06:14 -0800 (PST) MIME-Version: 1.0 References: <20201118082759.1413056-1-bharata@linux.ibm.com> <20210121053003.GB2587010@in.ibm.com> In-Reply-To: From: Jann Horn Date: Fri, 22 Jan 2021 14:05:47 +0100 Message-ID: Subject: Re: [RFC PATCH v0] mm/slub: Let number of online CPUs determine the slub page order To: Vlastimil Babka Cc: Christoph Lameter , Bharata B Rao , Vincent Guittot , linux-kernel , Linux-MM , David Rientjes , Joonsoo Kim , Andrew Morton , Roman Gushchin , Shakeel Butt , Johannes Weiner , aneesh.kumar@linux.ibm.com, Michal Hocko Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Jan 21, 2021 at 7:19 PM Vlastimil Babka wrote: > On 1/21/21 11:01 AM, Christoph Lameter wrote: > > On Thu, 21 Jan 2021, Bharata B Rao wrote: > > > >> > The problem is that calculate_order() is called a number of times > >> > before secondaries CPUs are booted and it returns 1 instead of 224. > >> > This makes the use of num_online_cpus() irrelevant for those cases > >> > > >> > After adding in my command line "slub_min_objects=36" which equals to > >> > 4 * (fls(num_online_cpus()) + 1) with a correct num_online_cpus == 224 > >> > , the regression diseapears: > >> > > >> > 9 iterations of hackbench -l 16000 -g 16: 3.201sec (+/- 0.90%) > > I'm surprised that hackbench is that sensitive to slab performance, anyway. It's > supposed to be a scheduler benchmark? What exactly is going on? Uuuh, I think powerpc doesn't have cmpxchg_double? "vgrep cmpxchg_double arch/" just spits out arm64, s390 and x86? And says under "POWERPC": "no DW LL/SC" So powerpc is probably hitting the page-bitlock-based implementation all the time for stuff like __slub_free()? Do you have detailed profiling results from "perf top" or something like that? (I actually have some WIP patches and a design document for getting rid of cmpxchg_double in struct page that I hacked together in the last couple days; I'm currently in the process of sending them over to some other folks in the company who hopefully have cycles to review/polish/benchmark them so that they can be upstreamed, assuming that those folks think they're important enough. I don't have the cycles for it...)