From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4224EC433DB for ; Fri, 22 Jan 2021 13:10:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id BDAE0206A4 for ; Fri, 22 Jan 2021 13:10:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BDAE0206A4 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3DF476B000A; Fri, 22 Jan 2021 08:10:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 38E316B000C; Fri, 22 Jan 2021 08:10:03 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 255D56B000D; Fri, 22 Jan 2021 08:10:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0242.hostedemail.com [216.40.44.242]) by kanga.kvack.org (Postfix) with ESMTP id 0C6586B000A for ; Fri, 22 Jan 2021 08:10:03 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id B696F613D for ; Fri, 22 Jan 2021 13:10:02 +0000 (UTC) X-FDA: 77733443844.08.bear88_0d0f4072756c Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin08.hostedemail.com (Postfix) with ESMTP id 960181819E76F for ; Fri, 22 Jan 2021 13:10:02 +0000 (UTC) X-HE-Tag: bear88_0d0f4072756c X-Filterd-Recvd-Size: 5297 Received: from mail-lf1-f49.google.com (mail-lf1-f49.google.com [209.85.167.49]) by imf26.hostedemail.com (Postfix) with ESMTP for ; Fri, 22 Jan 2021 13:10:01 +0000 (UTC) Received: by mail-lf1-f49.google.com with SMTP id v24so7448118lfr.7 for ; Fri, 22 Jan 2021 05:10:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=jM6zpCES+0HLn8XlUNi032+O57IMQ4RlIhY5sPMlY+Q=; b=sF5zAjDT56BXiHUey17OUSfQJ66O9gcNB/c6sn5CjzVA/TG8uudCB1ttxC7JwOpY3K qIB1A5e7fuCatdIf+5BL5DYq0UuC+ftUgAIEvi3YC5xTK9SBTvWSgrFOBn0lRFms7Xws 9YSdzTRVxlR3bmUlTGBzpMETm64mfRVCQkQ2LEL/+zTbIQSLC1MxJyX0y0/Mku+Uwr+g Po0AbrD406ZbDdueebjDmUt49f7PZoFwWFK/6oIhfRGB6J3GePuH//HPXFcypYtsnNGg teRWxa9ynMonmK9sWmd+YcmJci32W7kQOXEDs5ju0ygcEo5dtXPSYY9zqvnhGp+PNLCC bk/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=jM6zpCES+0HLn8XlUNi032+O57IMQ4RlIhY5sPMlY+Q=; b=e6bRnhUCpLqiifsp1z9TEFaFlySFfn0yl1VscqjRM17fhieLM1xHusnSJlmP7gE9A8 LJqBLcVtkgOF37G2Z16kXr4i0v0dyb5zAQREhoniyDRaJkS7BG9d67P+dYXu4iA9hmDf dLCjSXboCk027c8Kq5Jt9VTI4c2JzEOaVDbYkH2gkUDbYF8QWq0YXB12IZVMcBGNiF0y AM2DeMloj6MdyOZ0zPdMNuJhTFTx7+1R1Urih0aMtfrVsJuJmtr0QX407dS+bODfStuE YcuPeT3B1T33oMKIHAdkWn6E6JSXgcUs6wXuUkAxpPgeiwTKrwYmGpCNgbFZISzMoyzt dYmw== X-Gm-Message-State: AOAM533td57KUKggf5v/K7f4HR9DDTi3CpSp2gvTuPoNQlnugnRkuiR+ QzP/Z61lfQYE25eMdwD8l7sBcWTMtJseeIIdydK6eQ== X-Google-Smtp-Source: ABdhPJxHLxFQGKxSHSjonKXZfD+TP3Iizcnks53UjMPVH/itFaqhB5xo0zcrd/h+EYj3WPQIojePEaOg0ILdmqpfLcI= X-Received: by 2002:a05:6512:38c1:: with SMTP id p1mr2197526lft.193.1611321000384; Fri, 22 Jan 2021 05:10:00 -0800 (PST) MIME-Version: 1.0 References: <20201118082759.1413056-1-bharata@linux.ibm.com> <20210121053003.GB2587010@in.ibm.com> In-Reply-To: From: Jann Horn Date: Fri, 22 Jan 2021 14:09:33 +0100 Message-ID: Subject: Re: [RFC PATCH v0] mm/slub: Let number of online CPUs determine the slub page order To: Vlastimil Babka Cc: Christoph Lameter , Bharata B Rao , Vincent Guittot , linux-kernel , Linux-MM , David Rientjes , Joonsoo Kim , Andrew Morton , Roman Gushchin , Shakeel Butt , Johannes Weiner , aneesh.kumar@linux.ibm.com, Michal Hocko Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Jan 22, 2021 at 2:05 PM Jann Horn wrote: > On Thu, Jan 21, 2021 at 7:19 PM Vlastimil Babka wrote: > > On 1/21/21 11:01 AM, Christoph Lameter wrote: > > > On Thu, 21 Jan 2021, Bharata B Rao wrote: > > > > > >> > The problem is that calculate_order() is called a number of times > > >> > before secondaries CPUs are booted and it returns 1 instead of 224. > > >> > This makes the use of num_online_cpus() irrelevant for those cases > > >> > > > >> > After adding in my command line "slub_min_objects=36" which equals to > > >> > 4 * (fls(num_online_cpus()) + 1) with a correct num_online_cpus == 224 > > >> > , the regression diseapears: > > >> > > > >> > 9 iterations of hackbench -l 16000 -g 16: 3.201sec (+/- 0.90%) > > > > I'm surprised that hackbench is that sensitive to slab performance, anyway. It's > > supposed to be a scheduler benchmark? What exactly is going on? > > Uuuh, I think powerpc doesn't have cmpxchg_double? > > "vgrep cmpxchg_double arch/" just spits out arm64, s390 and x86? And > > says under "POWERPC": "no DW LL/SC" > > So powerpc is probably hitting the page-bitlock-based implementation > all the time for stuff like __slub_free()? Do you have detailed > profiling results from "perf top" or something like that? > > (I actually have some WIP patches and a design document for getting > rid of cmpxchg_double in struct page that I hacked together in the > last couple days; I'm currently in the process of sending them over to > some other folks in the company who hopefully have cycles to > review/polish/benchmark them so that they can be upstreamed, assuming > that those folks think they're important enough. I don't have the > cycles for it...) (The stuff I have in mind will only work on 64-bit though. We are talking about PPC64 here, right?)