From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B38EC433E6 for ; Tue, 19 Jan 2021 18:37:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E19FA20776 for ; Tue, 19 Jan 2021 18:37:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E19FA20776 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7D1BA6B0006; Tue, 19 Jan 2021 11:39:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 782836B0007; Tue, 19 Jan 2021 11:39:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 671BC6B0008; Tue, 19 Jan 2021 11:39:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0052.hostedemail.com [216.40.44.52]) by kanga.kvack.org (Postfix) with ESMTP id 4E8D86B0006 for ; Tue, 19 Jan 2021 11:39:45 -0500 (EST) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 149CB1EE6 for ; Tue, 19 Jan 2021 16:39:45 +0000 (UTC) X-FDA: 77723085930.03.shape74_330751527553 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin03.hostedemail.com (Postfix) with ESMTP id C0A5428A4E8 for ; Tue, 19 Jan 2021 16:39:44 +0000 (UTC) X-HE-Tag: shape74_330751527553 X-Filterd-Recvd-Size: 5484 Received: from mail-lf1-f48.google.com (mail-lf1-f48.google.com [209.85.167.48]) by imf42.hostedemail.com (Postfix) with ESMTP for ; Tue, 19 Jan 2021 16:39:42 +0000 (UTC) Received: by mail-lf1-f48.google.com with SMTP id s26so29915933lfc.8 for ; Tue, 19 Jan 2021 08:39:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=+5KQrnHXiI7OqCuos/sE3GNhfpeVWGLcvYlJ3q/gLNQ=; b=SAYm/oQ4w8PyWqsqATgX21dcMEFisH+jKAqzgYXuu1QVHd/bKa+saZpsKd0Ixp5kaK dtVPudvI5I/U+a75VmTLgGA4JLdIlmVG6G7NDd3i+0QbpglU435+x1/ZhSSPnal6m3Q7 BZokcWSHCd/aVi/+BJGmwVfY5wL8M9nHmddJwC7hyop08FGjrY8jpf2dLj8nn5idDhCj zYUOf7d4hocD4nvwEysBlcl4xz3Cnf/evjf9TU0EZS3EUZvbJhBce3elVCt7M46xBT3i H/4oHiqUdBfqrrghrCRwecGLWnJLddQDP77wCGd+nIltcdeYFxaxQoeGdG9MqGVUh8T1 EnmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=+5KQrnHXiI7OqCuos/sE3GNhfpeVWGLcvYlJ3q/gLNQ=; b=p8U+RjmbnJo23oKBKGeY/ab1L6c73oln3zQMn1FjKCAiYdQ9z1n1lJLIuZgCV8j4Nh 1aeUjs8ylp/h+nDzm4D/oeFiIflhjqzucvZQFsagb41zPkzCOeI2gmZ/tVfKa5WndJxu eYw83kDJmPg7kvxggpJQ/LNw8QqzIMZlhwvlrFkYzYwIrQ7MVtjGeKFMsF2MAL1Zqmo2 xceJkBfiH5MrFdvDT3meCS+1cvXniWghXyUZuFdD3vfGd0efLMo0325spLUTCKE2k6JI oMJSNjMdBtUJk4HqvYcYViIMRpU/obCFvQjQLlB65hz0cuLKHrJWeEYrWQEi/Geq73u7 Xv7Q== X-Gm-Message-State: AOAM5335uNFB7igISdKRRkZiu+hMc/QJw1FVm1MAYO+EkKMFgA5v3lVA H1KXNelF5JAjK9gEukShOsQ421xTfFJLqH29+6hSyg== X-Google-Smtp-Source: ABdhPJyJPE1X8NNqnvFUJKp0feCMNexDHMs9OdqGWydmBlXJHZ/MOOhnybjGdU0z68fDY5qnWRMsYScuqsXnnPBuuCo= X-Received: by 2002:a19:644b:: with SMTP id b11mr2187416lfj.358.1611074379421; Tue, 19 Jan 2021 08:39:39 -0800 (PST) MIME-Version: 1.0 References: <1611040814-33449-1-git-send-email-feng.tang@intel.com> In-Reply-To: <1611040814-33449-1-git-send-email-feng.tang@intel.com> From: Shakeel Butt Date: Tue, 19 Jan 2021 08:39:28 -0800 Message-ID: Subject: Re: [PATCH v2] mm: page_counter: relayout structure to reduce false sharing To: Feng Tang Cc: Andrew Morton , Michal Hocko , Johannes Weiner , Roman Gushchin , Linux MM , LKML Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Jan 18, 2021 at 11:20 PM Feng Tang wrote: > > When checking a memory cgroup related performance regression [1], > from the perf c2c profiling data, we found high false sharing for > accessing 'usage' and 'parent'. > > On 64 bit system, the 'usage' and 'parent' are close to each other, > and easy to be in one cacheline (for cacheline size =3D=3D 64+ B). 'usage= ' > is usally written, while 'parent' is usually read as the cgroup's > hierarchical counting nature. > > So move the 'parent' to the end of the structure to make sure they > are in different cache lines. > > Following are some performance data with the patch, against > v5.11-rc1. [ In the data, A means a platform with 2 sockets 48C/96T, > B is a platform of 4 sockests 72C/144T, and if a %stddev will be > shown bigger than 2%, P100/P50 means number of test tasks equals > to 100%/50% of nr_cpu] > > will-it-scale/malloc1 > --------------------- > v5.11-rc1 v5.11-rc1+patch > > A-P100 15782 =C2=B1 2% -0.1% 15765 =C2=B1 3% will-it-s= cale.per_process_ops > A-P50 21511 +8.9% 23432 will-it-scale.per_p= rocess_ops > B-P100 9155 +2.2% 9357 will-it-scale.per_p= rocess_ops > B-P50 10967 +7.1% 11751 =C2=B1 2% will-it-scale.= per_process_ops > > will-it-scale/pagefault2 > ------------------------ > v5.11-rc1 v5.11-rc1+patch > > A-P100 79028 +3.0% 81411 will-it-scale.per_p= rocess_ops > A-P50 183960 =C2=B1 2% +4.4% 192078 =C2=B1 2% will-it-s= cale.per_process_ops > B-P100 85966 +9.9% 94467 =C2=B1 3% will-it-scale.= per_process_ops > B-P50 198195 +9.8% 217526 will-it-scale.per_p= rocess_ops > > fio (4k/1M is block size) > ------------------------- > v5.11-rc1 v5.11-rc1+patch > > A-P50-r-4k 16881 =C2=B1 2% +1.2% 17081 =C2=B1 2% fio.read_= bw_MBps > A-P50-w-4k 3931 +4.5% 4111 =C2=B1 2% fio.write_bw_M= Bps > A-P50-r-1M 15178 -0.2% 15154 fio.read_bw_MBps > A-P50-w-1M 3924 +0.1% 3929 fio.write_bw_MBps > > [1].https://lore.kernel.org/lkml/20201102091543.GM31092@shao2-debian/ > Signed-off-by: Feng Tang > Reviewed-by: Roman Gushchin > Cc: Johannes Weiner > Cc: Michal Hocko Reviewed-by: Shakeel Butt