From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87CC9C433B4 for ; Thu, 15 Apr 2021 18:13:42 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 07F96611AB for ; Thu, 15 Apr 2021 18:13:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 07F96611AB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=cmpxchg.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 92C076B0070; Thu, 15 Apr 2021 14:13:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8DC6B6B0071; Thu, 15 Apr 2021 14:13:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 755B56B0072; Thu, 15 Apr 2021 14:13:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0169.hostedemail.com [216.40.44.169]) by kanga.kvack.org (Postfix) with ESMTP id 54CAB6B0070 for ; Thu, 15 Apr 2021 14:13:41 -0400 (EDT) Received: from smtpin34.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 05D408248D7C for ; Thu, 15 Apr 2021 18:13:41 +0000 (UTC) X-FDA: 78035399442.34.119DA0C Received: from mail-qt1-f182.google.com (mail-qt1-f182.google.com [209.85.160.182]) by imf13.hostedemail.com (Postfix) with ESMTP id 4D3A6E000122 for ; Thu, 15 Apr 2021 18:13:37 +0000 (UTC) Received: by mail-qt1-f182.google.com with SMTP id z15so10817856qtj.7 for ; Thu, 15 Apr 2021 11:13:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=EO1ZRnRE5AIix1KmwKUfdGLcr4z4RBOaJpoQ33nEJcU=; b=jcmAT3HPYBCrHP6jHCPcphUzr0fG91bIyePgBoblxnUY+uM+e0eGmbicEIUky5JdS0 3f+T18xTekj77ibaghPcyaYdxNQRjMPbL4Ntyc63OCy/SDczT2i+WSCRxh/sJc/Sb+jL zTGvaa45VbkcKTNF0+kZ/awf98n0XtQGdqXy5xnTcX+StcRGfboJyIAFPchfBTtSF5A9 VROXd7+VJIJA1jLng07E/m4Vx1z4MfGAhl74Z/6Z251sfegah5pmMek7SH5p2gLdC140 PPSH1qvSRmRMyxmzaa54BMa3AEJO4ukZlCVGoQ8YJmEH+HRKNHBmnXMdiv8fuv7DifI4 oqvQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=EO1ZRnRE5AIix1KmwKUfdGLcr4z4RBOaJpoQ33nEJcU=; b=svrvcAiVnyvW4X5but3PWQfGIkk6ftW6QxNER60CON/nGxsk7KgRfHazzabfyqxQT1 XYYGiE6mC5RLZkoGEQGsqynCArMnSjJO28d24ZBua4UzIS3pTSC5jGb7456z4hhCkKTi Y/mhgKjO6uvolRBf0LcYjq2kXz6ROrg10z27WNNfsZLgrdPhU2Kc0En4pW1apZHzz9fj 0wN4c1+yHY1o6b1/MsOXDxQQsGlgOfrdHg3tdoK/yxsd8Y/wcf/C6lz5MbbHV2JUr1GD sZ/q3QDHZ124yzocMZnWYxT1KpJCmjZBkcdk9RxdQeGnwjFz9VFZytpzJLTX84DpPer0 RODQ== X-Gm-Message-State: AOAM532j/0J5a0L/d6Aloq1CQx4/SE23a0QNZfLXQX3FwaDpqoLk0ZoS 8fGhz2M534eTSmqCiPQ8uEjl7w== X-Google-Smtp-Source: ABdhPJwZeV1EKidTKUJjnDui70hmgN5zSMh0mrr3zrj2n5DFwYlqRj8w5iOpw/kIBIcCxxRbU4cpSA== X-Received: by 2002:a05:622a:3c8:: with SMTP id k8mr4248564qtx.101.1618510419998; Thu, 15 Apr 2021 11:13:39 -0700 (PDT) Received: from localhost (70.44.39.90.res-cmts.bus.ptd.net. [70.44.39.90]) by smtp.gmail.com with ESMTPSA id 71sm2559708qkm.40.2021.04.15.11.13.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Apr 2021 11:13:39 -0700 (PDT) Date: Thu, 15 Apr 2021 14:13:38 -0400 From: Johannes Weiner To: Waiman Long Cc: Michal Hocko , Vladimir Davydov , Andrew Morton , Tejun Heo , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Vlastimil Babka , Roman Gushchin , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Shakeel Butt , Muchun Song , Alex Shi , Chris Down , Yafang Shao , Wei Yang , Masayoshi Mizuma , Xing Zhengjun Subject: Re: [PATCH v3 3/5] mm/memcg: Cache vmstat data in percpu memcg_stock_pcp Message-ID: References: <20210414012027.5352-1-longman@redhat.com> <20210414012027.5352-4-longman@redhat.com> <5abe499a-b1ad-fa22-3487-1a6e00e30e17@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5abe499a-b1ad-fa22-3487-1a6e00e30e17@redhat.com> X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 4D3A6E000122 X-Stat-Signature: pbhyry33nzns1yg565go8gtp4pe813r6 Received-SPF: none (cmpxchg.org>: No applicable sender policy available) receiver=imf13; identity=mailfrom; envelope-from=""; helo=mail-qt1-f182.google.com; client-ip=209.85.160.182 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618510417-859733 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Apr 15, 2021 at 01:08:29PM -0400, Waiman Long wrote: > On 4/15/21 12:50 PM, Johannes Weiner wrote: > > On Tue, Apr 13, 2021 at 09:20:25PM -0400, Waiman Long wrote: > > > Before the new slab memory controller with per object byte charging, > > > charging and vmstat data update happen only when new slab pages are > > > allocated or freed. Now they are done with every kmem_cache_alloc() > > > and kmem_cache_free(). This causes additional overhead for workloads > > > that generate a lot of alloc and free calls. > > > > > > The memcg_stock_pcp is used to cache byte charge for a specific > > > obj_cgroup to reduce that overhead. To further reducing it, this patch > > > makes the vmstat data cached in the memcg_stock_pcp structure as well > > > until it accumulates a page size worth of update or when other cached > > > data change. > > > > > > On a 2-socket Cascade Lake server with instrumentation enabled and this > > > patch applied, it was found that about 17% (946796 out of 5515184) of the > > > time when __mod_obj_stock_state() is called leads to an actual call to > > > mod_objcg_state() after initial boot. When doing parallel kernel build, > > > the figure was about 16% (21894614 out of 139780628). So caching the > > > vmstat data reduces the number of calls to mod_objcg_state() by more > > > than 80%. > > Right, but mod_objcg_state() is itself already percpu-cached. What's > > the benefit of avoiding calls to it with another percpu cache? > > > There are actually 2 set of vmstat data that have to be updated. One is > associated with the memcg and other one is for each lruvec within the > cgroup. Caching it in obj_stock, we replace 2 writes to two colder > cachelines with one write to a hot cacheline. If you look at patch 5, I > break obj_stock into two - one for task context and one for irq context. > Interrupt disable is no longer needed in task context, but that is not > possible when writing to the actual vmstat data arrays. Ah, thanks for the explanation. Both of these points are worth mentioning in the changelog of this patch.