From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.3 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A1C14C433ED for ; Thu, 15 Apr 2021 17:08:36 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 399C86117A for ; Thu, 15 Apr 2021 17:08:36 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 399C86117A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id AFEAA6B0036; Thu, 15 Apr 2021 13:08:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AB28F6B006C; Thu, 15 Apr 2021 13:08:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 903176B0070; Thu, 15 Apr 2021 13:08:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0054.hostedemail.com [216.40.44.54]) by kanga.kvack.org (Postfix) with ESMTP id 6FC306B0036 for ; Thu, 15 Apr 2021 13:08:35 -0400 (EDT) Received: from smtpin03.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 1704E4DC8 for ; Thu, 15 Apr 2021 17:08:35 +0000 (UTC) X-FDA: 78035235390.03.3F25BF2 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf18.hostedemail.com (Postfix) with ESMTP id 6A23B2000263 for ; Thu, 15 Apr 2021 17:08:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1618506514; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=c4bUwdWqx5JUrWJYR1rKkQlIPcDgNW5mI87L7J9m2Ko=; b=HMlDzOBsO9FtPKidhpmYUZVrSl40bs1GxJV36AqBOOk1V+tZclNUIKHHg/j0h4lIGZYKtI khLfF1o2cguWp2JLcYC1YoW+kB+nkw8T24toNmVkF1zlz6qwdFgCeqZPmDyOaeI8R8J0U6 DKv2MCqXegXcxLO1SBtZW6FteAQOvTI= Received: from mail-qt1-f199.google.com (mail-qt1-f199.google.com [209.85.160.199]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-26-l_nKJh8NM6y_z7SPqkRwow-1; Thu, 15 Apr 2021 13:08:32 -0400 X-MC-Unique: l_nKJh8NM6y_z7SPqkRwow-1 Received: by mail-qt1-f199.google.com with SMTP id y10-20020a05622a004ab029019d4ad3437cso4458458qtw.12 for ; Thu, 15 Apr 2021 10:08:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:subject:to:cc:references:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=c4bUwdWqx5JUrWJYR1rKkQlIPcDgNW5mI87L7J9m2Ko=; b=jjnuNGuo59LM00ag7ZvotoAmHFYOvRQsVeR3twfLTCiKH0x3F9nrS1zli+OcXwX05f tQUIMPmWp0te7ur+RnyIFgwjiHFZqptKfkICer/2UKWcRbmDmBicslxI8l9WzaENjxCZ hcxHvkWqY5afxP8fIWC2toApog21N17nvMRmtt94iM1PNGbhirH4qpGSYbsX4Jz6k/8B FCelpXwSmK1bOONcRGzEJwudrcEfHN+g/kfCEOlbYhfVLygdawDhuaFUWaf0CRM2kS8/ C7NlNrS3AhB7c8+Qhla72+6QcWsdXOIrD3xJAzzwduMspbJV2Ogq0287ajwAq7ub2eIT WPwQ== X-Gm-Message-State: AOAM5308m1QLQ9iiiPrltw0AWyCOV0ucft7JCvZ7aUpL7WU1eU3YdFzy hesQv0nvvnbCT7CRD4Xn48KZeOE0MKJyux+cXc1om/0A4rUDP3ohJOUc8YLH/KiuIZY8jhOxL5b Te+IwO9NepU4= X-Received: by 2002:a0c:e601:: with SMTP id z1mr4249146qvm.62.1618506511981; Thu, 15 Apr 2021 10:08:31 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyYKgNv627ZQRacj4Z0akpbwf6cDPraxoDdrUd5LGopDdVhUkGCp7HX14SZP2g3PIZr3viwAg== X-Received: by 2002:a0c:e601:: with SMTP id z1mr4249129qvm.62.1618506511823; Thu, 15 Apr 2021 10:08:31 -0700 (PDT) Received: from llong.remote.csb ([2601:191:8500:76c0::cdbc]) by smtp.gmail.com with ESMTPSA id u21sm2196347qtq.11.2021.04.15.10.08.30 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 15 Apr 2021 10:08:31 -0700 (PDT) From: Waiman Long X-Google-Original-From: Waiman Long Subject: Re: [PATCH v3 3/5] mm/memcg: Cache vmstat data in percpu memcg_stock_pcp To: Johannes Weiner Cc: Michal Hocko , Vladimir Davydov , Andrew Morton , Tejun Heo , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Vlastimil Babka , Roman Gushchin , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Shakeel Butt , Muchun Song , Alex Shi , Chris Down , Yafang Shao , Wei Yang , Masayoshi Mizuma , Xing Zhengjun References: <20210414012027.5352-1-longman@redhat.com> <20210414012027.5352-4-longman@redhat.com> Message-ID: <5abe499a-b1ad-fa22-3487-1a6e00e30e17@redhat.com> Date: Thu, 15 Apr 2021 13:08:29 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.0 MIME-Version: 1.0 In-Reply-To: Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=llong@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 6A23B2000263 X-Stat-Signature: ndu3ss7nfute39e43e5xd49kohnuak7o Received-SPF: none (redhat.com>: No applicable sender policy available) receiver=imf18; identity=mailfrom; envelope-from=""; helo=us-smtp-delivery-124.mimecast.com; client-ip=170.10.133.124 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618506515-892552 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 4/15/21 12:50 PM, Johannes Weiner wrote: > On Tue, Apr 13, 2021 at 09:20:25PM -0400, Waiman Long wrote: >> Before the new slab memory controller with per object byte charging, >> charging and vmstat data update happen only when new slab pages are >> allocated or freed. Now they are done with every kmem_cache_alloc() >> and kmem_cache_free(). This causes additional overhead for workloads >> that generate a lot of alloc and free calls. >> >> The memcg_stock_pcp is used to cache byte charge for a specific >> obj_cgroup to reduce that overhead. To further reducing it, this patch >> makes the vmstat data cached in the memcg_stock_pcp structure as well >> until it accumulates a page size worth of update or when other cached >> data change. >> >> On a 2-socket Cascade Lake server with instrumentation enabled and this >> patch applied, it was found that about 17% (946796 out of 5515184) of the >> time when __mod_obj_stock_state() is called leads to an actual call to >> mod_objcg_state() after initial boot. When doing parallel kernel build, >> the figure was about 16% (21894614 out of 139780628). So caching the >> vmstat data reduces the number of calls to mod_objcg_state() by more >> than 80%. > Right, but mod_objcg_state() is itself already percpu-cached. What's > the benefit of avoiding calls to it with another percpu cache? > There are actually 2 set of vmstat data that have to be updated. One is associated with the memcg and other one is for each lruvec within the cgroup. Caching it in obj_stock, we replace 2 writes to two colder cachelines with one write to a hot cacheline. If you look at patch 5, I break obj_stock into two - one for task context and one for irq context. Interrupt disable is no longer needed in task context, but that is not possible when writing to the actual vmstat data arrays. Cheers, Longman