From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4970FC433ED for ; Mon, 12 Apr 2021 22:55:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C934B61287 for ; Mon, 12 Apr 2021 22:55:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C934B61287 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 36DB66B0036; Mon, 12 Apr 2021 18:55:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 31DC56B006E; Mon, 12 Apr 2021 18:55:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1BF036B0070; Mon, 12 Apr 2021 18:55:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0044.hostedemail.com [216.40.44.44]) by kanga.kvack.org (Postfix) with ESMTP id F3C246B0036 for ; Mon, 12 Apr 2021 18:55:26 -0400 (EDT) Received: from smtpin32.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id A274583EF for ; Mon, 12 Apr 2021 22:55:26 +0000 (UTC) X-FDA: 78025223052.32.02E6A0B Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by imf23.hostedemail.com (Postfix) with ESMTP id D35D3A0000FD for ; Mon, 12 Apr 2021 22:55:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1618268125; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc; bh=TJxmmYkTPVvl2+ih4+GZ2DW2gMVPFn/WPrWSZF4KB64=; b=ZykdeXDLiSOPEnbtglMYbstKAggPXbWXsd0xjJ0DPYC150UgKa2lFG8Qe6QvbctTp9MhgC VTmv9jL1+UogcmZ5MuCHGn0pPDuduwo4TqYbI+GUytL08SGU+QD5oG4rYm4aE1SSfYSg4w o5Gv0drYBVaBbKeKNQtfG0dvjADov+8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-479-6X-3d14tPaKkDC1W7EYvFA-1; Mon, 12 Apr 2021 18:55:24 -0400 X-MC-Unique: 6X-3d14tPaKkDC1W7EYvFA-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E3C4410053E6; Mon, 12 Apr 2021 22:55:20 +0000 (UTC) Received: from llong.com (ovpn-114-18.rdu2.redhat.com [10.10.114.18]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1470D5D6D1; Mon, 12 Apr 2021 22:55:12 +0000 (UTC) From: Waiman Long To: Johannes Weiner , Michal Hocko , Vladimir Davydov , Andrew Morton , Tejun Heo , Christoph Lameter , Pekka Enberg , David Rientjes , Joonsoo Kim , Vlastimil Babka , Roman Gushchin Cc: linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Shakeel Butt , Muchun Song , Alex Shi , Chris Down , Yafang Shao , Wei Yang , Masayoshi Mizuma , Xing Zhengjun , Waiman Long Subject: [PATCH v2 0/5] mm/memcg: Reduce kmemcache memory accounting overhead Date: Mon, 12 Apr 2021 18:54:58 -0400 Message-Id: <20210412225503.15119-1-longman@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: D35D3A0000FD X-Stat-Signature: 4sm7fwjmp33csottxqf4e6nspj79k1jb Received-SPF: none (redhat.com>: No applicable sender policy available) receiver=imf23; identity=mailfrom; envelope-from=""; helo=us-smtp-delivery-124.mimecast.com; client-ip=216.205.24.124 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618268124-388260 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: v2: - Fix bug found by test robot in patch 5. - Update cover letter and commit logs. With the recent introduction of the new slab memory controller, we eliminate the need for having separate kmemcaches for each memory cgroup and reduce overall kernel memory usage. However, we also add additional memory accounting overhead to each call of kmem_cache_alloc() and kmem_cache_free(). For workloads that require a lot of kmemcache allocations and de-allocations, they may experience performance regression as illustrated in [1] and [2]. A simple kernel module that performs repeated loop of 100,000,000 kmem_cache_alloc() and kmem_cache_free() of a 64-byte object at module init time is used for benchmarking. The test was run on a CascadeLake server with turbo-boosting disable to reduce run-to-run variation. With memory accounting disable, the run time was 2.848s. With memory accounting enabled, the run times with the application of various patches in the patchset were: Applied patches Run time Accounting overhead Overhead %age --------------- -------- ------------------- ------------- None 10.800s 7.952s 100.0% 1-2 9.140s 6.292s 79.1% 1-3 7.641s 4.793s 60.3% 1-5 6.801s 3.953s 49.7% Note that this is the best case scenario where most updates happen only to the percpu stocks. Real workloads will likely have a certain amount of updates to the memcg charges and vmstats. So the performance benefit will be less. It was found that a big part of the memory accounting overhead was caused by the local_irq_save()/local_irq_restore() sequences in updating local stock charge bytes and vmstat array, at least in x86 systems. There are two such sequences in kmem_cache_alloc() and two in kmem_cache_free(). This patchset tries to reduce the use of such sequences as much as possible. In fact, it eliminates them in the common case. Another part of this patchset to cache the vmstat data update in the local stock as well which also helps. [1] https://lore.kernel.org/linux-mm/20210408193948.vfktg3azh2wrt56t@gabell/T/#u [2] https://lore.kernel.org/lkml/20210114025151.GA22932@xsang-OptiPlex-9020/ Waiman Long (5): mm/memcg: Pass both memcg and lruvec to mod_memcg_lruvec_state() mm/memcg: Introduce obj_cgroup_uncharge_mod_state() mm/memcg: Cache vmstat data in percpu memcg_stock_pcp mm/memcg: Separate out object stock data into its own struct mm/memcg: Optimize user context object stock access include/linux/memcontrol.h | 14 ++- mm/memcontrol.c | 200 ++++++++++++++++++++++++++++++++----- mm/percpu.c | 9 +- mm/slab.h | 32 +++--- 4 files changed, 197 insertions(+), 58 deletions(-) -- 2.18.1