From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.6 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0613C433DB for ; Wed, 6 Jan 2021 06:05:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 09F2D22C9F for ; Wed, 6 Jan 2021 06:05:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 09F2D22C9F Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 124546B00E4; Wed, 6 Jan 2021 01:05:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0D5C68D00EA; Wed, 6 Jan 2021 01:05:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0121E8D0090; Wed, 6 Jan 2021 01:05:33 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0234.hostedemail.com [216.40.44.234]) by kanga.kvack.org (Postfix) with ESMTP id DEDE86B00E4 for ; Wed, 6 Jan 2021 01:05:33 -0500 (EST) Received: from smtpin16.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id A78F91EE6 for ; Wed, 6 Jan 2021 06:05:33 +0000 (UTC) X-FDA: 77674313346.16.limit52_0711523274df Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin16.hostedemail.com (Postfix) with ESMTP id 7F5F4100E6903 for ; Wed, 6 Jan 2021 06:05:33 +0000 (UTC) X-HE-Tag: limit52_0711523274df X-Filterd-Recvd-Size: 5256 Received: from mail-lf1-f53.google.com (mail-lf1-f53.google.com [209.85.167.53]) by imf47.hostedemail.com (Postfix) with ESMTP for ; Wed, 6 Jan 2021 06:05:32 +0000 (UTC) Received: by mail-lf1-f53.google.com with SMTP id x20so4024748lfe.12 for ; Tue, 05 Jan 2021 22:05:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=tIJaOqbryClxfevroW/ia7XSphbowTWiua0Lkb9FHkA=; b=ZOcX4aGlTjVymjlFOaOvgg1un/DRzDU7SmBaYNTrnVhtpPEDGnU033l+QWfwDB0ZvC iVAJSvmWjESVp/0zenZR0N5ZakfQU2iGXQidJ+J8OQmYHj/KPma0juWZMsOjByvwQKos luI7FW2S6c+xfhLXldHZUvRSKqZ+hTCGxrXNNVzZg9/ZL5uR2dtR04DlkKOQ44+rbzS6 QDHyz7/w3poV7RefdJFg8KImtLcgV/nuffLhc4PUOYnUfCcLOJzS7eBCA/RETIm0vrM4 ctg9FurwupAapeGDslO1GGYY1x72+liakoMZefdELNiBFeMW9Ga5gnaztrDZjGyiSm6j FECA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=tIJaOqbryClxfevroW/ia7XSphbowTWiua0Lkb9FHkA=; b=Miu8Yd4GdEhNqOLjj9vyFlQ/dln1RrPWpIFfPK5ennnzWuVLtRBh4D7sIM/hCXvQGG hrxNCc1MjekPeGo2tnIOsitnJrriOTlbHCdLh8UznflHbDRSJuI4JL4vmcvLB4b92QWU QfQu7UXljbH2GDsaD3BKx0x+eSHSrl9OSzuAdih5WVUgmvnV8VeKXoGzhgEx5R4Q16cl DD/aLiWgoJXglAxHJBCWGNZ7ZBA44BL7SP6sDS0OGk9IWRnsCUz/2mGrWcdLitoRvzPv +yD0T4cG0ZeFNW4Q5mfbzsX14LaMkk+zNkSKnXjtvvyvGiV51PoL7S+6XnqgxcRVrE7r +pOg== X-Gm-Message-State: AOAM533ePJtuJymM6BXjaGFKTNBea4Lbqmr5cO5Hhb1CWAHswFr5MGBy W7Vlb9bQLPXAxcF4tp6vu1EazVBeY0t1b4l4OmD+jg== X-Google-Smtp-Source: ABdhPJzIyXxfjs3/KFS6JaC9/bOWQdFBqOzAqT+WUjEFzFby45K7RLUK8h8kZqk0W0n/DGJd7UWu2ioehB8yt55Volo= X-Received: by 2002:a2e:9cc3:: with SMTP id g3mr1519821ljj.0.1609913131155; Tue, 05 Jan 2021 22:05:31 -0800 (PST) MIME-Version: 1.0 References: <20210106042239.2860107-1-guro@fb.com> In-Reply-To: <20210106042239.2860107-1-guro@fb.com> From: Shakeel Butt Date: Tue, 5 Jan 2021 22:05:20 -0800 Message-ID: Subject: Re: [PATCH] mm: memcg/slab: optimize objcg stock draining To: Roman Gushchin Cc: Andrew Morton , Linux MM , Michal Hocko , Johannes Weiner , LKML , Kernel Team , Imran Khan Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Jan 5, 2021 at 8:22 PM Roman Gushchin wrote: > > Imran Khan reported a regression in hackbench results caused by the > commit f2fe7b09a52b ("mm: memcg/slab: charge individual slab objects > instead of pages"). The regression is noticeable in the case of > a consequent allocation of several relatively large slab objects, > e.g. skb's. As soon as the amount of stocked bytes exceeds PAGE_SIZE, > drain_obj_stock() and __memcg_kmem_uncharge() are called, and it leads > to a number of atomic operations in page_counter_uncharge(). > > The corresponding call graph is below (provided by Imran Khan): > |__alloc_skb > | | > | |__kmalloc_reserve.isra.61 > | | | > | | |__kmalloc_node_track_caller > | | | | > | | | |slab_pre_alloc_hook.constprop.88 > | | | obj_cgroup_charge > | | | | | > | | | | |__memcg_kmem_charge > | | | | | | > | | | | | |page_counter_try_charge > | | | | | > | | | | |refill_obj_stock > | | | | | | > | | | | | |drain_obj_stock.isra.68 > | | | | | | | > | | | | | | |__memcg_kmem_uncharge > | | | | | | | | > | | | | | | | |page_counter_uncharge > | | | | | | | | | > | | | | | | | | |page_counter_cancel > | | | | > | | | | > | | | |__slab_alloc > | | | | | > | | | | |___slab_alloc > | | | | | > | | | |slab_post_alloc_hook > > Instead of directly uncharging the accounted kernel memory, it's > possible to refill the generic page-sized per-cpu stock instead. > It's a much faster operation, especially on a default hierarchy. > As a bonus, __memcg_kmem_uncharge_page() will also get faster, > so the freeing of page-sized kernel allocations (e.g. large kmallocs) > will become faster. > > A similar change has been done earlier for the socket memory by > the commit 475d0487a2ad ("mm: memcontrol: use per-cpu stocks for > socket memory uncharging"). > > Signed-off-by: Roman Gushchin > Reported-by: Imran Khan I remember seeing this somewhere https://lore.kernel.org/linux-mm/20190423154405.259178-1-shakeelb@google.com/ Reviewed-by: Shakeel Butt