From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97370C433FE for ; Fri, 17 Sep 2021 10:50:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0D0B4610A7 for ; Fri, 17 Sep 2021 10:50:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 0D0B4610A7 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=bytedance.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 7C1E06B0071; Fri, 17 Sep 2021 06:50:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 770B56B0072; Fri, 17 Sep 2021 06:50:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 611846B0073; Fri, 17 Sep 2021 06:50:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 549396B0071 for ; Fri, 17 Sep 2021 06:50:05 -0400 (EDT) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 1582A824556B for ; Fri, 17 Sep 2021 10:50:05 +0000 (UTC) X-FDA: 78596745570.29.AEB05EA Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) by imf05.hostedemail.com (Postfix) with ESMTP id 174A1505D853 for ; Fri, 17 Sep 2021 10:50:03 +0000 (UTC) Received: by mail-pl1-f172.google.com with SMTP id f21so5939393plb.4 for ; Fri, 17 Sep 2021 03:50:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=g4vyRKEIUiu45D51r7XhzUKkB2Cebq0r9MOFU5Rsk84=; b=05M06HYqKZipEP5EwPjCKhM17S57oJhsjjeabLjo7ThsmfwwgWobY39fx8bB4rbl5r f27qU39b3vBOJZbG5Szk3HYi9CKPk9HmvirYZhihSqbsKJhnhz3Kj4th257EoM3tjICC xdULXOelJcdTOTU9v4u/5QrcqdkZJMLU3+gBSskuI4ZfnnuBz61TIjJneTJ08QAEXEBp e5y6GUJwfRyeYA0UCVVJLLMwkbiyBopnJTq8cydpck21il7CMGs+KbZKyWGpVvqqNN5H PPCAm+aSVenlblAEpyY7rpgTitTgndDKEm0ExwLujAGaAitt44kgUgze/0tsWGMtqUVk xMGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=g4vyRKEIUiu45D51r7XhzUKkB2Cebq0r9MOFU5Rsk84=; b=qgJSaOd0rzHnP2P1z62CKo9t0uZju93PVhb5yJSVfKMz0DQoGqUsQ7FLIVu5GdkBE4 vHv9ZYPQgMAo6Qj28F1dy9eP4FTP2+hWwKly1yym9X++fjEzlTDjGKQYK48z+aUXxi5b vyHCAl+PTSbhrNFT+uoWiCXQT948/DcVMVj8LNrUX6q09qHoXeKQ+u6Dq3unFV9XK2wm 9xRUkqz1Wae+XtIBT520+gROIn377YBApzg2iEdmTBdxJGMQ3R2ErXu1+SUFZslEd31I xLOuEXs4v3Zwx/SMRZHosKxlZaeTECeqAA6f/gCSXVRWHc0sZ7BzI2bm+2LDhD3STX+2 c8eQ== X-Gm-Message-State: AOAM532MAtFtsz/q2cEqGuYdzCGSVOf2kY1CM3ZGGJtVfg5rq5jUTdvx 9Z5Y0r/iBdhvsuNbubksYyPvUUSvWAc6Haez8Ahviw== X-Google-Smtp-Source: ABdhPJzIXgbIS7KQ1Pi0ina81GvSAlcfS4Ljtx7M0muOzGWHB11wGZ9eu5Ct1nNNvUlsshQkgiBs4B+XrWZAdEpmTOQ= X-Received: by 2002:a17:902:7107:b0:13d:93aa:a70a with SMTP id a7-20020a170902710700b0013d93aaa70amr827046pll.32.1631875802670; Fri, 17 Sep 2021 03:50:02 -0700 (PDT) MIME-Version: 1.0 References: <20210916134748.67712-1-songmuchun@bytedance.com> In-Reply-To: From: Muchun Song Date: Fri, 17 Sep 2021 18:49:21 +0800 Message-ID: Subject: Re: [PATCH v2 00/13] Use obj_cgroup APIs to charge the LRU pages To: Roman Gushchin Cc: Johannes Weiner , Michal Hocko , Andrew Morton , Shakeel Butt , Vladimir Davydov , LKML , Linux Memory Management List , Xiongchun duan , fam.zheng@bytedance.com, "Singh, Balbir" , Yang Shi , Alex Shi , Muchun Song , Qi Zheng Content-Type: text/plain; charset="UTF-8" Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=05M06HYq; spf=pass (imf05.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.214.172 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com X-Stat-Signature: 73njceg6mtn5y1p89t154t1cbw1wii3i X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 174A1505D853 X-HE-Tag: 1631875803-799449 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Sep 17, 2021 at 9:29 AM Roman Gushchin wrote: > > Hi Muchun! > > On Thu, Sep 16, 2021 at 09:47:35PM +0800, Muchun Song wrote: > > This version is rebased over linux 5.15-rc1, because Shakeel has asked me > > if I could do that. I rework some code suggested by Roman as well in this > > version. I have not removed the Acked-by tags which are from Roman, because > > this version is not based on the folio relevant. If Roman wants me to > > do this, please let me know, thanks. > > I'm fine with this, thanks for clarifying. > > > > > Since the following patchsets applied. All the kernel memory are charged > > with the new APIs of obj_cgroup. > > > > [v17,00/19] The new cgroup slab memory controller[1] > > [v5,0/7] Use obj_cgroup APIs to charge kmem pages[2] > > > > But user memory allocations (LRU pages) pinning memcgs for a long time - > > it exists at a larger scale and is causing recurring problems in the real > > world: page cache doesn't get reclaimed for a long time, or is used by the > > second, third, fourth, ... instance of the same job that was restarted into > > a new cgroup every time. Unreclaimable dying cgroups pile up, waste memory, > > and make page reclaim very inefficient. > > I've an idea: what if we use struct list_lru_memcg as an intermediate object > between an individual page and struct mem_cgroup? > > It could contain a pointer to a memory cgroup structure (not even sure if a > reference is needed), and a lru page can contain a pointer to the lruvec instead > of memcg/objcg. Hi Roman, If I understand properly, here you mean the struct page has a pointer to the struct lruvec not struct list_lru_memcg. What's the functionality of the struct list_lru_memcg? Would you mind exposing more details? > > This approach can probably simplify the locking scheme. But what's more > important, it can dramatically reduce the number of css_get()/put() calls. > The latter are not particularly cheap after the deletion of a cgroup: > they are atomic_dec()'s. As a result, the reclaim efficiency could be much > better. The downside: we will need to update page->lruvec_memcg pointers on > reparenting pages during the cgroup removal. Here we need to update page->lruvec_memcg pointers one by one, right? Because the lru lock is per lruvec, the locking scheme still need to be as proposed by this series when the page->lruvec_memcg is changed If I understand properly. It's likely that I don't get your point. Looking forward to your further details. Thanks. > > This is a rough idea, maybe there are significant reasons why it's not possible > or will be way worse. But I think it's worth discussing. What do you think? > > Thanks!