From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 822E9C433EF for ; Tue, 28 Sep 2021 22:14:32 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 0B9826128E for ; Tue, 28 Sep 2021 22:14:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 0B9826128E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 8A884900003; Tue, 28 Sep 2021 18:14:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 831AF900002; Tue, 28 Sep 2021 18:14:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6F9F6900003; Tue, 28 Sep 2021 18:14:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0139.hostedemail.com [216.40.44.139]) by kanga.kvack.org (Postfix) with ESMTP id 58913900002 for ; Tue, 28 Sep 2021 18:14:31 -0400 (EDT) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 092228249980 for ; Tue, 28 Sep 2021 22:14:31 +0000 (UTC) X-FDA: 78638387142.13.6929FBF Received: from mail-ed1-f44.google.com (mail-ed1-f44.google.com [209.85.208.44]) by imf18.hostedemail.com (Postfix) with ESMTP id BC6814002087 for ; Tue, 28 Sep 2021 22:14:30 +0000 (UTC) Received: by mail-ed1-f44.google.com with SMTP id ba1so848173edb.4 for ; Tue, 28 Sep 2021 15:14:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=x9zDwJRWIm139eLhv6Qo7TZGdOTE27JLM+druYyg5ZI=; b=qYa6jC0CPO7v3mfxhwIs9R3Pwvt+6UQsr/69yWr3RX8Q0Qy1yfDrgSUOJ6L0aCi5AH AWgvQKSfEkR9/nBtB3gJhn82TWy84/sbaRg2a0UIo9UHrpZf4jGRaGcZXf/fr4BKwigU XCciR7+dbBgdzzNvOxm2Oi4LXlbcYAgvEYSqyHxXSbUe184l4EFPk01aWIAWUFknnFJW cFgir3Hbv8hMcOLsfcDKgpuqNjyvvpCb63jHiZ7vw4MlX1av11n+wLVuVe+ZT1pOZ4g3 /OJIQWe5OyFjz2vauklQqXnSN1DM9OZuVtTrO+1vWWam1kkYvWtY9ovmbPB13155558R 7qBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=x9zDwJRWIm139eLhv6Qo7TZGdOTE27JLM+druYyg5ZI=; b=NAijLURyGDNYU2RnDcOgn9OsoEbMJrCnwABM5Z3NQXORz6+JzXxqvF4zrKS2jFAYwB SykKs2h00Yb71YdEzmKRZ0FndKmKqksdU3kickUWqBcQXrxBBtj+89stImvtufyxutlf S46Ignx9lEKxFbGP7DFFa6O5eMzsxfctLrTNrCBM/vGeyIqAjkOULaeRhkd2sRpx4naP Tfyljm4CLcLxjthS8lqOYPkwSBB0doPPnAdpPhASlavNEUbCTij1r1GN8J8hQ+WEQzY2 3q4iqpKzmxH+rV96xz7Zb0H2acw2EphX7UdaLeafAp/AnE3YXbXkwaFeJdbSDwkWYOQO mWHw== X-Gm-Message-State: AOAM533JjX4bdhoclo5/5Io3/yYK7LclLj/lqmPFYNN6efN8Z+H10Hl5 Yw77vd3ne5Z3UPgYM7PvoQPOxmss2gUaqQ6IKsA= X-Google-Smtp-Source: ABdhPJxeEO9O5JCyltmbwDuJH0dZza+zpUnIZ/I0Jy1+CdC9cqDxCKq0g+oPsfHKrMj138CvyRCNMq5qlx1HIMBxqGQ= X-Received: by 2002:a50:e044:: with SMTP id g4mr10497027edl.46.1632867269412; Tue, 28 Sep 2021 15:14:29 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Yang Shi Date: Tue, 28 Sep 2021 15:14:17 -0700 Message-ID: Subject: Re: [BUG] The usage of memory cgroup is not consistent with processes when using THP To: =?UTF-8?B?5Y+w6L+Q5pa5?= Cc: Johannes Weiner , Hugh Dickins , Tejun Heo , Cgroups , Linux MM Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: BC6814002087 X-Stat-Signature: 3sctkwtudw9m36g6oasqdgtsn4id3atm Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=qYa6jC0C; spf=pass (imf18.hostedemail.com: domain of shy828301@gmail.com designates 209.85.208.44 as permitted sender) smtp.mailfrom=shy828301@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam06 X-HE-Tag: 1632867270-838955 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Sep 28, 2021 at 12:15 AM =E5=8F=B0=E8=BF=90=E6=96=B9 wrote: > > Yang Shi =E4=BA=8E2021=E5=B9=B49=E6=9C=8828=E6=97= =A5=E5=91=A8=E4=BA=8C =E4=B8=8A=E5=8D=881:28=E5=86=99=E9=81=93=EF=BC=9A > > IMHO I don't think this is a bug. The disparity reflects the > > difference in how the page life cycle is viewed between process and > > cgroup. The usage of process comes from the rss_counter of mm. It > > tracks the per-process mapped memory usage. So it is updated once the > > page is zapped. > > > > But from the point of cgroup, the page is charged when it is allocated > > and uncharged when it is freed. The page may be zapped by one process, > > but there might be other users pin the page to prevent it from being > > freed. The pin may be very transient or may be indefinite. THP is one > > of the pins. It is gone when the THP is split, but the split may > > happen a long time after the page is zapped due to deferred split. > Thank you for reply. I agree that it reflects the difference between > process and cgroup. The memory usage of cgroup is usually used to > indicate the memory usage of the container. It can be used to avoid > the OOM and etc. The disparity will cause that the memory usage of > containers with the same processes are randomly different (we found > more than 30GB different). It is hard to manage them. Of course, > disable THP is a way to solve it. Can it have another way to solve it > ? I don't quite get what exactly you want to manage. If you want to get rid of the disparity, I don't have good idea other than splitting THP in place instead of using deferred split. But AFAIK it is not quite feasible due to some locking problems.