From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C98A9C433EF for ; Sun, 26 Sep 2021 07:35:47 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 5661960FDA for ; Sun, 26 Sep 2021 07:35:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 5661960FDA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 92F026B0071; Sun, 26 Sep 2021 03:35:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8DF096B0072; Sun, 26 Sep 2021 03:35:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7CDA06B0073; Sun, 26 Sep 2021 03:35:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0165.hostedemail.com [216.40.44.165]) by kanga.kvack.org (Postfix) with ESMTP id 6E04C6B0071 for ; Sun, 26 Sep 2021 03:35:46 -0400 (EDT) Received: from smtpin37.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 2C47D8249980 for ; Sun, 26 Sep 2021 07:35:46 +0000 (UTC) X-FDA: 78628915092.37.5E2B8D6 Received: from mail-ot1-f51.google.com (mail-ot1-f51.google.com [209.85.210.51]) by imf02.hostedemail.com (Postfix) with ESMTP id CEF587001776 for ; Sun, 26 Sep 2021 07:35:45 +0000 (UTC) Received: by mail-ot1-f51.google.com with SMTP id j11-20020a9d190b000000b00546fac94456so19720229ota.6 for ; Sun, 26 Sep 2021 00:35:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:from:date:message-id:subject:to:cc; bh=Sb0engf5YAW0rLpuy9gVSDhYbZE2i/wvwLZEgTdAMSM=; b=cx7/UfjAmHfby9ohbkgrVDoMiokDFk6S+GDY/Ux7skpOmmp/yHR7ezm8j+9jnnm1ej rCGyCtLAf6xOBt3Otqdg78LIc7Tiwx6cwEIWIFQtZhQQp3s32sXq93RVtCDuEHT4Hg+S DWmqgyrzRbTtlWAN2fQ2NKC5/YtqFK+ntPzPLuLK2OjWumsuDvnb7dxrRv7nVoReltLv Hzpl/XCI98c6EribS5aQiTECullt+Ls/SiiVpLUTAp6udHqwdyc+f2sP7Z2zG3oB1Qnc fY6q1rqsLXhBPY4PjLJGEdWg7xzTarQxaO8kUfVxg0MWXlMj5gEISh3f8ZPikodaN4DM TMJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=Sb0engf5YAW0rLpuy9gVSDhYbZE2i/wvwLZEgTdAMSM=; b=AxJSvG929oEwGXA9Hl5XvwYWdCBSjjIzymGV0o/jj4YJTkX43kCGzTzMmMs+QCw7Ox 4etPXy8MOEmkAWHLoT/FMyIRAHsmLeOMzNKBiS7B690Rhj3fkCL/66HLpAdTaYeTTcYA /i8zwEg/Kd2Q9CLf9SMM2KJYCyi7UOBTAy2gtnCuvJLzfipg2VI99tCI5kft6WyGjLVw 0qZhiQ6PRYjJCbTitt/vlh61RlT9seRqv65ory7s1jceiYQ2VbMga0rHv2MgKwVxih74 PZxtDPWhpNkaFxqRtmUJq8Wm60WwC1wWGlJYbYCZJLJzCbw/PCfz4k3RkTi2Yj9UeY8v kzmA== X-Gm-Message-State: AOAM533ip+3758/OmxAXsrbLvCcG8SODtA49PT3QwnJ9CcdMrQUD73/j MEAraJbbW9zg2QIXFEyscQxoObJxSzb289uBBmc= X-Google-Smtp-Source: ABdhPJysFmNqtv74irFeFG1Ugv0SrSTRN9Pa7geQflNUx2jOccedSfcL1jgLANf+vecM4ztF07w1QIXw8SWBWl9GV1U= X-Received: by 2002:a9d:7257:: with SMTP id a23mr11851796otk.311.1632641745137; Sun, 26 Sep 2021 00:35:45 -0700 (PDT) MIME-Version: 1.0 From: =?UTF-8?B?5Y+w6L+Q5pa5?= Date: Sun, 26 Sep 2021 15:35:34 +0800 Message-ID: Subject: [BUG] The usage of memory cgroup is not consistent with processes when using THP To: hannes@cmpxchg.org Cc: hughd@google.com, tj@kernel.org, vdavydov@parallels.com, cgroups@vger.kernel.org, linux-mm@kvack.org Content-Type: multipart/mixed; boundary="00000000000043c54105cce10702" X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: CEF587001776 X-Stat-Signature: paaba37k739ob8qx76hswrw1kgqcx79w Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="cx7/UfjA"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf02.hostedemail.com: domain of yunfangtai09@gmail.com designates 209.85.210.51 as permitted sender) smtp.mailfrom=yunfangtai09@gmail.com X-HE-Tag: 1632641745-394443 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: --00000000000043c54105cce10702 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi folks=EF=BC=8C We found that the usage counter of containers with memory cgroup v1 is not consistent with the memory usage of processes when using THP. It is introduced in upstream 0a31bc97c80 patch and still exists in Linux 5.14.5. The root cause is that mem_cgroup_uncharge is moved to the final put_page(). When freeing parts of huge pages in THP, the memory usage of process is updated when pte unmapped and the usage counter of memory cgroup is updated when splitting huge pages in deferred_split_scan. This causes the inconsistencies and we could find more than 30GB memory difference in our daily usage. It is reproduced with the following program and script. The program named "eat_memory_release" allocates every 8 MB memory and releases the last 1 MB memory using madvise. The script "test_thp.sh" creates a memory cgroup, runs "eat_memory_release 500" in it and loops the proceed by 10 times. The output shows the changing of memory, which should be about 500M memory less in theory. The outputs are varying randomly when using THP, while adding "echo 2 > /proc/sys/vm/drop_caches" before accounting can avoid this. Are there any patches to fix it or is it normal by design? Thanks, Yunfang Tai --00000000000043c54105cce10702 Content-Type: text/x-c-code; charset="US-ASCII"; name="eat_release_memory.c" Content-Disposition: attachment; filename="eat_release_memory.c" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_ku0py5fu0 I2luY2x1ZGUgPHN0ZGlvLmg+CiNpbmNsdWRlIDxzdGRsaWIuaD4KI2luY2x1ZGUgPHVuaXN0ZC5o PgojaW5jbHVkZSA8c3lzL3R5cGVzLmg+CiNpbmNsdWRlIDxzeXMvbW1hbi5oPgoKCmludCBtYWlu KGludCBhcmdjLCBjaGFyKiBhcmd2W10pCnsKICAgIGNoYXIqIG1lbWluZGV4WzEwMDBdID0gezB9 OwogICAgaW50IGVhdCA9IDA7CiAgICBpbnQgd2FpdCA9IDA7CiAgICBpbnQgaSA9IDA7CgogICAg aWYgKGFyZ2MgPCAyKSAgewogICAgICAgIHByaW50ZigiVXNhZ2U6IC4vZWF0X3JlbGVhc2VfbWVt b3J5IDxudW0+ICAgI2FsbG9jYXRlIG51bSAqIDggTUIgYW5kIGZyZWUgbnVtIE1CIG1lbW9yeVxu Iik7CiAgICAgICAgcmV0dXJuOwogICAgfQoKICAgIHNzY2FuZihhcmd2WzFdLCAiJWQiLCAmZWF0 KTsKICAgIGlmIChlYXQgPD0gMCB8fCBlYXQgPj0gMTAwMCkgewogICAgICAgIHByaW50ZigibnVt IHNob3VsZCBsYXJnZXIgdGhhbiAwIGFuZCBsZXNzIHRoYW4gMTAwMFxuIik7CiAgICAgICAgcmV0 dXJuOwogICAgfQogICAgcHJpbnRmKCJBbGxvY2F0ZSBtZW1vcnkgaW4gTUIgc2l6ZTogJWRcbiIs IGVhdCAqIDgpOwoKICAgIHByaW50ZigiQWxsb2NhdGlvbiBtZW1vcnkgQmVnaW4hXG4iKTsKICAg IGZvciAoaSA9IDA7IGkgPCBlYXQ7IGkrKykgewogICAgICAgIG1lbWluZGV4W2ldID0gKGNoYXIq KW1tYXAoTlVMTCwgOCoxMDI0KjEwMjQsIFBST1RfUkVBRHxQUk9UX1dSSVRFLCBNQVBfQU5PTllN T1VTfE1BUF9QUklWQVRFLCAtMSwgMCk7CiAgICAgICAgbWVtc2V0KG1lbWluZGV4W2ldLCAwLCA4 KjEwMjQqMTAyNCk7CiAgICB9CgogICAgcHJpbnRmKCJBbGxvY2F0aW9uIG1lbW9yeSBEb25lIVxu Iik7CiAgICBzbGVlcCgyKTsKICAgIHByaW50ZigiTm93IGJlZ2luIHRvIG1hZHZpc2UgZnJlZSBt ZW1vcnkhXG4iKTsKICAgIGZvciAoaSA9IDA7IGkgPCBlYXQ7IGkrKykgewogICAgICAgIG1hZHZp c2UobWVtaW5kZXhbaV0gKyA3KjEwMjQqMTAyNCwgMTAyNCoxMDI0LCBNQURWX0RPTlRORUVEKTsK ICAgIH0KICAgIHNsZWVwKDUpOwogICAgcHJpbnRmKCJOb3cgYmVnaW4gdG8gcmVsZWFzZSBtZW1v cnkhXG4iKTsKICAgIGZvciAoaSA9IDA7IGkgPCBlYXQ7IGkrKykgewogICAgICAgIG11bm1hcCht ZW1pbmRleFtpXSwgOCoxMDI0KjEwMjQpOwogICAgfQoKfQo= --00000000000043c54105cce10702 Content-Type: application/x-sh; name="test_thp.sh" Content-Disposition: attachment; filename="test_thp.sh" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_ku0vyq7e1 IyEgL2Jpbi9iYXNoCgptZW09NTAwCm1rZGlyIC9zeXMvZnMvY2dyb3VwL21lbW9yeS90aHB0ZXN0 Cmxvb3A9MAp3aGlsZSAoKGxvb3AgPCAxMCkpOwpkbwoJY2dleGVjIC1nIG1lbW9yeTovdGhwdGVz dCAuL2VhdF9yZWxlYXNlX21lbW9yeSAkbWVtID4gL2Rldi9udWxsJgoJbnVtPWBjYXQgL3N5cy9m cy9jZ3JvdXAvbWVtb3J5L3RocHRlc3QvbWVtb3J5LnVzYWdlX2luX2J5dGVzYAoJKChudW0gPSBu dW0vMTAyNC8xMDI0KSkKCWVjaG8gImJlZm9yZSBtZW1vcnk6ICRudW0gTSIKCXNsZWVwIDUKCSNl Y2hvIDIgPiAvcHJvYy9zeXMvdm0vZHJvcF9jYWNoZXMKCW51bTE9YGNhdCAvc3lzL2ZzL2Nncm91 cC9tZW1vcnkvdGhwdGVzdC9tZW1vcnkudXNhZ2VfaW5fYnl0ZXNgCgkoKG51bTEgPSBudW0xLzEw MjQvMTAyNCkpCgllY2hvICJhZnRlciBtZW1vcnk6ICRudW0xIE0iCgkoKG51bTEgPSBudW0xIC0g bnVtKSkKCXJldD0wCgkoKHJldD1tZW0qOCAtIG51bTEpKQoJZWNobyAibWVtb3J5IGxlc3M6ICRy ZXQgTSwgc3RhbmRhcmQ6JG1lbSBNIgogICAgICAgIHdhaXQKICAgICAgICAoKGxvb3ArKykpCmRv bmUKY2dkZWxldGUgbWVtb3J5Oi90aHB0ZXN0Cg== --00000000000043c54105cce10702--