From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34674C433E2 for ; Tue, 21 Jul 2020 14:47:19 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DE53B20717 for ; Tue, 21 Jul 2020 14:47:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="RJPk4UVm" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DE53B20717 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6E5FC6B0005; Tue, 21 Jul 2020 10:47:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 695906B0006; Tue, 21 Jul 2020 10:47:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 584D86B0007; Tue, 21 Jul 2020 10:47:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0066.hostedemail.com [216.40.44.66]) by kanga.kvack.org (Postfix) with ESMTP id 3FC056B0005 for ; Tue, 21 Jul 2020 10:47:18 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 8BFA3181C65E5 for ; Tue, 21 Jul 2020 14:47:17 +0000 (UTC) X-FDA: 77062360914.22.snow12_400582c26f2e Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin22.hostedemail.com (Postfix) with ESMTP id 5351418193421 for ; Tue, 21 Jul 2020 14:45:28 +0000 (UTC) X-HE-Tag: snow12_400582c26f2e X-Filterd-Recvd-Size: 4895 Received: from mail-lf1-f68.google.com (mail-lf1-f68.google.com [209.85.167.68]) by imf28.hostedemail.com (Postfix) with ESMTP for ; Tue, 21 Jul 2020 14:45:27 +0000 (UTC) Received: by mail-lf1-f68.google.com with SMTP id j21so11833665lfe.6 for ; Tue, 21 Jul 2020 07:45:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=0pAwByUR93Lcw7763OHJXsSvUvLTrNih6qhx4gjT+aM=; b=RJPk4UVm3QG1OSNEE3CsSct0ln7CZy0uNFBtNkbOJqVixfV+ABWf25M4H+TnYcPfce Ta+awLrH5yk/hEx+EREuJMfro+hKQpGe5CFLuXIivkB9MRt63Mgki6+ZSqsYmq60+KGR saMyvxi2Hmz1v1xQI2GSgfDNxXTFo1iMzKVWjBJjRBqfCJ1CmI8KNgJBdaDfPJ2tuRhc 1B2BJWsw4BWQe4cDBiWCe26f3Sag/jnK6MVlPixWUeRZaxdJW6M9dd3rU+dU/oYLU+7I NhW4hltGvXPB7wF1uCgYh08R3vYJ9kwAMy5b5J9dOBkBQSe9QqK7WkCSf+m3yniK49H9 NTuA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=0pAwByUR93Lcw7763OHJXsSvUvLTrNih6qhx4gjT+aM=; b=qn1NviN4pQ40bPA1Se82chU7BkLsXFf5dY0+amXZakXaI/oZk3E99U246j22TBPGHS uojn6rwrMGU/n7fIhtq0EmrO/m4Xbxfg8gTNuIS3BtWUwvaDMLni9L1CHFpRK5LTzLJm oWVW2l0N9AfJSJ7mASkbrqgBa6yoZOYEucOvzeWTMhgX1MFXb0zTWHYNXPVnDhzpRf/o ZUJBRBwep4v60s2ks5yVfQ0Oq0pDpdXAfrbpUyIInwMztmtNHGdERGa2RsOTjQlrMs4U Sew54X+WiIUuBGFYeI0+sbssQwcfyCfeQlbenTKEtg2oV/JWf/OVedpwWN6n1x8xK/yq aaaA== X-Gm-Message-State: AOAM533RH094QduyNP6tn193C0K/xtujv0BWS8litTesXLUD1REgndDl DJK0ypj2uHMQF0qEK2fo0qznLIunafEaeQhb6b2xhA== X-Google-Smtp-Source: ABdhPJx+6QMWrXHBeyyEBoTcyZr8xV8VGVPYDS7ORai+toxRwj0nYcBfTYvA3wm86GBp4I6uRaZTjfvBKCYXxnSWba4= X-Received: by 2002:ac2:4183:: with SMTP id z3mr10668274lfh.3.1595342725776; Tue, 21 Jul 2020 07:45:25 -0700 (PDT) MIME-Version: 1.0 References: <2E04DD7753BE0E4ABABF0B664610AD6F2620CAF7@dggeml528-mbx.china.huawei.com> In-Reply-To: <2E04DD7753BE0E4ABABF0B664610AD6F2620CAF7@dggeml528-mbx.china.huawei.com> From: Shakeel Butt Date: Tue, 21 Jul 2020 07:45:14 -0700 Message-ID: Subject: Re: PROBLEM: cgroup cost too much memory when transfer small files to tmpfs To: jingrui Cc: "tj@kernel.org" , Lizefan , "hannes@cmpxchg.org" , "mhocko@kernel.org" , "vdavydov.dev@gmail.com" , "akpm@linux-foundation.org" , "linux-mm@kvack.org" , "cgroups@vger.kernel.org" , "linux-kernel@vger.kernel.org" , caihaomin , "Weiwei (N)" Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 5351418193421 X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam05 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Jul 21, 2020 at 4:20 AM jingrui wrote: > > Cc: Johannes Weiner ; Michal Hocko ; Vladimir Davydov > > Thanks. > > --- > PROBLEM: cgroup cost too much memory when transfer small files to tmpfs. > > keywords: cgroup PERCPU/memory cost too much. > > description: > > We send small files from node-A to node-B tmpfs /tmp directory using sftp. On > node-B the systemd configured with pam on like below. > > cat /etc/pam.d/password-auth | grep systemd > -session optional pam_systemd.so > > So when transfer a file, a systemd session is created, that means a cgroup is > created, then file saved at /tmp will associated with a cgroup object. After > file transferred, session and cgroup-dir will be removed, but the file in /tmp > still associated with the cgroup object. Is there a way for you to re-use the cgroup instead of creating and deleting cgroup for each individual file transfer session? > The PERCPU memory in cgroup/css object > cost a lot(about 0.5MB/per-cgroup-object) on 200/cpus machine. > > When lot of small files transferred to tmpfs, the cgroup/css object memory > cost become huge in this scenes to be used. > > systemd related issue: https://github.com/systemd/systemd/issues/16499 > > kernel version: 4.19+ > > Problem: > > 1. Do we have any idea to descrease cgroup memory cost in this case? > 2. When user remove cgroup directory, does it possible associated file memory to root cgroup? No, the memory remains associated with the cgroup and the cgroup becomes zombie on deletion. > 3. Can we provide an option that do not associate memory with cgroup in tmpfs? Only way, if you don't want to disable memcg, is to move the file receiver process to root cgroup.