From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66264C07E97 for ; Sat, 3 Jul 2021 16:18:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id E9C2D61938 for ; Sat, 3 Jul 2021 16:18:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E9C2D61938 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 344A56B0071; Sat, 3 Jul 2021 12:18:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2CDC58D0001; Sat, 3 Jul 2021 12:18:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 120126B0073; Sat, 3 Jul 2021 12:18:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0129.hostedemail.com [216.40.44.129]) by kanga.kvack.org (Postfix) with ESMTP id DBE706B0071 for ; Sat, 3 Jul 2021 12:18:39 -0400 (EDT) Received: from smtpin33.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 83CC08248047 for ; Sat, 3 Jul 2021 16:18:39 +0000 (UTC) X-FDA: 78321784758.33.37A182A Received: from mail-ej1-f48.google.com (mail-ej1-f48.google.com [209.85.218.48]) by imf29.hostedemail.com (Postfix) with ESMTP id 2C92E90001BC for ; Sat, 3 Jul 2021 16:18:39 +0000 (UTC) Received: by mail-ej1-f48.google.com with SMTP id gn32so21665671ejc.2 for ; Sat, 03 Jul 2021 09:18:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=/SFx0l3EFfXNlGfyI4GVdZzRabxV5Q5+sQtiSDXNWyA=; b=Y89WEwsifC2v0VdErRLJjYeyJmIWBXHrtZzhT6IZHQdx/2Nsemmeil8SulvH9KRj22 A0Z4KglyhDv0BfV7EQtbvIGrJd6mAagDmlsICgrwU5UCwcSFBcz/ZNLIh/YNAaxYbqM0 7Y3SJFoXL6FokOSyo/uTlcMKTOk+lfmlRIn6oDp9MqwiiFZRpYHHtcaceEaWiTrKZxQj clRa0/2TAIx0uSBeawIVdmNqtU2GiIjfR6AQBbCIZIR1+tayHXy02exYi5VQNqPzKnbU fBY3jKfCTAPHStRotUekp2kOWYZ3Ml8xGNHDMLpd7CRelawB2FnloxzSY0Opt5YAojyg UpKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=/SFx0l3EFfXNlGfyI4GVdZzRabxV5Q5+sQtiSDXNWyA=; b=gSQ7iQ+7m80enHIr5wYGWm7l5dNssqZ9PA65VTgyD0JEKkHNkYis8BkBR4w/6iqxxS jVwrzB8CWV7MxEML+IwKrtCScEm/yx6EA7cxsYm38UEGVbfFFXSBXZIUbrGAAnzGO2d7 uovGGkpos9o0Eyb4wvJzXocLvEGPOlqf+i8f+L/c0xYxGlEtY+sb19rqeRpKl+f6RJ8/ 2/BvRRFDP24oafhomszZOwGPl/rMBZ3tIxdz/XS0jdkxGfQs+yp3MQsN/3tYHQxDq6wE tS85gJ8MWDasTNrN7SnU2MwgfTiNtvtjfhlgp+4QSMTAcICpNYdO3Ad9H616Vf8MHj2W tW4w== X-Gm-Message-State: AOAM530f8AG55aa6phattaKZ4qRA/PZWJy9dbO5ODq7/wwE7DtqzSHCM /KRlXpPeObAd0ZwZB/mGdzZJL0fFs377iI/nyw8= X-Google-Smtp-Source: ABdhPJyIW/gHmhWZpraIEnHpbIr8igMFbgS3aJkgvRtJleUmEjWgSls5GThsHsT/TQjnxCxv5qi9q8Jw2YfdfeIdlIA= X-Received: by 2002:a17:906:15c2:: with SMTP id l2mr5335696ejd.348.1625329116880; Sat, 03 Jul 2021 09:18:36 -0700 (PDT) MIME-Version: 1.0 References: <60decdb6.1c69fb81.6130e.7642@mx.google.com> <20210702200742.wuhdg4dhpolher3t@oracle.com> In-Reply-To: <20210702200742.wuhdg4dhpolher3t@oracle.com> From: Hao Lee Date: Sun, 4 Jul 2021 00:18:22 +0800 Message-ID: Subject: Re: [Question] Do we need remote charging for cpu and cpuacct subsys? To: Daniel Jordan Cc: linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, tj@kernel.org Content-Type: text/plain; charset="UTF-8" Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20161025 header.b=Y89WEwsi; spf=pass (imf29.hostedemail.com: domain of haoleeswjtu@gmail.com designates 209.85.218.48 as permitted sender) smtp.mailfrom=haoleeswjtu@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 2C92E90001BC X-Stat-Signature: hfzgufxto7xoucmc39um4yw4asth6fzc X-HE-Tag: 1625329119-5039 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000003, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sat, Jul 3, 2021 at 4:07 AM Daniel Jordan wrote: > > Hello, > > On Fri, Jul 02, 2021 at 08:26:27AM -0000, Hao Lee wrote: > > memcg currently has a remote charging mechanism that can charge usage to other > > memcg instead of the one the task belongs to. > > > > In our environment, we need to account the cpu usage consumed by some kworkers > > to a specific cgroup. Thus, we want to introduce a remote-charging mechanism to > > cpu and cpuacct subsys in our kernel. > > I also want to see this upstream, and am actually working on it right > now, have been for some time. > > So far, this is needed to properly account multithreaded padata jobs, > memory reclaim, and net rx. Android folks have raised this issue in the > past too, though I'm not aware of the specific kthreads that are giving > them problems. > > So naturally, I'm curious about your use case and how it may be > different from these others. What kworkers would you like to account? Thanks. We use a workqueue to perform asynchronous reclaim for cgroups. The kworker may consume lots of CPU cycles if the cgroup memory pressure is extremely high, so we want to charge the cpu usage to the related cgroup for which the kworker works. Otherwise, the reclaim kworker will steal cpu time from the system level, which breaks the resource isolation. > > > I want to know if the community has a plan to do this? > > What will the community approach look like? > > There has been discussion about this here, > > https://lore.kernel.org/lkml/20200219214112.4kt573kyzbvmbvn3@ca-dmjordan1.us.oracle.com/ > Yes, our work is very similar to Johannes'. > more recently here, > > https://lore.kernel.org/lkml/YGxjwKbec68sCcqo@slm.duckdns.org/ > > and we may talk about it at LPC: > > https://www.linuxplumbersconf.org/event/11/page/104-accepted-microconferences#cont-perform I also found that you posted a patchset in 2019 to introduce a cgroup-aware workqueue. In that discussion, back-charging is considered to be a suitable solution. https://lore.kernel.org/lkml/20190611195549.GL3341036@devbig004.ftw2.facebook.com/ I also have a question here. Are the back-charging and remote charging the same thing? > > > I think we need to move the active_memcg to a separated active_cgroup struct, > > and the latter will contain active_memcg, active_tg, and active_cpuacct. > > I'm not seeing how that could work for cases that don't know the cgroup > when the remote charging period begins. This is indeed a problem. Neither cgroup-aware kthread nor remote charging can address this. Maybe this is the biggest obstacle hindering fine-grained charging. > The only one I'm aware of > that's like that is net rx, where the work to process packets has to > start before their ultimate destination, and therefore cgroup, is known. Sorry. Is this a typo? It seems the word "known" should be "unknown"... Regards, Hao Lee > > thanks, > Daniel