From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 33B66C433EF for ; Tue, 10 May 2022 01:43:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6F4606B0072; Mon, 9 May 2022 21:43:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6A2676B0073; Mon, 9 May 2022 21:43:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 51D7D6B0074; Mon, 9 May 2022 21:43:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 41F756B0072 for ; Mon, 9 May 2022 21:43:52 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 1B9F361562 for ; Tue, 10 May 2022 01:43:52 +0000 (UTC) X-FDA: 79448137104.29.C709332 Received: from mail-pg1-f178.google.com (mail-pg1-f178.google.com [209.85.215.178]) by imf16.hostedemail.com (Postfix) with ESMTP id 4BCAA1800AC for ; Tue, 10 May 2022 01:43:43 +0000 (UTC) Received: by mail-pg1-f178.google.com with SMTP id v10so13434076pgl.11 for ; Mon, 09 May 2022 18:43:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:date:from:to:cc:subject:references:mime-version :content-disposition:in-reply-to; bh=HgGTA2L0WTR4KLBkAGFUu4sFQBHJ/DJdCql0JMw+UdA=; b=PQtM78l2Szcsfpy4oz0zTJGNYjLHXrbEDGhoEKp+SP/Pa1siwRO+rDz3t0BFl3resm dd8tcXGT5i6a82bz80DTxIKuLb3u8lAwq8yUsRIx0GhudcTqTvxtzIEeBaOjbjpCitNq q+VyfCneFhYGz4mP2xJ3gYDnKji0GVSkfrT2Oee3fXek4xuIo880FhrxH+lMXsrr5xwx Nq1pIdHfa41nMi5miCZmzs4D/xeLesHo5FfIvycHlMyBPbIUJeXhmdjr3j9zvs+wYUH5 o6+K3hpdHvKaBvRM4E/gsKQoEwDwy4zBLdaqaK3c6sqnny9rfkbdfe6hT5+LSAR8EzcN /8NQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:from:to:cc:subject:references :mime-version:content-disposition:in-reply-to; bh=HgGTA2L0WTR4KLBkAGFUu4sFQBHJ/DJdCql0JMw+UdA=; b=0p3EBqaZ/rcmcJcmZasKjOhLvwb7W5Opq7n5nZpgkyfW0O9h3SnS7IhU1e1SzWYP6p aGnE77ICLg0VkGkypnB+mHSRbnuvZIoys/19SdUGuyQRmms9zZUXVDPQJwd5IJ60p0/9 J01UpNLrTRrFTSKWFSCJmqkzDZ+n79N6LGYBYRuYDkAok4of2KI41YQ8sveD5A3AOXJh fQ4T46bxyRDCa5rHYZAX1sdw/PkQfU0LK1uohN6TwzU+iXmxvPztXAdAbkeZvhjv0MQF lbFW6Wina3V8lhRUjmElOjAIy8gdpvYC6+xclkzh1Yyaxz9FIbt7+MXEkZLXTNr/qKwM ySMg== X-Gm-Message-State: AOAM53035+upZMCTEtNNRsMFOq55NvCw37SL3odAURXjFyuQM9p2gnfB tFt482i+IoojycYF+Vn3Xfo= X-Google-Smtp-Source: ABdhPJy2FOePOQ00ab2y0snBDHiAx1OOUXyfOevwooeooZCPtaDvJEiNPq0H1xGkxOIzZYF9ARPTVw== X-Received: by 2002:a65:6e41:0:b0:39c:c97b:2aef with SMTP id be1-20020a656e41000000b0039cc97b2aefmr15690480pgb.473.1652147030614; Mon, 09 May 2022 18:43:50 -0700 (PDT) Received: from localhost ([193.203.214.57]) by smtp.gmail.com with ESMTPSA id g1-20020a17090a7d0100b001d93118827asm380903pjl.57.2022.05.09.18.43.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 May 2022 18:43:48 -0700 (PDT) Message-ID: <6279c354.1c69fb81.7f6c1.15e0@mx.google.com> X-Google-Original-Message-ID: <20220510014338.GA1267733@cgel.zte@gmail.com> Date: Tue, 10 May 2022 01:43:38 +0000 From: CGEL To: Michal Hocko Cc: akpm@linux-foundation.org, hannes@cmpxchg.org, willy@infradead.org, shy828301@gmail.com, roman.gushchin@linux.dev, shakeelb@google.com, linmiaohe@huawei.com, william.kucharski@oracle.com, peterx@redhat.com, hughd@google.com, vbabka@suse.cz, songmuchun@bytedance.com, surenb@google.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Yang Yang Subject: Re: [PATCH] mm/memcg: support control THP behaviour in cgroup References: <20220505033814.103256-1-xu.xin16@zte.com.cn> <6275d3e7.1c69fb81.1d62.4504@mx.google.com> <6278fa75.1c69fb81.9c598.f794@mx.google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=PQtM78l2; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf16.hostedemail.com: domain of cgel.zte@gmail.com designates 209.85.215.178 as permitted sender) smtp.mailfrom=cgel.zte@gmail.com X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 4BCAA1800AC X-Rspam-User: X-Stat-Signature: tf9btj3x5kpe5p37qcsha6oj95sxt6ej X-HE-Tag: 1652147023-857165 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, May 09, 2022 at 01:48:39PM +0200, Michal Hocko wrote: > On Mon 09-05-22 11:26:43, CGEL wrote: > > On Mon, May 09, 2022 at 12:00:28PM +0200, Michal Hocko wrote: > > > On Sat 07-05-22 02:05:25, CGEL wrote: > > > [...] > > > > If there are many containers to run on one host, and some of them have high > > > > performance requirements, administrator could turn on thp for them: > > > > # docker run -it --thp-enabled=always > > > > Then all the processes in those containers will always use thp. > > > > While other containers turn off thp by: > > > > # docker run -it --thp-enabled=never > > > > > > I do not know. The THP config space is already too confusing and complex > > > and this just adds on top. E.g. is the behavior of the knob > > > hierarchical? What is the policy if parent memcg says madivise while > > > child says always? How does the per-application configuration aligns > > > with all that (e.g. memcg policy madivise but application says never via > > > prctl while still uses some madvised - e.g. via library). > > > > > > > The cgroup THP behavior is align to host and totally independent just likes > > /sys/fs/cgroup/memory.swappiness. That means if one cgroup config 'always' > > for thp, it has no matter with host or other cgroup. This make it simple for > > user to understand or control. > > All controls in cgroup v2 should be hierarchical. This is really > required for a proper delegation semantic. > Could we align to the semantic of /sys/fs/cgroup/memory.swappiness? Some distributions like Ubuntu is still using cgroup v1. > > If memcg policy madivise but application says never, just like host, the result > > is no THP for that application. > > > > > > By doing this we could promote important containers's performance with less > > > > footprint of thp. > > > > > > Do we really want to provide something like THP based QoS? To me it > > > sounds like a bad idea and if the justification is "it might be useful" > > > then I would say no. So you really need to come with a very good usecase > > > to promote this further. > > > > At least on some 5G(communication technology) machine, it's useful to provide > > THP based QoS. Those 5G machine use micro-service software architecture, in > > other words one service application runs in one container. > > I am not really sure I understand. If this is one application per > container (cgroup) then why do you really need per-group setting? > Does the application is a set of different processes which are only very > loosely tight? > For micro-service architecture, the application in one container is not a set of loosely tight processes, it's aim at provide one certain service, so different containers means different service, and different service has different QoS demand. The reason why we need per-group(per-container) setting is because most container are managed by compose software, the compose software provide UI to decide how to run a container(likes setting swappiness value). For example the docker compose: https://docs.docker.com/compose/#compose-v2-and-the-new-docker-compose-command To make it clearer, I try to make a summary for why container needs this patch: 1.one machine can run different containers; 2.for some scenario, container runs only one service inside(can be only one application); 3.different containers provide different services, different services have different QoS demands; 4.THP has big influence on QoS. It's fast for memory access, but eat more memory; 5.containers usually managed by compose software, which treats container as base management unit; 6.this patch provide cgroup THP controller, which can be a method to adjust container memory QoS. > > Container becomes > > the suitable management unit but not the whole host. And some performance > > sensitive containers desiderate THP to provide low latency communication. > > But if we use THP with 'always', it will consume more memory(on our machine > > that is about 10% of total memory). And unnecessary huge pages will increase > > memory pressure, add latency for minor pages faults, and add overhead when > > splitting huge pages or coalescing normal sized pages into huge pages. > > It is still not really clear to me how do you achieve that the whole > workload in the said container has the same THP requirements. > -- > Michal Hocko > SUSE Labs