From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72E7CECAAD5 for ; Tue, 6 Sep 2022 10:37:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DC8AC80277; Tue, 6 Sep 2022 06:37:56 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D773980224; Tue, 6 Sep 2022 06:37:56 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C3F1380277; Tue, 6 Sep 2022 06:37:56 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B455D80224 for ; Tue, 6 Sep 2022 06:37:56 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 7351D160E5C for ; Tue, 6 Sep 2022 10:37:56 +0000 (UTC) X-FDA: 79881310152.19.F770B51 Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by imf16.hostedemail.com (Postfix) with ESMTP id 3080818006C for ; Tue, 6 Sep 2022 10:37:55 +0000 (UTC) Received: by mail-pl1-f169.google.com with SMTP id t3so5802911ply.2 for ; Tue, 06 Sep 2022 03:37:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to:subject :user-agent:mime-version:date:message-id:from:to:cc:subject:date; bh=Xz8bdPgR7exYL3R9SL9LcFFqWqgFZVil8ZYBWFtqEUU=; b=cA7Hl5ls5U/OscRp4DobHKMQjNA6D5C4jKC+7Av/PRDOkYb+XEGwmPKNz+0wLOS61X fhy0qVVTI5ldKJ0RhCmSDZqq/RQePtu4iAEe6umgDYn0MBNIb9KyPYXriWh+6o/CF8Ev 0SP2kzY3jn3Ut7Uwr3OOVvEv5KLxDMgFehbIVflowPFwUBRRMOLtK6Poj76c5WL10QlO UZZU0VV5LbhyEHUDIUhN0ptJM5SMq1MRRmLd4NfX9PAtqxwkwlH2NCOb6SDzLvD0tb4k HBIsZ8wCBCeGVl7izNloEowlXHHdNdvPd5dTbNnKUop2/NhG/cwTOuW0nFYVuh7pSAbQ i+6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to:subject :user-agent:mime-version:date:message-id:x-gm-message-state:from:to :cc:subject:date; bh=Xz8bdPgR7exYL3R9SL9LcFFqWqgFZVil8ZYBWFtqEUU=; b=Zy+4wCvgcpMUQ9mKBrd4X3X2Hvwcn4ee0YAMvkRx6pegwRyH+9wA+S01DexiODY501 4F9WxPN9rixjesr5tNv+zWvK+UUFHVkPm9kx5VaKpHh3hKRRSdU+VCpFmAQ30XlSjn7s 3eOjYrDAElDc91t3WB5GQ/feUimKKSN6NiG1dkeKgR2NDdvvlrJr8Bc/FKTloe2TLXRs FZ4XJTo8vpKuhDozssp7BenZpGOHcdnn6vaPsegVfrJRgLF78Dqgx5yVSMQbsAdXItv9 UXvAI5IBFLnwIBVfns9MDN/I4Q9T5hulbpvzblCbQffB+fLHCrSpSA0HLlsoLP40dLnr tSDw== X-Gm-Message-State: ACgBeo1Qmak39gOD4Gd7cKC3/S39CSQI2S/32HHgkS9rCU4l6+YcAYAx dbSBYEOt9HKnMYR/SisJtDsNCQ== X-Google-Smtp-Source: AA6agR58jEPcuQgEpaDM12qAcr3UOiSCl01qGenKN5dkhfPeBha2n6tYh8xoaVvbyMRX8m7ZKFA5dQ== X-Received: by 2002:a17:903:228c:b0:16e:df74:34e5 with SMTP id b12-20020a170903228c00b0016edf7434e5mr53043766plh.49.1662460673860; Tue, 06 Sep 2022 03:37:53 -0700 (PDT) Received: from [10.4.233.171] ([139.177.225.241]) by smtp.gmail.com with ESMTPSA id a16-20020a170902ecd000b00176db67ce98sm442737plh.3.2022.09.06.03.37.50 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 06 Sep 2022 03:37:53 -0700 (PDT) Message-ID: Date: Tue, 6 Sep 2022 18:37:40 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.13.0 Subject: Re: [External] Re: [PATCH] cgroup/cpuset: Add a new isolated mems.policy type. To: Michal Hocko Cc: hannes@cmpxchg.org, roman.gushchin@linux.dev, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, lizefan.x@bytedance.com, wuyun.abel@bytedance.com References: <20220904040241.1708-1-hezhongkun.hzk@bytedance.com> <0e5f380b-9201-0f56-9144-ce8449491fc8@bytedance.com> From: Zhongkun He In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1662460675; a=rsa-sha256; cv=none; b=q6spXnFGE5wDSCy6TfKESV8uPnVMuDrTmeWHBDtgNt1bEdl+n28hrIP3L8llFkgufa7niu EiAu8s9VDE0b0f0uloL+v5OHwP2259NFsrALp8/fMlaEKtji5QJtJ5HmaA3MehrwwoPI7v LnztPRkJi2PAiClme0vqTva+2Unujsc= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=cA7Hl5ls; spf=pass (imf16.hostedemail.com: domain of hezhongkun.hzk@bytedance.com designates 209.85.214.169 as permitted sender) smtp.mailfrom=hezhongkun.hzk@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1662460675; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Xz8bdPgR7exYL3R9SL9LcFFqWqgFZVil8ZYBWFtqEUU=; b=HorNDtwlQAYOv+xHUGc18ykIXHhPTM/qJpB+0Epn5NOBEVpBYRpACPUq9aZl9DDkGTI0VA YUjRoJzFPet+tqxNjV9R8eo0HOKeQBQidDY3kb9bejK+mAsCXUKdtho5SwOjQmK1F6oMpc drfVorAiPAJD9BdwZ82eC7CBXkC56MQ= X-Stat-Signature: 7mpgodzqq6mk6z1hn7jbtskr8wq6ag9o X-Rspamd-Queue-Id: 3080818006C X-Rspam-User: Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=cA7Hl5ls; spf=pass (imf16.hostedemail.com: domain of hezhongkun.hzk@bytedance.com designates 209.85.214.169 as permitted sender) smtp.mailfrom=hezhongkun.hzk@bytedance.com; dmarc=pass (policy=none) header.from=bytedance.com X-Rspamd-Server: rspam08 X-HE-Tag: 1662460674-641054 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: > On Mon 05-09-22 18:30:55, Zhongkun He wrote: >> Hi Michal, thanks for your reply. >> >> The current 'mempolicy' is hierarchically independent. The default value of >> the child is to inherit from the parent. The modification of the child >> policy will not be restricted by the parent. > > This breaks cgroup fundamental property of hierarchical enforcement of > each property. And as such it is a no go. > >> Of course, there are other options, such as the child's policy mode must be >> the same as the parent's. node can be the subset of parent's, but the >> interleave type will be complicated, that's why hierarchy independence is >> used. It would be better if you have other suggestions? > > Honestly, I am not really sure cgroup cpusets is a great fit for this > usecase. It would be probably better to elaborate some more what are the > existing shortcomings and what you would like to achieve. Just stating > the syscall is a hard to use interface is not quite clear on its own. > > Btw. have you noticed this question? > >>> What is the hierarchical behavior of the policy? Say parent has a >>> stronger requirement (say bind) than a child (prefer)? >>>> How to use the mempolicy interface: >>>> echo prefer:2 > /sys/fs/cgroup/zz/cpuset.mems.policy >>>> echo bind:1-3 > /sys/fs/cgroup/zz/cpuset.mems.policy >>>> echo interleave:0,1,2,3 >/sys/fs/cgroup/zz/cpuset.mems.policy >>> >>> Am I just confused or did you really mean to combine all these >>> together? > Hi Michal, thanks for your reply. >>Say parent has a stronger requirement (say bind) than a child(prefer)? Yes, combine all these together. The parent's task will use 'bind', child's use 'prefer'.This is the current implementation, and we can discuss and modify it together if there are other suggestions. 1:Existing shortcomings In our use case, the application and the control plane are two separate systems. When the application is created, it doesn't know how to use memory, and it doesn't care. The control plane will decide the memory usage policy based on different reasons (the attributes of the application itself, the priority, the remaining resources of the system). Currently, numactl is used to set it at program startup, and the child process will inherit the mempolicy. But we can't dynamically adjust the memory policy, except restart, the memory policy will not change. 2:Our goals For the above reasons, we want to create a mempolicy at the cgroup level. Usually processes under a cgroup have the same priority and attributes, and we can dynamically adjust the memory allocation strategy according to the remaining resources of the system. For example, a low-priority cgroup uses the 'bind:2-3' policy, and a high-priority cgroup uses bind:0-1. When resources are insufficient, it will be changed to bind:3, bind:0-2 by control plane, etc.Further more, more mempolicy can be extended, such as allocating memory according to node weight, etc. Thanks.