From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4FF24C352A1 for ; Wed, 7 Dec 2022 13:00:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9864E8E0003; Wed, 7 Dec 2022 08:00:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 935BC8E0001; Wed, 7 Dec 2022 08:00:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7D65E8E0003; Wed, 7 Dec 2022 08:00:44 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 6B4658E0001 for ; Wed, 7 Dec 2022 08:00:44 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 38FB240127 for ; Wed, 7 Dec 2022 13:00:44 +0000 (UTC) X-FDA: 80215519608.10.4E66E33 Received: from mail-wr1-f48.google.com (mail-wr1-f48.google.com [209.85.221.48]) by imf19.hostedemail.com (Postfix) with ESMTP id 5CC551A001F for ; Wed, 7 Dec 2022 13:00:43 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=cmpxchg-org.20210112.gappssmtp.com header.s=20210112 header.b=yeFZCjZB; spf=pass (imf19.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.221.48 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1670418043; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xR1noXbyPM5kt/sYfkF4dJpSqyLI733A+gni4V3Hl+g=; b=mI5QEwoWrMZp4Ex8+JTfzXLpNtnFWUZJRuF96YpUNrH8ZUfNcz/kX0qdSlyXIgis+Du0Rv X7OnghuqgME+p4VNmnCCFL6+JUgvwOOtHieKBpvq6d4KCHuxbMVFlMvny8Aah6fpvqLqmE ldkHSh4uGRm7V5s9EU4kKf9IqWsJYDY= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=cmpxchg-org.20210112.gappssmtp.com header.s=20210112 header.b=yeFZCjZB; spf=pass (imf19.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.221.48 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1670418043; a=rsa-sha256; cv=none; b=uMLLGw7vf05YwvYQVc202rMV1+cyf8fsSWQYXvqhAeFAvNAy2GJvGR5dHThxGgymP8HLCW aKMsEgnmIUGV/DZ8orbkCOzGztt3TGsXNGGMMNtF8hYbZgfkqIeftcLj5tTumHjX63UCvG AMXdJGoUqQxpNLWmcPjnytRfRA/Sch4= Received: by mail-wr1-f48.google.com with SMTP id f18so27974246wrj.5 for ; Wed, 07 Dec 2022 05:00:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20210112.gappssmtp.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=xR1noXbyPM5kt/sYfkF4dJpSqyLI733A+gni4V3Hl+g=; b=yeFZCjZBCI9Jl2kn0mMwm5h3PCNQV0qMVvCmJLEPczp/jvmDAWIfUZZZbfXW/EXohY p2XOzLBhN+FZBTdxRVYo44n24EZiyP96Jx5kd/7p3Hc+l8wdcRpierf6Dp49w8DJUQDy rlBFc+EHmIFOsMjJqc5U/27MwGIGi1dyA0aOpjKrEGMwSdLG15MR/hFUANlgdcLoVSMy tGxtn8JsPnRt/XqqdlJcxbVGkDLHo+cYI60OVm5g44w4aaSGFBdCTWJ5kcy1eAG75Mkg tJLpMweFuT/1HwNNf/oHnBgCm87mrMqKt+FPX76ImHI3UkxCCwnkVWry+O/oi/fCry9i XXNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=xR1noXbyPM5kt/sYfkF4dJpSqyLI733A+gni4V3Hl+g=; b=sbEI3zobbSaPl4QUWLXsJXZWJjYca+Nmx5rSmS3FGcPzVKdZ8OV4iU4p1KcR6RzUnm +0iKhpz1y8Q/YCa+6Gn4kmcbIPJf1LSK6KFJqe/d7l+0mPSS4PH2YoYrNlJDzh/xtyXv 5IbnzWZHVUzVrTcNELWA6vVfnUTiY4SO49mIaJxrw86xS8gBkPDfhCyR3U4rkLe1aHEH DH233/J4oM9Eys7OlvMdm7ZS+awyrwdNbSzOAitEv/WY2ndjqruZwCbyH2Oo21l84aen 8vezpzyk+4MJnHrd4RaMN7cFqE4RNj+gfPyMKQCGk9fQE0boqQ3zGTE7fmi59uSKpFZM uShg== X-Gm-Message-State: ANoB5pnE5VwpeVJMTjB1osQ2Lq1TbIM2TOmbxsHDO+cu7aBu5qwkiapW 3x96KliNmOt1J6ObvCW0RiIcjQ== X-Google-Smtp-Source: AA0mqf5ZqtIW3Nlw0Hn0bcSCxhPju08BUFN33Uu0nIt2jrv6oohoW/Xp5yzOalr4pHN+P7tCxG+4GA== X-Received: by 2002:a05:6000:1d92:b0:241:6e0a:bfe6 with SMTP id bk18-20020a0560001d9200b002416e0abfe6mr47301524wrb.34.1670418041694; Wed, 07 Dec 2022 05:00:41 -0800 (PST) Received: from localhost (ip-046-005-139-011.um12.pools.vodafone-ip.de. [46.5.139.11]) by smtp.gmail.com with ESMTPSA id t14-20020a05600c198e00b003cfd4a50d5asm1645753wmq.34.2022.12.07.05.00.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Dec 2022 05:00:41 -0800 (PST) Date: Wed, 7 Dec 2022 14:00:39 +0100 From: Johannes Weiner To: Hugh Dickins Cc: Andrew Morton , Linus Torvalds , Shakeel Butt , Michal Hocko , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 3/3] mm: memcontrol: deprecate charge moving Message-ID: References: <20221206171340.139790-1-hannes@cmpxchg.org> <20221206171340.139790-4-hannes@cmpxchg.org> <02b9663-4377-bd67-8da2-aad72240da@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <02b9663-4377-bd67-8da2-aad72240da@google.com> X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 5CC551A001F X-Rspam-User: X-Stat-Signature: 6dt6i96ze3bmjxwmbbcpk7xpj4deqdm6 X-Spamd-Result: default: False [-2.90 / 9.00]; BAYES_HAM(-6.00)[100.00%]; SORBS_IRL_BL(3.00)[209.85.221.48:from]; MIME_GOOD(-0.10)[text/plain]; RCVD_NO_TLS_LAST(0.10)[]; BAD_REP_POLICIES(0.10)[]; TO_DN_SOME(0.00)[]; RCPT_COUNT_SEVEN(0.00)[8]; DMARC_POLICY_ALLOW(0.00)[cmpxchg.org,none]; DKIM_TRACE(0.00)[cmpxchg-org.20210112.gappssmtp.com:+]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; R_SPF_ALLOW(0.00)[+ip4:209.85.128.0/17]; RCVD_COUNT_THREE(0.00)[3]; R_DKIM_ALLOW(0.00)[cmpxchg-org.20210112.gappssmtp.com:s=20210112]; ARC_SIGNED(0.00)[hostedemail.com:s=arc-20220608:i=1]; PREVIOUSLY_DELIVERED(0.00)[linux-mm@kvack.org]; RCVD_VIA_SMTP_AUTH(0.00)[] X-HE-Tag: 1670418043-649018 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Dec 06, 2022 at 05:58:14PM -0800, Hugh Dickins wrote: > On Tue, 6 Dec 2022, Johannes Weiner wrote: > > > Charge moving mode in cgroup1 allows memory to follow tasks as they > > migrate between cgroups. This is, and always has been, a questionable > > thing to do - for several reasons. > > > > First, it's expensive. Pages need to be identified, locked and > > isolated from various MM operations, and reassigned, one by one. > > > > Second, it's unreliable. Once pages are charged to a cgroup, there > > isn't always a clear owner task anymore. Cache isn't moved at all, for > > example. Mapped memory is moved - but if trylocking or isolating a > > page fails, it's arbitrarily left behind. Frequent moving between > > domains may leave a task's memory scattered all over the place. > > > > Third, it isn't really needed. Launcher tasks can kick off workload > > tasks directly in their target cgroup. Using dedicated per-workload > > groups allows fine-grained policy adjustments - no need to move tasks > > and their physical pages between control domains. The feature was > > never forward-ported to cgroup2, and it hasn't been missed. > > > > Despite it being a niche usecase, the maintenance overhead of > > supporting it is enormous. Because pages are moved while they are live > > and subject to various MM operations, the synchronization rules are > > complicated. There are lock_page_memcg() in MM and FS code, which > > non-cgroup people don't understand. In some cases we've been able to > > shift code and cgroup API calls around such that we can rely on native > > locking as much as possible. But that's fragile, and sometimes we need > > to hold MM locks for longer than we otherwise would (pte lock e.g.). > > > > Mark the feature deprecated. Hopefully we can remove it soon. > > > > Signed-off-by: Johannes Weiner > > Acked-by: Hugh Dickins Thanks > but I wonder if it would be helpful to mention move_charge_at_immigrate > in the deprecation message: maybe the first line should be > "Cgroup memory moving (move_charge_at_immigrate) is deprecated.\n" Fair enough! Here is the updated patch. --- >From 0e791e6ab8ba2f75dd4205684c06bcc7308d9867 Mon Sep 17 00:00:00 2001 From: Johannes Weiner Date: Mon, 5 Dec 2022 19:57:06 +0100 Subject: [PATCH] mm: memcontrol: deprecate charge moving Charge moving mode in cgroup1 allows memory to follow tasks as they migrate between cgroups. This is, and always has been, a questionable thing to do - for several reasons. First, it's expensive. Pages need to be identified, locked and isolated from various MM operations, and reassigned, one by one. Second, it's unreliable. Once pages are charged to a cgroup, there isn't always a clear owner task anymore. Cache isn't moved at all, for example. Mapped memory is moved - but if trylocking or isolating a page fails, it's arbitrarily left behind. Frequent moving between domains may leave a task's memory scattered all over the place. Third, it isn't really needed. Launcher tasks can kick off workload tasks directly in their target cgroup. Using dedicated per-workload groups allows fine-grained policy adjustments - no need to move tasks and their physical pages between control domains. The feature was never forward-ported to cgroup2, and it hasn't been missed. Despite it being a niche usecase, the maintenance overhead of supporting it is enormous. Because pages are moved while they are live and subject to various MM operations, the synchronization rules are complicated. There are lock_page_memcg() in MM and FS code, which non-cgroup people don't understand. In some cases we've been able to shift code and cgroup API calls around such that we can rely on native locking as much as possible. But that's fragile, and sometimes we need to hold MM locks for longer than we otherwise would (pte lock e.g.). Mark the feature deprecated. Hopefully we can remove it soon. Signed-off-by: Johannes Weiner Acked-by: Shakeel Butt Acked-by: Hugh Dickins Cc: stable@vger.kernel.org --- Documentation/admin-guide/cgroup-v1/memory.rst | 11 ++++++++++- mm/memcontrol.c | 4 ++++ 2 files changed, 14 insertions(+), 1 deletion(-) diff --git a/Documentation/admin-guide/cgroup-v1/memory.rst b/Documentation/admin-guide/cgroup-v1/memory.rst index 60370f2c67b9..87d7877b98ec 100644 --- a/Documentation/admin-guide/cgroup-v1/memory.rst +++ b/Documentation/admin-guide/cgroup-v1/memory.rst @@ -86,6 +86,8 @@ Brief summary of control files. memory.swappiness set/show swappiness parameter of vmscan (See sysctl's vm.swappiness) memory.move_charge_at_immigrate set/show controls of moving charges + This knob is deprecated and shouldn't be + used. memory.oom_control set/show oom controls. memory.numa_stat show the number of memory usage per numa node @@ -717,9 +719,16 @@ Soft limits can be setup by using the following commands (in this example we It is recommended to set the soft limit always below the hard limit, otherwise the hard limit will take precedence. -8. Move charges at task migration +8. Move charges at task migration (DEPRECATED!) ================================= +THIS IS DEPRECATED! + +It's expensive and unreliable! It's better practice to launch workload +tasks directly from inside their target cgroup. Use dedicated workload +cgroups to allow fine-grained policy adjustments without having to +move physical pages between control domains. + Users can move charges associated with a task along with task migration, that is, uncharge task's pages from the old cgroup and charge them to the new cgroup. This feature is not supported in !CONFIG_MMU environments because of lack of diff --git a/mm/memcontrol.c b/mm/memcontrol.c index b696354c1b21..9c9a42153b76 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -3919,6 +3919,10 @@ static int mem_cgroup_move_charge_write(struct cgroup_subsys_state *css, { struct mem_cgroup *memcg = mem_cgroup_from_css(css); + pr_warn_once("Cgroup memory moving (move_charge_at_immigrate) is deprecated. " + "Please report your usecase to linux-mm@kvack.org if you " + "depend on this functionality.\n"); + if (val & ~MOVE_MASK) return -EINVAL; -- 2.38.1