From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44FDAC02183 for ; Wed, 15 Jan 2025 01:33:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CCB3C6B007B; Tue, 14 Jan 2025 20:33:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C547C6B0082; Tue, 14 Jan 2025 20:33:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ACDDE6B0083; Tue, 14 Jan 2025 20:33:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 8B6A96B007B for ; Tue, 14 Jan 2025 20:33:23 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 43063B030F for ; Wed, 15 Jan 2025 01:33:23 +0000 (UTC) X-FDA: 83007963486.17.FE2A8C8 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) by imf25.hostedemail.com (Postfix) with ESMTP id 49C08A0003 for ; Wed, 15 Jan 2025 01:33:21 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=GTERuukV; spf=pass (imf25.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.171 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736904801; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=d9P5/KEhDafJlP2HFwU6Lo/haCAgWGhZE6rmWKAwwYg=; b=mpttFL2C2a5ypYpuPmNrNnmJa9dZqTYMt5JWwHYIYpNyhsmCzI9p6DIEPySulUCPMkJaVF kjCtSM397kCD+XFqDAdd+REcA577JdIF1G1c9OH1K0Z89yZO2khexgVeNn/jP1fYznTwg7 GvzkllUrzrRMH+PrJQDunFK886z+L5c= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=GTERuukV; spf=pass (imf25.hostedemail.com: domain of inwardvessel@gmail.com designates 209.85.214.171 as permitted sender) smtp.mailfrom=inwardvessel@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736904801; a=rsa-sha256; cv=none; b=5lsPa0U4Iz/KWqu11DTo6DYZuiwpddyx3XScR+r5dhXcT56BuJ8IpaW+Ew81zjW5zf1uf3 dKtOTFgWcwKL2avI7ToFg903zyDnZbSae2BLv3NuXVVoSeuOxW6qruGqNvG1EHCP4dGylL Dc2Y63CWI3cLzv9OxYsNofmweQN5YcI= Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-21670dce0a7so133797975ad.1 for ; Tue, 14 Jan 2025 17:33:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1736904800; x=1737509600; darn=kvack.org; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :from:to:cc:subject:date:message-id:reply-to; bh=d9P5/KEhDafJlP2HFwU6Lo/haCAgWGhZE6rmWKAwwYg=; b=GTERuukVVxeucLaGnFxvBvIsl0bPCZgOqhzJbdQK3WUd41AJW32Qhyz92xB0J3uJYR yrIkZTYkwgF/ukOQ0UKcUuxjmIO8gJhxGFSmd4AFyTuHHnlZKcVyeb3NhTdFQv1HDQtv gK2KQkws/8ByDDAellROCIlHnr2gUt1jQJZyW0nXsw1asBqBLIx5QKPThyeGBk25xf5z M9mWchh/mNrquF/gYfyXzSmDXvKi3iuCWTyK8Cvuwes6BJfe5uOjVi0v1JF8kegxXBwW JcvDHU999GxT5QRra7QdZjCXOsMOQP9wEAuICZuB9WXXkVXC+ZveN08pfoU3LrJM8lem OpqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736904800; x=1737509600; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=d9P5/KEhDafJlP2HFwU6Lo/haCAgWGhZE6rmWKAwwYg=; b=UhMwbRtyzZ+P1iIdsEM1R+6dtymLH7igTvL21caPR+A6EKy79Pjsp7y5v+Sus3dL+i DBNls0+f/DnGinmJNyFJ4l8I0VGZOgEt5fXLVx1WW08eGmxIDGX2mKp2y+aeTHpmEVnY D+TJzi9/QoxW9oEHx3kouFn5mnVsgISFQL1+i/Qex6bHj3A4G934v4z+U9gzefZDopcf 8XwwNKOViBoeELBuxftgibU66XK0wMtENuwXhxkcGLIyrkOrklAyaO5En+B0GB5xNk+m vW56JQ3FpGFgF0lbIeD2wUSpJMmjgiUJqbMjehdovRUKTnsvLTze/IBQswj4+8VSt0CV ysMw== X-Forwarded-Encrypted: i=1; AJvYcCWb73OSUOd0R2eLwWJ5N5ghNhTZT5Kya7wS9ZRw993tWimEt7L2zQWd9Ns243Ghz5PC0Sq/ird+4A==@kvack.org X-Gm-Message-State: AOJu0YxbPUPAby7L4Cc1Xjon0P9Y9G1DjGGJzTOsRMaHdZn5vf6YC91p NnfYNDzYCAfRd0AiB/pVoDE2HNWk4kJamjF1ywwwqja/YJ3WACIk X-Gm-Gg: ASbGncvHkWaQG+cJ0IjTD/vXGcGeYVHzjlzqmzLUlK1O4prn2zGfSgHcj28zGlvG3+G l/VAg6R0GEDscqH3hdqMn/5aqAnOb2qOT6WOWo7p0XD+j5zYILPNLsTcvArHa43ZMt0FoevBpZv TVIIQ9xKUBaug+VP1mzXGj/6C7PJbKjPcK6SKD+++ogmbgT6jTU1U2/76CmiUkGe6ut718DF/u1 zGYyMrTpfsfS6qpP1Cf2F5jbFWHKoOSSvNwDbv3AG511/1KPHLHONJys72SeOr8iY6HsPjySmpe NLRDIU4uh4E1sq5iZujJb+iQiQA= X-Google-Smtp-Source: AGHT+IF8WCGrsngtExiMEJPyGr2HFt1RxJSp++uB0HJ+ldZPbD42ZP+VfZFVFYg+hwKZeGQm/b/i5Q== X-Received: by 2002:a17:902:f68b:b0:216:55a1:35a with SMTP id d9443c01a7336-21a83f9cbedmr389805695ad.30.1736904799907; Tue, 14 Jan 2025 17:33:19 -0800 (PST) Received: from [192.168.2.117] (c-67-188-127-15.hsd1.ca.comcast.net. [67.188.127.15]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2f72c154868sm191436a91.8.2025.01.14.17.33.18 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 14 Jan 2025 17:33:19 -0800 (PST) Message-ID: <3348742b-4e49-44c1-b447-b21553ff704a@gmail.com> Date: Tue, 14 Jan 2025 17:33:17 -0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 0/9 RFC] cgroup: separate rstat trees To: Shakeel Butt , =?UTF-8?Q?Michal_Koutn=C3=BD?= Cc: hannes@cmpxchg.org, yosryahmed@google.com, akpm@linux-foundation.org, linux-mm@kvack.org, cgroups@vger.kernel.org, Tejun Heo References: <20241224011402.134009-1-inwardvessel@gmail.com> <3wew3ngaqq7cjqphpqltbq77de5rmqviolyqphneer4pfzu5h5@4ucytmd6rpfa> Content-Language: en-US From: JP Kobryn In-Reply-To: <3wew3ngaqq7cjqphpqltbq77de5rmqviolyqphneer4pfzu5h5@4ucytmd6rpfa> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 49C08A0003 X-Rspamd-Server: rspam12 X-Stat-Signature: wdauwdrgytp78wo4ow4gxinjc6n3knjn X-Rspam-User: X-HE-Tag: 1736904801-409812 X-HE-Meta: U2FsdGVkX1+d26tUZWAv7eD7+efPjBwb2Jo8lEsS2cu3KnRRXvKC4kVkcGGVNLL3b+DkM0OSU6g1Rv/dJyIHhXA5qW731E8LfNn7JonMN4BTt/qVCzl2AW1W+4H3zgxeqxeYTS7xe2JQ6oef/+QW8wQXay4XOCFrFFLTkERq0gVg8Ep3/vRTMRLVJIGeK7EpCaGViin+jzd/SssOPVugp1U0oVGzcb74KycqworJOKLBicf9tZMceEKY+fUoKRxIBjJcojJXlVi6A/0UcvALSvZMHlV5rJkoLb6zscrql1Ee8zV8BKXMaloe8qJJ+16TCPZqJm27Sbdbh+niQxVWEIfQBYZtXta+9bw6L6zNSJE2i0Kiou8uqODH3xZo8cNy/MLUzmlHkOxUxqi/tn0UqpqFNYbzkYhvnUQPCelmUOKNvsP4lMKrz0X8QC2WppQ7nNpikYT3VRNsKmX+iUq5+9SXIMhf/KuJpeN/h59pdnRAEYkAoH73oea6GJdoQ3XG8ZA0ftRoG5lsyLuYS3AGmC1nr8j07s5k2wPnZYFOWdiAzdEe0DEYd9w43PEIOpU5d/ah4t5s3+rS69v3krOgmYA7fZs3MIHOhxJScC9KwLAAqb7G4LW94U3ZTRUqeQkCxOt1HLEkc3asDviTifAjLCL0MRbYrrXBqBVwmFROQk7mNgYevqstKZcBbn11fSOPWUL2Fcv7ANQX0aSsyNLH1O4KFh+f21JK4KRHBP8Pdz7LrIAOtgD4w7RbXQuLP5k3mMs41NW8iEKWXQWDmxtZ9NZvvkNT8ok4/fIswirUMaas3XsK+su84k3kJXvaCXtr9WctolYuo3F7GDeh2AZ0kRTrVuO0tQoaOtiDxMASZpp0VaYqR1fohvb0lEfc68mG2OcRIIl/vfmSre5ESViyyEcSZATTS5q+UiGUxvOVkX9YGhoM4+uicJerhGnYF87+M4hL4lW3sw3xDviuj4w oIaQ8q4x 8nlc3XjoRxMnvccNy9hAgT4bSItfw11lspebphRs1CY/+0l+FcIoT/TJdo0rY6FgZa8WpF31OCzRCvWXrw1UG8mptfYpz/VJlHuyINWOOE+tI91jNJup0lrky+xu1psYXQzFizcz1R7nIadZRrepWtCXtuhiL085FaOGjWc1c0Zc0ijpJnD6ITgUrZUSfiPuqYGjJ3IOSxTkSn7OhWYR1GFeGBraol1DKeZkblSJfHTStcvIXepMAY7T6i6uld9mJOixo7hERBhUusmgkcz9shfS0tL1Oz2SmtKZeCljXkQBsuLzkX27PyTB/1hB49Iw1Fpjk0jvQLqjT2c9k0yMdH/6tOy3Pg4VwFIU3B3vlsusgPVrs5BFMOjhb9h/4Qw+937E+rwzFypYn5Mv2ChHMEBbwMLGIdFTbq+12YRMlwkuRSBxNhi5vlcfV6f7ld3mCc/C7 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi Michal, On 1/13/25 10:25 AM, Shakeel Butt wrote: > On Wed, Jan 08, 2025 at 07:16:47PM +0100, Michal Koutný wrote: >> Hello JP. >> >> On Mon, Dec 23, 2024 at 05:13:53PM -0800, JP Kobryn wrote: >>> I've been experimenting with these changes to allow for separate >>> updating/flushing of cgroup stats per-subsystem. >> >> Nice. >> >>> I reached a point where this started to feel stable in my local testing, so I >>> wanted to share and get feedback on this approach. >> >> The split is not straight-forwardly an improvement -- > > The major improvement in my opinion is the performance isolation for > stats readers i.e. cpu stats readers do not need to flush memory stats. > >> there's at least >> higher memory footprint > > Yes this is indeed the case and JP, can you please give a ballmark on > the memory overhead? Yes, the trade-off is using more memory to allow for separate trees. With these patches the changes in allocated memory for the cgroup_rstat_cpu instances and their associated locks are: static reduced by 58% dynamic increased by 344% The threefold increase on the dynamic side is attributed to now having 3 rstat trees per cgroup (1 for base stats, 1 for memory, 1 for io), instead of originally just 1. The number will change if more subsystems start or stop using rstat in the future. Feel free to let me know if you would like to see the detailed breakdown of these values. > >> and flushing efffectiveness depends on how >> individual readers are correlated, > > Sorry I am confused by the above statement, can you please expand on > what you meant by it? > >> OTOH writer correlation affects >> updaters when extending the update tree. > > Here I am confused about the difference between writer and updater. > >> So a workload dependent effect >> can go (in my theory) both sides. >> There are also in-kernel consumers of stats, namely memory controller >> that's been optimized over the years to balance the tradeoff between >> precision and latency. > > In-kernel memcg stats readers will be unaffected most of the time with > this change. The only difference will be when they flush, they will only > flush memcg stats. > >> >> So do you have any measurements (or expectations) that show how readers >> or writers are affected? >> > > Here I am assuming you meant measurements in terms of cpu cost or do you > have something else in mind? > > > Thanks a lot Michal for taking a look.