linux-mm.kvack.org archive mirror
 help / color / mirror / Atom feed
From: Roman Gushchin <roman.gushchin@linux.dev>
To: "Michal Koutný" <mkoutny@suse.com>
Cc: void@manifault.com, akpm@linux-foundation.org,
	cgroups@vger.kernel.org, hannes@cmpxchg.org, kernel-team@fb.com,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	mhocko@kernel.org, shakeelb@google.com, tj@kernel.org,
	Richard Palethorpe <rpalethorpe@suse.de>
Subject: Re: [PATCH 3/4] selftests: memcg: Adjust expected reclaim values of protected cgroups
Date: Fri, 13 May 2022 11:52:52 -0700	[thread overview]
Message-ID: <Yn6pBPq+lAXm9NG8@carbon> (raw)
In-Reply-To: <20220513171811.730-4-mkoutny@suse.com>

On Fri, May 13, 2022 at 07:18:10PM +0200, Michal Koutny wrote:
> The numbers are not easy to derive in a closed form (certainly mere
> protections ratios do not apply), therefore use a simulation to obtain
> expected numbers.
> 
> The new values make the protection tests succeed more precisely.
> 
> 	% run as: octave-cli script
> 	%
> 	% Input configurations
> 	% -------------------
> 	% E parent effective protection
> 	% n nominal protection of siblings set at the givel level
> 	% c current consumption -,,-
> 
> 	% example from testcase (values in GB)
> 	E = 50 / 1024;
> 	n = [75 25 0 500 ] / 1024;
> 	c = [50 50 50 0] / 1024;
> 
> 	% Reclaim parameters
> 	% ------------------
> 
> 	% Minimal reclaim amount (GB)
> 	cluster = 32*4 / 2**20;
> 
> 	% Reclaim coefficient (think as 0.5^sc->priority)
> 	alpha = .1
> 
> 	% Simulation parameters
> 	% ---------------------
> 	epsilon = 1e-7;
> 	timeout = 1000;
> 
> 	% Simulation loop
> 	% ---------------------
> 	% Simulation assumes siblings consumed the initial amount of memory (w/out
> 	% reclaim) and then the reclaim starts, all memory is reclaimable, i.e. treated
> 	% same. It simulates only non-low reclaim and assumes all memory.min = 0.
> 
> 	ch = [];
> 	eh = [];
> 	rh = [];
> 
> 	for t = 1:timeout
> 		% low_usage
> 		u = min(c, n);
> 		siblings = sum(u);
> 
> 		% effective_protection()
> 		protected = min(n, c);                % start with nominal
> 		e = protected * min(1, E / siblings); % normalize overcommit
> 
> 		% recursive protection
> 		unclaimed = max(0, E - siblings);
> 		parent_overuse = sum(c) - siblings;
> 		if (unclaimed > 0 && parent_overuse > 0)
> 			overuse = max(0, c - protected);
> 			e += unclaimed * (overuse / parent_overuse);
> 		endif
> 
> 		% get_scan_count()
> 		r = alpha * c;             % assume all memory is in a single LRU list
> 
> 		% commit 1bc63fb1272b ("mm, memcg: make scan aggression always exclude protection")
> 		sz = max(e, c);
> 		r .*= (1 - (e+epsilon) ./ (sz+epsilon));
> 
> 		% uncomment to debug prints
> 		% e, c, r
> 
> 		% nothing to reclaim, reached equilibrium
> 		if max(r) < epsilon
> 			break;
> 		endif
> 
> 		% SWAP_CLUSTER_MAX
> 		r = max(r, (r > epsilon) .* cluster);
> 		% XXX here I do parallel reclaim of all siblings
> 		% in reality reclaim is serialized and each sibling recalculates own residual
> 		c = max(c - r, 0);
> 
> 		ch = [ch ; c];
> 		eh = [eh ; e];
> 		rh = [rh ; r];
> 	endfor
> 
> 	t
> 	c, e

This is a cool stuff!

How about to place it into a separate file and add a comment into the code
with a reference?

Thanks!


  reply	other threads:[~2022-05-13 18:53 UTC|newest]

Thread overview: 37+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-04-23 15:56 [PATCH v2 0/5] Fix bugs in memcontroller cgroup tests David Vernet
2022-04-23 15:56 ` [PATCH v2 1/5] cgroups: Refactor children cgroups in memcg tests David Vernet
2022-04-26  1:56   ` Roman Gushchin
2022-04-23 15:56 ` [PATCH v2 2/5] cgroup: Account for memory_recursiveprot in test_memcg_low() David Vernet
2022-04-27 14:09   ` Michal Koutný
2022-04-29  1:03     ` David Vernet
2022-04-29  9:26       ` Michal Koutný
2022-05-06 16:40         ` David Vernet
2022-05-09 15:09           ` Johannes Weiner
2022-05-10  0:44             ` Andrew Morton
2022-05-10 17:43               ` Michal Koutný
2022-05-11 17:53                 ` Johannes Weiner
2022-05-12 17:27                   ` Michal Koutný
2022-04-23 15:56 ` [PATCH v2 3/5] cgroup: Account for memory_localevents in test_memcg_oom_group_leaf_events() David Vernet
2022-04-23 15:56 ` [PATCH v2 4/5] cgroup: Removing racy check in test_memcg_sock() David Vernet
2022-04-23 15:56 ` [PATCH v2 5/5] cgroup: Fix racy check in alloc_pagecache_max_30M() helper function David Vernet
2022-05-12 17:04 ` [PATCH v2 0/5] Fix bugs in memcontroller cgroup tests Michal Koutný
2022-05-12 17:30   ` David Vernet
2022-05-12 17:44     ` David Vernet
2022-05-13 17:18       ` [PATCH 0/4] memcontrol selftests fixups Michal Koutný
2022-05-13 17:18         ` [PATCH 1/4] selftests: memcg: Fix compilation Michal Koutný
2022-05-13 17:40           ` David Vernet
2022-05-13 18:53           ` Roman Gushchin
2022-05-13 19:09             ` Roman Gushchin
2022-05-13 17:18         ` [PATCH 2/4] selftests: memcg: Expect no low events in unprotected sibling Michal Koutný
2022-05-13 17:42           ` David Vernet
2022-05-13 18:54           ` Roman Gushchin
2022-05-18 15:54             ` Michal Koutný
2022-05-13 17:18         ` [PATCH 3/4] selftests: memcg: Adjust expected reclaim values of protected cgroups Michal Koutný
2022-05-13 18:52           ` Roman Gushchin [this message]
2022-05-13 17:18         ` [PATCH 4/4] selftests: memcg: Remove protection from top level memcg Michal Koutný
2022-05-13 18:59           ` Roman Gushchin
2022-05-18  0:24             ` Andrew Morton
2022-05-18  0:52               ` Roman Gushchin
2022-05-18 15:44                 ` Michal Koutný
2022-05-13 19:14           ` David Vernet
2022-05-18 15:40 [PATCH 0/4] memcontrol selftests fixups Michal Koutný
2022-05-18 15:40 ` [PATCH 3/4] selftests: memcg: Adjust expected reclaim values of protected cgroups Michal Koutný

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=Yn6pBPq+lAXm9NG8@carbon \
    --to=roman.gushchin@linux.dev \
    --cc=akpm@linux-foundation.org \
    --cc=cgroups@vger.kernel.org \
    --cc=hannes@cmpxchg.org \
    --cc=kernel-team@fb.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mhocko@kernel.org \
    --cc=mkoutny@suse.com \
    --cc=rpalethorpe@suse.de \
    --cc=shakeelb@google.com \
    --cc=tj@kernel.org \
    --cc=void@manifault.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox