Re: [PATCH] memcg: implement low limits

[Roman Gushchin <klamm@yandex-team.ru>]

[-- Attachment #1: Type: text/plain, Size: 4220 bytes --]

Please find my comments below.

> More comments on the code bellow.
>
> [...]
>
>> ?diff --git a/mm/memcontrol.c b/mm/memcontrol.c
>> ?index 53b8201..d8e6ee6 100644
>> ?--- a/mm/memcontrol.c
>> ?+++ b/mm/memcontrol.c
>> ?@@ -1743,6 +1743,53 @@ static void mem_cgroup_out_of_memory(struct mem_cgroup *memcg, gfp_t gfp_mask,
>> ???????????????????????????NULL, "Memory cgroup out of memory");
>> ??}
>>
>> ?+/*
>> ?+ * If a cgroup is under low limit or enough close to it,
>> ?+ * decrease speed of page scanning.
>> ?+ *
>> ?+ * mem_cgroup_low_limit_scale() returns a number
>> ?+ * from range [0, DEF_PRIORITY - 2], which is used
>> ?+ * in the reclaim code as a scanning priority modifier.
>> ?+ *
>> ?+ * If the low limit is not set, it returns 0;
>> ?+ *
>> ?+ * usage - low_limit > usage / 8 ?=> 0
>> ?+ * usage - low_limit > usage / 16 => 1
>> ?+ * usage - low_limit > usage / 32 => 2
>> ?+ * ...
>> ?+ * usage - low_limit > usage / (2 ^ DEF_PRIORITY - 3) => DEF_PRIORITY - 3
>> ?+ * usage < low_limit => DEF_PRIORITY - 2
>
> Could you clarify why you have used this calculation. The comment
> exlaims _what_ is done but not _why_ it is done.
>
> It is also strange (and unexplained) that the low limit will work
> differently depending on the memcg memory usage - bigger groups have a
> bigger chance to be reclaimed even if they are under the limit.

The idea is to decrease scanning speed smoothly.
It's hard to explain why I used exact these numbers. It' like why DEF_PRIORITY is 12?
Just because it works :). Of course, these numbers are an object for discussion/change.

There is a picture in attachment that illustrates how low limits work:
red line - memory usage of cgroup with low_limit set to 1Gb,
blue line - memory usage of another cgroup, where I ran cat <large file> > /dev/null.

>> ?+ *
>> ?+ */
>> ?+unsigned int mem_cgroup_low_limit_scale(struct lruvec *lruvec)
>> ?+{
>> ?+ struct mem_cgroup_per_zone *mz;
>> ?+ struct mem_cgroup *memcg;
>> ?+ unsigned long long low_limit;
>> ?+ unsigned long long usage;
>> ?+ unsigned int i;
>> ?+
>> ?+ mz = container_of(lruvec, struct mem_cgroup_per_zone, lruvec);
>> ?+ memcg = mz->memcg;
>> ?+ if (!memcg)
>> ?+ return 0;
>> ?+
>> ?+ low_limit = res_counter_read_u64(&memcg->res, RES_LOW_LIMIT);
>> ?+ if (!low_limit)
>> ?+ return 0;
>> ?+
>> ?+ usage = res_counter_read_u64(&memcg->res, RES_USAGE);
>> ?+
>> ?+ if (usage < low_limit)
>> ?+ return DEF_PRIORITY - 2;
>> ?+
>> ?+ for (i = 0; i < DEF_PRIORITY - 2; i++)
>> ?+ if (usage - low_limit > (usage >> (i + 3)))
>> ?+ break;
>
> why this doesn't depend in the current reclaim priority?

How do you want to use reclaim priority here?

I don't like an idea to start ignoring low limit on some priorities.

In my implementation low_limit_scale just "increases" scanning priority, 
but no more than for 10 (DEF_PRIORITY - 2). So, if priority is 0-2, 
the reclaim works as if the priority were 10-12, that means "normal" slow reclaim.

>
>> ?+
>> ?+ return i;
>> ?+}
>> ?+
>> ??static unsigned long mem_cgroup_reclaim(struct mem_cgroup *memcg,
>> ??????????????????????????????????????????gfp_t gfp_mask,
>> ??????????????????????????????????????????unsigned long flags)
>
> [...]
>
>> ?diff --git a/mm/vmscan.c b/mm/vmscan.c
>> ?index 88c5fed..9c1c702 100644
>> ?--- a/mm/vmscan.c
>> ?+++ b/mm/vmscan.c
>> ?@@ -1660,6 +1660,7 @@ static void get_scan_count(struct lruvec *lruvec, struct scan_control *sc,
>> ??????????bool force_scan = false;
>> ??????????unsigned long ap, fp;
>> ??????????enum lru_list lru;
>> ?+ unsigned int low_limit_scale = 0;
>>
>> ??????????/*
>> ???????????* If the zone or memcg is small, nr[l] can be 0. ?This
>> ?@@ -1779,6 +1780,9 @@ static void get_scan_count(struct lruvec *lruvec, struct scan_control *sc,
>> ??????????fraction[1] = fp;
>> ??????????denominator = ap + fp + 1;
>> ??out:
>> ?+ if (global_reclaim(sc))
>> ?+ low_limit_scale = mem_cgroup_low_limit_scale(lruvec);
>
> What if the group is reclaimed as a result from parent hitting its
> limit?

For now, low limits will work only for global reclaim. Enabling them for target reclaim will require some additional checks.
I plan to do this as a separate change.

Thank you for your comments!

--
Regards,
Roman

[-- Attachment #2: low_limit_memcg.gif --]
[-- Type: image/gif, Size: 11206 bytes --]