Re: [ckrm-tech] [RFC][PATCH 5/8] RSS controller task migration support

Re: [ckrm-tech] [RFC][PATCH 5/8] RSS controller task migration support

[Patrick.Le-Dot]

> ...
> For implementing guarantees, we can use limits. Please see
> http://wiki.openvz.org/Containers/Guarantees_for_resources.

Nack.

This seems to be correct for resources like cpu, disk or network
bandwidth but not for the memory just because nobody in this wiki
speaks about the kswapd and page reclaim (but it's true that a such
demon does not exist for cpu, disk or... then the problem is more
simple).

For a customer the main reason to use guarantee is to be sure that
some pages of a job remain in memory when the system is low on free
memory. This should be true even for a job in group/container A with
a smooth activity compared to a group/container B with a set of jobs
using memory more agressively...

What happens if we use limits to implement guarantees ?

>> ...
>> The idea of getting a guarantee is simple:
>> if any group gi requires a Gi units of resource from R units available
>> then limiting all the rest groups with R - Gi units provides a desired
>> guarantee

If the limit is a "hard limit" then we have implemented reservation and
this is too strict.

If the limit is a "soft limit" then group/container B is autorized to
use more than the limit and nothing is guaranteed for group/container A...

Patrick

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ckrm-tech] [RFC][PATCH 5/8] RSS controller task migration support

[Alan]

On Fri, 17 Nov 2006 14:25:33 +0100 (CET)
> For a customer the main reason to use guarantee is to be sure that
> some pages of a job remain in memory when the system is low on free
> memory. This should be true even for a job in group/container A with

That actually doesn't appear a very useful definition.

There are two reasons for wanting memory guarantees

#1	To be sure a user can't toast the entire box but just their own
	compartment (eg web hosting)

#2	To ensure all apps continue to make progress

The simple approach doesn't seem to work for either. There is a threshold
above which #1 and #2 are the same thing, below that trying to keep a few
pages in memory will thrash not make progress and will harm overall
behaviour thus failing to solve #1 or #2. At that point you have to
decide whether what you have is a misconfiguration or whether the system
should be prepared to do temporary cycling overcommits so containers take
it in turn to make progress when overcommitted.

> If the limit is a "hard limit" then we have implemented reservation and
> this is too strict.

Thats fundamentally a judgement based on your particular workload and
constraints. If I am web hosting then I don't generally care if my end
users compartment blows up under excess load, I care that the other 200
customers using the box don't suffer and all phone me to complain.

Alan

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ckrm-tech] [RFC][PATCH 5/8] RSS controller task migration support

[Balbir Singh]

Patrick.Le-Dot wrote:
>> ...
>> For implementing guarantees, we can use limits. Please see
>> http://wiki.openvz.org/Containers/Guarantees_for_resources.
> 
> Nack.
> 
> This seems to be correct for resources like cpu, disk or network
> bandwidth but not for the memory just because nobody in this wiki
> speaks about the kswapd and page reclaim (but it's true that a such
> demon does not exist for cpu, disk or... then the problem is more
> simple).
> 
> For a customer the main reason to use guarantee is to be sure that
> some pages of a job remain in memory when the system is low on free
> memory. This should be true even for a job in group/container A with
> a smooth activity compared to a group/container B with a set of jobs
> using memory more agressively...
> 

I am not against guarantees, but

Consider the following scenario, let's say we implement guarantees

1. If we account for kernel resources, how do you provide guarantees
   when you have non-reclaimable resources?
2. If a customer runs a system with swap turned off (which is quite
   common), then anonymous memory becomes irreclaimable. If a group
   takes more than it's fair share (exceeds its guarantee), you
   have scenario similar to 1 above.

> What happens if we use limits to implement guarantees ?
> 
>>> ...
>>> The idea of getting a guarantee is simple:
>>> if any group gi requires a Gi units of resource from R units available
>>> then limiting all the rest groups with R - Gi units provides a desired
>>> guarantee
> 
> If the limit is a "hard limit" then we have implemented reservation and
> this is too strict.
>
> If the limit is a "soft limit" then group/container B is autorized to
> use more than the limit and nothing is guaranteed for group/container A...
> 
> Patrick


Yes, but it is better than failing to meet a guarantee (if guarantees are
desired :))


-- 

	Balbir Singh,
	Linux Technology Center,
	IBM Software Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ckrm-tech] [RFC][PATCH 5/8] RSS controller task migration support

[Patrick.Le-Dot]

On Fri, 17 Nov 2006 14:05:13 +0000
> ...
> There are two reasons for wanting memory guarantees
> 
> #1      To be sure a user can't toast the entire box but just their own
>         compartment (eg web hosting)

Well, this seems not a situation to add a guarantee to this user
but a limit...

> ...
> #2      To ensure all apps continue to make progress

or to ensure that a job is ready to work without to have to pay the
cost of a lot of pagination in...


>> If the limit is a "hard limit" then we have implemented reservation and
>> this is too strict.
>
> Thats fundamentally a judgement based on your particular workload and
> constraints.
Nop.
You can read this on the wiki page...

I'm just saying that the implementation of guarantee with limits seems to
be not enough for #2.

> If I am web hosting then I don't generally care if my end
> users compartment blows up under excess load, I care that the other 200
> customers using the box don't suffer and all phone me to complain.

I agree : limit is necessary and should be a "hard limit" (even if the
controler needs an internal threeshold like a "soft limit" to decide to
wakeup the kswapd).
But this is not the topic (not yet:-)

Patrick

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ckrm-tech] [RFC][PATCH 5/8] RSS controller task migration support

[Balbir Singh]

Patrick.Le-Dot wrote:
> Hi Balbir,
> 
> The get_task_mm()/mmput(mm) usage is not correct.
> With CONFIG_DEBUG_SPINLOCK_SLEEP=y :
> 
> BUG: sleeping function called from invalid context at kernel/fork.c:390
> in_atomic():1, irqs_disabled():0
>  [<c0116620>] __might_sleep+0x97/0x9c
>  [<c0116a2e>] mmput+0x15/0x8b
>  [<c01582f6>] install_arg_page+0x72/0xa9
>  [<c01584b1>] setup_arg_pages+0x184/0x1a5
>  ...
> 
> BUG: sleeping function called from invalid context at kernel/fork.c:390
> in_atomic():1, irqs_disabled():0
>  [<c0116620>] __might_sleep+0x97/0x9c
>  [<c0116a2e>] mmput+0x15/0x8b
>  [<c01468ee>] do_no_page+0x255/0x2bd
>  [<c0146b8d>] __handle_mm_fault+0xed/0x1ef
>  [<c0111884>] do_page_fault+0x247/0x506
>  [<c011163d>] do_page_fault+0x0/0x506
>  [<c0348f99>] error_code+0x39/0x40
> 
> 
> current->mm seems to be enough here.

Excellent, thanks for catching this!

> 
> 
> 
> In patch4, memctlr_dec_rss(page, mm) should be memctlr_dec_rss(page)
> to compile correctly.
> 
> and in patch0 :
>> 4. Disable cpuset's (to simply assignment of tasks to resource groups)
>>         cd /container
>>         echo 0 > cpuset_enabled
> 
> should be :
>         echo 0 > cpuacct_enabled
> 
> Note : cpuacct_enabled is 0 by default.
> 

Thanks for pointing this out.

> 
> Now the big question : to implement guarantee, the LRU needs to know
> if a page can be removed from memory or not.
> Any ideas to do that without any change in the struct page ?
> 

For implementing guarantees, we can use limits. Please see
http://wiki.openvz.org/Containers/Guarantees_for_resources.


Thanks for the feedback!

-- 

	Balbir Singh,
	Linux Technology Center,
	IBM Software Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>