Re: "make -j" with memory.(memsw.)limit_in_bytes smaller than required -> livelock, even for unlimited processes

[Ying Han <yinghan@google.com>]

On Tue, Jun 21, 2011 at 7:51 AM, Lutz Vieweg <lvml@5t9.de> wrote:
> Dear Memory Ressource Controller maintainers,
>
> by using per-user control groups with a limit on memory (and swap) I am
> trying to secure a shared development server against memory exhaustion
> by any one single user - as it happened before when somebody imprudently
> issued "make -j" (which has the infamous habit to spawn an unlimited
> number of processes) on a large software project with many source files.
>
> The memory limitation using control groups works just fine when
> only a few processes sum up to a usage that exceeds the limits - the
> processes are OOM-killed, then, and the others users are unaffected.
>
> But the original cause, a "make -j" on many source files, leads to
> the following ugly symptom:
>
> - make starts numerous (~ 100 < x < 200) gcc processes
>
> - some of those gcc processes get OOM-killed quickly, then
>  a few more are killed, but with increasing pauses in between
>
> - then after a few seconds, no more gcc processes are killed, but
>  the "make" process and its childs do not show any progress anymore
>
> - at this time, top indicates 100% "system" CPU usage, mostly by
>  "[kworker/*]" threads (one per CPU). But processes from other
>  users, that only require CPU, proceed to run.

The following patch might not be the root-cause of livelock, but
should reduce the [kworker/*] in your case.

==