Re: [PATCH 2/2] memcg: fix livelock in try charge during readahead

[Ying Han <yinghan@google.com>]

On Mon, Dec 12, 2011 at 8:45 PM, KAMEZAWA Hiroyuki
<kamezawa.hiroyu@jp.fujitsu.com> wrote:
> On Mon, 12 Dec 2011 18:16:48 -0800
> Ying Han <yinghan@google.com> wrote:
>
>> Couple of kernel dumps are triggered by watchdog timeout. It turns out that two
>> processes within a memcg livelock on a same page lock. We believe this is not
>> memcg specific issue and the same livelock exists in non-memcg world as well.
>>
>> The sequence of triggering the livelock:
>> 1. Task_A enters pagefault (filemap_fault) and then starts readahead
>> filemap_fault
>>  -> do_sync_mmap_readahead
>>     -> ra_submit
>>        ->__do_page_cache_readahead // here we allocate the readahead pages
>>          ->read_pages
>>          ...
>>            ->add_to_page_cache_locked
>>              //for each page, we do the try charge and then add the page into
>>              //radix tree. If one of the try charge failed, it enters per-memcg
>>              //oom while holding the page lock of previous readahead pages.
>>
>>             // in the memcg oom killer, it picks a task within the same memcg
>>             // and mark it TIF_MEMDIE. then it goes back into retry loop and
>>             // hopes the task exits to free some memory.
>>
>> 2. Task_B enters pagefault (filemap_fault) and finds the page in radix tree (
>> one of the readahead pages from ProcessA)
>>
>> filemap_fault
>>  ->__lock_page // here it is marked as TIF_MEMDIE. but it can not proceed since
>>                // the page lock is hold by ProcessA looping at OOM.
>>
>
> Should this __lock_page() be lock_page_killable() ?
> Hmm, at seeing linux-next, it's now lock_page_or_retry() and FAULT_FLAG_KILLABLE
> is set. why not killed immediately ?

Hmm, thank you for pointing it out. It seems that we are missing the
following patch in the tree triggering the problem:

commit 37b23e0525d393d48a7d59f870b3bc061a30ccdb
Author: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Date:   Tue May 24 17:11:30 2011 -0700

    x86,mm: make pagefault killable

    When an oom killing occurs, almost all processes are getting stuck at the
    following two points.

        1) __alloc_pages_nodemask
        2) __lock_page_or_retry

By eye-balling the linux-next including the patch above, we should be
able to avoid the live-lock by checking the fatal_signal_pending in
the page fault path.

--Ying

>
> Thanks,
> -Kame
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>