Re: strange allocation failures

[Dmitry Vyukov <dvyukov@google.com>]

On Mon, Mar 13, 2017 at 11:37 AM, Dmitry Vyukov <dvyukov@google.com> wrote:
> On Mon, Mar 13, 2017 at 11:31 AM, Andrey Ryabinin
> <aryabinin@virtuozzo.com> wrote:
>>
>>
>> On 03/13/2017 01:10 PM, Dmitry Vyukov wrote:
>>> On Mon, Mar 13, 2017 at 11:08 AM, Dmitry Vyukov <dvyukov@google.com> wrote:
>>>> On Mon, Mar 13, 2017 at 11:04 AM, Andrey Ryabinin
>>>> <aryabinin@virtuozzo.com> wrote:
>>>>>
>>>>>
>>>>> On 03/13/2017 12:50 PM, Dmitry Vyukov wrote:
>>>>>> Hello Andrey, Kirill,
>>>>>>
>>>>>> Can you please help me understand where is all my memory?
>>>>>> I am running very moderate workload on a machine with 7.5GB of memory
>>>>>> with KASAN. And I see constant vmalloc allocation failures for very
>>>>>> moderate sizes. I am confused why it happens and where is all my
>>>>>> memory...
>>>>>>
>>>>>
>>>>>
>>>>> Perhaps it's SIGKILL generated by syzkaller?
>>>>>
>>>>> static void *__vmalloc_area_node()
>>>>> {
>>>>> .....
>>>>>                 if (fatal_signal_pending(current)) {
>>>>>                         area->nr_pages = i;
>>>>>                         goto fail;
>>>>>                 }
>>>>
>>>>
>>>> Ah, that would make sense. Syzkaller can indeed kill processes frequently.
>>>>
>>>> Perhaps we should not print the lengthy allocation failure message
>>>> with all the details in such. Not sure how easy it is to filter out
>>>> such cases.
>>>> I have constant stream of these messages that just make everything
>>>> else lost between them. And they are quite confusing. I've starred at
>>>> the numbers trying to understand why I am short on memory.
>>>
>>>
>>> Seems trivial. What do you think of:
>>>
>>
>> Makes sense. ACK.
>>
>>> diff --git a/mm/vmalloc.c b/mm/vmalloc.c
>>> index 0dd80222b20b..0b057628a7ba 100644
>>> --- a/mm/vmalloc.c
>>> +++ b/mm/vmalloc.c
>>> @@ -1683,7 +1683,7 @@ static void *__vmalloc_area_node(struct
>>> vm_struct *area, gfp_t gfp_mask,
>>>
>>>                 if (fatal_signal_pending(current)) {
>>>                         area->nr_pages = i;
>>> -                       goto fail;
>>> +                       goto fail_no_warn;
>>>                 }
>>>
>>>                 if (node == NUMA_NO_NODE)
>>> @@ -1709,6 +1709,7 @@ static void *__vmalloc_area_node(struct
>>> vm_struct *area, gfp_t gfp_mask,
>>>         warn_alloc(gfp_mask, NULL,
>>>                           "vmalloc: allocation failure, allocated %ld
>>> of %ld bytes",
>>>                           (area->nr_pages*PAGE_SIZE), area->size);
>>> +fail_no_warn:
>>>         vfree(area->addr);
>>>         return NULL;
>>>  }
>>>
>>>
>>> ?
>
>
> These failing vmalloc's provoked a bunch of bugs in kernel on error
> handling paths. And it was useful to see that there was an allocation
> failure in the same process right before the bug.
> And it was unexpected that I am killing processes that frequently, so
> I would like to see at least some information about this on console.
> So now I have:
>
>
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 6cbde310abed..418c80a76b4a 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -3073,6 +3073,11 @@ static inline bool should_suppress_show_mem(void)
>  #if NODES_SHIFT > 8
>         ret = in_interrupt();

/\/\/\/\/\/\/\/\
As a side note, looking at this line.
Can vmalloc be called from an interrupt? If so, won't we fail all
vmalloc's in an unlucky interrupt that hit a task with
fatal_signal_pending?



>  #endif
> +       /*
> +        * vmalloc() fails when fatal_signal_pending(),
> +        * but that's not because we are out of memory.
> +        */
> +       ret |= fatal_signal_pending(current);
>         return ret;
>  }
>
> @@ -3120,9 +3125,13 @@ void warn_alloc(gfp_t gfp_mask, nodemask_t
> *nodemask, const char *fmt, ...)
>
>         pr_cont(", mode:%#x(%pGg), nodemask=", gfp_mask, &gfp_mask);
>         if (nodemask)
> -               pr_cont("%*pbl\n", nodemask_pr_args(nodemask));
> +               pr_cont("%*pbl", nodemask_pr_args(nodemask));
> +       else
> +               pr_cont("(null)");
> +       if (fatal_signal_pending(current))
> +               pr_cont(", fatal signal pending\n");
>         else
> -               pr_cont("(null)\n");
> +               pr_cont("\n");
>
>         cpuset_print_current_mems_allowed();
>
> It's not so verbose, but explains things better.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>