Re: strange allocation failures

[Michal Hocko <mhocko@kernel.org>]

On Mon 13-03-17 11:37:41, Dmitry Vyukov wrote:
> On Mon, Mar 13, 2017 at 11:31 AM, Andrey Ryabinin
> <aryabinin@virtuozzo.com> wrote:
> >
> >
> > On 03/13/2017 01:10 PM, Dmitry Vyukov wrote:
> >> On Mon, Mar 13, 2017 at 11:08 AM, Dmitry Vyukov <dvyukov@google.com> wrote:
> >>> On Mon, Mar 13, 2017 at 11:04 AM, Andrey Ryabinin
> >>> <aryabinin@virtuozzo.com> wrote:
> >>>>
> >>>>
> >>>> On 03/13/2017 12:50 PM, Dmitry Vyukov wrote:
> >>>>> Hello Andrey, Kirill,
> >>>>>
> >>>>> Can you please help me understand where is all my memory?
> >>>>> I am running very moderate workload on a machine with 7.5GB of memory
> >>>>> with KASAN. And I see constant vmalloc allocation failures for very
> >>>>> moderate sizes. I am confused why it happens and where is all my
> >>>>> memory...
> >>>>>
> >>>>
> >>>>
> >>>> Perhaps it's SIGKILL generated by syzkaller?
> >>>>
> >>>> static void *__vmalloc_area_node()
> >>>> {
> >>>> .....
> >>>>                 if (fatal_signal_pending(current)) {
> >>>>                         area->nr_pages = i;
> >>>>                         goto fail;
> >>>>                 }
> >>>
> >>>
> >>> Ah, that would make sense. Syzkaller can indeed kill processes frequently.
> >>>
> >>> Perhaps we should not print the lengthy allocation failure message
> >>> with all the details in such. Not sure how easy it is to filter out
> >>> such cases.
> >>> I have constant stream of these messages that just make everything
> >>> else lost between them. And they are quite confusing. I've starred at
> >>> the numbers trying to understand why I am short on memory.
> >>
> >>
> >> Seems trivial. What do you think of:
> >>
> >
> > Makes sense. ACK.
> >
> >> diff --git a/mm/vmalloc.c b/mm/vmalloc.c
> >> index 0dd80222b20b..0b057628a7ba 100644
> >> --- a/mm/vmalloc.c
> >> +++ b/mm/vmalloc.c
> >> @@ -1683,7 +1683,7 @@ static void *__vmalloc_area_node(struct
> >> vm_struct *area, gfp_t gfp_mask,
> >>
> >>                 if (fatal_signal_pending(current)) {
> >>                         area->nr_pages = i;
> >> -                       goto fail;
> >> +                       goto fail_no_warn;
> >>                 }
> >>
> >>                 if (node == NUMA_NO_NODE)
> >> @@ -1709,6 +1709,7 @@ static void *__vmalloc_area_node(struct
> >> vm_struct *area, gfp_t gfp_mask,
> >>         warn_alloc(gfp_mask, NULL,
> >>                           "vmalloc: allocation failure, allocated %ld
> >> of %ld bytes",
> >>                           (area->nr_pages*PAGE_SIZE), area->size);
> >> +fail_no_warn:
> >>         vfree(area->addr);
> >>         return NULL;
> >>  }
> >>
> >>
> >> ?
> 
> 
> These failing vmalloc's provoked a bunch of bugs in kernel on error
> handling paths. And it was useful to see that there was an allocation
> failure in the same process right before the bug.
> And it was unexpected that I am killing processes that frequently, so
> I would like to see at least some information about this on console.
> So now I have:

Allocation failure should tell us the state of the memory subsystem when
the allocation failed. This failure is due to external condition rather
than the MM subsystem failing so I agree that skipping the warning makes
some sense. The warning will be mostly uninteresting or worse confusing.
If you absolutely need some information then put pr_debug  to the
failing path inside fatal_signal_pending branch.

> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 6cbde310abed..418c80a76b4a 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -3073,6 +3073,11 @@ static inline bool should_suppress_show_mem(void)
>  #if NODES_SHIFT > 8
>         ret = in_interrupt();
>  #endif
> +       /*
> +        * vmalloc() fails when fatal_signal_pending(),
> +        * but that's not because we are out of memory.
> +        */
> +       ret |= fatal_signal_pending(current);
>         return ret;
>  }

This will basically silent all the warnings for OOM victims failing the
allocation. I do think we want that.

-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>