Re: [PATCH] mm: ratelimit oversized kvmalloc warnings instead of once

[Mateusz Guzik <mjguzik@gmail.com>]

On Wed, Jun 19, 2024 at 2:49 PM Mateusz Guzik <mjguzik@gmail.com> wrote:
>
> On Tue, Jun 18, 2024 at 02:34:21PM -0700, Shakeel Butt wrote:
> > At the moment oversize kvmalloc warnings are triggered once using
> > WARN_ON_ONCE() macro. One issue with this approach is that it only
> > detects the first abuser and then ignores the remaining abusers which
> > complicates detecting all such abusers in a timely manner. The situation
> > becomes worse when the repro has low probability and requires production
> > traffic and thus require large set of machines to find such abusers. In
> > Mera production, this warn once is slowing down the detection of these
> > abusers. Simply replace WARN_ON_ONCE with WARN_RATELIMIT.
> >
> > Reported-by: Kyle McMartin <kyle@infradead.org>
> > Signed-off-by: Shakeel Butt <shakeel.butt@linux.dev>
> > ---
> >  mm/util.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> >
> > diff --git a/mm/util.c b/mm/util.c
> > index 10f215985fe5..de36344e8d53 100644
> > --- a/mm/util.c
> > +++ b/mm/util.c
> > @@ -649,7 +649,8 @@ void *kvmalloc_node_noprof(size_t size, gfp_t flags, int node)
> >
> >       /* Don't even allow crazy sizes */
> >       if (unlikely(size > INT_MAX)) {
> > -             WARN_ON_ONCE(!(flags & __GFP_NOWARN));
> > +             WARN_RATELIMIT(!(flags & __GFP_NOWARN), "size = %zu > INT_MAX",
> > +                            size);
> >               return NULL;
> >       }
> >
>
> I don't think this is necessary. From the description I think interested
> parties can get away with bpftrace.
>
> Suppose you have an abuser of the sort and you are worried there is more
> than one.
>
> Then this one-liner will catch *all* of them, not just the ones which
> were "lucky" to get logged with ratelimit:
> bpftrace -e 'kprobe:kvmalloc_node_noprof /arg0 > 2147483647/ { @[kstack()] = count(); }'
>
> Of course adding a probe is not free, but then again kvmalloc should not
> be used often to begin with so I doubt it is going to have material
> impact in terms of performance.
>
> While I concede it takes more effort to get this running on all affected
> machines, the result is much better than mere ratelimit. Also there is
> no need to patch the kernel.
>
> btw, I found drm keeps spamming kvmalloc, someone(tm) should look into
> it:
> @[
>     kvmalloc_node_noprof+5
>     drm_property_create_blob+76
>     drm_atomic_helper_dirtyfb+234
>     drm_fbdev_generic_helper_fb_dirty+509
>     drm_fb_helper_damage_work+139
>     process_one_work+376
>     worker_thread+753
>     kthread+207
>     ret_from_fork+49
>     ret_from_fork_asm+26
> , 104]: 12

I should clarify this is allocs of 104 bytes, not some outlandish size.

-- 
Mateusz Guzik <mjguzik gmail.com>