From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 258D7C27C53 for ; Wed, 19 Jun 2024 12:51:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9DF556B03FA; Wed, 19 Jun 2024 08:51:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 98F376B03FB; Wed, 19 Jun 2024 08:51:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 809A76B03FC; Wed, 19 Jun 2024 08:51:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 62D336B03FA for ; Wed, 19 Jun 2024 08:51:45 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 26232A3EA5 for ; Wed, 19 Jun 2024 12:51:45 +0000 (UTC) X-FDA: 82247624970.20.6A6CD89 Received: from mail-ej1-f41.google.com (mail-ej1-f41.google.com [209.85.218.41]) by imf13.hostedemail.com (Postfix) with ESMTP id 4E82E20002 for ; Wed, 19 Jun 2024 12:51:43 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=IU0PPdxZ; spf=pass (imf13.hostedemail.com: domain of mjguzik@gmail.com designates 209.85.218.41 as permitted sender) smtp.mailfrom=mjguzik@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718801497; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NMsD3rtYvv6DR/KAWNNE4xDIpVmpwwTpjIaG1Jg15rk=; b=iNVkg0c6SITUMkukAZuV+aLI/pJ3bA9Pd97YhFODqSMxoboLaQLUrbz2oDBAx0ezRIC36C NAu4vA5PtPss0EhirQbHh8XaOymT9aLivcHNaUlqZUN+IrEvGuRSRRD4lobqZf2LDrcgEh ZhPbq+SzN6nufMqF2inU55m1y5GVWe8= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=IU0PPdxZ; spf=pass (imf13.hostedemail.com: domain of mjguzik@gmail.com designates 209.85.218.41 as permitted sender) smtp.mailfrom=mjguzik@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718801497; a=rsa-sha256; cv=none; b=NeB38KCUG083MeaRQq7xjv8jqh0IPx1Vx4e+Jr4Z1c0Pm/j8mIC+VcdSRlSx8tqnBRqHDD mt8q2D6MJSroddlVeqX+/Dltqa9SZ682xalJ7jDO2ej8f90E/L++6+so+GXCK0HBD1o3vw /6Sy29qY9J9tL7nq1m49pqWvzqjUTXs= Received: by mail-ej1-f41.google.com with SMTP id a640c23a62f3a-a6cb130027aso444753966b.2 for ; Wed, 19 Jun 2024 05:51:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718801502; x=1719406302; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=NMsD3rtYvv6DR/KAWNNE4xDIpVmpwwTpjIaG1Jg15rk=; b=IU0PPdxZB5WqE9ZWsh79H3oeDIi2wbcY+LEL32+u6fjMGbu1V90zeXz+lMSl9m+rd2 5gbeMCOcETxW3mYg5NGG3UspOBSejdfPJcizrGgyMii/ec5F8jZ/phaOuriLe0cm/r0A 1UeOfX9TIkNDHbH1S8cI+DS0M2LsQwSmq+lXgvLUbFy5xbHC8Ufjf9vcDNOOKETZWkC/ lfwAuwenbp4qgKW5kYATbUShtU/7jT0uSrUHb0FVT2wULEYYZbS7wVoidAQcxc645cU4 EXaWw7MlegNsnExiU0tj0b7LEu+AVCZIS3+Gy1iOxKAmIzKOrRVBOwG0Z+0bEIMsOAjx vcKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718801502; x=1719406302; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NMsD3rtYvv6DR/KAWNNE4xDIpVmpwwTpjIaG1Jg15rk=; b=a9LJ0L1lgT9J/3Sfaizg1vt6VHokqb5pJHMAeTK8qrLLaVVxhIkrCPpZVF2+bNGbRx CkE0MYlyJga9SvmVJZUSikVAb0v1VhEBapVDdHRdQHh47f6jjHOCPhPIwc907C/8XAV5 ERw/9tZZxkLneoOEouQAPyQfREgF290Dcik0rCokfUI8GvgymRwBTA7HgGvWxKZbKziz xoNN+wTCbKZWF1NKs159EXa4TPln+GdBIcwQceDQyYS8oU1lEDtFTHsFamlkx6anYM5T 4vwoxQLHZti2Bvh0J2+zhGGteqtyuw9FzZFGLJpw9+gCXSHVJ0XfhugJUNlXeUBh3EoD LyTQ== X-Forwarded-Encrypted: i=1; AJvYcCVsOV0RxLyY45OXkD37pXEyRR7fMvdj0qTkB1fSuo+evFu5H2iV80hK0zIve1wwrXcxGoi2VVQ7GZQ1PoblInXJu2g= X-Gm-Message-State: AOJu0YzKaIr4zho5TlwI+u33WPBzNN0aA/vNro9kJIn6hfaWgV8mKY2e 7Rl9nKim7mHFziX96GcMDBy5V+DlFwGeK919d0KKWQokLdCkCVTLzg9Uwu990LIkxEdQucA054U wNLbHt/CHtCy4XlLbNouNjvNjYhAuQAiI X-Google-Smtp-Source: AGHT+IGXLMGlpPC9iSsrnp0dsArwZMyvFXIuQtkDUUvWmN0IBfIeW17SsTFZ1CjlCZvPvB4Lf7B234R8xL4PfqMmHl0= X-Received: by 2002:a50:9359:0:b0:57c:77a1:d1da with SMTP id 4fb4d7f45d1cf-57d07c37da2mr1742190a12.0.1718801501780; Wed, 19 Jun 2024 05:51:41 -0700 (PDT) MIME-Version: 1.0 References: <20240618213421.282381-1-shakeel.butt@linux.dev> In-Reply-To: From: Mateusz Guzik Date: Wed, 19 Jun 2024 14:51:28 +0200 Message-ID: Subject: Re: [PATCH] mm: ratelimit oversized kvmalloc warnings instead of once To: Shakeel Butt Cc: Andrew Morton , Michal Hocko , Linus Torvalds , kernel-team@meta.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Kyle McMartin Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: gf8mg9xxufna5wkqeyh4x4ggsrrggnfp X-Rspam-User: X-Rspamd-Queue-Id: 4E82E20002 X-Rspamd-Server: rspam02 X-HE-Tag: 1718801503-719253 X-HE-Meta: U2FsdGVkX1/ZGpxdYnO8S0cgzJkIUN63ixVPBaUPBc2M5krcSvlCuMEYJAZLru531f9AmcxNOr7Y5LfUJ6amyDputgjXimHsSwi5e8w7lLckwnAqNsWzSoFs+d3xSt2CGkIoHhn862eDFeqJlcp3AqD7dPE2YeBs76SL4JPI3haYn/ojpPncwa3eDCGqFIut83mZXpe9JIoMa30iRX0hwGOHEE6EfHfo98xb3DA86cKPUyVgjEs0BWtqY9UHpfW8L5GPH1KvaJ0JRYm2ZfOyxwDTs1pIreFEZe2aemWhecTxI3HnG+PWpiZwaT7FY3m3zr6XPgsn1Nh5rIj/eDrf2BlVlOYa4Kp95wEBxfTsVKE9MUlJWUjK2J4IotrFO4fovIAh/e2qh08krp0hb9OlQ7NpoiNUMPMeIc1IxIHzIwVsa1UunllM4QWJyaJDseaOI6CnHb1CA9f6496GfIQ6tTiGa5duMD9JbVnzp72XgUsHy1qPYXw0wcGf5S1boA/vxaNfs6ZTisKZj6aqj7hTV60IF3z0ys26pOuyLPrxkTTXtRby6cY+gNUK2j9Vw1qYIo0iN5lHsWKzr0aEdIbOLh6Ar737s+Q6q+qf1lHNn2NL7pN4HGIECRS7HlMDcXvRbSPpigmA081gopsYRJPXSP5IAj4oUKY/XHdR5diRIcQxn4x0T0J+4dqDqxOPjEzH+tpwxGonBSBwYKwdhQW2LBFqC66yRhZT5tOvz994Voi8x6wPy5tnIkwARSm/KPnQKLFk+pgv0jv3o8ICJ6ctn3lMEaadjtxBeJtZmcwThLl3K3YUbQ5WjMl3P6zz+omv4Nxe5n457vaBUM3iV1S8dVsToTnMP+nLHNPXG+VQ8E/0EvsKIdZ88wSmcP7xQaxganjMenGmZ4YaEmHqNFFR/5jfbXmfBPGjVwVjlwkEkpV3Qr5ZRPUSCd5dD04N/GwNlz1x3DajykfDDut81lr 6a4Wp3lU hf3tR/VUyR2dTrWUPgDcCf0Ae6Qpj6A43HXCoFCfHA3F86KSwCdlZL4WCbjbsA5cyYe8SxF94gnGEzDUYvFz5uHVkSCWf8PtbayH0GtdZ2fg3ABez59VWJ3Dy+C9L9SEwQi33b4DbpnJoton7vIheUxcVCxk1Qj6wcYxmKEAsEX0FqtQCMCS/6RqG0Xct3efOm97DGFC+ernTOk0a8LDrseEMcu4RN0qlobLyrHIUnWxW1o5eXwxx0H267Bvbriqoc+rnZlMu3ovEMveyqqtYQiAcr3MjU2IqYKDKbtpIQTjAA2UTpErcwrhEGQfyiTUk7skCyVRUX6uUAi5xEURJTm9ep30E+QpaizLV6wjl/ZDzb9gvuUO9n+20a5GgMZpuEGCDVCJZqRT1fiQxDOC/rMBR4t10Jmx8SBkIu0WGYH9hHr4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000019, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Jun 19, 2024 at 2:49=E2=80=AFPM Mateusz Guzik w= rote: > > On Tue, Jun 18, 2024 at 02:34:21PM -0700, Shakeel Butt wrote: > > At the moment oversize kvmalloc warnings are triggered once using > > WARN_ON_ONCE() macro. One issue with this approach is that it only > > detects the first abuser and then ignores the remaining abusers which > > complicates detecting all such abusers in a timely manner. The situatio= n > > becomes worse when the repro has low probability and requires productio= n > > traffic and thus require large set of machines to find such abusers. In > > Mera production, this warn once is slowing down the detection of these > > abusers. Simply replace WARN_ON_ONCE with WARN_RATELIMIT. > > > > Reported-by: Kyle McMartin > > Signed-off-by: Shakeel Butt > > --- > > mm/util.c | 3 ++- > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > diff --git a/mm/util.c b/mm/util.c > > index 10f215985fe5..de36344e8d53 100644 > > --- a/mm/util.c > > +++ b/mm/util.c > > @@ -649,7 +649,8 @@ void *kvmalloc_node_noprof(size_t size, gfp_t flags= , int node) > > > > /* Don't even allow crazy sizes */ > > if (unlikely(size > INT_MAX)) { > > - WARN_ON_ONCE(!(flags & __GFP_NOWARN)); > > + WARN_RATELIMIT(!(flags & __GFP_NOWARN), "size =3D %zu > I= NT_MAX", > > + size); > > return NULL; > > } > > > > I don't think this is necessary. From the description I think interested > parties can get away with bpftrace. > > Suppose you have an abuser of the sort and you are worried there is more > than one. > > Then this one-liner will catch *all* of them, not just the ones which > were "lucky" to get logged with ratelimit: > bpftrace -e 'kprobe:kvmalloc_node_noprof /arg0 > 2147483647/ { @[kstack()= ] =3D count(); }' > > Of course adding a probe is not free, but then again kvmalloc should not > be used often to begin with so I doubt it is going to have material > impact in terms of performance. > > While I concede it takes more effort to get this running on all affected > machines, the result is much better than mere ratelimit. Also there is > no need to patch the kernel. > > btw, I found drm keeps spamming kvmalloc, someone(tm) should look into > it: > @[ > kvmalloc_node_noprof+5 > drm_property_create_blob+76 > drm_atomic_helper_dirtyfb+234 > drm_fbdev_generic_helper_fb_dirty+509 > drm_fb_helper_damage_work+139 > process_one_work+376 > worker_thread+753 > kthread+207 > ret_from_fork+49 > ret_from_fork_asm+26 > , 104]: 12 I should clarify this is allocs of 104 bytes, not some outlandish size. --=20 Mateusz Guzik