From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A32EC433FE for ; Wed, 20 Oct 2021 06:11:28 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id F14856008E for ; Wed, 20 Oct 2021 06:11:27 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org F14856008E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=canonical.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id 846846B0071; Wed, 20 Oct 2021 02:11:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7CF756B0072; Wed, 20 Oct 2021 02:11:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 649736B0073; Wed, 20 Oct 2021 02:11:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0195.hostedemail.com [216.40.44.195]) by kanga.kvack.org (Postfix) with ESMTP id 4EDA26B0071 for ; Wed, 20 Oct 2021 02:11:27 -0400 (EDT) Received: from smtpin01.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id C8FA629E0F for ; Wed, 20 Oct 2021 06:11:26 +0000 (UTC) X-FDA: 78715793772.01.986FCCC Received: from smtp-relay-internal-1.canonical.com (smtp-relay-internal-1.canonical.com [185.125.188.123]) by imf06.hostedemail.com (Postfix) with ESMTP id 8B4C4801A8A0 for ; Wed, 20 Oct 2021 06:11:25 +0000 (UTC) Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-1.canonical.com (Postfix) with ESMTPS id 306F03F4BA for ; Wed, 20 Oct 2021 06:11:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1634710283; bh=1Pwbto3xzDOuj9AdmGAawkkhjI3NOhcoDznc6EjuA4E=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:In-Reply-To; b=TV6/5zrABAxI7gQxS83/Nk+2lL1HQB7BRGC0eYminhVl5tQW0jxNHw6arMvHACJJ4 WwSArGpnJRO2pZrlgCSLdBduPnLd1U6ow6wPZ/2I0WudKeLykajGMq9sFnZ3SC1j1W IafoLvBZO0yLXZxe1dp5rNkqFlc520kZHt9iT8VSWwx30Mo4iDTsL0Fmj9KyhkvslU s8qCdKDwoQIrQw9Tz98FC8k+8D46f0nJlKZlfMhtgnH4sgjNG/pcALydwsxsPuskwr K5fuCYUlHruHLAIAnTqx/aKUqllnmtvIH1BH97GE3n9UNdm48BflTQlmE/9gOfbVIT 2Oh7chBAoJFzQ== Received: by mail-ed1-f71.google.com with SMTP id d3-20020a056402516300b003db863a248eso19889185ede.16 for ; Tue, 19 Oct 2021 23:11:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=1Pwbto3xzDOuj9AdmGAawkkhjI3NOhcoDznc6EjuA4E=; b=BlQ4fFAQ/JWhWAbs0vWudNGEwXLd09+LJBDVJAjln71am/BDx69CbiTCuoTl4IquAE EWmFAOls2/7o+tqbm8S8QTElIWR7vbj6lfRHvF4I9JA4oNlyFvKICxRSryiz337HfqEh S7WFomBSr8QwbjWTnWBvdg3yEdYsCJic83OA+cUhxvU4ow4bhyDB3QSLt67oyLwkjIBi M9WYGKT6XJ6rDn7yekw3Gcd87+wkJ5L7eYZS7JLWOhurqlcmf3lYr6ugaUhdYiIlkXoY SxsQaxs1OJAtwx6nJniDPqgW6+//eNUZBn0PVDir0LMpv5kZTiXWZd7MJZYb4vfVwph2 lsKQ== X-Gm-Message-State: AOAM533nIuX1Mf3/XZDpm45gabt4Th3v0mqHqwBPKr4DfcsIqvXh+aZ3 Mzr3A0VKjxtSKRUvkeU77wYk0GfF6rQvmEDf7zxLvv9SohwHnvjPrZ4zpJpZdPMyAUN619DkOa/ 852q3AAZgfRjr6ogy6dcU/aiDefty X-Received: by 2002:a50:9d49:: with SMTP id j9mr59027135edk.39.1634710282865; Tue, 19 Oct 2021 23:11:22 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzy3CpnuvgbkTXuc9O1L/YV9fI7tJfyLLF6gi248jNNmlGukEUPfWeAFLRbyb7gPAFDmncXDQ== X-Received: by 2002:a50:9d49:: with SMTP id j9mr59027121edk.39.1634710282707; Tue, 19 Oct 2021 23:11:22 -0700 (PDT) Received: from localhost ([2001:67c:1560:8007::aac:c1b6]) by smtp.gmail.com with ESMTPSA id e7sm573903edz.95.2021.10.19.23.11.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Oct 2021 23:11:22 -0700 (PDT) Date: Wed, 20 Oct 2021 08:11:21 +0200 From: Andrea Righi To: Marco Elver Cc: Dmitry Vyukov , Alexander Potapenko , kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: Re: BUG: soft lockup in __kmalloc_node() with KFENCE enabled Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 8B4C4801A8A0 X-Stat-Signature: b4dye35hxyzsytbjxejcoipck5kcsz7q Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=canonical.com header.s=20210705 header.b="TV6/5zrA"; spf=pass (imf06.hostedemail.com: domain of andrea.righi@canonical.com designates 185.125.188.123 as permitted sender) smtp.mailfrom=andrea.righi@canonical.com; dmarc=pass (policy=none) header.from=canonical.com X-HE-Tag: 1634710285-41153 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Oct 20, 2021 at 08:00:00AM +0200, Marco Elver wrote: > On Mon, 11 Oct 2021 at 16:42, Andrea Righi wrote: > > On Mon, Oct 11, 2021 at 12:03:52PM +0200, Marco Elver wrote: > > > On Mon, 11 Oct 2021 at 11:53, Andrea Righi wrote: > > > > On Mon, Oct 11, 2021 at 11:23:32AM +0200, Andrea Righi wrote: > > > > ... > > > > > > You seem to use the default 20s stall timeout. FWIW syzbot uses 160 > > > > > > secs timeout for TCG emulation to avoid false positive warnings: > > > > > > https://github.com/google/syzkaller/blob/838e7e2cd9228583ca33c49a39aea4d863d3e36d/dashboard/config/linux/upstream-arm64-kasan.config#L509 > > > > > > There are a number of other timeouts raised as well, some as high as > > > > > > 420 seconds. > > > > > > > > > > I see, I'll try with these settings and see if I can still hit the soft > > > > > lockup messages. > > > > > > > > Still getting soft lockup messages even with the new timeout settings: > > > > > > > > [ 462.663766] watchdog: BUG: soft lockup - CPU#2 stuck for 430s! [systemd-udevd:168] > > > > [ 462.755758] watchdog: BUG: soft lockup - CPU#3 stuck for 430s! [systemd-udevd:171] > > > > [ 924.663765] watchdog: BUG: soft lockup - CPU#2 stuck for 861s! [systemd-udevd:168] > > > > [ 924.755767] watchdog: BUG: soft lockup - CPU#3 stuck for 861s! [systemd-udevd:171] > > > > > > The lockups are expected if you're hitting the TCG bug I linked. Try > > > to pass '-enable-kvm' to the inner qemu instance (my bad if you > > > already have), assuming that's somehow easy to do. > > > > If I add '-enable-kvm' I can triggering other random panics (almost > > immediately), like this one for example: > > Just FYI: https://lkml.kernel.org/r/20211019102524.2807208-2-elver@google.com > > But you can already flip that switch in your config > (CONFIG_KFENCE_STATIC_KEYS=n), which we recommend as a default now. > > As a side-effect it'd also make your QEMU TCG tests pass. Cool! Thanks for the update! And about the other panic that I was getting it seems to be fixed by this one: https://lore.kernel.org/lkml/YW6N2qXpBU3oc50q@arighi-desktop/T/#u -Andrea