From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E0CCC27C4F for ; Thu, 13 Jun 2024 21:10:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C7A576B008A; Thu, 13 Jun 2024 17:10:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C28CC6B008C; Thu, 13 Jun 2024 17:10:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AF0186B0092; Thu, 13 Jun 2024 17:10:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 928946B008A for ; Thu, 13 Jun 2024 17:10:49 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 359731A02F6 for ; Thu, 13 Jun 2024 21:10:49 +0000 (UTC) X-FDA: 82227109818.16.F787DCA Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf27.hostedemail.com (Postfix) with ESMTP id 8E6394000D for ; Thu, 13 Jun 2024 21:10:47 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=NdYumvUV; spf=none (imf27.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718313046; a=rsa-sha256; cv=none; b=q2a5iTFsIQt1bW0FZhYhmmkjE6xRXSim3bTckeVmiB0HsJSH+7UH7HuuxC8w9PzFQRP/3C fEGmob+IYoJvK5ikkjR+UTR4Ncvx3zPhPLCOOJQeutOD38lZkuFevWOvXmVtk8P/+1lgjv Wa5zVLMq95o+EuTGTiJ+zVcXICfAw8I= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=NdYumvUV; spf=none (imf27.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718313046; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0HALQjHlXOL6+WTYDn4qLGcW9dgjlpY1wKU66kRI+YQ=; b=wPEhxxtFhxE7n2qwiUHBIDPNPUwqBXgvdKkag33VgrXsOLdiIQEcC4V+JIKSziIwiWxgG8 3HYPQs5O67FowOV1sPO87j1SWLAiJ2ygTSF8T5efdTtxSdAsLJ/sAZOXUlTm+6e3LA+o7p R6ItgiYkfIUSYalEpuhPIxJ/1lladr0= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=0HALQjHlXOL6+WTYDn4qLGcW9dgjlpY1wKU66kRI+YQ=; b=NdYumvUVsJQZ9dm/svHJBAsMl1 FZdv0vNdmhDdlfWs2gOZhDxs2Szp+j59WUlMwBcxaqgXKhyRV6j7j5dLoWhv1OMrn3hfXAndskAQc 7H6t2QsZuJLki7Mv18C6rIEGzJB4XXtHA0XLirntWt/C2yaSazpSYj5mRaWrxZ0j8fWS8zRfgfVRo NMRBW1Tmmfsrb8SfxHrOskx9iWm7WRoVhr/FaW7f25N2RYIAkupO2bpSjlQROb02vi9pwoFJmqsrh BO/pAagTc1bM0pmVEBnff2vbzW2+WouAi0icO1FOuLvo/ajc2fP46vi9A2CS60sobc8tiI6uZEQhV XMh1JH1g==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHriU-00000000V0D-0Y84; Thu, 13 Jun 2024 21:10:42 +0000 Date: Thu, 13 Jun 2024 14:10:42 -0700 From: Luis Chamberlain To: Zorro Lang Cc: patches@lists.linux.dev, fstests@vger.kernel.org, linux-xfs@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, akpm@linux-foundation.org, ziy@nvidia.com, vbabka@suse.cz, seanjc@google.com, willy@infradead.org, david@redhat.com, hughd@google.com, linmiaohe@huawei.com, muchun.song@linux.dev, osalvador@suse.de, p.raghav@samsung.com, da.gomez@samsung.com, hare@suse.de, john.g.garry@oracle.com Subject: Re: [PATCH 3/5] fstests: add fsstress + compaction test Message-ID: References: <20240611030203.1719072-1-mcgrof@kernel.org> <20240611030203.1719072-4-mcgrof@kernel.org> <20240612080048.dnbc3rzmeo7jtubv@dell-per750-06-vm-08.rhts.eng.pek2.redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240612080048.dnbc3rzmeo7jtubv@dell-per750-06-vm-08.rhts.eng.pek2.redhat.com> X-Stat-Signature: 1x1y51osabhuo8i3dx94p5xwacprobwn X-Rspamd-Queue-Id: 8E6394000D X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1718313047-152181 X-HE-Meta: U2FsdGVkX1/UGnfQuz56cSzReCv02AkIX2ay+fcEZ2YjAssfTTjaX1ponnazy4ticoeFUAfKXsMOgaLGgMe5JmLEy9f3KsoNhAZtoAOkLyP05F7MZAzGtQIDeNrdd1teR9GH+ut+Z1EMrzn445XPp2rrJOl1cXHp/4l7KoK0KMR7cCkxUvk9+njueMnvyzzMlHWRKe9y1vZNO9oQV2Sj9TWYro3/TedfO64J6j3h28U1DkROayWpgxxUERy7Ddqnb79/Il9MmOmDocuw73Mfgg5ZlsvUn3/xL09IolpA/20BQUOBTETEDqAW69KWmmH40igonecbr3RC5EO6vrm/oAClJRNyNRBageMtXIj7cNd0kZ3MUYqh+6KlIIIyROdES6ExBey8/BtvxU+RxCzR5gerU+ExE9HwqTs6lBFrgEEaRHwwojKfJ1lQ/UWWjzwTlGsIDOfWXaSjCGF8zA5YK5LaEC9+oA4B6mYd2y+nMwI238jfe/D/38tiOWUFhsObyoQuEh/6983mqJCZqXIlpuIbOFH+d7LZOWzgq4B42mmUqzuOk5YuGbaguEGlSgBhfY2XNTTMn0vjSUdsNW1Vz1WgLHqQyjiIojce+TI8u4lvxQrL8lShxq+a7pxHzi8OeWvKN7xGUZR8DkVUzXdPbqCzJLFMM5GcQCrDghPilQ88PSkVKWBBNTGcPZAlt/vHu1EY8yI0ytzdxfw1FOyOmsSWmmEWnMJUk3qB/PCS+ZRHr82IVfRqhkxEPO7Y5uf0P6womM0Y7hYW7wWhOtQDqVF4Eo7BNZOR1QnaUSGMYtQC4L1Zz9jwtmd0ET+qagAcoZz7DVsvTd4JHPNIKHfYJvi0j494ExQ9SEs/JH7OxcXoYMFmOEKi2082KZfsPya4Dh+/RMfLOpBgxJqjbhfiOtJq5qkzjQTqGKftPamgRSNnNNFBD2nejusQWmjr9OeX0YEjvD4Vyz4+Iwzljm5 ayEZioHY vbHapqogfwkYTxdoG3TfjHh6mGLQKOcvdao5wTmqewenOTwto2etjA/OFNwXwjXqCdQcv8B4cppNxVY/nWf8q98AfEYRGffy5fkfbqhOax1NFRQ866Gse64jF9zQo6QStH0WtNKw45aKI3usZtt5JN3heruuzTnJs44opx7aDdiCzCahsllgUH3kHKA5iJ/eExrTlEZsOmF5q7B5a5hPWfZT2zWZxtyrLGTgv/3xdRHIttKbMLYdrZGLeHVzOKGHT6m+9suvDhb8EW2C3j0mhiQhqFheNNoPbKkwUMZCBX6b6On7eG4G2WV7pDpF5te8kFC9LJwUk4A3LkTSDvpqIpOz2Sl+c7Yn0U5ax33smS9ZRZI+ts5bPoDLpxr+N5DTH010iFZzPmjfC+R1Z6zzFKs6FHeS3WKHCQbye2pAvusFlJysbobzvWeofbeN7iHIv/92JnfpqVTV0o6KX1Y5QFIv8u3m7e+dzQ9ikV2/fdQPejnY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Jun 12, 2024 at 04:00:48PM +0800, Zorro Lang wrote: > On Mon, Jun 10, 2024 at 08:02:00PM -0700, Luis Chamberlain wrote: > > Running compaction while we run fsstress can crash older kernels as per > > korg#218227 [0], the fix for that [0] has been posted [1] that patch > > was merged on v6.9-rc6 fixed by commit d99e3140a4d3 ("mm: turn > > folio_test_hugetlb into a PageType"). However even on v6.10-rc2 where > > this kernel commit is already merged we can still deadlock when running > > fsstress and at the same time triggering compaction, this is a new > > issue being reported now this through patch, but this patch also > > serves as a reproducer with a high confidence. It always deadlocks. > > If you enable CONFIG_PROVE_LOCKING with the defaults you will end up > > with a complaint about increasing MAX_LOCKDEP_CHAIN_HLOCKS [1], if > > you adjust that you then end up with a few soft lockup complaints and > > some possible deadlock candidates to evaluate [2]. > > > > Provide a simple reproducer and pave the way so we keep on testing this. > > > > Without lockdep enabled we silently deadlock on the first run of the > > test without the fix applied. With lockdep enabled you get a splat about > > the possible deadlock on the first run of the test. > > > > [0] https://bugzilla.kernel.org/show_bug.cgi?id=218227 > > [1] https://gist.github.com/mcgrof/824913b645892214effeb1631df75072 > > [2] https://gist.github.com/mcgrof/926e183d21c5c4c55d74ec90197bd77a > > > > Signed-off-by: Luis Chamberlain > > --- > > common/rc | 7 +++++ > > tests/generic/750 | 62 +++++++++++++++++++++++++++++++++++++++++++ > > tests/generic/750.out | 2 ++ > > 3 files changed, 71 insertions(+) > > create mode 100755 tests/generic/750 > > create mode 100644 tests/generic/750.out > > > > diff --git a/common/rc b/common/rc > > index e812a2f7cc67..18ad25662d5c 100644 > > --- a/common/rc > > +++ b/common/rc > > @@ -151,6 +151,13 @@ _require_hugepages() > > _notrun "Kernel does not report huge page size" > > } > > > > +# Requires CONFIG_COMPACTION > > +_require_vm_compaction() > > +{ > > + if [ ! -f /proc/sys/vm/compact_memory ]; then > > + _notrun "Need compaction enabled CONFIG_COMPACTION=y" > > + fi > > +} > > # Get hugepagesize in bytes > > _get_hugepagesize() > > { > > diff --git a/tests/generic/750 b/tests/generic/750 > > new file mode 100755 > > index 000000000000..334ab011dfa0 > > --- /dev/null > > +++ b/tests/generic/750 > > @@ -0,0 +1,62 @@ > > +#! /bin/bash > > +# SPDX-License-Identifier: GPL-2.0 > > +# Copyright (c) 2024 Luis Chamberlain. All Rights Reserved. > > +# > > +# FS QA Test 750 > > +# > > +# fsstress + memory compaction test > > +# > > +. ./common/preamble > > +_begin_fstest auto rw long_rw stress soak smoketest > > + > > +_cleanup() > > +{ > > + cd / > > + rm -f $runfile > > + rm -f $tmp.* > > + kill -9 $trigger_compaction_pid > /dev/null 2>&1 > > + $KILLALL_PROG -9 fsstress > /dev/null 2>&1 > > + > > + wait > /dev/null 2>&1 > > +} > > + > > +# Import common functions. > > + > > +# real QA test starts here > > + > > +_supported_fs generic > > + > > +_require_scratch > > +_require_vm_compaction > > +_require_command "$KILLALL_PROG" "killall" > > + > > +# We still deadlock with this test on v6.10-rc2, we need more work. > > +# but the below makes things better. > > +_fixed_by_git_commit kernel d99e3140a4d3 \ > > + "mm: turn folio_test_hugetlb into a PageType" > > + > > +echo "Silence is golden" > > + > > +_scratch_mkfs > $seqres.full 2>&1 > > +_scratch_mount >> $seqres.full 2>&1 > > + > > +nr_cpus=$((LOAD_FACTOR * 4)) > > +nr_ops=$((25000 * nr_cpus * TIME_FACTOR)) > > +fsstress_args=(-w -d $SCRATCH_MNT -n $nr_ops -p $nr_cpus) > > + > > +# start a background trigger for memory compaction > > +runfile="$tmp.compaction" > > +touch $runfile > > +while [ -e $runfile ]; do > > + echo 1 > /proc/sys/vm/compact_memory > > + sleep 5 > > +done & > > +trigger_compaction_pid=$! > > + > > +test -n "$SOAK_DURATION" && fsstress_args+=(--duration="$SOAK_DURATION") > > + > > +$FSSTRESS_PROG $FSSTRESS_AVOID "${fsstress_args[@]}" >> $seqres.full > > +wait > /dev/null 2>&1 > > Won't this "wait" wait forever (except a ctrl+C), due to no one removes > the $runfile? Odd, pretty sure I tested it and it didn't wait forever, but I'll add the rm after the FSSTRESS call. Luis