From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F0839C27C4F for ; Tue, 11 Jun 2024 03:02:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A1F0D6B00A6; Mon, 10 Jun 2024 23:02:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7D8586B00A9; Mon, 10 Jun 2024 23:02:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4A20A6B00AA; Mon, 10 Jun 2024 23:02:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 0F8306B00A6 for ; Mon, 10 Jun 2024 23:02:11 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id B28CB14113C for ; Tue, 11 Jun 2024 03:02:10 +0000 (UTC) X-FDA: 82217108820.07.95644D3 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by imf18.hostedemail.com (Postfix) with ESMTP id 147971C000A for ; Tue, 11 Jun 2024 03:02:07 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=tyK80xDR; spf=none (imf18.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=none) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1718074929; a=rsa-sha256; cv=none; b=yqMzjhW1y31vY3K1h8Yl43HZD070bAYJys0/1JstxHDDcp9oV3oQbBkRwmigawYO/y2gSp a28oX5EWreWB9QV/SKRjX2v9JPDf0/0RkPKkpJug65X4V3zdbCb7PGiZRhTgt5dMyvFM9o GcEJ4pYURs4PkScztkY66d5gJXF4y4g= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=infradead.org header.s=bombadil.20210309 header.b=tyK80xDR; spf=none (imf18.hostedemail.com: domain of mcgrof@infradead.org has no SPF policy when checking 198.137.202.133) smtp.mailfrom=mcgrof@infradead.org; dmarc=fail reason="No valid SPF, DKIM not aligned (relaxed)" header.from=kernel.org (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1718074929; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=w5upqqK3le9bkA35Adh9sB0Tb3TWHmZ5mS+iqjEKnts=; b=M919FRPfwhPlIJ0U5tPxl/eQjzADflfZ/8OlVNtcYvXRFfPq2Z3NpRwcxVMBDbDfeKt+Ks hHJmc58gA7OpzuN/yH7v+6CbUD/zoFtcqZS35hA4yi6SXRSI+nwLv4ZSQ/0euvhqUhsm5x JGyDX39qi2AOK7U/3aPzhuasE6lqtHI= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=w5upqqK3le9bkA35Adh9sB0Tb3TWHmZ5mS+iqjEKnts=; b=tyK80xDR4FjkmXamvnGufFwee/ osrHpUf1HtQ7JnZlKRPjDbwP6VxoRLDshUf1idmTDgzPQ+o+IBHMJw4lIaXvBME2ey/SQOv4qJHE0 VoOAOYx1Dw7Xetp3OUDD8IhANx6OsQV38M5S+ialE3F3kBNW+2cWU/ynh9I7FtSqdx20fyr9ly/b/ JdZmLCHGKzswNupQYtEkfjAcNXED2JcUObhdgPkEmjgLTSmJm+An6HmkF8VY9B/DICy229Ek8ILcP /j/RAriVVoSgBiq96+VcYOrrOuvfIOEv2Q5UTveYEUa60fNtFjy28iq9i34Wl6bN+8AWd/CaiRwqx BaXeOayw==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1sGrls-00000007DDO-2Yy9; Tue, 11 Jun 2024 03:02:04 +0000 From: Luis Chamberlain To: patches@lists.linux.dev, fstests@vger.kernel.org Cc: linux-xfs@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, akpm@linux-foundation.org, ziy@nvidia.com, vbabka@suse.cz, seanjc@google.com, willy@infradead.org, david@redhat.com, hughd@google.com, linmiaohe@huawei.com, muchun.song@linux.dev, osalvador@suse.de, p.raghav@samsung.com, da.gomez@samsung.com, hare@suse.de, john.g.garry@oracle.com, mcgrof@kernel.org Subject: [PATCH 5/5] fstests: add stress truncation + writeback test Date: Mon, 10 Jun 2024 20:02:02 -0700 Message-ID: <20240611030203.1719072-6-mcgrof@kernel.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240611030203.1719072-1-mcgrof@kernel.org> References: <20240611030203.1719072-1-mcgrof@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Stat-Signature: iagsewa8t6khshq4fawkdkmxzqcxd7gz X-Rspamd-Queue-Id: 147971C000A X-Rspamd-Server: rspam04 X-Rspam-User: X-HE-Tag: 1718074927-440160 X-HE-Meta: U2FsdGVkX1+dpbcLaFYy6/kkbc1gy+B/Ys034DSGMI2nHaWZEoCpIgz3IZFYjAa9DzMrMpn09yLlqSZy83U3yRuGqVE6n7Q9UVEVgTHJXgc3Q9CJK+tzSBG5KqEi/t/b0bhA9IVsXWC9nnG/8UVtKGdwTwM1CTXsq51LHpcSMicMlanRO4h3LbLpYjW9ZgLPyJxK+b8yvUwhqVKBjEwjB09Y9/YEPdWL+mHM67YhFVAOkL1dpjSH0Dl4oRY4cEUY7hmkkpdpDZAsSIVyRN9nPUj6rR6Y83rOyzfF0C2n93Sx0WSY8FOz+X7+r/5sBIFNMOnlUKtuWR9rQCoVRUoRx+Ml5unXuUFUHfvSxslNucX4oQuJMQJJcS4CCAZDaHafVGIMPLFFWhRhfwHJhzeNM6H5VEHGSnLG7HrJtehqCtYgoFlKB3PickMBw3dULoiRvMK0OQ27N9JNMIc2tgwFA1UwKZooc2HSy7dIy80Hxna9Hb7ug8K1fJ0hhi/KsJLTsBpjpF33jHEzU2xnvVn268411JAwSPJWMe4/1Ujsor7piajHc/ZWBIQFJmcSBQuybwJ4vYCnFoUdZJYmzaPRIehAS9qKICGKitFOQBab8b/x6200eRSxPEU2/ZhYEeeEfZmO6kuJ5suUZ1gO8Gd9T67oSkZBKptix9NksEJKw+PTSYz2+vupmseqSszUIsvmPiqPEg2Ej8YrpDH/8X2gWXu9xkcw3WB3zZsKGjFzJom2WZafBnCTv+S7RMhmEwQM4fpUMmntxRGxY7x7mpWwmOOMg6VOshlsEOWWfP6nPYtQBSGDY3336dPSJcV4+rv6nn2M4hApP+Os+8PRPoLxh+rT9ruo/iKw/lVqKzT5j30BXbqHdX9286CLslzhRRSAPcbhPt2GtB15nvLytJhEHTnwasdKRsfPq8lheDa7VuMqx+EK/XRhTvHPlUYJlLjcpEIjB9Tzg4MurJ8AHE8 BmTibvTP PinoHAnnIjJRsJO+7eVABgtNIinf8b9rdY265tQ0tzUJpKLvQKySfvabF6gIHEj+WHP2UrtJ9MNCJWGVBG99G+1UX4zx+wNXrer2mWlAfK4CGkOj22b7PNhTZi4CgHGnhENvywlRIQ7GRBZg7PTLqWKhN8OLIDflJ1TIuXwDdN7RGbnF6zp1bWAT3+TjTNkjfexjlkthbDvSccmv4ZT+i83xEbndemgB+0ADaA+xruUdzeY1O3jWN4c7AGOfm/f0/QL7eqafxghzyqI81npNCMcQ+RqU8rEdE2rvYB1C58bRYj2SW+Yoel9aktlW3zAFIuuhcdOdqwQXDLSRUpJ8/o9Ous3WoATMGYXQJhuMmZVAP2x6IgSF/J3ZiSxxjDENnlxD6p4WfJhA5yEZyRz1L7aJmY7rIShoD0M+08mXFfxkFsudgIdz94etxHfSU3KdZm0yz X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Stress test folio splits by using the new debugfs interface to a target a new smaller folio order while triggering writeback at the same time. This is known to only creates a crash with min order enabled, so for example with a 16k block sized XFS test profile, an xarray fix for that is merged already. This issue is fixed by kernel commit 2a0774c2886d ("XArray: set the marks correctly when splitting an entry"). If inspecting more closely, you'll want to enable on your kernel boot: dyndbg='file mm/huge_memory.c +p' Since we want to race large folio splits we also augment the full test output log $seqres.full with the test specific number of successful splits from vmstat thp_split_page and thp_split_page_failed. The larger the vmstat thp_split_page the more we stress test this path. This test reproduces a really hard to reproduce crash immediately. Signed-off-by: Luis Chamberlain --- common/rc | 14 +++++ tests/generic/751 | 127 ++++++++++++++++++++++++++++++++++++++++++ tests/generic/751.out | 2 + 3 files changed, 143 insertions(+) create mode 100755 tests/generic/751 create mode 100644 tests/generic/751.out diff --git a/common/rc b/common/rc index 30beef4e5c02..60f572108818 100644 --- a/common/rc +++ b/common/rc @@ -158,6 +158,20 @@ _require_vm_compaction() _notrun "Need compaction enabled CONFIG_COMPACTION=y" fi } + +# Requires CONFIG_DEBUGFS and truncation knobs +_require_split_debugfs() +{ + if [ ! -f $DEBUGFS_MNT/split_huge_pages ]; then + _notrun "Needs CONFIG_DEBUGFS and split_huge_pages" + fi +} + +_split_huge_pages_all() +{ + echo 1 > $DEBUGFS_MNT/split_huge_pages +} + # Get hugepagesize in bytes _get_hugepagesize() { diff --git a/tests/generic/751 b/tests/generic/751 new file mode 100755 index 000000000000..7cc96054a5a9 --- /dev/null +++ b/tests/generic/751 @@ -0,0 +1,127 @@ +#! /bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2024 Luis Chamberlain. All Rights Reserved. +# +# FS QA Test No. 751 +# +# stress page cache truncation + writeback +# +# This aims at trying to reproduce a difficult to reproduce bug found with +# min order. The issue was root caused to an xarray bug when we split folios +# to another order other than 0. This functionality is used to support min +# order. The crash: +# +# https://gist.github.com/mcgrof/d12f586ec6ebe32b2472b5d634c397df +# +# This may also find future truncation bugs in the future, as truncating any +# mapped file through the collateral of using echo 1 > split_huge_pages will +# always respect the min order. Truncating to a larger order then is excercised +# when this test is run against any filesystem LBS profile or an LBS device. +# +# If you're enabling this and want to check underneath the hood you may want to +# enable: +# +# dyndbg='file mm/huge_memory.c +p' +# +# This tests aims at increasing the rate of successful truncations so we want +# to increase the value of thp_split_page in $seqres.full. Using echo 1 > +# split_huge_pages is extremely aggressive, and even accounts for anonymous +# memory on a system, however we accept that tradeoff for the efficiency of +# doing the work in-kernel for any mapped file too. Our general goal here is to +# race with folio truncation + writeback. + +. ./common/preamble + +_begin_fstest auto long_rw stress soak smoketest + +# Override the default cleanup function. +_cleanup() +{ + cd / + rm -f $tmp.* + rm -f $runfile + kill -9 $split_huge_pages_files_pid > /dev/null 2>&1 +} + +fio_config=$tmp.fio +fio_out=$tmp.fio.out + +# real QA test starts here +_supported_fs generic +_require_test +_require_scratch +_require_debugfs +_require_split_debugfs +_require_command "$KILLALL_PROG" "killall" +_fixed_by_git_commit kernel 2a0774c2886d \ + "XArray: set the marks correctly when splitting an entry" + +# we need buffered IO to force truncation races with writeback in the +# page cache +cat >$fio_config <>$seqres.full 2>&1 +_scratch_mount >> $seqres.full 2>&1 + +# used to let our loops know when to stop +runfile="$tmp.keep.running.loop" +touch $runfile + +# The background ops are out of bounds, the goal is to race with fsstress. + +# Force folio split if possible, this seems to be screaming for MADV_NOHUGEPAGE +# for large folios. +while [ -e $runfile ]; do + _split_huge_pages_all >/dev/null 2>&1 +done & +split_huge_pages_files_pid=$! + +split_count_before=0 +split_count_failed_before=0 + +if grep -q thp_split_page /proc/vmstat; then + split_count_before=$(grep ^thp_split_page /proc/vmstat | head -1 | awk '{print $2}') + split_count_failed_before=$(grep ^thp_split_page_failed /proc/vmstat | head -1 | awk '{print $2}') +else + echo "no thp_split_page in /proc/vmstat" >> $seqres.full +fi + +# we blast away with large writes to force large folio writes when +# possible. +echo -e "Running fio with config:\n" >> $seqres.full +cat $fio_config >> $seqres.full +$FIO_PROG $fio_config --alloc-size=$(( $(nproc) * 8192 )) --output=$fio_out + +rm -f $runfile + +wait > /dev/null 2>&1 + +if grep -q thp_split_page /proc/vmstat; then + split_count_after=$(grep ^thp_split_page /proc/vmstat | head -1 | awk '{print $2}') + split_count_failed_after=$(grep ^thp_split_page_failed /proc/vmstat | head -1 | awk '{print $2}') + thp_split_page=$((split_count_after - split_count_before)) + thp_split_page_failed=$((split_count_failed_after - split_count_failed_before)) + + echo "vmstat thp_split_page: $thp_split_page" >> $seqres.full + echo "vmstat thp_split_page_failed: $thp_split_page_failed" >> $seqres.full +fi + +status=0 +exit diff --git a/tests/generic/751.out b/tests/generic/751.out new file mode 100644 index 000000000000..6479fa6f1404 --- /dev/null +++ b/tests/generic/751.out @@ -0,0 +1,2 @@ +QA output created by 751 +Silence is golden -- 2.43.0