From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EAA9AC5B555 for ; Tue, 3 Jun 2025 13:19:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 85B3E6B0447; Tue, 3 Jun 2025 09:19:08 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 832ED6B0448; Tue, 3 Jun 2025 09:19:08 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7701C6B0449; Tue, 3 Jun 2025 09:19:08 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 558696B0447 for ; Tue, 3 Jun 2025 09:19:08 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 030ED809B6 for ; Tue, 3 Jun 2025 13:19:07 +0000 (UTC) X-FDA: 83514145176.01.D668461 Received: from mailout4.samsung.com (mailout4.samsung.com [203.254.224.34]) by imf03.hostedemail.com (Postfix) with ESMTP id 745DB2000F for ; Tue, 3 Jun 2025 13:19:04 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=samsung.com header.s=mail20170921 header.b=WBSQ6lK9; dmarc=pass (policy=none) header.from=samsung.com; spf=pass (imf03.hostedemail.com: domain of anuj20.g@samsung.com designates 203.254.224.34 as permitted sender) smtp.mailfrom=anuj20.g@samsung.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1748956745; a=rsa-sha256; cv=none; b=yEsPigNlxhYFV1garX5EsmMwq4opIf9PmGlCkQsbMSmtOn+RHk7OjhT7hrAQvntG5lPkPL X6RycNGprRwajmAPtvJqS49tWBcJeyWlA1KK4hGV/rjsjN2k6zT5VRgY5VsVzI2JkUZSjm hWpxDBLGpxuHsE27pXPjnXPealy+W24= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=samsung.com header.s=mail20170921 header.b=WBSQ6lK9; dmarc=pass (policy=none) header.from=samsung.com; spf=pass (imf03.hostedemail.com: domain of anuj20.g@samsung.com designates 203.254.224.34 as permitted sender) smtp.mailfrom=anuj20.g@samsung.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1748956745; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=lhg+yoxzmSUI95r9m4I7c09Wc8JZvBjC2Fu14XUQZiQ=; b=8o4eupI3xiSmlz9Dv6HOAZtghUXpodhZVOTozvEFBlb8pSTJf/3iax14+HEVd6IwQHxoA2 qiiD/0Pod66HZItyvverCxUWHT9SPddS4Z38tmUnum1YmGBTBhHbcbNgm5euk0YmyAdTX1 uprwkro35cvMkYtPKMsvrNoyCZ7BsxY= Received: from epcas5p2.samsung.com (unknown [182.195.41.40]) by mailout4.samsung.com (KnoxPortal) with ESMTP id 20250603131900epoutp04330019ad3f618f47627d26acdcc85323~FiuOns31u3179631796epoutp040 for ; Tue, 3 Jun 2025 13:19:00 +0000 (GMT) DKIM-Filter: OpenDKIM Filter v2.11.0 mailout4.samsung.com 20250603131900epoutp04330019ad3f618f47627d26acdcc85323~FiuOns31u3179631796epoutp040 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=samsung.com; s=mail20170921; t=1748956740; bh=lhg+yoxzmSUI95r9m4I7c09Wc8JZvBjC2Fu14XUQZiQ=; h=Date:Subject:To:Cc:From:In-Reply-To:References:From; b=WBSQ6lK9EWDgjnVk2UCtHoWdiBqZlqtb8XsrOvFf6dvZb1mPriE1FWBFdCTIU+Omt sfXwfjHouTr1qc88PlriEG7dT8fdI458jv1zxM7KNQWgvhbWmx/vMIisreunoFO5cx LXs0VIUkGSuE72lyDrDiyTlCYGseOg1a61E2t5IQ= Received: from epsnrtp01.localdomain (unknown [182.195.42.153]) by epcas5p3.samsung.com (KnoxPortal) with ESMTPS id 20250603131859epcas5p3adb125b08bbc9f901dde4353e77a8e0a~FiuNydMNP1272412724epcas5p3Q; Tue, 3 Jun 2025 13:18:59 +0000 (GMT) Received: from epcas5p3.samsung.com (unknown [182.195.38.178]) by epsnrtp01.localdomain (Postfix) with ESMTP id 4bBWWQ3Z6rz6B9m9; Tue, 3 Jun 2025 13:18:58 +0000 (GMT) Received: from epsmtip1.samsung.com (unknown [182.195.34.30]) by epcas5p3.samsung.com (KnoxPortal) with ESMTPA id 20250603091626epcas5p3c6680e3a112b654ee64a2a45ee05c29c~Ffab6u3oY3154931549epcas5p3V; Tue, 3 Jun 2025 09:16:26 +0000 (GMT) Received: from [107.122.10.194] (unknown [107.122.10.194]) by epsmtip1.samsung.com (KnoxPortal) with ESMTPA id 20250603091621epsmtip1b17ae68149000f4e610433391a5ab9c2~FfaXdde6I1152711527epsmtip1N; Tue, 3 Jun 2025 09:16:21 +0000 (GMT) Message-ID: Date: Tue, 3 Jun 2025 14:46:20 +0530 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH 00/13] Parallelizing filesystem writeback To: Christoph Hellwig , Kundan Kumar Cc: jaegeuk@kernel.org, chao@kernel.org, viro@zeniv.linux.org.uk, brauner@kernel.org, jack@suse.cz, miklos@szeredi.hu, agruenba@redhat.com, trondmy@kernel.org, anna@kernel.org, akpm@linux-foundation.org, willy@infradead.org, mcgrof@kernel.org, clm@meta.com, david@fromorbit.com, amir73il@gmail.com, axboe@kernel.dk, ritesh.list@gmail.com, djwong@kernel.org, dave@stgolabs.net, p.raghav@samsung.com, da.gomez@samsung.com, linux-f2fs-devel@lists.sourceforge.net, linux-fsdevel@vger.kernel.org, gfs2@lists.linux.dev, linux-nfs@vger.kernel.org, linux-mm@kvack.org, gost.dev@samsung.com, anuj1072538@gmail.com, kundanthebest@gmail.com Content-Language: en-US From: Anuj Gupta/Anuj Gupta In-Reply-To: <20250602141904.GA21996@lst.de> Content-Transfer-Encoding: 7bit X-CMS-MailID: 20250603091626epcas5p3c6680e3a112b654ee64a2a45ee05c29c X-Msg-Generator: CA Content-Type: text/plain; charset="utf-8" X-Sendblock-Type: REQ_APPROVE CMS-TYPE: 105P cpgsPolicy: CPGSC10-542,Y X-CFilter-Loop: Reflected X-CMS-RootMailID: 20250529113215epcas5p2edd67e7b129621f386be005fdba53378 References: <20250529111504.89912-1-kundan.kumar@samsung.com> <20250602141904.GA21996@lst.de> X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 745DB2000F X-Stat-Signature: 8xi3xk9w5fwqnujdo4k3wg8ksbxi8izr X-Rspam-User: X-HE-Tag: 1748956744-814174 X-HE-Meta: U2FsdGVkX1/vbEFlJMYtef+7cPJquTXQI35rERbDkhClTSv/TKWKGtTdnTJ1rvPm11i7XyLxi7qOQAVpu60h9iYfcBhvwo2mFmRk2TALi8e5125YBk36V6jYwixiwfyYaoCmAdSg3h4oaOhr1TVvYtvfnDP9rRvQrsuWE29gkp6DvXcZIvSaynM5+XMNYXBU3YCjraHErzVSsEiW4BNAaPaW7ls03Eh0wxhcXUGJz7Sgl2TmdmirjTFeTcqPnWKo9RYFcPMMvpvDDnWn4vSSKFPZ0QJrSziz4f4HBqaTTKgyeH5AS606eqWOISVPW4ikJcXqKLFPM7KnNdcq/V41HK4S4zAqWLYnuXpqeKFyp0oLKlzKH3BuA0XzMHmq/Scvt2c4bUTcb9eS0Ghq9o1nPF6jFHkXC4Lrbx0ayPGDayFPG/aZOaX3t9P/Pg45CfRSa82lALnmfKQqmUYOvd6psO/P5xL1GZ885d1Z0kyzxry+RAwIfk872tKZWuJzBD5fTBwda3HCXNxqACXD+eOBVHuyvkJ7jh7u8IsCaoUaAbnXei9hwJ+nezoYdqxn2Alh3/2ez7GhNrDOXgvfko7xyt5+E2AWgxz/wUvbU5kKyY7djyJQD/1YCL1nEiV5PIyX9ApN6FVDfrteL39j0NtmiSUCoWrqMBj0mR0N6SE73NF/ifu7Fed8ggtMbvvFy2GONjR0oNZnS2tkSWXM5VkHMgpzHYxoNB7Pkwj0YZCtzfV9XBMKOHLZrVkPanh1IhSE5worN8i7CCxSvLSYd0l7uUH6wFoaicQJeonisVp10wX+uUm0UqfjGKrJ6TOPMjJ84acRuL7Ys7ffqvgVjbC0Z1usi0XakSXI9wgeDhN0o0i5uxG6R4Bojlso5LUr9e3K7/qDZ+5rtdRmH5pRfL5jO2VVqhrFNzmvC0NyDEwZ9/trA2zO0zmxBaDcTtGa4rX1r4uF/fwmefQiTZRRsNO Dn8vAmuY h5Zr+8837Lqqray12yLdLiAwkLcjtyApsN4RurQJ+J4nJHT1QFC7ah4Rj8gqFqjI5Cp2DmhXD/78oxZO1+UwzYqX8S12TKKGvJke0kvT3prjHhc3OzAIjBlub/yGRSR+SrNuqsjDP5S5CqM0crUq8Zi12FfQcwHk2EqmCH0DDPAdYG5vlRt4u+7iJyTQKRRgOyEDl+E3i/RYuI48= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 6/2/2025 7:49 PM, Christoph Hellwig wrote: > On Thu, May 29, 2025 at 04:44:51PM +0530, Kundan Kumar wrote: > Well, the proper thing would be to figure out a good default and not > just keep things as-is, no? We observed that some filesystems, such as Btrfs, don't benefit from this infra due to their distinct writeback architecture. To preserve current behavior and avoid unintended changes for such filesystems, we have kept nr_wb_ctx=1 as the default. Filesystems that can take advantage of parallel writeback (xfs, ext4) can opt-in via a mount option. Also we wanted to reduce risk during initial integration and hence kept it as opt-in. > >> IOPS and throughput >> =================== >> We see significant improvement in IOPS across several filesystem on both >> PMEM and NVMe devices. >> >> Performance gains: >> - On PMEM: >> Base XFS : 544 MiB/s >> Parallel Writeback XFS : 1015 MiB/s (+86%) >> Base EXT4 : 536 MiB/s >> Parallel Writeback EXT4 : 1047 MiB/s (+95%) >> >> - On NVMe: >> Base XFS : 651 MiB/s >> Parallel Writeback XFS : 808 MiB/s (+24%) >> Base EXT4 : 494 MiB/s >> Parallel Writeback EXT4 : 797 MiB/s (+61%) > > What worksload was this? Number of CPUs = 12 System RAM = 16G For XFS number of AGs = 4 For EXT4 BG count = 28616 Used PMEM of 6G and NVMe SSD of 3.84 TB fio command line : fio --directory=/mnt --name=test --bs=4k --iodepth=1024 --rw=randwrite --ioengine=io_uring --time_based=1 -runtime=60 --numjobs=12 --size=450M --direct=0 --eta-interval=1 --eta-newline=1 --group_reporting Will measure the write-amp and share. > > How many CPU cores did the system have, how many AGs/BGs did the file > systems have? What SSD/Pmem was this? Did this change the write > amp as measure by the media writes on the NVMe SSD? > > Also I'd be really curious to see numbers on hard drives. > >> We also see that there is no increase in filesystem fragmentation >> # of extents: >> - On XFS (on PMEM): >> Base XFS : 1964 >> Parallel Writeback XFS : 1384 >> >> - On EXT4 (on PMEM): >> Base EXT4 : 21 >> Parallel Writeback EXT4 : 11 > > How were the number of extents counts given that they look so wildly > different? > > Issued random write of 1G using fio with fallocate=none and then measured the number of extents, after a delay of 30 secs : fio --filename=/mnt/testfile --name=test --bs=4k --iodepth=1024 --rw=randwrite --ioengine=io_uring --fallocate=none --numjobs=1 --size=1G --direct=0 --eta-interval=1 --eta-newline=1 --group_reporting For xfs used this command: xfs_io -c "stat" /mnt/testfile And for ext4 used this: filefrag /mnt/testfile