Re: [PATCH v2 00/16] Parallelizing filesystem writeback

[Kundan Kumar <kundan.kumar@samsung.com>]

On 11/7/2025 7:07 PM, Christoph Hellwig wrote:
> On Fri, Nov 07, 2025 at 02:54:42PM +0530, Kundan Kumar wrote:
>> Predicting the Allocation Group (AG) for aged filesystems and passing
>> this information to per-AG writeback threads appears to be a complex
>> task.
> 
> Yes.  But in the end aged file systems are what will see most usage.
> Fresh file systems look nice in benchmarks, but they aren't what
> users will mostly deal with.
> 
>> To segregate these I/O requests by AG, it is necessary to associate
>> AG-specific information with the pages/folios in the page cache. Two
>> possible approaches are:
>> (1) storing AG information in the folio->private field, or
>> (2) introducing new markers in the xarray to track AG-specific data.
>>
>> The AG-affined writeback thread processes specific pages from the page
>> cache marked for its AG. Is this a viable approach, or are there
>> alternative solutions that could be more effective?
> 
> Or maybe the per-AG scheme isn't that great after all and we just
> need some other simple sharding scheme?  Of course lock contention
> will be nicer on a per-AG basis, but as you found out actually
> mapping high-level writeback to AGs is pretty hard.
> 
> 
Thank you for your insightful comments, Christoph. I'm considering using
folio private to incorporate IOMAP type and predicted AG information.
The prediction for DELALLOC, using pagf_freeblks etc., and for UNWRITTEN
and MAPPED, using the actual location of allocated blocks.

Subsequently, schedule all writeback threads for the inode. With all the
necessary information, these threads will be able to filter AG-specific
folios and focus on those marked for the corresponding AG. Although this
approach may seem complex, it should effectively address various use
cases, including aged filesystems, filesystem fragmentation, and locking
concerns.

We tried CPU and inode based sharding, CPU based sharding increases
filesystem fragmentation, and inode based sharding results in AG lock
contention. We adopted AG-based sharding to resolve these issues.