Re: [PATCH 8/8] RFC: use a TASK_FIFO kthread for read completion support

[Dave Chinner <dgc@kernel.org>]

On Sat, Apr 11, 2026 at 07:44:43AM +0800, Gao Xiang wrote:
> 
> 
> On 2026/4/11 06:11, Dave Chinner wrote:
> > On Thu, Apr 09, 2026 at 06:02:21PM +0200, Christoph Hellwig wrote:
> > > Commit 3fffb589b9a6 ("erofs: add per-cpu threads for decompression as an
> > > option") explains why workqueue aren't great for low-latency completion
> > > handling.  Switch to a per-cpu kthread to handle it instead.  This code
> > > is based on the erofs code in the above commit, but further simplified
> > > by directly using a kthread instead of a kthread_work.
> > > 
> > > Signed-off-by: Christoph Hellwig <hch@lst.de>
> > 
> > Can we please not go back to the (bad) old days of individual
> > subsystems needing their own set of per-cpu kernel tasks just
> > sitting around idle most of of the time?  The whole point of the
> > workqueue infrastructure was to get rid of this widely repeated
> > anti-pattern.
> > 
> > If there's a latency problem with workqueue scheduling, then we
> > should be fixing that problem rather than working around it in every
> > subsystem that thinkgs it has a workqueue scheduling latency
> > issue...
> 
> It has been "fixed" but never actually get fixed:
> https://lore.kernel.org/r/CAB=BE-QaNBn1cVK6c7LM2cLpH_Ck_9SYw-YDYEnNrtwfoyu81Q@mail.gmail.com
> 
> and workqueues don't have any plan to introduce RT threads;

They don't need to (or should) introduce RT threads. Per-cpu kernel
threads already get priority over normal user tasks on scheduling
decisions.  However, they do not pre-empt running kernel tasks of
the same priority.

In general, kernel threads should not use RT scheduling at all - if
the kernel uses RT prioprity tasks then that can interfere with user
scheduled RT tasks. This is especially true in this case where a
non-RT tasks issue the IO, and the IO completion is then scheduled
with RT priority. IOWs, any unprivileged user can now impact the
processing time available to, and the response latency of, other
RT scheduled tasks the system is running.

Tejun asked Sandeep if setting the workqueue thread priority to
-19 through sysfs (i.e. making them higher priority than normal
kernel threads) had the same effect on latency as using a dedicated
per-cpu RT task thread. THere was no followup.

In theory, this should provide the same benefit, because what RT
scheduling is doing is pre-empting any user and kernel task that was
running when the interrupt was delivered to execute the completion
task immediately.

Setting the workqueue to use kernel threads of a higher scheduler
prioirty should do the same thing, without the need to use dedicated
per-cpu RT threads.

> If Sandeep has more time, I hope he could have more time to
> test since I don't work on Android anymore: In principle,
> I still think RT thread is needed somewhere for such usage
> since lowest latencies is needed.

All that is needed is for the kworker thread to be scheduled to run
immeidately after the interrupt that scheduled the work exits. This
does not require dedicated per-cpu kernel tasks or RT scheduling....

> Compared to the scheduling latency issues, interested users
> don't care "individual subsystems needing their own set of
> per-cpu kernel tasks just sitting around idle most of of
> the time". If end users care it more, they can just turn
> it off by Kconfig.

Distros enable all these subsystems all the time, so saying
"turn it off via kconfig" is not a viable mitigation
strategy. Proliferation of dedicated per-CPU worker task pools is a
known problem, and we really don't want to regress back to those
days when a typical system had thousands of dedicated per-cpu work
queues that largely did nothing most of the time.

-Dave.

-- 
Dave Chinner
dgc@kernel.org