Re: sharing rescuer threads when WQ_MEM_RECLAIM needed? [was: Re: dm verity: don't use WQ_MEM_RECLAIM]

[Mikulas Patocka <mpatocka@redhat.com>]

On Thu, 5 Sep 2024, Tejun Heo wrote:

> Hello,
> 
> On Thu, Sep 05, 2024 at 07:35:41PM -0400, Mike Snitzer wrote:
> ...
> > > I wonder if there's any way to safely share the rescuer threads.
> > 
> > Oh, I like that idea, yes please! (would be surprised if it exists,
> > but I love being surprised!).  Like Mikulas pointed out, we have had
> > to deal with fundamental deadlocks due to resource sharing in DM.
> > Hence the need for guaranteed forward progress that only
> > WQ_MEM_RECLAIM can provide.

I remember that one of the first thing that I did when I started at Red 
Hat was to remove shared resources from device mapper :) There were shared 
mempools and shared kernel threads.

You can see this piece of code in mm/mempool.c that was a workaround for 
shared mempool bugs:
        /*
         * FIXME: this should be io_schedule().  The timeout is there as a
         * workaround for some DM problems in 2.6.18.
         */
        io_schedule_timeout(5*HZ);

> The most straightforward way to do this would be simply sharing the
> workqueue across the entities that wanna be in the same forward progress
> guarantee domain. It shouldn't be that difficult to make workqueues share a
> rescuer either but may be a bit of an overkill.
> 
> Taking a step back tho, how would you determine which ones can share a
> rescuer? Things which stack on top of each other can't share the rescuer cuz
> higher layer occupying the rescuer and stall lower layers and thus deadlock.
> The rescuers can be shared across independent stacks of dm devices but that
> sounds like that will probably involve some graph walking. Also, is this a
> real problem?
> 
> Thanks.

It would be nice if we could know dependencies of every Linux driver. But 
we are not quite there. We know the dependencies inside device mapper, but 
when you use some non-dm device (like md, loop), we don't have a dependecy 
graph for that.

Mikulas