Re: [PATCH v6 4/5] mm/migrate: skip migrating folios under writeback with AS_WRITEBACK_INDETERMINATE mappings

[David Hildenbrand <david@redhat.com>]

On 20.12.24 19:01, Shakeel Butt wrote:
> On Fri, Dec 20, 2024 at 03:49:39PM +0100, David Hildenbrand wrote:
>>>> I'm wondering if there would be a way to just "cancel" the writeback and
>>>> mark the folio dirty again. That way it could be migrated, but not
>>>> reclaimed. At least we could avoid the whole AS_WRITEBACK_INDETERMINATE
>>>> thing.
>>>>
>>>
>>> That is what I basically meant with short timeouts. Obviously it is not
>>> that simple to cancel the request and to retry - it would add in quite
>>> some complexity, if all the issues that arise can be solved at all.
>>
>> At least it would keep that out of core-mm.
>>
>> AS_WRITEBACK_INDETERMINATE really has weird smell to it ... we should try to
>> improve such scenarios, not acknowledge and integrate them, then work around
>> using timeouts that must be manually configured, and ca likely no be default
>> enabled because it could hurt reasonable use cases :(
> 
> Just to be clear AS_WRITEBACK_INDETERMINATE is being used in two core-mm
> parts. First is reclaim and second is compaction/migration. For reclaim,
> it is a must have as explained by Jingbo in [1] i.e. due to potential
> self deadlock by fuse server. If I understand you correctly, the main
> concern you have is its usage in the second case.

Yes, so I can see fuse

(1) Breaking memory reclaim (memory cannot get freed up)

(2) Breaking page migration (memory cannot be migrated)

Due to (1) we might experience bigger memory pressure in the system I 
guess. A handful of these pages don't really hurt, I have no idea how 
bad having many of these pages can be. But yes, inherently we cannot 
throw away the data as long as it is dirty without causing harm. (maybe 
we could move it to some other cache, like swap/zswap; but that smells 
like a big and complicated project)

Due to (2) we turn pages that are supposed to be movable possibly for a 
long time unmovable. Even a *single* such page will mean that CMA 
allocations / memory unplug can start failing.

We have similar situations with page pinning. With things like O_DIRECT, 
our assumption/experience so far is that it will only take a couple of 
seconds max, and retry loops are sufficient to handle it. That's why 
only long-term pinning ("indeterminate", e.g., vfio) migrate these pages 
out of ZONE_MOVABLE/MIGRATE_CMA areas in order to long-term pin them.


The biggest concern I have is that timeouts, while likely reasonable it 
many scenarios, might not be desirable even for some sane workloads, and 
the default in all system will be "no timeout", letting the clueless 
admin of each and every system out there that might support fuse to make 
a decision.

I might have misunderstood something, in which case I am very sorry, but 
we also don't want CMA allocations to start failing simply because a 
network connection is down for a couple of minutes such that a fuse 
daemon cannot make progress.


> 
> The reason for adding AS_WRITEBACK_INDETERMINATE in the second case was
> to avoid untrusted fuse server causing pain to unrelated jobs on the
> machine (fuse folks please correct me if I am wrong here). Now we are
> discussing how to better handle that scenario.
> 
> I just wanted to point out that irrespective of that discussion, the
> reclaim will have handle the potential recursive deadlock and thus will
> be using AS_WRITEBACK_INDETERMINATE or something similar.

Yes, I see no way to throw away dirty data without causing harm.

Migration was kept working for now, although in a hacky fashion I admit. 
I do enjoy that "writeback" on the folio actually matches the reality now.

I guess an alternative to "aborting writeback" would be to make fuse 
allow for migrating folios that are under writeback. I would assume that 
with fuse we have very good control over who is currently 
reading/writing that folio, and we could swap it out? Again, just an 
idea ...


-- 
Cheers,

David / dhildenb