From: "David Hildenbrand (Red Hat)" <david@kernel.org>
To: Joanne Koong <joannelkoong@gmail.com>
Cc: akpm@linux-foundation.org, linux-mm@kvack.org,
shakeel.butt@linux.dev, athul.krishna.kr@protonmail.com,
miklos@szeredi.hu, stable@vger.kernel.org
Subject: Re: [PATCH v1 2/2] fs/writeback: skip inodes with potential writeback hang in wait_sb_inodes()
Date: Mon, 24 Nov 2025 14:58:03 +0100 [thread overview]
Message-ID: <f8da9ee0-f136-4366-b63a-1812fda11304@kernel.org> (raw)
In-Reply-To: <CAJnrk1Zsdw9Uvb44ynkfWLBvs2vw7he-opVu6mzJqokphMiLSQ@mail.gmail.com>
On 11/20/25 22:20, Joanne Koong wrote:
> On Thu, Nov 20, 2025 at 12:23 PM David Hildenbrand (Red Hat)
> <david@kernel.org> wrote:
>>
>> On 11/20/25 19:42, Joanne Koong wrote:
>>> During superblock writeback waiting, skip inodes where writeback may
>>> take an indefinite amount of time or hang, as denoted by the
>>> AS_WRITEBACK_MAY_HANG mapping flag.
>>>
>>> Currently, fuse is the only filesystem with this flag set. For a
>>> properly functioning fuse server, writeback requests are completed and
>>> there is no issue. However, if there is a bug in the fuse server and it
>>> hangs on writeback, then without this change, wait_sb_inodes() will wait
>>> forever.
>>>
>>> Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
>>> Fixes: 0c58a97f919c ("fuse: remove tmp folio for writebacks and internal rb tree")
>>> Reported-by: Athul Krishna <athul.krishna.kr@protonmail.com>
>>> ---
>>> fs/fs-writeback.c | 3 +++
>>> 1 file changed, 3 insertions(+)
>>>
>>> diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c
>>> index 2b35e80037fe..eb246e9fbf3d 100644
>>> --- a/fs/fs-writeback.c
>>> +++ b/fs/fs-writeback.c
>>> @@ -2733,6 +2733,9 @@ static void wait_sb_inodes(struct super_block *sb)
>>> if (!mapping_tagged(mapping, PAGECACHE_TAG_WRITEBACK))
>>> continue;
>>>
>>> + if (mapping_writeback_may_hang(mapping))
>>> + continue;
>>
>> I think I raised it in the past, but simply because it could happen, why
>> would we unconditionally want to do that for all fuse mounts? That just
>> seems wrong :(
>
> I think it's considered a userspace regression if we don't revert the
> program behavior back to its previous version, even if it is from the
> program being incorrectly written, as per the conversation in [1].
>
> [1] https://lore.kernel.org/regressions/CAJnrk1Yh4GtF-wxWo_2ffbr90R44u0WDmMAEn9vr9pFgU0Nc6w@mail.gmail.com/T/#m73cf4b4828d51553caad3209a5ac92bca78e15d2
>
>>
>> To phrase it in a different way, if any writeback could theoretically
>> hang, why are we even waiting on writeback in the first place?
>>
>
> I think it's because on other filesystems, something has to go
> seriously wrong for writeback to hang, but on fuse a server can easily
> make writeback hang and as it turns out, there are already existing
> userspace programs that do this accidentally.
Sorry, I only found the time to reply now. I wanted to reply in more
detail why what you propose here does not make sense to me.
I understand that it might make one of the weird fuse scenarios (buggy
fuse server) work again, but it sounds like we are adding more hacks on
top of broken semantics. If we want to tackle the writeback problem, we
should find a proper way to deal with that for good.
(1) AS_WRITEBACK_MAY_HANG semantics
As discussed in the past, writeeback of pretty much any filesystem might
hang forever on I/O errors.
On network filesystems apparently as well fairly easily.
It's completely unclear when to set AS_WRITEBACK_MAY_HANG.
So as writeback on any filesystem may hang, AS_WRITEBACK_MAY_HANG would
theoretically have to be set on any mapping out there.
The semantics don't make sense to me, unfortuantely.
(2) AS_WRITEBACK_MAY_HANG usage
It's unclear in which scenarios we would not want to wait for writeback,
and what the effects of that are.
For example, wait_sb_inodes() documents "Data integrity sync. Must wait
for all pages under writeback, because there may have been pages dirtied
before our sync call ...".
It's completely unclear why it might be okay to skip that simply because
a mapping indicated that waiting for writeback is maybe more sketchy
than on other filesystems.
But what concerns me more is what we do about other
folio_wait_writeback() callers. Throwing in AS_WRITEBACK_MAY_HANG
wherever somebody reproduced a hang is not a good approach.
We need something more robust where we can just not break the kernel in
weird ways because user space is buggy or malicious.
(3) Other operations
If my memory serves me right, there are similar issues on readahead. It
wouldn't surprise me if there are yet other operations where fuse Et al
can trick the kernel into hanging forever.
So I'm wondering if there is more to this than just "writeback may hang".
Obviously, getting the kernel to hang, controlled by user space that
easily, is extremely unpleasant and probably the thing that I really
dislike about fuse. Amir mentioned that maybe the iomap changes from
Darrick might improve the situation in the long run, I would hope it
would allow for de-nastifying fuse in that sense, at least in some
scenarios.
I cannot really say what would be better here (maybe aborting writeback
after a short timeout), but AS_WRITEBACK_MAY_HANG to then just skip
selected waits for writeback is certainly something that does not make
sense to me.
Regarding the patch here, is there a good reason why fuse does not have
to wait for the "Data integrity sync. Must wait for all pages under
writeback ..."?
IOW, is the documented "must" not a "must" for fuse? In that case,
having a flag that states something like that that
"AS_NO_WRITEBACK_WAIT_ON_DATA_SYNC" would probable be what we would want
to add to avoid waiting for writeback with clear semantics why it is ok
in that specific scenario.
Hope that helps, and happy to be convinced why AS_WRITEBACK_MAY_HANG is
the right thing to do in this way proposed here.
--
Cheers
David
next prev parent reply other threads:[~2025-11-24 13:58 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-11-20 18:42 [PATCH v1 0/2] mm: skip wait in wait_sb_inodes() for hangable-writeback mappings Joanne Koong
2025-11-20 18:42 ` [PATCH v1 1/2] mm: rename AS_WRITEBACK_MAY_DEADLOCK_ON_RECLAIM to AS_WRITEBACK_MAY_HANG Joanne Koong
2025-11-20 20:08 ` David Hildenbrand (Red Hat)
2025-11-20 21:28 ` Joanne Koong
2025-11-20 18:42 ` [PATCH v1 2/2] fs/writeback: skip inodes with potential writeback hang in wait_sb_inodes() Joanne Koong
2025-11-20 20:23 ` David Hildenbrand (Red Hat)
2025-11-20 21:20 ` Joanne Koong
2025-11-24 13:58 ` David Hildenbrand (Red Hat) [this message]
2025-11-25 1:10 ` Joanne Koong
2025-11-26 10:19 ` Miklos Szeredi
2025-11-26 10:41 ` David Hildenbrand (Red Hat)
2025-11-26 10:55 ` David Hildenbrand (Red Hat)
2025-11-26 17:58 ` Joanne Koong
2025-12-03 9:28 ` Miklos Szeredi
2025-12-04 18:06 ` Joanne Koong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=f8da9ee0-f136-4366-b63a-1812fda11304@kernel.org \
--to=david@kernel.org \
--cc=akpm@linux-foundation.org \
--cc=athul.krishna.kr@protonmail.com \
--cc=joannelkoong@gmail.com \
--cc=linux-mm@kvack.org \
--cc=miklos@szeredi.hu \
--cc=shakeel.butt@linux.dev \
--cc=stable@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox