Re: [Lsf-pc] [LSF/MM/BPF TOPIC] Reclamation interactions with RCU

[Dave Chinner <david@fromorbit.com>]

On Mon, Mar 04, 2024 at 09:45:48AM +1100, NeilBrown wrote:
> I have in mind a more explicit statement of how much waiting is
> acceptable.
> 
> GFP_NOFAIL - wait indefinitely

Make this the default, and we don't need a flag for it at all.

> GFP_KILLABLE - wait indefinitely unless fatal signal is pending.
> GFP_RETRY - may retry but deadlock, though unlikely, is possible.  So
>             don't wait indefinitely.  May abort more quickly if fatal
>             signal is pending.

KILLABLE and RETRY are the same thing from the caller POV.
Effectively "GFP_MAY_FAIL", where it will try really hard, but if it
there is a risk of deadlock or a fatal signal pending, it will fail.

> GFP_NO_RETRY - only try things once.  This may sleep, but will give up
>             fairly quickly.  Either deadlock is a significant
>             possibility, or alternate strategy is fairly cheap.
> GFP_ATOMIC - don't sleep - same as current.

We're talking about wait semantics, so GFP_ATOMIC should be named
GFP_NO_WAIT and described as "same as NO_RETRY but will not sleep".

That gives us three modifying flags to the default behaviour of
sleeping until success: GFP_MAY_FAIL, GFP_NO_RETRY and GFP_NO_WAIT.

I will note there is one more case callers might really want to
avoid: direct reclaim. That sort of behaviour might be worth folding
into GFP_NO_WAIT, as there are cases where we want the allocation
attempt to fail without trying to reclaim memory because it's *much*
faster to simply use the fallback mechanism than it is to attempt
memory reclaim (e.g.  xlog_kvmalloc()).

> I don't see how "GFP_KERNEL" fits into that spectrum.

Agreed.

> The definition of
> "this will try really hard, but might fail and we can't really tell you
> what circumstances it might fail in" isn't fun to work with.

Yup, XFS was designed for NO_FAIL and MAY_FAIL behaviour, and in more
recent times we also use NO_RECLAIM to provide our own kvmalloc
semantics because the current kvmalloc API really only supports
"GFP_KERNEL" allocation.

> > Deprecating GFP_NOFS and GFP_NOIO would be wonderful - those should
> > really just be PF_MEMALLOC_NOFS and PF_MEMALLOC_NOIO, now that we're
> > pushing for memalloc_flags_(save|restore) more.

This is largely now subsystem maintenance work - the infrastructure
is there, and some subsystems have been converted over entirely to
use it. The remaining work either needs to be mandated or have
someone explicitly tasked with completing that work.

IOWs, the process in which we change APIs and then leave the long
tail of conversions to subsystem maintainers is just a mechanism for
creating technical debt that takes forever to clean up...

> > Getting rid of those would be a really nice cleanup beacuse then gfp
> > flags would mostly just be:
> >  - the type of memory to allocate (highmem, zeroed, etc.)
> >  - how hard to try (don't block at all, block some, block forever)

Yup, would be a very good improvement.

-Dave.
-- 
Dave Chinner
david@fromorbit.com