Re: [PATCH 0/2] fix MADV_COLLAPSE issue if THP settings are disabled

[Lorenzo Stoakes <lorenzo.stoakes@oracle.com>]

On Fri, May 30, 2025 at 10:07:11AM +0100, Ryan Roberts wrote:
> On 30/05/2025 09:52, David Hildenbrand wrote:
> > On 30.05.25 10:47, Lorenzo Stoakes wrote:
> >> On Fri, May 30, 2025 at 10:44:36AM +0200, David Hildenbrand wrote:
> >>> On 30.05.25 10:04, Ryan Roberts wrote:
> >>>> On 29/05/2025 09:23, Baolin Wang wrote:
> >>>>> As we discussed in the previous thread [1], the MADV_COLLAPSE will ignore
> >>>>> the system-wide anon/shmem THP sysfs settings, which means that even though
> >>>>> we have disabled the anon/shmem THP configuration, MADV_COLLAPSE will still
> >>>>> attempt to collapse into a anon/shmem THP. This violates the rule we have
> >>>>> agreed upon: never means never. This patch set will address this issue.
> >>>>
> >>>> This is a drive-by comment from me without having the previous context, but...
> >>>>
> >>>> Surely MADV_COLLAPSE *should* ignore the THP sysfs settings? It's a deliberate
> >>>> user-initiated, synchonous request to use huge pages for a range of memory.
> >>>> There is nothing *transparent* about it, it just happens to be implemented
> >>>> using
> >>>> the same logic that THP uses.
> >>>>
> >>>> I always thought this was a deliberate design decision.
> >>>
> >>> If the admin said "never", then why should a user be able to overwrite that?
> >>>
> >>> The design decision I recall is that if VM_NOHUGEPAGE is set, we'll ignore
> >>> that. Because that was set by the app itself (MADV_NOHUEPAGE).
> >>>
> >>
> >> I'm with David on this one.
> >>
> >> I think it's principal of least surprise - to me 'never' is pretty
> >> emphatic, and keep in mind the other choices are 'always' and...  'madvise'
> >> :) which explicitly is 'hey only do this if madvise tells you to'.
>
> I think it's a bit reductive to suggest that enabled=madvise means all madvise
> calls though. I don't think anyone would suggest MADV_DONTNEED should be ignored
> if enabled=never. MADV_COLLAPSE just happens to be implemented on top of the THP
> logic. But it's a different feature in my view.

No I absolutely take your point, and indeed this is very reductive, but I think
that's a product of this interface being... sub-optimal.

if you dig into the docs for instance it's explicit about that referring to
MADV_[NO]HUGEPAGE.

But, as a user/sys-admin, I'd definitely find that surprising.

I think the intent of 'never' people is 'THP bad I don't want it' for whatever
reason that might be the case.

>
> >>
> >> I'd be rather surprised if I hadn't set madvise and a user uses madvise to
> >> in some fashion override the never.
> >>
> >> I mean I think we all agree this interface is to use a technical term -
> >> crap - and we need something a lot more fine-grained and smart,
>
> Yes agreed there!
>
> >> but I think
> >> given the situation we're in we should make it at least as least surprising
> >> as possible.
>
> >
> > Yes. If you configure "never" you are supposed to suffer, consistently.
> >
>
> OK fair enough. Just giving my 2 cents.
>
>

Your input is very welcome! We have made a mess here so it's good to talk it
through.