Re: [PATCH] mm: mmap: map MAP_STACK to VM_NOHUGEPAGE only if THP is enabled

[Lorenzo Stoakes <lorenzo.stoakes@oracle.com>]

On Fri, May 02, 2025 at 02:03:48PM +0100, Matthew Wilcox wrote:
> On Fri, May 02, 2025 at 11:31:41AM +0200, Ignacio Moreno Gonzalez via B4 Relay wrote:
> > From: Ignacio Moreno Gonzalez <Ignacio.MorenoGonzalez@kuka.com>
> >
> > commit c4608d1bf7c6 ("mm: mmap: map MAP_STACK to VM_NOHUGEPAGE") maps
> > the mmap option MAP_STACK to VM_NOHUGEPAGE. This is also done if
> > CONFIG_TRANSPARENT_HUGETABLES is not defined. But in that case, the
> > VM_NOHUGEPAGE does not make sense. For instance, when calling madvise()
> > with MADV_NOHUGEPAGE, an error is always returned.
>
> Isn't that the real problem though?

Hmm, but wouldn't we want users who are trying to set MADV_[NO]HUGEPAGE to
be made aware that it isn't going to do anything?

And wouldn't changing this be a possibly 'breaking userspace' thing if
somebody somewhere relies on this?

Also this will make this inconsistent with e.g. MADV_COLLAPSE also?

I guess you could argue MADV_NOHUGEPAGE with !THP is just a no-op... But
definitely not MADV_HUGEPAGE.

Another angle to this as well is - we are causing these VMAs to have a
pointless VMA flag set that presumably can't be set any other way.

So I do think the proposed solution is right, and is the least impactful
one.

>
> +++ b/include/linux/huge_mm.h
> @@ -596,6 +596,8 @@ static inline bool unmap_huge_pmd_locked(struct vm_area_struct *vma,
>  static inline int hugepage_madvise(struct vm_area_struct *vma,
>                                    unsigned long *vm_flags, int advice)
>  {
> +       if (advice == MADV_NOHUGEPAGE)
> +               return 0;
>         return -EINVAL;
>  }
>
>