Re: [v5 PATCH 4/7] mm: thp: kill transparent_hugepage_active()

[Andrew Morton <akpm@linux-foundation.org>]

On Tue, 21 Jun 2022 11:58:34 -0700 "Zach O'Keefe" <zokeefe@google.com> wrote:

> > -bool transparent_hugepage_active(struct vm_area_struct *vma)
> > +bool hugepage_vma_check(struct vm_area_struct *vma,
> > +			unsigned long vm_flags,
> > +			bool smaps)
> >  {
> > -	/* The addr is used to check if the vma size fits */
> > -	unsigned long addr = (vma->vm_end & HPAGE_PMD_MASK) - HPAGE_PMD_SIZE;
> > +	if (!transhuge_vma_enabled(vma, vm_flags))
> > +		return false;
> > +
> 
> During testing my work on top this patch, I found a small bug here.
> 
> Namely, transhuge_vma_enabled() will check vma->vm_mm->flags (to see if
> MMF_DISABLE_THP is set); however, for vDSO vmas, vma->vm_mm is NULL.
> 
> Previously, transparent_hugepage_active() in smaps path would check
> transhuge_vma_suitable() before checking these flags, which would fail for vDSO
> vma since we'd take the !vma_is_anonymous() branch and find the vma (most
> likely) wasn't suitably aligned (by chance ?).
> 
> Anyways, I think we need to check vma->vm_mm.

Like this?

--- a/mm/huge_memory.c~mm-thp-kill-transparent_hugepage_active-fix
+++ a/mm/huge_memory.c
@@ -73,6 +73,9 @@ bool hugepage_vma_check(struct vm_area_s
 			unsigned long vm_flags,
 			bool smaps)
 {
+	if (!vma->vm_mm)
+		return false;
+
 	if (!transhuge_vma_enabled(vma, vm_flags))
 		return false;
 
_