Re: [Bug 202089] New: transparent hugepage not compatable with madvise(MADV_DONTNEED)

[Michal Hocko <mhocko@kernel.org>]

On Sun 30-12-18 01:48:43, Kirill A. Shutemov wrote:
> On Sat, Dec 29, 2018 at 12:53:16PM -0800, Andrew Morton wrote:
> > 
> > (switched to email.  Please respond via emailed reply-to-all, not via the
> > bugzilla web interface).
> > 
> > On Sat, 29 Dec 2018 09:00:22 +0000 bugzilla-daemon@bugzilla.kernel.org wrote:
> > 
> > > https://bugzilla.kernel.org/show_bug.cgi?id=202089
> > > 
> > >             Bug ID: 202089
> > >            Summary: transparent hugepage not compatable with
> > >                     madvise(MADV_DONTNEED)
> > >            Product: Memory Management
> > >            Version: 2.5
> > >     Kernel Version: 4.4.0-117
> > >           Hardware: x86-64
> > >                 OS: Linux
> > >               Tree: Mainline
> > >             Status: NEW
> > >           Severity: high
> > >           Priority: P1
> > >          Component: Other
> > >           Assignee: akpm@linux-foundation.org
> > >           Reporter: jianpanlanyue@163.com
> > >         Regression: No
> > > 
> > > environment:  
> > >   1.kernel 4.4.0 on x86_64
> > >   2.echo always > /sys/kernel/mm/transparent_hugepage/enable
> > >     echo always > /sys/kernel/mm/transparent_hugepage/defrag
> > >     echo 2000000 > /sys/kernel/mm/transparent_hugepage/khugepaged/pages_to_scan
> > > ( faster defrag pages to reproduce problem)
> > > 
> > > problem: 
> > >   1. use mmap() to allocate 4096 bytes for 1024*512 times (4096*1024*512=2G).
> > >   2. use madvise(MADV_DONTNEED) to free most of the above pages, but reserve a
> > > few pages(by if（i%33==0) continue;), then process's physical memory firstly
> > > come down, but after a few seconds, it rise back to 2G again, and can't come
> > > down forever.
> > >   3. if i delete this condition(if（i%33==0) continue;) or disable
> > > transparent_hugepage by setting 'enable' and 'defrag' to never, all go well and
> > > the physical memory can come down expectly.
> > > 
> > >   It seems like transparent_hugepage has problems with non-contiguous
> > > madvise(MADV_DONTEED).
> 
> It's expected behaviour.
> 
> MADV_DONTNEED doesn't guarantee that the range will not be repopulated
> (with or without direct action on application behalf). It's just a hint
> for the kernel.

I agree with Kirill here but I would be interested in the underlying
usecase that triggered this. The test case is clearly artificial but is
any userspace actually relying on MADV_DONTNEED reducing the rss
longterm?

> For sparse mappings, consider using MADV_NOHUGEPAGE.

Yes or use a high threshold for khugepaged for collapsing.
-- 
Michal Hocko
SUSE Labs