Re: Possible regression with file madvise(MADV_COLLAPSE)

[Yang Shi <shy828301@gmail.com>]

On Sat, Oct 12, 2024 at 8:38 AM Avi Kivity <avi@scylladb.com> wrote:
>
> On Fri, 2024-10-11 at 15:29 -0700, Yang Shi wrote:
> > On Wed, Oct 9, 2024 at 9:04 AM Avi Kivity <avi@scylladb.com> wrote:
> > >
> > > On Linux 6.10.10 with CONFIG_READ_ONLY_THP_FOR_FS=y,
> > > madvise(MADV_COLLAPSE) on  program text fails with EINVAL.
> > >
> > > To reproduce, compile the reproducer with
> > >
> > > clang -g -o text-hugepage  text-hugepage.c \
> > >         -fuse-ld=lld \
> > >         -Wl,-zcommon-page-size=2097152 -Wl,-zmax-page-size=2097152
> > > \
> > >         -Wl,-z,separate-loadable-segments
> > >
> > > and run:
> >
> > Didn't clang make the page cache dirty?
> >
> > Having sync between clang and the execution made the problem go away
> > for me.
> >
>
> I see it even with sync (and msync just before the madvise calls).

Did you stop khugepaged? It may race with MADV_COLLAPSE. If it failed
due to race with khugepaged, you should see -EAGAIN instead of
-EINVAL.

I did the below commands in a loop for 1000 times, it never failed (I
modified the test program a little bit to print out failure if
MADV_COLLAPSE returns failure). I had khugepaged stopped and ran the
test on v6.12-rc1 kernel on my AmpereOne machine.

rm text-hugepage
clang -g -o text-hugepage  text-hugepage.c -fuse-ld=lld
-Wl,-zcommon-page-size=2097152 -Wl,-zmax-page-size=2097152
-Wl,-z,separate-loadable-segments
sync
./text-hugepage

>
>
> Tracing shows this (last lines before syscall exit):
>
> |          hpage_collapse_scan_file() {
> |            __rcu_read_lock();
> |            __rcu_read_unlock();
> |          }

It meant collapse_file() was not called at all.
hpage_collapse_scan_file() failed. A couple of reasons may fail it,
for example, refcount is not expected, not on lru, etc. You can trace
huge_memory:mm_khugepaged_scan_file to get more information about the
failure.

>
>
> so, it's not clear what the root cause is.
>