[PATCH] mm: remove VM_EXEC requirement for THP eligibility

[PATCH] mm: remove VM_EXEC requirement for THP eligibility

[Fangrui Song]

Commit e6be37b2e7bd ("mm/huge_memory.c: add missing read-only THP
checking in transparent_hugepage_enabled()") introduced the VM_EXEC
requirement, which is not strictly needed.

lld's default --rosegment option and GNU ld's -z separate-code option
(default on Linux/x86 since binutils 2.31) create a read-only PT_LOAD
segment without the PF_X flag, which should be eligible for THP.

Certain architectures support medium and large code models, where
.lrodata may be placed in a separate read-only PT_LOAD segment, which
should be eligible for THP as well.

Signed-off-by: Fangrui Song <maskray@google.com>
---
 include/linux/huge_mm.h | 1 -
 1 file changed, 1 deletion(-)

diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
index fa0350b0812a..4c9e67e9000f 100644
--- a/include/linux/huge_mm.h
+++ b/include/linux/huge_mm.h
@@ -126,7 +126,6 @@ static inline bool file_thp_enabled(struct vm_area_struct *vma)
 	inode = vma->vm_file->f_inode;
 
 	return (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS)) &&
-	       (vma->vm_flags & VM_EXEC) &&
 	       !inode_is_open_for_write(inode) && S_ISREG(inode->i_mode);
 }
 
-- 
2.43.0.472.g3155946c3a-goog

Re: [PATCH] mm: remove VM_EXEC requirement for THP eligibility

[Yang Shi]

On Tue, Dec 19, 2023 at 9:41 PM Fangrui Song <maskray@google.com> wrote:
>
> Commit e6be37b2e7bd ("mm/huge_memory.c: add missing read-only THP
> checking in transparent_hugepage_enabled()") introduced the VM_EXEC
> requirement, which is not strictly needed.
>
> lld's default --rosegment option and GNU ld's -z separate-code option
> (default on Linux/x86 since binutils 2.31) create a read-only PT_LOAD
> segment without the PF_X flag, which should be eligible for THP.
>
> Certain architectures support medium and large code models, where
> .lrodata may be placed in a separate read-only PT_LOAD segment, which
> should be eligible for THP as well.

Yeah, it doesn't have to be VM_EXEC. The original implementation was
restricted to VM_EXEC to minimize the blast radius and the targe use
case is for large text segments. Out of curiosity, did you see any
noticeable improvement with this change?

>
> Signed-off-by: Fangrui Song <maskray@google.com>
> ---
>  include/linux/huge_mm.h | 1 -
>  1 file changed, 1 deletion(-)
>
> diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
> index fa0350b0812a..4c9e67e9000f 100644
> --- a/include/linux/huge_mm.h
> +++ b/include/linux/huge_mm.h
> @@ -126,7 +126,6 @@ static inline bool file_thp_enabled(struct vm_area_struct *vma)
>         inode = vma->vm_file->f_inode;
>
>         return (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS)) &&
> -              (vma->vm_flags & VM_EXEC) &&
>                !inode_is_open_for_write(inode) && S_ISREG(inode->i_mode);
>  }
>
> --
> 2.43.0.472.g3155946c3a-goog
>

Re: [PATCH] mm: remove VM_EXEC requirement for THP eligibility

[Fangrui Song]

On Wed, Dec 20, 2023 at 3:42 PM Yang Shi <shy828301@gmail.com> wrote:
>
> On Tue, Dec 19, 2023 at 9:41 PM Fangrui Song <maskray@google.com> wrote:
> >
> > Commit e6be37b2e7bd ("mm/huge_memory.c: add missing read-only THP
> > checking in transparent_hugepage_enabled()") introduced the VM_EXEC
> > requirement, which is not strictly needed.
> >
> > lld's default --rosegment option and GNU ld's -z separate-code option
> > (default on Linux/x86 since binutils 2.31) create a read-only PT_LOAD
> > segment without the PF_X flag, which should be eligible for THP.
> >
> > Certain architectures support medium and large code models, where
> > .lrodata may be placed in a separate read-only PT_LOAD segment, which
> > should be eligible for THP as well.
>
> Yeah, it doesn't have to be VM_EXEC. The original implementation was
> restricted to VM_EXEC to minimize the blast radius and the targe use
> case is for large text segments. Out of curiosity, did you see any
> noticeable improvement with this change?

Hi Yang,

Thanks for the comment. Frankly, I am not familiar with huge pages...
I noticed this VM_EXEC condition when I was writing this
hugepage-related section in
https://maskray.me/blog/2023-12-17-exploring-the-section-layout-in-linker-output#transparent-huge-pages-for-mapped-files
(Thanks to Alexander Monakov's comment about
CONFIG_READ_ONLY_THP_FOR_FS in
https://mazzo.li/posts/check-huge-page.html).

As dTLB for read-only data is also an important optimization of
file-backed THP, it seems straightforward that we should drop the
VM_EXEC condition :)

On my Arch linux machine, the r--p page gets split if I invoke
madvise(__ehdr_start, HPAGE_SIZE, MADV_HUGEPAGE) I haven't figured out
why it behaves so in the presence of the VM_EXEC check.

% g++ test.cc -o ~/tmp/test -O2 -fuse-ld=lld
-Wl,-z,max-page-size=2097152 && sudo ~/tmp/test
__ehdr_start: 0x55f3b1c00000
55f3b1c00000-55f3b1e00000 r--p 00000000 103:03 555277119
  /home/ray/tmp/test
55f3b1e00000-55f3b1e01000 r--p 00200000 103:03 555277119
  /home/ray/tmp/test
55f3b2000000-55f3b2002000 r-xp 00200000 103:03 555277119
  /home/ray/tmp/test
55f3b2201000-55f3b2202000 r--p 00201000 103:03 555277119
  /home/ray/tmp/test
55f3b2401000-55f3b2402000 rw-p 00201000 103:03 555277119
  /home/ray/tmp/test
55f3b3a9a000-55f3b3abb000 rw-p 00000000 00:00 0                          [heap]


It'd be greatly appreciated if someone familiar with
CONFIG_READ_ONLY_THP_FOR_FS could provide some notes on how to use
this feature:)

> >
> > Signed-off-by: Fangrui Song <maskray@google.com>
> > ---
> >  include/linux/huge_mm.h | 1 -
> >  1 file changed, 1 deletion(-)
> >
> > diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
> > index fa0350b0812a..4c9e67e9000f 100644
> > --- a/include/linux/huge_mm.h
> > +++ b/include/linux/huge_mm.h
> > @@ -126,7 +126,6 @@ static inline bool file_thp_enabled(struct vm_area_struct *vma)
> >         inode = vma->vm_file->f_inode;
> >
> >         return (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS)) &&
> > -              (vma->vm_flags & VM_EXEC) &&
> >                !inode_is_open_for_write(inode) && S_ISREG(inode->i_mode);
> >  }
> >
> > --
> > 2.43.0.472.g3155946c3a-goog
> >



-- 
宋方睿

Re: [PATCH] mm: remove VM_EXEC requirement for THP eligibility

[Yang Shi]

On Wed, Dec 20, 2023 at 8:53 PM Fangrui Song <maskray@google.com> wrote:
>
> On Wed, Dec 20, 2023 at 3:42 PM Yang Shi <shy828301@gmail.com> wrote:
> >
> > On Tue, Dec 19, 2023 at 9:41 PM Fangrui Song <maskray@google.com> wrote:
> > >
> > > Commit e6be37b2e7bd ("mm/huge_memory.c: add missing read-only THP
> > > checking in transparent_hugepage_enabled()") introduced the VM_EXEC
> > > requirement, which is not strictly needed.
> > >
> > > lld's default --rosegment option and GNU ld's -z separate-code option
> > > (default on Linux/x86 since binutils 2.31) create a read-only PT_LOAD
> > > segment without the PF_X flag, which should be eligible for THP.
> > >
> > > Certain architectures support medium and large code models, where
> > > .lrodata may be placed in a separate read-only PT_LOAD segment, which
> > > should be eligible for THP as well.
> >
> > Yeah, it doesn't have to be VM_EXEC. The original implementation was
> > restricted to VM_EXEC to minimize the blast radius and the targe use
> > case is for large text segments. Out of curiosity, did you see any
> > noticeable improvement with this change?
>
> Hi Yang,
>
> Thanks for the comment. Frankly, I am not familiar with huge pages...
> I noticed this VM_EXEC condition when I was writing this
> hugepage-related section in
> https://maskray.me/blog/2023-12-17-exploring-the-section-layout-in-linker-output#transparent-huge-pages-for-mapped-files
> (Thanks to Alexander Monakov's comment about
> CONFIG_READ_ONLY_THP_FOR_FS in
> https://mazzo.li/posts/check-huge-page.html).

Thanks for sharing the article, learnt something about linker and loader.

>
> As dTLB for read-only data is also an important optimization of
> file-backed THP, it seems straightforward that we should drop the
> VM_EXEC condition :)

Yeah, as long as the use case is valid, it is definitely fine to lift
the restriction.

>
> On my Arch linux machine, the r--p page gets split if I invoke
> madvise(__ehdr_start, HPAGE_SIZE, MADV_HUGEPAGE) I haven't figured out
> why it behaves so in the presence of the VM_EXEC check.

What do you mean about "split"? THP got split into small pages? It
depends on the address of __ehdr_start. If it is in the middle of a
VMA, the VMA is going to be split due to the different huge page
attributes.

>
> % g++ test.cc -o ~/tmp/test -O2 -fuse-ld=lld
> -Wl,-z,max-page-size=2097152 && sudo ~/tmp/test
> __ehdr_start: 0x55f3b1c00000
> 55f3b1c00000-55f3b1e00000 r--p 00000000 103:03 555277119
>   /home/ray/tmp/test
> 55f3b1e00000-55f3b1e01000 r--p 00200000 103:03 555277119
>   /home/ray/tmp/test
> 55f3b2000000-55f3b2002000 r-xp 00200000 103:03 555277119
>   /home/ray/tmp/test
> 55f3b2201000-55f3b2202000 r--p 00201000 103:03 555277119
>   /home/ray/tmp/test
> 55f3b2401000-55f3b2402000 rw-p 00201000 103:03 555277119
>   /home/ray/tmp/test
> 55f3b3a9a000-55f3b3abb000 rw-p 00000000 00:00 0                          [heap]
>
>
> It'd be greatly appreciated if someone familiar with
> CONFIG_READ_ONLY_THP_FOR_FS could provide some notes on how to use
> this feature:)

I think your blog covered all the points. If you don't mind, you could
add some notes in Documentation/admin-guide/mm/transhuge.rst.

>
> > >
> > > Signed-off-by: Fangrui Song <maskray@google.com>
> > > ---
> > >  include/linux/huge_mm.h | 1 -
> > >  1 file changed, 1 deletion(-)
> > >
> > > diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
> > > index fa0350b0812a..4c9e67e9000f 100644
> > > --- a/include/linux/huge_mm.h
> > > +++ b/include/linux/huge_mm.h
> > > @@ -126,7 +126,6 @@ static inline bool file_thp_enabled(struct vm_area_struct *vma)
> > >         inode = vma->vm_file->f_inode;
> > >
> > >         return (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS)) &&
> > > -              (vma->vm_flags & VM_EXEC) &&
> > >                !inode_is_open_for_write(inode) && S_ISREG(inode->i_mode);
> > >  }
> > >
> > > --
> > > 2.43.0.472.g3155946c3a-goog
> > >
>
>
>
> --
> 宋方睿

Re: [PATCH] mm: remove VM_EXEC requirement for THP eligibility

[Yang Shi]

On Thu, Dec 21, 2023 at 11:31 AM Yang Shi <shy828301@gmail.com> wrote:
>
> On Wed, Dec 20, 2023 at 8:53 PM Fangrui Song <maskray@google.com> wrote:
> >
> > On Wed, Dec 20, 2023 at 3:42 PM Yang Shi <shy828301@gmail.com> wrote:
> > >
> > > On Tue, Dec 19, 2023 at 9:41 PM Fangrui Song <maskray@google.com> wrote:
> > > >
> > > > Commit e6be37b2e7bd ("mm/huge_memory.c: add missing read-only THP
> > > > checking in transparent_hugepage_enabled()") introduced the VM_EXEC
> > > > requirement, which is not strictly needed.
> > > >
> > > > lld's default --rosegment option and GNU ld's -z separate-code option
> > > > (default on Linux/x86 since binutils 2.31) create a read-only PT_LOAD
> > > > segment without the PF_X flag, which should be eligible for THP.
> > > >
> > > > Certain architectures support medium and large code models, where
> > > > .lrodata may be placed in a separate read-only PT_LOAD segment, which
> > > > should be eligible for THP as well.
> > >
> > > Yeah, it doesn't have to be VM_EXEC. The original implementation was
> > > restricted to VM_EXEC to minimize the blast radius and the targe use
> > > case is for large text segments. Out of curiosity, did you see any
> > > noticeable improvement with this change?
> >
> > Hi Yang,
> >
> > Thanks for the comment. Frankly, I am not familiar with huge pages...
> > I noticed this VM_EXEC condition when I was writing this
> > hugepage-related section in
> > https://maskray.me/blog/2023-12-17-exploring-the-section-layout-in-linker-output#transparent-huge-pages-for-mapped-files
> > (Thanks to Alexander Monakov's comment about
> > CONFIG_READ_ONLY_THP_FOR_FS in
> > https://mazzo.li/posts/check-huge-page.html).
>
> Thanks for sharing the article, learnt something about linker and loader.

BTW, kernel should try to map the segments (the size has to be >= 2M)
to 2M aligned address even though the loading address is not 2M
aligned for ext4/xfs/btrfs since v5.18. See commit 1854bc6e2420
("mm/readahead: Align file mappings for non-DAX"). Did you see this
behavior?

>
> >
> > As dTLB for read-only data is also an important optimization of
> > file-backed THP, it seems straightforward that we should drop the
> > VM_EXEC condition :)
>
> Yeah, as long as the use case is valid, it is definitely fine to lift
> the restriction.
>
> >
> > On my Arch linux machine, the r--p page gets split if I invoke
> > madvise(__ehdr_start, HPAGE_SIZE, MADV_HUGEPAGE) I haven't figured out
> > why it behaves so in the presence of the VM_EXEC check.
>
> What do you mean about "split"? THP got split into small pages? It
> depends on the address of __ehdr_start. If it is in the middle of a
> VMA, the VMA is going to be split due to the different huge page
> attributes.
>
> >
> > % g++ test.cc -o ~/tmp/test -O2 -fuse-ld=lld
> > -Wl,-z,max-page-size=2097152 && sudo ~/tmp/test
> > __ehdr_start: 0x55f3b1c00000
> > 55f3b1c00000-55f3b1e00000 r--p 00000000 103:03 555277119
> >   /home/ray/tmp/test
> > 55f3b1e00000-55f3b1e01000 r--p 00200000 103:03 555277119
> >   /home/ray/tmp/test
> > 55f3b2000000-55f3b2002000 r-xp 00200000 103:03 555277119
> >   /home/ray/tmp/test
> > 55f3b2201000-55f3b2202000 r--p 00201000 103:03 555277119
> >   /home/ray/tmp/test
> > 55f3b2401000-55f3b2402000 rw-p 00201000 103:03 555277119
> >   /home/ray/tmp/test
> > 55f3b3a9a000-55f3b3abb000 rw-p 00000000 00:00 0                          [heap]
> >
> >
> > It'd be greatly appreciated if someone familiar with
> > CONFIG_READ_ONLY_THP_FOR_FS could provide some notes on how to use
> > this feature:)
>
> I think your blog covered all the points. If you don't mind, you could
> add some notes in Documentation/admin-guide/mm/transhuge.rst.
>
> >
> > > >
> > > > Signed-off-by: Fangrui Song <maskray@google.com>
> > > > ---
> > > >  include/linux/huge_mm.h | 1 -
> > > >  1 file changed, 1 deletion(-)
> > > >
> > > > diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
> > > > index fa0350b0812a..4c9e67e9000f 100644
> > > > --- a/include/linux/huge_mm.h
> > > > +++ b/include/linux/huge_mm.h
> > > > @@ -126,7 +126,6 @@ static inline bool file_thp_enabled(struct vm_area_struct *vma)
> > > >         inode = vma->vm_file->f_inode;
> > > >
> > > >         return (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS)) &&
> > > > -              (vma->vm_flags & VM_EXEC) &&
> > > >                !inode_is_open_for_write(inode) && S_ISREG(inode->i_mode);
> > > >  }
> > > >
> > > > --
> > > > 2.43.0.472.g3155946c3a-goog
> > > >
> >
> >
> >
> > --
> > 宋方睿

Re: [PATCH] mm: remove VM_EXEC requirement for THP eligibility

[Matthew Wilcox]

On Wed, Dec 20, 2023 at 08:53:38PM -0800, Fangrui Song wrote:
> Thanks for the comment. Frankly, I am not familiar with huge pages...
> I noticed this VM_EXEC condition when I was writing this
> hugepage-related section in
> https://maskray.me/blog/2023-12-17-exploring-the-section-layout-in-linker-output#transparent-huge-pages-for-mapped-files
> (Thanks to Alexander Monakov's comment about
> CONFIG_READ_ONLY_THP_FOR_FS in
> https://mazzo.li/posts/check-huge-page.html).

CONFIG_READ_ONLY_THP_FOR_FS is a preliminary hack which solves some
problems.  The real solution is using large folios, which at the moment
means that you should test on XFS or AFS; filesystem authors have not
been enthusiastic about adding support to their filesystems so far.

In your blog, you write:

: In -z noseparate-code layouts, the file content starts somewhere at
: the first page, potentially wasting half a huge page on unrelated
: content. Switching to -z separate-code allows reclaiming the benefits
: of the half huge page but increases the file size. Balancing
: these aspects poses a challenge. One potential solution is using
: fallocate(FALLOC_FL_PUNCH_HOLE), which introduces complexity into the
: linker. However, this approach feels like a workaround to address a
: kernel limitation. It would be preferable if a file-backed huge page
: didn't necessitate a file offset aligned to a huge page boundary.

You should distinguish between file size (ie st_size in stat(3)) and
amount of space occupied on storage (st_blocks).  The linker should be
fine with creating a sparse file.  If it doesn't, cp --sparse will do
the trick.

Yes, it's a kernel limitation that folios have to be aligned within the
file as well as in both virtual and physical address space.  It's a huge
complexity win to do that; I don't think we'd be able to tile the page
cache effectively if we allowed folios to be placed at arbitrary offsets
(I think it turns into a knapsack problem at that point).

> As dTLB for read-only data is also an important optimization of
> file-backed THP, it seems straightforward that we should drop the
> VM_EXEC condition :)

I'm not particularly enthusiastic about making CONFIG_READ_ONLY_THP_FOR_FS
better.  Large folios are the future.  Indeed, I'd like to see
CONFIG_READ_ONLY_THP_FOR_FS go away in the next year or two once
btrfs and ext4 have support for large folios.