* [PATCH v3 0/2] mm: introduce per-order mTHP split counters
@ 2024-07-04 1:29 Lance Yang
2024-07-04 1:29 ` [PATCH v3 1/2] mm: add " Lance Yang
2024-07-04 1:29 ` [PATCH v3 2/2] mm: add docs for " Lance Yang
0 siblings, 2 replies; 15+ messages in thread
From: Lance Yang @ 2024-07-04 1:29 UTC (permalink / raw)
To: akpm
Cc: dj456119, 21cnbao, ryan.roberts, david, shy828301, ziy,
libang.li, baolin.wang, linux-kernel, linux-mm, Lance Yang
Hi all,
At present, the split counters in THP statistics no longer include
PTE-mapped mTHP. Therefore, we want to introduce per-order mTHP split
counters to monitor the frequency of mTHP splits. This will assist
developers in better analyzing and optimizing system performance.
/sys/kernel/mm/transparent_hugepage/hugepages-<size>/stats
split
split_failed
split_deferred
---
Changes since v2 [2]
====================
- mm: add per-order mTHP split counters
- Pick AB from Barry - thanks!
- Pick RB from Baolin - thanks!
- Pick RB from Ryan - thanks!
- Make things more readable (per Barry and Baolin)
- mm: add docs for per-order mTHP split counters
- Pick RB from Barry - thanks!
- Improve the doc as suggested by Ryan
- Remove the outdated note (per Ryan)
Changes since v1 [1]
====================
- mm: add per-order mTHP split counters
- Update the changelog
- Drop '_page' from mTHP split counter names (per David and Ryan)
- Store the order of the folio in a variable and reuse it later
(per Bang)
- mm: add docs for per-order mTHP split counters
- Improve the doc as suggested by Ryan
[1] https://lore.kernel.org/linux-mm/20240424135148.30422-1-ioworker0@gmail.com
[2] https://lore.kernel.org/linux-mm/20240628130750.73097-1-ioworker0@gmail.com
Lance Yang (2):
mm: add per-order mTHP split counters
mm: add docs for per-order mTHP split counters
Documentation/admin-guide/mm/transhuge.rst | 20 ++++++++++++++++----
include/linux/huge_mm.h | 3 +++
mm/huge_memory.c | 12 ++++++++++--
3 files changed, 29 insertions(+), 6 deletions(-)
base-commit: ec34ecf3924f09a694e5c7d8d7e785b82f67f9f0
--
2.45.2
^ permalink raw reply [flat|nested] 15+ messages in thread* [PATCH v3 1/2] mm: add per-order mTHP split counters 2024-07-04 1:29 [PATCH v3 0/2] mm: introduce per-order mTHP split counters Lance Yang @ 2024-07-04 1:29 ` Lance Yang 2024-07-05 9:08 ` David Hildenbrand 2024-07-04 1:29 ` [PATCH v3 2/2] mm: add docs for " Lance Yang 1 sibling, 1 reply; 15+ messages in thread From: Lance Yang @ 2024-07-04 1:29 UTC (permalink / raw) To: akpm Cc: dj456119, 21cnbao, ryan.roberts, david, shy828301, ziy, libang.li, baolin.wang, linux-kernel, linux-mm, Lance Yang, Barry Song, Mingzhe Yang Currently, the split counters in THP statistics no longer include PTE-mapped mTHP. Therefore, we propose introducing per-order mTHP split counters to monitor the frequency of mTHP splits. This will help developers better analyze and optimize system performance. /sys/kernel/mm/transparent_hugepage/hugepages-<size>/stats split split_failed split_deferred Reviewed-by: Ryan Roberts <ryan.roberts@arm.com> Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com> Acked-by: Barry Song <baohua@kernel.org> Signed-off-by: Mingzhe Yang <mingzhe.yang@ly.com> Signed-off-by: Lance Yang <ioworker0@gmail.com> --- include/linux/huge_mm.h | 3 +++ mm/huge_memory.c | 12 ++++++++++-- 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 212cca384d7e..cee3c5da8f0e 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -284,6 +284,9 @@ enum mthp_stat_item { MTHP_STAT_FILE_ALLOC, MTHP_STAT_FILE_FALLBACK, MTHP_STAT_FILE_FALLBACK_CHARGE, + MTHP_STAT_SPLIT, + MTHP_STAT_SPLIT_FAILED, + MTHP_STAT_SPLIT_DEFERRED, __MTHP_STAT_COUNT }; diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 954c63575917..5cbd838e96e6 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -559,6 +559,9 @@ DEFINE_MTHP_STAT_ATTR(swpout_fallback, MTHP_STAT_SWPOUT_FALLBACK); DEFINE_MTHP_STAT_ATTR(file_alloc, MTHP_STAT_FILE_ALLOC); DEFINE_MTHP_STAT_ATTR(file_fallback, MTHP_STAT_FILE_FALLBACK); DEFINE_MTHP_STAT_ATTR(file_fallback_charge, MTHP_STAT_FILE_FALLBACK_CHARGE); +DEFINE_MTHP_STAT_ATTR(split, MTHP_STAT_SPLIT); +DEFINE_MTHP_STAT_ATTR(split_failed, MTHP_STAT_SPLIT_FAILED); +DEFINE_MTHP_STAT_ATTR(split_deferred, MTHP_STAT_SPLIT_DEFERRED); static struct attribute *stats_attrs[] = { &anon_fault_alloc_attr.attr, @@ -569,6 +572,9 @@ static struct attribute *stats_attrs[] = { &file_alloc_attr.attr, &file_fallback_attr.attr, &file_fallback_charge_attr.attr, + &split_attr.attr, + &split_failed_attr.attr, + &split_deferred_attr.attr, NULL, }; @@ -3068,7 +3074,7 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, XA_STATE_ORDER(xas, &folio->mapping->i_pages, folio->index, new_order); struct anon_vma *anon_vma = NULL; struct address_space *mapping = NULL; - bool is_thp = folio_test_pmd_mappable(folio); + int order = folio_order(folio); int extra_pins, ret; pgoff_t end; bool is_hzp; @@ -3253,8 +3259,9 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, i_mmap_unlock_read(mapping); out: xas_destroy(&xas); - if (is_thp) + if (order >= HPAGE_PMD_ORDER) count_vm_event(!ret ? THP_SPLIT_PAGE : THP_SPLIT_PAGE_FAILED); + count_mthp_stat(order, !ret ? MTHP_STAT_SPLIT : MTHP_STAT_SPLIT_FAILED); return ret; } @@ -3307,6 +3314,7 @@ void deferred_split_folio(struct folio *folio) if (list_empty(&folio->_deferred_list)) { if (folio_test_pmd_mappable(folio)) count_vm_event(THP_DEFERRED_SPLIT_PAGE); + count_mthp_stat(folio_order(folio), MTHP_STAT_SPLIT_DEFERRED); list_add_tail(&folio->_deferred_list, &ds_queue->split_queue); ds_queue->split_queue_len++; #ifdef CONFIG_MEMCG -- 2.45.2 ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 1/2] mm: add per-order mTHP split counters 2024-07-04 1:29 ` [PATCH v3 1/2] mm: add " Lance Yang @ 2024-07-05 9:08 ` David Hildenbrand 2024-07-05 10:12 ` Barry Song 0 siblings, 1 reply; 15+ messages in thread From: David Hildenbrand @ 2024-07-05 9:08 UTC (permalink / raw) To: Lance Yang, akpm Cc: dj456119, 21cnbao, ryan.roberts, shy828301, ziy, libang.li, baolin.wang, linux-kernel, linux-mm, Barry Song, Mingzhe Yang > @@ -3253,8 +3259,9 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, > i_mmap_unlock_read(mapping); > out: > xas_destroy(&xas); > - if (is_thp) > + if (order >= HPAGE_PMD_ORDER) We likely should be using "== HPAGE_PMD_ORDER" here, to be safe for the future. Acked-by: David Hildenbrand <david@redhat.com> -- Cheers, David / dhildenb ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 1/2] mm: add per-order mTHP split counters 2024-07-05 9:08 ` David Hildenbrand @ 2024-07-05 10:12 ` Barry Song 2024-07-05 10:14 ` David Hildenbrand 0 siblings, 1 reply; 15+ messages in thread From: Barry Song @ 2024-07-05 10:12 UTC (permalink / raw) To: David Hildenbrand Cc: Lance Yang, akpm, dj456119, ryan.roberts, shy828301, ziy, libang.li, baolin.wang, linux-kernel, linux-mm, Mingzhe Yang On Fri, Jul 5, 2024 at 9:08 PM David Hildenbrand <david@redhat.com> wrote: > > > @@ -3253,8 +3259,9 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, > > i_mmap_unlock_read(mapping); > > out: > > xas_destroy(&xas); > > - if (is_thp) > > + if (order >= HPAGE_PMD_ORDER) > > We likely should be using "== HPAGE_PMD_ORDER" here, to be safe for the > future. I feel this might need to be separate since all other places are using folio_test_pmd_mappable() ? > > Acked-by: David Hildenbrand <david@redhat.com> > > -- > Cheers, > > David / dhildenb > ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 1/2] mm: add per-order mTHP split counters 2024-07-05 10:12 ` Barry Song @ 2024-07-05 10:14 ` David Hildenbrand 2024-07-05 10:48 ` Lance Yang 0 siblings, 1 reply; 15+ messages in thread From: David Hildenbrand @ 2024-07-05 10:14 UTC (permalink / raw) To: Barry Song Cc: Lance Yang, akpm, dj456119, ryan.roberts, shy828301, ziy, libang.li, baolin.wang, linux-kernel, linux-mm, Mingzhe Yang On 05.07.24 12:12, Barry Song wrote: > On Fri, Jul 5, 2024 at 9:08 PM David Hildenbrand <david@redhat.com> wrote: >> >>> @@ -3253,8 +3259,9 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, >>> i_mmap_unlock_read(mapping); >>> out: >>> xas_destroy(&xas); >>> - if (is_thp) >>> + if (order >= HPAGE_PMD_ORDER) >> >> We likely should be using "== HPAGE_PMD_ORDER" here, to be safe for the >> future. > > I feel this might need to be separate since all other places are using > folio_test_pmd_mappable() ? Likely, but as you are moving away from this ... this counter here does and will always only care about HPAGE_PMD_ORDER. -- Cheers, David / dhildenb ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 1/2] mm: add per-order mTHP split counters 2024-07-05 10:14 ` David Hildenbrand @ 2024-07-05 10:48 ` Lance Yang 2024-07-05 10:55 ` David Hildenbrand 0 siblings, 1 reply; 15+ messages in thread From: Lance Yang @ 2024-07-05 10:48 UTC (permalink / raw) To: David Hildenbrand Cc: Barry Song, akpm, dj456119, ryan.roberts, shy828301, ziy, libang.li, baolin.wang, linux-kernel, linux-mm, Mingzhe Yang Hi David and Barry, Thanks a lot for paying attention! On Fri, Jul 5, 2024 at 6:14 PM David Hildenbrand <david@redhat.com> wrote: > > On 05.07.24 12:12, Barry Song wrote: > > On Fri, Jul 5, 2024 at 9:08 PM David Hildenbrand <david@redhat.com> wrote: > >> > >>> @@ -3253,8 +3259,9 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, > >>> i_mmap_unlock_read(mapping); > >>> out: > >>> xas_destroy(&xas); > >>> - if (is_thp) > >>> + if (order >= HPAGE_PMD_ORDER) > >> > >> We likely should be using "== HPAGE_PMD_ORDER" here, to be safe for the > >> future. > > > > I feel this might need to be separate since all other places are using > > folio_test_pmd_mappable() ? > > Likely, but as you are moving away from this ... this counter here does > and will always only care about HPAGE_PMD_ORDER. I appreciate the different opinions on whether we should use ">= HPAGE_PMD_ORDER" or "==" for this check. In this context, let's leave it as is and stay consistent with folio_test_pmd_mappable() by using ">= HPAGE_PMD_ORDER", what do you think? Thanks, Lance > > -- > Cheers, > > David / dhildenb > ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 1/2] mm: add per-order mTHP split counters 2024-07-05 10:48 ` Lance Yang @ 2024-07-05 10:55 ` David Hildenbrand 2024-07-05 11:19 ` Lance Yang 0 siblings, 1 reply; 15+ messages in thread From: David Hildenbrand @ 2024-07-05 10:55 UTC (permalink / raw) To: Lance Yang Cc: Barry Song, akpm, dj456119, ryan.roberts, shy828301, ziy, libang.li, baolin.wang, linux-kernel, linux-mm, Mingzhe Yang On 05.07.24 12:48, Lance Yang wrote: > Hi David and Barry, > > Thanks a lot for paying attention! > > On Fri, Jul 5, 2024 at 6:14 PM David Hildenbrand <david@redhat.com> wrote: >> >> On 05.07.24 12:12, Barry Song wrote: >>> On Fri, Jul 5, 2024 at 9:08 PM David Hildenbrand <david@redhat.com> wrote: >>>> >>>>> @@ -3253,8 +3259,9 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, >>>>> i_mmap_unlock_read(mapping); >>>>> out: >>>>> xas_destroy(&xas); >>>>> - if (is_thp) >>>>> + if (order >= HPAGE_PMD_ORDER) >>>> >>>> We likely should be using "== HPAGE_PMD_ORDER" here, to be safe for the >>>> future. >>> >>> I feel this might need to be separate since all other places are using >>> folio_test_pmd_mappable() ? >> >> Likely, but as you are moving away from this ... this counter here does >> and will always only care about HPAGE_PMD_ORDER. > > I appreciate the different opinions on whether we should use > ">= HPAGE_PMD_ORDER" or "==" for this check. > > In this context, let's leave it as is and stay consistent with > folio_test_pmd_mappable() by using ">= HPAGE_PMD_ORDER", > what do you think? I don't think it's a good idea to add more wrong code that is even harder to grep (folio_test_pmd_mappable would give you candidates that might need attention). But I don't care too much. Maybe someone here can volunteer to clean up these instances to make sure we check PMD-size and not PMD-mappable for these counters that are for PMD-sized folios only, even in the future with larger folios? -- Cheers, David / dhildenb ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 1/2] mm: add per-order mTHP split counters 2024-07-05 10:55 ` David Hildenbrand @ 2024-07-05 11:19 ` Lance Yang 2024-07-05 11:31 ` Lance Yang 0 siblings, 1 reply; 15+ messages in thread From: Lance Yang @ 2024-07-05 11:19 UTC (permalink / raw) To: David Hildenbrand Cc: Barry Song, akpm, dj456119, ryan.roberts, shy828301, ziy, libang.li, baolin.wang, linux-kernel, linux-mm, Mingzhe Yang On Fri, Jul 5, 2024 at 6:56 PM David Hildenbrand <david@redhat.com> wrote: > > On 05.07.24 12:48, Lance Yang wrote: > > Hi David and Barry, > > > > Thanks a lot for paying attention! > > > > On Fri, Jul 5, 2024 at 6:14 PM David Hildenbrand <david@redhat.com> wrote: > >> > >> On 05.07.24 12:12, Barry Song wrote: > >>> On Fri, Jul 5, 2024 at 9:08 PM David Hildenbrand <david@redhat.com> wrote: > >>>> > >>>>> @@ -3253,8 +3259,9 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, > >>>>> i_mmap_unlock_read(mapping); > >>>>> out: > >>>>> xas_destroy(&xas); > >>>>> - if (is_thp) > >>>>> + if (order >= HPAGE_PMD_ORDER) > >>>> > >>>> We likely should be using "== HPAGE_PMD_ORDER" here, to be safe for the > >>>> future. > >>> > >>> I feel this might need to be separate since all other places are using > >>> folio_test_pmd_mappable() ? > >> > >> Likely, but as you are moving away from this ... this counter here does > >> and will always only care about HPAGE_PMD_ORDER. > > > > I appreciate the different opinions on whether we should use > > ">= HPAGE_PMD_ORDER" or "==" for this check. > > > > In this context, let's leave it as is and stay consistent with > > folio_test_pmd_mappable() by using ">= HPAGE_PMD_ORDER", > > what do you think? > > I don't think it's a good idea to add more wrong code that is even > harder to grep (folio_test_pmd_mappable would give you candidates that > might need attention). But I don't care too much. Maybe someone here can > volunteer to clean up these instances to make sure we check PMD-size and > not PMD-mappable for these counters that are for PMD-sized folios only, > even in the future with larger folios? Thanks for clarifying! Yes, agreed. We should ensure we check PMD-size, not PMD-mappable here, especially as we consider large folios in the future. So, let's use "== HPAGE_PMD_ORDER" here ;) Thanks, Lance > > -- > Cheers, > > David / dhildenb > ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 1/2] mm: add per-order mTHP split counters 2024-07-05 11:19 ` Lance Yang @ 2024-07-05 11:31 ` Lance Yang 0 siblings, 0 replies; 15+ messages in thread From: Lance Yang @ 2024-07-05 11:31 UTC (permalink / raw) To: akpm Cc: ioworker0, 21cnbao, baolin.wang, david, dj456119, libang.li, linux-kernel, linux-mm, mingzhe.yang, ryan.roberts, shy828301, ziy Hi Andrew, Could you please fold the following changes into this patch? diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 993f0dd2a1ed..f8b5cbd4dd71 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -3266,7 +3266,7 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list, i_mmap_unlock_read(mapping); out: xas_destroy(&xas); - if (order >= HPAGE_PMD_ORDER) + if (order == HPAGE_PMD_ORDER) count_vm_event(!ret ? THP_SPLIT_PAGE : THP_SPLIT_PAGE_FAILED); count_mthp_stat(order, !ret ? MTHP_STAT_SPLIT : MTHP_STAT_SPLIT_FAILED); return ret; Thanks, Lance ^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v3 2/2] mm: add docs for per-order mTHP split counters 2024-07-04 1:29 [PATCH v3 0/2] mm: introduce per-order mTHP split counters Lance Yang 2024-07-04 1:29 ` [PATCH v3 1/2] mm: add " Lance Yang @ 2024-07-04 1:29 ` Lance Yang 2024-07-05 9:09 ` David Hildenbrand ` (2 more replies) 1 sibling, 3 replies; 15+ messages in thread From: Lance Yang @ 2024-07-04 1:29 UTC (permalink / raw) To: akpm Cc: dj456119, 21cnbao, ryan.roberts, david, shy828301, ziy, libang.li, baolin.wang, linux-kernel, linux-mm, Lance Yang, Barry Song, Mingzhe Yang This commit introduces documentation for mTHP split counters in transhuge.rst. Reviewed-by: Barry Song <baohua@kernel.org> Signed-off-by: Mingzhe Yang <mingzhe.yang@ly.com> Signed-off-by: Lance Yang <ioworker0@gmail.com> --- Documentation/admin-guide/mm/transhuge.rst | 20 ++++++++++++++++---- 1 file changed, 16 insertions(+), 4 deletions(-) diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst index 1f72b00af5d3..0830aa173a8b 100644 --- a/Documentation/admin-guide/mm/transhuge.rst +++ b/Documentation/admin-guide/mm/transhuge.rst @@ -369,10 +369,6 @@ also applies to the regions registered in khugepaged. Monitoring usage ================ -.. note:: - Currently the below counters only record events relating to - PMD-sized THP. Events relating to other THP sizes are not included. - The number of PMD-sized anonymous transparent huge pages currently used by the system is available by reading the AnonHugePages field in ``/proc/meminfo``. To identify what applications are using PMD-sized anonymous transparent huge @@ -514,6 +510,22 @@ file_fallback_charge falls back to using small pages even though the allocation was successful. +split + is incremented every time a huge page is successfully split into + smaller orders. This can happen for a variety of reasons but a + common reason is that a huge page is old and is being reclaimed. + This action implies splitting any block mappings into PTEs. + +split_failed + is incremented if kernel fails to split huge + page. This can happen if the page was pinned by somebody. + +split_deferred + is incremented when a huge page is put onto split + queue. This happens when a huge page is partially unmapped and + splitting it would free up some memory. Pages on split queue are + going to be split under memory pressure. + As the system ages, allocating huge pages may be expensive as the system uses memory compaction to copy data around memory to free a huge page for use. There are some counters in ``/proc/vmstat`` to help -- 2.45.2 ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 2/2] mm: add docs for per-order mTHP split counters 2024-07-04 1:29 ` [PATCH v3 2/2] mm: add docs for " Lance Yang @ 2024-07-05 9:09 ` David Hildenbrand 2024-07-05 9:16 ` Ryan Roberts 2024-07-07 1:36 ` Lance Yang 2 siblings, 0 replies; 15+ messages in thread From: David Hildenbrand @ 2024-07-05 9:09 UTC (permalink / raw) To: Lance Yang, akpm Cc: dj456119, 21cnbao, ryan.roberts, shy828301, ziy, libang.li, baolin.wang, linux-kernel, linux-mm, Barry Song, Mingzhe Yang On 04.07.24 03:29, Lance Yang wrote: > This commit introduces documentation for mTHP split counters in > transhuge.rst. > > Reviewed-by: Barry Song <baohua@kernel.org> > Signed-off-by: Mingzhe Yang <mingzhe.yang@ly.com> > Signed-off-by: Lance Yang <ioworker0@gmail.com> > --- > Documentation/admin-guide/mm/transhuge.rst | 20 ++++++++++++++++---- > 1 file changed, 16 insertions(+), 4 deletions(-) > > diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst > index 1f72b00af5d3..0830aa173a8b 100644 > --- a/Documentation/admin-guide/mm/transhuge.rst > +++ b/Documentation/admin-guide/mm/transhuge.rst > @@ -369,10 +369,6 @@ also applies to the regions registered in khugepaged. > Monitoring usage > ================ > > -.. note:: > - Currently the below counters only record events relating to > - PMD-sized THP. Events relating to other THP sizes are not included. > - > The number of PMD-sized anonymous transparent huge pages currently used by the > system is available by reading the AnonHugePages field in ``/proc/meminfo``. > To identify what applications are using PMD-sized anonymous transparent huge > @@ -514,6 +510,22 @@ file_fallback_charge > falls back to using small pages even though the allocation was > successful. > > +split > + is incremented every time a huge page is successfully split into > + smaller orders. This can happen for a variety of reasons but a > + common reason is that a huge page is old and is being reclaimed. > + This action implies splitting any block mappings into PTEs. > + > +split_failed > + is incremented if kernel fails to split huge > + page. This can happen if the page was pinned by somebody. > + > +split_deferred > + is incremented when a huge page is put onto split > + queue. This happens when a huge page is partially unmapped and > + splitting it would free up some memory. Pages on split queue are > + going to be split under memory pressure. ".. if splitting is possible." ;) Acked-by: David Hildenbrand <david@redhat.com> -- Cheers, David / dhildenb ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 2/2] mm: add docs for per-order mTHP split counters 2024-07-04 1:29 ` [PATCH v3 2/2] mm: add docs for " Lance Yang 2024-07-05 9:09 ` David Hildenbrand @ 2024-07-05 9:16 ` Ryan Roberts 2024-07-05 9:25 ` David Hildenbrand 2024-07-07 1:36 ` Lance Yang 2 siblings, 1 reply; 15+ messages in thread From: Ryan Roberts @ 2024-07-05 9:16 UTC (permalink / raw) To: Lance Yang, akpm Cc: dj456119, 21cnbao, david, shy828301, ziy, libang.li, baolin.wang, linux-kernel, linux-mm, Barry Song, Mingzhe Yang On 04/07/2024 02:29, Lance Yang wrote: > This commit introduces documentation for mTHP split counters in > transhuge.rst. > > Reviewed-by: Barry Song <baohua@kernel.org> > Signed-off-by: Mingzhe Yang <mingzhe.yang@ly.com> > Signed-off-by: Lance Yang <ioworker0@gmail.com> > --- > Documentation/admin-guide/mm/transhuge.rst | 20 ++++++++++++++++---- > 1 file changed, 16 insertions(+), 4 deletions(-) > > diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst > index 1f72b00af5d3..0830aa173a8b 100644 > --- a/Documentation/admin-guide/mm/transhuge.rst > +++ b/Documentation/admin-guide/mm/transhuge.rst > @@ -369,10 +369,6 @@ also applies to the regions registered in khugepaged. > Monitoring usage > ================ > > -.. note:: > - Currently the below counters only record events relating to > - PMD-sized THP. Events relating to other THP sizes are not included. > - > The number of PMD-sized anonymous transparent huge pages currently used by the > system is available by reading the AnonHugePages field in ``/proc/meminfo``. > To identify what applications are using PMD-sized anonymous transparent huge > @@ -514,6 +510,22 @@ file_fallback_charge > falls back to using small pages even though the allocation was > successful. > > +split > + is incremented every time a huge page is successfully split into > + smaller orders. This can happen for a variety of reasons but a > + common reason is that a huge page is old and is being reclaimed. > + This action implies splitting any block mappings into PTEs. nit: the block mappings will already be PTEs if starting with mTHP? regardless: Reviewed-by: Ryan Roberts <ryan.roberts@arm.com> > + > +split_failed > + is incremented if kernel fails to split huge > + page. This can happen if the page was pinned by somebody. > + > +split_deferred > + is incremented when a huge page is put onto split > + queue. This happens when a huge page is partially unmapped and > + splitting it would free up some memory. Pages on split queue are > + going to be split under memory pressure. > + > As the system ages, allocating huge pages may be expensive as the > system uses memory compaction to copy data around memory to free a > huge page for use. There are some counters in ``/proc/vmstat`` to help ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 2/2] mm: add docs for per-order mTHP split counters 2024-07-05 9:16 ` Ryan Roberts @ 2024-07-05 9:25 ` David Hildenbrand 2024-07-05 11:08 ` Lance Yang 0 siblings, 1 reply; 15+ messages in thread From: David Hildenbrand @ 2024-07-05 9:25 UTC (permalink / raw) To: Ryan Roberts, Lance Yang, akpm Cc: dj456119, 21cnbao, shy828301, ziy, libang.li, baolin.wang, linux-kernel, linux-mm, Barry Song, Mingzhe Yang On 05.07.24 11:16, Ryan Roberts wrote: > On 04/07/2024 02:29, Lance Yang wrote: >> This commit introduces documentation for mTHP split counters in >> transhuge.rst. >> >> Reviewed-by: Barry Song <baohua@kernel.org> >> Signed-off-by: Mingzhe Yang <mingzhe.yang@ly.com> >> Signed-off-by: Lance Yang <ioworker0@gmail.com> >> --- >> Documentation/admin-guide/mm/transhuge.rst | 20 ++++++++++++++++---- >> 1 file changed, 16 insertions(+), 4 deletions(-) >> >> diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst >> index 1f72b00af5d3..0830aa173a8b 100644 >> --- a/Documentation/admin-guide/mm/transhuge.rst >> +++ b/Documentation/admin-guide/mm/transhuge.rst >> @@ -369,10 +369,6 @@ also applies to the regions registered in khugepaged. >> Monitoring usage >> ================ >> >> -.. note:: >> - Currently the below counters only record events relating to >> - PMD-sized THP. Events relating to other THP sizes are not included. >> - >> The number of PMD-sized anonymous transparent huge pages currently used by the >> system is available by reading the AnonHugePages field in ``/proc/meminfo``. >> To identify what applications are using PMD-sized anonymous transparent huge >> @@ -514,6 +510,22 @@ file_fallback_charge >> falls back to using small pages even though the allocation was >> successful. >> >> +split >> + is incremented every time a huge page is successfully split into >> + smaller orders. This can happen for a variety of reasons but a >> + common reason is that a huge page is old and is being reclaimed. >> + This action implies splitting any block mappings into PTEs. > > nit: the block mappings will already be PTEs if starting with mTHP? Was confused by that as well, so maybe just drop that detail here. -- Cheers, David / dhildenb ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 2/2] mm: add docs for per-order mTHP split counters 2024-07-05 9:25 ` David Hildenbrand @ 2024-07-05 11:08 ` Lance Yang 0 siblings, 0 replies; 15+ messages in thread From: Lance Yang @ 2024-07-05 11:08 UTC (permalink / raw) To: david Cc: 21cnbao, akpm, baohua, baolin.wang, dj456119, ioworker0, libang.li, linux-kernel, linux-mm, mingzhe.yang, ryan.roberts, shy828301, ziy Hi David and Ryan, Thanks for taking time to review! Updated the doc. How about the following? split is incremented every time a huge page is successfully split into smaller orders. This can happen for a variety of reasons but a common reason is that a huge page is old and is being reclaimed. split_failed is incremented if kernel fails to split huge page. This can happen if the page was pinned by somebody. split_deferred is incremented when a huge page is put onto split queue. This happens when a huge page is partially unmapped and splitting it would free up some memory. Pages on split queue are going to be split under memory pressure, if splitting is possible. Thanks, Lance ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 2/2] mm: add docs for per-order mTHP split counters 2024-07-04 1:29 ` [PATCH v3 2/2] mm: add docs for " Lance Yang 2024-07-05 9:09 ` David Hildenbrand 2024-07-05 9:16 ` Ryan Roberts @ 2024-07-07 1:36 ` Lance Yang 2 siblings, 0 replies; 15+ messages in thread From: Lance Yang @ 2024-07-07 1:36 UTC (permalink / raw) To: akpm Cc: ioworker0, 21cnbao, baohua, baolin.wang, david, dj456119, libang.li, linux-kernel, linux-mm, mingzhe.yang, ryan.roberts, shy828301, ziy Hi Andrew, Could you please fold the following changes into this patch? diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation/admin-guide/mm/transhuge.rst index 747c811ee8f1..fe237825b95c 100644 --- a/Documentation/admin-guide/mm/transhuge.rst +++ b/Documentation/admin-guide/mm/transhuge.rst @@ -513,17 +513,16 @@ split is incremented every time a huge page is successfully split into smaller orders. This can happen for a variety of reasons but a common reason is that a huge page is old and is being reclaimed. - This action implies splitting any block mappings into PTEs. split_failed is incremented if kernel fails to split huge page. This can happen if the page was pinned by somebody. split_deferred - is incremented when a huge page is put onto split - queue. This happens when a huge page is partially unmapped and - splitting it would free up some memory. Pages on split queue are - going to be split under memory pressure. + is incremented when a huge page is put onto split queue. + This happens when a huge page is partially unmapped and splitting + it would free up some memory. Pages on split queue are going to + be split under memory pressure, if splitting is possible. As the system ages, allocating huge pages may be expensive as the system uses memory compaction to copy data around memory to free a -- Thanks, Lance ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2024-07-07 1:37 UTC | newest] Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2024-07-04 1:29 [PATCH v3 0/2] mm: introduce per-order mTHP split counters Lance Yang 2024-07-04 1:29 ` [PATCH v3 1/2] mm: add " Lance Yang 2024-07-05 9:08 ` David Hildenbrand 2024-07-05 10:12 ` Barry Song 2024-07-05 10:14 ` David Hildenbrand 2024-07-05 10:48 ` Lance Yang 2024-07-05 10:55 ` David Hildenbrand 2024-07-05 11:19 ` Lance Yang 2024-07-05 11:31 ` Lance Yang 2024-07-04 1:29 ` [PATCH v3 2/2] mm: add docs for " Lance Yang 2024-07-05 9:09 ` David Hildenbrand 2024-07-05 9:16 ` Ryan Roberts 2024-07-05 9:25 ` David Hildenbrand 2024-07-05 11:08 ` Lance Yang 2024-07-07 1:36 ` Lance Yang
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox