* [PATCH] mm: add stack trace when bad rss-counter state is detected @ 2025-07-23 7:23 Xuanye Liu 2025-07-23 7:31 ` Kees Cook 0 siblings, 1 reply; 12+ messages in thread From: Xuanye Liu @ 2025-07-23 7:23 UTC (permalink / raw) To: Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot, Kees Cook, Andrew Morton, David Hildenbrand Cc: Xuanye Liu, Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko, Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman, Valentin Schneider, linux-mm, linux-kernel The check_mm() function verifies the correctness of rss counters in struct mm_struct. Currently, it only prints an alert when a bad rss-counter state is detected, but lacks sufficient context for debugging. This patch adds a dump_stack() call to provide a stack trace when the rss-counter state is invalid. This helps developers identify where the corrupted mm_struct is being checked and trace the underlying cause of the inconsistency. Signed-off-by: Xuanye Liu <liuqiye2025@163.com> --- kernel/fork.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/kernel/fork.c b/kernel/fork.c index cfe2f1df5f27..d38f1c5270ea 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -584,9 +584,11 @@ static void check_mm(struct mm_struct *mm) for (i = 0; i < NR_MM_COUNTERS; i++) { long x = percpu_counter_sum(&mm->rss_stat[i]); - if (unlikely(x)) + if (unlikely(x)) { pr_alert("BUG: Bad rss-counter state mm:%p type:%s val:%ld\n", mm, resident_page_types[i], x); + dump_stack(); + } } if (mm_pgtables_bytes(mm)) -- 2.43.0 ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] mm: add stack trace when bad rss-counter state is detected 2025-07-23 7:23 [PATCH] mm: add stack trace when bad rss-counter state is detected Xuanye Liu @ 2025-07-23 7:31 ` Kees Cook 2025-07-23 7:45 ` Xuanye Liu 0 siblings, 1 reply; 12+ messages in thread From: Kees Cook @ 2025-07-23 7:31 UTC (permalink / raw) To: Xuanye Liu Cc: Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot, Andrew Morton, David Hildenbrand, Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko, Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman, Valentin Schneider, linux-mm, linux-kernel On Wed, Jul 23, 2025 at 03:23:49PM +0800, Xuanye Liu wrote: > The check_mm() function verifies the correctness of rss counters in > struct mm_struct. Currently, it only prints an alert when a bad > rss-counter state is detected, but lacks sufficient context for > debugging. > > This patch adds a dump_stack() call to provide a stack trace when > the rss-counter state is invalid. This helps developers identify > where the corrupted mm_struct is being checked and trace the > underlying cause of the inconsistency. Why not just convert the pr_alert to a WARN? -Kees > > Signed-off-by: Xuanye Liu <liuqiye2025@163.com> > --- > kernel/fork.c | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) > > diff --git a/kernel/fork.c b/kernel/fork.c > index cfe2f1df5f27..d38f1c5270ea 100644 > --- a/kernel/fork.c > +++ b/kernel/fork.c > @@ -584,9 +584,11 @@ static void check_mm(struct mm_struct *mm) > for (i = 0; i < NR_MM_COUNTERS; i++) { > long x = percpu_counter_sum(&mm->rss_stat[i]); > > - if (unlikely(x)) > + if (unlikely(x)) { > pr_alert("BUG: Bad rss-counter state mm:%p type:%s val:%ld\n", > mm, resident_page_types[i], x); > + dump_stack(); > + } > } > > if (mm_pgtables_bytes(mm)) > -- > 2.43.0 > -- Kees Cook ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] mm: add stack trace when bad rss-counter state is detected 2025-07-23 7:31 ` Kees Cook @ 2025-07-23 7:45 ` Xuanye Liu 2025-07-23 8:05 ` David Hildenbrand 0 siblings, 1 reply; 12+ messages in thread From: Xuanye Liu @ 2025-07-23 7:45 UTC (permalink / raw) To: Kees Cook Cc: Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot, Andrew Morton, David Hildenbrand, Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko, Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman, Valentin Schneider, linux-mm, linux-kernel 在 2025/7/23 15:31, Kees Cook 写道: > On Wed, Jul 23, 2025 at 03:23:49PM +0800, Xuanye Liu wrote: >> The check_mm() function verifies the correctness of rss counters in >> struct mm_struct. Currently, it only prints an alert when a bad >> rss-counter state is detected, but lacks sufficient context for >> debugging. >> >> This patch adds a dump_stack() call to provide a stack trace when >> the rss-counter state is invalid. This helps developers identify >> where the corrupted mm_struct is being checked and trace the >> underlying cause of the inconsistency. > Why not just convert the pr_alert to a WARN? Good idea! I'll gather more feedback from others and then update to v2. > > -Kees > >> Signed-off-by: Xuanye Liu <liuqiye2025@163.com> >> --- >> kernel/fork.c | 4 +++- >> 1 file changed, 3 insertions(+), 1 deletion(-) >> >> diff --git a/kernel/fork.c b/kernel/fork.c >> index cfe2f1df5f27..d38f1c5270ea 100644 >> --- a/kernel/fork.c >> +++ b/kernel/fork.c >> @@ -584,9 +584,11 @@ static void check_mm(struct mm_struct *mm) >> for (i = 0; i < NR_MM_COUNTERS; i++) { >> long x = percpu_counter_sum(&mm->rss_stat[i]); >> >> - if (unlikely(x)) >> + if (unlikely(x)) { >> pr_alert("BUG: Bad rss-counter state mm:%p type:%s val:%ld\n", >> mm, resident_page_types[i], x); >> + dump_stack(); >> + } >> } >> >> if (mm_pgtables_bytes(mm)) >> -- >> 2.43.0 >> -- Thanks, Xuanye ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] mm: add stack trace when bad rss-counter state is detected 2025-07-23 7:45 ` Xuanye Liu @ 2025-07-23 8:05 ` David Hildenbrand 2025-07-23 8:42 ` David Hildenbrand 0 siblings, 1 reply; 12+ messages in thread From: David Hildenbrand @ 2025-07-23 8:05 UTC (permalink / raw) To: Xuanye Liu, Kees Cook Cc: Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot, Andrew Morton, Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko, Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman, Valentin Schneider, linux-mm, linux-kernel On 23.07.25 09:45, Xuanye Liu wrote: > > 在 2025/7/23 15:31, Kees Cook 写道: >> On Wed, Jul 23, 2025 at 03:23:49PM +0800, Xuanye Liu wrote: >>> The check_mm() function verifies the correctness of rss counters in >>> struct mm_struct. Currently, it only prints an alert when a bad >>> rss-counter state is detected, but lacks sufficient context for >>> debugging. >>> >>> This patch adds a dump_stack() call to provide a stack trace when >>> the rss-counter state is invalid. This helps developers identify >>> where the corrupted mm_struct is being checked and trace the >>> underlying cause of the inconsistency. >> Why not just convert the pr_alert to a WARN? > Good idea! I'll gather more feedback from others and then update to v2. Makes sense to me. -- Cheers, David / dhildenb ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] mm: add stack trace when bad rss-counter state is detected 2025-07-23 8:05 ` David Hildenbrand @ 2025-07-23 8:42 ` David Hildenbrand 2025-07-23 9:10 ` Xuanye Liu 0 siblings, 1 reply; 12+ messages in thread From: David Hildenbrand @ 2025-07-23 8:42 UTC (permalink / raw) To: Xuanye Liu, Kees Cook Cc: Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot, Andrew Morton, Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko, Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman, Valentin Schneider, linux-mm, linux-kernel On 23.07.25 10:05, David Hildenbrand wrote: > On 23.07.25 09:45, Xuanye Liu wrote: >> >> 在 2025/7/23 15:31, Kees Cook 写道: >>> On Wed, Jul 23, 2025 at 03:23:49PM +0800, Xuanye Liu wrote: >>>> The check_mm() function verifies the correctness of rss counters in >>>> struct mm_struct. Currently, it only prints an alert when a bad >>>> rss-counter state is detected, but lacks sufficient context for >>>> debugging. >>>> >>>> This patch adds a dump_stack() call to provide a stack trace when >>>> the rss-counter state is invalid. This helps developers identify >>>> where the corrupted mm_struct is being checked and trace the >>>> underlying cause of the inconsistency. >>> Why not just convert the pr_alert to a WARN? >> Good idea! I'll gather more feedback from others and then update to v2. > > Makes sense to me. After discussion this with Lorenzo off-list, isn't the stack completely misleading/useless in that case? Whatever caused the RSS counter mismatch (e.g., unmapped the wrong pages, missed to unmap pages) quite possibly happened in different context, way way earlier. Why would you think the stack trace would be of any value when destroying an MM (__mmdrop)? Having that said, I really hate these "pr_*("BUG: ...") with passion. Probably we'd want to invoke the panic_on_warn machinery, because something unexpected happened. -- Cheers, David / dhildenb ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] mm: add stack trace when bad rss-counter state is detected 2025-07-23 8:42 ` David Hildenbrand @ 2025-07-23 9:10 ` Xuanye Liu 2025-07-23 9:14 ` Xuanye Liu 2025-07-23 9:17 ` Vlastimil Babka 0 siblings, 2 replies; 12+ messages in thread From: Xuanye Liu @ 2025-07-23 9:10 UTC (permalink / raw) To: David Hildenbrand, Kees Cook Cc: Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot, Andrew Morton, Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko, Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman, Valentin Schneider, linux-mm, linux-kernel 在 2025/7/23 16:42, David Hildenbrand 写道: > On 23.07.25 10:05, David Hildenbrand wrote: >> On 23.07.25 09:45, Xuanye Liu wrote: >>> >>> 在 2025/7/23 15:31, Kees Cook 写道: >>>> On Wed, Jul 23, 2025 at 03:23:49PM +0800, Xuanye Liu wrote: >>>>> The check_mm() function verifies the correctness of rss counters in >>>>> struct mm_struct. Currently, it only prints an alert when a bad >>>>> rss-counter state is detected, but lacks sufficient context for >>>>> debugging. >>>>> >>>>> This patch adds a dump_stack() call to provide a stack trace when >>>>> the rss-counter state is invalid. This helps developers identify >>>>> where the corrupted mm_struct is being checked and trace the >>>>> underlying cause of the inconsistency. >>>> Why not just convert the pr_alert to a WARN? >>> Good idea! I'll gather more feedback from others and then update to v2. >> >> Makes sense to me. > > After discussion this with Lorenzo off-list, isn't the stack completely misleading/useless in that case? > > Whatever caused the RSS counter mismatch (e.g., unmapped the wrong pages, missed to unmap pages) quite possibly happened in different context, way way earlier. > > Why would you think the stack trace would be of any value when destroying an MM (__mmdrop)? > > Having that said, I really hate these "pr_*("BUG: ...") with passion. Probably we'd want to invoke the panic_on_warn machinery, because something unexpected happened. > The stack trace dumped here may indeed not reflect the root cause —— the actual error could have occurred much earlier, for example during a failed or missing page map/unmap operation. The current stack (e.g., in __mmdrop() or exit_mmap()) is merely part of the cleanup phase. Given that, how should we go about identifying the root cause when such an issue occurs? Is there any existing way to trace it more effectively, or could we introduce a new mechanism to monitor and detect these inconsistencies earlier? Let’s brainstorm possible solutions together. -- Thanks, Xuanye ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] mm: add stack trace when bad rss-counter state is detected 2025-07-23 9:10 ` Xuanye Liu @ 2025-07-23 9:14 ` Xuanye Liu 2025-07-23 9:22 ` Lorenzo Stoakes 2025-07-23 9:17 ` Vlastimil Babka 1 sibling, 1 reply; 12+ messages in thread From: Xuanye Liu @ 2025-07-23 9:14 UTC (permalink / raw) To: David Hildenbrand, Kees Cook Cc: Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot, Andrew Morton, Lorenzo Stoakes, Liam R. Howlett, Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko, Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman, Valentin Schneider, linux-mm, linux-kernel 在 2025/7/23 17:10, Xuanye Liu 写道: > 在 2025/7/23 16:42, David Hildenbrand 写道: >> On 23.07.25 10:05, David Hildenbrand wrote: >>> On 23.07.25 09:45, Xuanye Liu wrote: >>>> 在 2025/7/23 15:31, Kees Cook 写道: >>>>> On Wed, Jul 23, 2025 at 03:23:49PM +0800, Xuanye Liu wrote: >>>>>> The check_mm() function verifies the correctness of rss counters in >>>>>> struct mm_struct. Currently, it only prints an alert when a bad >>>>>> rss-counter state is detected, but lacks sufficient context for >>>>>> debugging. >>>>>> >>>>>> This patch adds a dump_stack() call to provide a stack trace when >>>>>> the rss-counter state is invalid. This helps developers identify >>>>>> where the corrupted mm_struct is being checked and trace the >>>>>> underlying cause of the inconsistency. >>>>> Why not just convert the pr_alert to a WARN? >>>> Good idea! I'll gather more feedback from others and then update to v2. >>> Makes sense to me. >> After discussion this with Lorenzo off-list, isn't the stack completely misleading/useless in that case? >> >> Whatever caused the RSS counter mismatch (e.g., unmapped the wrong pages, missed to unmap pages) quite possibly happened in different context, way way earlier. >> >> Why would you think the stack trace would be of any value when destroying an MM (__mmdrop)? >> >> Having that said, I really hate these "pr_*("BUG: ...") with passion. Probably we'd want to invoke the panic_on_warn machinery, because something unexpected happened. >> > The stack trace dumped here may indeed not reflect the root cause —— > the actual error could have occurred much earlier, for example during a > failed or missing page map/unmap operation. > The current stack (e.g., in __mmdrop() or exit_mmap()) is merely part > of the cleanup phase. Dumping the stack still has some chance of helping identify the issue — at the very least, it shows which task triggered the check. > > Given that, how should we go about identifying the root cause when such an issue occurs? > > Is there any existing way to trace it more effectively, or could we introduce a new mechanism > to monitor and detect these inconsistencies earlier? > > Let’s brainstorm possible solutions together. > -- Thanks, Xuanye ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] mm: add stack trace when bad rss-counter state is detected 2025-07-23 9:14 ` Xuanye Liu @ 2025-07-23 9:22 ` Lorenzo Stoakes 2025-07-23 9:31 ` Xuanye Liu 0 siblings, 1 reply; 12+ messages in thread From: Lorenzo Stoakes @ 2025-07-23 9:22 UTC (permalink / raw) To: Xuanye Liu Cc: David Hildenbrand, Kees Cook, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot, Andrew Morton, Liam R. Howlett, Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko, Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman, Valentin Schneider, linux-mm, linux-kernel On Wed, Jul 23, 2025 at 05:14:19PM +0800, Xuanye Liu wrote: > > 在 2025/7/23 17:10, Xuanye Liu 写道: > > 在 2025/7/23 16:42, David Hildenbrand 写道: > >> On 23.07.25 10:05, David Hildenbrand wrote: > >>> On 23.07.25 09:45, Xuanye Liu wrote: > >>>> 在 2025/7/23 15:31, Kees Cook 写道: > >>>>> On Wed, Jul 23, 2025 at 03:23:49PM +0800, Xuanye Liu wrote: > >>>>>> The check_mm() function verifies the correctness of rss counters in > >>>>>> struct mm_struct. Currently, it only prints an alert when a bad > >>>>>> rss-counter state is detected, but lacks sufficient context for > >>>>>> debugging. > >>>>>> > >>>>>> This patch adds a dump_stack() call to provide a stack trace when > >>>>>> the rss-counter state is invalid. This helps developers identify > >>>>>> where the corrupted mm_struct is being checked and trace the > >>>>>> underlying cause of the inconsistency. > >>>>> Why not just convert the pr_alert to a WARN? > >>>> Good idea! I'll gather more feedback from others and then update to v2. > >>> Makes sense to me. > >> After discussion this with Lorenzo off-list, isn't the stack completely misleading/useless in that case? > >> > >> Whatever caused the RSS counter mismatch (e.g., unmapped the wrong pages, missed to unmap pages) quite possibly happened in different context, way way earlier. > >> > >> Why would you think the stack trace would be of any value when destroying an MM (__mmdrop)? > >> > >> Having that said, I really hate these "pr_*("BUG: ...") with passion. Probably we'd want to invoke the panic_on_warn machinery, because something unexpected happened. > >> > > The stack trace dumped here may indeed not reflect the root cause —— > > the actual error could have occurred much earlier, for example during a > > failed or missing page map/unmap operation. > > The current stack (e.g., in __mmdrop() or exit_mmap()) is merely part > > of the cleanup phase. > > Dumping the stack still has some chance of helping identify the issue — at the very least, it > shows which task triggered the check. The stack will be actively misleading because it's highly likely to be totally unrelated. if you want to know the task, just output current->comm :) I think it's not only of no value, it's _ACTIVELY_ misleading. So it's definitely a no to a dump_stack(). I am also not in favour of a WARN_ON() for the same reason. Really we should be catching these elsewhere. If you want to send the patch just outputting thet ask then all good. > > > > > Given that, how should we go about identifying the root cause when such an issue occurs? > > > > Is there any existing way to trace it more effectively, or could we introduce a new mechanism > > to monitor and detect these inconsistencies earlier? > > > > Let’s brainstorm possible solutions together. > > > -- > Thanks, > Xuanye > ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] mm: add stack trace when bad rss-counter state is detected 2025-07-23 9:22 ` Lorenzo Stoakes @ 2025-07-23 9:31 ` Xuanye Liu 0 siblings, 0 replies; 12+ messages in thread From: Xuanye Liu @ 2025-07-23 9:31 UTC (permalink / raw) To: Lorenzo Stoakes Cc: David Hildenbrand, Kees Cook, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot, Andrew Morton, Liam R. Howlett, Vlastimil Babka, Mike Rapoport, Suren Baghdasaryan, Michal Hocko, Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman, Valentin Schneider, linux-mm, linux-kernel [-- Attachment #1: Type: text/plain, Size: 3104 bytes --] 在 2025/7/23 17:22, Lorenzo Stoakes 写道: > On Wed, Jul 23, 2025 at 05:14:19PM +0800, Xuanye Liu wrote: >> 在 2025/7/23 17:10, Xuanye Liu 写道: >>> 在 2025/7/23 16:42, David Hildenbrand 写道: >>>> On 23.07.25 10:05, David Hildenbrand wrote: >>>>> On 23.07.25 09:45, Xuanye Liu wrote: >>>>>> 在 2025/7/23 15:31, Kees Cook 写道: >>>>>>> On Wed, Jul 23, 2025 at 03:23:49PM +0800, Xuanye Liu wrote: >>>>>>>> The check_mm() function verifies the correctness of rss counters in >>>>>>>> struct mm_struct. Currently, it only prints an alert when a bad >>>>>>>> rss-counter state is detected, but lacks sufficient context for >>>>>>>> debugging. >>>>>>>> >>>>>>>> This patch adds a dump_stack() call to provide a stack trace when >>>>>>>> the rss-counter state is invalid. This helps developers identify >>>>>>>> where the corrupted mm_struct is being checked and trace the >>>>>>>> underlying cause of the inconsistency. >>>>>>> Why not just convert the pr_alert to a WARN? >>>>>> Good idea! I'll gather more feedback from others and then update to v2. >>>>> Makes sense to me. >>>> After discussion this with Lorenzo off-list, isn't the stack completely misleading/useless in that case? >>>> >>>> Whatever caused the RSS counter mismatch (e.g., unmapped the wrong pages, missed to unmap pages) quite possibly happened in different context, way way earlier. >>>> >>>> Why would you think the stack trace would be of any value when destroying an MM (__mmdrop)? >>>> >>>> Having that said, I really hate these "pr_*("BUG: ...") with passion. Probably we'd want to invoke the panic_on_warn machinery, because something unexpected happened. >>>> >>> The stack trace dumped here may indeed not reflect the root cause —— >>> the actual error could have occurred much earlier, for example during a >>> failed or missing page map/unmap operation. >>> The current stack (e.g., in __mmdrop() or exit_mmap()) is merely part >>> of the cleanup phase. >> Dumping the stack still has some chance of helping identify the issue — at the very least, it >> shows which task triggered the check. > The stack will be actively misleading because it's highly likely to be totally > unrelated. > > if you want to know the task, just output current->comm :) > > I think it's not only of no value, it's _ACTIVELY_ misleading. So it's > definitely a no to a dump_stack(). > > I am also not in favour of a WARN_ON() for the same reason. > > Really we should be catching these elsewhere. > > If you want to send the patch just outputting thet ask then all good. we can start by adding |current->comm| and |task_pid_nr(current)| to help identify the triggering task. As for possible detection or monitoring mechanisms, we can continue the discussion. > >>> Given that, how should we go about identifying the root cause when such an issue occurs? >>> >>> Is there any existing way to trace it more effectively, or could we introduce a new mechanism >>> to monitor and detect these inconsistencies earlier? >>> >>> Let’s brainstorm possible solutions together. >>> >> -- >> Thanks, >> Xuanye >> -- Thanks, Xuanye [-- Attachment #2: Type: text/html, Size: 5049 bytes --] ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] mm: add stack trace when bad rss-counter state is detected 2025-07-23 9:10 ` Xuanye Liu 2025-07-23 9:14 ` Xuanye Liu @ 2025-07-23 9:17 ` Vlastimil Babka 2025-07-23 9:23 ` Lorenzo Stoakes 2025-07-23 9:25 ` David Hildenbrand 1 sibling, 2 replies; 12+ messages in thread From: Vlastimil Babka @ 2025-07-23 9:17 UTC (permalink / raw) To: Xuanye Liu, David Hildenbrand, Kees Cook Cc: Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot, Andrew Morton, Lorenzo Stoakes, Liam R. Howlett, Mike Rapoport, Suren Baghdasaryan, Michal Hocko, Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman, Valentin Schneider, linux-mm, linux-kernel On 7/23/25 11:10, Xuanye Liu wrote: > > 在 2025/7/23 16:42, David Hildenbrand 写道: >> On 23.07.25 10:05, David Hildenbrand wrote: >>> On 23.07.25 09:45, Xuanye Liu wrote: >>>> >>>> 在 2025/7/23 15:31, Kees Cook 写道: >>>>> On Wed, Jul 23, 2025 at 03:23:49PM +0800, Xuanye Liu wrote: >>>>>> The check_mm() function verifies the correctness of rss counters in >>>>>> struct mm_struct. Currently, it only prints an alert when a bad >>>>>> rss-counter state is detected, but lacks sufficient context for >>>>>> debugging. >>>>>> >>>>>> This patch adds a dump_stack() call to provide a stack trace when >>>>>> the rss-counter state is invalid. This helps developers identify >>>>>> where the corrupted mm_struct is being checked and trace the >>>>>> underlying cause of the inconsistency. >>>>> Why not just convert the pr_alert to a WARN? >>>> Good idea! I'll gather more feedback from others and then update to v2. >>> >>> Makes sense to me. >> >> After discussion this with Lorenzo off-list, isn't the stack completely misleading/useless in that case? >> >> Whatever caused the RSS counter mismatch (e.g., unmapped the wrong pages, missed to unmap pages) quite possibly happened in different context, way way earlier. >> >> Why would you think the stack trace would be of any value when destroying an MM (__mmdrop)? >> >> Having that said, I really hate these "pr_*("BUG: ...") with passion. Probably we'd want to invoke the panic_on_warn machinery, because something unexpected happened. >> > The stack trace dumped here may indeed not reflect the root cause —— > the actual error could have occurred much earlier, for example during a > failed or missing page map/unmap operation. > The current stack (e.g., in __mmdrop() or exit_mmap()) is merely part > of the cleanup phase. > > Given that, how should we go about identifying the root cause when such an issue occurs? > > Is there any existing way to trace it more effectively, or could we introduce a new mechanism > to monitor and detect these inconsistencies earlier? > > Let’s brainstorm possible solutions together. Excellent idea! How about we introduce a function that walks the whole page tables and checks the numbers of individual pte types against the rss counters. And if we invoke it before and after every single pte update, we can pinpoint much sooner the moment it went wrong and the stack that lead to it? ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] mm: add stack trace when bad rss-counter state is detected 2025-07-23 9:17 ` Vlastimil Babka @ 2025-07-23 9:23 ` Lorenzo Stoakes 2025-07-23 9:25 ` David Hildenbrand 1 sibling, 0 replies; 12+ messages in thread From: Lorenzo Stoakes @ 2025-07-23 9:23 UTC (permalink / raw) To: Vlastimil Babka Cc: Xuanye Liu, David Hildenbrand, Kees Cook, Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot, Andrew Morton, Liam R. Howlett, Mike Rapoport, Suren Baghdasaryan, Michal Hocko, Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman, Valentin Schneider, linux-mm, linux-kernel On Wed, Jul 23, 2025 at 11:17:16AM +0200, Vlastimil Babka wrote: > On 7/23/25 11:10, Xuanye Liu wrote: > > > > 在 2025/7/23 16:42, David Hildenbrand 写道: > >> On 23.07.25 10:05, David Hildenbrand wrote: > >>> On 23.07.25 09:45, Xuanye Liu wrote: > >>>> > >>>> 在 2025/7/23 15:31, Kees Cook 写道: > >>>>> On Wed, Jul 23, 2025 at 03:23:49PM +0800, Xuanye Liu wrote: > >>>>>> The check_mm() function verifies the correctness of rss counters in > >>>>>> struct mm_struct. Currently, it only prints an alert when a bad > >>>>>> rss-counter state is detected, but lacks sufficient context for > >>>>>> debugging. > >>>>>> > >>>>>> This patch adds a dump_stack() call to provide a stack trace when > >>>>>> the rss-counter state is invalid. This helps developers identify > >>>>>> where the corrupted mm_struct is being checked and trace the > >>>>>> underlying cause of the inconsistency. > >>>>> Why not just convert the pr_alert to a WARN? > >>>> Good idea! I'll gather more feedback from others and then update to v2. > >>> > >>> Makes sense to me. > >> > >> After discussion this with Lorenzo off-list, isn't the stack completely misleading/useless in that case? > >> > >> Whatever caused the RSS counter mismatch (e.g., unmapped the wrong pages, missed to unmap pages) quite possibly happened in different context, way way earlier. > >> > >> Why would you think the stack trace would be of any value when destroying an MM (__mmdrop)? > >> > >> Having that said, I really hate these "pr_*("BUG: ...") with passion. Probably we'd want to invoke the panic_on_warn machinery, because something unexpected happened. > >> > > The stack trace dumped here may indeed not reflect the root cause —— > > the actual error could have occurred much earlier, for example during a > > failed or missing page map/unmap operation. > > The current stack (e.g., in __mmdrop() or exit_mmap()) is merely part > > of the cleanup phase. > > > > Given that, how should we go about identifying the root cause when such an issue occurs? > > > > Is there any existing way to trace it more effectively, or could we introduce a new mechanism > > to monitor and detect these inconsistencies earlier? > > > > Let’s brainstorm possible solutions together. > > Excellent idea! How about we introduce a function that walks the whole page > tables and checks the numbers of individual pte types against the rss > counters. And if we invoke it before and after every single pte update, we > can pinpoint much sooner the moment it went wrong and the stack that lead to it? :))))))) Haha well, I think this might be one of those reductio ad absurdum arguments :P Yes the point is we can't figure out where this happened in any sane way if the root cause doesn't have sufficient checks. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH] mm: add stack trace when bad rss-counter state is detected 2025-07-23 9:17 ` Vlastimil Babka 2025-07-23 9:23 ` Lorenzo Stoakes @ 2025-07-23 9:25 ` David Hildenbrand 1 sibling, 0 replies; 12+ messages in thread From: David Hildenbrand @ 2025-07-23 9:25 UTC (permalink / raw) To: Vlastimil Babka, Xuanye Liu, Kees Cook Cc: Ingo Molnar, Peter Zijlstra, Juri Lelli, Vincent Guittot, Andrew Morton, Lorenzo Stoakes, Liam R. Howlett, Mike Rapoport, Suren Baghdasaryan, Michal Hocko, Dietmar Eggemann, Steven Rostedt, Ben Segall, Mel Gorman, Valentin Schneider, linux-mm, linux-kernel On 23.07.25 11:17, Vlastimil Babka wrote: > On 7/23/25 11:10, Xuanye Liu wrote: >> >> 在 2025/7/23 16:42, David Hildenbrand 写道: >>> On 23.07.25 10:05, David Hildenbrand wrote: >>>> On 23.07.25 09:45, Xuanye Liu wrote: >>>>> >>>>> 在 2025/7/23 15:31, Kees Cook 写道: >>>>>> On Wed, Jul 23, 2025 at 03:23:49PM +0800, Xuanye Liu wrote: >>>>>>> The check_mm() function verifies the correctness of rss counters in >>>>>>> struct mm_struct. Currently, it only prints an alert when a bad >>>>>>> rss-counter state is detected, but lacks sufficient context for >>>>>>> debugging. >>>>>>> >>>>>>> This patch adds a dump_stack() call to provide a stack trace when >>>>>>> the rss-counter state is invalid. This helps developers identify >>>>>>> where the corrupted mm_struct is being checked and trace the >>>>>>> underlying cause of the inconsistency. >>>>>> Why not just convert the pr_alert to a WARN? >>>>> Good idea! I'll gather more feedback from others and then update to v2. >>>> >>>> Makes sense to me. >>> >>> After discussion this with Lorenzo off-list, isn't the stack completely misleading/useless in that case? >>> >>> Whatever caused the RSS counter mismatch (e.g., unmapped the wrong pages, missed to unmap pages) quite possibly happened in different context, way way earlier. >>> >>> Why would you think the stack trace would be of any value when destroying an MM (__mmdrop)? >>> >>> Having that said, I really hate these "pr_*("BUG: ...") with passion. Probably we'd want to invoke the panic_on_warn machinery, because something unexpected happened. >>> >> The stack trace dumped here may indeed not reflect the root cause —— >> the actual error could have occurred much earlier, for example during a >> failed or missing page map/unmap operation. >> The current stack (e.g., in __mmdrop() or exit_mmap()) is merely part >> of the cleanup phase. >> >> Given that, how should we go about identifying the root cause when such an issue occurs? >> >> Is there any existing way to trace it more effectively, or could we introduce a new mechanism >> to monitor and detect these inconsistencies earlier? >> >> Let’s brainstorm possible solutions together. > > Excellent idea! How about we introduce a function that walks the whole page > tables and checks the numbers of individual pte types against the rss > counters. And if we invoke it before and after every single pte update, we > can pinpoint much sooner the moment it went wrong and the stack that lead to it? > :) On a more serious note, I ran into that usually after hitting a bunch of print_bad_pte() statements: vm_normal_page() would return NULL where it shouldn't, making us not adjust the RSS. In which context would you run into this issue? Usually it really indicates some fundamental page table handling flaw. E.g., page table corruption leading to print_bad_pte() earlier. -- Cheers, David / dhildenb ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2025-07-23 9:32 UTC | newest] Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2025-07-23 7:23 [PATCH] mm: add stack trace when bad rss-counter state is detected Xuanye Liu 2025-07-23 7:31 ` Kees Cook 2025-07-23 7:45 ` Xuanye Liu 2025-07-23 8:05 ` David Hildenbrand 2025-07-23 8:42 ` David Hildenbrand 2025-07-23 9:10 ` Xuanye Liu 2025-07-23 9:14 ` Xuanye Liu 2025-07-23 9:22 ` Lorenzo Stoakes 2025-07-23 9:31 ` Xuanye Liu 2025-07-23 9:17 ` Vlastimil Babka 2025-07-23 9:23 ` Lorenzo Stoakes 2025-07-23 9:25 ` David Hildenbrand
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox