Re: [PATCH v4 08/15] kunit: tool: Don't overwrite test status based on subtest counts

["Thomas Weißschuh" <thomas.weissschuh@linutronix.de>]

Hi Rae,

On Thu, Jul 03, 2025 at 05:30:02PM +0200, Thomas Weißschuh wrote:
> On Tue, Jul 01, 2025 at 05:11:59PM -0400, Rae Moar wrote:

<snip>

> > > Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
> > > Reviewed-by: David Gow <davidgow@google.com>
> > > ---
> > >  tools/testing/kunit/kunit_parser.py                                  | 5 +++++
> > >  tools/testing/kunit/kunit_tool_test.py                               | 3 ++-
> > >  tools/testing/kunit/test_data/test_is_test_passed-failure-nested.log | 3 +++
> > >  3 files changed, 10 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py
> > > index c176487356e6c94882046b19ea696d750905b8d5..2478beb28fc3db825855ad46200340e884da7df1 100644
> > > --- a/tools/testing/kunit/kunit_parser.py
> > > +++ b/tools/testing/kunit/kunit_parser.py
> > > @@ -686,6 +686,11 @@ def bubble_up_test_results(test: Test) -> None:
> > >                 counts.add_status(status)
> > >         elif test.counts.get_status() == TestStatus.TEST_CRASHED:
> > >                 test.status = TestStatus.TEST_CRASHED
> > > +       if not test.ok_status():
> > > +               for t in subtests:
> > > +                       if not t.ok_status():
> > > +                               counts.add_status(t.status)
> > > +                               break
> > 
> > Here instead I recommend checking if not test.ok_status() and
> > test.counts.get_status() == TestStatus.SUCCESS and if so
> > counts.add_status(status)
> 
> Thanks for the recommendation. I tried this and it works well for this specific
> testcase, but unfortunately all kinds of othes tests are now broken.
> I'll look into it some more, but any hints are highly appreciated.
> It has been a while since I looked at the code.

The following variant passes all tests. What do you think?

diff --git a/tools/testing/kunit/kunit_parser.py b/tools/testing/kunit/kunit_parser.py
index 333cd3a4a56b..5338489dcbe4 100644
--- a/tools/testing/kunit/kunit_parser.py
+++ b/tools/testing/kunit/kunit_parser.py
@@ -689,6 +689,9 @@ def bubble_up_test_results(test: Test) -> None:
        elif test.counts.get_status() == TestStatus.TEST_CRASHED:
                test.status = TestStatus.TEST_CRASHED
 
+       if status == TestStatus.FAILURE and test.counts.get_status() == TestStatus.SUCCESS:
+               counts.add_status(status)
+