Re: hmm_test issues with latest mainline

[Alistair Popple <apopple@nvidia.com>]

Vlastimil Babka <vbabka@suse.cz> writes:

> On 10/13/2022 9:38 PM, Shuah Khan wrote:
>> On 10/13/22 12:00, David Hildenbrand wrote:
>>>>>>> When did that test start failing? Was it still ok for 6.0?
>>>>>
>>>>> Didn't test yet, will try, in case it's my system/config specific thing.
>>>>
>>>> So it's actually all the same with v6.0 for me. The infinite loops, the test
>>>> failures, the misreported SKIPs.
>>>>
>>
>> I am not seeing infinite loops and seeing 25 failures which could
>> be skips.
>>
>>>
>>> Is the kernel compiled with support. I have the feeling that we might simply miss kernel support and it's not handled gracefully ...
>>>
>>
>> Here is my config
>> CONFIG_HMM_MIRROR=y
>> # CONFIG_TEST_HMM is not set
>>
>> Okay here is what is going on - hmm_tests are supposed to be run
>> from test_hmm.sh script. When I run this I see a message that tells
>> me what to do.
>>
>> sudo ./test_hmm.sh
>> ./test_hmm.sh: You must have the following enabled in your kernel:
>> CONFIG_TEST_HMM=m
>>
>> Running ./hmm_tests gives me all the failures. So it appears running
>> hmm_tests executable won't work. This is expected as test_hmm.sh does
>> the right setup before running the test. We have several tests that do
>> that.
>>
>> Vlastimil, can you try this and let me know what you see. I will compile
>> with CONFIG_TEST_HMM=m and let you know what I see on my system.
>
> Right, I didn't mention it, sorry. I did have CONFIG_TEST_HMM=m and was running
> "test_hmm.sh smoke"

FWIW I tend not to use that script on my development machine, mainly
because I either have the module built in or otherwise don't have
modules installed in a place modprobe knows about.

Anyway I am not seeing test failures running hmm-tests directly. However
I do observe both the issue of SKIP in FIXTURE_SETUP() being reported as
a pass in the summary, and the infinite loop on ASSERT failure in
FIXTURE_TEARDOWN.

There does seem to be some framework issues here which are causing this
behaviour. Consider the following representitive snippet:

#include "../kselftest_harness.h"

#include <stdio.h>

FIXTURE(test) {};

FIXTURE_SETUP(test)
{
	SKIP(return, "skip");
}

FIXTURE_TEARDOWN(test)
{
	ASSERT_TRUE(0);
}

TEST_F(test, test)
{
	printf("Running test\n");
}

TEST_HARNESS_MAIN

In this case the test will still be run even though SKIP() was called in
FIXTURE_SETUP. The ASSERT_TRUE() during FIXTURE_TEARDOWN results in the
infinite loop. So it looks to me like calling SKIP from FIXTURE_SETUP
isn't supported, and calling ASSERT_*() in FIXTURE_TEARDOWN is also not
allowed/supported by the kselftest framework.

Unlike hmm-tests though the above snippet reports correct pass/skip
statistics with the teardown assertion removed. This is because there is
also a bug in hmm-tests. Currently we have:

   SKIP(exit(0), "DEVICE_COHERENT not available");

Which should really be:

   SKIP(return, "DEVICE_COHERENT not available");

Of course that results in an infinite loop due to the associated
assertion failure during teardown which is still called despite the SKIP
in setup. Not sure if this is why it was originally coded this way.

 - Alistair

>> thanks,
>> -- Shuah
>>
>>
>>