Re: hmm_test issues with latest mainline

[Vlastimil Babka <vbabka@suse.cz>]

On 10/13/22 19:03, Shuah Khan wrote:
> On 10/13/22 10:54, Vlastimil Babka wrote:
>> Hi,
>>
>> I've been trying the hmm_tests as of today's commit:
>>
>> a185a0995518 ("Merge tag 'linux-kselftest-kunit-6.1-rc1-2' ...)
>>
>> and run into several issues that seemed worth reporting.
>>
>> First, it seems the FIXTURE_TEARDOWN(hmm) in
>> tools/testing/selftests/vm/hmm-tests.c
>> using ASSERT_EQ(ret, 0); can run into an infinite loop of reporting the
>> assertion failure. Dunno if it's a kselftests issue or it's a bug to
>> use asserts in teardown. I hacked it up like this locally to proceed:
>>
> 
> kselftest pull requests didn't include any framework changes. I doubt
> that it is framework related.

But is it OK to use e.g. ASSERT_EQ() in FIXTURE_TEARDOWN()?

>> --- a/tools/testing/selftests/vm/hmm-tests.c
>> +++ b/tools/testing/selftests/vm/hmm-tests.c
>> @@ -154,6 +154,11 @@ FIXTURE_TEARDOWN(hmm)
>>   {
>>       int ret = close(self->fd);
>>   +    if (ret != 0) {
>> +        fprintf(stderr, "close returned (%d) fd is (%d)\n", ret,self->fd);
>> +        exit(1);
>> +    }
>> +
>>       ASSERT_EQ(ret, 0);
>>       self->fd = -1;
>>   }
>>
>> Next, there are some tests that fail (and thus also trigger the issue above)
>>
>> #  RUN           hmm.hmm_device_private.exclusive ...
>> # hmm-tests.c:1702:exclusive:Expected ret (-16) == 0 (0)
>> close returned (-1) fd is (3)
>> # exclusive: Test failed at step #1
>> #          FAIL  hmm.hmm_device_private.exclusive
>> not ok 20 hmm.hmm_device_private.exclusive
>> #  RUN           hmm.hmm_device_private.exclusive_mprotect ...
>> # hmm-tests.c:1756:exclusive_mprotect:Expected ret (-16) == 0 (0)
>> close returned (-1) fd is (3)
>> # exclusive_mprotect: Test failed at step #1
>> #          FAIL  hmm.hmm_device_private.exclusive_mprotect
>> not ok 21 hmm.hmm_device_private.exclusive_mprotect
>> #  RUN           hmm.hmm_device_private.exclusive_cow ...
>> # hmm-tests.c:1809:exclusive_cow:Expected ret (-16) == 0 (0)
>> close returned (-1) fd is (3)
>> # exclusive_cow: Test failed at step #1
>> #          FAIL  hmm.hmm_device_private.exclusive_cow
>> not ok 22 hmm.hmm_device_private.exclusive_cow
>>
>> I'll try to check more closely but maybe if you can reproduce it too, you'll
>> have more idea what's going on.
>>
> 
> Sounds good.
> 
>> The next thing is more of a question/documentation suggestion. Tons of tests
>> fail like this:
>>
>> ok 24 hmm.hmm_device_private.hmm_cow_in_device
>> #  RUN           hmm.hmm_device_coherent.open_close ...
>> could not open hmm dmirror driver (/dev/hmm_dmirror2)
>> #      SKIP      DEVICE_COHERENT not available
>> #            OK  hmm.hmm_device_coherent.open_close
>>
>> I assume this is because I run "test_hmm.sh smoke" without the SPM
>> parameters.
>> The help message doesn't say much about what to specify there for
>> <spm_addr_dev0> <spm_addr_dev1>. Do these tests need a particular hardware?
>> (unlike the rest?) Maybe it could be clarified.
>>
>> Last thing, I noticed all these DEVICE_COHERENT tests ultimately count as OK,
>> not SKIPPED, which would probably be more appropriate?
>>
> 
> Anytime a test can't be run due to missing config, the result should be a SKIP.
> If that is not the case let's fix these cases.
> 
>> # FAILED: 51 / 54 tests passed.
>> # Totals: pass:50 fail:3 xfail:0 xpass:0 skip:1 error:0
>>
>> (the skip:1 is due to test 9 "#      SKIP      Huge page could not be
>> allocated"
>> which is probably a misconfiguration on my part so I don't report that as
>> an issue)
>>
> 
> Skip is the right result in this case if it is indeed the result of misconfig.

Right. My point is that there were more than 20 more reporting "#      SKIP
    DEVICE_COHERENT not available" that were not counted as skip: but pass:

> thanks,
> -- Shuah