Re: [RFC PATCH 7/7] tools: add skeleton code for userland testing of VMA logic

["Liam R. Howlett" <Liam.Howlett@oracle.com>]

* Kees Cook <kees@kernel.org> [240627 12:58]:
> On Thu, Jun 27, 2024 at 11:39:32AM +0100, Lorenzo Stoakes wrote:
> > Establish a new userland VMA unit testing implementation under
> > tools/testing which utilises existing logic providing maple tree support in
> > userland utilising the now-shared code previously exclusive to radix tree
> > testing.
> > 
> > This provides fundamental VMA operations whose API is defined in mm/vma.h,
> > while stubbing out superfluous functionality.
> > 
> > This exists as a proof-of-concept, with the test implementation functional
> > and sufficient to allow userland compilation of vma.c, but containing only
> > cursory tests to demonstrate basic functionality.
> 
> Interesting! Why do you want to have this in userspace instead of just
> wiring up what you have here to KUnit so testing can be performed by
> existing CI systems that are running all the KUnit tests?

The primary reason we did the maple tree testing in userspace was for
speed of testing.  We don't need to build the kernel, but a subset of
APIs.  Debugging problems is also much quicker since we can instrument
and rebuild, iterate down faster.  Tracing every call to the maple tree
on boot alone is massive.

It is also difficult to verify the vma correctness without exposing APIs
we don't want exported (or, I guess, parse proc files..).  On my side, I
have a module for testing the overall interface while I have more tests
on the userspace side that poke and prod on internal states, and
userspace rcu testing is possible.  I expect the same issues on the vma
side.

Adding tests can also be made very efficient with tracepoints dumping
something to add to an array, for example.

Finally, you have ultimate control on what other functions return (or
do) - so you can fail allocations to test error paths, for example.  Or
set the external function to fail after N allocations.  This comes in
handy when a syzbot reports a failed allocation at line X caused a
crash.

This has worked out phenomenally on the maple tree side.  I've been able
to record boot failures and import them, syzbot tests, and fuzzer tests.
The result is a huge list of tests that allowed me to rewrite my node
replacement algorithm and have it just work, once it passed the
collected tests.

I haven't used kunit as much as I have userspace testing, so I cannot
say if all of these points are not possible, but I didn't see a way to
test races like I do with rcu in userspace.

Thanks,
Liam