Re: [RFC PATCH v2 06/10] MAINTAINERS: Support referencing test docs in V:

[David Gow <davidgow@google.com>]

[-- Attachment #1: Type: text/plain, Size: 2675 bytes --]

On Wed, 6 Dec 2023 at 02:45, Nikolai Kondrashov
<Nikolai.Kondrashov@redhat.com> wrote:
>
> Support referencing test suite documentation in the V: entries of
> MAINTAINERS file. Use the '*<name>' syntax (like C pointer dereference),
> where '<name>' is a second-level heading in the new
> Documentation/process/tests.rst file, with the suite's description.
> This syntax allows distinguishing the references from test commands.
>
> Add a boiler-plate Documentation/process/tests.rst file, describing a
> way to add structured info to the test suites in the form of field
> lists. Apart from a "summary" and "command" fields, they can also
> contain a "superset" field specifying the superset of the test suite,
> helping reuse documentation and express both wider and narrower test
> sets.
>
> Make scripts/checkpatch.pl load the tests from the file, along with the
> structured data, validate the references in MAINTAINERS, dereference
> them, and output the test suite information in the CHECK messages
> whenever the corresponding subsystems are changed. But only if there was
> no corresponding Tested-with: tag in the commit message, certifying it
> was executed successfully already.
>
> This is supposed to help propose executing test suites which cannot be
> executed immediately, and need extra setup, as well as provide a place
> for extra documentation and information on directly-available suites.
>
> Signed-off-by: Nikolai Kondrashov <Nikolai.Kondrashov@redhat.com>
> ---

I like the idea here, but wonder whether it makes sense to put all of
these tests into a single 'tests.rst' file. There's already lots of
existing documentation scattered around the tree, and while keeping
all of the testing information in one place does have advantages, I
think there's a lot to be said for keeping subsystem-specific test
docs alongside the rest of the documentation for the subsystem itself.
And it'd be less work, as the docs are already there.

So, could we just make this a path under Documentation/ (possibly with
an #anchor if we need to reference just one part of a file)?

e.g., something like these, all of which are existing docs:
V: *Documentation/dev-tools/kasan.rst#Tests
or
V: *Dcoumentation/RCU/torture.rst
or
V: *Documentation/gpu/automated_testing.rst
or
V: *Documentation/process/maintainer-kvm-x86.rst#Testing

(We could even get rid of the '*' and just use 'Documentation/' as a
prefix, or the executable bit on the file, or similar to distinguish
these from scripts.)

If we wanted to be very brave, we could extend this further to
arbitrary webpages, like:
V: https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git/tree/README

Thoughts?

-- David

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4003 bytes --]