Re: [PATCH 2/3] MAINTAINERS: Require kvm-xfstests smoke for ext4

["Darrick J. Wong" <djwong@kernel.org>]

On Wed, Nov 22, 2023 at 04:44:58PM +0200, Nikolai Kondrashov wrote:
> On 11/20/23 00:54, Theodore Ts'o wrote:
> > So as for *me*, I'm going to point people at:
> > 
> > https://github.com/tytso/xfstests-bld/blob/master/Documentation/kvm-quickstart.md
> 
> ...
> 
> > (And note that I keep the xfstests-bld repo's on kernel.org and
> > github.com both uptodate, and I prefer using the using the github.com
> > URL because it's easier for the new developer to read and understand
> > it.)
> 
> I already queued a switch to the kernel.org URL, which Darrick has suggested.
> I'll drop it now, but you guys would have to figure it out between yourselves,
> which one you want :D
> 
> Personally, I agree that the one on GitHub is more reader-friendly, FWIW.

For xfstests-bld links, I'm ok with whichever domain Ted wants.

> > And similarly, just because the V: line might say, "kvm-xfstests
> > smoke", someone could certainly use kdevops if they wanted to.  So
> > perhaps we need to be a bit clearer about what we expect the V: line
> > to mean?
> 
> I tried to handle some of that with the "subsets", so that you can run a wider
> test suite and still pass the Tested-with: check. I think this has to be
> balanced between allowing all the possible ways to run the tests and a
> reasonable way to certify the commit was tested automatically.
> 
> E.g. name the test "xfstests", and list all the ways it can be executed, thus
> communicating that it should still say "Tested-with: xfstests" regardless of
> the way. And if there is a smaller required subset, name it just "xfstests
> smoke" and list all the ways it can be run, including the simplest
> "kvm-xfstests smoke", but accept just "Tested-with: xfstests-smoke".
> 
> I'm likely getting things wrong, but I hope you get what I'm trying to say.

Not entirely -- for drive-by contributions and obvious bugfixes, a quick
"V: xfstests-bld: kvm-xfstests smoke" / "V: fstests: ./check -g smoke"
run is probably sufficient.

(Insofar as n00bs running ./check isn't sufficient, but that's something
that fstests needs to solve...)

For nontrivial code tidying, the author really ought to run the whole
test suite.  It's still an open question as to whether xfs tidying
should run the full fuzz suite too, since that increases the runtime
from overnightish to a week.

For /new features/, the developer(s) ought to come up with a testing
plan and run that by the community.  Eventually those will merge into
fstests or ktest or wherever.

--D

> > Along these lines, we should perhaps be a bit more thoughtful about
> > the intended audience for Documentation/process/tests.rst.  I
> > personally wouldn't try ask first-time kernel developers to look at
> > the xfstests group files, because that's just way too complicated for
> > them.
> > 
> > And I had *assumed* that Documentation/process/tests.rst was not
> > primarily intended for sophistiocated file system developers who
> > wouldn't be afraid to start looking at the many ways that xfstests
> > could be configured.  But we should perhaps be a bit more explicit
> > about who the intended audience would be for a certain set up
> > Documentation files, since that will make it easier for us to come to
> > consensus about how that particular piece of documentation would be
> > worded.
> > 
> > As E.B. White (author of the book "The Elements of Style" was reputed
> > to have once said, "Always write with deep sympathy for the reader".
> > Which means we need to understand who the reader is, and to try to
> > walk in their shoes, first.
> 
> Amen to that! Apart from the newbies and just people working on other
> subsystems, we should also remember to be kinder to ourselves and keep our own
> tools easier to use. So perhaps just say "newbies should be able to follow
> tests.rst", and enjoy it :D
> 
> Ultimately, I think the (admittedly elusive) target should be the ability to
> just plop a command line into every V: entry, running something from the tree
> itself. Meanwhile, we would need the stepping stone of tests.rst, or something
> like that, to walk people through whatever setup is required.
> 
> I'll see how we can accommodate the commands in the V: directly, though.
> 
> Nick
>