Re: [MAINTAINERS SUMMIT] Quality standards for embargoed code

[Sasha Levin <sashal@kernel.org>]

On Tue, Aug 15, 2023 at 06:18:16PM +0100, Mark Brown wrote:
>On Tue, Aug 15, 2023 at 12:58:37PM -0400, Sasha Levin wrote:
>
>> 1. Ask (require) organizations that repeatedly go through this mechanism
>> to create a test environment that can demonstrate how the embargoed code
>> passes different build/validation tests. We should set a minimal bar to
>> the demonstrated quality of code that we'll "sneak" behind the backs of
>> community members.
>
>This would be great, it's especially frustrating when the issues people
>find are readily visible either in build testing or with virtual
>environments and therefore even if people want to keep things secret
>they should be able to do the testing themselves.  I'm not sure what the
>consequences would be for messing up other than a bit of yelling but
>perhaps that's enough.

My thinking was that the community could define a set of requirements
that we expect to be tested before we're willing to let code sneak in,
something along the lines of:

1. Built with GCC v1, v2, v3 for platforms x86, arm64, ... using
allmodconfig/allyesconfig/allnoconfig/...

2. Built with Clang v1, v2, v3 for platforms x86, arm64, ... using
allmodconfig/allyesconfig/allnoconfig/...

3. Run through LTP released in the past month.

4. Custom community provided tests, etc...

The consequence of letting something sneak through will then be on us,
and will probably trigger the addition of a test to the list of tests
above.

-- 
Thanks,
Sasha