Re: [MAINTAINERS SUMMIT] Quality standards for embargoed code

[Greg KH <greg@kroah.com>]

On Tue, Aug 15, 2023 at 11:34:37AM -0700, Dave Hansen wrote:
> On 8/15/23 11:19, Sasha Levin wrote:
> > On Tue, Aug 15, 2023 at 10:19:21AM -0700, Dave Hansen wrote:
> >> On 8/15/23 09:58, Sasha Levin wrote:
> ...>>> 1. Ask (require) organizations that repeatedly go through this
> mechanism
> >>> to create a test environment that can demonstrate how the embargoed code
> >>> passes different build/validation tests. We should set a minimal bar to
> >>> the demonstrated quality of code that we'll "sneak" behind the backs of
> >>> community members.
> >>
> >> Intel does send things through 0day internally, with a few minor
> >> differences from how public stuff gets tested.  But, I don't think any
> >> information about that internal testing ever makes it into the material
> >> that get merged.  We'll fix that.
> > 
> > Beyond running tests, it would also be great to standardize on what we
> > need to run, and if Intel wants to start the discussion by openning up
> > it's tests for embargoed code then it'll e a great start!
> 
> I'll go rattle some cages.  It might be boring old 0day, but I'll find out.
> 
> >>> 2. Create a group of trusted "testers" who can test embargoed code with
> >>> different (ideally "real") workloads and environments. I think that
> >>> we're overly focused on keeping the circle of people in the know small.
> >>
> >> The docs:
> >>
> >>> https://www.kernel.org/doc/html/latest/process/embargoed-hardware-issues.html
> >>
> >> _should_ allow the "hardware security team" to add testers today:
> > 
> > It probably does, but the way it's written now you'd need a lawyer to
> > confirm that.
> 
> How about something like the attached patch for that doc?  Does that
> help ensure we leave the lawyers alone? :)

The "trick" here is that lawyers agreed with the original wording,
changing it without also getting their approval might be tough :(

Anyway, the main reason we have NOT added testers (nor allowed
developers to use the test systems from their employer) is that those
test systems are able to be accessed by a huge/unknown number of other
people, none of who should have access to the potential changes under
development.

If that can be solved, with a "private" kernelci/lkft/openssf/whatever
test instance, that would be wonderful.  Ideally it should be the
responsibility of the hardware vendor for which we are fixing their
broken hardware with kernel changes to provide this for us.

I know that Linaro has made some lkft access available to some of us in
the past with "private" test systems, but that was a long time ago and I
don't think I have access to that anymore with their most recent rewrite
of their backend.  Oh, and their systems primarily test ARM cpus, of
which we generally do NOT use the embargoed-hw system because those CPUs
usually don't have these types of problems :)

So if we do get access to private testing systems, that do not leak
information to non-members of the group, be it however, I see no reason
why the current wording of the document would not be applicable as we
are allowed to do so:

	The initial response team can bring in further developers
	(domain experts) to address the issue in the best technical way.

And testing is the best technical way to ensure fixes are correct :)

thanks,

greg k-h