Re: [MAINTAINERS SUMMIT] Quality standards for embargoed code

[Dave Hansen <dave@sr71.net>]

On 8/15/23 09:58, Sasha Levin wrote:
> I'd like to have a discussion about how the community handles code
> drops to address embargoed security issues: my concern is that we
> sidestap our regular development workflow (post patches, review,
> test, bots, etc...)

I couldn't agree more.  Working on these issues feels like you're
hacking with one arm tied behind your back.  Things are _way_ better
than they used to be, but the closer the folks working behind closed
doors get to the "regular" workflows, the better off everyone is.

> 1. Ask (require) organizations that repeatedly go through this mechanism
> to create a test environment that can demonstrate how the embargoed code
> passes different build/validation tests. We should set a minimal bar to
> the demonstrated quality of code that we'll "sneak" behind the backs of
> community members.

Intel does send things through 0day internally, with a few minor
differences from how public stuff gets tested.  But, I don't think any
information about that internal testing ever makes it into the material
that get merged.  We'll fix that.

> 2. Create a group of trusted "testers" who can test embargoed code with
> different (ideally "real") workloads and environments. I think that
> we're overly focused on keeping the circle of people in the know small.

The docs:

> https://www.kernel.org/doc/html/latest/process/embargoed-hardware-issues.html

_should_ allow the "hardware security team" to add testers today:

> The hardware security team identifies the developers (domain experts)
> who will form the initial response team for a particular issue. The
> initial response team can bring in further developers (domain
> experts) to address the issue in the best technical way.
Do we need to make this more explicit that some of those developers
might be focused on testing?

> 3. Work with KernelCI/OpenSSF on setting up a (small) environment
> similar to the public one that we could run embargoed code through.

That would be really nice.