On Tue, Aug 15, 2023 at 12:58:37PM -0400, Sasha Levin wrote:

> 1. Ask (require) organizations that repeatedly go through this mechanism
> to create a test environment that can demonstrate how the embargoed code
> passes different build/validation tests. We should set a minimal bar to
> the demonstrated quality of code that we'll "sneak" behind the backs of
> community members.

This would be great, it's especially frustrating when the issues people
find are readily visible either in build testing or with virtual
environments and therefore even if people want to keep things secret
they should be able to do the testing themselves.  I'm not sure what the
consequences would be for messing up other than a bit of yelling but
perhaps that's enough.

> 2. Create a group of trusted "testers" who can test embargoed code with
> different (ideally "real") workloads and environments. I think that
> we're overly focused on keeping the circle of people in the know small.

> 3. Work with KernelCI/OpenSSF on setting up a (small) environment
> similar to the public one that we could run embargoed code through.

If these environments are documented and based on available code (they
should be) that could be a good way of setting the requirements for
people who want to do everything in house.