[Ksummit-discuss] [TOPIC] Services needed from kernel.org

[Ksummit-discuss] [TOPIC] Services needed from kernel.org

[H. Peter Anvin]

I have been discussing with the Linux Foundation about if there are any
new IT services needed or desired for kernel developers (maybe or maybe
not under the kernel.org.)  The kernel.org cleanup and inevitable
changes meant we couldn't service some of the needs that we had in the
past, some of which has moved to privately owned equipment and some of
which has migrated to cloud services or discontinued.

However, the big question is: what IT services do kernel developers
need, and how can we best provide them?  One idea bandied about would be
to provide either managed or unmanaged VMs to kernel developers.  Would
that reduce the workload on the upper tier of kernel developers so that
we can focus more on our time on what actually provides value to our
users and employers, and which we hopefully find more interesting/fun?

	-hpa

Re: [Ksummit-discuss] [TOPIC] Services needed from kernel.org

[josh]

On Tue, May 20, 2014 at 11:48:37AM -0700, H. Peter Anvin wrote:
> I have been discussing with the Linux Foundation about if there are any
> new IT services needed or desired for kernel developers (maybe or maybe
> not under the kernel.org.)  The kernel.org cleanup and inevitable
> changes meant we couldn't service some of the needs that we had in the
> past, some of which has moved to privately owned equipment and some of
> which has migrated to cloud services or discontinued.
> 
> However, the big question is: what IT services do kernel developers
> need, and how can we best provide them?  One idea bandied about would be
> to provide either managed or unmanaged VMs to kernel developers.  Would
> that reduce the workload on the upper tier of kernel developers so that
> we can focus more on our time on what actually provides value to our
> users and employers, and which we hopefully find more interesting/fun?

How feasible is it to support git hooks that want to construct and send
(significant volumes of) email, while retaining the security and
sandboxing currently being applied to git repositories?

- Josh Triplett

Re: [Ksummit-discuss] [TOPIC] Services needed from kernel.org

[James Bottomley]

On Tue, 2014-05-20 at 15:53 -0700, josh@joshtriplett.org wrote:
> On Tue, May 20, 2014 at 11:48:37AM -0700, H. Peter Anvin wrote:
> > I have been discussing with the Linux Foundation about if there are any
> > new IT services needed or desired for kernel developers (maybe or maybe
> > not under the kernel.org.)  The kernel.org cleanup and inevitable
> > changes meant we couldn't service some of the needs that we had in the
> > past, some of which has moved to privately owned equipment and some of
> > which has migrated to cloud services or discontinued.
> > 
> > However, the big question is: what IT services do kernel developers
> > need, and how can we best provide them?  One idea bandied about would be
> > to provide either managed or unmanaged VMs to kernel developers.  Would
> > that reduce the workload on the upper tier of kernel developers so that
> > we can focus more on our time on what actually provides value to our
> > users and employers, and which we hopefully find more interesting/fun?
> 
> How feasible is it to support git hooks that want to construct and send
> (significant volumes of) email, while retaining the security and
> sandboxing currently being applied to git repositories?

Seconded; it's a pain having to run my commit hooks on a different
system, particularly as the email is set up to come from kernel.org

James

Re: [Ksummit-discuss] [TOPIC] Services needed from kernel.org

[Jiri Kosina]

On Wed, 21 May 2014, James Bottomley wrote:

> > How feasible is it to support git hooks that want to construct and 
> > send (significant volumes of) email, while retaining the security and 
> > sandboxing currently being applied to git repositories?
> 
> Seconded; it's a pain having to run my commit hooks on a different
> system, particularly as the email is set up to come from kernel.org

I hate e-mails which contain just '+1' and nothing else, so I'd also add 
that I find this very nice-to-have feature as well, as discussed at [1] 
already :)

[1] http://lists.linuxfoundation.org/pipermail/ksummit-discuss/2014-May/000489.html

Thanks,

-- 
Jiri Kosina
SUSE Labs

Re: [Ksummit-discuss] [TOPIC] Services needed from kernel.org

[Konstantin Ryabitsev]

[-- Attachment #1: Type: text/plain, Size: 1164 bytes --]

On 20/05/14 06:53 PM, josh@joshtriplett.org wrote:
> How feasible is it to support git hooks that want to construct and send
> (significant volumes of) email, while retaining the security and
> sandboxing currently being applied to git repositories?

It's feasible, but I can't make this a free-for-all, for obvious
reasons. :) With gitolite, hooks execute as the same user "git" and
therefore any code running inside a hook has unfettered access to all
git repositories regardless of in-gitolite repository permissions.

What we can do is have a peer-reviewed collection of "blessed" hooks
available to developers. Not being a kernel dev myself, I'm not the one
to put such a collection together, though -- I'm not even sure if such a
cookie-cutter approach would be suitable.

I guess what I'm really trying to say is that I'd rather avoid having to
continuously review code written by people to whom "you write perl like
it's C" is a compliment. ;) If we can get by with a handful of standard
hooks, I'm for it, though.

Best,
-- 
Konstantin Ryabitsev
Senior Systems Administrator
Linux Foundation Collab Projects
Montréal, Québec


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 713 bytes --]

Re: [Ksummit-discuss] [TOPIC] Services needed from kernel.org

[H. Peter Anvin]

On 05/20/2014 05:40 PM, Konstantin Ryabitsev wrote:
> On 20/05/14 06:53 PM, josh@joshtriplett.org wrote:
>> How feasible is it to support git hooks that want to construct
>> and send (significant volumes of) email, while retaining the
>> security and sandboxing currently being applied to git
>> repositories?
> 
> It's feasible, but I can't make this a free-for-all, for obvious 
> reasons. :) With gitolite, hooks execute as the same user "git"
> and therefore any code running inside a hook has unfettered access
> to all git repositories regardless of in-gitolite repository
> permissions.
> 
> What we can do is have a peer-reviewed collection of "blessed"
> hooks available to developers. Not being a kernel dev myself, I'm
> not the one to put such a collection together, though -- I'm not
> even sure if such a cookie-cutter approach would be suitable.
> 
> I guess what I'm really trying to say is that I'd rather avoid
> having to continuously review code written by people to whom "you
> write perl like it's C" is a compliment. ;) If we can get by with a
> handful of standard hooks, I'm for it, though.
> 

Hmm... I resemble that remark.

That being said, there is no fundamental reason that email generator
have to be run on the master copy.  It can be run on an entirely
different box, which doesn't even need any kind of privileged access
to the machinery; this is how the tip-bot runs: it is just a "regular
puller" of the tip tree, using the publicly accessible git port.  This
does add some latency, but has the advantage that it sends out the
emails once the commits are actually visible to general users upstream.

Generalizing the tip-bot is probably not that hard.  I already use
variants of it for multiple projects.  It does have a *lot* of
configuration options, though, some of which are in the form of
scripts.  This pretty much means that in its current form it is
equivalent to a shell account and would have to be sandboxed accordingly.

	-hpa

Re: [Ksummit-discuss] [TOPIC] Services needed from kernel.org

[Andy Lutomirski]

On Wed, May 21, 2014 at 10:17 AM, H. Peter Anvin <hpa@zytor.com> wrote:
> On 05/20/2014 05:40 PM, Konstantin Ryabitsev wrote:
>> On 20/05/14 06:53 PM, josh@joshtriplett.org wrote:
>>> How feasible is it to support git hooks that want to construct
>>> and send (significant volumes of) email, while retaining the
>>> security and sandboxing currently being applied to git
>>> repositories?
>>
>> It's feasible, but I can't make this a free-for-all, for obvious
>> reasons. :) With gitolite, hooks execute as the same user "git"
>> and therefore any code running inside a hook has unfettered access
>> to all git repositories regardless of in-gitolite repository
>> permissions.
>>
>> What we can do is have a peer-reviewed collection of "blessed"
>> hooks available to developers. Not being a kernel dev myself, I'm
>> not the one to put such a collection together, though -- I'm not
>> even sure if such a cookie-cutter approach would be suitable.
>>
>> I guess what I'm really trying to say is that I'd rather avoid
>> having to continuously review code written by people to whom "you
>> write perl like it's C" is a compliment. ;) If we can get by with a
>> handful of standard hooks, I'm for it, though.
>>
>
> Hmm... I resemble that remark.
>
> That being said, there is no fundamental reason that email generator
> have to be run on the master copy.  It can be run on an entirely
> different box, which doesn't even need any kind of privileged access
> to the machinery; this is how the tip-bot runs: it is just a "regular
> puller" of the tip tree, using the publicly accessible git port.  This
> does add some latency, but has the advantage that it sends out the
> emails once the commits are actually visible to general users upstream.
>
> Generalizing the tip-bot is probably not that hard.  I already use
> variants of it for multiple projects.  It does have a *lot* of
> configuration options, though, some of which are in the form of
> scripts.  This pretty much means that in its current form it is
> equivalent to a shell account and would have to be sandboxed accordingly.
>

There *must* be a sensible way to construct a sandbox these days.
Some combination of user namespaces, seccomp, and maybe even a VM
should be quite secure and even fairly easy to use.  Not to mention
that various scripting languages should have viable sandbox modes.

I'm sort of working on a userns-based general-purpose sandbox.  Maybe
I'll have something usable to play with at the kernel summit.

--Andy

Re: [Ksummit-discuss] [TOPIC] Services needed from kernel.org

[Josh Triplett]

On Wed, May 21, 2014 at 10:17:24AM -0700, H. Peter Anvin wrote:
> On 05/20/2014 05:40 PM, Konstantin Ryabitsev wrote:
> > On 20/05/14 06:53 PM, josh@joshtriplett.org wrote:
> >> How feasible is it to support git hooks that want to construct
> >> and send (significant volumes of) email, while retaining the
> >> security and sandboxing currently being applied to git
> >> repositories?
> > 
> > It's feasible, but I can't make this a free-for-all, for obvious 
> > reasons. :) With gitolite, hooks execute as the same user "git"
> > and therefore any code running inside a hook has unfettered access
> > to all git repositories regardless of in-gitolite repository
> > permissions.
> > 
> > What we can do is have a peer-reviewed collection of "blessed"
> > hooks available to developers. Not being a kernel dev myself, I'm
> > not the one to put such a collection together, though -- I'm not
> > even sure if such a cookie-cutter approach would be suitable.
> > 
> > I guess what I'm really trying to say is that I'd rather avoid
> > having to continuously review code written by people to whom "you
> > write perl like it's C" is a compliment. ;) If we can get by with a
> > handful of standard hooks, I'm for it, though.
> > 
> 
> Hmm... I resemble that remark.
> 
> That being said, there is no fundamental reason that email generator
> have to be run on the master copy.  It can be run on an entirely
> different box, which doesn't even need any kind of privileged access
> to the machinery; this is how the tip-bot runs: it is just a "regular
> puller" of the tip tree, using the publicly accessible git port.  This
> does add some latency, but has the advantage that it sends out the
> emails once the commits are actually visible to general users upstream.
> 
> Generalizing the tip-bot is probably not that hard.  I already use
> variants of it for multiple projects.  It does have a *lot* of
> configuration options, though, some of which are in the form of
> scripts.  This pretty much means that in its current form it is
> equivalent to a shell account and would have to be sandboxed accordingly.

That's exactly the kind of approach I had in mind: let's not run this on
git.kernel.org or gitolite.kernel.org, but rather in a separate sandbox.

However, I don't think we need the full generality of tip-bot's shell
configuration; I'd personally be fine with Konstantin's proposal of a
curated set of hook scripts, perhaps enabled/disabled and configured via
a git branch in each repository.

- Josh Triplett

Re: [Ksummit-discuss] [TOPIC] Services needed from kernel.org

[David Howells]

I posted a list of niggles about the cgit interface over the gitweb interface
a while back (see below).  As far as I can tell, none of them got addressed.

David
---
There some things in gitweb that are missing from the cgit interface on
git.kernel.org that would be nice to have:

(1) Could the patch view screen be made to show tabs expanded?  (Both in the
    diff and in the description).

(2) If an additional remote was pushed into a kernel tree, IIRC with gitweb
    its branches used to show up in the tree in pink boxes - see:

	http://git.infradead.org/users/dhowells/linux-headers.git/shortlog/refs/tags/disintegrate-fbdev-20121220

    where you can see:

	linus/master

    in a pink box.  Could this be brought back please?

 (3) Tag entries in the tag list on the front page had a "shortlog" button
     associated with them in gitweb that would show the log starting from that
     tag - but that's no longer so easy to do.

 (4) Author names are no longer clickable to get a list of patches by that
     author.

Re: [Ksummit-discuss] [TOPIC] Services needed from kernel.org

[Konstantin Ryabitsev]

[-- Attachment #1: Type: text/plain, Size: 1288 bytes --]

On 21/05/14 11:18 AM, David Howells wrote:
> I posted a list of niggles about the cgit interface over the gitweb interface
> a while back (see below).  As far as I can tell, none of them got addressed.

Cgit is an upstream project, so best I can do is pass these feature
requests upstream -- which I did.

> (2) If an additional remote was pushed into a kernel tree, IIRC with gitweb
>     its branches used to show up in the tree in pink boxes - see:
> 
> 	http://git.infradead.org/users/dhowells/linux-headers.git/shortlog/refs/tags/disintegrate-fbdev-20121220
> 
>     where you can see:
> 
> 	linus/master
> 
>     in a pink box.  Could this be brought back please?

I'm not sure we can do that, as git.kernel.org is a "git clone --mirror"
of the actual repositories on gitolite.kernel.org, so are a) bare, and
b) wouldn't have any remotes other than origin. Unless I'm
misunderstanding the functionality.

>  (4) Author names are no longer clickable to get a list of patches by that
>      author.

I can add this easily via a filter. Currently, to do the same, you just
need to use the search box and copy-paste the author name.

Regards,
-- 
Konstantin Ryabitsev
Senior Systems Administrator
Linux Foundation Collab Projects
Montréal, Québec


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 713 bytes --]