[Ksummit-discuss] Self nomination - Sasha Levin

[Ksummit-discuss] Self nomination - Sasha Levin

[Levin, Alexander]

Hi all,

Appologies for the late mail - betweem workplace change (and ML mails bouncing off the old @oracle.com mail), and some personal stuff, I've missed the fact that planning and discussions have already started!

I wanted to discuss a variety of topics regarding stable kernels, testing, and testing stable kernels:

 - My new employer is a heavy user of "cutting edge" stable kernels, and as such I'm looking into ways to improve the process. Making people complain and express their unhappiness with concrete workflows is a good way for me to make improvements on my end.

 - Automating the while -stable process further: it's too dependant on humans getting things right which is a recipe for disaster. I have a few proposals about how to reduce the human aspect of it further to improve the quality of the resulting trees (expanding stable-tools even further!).

 - We need to figure out ways to integrate tests better, beyond the usual 3 day review window we have for "testing" where no *actual* user tests anything. In the example of Verizon, it simply doesn't fit in the workflow and we'd need to make a good amount of modifications to make it happen, so I bet it's the same issue for other users of the tree.

 - Reviewing the stable rules and adjusting them to reality. There's a mismatch between what the rules say and what actually happens.

 - A few smaller ideas I want to bounce off maintainers:

   - Adding stable@ tags to commits is "complicated", it requires editing the commit message which is an extra step that people are too lazy to take, how can we improve that?

   - Making checkpatch check for (some) of the stable kernel rules (and possibly recommend adding the stable@ tag in certain cases?).
     - Depends on: making checkpatch sane again.

   - Improving tagging for stable. The "version tag" option is broken and the "Fixes:" tag is always preferable, how do we get people to use that more often? (script it somehow? scripts/find-version-it-fixes ?).

-- 

Thanks,
Sasha

Re: [Ksummit-discuss] Self nomination - Sasha Levin

[Greg KH]

On Fri, Aug 26, 2016 at 12:46:51AM -0400, Levin, Alexander wrote:
>    - Making checkpatch check for (some) of the stable kernel rules
>    (and possibly recommend adding the stable@ tag in certain cases?).
>      - Depends on: making checkpatch sane again.

This sounds interesting.  What do you mean by "sane"?  "sane" as in
rewriting it so that someone who isn't a perl master can modify it?  Or
"sane" as in "don't be so pedantic about things that don't matter"?

>    - Improving tagging for stable. The "version tag" option is broken
>    and the "Fixes:" tag is always preferable, how do we get people to
>    use that more often? (script it somehow?
>    scripts/find-version-it-fixes ?).

Oh a script like that would be nice, but how would that work in reality?

thanks,

greg k-h

Re: [Ksummit-discuss] Self nomination - Sasha Levin

[James Hogan]

[-- Attachment #1: Type: text/plain, Size: 1983 bytes --]

On Fri, Aug 26, 2016 at 01:26:35PM +0200, Greg KH wrote:
> On Fri, Aug 26, 2016 at 12:46:51AM -0400, Levin, Alexander wrote:
> >    - Improving tagging for stable. The "version tag" option is broken
> >    and the "Fixes:" tag is always preferable, how do we get people to
> >    use that more often? (script it somehow?
> >    scripts/find-version-it-fixes ?).
> 
> Oh a script like that would be nice, but how would that work in reality?

Not all Fixes: tags are suitable for stable though. I've been caught out
by patches being applied to stable (4.2 maybe) due to a Fixes tag,
without prerequisite patches being applied.

FWIW I have a git alias that blames a git diff or commit (I think based
on an old version of [1] with some fixes). That doesn't necessarily tell
you where a problem was introduced, but it sure can help if the problem
was introduced nearby the fix (and for fixup commits before a rebase).

Food for thought if nothing else.

E.g. 

$ git blame-show v4.8-rc2
commit 694d0d0bb2030d2e36df73e2d23d5770511dbc8d
Author:     Linus Torvalds <torvalds@linux-foundation.org>
AuthorDate: Sun Aug 14 19:11:36 2016 -0700
Commit:     Linus Torvalds <torvalds@linux-foundation.org>
CommitDate: Sun Aug 14 19:11:36 2016 -0700

    Linux 4.8-rc2

diff --git a/Makefile b/Makefile
index 8c504f324154..5c18baad7218 100644
--- a/Makefile
+++ b/Makefile
@@ -1,7 +1,7 @@
Blaming lines:   0% (7/1684), done.
Blaming lines:   0% (7/1684), done.
  c517d838eb7d0 (Linus Torvalds   VERSION = 4
  29b4817d4018d (Linus Torvalds   PATCHLEVEL = 8
  55922c9d1b84b (Linus Torvalds   SUBLEVEL = 0
- 29b4817d4018d (Linus Torvalds  -EXTRAVERSION = -rc1
+ 694d0d0bb2030 (Linus Torvalds  +EXTRAVERSION = -rc2
  1a695a905c185 (Linus Torvalds   NAME = Psychotic Stoned Sheep
  ^1da177e4c3f4 (Linus Torvalds   
  ^1da177e4c3f4 (Linus Torvalds   # *DOCUMENTATION*

Cheers
James

[1] https://github.com/dmnd/git-diff-blame/blob/master/git-diff-blame

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [Ksummit-discuss] Self nomination - Sasha Levin

[James Hogan]

[-- Attachment #1: Type: text/plain, Size: 1290 bytes --]

On Fri, Aug 26, 2016 at 12:42:05PM +0100, James Hogan wrote:
> On Fri, Aug 26, 2016 at 01:26:35PM +0200, Greg KH wrote:
> > On Fri, Aug 26, 2016 at 12:46:51AM -0400, Levin, Alexander wrote:
> > >    - Improving tagging for stable. The "version tag" option is broken
> > >    and the "Fixes:" tag is always preferable, how do we get people to
> > >    use that more often? (script it somehow?
> > >    scripts/find-version-it-fixes ?).
> > 
> > Oh a script like that would be nice, but how would that work in reality?
> 
> Not all Fixes: tags are suitable for stable though. I've been caught out
> by patches being applied to stable (4.2 maybe) due to a Fixes tag,
> without prerequisite patches being applied.

I also find the following alias useful to find the version number a
commit is first merged in:

vc = "!vc() { for i in `git tag --contains \"$@\" | grep '^v'`; do echo \"$(git log -1 --pretty='%ct' $i) $i\"; done | sort -n | head -n1 | sed 's/^[0-9]* //g'; }; vc"

Its a bit slow and hacky and there's probably better ways, but it picks
the tag a bit more reliably than "git describe --contains". Maybe that
could be wrapped in a script that generates a Stable tag automatically
from a Fixes tag for when the patch is suitable for stable.

Cheers
James

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [Ksummit-discuss] Self nomination - Sasha Levin

[Jani Nikula]

On Fri, 26 Aug 2016, James Hogan <james.hogan@imgtec.com> wrote:
> [ Unknown signature status ]
> On Fri, Aug 26, 2016 at 12:42:05PM +0100, James Hogan wrote:
>> On Fri, Aug 26, 2016 at 01:26:35PM +0200, Greg KH wrote:
>> > On Fri, Aug 26, 2016 at 12:46:51AM -0400, Levin, Alexander wrote:
>> > >    - Improving tagging for stable. The "version tag" option is broken
>> > >    and the "Fixes:" tag is always preferable, how do we get people to
>> > >    use that more often? (script it somehow?
>> > >    scripts/find-version-it-fixes ?).
>> > 
>> > Oh a script like that would be nice, but how would that work in reality?
>> 
>> Not all Fixes: tags are suitable for stable though. I've been caught out
>> by patches being applied to stable (4.2 maybe) due to a Fixes tag,
>> without prerequisite patches being applied.
>
> I also find the following alias useful to find the version number a
> commit is first merged in:
>
> vc = "!vc() { for i in `git tag --contains \"$@\" | grep '^v'`; do echo \"$(git log -1 --pretty='%ct' $i) $i\"; done | sort -n | head -n1 | sed 's/^[0-9]* //g'; }; vc"
>
> Its a bit slow and hacky and there's probably better ways, but it picks
> the tag a bit more reliably than "git describe --contains". Maybe that
> could be wrapped in a script that generates a Stable tag automatically
> from a Fixes tag for when the patch is suitable for stable.

If you can trust your tags to contain versions like in the kernel, I
think you can get away with just:

	git tag --contains <committish> | grep ^v | sort -V | head -n 1

BR,
Jani.


-- 
Jani Nikula, Intel Open Source Technology Center

Re: [Ksummit-discuss] Self nomination - Sasha Levin

[James Hogan]

[-- Attachment #1: Type: text/plain, Size: 2345 bytes --]

On Fri, Aug 26, 2016 at 03:27:50PM +0300, Jani Nikula wrote:
> On Fri, 26 Aug 2016, James Hogan <james.hogan@imgtec.com> wrote:
> > [ Unknown signature status ]
> > On Fri, Aug 26, 2016 at 12:42:05PM +0100, James Hogan wrote:
> >> On Fri, Aug 26, 2016 at 01:26:35PM +0200, Greg KH wrote:
> >> > On Fri, Aug 26, 2016 at 12:46:51AM -0400, Levin, Alexander wrote:
> >> > >    - Improving tagging for stable. The "version tag" option is broken
> >> > >    and the "Fixes:" tag is always preferable, how do we get people to
> >> > >    use that more often? (script it somehow?
> >> > >    scripts/find-version-it-fixes ?).
> >> > 
> >> > Oh a script like that would be nice, but how would that work in reality?
> >> 
> >> Not all Fixes: tags are suitable for stable though. I've been caught out
> >> by patches being applied to stable (4.2 maybe) due to a Fixes tag,
> >> without prerequisite patches being applied.
> >
> > I also find the following alias useful to find the version number a
> > commit is first merged in:
> >
> > vc = "!vc() { for i in `git tag --contains \"$@\" | grep '^v'`; do echo \"$(git log -1 --pretty='%ct' $i) $i\"; done | sort -n | head -n1 | sed 's/^[0-9]* //g'; }; vc"
> >
> > Its a bit slow and hacky and there's probably better ways, but it picks
> > the tag a bit more reliably than "git describe --contains". Maybe that
> > could be wrapped in a script that generates a Stable tag automatically
> > from a Fixes tag for when the patch is suitable for stable.
> 
> If you can trust your tags to contain versions like in the kernel, I
> think you can get away with just:
> 
> 	git tag --contains <committish> | grep ^v | sort -V | head -n 1

Yes. I think the reason I avoided that is because it wouldn't tell me
the rc release a commit was merged in (which is sometimes of interest),
hence sorting by commit timestamp instead, but for the purposes of a
stable tag that is indeed irrelevant:

$ git tag --contains v4.7-rc2 | grep ^v | sort -V | head -n 1                                                                                  
v4.7
$ git tag --contains v4.8-rc2 | grep ^v | sort -V | head -n 1
v4.8-rc2

Looking again at the man pages though, the following is even nicer:

git tag --contains <committish> --sort=taggerdate | grep '^v' | head -n 1

Cheers
James

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [Ksummit-discuss] Self nomination - Sasha Levin

[Greg KH]

On Fri, Aug 26, 2016 at 12:42:05PM +0100, James Hogan wrote:
> On Fri, Aug 26, 2016 at 01:26:35PM +0200, Greg KH wrote:
> > On Fri, Aug 26, 2016 at 12:46:51AM -0400, Levin, Alexander wrote:
> > >    - Improving tagging for stable. The "version tag" option is broken
> > >    and the "Fixes:" tag is always preferable, how do we get people to
> > >    use that more often? (script it somehow?
> > >    scripts/find-version-it-fixes ?).
> > 
> > Oh a script like that would be nice, but how would that work in reality?
> 
> Not all Fixes: tags are suitable for stable though. I've been caught out
> by patches being applied to stable (4.2 maybe) due to a Fixes tag,
> without prerequisite patches being applied.

Yeah, it is true, but it gives me a hint as to where I should stop at,
or where I should look around at.  If I see a "3.14" mark, and the patch
doesn't apply at all there, then I'll push back on the developer of the
patch to see if they can provide a version for that kernel.  If I don't
have that mark, and it doesn't apply to 3.14, I'll just drop it on the
floor as "obviously" it doesn't apply there.

> FWIW I have a git alias that blames a git diff or commit (I think based
> on an old version of [1] with some fixes). That doesn't necessarily tell
> you where a problem was introduced, but it sure can help if the problem
> was introduced nearby the fix (and for fixup commits before a rebase).
> 
> Food for thought if nothing else.
> 
> E.g. 
> 
> $ git blame-show v4.8-rc2
> commit 694d0d0bb2030d2e36df73e2d23d5770511dbc8d
> Author:     Linus Torvalds <torvalds@linux-foundation.org>
> AuthorDate: Sun Aug 14 19:11:36 2016 -0700
> Commit:     Linus Torvalds <torvalds@linux-foundation.org>
> CommitDate: Sun Aug 14 19:11:36 2016 -0700
> 
>     Linux 4.8-rc2
> 
> diff --git a/Makefile b/Makefile
> index 8c504f324154..5c18baad7218 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -1,7 +1,7 @@
> Blaming lines:   0% (7/1684), done.
> Blaming lines:   0% (7/1684), done.
>   c517d838eb7d0 (Linus Torvalds   VERSION = 4
>   29b4817d4018d (Linus Torvalds   PATCHLEVEL = 8
>   55922c9d1b84b (Linus Torvalds   SUBLEVEL = 0
> - 29b4817d4018d (Linus Torvalds  -EXTRAVERSION = -rc1
> + 694d0d0bb2030 (Linus Torvalds  +EXTRAVERSION = -rc2
>   1a695a905c185 (Linus Torvalds   NAME = Psychotic Stoned Sheep
>   ^1da177e4c3f4 (Linus Torvalds   
>   ^1da177e4c3f4 (Linus Torvalds   # *DOCUMENTATION*
> 
> Cheers
> James
> 
> [1] https://github.com/dmnd/git-diff-blame/blob/master/git-diff-blame

Oooh, nice!  Care to push this "upstream" to get it merged into git for
everyone to use?  It looks helpful to me.

thanks,

greg k-h

Re: [Ksummit-discuss] Self nomination - Sasha Levin

[James Hogan]

[-- Attachment #1: Type: text/plain, Size: 2031 bytes --]

On Fri, Aug 26, 2016 at 01:56:53PM +0200, Greg KH wrote:
> On Fri, Aug 26, 2016 at 12:42:05PM +0100, James Hogan wrote:
> > FWIW I have a git alias that blames a git diff or commit (I think based
> > on an old version of [1] with some fixes). That doesn't necessarily tell
> > you where a problem was introduced, but it sure can help if the problem
> > was introduced nearby the fix (and for fixup commits before a rebase).
> > 
> > Food for thought if nothing else.
> > 
> > E.g. 
> > 
> > $ git blame-show v4.8-rc2
> > commit 694d0d0bb2030d2e36df73e2d23d5770511dbc8d
> > Author:     Linus Torvalds <torvalds@linux-foundation.org>
> > AuthorDate: Sun Aug 14 19:11:36 2016 -0700
> > Commit:     Linus Torvalds <torvalds@linux-foundation.org>
> > CommitDate: Sun Aug 14 19:11:36 2016 -0700
> > 
> >     Linux 4.8-rc2
> > 
> > diff --git a/Makefile b/Makefile
> > index 8c504f324154..5c18baad7218 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -1,7 +1,7 @@
> > Blaming lines:   0% (7/1684), done.
> > Blaming lines:   0% (7/1684), done.
> >   c517d838eb7d0 (Linus Torvalds   VERSION = 4
> >   29b4817d4018d (Linus Torvalds   PATCHLEVEL = 8
> >   55922c9d1b84b (Linus Torvalds   SUBLEVEL = 0
> > - 29b4817d4018d (Linus Torvalds  -EXTRAVERSION = -rc1
> > + 694d0d0bb2030 (Linus Torvalds  +EXTRAVERSION = -rc2
> >   1a695a905c185 (Linus Torvalds   NAME = Psychotic Stoned Sheep
> >   ^1da177e4c3f4 (Linus Torvalds   
> >   ^1da177e4c3f4 (Linus Torvalds   # *DOCUMENTATION*
> > 
> > Cheers
> > James
> > 
> > [1] https://github.com/dmnd/git-diff-blame/blob/master/git-diff-blame
> 
> Oooh, nice!  Care to push this "upstream" to get it merged into git for
> everyone to use?  It looks helpful to me.

I wasn't planning to, mainly because I'm not the author and presumed
it'd need rewriting to work properly/efficiently with git rather than
just being a wrapper script.

I'd love to see that functionality in git though, maybe as a --blame
argument to git diff/show

Cheers
James

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [Ksummit-discuss] Self nomination - Sasha Levin

[Levin, Alexander]

On Fri, Aug 26, 2016 at 07:56:53AM -0400, Greg KH wrote:
> On Fri, Aug 26, 2016 at 12:42:05PM +0100, James Hogan wrote:
> > On Fri, Aug 26, 2016 at 01:26:35PM +0200, Greg KH wrote:
> > > On Fri, Aug 26, 2016 at 12:46:51AM -0400, Levin, Alexander wrote:
> > > >    - Improving tagging for stable. The "version tag" option is broken
> > > >    and the "Fixes:" tag is always preferable, how do we get people to
> > > >    use that more often? (script it somehow?
> > > >    scripts/find-version-it-fixes ?).
> > > 
> > > Oh a script like that would be nice, but how would that work in reality?
> > 
> > Not all Fixes: tags are suitable for stable though. I've been caught out
> > by patches being applied to stable (4.2 maybe) due to a Fixes tag,
> > without prerequisite patches being applied.
> 
> Yeah, it is true, but it gives me a hint as to where I should stop at,
> or where I should look around at.  If I see a "3.14" mark, and the patch
> doesn't apply at all there, then I'll push back on the developer of the
> patch to see if they can provide a version for that kernel.  If I don't
> have that mark, and it doesn't apply to 3.14, I'll just drop it on the
> floor as "obviously" it doesn't apply there.

Yup, I fully agree that "Fixes:" doesn't guarantee that the commit is stable
material, but in reality neither does the stable@ tag. It's just another
marker for us.

What I was originally talking about is "cc: stable@vger.kernel.org # vX.Y+" vs
"Fixes: hash ("subject"): It's obvious that for and commit that fixes something
there's another commit that broke that thing, but it's also possible that that
commit that broke something fixed something else and made it's way into stable.

Therefore, by using version tags instead of "Fixes:" we have a very good chance
of missing the fact that a certain commit might need to be applied to older
stable trees as well.

-- 

Thanks,
Sasha

Re: [Ksummit-discuss] Self nomination - Sasha Levin

[Julia Lawall]

Le 26.08.2016 13:26, Greg KH a écrit :
> On Fri, Aug 26, 2016 at 12:46:51AM -0400, Levin, Alexander wrote:
>>    - Making checkpatch check for (some) of the stable kernel rules
>>    (and possibly recommend adding the stable@ tag in certain cases?).
>>      - Depends on: making checkpatch sane again.
> 
> This sounds interesting.  What do you mean by "sane"?  "sane" as in
> rewriting it so that someone who isn't a perl master can modify it?  Or
> "sane" as in "don't be so pedantic about things that don't matter"?

A few years ago, I did some work on automatically identifying bug fixing 
patches, based on combining information from the log message and the 
code.  For example, one issue would be the number of changes in the 
code, because a big change, even if it was a bug fix, might be too 
complex for stable.  Machine learning was involved.  The obvious weak 
point of the work was that we only had me to label the data, as opposed 
to a real stable maintainer.  If you think it could be useful to pursue 
this work, and do a better job, and would be willing to help out in 
creating the datasets, then perhaps it could be something to pursue 
again.  My collaborator on this work is David Lo of Singapore Management 
University.  I will be visiting him in the first week of September and 
could discuss with him further.  If by chance you think of something 
else where machine learning could be relevant to kernel development, I 
could bring that up as well.  He knows everything about applying machine 
learning to software development issues, but nothing about the Linux 
kernel, or even C code.

julia

Re: [Ksummit-discuss] Self nomination - Sasha Levin

[Julia Lawall]

Not sure if I was clear about what I was asking you to agree to :)

Basically, we can take the patches sent to stable and the patches not 
sent to stable as a training set, but then the machine learning comes up 
with some algorithm that produces some results.  An expert is needed to 
evaluate the results.  Ie for a thousand (number chosen at random) 
patches, if the algorithm says it is a bug fixing patch, is it or isn't 
it, and vice versa.  Of course, we could also evaluate on patches that 
previously have and have not been sent too stable, but there is a 
problem there, because our goal is to have more patches sent to stable 
than are already being sent there, so we need to show that the algorithm 
can capture what humans are missing.

julia

Re: [Ksummit-discuss] Self nomination - Sasha Levin

[Greg KH]

On Fri, Aug 26, 2016 at 01:55:27PM +0200, Julia Lawall wrote:
> Not sure if I was clear about what I was asking you to agree to :)
> 
> Basically, we can take the patches sent to stable and the patches not sent
> to stable as a training set, but then the machine learning comes up with
> some algorithm that produces some results.  An expert is needed to evaluate
> the results.  Ie for a thousand (number chosen at random) patches, if the
> algorithm says it is a bug fixing patch, is it or isn't it, and vice versa.
> Of course, we could also evaluate on patches that previously have and have
> not been sent too stable, but there is a problem there, because our goal is
> to have more patches sent to stable than are already being sent there, so we
> need to show that the algorithm can capture what humans are missing.

I think that is very interesting research and I would be glad to help
out with it how ever I can, as the result might be very useful for us.

So sign me up!

thanks,

greg k-h

Re: [Ksummit-discuss] Self nomination - Sasha Levin

[Levin, Alexander]

On Fri, Aug 26, 2016 at 08:11:19AM -0400, Greg KH wrote:
> On Fri, Aug 26, 2016 at 01:55:27PM +0200, Julia Lawall wrote:
> > Not sure if I was clear about what I was asking you to agree to :)
> > 
> > Basically, we can take the patches sent to stable and the patches not sent
> > to stable as a training set, but then the machine learning comes up with
> > some algorithm that produces some results.  An expert is needed to evaluate
> > the results.  Ie for a thousand (number chosen at random) patches, if the
> > algorithm says it is a bug fixing patch, is it or isn't it, and vice versa.
> > Of course, we could also evaluate on patches that previously have and have
> > not been sent too stable, but there is a problem there, because our goal is
> > to have more patches sent to stable than are already being sent there, so we
> > need to show that the algorithm can capture what humans are missing.
> 
> I think that is very interesting research and I would be glad to help
> out with it how ever I can, as the result might be very useful for us.
> 
> So sign me up!

I'd be happy to do this as well.

Per Greg's advice, I'm reviewing distro kernels for upstream commits that
they carry which should have been in our LTS trees, those commits usually
aren't tagged in any way and can be a good set of commits for training or
validation.

I do think that we should be using the algorithm to produce a list of authors
and maintainers who don't provide proper tags when they should and have a
discussion with them about why that doesn't happen and how we can help them
to get it "right" (vs just using the algorithm to apply patches).

-- 

Thanks,
Sasha

Re: [Ksummit-discuss] Self nomination - Sasha Levin

[Julia Lawall]

On Fri, 26 Aug 2016, Levin, Alexander wrote:

> On Fri, Aug 26, 2016 at 08:11:19AM -0400, Greg KH wrote:
> > On Fri, Aug 26, 2016 at 01:55:27PM +0200, Julia Lawall wrote:
> > > Not sure if I was clear about what I was asking you to agree to :)
> > >
> > > Basically, we can take the patches sent to stable and the patches not sent
> > > to stable as a training set, but then the machine learning comes up with
> > > some algorithm that produces some results.  An expert is needed to evaluate
> > > the results.  Ie for a thousand (number chosen at random) patches, if the
> > > algorithm says it is a bug fixing patch, is it or isn't it, and vice versa.
> > > Of course, we could also evaluate on patches that previously have and have
> > > not been sent too stable, but there is a problem there, because our goal is
> > > to have more patches sent to stable than are already being sent there, so we
> > > need to show that the algorithm can capture what humans are missing.
> >
> > I think that is very interesting research and I would be glad to help
> > out with it how ever I can, as the result might be very useful for us.
> >
> > So sign me up!
>
> I'd be happy to do this as well.
>
> Per Greg's advice, I'm reviewing distro kernels for upstream commits that
> they carry which should have been in our LTS trees, those commits usually
> aren't tagged in any way and can be a good set of commits for training or
> validation.
>
> I do think that we should be using the algorithm to produce a list of authors
> and maintainers who don't provide proper tags when they should and have a
> discussion with them about why that doesn't happen and how we can help them
> to get it "right" (vs just using the algorithm to apply patches).

Thanks for volunteering and for the suggestions.  I will get back to you
about this shortly.

julia

Re: [Ksummit-discuss] Self nomination - Sasha Levin

[Levin, Alexander]

On Fri, Aug 26, 2016 at 09:55:18AM -0400, Julia Lawall wrote:
> On Fri, 26 Aug 2016, Levin, Alexander wrote:
> > On Fri, Aug 26, 2016 at 08:11:19AM -0400, Greg KH wrote:
> > > On Fri, Aug 26, 2016 at 01:55:27PM +0200, Julia Lawall wrote:
> > > > Not sure if I was clear about what I was asking you to agree to :)
> > > >
> > > > Basically, we can take the patches sent to stable and the patches not sent
> > > > to stable as a training set, but then the machine learning comes up with
> > > > some algorithm that produces some results.  An expert is needed to evaluate
> > > > the results.  Ie for a thousand (number chosen at random) patches, if the
> > > > algorithm says it is a bug fixing patch, is it or isn't it, and vice versa.
> > > > Of course, we could also evaluate on patches that previously have and have
> > > > not been sent too stable, but there is a problem there, because our goal is
> > > > to have more patches sent to stable than are already being sent there, so we
> > > > need to show that the algorithm can capture what humans are missing.
> > >
> > > I think that is very interesting research and I would be glad to help
> > > out with it how ever I can, as the result might be very useful for us.
> > >
> > > So sign me up!
> >
> > I'd be happy to do this as well.
> >
> > Per Greg's advice, I'm reviewing distro kernels for upstream commits that
> > they carry which should have been in our LTS trees, those commits usually
> > aren't tagged in any way and can be a good set of commits for training or
> > validation.
> >
> > I do think that we should be using the algorithm to produce a list of authors
> > and maintainers who don't provide proper tags when they should and have a
> > discussion with them about why that doesn't happen and how we can help them
> > to get it "right" (vs just using the algorithm to apply patches).
> 
> Thanks for volunteering and for the suggestions.  I will get back to you
> about this shortly.

I think it's also a great opportunity to discuss the differences between the
commits Greg marked for stable and the ones I did. As with everything else we
see things differently so it'll be useful to learn more about why each of us
picked (or didn't pick) a given commit.

-- 

Thanks,
Sasha

Re: [Ksummit-discuss] Self nomination - Sasha Levin

[Julia Lawall]

On Fri, 26 Aug 2016, Levin, Alexander wrote:

> On Fri, Aug 26, 2016 at 09:55:18AM -0400, Julia Lawall wrote:
> > On Fri, 26 Aug 2016, Levin, Alexander wrote:
> > > On Fri, Aug 26, 2016 at 08:11:19AM -0400, Greg KH wrote:
> > > > On Fri, Aug 26, 2016 at 01:55:27PM +0200, Julia Lawall wrote:
> > > > > Not sure if I was clear about what I was asking you to agree to :)
> > > > >
> > > > > Basically, we can take the patches sent to stable and the patches not sent
> > > > > to stable as a training set, but then the machine learning comes up with
> > > > > some algorithm that produces some results.  An expert is needed to evaluate
> > > > > the results.  Ie for a thousand (number chosen at random) patches, if the
> > > > > algorithm says it is a bug fixing patch, is it or isn't it, and vice versa.
> > > > > Of course, we could also evaluate on patches that previously have and have
> > > > > not been sent too stable, but there is a problem there, because our goal is
> > > > > to have more patches sent to stable than are already being sent there, so we
> > > > > need to show that the algorithm can capture what humans are missing.
> > > >
> > > > I think that is very interesting research and I would be glad to help
> > > > out with it how ever I can, as the result might be very useful for us.
> > > >
> > > > So sign me up!
> > >
> > > I'd be happy to do this as well.
> > >
> > > Per Greg's advice, I'm reviewing distro kernels for upstream commits that
> > > they carry which should have been in our LTS trees, those commits usually
> > > aren't tagged in any way and can be a good set of commits for training or
> > > validation.
> > >
> > > I do think that we should be using the algorithm to produce a list of authors
> > > and maintainers who don't provide proper tags when they should and have a
> > > discussion with them about why that doesn't happen and how we can help them
> > > to get it "right" (vs just using the algorithm to apply patches).
> >
> > Thanks for volunteering and for the suggestions.  I will get back to you
> > about this shortly.
>
> I think it's also a great opportunity to discuss the differences between the
> commits Greg marked for stable and the ones I did. As with everything else we
> see things differently so it'll be useful to learn more about why each of us
> picked (or didn't pick) a given commit.

I agree.  This occurred to me also.

julia

Re: [Ksummit-discuss] Self nomination - Sasha Levin

[Julia Lawall]

Le 26.08.2016 13:26, Greg KH a écrit :
> On Fri, Aug 26, 2016 at 12:46:51AM -0400, Levin, Alexander wrote:
>>    - Making checkpatch check for (some) of the stable kernel rules
>>    (and possibly recommend adding the stable@ tag in certain cases?).
>>      - Depends on: making checkpatch sane again.
> 
> This sounds interesting.  What do you mean by "sane"?  "sane" as in
> rewriting it so that someone who isn't a perl master can modify it?  Or
> "sane" as in "don't be so pedantic about things that don't matter"?

Wouldn't that require interpreting the log message as well?

julia

Re: [Ksummit-discuss] Self nomination - Sasha Levin

[Levin, Alexander]

On Fri, Aug 26, 2016 at 08:08:49AM -0400, Julia Lawall wrote:
> Le 26.08.2016 13:26, Greg KH a écrit :
> > On Fri, Aug 26, 2016 at 12:46:51AM -0400, Levin, Alexander wrote:
> >>    - Making checkpatch check for (some) of the stable kernel rules
> >>    (and possibly recommend adding the stable@ tag in certain cases?).
> >>      - Depends on: making checkpatch sane again.
> > 
> > This sounds interesting.  What do you mean by "sane"?  "sane" as in
> > rewriting it so that someone who isn't a perl master can modify it?  Or
> > "sane" as in "don't be so pedantic about things that don't matter"?
> 
> Wouldn't that require interpreting the log message as well?

It already does.

-- 

Thanks,
Sasha

Re: [Ksummit-discuss] Self nomination - Sasha Levin

[Levin, Alexander]

On Fri, Aug 26, 2016 at 07:26:35AM -0400, Greg KH wrote:
> On Fri, Aug 26, 2016 at 12:46:51AM -0400, Levin, Alexander wrote:
> >    - Making checkpatch check for (some) of the stable kernel rules
> >    (and possibly recommend adding the stable@ tag in certain cases?).
> >      - Depends on: making checkpatch sane again.
> 
> This sounds interesting.  What do you mean by "sane"?  "sane" as in
> rewriting it so that someone who isn't a perl master can modify it?  Or
> "sane" as in "don't be so pedantic about things that don't matter"?

Both.

It actually came up during my talk: in theory we can enforce with checkpatch to produce proper stable@ tags with correct version/Fixes tagging to simplify our lives, but my claim was that the amount of actual people who use checkpatch is dropping due to the amount of warnings it spews that people just proceed to ignore.

The perl bit is also annoying, there's no way I can do what I suggested above myself.
 
> >    - Improving tagging for stable. The "version tag" option is broken
> >    and the "Fixes:" tag is always preferable, how do we get people to
> >    use that more often? (script it somehow?
> >    scripts/find-version-it-fixes ?).
> 
> Oh a script like that would be nice, but how would that work in reality?

The stable-deps script I have can find the commit dependencies you need to have to apply a give patch on top of that.

Since usually the broken fix is a dependency for the fix, we can figure out which commit we fix by looking at the dependency tree.

It's not going to be *perfect* (and since nothing is that's fine), but it could produce a very short list of possible options for a "Fixes:" tag.

-- 

Thanks,
Sasha