From: Sasha Levin <sashal@kernel.org>
To: James Bottomley <James.Bottomley@hansenpartnership.com>
Cc: Thorsten Leemhuis <linux@leemhuis.info>,
"ksummit@lists.linux.dev" <ksummit@lists.linux.dev>
Subject: Re: [MAINTAINERS SUMMIT] [4/4] Discuss how to better prevent backports of commits that turn out to cause regressions
Date: Thu, 13 Jun 2024 09:06:54 -0400 [thread overview]
Message-ID: <ZmruqWDTG2PK-rbu@sashalap> (raw)
In-Reply-To: <7794a2b09ae4fa73ac35fdaec4858145a665efea.camel@HansenPartnership.com>
On Thu, Jun 13, 2024 at 07:58:58AM -0400, James Bottomley wrote:
>On Thu, 2024-06-13 at 10:42 +0200, Thorsten Leemhuis wrote:
>> The scenario shown at the start of the thread illustrates a problem I
>> see frequently: commits with a Fixes: tag end up in new to stable
>> series releases just days after being mainlined and cause regressions
>> -- just like they do in mainline, which just was not known yet at the
>> time of backporting. This happens extremely often right after merge
>> windows when huge piles of changes are backported to the stable trees
>> each cycle shortly after -rc1 is out (which even some kernel
>> developers apparently are somewhat afraid to test from what I've
>> seen).
>
>I haven't really observed this for curated fixes. For most subsystems,
>patches with Fixes tags that are cc'd to stable tend to go steadily
>outside the merge window. Obviously a few arrive within it, but
>usually at roughly the rate they arrive outside it.
>
>What I observe in the merge window is huge piles of patches go into
>stable *without* a cc:stable tag from the autosel machinery (and quite
>a few even without fixes: tags).
Could you provide a concrete example? This shouldn't happen.
>So this does beg a couple of questions:
>
>Since you have the figures, what's the proportion of regressions caused
>by backports to stable without cc:stable tags?
This question came up two years ago and we had statistics around this.
Autosel patches didn't cause more (actually, it was *less*) regressions
than stable tagged ones.
>Could we fix a lot of this by delaying autosel? It tends to ramp up in
>the merge window when everyone is concentrating on other things, so any
>regressions it causes naturally get ignored for a couple of weeks.
autosel is currently delayed about 3-4 weeks, sometimes more.
--
Thanks,
Sasha
next prev parent reply other threads:[~2024-06-13 13:27 UTC|newest]
Thread overview: 107+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-06-13 8:22 [MAINTAINERS SUMMIT] [0/4] Common scenario for four proposals regarding regressions Thorsten Leemhuis
2024-06-13 8:26 ` [MAINTAINERS SUMMIT] [1/4] Create written down guidelines for handling regressions Thorsten Leemhuis
2024-09-12 13:33 ` Thorsten Leemhuis
2024-06-13 8:32 ` [MAINTAINERS SUMMIT] [2/4] Ensure recent mainline regression are fixed in latest stable series Thorsten Leemhuis
2024-06-13 11:02 ` Johannes Berg
2024-06-13 11:21 ` Greg KH
2024-06-13 13:18 ` Sasha Levin
2024-06-13 11:17 ` Jiri Kosina
2024-06-13 11:28 ` Laurent Pinchart
2024-06-14 0:50 ` Steven Rostedt
2024-06-14 14:01 ` Mark Brown
2024-06-14 14:32 ` Rafael J. Wysocki
2024-06-13 8:34 ` [MAINTAINERS SUMMIT] [3/4] Elevate handling of regressions that made it to releases deemed for end users Thorsten Leemhuis
2024-06-13 11:34 ` Laurent Pinchart
2024-06-13 11:39 ` Jiri Kosina
2024-06-14 14:10 ` Mark Brown
2024-06-18 12:58 ` Thorsten Leemhuis
2024-06-19 20:25 ` Laurent Pinchart
2024-06-20 10:47 ` Thorsten Leemhuis
2024-06-13 15:56 ` Liam R. Howlett
2024-06-18 12:24 ` Thorsten Leemhuis
2024-06-20 13:20 ` Jani Nikula
2024-06-20 13:35 ` Thorsten Leemhuis
2024-06-20 14:16 ` Mark Brown
2024-06-21 6:47 ` Jiri Kosina
2024-06-21 10:19 ` Thorsten Leemhuis
2024-06-13 8:42 ` [MAINTAINERS SUMMIT] [4/4] Discuss how to better prevent backports of commits that turn out to cause regressions Thorsten Leemhuis
2024-06-13 9:59 ` Jan Kara
2024-06-13 10:18 ` Thorsten Leemhuis
2024-06-13 14:08 ` Konstantin Ryabitsev
2024-06-14 9:19 ` Lee Jones
2024-06-14 9:24 ` Lee Jones
2024-06-14 12:27 ` Konstantin Ryabitsev
2024-06-14 14:26 ` Konstantin Ryabitsev
2024-06-14 14:36 ` Lee Jones
2024-06-14 14:29 ` Michael Ellerman
2024-06-14 14:38 ` Konstantin Ryabitsev
2024-06-14 14:44 ` Rafael J. Wysocki
2024-06-14 15:08 ` Geert Uytterhoeven
2024-06-15 11:29 ` Michael Ellerman
2024-06-17 10:15 ` Jani Nikula
2024-06-17 12:42 ` Geert Uytterhoeven
2024-06-14 15:45 ` Mark Brown
2024-06-14 14:43 ` Mark Brown
2024-06-14 14:51 ` Konstantin Ryabitsev
2024-06-14 15:42 ` Mark Brown
2024-06-14 14:43 ` Steven Rostedt
2024-06-14 14:57 ` Laurent Pinchart
2024-06-16 1:13 ` Linus Torvalds
2024-06-16 3:28 ` Steven Rostedt
2024-06-16 4:59 ` Linus Torvalds
2024-06-16 8:22 ` Paolo Bonzini
2024-06-16 9:05 ` Geert Uytterhoeven
2024-06-16 15:07 ` Steven Rostedt
2024-06-17 13:48 ` Dan Carpenter
2024-06-17 15:23 ` Dan Carpenter
2024-06-17 14:39 ` Konstantin Ryabitsev
2024-06-17 16:04 ` Paul E. McKenney
2024-06-17 16:06 ` Konstantin Ryabitsev
2024-06-17 16:14 ` Paolo Bonzini
2024-06-17 16:18 ` Konstantin Ryabitsev
2024-06-17 17:11 ` Geert Uytterhoeven
2024-06-18 12:05 ` Michael Ellerman
2024-06-16 7:26 ` Takashi Iwai
2024-06-16 8:10 ` Paolo Bonzini
2024-06-16 11:31 ` Laurent Pinchart
2024-06-16 11:39 ` Takashi Iwai
2024-06-16 16:40 ` Linus Torvalds
2024-06-16 8:31 ` Jiri Kosina
2024-06-16 8:54 ` Geert Uytterhoeven
2024-06-13 19:39 ` Dan Carpenter
2024-06-14 1:00 ` Steven Rostedt
2024-06-13 11:58 ` James Bottomley
2024-06-13 13:06 ` Sasha Levin [this message]
2024-06-13 13:56 ` James Bottomley
2024-06-13 14:02 ` Greg KH
2024-06-13 15:11 ` James Bottomley
2024-06-13 16:27 ` Greg KH
2024-06-14 18:47 ` Sasha Levin
2024-06-17 10:59 ` Vlastimil Babka
2024-06-13 18:08 ` Sasha Levin
2024-06-13 13:45 ` Greg KH
2024-06-13 13:40 ` Sasha Levin
2024-06-18 13:12 ` Thorsten Leemhuis
2024-06-13 14:28 ` Andrew Lunn
2024-06-13 18:14 ` Sasha Levin
2024-06-14 14:41 ` Jan Kara
2024-06-14 15:03 ` Rafael J. Wysocki
2024-06-14 17:46 ` Sasha Levin
2024-06-18 14:43 ` [MAINTAINERS SUMMIT] [0/4] Common scenario for four proposals regarding regressions James Bottomley
2024-06-18 15:50 ` Mark Brown
2024-06-20 10:32 ` Thorsten Leemhuis
2024-06-20 12:57 ` James Bottomley
2024-06-20 13:55 ` Mark Brown
2024-06-20 14:01 ` James Bottomley
2024-06-20 14:42 ` Mark Brown
2024-06-20 16:02 ` James Bottomley
2024-06-20 17:15 ` Mark Brown
2024-06-20 23:25 ` Sasha Levin
2024-06-21 6:33 ` Thorsten Leemhuis
[not found] ` <20240625175131.672d14a4@rorschach.local.home>
2024-06-26 7:36 ` Greg KH
2024-06-26 18:32 ` Steven Rostedt
2024-06-26 19:05 ` James Bottomley
2024-07-25 10:14 ` Thorsten Leemhuis
2024-07-25 13:14 ` Greg KH
2024-06-20 16:59 ` Thorsten Leemhuis
2024-06-20 23:18 ` Sasha Levin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZmruqWDTG2PK-rbu@sashalap \
--to=sashal@kernel.org \
--cc=James.Bottomley@hansenpartnership.com \
--cc=ksummit@lists.linux.dev \
--cc=linux@leemhuis.info \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox