Re: [RFC PATCH v2 1/2] docs: add a document about regression handling

["Matthias Brugger (SUSE)" <mbrugger@suse.com>]

On 1/10/22 12:40, Thorsten Leemhuis wrote:
> On 07.01.22 18:44, Matthias Brugger wrote:
>> On 07/01/2022 17:51, Thorsten Leemhuis wrote:
>>> On 07.01.22 16:37, Matthias Brugger wrote:
>>>> On 07/01/2022 15:21, Thorsten Leemhuis wrote:
>>>>> Create a document explaining various aspects around regression handling
>>>>> and tracking both for users and developers. Among others describe the
>>>>> first rule of Linux kernel development and what it means in practice.
>>>>> Also explain what a regression actually is and how to report one
>>>>> properly. The text additionally provides a brief introduction to the
>>>>> bot
>>>>> the kernel's regression tracker uses to facilitate his work. To sum
>>>>> things up, provide a few quotes from Linus to show how serious he takes
>>>>> regressions.
>>>>>
>>>>> Signed-off-by: Thorsten Leemhuis <linux@leemhuis.info>
>>>> [...]
>>>>> +The important bits for people fixing regressions
>>>>> +================================================
>>>>> +
>>>>> +When receiving regression reports by mail, check if the reporter CCed
>>>>> `the
>>>>> +regression mailing list <https://lore.kernel.org/regressions/>`_
>>>>> +(regressions@lists.linux.dev). If not, forward or bounce the report
>>>>> to the Linux
>>>>> +kernel's regression tracker (regressions@leemhuis.info), unless you
>>>>> plan on
>>>>
>>>> I would have expected it to be the same mailing list
>>>> (regressions@lists.linux.dev), is this a typo maybe?
>>>
>>> Thx for taking a look. Hmm. That's possible, but I (and the grep call I
>>> just ran) fail to spot the typo.
>>>

I think that structure is much better, thanks!

Regards,
Matthias

>>> Maybe the wording is to confusing: regressions@lists.linux.dev is the
>>> list, regressions@leemhuis.info is a dedicated email address I (the
>>> kernel's regression tracker) use for regression tracking (which reminds
>>> me: maybe I should ask for a alias like regressions@kernel.org or
>>> regression-tracker@kernel.org).
>>
>> Yes it's the wording then :)
>> Anyway I just wonder why you we should send the regression to the
>> regressions email list, but only to the tracker email address. For me
>> that's the confusing part. I'd expect to send it to the list as well and
>> the tracker takes it from there. If for any reason someone does not want
>> to send a regression to the list, then he can send it to the tracker
>> directly. That's my understanding of how this works. If that's correct
>> then I'd say we should just explain the difference.
> 
> You comments made be revisit the section and made me spot a few other
> issues I found less than ideal. So I rewrote it over the weekend (more
> than once, to be precise...). I hope this clears things up.
> 
> ```
> The important bits for people fixing regressions
> ================================================
> 
> When submitting fixes for regressions, add "Link:" tags pointing to all
> places where the issue was reported, as tools like the Linux kernel
> regression bot 'regzbot' heavily rely on these; these pointers are also
> of great value for people looking into the issue some time in the
> future, as explained in `Documentation/process/submitting-patches.rst`
> and :ref:`Documentation/process/5.Posting.rst <development_posting>`::
> 
>         Link:
> https://lore.kernel.org/r/30th.anniversary.repost@klaava.Helsinki.FI/
>         Link: https://bugzilla.kernel.org/show_bug.cgi?id=1234567890
> 
> Let the Linux kernel's regression tracker and all other subscribers of
> the `regression mailing list <https://lore.kernel.org/regressions/>`_
> (regressions@lists.linux.dev) quickly known about newly reported
> regressions:
> 
>   * When receiving a mailed report that did not CC the list, immediately
> send at least a brief "Reply-all" with the list CCed to get it into the
> loop; also ensure its CCed on all future replies, in case it got lost.
> 
>   * If you receive a report from a bug tracker, forward or bounce the
> report to the list, unless the reporter followed
> `Documentation/admin-guide/reporting-issues.rst` instructions and did it
> already.
> 
> [Optional] Ensure the Linux kernel regression bot 'regzbot' tracks the
> issue:
> 
>   * For mailed reports, check if the reporter included a 'regzbot
> command' like the ``#regzbot introduced v5.13..v5.14-rc1`` described
> above. If not, send a reply with the regressions list in CC, which
> includes a paragraph like the following:
> 
>         #regzbot ^introduced v5.13..v5.14-rc1
> 
>    Note, in this case there is a caret (^) before the `introduced` to
> make regzbot treat the parent mail (the one you reply to) as the report
> for the regression you want to see tracked.
> 
>    Instead of specifying a version range you can also state a commit-id,
> in case the reporter identified the culprit.
> 
>   * When receiving a report from a bug tracker and forwarding it to the
> regressions list (see above), include a paragraph like this:
> 
>         #regzbot introduced: v5.13..v5.14-rc1
>         #regzbot from: Some N. Ice Human <some.human@example.com>
>         #regzbot monitor:
> http://some.bugtracker.example.com/ticket?id=123456789
> ```
> 
> Note, regzbot does not yet support "#regzbot from" and "#regzbot monitor
> <bugtracker>", but I wanted to work on that soon anyway -- and this text
> will likely take weeks before it hits mailine, so this shouldn't be a
> problem.
> 
> Ciao, Thorsten
>