Re: [Ksummit-discuss] [GIT PULL] ARM: SoC fixes (and a cross-arch dt-include fix)

Re: [Ksummit-discuss] [GIT PULL] ARM: SoC fixes (and a cross-arch dt-include fix)

[Olof Johansson]

[adding ksummit-discuss]

On Mon, May 22, 2017 at 4:44 AM, Linus Walleij <linus.walleij@linaro.org> wrote:
> On Fri, May 19, 2017 at 9:34 PM, Olof Johansson <olof@lixom.net> wrote:
>
>>  - We've started telling people to avoid cross-tree shared branches if all
>>    they're doing is picking up one or two DT-used constants from a
>>    shared include file, and instead to use the numeric values on first
>>    submission. Follow-up moving over to symbolic names are sent in right
>>    after -rc1, i.e. here. It's only a few minor patches of this type.
>
> OK it seems like a reasonable process.
>
> It's not like I can think about anything better.
>
> I was more opting for doing things this way:
>
> - Create bindings and <dt-bindings/foo/bar.h> merge it into the
>   foo subsystem.
>
> - Merge driver into drivers/foo/bar.c that use <dt-bindings/foo/bar.h>
>
> - Submit DTS patch to ARM SoC (or whetever) using only numerical
>   values.
>
> - After the merge window, follow up with a patch replacing the
>   numerical constants with #defines from <dt-bindings/foo/bar.h>

You're describing exactly what we've been pushing people towards.

If it's just a few simple references it's not significantly more
awkward, and decouples merges and removes need for stable branches.
Essentially we've become slightly more lax in what we're willing to
consider _right after_ -rc1 to include these tweaks (and often patches
to turn on new drivers in defconfigs).

If the amount of contents grows too large we might need to tweak
things further, maybe with something pre-rc1 but that's more awkward.

> An alternative would obviously be to wait for the next merge window
> after merging the driver patch but I guess people are too impatient
> to do that (including me).

Yeah, asking people to spread out across releases would remove
dependencies a lot, but it would also slow down progress and frustrate
a lot of contributors so we don't do that.

> We discussed cross-tree dependencies a bit on ksummit-discuss
> but this solution was not mentioned back then.

I thought it was, but I wasn't fully engaged in the discussion. We've
also only started this over the last release or two so it's early to
tell just how well it'll work in reality. Cc:ing the list.



-Olof

Re: [Ksummit-discuss] [GIT PULL] ARM: SoC fixes (and a cross-arch dt-include fix)

[Daniel Vetter]

On Mon, May 22, 2017 at 5:11 PM, Olof Johansson <olof@lixom.net> wrote:
> [adding ksummit-discuss]
>
> On Mon, May 22, 2017 at 4:44 AM, Linus Walleij <linus.walleij@linaro.org> wrote:
>> On Fri, May 19, 2017 at 9:34 PM, Olof Johansson <olof@lixom.net> wrote:
>>
>>>  - We've started telling people to avoid cross-tree shared branches if all
>>>    they're doing is picking up one or two DT-used constants from a
>>>    shared include file, and instead to use the numeric values on first
>>>    submission. Follow-up moving over to symbolic names are sent in right
>>>    after -rc1, i.e. here. It's only a few minor patches of this type.
>>
>> OK it seems like a reasonable process.
>>
>> It's not like I can think about anything better.
>>
>> I was more opting for doing things this way:
>>
>> - Create bindings and <dt-bindings/foo/bar.h> merge it into the
>>   foo subsystem.
>>
>> - Merge driver into drivers/foo/bar.c that use <dt-bindings/foo/bar.h>
>>
>> - Submit DTS patch to ARM SoC (or whetever) using only numerical
>>   values.
>>
>> - After the merge window, follow up with a patch replacing the
>>   numerical constants with #defines from <dt-bindings/foo/bar.h>
>
> You're describing exactly what we've been pushing people towards.
>
> If it's just a few simple references it's not significantly more
> awkward, and decouples merges and removes need for stable branches.
> Essentially we've become slightly more lax in what we're willing to
> consider _right after_ -rc1 to include these tweaks (and often patches
> to turn on new drivers in defconfigs).
>
> If the amount of contents grows too large we might need to tweak
> things further, maybe with something pre-rc1 but that's more awkward.
>
>> An alternative would obviously be to wait for the next merge window
>> after merging the driver patch but I guess people are too impatient
>> to do that (including me).
>
> Yeah, asking people to spread out across releases would remove
> dependencies a lot, but it would also slow down progress and frustrate
> a lot of contributors so we don't do that.
>
>> We discussed cross-tree dependencies a bit on ksummit-discuss
>> but this solution was not mentioned back then.
>
> I thought it was, but I wasn't fully engaged in the discussion. We've
> also only started this over the last release or two so it's early to
> tell just how well it'll work in reality. Cc:ing the list.

Yeah, this sounds like a good step towards improving what I've
complained about, while still keeping topic/shared stable branch
proliferation under control.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Re: [Ksummit-discuss] [GIT PULL] ARM: SoC fixes (and a cross-arch dt-include fix)

[Geert Uytterhoeven]

Hi Olof,

On Mon, May 22, 2017 at 5:11 PM, Olof Johansson <olof@lixom.net> wrote:
> On Mon, May 22, 2017 at 4:44 AM, Linus Walleij <linus.walleij@linaro.org> wrote:
>> On Fri, May 19, 2017 at 9:34 PM, Olof Johansson <olof@lixom.net> wrote:
>>>  - We've started telling people to avoid cross-tree shared branches if all
>>>    they're doing is picking up one or two DT-used constants from a
>>>    shared include file, and instead to use the numeric values on first
>>>    submission. Follow-up moving over to symbolic names are sent in right
>>>    after -rc1, i.e. here. It's only a few minor patches of this type.
>>
>> OK it seems like a reasonable process.
>>
>> It's not like I can think about anything better.
>>
>> I was more opting for doing things this way:
>>
>> - Create bindings and <dt-bindings/foo/bar.h> merge it into the
>>   foo subsystem.
>>
>> - Merge driver into drivers/foo/bar.c that use <dt-bindings/foo/bar.h>
>>
>> - Submit DTS patch to ARM SoC (or whetever) using only numerical
>>   values.
>>
>> - After the merge window, follow up with a patch replacing the
>>   numerical constants with #defines from <dt-bindings/foo/bar.h>
>
> You're describing exactly what we've been pushing people towards.
>
> If it's just a few simple references it's not significantly more
> awkward, and decouples merges and removes need for stable branches.
> Essentially we've become slightly more lax in what we're willing to
> consider _right after_ -rc1 to include these tweaks (and often patches
> to turn on new drivers in defconfigs).
>
> If the amount of contents grows too large we might need to tweak
> things further, maybe with something pre-rc1 but that's more awkward.
>
>> An alternative would obviously be to wait for the next merge window
>> after merging the driver patch but I guess people are too impatient
>> to do that (including me).
>
> Yeah, asking people to spread out across releases would remove
> dependencies a lot, but it would also slow down progress and frustrate
> a lot of contributors so we don't do that.

The above works fine for new support, or for new platforms.

There's still support being migrated from platform code to DT, which
requires three steps:
  1. New DT-aware driver support,
  2. DT update to use the new driver support,
  3. Clean up platform code after optional DTB backwards compatibility
     grace period,
To make matters worse, 1 may conflict with the existing platform code,
and 2 must sometimes not be done before 1. Hence you may need three kernel
releases.
So we're already planning now what to clean up for v4.15 ;-)

Would it be acceptable to do step 2 in the same release, after the driver
support has entered in -rc1? I know this is more than just replacing
numbers by symbolic values.

Thanks!

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [Ksummit-discuss] [GIT PULL] ARM: SoC fixes (and a cross-arch dt-include fix)

[Arnd Bergmann]

On Tue, May 23, 2017 at 11:49 AM, Geert Uytterhoeven
<geert@linux-m68k.org> wrote:
>
> On Mon, May 22, 2017 at 5:11 PM, Olof Johansson <olof@lixom.net> wrote:
>> On Mon, May 22, 2017 at 4:44 AM, Linus Walleij <linus.walleij@linaro.org> wrote:
>>> On Fri, May 19, 2017 at 9:34 PM, Olof Johansson <olof@lixom.net> wrote:
>>
>> Yeah, asking people to spread out across releases would remove
>> dependencies a lot, but it would also slow down progress and frustrate
>> a lot of contributors so we don't do that.
>
> The above works fine for new support, or for new platforms.
>
> There's still support being migrated from platform code to DT, which
> requires three steps:
>   1. New DT-aware driver support,
>   2. DT update to use the new driver support,
>   3. Clean up platform code after optional DTB backwards compatibility
>      grace period,
> To make matters worse, 1 may conflict with the existing platform code,
> and 2 must sometimes not be done before 1. Hence you may need three kernel
> releases.
> So we're already planning now what to clean up for v4.15 ;-)
>
> Would it be acceptable to do step 2 in the same release, after the driver
> support has entered in -rc1? I know this is more than just replacing
> numbers by symbolic values.

I'd say it really depends on the individual case. Do you have a particular
platform in mind? E.g. For some of the more obsolete platforms that
Linus Walleij has worked on over time, we have sometimes relaxed the
rules about clean bisection and just merged everything in parallel, knowing
that nobody else was likely to run that code on a vanilla kernel anyway.

       Arnd

Re: [Ksummit-discuss] [GIT PULL] ARM: SoC fixes (and a cross-arch dt-include fix)

[Geert Uytterhoeven]

Hi Arnd,

On Tue, May 23, 2017 at 4:17 PM, Arnd Bergmann <arnd@arndb.de> wrote:
> On Tue, May 23, 2017 at 11:49 AM, Geert Uytterhoeven
> <geert@linux-m68k.org> wrote:
>> On Mon, May 22, 2017 at 5:11 PM, Olof Johansson <olof@lixom.net> wrote:
>>> On Mon, May 22, 2017 at 4:44 AM, Linus Walleij <linus.walleij@linaro.org> wrote:
>>>> On Fri, May 19, 2017 at 9:34 PM, Olof Johansson <olof@lixom.net> wrote:
>>>
>>> Yeah, asking people to spread out across releases would remove
>>> dependencies a lot, but it would also slow down progress and frustrate
>>> a lot of contributors so we don't do that.
>>
>> The above works fine for new support, or for new platforms.
>>
>> There's still support being migrated from platform code to DT, which
>> requires three steps:
>>   1. New DT-aware driver support,
>>   2. DT update to use the new driver support,
>>   3. Clean up platform code after optional DTB backwards compatibility
>>      grace period,
>> To make matters worse, 1 may conflict with the existing platform code,
>> and 2 must sometimes not be done before 1. Hence you may need three kernel
>> releases.
>> So we're already planning now what to clean up for v4.15 ;-)
>>
>> Would it be acceptable to do step 2 in the same release, after the driver
>> support has entered in -rc1? I know this is more than just replacing
>> numbers by symbolic values.
>
> I'd say it really depends on the individual case. Do you have a particular
> platform in mind? E.g. For some of the more obsolete platforms that
> Linus Walleij has worked on over time, we have sometimes relaxed the
> rules about clean bisection and just merged everything in parallel, knowing
> that nobody else was likely to run that code on a vanilla kernel anyway.

This is for Renesas R-Car Gen2 SoCs, so we do care about DT backward
compatibility (for a while), and about bisection.

Last headache was "[PATCH v4 00/23] soc: renesas: Add R-Car RST driver for
obtaining mode pin state" (https://lkml.org/lkml/2016/10/21/500). This used
a shared immutable branch, but that caused several merge conflicts.

Next one will be smaller, as it's not really moving functionality from
platform code to DT, but switching to a new and better clock driver framework,
so there's only the dependency of DT on the new driver
"[PATCH v2 00/10] clk: renesas: rcar-gen2: Add new CPG/MSSR drivers"
(https://lkml.org/lkml/2017/5/19/212)

I'll go create an inventory of stuff in platform code for Renesas SoCs that
still needs to be converted to DT...

Thanks!

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [Ksummit-discuss] [GIT PULL] ARM: SoC fixes (and a cross-arch dt-include fix)

[Arnd Bergmann]

On Tue, May 30, 2017 at 11:13 AM, Geert Uytterhoeven
<geert@linux-m68k.org> wrote:
> Hi Arnd,
>
> On Tue, May 23, 2017 at 4:17 PM, Arnd Bergmann <arnd@arndb.de> wrote:
>> On Tue, May 23, 2017 at 11:49 AM, Geert Uytterhoeven
>> <geert@linux-m68k.org> wrote:
>>> On Mon, May 22, 2017 at 5:11 PM, Olof Johansson <olof@lixom.net> wrote:
>>>> On Mon, May 22, 2017 at 4:44 AM, Linus Walleij <linus.walleij@linaro.org> wrote:
>>>>> On Fri, May 19, 2017 at 9:34 PM, Olof Johansson <olof@lixom.net> wrote:
>>>>
>>>> Yeah, asking people to spread out across releases would remove
>>>> dependencies a lot, but it would also slow down progress and frustrate
>>>> a lot of contributors so we don't do that.
>>>
>>> The above works fine for new support, or for new platforms.
>>>
>>> There's still support being migrated from platform code to DT, which
>>> requires three steps:
>>>   1. New DT-aware driver support,
>>>   2. DT update to use the new driver support,
>>>   3. Clean up platform code after optional DTB backwards compatibility
>>>      grace period,
>>> To make matters worse, 1 may conflict with the existing platform code,
>>> and 2 must sometimes not be done before 1. Hence you may need three kernel
>>> releases.
>>> So we're already planning now what to clean up for v4.15 ;-)
>>>
>>> Would it be acceptable to do step 2 in the same release, after the driver
>>> support has entered in -rc1? I know this is more than just replacing
>>> numbers by symbolic values.
>>
>> I'd say it really depends on the individual case. Do you have a particular
>> platform in mind? E.g. For some of the more obsolete platforms that
>> Linus Walleij has worked on over time, we have sometimes relaxed the
>> rules about clean bisection and just merged everything in parallel, knowing
>> that nobody else was likely to run that code on a vanilla kernel anyway.
>
> This is for Renesas R-Car Gen2 SoCs, so we do care about DT backward
> compatibility (for a while), and about bisection.

Ok, I see.

> Last headache was "[PATCH v4 00/23] soc: renesas: Add R-Car RST driver for
> obtaining mode pin state" (https://lkml.org/lkml/2016/10/21/500). This used
> a shared immutable branch, but that caused several merge conflicts.
>
> Next one will be smaller, as it's not really moving functionality from
> platform code to DT, but switching to a new and better clock driver framework,
> so there's only the dependency of DT on the new driver
> "[PATCH v2 00/10] clk: renesas: rcar-gen2: Add new CPG/MSSR drivers"
> (https://lkml.org/lkml/2017/5/19/212)

Ok, so there is already a working driver for these chips, but you want
to move to a better driver and binding. That's all fine, but I'm not sure
about why you want to do this faster at all. Presumably you want to use
the new driver quickly so you can add new platforms not using the old
driver, but then you'd have a couple of years of grace period before
removing the old driver for customers with existing out of tree dts files,
so I would assume that waiting another release or two before moving
the DT files over is not adding a huge burden.

> I'll go create an inventory of stuff in platform code for Renesas SoCs that
> still needs to be converted to DT...

That would certainly be helpful, yes.

       Arnd

Re: [Ksummit-discuss] [GIT PULL] ARM: SoC fixes (and a cross-arch dt-include fix)

[Olof Johansson]

On Tue, May 23, 2017 at 2:49 AM, Geert Uytterhoeven
<geert@linux-m68k.org> wrote:
> Hi Olof,
>
> On Mon, May 22, 2017 at 5:11 PM, Olof Johansson <olof@lixom.net> wrote:
>> On Mon, May 22, 2017 at 4:44 AM, Linus Walleij <linus.walleij@linaro.org> wrote:
>>> On Fri, May 19, 2017 at 9:34 PM, Olof Johansson <olof@lixom.net> wrote:
>>>>  - We've started telling people to avoid cross-tree shared branches if all
>>>>    they're doing is picking up one or two DT-used constants from a
>>>>    shared include file, and instead to use the numeric values on first
>>>>    submission. Follow-up moving over to symbolic names are sent in right
>>>>    after -rc1, i.e. here. It's only a few minor patches of this type.
>>>
>>> OK it seems like a reasonable process.
>>>
>>> It's not like I can think about anything better.
>>>
>>> I was more opting for doing things this way:
>>>
>>> - Create bindings and <dt-bindings/foo/bar.h> merge it into the
>>>   foo subsystem.
>>>
>>> - Merge driver into drivers/foo/bar.c that use <dt-bindings/foo/bar.h>
>>>
>>> - Submit DTS patch to ARM SoC (or whetever) using only numerical
>>>   values.
>>>
>>> - After the merge window, follow up with a patch replacing the
>>>   numerical constants with #defines from <dt-bindings/foo/bar.h>
>>
>> You're describing exactly what we've been pushing people towards.
>>
>> If it's just a few simple references it's not significantly more
>> awkward, and decouples merges and removes need for stable branches.
>> Essentially we've become slightly more lax in what we're willing to
>> consider _right after_ -rc1 to include these tweaks (and often patches
>> to turn on new drivers in defconfigs).
>>
>> If the amount of contents grows too large we might need to tweak
>> things further, maybe with something pre-rc1 but that's more awkward.
>>
>>> An alternative would obviously be to wait for the next merge window
>>> after merging the driver patch but I guess people are too impatient
>>> to do that (including me).
>>
>> Yeah, asking people to spread out across releases would remove
>> dependencies a lot, but it would also slow down progress and frustrate
>> a lot of contributors so we don't do that.
>
> The above works fine for new support, or for new platforms.
>
> There's still support being migrated from platform code to DT, which
> requires three steps:
>   1. New DT-aware driver support,
>   2. DT update to use the new driver support,
>   3. Clean up platform code after optional DTB backwards compatibility
>      grace period,
> To make matters worse, 1 may conflict with the existing platform code,
> and 2 must sometimes not be done before 1. Hence you may need three kernel
> releases.
> So we're already planning now what to clean up for v4.15 ;-)
>
> Would it be acceptable to do step 2 in the same release, after the driver
> support has entered in -rc1? I know this is more than just replacing
> numbers by symbolic values.

Part of why we encouraged a more stretched-out approach in your case
was that we used to get quite a few dependent branches for Renesas
platforms, and when there were things to fixup, a lot of respins and
churn. So we asked to move a little slower to avoid DoS:ing ourselves.

As far as the above process goes: 1+2 is exactly what we're
encouraging others to do. The main discussion point here is when
there's substantial shared dt-includes between driver and the DTS
contents. Most of the time, for most drivers, that's usually not the
case. As Arnd suggested, it might be helpful to have specific examples
to discuss here.


-Olof