[MAINTAINERS SUMMIT] State of dt-bindings and DT source files, and invitation request

[MAINTAINERS SUMMIT] State of dt-bindings and DT source files, and invitation request

[Arınç ÜNAL]

Hello.

I maintain the MediaTek DSA subdriver and some devicetree bindings and
source files for MediaTek hardware.

I am especially interested in the best practices of maintaining dt-bindings
and DT source files.

There's this false impression with some maintainers that, as the
dt-bindings and the DT source files are being hosted on the Linux
repository, Linux drivers have influence over the design of bindings or
fixing DT source files that did not comply with the bindings.

I'd be very interested to be involved in or kick start the efforts to take
dt-bindings and DT source files out of the Linux repository into its own,
separate repository. I believe, this would be a great step in addressing
all the project-dependent bindings of Linux, U-Boot, OpenWrt, and all other
projects, to have a single, unified repository to describe all the hardware
that exists in the world. I am already working towards this goal by
improving the dt-bindings and DT source files on the Linux repository
whenever I can.

I must be quite late to make a topic suggestion but I'd be very happy to be
able to attend to the maintainers summit. I've already registered for the
Linux Plumbers Conference 2024.

Arınç

Re: [MAINTAINERS SUMMIT] State of dt-bindings and DT source files, and invitation request

[Rob Herring]

On Tue, Sep 10, 2024 at 5:53 AM Arınç ÜNAL <arinc.unal@arinc9.com> wrote:
>
> Hello.
>
> I maintain the MediaTek DSA subdriver and some devicetree bindings and
> source files for MediaTek hardware.
>
> I am especially interested in the best practices of maintaining dt-bindings
> and DT source files.
>
> There's this false impression with some maintainers that, as the
> dt-bindings and the DT source files are being hosted on the Linux
> repository, Linux drivers have influence over the design of bindings or
> fixing DT source files that did not comply with the bindings.
>
> I'd be very interested to be involved in or kick start the efforts to take
> dt-bindings and DT source files out of the Linux repository into its own,
> separate repository. I believe, this would be a great step in addressing
> all the project-dependent bindings of Linux, U-Boot, OpenWrt, and all other
> projects, to have a single, unified repository to describe all the hardware
> that exists in the world.

This! This is precisely why we don't move things out of the kernel.
The kernel is the location that has the most hardware support in the
world by far. It is not even close. Really, the only h/w missing are
things too small to run Linux. And with all that h/w support, comes
the people who understand the various classes of h/w. Those people are
not going to come along to a separate project. It would be more work
and there aren't any maintainers looking for extra work.

We already have a separate repository[1]. U-boot has recently
incorporated it and is happily (AFAIK) using it. It happens to be
generated from the kernel tree, but what doesn't work for you there?
I'm happy to discuss what it needs to work better.

> I am already working towards this goal by
> improving the dt-bindings and DT source files on the Linux repository
> whenever I can.

That's great, still plenty of work to do there no matter what
repository is hosting it.

> I must be quite late to make a topic suggestion but I'd be very happy to be
> able to attend to the maintainers summit. I've already registered for the
> Linux Plumbers Conference 2024.

This is probably not a maintainers summit topic. There's a DT BoF
scheduled already that Krzysztof is running and supporting other
projects is on the agenda already. I won't be there in person nor will
I be awake at the scheduled time.

Rob

[1] https://git.kernel.org/pub/scm/linux/kernel/git/devicetree/devicetree-rebasing.git/

Re: [MAINTAINERS SUMMIT] State of dt-bindings and DT source files, and invitation request

[Conor Dooley]

[-- Attachment #1: Type: text/plain, Size: 2966 bytes --]

On Tue, Sep 10, 2024 at 10:46:03AM -0500, Rob Herring wrote:
> On Tue, Sep 10, 2024 at 5:53 AM Arınç ÜNAL <arinc.unal@arinc9.com> wrote:
> >
> > Hello.
> >
> > I maintain the MediaTek DSA subdriver and some devicetree bindings and
> > source files for MediaTek hardware.
> >
> > I am especially interested in the best practices of maintaining dt-bindings
> > and DT source files.
> >
> > There's this false impression with some maintainers that, as the
> > dt-bindings and the DT source files are being hosted on the Linux
> > repository, Linux drivers have influence over the design of bindings or

> > fixing DT source files that did not comply with the bindings.

What does "fixing DT source files that did not comply" have to do with
Linux, I'm afraid I do not understand what your point is there. The
bindings are the ABI, and fixing incorrect source files would happen
regardless of how the project is hosted?

> > I'd be very interested to be involved in or kick start the efforts to take
> > dt-bindings and DT source files out of the Linux repository into its own,
> > separate repository. I believe, this would be a great step in addressing
> > all the project-dependent bindings of Linux, U-Boot, OpenWrt, and all other
> > projects, to have a single, unified repository to describe all the hardware
> > that exists in the world.
> 
> This! This is precisely why we don't move things out of the kernel.
> The kernel is the location that has the most hardware support in the
> world by far. It is not even close. Really, the only h/w missing are
> things too small to run Linux. And with all that h/w support, comes
> the people who understand the various classes of h/w. Those people are
> not going to come along to a separate project. It would be more work
> and there aren't any maintainers looking for extra work.
> 
> We already have a separate repository[1]. U-boot has recently
> incorporated it and is happily (AFAIK) using it. It happens to be
> generated from the kernel tree, but what doesn't work for you there?
> I'm happy to discuss what it needs to work better.
> 
> > I am already working towards this goal by
> > improving the dt-bindings and DT source files on the Linux repository
> > whenever I can.
> 
> That's great, still plenty of work to do there no matter what
> repository is hosting it.
> 
> > I must be quite late to make a topic suggestion but I'd be very happy to be
> > able to attend to the maintainers summit. I've already registered for the
> > Linux Plumbers Conference 2024.
> 
> This is probably not a maintainers summit topic. There's a DT BoF
> scheduled already that Krzysztof is running and supporting other
> projects is on the agenda already. I won't be there in person nor will
> I be awake at the scheduled time.

FWIW, I will be there.

> 
> Rob
> 
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/devicetree/devicetree-rebasing.git/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [MAINTAINERS SUMMIT] State of dt-bindings and DT source files, and invitation request

[Arınç ÜNAL]

On 10/09/2024 18:46, Rob Herring wrote:
> On Tue, Sep 10, 2024 at 5:53 AM Arınç ÜNAL<arinc.unal@arinc9.com> wrote:
>> Hello.
>>
>> I maintain the MediaTek DSA subdriver and some devicetree bindings and
>> source files for MediaTek hardware.
>>
>> I am especially interested in the best practices of maintaining dt-bindings
>> and DT source files.
>>
>> There's this false impression with some maintainers that, as the
>> dt-bindings and the DT source files are being hosted on the Linux
>> repository, Linux drivers have influence over the design of bindings or
>> fixing DT source files that did not comply with the bindings.
>>
>> I'd be very interested to be involved in or kick start the efforts to take
>> dt-bindings and DT source files out of the Linux repository into its own,
>> separate repository. I believe, this would be a great step in addressing
>> all the project-dependent bindings of Linux, U-Boot, OpenWrt, and all other
>> projects, to have a single, unified repository to describe all the hardware
>> that exists in the world.
> This! This is precisely why we don't move things out of the kernel.
> The kernel is the location that has the most hardware support in the
> world by far. It is not even close. Really, the only h/w missing are
> things too small to run Linux. And with all that h/w support, comes
> the people who understand the various classes of h/w. Those people are
> not going to come along to a separate project. It would be more work
> and there aren't any maintainers looking for extra work.

Understood. I can only speculate the reasons for having the device tree
binding documents and source files - let's call them device tree
definitions - on the Linux repository in the first place. It seems
convenience was a big if not the only factor.

On top of what you've pointed out, device tree source files are associated
with Linux architecture code. Therefore, there is a single menu to choose
both the architecture code and the device tree. This and the fact that
device tree source files are compiled along with the architecture code is
quite convenient. However, there is a case in which I just want to compile
a source file. I shouldn't need to have anything to do with Linux drivers
to achieve that.

> 
> We already have a separate repository[1]. U-boot has recently
> incorporated it and is happily (AFAIK) using it. It happens to be
> generated from the kernel tree, but what doesn't work for you there?
> I'm happy to discuss what it needs to work better.

The separate repository with the device tree definitions extracted is
excellent. It seems to bring the best of both worlds. I can build a single
device tree binary without having anything to do with the Linux repository.
And the device tree definitions on the Linux repository is kept so the
submissions for them still work through the means laid out by decades of
Linux and device tree development. Though, the repository seems to be
missing the functionality to validate the device tree binding documents and
source files.

However, this doesn't address the fact that device tree definitions have
nothing to do with Linux as a kernel. Because that the device tree
definitions are part of the Linux repository, on the surface, this gives
the false impression that Linux drivers have influence over the device tree
definitions. This is what I am in discontent with.

Over the course of years, I've had maintainers resisting to or completely
blocking my changes on the device tree definitions because of Linux driver
related reasons. I couldn't have patches that fixed incorrect hardware
definitions to be applied, because the maintainer would demand a change on
Linux driver to happen beforehand. I've stumbled upon misconceptions such
as thinking that a Linux driver change could break ABI. In reality, that is
nonsense because a driver change represents the implementation being
changed, not the bindings. The implementation change can only be so that it
breaks compliance with the bindings.

I understand that developers naturally become maintainers of the device
tree definitions for the hardware they've developed the driver for. So they
may not know better and fall into these false understandings. There are
currently Linux-driver-specific device tree definitions on the repository.
Device tree bindings for "DSA" switches which are actually Ethernet
switches with CPU ports. I hope to address this when I can.

I'd like to improve the Linux documentation so that I can refer to it in
the future instead of arguing the same points with folks every single time.
I'm thinking of making changes on two documents [1] [2]. I intend to
include the points below in these documents.

The device tree binding documents and device tree source files, to be
collectively referred to as the device tree definitions, on the Linux
repository are to be treated completely separately from the Linux kernel
development. The design of the device tree definitions is not to be
influenced by Linux or any other project. That means the device tree
definitions are described first, then the implementation is made by any OS
or project. Maintainers, do not block device tree patches until a patch for
a Linux driver is taken in.

> 
>> I am already working towards this goal by
>> improving the dt-bindings and DT source files on the Linux repository
>> whenever I can.
> That's great, still plenty of work to do there no matter what
> repository is hosting it.

I agree. There is serious, hard work to do here. I'm looking to turn this
into a full time job if there's such interest from Linaro to further employ
folks with experience and interest in this line of work.

> 
>> I must be quite late to make a topic suggestion but I'd be very happy to be
>> able to attend to the maintainers summit. I've already registered for the
>> Linux Plumbers Conference 2024.
> This is probably not a maintainers summit topic. There's a DT BoF
> scheduled already that Krzysztof is running and supporting other
> projects is on the agenda already. I won't be there in person nor will
> I be awake at the scheduled time.

I see it. I'll be there.

Arınç

Re: [MAINTAINERS SUMMIT] State of dt-bindings and DT source files, and invitation request

[Arınç ÜNAL]

On 11/09/2024 00:53, Conor Dooley wrote:
> On Tue, Sep 10, 2024 at 10:46:03AM -0500, Rob Herring wrote:
>> On Tue, Sep 10, 2024 at 5:53 AM Arınç ÜNAL <arinc.unal@arinc9.com> wrote:
>>>
>>> Hello.
>>>
>>> I maintain the MediaTek DSA subdriver and some devicetree bindings and
>>> source files for MediaTek hardware.
>>>
>>> I am especially interested in the best practices of maintaining dt-bindings
>>> and DT source files.
>>>
>>> There's this false impression with some maintainers that, as the
>>> dt-bindings and the DT source files are being hosted on the Linux
>>> repository, Linux drivers have influence over the design of bindings or
> 
>>> fixing DT source files that did not comply with the bindings.
> 
> What does "fixing DT source files that did not comply" have to do with
> Linux, I'm afraid I do not understand what your point is there. The
> bindings are the ABI, and fixing incorrect source files would happen
> regardless of how the project is hosted?

That's exactly what I think. I had a maintainer that argued otherwise is my
point. Which is why I want to strengthen the Linux documentation.

> 
>>> I'd be very interested to be involved in or kick start the efforts to take
>>> dt-bindings and DT source files out of the Linux repository into its own,
>>> separate repository. I believe, this would be a great step in addressing
>>> all the project-dependent bindings of Linux, U-Boot, OpenWrt, and all other
>>> projects, to have a single, unified repository to describe all the hardware
>>> that exists in the world.
>>
>> This! This is precisely why we don't move things out of the kernel.
>> The kernel is the location that has the most hardware support in the
>> world by far. It is not even close. Really, the only h/w missing are
>> things too small to run Linux. And with all that h/w support, comes
>> the people who understand the various classes of h/w. Those people are
>> not going to come along to a separate project. It would be more work
>> and there aren't any maintainers looking for extra work.
>>
>> We already have a separate repository[1]. U-boot has recently
>> incorporated it and is happily (AFAIK) using it. It happens to be
>> generated from the kernel tree, but what doesn't work for you there?
>> I'm happy to discuss what it needs to work better.
>>
>>> I am already working towards this goal by
>>> improving the dt-bindings and DT source files on the Linux repository
>>> whenever I can.
>>
>> That's great, still plenty of work to do there no matter what
>> repository is hosting it.
>>
>>> I must be quite late to make a topic suggestion but I'd be very happy to be
>>> able to attend to the maintainers summit. I've already registered for the
>>> Linux Plumbers Conference 2024.
>>
>> This is probably not a maintainers summit topic. There's a DT BoF
>> scheduled already that Krzysztof is running and supporting other
>> projects is on the agenda already. I won't be there in person nor will
>> I be awake at the scheduled time.
> 
> FWIW, I will be there.

Me too, I'd be very happy to meet you!

Arınç

Re: [MAINTAINERS SUMMIT] State of dt-bindings and DT source files, and invitation request

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 711 bytes --]

On Thu, Sep 12, 2024 at 04:01:20PM +0300, Arınç ÜNAL wrote:
> On 11/09/2024 00:53, Conor Dooley wrote:

> > What does "fixing DT source files that did not comply" have to do with
> > Linux, I'm afraid I do not understand what your point is there. The
> > bindings are the ABI, and fixing incorrect source files would happen
> > regardless of how the project is hosted?

> That's exactly what I think. I had a maintainer that argued otherwise is my
> point. Which is why I want to strengthen the Linux documentation.

TBH I'm not sure an issue like that would be particularly fixed with
documentation, that's more of a people thing given that the DT is ABI
thing is already widely communicated.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [MAINTAINERS SUMMIT] State of dt-bindings and DT source files, and invitation request

[Conor Dooley]

[-- Attachment #1: Type: text/plain, Size: 2058 bytes --]

On Thu, Sep 12, 2024 at 04:01:20PM +0300, Arınç ÜNAL wrote:
> On 11/09/2024 00:53, Conor Dooley wrote:
> > On Tue, Sep 10, 2024 at 10:46:03AM -0500, Rob Herring wrote:
> > > On Tue, Sep 10, 2024 at 5:53 AM Arınç ÜNAL <arinc.unal@arinc9.com> wrote:
> > > > 
> > > > Hello.
> > > > 
> > > > I maintain the MediaTek DSA subdriver and some devicetree bindings and
> > > > source files for MediaTek hardware.
> > > > 
> > > > I am especially interested in the best practices of maintaining dt-bindings
> > > > and DT source files.
> > > > 
> > > > There's this false impression with some maintainers that, as the
> > > > dt-bindings and the DT source files are being hosted on the Linux
> > > > repository, Linux drivers have influence over the design of bindings or
> > 
> > > > fixing DT source files that did not comply with the bindings.
> > 
> > What does "fixing DT source files that did not comply" have to do with
> > Linux, I'm afraid I do not understand what your point is there. The
> > bindings are the ABI, and fixing incorrect source files would happen
> > regardless of how the project is hosted?
> 
> That's exactly what I think. I had a maintainer that argued otherwise is my
> point. Which is why I want to strengthen the Linux documentation.

On a bunch of older platforms it's pretty common that the bindings were
lacklustre or didn't match the devicetree and kernel source code, and in
those cases (which are almost always text bindings) two outweighs one.
Ordinarily though, if the kernel or dts don't match the binding they get
adjusted, and if there are maintainers resisting this, then point them
our way.

If things have been wrong for enough time for it to affect users,
usually the correct thing to do is fix the kernel to support the
incorrect as well as the correct. The same, however, goes for other
projects: if something long established is being fixed, the other users
need to be accounted for, particularly those that automatically import
from the devicetree-rebasing repository.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [MAINTAINERS SUMMIT] State of dt-bindings and DT source files, and invitation request

[Arınç ÜNAL]

On 12/09/2024 20:05, Conor Dooley wrote:
> On Thu, Sep 12, 2024 at 04:01:20PM +0300, Arınç ÜNAL wrote:
>> On 11/09/2024 00:53, Conor Dooley wrote:
>>> On Tue, Sep 10, 2024 at 10:46:03AM -0500, Rob Herring wrote:
>>>> On Tue, Sep 10, 2024 at 5:53 AM Arınç ÜNAL <arinc.unal@arinc9.com> wrote:
>>>>>
>>>>> Hello.
>>>>>
>>>>> I maintain the MediaTek DSA subdriver and some devicetree bindings and
>>>>> source files for MediaTek hardware.
>>>>>
>>>>> I am especially interested in the best practices of maintaining dt-bindings
>>>>> and DT source files.
>>>>>
>>>>> There's this false impression with some maintainers that, as the
>>>>> dt-bindings and the DT source files are being hosted on the Linux
>>>>> repository, Linux drivers have influence over the design of bindings or
>>>
>>>>> fixing DT source files that did not comply with the bindings.
>>>
>>> What does "fixing DT source files that did not comply" have to do with
>>> Linux, I'm afraid I do not understand what your point is there. The
>>> bindings are the ABI, and fixing incorrect source files would happen
>>> regardless of how the project is hosted?
>>
>> That's exactly what I think. I had a maintainer that argued otherwise is my
>> point. Which is why I want to strengthen the Linux documentation.
> 
> On a bunch of older platforms it's pretty common that the bindings were
> lacklustre or didn't match the devicetree and kernel source code, and in
> those cases (which are almost always text bindings) two outweighs one.
> Ordinarily though, if the kernel or dts don't match the binding they get
> adjusted, and if there are maintainers resisting this, then point them
> our way.
> 
> If things have been wrong for enough time for it to affect users,
> usually the correct thing to do is fix the kernel to support the
> incorrect as well as the correct. The same, however, goes for other
> projects: if something long established is being fixed, the other users
> need to be accounted for, particularly those that automatically import
> from the devicetree-rebasing repository.

The users in mention are the Linux kernel users, of course. So a maintainer
must not demand patches fixing the kernel to support the incorrect to be
submitted before accepting device tree patches; do we agree? This is what I
mean when I say complete autonomy from the Linux kernel development.

Arınç

Re: [MAINTAINERS SUMMIT] State of dt-bindings and DT source files, and invitation request

[David Woodhouse]

[-- Attachment #1: Type: text/plain, Size: 1439 bytes --]

On Thu, 2024-09-12 at 15:57 +0300, Arınç ÜNAL wrote
> Over the course of years, I've had maintainers resisting to or completely
> blocking my changes on the device tree definitions because of Linux driver
> related reasons. I couldn't have patches that fixed incorrect hardware
> definitions to be applied, because the maintainer would demand a change on
> Linux driver to happen beforehand. I've stumbled upon misconceptions such
> as thinking that a Linux driver change could break ABI. In reality, that is
> nonsense because a driver change represents the implementation being
> changed, not the bindings. The implementation change can only be so that it
> breaks compliance with the bindings.


We should be careful here.

The device-tree bindings are the definition of the ABI. But they are
only words; what matters is the interface between the DT blob itself
and the OS drivers which interpret them.

If we want to *change* that ABI in a way which breaks users of it, then
of *course* we have to consider a transition path for those users.

That's true of *any* ABI, be it a command line, a library ABI, or the
device-tree bindings.

So where you say, "blocking my changes on the device tree definitions
because of Linux driver related reasons", that isn't necessarily wrong.
A breaking change to an ABI *needs* to have a transition plan for how
its users get from old to new without a flag day.




[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5965 bytes --]

Re: [MAINTAINERS SUMMIT] State of dt-bindings and DT source files, and invitation request

[Arınç ÜNAL]

On 13 September 2024 10:08:41 CEST, David Woodhouse <dwmw2@infradead.org> wrote:
>On Thu, 2024-09-12 at 15:57 +0300, Arınç ÜNAL wrote
>> Over the course of years, I've had maintainers resisting to or completely
>> blocking my changes on the device tree definitions because of Linux driver
>> related reasons. I couldn't have patches that fixed incorrect hardware
>> definitions to be applied, because the maintainer would demand a change on
>> Linux driver to happen beforehand. I've stumbled upon misconceptions such
>> as thinking that a Linux driver change could break ABI. In reality, that is
>> nonsense because a driver change represents the implementation being
>> changed, not the bindings. The implementation change can only be so that it
>> breaks compliance with the bindings.
>
>
>We should be careful here.
>
>The device-tree bindings are the definition of the ABI. But they are
>only words; what matters is the interface between the DT blob itself
>and the OS drivers which interpret them.
>
>If we want to *change* that ABI in a way which breaks users of it, then
>of *course* we have to consider a transition path for those users.
>
>That's true of *any* ABI, be it a command line, a library ABI, or the
>device-tree bindings.

First, let's agree on the two cases of changing the ABI. You either add
new properties and rules (let's call them definitions) to describe the
hardware more in detail, or you change the existing definitions which
would break the ABI. As it's irrelevant to my point, I'll simplify the
valid reasons to break the ABI as: The existing definitions wouldn't allow
the hardware to be properly described.

>
>So where you say, "blocking my changes on the device tree definitions
>because of Linux driver related reasons", that isn't necessarily wrong.
>A breaking change to an ABI *needs* to have a transition plan for how
>its users get from old to new without a flag day.

This is a concern for the Linux kernel. If we demand the compliance of the
Linux kernel with the changed device tree definitions from the people that
made the change, then we can't talk about a complete autonomy of the
device tree development from the Linux kernel development. I should be
able to submit patches with the only goal of adding or fixing hardware
definitions. Either I've broken the ABI with a valid reason or added
hardware definitions, I must not be forced to do Linux kernel development
for my device tree patches to be applied. I should not need to know the C
language to do device tree work. If we want more folks to do device tree
janitor work, let's not add in unnecessary requirements.

Device tree definitions are not just for being compiled into a blob for
drivers to interpret. For example, I do regularly read device tree
definitions to learn about the hardware being described. So it has a use
for documentation as well.

Arınç

Re: [MAINTAINERS SUMMIT] State of dt-bindings and DT source files, and invitation request

[Conor Dooley]

[-- Attachment #1: Type: text/plain, Size: 4200 bytes --]

On Fri, Sep 13, 2024 at 05:38:44PM +0200, Arınç ÜNAL wrote:
> On 13 September 2024 10:08:41 CEST, David Woodhouse <dwmw2@infradead.org> wrote:
> >On Thu, 2024-09-12 at 15:57 +0300, Arınç ÜNAL wrote
> >> Over the course of years, I've had maintainers resisting to or completely
> >> blocking my changes on the device tree definitions because of Linux driver
> >> related reasons. I couldn't have patches that fixed incorrect hardware
> >> definitions to be applied, because the maintainer would demand a change on
> >> Linux driver to happen beforehand. I've stumbled upon misconceptions such
> >> as thinking that a Linux driver change could break ABI. In reality, that is
> >> nonsense because a driver change represents the implementation being
> >> changed, not the bindings. The implementation change can only be so that it
> >> breaks compliance with the bindings.
> >
> >
> >We should be careful here.
> >
> >The device-tree bindings are the definition of the ABI. But they are
> >only words; what matters is the interface between the DT blob itself
> >and the OS drivers which interpret them.
> >
> >If we want to *change* that ABI in a way which breaks users of it, then
> >of *course* we have to consider a transition path for those users.
> >
> >That's true of *any* ABI, be it a command line, a library ABI, or the
> >device-tree bindings.
> 
> First, let's agree on the two cases of changing the ABI. You either add
> new properties and rules (let's call them definitions) to describe the
> hardware more in detail, or you change the existing definitions which
> would break the ABI. As it's irrelevant to my point, I'll simplify the
> valid reasons to break the ABI as: The existing definitions wouldn't allow
> the hardware to be properly described.
> 
> >
> >So where you say, "blocking my changes on the device tree definitions
> >because of Linux driver related reasons", that isn't necessarily wrong.
> >A breaking change to an ABI *needs* to have a transition plan for how
> >its users get from old to new without a flag day.
> 
> This is a concern for the Linux kernel. If we demand the compliance of the
> Linux kernel with the changed device tree definitions from the people that
> made the change, then we can't talk about a complete autonomy of the
> device tree development from the Linux kernel development. I should be
> able to submit patches with the only goal of adding or fixing hardware
> definitions. Either I've broken the ABI with a valid reason or added
> hardware definitions, I must not be forced to do Linux kernel development
> for my device tree patches to be applied.

I'm going to require that it is done, by someone, before I ack a
dt-binding change. I'm also going to require that it's not breaking
other projects, like U-Boot, that pull in the dt-rebasing repository.
If you change the ABI, and therefore change the source files, you also
need to change the users.

And I would have those requirements whether or not the bindings lie within,
or outside of, the kernel tree :) Suggesting that decoupling the bindings
and kernel would allow that to be relaxed is, in my view, silly - it
would just prevent anyone from being "blindly" able to pull in whatever
the latest revision of platform devicetree source files is, without
substantial risk of regressions. That's a downgrade from the status quo
in my view!

The far harder thing to deal with at present is ABI "compliant" changes
to devicetree source files, for example where a new dedicated clock node
replaces a set of fixed-clocks, which is something we need platform
maintainers to become more aware of as use of the dt-rebasing
repo/OF_UPSTREAM grows in U-Boot.

Cheers,
Conor.

> I should not need to know the C
> language to do device tree work. If we want more folks to do device tree
> janitor work, let's not add in unnecessary requirements.
> 
> Device tree definitions are not just for being compiled into a blob for
> drivers to interpret. For example, I do regularly read device tree
> definitions to learn about the hardware being described. So it has a use
> for documentation as well.
> 
> Arınç

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [MAINTAINERS SUMMIT] State of dt-bindings and DT source files, and invitation request

[Arınç ÜNAL]

On 13/09/2024 19:13, Conor Dooley wrote:
> On Fri, Sep 13, 2024 at 05:38:44PM +0200, Arınç ÜNAL wrote:
>> On 13 September 2024 10:08:41 CEST, David Woodhouse <dwmw2@infradead.org> wrote:
>>> On Thu, 2024-09-12 at 15:57 +0300, Arınç ÜNAL wrote
>>>> Over the course of years, I've had maintainers resisting to or completely
>>>> blocking my changes on the device tree definitions because of Linux driver
>>>> related reasons. I couldn't have patches that fixed incorrect hardware
>>>> definitions to be applied, because the maintainer would demand a change on
>>>> Linux driver to happen beforehand. I've stumbled upon misconceptions such
>>>> as thinking that a Linux driver change could break ABI. In reality, that is
>>>> nonsense because a driver change represents the implementation being
>>>> changed, not the bindings. The implementation change can only be so that it
>>>> breaks compliance with the bindings.
>>>
>>>
>>> We should be careful here.
>>>
>>> The device-tree bindings are the definition of the ABI. But they are
>>> only words; what matters is the interface between the DT blob itself
>>> and the OS drivers which interpret them.
>>>
>>> If we want to *change* that ABI in a way which breaks users of it, then
>>> of *course* we have to consider a transition path for those users.
>>>
>>> That's true of *any* ABI, be it a command line, a library ABI, or the
>>> device-tree bindings.
>>
>> First, let's agree on the two cases of changing the ABI. You either add
>> new properties and rules (let's call them definitions) to describe the
>> hardware more in detail, or you change the existing definitions which
>> would break the ABI. As it's irrelevant to my point, I'll simplify the
>> valid reasons to break the ABI as: The existing definitions wouldn't allow
>> the hardware to be properly described.
>>
>>>
>>> So where you say, "blocking my changes on the device tree definitions
>>> because of Linux driver related reasons", that isn't necessarily wrong.
>>> A breaking change to an ABI *needs* to have a transition plan for how
>>> its users get from old to new without a flag day.
>>
>> This is a concern for the Linux kernel. If we demand the compliance of the
>> Linux kernel with the changed device tree definitions from the people that
>> made the change, then we can't talk about a complete autonomy of the
>> device tree development from the Linux kernel development. I should be
>> able to submit patches with the only goal of adding or fixing hardware
>> definitions. Either I've broken the ABI with a valid reason or added
>> hardware definitions, I must not be forced to do Linux kernel development
>> for my device tree patches to be applied.
> 
> I'm going to require that it is done, by someone, before I ack a
> dt-binding change. I'm also going to require that it's not breaking
> other projects, like U-Boot, that pull in the dt-rebasing repository.
> If you change the ABI, and therefore change the source files, you also
> need to change the users.
> 
> And I would have those requirements whether or not the bindings lie within,
> or outside of, the kernel tree :) Suggesting that decoupling the bindings
> and kernel would allow that to be relaxed is, in my view, silly - it
> would just prevent anyone from being "blindly" able to pull in whatever
> the latest revision of platform devicetree source files is, without
> substantial risk of regressions. That's a downgrade from the status quo
> in my view!

I agree. I have an idea for an approach that will keep the status quo and
allow us to have proper device tree bindings. Allow changing the device
tree binding documents; require the driver to be tested and fixed only if
you're changing the device tree source files. This way, we will have proper
device tree definitions without affecting the users and have upcoming
drivers implement the proper device tree bindings. The non-compliant device
tree source files will get a nice complaint when checked with the bindings,
encouraging to fix them.

We explain on the kernel documentation that if you change the device tree
source file to be compliant with the binding document, you are required to
test the driver and let the maintainers know that there are no regressions.
If there are regressions, a patch to fix the driver must be be included on
the patch series and come after the device tree source file patch.

> 
> The far harder thing to deal with at present is ABI "compliant" changes
> to devicetree source files, for example where a new dedicated clock node
> replaces a set of fixed-clocks, which is something we need platform
> maintainers to become more aware of as use of the dt-rebasing
> repo/OF_UPSTREAM grows in U-Boot.

I think this falls under the case of breaking the ABI to be able to
properly describe the hardware. In any case, I don't see this interfere
with my suggestion.

> 
> Cheers,
> Conor.
> 
>> I should not need to know the C
>> language to do device tree work. If we want more folks to do device tree
>> janitor work, let's not add in unnecessary requirements.
>>
>> Device tree definitions are not just for being compiled into a blob for
>> drivers to interpret. For example, I do regularly read device tree
>> definitions to learn about the hardware being described. So it has a use
>> for documentation as well.
>>
>> Arınç

Arınç

Re: [MAINTAINERS SUMMIT] State of dt-bindings and DT source files, and invitation request

[Rob Herring]

On Sat, Sep 14, 2024 at 8:49 AM Arınç ÜNAL <arinc.unal@arinc9.com> wrote:
>
> On 13/09/2024 19:13, Conor Dooley wrote:
> > On Fri, Sep 13, 2024 at 05:38:44PM +0200, Arınç ÜNAL wrote:
> >> On 13 September 2024 10:08:41 CEST, David Woodhouse <dwmw2@infradead.org> wrote:
> >>> On Thu, 2024-09-12 at 15:57 +0300, Arınç ÜNAL wrote
> >>>> Over the course of years, I've had maintainers resisting to or completely
> >>>> blocking my changes on the device tree definitions because of Linux driver
> >>>> related reasons. I couldn't have patches that fixed incorrect hardware
> >>>> definitions to be applied, because the maintainer would demand a change on
> >>>> Linux driver to happen beforehand. I've stumbled upon misconceptions such
> >>>> as thinking that a Linux driver change could break ABI. In reality, that is
> >>>> nonsense because a driver change represents the implementation being
> >>>> changed, not the bindings. The implementation change can only be so that it
> >>>> breaks compliance with the bindings.
> >>>
> >>>
> >>> We should be careful here.
> >>>
> >>> The device-tree bindings are the definition of the ABI. But they are
> >>> only words; what matters is the interface between the DT blob itself
> >>> and the OS drivers which interpret them.
> >>>
> >>> If we want to *change* that ABI in a way which breaks users of it, then
> >>> of *course* we have to consider a transition path for those users.
> >>>
> >>> That's true of *any* ABI, be it a command line, a library ABI, or the
> >>> device-tree bindings.
> >>
> >> First, let's agree on the two cases of changing the ABI. You either add
> >> new properties and rules (let's call them definitions) to describe the
> >> hardware more in detail, or you change the existing definitions which
> >> would break the ABI. As it's irrelevant to my point, I'll simplify the
> >> valid reasons to break the ABI as: The existing definitions wouldn't allow
> >> the hardware to be properly described.
> >>
> >>>
> >>> So where you say, "blocking my changes on the device tree definitions
> >>> because of Linux driver related reasons", that isn't necessarily wrong.
> >>> A breaking change to an ABI *needs* to have a transition plan for how
> >>> its users get from old to new without a flag day.
> >>
> >> This is a concern for the Linux kernel. If we demand the compliance of the
> >> Linux kernel with the changed device tree definitions from the people that
> >> made the change, then we can't talk about a complete autonomy of the
> >> device tree development from the Linux kernel development. I should be
> >> able to submit patches with the only goal of adding or fixing hardware
> >> definitions. Either I've broken the ABI with a valid reason or added
> >> hardware definitions, I must not be forced to do Linux kernel development
> >> for my device tree patches to be applied.
> >
> > I'm going to require that it is done, by someone, before I ack a
> > dt-binding change. I'm also going to require that it's not breaking
> > other projects, like U-Boot, that pull in the dt-rebasing repository.
> > If you change the ABI, and therefore change the source files, you also
> > need to change the users.
> >
> > And I would have those requirements whether or not the bindings lie within,
> > or outside of, the kernel tree :) Suggesting that decoupling the bindings
> > and kernel would allow that to be relaxed is, in my view, silly - it
> > would just prevent anyone from being "blindly" able to pull in whatever
> > the latest revision of platform devicetree source files is, without
> > substantial risk of regressions. That's a downgrade from the status quo
> > in my view!
>
> I agree. I have an idea for an approach that will keep the status quo and
> allow us to have proper device tree bindings. Allow changing the device
> tree binding documents; require the driver to be tested and fixed only if
> you're changing the device tree source files. This way, we will have proper
> device tree definitions without affecting the users and have upcoming
> drivers implement the proper device tree bindings. The non-compliant device
> tree source files will get a nice complaint when checked with the bindings,
> encouraging to fix them.
>
> We explain on the kernel documentation that if you change the device tree
> source file to be compliant with the binding document, you are required to
> test the driver and let the maintainers know that there are no regressions.
> If there are regressions, a patch to fix the driver must be be included on
> the patch series and come after the device tree source file patch.

This is not how an ABI works. If you have to change both sides for
things to work, then you've broken the ABI.

With no ABI, the order you suggest for the series is backwards. The
series is not bisectable. That assumes you mean the driver change
supports a DT with and without the DT change. If not, it's just
completely broken.

Rob

Re: [MAINTAINERS SUMMIT] State of dt-bindings and DT source files, and invitation request

[Arınç ÜNAL]

On 19/09/2024 17:55, Rob Herring wrote:
> On Sat, Sep 14, 2024 at 8:49 AM Arınç ÜNAL <arinc.unal@arinc9.com> wrote:
>>
>> On 13/09/2024 19:13, Conor Dooley wrote:
>>> On Fri, Sep 13, 2024 at 05:38:44PM +0200, Arınç ÜNAL wrote:
>>>> On 13 September 2024 10:08:41 CEST, David Woodhouse <dwmw2@infradead.org> wrote:
>>>>> On Thu, 2024-09-12 at 15:57 +0300, Arınç ÜNAL wrote
>>>>>> Over the course of years, I've had maintainers resisting to or completely
>>>>>> blocking my changes on the device tree definitions because of Linux driver
>>>>>> related reasons. I couldn't have patches that fixed incorrect hardware
>>>>>> definitions to be applied, because the maintainer would demand a change on
>>>>>> Linux driver to happen beforehand. I've stumbled upon misconceptions such
>>>>>> as thinking that a Linux driver change could break ABI. In reality, that is
>>>>>> nonsense because a driver change represents the implementation being
>>>>>> changed, not the bindings. The implementation change can only be so that it
>>>>>> breaks compliance with the bindings.
>>>>>
>>>>>
>>>>> We should be careful here.
>>>>>
>>>>> The device-tree bindings are the definition of the ABI. But they are
>>>>> only words; what matters is the interface between the DT blob itself
>>>>> and the OS drivers which interpret them.
>>>>>
>>>>> If we want to *change* that ABI in a way which breaks users of it, then
>>>>> of *course* we have to consider a transition path for those users.
>>>>>
>>>>> That's true of *any* ABI, be it a command line, a library ABI, or the
>>>>> device-tree bindings.
>>>>
>>>> First, let's agree on the two cases of changing the ABI. You either add
>>>> new properties and rules (let's call them definitions) to describe the
>>>> hardware more in detail, or you change the existing definitions which
>>>> would break the ABI. As it's irrelevant to my point, I'll simplify the
>>>> valid reasons to break the ABI as: The existing definitions wouldn't allow
>>>> the hardware to be properly described.
>>>>
>>>>>
>>>>> So where you say, "blocking my changes on the device tree definitions
>>>>> because of Linux driver related reasons", that isn't necessarily wrong.
>>>>> A breaking change to an ABI *needs* to have a transition plan for how
>>>>> its users get from old to new without a flag day.
>>>>
>>>> This is a concern for the Linux kernel. If we demand the compliance of the
>>>> Linux kernel with the changed device tree definitions from the people that
>>>> made the change, then we can't talk about a complete autonomy of the
>>>> device tree development from the Linux kernel development. I should be
>>>> able to submit patches with the only goal of adding or fixing hardware
>>>> definitions. Either I've broken the ABI with a valid reason or added
>>>> hardware definitions, I must not be forced to do Linux kernel development
>>>> for my device tree patches to be applied.
>>>
>>> I'm going to require that it is done, by someone, before I ack a
>>> dt-binding change. I'm also going to require that it's not breaking
>>> other projects, like U-Boot, that pull in the dt-rebasing repository.
>>> If you change the ABI, and therefore change the source files, you also
>>> need to change the users.
>>>
>>> And I would have those requirements whether or not the bindings lie within,
>>> or outside of, the kernel tree :) Suggesting that decoupling the bindings
>>> and kernel would allow that to be relaxed is, in my view, silly - it
>>> would just prevent anyone from being "blindly" able to pull in whatever
>>> the latest revision of platform devicetree source files is, without
>>> substantial risk of regressions. That's a downgrade from the status quo
>>> in my view!
>>
>> I agree. I have an idea for an approach that will keep the status quo and
>> allow us to have proper device tree bindings. Allow changing the device
>> tree binding documents; require the driver to be tested and fixed only if
>> you're changing the device tree source files. This way, we will have proper
>> device tree definitions without affecting the users and have upcoming
>> drivers implement the proper device tree bindings. The non-compliant device
>> tree source files will get a nice complaint when checked with the bindings,
>> encouraging to fix them.
>>
>> We explain on the kernel documentation that if you change the device tree
>> source file to be compliant with the binding document, you are required to
>> test the driver and let the maintainers know that there are no regressions.
>> If there are regressions, a patch to fix the driver must be be included on
>> the patch series and come after the device tree source file patch.
> 
> This is not how an ABI works. If you have to change both sides for
> things to work, then you've broken the ABI.
> 
> With no ABI, the order you suggest for the series is backwards. The
> series is not bisectable. That assumes you mean the driver change
> supports a DT with and without the DT change. If not, it's just
> completely broken.

Thanks for pointing that out Rob. Yes, the order should be that the driver
is fixed first to comply with the binding document, then the device tree
source file is fixed to do the same. With that addressed, what's your
opinion on my proposal overall?

Arınç