[TECH TOPIC] UALink driver upstreaming

[TECH TOPIC] UALink driver upstreaming

[Sinyuk, Konstantin]

Hi All,

The UALink Consortium is defining an open, vendor‑neutral interconnect aimed
at scaling AI workloads with low‑latency, memory‑semantic communication
beyond PCIe. Unlike proprietary solutions such as NVLink (NVIDIA) or
Infinity Fabric (AMD), UALink is a cross‑vendor standard and was recently
recognized at FMS 2025.

I would like to present a proposal on what UALink support could look like in
the upstream Linux kernel.

Key areas for discussion:
- Core driver design: proposed start under drivers/misc/ual/ for discovery,
  topology, and resource management.
- Memory semantics: same‑OS and multi‑OS rack scenarios, leveraging dma_buf,
  HMM, and NUMA.
- Control path: AUX bus for vendor extensions, offloading real‑time sequences
  to device microcontrollers, generic UALink interface.
- Security: confidential compute support and a userspace daemon for topology
  and authentication.
- Upstreaming strategy: begin with a minimal core driver, then incrementally
  extend toward MM integration, dma_buf support, security, and
  cross‑subsystem work.

The goal is to decide how UALink should be represented as a first‑class
interconnect in Linux, complementing CXL while remaining vendor‑neutral,
ABI‑stable, and maintainable.

Best Regards,
Konstantin Sinyuk
Habana Labs Gaudi driver maintainer

---------------------------------------------------------------------
Intel Israel (74) Limited

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Re: [TECH TOPIC] UALink driver upstreaming

[dan.j.williams]

Sinyuk, Konstantin wrote:
> Hi All,
> 
> The UALink Consortium is defining an open, vendor‑neutral interconnect aimed
> at scaling AI workloads with low‑latency, memory‑semantic communication
> beyond PCIe. Unlike proprietary solutions such as NVLink (NVIDIA) or
> Infinity Fabric (AMD), UALink is a cross‑vendor standard and was recently
> recognized at FMS 2025.
> 
> I would like to present a proposal on what UALink support could look like in
> the upstream Linux kernel.
> 
> Key areas for discussion:
> - Core driver design: proposed start under drivers/misc/ual/ for discovery,
>   topology, and resource management.
> - Memory semantics: same‑OS and multi‑OS rack scenarios, leveraging dma_buf,
>   HMM, and NUMA.
> - Control path: AUX bus for vendor extensions, offloading real‑time sequences
>   to device microcontrollers, generic UALink interface.
> - Security: confidential compute support and a userspace daemon for topology
>   and authentication.
> - Upstreaming strategy: begin with a minimal core driver, then incrementally
>   extend toward MM integration, dma_buf support, security, and
>   cross‑subsystem work.
> 
> The goal is to decide how UALink should be represented as a first‑class
> interconnect in Linux, complementing CXL while remaining vendor‑neutral,
> ABI‑stable, and maintainable.

Hi Konstantin,

Please do also consider submitting this as a topic for the Device Memory
Microconference (https://lpc.events/event/19/contributions/2009/).

Many of folks with experience with CXL, NVLink, Infinity Fabric, RDMA,
HMM, NUMA, dma_buf, Confidential Computing etc... will also be in the
room for a focused topic like this.

Re: [TECH TOPIC] UALink driver upstreaming

[Greg KH]

On Wed, Sep 10, 2025 at 10:37:03PM +0300, Sinyuk, Konstantin wrote:
> Hi All,
> 
> The UALink Consortium is defining an open, vendor‑neutral interconnect aimed
> at scaling AI workloads with low‑latency, memory‑semantic communication
> beyond PCIe. Unlike proprietary solutions such as NVLink (NVIDIA) or
> Infinity Fabric (AMD), UALink is a cross‑vendor standard and was recently
> recognized at FMS 2025.
> 
> I would like to present a proposal on what UALink support could look like in
> the upstream Linux kernel.
> 
> Key areas for discussion:
> - Core driver design: proposed start under drivers/misc/ual/ for discovery,
>   topology, and resource management.
> - Memory semantics: same‑OS and multi‑OS rack scenarios, leveraging dma_buf,
>   HMM, and NUMA.
> - Control path: AUX bus for vendor extensions, offloading real‑time sequences
>   to device microcontrollers, generic UALink interface.
> - Security: confidential compute support and a userspace daemon for topology
>   and authentication.
> - Upstreaming strategy: begin with a minimal core driver, then incrementally
>   extend toward MM integration, dma_buf support, security, and
>   cross‑subsystem work.

Do you have patches today for this new bus?  Why not start submitting
them already?  Why wait till December?

> ---------------------------------------------------------------------
> Intel Israel (74) Limited
> 
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.

You are going to have to fix that footer up to be able to send anything
to public lists :)

thanks,

greg k-h

Re: [TECH TOPIC] UALink driver upstreaming

[Mimi Zohar]

On Wed, 2025-09-10 at 22:37 +0300, Sinyuk, Konstantin wrote:
> Hi All,
> 
> The UALink Consortium is defining an open, vendor‑neutral interconnect aimed
> at scaling AI workloads with low‑latency, memory‑semantic communication
> beyond PCIe. Unlike proprietary solutions such as NVLink (NVIDIA) or
> Infinity Fabric (AMD), UALink is a cross‑vendor standard and was recently
> recognized at FMS 2025.
> 
> I would like to present a proposal on what UALink support could look like in
> the upstream Linux kernel.
> 
> Key areas for discussion:
> - Core driver design: proposed start under drivers/misc/ual/ for discovery,
>   topology, and resource management.
> - Memory semantics: same‑OS and multi‑OS rack scenarios, leveraging dma_buf,
>   HMM, and NUMA.
> - Control path: AUX bus for vendor extensions, offloading real‑time sequences
>   to device microcontrollers, generic UALink interface.
> - Security: confidential compute support and a userspace daemon for topology
>   and authentication.
> - Upstreaming strategy: begin with a minimal core driver, then incrementally
>   extend toward MM integration, dma_buf support, security, and
>   cross‑subsystem work.
> 
> The goal is to decide how UALink should be represented as a first‑class
> interconnect in Linux, complementing CXL while remaining vendor‑neutral,
> ABI‑stable, and maintainable.

Hi Konstantin,

It would be nice to have all the Linux Security Module (LSM) hooks in all the
right places from the beginning for integrity (e.g. signature verification).

thanks,

Mimi

Re: FW: [TECH TOPIC] UALink driver upstreaming

[Konstantin Sinyuk]

On Thu, Sep 11, 2025 at 08:12:45AM +0200, Mimi Zohar wrote:
> Hi Konstantin,
>
> It would be nice to have all the Linux Security Module (LSM) hooks in
> all the right places from the beginning for integrity (e.g. signature
> verification).

Hi Mimi,

Good idea, thanks for pointing that out. We'll ensure appropriate LSM
hooks are considered from the initial design.

Best Regards,
 Konstantin.

Re: [TECH TOPIC] UALink driver upstreaming

[Konstantin Sinyuk]

On Thu, Sep 11, 2025 at 08:12:45AM +0200, Greg Kroah-Hartman wrote:
> Do you have patches today for this new bus? Why not start submitting
> them already? Why wait till December?
>
> You are going to have to fix that footer up to be able to send
> anything to public lists :)

Hi Greg,

Good idea, we'll prioritize cleanup so an RFC skeleton can go out before
December for early feedback.

For the footer: understood. I'm on personal Gmail now and will switch to
proper SMTP config once it's set up.

Best Regards,
 Konstantin

Re: [TECH TOPIC] UALink driver upstreaming

[Linus Walleij]

On Wed, Sep 10, 2025 at 9:37 PM Sinyuk, Konstantin
<konstantin.sinyuk@intel.com> wrote:

> - Core driver design: proposed start under drivers/misc/ual/ for discovery,
>   topology, and resource management.

So this gives at hand that since this is no "ordinary" memory-mapped
driver, it needs its own bus and therefore intuitively its own subsystem?

What about drivers/accel/ual where other accelerators live?

Or if that is somehow inappropriate, just drivers/ual, don't be shy.

Yours,
Linus Walleij

Re: [TECH TOPIC] UALink driver upstreaming

[Konstantin Sinyuk]

On Thu, Sep 11, 2025 at 10:42:01AM +0200, Linus Walleij wrote:
> So this gives at hand that since this is no "ordinary" memory-mapped
> driver, it needs its own bus and therefore intuitively its own subsystem?
>
> What about drivers/accel/ual where other accelerators live?
>
> Or if that is somehow inappropriate, just drivers/ual, don't be shy.

Hi Linus,

For the initial RFC, I thought to start under drivers/misc/ just as a
lightweight entry point to get early review. But I agree with you that
UALink fits more naturally as its own subsystem, similar to how CXL is
handled, rather than being grouped under accel. The long-term plan
should definitely be drivers/ual/.

Best Regards,
 Konstantin

Re: [TECH TOPIC] UALink driver upstreaming

[Leon Romanovsky]

On Thu, Sep 11, 2025, at 21:10, Konstantin Sinyuk wrote:
> On Thu, Sep 11, 2025 at 10:42:01AM +0200, Linus Walleij wrote:
>> So this gives at hand that since this is no "ordinary" memory-mapped
>> driver, it needs its own bus and therefore intuitively its own subsystem?
>>
>> What about drivers/accel/ual where other accelerators live?
>>
>> Or if that is somehow inappropriate, just drivers/ual, don't be shy.
>
> Hi Linus,
>
> For the initial RFC, I thought to start under drivers/misc/ just as a
> lightweight entry point to get early review. But I agree with you that
> UALink fits more naturally as its own subsystem, similar to how CXL is
> handled, rather than being grouped under accel. The long-term plan
> should definitely be drivers/ual/.

So do it right from the beginning and save from us unnecessary review iterations.

Thanks 

>
> Best Regards,
>  Konstantin

Re: [TECH TOPIC] UALink driver upstreaming

[Linus Walleij]

On Thu, Sep 11, 2025 at 8:10 PM Konstantin Sinyuk <sinyuk@gmail.com> wrote:
> On Thu, Sep 11, 2025 at 10:42:01AM +0200, Linus Walleij wrote:

> > So this gives at hand that since this is no "ordinary" memory-mapped
> > driver, it needs its own bus and therefore intuitively its own subsystem?
> >
> > What about drivers/accel/ual where other accelerators live?
> >
> > Or if that is somehow inappropriate, just drivers/ual, don't be shy.
>
> Hi Linus,
>
> For the initial RFC, I thought to start under drivers/misc/ just as a
> lightweight entry point to get early review. But I agree with you that
> UALink fits more naturally as its own subsystem, similar to how CXL is
> handled, rather than being grouped under accel. The long-term plan
> should definitely be drivers/ual/.

If you want to "ease in" drivers the appropriate place is drivers/staging.

But if you have a focused team and you are going to start small
and work on this then just use drivers/accel/ual from day 1.

Yours,
Linus Walleij

Re: [TECH TOPIC] UALink driver upstreaming

[Greg KH]

On Fri, Sep 12, 2025 at 09:22:29AM +0200, Linus Walleij wrote:
> On Thu, Sep 11, 2025 at 8:10 PM Konstantin Sinyuk <sinyuk@gmail.com> wrote:
> > On Thu, Sep 11, 2025 at 10:42:01AM +0200, Linus Walleij wrote:
> 
> > > So this gives at hand that since this is no "ordinary" memory-mapped
> > > driver, it needs its own bus and therefore intuitively its own subsystem?
> > >
> > > What about drivers/accel/ual where other accelerators live?
> > >
> > > Or if that is somehow inappropriate, just drivers/ual, don't be shy.
> >
> > Hi Linus,
> >
> > For the initial RFC, I thought to start under drivers/misc/ just as a
> > lightweight entry point to get early review. But I agree with you that
> > UALink fits more naturally as its own subsystem, similar to how CXL is
> > handled, rather than being grouped under accel. The long-term plan
> > should definitely be drivers/ual/.
> 
> If you want to "ease in" drivers the appropriate place is drivers/staging.

Not really.  Staging is for "stuff that is not cleaned up yet and we
want to do that work in-tree".  It's best for existing code bases that
have been around for a while to get dropped in there and take advantage
of people wanting to do simple "first task" type of kernel patches
(coding style cleanups, shim layer removals, spelling fixes.)

It's almost never a good idea to use staging for a new subsystem as the
work it takes to get it merged out of staging is almost always more than
it would be to just do "all of the coding style cleanups first" out of
the tree and then merge it properly.

So I do not recommend staging for anything with an "active" developer
community, as it would just slow down the acceptance of the code to the
real part of the kernel.

> But if you have a focused team and you are going to start small
> and work on this then just use drivers/accel/ual from day 1.

Totally agreed.

thanks,

greg k-h

Re: [TECH TOPIC] UALink driver upstreaming

[Konstantin Sinyuk]

On Fri, Sep 12, 2025 at 09:22:29AM +0200, Linus Walleij wrote:
> On Thu, Sep 11, 2025 at 8:10 PM Konstantin Sinyuk <sinyuk@gmail.com> wrote:
>> On Thu, Sep 11, 2025 at 10:42:01AM +0200, Linus Walleij wrote:
>
>>> So this gives at hand that since this is no "ordinary" memory-mapped
>>> driver, it needs its own bus and therefore intuitively its own subsystem?
>>>
>>> What about drivers/accel/ual where other accelerators live?
>>>
>>> Or if that is somehow inappropriate, just drivers/ual, don't be shy.
>>
>> Hi Linus,
>>
>> For the initial RFC, I thought to start under drivers/misc/ just as a
>> lightweight entry point to get early review. But I agree with you that
>> UALink fits more naturally as its own subsystem, similar to how CXL is
>> handled, rather than being grouped under accel. The long-term plan
>> should definitely be drivers/ual/.
>
> If you want to "ease in" drivers the appropriate place is drivers/staging.

Not really.  Staging is for "stuff that is not cleaned up yet and we
want to do that work in-tree".  It's best for existing code bases that
have been around for a while to get dropped in there and take advantage
of people wanting to do simple "first task" type of kernel patches
(coding style cleanups, shim layer removals, spelling fixes.)

It's almost never a good idea to use staging for a new subsystem as the
work it takes to get it merged out of staging is almost always more than
it would be to just do "all of the coding style cleanups first" out of
the tree and then merge it properly.

So I do not recommend staging for anything with an "active" developer
community, as it would just slow down the acceptance of the code to the
real part of the kernel.

> But if you have a focused team and you are going to start small
> and work on this then just use drivers/accel/ual from day 1.

Totally agreed.

---

Hi Greg, Linus,

Thanks for clarifying. I agree staging and misc are not good homes.

Strictly speaking, UALink is an interconnect fabric (rack‑scale memory
semantic bus, closer to how CXL is structured) rather than an
accelerator device driver. The existing accel/ drivers (Gaudi, QAIC,
ivpu) are compute engines, while UAL should provide a cross‑device bus
framework, so accel/ is not a perfect architectural fit.

That said, I understand the concern about creating a top‑level
drivers/ual/ directory too early. Starting small under
drivers/accel/ual/ is clearly better than misc/, and I will coordinate
with Oded  to make sure it integrates cleanly there.

Longer term, as UAL adoption grows and multiple vendors hook into the
framework, the natural home would be a dedicated drivers/ual/, just as
CXL evolved into its own subsystem.

Best Regards,
 Konstantin

Re: [TECH TOPIC] UALink driver upstreaming

[Linus Walleij]

On Fri, Sep 12, 2025 at 2:07 PM Konstantin Sinyuk <sinyuk@gmail.com> wrote:

> Longer term, as UAL adoption grows and multiple vendors hook into the
> framework, the natural home would be a dedicated drivers/ual/, just as
> CXL evolved into its own subsystem.

If you already know there will be other things than accelerators there,
so it's more like, i don't know, PCI or greybus, then put it into its own
subsystem in drivers/ualink from day one, I think drivers/ual is a bit
terse, the world is full och TLA:s already, also that is its actual name
isn't it? Hard to miss if someone is looking for it.

Your merges can still go through some submaintainer like Greg, or
DRM, if they need some shepherding to start out. After all that's how
drivers/accel is managed, through DRM.

Just my €0.01.

Yours,
Linus Walleij

Re: [TECH TOPIC] UALink driver upstreaming

[Rodrigo Vivi]

On Fri, Sep 12, 2025 at 02:18:40PM +0200, Linus Walleij wrote:
> On Fri, Sep 12, 2025 at 2:07 PM Konstantin Sinyuk <sinyuk@gmail.com> wrote:
> 
> > Longer term, as UAL adoption grows and multiple vendors hook into the
> > framework, the natural home would be a dedicated drivers/ual/, just as
> > CXL evolved into its own subsystem.
> 
> If you already know there will be other things than accelerators there,
> so it's more like, i don't know, PCI or greybus, then put it into its own
> subsystem in drivers/ualink from day one, I think drivers/ual is a bit
> terse, the world is full och TLA:s already, also that is its actual name
> isn't it? Hard to miss if someone is looking for it.
> 
> Your merges can still go through some submaintainer like Greg, or
> DRM, if they need some shepherding to start out. After all that's how
> drivers/accel is managed, through DRM.

I agree it should be drivers/ualink from day 1.

I like the idea of the flow through drm.

> 
> Just my €0.01.
> 
> Yours,
> Linus Walleij
> 

Re: [TECH TOPIC] UALink driver upstreaming

[Matthew Brost]

On Fri, Sep 12, 2025 at 09:49:56AM -0400, Rodrigo Vivi wrote:
> On Fri, Sep 12, 2025 at 02:18:40PM +0200, Linus Walleij wrote:
> > On Fri, Sep 12, 2025 at 2:07 PM Konstantin Sinyuk <sinyuk@gmail.com> wrote:
> > 
> > > Longer term, as UAL adoption grows and multiple vendors hook into the
> > > framework, the natural home would be a dedicated drivers/ual/, just as
> > > CXL evolved into its own subsystem.
> > 
> > If you already know there will be other things than accelerators there,
> > so it's more like, i don't know, PCI or greybus, then put it into its own
> > subsystem in drivers/ualink from day one, I think drivers/ual is a bit
> > terse, the world is full och TLA:s already, also that is its actual name
> > isn't it? Hard to miss if someone is looking for it.
> > 
> > Your merges can still go through some submaintainer like Greg, or
> > DRM, if they need some shepherding to start out. After all that's how
> > drivers/accel is managed, through DRM.
> 
> I agree it should be drivers/ualink from day 1.
> 
> I like the idea of the flow through drm.
> 

+1 to both. I like the idea of creating a mini subsystem for ualink,
going through DRM. With that, let's include dri-devel on all future
ualink discussions.

Matt

> > 
> > Just my €0.01.
> > 
> > Yours,
> > Linus Walleij
> > 

Re: [TECH TOPIC] UALink driver upstreaming

[Jason Gunthorpe]

On Fri, Sep 12, 2025 at 09:13:23AM -0700, Matthew Brost wrote:
> On Fri, Sep 12, 2025 at 09:49:56AM -0400, Rodrigo Vivi wrote:
> > On Fri, Sep 12, 2025 at 02:18:40PM +0200, Linus Walleij wrote:
> > > On Fri, Sep 12, 2025 at 2:07 PM Konstantin Sinyuk <sinyuk@gmail.com> wrote:
> > > 
> > > > Longer term, as UAL adoption grows and multiple vendors hook into the
> > > > framework, the natural home would be a dedicated drivers/ual/, just as
> > > > CXL evolved into its own subsystem.
> > > 
> > > If you already know there will be other things than accelerators there,
> > > so it's more like, i don't know, PCI or greybus, then put it into its own
> > > subsystem in drivers/ualink from day one, I think drivers/ual is a bit
> > > terse, the world is full och TLA:s already, also that is its actual name
> > > isn't it? Hard to miss if someone is looking for it.
> > > 
> > > Your merges can still go through some submaintainer like Greg, or
> > > DRM, if they need some shepherding to start out. After all that's how
> > > drivers/accel is managed, through DRM.
> > 
> > I agree it should be drivers/ualink from day 1.
> > 
> > I like the idea of the flow through drm.
> > 
> 
> +1 to both. I like the idea of creating a mini subsystem for ualink,
> going through DRM. With that, let's include dri-devel on all future
> ualink discussions.

Something like ualink has a lot less to do with DRM than it does with
networking and RDMA. These things are not "CPU buses" like PCI, they
are full multi-host network fabrics with switches, network addressing,
fabric management, and so on.

IMHO what principally distinguishes ualink from something like
classical RDMA is the expectation that the RMAs are triggered directly
in accelerator HW through load/store operations, and not from some
userspace CPU application in Linux.

But I would still expect to see the usual range of networky UAPIs
around link state, phy debugging, telemetry and general management
operations.

A normal subsystem seems like the right thing, but don't flow it
through DRM, just send it to Linus.

I think my main ask would be to have an intree user driver before
sending anything. Too much of this space relies on out of tree
accelerator drivers :(

Jason