Re: Memory management facing a 400Gpbs network link

[Christopher Lameter <cl@linux.com>]

On Tue, 19 Feb 2019, Michal Hocko wrote:

> > Well the hardware is one problem. The problem that a single core cannot
> > handle the full memory bandwidth can be solved by spreading the
> > processing of the data to multiple processors. So I think the memory
> > subsystem could be aware of that? How do we load balance between cores so
> > that we can handle the full bandwidth?
>
> Isn't that something that poeple already do from userspace?

Yes. We can certainly do a lot from userspace manually but this is hard
and involves working around memory management to some extend. The higher
the I/O bandwidth become the more memory management becomes not that
useful anymore.

Can we improve the situation? A 2M VM was repeatedly discussed f.e.

Or some kind of memory management extension that allows working with large
contiguous blocks of memory? Which are problematic in their own
because large contiguous blocks may not be obtainable due to
fragmentation. Therefore the need to reboot the system if the
load changes.

> > The other is that the memory needs to be pinned and all sorts of special
> > measures and tuning needs to be done to make this actually work. Is there
> > any way to simplify this?
> >
> > Also the need for page pinning becomes a problem since the majority of the
> > memory of a system would need to be pinned. Actually the application seems
> > to be doing the memory management then?
>
> I am sorry but this still sounds too vague. There are certainly
> possibilities to handle part the MM functionality in the userspace.
> But why should we discuss that at the MM track. Do you envision any
> in kernel changes that would be needed?

Without adapting to these trends memory management may become just a
part of the system that is mainly useful for running executables, handling
configuration files etc but not for handling the data going through the
system.

We end up with data fully bypassing the kernel. Its difficult to handle
that way.

Sorry this is fuzzy. I wonder if there are other solutions than those
that I know of for these issues. The solutions mostly mean going directly
to hardware because the performance is just not available if the kernel is
involved. If that is unavoidable then we need clean APIs to be able to
carve out memory for these needs.

I can make this more concrete by listing some of the approaches that I am
seeing?

F.e.

A 400G NIC has the ability to route traffic to certain endpoints on
specific cores. Thus traffic volume can be segmented into multiple
streams that are able to be handled by single cores. However, many
data streams (video, audio) have implicit ordering constraints between
packets.