Re: Memory management facing a 400Gpbs network link

[Christopher Lameter <cl@linux.com>]

On Tue, 19 Feb 2019, Michal Hocko wrote:

> On Tue 12-02-19 18:25:50, Cristopher Lameter wrote:
> > 400G Infiniband will become available this year. This means that the data
> > ingest speeds can be higher than the bandwidth of the processor
> > interacting with its own memory.
> >
> > For example a single hardware thread is limited to 20Gbyte/sec whereas the
> > network interface provides 50Gbytes/sec. These rates can only be obtained
> > currently with pinned memory.
> >
> > How can we evolve the memory management subsystem to operate at higher
> > speeds with more the comforts of paging and system calls that we are used
> > to?
>
> Realistically, is there anything we _can_ do when the HW is the
> bottleneck?

Well the hardware is one problem. The problem that a single core cannot
handle the full memory bandwidth can be solved by spreading the
processing of the data to multiple processors. So I think the memory
subsystem could be aware of that? How do we load balance between cores so
that we can handle the full bandwidth?

The other is that the memory needs to be pinned and all sorts of special
measures and tuning needs to be done to make this actually work. Is there
any way to simplify this?

Also the need for page pinning becomes a problem since the majority of the
memory of a system would need to be pinned. Actually the application seems
to be doing the memory management then?