Re: [Ksummit-discuss] [TOPIC] userspace infrastructure services

[James Bottomley <James.Bottomley@HansenPartnership.com>]

On Wed, 2015-08-12 at 11:21 -0700, Christoph Hellwig wrote:
> On Wed, Aug 12, 2015 at 11:15:52AM -0700, Stephen Hemminger wrote:
> > I expected someone else to bring this topic up...
> > 
> > One thing that is happening is that there is lots of activity in moving core features
> > of the kernel into userspace (networking, storage, security). I don't want to get into
> > an argument over whether that is good or bad;
> 
> Which I think is the most important question of them all.  From all
> the pain caused and the little gains I'm a believer it's mostly bad,
> especially for cases like:
> 
> > The area I am most familiar with is the DPDK which has to have its own UIO drivers
> > to work in all environments (get device into userspace). And then has to have its
> > own drivers to simulate network device (put device back into kernel).  This leads
> > to unmanageable ABI and development technical debt.
> 
> Which interacts on both sides.  In that case "let them suffer for their
> stupidity" is the only valid answer.
> 
> For a case where it's more reasonable there might be a different answer.

What about something like CRIU:

http://www.criu.org/Main_Page

?

in-kernel checkpoint/restore had been proposed many times before and
always rejected as being hugely complex (most publicly at the Kernel
Summit in Cambridge, where Oren Laadan proposed it).  CRIU took a
completely different approach where we broke the problem down into
sections and extended kernel interfaces to export state where it wasn't
available.  The net result is that we now have checkpoint/restore (and
thus process migration) for Linux at a significant reduction in the
complexity over the original code and realistically not a huge technical
debt.

What I like to think CRIU shows us is that if we look hard enough, there
is a way to separate the problem up so that a userspace becomes workable
without a massive increase in the kernel ABI surface.

The big problem is finding the way to do the split.  You can tell if you
get it wrong because the userspace component has a huge amount of state
shared with the kernel and the ABI balloons to cope with this (cough,
DPDK, cough)... however, what's not clear is how to get it right in the
first place.

Perhaps we should look first at how CRIU got it right and then discuss
how we might use those same principles to fix other projects, like DPDK.

James