Re: [LSF/MM/BPF TOPIC] block drivers in user space

[Gabriel Krisman Bertazi <krisman@collabora.com>]

Ming Lei <ming.lei@redhat.com> writes:

>> I was thinking of something like this, or having a way for the server to
>> only operate on the fds and do splice/sendfile.  But, I don't know if it
>> would be useful for many use cases.  We also want to be able to send the
>> data to userspace, for instance, for userspace networking.
>
> I understand the big point is that how to pass the io data to ubd driver's
> request/bio pages. But splice/sendfile just transfers data between two FDs,
> then how can the block request/bio's pages get filled with expected data?
> Can you explain a bit in detail?

Hi Ming,

My idea was to split the control and dataplanes in different file
descriptors.

A queue has a fd that is mapped to a shared memory area where the
request descriptors are.  Submission/completion are done by read/writing
the index of the request on the shared memory area.

For the data plane, each request descriptor in the queue has an
associated file descriptor to be used for data transfer, which is
preallocated at queue creation time.  I'm mapping the bio linearly, from
offset 0, on these descriptors on .queue_rq().  Userspace operates on
these data file descriptors with regular RW syscalls, direct splice to
another fd or pipe, or mmap it to move data around. The data is
available on that fd until IO is completed through the queue fd.  After
an operation is completed, the fds are reused for the next IO on that
queue position.

Hannes has pointed out the issues with fd limits. :)

> If block layer is bypassed, it won't be exposed as block disk to userspace.

I implemented it as a block-mq driver, but it still only supports one
queue.

-- 
Gabriel Krisman Bertazi