Re: [Ksummit-discuss] [TECH TOPIC] Sensors and similar - subsystem interactions, divisions, bindings etc.

[Laurent Pinchart <laurent.pinchart@ideasonboard.com>]

Hi Hans,

On Friday 29 Jul 2016 09:28:06 Hans Verkuil wrote:
> On 07/28/2016 06:42 PM, Laurent Pinchart wrote:
> > On Thursday 28 Jul 2016 08:58:46 Mauro Carvalho Chehab wrote:
> >> Em Wed, 20 Jul 2016 22:18:11 +0100 Jonathan Cameron escreveu:
> >>> Hi All,
> >>> 
> >>> This topic would be around the way the various subsystems interact, in
> >>> the rough area of 'sensors' (I haven't yet had much of an issue with
> >>> subsystem crossing with output devices but maybe that's just over the
> >>> next hill!)
> >> 
> >> Yeah, there is a gray area here as devices become more complex.
> >> So, I'm interested on such topic.
> >> 
> >>> Scope may well be wider but includes:
> >>> * input (some of it)
> >>> * hwmon
> >>> * iio
> >>> * comedi(?)
> >>> * thermal
> >>> * power/battery
> >>> * gpio - the blur between gpios and beaglescope / PLC type I/O.
> >>> * v4l - when does a device jump from being a multi pixel thermopile
> >>> to a thermal camera? Smart fingerprint scanners etc.  Gesture
> >>> recognition sensors (effectively 9ish pixel cameras)
> >> 
> >> For devices that provide images somehow, I'd say that the best would be
> >> to implement them as via the V4L2 API.
> > 
> > This requires defining what an image is. Furthermore, we very low image
> > resolution devices, we will need to deal with high frame rates (1k-10k is
> > a range we need to consider). The V4L2 API will likely show performance
> > issues.
>
> The main problems there are the limit to the number of buffers (32) in v4l2
> and that the number of qbuf/dqbuf ioctls is large for these high
> framerates. You would probably want to introduce mqbuf/mdqbuf ioctls to
> queue/dequeue multiple buffers. The core internals of v4l2 (vb2) are all
> pretty low overhead.

Or, just brainstorming, using some kind of ring-buffer based API that doesn't 
require context switches between userspace and kernelspace. That could also be 
somehow useful for larger images, as low-latency video streaming could benefit 
from the ability to start encoding the image before it's fully captured (or 
the other way around, we could start consuming it before it's fully decoded).

> >> We'll likely need to discuss it case by case, specially for early cross-
> >> subsystem drivers, though, as it is not trivial to identify the subsystem
> >> boundaries, and sometimes multiple APIs from different subsystems is
> >> needed. Also, it is not clear where such drivers would fit at the Kernel
> >> tree.
> >> 
> >> One interesting case is an input device driver with multi-finger
> >> support (drivers/input/touchscreen/sur40.c). I suspect we'll have
> >> more cases like that over time.
> >> 
> >>> * Lots of random things we haven't seen yet.

-- 
Regards,

Laurent Pinchart