Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] & [TECH TOPIC] Improve regression tracking

[Laurent Pinchart <laurent.pinchart@ideasonboard.com>]

On Wednesday 05 Jul 2017 12:10:00 Steven Rostedt wrote:
> On Wed, 5 Jul 2017 11:08:48 -0400 Carlos O'Donell wrote:
> > On 07/05/2017 10:33 AM, Steven Rostedt wrote:
> > > No test should be written for a single specific hardware. It should be a
> > > general functionality that different hardware can execute.
> > 
> > Why? We test all sorts of hardware in userspace and we see value in that
> > testing.
> 
> One reason is for bit rot. I'm not totally against it. But I envision
> that if we have hundreds of tests for very specific pieces of hardware,
> it's value will diminish over time. Unless we can get a good
> infrastructure written where the hardware info is more of a data sheet
> then a single test itself.

That's all nice, but when the hardware is complex and not fully abstracted 
behind a kernel API, tests are bound to be hardware-specific. Of course, a bug 
or regression observed only with a specific device, but triggered through the 
usage of abstract APIs only, can lead to a test case written for that device 
but runnable with any device in the same category. In that case the test case 
should certainly be added to a test suite for the corresponding API/subsystem, 
not to an hidden test suite for a particular device.

-- 
Regards,

Laurent Pinchart