From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id B59D8E8F for ; Fri, 7 Sep 2018 21:06:38 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 5D8047C6 for ; Fri, 7 Sep 2018 21:06:38 +0000 (UTC) Date: Fri, 7 Sep 2018 18:06:33 -0300 From: Mauro Carvalho Chehab To: Daniel Vetter Message-ID: <20180907180633.564b798c@coco.lan> In-Reply-To: References: <20180904201620.GC16300@sasha-vm> <20180905101710.73137669@gandalf.local.home> <20180907004944.GD16300@sasha-vm> <20180907014930.GE16300@sasha-vm> <20180907042754.GL5098@thunk.org> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: ksummit Subject: Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Bug-introducing patches List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Em Fri, 7 Sep 2018 11:13:20 +0200 Daniel Vetter escreveu: > On Fri, Sep 7, 2018 at 6:27 AM, Theodore Y. Ts'o wrote: > > On Fri, Sep 07, 2018 at 01:49:31AM +0000, Sasha Levin via Ksummit-discuss wrote: > > There actually is a perverse incentive to having all of the test > > 'bots, which is that I suspect some people have come to rely on it to > > catch problems. I generally run a full set of regression tests before > > I push an update to git.kernel.org (it only takes about 2 hours, and > > 12 VM's :-); and by the time we get to the late -rc's I *always* will > > do a full regression test. > > This is what imo a well-run subsystem should sound like from a testing > pov. All the subsystem specific testing should be done before merging. > Post-merge is only for integration testing and catching the long-tail > issues that need months/years of machine time to surface. > > Of course this is much harder for anything that needs physical > hardware, but even for driver subsystems there's lots you can do with > test-drivers, selftests and a pile of emulation, to at least catch > bugs in generic code. And for reasonably sized teams like drm/i915 > building a proper CI is a very obvious investement that will pay off. IMHO, CI would do even a better job for smaller teams, as they won't have much resources for testing, but the problem here is that those teams probably lack resources and money to invest on a physical hardware to setup a CI infra and to buy the myriad of different hardware to do regression testing. Also, some devices are harder to test: how would you check if a camera microphone is working? How to check if the camera captured images are ok? Thanks, Mauro