From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 691B294C for ; Tue, 12 Jul 2016 09:35:22 +0000 (UTC) Received: from mx2.suse.de (mx2.suse.de [195.135.220.15]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id C789FA1 for ; Tue, 12 Jul 2016 09:35:21 +0000 (UTC) Date: Tue, 12 Jul 2016 11:35:18 +0200 From: Jan Kara To: Dan Williams Message-ID: <20160712093518.GD16460@quack2.suse.cz> References: <91774112.AKkGksYjl6@vostro.rjw.lan> <20160709004352.GK28589@dtor-ws> <1468058721.2557.9.camel@HansenPartnership.com> <0ED98206-0A66-48A4-B5A4-A0BC53FDBF05@primarydata.com> <1468114447.2333.12.camel@HansenPartnership.com> <1468115770.2333.15.camel@HansenPartnership.com> <718BE1FD-6169-4205-A905-53F997D5943A@primarydata.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Cc: James Bottomley , Trond Myklebust , "ksummit-discuss@lists.linuxfoundation.org" Subject: Re: [Ksummit-discuss] [CORE TOPIC] kernel unit testing List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , On Mon 11-07-16 12:58:50, Dan Williams wrote: > * What does "we never regress userspace" mean when we have unit tests > that are tightly coupled to the kernel? > For example, there have been occasions where I have "regressed" a test > case to improve kernel behavior with the knowledge that no real > (non-test) application was dependent on the old behavior. It's always a judgement call. Linus has always stated that the rule only holds for real users using real applications. So if you are confident that it is only the test that regresses, then that is fine. > * Are tests only for developers, or should an end consumer of the > kernel expect to be able to run the tests and baseline their kernel? > The unit tests for libnvdimm [2] are meant to run against latest > mainline. There's some support for checking the kernel version, but I > don't promise that tests for new kernel functionality will be skipped > on older kernels without that specific enabling. I'd revisit that > stance if -stable tree maintainers or distros were looking to run the > tests. Well, in the end it is your call but my experience with xfstests shows that it is good to detect the case that the functionality is not supported by the kernel and fail gracefully. Because especially with enterprise distributions you get a strange mix of kernel and userspace and it is *very* useful to be able to run testsuites there when testing backports etc. It is a pita to figure out whether the test failed because of missing functionality of because of a bug in your backport... Honza -- Jan Kara SUSE Labs, CR