From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32621C3ABC9 for ; Fri, 16 May 2025 13:09:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1F98C6B0165; Fri, 16 May 2025 09:09:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1A8F16B0166; Fri, 16 May 2025 09:09:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0708E6B0167; Fri, 16 May 2025 09:09:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id DDE516B0165 for ; Fri, 16 May 2025 09:09:43 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id CFDB5804A0 for ; Fri, 16 May 2025 13:09:44 +0000 (UTC) X-FDA: 83448803088.03.6AD29EA Received: from tor.source.kernel.org (tor.source.kernel.org [172.105.4.254]) by imf22.hostedemail.com (Postfix) with ESMTP id 3CEC2C0011 for ; Fri, 16 May 2025 13:09:43 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=tO6DLC78; spf=pass (imf22.hostedemail.com: domain of broonie@kernel.org designates 172.105.4.254 as permitted sender) smtp.mailfrom=broonie@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=tO6DLC78; spf=pass (imf22.hostedemail.com: domain of broonie@kernel.org designates 172.105.4.254 as permitted sender) smtp.mailfrom=broonie@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1747400983; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=S2LbGUSPMqOGJJI17ms51nRbPfumG6Mb9OIjCEltbn8=; b=fmIlPv4/QvvtA1WTzv2vJdDf22CVGwK7yIzzhCVhxf1CxuUiZue1r2eX6X4lLasxpoljgP dwXkjAu0SspQPgGveCoU3FrBm8UFxMvidSdeiKrfWuGxQQ8XJRYT5sBZS6oRaLhmJmgdhk UrOaGFdCnXDfnCx7zgguW6A+wxn0eoM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1747400983; a=rsa-sha256; cv=none; b=E4kaUIV66iuMwaj4A7k1WBTVN2sTG+tlLUBxnOYqoppV9JqbYFHbpJRaTyZxB/T0V0jHFf dWBjlCx6iDxPdD7ZtrQznsEJ5/VWYBgpybbaTZEhSHUVlwNhWCyialgScMrUHHJdJt8sDm Xl7gFmOUwajRzQDma76+11xKxwPBFps= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by tor.source.kernel.org (Postfix) with ESMTP id 7E3E4629F9; Fri, 16 May 2025 13:09:42 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CD240C4CEE4; Fri, 16 May 2025 13:09:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1747400982; bh=eTlu4M+xbKe7S6ap+yHpZH/r7OM70xHL1rfiqO94rFY=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=tO6DLC78WaCvA9TWV/6GhHmgol9yYAgAo+jD6bzHVZabXzg3nP6cHaGyh29M3J4n5 TgCcOKrDAscv3Iv6wJIuomZwAbxDCM+tQsYLyEgpdZcqd1ue+bpB0n33qRcXqQcfYo dcz9N976+ynkHRqCKDX8lq9xVil0hJC5BAG/hDqoxCPX/Sua/JoUbK2mzsD+mKTeoC o5L/fLh6nwcB0Hrr0hyA5O0Vwmj/e4nzW5ryJ9oiDBdFhrTzVROoNz06S7R6Od38Kr JRdb9USomzZ2D8H/gUtODeuZkdJ0tQtqM/zFt3FuOeqcEG+M65tEVintEvYRoKkNai 0HD2iChLsPVQg== Date: Fri, 16 May 2025 15:09:39 +0200 From: Mark Brown To: David Hildenbrand Cc: Andrew Morton , Shuah Khan , linux-mm@kvack.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH] selftests/mm: Fix test result reporting in gup_longterm Message-ID: References: <20250515-selftests-mm-gup-longterm-dups-v1-1-05f8f731cf63@kernel.org> <58dbef73-6e37-46de-9092-365456306b27@redhat.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="d6KOq/7pGm4K+LcT" Content-Disposition: inline In-Reply-To: <58dbef73-6e37-46de-9092-365456306b27@redhat.com> X-Cookie: Well begun is half done. X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 3CEC2C0011 X-Stat-Signature: wuqc84yih6xzucs3c7fc7bzy7xt3qq7d X-Rspam-User: X-HE-Tag: 1747400983-6212 X-HE-Meta: U2FsdGVkX1/xMHM2YgtRSHNGQyAYmwKekp9LvSgLY8rw7CZ8D/5Z7ITeFyRCNU5/G30ASG9MubiAHDzf0wC1z9e5mK5jL8hCQphIyIk+rKxQZkFTwfl+Qg1iELFB7mjCmG5cjWR5ZWSR+4J7oPw3Ez0ewwouHgM98JE6UBq7dFmBDHUm6OI36LG+OumbsFV4fTjd8Ttb5HMFhCsqHceTmM7O0cKTcanU/HLqGh+c+BbbllFfZTnySUY9u3DBzhxkyyuTDkgEVUH61C0p6Kaxlrr7K01lfG2RNcOBgHmIyPyclD3onnl9OsLZDoXCegX6RboxlZ+CrAPEC9XbSKAeFnj3gSCBWvK8z4UQ6ZhGbJLz+TJAdbrzVlBcb66r0klE+niUBPX/apcvZMPeIHNpgJuvac4l9NjY2Q/b/jeQT/BDM4ILa6F2RYYDh0pRSIdBQsoybPTyMm/Un3/u3aPYTZ6hN4oBM/S4DWLsGOSSNoD2wVR1E8KkNMjBLglM+PIPNMNQ2jDSNhur/ch4bK/x3ZZTKOD7FwiaDjlfLlmGWvBjBzNOqaNHIZxpJr28ifcWwjV9ga+3QZmg7wwIgTelmUhuhIQV8DvSxj5k0mv84kTVAyIs67kcvMivSDWXeJtr/g+pJC4XZB683Metdun9MWNU8x+Lw98wAjMvAJCzJZxK9Ke7oV2Sq9YwiNMkv8Db2FrQ7vjvJBvauISoE3UiDLloLOCnK2zrcO4Enq7mNJH4Kndm5y0DPOCfnzcdEUtdUr8FCkRsQtyIYLJRslhtPbkz5Xe5wO925NKODGSdENidDPBdKvn/HVLHtee7VbjA027bZLKl1plt7+VVoWjtgPJSsIfRaGYBRJE5U+sMX2R3lhM1LPPoYuKQOFSFrFEGLtdS155Qsxcp72ISr50v/VsWwWuJdhDmB67RZI0gyF1r31z2J0MvrnATTmRNbQwgEX5PhNfixWngkM6tL5A 9seVj+YP AePhvDsutpDYLyIsBeFNJo/4ma2WYIBy25uDh+8uqOnsYC28a7n5F7xDYTozSvWdEWq+ScyY+hlBaVZMzpNeOoWlbzz3j6LwNETG/rAN7lO+SuiJl5snNXVIFYr4dRjz7SL6ZbDwrQMURhHctpB0tRVToRzivq4mTNd9R/j97KzptdqzAfggSo1LqKvGzQubUj+FyRh+zg9wOLgVRiA9y0BqcfU5Y1zsEqwMyCYv94jjAmYSutGh0v+ljnslJ4bQEryFX8FBdxgAgqOZUkYtIGcHDWGbam+yNEh8tSkug1eZnKMSHiP8e6jmxRKR+LioqpJ+vZfKfEf1XYQhkB6ZxmLapHahJd85GwKTX X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: --d6KOq/7pGm4K+LcT Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Fri, May 16, 2025 at 02:55:24PM +0200, David Hildenbrand wrote: > On 16.05.25 14:29, Mark Brown wrote: > > On Fri, May 16, 2025 at 10:02:16AM +0200, David Hildenbrand wrote: > > > reason), what exactly is the problem with that? > > > We run tests. If all pass, we're happy, if one fails, we investigate. > > None of the tooling is able to either distinguish between the multiple > > tests that are being run in gup_longterm, nor compare the results of > > multiple runs effectively. If all the tests run they report themselves > Okay, so this is purely to make tooling happy. Humans are smart enough to > figure it out. Not just the tools, humans interact with the selftests and their results via tools (unless I'm actively working on something and running the specific test for that thing I'm unlikely to ever directly look at results...). > What mechanism do we have in place to reliably prevent that from happening? > And is this at least documented somewhere ("unique identifier for a test")> It comes from TAP, I can't see a direct reference to anything in the kernel documentation. The main thing enforcing this is people running tooling noticing bad output, unfortunately. > I guess when using kselftest_harness, we get a single identifier per tests > (and much less output) just automatically. Nothing stops something using the harness from logging during the test, the harness tests actually tend to be a little chattier than a lot of the things written directly to kselftest.h as they log the start and end of tests as well as the actual TAP result line as standard. > > If a selftest is reporting multiple tests it should report them with > > names that are stable and unique. > I'm afraid we have other such tests that report duplicate conditions. cow.c > is likely another candidate (written by me ;) ). That one's not come up for me (this was one of four different patches for mm selftests I sent the other day cleaning up duplicate test names). > Probably, the affected tests should be converted to use kselftest_harness, > where we just report the result for a single tests, and not the individual > assertions. > That would reduce the output of these tests drastically as well. > So that is likely the way to clean this up properly and make tooling happy? That'd certainly work, though doing that is more surgery on the test than I personally have the time/enthusiasm for right now. Having the tests being chatty isn't a terrible thing, so long as they're not so chatty they cause execution time problems on serial console - it can be useful if they do blow up and you're looking at a failure on a machine you only have automated access to. --d6KOq/7pGm4K+LcT Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAABCgAdFiEEreZoqmdXGLWf4p/qJNaLcl1Uh9AFAmgnORAACgkQJNaLcl1U h9B+pwgAgQVI4RTsNN2lzXvLnrlwDdIqpmgNGC3XSHUJbPoJ2nAfwS/2l7Nawj1y 6V4ovdGnJ8pxwYHjz3kNcKK6+3ECE71vuP3EG3EcVjL4U/qQjZHqp1UgKKGflDjj 1ywYE3rpjVVXjJAkCjatqoOkmhoD+QHL98eHsdlGCeryC6+Pk3fBW8XthiSUHVL6 gjJrut71YcAa4hB2RDr5QHsYSFzfNQ1VGJCkBgPQ2KhBCMLIW5XbAKdgmTKcLppI h2CEGFp44SVYVLVMKh7xDZHOnRC5Rn630a7Agm+DNX++otprAW3tJom/xUkWmrS7 o8Bd4y0MXgryLagMcd9fid7VAgtKMw== =LiQi -----END PGP SIGNATURE----- --d6KOq/7pGm4K+LcT--