From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E468D15DA8 for ; Mon, 21 Oct 2024 14:59:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C49E76B007B; Mon, 21 Oct 2024 10:59:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BD29C6B0083; Mon, 21 Oct 2024 10:59:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A4CA46B0089; Mon, 21 Oct 2024 10:59:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 82A5B6B007B for ; Mon, 21 Oct 2024 10:59:34 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id B2203C1A7D for ; Mon, 21 Oct 2024 14:59:17 +0000 (UTC) X-FDA: 82697917806.21.43647A5 Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf01.hostedemail.com (Postfix) with ESMTP id 6E6D740018 for ; Mon, 21 Oct 2024 14:59:20 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=i0Ii4OXY; spf=pass (imf01.hostedemail.com: domain of broonie@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=broonie@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729522622; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=mm1jHxwCNpwzmKFkvhGQir01Ax3vhVoY9P9cPkY7nLY=; b=FwrWw8fFjr1fZiLevp2wehVHQHt1i+2FJUvdpo147Dah5tiyLuLMmnazETYQVdE8MqFZvm /k+HUu1bNcghAtqC2ov4i+tcv39r3hgtMu3xmtlJI8meAS4J03VXs2gTV8PjagXBpiCL56 UCnuKlXyS9wYLViHVRR4Ws/aFC/pyBQ= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1729522622; a=rsa-sha256; cv=none; b=IqLVxuAIq6u67K7md51rnQlwVYVM7CvZqnqpmzV+QjeVsQcmLoOpzfmt4gF9EKnxl5wRzU gVc7YZ7mIUl87uvUNpVCZowFBLnfQf1Zrf/CvHHzLKRUejy/2j+o0NCaslMwNQ+po/Waq2 uiNFmICRCSFpaacxuLnD1Ab4xSZ3jy0= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=i0Ii4OXY; spf=pass (imf01.hostedemail.com: domain of broonie@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=broonie@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 6AB8AA40444; Mon, 21 Oct 2024 14:59:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6C477C4CEC3; Mon, 21 Oct 2024 14:59:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1729522771; bh=albO36l24SFKiOm+yXAcpi7rfrjp7joQOfomUwb920c=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=i0Ii4OXYK3I8yRGtboVVVGKd4BxaOPScyZm4KX4qupSOZnr+P2JZB8hZNEDc4nNI7 o5nH/bTrY0jxuNWE/J3FFUlQNEK90VBzMbmBTQRIbfv/QdqNZtVaVzMbCpmw6EOUep kDGDUBSzKPgy3qFwVoqEUXWhqQowew3les07r5VJZ2t3gpBxtnZ4ZmCcMUxVS+3061 Qplyp9jHTIzNkdeLGJcRMP50OhEW09UBMelDCid0Xe7DpPu7x7ordT5W0xd5LXcBcD PPdmLNgPN/peOctn3WZJeaUC889Sy7qu/V9zHL0bW+rYtw4LMT0vP30yjKbsB4Rp/S V67SXlF4POitA== Date: Mon, 21 Oct 2024 15:59:25 +0100 From: Mark Brown To: Jeff Xu Cc: Muhammad Usama Anjum , Lorenzo Stoakes , akpm@linux-foundation.org, linux-kselftest@vger.kernel.org, linux-mm@kvack.org, linux-hardening@vger.kernel.org, linux-kernel@vger.kernel.org, pedro.falcato@gmail.com, willy@infradead.org, vbabka@suse.cz, Liam.Howlett@oracle.com, rientjes@google.com, keescook@chromium.org Subject: Re: [PATCH v3 4/5] selftests/mseal: add more tests for mmap Message-ID: References: <1f8eff74-005b-4fa9-9446-47f4cdbf3e8d@sirena.org.uk> <6aefd38b-d758-4e7c-a910-254251c2a294@sirena.org.uk> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="2vh7jJ4clxa55i2K" Content-Disposition: inline In-Reply-To: X-Cookie: Most people prefer certainty to truth. X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 6E6D740018 X-Stat-Signature: fwkpcmcz7fzn5adr5tkp9buwgaw9j4uy X-HE-Tag: 1729522760-688650 X-HE-Meta: U2FsdGVkX1/mG/tZXzFtzE+VmXeNXFOy3DVEzD7L11VJLqV/NnABr1GukZ6kR7/Xi+nJDd8KNEkXya1w20A4uRAPwBUZEcsj+HsVCgULnQWQMfkxchfw50hWijASDTg8HIR1ZXEH6je7GyGAxJ2r+n8SkfK86J78qSj9kUaNq5rsO/N5s8NG/Qr9BVjvAP8Adw8HJ3dwxtbFqsotBzhXicKMLePMIu46WEpXl+Xzgukfjdrm/vMwt56Tj7RqzTEg3JU1XwtXl6JMS+qMrTaWDizuTpZirmpF4ah7rOh2L/PClQo+/tThTDEtOgBBkzyj54DRroELQDwwQYG8JCAafXCXhJjlwfoCRPAIoL5TZLWdM2LpzS9XSrHcpTlsOkVXBNhNJtOg+kW25Fe7AOfRPF9G+KmwNAO/Q3EUsCebptXJF3zw/CAtaiV1tzz6QGUk/CNNyf9SnH1AjNRP5A0Dqw889SBJjX1YZyOXxy4qKsIgn0LgmWwwyk8znrRd4/kXI2jK/Aghe5ueJwgwQoAOts5r2j3cLxQXeLIrROYenAIrsRurcvp8wHMVrzm67ItExasgdq/oM+lAm6t17YbDrA13H3cbMdmBs3f05ehdiaNWZjVrYQU91W7YKNe90JQibD28xIv4Bcg+bD9MyrVyyHRiXwWFqCNPax4LFIdZqW1vmUDEXHxsD8II5I7/4WFIxFigEewswmjYg7Mwnl+SN4aEMi1gVXSyunKDLqlvZTUHS2EBF7QQbHKfNk3NqdlPtfpqESDGUqbXV56PlxLA4lsBesQHt5eQwZyq0G1B4stkMjwyWLZ+zihl9SjN7s3r/rlyHL4Zh72kwPfO+geBFyW2iWDeznMcmeQ9Q5wz9NChqPOVJzBIYKLuGO+WNgnKKD93erO6C+t/3JfQga0g0SGMDuX/zZ4HEnB9XvpexzZIjKNFB6vM1iyGUcAL/QQoG9hCGa8YDkyGnLe9xsu k+67PHrY Ib0nzt69e1lw8D2cNxqTynSmuylqsAsgzzOA9xeF3QCa5aeqWTfXa7xg7RepsP/D8P04FegClkNl8S/z7D3qIgVjRudIgJRstFYwcdWaunGIqsiQCiAEp1ZLel+70DX13TkVijO2u/owZo+tyJmWAa/YHvVCw0bewKd+kpinw8XLHZebaSpUtPet+XJmbinwW8T92c+eiBAIZn803kei09HZA12kHXCRtCzjv9sQDA23iktOYh52QMoOH3xBoxVLpZ0eAQTdJogs72ed4U1gOY5sYWraM1ImW7ggjyLUA0A9d+0+8veQPuxv7sP2E9otiuO8FTBoAgtNfvz47sfbeobvn8zMT9QgexcUIFvGl+Nmevt1MSM9cysgQYmEEJZVFsf0iMBNzxMP9/jsUl2RCreIUjzuYKOrTTOFV X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: --2vh7jJ4clxa55i2K Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Oct 18, 2024 at 05:10:01PM -0700, Jeff Xu wrote: > On Fri, Oct 18, 2024 at 2:05=E2=80=AFPM Mark Brown w= rote: > > That's not the entire issue - it is also a problem that the test name > > is not the same between passes and failures so automated systems can't > > associate the failures with the passes. > I failed to understand this part. > Maybe you meant the failing logging is not the same across the > multiple versions of test code, by testname you meant "failing > logging" Tests are identified by the string given in the line reporting their result, that's not *really* a log message but rather a test name. The strings for a given test need to be the same between different runs of the test program for tooling to be able to see that it's the same test. > >When a test starts failing they > > will see the passing test disappear and a new test appears that has nev= er > > worked. > > This will mean that for example if they have bisection support > > or UI for showing when a test started regressing those won't work. The > > test name needs to be stable, diagnostics identifying why or where it > > failed should be separate prints. > If the test hasn't been changed for a while, and start failing. Then Well, you'd hope that the tests never start failing but yet we still have tests and keep running them. =20 > it is quite easy to run the same test on recent code changes. I think > you might agree with me on this. The only thing that bisec needs to > check is if the entire tests are failing or not. Unfortunately we're not in a position where people can reliably assume that every test program will always work everywhere so people work on individual tests, and it's certainly useful for UIs to be able to give an overview of what specifically failed. A bunch of that is tests that just don't implement feature detection/skipping properly. > I haven't used the biset functionality, so I'm not sure how it works > exactly, e.g. when it runs on the old version of kernel, does it use > the test binary from the old kernel ? or the test binary provided by > dev ? That's up to whoever is doing the testing, but I think most people run the selftests from the version of the code they're testing. Some of the subsystems aren't very enthusiastic about supporting running on older kernels. > > > how do I pass the "seal" flag to it ? > > > e.g. how do I run the same test twice, first seal =3D true, and secon= d seal=3Dfalse. > > > > > test_seal_mmap_shrink(false); > > > test_seal_mmap_shrink(true); > > That looks like fixture variants to me, using those with > > kselftest_harness.h will also fix the problem with duplicate test names > > being used since it generates different names for each instance of the > > test. Something like: > Thanks! This is really helpful, I think the existing mseal_test can be > quickly converted using this example. Great! > (A side note: if selftest documentation is updated to include this > example, it will be much easier to future dev to follow) Possibly send a patch adding that wherever you were looking? That was just a quick hack down of the gcs-locking program verifying that it had what I thought you needed. --2vh7jJ4clxa55i2K Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAABCgAdFiEEreZoqmdXGLWf4p/qJNaLcl1Uh9AFAmcWbEwACgkQJNaLcl1U h9D7kgf/Q8O65O7KSc2ZO90h5zDrTW5/re+yyLl6rrA33pKJ3+HBeThhcoovMeVD UGIP6+ppSvCsERN01Havz3+k+ekJefqaaZFRnRn7PKoD4SyxguuojkO6hJMASnsK QUSlPb7+jf+VjQA09AZO7Rt65pr3drmlPjZk7eVXh4Gi0y+oPsV/TZ3CZA74Tnc1 3mGmetqcL4LsStRWPrO25RoJqkZrayX9P/WnjF5duR7r82pIIt7e5FdwyZIsWeia q3Reo7UQrCYnPWrZqJg64W1IKsVneN0FYMyUgtVBzlcQ2tR/Ncz5D/ySDxK6p+/r KZawNXyH8jQfs2HcpHmhVShtAw+/cQ== =ivod -----END PGP SIGNATURE----- --2vh7jJ4clxa55i2K--