From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 529C5C35274 for ; Fri, 15 Dec 2023 13:54:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C9A286B061E; Fri, 15 Dec 2023 08:54:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C49A06B061F; Fri, 15 Dec 2023 08:54:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B10A56B0620; Fri, 15 Dec 2023 08:54:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 97FDC6B061E for ; Fri, 15 Dec 2023 08:54:19 -0500 (EST) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 5D2AB140DD0 for ; Fri, 15 Dec 2023 13:54:19 +0000 (UTC) X-FDA: 81569197038.18.CE0B4F8 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf06.hostedemail.com (Postfix) with ESMTP id 47C1B18000B for ; Fri, 15 Dec 2023 13:54:15 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=RDd1hVLT; spf=pass (imf06.hostedemail.com: domain of broonie@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=broonie@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702648456; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=mrN51PGQvNvkxoQ7GDmX9J5vAkpNVhtppDL/sLGttW8=; b=Pfz+gurvsGED+5Cn/zd1fPrafFSx9o5jJCGBNygA1TyqgUwTT3J582x1Mi2v8VfzT64DFr P7LNYzRW/ENTZZYXHVlyB1tqlzOqcO/Z+eCYtKpF53U5OL5HMOQDRtZqkFbJ9lvOQryUZJ 0bZaaj5ZSJWlffNN9zWKu/gtmLRvzS0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1702648456; a=rsa-sha256; cv=none; b=HHTDBvJcr9Lt9wdg+XaOFkg8fmQ1HYll+jwNodvKJpeKhTRfndUhh3a3XakLzd22vIZfxr bBWrSvdF61O45ztzPhEAdXkuTCjlIE9fhFco6JdIWF51TVEwlV00gNmVv4E+iVmqtTeECy VwLW5N+G/Fcd6wRc2yjs+L2osW8+xpw= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=RDd1hVLT; spf=pass (imf06.hostedemail.com: domain of broonie@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=broonie@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 2B569CE1926; Fri, 15 Dec 2023 13:54:12 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9854AC433C8; Fri, 15 Dec 2023 13:54:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1702648451; bh=PfST1jdzhxOAteHkX/4wywKOjFP7hPGJ499h5PgtccM=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=RDd1hVLTE5QHaHhTZRN4PvozaEVV1AmCPxz6AuNirY1GRfojrTzHgFcOdlSEyMq5t Nd1V6jYntCPlDRpwjr+4WkfQqTABI4xRxuG+7hn2x8DtwYBHgKYo3p3A7TvgM45egd z1PBCf/M68ZkBK6gJvnY6OU/NTOadJSD2YEpMrbIzc33pKDZd2ymafzN1X7u1uG4kL A3IKZEzeiabD4lLDOSdW4B680Vx7Ca4MAewqttMd3rGdYcKCImxxIuGYPbUDP3NSKk 1OQuZ4iz+bri2B1gSSvsAt94SrvAVxKsM/aXoSmaHqYC02JW2ggikVZuSJkmvvtme0 JBLt+RV1kgvPg== Date: Fri, 15 Dec 2023 13:54:06 +0000 From: Mark Brown To: Ryan Roberts Cc: Andrew Morton , Shuah Khan , Peter Xu , Aishwarya TCV , linux-mm@kvack.org, linux-kselftest@vger.kernel.org Subject: Re: [PATCH v1] selftests/mm: Log run_vmtests.sh results in TAP format Message-ID: References: <20231214162434.3580009-1-ryan.roberts@arm.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="rv2JuR2krLZfp0G5" Content-Disposition: inline In-Reply-To: <20231214162434.3580009-1-ryan.roberts@arm.com> X-Cookie: PARDON me, am I speaking ENGLISH? X-Stat-Signature: 4tqupjfipth9jm4srxkdk4hsf9n3zwxf X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 47C1B18000B X-Rspam-User: X-HE-Tag: 1702648455-528780 X-HE-Meta: U2FsdGVkX1/sazLFGvPFq9dlo8XkfxH8L92sjX/M2jfRWkMiRfc175Sm0PTWZncsDPqIb3qpjTokl3xPdW17QM+07MPH3nRfjnEkINo1CYoYALg0IoCFiDwk09mj6gPcV/vuu+Wyvki1OAhs0teoQbLlfYlpfuG2lMI+1iTWo9zPT5NUAEiD8C5uNnI3aCcSbG8TKA6uaJxwwApnPV3BgudywLiLZYmfp9uYI6AIyaCYwvlb75SRZjp2FSU6ddv7Olqf51gEskDD2f3piV9+rtPgxn7emQVnbFa2M8KdbTnz5cA5E1vUtF8GEGutG0Wbe8fNndwiKghn5ACBssRRnEkUCMCIGJVDv8NapJGjnX951bZJNGpuocttWIlIOePv5D33btd0zNe1nnR2MwS4LK/LQe5hrtGPEAOBLMxiAVyFlS2nznKUdQsLPhaNDmcGXA/akJUluMTFylGEPqX4pT+H+GUQ6ROugzX7v+kXLEm01gEJyhSFFmrw9q/bNo/xYCHp/AqaBoSDeBgNPptT56x7uVt8oxkgoh4nPYvsfq2M0Arm0tLtUsj+lmpSQ50ya4oAexH3+CrCL2p32XlFsddbJcR5auOl13f31AaqcTwXZSpzCYegkL+Byh+1feejWIgRdEqROQMF/S+txhmZn3alpANiSW4PuxA+Evi2aY8z3LOe7EtN3aTyz5P/y1lHFuGsswzUGFTX12u9vgaE/9K3+2OBRmS8+lpSt7Cp4FBYdU2yGsp0BjzqVyrWEQhv3KxCLD2hXr2xDmtumwGW6VBFsGELng3Cyn2HmquHTtA67uNafUpPW2gbLckFBeXhvbpWi3BVfdZ+nLq/pHXeUDpe6EgB44uWeFR2Jsp5EPYZ/4o5NpuqQAC456ExVI2pNfCE/aTvcvrgHfkWDT7+3bMUON7KDmCJFsDcJf63lpaq+SrxS+Iqc3/higJgBQzE3HPc2IKpnNw7f/7mJ/Q sVEw62NC HcIdW7KwE6Ll3cmBBEDumweKamNuBfyHJFLlCBDz3TM9Hp57xBmvGBdUJqln9gKRph5JrzaSQh2RczBHXRQ/yCS2m6FAEiKxjCwxpzBY3Ei8007cA6sjRr6IAgAa/YEonVI7+QX4ARrWaobTij69xIRUf6ugLCaZLJtbcpC/tTIJg3gGmGtzldWavMxau3yXWhlTVOqMrU0d9KdH/DoUHkqpiUtwNtOD3HI+YnFM+OKUbkzJ2dNbyZicVrR4rCsuNoM3Gdc47BI5xXOuZt/dTUjli655kTP3fZsKqzMwoEr9P9ApjGigtlg14x+d5xK5aSQipm+Fg2BvfWfum57hcvG8WlOEfBh6laoqk X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: --rv2JuR2krLZfp0G5 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Thu, Dec 14, 2023 at 04:24:34PM +0000, Ryan Roberts wrote: > When running tests on a CI system (e.g. LAVA) it is useful to output > test results in TAP format so that the CI can parse the fine-grained > results to show regressions. Many of the mm selftest binaries already > output using the TAP format. And the kselftests runner > (run_kselftest.sh) also uses the format. CI systems such as LAVA can > already handle nested TAP reports. However, with the mm selftests we > have 3 levels of nesting (run_kselftest.sh -> run_vmtests.sh -> > individual test binaries) and the middle level did not previously > support TAP, which breaks the parser. Reviewed-by: Mark Brown > Let's fix that by teaching run_vmtests.sh to output using the TAP > format. Ideally this would be opt-in via a command line argument to > avoid the possibility of breaking anyone's existing scripts that might > scrape the output. However, it is not possible to pass arguments to > tests invoked via run_kselftest.sh. So I've implemented an opt-out > option (-n), which will revert to the existing output format. What I did for ftrace which had a similar situation was make a wrapper script which invokes the test runner, make the test runner a TEST_PROGS_EXTENDED so it's not run by the kselftest infrastructure automatically and make the wrapper a normal TEST_PROGS. Neither option is especially lovely. --rv2JuR2krLZfp0G5 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAABCgAdFiEEreZoqmdXGLWf4p/qJNaLcl1Uh9AFAmV8Wn0ACgkQJNaLcl1U h9CjPwf/cdFp1eTRgPPag0UWpm/AWBET2FW0ZyA+FmQzYiPHI/P8Y8PqBQ1Kv3Ba fNl0WRn1C4TNd5RhpjYWXqcLnd+aLZ0hsF22GuXo/lu6sTLz6AbZ+YyAa27vXZqO hiH8TriNzHwGrWolo+rIxEHoPlIBiX6TWORSTIZwW6tMHHq4z01XduvTnYrknX8l cJMM7lSmML+0jzUce0tSw7iaKtIhhcK9jthZaHnwkf5jZIpxH6TTtSU4I1hHgfBb ZDmKai3mOcpHxUlLj0+lp0uju/TNgr3CDNvLPfh7UxIQy+hEkaZgH4XHsU7WdOh9 tJnXBcqtIASP5QRyVeKvyJ5rLPxsLg== =cTyH -----END PGP SIGNATURE----- --rv2JuR2krLZfp0G5--