From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8A99C0015E for ; Thu, 13 Jul 2023 15:43:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 56936900041; Thu, 13 Jul 2023 11:43:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 51836900037; Thu, 13 Jul 2023 11:43:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3E027900041; Thu, 13 Jul 2023 11:43:38 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 2EB81900037 for ; Thu, 13 Jul 2023 11:43:38 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id E902B160325 for ; Thu, 13 Jul 2023 15:43:37 +0000 (UTC) X-FDA: 81007008474.25.4F3EBAD Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf28.hostedemail.com (Postfix) with ESMTP id E221AC0007 for ; Thu, 13 Jul 2023 15:43:35 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=PfOVpNnc; spf=pass (imf28.hostedemail.com: domain of broonie@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=broonie@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689263016; a=rsa-sha256; cv=none; b=wYKYyLEdZIHgdQf7D3rHfqdX2MbdHlnBJ+STD3x2ltgtItUET6Um7ObtGUZxMpg/YiYRC3 R/+H4bmjM2RoeK8hhLZ9YW40hTJXmmwgrcMqR92huXqSmQL+MjR/ZbBwFT4nXJuIwY04G0 6ztEIFHR0SAtyTBpH4UTas0GRz+I9lo= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=PfOVpNnc; spf=pass (imf28.hostedemail.com: domain of broonie@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=broonie@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689263016; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=a02n8F7I0BC9O3H7WAzCyYW5N9p3gBn7c9lJd2JE190=; b=Yi2q/IqYl3iBogke8uacP6Q40xnlQ5AHItdgrruQPG5ot7VtYVNeXKI2AXchKROLYeo8wD ZUQPNIjbQP2v4z134d/xXRkV4SIcun3J6YZN5xHYlJGPGA3BJiZW+079odr/6h+948dJUJ +EdMlGWJJvC1xXKtm53zZuODmEovFp8= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id D82E261993; Thu, 13 Jul 2023 15:43:34 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id F2B77C433C9; Thu, 13 Jul 2023 15:43:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1689263014; bh=a02n8F7I0BC9O3H7WAzCyYW5N9p3gBn7c9lJd2JE190=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=PfOVpNnct00JOEZz8f6ofK8Np5RrswDO/LANYthjXblaP6MJvrz/J75iIInrPrR8A mLZJndIqh6jREZtgoEYRRL1NkydUTupj7PaEPuWAilZzfh0tiM4LQt1FIJBXvafRo7 7HWqi9v6tJ4auKKUhxdpUe++/9+npal1bkM+BhSgftDd9IEyQhP860s680yuMEEmhW ssHEjcOVyjgycdZ10Q+egHwFklDxSg+pVJ6LUlKSPGqCH1o5cr6otrGzP62eYdsY3s mZVjUyXSC3sXkhHIOZX9+fu/FvZreIO0c0RVnhfWa0zrHS6vHLrICM1lDoPo+efxLT u2eAaq9J6wNyQ== Date: Thu, 13 Jul 2023 16:43:29 +0100 From: Mark Brown To: Ryan Roberts Cc: David Hildenbrand , Andrew Morton , Shuah Khan , =?iso-8859-1?B?Suly9G1l?= Glisse , John Hubbard , Florent Revest , "Liam R. Howlett" , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org Subject: Re: [PATCH v1 9/9] selftests/mm: Run all tests from run_vmtests.sh Message-ID: <5b4fcf62-98c3-458e-a0e7-8e86ec354cb9@sirena.org.uk> References: <20230713135440.3651409-1-ryan.roberts@arm.com> <20230713135440.3651409-10-ryan.roberts@arm.com> <2b586ba2-7522-a823-afd6-7b4d978f18c2@arm.com> <97742685-e026-417b-8c8f-938330027636@sirena.org.uk> <8d2e75e7-0d38-6e6c-a02a-b66a18515dfb@arm.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="y9sB0v2MUUQiI5hI" Content-Disposition: inline In-Reply-To: <8d2e75e7-0d38-6e6c-a02a-b66a18515dfb@arm.com> X-Cookie: MOUNT TAPE U1439 ON B3, NO RING X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: E221AC0007 X-Stat-Signature: jcf3bc576owrhmz3pndqaa1t4hrmrugb X-Rspam-User: X-HE-Tag: 1689263015-324721 X-HE-Meta: U2FsdGVkX19FE7PoiGrkvJ7sLAVs9iPO58PBWr9BOidX7F4/V/tiSBFDWVSMez8iibgl3EvrUwpHMOdjuAcPTKY3RkjfqeMf2h7/3iL/lSqFnqFPnc6j+bnHlquF+ynijqqBADbxn9UpzauduKrHqnFN6LLBomXFijnE8X1RP19BJE1e5S098VjBk3aHDsVTIC1Ba4/saA9h9tzav8BUsDukwrfdxIKJ7Ae7l/DMcpGjdrWqgFrNW/c1idASelVwchlnfe1x4TAz2UstHOCCPOWK/6Z9SyLXQajqdF02E9wsm843KsfN2U+kF4rqzvp4cspm0tRjjzrmzlYhsRlv2S4U+0Ph1LrMpcikpZ59SDsFklH6vv9QxI6rlx0P7/Dd+eHs7vHca6/fEzHL6L5x9NWKAsfl8EIOTYpy1DA2Bo9e3raOGT/qDMJU3d07bVs7EDd1NWnaFaAxvtpeqS6eh1ralpgdNxrXGSxhEfEzqh4O1yCRVASB+gF4BGfMNMVNoD73Z9XBM6KdNj3/K2ciNEhxeSvA4vhLYXZPS6phwU4TZ3lA4zqROU7dgONiXvUd8k8mUrEx+4uHIFZ69oLEPENDUqT7cjp8sQYCwdV3atVzDkloYel5ZCwTp4hbwbN9Cxyx57WQYxhvv0PVMH0x+FTZnhjdIuJyObFEofsqEPZZFwozghdCfttiBJmuxZM+IdRugQgkOu15VEFsXTzX5qUVzVKNuRBVDX3nbzqkXXJJAy0OefbDInKO4+kt7jm6dIzHU3c8F8/nilEFm9D0mE1hpQNi1Y2iocbHRI7bBUyANOMlI5XhmiT5s2/KtjqtAeSdtJw0F4pPtadpec7KBlO2y+gdfpFcAYx52Vnird0ilOYndEokYCQUmNs5MtY83Di9goqqkIOTsDIzLqetb9OLjbsNXwxdrMUxqgfvCX43Ri46cyzHk8CUJbBWIgPHTFXybiHmiKNCtQDWCVb PVC9uhO8 /MQfn7CmgtVL/p2BgcSPbPMPCuVOQIjV0bXQXs3+SjVjFTgUIaeUe/jHjHrnqLtwsSBFKBxHDnG899TgfE70+pCx43sz6W7cjFYGhw3sAjP5dG+hsBymuuWl+k3Omsh71CV6ubRvQPoE6MpPQ9rbZ+C66yLui5uVngyhvTDP0IOg8Q8hwTUI2q2v8wIOyrNFTFPg5wkhnSROkLhxJZfXS8mXi1LvqgU1hnfAm0HhgSwJPmLW3btjc7ZAte41jndfum7bOl7hcwZTsSJ5UjmVLhaxBWbxHP8A/QpqgoKp4OANW8dZHPnBk+qOUJXO2aZbHR6Se8NOpFtf4emldJ9Rjh8nCUnKVFmJCqOiibg9nRtun0Su4gcBUldPamtSYAEmDj9QyQm9tjm5mn6Y= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: --y9sB0v2MUUQiI5hI Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Thu, Jul 13, 2023 at 04:36:18PM +0100, Ryan Roberts wrote: > On 13/07/2023 16:30, Mark Brown wrote: > > The results parsers I'm aware of like the LAVA one will DTRT with nested > > kselftests since that's required to pull see individual test cases run > > by a single binary so it's the common case to see at least one level of > > nesting. > That's good to hear. But bear in mind that run_vmtests.sh does not use TAP. So > you end up with a single top-level test who's result is reported with > run_kselftest.sh's TAP output. Then you have a second level (run_vmtests.sh) > using custom reporting, then _some_ of the tests invoked use TAP so you > sometimes have TAP at level 3. But those tests at level 2 that don't do their > own TAP output probably won't be parsed by LAVA? I think that should mostly mean that all the tests that don't individually produce KTAP output get ignored by parsers and those which do produce KTAP output will be seen as nesting one level up from where they are (ie, the individual cases will run directly from vmtest), though there's likely to be confusion about expected run numbers for things that actually pay attention to that. > Since you agreed to put this into the CI, I was going to call this part "your > problem" ;-) It'll run, the results are a different story. :P --y9sB0v2MUUQiI5hI Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAABCgAdFiEEreZoqmdXGLWf4p/qJNaLcl1Uh9AFAmSwG6AACgkQJNaLcl1U h9ATFQf8CDkX97XcCz4UtVfkVlN9UXmTmpyuRQjNSh2CHkbvSzihJqhlE/J1YTOe vBavd1wDJTa2HkgTgFKVrbZe5+oME3wGPSR9Hu+uhm1zQfCkkHhR0rsJjJqGZ9hN l2YMM7cqSl9UC0QzX8lcTUTAOb14pAC5ikmnWd4LlMFVtR6sZHoqZk60xsxvZymb evGt7ytkwQxliaEKv4kVsA2NDeiQ/awIcIeg26VNABqHa/T8MoGteDxdx2ZkeFAI 2Lp8OV6YGhDmBn0lnuY+qoTUWdgceIuau7ISav3S6udB8Useinu7ab19HPw48nDL oq5PM+NJhvqYKm9ZsLM7JNhMbQi4bQ== =HNhe -----END PGP SIGNATURE----- --y9sB0v2MUUQiI5hI--