From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BD21DC35274 for ; Tue, 19 Dec 2023 00:55:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 478AA6B0071; Mon, 18 Dec 2023 19:55:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 428326B007E; Mon, 18 Dec 2023 19:55:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2A1836B0080; Mon, 18 Dec 2023 19:55:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 1B8A56B0071 for ; Mon, 18 Dec 2023 19:55:32 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id D7695C02C3 for ; Tue, 19 Dec 2023 00:55:31 +0000 (UTC) X-FDA: 81581749662.24.23E3ADC Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2079.outbound.protection.outlook.com [40.107.243.79]) by imf20.hostedemail.com (Postfix) with ESMTP id D1A7E1C0011 for ; Tue, 19 Dec 2023 00:55:28 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=awDnjkzb; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf20.hostedemail.com: domain of jhubbard@nvidia.com designates 40.107.243.79 as permitted sender) smtp.mailfrom=jhubbard@nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1702947328; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2Uy4jeHer/VGh0uS36ZXUQfs5QCoK4AmPyey/aZ06tI=; b=hSpMF0Gm2uyk13ou4Fo7gYs5mTSWqaXn7e9tyoqaJFr+LKOiGLU8jCgXaKnE1koooGFhgP CaZ1tCWhA2J4qsE0FcvG4mWPDLs99xoPg/7UYr5/Xmht/+obL0YsqvUTcZmZvUArv3Rll9 NW6pFwsvCebBmwkVmHjisZ6AobOHuDc= ARC-Authentication-Results: i=2; imf20.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=awDnjkzb; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf20.hostedemail.com: domain of jhubbard@nvidia.com designates 40.107.243.79 as permitted sender) smtp.mailfrom=jhubbard@nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1702947328; a=rsa-sha256; cv=pass; b=7aqnFrA06MP7dzYVcT2GCxAvk9l6xHgjyXecRWJbvIEnNoGFlog1wSUN6lZhmJjhfOtCEJ 7Sm1vf9pwsuhEBGAexhPS+8q06hcpMTX+0YX9k3B2Yxg3huoMlzCMl2p+mSyOkUUW3xGsM KI0R/EKVph7zjjN+dr3RKz8CS8PF1Jg= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=K9gTIiiG5eaNtpiypYopLf7mOBY+ggoBtBW1pEKOYYBP2FrZvOcFMxjJO+mhgXNIZD1mq1Izg1azX/NmU1kCziWkKBuMreSN4P9omF0OCvE9L/qM660bF4zDpFA9zeEmJeX+x2gaIrjcdNbefbiBbyO4Lx33QfKQAGWe59ShxQRbPLy1XiOdt3L63NUznSjN3RKN8N4aNUoj245p7OGAcFfCkdZIYFTIgS9kmMEYVJaJUQK1Km/0HrPROKLIcNXIDk3HHzR8GFEpmXIKcvTBVhraZe+mksqLjqMTJpsbeAmmBd8CJT+179kOj93szHZkXjRtqKP3J8ddlOl8mBgycg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2Uy4jeHer/VGh0uS36ZXUQfs5QCoK4AmPyey/aZ06tI=; b=GzDLlT6tPAEBjsfmbe0UXs9UPnRQuJIcRCcMw0fn5PRR+D9Ud3ma+aoarwaB/S5BNLuA58gOTMs2r+Ku6irGM9DhiCTYrSuL6bBl9Lpl/eW6iE37En3RwhYBvpwPLVrk3z04XvHwGdUQYIWvekIYBeFXPT4cHAywiSNOHY2tfsB5hrdmttGUMGLwEOYkMx5FdmjD+TUF//I1mhfMbmbQzxqvceKEuRysUAsRTsw1t7j6eZBJQGEc/JxdCIFMvb5/9CBbOMxSS29oHV4xTwAtR7VSSdhgUn1ceqI4tvi2qKadimcR0iXuZ5QaNB30B2zM1WCFLDsRvckQhQx/R1hssw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=arm.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2Uy4jeHer/VGh0uS36ZXUQfs5QCoK4AmPyey/aZ06tI=; b=awDnjkzbKly/n1wXbL9QpIjeVTw5a+ynyRwgD4UscwRct6FivSt/xqgATve2fQmdJxPyH93ZOO5yfLG1FRLaHE7ETsYGVarDmrx2gAJNsJSVq43WXKh1PVAWno6TYggcTVhm9HwFrEaibITmHkmbcasTQ5A/gGN20agC7IhoFa2DIZ3+cim4xlJbrFMgI8H98OFGN7k/R1fVFvJOuYPJcDgbxD3sA71hMv2cLv7Lw+jAsGV8wUxlvQijMI8whgboh2aCdMdZTvcxCkFp2y7mgsbvFA07OxH4PsSJ86DKTH2OB2UZhSJT6+43Gv7Tr8Wp3kKreBuhXq93M1HO/mqRuA== Received: from BLAP220CA0023.NAMP220.PROD.OUTLOOK.COM (2603:10b6:208:32c::28) by LV2PR12MB5798.namprd12.prod.outlook.com (2603:10b6:408:17a::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7091.37; Tue, 19 Dec 2023 00:55:24 +0000 Received: from BL6PEPF0001AB4E.namprd04.prod.outlook.com (2603:10b6:208:32c:cafe::cd) by BLAP220CA0023.outlook.office365.com (2603:10b6:208:32c::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7091.37 via Frontend Transport; Tue, 19 Dec 2023 00:55:24 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BL6PEPF0001AB4E.mail.protection.outlook.com (10.167.242.72) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7113.14 via Frontend Transport; Tue, 19 Dec 2023 00:55:24 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Mon, 18 Dec 2023 16:55:07 -0800 Received: from [10.110.48.28] (10.126.230.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.41; Mon, 18 Dec 2023 16:55:06 -0800 Message-ID: Date: Mon, 18 Dec 2023 16:55:01 -0800 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v1] selftests/mm: Log run_vmtests.sh results in TAP format Content-Language: en-US From: John Hubbard To: Ryan Roberts , Mark Brown CC: Andrew Morton , Shuah Khan , Peter Xu , Aishwarya TCV , , References: <20231214162434.3580009-1-ryan.roberts@arm.com> <71228821-cbd3-4a3c-9ed5-18f6d5ebcfc0@arm.com> <07193932-941d-46f6-b152-d6c5fe09b26b@sirena.org.uk> <76abe3b9-3f66-4336-b09d-d5c137ff6582@arm.com> <7663374f-ed7e-4d86-a07f-e71c6bcef841@nvidia.com> In-Reply-To: <7663374f-ed7e-4d86-a07f-e71c6bcef841@nvidia.com> Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-Originating-IP: [10.126.230.35] X-ClientProxiedBy: rnnvmail203.nvidia.com (10.129.68.9) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB4E:EE_|LV2PR12MB5798:EE_ X-MS-Office365-Filtering-Correlation-Id: 1f4b60ff-8fc7-4e57-50e3-08dc002d2f38 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: AwOUNTSoUm7ZuxPHj5eYxM/HVD3T+vvfogbdF5kov5YXCdTgSbaPKR0ikfG8Fdy9CDv0KHe5TWXNHuhor9Ka+Vm0PYHEEXvfQZvZeST6vMppJVWUh5ONktBwbMzUPRHIMCRpvLZ60H7UUXH9D+2ozIfDfASU/a+un2qBkPoNORIj7gG8Oe80DQce8xrJS8zPOsWu+2JIODOIP+zjcIedGZYspV3M9HkWWGYWVfwHJddJoscenAOuXOjvw2BCRrzeLJwbuh3Lm/OGnwS1cioOJ//cGzfOdnP6V92rLqAVJSjhX2Em0CPWsNe8OJY0nGc69OOTxoX4hHNnkdNZSvUgN3AKpFtdcutPZr+QsAzvwxtQnj5WE3qNPEiPN9V2Ub/WNP9L2Aj2wa8aQ8fPo5/49p86DrofRD4nq3evIxTO/LPhrybE9tHKrzRO7/Ng0oQ7chIK2kO7KU+bjfPJEqQuyVPdAnmP1zXx387C3S/inTkydnsHL5ckfCMzkRGHZ0ed6nViVzreIFu4meN/P+Yr59lnmVWZsZsYaLDDUsDrYlmPOwTaTEIuz3o35bfxlEvAd2r86gyRk5yjL8t92A8V6yQXCWnnpTs9eHIwFcHZEprY8tztT2Eyz60iEoYxczTmHTWih6s4GYYvam4EoDR5bUVuLUNillZaOskLCziSbCj1r+lSBekQwBMveSCF7UCOjfSUHC4ziSBCUyyS1szwdNdpazgtWZK+33jzcMtR6a9UYwO/4hLf1KRY8hHH/YhA7Nw5JXUqjuCqLRJg3LTYyg== X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230031)(4636009)(346002)(136003)(396003)(39860400002)(376002)(230922051799003)(451199024)(64100799003)(82310400011)(186009)(1800799012)(46966006)(36840700001)(40470700004)(2906002)(5660300002)(66899024)(31696002)(41300700001)(36756003)(86362001)(356005)(82740400003)(7636003)(16526019)(336012)(426003)(26005)(40480700001)(54906003)(2616005)(36860700001)(70206006)(16576012)(70586007)(110136005)(478600001)(53546011)(31686004)(40460700003)(6666004)(4326008)(8936002)(8676002)(316002)(47076005)(43740500002);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Dec 2023 00:55:24.3485 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1f4b60ff-8fc7-4e57-50e3-08dc002d2f38 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF0001AB4E.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR12MB5798 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: D1A7E1C0011 X-Stat-Signature: wne1m1gdpbiu8sr5hwbw598zx9qkc4gk X-HE-Tag: 1702947328-511375 X-HE-Meta: U2FsdGVkX1/tGY/YoENdTjYuYMGcI2k+p+VuE9Er+ZgJ/Sy2imOT3gcSnEsYLKo9ZGpUjvZ2YSucYPyrCgs81z3gCM3PJCwPJFeUycGiuMrlsRlKBXOtgBdHYPCUojfqn0Mx/ft8gZZAYFWB7OMy76qjqAyVGUStIpPD7sVTECPa9SFVP9qxEjUMT1+ozpAAcRIE53399NS3ufeOVEqbYfzVoSGytBb59Vp7gHmI6B4PbqDezJ9sg8PaGJc3Xyaam74uGmIFsWqb7Jdpb5mUrXG6/vP1E4UqPb/YoZjekDCQRGMNAlCKd9x6ny2UIIB4JKskVg1IY5E/g9hW02ZCHxwDp00yHmqm++HPmlkzbaVVx119FJesZpUN3uVrbNsicQw0FsSPss1WcilExNWcTdJPoepQNBPIADfWBEKcqUn6Z26fyLKHk5TSzVvrueuW2bo1TOy6/vFqBEeyT85pqmnv0qM+73GsPeWi5H+UoxhyDavDEOLE9XVdhuAB1p8FIIPy3SQqxD3H1F6q5S/OfAcvCfRg48aSkY+9VeopcRu1kqeM3ZIUU9SKXBR1IMdHh1ub26YLikKuUYOf9WUNVvQYjnZvK+4T3T2vcLboUIYT7u/9fewNTqXdr9J+VUSKaLsjPtF1+4Xos5G/n4c1eKCb2koO2Y7ffB88wgaIqD/tskunU/67CA/O/wmhSxQjilJgPSdsXg7mL8RHlQtueWHVei5bV//q/fOCHxpJe6sppfwkuogt+oGImNsDHHKb2ojXgIx+Sn3tdOJE8o5j+8eYqLKNpw6PhGRD8nYvfQWypaVccvzdEfLutuSjoQrgpu2s5MAco+sRkZxtDb5K+oOg/yNS/iqbvUBY0qiQI4sF0zRV5cJj6+0zdXJXRdeHFjZ0hTbqJMD6tO2sWRf/0BG28YwV9X6Seo0cYGsT/4ln4iNX8rVTrhM2njjZx16E3XepcDe4dJXh/tCyDmc bqwJGJXp 3Y1b/ZvbrayjjuE8yHaKqAKO2owdZ4hARfQMMU4PkVuO9EuB1kz+nCkA1fEX7WXABMfUMhEmro566uLZxGm2hB1OXD4wZbR9eN6scWgu7pBS+0NUyxIGkbvBxedDlzY9qwdjJjVMDYqkxvjub6X2xUeXKF8sizqodfMSx5RokMJVYG7kTeYYjRxsfwJZyGjpinjqPTbE7G83O6tYFq+be5vzBT08+H9iX9rpGcxPRpbA8NP9YL9u93p8v9bnlmIu+pgPtEtutHclYiTW5Xceq+8OVxxwcqiQvIRxi1u0kOBezDi7TGu2fIjyMiAWMToWH/vfYGfjfozsvHoeF6v2ME50jPfRBg2KiVBkJj8uTJTUBAfW15duhAUKIehcvry1H00sk6BUDI0AWv1s6g101deqYKOychMISEeshUlKyUTPufQU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On 12/18/23 16:51, John Hubbard wrote: > On 12/18/23 03:32, Ryan Roberts wrote: > ... >>> I should also point out that some of the subtests already attempt a TAP >>> output. So now we end up with TAP-within-TAP output for those programs. >> >> It's actually TAP-in-TAP-in-TAP if you're running from run_kselftest.sh :) >> >>> >>> For example: >>>      # ----------------------- >>>      # running ./madv_populate >>>      # ----------------------- >>>      # TAP version 13 >>>      # 1..21 >>>      # # [RUN] test_prot_read >>>      # ok 1 MADV_POPULATE_READ with PROT_READ >>>      # ok 2 MADV_POPULATE_WRITE with PROT_READ >>>      # # [RUN] test_prot_write >>>      # ok 3 MADV_POPULATE_READ with PROT_WRITE >>>      ...etc... >>> >>> Note the double level of leading '#' characters. >>> >>> Again, this is still readable enough for humans. But it should probably >>> be removed in subsequent patches to the subtests. >> >> I personally don't agree with this. It would be difficult to flatten to a single >> TAP instance because the top level doesn't have a clue how many test cases the > > That's not quite what I had in mind... > >> child is running. Trying to do this will make things more fragile and less >> modular. LAVA can certainly deal with nested test cases and correctly parses >> everything to test case names that contain the test name at each level of >> nesting. The thing I was trying to solve with this patch was that previously the >> top level (run_kselftest.sh) and the bottom level (individual mm test binaries) >> were using TAP, but the middle level (run_vmtests.sh) wasn't, and this was >> confusing the LAVA parser. >> > > I was thinking more along these lines: > > a) For the individual programs (binaries), there is actually neither need nor > desire to create TAP output at that level, because frameworks like LAVA only > care about running a lot of tests and parsing the output. > > b) Therefore, just stop specifying TAP output at the leaf level, and let > run_vmtests.sh and run_kselftest.sh do it. > > Looking at madv_populate.c, I see that it scatters calls to ksft_*() around. > And I was thinking that this is all just redundant, isn't it? > Although I suppose that the counter argument is that the subtests in madv_populate.c really *do* want to be specifically printed in TAP format. arggh, I guess this is just not worth fooling around with after all. thanks, -- John Hubbard NVIDIA