From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 93DA8C001B0 for ; Thu, 13 Jul 2023 15:30:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0D92C6B00AD; Thu, 13 Jul 2023 11:30:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 08B8B90003E; Thu, 13 Jul 2023 11:30:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E92F6900037; Thu, 13 Jul 2023 11:30:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D92C76B00AD for ; Thu, 13 Jul 2023 11:30:28 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 833FF1A02F2 for ; Thu, 13 Jul 2023 15:30:25 +0000 (UTC) X-FDA: 81006975210.25.D5F2E66 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf24.hostedemail.com (Postfix) with ESMTP id 6938C180025 for ; Thu, 13 Jul 2023 15:30:22 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=none; spf=pass (imf24.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689262222; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=W4TmHQDxaK/hwD9v2fKCp7kMt9IRHZzsCxGNcDBJR/o=; b=DH1vDcEHoK9J45se9O+qmTLJVEAt1qw6qGyB4wWTrwUc1y2hWGdjaE0QIsh9SA0fyrb6DA OnK3n3iPgrbtB9VpCQAMp4dtrYvFHyR0erdrG2VYCsVvFY85lQLzJQ+wl5kiSwWjWd2hLD wjbQVWQbbSZRW9W8/lXR9k0F3znQnvA= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689262222; a=rsa-sha256; cv=none; b=sEBibIhUeknws10nzNxoADd5wTiZQmLb3RYjIQJlwALeyoJQvGLi6U60Cm+RryUCqpRHG3 NqI+vq4qYhBOxOHiar+p8w3Fg//Um0d/DcDRwfK28yx6UdoJxuYQ0DgsstWjaxoOw7HeB4 lYlgWtyheoficquikIp53dXtQN2bYoU= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=none; spf=pass (imf24.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A104C1570; Thu, 13 Jul 2023 08:31:03 -0700 (PDT) Received: from [10.1.30.48] (C02Z41KALVDN.cambridge.arm.com [10.1.30.48]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id A37C33F73F; Thu, 13 Jul 2023 08:30:19 -0700 (PDT) Message-ID: Date: Thu, 13 Jul 2023 16:30:18 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: [PATCH v1 9/9] selftests/mm: Run all tests from run_vmtests.sh To: David Hildenbrand , Andrew Morton , Shuah Khan , =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= , Mark Brown , John Hubbard , Florent Revest , "Liam R. Howlett" Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, Peter Xu References: <20230713135440.3651409-1-ryan.roberts@arm.com> <20230713135440.3651409-10-ryan.roberts@arm.com> <2b586ba2-7522-a823-afd6-7b4d978f18c2@arm.com> <57f4cfe0-2f20-c50a-439f-ee914dde1693@redhat.com> From: Ryan Roberts In-Reply-To: <57f4cfe0-2f20-c50a-439f-ee914dde1693@redhat.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 6938C180025 X-Rspam-User: X-Stat-Signature: 4gaj378t9j3rqf5ncf1ms1za1cudhpt5 X-Rspamd-Server: rspam03 X-HE-Tag: 1689262222-236812 X-HE-Meta: U2FsdGVkX1/iOQ+HT2SuVRVFUuWUQmVy3rw31I07igT08Dg56xMrcmdl5JnB/EVLj6t1A9UMSkwGsBNgb+fkyOcPN5wSteMNNJq8RqU+uPQpzgQnJq3nsxJBNaOlNChFqCRl2gAdVFsr9DFDjiH6q3G+63/WR0IrxhUlDMQpVQm7fzOngMFIEu9LbI79AQtqtY6ZqL9GnRTuZe/1DgQ5b+Asjfefn6CPnunCBluKJ1xQd8/2dirk3V5nOM0QB0R/jrBQHEn76cvOQ9GcTpbSkzwGaB78KDrt8HRD1+Q7lF8lRYV5Hz9xTcn/1vFeZjLCERI3AVtYLKLt6Qjs/3z6ELiz/srjQKPbCgKA+JKAjXhX0Uk/m+xuUU1BHaXpoZz1BkH0vEQPSefMsl0xoB4HlMR8nyMSAVVrup6FZ514adMGPkzwyCbz0QOtIMndznS0ci7dUOy4DQmLFPxy9OMsjmta6RorXBiGLyCWqdshje7rg3NsfB4iBSEM6PqZp2fU212dK5MeVngY9iaOAQd24rZZmf9oss2cEj0OVe/K7cGq7QuCQwQ14PKECvJ5Hj3hfkEVcYazIhRAJAiOq4XDD2FtO1wWzJMew3WN3keiLvG5h8UXXEc4gFaekwUTd6IhVz7C7EW2dTmy81jAzbT9MHNdV4cc/dkhVwP40qjLq4leszSdAamM9IYrgBR8gY9VYbby5VhU+0sdsKM1VjQjZG+5vuBbzHA0yj+uglB9taMQS5HcMcHmfd7cn+RtwNeOc607nyush8hnQ1pT+0XC21pgz4D2YL2lc9sPpbwsr27RKw9Z9zc+F3LD8/0pZztdRt8Y+/Bw47HpwZBbKxqpf1hcG6mhDR4PcCexFl47hr90KnFafZaHrqtVu/UpTP6B0HKMKw49gJph4s2fzRgZa53ZEbpRBjXYkVH/5HEmqeVKRFSN/7Ti/rLAEX/yXM8E+94Qdw8kF/bZRpbYpo5 jJkIY7RC bP10gHuBtHSy4DPxL/WNtDCUd4jcfQ30LW58fAeRb5ogdvp3K0HcZRBYsXArraAN9lk1rFvQMQC8RzUUSz05dzeZWg4fI1i6dY8FORC8TnYeZMTp4L8qEWVgWRNseT5nx0ZbLFrazUQSxVVQD46lyqlk1zt0STkOdBCcmTVsrMr8YiZtteGmIb4AezfAo5nXBDJBaxyF/PYk01gRtoWoWhS6uY0+7zSnno9hmev3dFAJA3XE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 13/07/2023 16:25, David Hildenbrand wrote: >>> >>> Which run_kselftest.sh are you referring to, the one in the parent directory? >> >> run_kselftest.sh is the uniform way of executing all the kselftests. mm seems to >> be trying to be special as far as I can see. Certainly if you run the `install` >> make target, kselftests will create a list of all the tests (including non-mm >> tests if you have included them in the TARGETS variable) and copy that test list >> and run_kselftest.sh to the install path along with all the test binaries. Then >> the user can invoke any of the collections or specific tests in the collections >> using that tool. It also wraps everything with tap output, runs tests with a >> timeout, etc. >> >> See Documentation/dev-tools/kselftest.rst >> > > Got it, thanks! > >>> >>> How to invoke it to run these mm tests? >>> >>> (I never dared invoking something different than >>> run_vmtests.sh ;) ) >> >> # single test: >> $ sudo ./run_kselftest.sh -t mm: >> >> or >> >> # all tests in collection: >> $ sudo ./run_kselftest.sh -c mm >> > > Ah, that makes sense. So I guess mm is then one "collection". yep, the collections are the directories under tools/testing/selftests with a few special exceptions. > >>> >>> [...] >>> >>>> >>>> Signed-off-by: Ryan Roberts >>>> --- >>>>    tools/testing/selftests/mm/Makefile       | 79 ++++++++++++----------- >>>>    tools/testing/selftests/mm/run_vmtests.sh | 23 +++++++ >>>>    tools/testing/selftests/mm/settings       |  2 +- >>>>    3 files changed, 64 insertions(+), 40 deletions(-) >>>> >>>> diff --git a/tools/testing/selftests/mm/Makefile >>>> b/tools/testing/selftests/mm/Makefile >>>> index 66d7c07dc177..881ed96d96fd 100644 >>>> --- a/tools/testing/selftests/mm/Makefile >>>> +++ b/tools/testing/selftests/mm/Makefile >>>> @@ -35,39 +35,39 @@ MAKEFLAGS += --no-builtin-rules >>>>    CFLAGS = -Wall -I $(top_srcdir) $(EXTRA_CFLAGS) $(KHDR_INCLUDES) >>>>    LDLIBS = -lrt -lpthread >>>>    -TEST_GEN_PROGS = cow >>>> -TEST_GEN_PROGS += compaction_test >>>> -TEST_GEN_PROGS += gup_longterm >>>> -TEST_GEN_PROGS += gup_test >>>> -TEST_GEN_PROGS += hmm-tests >>>> -TEST_GEN_PROGS += hugetlb-madvise >>>> -TEST_GEN_PROGS += hugepage-mmap >>>> -TEST_GEN_PROGS += hugepage-mremap >>>> -TEST_GEN_PROGS += hugepage-shm >>>> -TEST_GEN_PROGS += hugepage-vmemmap >>>> -TEST_GEN_PROGS += khugepaged >>>> -TEST_GEN_PROGS += madv_populate >>>> -TEST_GEN_PROGS += map_fixed_noreplace >>>> -TEST_GEN_PROGS += map_hugetlb >>>> -TEST_GEN_PROGS += map_populate >>>> -TEST_GEN_PROGS += memfd_secret >>>> -TEST_GEN_PROGS += migration >>>> -TEST_GEN_PROGS += mkdirty >>>> -TEST_GEN_PROGS += mlock-random-test >>>> -TEST_GEN_PROGS += mlock2-tests >>>> -TEST_GEN_PROGS += mrelease_test >>>> -TEST_GEN_PROGS += mremap_dontunmap >>>> -TEST_GEN_PROGS += mremap_test >>>> -TEST_GEN_PROGS += on-fault-limit >>>> -TEST_GEN_PROGS += thuge-gen >>>> -TEST_GEN_PROGS += transhuge-stress >>>> -TEST_GEN_PROGS += uffd-stress >>>> -TEST_GEN_PROGS += uffd-unit-tests >>>> -TEST_GEN_PROGS += soft-dirty >>>> -TEST_GEN_PROGS += split_huge_page_test >>>> -TEST_GEN_PROGS += ksm_tests >>>> -TEST_GEN_PROGS += ksm_functional_tests >>>> -TEST_GEN_PROGS += mdwe_test >>>> +TEST_GEN_FILES = cow >>>> +TEST_GEN_FILES += compaction_test >>>> +TEST_GEN_FILES += gup_longterm >>>> +TEST_GEN_FILES += gup_test >>>> +TEST_GEN_FILES += hmm-tests >>>> +TEST_GEN_FILES += hugetlb-madvise >>>> +TEST_GEN_FILES += hugepage-mmap >>>> +TEST_GEN_FILES += hugepage-mremap >>>> +TEST_GEN_FILES += hugepage-shm >>>> +TEST_GEN_FILES += hugepage-vmemmap >>>> +TEST_GEN_FILES += khugepaged >>>> +TEST_GEN_FILES += madv_populate >>>> +TEST_GEN_FILES += map_fixed_noreplace >>>> +TEST_GEN_FILES += map_hugetlb >>>> +TEST_GEN_FILES += map_populate >>>> +TEST_GEN_FILES += memfd_secret >>>> +TEST_GEN_FILES += migration >>>> +TEST_GEN_FILES += mkdirty >>>> +TEST_GEN_FILES += mlock-random-test >>>> +TEST_GEN_FILES += mlock2-tests >>>> +TEST_GEN_FILES += mrelease_test >>>> +TEST_GEN_FILES += mremap_dontunmap >>>> +TEST_GEN_FILES += mremap_test >>>> +TEST_GEN_FILES += on-fault-limit >>>> +TEST_GEN_FILES += thuge-gen >>>> +TEST_GEN_FILES += transhuge-stress >>>> +TEST_GEN_FILES += uffd-stress >>>> +TEST_GEN_FILES += uffd-unit-tests >>>> +TEST_GEN_FILES += soft-dirty >>>> +TEST_GEN_FILES += split_huge_page_test >>>> +TEST_GEN_FILES += ksm_tests >>>> +TEST_GEN_FILES += ksm_functional_tests >>>> +TEST_GEN_FILES += mdwe_test >>> >>> IIRC, we recently converted all to TEST_GEN_PROGS. See >>> >>> commit aef6fde75d8c6c1cad4a0e017a8d4cbee2143723 >>> Author: Peter Xu >>> Date:   Wed Apr 12 12:42:18 2023 -0400 >>> >>>      selftests/mm: use TEST_GEN_PROGS where proper >>>          TEST_GEN_PROGS and TEST_GEN_FILES are used randomly in the >>> mm/Makefile to >>>      specify programs that need to build.  Logically all these binaries should >>>      all fall into TEST_GEN_PROGS. >>>          Replace those TEST_GEN_FILES with TEST_GEN_PROGS, so that we can >>> reference >>>      all the tests easily later. >>> >>> >>> Why is that change required, and how does it interact with >>> run_kselftest.sh? (Not clear from you patch description.) >> >> TEST_GEN_PROGS will compile and install the tests and will add them to the list >> of tests that run_kselftest.sh will run. TEST_GEN_FILES will compile and install >> the tests but will not add them to the test list. >> >> Note that run_vmtests.sh is added to TEST_PROGS, which means it ends up in the >> test list. (the lack of "_GEN" means it won't be compiled, but simply copied). >> >> So with this change at the kselftest level, there is a single test in its list; >> run_vmtests.sh. And all the other tests that were previously in that list are >> moved into run_vmtests.sh (if they weren't there already). > > That sound good to me. (worth adding to the patch description) > > Let me CC Peter, so he's aware. Thanks - would be good to hear his opinion!