From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F00D5C636CC for ; Tue, 31 Jan 2023 20:52:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 328F86B0071; Tue, 31 Jan 2023 15:52:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2D8D46B0072; Tue, 31 Jan 2023 15:52:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1A04E6B0078; Tue, 31 Jan 2023 15:52:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 0A2E56B0071 for ; Tue, 31 Jan 2023 15:52:26 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id A516740CDC for ; Tue, 31 Jan 2023 20:52:25 +0000 (UTC) X-FDA: 80416292250.06.787AB2E Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf04.hostedemail.com (Postfix) with ESMTP id EDD1D40004 for ; Tue, 31 Jan 2023 20:52:23 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Xsaf6jaI; spf=pass (imf04.hostedemail.com: domain of "SRS0=0G0N=54=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org" designates 139.178.84.217 as permitted sender) smtp.mailfrom="SRS0=0G0N=54=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org"; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675198344; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RxyXq/5Xr6RG4WzK1DuYD5VvVo2gLi8GzYD8GY2UNpQ=; b=RPV2zRtUlX6Lbzp61m/A9/s5Sd4bGPQK1hnBerub+kak77eq5tSrN+QesORUP/cmynJn7I ZjwwU4NmWpURfGcKWe9J3wxFIB951mAtSBh0BRTjd53QqhauZ1VX+7WzCFBt8dyEFG/CIT mSxnlU59FaqCWGHEfwtzm2pH43A/xGk= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Xsaf6jaI; spf=pass (imf04.hostedemail.com: domain of "SRS0=0G0N=54=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org" designates 139.178.84.217 as permitted sender) smtp.mailfrom="SRS0=0G0N=54=paulmck-ThinkPad-P17-Gen-1.home=paulmck@kernel.org"; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675198344; a=rsa-sha256; cv=none; b=z5sp7YPMQfU2ikpKhgNZg6BESxc4cQy9TS3YPm8FSQFlajqi1WWKozDCYH5w9DZkeuzLaS sHQDTyWmZKlfokmQ7Mv5M6DETIoWW8LxnJWmOc5sg3JARjxe0Jw6uJ6jjo6dOSb2xY86zG Y7F42Q3/JP1ZU5BsrZSOQowZhnysIDk= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id F3A4B61537; Tue, 31 Jan 2023 20:52:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 55263C4339C; Tue, 31 Jan 2023 20:52:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1675198342; bh=OGoBKtZbg1Gxn6addTD9QmWM3qC69M1PJemUynGtSEY=; h=Date:From:To:Subject:Reply-To:References:In-Reply-To:From; b=Xsaf6jaIeAdOVNxk4xKGxZbbmwZpvLkujHyHNOpqNZISVqEFAEx6THr4cErEdoIXJ Wtmi6euUpqmjQxV+4Yo+cKawmw1cUOQxfkMZDTv8kekLT8/sWOnYICYwGCSu64aBLd vLpJqCHdp1YYenon4bSAhgik4jb+GxrrSYIywhub++KnM7OVr1jyLAwkpzp2xFbwDx q3g7FHIKkYThfi1/QA8VL85WJK5omxV2itfE3JrPV7mm78EAJyVvD0xTqu4at/mvd+ 0U17HH/1JPdXGeZnx4kR6gpzOTLu7kv6Z3vlYzj3LvED1QSaVc2rk61Qkjs7bNgYPy EFO2/DrVD64og== Received: by paulmck-ThinkPad-P17-Gen-1.home (Postfix, from userid 1000) id F1DD75C0510; Tue, 31 Jan 2023 12:52:21 -0800 (PST) Date: Tue, 31 Jan 2023 12:52:21 -0800 From: "Paul E. McKenney" To: "Liam R. Howlett" , kernel test robot , oe-lkp@lists.linux.dev, lkp@intel.com, linux-kernel@vger.kernel.org, Andrew Morton , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Subject: Re: [linus:master] [maple_tree] 120b116208: INFO:task_blocked_for_more_than#seconds Message-ID: <20230131205221.GX2948950@paulmck-ThinkPad-P17-Gen-1> Reply-To: paulmck@kernel.org References: <202301310940.4a37c7af-yujie.liu@intel.com> <20230131202635.GA3019407@paulmck-ThinkPad-P17-Gen-1> <20230131204520.ad6cf4lvtw5uf27s@revolver> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230131204520.ad6cf4lvtw5uf27s@revolver> X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: h6djzcmcajzqxst4t1p788who3n7u1sc X-Rspamd-Queue-Id: EDD1D40004 X-HE-Tag: 1675198343-585318 X-HE-Meta: U2FsdGVkX1/LR1qnV8k86/hZQbOMKQIzBgVvO4jrHmm92XBTA4dU7P04FSbdkcmz7NLJwyVaGh9CLTkT3MGU0O5XRYYKJ/6VLFQByTN3V6/v2jdbak1X9CyOb8hEEW3tDliW579eVcKUuPjyoeR5wNIRvhDGPfDiSN/Pp2A6NpOAwpEFqIjbbnNke5H1VGEt6yTziJ+50BnpyjjD3O+PH/ULr4Nx2URz+XKy85WfItXQvrjRcUifS+Fy2xOZlGGxEyhCTbGmm7FPIes2Qy4XAFatzg1S9wLQKYb2MYEh5a6B9z35VE8lBa4O9CCoiHixjceBZCD5Ldy0xbAWu9AvyDi+BcoStoA1H1PctVxYhLjIqSbxOMyV+piuwVXChW6vOxdYuQumoJRqfL8f1n5kf6fZjOyTf0bFBsOI1CI0aAhm0cpwN0Ll5cBfPtejymjRkzJNf+aqScVmNVJrsa0X/Fs9WgC4OYIwegbbk1lhHnpkV0pI/Ti5VhF/VhWV2d5c+ZYsqJlnATeVxeOkrDhSlujNEsksjtC4wTtQhnSP6PSFFkIWTgVBFMPC3ozCZmXyGeIVwMdZOULyIu7Emj/gW8pAXzpgjvL+wu5WVxbq85ZLYMjbI5iGkAZrafbwQMsQYmUkZMkg7TeRaoeEqzFZoeQtGRoJabSvIpEoa+yXyPvxjPgOkHaxDIe8eV5NhzFnhkiwKIOzWLF+utqHKclW659cwMXjJHq+4K/fg89LYaYWxLVh97qtrf7Yx87jZ9Ym4O1dXEX200M3+dCyNV+Ds2/HSmCqHz7pIpX9QFop5UutGkqwVnZ2ktJB2UThDqWIHC4YXej6/DRVtzxDLeIdS56JJW4lNfXHpup6KEdNDqraEPARhc2JWNvsIUk8FOTQtN2bq5zHcx/aPcmh7uDQZUH1ltGYYdFycRtZ328vHypN67ek+MkQjpeRILPb/hN2wI9VvttCbesPZf60prw BfyS0C0S tH5/t5BuADyrabEsatPT1H5oQmukNiDCK0bnFtjUy9H2Mr+sgnEKE79lsd6XyGhHcxpGAwch1eW7xXUCEql+r/FMUFt5cXb8g7ZRXe1+llzJEYpjZNTCX1+0yPSyPqnadNNZse5B41j4I82FUsooHjkvDmtLPkL2MJGWQ8azqzAdu+J2gFSCj8Kk9zhnUubA4jewLs3eHBAq7OHVgYv6FLg4YfTnNimz0s1FFqSZ7qX/qHQ2UVD6nFe7Pv/8kL+VF4iD7ka1r38cjKyKAo8cuDJlG6kkH46+V7q51YXXXOkh/g07tSHhc5+uRys2IPrykHSPgjcK8YFzxonpbb8i15oEPEU5uwzVtnwA7/W3FVdOedV+45+rZS0/snXq4rEQJPESs+XIv2NzwfoV/rsvFWg3N2d0gcN+Fz6+bBtlZdKIh8QH5AYBW03N/LaQyxkRWsYkukloyaCWYfWhrpuBu8OqEo2Y+4G6QesyOo599J3SrhKz9QoO2/7BjsO93RBEDPDHObMtTXASAagCAoRatix2fc2FVFMwnOu+8e/8ZE6H47SXdvlXeqqBZUrKO9XKbsbmUQWkMxUwS7zfASb6Oia0FhebRAXj0WMwOTowvU3ecdw9kGfOfkl3YHSgxxasGZG+vDzQ+nAXOL8hdywMRgBGlTzqOd1ngC3UuOvuZ4AQ4xvOWTLrEyTRWDnSmwUQwTd20m+2DBShu6+H76L5AJenZ/u6890+glzFd/v2jweLpDuXkb46hccVMHKrycAsEq2uFvEMSqmcjAyrU8ikDQ8rWEKYRemksIvFiCKznVq0gQnU/AniGdcOhyewXvJG1ciFj X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Jan 31, 2023 at 03:45:20PM -0500, Liam R. Howlett wrote: > * Paul E. McKenney [230131 15:26]: > > On Tue, Jan 31, 2023 at 03:18:22PM +0800, kernel test robot wrote: > > > Hi Liam, > > > > > > We caught a "task blocked" dmesg in maple tree test. Not sure if this > > > is expected for maple tree test, so we are sending this report for > > > your information. Thanks. > > > > > > Greeting, > > > > > > FYI, we noticed INFO:task_blocked_for_more_than#seconds due to commit (built with clang-14): > > > > > > commit: 120b116208a0877227fc82e3f0df81e7a3ed4ab1 ("maple_tree: reorganize testing to restore module testing") > > > https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master > > > > > > in testcase: boot > > > > > > on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 16G > > > > > > caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): > > > > > > > > > [ 17.318428][ T1] calling maple_tree_seed+0x0/0x15d0 @ 1 > > > [ 17.319219][ T1] > > > [ 17.319219][ T1] TEST STARTING > > > [ 17.319219][ T1] > > > [ 999.249871][ T23] INFO: task rcu_scale_shutd:59 blocked for more than 491 seconds. > > > [ 999.253363][ T23] Not tainted 6.1.0-rc4-00003-g120b116208a0 #1 > > > [ 999.254249][ T23] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. > > > [ 999.255390][ T23] task:rcu_scale_shutd state:D stack:30968 pid:59 ppid:2 flags:0x00004000 > > > [ 999.256934][ T23] Call Trace: > > > [ 999.257418][ T23] > > > [ 999.257900][ T23] __schedule+0x169b/0x1f90 > > > [ 999.261677][ T23] schedule+0x151/0x300 > > > [ 999.262281][ T23] ? compute_real+0xe0/0xe0 > > > [ 999.263364][ T23] rcu_scale_shutdown+0xdd/0x130 > > > [ 999.264093][ T23] ? wake_bit_function+0x2c0/0x2c0 > > > [ 999.268985][ T23] kthread+0x309/0x3a0 > > > [ 999.269958][ T23] ? compute_real+0xe0/0xe0 > > > [ 999.270552][ T23] ? kthread_unuse_mm+0x200/0x200 > > > [ 999.271281][ T23] ret_from_fork+0x1f/0x30 > > > [ 999.272385][ T23] > > > [ 999.272865][ T23] > > > [ 999.272865][ T23] Showing all locks held in the system: > > > [ 999.273988][ T23] 2 locks held by swapper/0/1: > > > [ 999.274684][ T23] 1 lock held by khungtaskd/23: > > > [ 999.275400][ T23] #0: ffffffff88346e00 (rcu_read_lock){....}-{1:2}, at: rcu_lock_acquire+0x8/0x30 > > > [ 999.277171][ T23] > > > [ 999.277525][ T23] ============================================= > > > [ 999.277525][ T23] > > > [ 1049.050884][ T1] maple_tree: 12610686 of 12610686 tests passed > > > > > > > > > If you fix the issue, kindly add following tag > > > | Reported-by: kernel test robot > > > | Link: https://lore.kernel.org/oe-lkp/202301310940.4a37c7af-yujie.liu@intel.com > > > > Liam brought this to my attention on IRC, and it looks like the root > > cause is that the rcuscale code does not deal gracefully with grace > > periods that are in much excess of a second in duration. > > > > Now, it might well be worth looking into why the grace periods were taking > > that long, but if you were running Maple Tree stress tests concurrently > > with rcuscale, this might well be expected behavior. > > > > This could be simply cpu starvation causing no foward progress in your > tests with the number of concurrent running tests and "-smp 2". > > It's also worth noting that building in the rcu test module makes the > machine turn off once the test is complete. This can be seen in your > console message: > [ 13.254240][ T1] rcu-scale:--- Start of test: nreaders=2 nwriters=2 verbose=1 shutdown=1 > > so your machine may not have finished running through the array of tests > you have specified to build in - which is a lot. I'm not sure if this > is the best approach considering the load that produces on the system > and how difficult it is (was) to figure out which test is causing a > stall, or other issue. Agreed, both rcuscale and refscale when built in turn the machine off at the end of the test. For providing background stress for some other test (in this case Maple Tree tests), rcutorture, locktorture, or scftorture might be better choices. Thanx, Paul