From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1107E77188 for ; Mon, 6 Jan 2025 10:18:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3AD696B0088; Mon, 6 Jan 2025 05:18:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 35D326B008A; Mon, 6 Jan 2025 05:18:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 224EC6B008C; Mon, 6 Jan 2025 05:18:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 071F96B0088 for ; Mon, 6 Jan 2025 05:18:26 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id A3E02140506 for ; Mon, 6 Jan 2025 10:18:25 +0000 (UTC) X-FDA: 82976627370.21.B552F60 Received: from mail-vk1-f177.google.com (mail-vk1-f177.google.com [209.85.221.177]) by imf08.hostedemail.com (Postfix) with ESMTP id 834D2160005 for ; Mon, 6 Jan 2025 10:18:23 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=none; spf=pass (imf08.hostedemail.com: domain of geert.uytterhoeven@gmail.com designates 209.85.221.177 as permitted sender) smtp.mailfrom=geert.uytterhoeven@gmail.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1736158703; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VFiw2m0qYuDUxpznSFuo6WroJW1SYr3XFORf2X9DILQ=; b=A03wwua45X5vfacEVSnm89qqSnVVGXh4amWCT9HdCX/TPBDNTg1IjorEuVoXsk/AkPsjKd i5MSc0TJf2wpoZWHO4XvmQXUWKoZNFy2bHKOZ65yjKN/SinJUXZVxITm87Pzf/VeF+HSx4 XQHWHz9HvPocuhx9P6Q0lrqAKl4ZJwM= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=none; spf=pass (imf08.hostedemail.com: domain of geert.uytterhoeven@gmail.com designates 209.85.221.177 as permitted sender) smtp.mailfrom=geert.uytterhoeven@gmail.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1736158703; a=rsa-sha256; cv=none; b=E8iMOkqhUpKY0LhbONeo0S4ZeL5FWGIDgB1cmhiaQTabqGQDAV4XJ1QcvOgUlaMHP+wvBV 80CSlCpos1CbuEXComT0m8o7nz0atiwHneFciWcdS9K6k/7uRbjkjPtpjWVHZR56fL0Vmu O+Sf/zXxbH9mDD68DZkzt6eCBMsIYBw= Received: by mail-vk1-f177.google.com with SMTP id 71dfb90a1353d-5174f9c0d2aso4328283e0c.1 for ; Mon, 06 Jan 2025 02:18:23 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736158701; x=1736763501; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VFiw2m0qYuDUxpznSFuo6WroJW1SYr3XFORf2X9DILQ=; b=Xcn2Pn/XZzYjOLOOYPdK/9MvFP+4TmDo9vIseY0M5iHz/cDwBegu6eBxpfdYR/VlbR bAOtVsH4XZeeBrctIAYXF8a37yIQILvjSozTRkHdGC/Afn3C4JcDjams/AHAKmu1rY9x FozFpqGFenbZLpDzNcF+TZm4c+6+Iqj/n07JrFMKK8/s2snyADMkRNTK6HssH15uAyJ7 4qL3JROrbc3OSz0YcsUpc5TZyW74AupBYjYmsr0Gy8ksesNxUFID318XNA6uXfT4EfSG lbXmXXlXPcmwq2182h1hZBX5pQxXitnTEfJ5fsGu6QpPwdi3TcJKEYKdCwOmFTsFCDH2 DRaA== X-Forwarded-Encrypted: i=1; AJvYcCXztm1NiatW5tbLPh5LxLTcgK3LNT3ZbWaWqB4zwkj8lJLi+ApV11ArG7m2Yxd2RiyljooyAe4TCw==@kvack.org X-Gm-Message-State: AOJu0Yzl01p+qJZK2Qytx+3feRnigFYBYflrGvRz9+2g4f+RTzrI9nKK 4gTT2JYk1Kd10ETGjUzDkp9Sq93FYvBpYbAF/9C/aVr24JyYk9UNlvHtRULU X-Gm-Gg: ASbGnctsCLqCpkUVXBd0MMxUsFNi5UxclI++dMplNpO7NkjEZswbEHV9sU9KGJ3UV9y j5FDRZv4DU9HMQGEJHTJWcxIdEDA63eWeaO9e09szLt6biCw+jQq8wlD4BsDW1IBmZiaae5juPg AyMEP3oU4Xn3k41K6TTAdvhG2Wx+qE+rB41sOaw6tiWPCbWgZgGg0zPU4iNllqvmhAsIWFvYZtj 28bxYH3g6u81EC+6VAgiMNA3bU7xhtj8bA7et4CZ+T/vdFzgdPEUqXziFrPwJ3dY8EiDZXQzStc WxfIosiFVIDd4JjE8ZRuHgE= X-Google-Smtp-Source: AGHT+IGOH0pVqD53TXIIs5Uf5uxtXoWYTpzeHSQpJPuRaXEEPKploSpCyR8JtOGwsCZNDQh+SVJS2A== X-Received: by 2002:a05:6102:548f:b0:4af:f740:c1b4 with SMTP id ada2fe7eead31-4b2cc319f2fmr43409044137.5.1736158700900; Mon, 06 Jan 2025 02:18:20 -0800 (PST) Received: from mail-vk1-f175.google.com (mail-vk1-f175.google.com. [209.85.221.175]) by smtp.gmail.com with ESMTPSA id a1e0cc1a2514c-8610ac4c1e5sm7265968241.10.2025.01.06.02.18.20 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 06 Jan 2025 02:18:20 -0800 (PST) Received: by mail-vk1-f175.google.com with SMTP id 71dfb90a1353d-51ba75ee2f3so3747536e0c.2 for ; Mon, 06 Jan 2025 02:18:20 -0800 (PST) X-Forwarded-Encrypted: i=1; AJvYcCU62Es+e56e6XuxeRt5o4q0XsnvuTCI41QV8rNuNl1yf20iGDfoVr3W6T9Zdnltp+A8dd7Cp4cwkQ==@kvack.org X-Received: by 2002:a05:6122:318f:b0:518:a2af:d666 with SMTP id 71dfb90a1353d-51b75c30618mr35360461e0c.1.1736158700259; Mon, 06 Jan 2025 02:18:20 -0800 (PST) MIME-Version: 1.0 References: <20241221033321.4154409-1-koichiro.den@canonical.com> <2q7ge6cgzeowqffyn6w6ed4trhaaumv5ubdgud2tsoolen7wpw@4akuomhbacyh> In-Reply-To: <2q7ge6cgzeowqffyn6w6ed4trhaaumv5ubdgud2tsoolen7wpw@4akuomhbacyh> From: Geert Uytterhoeven Date: Mon, 6 Jan 2025 11:18:08 +0100 X-Gmail-Original-Message-ID: X-Gm-Features: AbW1kvbMfnk2SgBXKVUIV0YROpavUz-OLtnVYxy0TWeDhPN2Kq31j9pnnSSq8o0 Message-ID: Subject: Re: [PATCH v2] vmstat: disable vmstat_work on vmstat_cpu_down_prep() To: Koichiro Den Cc: Lorenzo Stoakes , linux-mm@kvack.org, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, Linux-Renesas Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Server: rspam05 X-Stat-Signature: pai6syzra6y4obq3jfasjjeu31uytpog X-Rspamd-Queue-Id: 834D2160005 X-Rspam-User: X-HE-Tag: 1736158703-655023 X-HE-Meta: U2FsdGVkX1/H9FEX7eFl2CwiV141nAhHieD1SQ1xJmMsbnfcQ5drDM/+Nvb8YBAD0EqU9EnlLKyM1PdPeypYkTkFiuOqgOI6QZPrfPictJVC+mL2WPBdC2pu91AddXC/IWyidRD1nCXroAMpeD31J+g7JjsmsviOEjjXQqHqSwZ/4AndovAo7haVHM/L/khJ3WtIQCYWeJCJMG28LHtTfxPIkUGXSNp49V+n4TjVJd1DW6yMvoWTwdznBYKJTIcOp7kjjMZEU9ryNchJH4gcUNwwqK88qBmP1VwMNoisH45knSQ46fwxpU3XKDIFJ07exH0j31IHXNdgq6Jwpbkz75WPpEsh9HQrm5KFrwnwNhs0w/EMQro8PbHmATjZiGtC17RED592rpXWg5hW5uj1lcTVxhvd2w1hUKiKw7l/dPZAHUv2ZQJxxnSugzsTxTQ/aaYpPzBQBt7nCHjQe8zkR17qTriD8/PM0M/HV0EfzYS/zENIvK3ysxhXdM8w4gnZ9z2oqIpRRI83V2mTb9HdefdAajhqrzHg83afq8YA9k9Fu20fZrDglQ4Fm4hd5yzXayH7bF7ZW358QEuBSt4KGooyIglOjBAGZWfIPJz4Vqi7aMKQ/77mRtWJQCrKdfnu4yk75t6lv4Dq3WeqJb+gFPos0K+TVvuxikov8Uv+L/tCrNsha5kUkwZst8+7IDyxUd6j4mOTynmK6nbG5Elkaop9cydeO0ml0JCyUsgu5ixlfHWZZCZFqftUj8Mz0ZffxbunkygGQiaQQZXrt4gpUq5+sKI/bHWYg/k0x3X0DCH1Eu6IynytRoVeqSkb2DUWv8YKO8k/e+zTUTVxzFcKgW3Bk7MaU+QVCmB/bU2RELerueBd7bS9J/+68ab2sLCcluRCTCwK8znWfESV0glPHBFQjTRJNgM/rzaol1c9pAlvYJr76YIh8+aVE+1wKont9ROejBb/fIxpWx0ckJD uo7ge4sz zZP6zP10Fx43ecrWWHqmk0fSXDKxxLbqVBrVxz7zr8OKZWgHESc+wZkd+q4FIHOUpYAGqA4z2X0PU795jgZfAXOeevrIcqR6ZafuwVywaZLa4gUimT0TK8dZnBa6gwR+5Mn03pnZ/49qnID1hAgqhxAR8br73zx5vPtaQKbsmdb5chfo9j5vCaHQsUwVqKGuVacDROxZ8KVhGfXFDNNWw5DtbLQ7ZiiAbgn26TiC5CV6NYuFArnCYX7dX9oGak3NkXJeLlilf+kWhey6vw0edxgMdMtf21I6wnD+rCzCoc9uxtdrXtS1nN1UNBPIK9zepiYOaGn42OcZbJYZWHZgfMytiJY4FCP3g64dyYbx1BtbVo/lrMOtjgrXtMLurKDU5QCF+HO7DETn0myDfhMJJAoPQS9A6J7sazLsI8rzLJZN/uWJSdcB5btmDaWLjHWanudCEDccEwtk8IW+xXCmVsnbMxpK2B101Qv5laHvnDj41JYSX8wWVLS9yF16+lmJ1dF2c X-Bogosity: Ham, tests=bogofilter, spamicity=0.000005, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hi Koichiro, On Sat, Jan 4, 2025 at 5:00=E2=80=AFAM Koichiro Den wrote: > On Fri, Jan 03, 2025 at 11:33:19PM +0000, Lorenzo Stoakes wrote: > > On Sat, Dec 21, 2024 at 12:33:20PM +0900, Koichiro Den wrote: > > > Even after mm/vmstat:online teardown, shepherd may still queue work f= or > > > the dying cpu until the cpu is removed from online mask. While it's > > > quite rare, this means that after unbind_workers() unbinds a per-cpu > > > kworker, it potentially runs vmstat_update for the dying CPU on an > > > irrelevant cpu before entering atomic AP states. > > > When CONFIG_DEBUG_PREEMPT=3Dy, it results in the following error with= the > > > backtrace. > > > > > > BUG: using smp_processor_id() in preemptible [00000000] code: \ > > > kworker/7:3/1702 > > > caller is refresh_cpu_vm_stats+0x235/0x5f0 > > > CPU: 0 UID: 0 PID: 1702 Comm: kworker/7:3 Tainted: G > > > Tainted: [N]=3DTEST > > > Workqueue: mm_percpu_wq vmstat_update > > > Call Trace: > > > > > > dump_stack_lvl+0x8d/0xb0 > > > check_preemption_disabled+0xce/0xe0 > > > refresh_cpu_vm_stats+0x235/0x5f0 > > > vmstat_update+0x17/0xa0 > > > process_one_work+0x869/0x1aa0 > > > worker_thread+0x5e5/0x1100 > > > kthread+0x29e/0x380 > > > ret_from_fork+0x2d/0x70 > > > ret_from_fork_asm+0x1a/0x30 > > > > > > > > > So, for mm/vmstat:online, disable vmstat_work reliably on teardown an= d > > > symmetrically enable it on startup. > > > > > > Signed-off-by: Koichiro Den > > > > I observed a warning in my qemu and real hardware, which I bisected to = this commit: > > > > [ 0.087733] ------------[ cut here ]------------ > > [ 0.087733] workqueue: work disable count underflowed > > [ 0.087733] WARNING: CPU: 1 PID: 21 at kernel/workqueue.c:4313 enabl= e_work+0xb5/0xc0 I am seeing the same on arm32 (R-Car M2-W) and arm64 (R-Car H3 ES2.0). > Thank you for the report. I was able to reproduce the warning and now > wonder how I missed it.. My oversight, apologies. > > In my current view, the simplest solution would be to make sure a local > vmstat_work is disabled until vmstat_cpu_online() runs for the cpu, even > during boot-up. The following patch suppresses the warning: > > diff --git a/mm/vmstat.c b/mm/vmstat.c > index 0889b75cef14..19ceed5d34bf 100644 > --- a/mm/vmstat.c > +++ b/mm/vmstat.c > @@ -2122,10 +2122,14 @@ static void __init start_shepherd_timer(void) > { > int cpu; > > - for_each_possible_cpu(cpu) > + for_each_possible_cpu(cpu) { > INIT_DEFERRABLE_WORK(per_cpu_ptr(&vmstat_work, cpu), > vmstat_update); > > + /* will be enabled on vmstat_cpu_online */ > + disable_delayed_work_sync(&per_cpu(vmstat_work, cpu)); > + } > + > schedule_delayed_work(&shepherd, > round_jiffies_relative(sysctl_stat_interval)); > } > > If you think of a better solution later, please let me know. Otherwise, > I'll submit a follow-up fix patch with the above diff. Thank you, that fixes the warnings for me! Tested-by: Geert Uytterhoeven Gr{oetje,eeting}s, Geert -- Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k= .org In personal conversations with technical people, I call myself a hacker. Bu= t when I'm talking to journalists I just say "programmer" or something like t= hat. -- Linus Torvalds