From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57E2CC46CD2 for ; Sat, 30 Dec 2023 22:39:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5DFAE6B0083; Sat, 30 Dec 2023 17:39:27 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 58EB36B00BD; Sat, 30 Dec 2023 17:39:27 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 409216B00D2; Sat, 30 Dec 2023 17:39:27 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 29F0F6B0083 for ; Sat, 30 Dec 2023 17:39:27 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id D3887C040F for ; Sat, 30 Dec 2023 22:39:26 +0000 (UTC) X-FDA: 81624952332.12.3FDB2AE Received: from mail-oa1-f43.google.com (mail-oa1-f43.google.com [209.85.160.43]) by imf01.hostedemail.com (Postfix) with ESMTP id 1C5D040014 for ; Sat, 30 Dec 2023 22:39:23 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=none; spf=pass (imf01.hostedemail.com: domain of dennisszhou@gmail.com designates 209.85.160.43 as permitted sender) smtp.mailfrom=dennisszhou@gmail.com; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=kernel.org (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1703975964; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=s95fd8CWPyRAX6w7x1MUkDahCkeW5Ikcu9kmhfogHKg=; b=OmCyhse//3TeF0pQrqmGfU0p/cy8MLt3sLg21KBG8Y9EFpyfyIL+SYCf5yjzlxljvtjI6a /NsKNRFx50CygeVWH9TGPC6MwNFUX0F48AOpX/eTuiJaOYLRAAx4anCWLFBpwticgp7EI+ Fa7koL7VLHMf6QUJn3BtzawROt8a2Rk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1703975964; a=rsa-sha256; cv=none; b=kDEOT4HeuHZRQlWQirTj8xwm9LLJeG3SZveNbGZ2s1Jj1j2RQBGgBAQUQnGesP7f72snfO GCP8+4csAJPVmHEST0bscOgObbycmbXM9O1OCpk8czwo6MashBnTnWcaX9fOvNFmsuPbHO qyZUkBSofgAdSPL5KJ5EgBShAYIYWSA= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=none; spf=pass (imf01.hostedemail.com: domain of dennisszhou@gmail.com designates 209.85.160.43 as permitted sender) smtp.mailfrom=dennisszhou@gmail.com; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=kernel.org (policy=none) Received: by mail-oa1-f43.google.com with SMTP id 586e51a60fabf-204fdd685fdso1812113fac.2 for ; Sat, 30 Dec 2023 14:39:23 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703975963; x=1704580763; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=s95fd8CWPyRAX6w7x1MUkDahCkeW5Ikcu9kmhfogHKg=; b=Vk3yWjf/zambpiO2PeS3JKXcCl8B+Vc4LZZQR+dYaj2xBR7BFj9XCbRSxCrXeagvTz 4O+Z2JBU08lhBOH2VEK/v0/eJek4/+e6rKxWOWGilc3czc/TgicgP/8bHIaJV3sgO2Xa rXvVztaYrV3en2oVYi+MDkvTE0qif1A7mtR+JLxtOJlg+WBaBRtpSjE/48Gw8YfbJ5Dv s/u1uY4AlGPAIdmCMJvbki6+RULXge4Sa0CWkDsw6juEnBQVEt99Tzltv4yd1UpCaRyD 3CZ3AIuRuutMSNaSg6IyooAga88g49QoHcEq8MwejKeUxjP7Xd5UwpgFlefkZVnLWqMp FU5Q== X-Gm-Message-State: AOJu0YzA6+/WgwmzgIAU+lBwCGgeNX/JI+74WO+/OKMR+6a2XvLdbjYE xH4MCUq+qCl5qxTN7I8AjIg= X-Google-Smtp-Source: AGHT+IFrO7weYWKnkqic556yhkSML59TQmslPEqzkVBx96V3jRUrKOBahBWgHXSgw4PuAoG+DF2AYA== X-Received: by 2002:a05:6870:65a4:b0:204:805:54ef with SMTP id fp36-20020a05687065a400b00204080554efmr17861120oab.86.1703975963073; Sat, 30 Dec 2023 14:39:23 -0800 (PST) Received: from snowbird ([136.25.84.107]) by smtp.gmail.com with ESMTPSA id l16-20020a656810000000b005cdfa6f5aeesm12361472pgt.94.2023.12.30.14.39.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 30 Dec 2023 14:39:22 -0800 (PST) Date: Sat, 30 Dec 2023 14:39:20 -0800 From: Dennis Zhou To: Thomas Gleixner Cc: LKML , Peter Zijlstra , Valentin Schneider , Tejun Heo , Christoph Lameter , Dave Chinner , Yury Norov , Andy Shevchenko , Rasmus Villemoes , Ye Bin , linux-mm@kvack.org Subject: Re: [patch 0/3] lib/percpu_counter, cpu/hotplug: Cure the cpu_dying_mask woes Message-ID: References: <20230414162755.281993820@linutronix.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230414162755.281993820@linutronix.de> X-Stat-Signature: 5rmnfuq1qokkpefxji6k963kux4iu77q X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 1C5D040014 X-Rspam-User: X-HE-Tag: 1703975963-816167 X-HE-Meta: U2FsdGVkX19hZIq2+5iemkb0XB3WEDW8Z4bYUii99NcAvX7N+PRCmUFkGAZiUQEQYdAHeyyYSX0/FaledAd/kmfdW7vjnhP4qtsrDdoZ423lQXJxUCDJIH9M6A3KBcGUYoY2QRiIsrojOvG7DMwWT7/YR2cjAshBXAZ64IglWMv/PrVeG13HgOISbxhiSlPkyIQeBxm8Wx9bRSgHEkMlannUHAp7CrZc1Qg8FFRc/F4TLCQcWMYq6OhB51a89NOttp1wGajDLRz8vHSu9VqmXoe1iv5fMQWk25h+eVoxYNZpkkjs1AQO2vLD/+PIfPiXxXI1ilFWP33ZetQhF5KnX1w202EaZ0SwoEzvZhB7978odbmqRl5sUFo2Hq8AtHaqgHGe75DMJCoDxNL4PhXBsSyDAQf4pF7+0Tw3Jr533fW6a4aMtqw0Rz0MKqsDe2l8fs2VkoKv2cHLnbb0ZyJfyl4u7s/zACdmDMPizBsDFMwpgpssWO0XnchELrmMuFAYvIKYuGUCOa9Uy6cV5YNbBATciBIDFcOkLVdGwD/MgMv5IMmYxnN4MMd1B6rD8E/TJA9fLLYQxHx0TdtlSsS/MnYOPQFE731SjhRbfPsQNlcURIch36iJBHQWSza+6Cm3/Ny/oWS9pk9onftQYlv4y5XQ/NcX70sDW+JI5qQnnXlDQ/WQT6j/4Lp0jA49bg2GhJPucki4+Etq9sn2++lDCBO3nraMnMU729r9dBvcXKhhxlluw+UtHXlJtd6tCRByPMtC96B4rF34nUNFY0drZn0Kkx/E4I9JIxxs6tPPMtSW8sLm2x6S9DngmH9x/yJS8FsdC8o/J8EKYGKgzFIN4EbKeUo72hczglMwXVfnsOCHxwNCsEmVWMQobsRXMfdTrLVfDa4FhroOMZW72qAf1xtPjV3Vh2/X2dk7ucgxT3foD2n7fnTIrZ5PZpNUR6AnSJVKER4fNWSKM8lB9Gd MnBhOCNI wptFTesqOnm9ulHvDq8tAxr9xXT+YsX0fBzP8nnoUccs0TX74bmLNo5LxRy4G4u6aIHVSXNaNkT3QjljgKM7W95GD7uor8c6Kaj717M7YXiii56NN9Q3vMvwEhdNCky86tR8k8XfZEs0GVaQXNGRWfMz3zrdGnxTkGYK3N47JJD2ier/nXyumgBtsBjLTamZqOYs8RDxe67H0NUUcsnmqoSG3CfZ1g1JygmMzlbKAa3HTM50Mgld+hcVfTOd9n7H744jZxwlSo55wwCGTHb2NbjaBxieZq+RswiDNL9d1DMS4/v1LgGkIxmFbf1Ob5jGoZ0mk4ufFM7sTAt4EgLHwg7vB4cwH2s9EUiTNn5QvbG2QIaPEtTx0nYoPiSGOM9h36iPDG/OsQEhJLyh2dPjVjk/FVTg8+SA7De3CRYNIQDxN2Ch8JkDv0p6lpOz+dRLJQKRMcKJY9rKco9mJ5kDJ+jHD4X7CeeUa7bwfiAg6wq++8gTDZ2cXHS/cfflvJRhPPXmiLzwzye11ON3k847ANnDdeAKB6sELXjV5C0GS/dPgpf7pKrKnUJNCwqMKNrOm6vZrHTHfyRIYwzFT/Qh1YajxWCfm1F3502XmVg01673eNVjw8R0EYYwHWOqTLgJNcM9sU2DUfxFLg78= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Hello, On Fri, Apr 14, 2023 at 06:30:42PM +0200, Thomas Gleixner wrote: > Hi! > > The cpu_dying_mask is not only undocumented but also to some extent a > misnomer. It's purpose is to capture the last direction of a cpu_up() or > cpu_down() operation taking eventual rollback operations into account. > > cpu_dying mask is not really useful for general consumption. The > cpu_dying_mask bits are sticky even after cpu_up() or cpu_down() completes. > > A recent fix to plug a race in the per CPU counter code picked > cpu_dying_mask to cure it. Unfortunately this does not work as the author > probably expected and the behaviour of cpu_dying_mask is not easy to change > without breaking the only other and initial user, the scheduler. > > This series addresses this by: > > 1) Reworking the per CPU counter hotplug mechanism so the race is fully > plugged without using cpu_dying_mask > > 2) Replacing the cpu_dying_mask logic with hotplug core internal state > which is exposed to the scheduler with a properly documented > function. > > The series is also available from git: > > git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git smp/dying_mask > > Thanks > > tglx > --- > include/linux/cpuhotplug.h | 2 - > include/linux/cpumask.h | 21 ---------------- > kernel/cpu.c | 45 +++++++++++++++++++++++++++++------ > kernel/sched/core.c | 4 +-- > kernel/smpboot.h | 2 + > lib/percpu_counter.c | 57 +++++++++++++++++++-------------------------- > 6 files changed, 67 insertions(+), 64 deletions(-) This has been on my mind and regretfully it's been a busy year for me. I know the merge window is around the corner, but I rebased this series onto percpu#for-6.8 [1]. I had to massage percpu_counter slightly due to some changes but other than that it largely is intact. I need to do a little bit of a more thorough pass and re-send it out, but I think it remains correct to merge. I can then pull it, give it a few days to soak in for-next and then send it to Linus either in a follow up PR or in the 2nd week of the merge window. Thomas, how does this sound to you? [1] https://git.kernel.org/pub/scm/linux/kernel/git/dennis/percpu.git/log/?h=percpu-hotplug Thanks, Dennis