From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49041C7618D for ; Tue, 4 Apr 2023 02:50:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 98EBB6B0075; Mon, 3 Apr 2023 22:50:49 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 917DF6B0078; Mon, 3 Apr 2023 22:50:49 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7C17F6B007B; Mon, 3 Apr 2023 22:50:49 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 688AD6B0075 for ; Mon, 3 Apr 2023 22:50:49 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 30BA1140407 for ; Tue, 4 Apr 2023 02:50:49 +0000 (UTC) X-FDA: 80642181018.02.95F6F8C Received: from mail-oi1-f174.google.com (mail-oi1-f174.google.com [209.85.167.174]) by imf05.hostedemail.com (Postfix) with ESMTP id 69EDB100006 for ; Tue, 4 Apr 2023 02:50:47 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=MPXKFvVI; spf=pass (imf05.hostedemail.com: domain of yury.norov@gmail.com designates 209.85.167.174 as permitted sender) smtp.mailfrom=yury.norov@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680576647; a=rsa-sha256; cv=none; b=BXVpLGfN4Pa95sPzPZ3e9ks5rJTo89gCK3alVt5wHFWcwERyaghEBoB7QliJiu349ipcQT qwlB4WLsUg7ZxpwwrvAzmzV5huqElPxvUm7ViAadi7/P36w8LxCc/JvdGltIoSAr/Gamzh n23e6w7tevWbHSbaoLPdyP6y8t9ApDY= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=MPXKFvVI; spf=pass (imf05.hostedemail.com: domain of yury.norov@gmail.com designates 209.85.167.174 as permitted sender) smtp.mailfrom=yury.norov@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680576647; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XofY8x5MeQRpt2Pp1DmyLIEcV5jICvunInGd53WZu+o=; b=RemU2yJUG/vEq/y0FnCfdi+4oLHQpeDcs1AYiDS5kcWsM/zupf+ROmBIvtB4MobbCf/Xot 8PjGg/7iEUe9h1hD2fAf6TGvsTz6RKfnXJcUQ1lRFQDA5l1q2uS01tLvQdY258WJ7jUvpH 8NAr/8Fc0Z854+5UTSZ9yQ6XlseDeLU= Received: by mail-oi1-f174.google.com with SMTP id f14so9737689oiw.10 for ; Mon, 03 Apr 2023 19:50:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1680576646; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=XofY8x5MeQRpt2Pp1DmyLIEcV5jICvunInGd53WZu+o=; b=MPXKFvVIvzBj3nPjmR6E5eI+tv/dKzIV7fsnj5e0SNo3sXxOIfbBbMNPdL3lAydrpO 7zXFSHlQsfoy9seplKMVEyzLw6V/O7TqA+KfmsmJEuaFqFBFBnUF+3hDyUhWZjYhU5Vn KsFOqlTTtJPPBWbHjppI8Z0r1ctzgyPX2FLtNWOIGtHrEcpw4mqWQbQ1GoterWVdC+uk 0AOvTom9KbMDl3WmAEoJtDftEKeBGrLaLkzdh2yOrZAStVFJY6Uu7sViCCcLI9iCOYrr tzxN6r+5dc9HatSy5yHC04RXoEkwBC/GVyfgljAcuNvakDxHeLo5qACX9SR5aI83+xs6 1MJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680576646; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=XofY8x5MeQRpt2Pp1DmyLIEcV5jICvunInGd53WZu+o=; b=yojxNLytWNvIilJE5WmdsGVPzijOLPySNSDquT6gwaEVt62bGy8RyOwhVS6YniSlI0 m2Y0SLbX0VepH1ma8W1TZnt4u3jsNhwRWhb8Ek0kxwID05azk3AtVyj3TEzzHNZxT5PA hstSdF5wWjUJ2U+FWXukmNKX2DnkVjziTRbEv7MG4JQu3AHJOaPqkcbPIWLqPTzk/yBB YlH1EAQ9VLCRkEgLdys405Ef8vJwp92wfLZchFcGj1Cb4EPDMNVGJDu5KKewIRsaa0Vv MDofjc6dlYA6jSmHE3+K6FPFrYzOAjJ5K8uPZ0iRPx4Pmy19kdV5hyXlf2cJ9iL0Cd6I uXNg== X-Gm-Message-State: AAQBX9fi5JJYzbxyJf+dyzWvQ6Aqu3QDyQFObHUPcf4So19HgAbkngqJ JxSd1ipy++BJ9rW73WuKFiU= X-Google-Smtp-Source: AKy350Zu+r7jlAU33Q2ZbeqlfS7gzvqDQC5DO3QuZAPqWgw18Y+CQ1y/OlZ6bq27RCi5tqoiNHjtnA== X-Received: by 2002:aca:1812:0:b0:389:93a5:5b03 with SMTP id h18-20020aca1812000000b0038993a55b03mr652229oih.3.1680576646442; Mon, 03 Apr 2023 19:50:46 -0700 (PDT) Received: from localhost ([12.97.180.36]) by smtp.gmail.com with ESMTPSA id x14-20020a056808144e00b00387160bcd46sm4541289oiv.46.2023.04.03.19.50.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Apr 2023 19:50:45 -0700 (PDT) Date: Mon, 3 Apr 2023 19:50:44 -0700 From: Yury Norov To: Ye Bin Cc: dennis@kernel.org, tj@kernel.org, cl@linux.com, linux-mm@kvack.org, andriy.shevchenko@linux.intel.com, linux@rasmusvillemoes.dk, linux-kernel@vger.kernel.org, dchinner@redhat.com, yebin10@huawei.com Subject: Re: [PATCH 2/2] lib/percpu_counter: fix dying cpu compare race Message-ID: References: <20230404014206.3752945-1-yebin@huaweicloud.com> <20230404014206.3752945-3-yebin@huaweicloud.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230404014206.3752945-3-yebin@huaweicloud.com> X-Rspam-User: X-Rspamd-Queue-Id: 69EDB100006 X-Rspamd-Server: rspam01 X-Stat-Signature: 5eejqzn6eks6fs4g6afotja5whhb9qtk X-HE-Tag: 1680576647-362202 X-HE-Meta: U2FsdGVkX19lK4qn9xUSWoIl475lNoihF8AAojcQ144bj963pSPJEaiFurX4/llfZJCZ8oxSOnevAlLwaIShKshAX/5kPeWHg6UCNdc1ZK9YlAikcocnOJ9ONqqkfw+I6+JP/9xWwNrINNQpOrN5xYF9C/lOclDTVwEMU42H8QStqi/gwl7FU9WboW4eHZEPhjt9YD8sD45HjQJqICipKZYbRrYIQ3dUJnR8B/eNLinpbwz49uNXVoY4Ep1+pBqmur0b+g5EQTW3cOT28S0lnyKL3H8ywTGGMmaG4Ph6fvCPTRbbmdRRQyW5Zi1z15bXyu3D7i4Ly5+01T32SPh4M4fU+C1tna3PNde5N1Z/AelYQubFUT9zfDCnWjF7/lO/lmg4lxejnm376XuOKPIFBbj/iUIa2xAWDh0PAq9YY+2sW4NmAlN6hc9U3QpXXKrx7W9X/TOwyjqMK8kWJNUYD3IS1Zi9bpUH1dKzqUdAlD6NlyBLsPnrzC9+/eq8ixRxanVeP1kw6zeFqZCJdLLTVWnDt/UDXWs+Etehalsflijdzw/dqVzMuzeG8QaJWNnZjV1TvG6Bkc/ZEY4gnzgSMJrvNMcuOdH2TUY6h92kFnAfSCRQptH6KUrJ7vftGDm/lTIfUAodhsc+wsMsYN/6vY1X11ZdBdgiTBfeCMkc2SECdDhJRlyNG6mWylxCiduwzosnzIlZGWZgy2IwYYYb1Z4Wpj8++w8kDFgbxxzSpMpxlsEU7ULyqY7JGKf0qh7N3KRlckSBuF+sszxz0HIqTboK6RafRVS+Ra9oG3+dlhyQZTRYfZxl5sj7ZujOO3skgpW1a4yPmGquYTO1j2RfoZ7/7NpLF4UUoAIvnMaD4fYtprvBLJssK92TQEt4rwaba6D/+ElxuGTRNAJkwqjhi5TRAuMSI2+ACpAD3sd5jVuAvM1rMeOEgUQ1KLcWIPXpAIXcjmv7N2dzqSPTPAP MYI5sEjc uPPdUuJ4aCclGjJNSywpIz5WsFCRr+wg17eZOLtbVjje86lDAXz/Jcv5A8C1hB985b7Oa/VPHb2CI2o1lev5rUFWfUoigHzBIqfa0ylcQv1FCWXyXUC+N5rEgN8CdzVJEh4xQQjmqmVvI2CpsCZ8zygi9XzXBRyHpdEAfJ1EM01/KsL/t09kDeFAn+6y2XFibKPJ1c0a3fHpioD4yC2FzoHJJMEXZ5DyvBPK4bDHZZ+qTuZEmqqKbIo+9kYIlbRXu+1cgGOpqrLPtEUwvFs1GlGU/gP+7nWBqXampOpnGoOh0FrP/eSgpGqiziEFCG67J9xk1L3H6KeY2G9v4ouDKuYPjegGnnUa345Megc0tuiNprwUdHUcS4msmEwktbg7Y8eEkl4MaOyCRu+p/s5+NCCjvh3OyNvtSm0xHV6KXZQWxEw55q53yGXhk/GVIzVyTZ2rINktFIS0JFIxcryG+im3HvH+TgIIWT1u5+iA8wfsG2Ee38wpHNNHMgugYXbLhLPMGhpnX7XjHldjO66/xqF0RKVpBO5ktF0Nm64Qodeqx4Iw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000017, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Apr 04, 2023 at 09:42:06AM +0800, Ye Bin wrote: > From: Ye Bin > > In commit 8b57b11cca88 ("pcpcntrs: fix dying cpu summation race") a race > condition between a cpu dying and percpu_counter_sum() iterating online CPUs > was identified. > Acctually, there's the same race condition between a cpu dying and > __percpu_counter_compare(). Here, use 'num_online_cpus()' for quick judgment. > But 'num_online_cpus()' will be decreased before call 'percpu_counter_cpu_dead()', > then maybe return incorrect result. > To solve above issue, also need to add dying CPUs count when do quick judgment > in __percpu_counter_compare(). Not sure I completely understood the race you are describing. All CPU accounting is protected with percpu_counters_lock. Is it a real race that you've faced, or hypothetical? If it's real, can you share stack traces? > Signed-off-by: Ye Bin > --- > lib/percpu_counter.c | 11 ++++++++++- > 1 file changed, 10 insertions(+), 1 deletion(-) > > diff --git a/lib/percpu_counter.c b/lib/percpu_counter.c > index 5004463c4f9f..399840cb0012 100644 > --- a/lib/percpu_counter.c > +++ b/lib/percpu_counter.c > @@ -227,6 +227,15 @@ static int percpu_counter_cpu_dead(unsigned int cpu) > return 0; > } > > +static __always_inline unsigned int num_count_cpus(void) This doesn't look like a good name. Maybe num_offline_cpus? > +{ > +#ifdef CONFIG_HOTPLUG_CPU > + return (num_online_cpus() + num_dying_cpus()); ^ ^ 'return' is not a function. Braces are not needed Generally speaking, a sequence of atomic operations is not an atomic operation, so the above doesn't look correct. I don't think that it would be possible to implement raceless accounting based on 2 separate counters. Most probably, you'd have to use the same approach as in 8b57b11cca88: lock(); for_each_cpu_or(cpu, cpu_online_mask, cpu_dying_mask) cnt++; unlock(); And if so, I'd suggest to implement cpumask_weight_or() for that. > +#else > + return num_online_cpus(); > +#endif > +} > + > /* > * Compare counter against given value. > * Return 1 if greater, 0 if equal and -1 if less > @@ -237,7 +246,7 @@ int __percpu_counter_compare(struct percpu_counter *fbc, s64 rhs, s32 batch) > > count = percpu_counter_read(fbc); > /* Check to see if rough count will be sufficient for comparison */ > - if (abs(count - rhs) > (batch * num_online_cpus())) { > + if (abs(count - rhs) > (batch * num_count_cpus())) { > if (count > rhs) > return 1; > else > -- > 2.31.1