From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D36D6C433EF for ; Thu, 21 Oct 2021 13:31:59 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 80C5E60EE2 for ; Thu, 21 Oct 2021 13:31:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 80C5E60EE2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvack.org Received: by kanga.kvack.org (Postfix) id D487A940008; Thu, 21 Oct 2021 09:31:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CF8BA940007; Thu, 21 Oct 2021 09:31:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BE7B9940008; Thu, 21 Oct 2021 09:31:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0239.hostedemail.com [216.40.44.239]) by kanga.kvack.org (Postfix) with ESMTP id B11A8940007 for ; Thu, 21 Oct 2021 09:31:58 -0400 (EDT) Received: from smtpin38.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 6C7FC29305 for ; Thu, 21 Oct 2021 13:31:58 +0000 (UTC) X-FDA: 78720532716.38.06A2F30 Received: from mail-wm1-f44.google.com (mail-wm1-f44.google.com [209.85.128.44]) by imf21.hostedemail.com (Postfix) with ESMTP id E35AED046481 for ; Thu, 21 Oct 2021 13:30:55 +0000 (UTC) Received: by mail-wm1-f44.google.com with SMTP id y16-20020a05600c17d000b0030db7a51ee2so7266983wmo.0 for ; Thu, 21 Oct 2021 06:30:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=E65W5rHcRhwN48L7vjwZwLiwyU/1qg2FCqyrEx/BzFo=; b=A4Db4wOCuN4vfKzOeCn6fAKweaT7ITHTa5kuw6/YGTHV6ttOFo4kWjnssYmv6jaMRF Gfdn8DL2mVcKkW0lSly0/ROor4GCHXnrCLqdMfxgfi3dJ1lKXAXAf6tMX5QCfQidGUDE 0OaQh9/xTdsXXR7jMZFU2Vsz5Yrrt0HFMwWABV7sPUbhezb/e0+SDUu34ytTTM1T3FG4 PBSpUjg0jhAn+7XY7icgULt3E/wMvO9xuUvG04YXVTZSuix9L23CxbxMWIA2TAsWijzO AkTTnnvjdR30umdjFNPLqg2mgn63z4bcPUh7IsIP5FGG7JXHe7aus0aVs38HCSbJj5O4 vuhQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=E65W5rHcRhwN48L7vjwZwLiwyU/1qg2FCqyrEx/BzFo=; b=xvPCFewUyRrmQXwkH6KtOijJkULTO3gc35CW+E7XjcYpuo8yZzvrSnKydlVuT47UJ9 3UnlZ8gW6RPOSeUxnOOZuiBQs7HHpCqqqGVJ/3s3z7T5SqLyi/yklj8KNC/HQJPmE/Yw W/i66tMKCnRcyWxhy/iC1AjZSz6aPg9N1V6L9xebc17YZtigJYjeHAIyPHgP+OIs8pAz Lg0PbjEUxUVw/lvMjrArd+/bf+13LCf4iXrFToYIlTwhOO+7mUj5ci6CkQhNt4b1O/Qc wqXMz+x0Q9Os9LZy4ATY+Yv+rgOcDJTHTNvJ33jd/GLb+CPzebDZ3AZvEADh58pGln6s sU7Q== X-Gm-Message-State: AOAM532a/diNW9wsUwjt7M40urYr3xOQAmxXyhulpK/SFJRBcvpVl6Cr HvdT8uMx8sXkdVbPYQaKnc8= X-Google-Smtp-Source: ABdhPJyNo19TUXl1tAqCwlEvPsAOMdycnYyGTzCu7W5uXLrZT4E7XWnC+iUocVNRyw6J8dkZ2OfdNw== X-Received: by 2002:a1c:f609:: with SMTP id w9mr6721166wmc.24.1634823056793; Thu, 21 Oct 2021 06:30:56 -0700 (PDT) Received: from 127.0.0.1localhost ([185.69.145.206]) by smtp.gmail.com with ESMTPSA id q188sm1753987wma.1.2021.10.21.06.30.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Oct 2021 06:30:56 -0700 (PDT) From: Pavel Begunkov To: linux-block@vger.kernel.org Cc: Jens Axboe , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Christoph Lameter , Tejun Heo , Dennis Zhou , Pavel Begunkov Subject: [PATCH v2 1/2] percpu_ref: percpu_ref_tryget_live() version holding RCU Date: Thu, 21 Oct 2021 14:30:51 +0100 Message-Id: <3066500d7a6eb3e03f10adf98b87fdb3b1c49db8.1634822969.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.33.1 In-Reply-To: References: MIME-Version: 1.0 X-Stat-Signature: aw3m5pe7rw1if5d8chr8ogynz33oakyb Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=A4Db4wOC; spf=pass (imf21.hostedemail.com: domain of asml.silence@gmail.com designates 209.85.128.44 as permitted sender) smtp.mailfrom=asml.silence@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: E35AED046481 X-HE-Tag: 1634823055-823283 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add percpu_ref_tryget_live_rcu(), which is a version of percpu_ref_tryget_live() but the user is responsible for enclosing it in a RCU read lock section. Signed-off-by: Pavel Begunkov --- include/linux/percpu-refcount.h | 33 +++++++++++++++++++++++---------- 1 file changed, 23 insertions(+), 10 deletions(-) diff --git a/include/linux/percpu-refcount.h b/include/linux/percpu-refco= unt.h index ae16a9856305..b31d3f3312ce 100644 --- a/include/linux/percpu-refcount.h +++ b/include/linux/percpu-refcount.h @@ -266,6 +266,28 @@ static inline bool percpu_ref_tryget(struct percpu_r= ef *ref) return percpu_ref_tryget_many(ref, 1); } =20 +/** + * percpu_ref_tryget_live_rcu - same as percpu_ref_tryget_live() but the + * caller is responsible for taking RCU. + * + * This function is safe to call as long as @ref is between init and exi= t. + */ +static inline bool percpu_ref_tryget_live_rcu(struct percpu_ref *ref) +{ + unsigned long __percpu *percpu_count; + bool ret =3D false; + + WARN_ON_ONCE(!rcu_read_lock_held()); + + if (likely(__ref_is_percpu(ref, &percpu_count))) { + this_cpu_inc(*percpu_count); + ret =3D true; + } else if (!(ref->percpu_count_ptr & __PERCPU_REF_DEAD)) { + ret =3D atomic_long_inc_not_zero(&ref->data->count); + } + return ret; +} + /** * percpu_ref_tryget_live - try to increment a live percpu refcount * @ref: percpu_ref to try-get @@ -283,20 +305,11 @@ static inline bool percpu_ref_tryget(struct percpu_= ref *ref) */ static inline bool percpu_ref_tryget_live(struct percpu_ref *ref) { - unsigned long __percpu *percpu_count; bool ret =3D false; =20 rcu_read_lock(); - - if (__ref_is_percpu(ref, &percpu_count)) { - this_cpu_inc(*percpu_count); - ret =3D true; - } else if (!(ref->percpu_count_ptr & __PERCPU_REF_DEAD)) { - ret =3D atomic_long_inc_not_zero(&ref->data->count); - } - + ret =3D percpu_ref_tryget_live_rcu(ref); rcu_read_unlock(); - return ret; } =20 --=20 2.33.1