From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE8A0C02183 for ; Fri, 17 Jan 2025 15:41:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 539F06B0083; Fri, 17 Jan 2025 10:41:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4C3036B0085; Fri, 17 Jan 2025 10:41:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 33CD66B0088; Fri, 17 Jan 2025 10:41:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 14C836B0083 for ; Fri, 17 Jan 2025 10:41:49 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id BD2B51C75D2 for ; Fri, 17 Jan 2025 15:41:48 +0000 (UTC) X-FDA: 83017359096.15.3FE1F8C Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf21.hostedemail.com (Postfix) with ESMTP id 13E2A1C0003 for ; Fri, 17 Jan 2025 15:41:46 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=NFrrR7Tg; spf=pass (imf21.hostedemail.com: domain of will@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=will@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1737128507; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MJVay4JtNm+ElK3Qr+3HI+pwv2NwTf1AjBoH8vT6WQY=; b=L9t+3cc9Njx43hcOdAXqfCGr75z8aLLVDZPzyLobFeZ42zrU78Nps9dAJ91EbOZ/7e1tdb OZC5PXCXOfRulwrZ3H/IZybWsuHZp+4dp9g8scDXm45DHky2J/ZumMNzl+BprSmK7bGvxz pjw5zyTJvt5JF3u+F/J2ZoCk5rEeVck= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1737128507; a=rsa-sha256; cv=none; b=oT05LStpRwMaKwfmCow1MRWYFouyOxN6jT2wdYe5ymuU3JHK4ziM8VV792ualOLm09uux5 laX/rDW6Y4A7ucb+eRyM6K4vyDenmwGs4RxhjS6wGfQU3jLqxPzKRae0Ksg7CGP8LxqNIL 56Xkj1e06bSKIgATW+doQ8lIeVZRKis= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=NFrrR7Tg; spf=pass (imf21.hostedemail.com: domain of will@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=will@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 3591C5C6316; Fri, 17 Jan 2025 15:41:05 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 87D6AC4CEE0; Fri, 17 Jan 2025 15:41:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1737128505; bh=FMnEk5Wb+aQc5KN4VfScvTuPGnJ2n6y7XApOquJhgno=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=NFrrR7Tg1rMUFS1hWIZpjkLdInNmQzZSav5qtnreAH2cIeJCJ2KudEi7kqTndWV3n koZCL7z5zPO7vzlMfn4pL9D9y76u2r8g+Q1GUVV1nFrIwPS/wfulcec6eIpWAzXfQR 26gBxbjejRIEzCHdJgObLdRPJofXqZRbWRMaFV1MjSRQrwBcu20sFeKmZ+wcE1fKNY UkB5Tkwe6goPbVt1KIvhpJD/pafMZdq9SzLb3Eddxoducb6iVWAqCUx7HdbxzbvP4C fKt9lGQn+VBocWQxoEOaKbt4ynuZNbQEypL6dA1naTHcPkbr2S21y26K+YBE/9NKA0 TQuKVfx/OEOvQ== Date: Fri, 17 Jan 2025 15:41:36 +0000 From: Will Deacon To: Peter Zijlstra Cc: Suren Baghdasaryan , boqun.feng@gmail.com, mark.rutland@arm.com, Mateusz Guzik , akpm@linux-foundation.org, willy@infradead.org, liam.howlett@oracle.com, lorenzo.stoakes@oracle.com, david.laight.linux@gmail.com, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, oliver.sang@intel.com, mgorman@techsingularity.net, david@redhat.com, peterx@redhat.com, oleg@redhat.com, dave@stgolabs.net, paulmck@kernel.org, brauner@kernel.org, dhowells@redhat.com, hdanton@sina.com, hughd@google.com, lokeshgidra@google.com, minchan@google.com, jannh@google.com, shakeel.butt@linux.dev, souravpanda@google.com, pasha.tatashin@soleen.com, klarasmodin@gmail.com, richard.weiyang@gmail.com, corbet@lwn.net, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@android.com Subject: Re: [PATCH] refcount: Strengthen inc_not_zero() Message-ID: <20250117154135.GA17569@willie-the-truck> References: <20250111042604.3230628-1-surenb@google.com> <20250111042604.3230628-12-surenb@google.com> <20250115104841.GX5388@noisy.programming.kicks-ass.net> <20250115111334.GE8385@noisy.programming.kicks-ass.net> <20250115160011.GG8385@noisy.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20250115160011.GG8385@noisy.programming.kicks-ass.net> User-Agent: Mutt/1.10.1 (2018-07-13) X-Rspamd-Queue-Id: 13E2A1C0003 X-Stat-Signature: fnd6yc74sb9eskkk7qofk313miosnuxh X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1737128506-177788 X-HE-Meta: U2FsdGVkX18siFJuxU978uu4IXl7t0jgFAN7d8rdmW7jz8JcXU+O35IlDwjpImy+YoJ4QzOoIZDUDVPVRlm6mejSHpXqPjG4Zs1tM/8iUJDj4U0Z7wjBLfImISPA51l0TFwpxfnr1h6oImcy5EwWkvkW3kweW7uvK3r9Usugah6/5qs1mq0Sqo7vZLcPZhrgVqi90Ulss9dJywdw4or0A+bIEZgT4TiszM7AxJXQthveUb7TfnL4pfCDg08Q5D5fVLbTu3TZ066cdHHIAf9vxfWMtedtTbP7BpkzxFSWUKtbAZq66GsC7OW0kXEYVRqH5nWRicWFLsTXFW7Mg3aIR+D3cet6sP5kRaHAIzGYpXZKFHepklODUNJwgnXX0crh+Tg/I8WAVmdq9kcWsKUE6dGd8o8xRyUT+DddImdRSd5tB7gbZ8TtS7snCcbx1MRNWS3khrnbxIi2BHKrRMnh+YqYnhxpdscsJjRtU4creVLvZr6FWgwCA0psOOuYRx7gpU3Zu4roWRShbYEtcpcPJN3RL85B38oejt2+UlaPv5QAKM6om2eRKWbKJYhJI3vSdbRX4TZcZT5OVNsAta2ug3N5qgnIDiRSHh2eQoc/lCSnt2IlqWusu7XHLwSC4WlJJLMvr3CS1UzCkm+zO7psPtUEYmArD1a1eAo6boNzPUaf0B8NRzMLGeh5Qtlx2pvivrjHjuH1taA8SmdzUQBHzP5BGP2Q4ghK1ekzY8ffJoYIZxzldkncpSWjLgpGXFJvjLr1xfpWhe+ESajRrpCIXg2EyM9YwFsAivvHj8OMvAsvRmMLndXXcQfxg29zxpbLpL/MZje52bocFJ8q7sStHSpITxc2Yt3eWhXx4xcTwRy70hOn0yjWoxqgX9Urrf6OHO30Zo4SRKzMtyrHUiQfDr59mZb5/siEzin/PTewAAWrczpBwyp2xGObba5Vm2GJ8BuQgyJUlJeeSeg95zW sl+Bdnv8 ICop5+H0/xke6VafkrcwoObMLhX8kVRANLO33zg94lLTFy/Q9GaGMRsXJpj0aHci37RAyvJdlivqerh1uzYxRpPLbagHcF/SgRwnXfpIT8TrLqOvSeslKOJ+q+IVbrVpkQxfODJ45gZtmJdddy7fid4Rvp7GJpcvgjWdFRR8N/t0J0BziQvWjqdywCfqEyDMoZZqP1NUcUwTb2AI4vkx6Me8Nb0K9V4plG6/93tBKfqUk32s3dX7tc6NwTPs1Rp+8zfEJN1/ctoiWvYQr41yfIELEVR0JKTtJzHz4q5ffUOk3rFsvb+gf/SngkoDTLNbzTJN9NuycG5iTSt6OIK17/iApE82rUojB9tUUwwWtrZnri36e5DwcCUOhfBWzXxVZ/JMlLUEb8VM7R0o= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000023, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Jan 15, 2025 at 05:00:11PM +0100, Peter Zijlstra wrote: > On Wed, Jan 15, 2025 at 12:13:34PM +0100, Peter Zijlstra wrote: > > > Notably, it means refcount_t is entirely unsuitable for anything > > SLAB_TYPESAFE_BY_RCU, since they all will need secondary validation > > conditions after the refcount succeeds. > > > > And this is probably fine, but let me ponder this all a little more. > > Even though SLAB_TYPESAFE_BY_RCU is relatively rare, I think we'd better > fix this, these things are hard enough as they are. > > Will, others, wdyt? We should also update the Documentation (atomic_t.txt and refcount-vs-atomic.rst) if we strengthen this. > --- > Subject: refcount: Strengthen inc_not_zero() > > For speculative lookups where a successful inc_not_zero() pins the > object, but where we still need to double check if the object acquired > is indeed the one we set out to aquire, needs this validation to happen > *after* the increment. > > Notably SLAB_TYPESAFE_BY_RCU is one such an example. > > Signed-off-by: Peter Zijlstra (Intel) > --- > include/linux/refcount.h | 15 ++++++++------- > 1 file changed, 8 insertions(+), 7 deletions(-) > > diff --git a/include/linux/refcount.h b/include/linux/refcount.h > index 35f039ecb272..340e7ffa445e 100644 > --- a/include/linux/refcount.h > +++ b/include/linux/refcount.h > @@ -69,9 +69,10 @@ > * its the lock acquire, for RCU/lockless data structures its the dependent > * load. > * > - * Do note that inc_not_zero() provides a control dependency which will order > - * future stores against the inc, this ensures we'll never modify the object > - * if we did not in fact acquire a reference. > + * Do note that inc_not_zero() does provide acquire order, which will order > + * future load and stores against the inc, this ensures all subsequent accesses > + * are from this object and not anything previously occupying this memory -- > + * consider SLAB_TYPESAFE_BY_RCU. > * > * The decrements will provide release order, such that all the prior loads and > * stores will be issued before, it also provides a control dependency, which > @@ -144,7 +145,7 @@ bool __refcount_add_not_zero(int i, refcount_t *r, int *oldp) > do { > if (!old) > break; > - } while (!atomic_try_cmpxchg_relaxed(&r->refs, &old, old + i)); > + } while (!atomic_try_cmpxchg_acquire(&r->refs, &old, old + i)); Hmm, do the later memory accesses need to be ordered against the store part of the increment or just the read? If it's the former, then I don't think that _acquire is sufficient -- accesses can still get in-between the read and write parts of the RmW. Will