From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 62865C5472C for ; Sun, 25 Aug 2024 17:31:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C903F8D000E; Sun, 25 Aug 2024 13:31:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8ECFC8D000D; Sun, 25 Aug 2024 13:31:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6F2008D000E; Sun, 25 Aug 2024 13:31:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 30BE78D000E for ; Sun, 25 Aug 2024 13:31:50 -0400 (EDT) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id B72711C3D48 for ; Sun, 25 Aug 2024 17:31:49 +0000 (UTC) X-FDA: 82491460338.14.184C903 Received: from nyc.source.kernel.org (nyc.source.kernel.org [147.75.193.91]) by imf20.hostedemail.com (Postfix) with ESMTP id 1F07D1C000F for ; Sun, 25 Aug 2024 17:31:47 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=bKDVzr3+; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf20.hostedemail.com: domain of will@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=will@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724607089; a=rsa-sha256; cv=none; b=U0upaYO8rqyj3brVCYe1HJrHYC+h9aIXi+mUrl7uXNXQesHVG4FNdLXMl6rr+HebRtAVF8 SYy/aU7JXDoRvY57l9ixXbH/g7tnrohl8fegU1ZOuXl6Aqf2A76NUiTsIe12ralwkrtlbz QASPGFLUBVtLdQWWyaiTYTVWbv8PfkU= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=bKDVzr3+; dmarc=pass (policy=quarantine) header.from=kernel.org; spf=pass (imf20.hostedemail.com: domain of will@kernel.org designates 147.75.193.91 as permitted sender) smtp.mailfrom=will@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724607089; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vtam5RvHjK4SML60YYQw89qII1QxrrjuShYWxr06+Ok=; b=IUv9wSwPWgRv/FA196Sz28aWfSvrzB8EsU8mZPSz8y6ndzmT5pcEGP1cK9yx2IZBWgsb41 khxMKattnC5xf41sOLB7UEAczArXu8knJbAC45bkhr0i1RPrSUjTYqnOt9RZajyTih0dZm FqjoOpYKJA79JC64NByQFg7uQb9Rl0o= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 5E7DCA400A1; Fri, 23 Aug 2024 10:32:10 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id BE55CC32786; Fri, 23 Aug 2024 10:32:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724409136; bh=+TdwiAS6oo7wWEadkXTiyzSakrmyZjhYan1C9YRRgug=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=bKDVzr3+xoQNHI1mEkBcxBhyyOjdbFC9sKLrM+TZP/aFgP/3h02VE7a8TG8C25T0O C4+SoHQrL+QYhUY32G6Gv2wIfSoHgNmGvWVASTJjK0eAquCXLBZhJQGGzthVRXPmx6 zPBFRuonuzbw9QzRFWEihMk0eUjmNXul4TZdgYP+C+lw+9y9txCC4JUVNowFYLnf8U e52r03GztBcGl5KJ+abU00lsCAm1y3/K/rJNbXBf4mOcJwYIu65S0dkzcHcX/ssMFT pZmjHmudm7kgVGyu4klnLXCRVu5voSRvVgRqoAjkpqcN7GSfFAE/ubg3T8VARSLcax kHTyDe+XDdMTQ== Date: Fri, 23 Aug 2024 11:32:11 +0100 From: Will Deacon To: cl@gentwo.org Cc: Catalin Marinas , Peter Zijlstra , Ingo Molnar , Waiman Long , Boqun Feng , Linus Torvalds , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-arch@vger.kernel.org Subject: Re: [PATCH v2] Avoid memory barrier in read_seqcount() through load acquire Message-ID: <20240823103205.GA31866@willie-the-truck> References: <20240819-seq_optimize-v2-1-9d0da82b022f@gentwo.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240819-seq_optimize-v2-1-9d0da82b022f@gentwo.org> User-Agent: Mutt/1.10.1 (2018-07-13) X-Rspam-User: X-Rspamd-Queue-Id: 1F07D1C000F X-Rspamd-Server: rspam01 X-Stat-Signature: d6kmax46o5xtuqkpiagks6un996r8nr1 X-HE-Tag: 1724607107-796695 X-HE-Meta: U2FsdGVkX1/ob4bI8bhr7n3STFTWCZXUNQBGTuKXDxo0Su7A/lUhFX62xuO1vy0Hcxs8qj6RqG5RVvmxEpay6V7E4MCl7iDvKFmYYdPqGEHwPu6Hcy+kAN/wYiNgJSse3m38oGZi4X5gBIY+doiGqyRh8Px/pNL1xoGAqHBsl28zX1RXNIo8ptR+avoy9FLBY6dWDZ077OX/bftntQH0x09M8C9LHbaecNjyW1E1a64a2T7dhcIhVRimeXThhxKlmuu0UiioSg7aSWRb5j4xsiqQw87BzuNkE7QBt8UVYW2AlpvbO+tClF7DPegUrbZVIiNI3drgVNNuNP/aOJ9vHw8Xxo/6+Iy96N3cDELGrszLn1+LV8sxy2yQKzh4zWn99SHonM06Ny3SGhpfffr/AFLSAbJDVQvkTLw79x4H0XzhKk9hu2f/GxNa4GrlLm/zf1GrUzbavnWNMjwDZClo97zDMtR9nF0JwuQro0HWJGtPMUaK4PMd1+bhcOAw3gWtR51zu6L0ml5oPGF11qVuoDUbizsTf2lxXsqocmtfz9qiwyD1FvtwgF0BtXbGVKuioZg4zTOHJTKyEbKBSZdMyK9saUG72LeHocw/31CgRDweQqzfmEsOHnlvVg6ipMpSavoJQQO8t5pGZgUpybxxztUxB1FR4GGdPY2HYa5Tk28tFCLSzv9uu2CZXrgZalRvW4Hwio/At1PYKrRjWkiIBpqZSCG2lyrT7FTdKekTzDeIWi/64HtvONFnbCwAclyCMhlx3cwcESJOf8usHEYdA+fFLoB1rFdVXG+w7kCjvb4Hl7t8815aJCkDk+61SbQ/foAhmXV2YSMdv5+/LsatfyfLvNaPM3PG0RxQ66XNTzlsSxIJmY4RLiI3A1fmd4j28Zx5uKmJK69Qtmlhg1lu9neD4+6HsxNKrI11Bi6oZ2mtw+6wdPJUAm4QXPfoZMRWy3vxFWlqxiGtLG3NM1h /C0YNTlb RciNNEdMVVy03IrQ9kBrbX5URH8atVfVFI+Pk+9JRvOFQEWKabwKnonko6pxJPXAYJIWutXARSfoSeRXu89emhkWWbIaWu7sf+NPiSC2HIJTmJ/h76VMPU8LTthkAwU2ClELufYS7up2i7HQgoTJHRNRB/WniYRdMqn0QvkPJBAx4d5YFjfj1ByVA9Ij55w7JNpd0A0eAqYYYz6BWPLppNVd409KNRZbTEyRH7+VHWK0VLSyDRVclAzXqF2JpXnsDfUDKOW4Hlbe/x8fAJMDWmtWt5jJ3+eZ5OV11UrAb/U+D1IPYpBq/gQx/inQibabsohTms05RiBpsIHPul2DmwwhdYKsQxzmiZ94OkxVGQMX9Ufsu4oidinaGygaDISA6Na9Aq8Q+XVpPMfRxEXcU238Gyg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Mon, Aug 19, 2024 at 11:30:15AM -0700, Christoph Lameter via B4 Relay wrote: > diff --git a/include/linux/seqlock.h b/include/linux/seqlock.h > index d90d8ee29d81..353fcf32b800 100644 > --- a/include/linux/seqlock.h > +++ b/include/linux/seqlock.h > @@ -176,6 +176,28 @@ __seqprop_##lockname##_sequence(const seqcount_##lockname##_t *s) \ > return seq; \ > } \ > \ > +static __always_inline unsigned \ > +__seqprop_##lockname##_sequence_acquire(const seqcount_##lockname##_t *s) \ > +{ \ > + unsigned seq = smp_load_acquire(&s->seqcount.sequence); \ > + \ > + if (!IS_ENABLED(CONFIG_PREEMPT_RT)) \ > + return seq; \ > + \ > + if (preemptible && unlikely(seq & 1)) { \ > + __SEQ_LOCK(lockbase##_lock(s->lock)); \ > + __SEQ_LOCK(lockbase##_unlock(s->lock)); \ > + \ > + /* \ > + * Re-read the sequence counter since the (possibly \ > + * preempted) writer made progress. \ > + */ \ > + seq = smp_load_acquire(&s->seqcount.sequence); \ We could probably do even better with LDAPR here, as that should be sufficient for this. It's a can of worms though, as it's not implemented on all CPUs and relaxing smp_load_acquire() might introduce subtle breakage in places where it's used to build other types of lock. Maybe you can hack something to see if there's any performance left behind without it? > + } \ > + \ > + return seq; \ > +} \ > + \ > static __always_inline bool \ > __seqprop_##lockname##_preemptible(const seqcount_##lockname##_t *s) \ > { \ > @@ -211,6 +233,11 @@ static inline unsigned __seqprop_sequence(const seqcount_t *s) > return READ_ONCE(s->sequence); > } > > +static inline unsigned __seqprop_sequence_acquire(const seqcount_t *s) > +{ > + return smp_load_acquire(&s->sequence); > +} > + > static inline bool __seqprop_preemptible(const seqcount_t *s) > { > return false; > @@ -259,6 +286,7 @@ SEQCOUNT_LOCKNAME(mutex, struct mutex, true, mutex) > #define seqprop_ptr(s) __seqprop(s, ptr)(s) > #define seqprop_const_ptr(s) __seqprop(s, const_ptr)(s) > #define seqprop_sequence(s) __seqprop(s, sequence)(s) > +#define seqprop_sequence_acquire(s) __seqprop(s, sequence_acquire)(s) > #define seqprop_preemptible(s) __seqprop(s, preemptible)(s) > #define seqprop_assert(s) __seqprop(s, assert)(s) > > @@ -293,6 +321,18 @@ SEQCOUNT_LOCKNAME(mutex, struct mutex, true, mutex) > * > * Return: count to be passed to read_seqcount_retry() > */ > +#ifdef CONFIG_ARCH_HAS_ACQUIRE_RELEASE > +#define raw_read_seqcount_begin(s) \ > +({ \ > + unsigned _seq; \ > + \ > + while ((_seq = seqprop_sequence_acquire(s)) & 1) \ > + cpu_relax(); \ It would also be interesting to see whether smp_cond_load_acquire() performs any better that this loop in the !RT case. Both things to look at separately though, so: Acked-by: Will Deacon I assume this will go via -tip. Will