From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E819DCD342B for ; Mon, 2 Sep 2024 11:55:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 81C436B008A; Mon, 2 Sep 2024 07:55:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7CAB08D0065; Mon, 2 Sep 2024 07:55:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 694C66B00D7; Mon, 2 Sep 2024 07:55:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 493218D0065 for ; Mon, 2 Sep 2024 07:55:07 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id ABBE681D02 for ; Mon, 2 Sep 2024 11:55:06 +0000 (UTC) X-FDA: 82519642212.26.86C4E3C Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) by imf26.hostedemail.com (Postfix) with ESMTP id CD924140007 for ; Mon, 2 Sep 2024 11:55:04 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=linutronix.de header.s=2020 header.b=0cQML09U; dkim=pass header.d=linutronix.de header.s=2020e header.b=lkIJ4uXa; dmarc=pass (policy=none) header.from=linutronix.de; spf=pass (imf26.hostedemail.com: domain of tglx@linutronix.de designates 193.142.43.55 as permitted sender) smtp.mailfrom=tglx@linutronix.de ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725278029; a=rsa-sha256; cv=none; b=hg58qeeaGEjTzwsUNyRYOtPcjAnPSaQfCxV6rZDaSjFCsE3RT5uX4y699RZPEPA9pz1XBC S8PJWA5AnneROKdv1GfbNVZSmuFeL7jZNR1rIAHIq+trdfA2xWaAhYEGla41XbBdNmWFnR LhbB9LbbF6XhO6B36LsNQuRmF08tXpI= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=linutronix.de header.s=2020 header.b=0cQML09U; dkim=pass header.d=linutronix.de header.s=2020e header.b=lkIJ4uXa; dmarc=pass (policy=none) header.from=linutronix.de; spf=pass (imf26.hostedemail.com: domain of tglx@linutronix.de designates 193.142.43.55 as permitted sender) smtp.mailfrom=tglx@linutronix.de ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725278029; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=sEgsc5hfwM3ia/sAR7ebt4TKSr5Pvzs7BQG/4VSVeIk=; b=oYfXi+y/zeJz7yRDBguD9JQCUKFrxNX7v5e3wJEwl+y3w4XPTAK5i0CcqIiIcfwFm1MQj8 hOMGT6Wa0do8UR4MFLc/Z7e5vxk2baMTeH98YVUtG3Q/PbGzRRyr4HFle//5YqdBW+0fjA LumXxI8+16K+Cbwda/Zk9kWmarAWET4= From: Thomas Gleixner DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1725278102; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=sEgsc5hfwM3ia/sAR7ebt4TKSr5Pvzs7BQG/4VSVeIk=; b=0cQML09UZi5csPnYHKk9/AcDeo9RrboR8toJS2MoJJxZtJjbZQO4er92aqP37pf3tF3KYo B7CsiytHbvp5JAwoUmaIFKDOljiwHBgHKsSbpSFyGL77SHKiXzKbc7rIuOBbjjJletV5b5 +vQAtzPInUxUb96foq9u3SAulT2wCQ3fTM4ktJcOtpBTTJY2KF69ybew7eI9dU8mPwtkjc l9h4jNj+5gzVVrr8YA+VUFwl6tCDu5tSbfP71X/5oRYupvqqqBsNyp4SSjpJhTvSMDK/Ov VZTUa0/3lgBgYuGrbaZ6GPNdxKy1fsCZ6lWqX8xPEan5Agc2UzhfbDcontaS9Q== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1725278102; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=sEgsc5hfwM3ia/sAR7ebt4TKSr5Pvzs7BQG/4VSVeIk=; b=lkIJ4uXaxPac8zjaUxEtoY+WcZYF7rh33Smg10Am/Xgg45lhdJBXhlNLzsejtGadHI4nEz TuBxXTEcTfvmpXCA== To: "Christoph Lameter (Ampere)" Cc: Christoph Lameter via B4 Relay , Catalin Marinas , Will Deacon , Peter Zijlstra , Ingo Molnar , Waiman Long , Boqun Feng , Linus Torvalds , linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-arch@vger.kernel.org Subject: Re: [PATCH v2] Avoid memory barrier in read_seqcount() through load acquire In-Reply-To: References: <20240819-seq_optimize-v2-1-9d0da82b022f@gentwo.org> <87ttfbeyqt.ffs@tglx> Date: Mon, 02 Sep 2024 13:55:01 +0200 Message-ID: <871q226zje.ffs@tglx> MIME-Version: 1.0 Content-Type: text/plain X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: CD924140007 X-Stat-Signature: 6aibkrkuaan1iono1fjdxkdhkt3jwmzh X-Rspam-User: X-HE-Tag: 1725278104-928264 X-HE-Meta: U2FsdGVkX19gLtKZif7D2tE2MAxHq0ImVXb1iYzgjBdWTZv/szv0bUdLnawhynV+3GBFLcUKZWtqWmnFv7zaB76kqyKUOzHygKIpGZvubcnbZRDAQSoMzK7ahaxHnOCK+3ZxCySTw+J1gKKT1o6428TsV53CqzZjld3AFX7EB6811CTX7admkKXDsFs9aAhWnRAwANZWo3fDbCqWlQd1ZPh2xNQB4FvIIFlP6ci1MnjhoF/yWstycpsQjFvjCBw1UJgPE+gJaAr+Cy8eR+g3yfUD6TKhntNy9HBVpaEH6yIOHydTiCEmByB9kygfkLRS17wG6FB/UvGJNr607UEdG0lb2l4lJYPknPFnLCapmtK8lZ4T0ebKOoTFhp9r23DJY47SvNl2lqbCrsK0wKxbSxYh9HR8iIhZMYmiqr0L3sMykPOl1QKwAyYBcMU/GlaVmn5G/8rxkJTrDeH/uSj2cu9DrkCopPC1YvVH8dUjEARXxUKpOc7fbIbSJX/V3s+D+uuQ5XGck9dCTnrRAfffKL1vJSwDUC8JsaxuPTDUjzka8kWZ4LzQKWNOrrOYAnOMQRIdZihLlQMAKQ1UnUXcRbnhKn7mqBhtqGa47RInGTaMZrqir7Q4PsN0ghs5vmpGGBWGTPpY1518dOgqgocpcsmLICJ9big3ehPI2oBXpVz/WJAB765Sl9kcmPdlrKycGvhmBJUj/ZOAxJ5f9mhpmbtf7avY4TPqeQm99bqE6EmhCZFCo8qdPH0Ar0HAdhBQ7gqg3Xip/ztAcN38/7+KSmYtaAmKBGqdfkTyBV2aq9IrFqfgmz7uzi7tbxEJkiwRGXucmBjEwOk/Qi0FRTvZcLxjSehlMIxfUSKpQYcb7a0sSUtq6pAjXdGzUBvRTvFfQlJjLrtQo2I+EPKhbN7Dlku0jyNTmPCaWLaiY69zdNIfdR/kJGPaALxV4mgqNM+S3rEjFYJZV+M+oXZMK9t 7FK5Em8o RtlW6e8Zlt7+Ji3jhDFIn8YmWn+pqs0U6kgHjlXfBRkJb5kY59b11ElZoKWtIKodDq9cm72d/QSBZo/ukXJiN2pc8ivPxUn3ZZ8FvGYxyKJJ7lBahdEO7QExfveDYrKHz4Xc8q/ZaNFcrkOAc+jPLlEEqlkC/8n2blC6w1vvN+OgJdfxCadOeEVn3rlXb6Wcw7WnBzLahtesI4g4XtvKAqG5yWp0XPZq9IfNiSpGA5go2rb23xUyp2IOeffOFQvvqZTLr02UHxshCNkeycrfMZMhql0NBKTd79w9Q X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, Aug 28 2024 at 10:15, Christoph Lameter wrote: > On Fri, 23 Aug 2024, Thomas Gleixner wrote: > >> This all can be done without the extra copies of the counter >> accessors. Uncompiled patch below. > > Great. Thanks. Tried it too initially but could not make it work right. > > One thing that we also want is the use of the smp_cond_load_acquire to > have the cpu power down while waiting for a cacheline change. > > The code has several places where loops occur when the last bit is set in > the seqcount. > > We could use smp_cond_load_acquire in load_sequence() but what do we do > about the loops at the higher level? Also this does not sync with the lock > checking logic. Come on. It's not rocket science to figure that out. Uncompiled delta patch below. Thanks, tglx --- --- a/include/linux/seqlock.h +++ b/include/linux/seqlock.h @@ -23,6 +23,13 @@ #include +#ifdef CONFIG_ARCH_HAS_ACQUIRE_RELEASE +# define USE_LOAD_ACQUIRE true +# define USE_COND_LOAD_ACQUIRE !IS_ENABLED(CONFIG_PREEMPT_RT) +#else +# define USE_LOAD_ACQUIRE false +# define USE_COND_LOAD_ACQUIRE false +#endif /* * The seqlock seqcount_t interface does not prescribe a precise sequence of * read begin/retry/end. For readers, typically there is a call to @@ -134,10 +141,13 @@ static inline void seqcount_lockdep_read static __always_inline unsigned __seqprop_load_sequence(const seqcount_t *s, bool acquire) { - if (acquire && IS_ENABLED(CONFIG_ARCH_HAS_ACQUIRE_RELEASE)) - return smp_load_acquire(&s->sequence); - else + if (!acquire || !USE_LOAD_ACQUIRE) return READ_ONCE(s->sequence); + + if (USE_COND_LOAD_ACQUIRE) + return smp_cond_load_acquire(&s->sequence, (s->sequence & 1) == 0); + + return smp_load_acquire(&s->sequence); } /* @@ -283,8 +293,12 @@ SEQCOUNT_LOCKNAME(mutex, struct m ({ \ unsigned __seq; \ \ - while ((__seq = seqprop_sequence(s, acquire)) & 1) \ - cpu_relax(); \ + if (acquire && USE_COND_LOAD_ACQUIRE) { \ + __seq = seqprop_sequence(s, acquire); \ + } else { \ + while ((__seq = seqprop_sequence(s, acquire)) & 1) \ + cpu_relax(); \ + } \ \ kcsan_atomic_next(KCSAN_SEQLOCK_REGION_MAX); \ __seq; \