From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF438C4332F for ; Tue, 20 Dec 2022 11:08:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 446168E0002; Tue, 20 Dec 2022 06:08:56 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 3F62D8E0001; Tue, 20 Dec 2022 06:08:56 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2BD2D8E0002; Tue, 20 Dec 2022 06:08:56 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 1EA8C8E0001 for ; Tue, 20 Dec 2022 06:08:56 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id E00A3AB24F for ; Tue, 20 Dec 2022 11:08:55 +0000 (UTC) X-FDA: 80262412230.05.907C9AF Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) by imf01.hostedemail.com (Postfix) with ESMTP id 0766B4001B for ; Tue, 20 Dec 2022 11:08:51 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=IagLEV2U; dmarc=none; spf=none (imf01.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=peterz@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1671534534; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=WnDr2T6Lz6o8EGXfFOLVGTm1EwVLYgoHb8GLpQHXt1M=; b=KGuy1XtYHowhAoTfwWbYU0T3qgo7ip3TPlswpm239wx1upzHwrBp7t9c9uqXhZyjrY7ZF0 ZsPggG2GIoYbpFPv87rGC4TSllZYhGn9QTxanECL6zi85Xdr8dFqX3zRXGasO5ENhN/loF pqgj0tfFg6I5lDDoh7cORBixsl1byi8= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=infradead.org header.s=desiato.20200630 header.b=IagLEV2U; dmarc=none; spf=none (imf01.hostedemail.com: domain of peterz@infradead.org has no SPF policy when checking 90.155.92.199) smtp.mailfrom=peterz@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1671534534; a=rsa-sha256; cv=none; b=KBuyCygN5Z/OAm4JKGR/FJzqF1xReoNJrNuNuFvlaULVJKc+9F2Ljp1Q21byH54QuQk+2R Ov6dfs79LrG43ZgqxUUg8Erh1yxSIYu8tmL/aFqcJSbP2LpkQA0Z7ItetBzzbKgEocnfVl cDxMvLyuXGf6S59D7ABg9rwYjPMFaH0= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=WnDr2T6Lz6o8EGXfFOLVGTm1EwVLYgoHb8GLpQHXt1M=; b=IagLEV2UTnOH/1oiatZU2t3fd9 wIPIif+7ieQr0xntpK0GaWRRyiwDimnm1M6YG2G9sgsXB6FHTn576bYXZ2XsZQu0X8SVskwk0RdED sUugD1rIZDNEonT+Wrqt9z6gKuYGTXksjAvjtBZde5JzbCpvKPzWPD1B3Cr1mg/gBpkJd++WwKlHn CyrczxUMFGQp9teEMndrfFHW1fsEnOaw7HKW7iIuwSjcBQVnSoFpre5ReinrnsDbQ5HAqkui68Umh mhCAarnFYTE260WL9Igxo03K9hMfZoyNoHum8h5NGIXp5g8yzrlrLDnscp3XKQ468GvNz5tPnXOvK 5BahRGCw==; Received: from j130084.upc-j.chello.nl ([24.132.130.84] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.96 #2 (Red Hat Linux)) id 1p7aTs-00Cy6n-0d; Tue, 20 Dec 2022 11:08:20 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id E507F300023; Tue, 20 Dec 2022 12:08:16 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 1000) id 767AE201D1375; Tue, 20 Dec 2022 12:08:16 +0100 (CET) Date: Tue, 20 Dec 2022 12:08:16 +0100 From: Peter Zijlstra To: Boqun Feng Cc: torvalds@linux-foundation.org, corbet@lwn.net, will@kernel.org, mark.rutland@arm.com, catalin.marinas@arm.com, dennis@kernel.org, tj@kernel.org, cl@linux.com, hca@linux.ibm.com, gor@linux.ibm.com, agordeev@linux.ibm.com, borntraeger@linux.ibm.com, svens@linux.ibm.com, Herbert Xu , davem@davemloft.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, joro@8bytes.org, suravee.suthikulpanit@amd.com, robin.murphy@arm.com, dwmw2@infradead.org, baolu.lu@linux.intel.com, Arnd Bergmann , penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, Andrew Morton , vbabka@suse.cz, roman.gushchin@linux.dev, 42.hyeyoo@gmail.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-s390@vger.kernel.org, linux-crypto@vger.kernel.org, iommu@lists.linux.dev, linux-arch@vger.kernel.org Subject: Re: [RFC][PATCH 05/12] arch: Introduce arch_{,try_}_cmpxchg128{,_local}() Message-ID: References: <20221219153525.632521981@infradead.org> <20221219154119.154045458@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 0766B4001B X-Stat-Signature: jtufrwfkjhbc1wzmj93c6jq5f5hcarko X-HE-Tag: 1671534531-254963 X-HE-Meta: U2FsdGVkX1+Bl/3zaIAbEZ8cscT23Ft+/XiFKSBFHpnlcHqXVFpCxQXWGK33ECrkeEdnjerh7NhNOBpoJQOWpnzUM6yKu2GWKocFInYUIhYFIDYN9UyG2QkofMmVTIGI0AIOrYe/x3LXbjSenIJVQw4EZ2uH9QcVOll8hrI1EwfVCZLfHyoP/yjIKomojDNd1wLQcPOEbqpVQpEBFisbHv9oyUNug/T3stVgCyaUM8vyvXPuwzsxbp8nZK7mgiseuqqmDEXM4yXumNjDbFg4ACARJQbCYKY1lEHj22ZEb7DD1d8ZBRY0a3Yg1AwS4euPyU5TdKyOKzLiZQBO12n1b88ueTyfOXum38mxVJ/9sApNgmIK0q29CphUVuLPSGFi6i3t8BIyf704k5Yi91LWNgdcTQ2VAyFsK06B67F5D2Ql85ZqoAdjXk/NGoYXznK+PR06ai5EsDJlQ8H0DOEgxRFQbq78wKlhIDNPAa2TDlmGAeD3IwWzPwOosVaa3OQF7JQUFVoeaw5vP8faVuL0LD7bIb7it9CCZwQr9flX+JHxy+qtPKj3K9t7Z6xA580Be+UvnXRGEIljjMx2CLqKIm212/pudlAVBqBpsk0Top33QS/JLbrgw/VT1COdI0ZDX5SFWFbzrfid2mf0b2GnwFbkQxofUlODnYjPfJHnpxHTvUTD+oexhF67frzjX+D8l0ytebdO/LUl286e2IQOyNrIjY5fzb+q81Yg84jFeJkj6+iRvEfwVNbmkQoao3Z2EDnxQlTokZP9BHITUXnDFDkeevDzm28+G1kz/QINkQ4iDJUmkw64VTx2TMR+4Z/+qIOQVZV2yxvf4wBx2NMFmZYjYn5yZfsBULLoyxFVoRJF5Jol/EHhqB4Kxz6qN1PwXQiQzjD048jgJvxZld1tO6zbpm7Zl5g7/l8OY8aJ4wABCsj9LoszholaU37POsLAaMPdyYxBXj6i/LgIbps ETBsa1Aq kfR7R X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Dec 19, 2022 at 12:07:25PM -0800, Boqun Feng wrote: > On Mon, Dec 19, 2022 at 04:35:30PM +0100, Peter Zijlstra wrote: > > For all architectures that currently support cmpxchg_double() > > implement the cmpxchg128() family of functions that is basically the > > same but with a saner interface. > > > > Signed-off-by: Peter Zijlstra (Intel) > > --- > > arch/arm64/include/asm/atomic_ll_sc.h | 38 +++++++++++++++++++++++ > > arch/arm64/include/asm/atomic_lse.h | 33 +++++++++++++++++++- > > arch/arm64/include/asm/cmpxchg.h | 26 ++++++++++++++++ > > arch/s390/include/asm/cmpxchg.h | 33 ++++++++++++++++++++ > > arch/x86/include/asm/cmpxchg_32.h | 3 + > > arch/x86/include/asm/cmpxchg_64.h | 55 +++++++++++++++++++++++++++++++++- > > 6 files changed, 185 insertions(+), 3 deletions(-) > > > > --- a/arch/arm64/include/asm/atomic_ll_sc.h > > +++ b/arch/arm64/include/asm/atomic_ll_sc.h > > @@ -326,6 +326,44 @@ __CMPXCHG_DBL( , , , ) > > __CMPXCHG_DBL(_mb, dmb ish, l, "memory") > > > > #undef __CMPXCHG_DBL > > + > > +union __u128_halves { > > + u128 full; > > + struct { > > + u64 low, high; > > + }; > > +}; > > + > > +#define __CMPXCHG128(name, mb, rel, cl) \ > > +static __always_inline u128 \ > > +__ll_sc__cmpxchg128##name(volatile u128 *ptr, u128 old, u128 new) \ > > +{ \ > > + union __u128_halves r, o = { .full = (old) }, \ > > + n = { .full = (new) }; \ > > + \ > > + asm volatile("// __cmpxchg128" #name "\n" \ > > + " prfm pstl1strm, %2\n" \ > > + "1: ldxp %0, %1, %2\n" \ > > + " eor %3, %0, %3\n" \ > > + " eor %4, %1, %4\n" \ > > + " orr %3, %4, %3\n" \ > > + " cbnz %3, 2f\n" \ > > + " st" #rel "xp %w3, %5, %6, %2\n" \ > > + " cbnz %w3, 1b\n" \ > > + " " #mb "\n" \ > > + "2:" \ > > + : "=&r" (r.low), "=&r" (r.high), "+Q" (*(unsigned long *)ptr) \ > > I wonder whether we should use "(*(u128 *)ptr)" instead of "(*(unsigned > long *) ptr)"? Because compilers may think only 64bit value pointed by > "ptr" gets modified, and they are allowed to do "useful" optimization. In this I've copied the existing cmpxchg_double() code; I'll have to let the arch folks speak here, I've no clue.