From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0164C4332F for ; Mon, 19 Dec 2022 20:07:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 31C3C8E0002; Mon, 19 Dec 2022 15:07:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2CCA18E0001; Mon, 19 Dec 2022 15:07:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 16D208E0002; Mon, 19 Dec 2022 15:07:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 03AB48E0001 for ; Mon, 19 Dec 2022 15:07:50 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id A6FDDC0C22 for ; Mon, 19 Dec 2022 20:07:49 +0000 (UTC) X-FDA: 80260141458.03.E7825B0 Received: from mail-vs1-f46.google.com (mail-vs1-f46.google.com [209.85.217.46]) by imf18.hostedemail.com (Postfix) with ESMTP id 9F20F1C0005 for ; Mon, 19 Dec 2022 20:07:47 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="i7/Vm7xS"; spf=pass (imf18.hostedemail.com: domain of boqun.feng@gmail.com designates 209.85.217.46 as permitted sender) smtp.mailfrom=boqun.feng@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1671480467; a=rsa-sha256; cv=none; b=W6zMI2Y6fqP3JWVAtpfyJSpVuKOXx+aMyxR1echC9KGwvHZJA8eDXsN5euFcOvjEByEB/V JIjhu5v1ZPYIMCsvxsTJnHvIVcuaKtIvTj8aDUch2PQWmL55XgxaLhOZKo2VQscVaRtxIA H+wPhi8cSW2feU0EJWe1v0WjCO5EN/s= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b="i7/Vm7xS"; spf=pass (imf18.hostedemail.com: domain of boqun.feng@gmail.com designates 209.85.217.46 as permitted sender) smtp.mailfrom=boqun.feng@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1671480467; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=aYnmq6YUKA20HwARN8JDq4iYCZXHUrcLPE+3+zJtSe8=; b=VTgS1wbyl0lUxf7SeTdy1sC3aVbSK4Q7L/lgZqX7FiRlgU5zhEtrlJsPQPIWiuPno2Gdow INQzayVCVGZjR/5m/u0HgVKdKF/7/L4JITv6L4dKbueGGZ2MnlaF+YfltMKjwFhTLi33Ao qI3NZDB5nqr1aepSxV4yFHlRkL0slHM= Received: by mail-vs1-f46.google.com with SMTP id f189so9820395vsc.11 for ; Mon, 19 Dec 2022 12:07:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:feedback-id:from:to:cc:subject:date :message-id:reply-to; bh=aYnmq6YUKA20HwARN8JDq4iYCZXHUrcLPE+3+zJtSe8=; b=i7/Vm7xSYqsqXh54mibJCti0pLzVoIhTVNddh7lRN0PaJyiiHTO4Z7mude3UV0x7sH o9qJhBr1EMi3JrKMpCs8dfVw1tDc8LEX5bz5p4JfoIu+Ubfgmh2KLFsiyDsdN9LK5h9S aOXB8UnDWohgvlcqIRFBfBQRnsKw2VLOyjNl8v1FOkym3DleDRlleC7/ecr0ObQkq0/d bX+25KX8LYxq8C5hEy9LaOcjKyOoSGx0GHzfLt5aim0Q4EAjFbBFvrr2SGp/9bk59SVq y5UjOIAs6LYPpNHdRv8bg+UukznF+pTPetGwYEAkB5Zy/z1/A4C87gLx8VgzW2iGo+wI rU4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:feedback-id:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=aYnmq6YUKA20HwARN8JDq4iYCZXHUrcLPE+3+zJtSe8=; b=0UjWQYDeKKd4C5zRBFoQUNlhsWlqoxioODMzJETTMBKasNVvRe3ozG8lk5Ijz3rBPq CJ3HpLew6tHT0fsfayDAVBNBO4PXdVdNswjYWktn0HeYaDR1Nw3QhqCS6ZzUi4ugoGun avLro6+sPI/TuwNrVkSLUykmEcGI2rfSPELBrbSJfVC2qjboZ+IARojtxNs6F6tCPaMx n5DFDf19w1e77vhUWPt3IiV7rxVDrFlYtCEaJDhJcLaXrqmUTSpu1nTCBHRyTv2nscek EKW7EDcL7eF8Fiazt4U0ku+47h+xbaC1HnkuuhL4lRRIgskdi21mWv/hpDvCH7U9Xbkm /00Q== X-Gm-Message-State: AFqh2kqVGFoS1Lmt0maru1MnECF0o6u9IhydthpJUBt+R3Df3+HvvVRm DOLwuy+pIPTbPty+P2f4yGo= X-Google-Smtp-Source: AMrXdXsDzqfCRNTjjkXS9BQJxwqeAJtBOMlBh19EFdlQeLXBnuR5h323MBGGcxoDg51DibTTYx3UBQ== X-Received: by 2002:a67:ce0f:0:b0:3b1:2b40:5213 with SMTP id s15-20020a67ce0f000000b003b12b405213mr3344235vsl.18.1671480466748; Mon, 19 Dec 2022 12:07:46 -0800 (PST) Received: from auth1-smtp.messagingengine.com (auth1-smtp.messagingengine.com. [66.111.4.227]) by smtp.gmail.com with ESMTPSA id f19-20020a05620a15b300b006ce9e880c6fsm7411568qkk.111.2022.12.19.12.07.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 19 Dec 2022 12:07:45 -0800 (PST) Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailauth.nyi.internal (Postfix) with ESMTP id 03F3527C0054; Mon, 19 Dec 2022 15:07:44 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Mon, 19 Dec 2022 15:07:45 -0500 X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrgeefgddutdeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepfffhvfevuffkfhggtggujgesthdtredttddtvdenucfhrhhomhepuehoqhhu nhcuhfgvnhhguceosghoqhhunhdrfhgvnhhgsehgmhgrihhlrdgtohhmqeenucggtffrrg htthgvrhhnpeehudfgudffffetuedtvdehueevledvhfelleeivedtgeeuhfegueeviedu ffeivdenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe gsohhquhhnodhmvghsmhhtphgruhhthhhpvghrshhonhgrlhhithihqdeiledvgeehtdei gedqudejjeekheehhedvqdgsohhquhhnrdhfvghngheppehgmhgrihhlrdgtohhmsehfih igmhgvrdhnrghmvg X-ME-Proxy: Feedback-ID: iad51458e:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 19 Dec 2022 15:07:41 -0500 (EST) Date: Mon, 19 Dec 2022 12:07:25 -0800 From: Boqun Feng To: Peter Zijlstra Cc: torvalds@linux-foundation.org, corbet@lwn.net, will@kernel.org, mark.rutland@arm.com, catalin.marinas@arm.com, dennis@kernel.org, tj@kernel.org, cl@linux.com, hca@linux.ibm.com, gor@linux.ibm.com, agordeev@linux.ibm.com, borntraeger@linux.ibm.com, svens@linux.ibm.com, Herbert Xu , davem@davemloft.net, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com, joro@8bytes.org, suravee.suthikulpanit@amd.com, robin.murphy@arm.com, dwmw2@infradead.org, baolu.lu@linux.intel.com, Arnd Bergmann , penberg@kernel.org, rientjes@google.com, iamjoonsoo.kim@lge.com, Andrew Morton , vbabka@suse.cz, roman.gushchin@linux.dev, 42.hyeyoo@gmail.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-s390@vger.kernel.org, linux-crypto@vger.kernel.org, iommu@lists.linux.dev, linux-arch@vger.kernel.org Subject: Re: [RFC][PATCH 05/12] arch: Introduce arch_{,try_}_cmpxchg128{,_local}() Message-ID: References: <20221219153525.632521981@infradead.org> <20221219154119.154045458@infradead.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20221219154119.154045458@infradead.org> X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 9F20F1C0005 X-Stat-Signature: ybbs7pp71jqodobxy785gaszeqegsqqz X-HE-Tag: 1671480467-335537 X-HE-Meta: U2FsdGVkX1+5smJEKj43A4as42wvFy2USM5davDsNOmnKzQNcQReblY8r0m4Tgz9+7huzOwoQeHkShJXGSOutbz174+4B3b0RA7wWmPJEeKwY3ZCS/1mVeNGkgX+HAt/b5dDIbOUdzcRxptZh2vnSpadNoLpsqDId4rzlQgggp8kAntIVfDT1PPGaRXXbnoAwOlpBevsI6/E5NDfhAMFgcbhBpQCXKPCIdeRodwreZJId9m/lQvPjI9ozqEtEbqIf7ZzwNIDM5fGayJx5WQ61nZ7PFwmp89zbImzEopnjwZ3J8eH2hFS4xud5Y3QaRxajlGM4nIsx2JWlODFbS023cVqDDSEe03VHfUbkzrwerWg+iEuQyZ4EgrCvuC5jgnEKZkZy7sV+Naf5EJD4s7sAg4luLSdxHIs4aFzuAOCeQYffEQnYzML1sduXnIfxELao+wC20WHGnq7PFm21Va1bzz4SEP4Vp9HQMh5nKG4LXtWS2y4FOQUANxFz85DiXUr3kBrJgwVvNYld3cme6S2BQXwt9UNmtFjDOtnHi3A/dW6i/tRvHYrGp/j50ZtOUIkLZ0b8/yTvtydix6JZyDDrRwekDnIpRBRRcBZr3tcvjsrZj4xI8KQiq8VRkUAPbfciV62dbhvHm4DF92C4zoB60+VLF07fosQD7JZIJ7sAo3G9mEMPxnYbh7xShvuKmYZaye0M8JDz7jYfHQnx29MR6lGbk2YJYnEGTRhUC5cO7w+/C5cLYZvKS+EdJS0IhInEgQuU666l+adGcSBYTEXcGf9kGxTp5F3l1aPFkt/QT25toSfY57MsNv/lz6sNo0nbxvh9EcPqyxVZby7sOhgndn/DXvsmxgriQn7t6dobhxQsVtSn9WKU/DGFZJ4Uax9sUtlsJOXVyMXsEUOprjDVIVS0v5etRh5dr+fT3fTnUIINj7CN3ltd9twwjUJ4f7ywRp+G55v2+fLQVNkty7 Av1xZAfZ YRLVQ3OQo1mOP+DcSF4ODaEHB/c0Xu2rrnGUEcMGJjQg6KmPpS07GRlulkWuEqaE1+nxehhatIMR2wEnfPaG2WOBzu79DXB17RtdYilSBJxX+aQiRC/XezJPxvNUiqhnWy5Bebqcx2pQK1hSZPBpEw/1zqCAXOr0Y8dOLvEzjn641tzI74kpBQg9rMe53eU/XO8hqx4yz30UDmR54xbmqjgi0nHsLNhURK2Pmim7baIk8xs4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Dec 19, 2022 at 04:35:30PM +0100, Peter Zijlstra wrote: > For all architectures that currently support cmpxchg_double() > implement the cmpxchg128() family of functions that is basically the > same but with a saner interface. > > Signed-off-by: Peter Zijlstra (Intel) > --- > arch/arm64/include/asm/atomic_ll_sc.h | 38 +++++++++++++++++++++++ > arch/arm64/include/asm/atomic_lse.h | 33 +++++++++++++++++++- > arch/arm64/include/asm/cmpxchg.h | 26 ++++++++++++++++ > arch/s390/include/asm/cmpxchg.h | 33 ++++++++++++++++++++ > arch/x86/include/asm/cmpxchg_32.h | 3 + > arch/x86/include/asm/cmpxchg_64.h | 55 +++++++++++++++++++++++++++++++++- > 6 files changed, 185 insertions(+), 3 deletions(-) > > --- a/arch/arm64/include/asm/atomic_ll_sc.h > +++ b/arch/arm64/include/asm/atomic_ll_sc.h > @@ -326,6 +326,44 @@ __CMPXCHG_DBL( , , , ) > __CMPXCHG_DBL(_mb, dmb ish, l, "memory") > > #undef __CMPXCHG_DBL > + > +union __u128_halves { > + u128 full; > + struct { > + u64 low, high; > + }; > +}; > + > +#define __CMPXCHG128(name, mb, rel, cl) \ > +static __always_inline u128 \ > +__ll_sc__cmpxchg128##name(volatile u128 *ptr, u128 old, u128 new) \ > +{ \ > + union __u128_halves r, o = { .full = (old) }, \ > + n = { .full = (new) }; \ > + \ > + asm volatile("// __cmpxchg128" #name "\n" \ > + " prfm pstl1strm, %2\n" \ > + "1: ldxp %0, %1, %2\n" \ > + " eor %3, %0, %3\n" \ > + " eor %4, %1, %4\n" \ > + " orr %3, %4, %3\n" \ > + " cbnz %3, 2f\n" \ > + " st" #rel "xp %w3, %5, %6, %2\n" \ > + " cbnz %w3, 1b\n" \ > + " " #mb "\n" \ > + "2:" \ > + : "=&r" (r.low), "=&r" (r.high), "+Q" (*(unsigned long *)ptr) \ I wonder whether we should use "(*(u128 *)ptr)" instead of "(*(unsigned long *) ptr)"? Because compilers may think only 64bit value pointed by "ptr" gets modified, and they are allowed to do "useful" optimization. Same for lse and s390. Regards, Boqun > + : "r" (o.low), "r" (o.high), "r" (n.low), "r" (n.high) \ > + : cl); \ > + \ > + return r.full; \ > +} > + > +__CMPXCHG128( , , , ) > +__CMPXCHG128(_mb, dmb ish, l, "memory") > + > +#undef __CMPXCHG128 > + > #undef K > > #endif /* __ASM_ATOMIC_LL_SC_H */