From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49975C678D4 for ; Thu, 2 Mar 2023 21:25:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DD7966B0071; Thu, 2 Mar 2023 16:25:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D60826B0073; Thu, 2 Mar 2023 16:25:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C01046B0074; Thu, 2 Mar 2023 16:25:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id AB8946B0071 for ; Thu, 2 Mar 2023 16:25:15 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 752771201D8 for ; Thu, 2 Mar 2023 21:25:15 +0000 (UTC) X-FDA: 80525238990.13.F73266B Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf03.hostedemail.com (Postfix) with ESMTP id 6E47B2000F for ; Thu, 2 Mar 2023 21:25:13 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=DJHfcCrf; spf=pass (imf03.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1677792313; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=UFq8qp6M7dSmGx8Gz93tP0jf4xtRfEYWcv1XZPRpaEw=; b=sPVzX3j6zzBYGnwNUbXJI5LOuw3qbLCvj6lGcoPmp+r+lL5KdkIscl0iwldkQUieN5Nn9i OyxgaajfeA43/e6pQXW/B8G3ajoU7G9uF+CE88lKBOx2EyejzbJkyobkNZwaC2d36bBBCF C7z+OOUrMaPY4UWDZc7XBUVTUxNixSg= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=DJHfcCrf; spf=pass (imf03.hostedemail.com: domain of peterx@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=peterx@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1677792313; a=rsa-sha256; cv=none; b=Axa3y5cjzlZaX78aJoH5LvwIdiTEq725gx/eLbmO/oe4CSKstArsl5LyTya5DRu8XqBcUG eYAtf2rfm4bisEGE3OlQeTKeg47Q65HhmjN9Oe9hdcXCzHW4hpSVHsKoQ3q/B+OzNLyLha thm5BQQ+Uk9C+eQK8/brJSncrHaA2Z8= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677792312; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=UFq8qp6M7dSmGx8Gz93tP0jf4xtRfEYWcv1XZPRpaEw=; b=DJHfcCrfbtUxYhQIlINwNBA01BWAf8/DBL2MSPzWQkXAxhbBlQJODrQJUKT5md9GvluWzs L4Ojbu3gTlo1qKsxHJlrh9I9Ab8j9inAy+E45GnR+qNB6h6qi7pBZkp6EnUbyYrxOWTnzY ASIyrqQAPJrHZgOowSxrNfm2NsniLfA= Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-206-B9rux6UnMS2_vcoHCP6rXw-1; Thu, 02 Mar 2023 16:25:11 -0500 X-MC-Unique: B9rux6UnMS2_vcoHCP6rXw-1 Received: by mail-qk1-f200.google.com with SMTP id m25-20020ae9e019000000b007421ddd945eso211189qkk.6 for ; Thu, 02 Mar 2023 13:25:11 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677792311; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=UFq8qp6M7dSmGx8Gz93tP0jf4xtRfEYWcv1XZPRpaEw=; b=RYHtaTe5zcTT6dsZw38v5A6EZrOXaKG+B2Ec0Kbf3v45j6WVYPyrzmR2HW0nV8XwVN RZV9RVTBYDMQmiZCHCp0mOFAblp2aU5fEhqSlIhDe73rzocjM3qrApNRIQqU/+NjDeZf wSmOg+zhCQokrGqBBrF2v1C87+Hr+O4TaxwWp//q0q0HEY+OyIrZ1BhVZQ7k/jiWZS2C cXSOgcvwkncWl75fAfVoIxlcopGiPYTfDYe7xMyb2MT4s7XbnyK8Ou2n+72TdjXEhO9k 1caIFrf5e7Fg6Li8nQLcO1xhBr+t5WyG+rMLlRzyJda7Lli0+Lnl+zO5oRe+gd1h7JOm C2UQ== X-Gm-Message-State: AO0yUKVUQVgEoRPOep4VIuZXVwco6PD+D7HflBia5c46vln7jWFrfixf Ladh9DGcGa5XV/8ykcLmcZcefqWhv32U/ndhfBvlEkJoyWU6Vj+uZMyB/e1c7YJkzNd/HeEY2CO Tvx3Epgq3LVKtEHZF X-Received: by 2002:a05:622a:1a99:b0:3bf:a061:6cae with SMTP id s25-20020a05622a1a9900b003bfa0616caemr22098282qtc.5.1677792310867; Thu, 02 Mar 2023 13:25:10 -0800 (PST) X-Google-Smtp-Source: AK7set9knXpqPtJm74PSh3rkR9iwQNfM4d1iprlf1Mh6s41OicEkxWL4aHCA/4Fo4DOaxsfggfbMRA== X-Received: by 2002:a05:622a:1a99:b0:3bf:a061:6cae with SMTP id s25-20020a05622a1a9900b003bfa0616caemr22098258qtc.5.1677792310534; Thu, 02 Mar 2023 13:25:10 -0800 (PST) Received: from x1n (bras-base-aurron9127w-grc-56-70-30-145-63.dsl.bell.ca. [70.30.145.63]) by smtp.gmail.com with ESMTPSA id b1-20020ac84f01000000b003bb8c60cdf1sm422684qte.78.2023.03.02.13.25.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Mar 2023 13:25:09 -0800 (PST) Date: Thu, 2 Mar 2023 16:25:08 -0500 From: Peter Xu To: Marcelo Tosatti Cc: Christoph Lameter , Aaron Tomlin , Frederic Weisbecker , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH v2 02/11] this_cpu_cmpxchg: ARM64: switch this_cpu_cmpxchg to locked, add _local function Message-ID: References: <20230209150150.380060673@redhat.com> <20230209153204.683821550@redhat.com> MIME-Version: 1.0 In-Reply-To: X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Disposition: inline X-Rspamd-Queue-Id: 6E47B2000F X-Stat-Signature: 9t41x67x1ujz5d88dygne88wmkao3b9x X-Rspam-User: X-Rspamd-Server: rspam08 X-HE-Tag: 1677792313-274111 X-HE-Meta: U2FsdGVkX1/j6DrtY8bWLT/EFup7coP9uuV3JS4oPaViziS73wMh1x5R5jHnOw+ee6WqUCCc1kKQf8aHTZ0+Z5Cq86JKVXirI6JhRgMEJfuSjfR2iuqb7+eJrFPetf3ppjpFniaM8WVUPcB9KZA4SIbQ5Zg0+T5lOBp7O4HXHiKf7vsaRIugm76JgtmxsFt+HdME6oM+W8k8rAvjBli7+2fWj3E+2sgDS25q1G1COOk/PcftDwYF1hVo9+iiGFZxSlOk8QmqVp4sE3zSMdvSOgmZiNZu6zC3gHHDvDtPQedDF8avimgp8ZAXtfW9LxwaJlmcr/hNzUYeNFrwCuGEHIBfIB97aeWTNzjVTEpPM+7rkLd+tNjlMjm7lNxDODizy9zWYgFoApHQzcuw4Ch3M7F0F+7LOrJIeHLmKSAEwmnNJdzgDHBSz56Cx+nu+MVQGMUWEZ2VEzwGJGM479V4U4wLDT5sKb9DJQojeZE3yuV+dhbdQQ1dI1KCM8Ay1yjJb0TvkJSxKGrOd6yLI+8zCW2vS/mTN1Ge8N6dE0n4h2c8BlLHLMw1HRBHBapZAxdP7Qqt07BjEXQJhe7k8mlv9obaVD05h8n790nSx0L5JXAQ5eHKdhLYxa/9Dg0q7DwhQN7+Y0tHCii1z5ga6CYNMfU4yRIlmRU/Wap3Ha/yuSPmWQcB24YilK+U04c79lZ9+JP+jauvskPj5Iuk7WYhhe6+Ujw+zVSSP8nXhN3exxVyZ7MrKV7/J19zXhj51VIEtiZIAFLlrVrLoP7xYMF90C/KRtmpiJ3OBfW3JViNJHg1TPkoAskS47bTU2lPSg19b/NYXIkhwjY7dCK4qe89hWqEqnff2WChg38x8Cozdr5ETjVErkER2mmOQBF/IXJIf5gvKQGIxeq9ZF9iV1oSfZjWXcIqA4P+TWu5PaTvfJzckJmCrbJsOnZMCxbZD2Qa5PlOWLBjTz4gek2Aa0C EVFLSPZO P0eE+6dhLvZp3NVbzw1TY8LLG+3GMNoAcje7Ikwgh98VYr0Po22IGWAfieRBMXaZKYs5UfXhSwD4RyI/hFEVgMcBnD/Xbh/CErQetwdqoWoWi/VTT27oLiWmfZOingvACEyh6byn74rAdarBUamrSiYqjw7r5LQKps8mYuxVdh0oFftIr/b5XX8m3cE83Vk9Y0i3toznBTFIWZSLnwaJOXh7hzMPSH2IAAYLtsOEK0FRhIvP7q1ur1OBK/JSDgBcH09AYoJYuh2gENBr+W9IYRSghKcOr/G9GPumA2FlE5tgwoGSgRnyOtsr8n8FlRI7WJmqOGZwYpW8/s7A2reC6l3Kxvm5WCk17k1rYy75d6aNg5pObVJzOHrnmdVgPLDnMgV8H/2bwvoh+QWzQktBgRkzvYu93CA51HkPOw15C7go06pmCTUapjppaRv3PvQLNPHmXC7o7fahjyVk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Mar 02, 2023 at 06:04:25PM -0300, Marcelo Tosatti wrote: > On Thu, Mar 02, 2023 at 03:53:12PM -0500, Peter Xu wrote: > > On Thu, Feb 09, 2023 at 12:01:52PM -0300, Marcelo Tosatti wrote: > > > Goal is to have vmstat_shepherd to transfer from > > > per-CPU counters to global counters remotely. For this, > > > an atomic this_cpu_cmpxchg is necessary. > > > > > > Following the kernel convention for cmpxchg/cmpxchg_local, > > > change ARM's this_cpu_cmpxchg_ helpers to be atomic, > > > and add this_cpu_cmpxchg_local_ helpers which are not atomic. > > > > I can follow on the necessity of having the _local version, however two > > questions below. > > > > > > > > Signed-off-by: Marcelo Tosatti > > > > > > Index: linux-vmstat-remote/arch/arm64/include/asm/percpu.h > > > =================================================================== > > > --- linux-vmstat-remote.orig/arch/arm64/include/asm/percpu.h > > > +++ linux-vmstat-remote/arch/arm64/include/asm/percpu.h > > > @@ -232,13 +232,23 @@ PERCPU_RET_OP(add, add, ldadd) > > > _pcp_protect_return(xchg_relaxed, pcp, val) > > > > > > #define this_cpu_cmpxchg_1(pcp, o, n) \ > > > - _pcp_protect_return(cmpxchg_relaxed, pcp, o, n) > > > + _pcp_protect_return(cmpxchg, pcp, o, n) > > > #define this_cpu_cmpxchg_2(pcp, o, n) \ > > > - _pcp_protect_return(cmpxchg_relaxed, pcp, o, n) > > > + _pcp_protect_return(cmpxchg, pcp, o, n) > > > #define this_cpu_cmpxchg_4(pcp, o, n) \ > > > - _pcp_protect_return(cmpxchg_relaxed, pcp, o, n) > > > + _pcp_protect_return(cmpxchg, pcp, o, n) > > > #define this_cpu_cmpxchg_8(pcp, o, n) \ > > > + _pcp_protect_return(cmpxchg, pcp, o, n) > > > > This makes this_cpu_cmpxchg_*() not only non-local, but also (especially > > for arm64) memory barrier implications since cmpxchg() has a strong memory > > barrier, while the old this_cpu_cmpxchg*() doesn't have, afaiu. > > > > Maybe it's not a big deal if the audience of this helper is still limited > > (e.g. we can add memory barriers if we don't want strict ordering > > implication), but just to check with you on whether it's intended, and if > > so whether it may worth some comments. > > It happens that on ARM-64 cmpxchg_local == cmpxchg_relaxed. > > See cf10b79a7d88edc689479af989b3a88e9adf07ff. This is more or less a comment in general, rather than for arm only. Fundamentally starting from this patch it's redefining this_cpu_cmpxchg(). What I meant is whether we should define it properly then implement the arch patches with what is defined. We're adding non-local semantics into it, which is obvious to me. We're (silently, in this patch for aarch64) adding memory barrier semantics too, this is not obvious to me on whether all archs should implement this api the same way. It will make a difference IMHO when the helpers are used in any other code clips, because IIUC proper definition of memory barrier implications will decide whether the callers need explicit barriers when ordering is required. > > This patchset maintains the current behaviour > of this_cpu_cmpxch (for this_cpu_cmpxch_local), which was: > > #define this_cpu_cmpxchg_1(pcp, o, n) \ > - _pcp_protect_return(cmpxchg_relaxed, pcp, o, n) > + _pcp_protect_return(cmpxchg, pcp, o, n) > #define this_cpu_cmpxchg_2(pcp, o, n) \ > - _pcp_protect_return(cmpxchg_relaxed, pcp, o, n) > + _pcp_protect_return(cmpxchg, pcp, o, n) > #define this_cpu_cmpxchg_4(pcp, o, n) \ > - _pcp_protect_return(cmpxchg_relaxed, pcp, o, n) > + _pcp_protect_return(cmpxchg, pcp, o, n) > #define this_cpu_cmpxchg_8(pcp, o, n) \ > + _pcp_protect_return(cmpxchg, pcp, o, n) > > > > + > > > +#define this_cpu_cmpxchg_local_1(pcp, o, n) \ > > > _pcp_protect_return(cmpxchg_relaxed, pcp, o, n) > > > +#define this_cpu_cmpxchg_local_2(pcp, o, n) \ > > > + _pcp_protect_return(cmpxchg_relaxed, pcp, o, n) > > > +#define this_cpu_cmpxchg_local_4(pcp, o, n) \ > > > + _pcp_protect_return(cmpxchg_relaxed, pcp, o, n) > > > +#define this_cpu_cmpxchg_local_8(pcp, o, n) \ > > > + _pcp_protect_return(cmpxchg_relaxed, pcp, o, n) > > > > I think cmpxchg_relaxed()==cmpxchg_local() here for aarch64, however should > > we still use cmpxchg_local() to pair with this_cpu_cmpxchg_local_*()? > > Since cmpxchg_local = cmpxchg_relaxed, seems like this is not necessary. > > > Nothing about your patch along since it was the same before, but I'm > > wondering whether this is a good time to switchover. > > I would say that another patch is more appropriate to change this, > if desired. Sure on this one. Thanks, -- Peter Xu