From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16B16C64EC4 for ; Wed, 8 Mar 2023 21:43:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A776B6B0075; Wed, 8 Mar 2023 16:42:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A27A6280002; Wed, 8 Mar 2023 16:42:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 91623280001; Wed, 8 Mar 2023 16:42:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 7E7F96B0075 for ; Wed, 8 Mar 2023 16:42:59 -0500 (EST) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 3D364C0FCD for ; Wed, 8 Mar 2023 21:42:59 +0000 (UTC) X-FDA: 80547056478.06.82BD4CC Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf03.hostedemail.com (Postfix) with ESMTP id 5391420013 for ; Wed, 8 Mar 2023 21:42:57 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=QdVIDKfY; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf03.hostedemail.com: domain of mtosatti@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=mtosatti@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678311777; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=FdnsXLolUJxCc29xMKmh4dWAYjznbbiQcfHEwgveAgs=; b=0RtTYOheXw3clyPgLcjnTSP3pEd67Hb9f5Ul5IJbGOvrBHhMDAP5k/K4UZl/zGBrCNI2ZA DQbI2FwErLyifJhOFq/vGAOQQVDcG75hMHzJODAmNOq8bEoIBPd7B4AKS6N2ipvfaPRCuQ 6+A/Sr1aeCq32oTVhhdveWZlBKybvg8= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=QdVIDKfY; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf03.hostedemail.com: domain of mtosatti@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=mtosatti@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678311777; a=rsa-sha256; cv=none; b=7vDddgZx6TVKjWeYUuiO/I0JLNA1s0c5I+XL1Jk17qaKl3y2DA8Br68w1xF9d1y1pjP1SI Fzy/1Zsj77iQLVFEC+qL5JQoTc9Om74agRaZol1gR5PTpmPIh2/6aQiV3dzFbU5szBNQ57 aAbJHVBVoyoz3gl7+tkf3EfZPelnv68= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1678311776; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=FdnsXLolUJxCc29xMKmh4dWAYjznbbiQcfHEwgveAgs=; b=QdVIDKfYR/aQ6cOWVcBvYCwXlsGSypIPCRSXAnhHfKcFiM0+YF4JPNFN8hrbWiUU7qFdvF ktkSRbzCeThodoIx4Ah5nbMVXFlNK8QahQepfLWsacW7PvpwWjwZOt1Dpn2jKe00x/4su/ KEkvNN3HBd1QidWyiG1QPm9c+tgZkYU= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-100-AvipUr7jOD25WLIQ8vSGaQ-1; Wed, 08 Mar 2023 16:42:53 -0500 X-MC-Unique: AvipUr7jOD25WLIQ8vSGaQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7931185A588; Wed, 8 Mar 2023 21:42:52 +0000 (UTC) Received: from tpad.localdomain (ovpn-112-3.gru2.redhat.com [10.97.112.3]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3F0932166B26; Wed, 8 Mar 2023 21:42:52 +0000 (UTC) Received: by tpad.localdomain (Postfix, from userid 1000) id E5BF74015389B; Wed, 8 Mar 2023 18:42:27 -0300 (-03) Date: Wed, 8 Mar 2023 18:42:27 -0300 From: Marcelo Tosatti To: Peter Zijlstra Cc: Christoph Lameter , Aaron Tomlin , Frederic Weisbecker , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Russell King , Huacai Chen , Heiko Carstens , x86@kernel.org Subject: Re: [PATCH v4 05/12] this_cpu_cmpxchg: x86: switch this_cpu_cmpxchg to locked, add _local function Message-ID: References: <20230305133657.255737580@redhat.com> <20230305134053.537803923@redhat.com> <20230306112240.GB1267364@hirez.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230306112240.GB1267364@hirez.programming.kicks-ass.net> X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 5391420013 X-Stat-Signature: sk3utijrx6i5zi6qy7hjipewpc18n4ze X-HE-Tag: 1678311777-811605 X-HE-Meta: U2FsdGVkX18kkAKSSACJ+YUDs1aTl/U0ih01yqhZcJZ711fMwk1H2oXO2+KrzO/7j8OxBQ1USodeYRdIsGT+1wNV6qkiarJTyosdaeR5B69WUO0i3SetSZIczvP+8V3b+lDwRasvRUYXdbxCfM0gjf6KcI/Px2rOVfCvGo0rMfzwK8Yzzy3XT03pxxqxWwsgu0yR042PuUho6z5rfNanr8sPbVAMNQFQb6uhsLEDJm5+bfTQ1yNEhNs4DyCCP7qBe7EGQZBht9BJgdrVGdzRFGQjd3g9rdfU1fZVzKIIL39tE0FCecyKEGPzzyEqaT/wiWwwz8F6oMbe6CCCOGf3/3HFzzb1sHkkGf89w7hno7RT7yas4bieejSlgLHbhVN2xSsMdZlDH0caWIToDZ8YhnWABWDsgMXaJMQ8K/qbS88Z/EkqbqpssQ2CBTMQzJDeNYsw1nY9mRlB2VPoiliUmjK1xaT+uJYVqTAiT8mSVZfPqb26xCmmrV1QLCitWUIi+HmbmGklriQJkTahekYfDV54Hi7M/p7iJrlY++gWAwLqvm16Sz0OFBlqEWW61xUJB7pMVzDt+84YzIVFKut9ZrOZr+Yw766+LiHOHqp1kVThOxqHydJWV2KsI017jRqha19QqCF/lDQ66ieShfRru3WqwTu7c1YV6OAw3/TtWWjGUAzUEMQfX6VN8kQGkG6wE2owJ9x84XYLBmUsMxEj46cndVZSi4Zz5ptL05j1KNVq2dB2CfaSd9oCTqAgzjAhG0S9W4BsVt5NF4Ui9SmPBvPCAg1RDBuKCUzrL5IpA06E/389d/tQatnrPZZKltkgn1vKrIofCQaRjtQ+JIlsBCHuIe0chcGjlPvM9iB18s+Zn/7zi2rT7v2WXY5DztNz9PRwkERCJjsP1DIsOMwo2eMAzmkhWBaCRxYvXtlCNP5nyn11MGeHTSEgI5zygbI+TK9mUwvLWgY+WWIZ83t 9Y/b/06z Us7aItd0YtdEGk+3bFxzRNEpWfzk4SHA6KMm+2SFZbltKb8Qk1Q1S+gj2dETV5UPsvV8lLRznnZU/EIcvUsf+CH/p+apdIP9d3M9mA9wuAGqclYmnEf1N2Jv5CksuCZ9dX2eKqhmhsEy0uClA6jkv6GOe6JeRkYez+WaJFH70rGAiOcQuH+ffQmcn0xW887zmcSOzU5pQpVQPidPvw+RbqpaHicunlYPyGGNm X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Mon, Mar 06, 2023 at 12:22:40PM +0100, Peter Zijlstra wrote: > On Sun, Mar 05, 2023 at 10:37:02AM -0300, Marcelo Tosatti wrote: > > Goal is to have vmstat_shepherd to transfer from > > per-CPU counters to global counters remotely. For this, > > an atomic this_cpu_cmpxchg is necessary. > > > > Following the kernel convention for cmpxchg/cmpxchg_local, > > change x86's this_cpu_cmpxchg_ helpers to be atomic. > > and add this_cpu_cmpxchg_local_ helpers which are not atomic. > > Urgh.. much hate for this. this_cpu_*() is local, per definition, > always. :Author: Christoph Lameter, August 4th, 2014 :Author: Pranith Kumar, Aug 2nd, 2014 this_cpu operations are a way of optimizing access to per cpu variables associated with the *currently* executing processor. This is done through the use of segment registers (or a dedicated register where the cpu permanently stored the beginning of the per cpu area for a specific processor). this_cpu operations add a per cpu variable offset to the processor specific per cpu base and encode that operation in the instruction operating on the per cpu variable. This means that there are no atomicity issues between the calculation of the offset and the operation on the data. Therefore it is not necessary to disable preemption or interrupts to ensure that the processor is not changed between the calculation of the address and the operation on the data. >> Call the above [1]. >> Up to this point, everything remains valid with adding lock >> to cmpxchg. That is, it makes sense to have a locked >> this_cpu_cmpxchg, operating on per-CPU data, without the need >> to disable preemption/interrupts to ensure processor is not >> changed. Read-modify-write operations are of particular interest. Frequently processors have special lower latency instructions that can operate without the typical synchronization overhead, but still provide some sort of relaxed atomicity guarantees. The x86, for example, can execute RMW (Read Modify Write) instructions like inc/dec/cmpxchg without the lock prefix and the associated latency penalty. >> This sentence above makes sense if the data is accessed locally. >> However, we are extending it (this_cpu_cmpxchg) to the case >> where we want the data to be accessed remotely (therefore need >> to be a locked access), but still want the benefits of [1]. >> >> For example, this_cpu_xchg implies LOCK semantics. Access to the variable without the lock prefix is not synchronized but synchronization is not necessary since we are dealing with per cpu data specific to the currently executing processor. Only the current processor should be accessing that variable and therefore there are no concurrency issues with other processors in the system. ---- So in summary, i understand your POV, but: 1) Have outlined arguments above which point that this_cpu_xxx and locked instructions can co-exist (in fact, they are already do, for this_cpu_xchg). 2) In case you still think this is a bad idea, any suggestions for improvements? Thanks