From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14D6CC5AE59 for ; Tue, 3 Jun 2025 20:09:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8556B6B03C8; Tue, 3 Jun 2025 16:09:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7DF536B03CA; Tue, 3 Jun 2025 16:09:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6F7196B03D4; Tue, 3 Jun 2025 16:09:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 4C73B6B03C8 for ; Tue, 3 Jun 2025 16:09:07 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id E7B625EDCF for ; Tue, 3 Jun 2025 20:09:06 +0000 (UTC) X-FDA: 83515178292.16.592D9FF Received: from shelob.surriel.com (shelob.surriel.com [96.67.55.147]) by imf11.hostedemail.com (Postfix) with ESMTP id 37BAC40006 for ; Tue, 3 Jun 2025 20:09:05 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=surriel.com header.s=mail header.b=ZcXjG0KD; spf=pass (imf11.hostedemail.com: domain of riel@surriel.com designates 96.67.55.147 as permitted sender) smtp.mailfrom=riel@surriel.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1748981345; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=V0i6e9LPw7iBH4jD0X5pPM2azoxakouDkAG07q5MY6w=; b=f5i6KPNdKYP0HjrRO9guIsGURb4qqCowclhpfqwLZNF6U2Z+Wq75Z30J2jULgMxyEvT/mv eU3q0O1fGX2UE9hUf357GlD0MMsEhLZiaJKnnfvSOWE6cAZmmzQwSAnoE++Qr8bBvFWh6E dXh1JOtFdNHFQ5eVk55m9eO6dQGN1IM= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=surriel.com header.s=mail header.b=ZcXjG0KD; spf=pass (imf11.hostedemail.com: domain of riel@surriel.com designates 96.67.55.147 as permitted sender) smtp.mailfrom=riel@surriel.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1748981345; a=rsa-sha256; cv=none; b=W06VjRABVIAaz8pbbNFCBalVYJEq4BP93c5DL7bnykUBefokyhwY6yS68yCjxPipTtusvN 1hCUW/O/BY6VcafKLYmnVNPOC7e5SfjkkFM4SG7ybMDAgG5t/MWS/QOj1/DtxNykaRyl+6 qgDTiaFGgvogao6pvnA2ROp1ZaJT320= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=surriel.com ; s=mail; h=MIME-Version:Content-Transfer-Encoding:Content-Type:References: In-Reply-To:Date:Cc:To:From:Subject:Message-ID:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=V0i6e9LPw7iBH4jD0X5pPM2azoxakouDkAG07q5MY6w=; b=ZcXjG0KDYbuAJo6rln+EtHv/FY qqT+SZMJsAXXuh27BBL8lJW3aywO+1efqjvO6pxPI2ggVDKWZZ3xi08IE98lCRYVkjhtDkRIWUKy9 7Hk55WEfLI7gU8XaQP+oXtzZRC7tAlXWLdQJ3kC2kwgCdwtbIlI306/82tkkAAY1vB1uIikvP+Yqo I/Szw8JvTHH8jIVTKn87EOm09N4iYZJ7kDiG32PZ0jDPEvwCxJT5VRXf6w6LuwRavM0LwgrgeizXA xQB5ip56VpZGUMgwXsB/KgzoEoMlTNNq4EWhMUtxb91xJZz3hRob8gFwaoJTCOttzQsk+3Y3Yy0D8 YmiJ7AVg==; Received: from fangorn.home.surriel.com ([10.0.13.7]) by shelob.surriel.com with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.97.1) (envelope-from ) id 1uMXw5-000000001lz-2yLc; Tue, 03 Jun 2025 16:08:37 -0400 Message-ID: <513b0698bab160228c642598ba6cc7abdad1b694.camel@surriel.com> Subject: Re: [RFC v2 7/9] x86/mm: Introduce Remote Action Request From: Rik van Riel To: Dave Hansen , linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, x86@kernel.org, kernel-team@meta.com, dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, nadav.amit@gmail.com, Yu-cheng Yu Date: Tue, 03 Jun 2025 16:08:37 -0400 In-Reply-To: <2385d4ed-48d5-4d50-ae95-dbeb23432b71@intel.com> References: <20250520010350.1740223-1-riel@surriel.com> <20250520010350.1740223-8-riel@surriel.com> <2385d4ed-48d5-4d50-ae95-dbeb23432b71@intel.com> Autocrypt: addr=riel@surriel.com; prefer-encrypt=mutual; keydata=mQENBFIt3aUBCADCK0LicyCYyMa0E1lodCDUBf6G+6C5UXKG1jEYwQu49cc/gUBTTk33A eo2hjn4JinVaPF3zfZprnKMEGGv4dHvEOCPWiNhlz5RtqH3SKJllq2dpeMS9RqbMvDA36rlJIIo47 Z/nl6IA8MDhSqyqdnTY8z7LnQHqq16jAqwo7Ll9qALXz4yG1ZdSCmo80VPetBZZPw7WMjo+1hByv/ lvdFnLfiQ52tayuuC1r9x2qZ/SYWd2M4p/f5CLmvG9UcnkbYFsKWz8bwOBWKg1PQcaYHLx06sHGdY dIDaeVvkIfMFwAprSo5EFU+aes2VB2ZjugOTbkkW2aPSWTRsBhPHhV6dABEBAAG0HlJpayB2YW4gU mllbCA8cmllbEByZWRoYXQuY29tPokBHwQwAQIACQUCW5LcVgIdIAAKCRDOed6ShMTeg05SB/986o gEgdq4byrtaBQKFg5LWfd8e+h+QzLOg/T8mSS3dJzFXe5JBOfvYg7Bj47xXi9I5sM+I9Lu9+1XVb/ r2rGJrU1DwA09TnmyFtK76bgMF0sBEh1ECILYNQTEIemzNFwOWLZZlEhZFRJsZyX+mtEp/WQIygHV WjwuP69VJw+fPQvLOGn4j8W9QXuvhha7u1QJ7mYx4dLGHrZlHdwDsqpvWsW+3rsIqs1BBe5/Itz9o 6y9gLNtQzwmSDioV8KhF85VmYInslhv5tUtMEppfdTLyX4SUKh8ftNIVmH9mXyRCZclSoa6IMd635 Jq1Pj2/Lp64tOzSvN5Y9zaiCc5FucXtB9SaWsgdmFuIFJpZWwgPHJpZWxAc3VycmllbC5jb20+iQE +BBMBAgAoBQJSLd2lAhsjBQkSzAMABgsJCAcDAgYVCAIJCgsEFgIDAQIeAQIXgAAKCRDOed6ShMTe g4PpB/0ZivKYFt0LaB22ssWUrBoeNWCP1NY/lkq2QbPhR3agLB7ZXI97PF2z/5QD9Fuy/FD/jddPx KRTvFCtHcEzTOcFjBmf52uqgt3U40H9GM++0IM0yHusd9EzlaWsbp09vsAV2DwdqS69x9RPbvE/Ne fO5subhocH76okcF/aQiQ+oj2j6LJZGBJBVigOHg+4zyzdDgKM+jp0bvDI51KQ4XfxV593OhvkS3z 3FPx0CE7l62WhWrieHyBblqvkTYgJ6dq4bsYpqxxGJOkQ47WpEUx6onH+rImWmPJbSYGhwBzTo0Mm G1Nb1qGPG+mTrSmJjDRxrwf1zjmYqQreWVSFEt26tBpSaWsgdmFuIFJpZWwgPHJpZWxAZmIuY29tP okBPgQTAQIAKAUCW5LbiAIbIwUJEswDAAYLCQgHAwIGFQgCCQoLBBYCAwECHgECF4AACgkQznneko TE3oOUEQgAsrGxjTC1bGtZyuvyQPcXclap11Ogib6rQywGYu6/Mnkbd6hbyY3wpdyQii/cas2S44N cQj8HkGv91JLVE24/Wt0gITPCH3rLVJJDGQxprHTVDs1t1RAbsbp0XTksZPCNWDGYIBo2aHDwErhI omYQ0Xluo1WBtH/UmHgirHvclsou1Ks9jyTxiPyUKRfae7GNOFiX99+ZlB27P3t8CjtSO831Ij0Ip QrfooZ21YVlUKw0Wy6Ll8EyefyrEYSh8KTm8dQj4O7xxvdg865TLeLpho5PwDRF+/mR3qi8CdGbkE c4pYZQO8UDXUN4S+pe0aTeTqlYw8rRHWF9TnvtpcNzZw== Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.54.3 (3.54.3-1.fc41) MIME-Version: 1.0 X-Rspamd-Queue-Id: 37BAC40006 X-Stat-Signature: ideq71obgxoqthcpg7cnjy9qaarnmk38 X-Rspam-User: X-Rspamd-Server: rspam04 X-HE-Tag: 1748981345-372158 X-HE-Meta: U2FsdGVkX1/hOCftFrO9dMgSXuw5y3guCltu2CViVwDTMDH4MPlgucdtlLF3CkQUW7Lgz+5YOSAohHEVHbQjfuHUGEcmkb6wZZAODxl2pCbqB0ZC7fDegrLmV5GV7aMmlnFJmp8ZZpGNGe5orKW50Rs3cqZw/nct0SkNdbobZ0iAJhzQiH32cCB/c7/IE1QUHSrbD+s6cJENncpsmHuyJhGSprL5GDzLt/1mAgAZQR9ReTsv3CSkuzgEPL6arbXXziVBa8Cgh6//RGcz8k++jExd5oRvOZdOCke8TPGJVEBdqBqjVa7rsuZelCiQq06NuW7A7nZkok1fgoDZnS8bvvBWyM68ne4FDIsSens3OjT++BKhaId/jNK3HKxkTXxFOu2Ch7AKl0gEITOQ6AOI8RXxz7AOAbhzFkb2fwaOMqzg6CIMhUaRM8KshCb0AAt8UXZuEEhviXjSLjAvVn5fLuIYs/rBm7TKbXKT1U4e0+bBGBgobev+EqKxZ/YGnTiLnqE1Uk6sV3TeSTeKxCLdEdGF80fCLy0UfvEor5bl/LI4qf6QrWligTK9fa+qyvn1IAa3hqMaNlX3Vat8an3bq4oMN4h/tHq0P/FEVUVVgtbJfQHXChMmPND1DHd1GDa5YhmJd3BY8nSKTRBqX9sa/h8E2+2sWLJe+53rNd2VdrtDGMhTX5t+B+m5XqKTYsw8nGUwOE+n0EFEIXTOYSYKLMHP0Rnnt+hPuadqXnqpJazgTX4RNvG33/c+t+QXidfW550I2Z5kJPdCxiYx7ZDNQLV8ubKUgmWQakMHTHtXDEy7ny6e7N8NfNZfQKmCagwgP9UFki27KDPxIvr4lpKjGuZKx3Dewv0Ll0llR+/h3U8iFv+OIkb9cQev3k3ClvvUHp2NeEXo0RWgDGcaCjmZnE1+g2+L+ZEDkGNPC/HlUAlfASPIgtBx+njyOIRfNHsd3honIt+uqRH6iBOCYpq VHrQHuCm GUGHOuRhfi7AWCjyjkJ63EVWazHtGryFr0AuuO3VtzKifyjcvYVOiXq2IcrL2TuLusTFt+sGM8SDnoRQ04TG8HiCSO6ZiwdjBDj6OI3zEZiGY+2UNrVQx0GCGFADyQKdMpIJbOm1S8fY2BczGABWdrE6umPQ2NMnm7IJwAkjIBeJ2+PCcekXxD8piVn1jkRekOhb+6hALCXlOt5F60T0dx3ltD4EkGbNunU8s4iNrAtRVMpLYApmqwUcF48g+QYhk0zSMfBc2XppDWqoNHB8Sm0RFRpVNau8tXrcGZKQez4b43xE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Wed, 2025-05-21 at 09:38 -0700, Dave Hansen wrote: >=20 > > +static void wait_for_done(unsigned long idx, int target_cpu) > > +{ > > + u8 status; > > + u8 *rar_actions =3D per_cpu(rar_action, target_cpu); > > + > > + status =3D READ_ONCE(rar_actions[idx]); > > + > > + while ((status !=3D RAR_ACTION_OK) && (status !=3D > > RAR_ACTION_FAIL)) { >=20 > Should this be: >=20 > while (status =3D=3D RAR_ACTION_START) { > ... >=20 > ? That would more clearly link it to set_action_entry() and would > also > be shorter. >=20 That is a very good question. The old RAR code suggests there might be some intermediate state when the target CPU works on processing the RAR entry, but the current documentation only shows RAR_SUCCESS, RAR_PENDING, and RAR_FAILURE as possible values. Lets try with status =3D=3D RAR_ACTION_PENDING. > >=20 > > +void rar_cpu_init(void) > > +{ > > + u64 r; > > + u8 *bitmap; > > + int this_cpu =3D smp_processor_id(); > > + > > + cpumask_clear(&per_cpu(rar_cpu_mask, this_cpu)); > > + > > + rdmsrl(MSR_IA32_RAR_INFO, r); > > + pr_info_once("RAR: support %lld payloads\n", r >> 32); >=20 > Doesn't this need to get coordinated or checked against > RAR_MAX_PAYLOADS? I just added that in, and also applied all the cleanups from your email. >=20 > > + // reserved bits!!! r |=3D (RAR_VECTOR & 0xff); >=20 > Is this just some cruft from testing? >=20 I'm kind of guessing the old code might have used this value to specify which IRQ vector to use for RAR, but modern microcode hardcodes the RAR_VECTOR value. > > + wrmsrl(MSR_IA32_RAR_CTRL, r); > > +} > > + > > +/* > > + * This is a modified version of smp_call_function_many() of > > kernel/smp.c, > > + * without a function pointer, because the RAR handler is the > > ucode. > > + */ >=20 > It doesn't look _that_ much like smp_call_function_many(). I don't > see > much that can be consolidated. Agreed. It looks even less like it after some more simplifications. >=20 > > + /* No online cpus?=C2=A0 We're done. */ > > + if (cpu >=3D nr_cpu_ids) > > + return; >=20 > This little idiom _is_ in smp_call_function_many_cond(). I wonder if > it > can be refactored out. Removing the arch_send_rar_single_ipi fast path gets rid of this code completely. Once we cpumask_and with the cpu_online_mask, the cpumask_weight should end up as 0 if no online CPUs are in the mask. Thank you for all the cleanup suggestions. I've tried to address them all for v3. --=20 All Rights Reversed.