From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66EE6C3ABBC for ; Tue, 6 May 2025 15:50:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EE0F26B000A; Tue, 6 May 2025 11:50:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E90756B0082; Tue, 6 May 2025 11:50:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D317C6B0085; Tue, 6 May 2025 11:50:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id B2C266B000A for ; Tue, 6 May 2025 11:50:35 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 1B4C4140558 for ; Tue, 6 May 2025 15:50:37 +0000 (UTC) X-FDA: 83412920514.30.502420E Received: from mail-ej1-f50.google.com (mail-ej1-f50.google.com [209.85.218.50]) by imf11.hostedemail.com (Postfix) with ESMTP id 0E1F140003 for ; Tue, 6 May 2025 15:50:34 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=BV+SSnfr; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf11.hostedemail.com: domain of nadav.amit@gmail.com designates 209.85.218.50 as permitted sender) smtp.mailfrom=nadav.amit@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1746546635; a=rsa-sha256; cv=none; b=4yAC15WYQBD/FjWlMbKwLZwibNxSy7n30FFjkwNiMaEQ0csaSinpMnjovTsL0TsuksCdtl dwiSxX3DQbZsaChdhu2yZ0iyMdCOu4qfkswMQrgdSTxfYoT/Bj9q56mu6z6OYrdFsNmRjk VXbmBJ45YfCJy8TSIzfSqKm54BbwDnU= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=BV+SSnfr; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf11.hostedemail.com: domain of nadav.amit@gmail.com designates 209.85.218.50 as permitted sender) smtp.mailfrom=nadav.amit@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1746546635; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=v/cd32lR18qNaF2U9DfiKee4Kiegk/8OhHBbE5vcd/o=; b=ry9lxjuxzVxmnWQn1d5Rc6/KEuPDQdllGUX3dOyoSbX149MRdEKco5Xu3nJnpqIiJoUtZh G3zWuzNZ228yOv7QkU0euNnz3LheJDFQC6ZrSbj4eSBqsiJecyxdfrsqJKYaXbTr7pYaKh k9dBIeGdC6QXXWZ04DN5A1zxhysryRY= Received: by mail-ej1-f50.google.com with SMTP id a640c23a62f3a-ace94273f0dso335250766b.3 for ; Tue, 06 May 2025 08:50:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1746546633; x=1747151433; darn=kvack.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=v/cd32lR18qNaF2U9DfiKee4Kiegk/8OhHBbE5vcd/o=; b=BV+SSnfr6giMOdlizGhCC8BD/VBNCMfUzvAhXcgWxem1r1o2JwhZ0Hd/wopWv7RD65 +zZIGoAqm9YVaW1KeH7vwKcqFByzksz/G9PgGOx8yXP0c8zd74s+XbWNOH9L91Hpf/U4 flW3H3madhz+gpPTfeykgs9dCmHvskDrdym4SXw5I5M8zYLm7PSA5AX5MzwLsGk1Rw9y xRlj9EVS7nzP71NLCw1iWhTu2ULomhMkb488vkBcBw7FnGocXkOnubqlx3BjkenVWc9t IvquqPMLhrnjYZoY3dc3bFapB8YG7Y3BSWaY0CozJSL4EKQU2bMzL1hSeUVxjK0TE5Lp Jepw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1746546633; x=1747151433; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=v/cd32lR18qNaF2U9DfiKee4Kiegk/8OhHBbE5vcd/o=; b=mM4DKE+kluPrd2WN7/8o9EDYSnVTHexYwuOAVs3gBBCCfuJDNYsG4MPbyL/bSp9ye1 vxsmSEPJuD6Yjn/AVxcWgUvTFG9vHQnEmbnAboyUFO83fbS9ZCTH334ET6usC6VjvR3a ApBWCAF80YZAYKmvOrBujGgGFn9opC2L5gDvCeVldUpcmzHISeKU3FyVmwRrsQrsqCeE BNKTOXyQAxRgeAUUFhb9nWEDDDTo8gyCMrgdO/wss2LycdT4/b+qDU7C91PPrQRDUFbL w8xaT4c7PKTURL6AJmUPQQe/hdS1UDVx8rsQXtXjROnMkzOuFNJcGjdEhIphcl9gNy3+ gtZQ== X-Forwarded-Encrypted: i=1; AJvYcCUdtaEEelvlCElXsbya9bz+zZmuEKZbQVGWS+0lrSZ3KX7vkiF8FO3vhj4yWKhisthu6Pkv37nuJw==@kvack.org X-Gm-Message-State: AOJu0YysWQOlLDwr1CpyMHQhKDKVgu1Y/zsa6BRkYKpOX34FEQuqNNQF 0LvD/I2qBm4Nkuf61Ju2rt5GAogV77ZO1E/Kyi3vfZyjUrCWJTUa X-Gm-Gg: ASbGncvqcEjHJIPrbHpzBev+JfQ0pkiGp5xWCMcVs8/BjiZvBeI+TbPC/fWT/cec6jn jnXsE9FXzxRsekClcAX8YDFaCQzTFBaQFiRpwhHHInLApZC3smxAzDCWs6H0Do5tuZG1+z4gPM1 KF2Lg8tfE0KKi6Qh49MVw1PvAmNcM3jLADn0y792Yw94z5Ku8LbuablvVFP8CA+HRtgw+iUhk7w 0nOpoGmS3/CXJhbrnhLKCNm70hp8lu+7za2fZod2qLSSwnhf3sdvYt72ua/qXVfklqWrHJ+yEL+ tVUDjWqUAzCAHpgazqOwFyxTvbKoSoGA3T/W1ud77WE28/7zZZGIYs53 X-Google-Smtp-Source: AGHT+IHI07h3SWN4IWAmyQVPi93u++g4HV+GJJA/f5Bx3Mmaz6zA7XcY0iCTzxN3fSgKlR7yFWQWug== X-Received: by 2002:a17:907:1c1e:b0:ad1:77ae:d18e with SMTP id a640c23a62f3a-ad1a4b1119fmr1056367866b.56.1746546632859; Tue, 06 May 2025 08:50:32 -0700 (PDT) Received: from smtpclient.apple ([212.59.70.37]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ad1894c02f3sm735850766b.92.2025.05.06.08.50.30 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 06 May 2025 08:50:32 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.500.181.1.5\)) Subject: Re: [RFC PATCH 7/9] x86/mm: Introduce Remote Action Request From: Nadav Amit In-Reply-To: <09b6eb12ede47b2e2be69bdd68a8732104b26eb0.camel@surriel.com> Date: Tue, 6 May 2025 18:50:19 +0300 Cc: Linux Kernel Mailing List , "open list:MEMORY MANAGEMENT" , the arch/x86 maintainers , kernel-team@meta.com, Dave Hansen , luto@kernel.org, peterz@infradead.org, Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , Yu-cheng Yu Content-Transfer-Encoding: quoted-printable Message-Id: References: <20250506003811.92405-1-riel@surriel.com> <20250506003811.92405-8-riel@surriel.com> <03E5F4D7-3E3F-4809-87FE-6BD0B792E90F@gmail.com> <09b6eb12ede47b2e2be69bdd68a8732104b26eb0.camel@surriel.com> To: Rik van Riel X-Mailer: Apple Mail (2.3826.500.181.1.5) X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 0E1F140003 X-Stat-Signature: tzzmibxe1pjcmj5abt7t7zej48ocqo6p X-Rspam-User: X-HE-Tag: 1746546634-263242 X-HE-Meta: U2FsdGVkX19I3jhGTEeIFfKCRdZJ1hiF3EYWXYioV+M7w759QVPkDedeFvJUV0BclWQH6rzGuLyc3xHKBFduiXF+e6mfeEz9DIau1TB+vqKETqKekJh+3w+sgz63WVQMVJ5ALX8PUSvOWCfdr4o+TFhGRZpt0miTKiMCEESarXUpo4aniDDGiUi2vfeHe7tSS+0S9D7SJ94NYR/9umCH81UUu9bczi5XztKuwXL5Su0NuRsEaWFwQEFl1iZAA0qC8YeKaEvuHxuQpFWBA5IS7HdWLLReBzR7uhRm6um9j3EMo+E2kyp3BIbETGbj+dVNekiQlLv+ILIsbUG1qPgnLYZms4PDXZ7/32nBwMpkpiSUrZxiOKWLq8qcuwcdpXW7FfzkVo1Jcpw2hx2NicBTAz9I18spo4oAju5L+Bfs6Q2WaVMqS1RVtCvY9NO2XF3TpVScPnzozOPhP2wBDvkABn1JAMb/LjY711x8mtLMvUEbpcQx1/ARth8z0filDiKSvPPQYZ4DbU08Pajh2RiM74JQyRQJJQIQOvxM6BKUDZMnu3zMaaRXpBjHA6s3UdcItClp9yIS8aTIYQ1jQI1ZcBsAOTIzOWc0BXGGGdGyYESObU3KEhHuD50TNZGCbUlEmHJIdllO4l4l/0OoloPKQZPCt3nHtdUFxql9yCzymBaJ0SM+Ou1nXzBCRauGVHuDAmIl38gDvSYC64Jq/gYtTe9zD+ldVbNOiKhOAAz5mXx1/TUTC0DdSD2QpV9a1N4nSt/BE60XFdJdntKoqg8R6wWpfaLcH/QeLxQG7JRTIaYSlWuOJdr8eUhRNgp4KDasTCa3gXej31dwtlHqD2uU3aFwmwavqdJzWw9aK3N9uFOvYCkfaEAAfAprdOd1ECa3a5lqdhTfeLIe/cm3qB8K2In8hd6Wi7A0qkHhqQtVK2IP38rlGPdTIljXlCAEVU04MXJCqZnO05DI80z9lAf Z9A0qsof 4edkm6oOIhRM8HWqaI/E1T0eRBxQAAye/BQiGzZ9FuapDaXBjI5F5oxHRy2IIKbKjSc6jyblaSX6TdB4ZdkovqfbxE1hb8oPWq5l/Yc5mF0aNfiuwdbJsO1tggkqldDIhaPoGbrYcPBAMVTcloTBIwn8DnFAujH4bJ2/xtyNQJCNPRX+zJxX6ITTj0q/SYDiKZ8GNFn/kvPACJH6zQgKGNXNjNnlxVRv94/X/2UqnweNF1bzchPmUisoOFSuCeZxOiGw+2mLF51G/aYSVa5CILVJ6hxm9drBZAcCN2myTImervEftZ26JP5ovK5wGLIkBE4nM331VEa+vbqvvWD7fTDoCv0c1ryVM7gS1+IUeUy9b+LhUnnK3Z4GmcBWvctKNoNos0vg/fGDg/Wqufoy6WpptBcqk+9E7T4g9rAJuipdYFrfeseFYKo8MkkbqIFOutl37FEQRKHLSUkGEk5oH9Q23uC30AfjAxsTF X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: > On 6 May 2025, at 18:16, Rik van Riel wrote: >=20 > It gets better. Page 8 of the RAR whitepaper tells > us that we can simply use RAR to have a CPU send > itself TLB flush instructions, and the microcode > will do the flush at the same time the other CPUs > handle theirs. >=20 > "At this point, the ILP may invalidate its own TLB by=20 > signaling RAR to itself in order to invoke the RAR handler > locally as well" >=20 > I tried this, but things blew up very early in > boot, presumably due to the CPU trying to send > itself a RAR before it was fully configured to > handle them. >=20 > The code may need a better decision point than > cpu_feature_enabled(X86_FEATURE_RAR) to decide > whether or not to use RAR. >=20 > Probably something that indicates RAR is actually > ready to use on all CPUs. >=20 Once you get something working (perhaps with a branch for now) you can take the static-key/static-call path, presumably. I would first try to get something working properly. BTW: I suspect that the RAR approach might not handle TLB storms worse than the IPI approach, in which once the handler sees such a storm, it does full TLB flush and skips flushes of =E2=80=9Colder=E2=80=9D generations. You may want to benchmark this = scenario (IIRC one of the will-it-scale does something similar). > I think we have 3 cases here: >=20 > 1) Only the local TLB needs to be flushed. > In this case we can INVPCID locally, and skip any > potential contention on the RAR payload table. More like INVLPG (and INVPCID to the user PTI). AFAIK, Andy said INVLPG performs better than INVPCID for a single entry. But yes, this is a simple and hot scenario that should have a separate code-path. >=20 > 2) Only one remote CPU needs to be flushed (no local). > This can use the arch_rar_send_single_ipi() thing. >=20 > 3) Multiple CPUs need to be flushed. This could include > the local CPU, or be only multiple remote CPUs. > For this case we could just use arch_send_rar_ipi_mask(), > including sending a RAR request to the local CPU, which > should handle it concurrently with the other CPUs. >=20 > Does that seem like a reasonable way to handle things? It it. It is just that code-wise, I think the 2nd and 3rd cases are similar, and it can be better to distinguish the differences between them without creating two completely separate code-paths. This makes maintenance and reasoning more simple, I think. Consider having a look at smp_call_function_many_cond(). I think it handles the 2nd and 3rd cases nicely in the manner I just described. Admittedly, I am a bit biased=E2=80=A6=