From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E787C4332F for ; Tue, 7 Nov 2023 08:38:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 901C58D0035; Tue, 7 Nov 2023 03:38:23 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8D8FB8D0001; Tue, 7 Nov 2023 03:38:23 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 779858D0035; Tue, 7 Nov 2023 03:38:23 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 673138D0001 for ; Tue, 7 Nov 2023 03:38:23 -0500 (EST) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 373AB1A058E for ; Tue, 7 Nov 2023 08:38:23 +0000 (UTC) X-FDA: 81430506486.30.D40A82D Received: from mail-ed1-f50.google.com (mail-ed1-f50.google.com [209.85.208.50]) by imf17.hostedemail.com (Postfix) with ESMTP id 5409C40012 for ; Tue, 7 Nov 2023 08:38:21 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=iVobmsPo; spf=pass (imf17.hostedemail.com: domain of nadav.amit@gmail.com designates 209.85.208.50 as permitted sender) smtp.mailfrom=nadav.amit@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1699346301; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=y6dvthR2+uXXaKtZhfu19I8uRaF5Ucp/hKjGsFNZse0=; b=jTnCOdEAecIjrPev2H9D399o8ffrezs2qziiWBsWGs9IKi7q97pEj64dYVVsJMjANHIc3+ p/T0QDP5yKtHzLrSNhZQE7ERH4rX7IeAxV2rgKfcIxYEvEszekq5LhNmO+TYXYK/bFnyC5 XUr3cJNC9PrNiT9fxJHdFmi18sPspPY= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1699346301; a=rsa-sha256; cv=none; b=jMk0RTYfvVV6s9a+PFf/tWk/r70ji5DXa39XMUPj9KzwyvhVyF2HXnqDXPdHfl1wQWb6bA FDbjgkEQGhJLK0a7aLQPtsxgBEOztPLlqvMTnsa8WZhmeRG+ImcX6UX6gF17ik7MzGQrfl Owb4vNpylLFFs/Wjx3sprwnKmOb6fjc= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=iVobmsPo; spf=pass (imf17.hostedemail.com: domain of nadav.amit@gmail.com designates 209.85.208.50 as permitted sender) smtp.mailfrom=nadav.amit@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-ed1-f50.google.com with SMTP id 4fb4d7f45d1cf-5406c099cebso8885120a12.2 for ; Tue, 07 Nov 2023 00:38:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699346300; x=1699951100; darn=kvack.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=y6dvthR2+uXXaKtZhfu19I8uRaF5Ucp/hKjGsFNZse0=; b=iVobmsPopHsurTX161It06542nUSMVPbrRjANvkc47ecWWZehc3xGKdwkuMTrKaEcW CAH8pshFiV9uieHVOsn4msUqlUc7dc1rAMLecSuVYdk6fQ5mfpaizk7igS/KRKCAgMUr XL/28Jt9ICdvOaEsZNdTtfSJF26j7ulcQCOlUWl77vZcZtShNUcg1zH0C2r1xL32i09b +e4QWuNczzdjMkkXGk0GBd+WHlghgwk+h6qiSjmSxzIfDhbTtuUbz1JlTKLxqIfQGduo 3T507gEnaYuAurXpWkSrWHbx/DT4xXwci3YBVKEWbeEYB6eu+S46s+OajPutHfzzRaqI aqpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699346300; x=1699951100; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=y6dvthR2+uXXaKtZhfu19I8uRaF5Ucp/hKjGsFNZse0=; b=ANoAul/ZMb4ZZV5FatsGbfXEX7KbMBx4f8Lt8aOGZWLe616oakYwP4oJ/B44QxWtGp FSQf1NDhI3W6PeyZAvKhlEQscSrp/NoKQI5XQCU/XQSSzANPM4C7PUmTUHNP07XHCWl4 1SlAtNuoMGBwnwbCmPzamS0FOigSuV9guuVqN3pdK4/yog/D1xIaoO1/8UysicWJkkaa EM14906ChxPxMLRbDtdyV28nyPxL6EKC8XVz+5Zt34iwm7d4ng7rPRL9Wr4yntMdLajG 8cRjvmwv+pEzlLnjIMvXrn+No+sBzF8cDjK5yf4Ka7feQgWEUQgkhV61W+pJnN5G24uU j0jw== X-Gm-Message-State: AOJu0Yz2y4RjNbb6V8/6tQjBpVAgKrBhJMpYJLUI5KX1aUKVlRWxxCW0 RMKP5RcM/eLnFkLEDdh+5A8= X-Google-Smtp-Source: AGHT+IFAT164A5+0+DDOFocrTagB32BVc9/FCJDm9227j7/4JmRH7adGpr3sTfoKn2svR2FIqdu2QQ== X-Received: by 2002:a05:6402:28b3:b0:540:118:e8f with SMTP id eg51-20020a05640228b300b0054001180e8fmr25487702edb.24.1699346299233; Tue, 07 Nov 2023 00:38:19 -0800 (PST) Received: from smtpclient.apple ([132.69.236.77]) by smtp.gmail.com with ESMTPSA id r15-20020aa7cfcf000000b0053e469f6505sm5095738edy.26.2023.11.07.00.38.13 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 07 Nov 2023 00:38:17 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.200.91.1.1\)) Subject: Re: [PATCH v6 0/4] riscv: tlb flush improvements From: Nadav Amit In-Reply-To: Date: Tue, 7 Nov 2023 10:38:01 +0200 Cc: Alexandre Ghiti , Will Deacon , "Aneesh Kumar K . V" , Andrew Morton , Nick Piggin , Peter Zijlstra , Mayuresh Chitale , Vincent Chen , Paul Walmsley , Albert Ou , linux-arch@vger.kernel.org, linux-mm , linux-riscv@lists.infradead.org, Linux Kernel Mailing List , Samuel Holland , Lad Prabhakar Content-Transfer-Encoding: quoted-printable Message-Id: References: To: Palmer Dabbelt X-Mailer: Apple Mail (2.3774.200.91.1.1) X-Rspamd-Queue-Id: 5409C40012 X-Rspam-User: X-Stat-Signature: aj1tgcqi4rhiswixnogoaq3msf63sg5u X-Rspamd-Server: rspam03 X-HE-Tag: 1699346301-501827 X-HE-Meta: U2FsdGVkX187dwfSWlQqZm40Z+/kLUvd1pvB8nD+kRNqx5EjVnellNEcaHhX+mujzwm/mCozrmKhukBVNkKWfFruEPuRFR5U46SQ0tbiVEe9ClAyONHc8knyMp+Pp2IYJwfY4DYjSkKl/vIngyxrpm8ObmsXsQskceYyzbyEMND2tuBs5yxxVuYgmnsgdzHovrpxcsjbuJQeHAepuYU24FyAkKYS3CNe8aLZgz0swmosOOkOjIMTs8AbyGTijo7XZfo1cI6bQMsGDFQgYMvglu8wubnPMVd0SuzU+b5rmByfdx/z3wnsaDQ0GslNNmcgKQItzgWGTBJXdV70EIUM8mBlTPm/tj/PoUVRT9dESC5Yk55PBzJboi+5EHFTrGXj3n25HkFyRy3611GYK76+9ahZ00EKJ3F/dFjv9EgkLxKIwII6/050kiNd0YwM7Rx69hCDwTxePzrV0r62r88gxoh7ir1nftjwldLqpFqRt2I90t7iUzi/iIb7pN1DDb8LlgBLW9rTnx+j0Q1KwSTB/xbwB3CA95bMUvp5NsTXYDSP5JXokXIdWlz+WJSdNO/hsJP4fTlSBzgM7TGhuF2KoWGYS/3SHcDIG7rEeFSMuTBIsPbte56i0Eg+1u8Px/que/sFsPhoUgIYtpaV0rMPGZAd2Vm/RroVKo3n8lfPtA5VvEJbD3mzhiNf/4lB2mCSEzygW4g/nCkJdOgPLhePbZ6zCbRpxlQ/C9LyxHbGXYYS8vHWOlYROUBMWJzlkphAADVt5LdqTIN9WAWw0Z89d0UBZzNpJBzWEVvxy5j4/D5xeNyUtn/upvrRBlDXv/7559xlC3+vcUTqWd3Fko/mgdqoRchmbelTZRUGkwqRuBdqRXN6CqR8Qa8JHT1gLT2DWKyMU0wbhOZ8sgHiSSSm04s21/wiyV7mz9j6vpDaoy67ehnRLjgO7H4KsIwADlEArGkztVygrFybVMpyqFZ znsBr6oi HYU317J6Jliu+6kbx1sVJNu7BtLfNeYWN0zxtmy+m/xxcX0BmwCeD4/FQ4eTy8Hx7oVxveJF2ah7UR9X3AoTTdq7g8OTOxQMiLAEHy0FPRnqGAGkLgJn28r6NqRjlv5RtYq7fcZ0SmyHCkQ74Hj5texeb1nBm7qiPLa1zWV6xYf6MvdNP1lfa5BXrkjG1RzlaREpCUrxVc6i/k1or+7kzNF8TTXHsBytCqwsCoAjXtjstC1NcmPscollTvfS3MfCErzClIxZdehIvW6EZeeGlv0WxdSIkA6JclAUbzHYhxHywgfkhN6krvJ8X0ACUV2cy/Vyvm5/6UD6IZeAth4EmspeG/9iiQsWoBlCVY+EB+m2LQSmjscY1+ehdd0nMCpQ0G3cMLjaNashaMHNlkMJVWVcR4EftcahflH1IpphZTkLR8JjfhiRmfVumYyIidKueCiHGVRkHjw/V2/q9C6pPVLDIwTPQz6b+vwUtue4LPPfrVCyWMLQlQmZoptHJeAYqm12qSK0H7hjwpnK87dl/Yce6SF/ywj7xfU4b2GdFttQTO9Ribrd2GjlNVJ+qNfGLVhT43oRftLBN2YUsgr+tqZ6tfa6LDpxh/bV2o+ai2e/lUTx3EOQ/AktHoK/X1ZK4tYIu8apsnn3CjZtH9au62gGC190VciimcOxxKKqJVztJo3TZNKHPyOx9KgB7VWS+vv0T X-Bogosity: Ham, tests=bogofilter, spamicity=0.000003, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: > On Nov 7, 2023, at 9:00=E2=80=AFAM, Palmer Dabbelt = wrote: >=20 > On Mon, 30 Oct 2023 07:01:48 PDT (-0700), nadav.amit@gmail.com wrote: >>=20 >>> On Oct 30, 2023, at 3:30 PM, Alexandre Ghiti = wrote: >>> + on_each_cpu_mask(cmask, >>> + __ipi_flush_tlb_range_asid, >>> + &ftd, 1); >>=20 >> Unrelated, but having fed >=20 > Do you mean `ftd`? >=20 > If so I'm not all that convinced that's a problem: sure it's 4x`long`, = so we pass it on the stack instead of registers, but otherwise we'd need = another `on_each_cpu_mask()` callback to shim stuff through via = registers. I have no idea why you need to move stuff through the registers. >> Actually, it is best not to put it on the stack, if possible to = reduce >> cache traffic. >=20 > Sorry if I'm just missing something, but I'm not convinced this is a = measurable performance problem. I am not going to try to convince you (I ran the numbers on x86 a long time ago). There is a cost of bouncing cache-lines (because multiple cores access the stack), TLB-miss on remote cores (which is mostly avoidable if ftd is global). Having said that, the optimizations you added now and intend to add in the next steps are definitely more important for performance.