From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46837C001DC for ; Wed, 12 Jul 2023 17:24:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CB87A6B007D; Wed, 12 Jul 2023 13:23:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C41596B007E; Wed, 12 Jul 2023 13:23:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ABBFD6B0080; Wed, 12 Jul 2023 13:23:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 965056B007D for ; Wed, 12 Jul 2023 13:23:59 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 72745B02FD for ; Wed, 12 Jul 2023 17:23:59 +0000 (UTC) X-FDA: 81003632598.18.E591A3E Received: from mail-pf1-f175.google.com (mail-pf1-f175.google.com [209.85.210.175]) by imf29.hostedemail.com (Postfix) with ESMTP id 08A8C120014 for ; Wed, 12 Jul 2023 17:23:56 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=dabbelt-com.20221208.gappssmtp.com header.s=20221208 header.b=hZjwQME8; dmarc=none; spf=pass (imf29.hostedemail.com: domain of palmer@dabbelt.com designates 209.85.210.175 as permitted sender) smtp.mailfrom=palmer@dabbelt.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689182637; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:dkim-signature; bh=9tQ2ZbbyLLdcDDhvAxA5qEVgxy8o1H8UkklbLpk31ig=; b=oNl4Yc/xkUL6URNBSNdmh93pGUeP4dXDY98q49IU3EPJZ5/FTNK9cfka55pqeLaJ9T3JiU YFrZMgx/TCOrWq8NSmRrFCWfm6xU1kWjXLntAgNjE/NAakVOZBfMhnMnKER43Jq1NU1h4r EfnwL8Ufn7oLwdWfxO0x7JoHd3Bcvzs= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=dabbelt-com.20221208.gappssmtp.com header.s=20221208 header.b=hZjwQME8; dmarc=none; spf=pass (imf29.hostedemail.com: domain of palmer@dabbelt.com designates 209.85.210.175 as permitted sender) smtp.mailfrom=palmer@dabbelt.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689182637; a=rsa-sha256; cv=none; b=AhpauDvSZxUEDiLenx99sA/8uk/+rxtfQh9yAImehrzVoYS3g2gLW50zFrC2hI82Ttl4y7 /wJum9FhGMAyxZ19/+bbBt/aq8WDDiHO6D+wQh7bMgpM4QHrudXFmA2NnO+u/iDac/Apqo zCr7cZ4OEjIL/nZVzTfF4Jhk5hYEmZU= Received: by mail-pf1-f175.google.com with SMTP id d2e1a72fcca58-668704a5b5bso6466910b3a.0 for ; Wed, 12 Jul 2023 10:23:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dabbelt-com.20221208.gappssmtp.com; s=20221208; t=1689182636; x=1691774636; h=content-transfer-encoding:mime-version:message-id:to:from:cc :in-reply-to:subject:date:from:to:cc:subject:date:message-id :reply-to; bh=9tQ2ZbbyLLdcDDhvAxA5qEVgxy8o1H8UkklbLpk31ig=; b=hZjwQME8K62afCa/lKxAV5oDAQbtGEBFMc9MArh/PEz7oE9G6vUVAfOnb/9R4vBGxK 1p1NcH+qnX13Ru4H4jouGvPcdhK1h6k+qCLXI8YVgtZPBu4W+jMhiqB+u/ZVxaEw0Hp6 ihr02PgYKEWNMlwcQfr2/i7L4yiGRocd/mcHeDedHPSxJpwwaeiIvvgp1H7KSoPD7f+8 o0jddnMArxfV1MgrK7mGxmRLv6ix4n+Nl1ki4KAFn1VZOXE1Fc/kMqnaq33dhWTBTKYY g2APClzhteWp3vYwyHS6lDvUJbrZdjwu2siZMPrkDOwHRcHeAF+ln9H611Ur9MnJmfCJ lZFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689182636; x=1691774636; h=content-transfer-encoding:mime-version:message-id:to:from:cc :in-reply-to:subject:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=9tQ2ZbbyLLdcDDhvAxA5qEVgxy8o1H8UkklbLpk31ig=; b=RFUnPDc8mXUu88dbQ85txCh/bSo4KAkXgGRgFvw3S93lD1n1FIqW//ewlRB0gvFYZx avL4GRnqW2cCAgM2CZePBVs83BvKe+fNDvnjkERGSCf3v5auq0x7rXR5NBJdLYn1MP2X Yr4rxZ4skFSiEv0JKOb1tYmevDvXBoCuPCYJQjhmoreTtTNnCPdGySpDOKBm3c+dD/+h UMHfoDGkoEfHbSb2+1K1mMyprIUmNWIjKQMWCgPThmlSU+ueFe5RbGDFp7BG/rf6dX8f TR5obIWeDqRBgtCz6apBslhwzFLRD86hq+zXGsxccIGyP6OGfv8qBpbyX0CgQRllU8Wn IWzg== X-Gm-Message-State: ABy/qLZHlhi98XvVLV9GNuokb8EDlzijUZ6LppyntJ7Zi/P/ymQQ4jFl HZm7A8U1IpB725wjZAU7lBNZ8A== X-Google-Smtp-Source: APBJJlGtz88dgA9P7FtTfAm49xR0+wVRjyt/ZcPAukIFsqkezm35JqgXO0FcyxCpfwYFDyT4jJgTMg== X-Received: by 2002:a05:6a00:84d:b0:682:54b9:1093 with SMTP id q13-20020a056a00084d00b0068254b91093mr25574401pfk.15.1689182635685; Wed, 12 Jul 2023 10:23:55 -0700 (PDT) Received: from localhost ([50.38.6.230]) by smtp.gmail.com with ESMTPSA id ey24-20020a056a0038d800b00666add7f047sm3821151pfb.207.2023.07.12.10.23.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 12 Jul 2023 10:23:55 -0700 (PDT) Date: Wed, 12 Jul 2023 10:23:55 -0700 (PDT) X-Google-Original-Date: Wed, 12 Jul 2023 10:23:08 PDT (-0700) Subject: Re: [PATCH 0/4] riscv: tlb flush improvements In-Reply-To: <20230712-frying-unaired-e3acb5150e8b@spud> CC: alex@ghiti.fr, Conor Dooley , alexghiti@rivosinc.com, Will Deacon , aneesh.kumar@linux.ibm.com, akpm@linux-foundation.org, npiggin@gmail.com, peterz@infradead.org, mchitale@ventanamicro.com, vincent.chen@sifive.com, Paul Walmsley , aou@eecs.berkeley.edu, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org From: Palmer Dabbelt To: Conor Dooley Message-ID: Mime-Version: 1.0 (MHng) Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 08A8C120014 X-Stat-Signature: gjrnfbcw8c1ipsj3h43ppusi79aqssdx X-Rspam-User: X-HE-Tag: 1689182636-165715 X-HE-Meta: U2FsdGVkX1+b5I/lu/tpnTuclyQ4tHMsuQLAx3CtVzJ64LcLbVNdYgntCq1951bf6+uLS3xAm+EI++ULcwde0KuhiE4qBBEBECbzTZ/+i6DIQeTcg5h8wtrRCiHkMUixuYdhYMKw/OZVz3xqWCzp4Se5vhZqfbxWD9RlU8l7Bu2pu9QAbyR+5IZWzJRv5PaDkVGNN/jQ84rpZF2NFqxJi5bKOneCneEWwLInvGHzHEMM7sC0kc3p8mM+dMlYDVjb7uz3r6D2SyhC84yIwnckkBJ1hKbIxanTNK7nUZyVtSx4t7TdslZyDihcgswYtzZboJV0c81+oJ+0lAg/P1tlNvvIvBmiXtczXPPffFVcrl7LczGU8yK7jK/0b7u6MGzTa6XT8AUmIm/bEOU4lF7Q6GWJOqz6kTKIb+lvHJs51ZQWMBm7t4ROrawaWH4gp8a4loTM/I387+aKJFMt/lzb3q95fXmLQJYx46bPqKK2S4ZDcFWo19DJ/ON1yeOBD+428S5JejQrXZRTbkUSQhcJfTQTAYgJ/BHoLGG4aYxMGDnuFAIy7stoOPNdpf83JTy2ma3JivNUO12AGcoQQ1ClSZm+5pj8eL28dtzEFNo/4Y3A5ZpLr52KGPQiaLeL7n+90i3qvBYzOv62g1THBJLnmUOt1tkFGZZDp/C/JK5jytyTxMakBUnSdiHbmpEBRHf3Plpnyv7YqhLKT6T54H7TgmLa4ofyQmlXVzMYEhbi8w0sCUdSfDjvWmPIaq6N+MkhfLRtpGVknUIexPHapqsZ/CfhDuRztMdQzdHSoa1COms4FijD4eCNG9D5H3HbP2mfS4dBkH/fcyC/kAW86ub19yztV4mjpVvpHhnRE4/OAF81Z80RBsDIi9gHR0Q6nZeE9T/lDulm5QzD7BAWOKbKjayONTVbXNKK3dnIr7KchpqYtk8NLpI6TGcbiK4/2TlTll7L9BoE24hR6y3XCjT 9Rnn7bSQ iTaIaJk3pyQnby4FdhiYyEIbzd/CRr8Ap435LW1UiRPLzrf49gRPLg/pFifHMgGAvJ0N5sqLkr97GTFb9+tIEanm96hboPkjSK7NOSitqcEH4pxjUSEFC66buryEokAkNA4cm5Dq1qFerJwkaRSoE06SxZvc0yPpPiX1OBpIYLn/jcRMRv3wUUuNHoLekZihtm8ZxGwous19iygyAaF+4oC8NJhnhDvRV/WFLJVauRAH1K3+5zcmHHpaam2+AytupvHnsZhMou/KOkhouDPV8OdCnt5/4rZ6/O5MAZXafceihxgiW0vMXuon4Awmt9wPNY32uWRrsZUWHGD7PohWfGzYA2McQiXBHHyb6nLSxTplv0Rf9t5Ac58edNGmDyVR2qEiNhQDC2FCTlxn/H3ypvyE7jWN6UUJJ2u79C2HkSjlPvBkIpddf1RYAwQgB7fI9/PNn7JzR85chtpgozW0nFbvH/JysAjn45QQ82W7pST5JCdtvSMLBkwnCLbsKGMiwFlSKIbZpiLOJrkJMNwSh9T23y1aPJ2AiSm8t X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, 12 Jul 2023 10:19:47 PDT (-0700), Conor Dooley wrote: > On Wed, Jul 12, 2023 at 05:18:00PM +0200, Alexandre Ghiti wrote: >> On 12/07/2023 09:08, Conor Dooley wrote: >> > On Tue, Jul 11, 2023 at 09:54:30AM +0200, Alexandre Ghiti wrote: >> > > This series optimizes the tlb flushes on riscv which used to simply >> > > flush the whole tlb whatever the size of the range to flush or the size >> > > of the stride. >> > > >> > > Patch 3 introduces a threshold that is microarchitecture specific and >> > > will very likely be modified by vendors, not sure though which mechanism >> > > we'll use to do that (dt? alternatives? vendor initialization code?). >> >> >> @Conor any idea how to achieve this? > > It's in my queue of things to look at, just been prioritising the > extension related stuff the last few days. Hopefully I'll have a chance > to think about this tomorrow.. Famous last words probably. > >> > > Next steps would be to implement: >> > > - svinval extension as Mayuresh did here [1] >> > > - BATCHED_UNMAP_TLB_FLUSH (I'll wait for arm64 patchset to land) >> > > - MMU_GATHER_RCU_TABLE_FREE >> > > - MMU_GATHER_MERGE_VMAS >> > > >> > > Any other idea welcome. >> > > >> > > [1] https://lore.kernel.org/linux-riscv/20230623123849.1425805-1-mchitale@ventanamicro.com/ >> > > >> > > Alexandre Ghiti (4): >> > > riscv: Improve flush_tlb() >> > > riscv: Improve flush_tlb_range() for hugetlb pages >> > > riscv: Make __flush_tlb_range() loop over pte instead of flushing the >> > > whole tlb >> > The whole series does not build on nommu & this one adds a build warning >> > for regular builds: >> > + 1 ../arch/riscv/mm/tlbflush.c:32:15: warning: symbol 'tlb_flush_all_threshold' was not declared. Should it be static? >> > >> > Cheers, >> > Conor. >> >> >> I'll fix the nommu build, sorry about that. Weird I missed this warning, >> that's an LLVM build right? That variable will need to overwritten by the >> vendors, so that should not be static (but it will depend on what solution >> we implement). > > Just make it static until we actually have a vendor implementation of > this stuff please, since we don't know what that will look like yet. It's just a performance issue, right? IIRC the SiFive errata wasn't actually based on how many TLB flushes happen, they're just broken in general so it was a probability thing. If that's the case I agree we can just start with something arbitrary to start and then figure out how to set the tunable later. It's probably going to be workload-specific too, so we'll probably end up with both a firmware default and a userspace override (maybe a sys entry or whatever).