From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BFD75C433EF for ; Sun, 9 Jan 2022 20:34:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D9A856B0072; Sun, 9 Jan 2022 15:34:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D4A2E6B0073; Sun, 9 Jan 2022 15:34:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BEA316B0074; Sun, 9 Jan 2022 15:34:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0234.hostedemail.com [216.40.44.234]) by kanga.kvack.org (Postfix) with ESMTP id B00CA6B0072 for ; Sun, 9 Jan 2022 15:34:19 -0500 (EST) Received: from smtpin11.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 68E44180C4E48 for ; Sun, 9 Jan 2022 20:34:19 +0000 (UTC) X-FDA: 79011901038.11.862C57D Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) by imf06.hostedemail.com (Postfix) with ESMTP id 0C7CF180008 for ; Sun, 9 Jan 2022 20:34:18 +0000 (UTC) Received: by mail-pf1-f172.google.com with SMTP id m1so8972037pfk.8 for ; Sun, 09 Jan 2022 12:34:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=IyjVliw9s696kRZB/QLeKMSxq5UJgw6Khm/VAr8RauQ=; b=QkwjEWKRB54fs1Ng+rV42lgLJVyLc7Nx+rISn2U5YvfLIMmK9jEQ05t4wFKGeCR3BP AJ0t12d7y1lkoZhJ1EPe6NKr4RPFd9VO7wxHk0Vu8xUfVmV7dq6CTxCj/K790UI+D/Vp oA4FqP0n3zTueZVfUh4+GMePK4tP6JfiB2a4/JB6MK0JgJC5k85OLJ45fMdyiyE76WG5 x4/kkorbYeOWEyuZT4sTRpKkf1Kpf8TQ98sztnrCEHNfwkzZGgLahn7IMaGKQwqpzHZN ZASobw9UvA2zGGoQlzCC/wrmt8z6gI9EfP+3Jy7MbsIpXROlB/SSdrcaFGgKdURv1EmH 6quQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=IyjVliw9s696kRZB/QLeKMSxq5UJgw6Khm/VAr8RauQ=; b=q8Py0rHFVPSVkM9lyxuC59eSFCOMENc6UaQjiwdAidrQ4Zfb2qSIU582ql/8aSnn4T 34QmxRm01FWGd45xs91ikWUvjrOE0IETWHA1ERcJjPaz87RtN9MHDQHH9vL8XEz+1LmD UBS3ZvD48uG0SItaY8b/Dwo4zq6WA0Tev1//wgfNVigBhQ50IMNTh6/beSZUjN4g6fvK D2MfRiqMirMXJ7HEx9bZAf2jID9J6qnqa+O5ZK/CCy6XjfK4cfZg0i6RnJZyLmyM2O9i sfB8XJcAvzSAVnPyy1Jui3qmMpDyCozUPXd4UX6zCtjNZkRPadOsK7z6vtqmG+cF4Lct fnJQ== X-Gm-Message-State: AOAM533mcwpSPG3vbO8F7vTfY2g4o0fqJeTXm6vGH7z5dVbO+y74XgTy QoHGuQAGjqvbBidliOdMGzw= X-Google-Smtp-Source: ABdhPJzCOtBYtlvHzWgZIm0xZ3/oUi3BFDBMsuS9sjcW7ngBA5yQpVoURDt3XbrmUithkD0A/K1DSA== X-Received: by 2002:a63:78cd:: with SMTP id t196mr55155833pgc.503.1641760457978; Sun, 09 Jan 2022 12:34:17 -0800 (PST) Received: from smtpclient.apple (c-24-6-216-183.hsd1.ca.comcast.net. [24.6.216.183]) by smtp.gmail.com with ESMTPSA id g5sm4689939pfj.143.2022.01.09.12.34.16 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 09 Jan 2022 12:34:17 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Subject: Re: [PATCH 16/23] sched: Use lightweight hazard pointers to grab lazy mms From: Nadav Amit In-Reply-To: <355c148c-06a8-4e15-a77b-0ea2e22bf708@www.fastmail.com> Date: Sun, 9 Jan 2022 12:34:15 -0800 Cc: Linus Torvalds , Andrew Morton , Linux-MM , Nicholas Piggin , Anton Blanchard , Benjamin Herrenschmidt , Paul Mackerras , Randy Dunlap , linux-arch , the arch/x86 maintainers , Rik van Riel , Dave Hansen , "Peter Zijlstra (Intel)" , Mathieu Desnoyers Content-Transfer-Encoding: quoted-printable Message-Id: <0B23F7F5-96F1-4E2C-8C17-0ECC21CA14C6@gmail.com> References: <7c9c388c388df8e88bb5d14828053ac0cb11cf69.1641659630.git.luto@kernel.org> <739A3109-04DD-4BA5-A02B-52EE30E820AE@gmail.com> <355c148c-06a8-4e15-a77b-0ea2e22bf708@www.fastmail.com> To: Andy Lutomirski X-Mailer: Apple Mail (2.3654.120.0.1.13) X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 0C7CF180008 X-Stat-Signature: 7wr16h99jyyfe7tf1qaak888e1r38fn8 Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=QkwjEWKR; spf=pass (imf06.hostedemail.com: domain of nadav.amit@gmail.com designates 209.85.210.172 as permitted sender) smtp.mailfrom=nadav.amit@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-HE-Tag: 1641760458-254632 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: > On Jan 9, 2022, at 11:52 AM, Andy Lutomirski wrote: >=20 > On Sun, Jan 9, 2022, at 11:10 AM, Linus Torvalds wrote: >> On Sun, Jan 9, 2022 at 12:49 AM Nadav Amit = wrote: >>>=20 >>> I do not know whether it is a pure win, but there is a tradeoff. >>=20 >> Hmm. I guess only some serious testing would tell. >>=20 >> On x86, I'd be a bit worried about removing lazy TLB simply because >> even with ASID support there (called PCIDs by Intel for NIH reasons), >> the actual ASID space on x86 was at least originally very very >> limited. >>=20 >> Architecturally, x86 may expose 12 bits of ASID space, but iirc at >> least the first few implementations actually only internally had one >> or two bits, and hashed the 12 bits down to that internal very = limited >> hardware TLB ID space. >>=20 >> We only use a handful of ASIDs per CPU on x86 partly for this reason >> (but also since there's no remote hardware TLB shootdown, there's no >> reason to have a bigger global ASID space, so ASIDs aren't _that_ >> common). >>=20 >> And I don't know how many non-PCID x86 systems (perhaps virtualized?) >> there might be out there. >>=20 >> But it would be very interesting to test some "disable lazy tlb" >> patch. The main problem workloads tend to be IO, and I'm not sure how >> many of the automated performance tests would catch issues. I guess >> some threaded pipe ping-pong test (with each thread pinned to >> different cores) would show it. >=20 > My original PCID series actually did remove lazy TLB on x86. I don't = remember why, but people objected. The issue isn't the limited PCID = space -- IIRC it's just that MOV CR3 is slooooow. If we get rid of lazy = TLB on x86, then we are writing CR3 twice on even a very short idle. = That adds maybe 1k cycles, which isn't great. Just for the record: I just ran a short test when CPUs are on max freq on my skylake. MOV-CR3 without flush is 250-300 cycles. One can argue that you mostly only care for one of the switches for the idle thread (once you wake up). And waking up by itself has its overheads. But you are the master of micro optimizations, and as Rik said, I mostly think of TLB shootdowns and might miss the big picture. Just trying to make your life easier by less coding and my life simpler in understanding your super-smart code. ;-)