From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 026DAC433EF for ; Sun, 9 Jan 2022 19:11:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E92D96B0071; Sun, 9 Jan 2022 14:11:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E436E6B0073; Sun, 9 Jan 2022 14:11:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CE3026B0074; Sun, 9 Jan 2022 14:11:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0021.hostedemail.com [216.40.44.21]) by kanga.kvack.org (Postfix) with ESMTP id BCE9B6B0071 for ; Sun, 9 Jan 2022 14:11:15 -0500 (EST) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 7940C90071 for ; Sun, 9 Jan 2022 19:11:15 +0000 (UTC) X-FDA: 79011691710.07.524D56E Received: from mail-ed1-f41.google.com (mail-ed1-f41.google.com [209.85.208.41]) by imf05.hostedemail.com (Postfix) with ESMTP id 2A01D10000D for ; Sun, 9 Jan 2022 19:11:15 +0000 (UTC) Received: by mail-ed1-f41.google.com with SMTP id o6so45079875edc.4 for ; Sun, 09 Jan 2022 11:11:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux-foundation.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=YPOuTFQ/EFRlwfSEA3ipij++Ulf3u+5g+8CZYfsItBk=; b=AWGxyD2i+0VTrdb8KlRqAQ5n6fHAko3chHEhczjSNVua/1NPfcI4nbx0hg+4qt6xiX sS/zBrw6wnKvw92Ksmt5nmllSDLmc3J8Ra1HcjYvjIUoJZ8AzFA72hfzrOyU1vpJJ1ex IcvSo8dXW4uOAiE4lOmNhJmQ68qhxiDj69vU8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=YPOuTFQ/EFRlwfSEA3ipij++Ulf3u+5g+8CZYfsItBk=; b=U8AKufwi3khZGWgs3ol3UnmJTWws1Pj3KkNx80+LB2SZ9xDwJumGzsPwsWabSRa71X CZzckW+leqaFrKreNbL9nE9uI1+P34VtBExnfmyW7vtsQMT0cF/kEn5RfShjgxzmLK+c lRDcEw87Dcdqq3XJkEEwg/xaZuwQB+EHPDUWwhVOWOkcAPdeNSxgYFOxLm1WuxO6YZVw plJ1RIhk1WVANUW9WHRmpjtddFjwandOBPHpaeCCNKd+/cuDfFXb3ldEoYTNOUqqd8oC I+6/wvp+uGC2HD39DOgfIzDfCEMzskEv+M4mc+0FhC1vhS651/EFUqG1U2EnuunTcKxG ZCaQ== X-Gm-Message-State: AOAM530C1zAQ9kEusJCHx1MEQlosSoWvf63JdVQG2u1Fu4E5ND1raDah ju6nNJjYCa3huT25acAxjnest7xgmSGqWxK+yX8= X-Google-Smtp-Source: ABdhPJwEW98uvGAVYXeUqPXZS04GdfIvtgdIodvHrwn5byPvDhD2rIwsrIfa8rwWffAf8FbCTsIDog== X-Received: by 2002:a17:906:2f08:: with SMTP id v8mr2737230eji.708.1641755473461; Sun, 09 Jan 2022 11:11:13 -0800 (PST) Received: from mail-wm1-f48.google.com (mail-wm1-f48.google.com. [209.85.128.48]) by smtp.gmail.com with ESMTPSA id 18sm1625972ejw.187.2022.01.09.11.11.11 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 09 Jan 2022 11:11:11 -0800 (PST) Received: by mail-wm1-f48.google.com with SMTP id p1-20020a1c7401000000b00345c2d068bdso8483905wmc.3 for ; Sun, 09 Jan 2022 11:11:11 -0800 (PST) X-Received: by 2002:a05:600c:4f13:: with SMTP id l19mr19329898wmq.152.1641755471150; Sun, 09 Jan 2022 11:11:11 -0800 (PST) MIME-Version: 1.0 References: <7c9c388c388df8e88bb5d14828053ac0cb11cf69.1641659630.git.luto@kernel.org> <739A3109-04DD-4BA5-A02B-52EE30E820AE@gmail.com> In-Reply-To: From: Linus Torvalds Date: Sun, 9 Jan 2022 11:10:55 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH 16/23] sched: Use lightweight hazard pointers to grab lazy mms To: Nadav Amit Cc: Andy Lutomirski , Andrew Morton , Linux-MM , Nicholas Piggin , Anton Blanchard , Benjamin Herrenschmidt , Paul Mackerras , Randy Dunlap , linux-arch , X86 ML , Rik van Riel , Dave Hansen , Peter Zijlstra , Mathieu Desnoyers Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 2A01D10000D X-Stat-Signature: cpbkqybh4iokyf4ome1m8ekuytat5h7u Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=google header.b=AWGxyD2i; dmarc=none; spf=pass (imf05.hostedemail.com: domain of torvalds@linuxfoundation.org designates 209.85.208.41 as permitted sender) smtp.mailfrom=torvalds@linuxfoundation.org X-Rspamd-Server: rspam11 X-HE-Tag: 1641755475-472671 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Sun, Jan 9, 2022 at 12:49 AM Nadav Amit wrote: > > I do not know whether it is a pure win, but there is a tradeoff. Hmm. I guess only some serious testing would tell. On x86, I'd be a bit worried about removing lazy TLB simply because even with ASID support there (called PCIDs by Intel for NIH reasons), the actual ASID space on x86 was at least originally very very limited. Architecturally, x86 may expose 12 bits of ASID space, but iirc at least the first few implementations actually only internally had one or two bits, and hashed the 12 bits down to that internal very limited hardware TLB ID space. We only use a handful of ASIDs per CPU on x86 partly for this reason (but also since there's no remote hardware TLB shootdown, there's no reason to have a bigger global ASID space, so ASIDs aren't _that_ common). And I don't know how many non-PCID x86 systems (perhaps virtualized?) there might be out there. But it would be very interesting to test some "disable lazy tlb" patch. The main problem workloads tend to be IO, and I'm not sure how many of the automated performance tests would catch issues. I guess some threaded pipe ping-pong test (with each thread pinned to different cores) would show it. And I guess there is some load that triggered the original powerpc patch by Nick&co, and that Andy has been using.. Anybody willing to cook up a patch and run some benchmarks? Perhaps one that basically just replaces "set ->mm to NULL" with "set ->mm to &init_mm" - so that the lazy TLB code is still *there*, but it never triggers.. I think it's mainly 'copy_thread()' in kernel/fork.c and the 'init_mm' initializer in mm/init-mm.c, but there's probably other things too that have that knowledge of the special "tsk->mm = NULL" situation. Linus