From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95A54C4361A for ; Thu, 3 Dec 2020 17:14:29 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 13DED206D5 for ; Thu, 3 Dec 2020 17:14:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 13DED206D5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=amacapital.net Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 3BDB86B005C; Thu, 3 Dec 2020 12:14:28 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 36E456B0068; Thu, 3 Dec 2020 12:14:28 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 25E2B6B006C; Thu, 3 Dec 2020 12:14:28 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0254.hostedemail.com [216.40.44.254]) by kanga.kvack.org (Postfix) with ESMTP id 1015D6B005C for ; Thu, 3 Dec 2020 12:14:28 -0500 (EST) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id C8316180AD807 for ; Thu, 3 Dec 2020 17:14:27 +0000 (UTC) X-FDA: 77552619774.14.cake35_38131b3273bd Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin14.hostedemail.com (Postfix) with ESMTP id 22543182299A7 for ; Thu, 3 Dec 2020 17:14:26 +0000 (UTC) X-HE-Tag: cake35_38131b3273bd X-Filterd-Recvd-Size: 5878 Received: from mail-pj1-f65.google.com (mail-pj1-f65.google.com [209.85.216.65]) by imf45.hostedemail.com (Postfix) with ESMTP for ; Thu, 3 Dec 2020 17:14:25 +0000 (UTC) Received: by mail-pj1-f65.google.com with SMTP id hk16so1482004pjb.4 for ; Thu, 03 Dec 2020 09:14:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amacapital-net.20150623.gappssmtp.com; s=20150623; h=content-transfer-encoding:from:mime-version:subject:date:message-id :references:cc:in-reply-to:to; bh=jpxIDRJby02bDSmzqib9oh8ZXDBqI+2pH5k0P2lzdWM=; b=aRJu1HO4xnNzb8jocw95sh/+CjsELYLpn2jhaJctdiBvV0Fo/IKe2d9DUjnFTVLurX Zz5jY6puhccVi4/XYJcF7M1WWjBZ/WYrYvOH0BlxpJnlRQ5Gv20I5Qa54qDavxIpMqu7 46YMj6QXtaEh429/CwE53Heyk8rCED0kC2GVSiksY3g51x/NcNUwhV8nC9XoV+cO/mxt 4h3T5yaHfoFjomEoJs048LrdfUJhUseSC4iy7RdIuoRUVb60jBp4vPTph+8LXJR4h17K q4oZzEwHYNz5a1MVWTTRm26Dfo5pZpVJIp2icQJRZ7CVgm0Yy3vM/31Kuj9RXfgsUNyL FvEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:content-transfer-encoding:from:mime-version :subject:date:message-id:references:cc:in-reply-to:to; bh=jpxIDRJby02bDSmzqib9oh8ZXDBqI+2pH5k0P2lzdWM=; b=bGgwjH7WvcEZJwUafxzWoAjxOhgU1mEEYoyFEHxw+FytC43W/95J5vcneImdAjo1OL pLHvTe9WX0/iCP8AZ0HTunyCaRlFMTR/bdtZHYyKgAi3PTYJ8oSF857yH3qRGXxlZqbI S/SJzjjnKEabHrgCGNxcr6MogV0ziuahrzxeK5bg0yA2OnHpvUw2zZCbklTjy/oNwOo8 lY1wSXNCyEAVgdG3EluIZlczVtJlG8Ll+FXByaH9dVKq1C8926e3i0qyc826PPtDapbk o2LYC5GS0a/cgh+HNhLDUj9fIjoI0mUBS0SRoBHNHlSqXvqyNW3TOZldaL10djX8459K WPjw== X-Gm-Message-State: AOAM532isZAuxBfuB+lmLIJKTyIqG/9Greu+NuCi35ZfTmqp0LSE0JPA ukYWyRT17cZaATOJufVxbtBEBg== X-Google-Smtp-Source: ABdhPJwE4etOoyGMx9UOKs7yAhw7hgOhsifHtas0tGfE12clfk9l2UH+s+JZSLZnFMT+6hGUVEc2Aw== X-Received: by 2002:a17:90a:5988:: with SMTP id l8mr120805pji.82.1607015664612; Thu, 03 Dec 2020 09:14:24 -0800 (PST) Received: from ?IPv6:2600:1010:b02c:6432:59d6:b4ed:32aa:4315? ([2600:1010:b02c:6432:59d6:b4ed:32aa:4315]) by smtp.gmail.com with ESMTPSA id t9sm30146pjq.46.2020.12.03.09.14.23 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 03 Dec 2020 09:14:23 -0800 (PST) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable From: Andy Lutomirski Mime-Version: 1.0 (1.0) Subject: Re: [PATCH 6/8] lazy tlb: shoot lazies, a non-refcounting lazy tlb option Date: Thu, 3 Dec 2020 09:14:22 -0800 Message-Id: References: <20201203170332.GA27195@oc3871087118.ibm.com> Cc: Andy Lutomirski , Will Deacon , Catalin Marinas , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Dave Hansen , Nicholas Piggin , LKML , X86 ML , Mathieu Desnoyers , Arnd Bergmann , Peter Zijlstra , linux-arch , linuxppc-dev , Linux-MM , Anton Blanchard In-Reply-To: <20201203170332.GA27195@oc3871087118.ibm.com> To: Alexander Gordeev X-Mailer: iPhone Mail (18B121) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: > On Dec 3, 2020, at 9:09 AM, Alexander Gordeev wro= te: >=20 > =EF=BB=BFOn Mon, Nov 30, 2020 at 10:31:51AM -0800, Andy Lutomirski wrote: >> other arch folk: there's some background here: >=20 >>=20 >> power: Ridiculously complicated, seems to vary by system and kernel confi= g. >>=20 >> So, Nick, your unconditional IPI scheme is apparently a big >> improvement for power, and it should be an improvement and have low >> cost for x86. On arm64 and s390x it will add more IPIs on process >> exit but reduce contention on context switching depending on how lazy >=20 > s390 does not invalidate TLBs per-CPU explicitly - we have special > instructions for that. Those in turn initiate signalling to other > CPUs, completely transparent to OS. Just to make sure I understand: this means that you broadcast flushes to all= CPUs, not just a subset? >=20 > Apart from mm_count, I am struggling to realize how the suggested > scheme could change the the contention on s390 in connection with > TLB. Could you clarify a bit here, please? I=E2=80=99m just talking about mm_count. Maintaining mm_count is quite expen= sive on some workloads. >=20 >> TLB works. I suppose we could try it for all architectures without >> any further optimizations. Or we could try one of the perhaps >> excessively clever improvements I linked above. arm64, s390x people, >> what do you think? >=20 > I do not immediately see anything in the series that would harm > performance on s390. >=20 > We however use mm_cpumask to distinguish between local and global TLB > flushes. With this series it looks like mm_cpumask is *required* to > be consistent with lazy users. And that is something quite diffucult > for us to adhere (at least in the foreseeable future). You don=E2=80=99t actually need to maintain mm_cpumask =E2=80=94 we could sc= an all CPUs instead. >=20 > But actually keeping track of lazy users in a cpumask is something > the generic code would rather do AFAICT. The problem is that arches don=E2=80=99t agree on what the contents of mm_cp= umask should be. Tracking a mask of exactly what the arch wants in generic c= ode is a nontrivial operation.