From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0705DC32793 for ; Wed, 18 Jan 2023 17:30:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 927DC6B0071; Wed, 18 Jan 2023 12:30:44 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8D82F6B0072; Wed, 18 Jan 2023 12:30:44 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 79FE36B0075; Wed, 18 Jan 2023 12:30:44 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 67B046B0071 for ; Wed, 18 Jan 2023 12:30:44 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 0A683A0B6F for ; Wed, 18 Jan 2023 17:30:44 +0000 (UTC) X-FDA: 80368609608.07.CB85F6D Received: from mail-vk1-f176.google.com (mail-vk1-f176.google.com [209.85.221.176]) by imf14.hostedemail.com (Postfix) with ESMTP id 1CF07100024 for ; Wed, 18 Jan 2023 17:30:40 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=google header.b=KHKCIU14; spf=pass (imf14.hostedemail.com: domain of torvalds@linuxfoundation.org designates 209.85.221.176 as permitted sender) smtp.mailfrom=torvalds@linuxfoundation.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674063041; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=45VTI8G2wI4nS2eH3K9pAMjugnugZ8TVFHXhCaZ9BN4=; b=tltl2lRJVxtyDgPyyqo5kPu+DnzLGgk2nDdoV//EZQXQWoX0UkxObITEI1SKDuE0cF7bbw jG1REaFhYs0jrSZGCojGj4KwZo2tVJtp+FizraaBknGw3OcJIxT72uXqvK6Papujffh2Tf mUN7OHRi2CaNj7wE9psBtWQ6xFlf9BQ= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=google header.b=KHKCIU14; spf=pass (imf14.hostedemail.com: domain of torvalds@linuxfoundation.org designates 209.85.221.176 as permitted sender) smtp.mailfrom=torvalds@linuxfoundation.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674063041; a=rsa-sha256; cv=none; b=v4LCLqjRVnVyyvBmsZpSlcP2KzILWZyojw1kTiQ3eOfG9LJiqFEjSfLl9mQuX2kEEllazB 10LHW1cpm8351+mn+wwNUKzuYmkIFW0MNnsr4rAPUKaHnUwRm9iEkneVZXhGsArKVe2el9 Mrq771keUUeJyA8flw3DegyzBrhm13Y= Received: by mail-vk1-f176.google.com with SMTP id q141so14146222vkb.13 for ; Wed, 18 Jan 2023 09:30:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux-foundation.org; s=google; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=45VTI8G2wI4nS2eH3K9pAMjugnugZ8TVFHXhCaZ9BN4=; b=KHKCIU14OK3pQZodreyf7cATu0CnBQ6Qp/eyFOrDD7AtFbqgOhemxuzdFJljGaeQVx fFRqwtRDmL31BGjxA2umkxBS2bzIUxJXG7wccQXsOU1uKFkErI9PWKf3gKBRSCN1cRFu fZ7h59H1HqHCAQh4z6y/0HqZ7cCPYHBKbh2DM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=45VTI8G2wI4nS2eH3K9pAMjugnugZ8TVFHXhCaZ9BN4=; b=dnjN1MEiA03cNYRZkCV49RGeUtxvbXUjRExhr0h3U/8tILezROEZF+c40B+5OaR85E 3VLqOX5cC8+4Wxk3onE1Bplp0Fccr1PQTLn8cYJZG3Oo9oHKKkLT/AZOKjyz/Knk9c3S loD4tD4Jrf3Jb3rwIzb79vjR3qkeRwY53i5J/7ZRCX5nKtV+db2bNlE43sSRjYTAgcMZ Mmdlj7w4SO8r5+Ow0S3w+Qr1C60a3QePMI1EMGaTlqN8WW9ha0dwlgGRahjDRslU4gJH g9QXQfeQKFRTOJNTEOF5UxOFq7UUbhRoFjruwZfS7y+EihdXmL94NKzXRCElNVjgvbzS 5ABA== X-Gm-Message-State: AFqh2koHHO7lo6TFOmtqDGyYW1Ai+84tUuKiBLpBqajBwghuS33aI+WX FghkV8M2Qrpq1bLt33s5ryJZs3bOW/DBzfzf X-Google-Smtp-Source: AMrXdXv+AQkUGgG/xsZpJSGQsCnMMlhpUbaOwu3q4eodvS+CG6O/ZJlffmA9G46zGjuDXZzz1sCnAQ== X-Received: by 2002:a1f:a194:0:b0:3e1:b027:833b with SMTP id k142-20020a1fa194000000b003e1b027833bmr3496689vke.16.1674063039775; Wed, 18 Jan 2023 09:30:39 -0800 (PST) Received: from mail-qk1-f180.google.com (mail-qk1-f180.google.com. [209.85.222.180]) by smtp.gmail.com with ESMTPSA id u15-20020a05620a454f00b006ce76811a07sm23116053qkp.75.2023.01.18.09.30.39 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 18 Jan 2023 09:30:39 -0800 (PST) Received: by mail-qk1-f180.google.com with SMTP id w21so5055896qkf.8 for ; Wed, 18 Jan 2023 09:30:39 -0800 (PST) X-Received: by 2002:ae9:efd8:0:b0:706:e593:2598 with SMTP id d207-20020ae9efd8000000b00706e5932598mr50927qkg.216.1674063038833; Wed, 18 Jan 2023 09:30:38 -0800 (PST) MIME-Version: 1.0 References: <20230118080011.2258375-1-npiggin@gmail.com> <20230118080011.2258375-5-npiggin@gmail.com> In-Reply-To: <20230118080011.2258375-5-npiggin@gmail.com> From: Linus Torvalds Date: Wed, 18 Jan 2023 09:30:22 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v6 4/5] powerpc/64s: enable MMU_LAZY_TLB_SHOOTDOWN To: Nicholas Piggin , Dave Hansen , Andy Lutomirski , Peter Zijlstra , Catalin Marinas , Will Deacon Cc: Andrew Morton , linux-arch , linux-mm , linuxppc-dev@lists.ozlabs.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 1CF07100024 X-Rspam-User: X-Stat-Signature: aet9emoicishu9uhdit9qhfajr1cao33 X-HE-Tag: 1674063040-129547 X-HE-Meta: U2FsdGVkX1/XZDfN45OvPSHJAWsTDqanWkiAwcgykf7PldzdBZZ4VrkPDstpfg7EklZG/iUzUMh/Z33PpzOyz1mYgQEajxf9WEHMVsHyOo2+xOM0pq8aFQ9c6k7w6pJG6EAfQfHpwvNX/iL7SJS+byBy44RbfAJaUP+qhiNNMEQhtSSJ0BhoRxyt6qAm0Qsb9WxoBMetM2pPeFSnvLux7UZ8P4bRcLXczCYpzl+yDb1RgLX+ciPVdavEExflW0YkLOhIpyfiMWQopsUOg+sePPjvXxZwcUrbLSKsllAgu1hYtApllw5CrantqGHv8Zmh19cTpbF97Ix5z7DAL/p9oaSYCpYb3fsY+vFiw7UWsgZOy5B8W8q+4cQEi0GHy36qFrdhGoaP63rGoNDlfQlPDwV3ACZbc/1CVR+I8DhPhIChy3D6rVK+ugbtsNm9irLHP25yUAjCi3hBp8KBWViL01Rxd5DD8Qq0wNTn9R6TR3Zg9pefEGILRN8zK/q7WKjDijhtVt6pYj1woABuaA8wBRt9C4iAhE95G6s6u6Cbcaw5kLCwnfiXSG0DxhOCcMhHE1hMuAgalsJPEY9U71wC8qZMQe7S1F8s3T63vtVtMilAJeGhuEwBomyxUD7eaa1l9OTKTatIZf5A+xk2QFKCyJtVxZBtfGHPAPreyaIMDz1dyNyxsuf9LUmD1G9Fv6aTuWSkJsUcQkAVq+uB4s3FjqyACOqrFRYxlBKN7P9gwt83g5A4KvOIvu/QYlGQKD961i/92l01nY2jFs9OP+5lsZ0LjFuurhjdJwb4faZPCQwSsiv40W+aa15Xsxx1pO9jE51GQqBqj2x//D2GrKX0HXmjlKB1AsLqqgkDyPDDzSRFmOKoytJ6h3oi+xrxktEOxSEn1HbHxfIp0XIDy7VrAJ10hVL6rRhbRKyfKImELMRpV+jRatSEzBxJTC3Pd0vZKyluKZ+K7HlKaVurWbd dw5ji2bb s+7j3dbNz7AcXjllcI8HKXPjYOLNyOJjynwIMf2c4mQT2zQwPh5DfwQoBV0VnTnx8gbrnTo3n6DJo5qO0vFPY8om9aZM1D1HQ6k+BLiVcz1Tjwt46BpSWeWjXfi0m7Zn2AuN8MOU1GH0RtHaUPyjP7XDCaKbNf53preah4DCsX8/jvSYm3/UOWcS5rjBJP5wiv2ilmTfeVdLr6XWbfFblF3c/mhimwqXdEJmVGhaWhCfP5F2GFXeBgaw3NabffeMg5G33Rw15T1S7wCVxNeW6zn4U07B1CCzQm81ZYc1vnNqJnmZWYJqFr4Ku2UPPil4m9NPfdMR5HjNL80sxW9geUgME2w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: [ Adding a few more x86 and arm64 maintainers - while linux-arch is the right mailing list, I'm not convinced people actually follow it all that closely ] On Wed, Jan 18, 2023 at 12:00 AM Nicholas Piggin wrote: > > On a 16-socket 192-core POWER8 system, a context switching benchmark > with as many software threads as CPUs (so each switch will go in and > out of idle), upstream can achieve a rate of about 1 million context > switches per second, due to contention on the mm refcount. > > 64s meets the prerequisites for CONFIG_MMU_LAZY_TLB_SHOOTDOWN, so enable > the option. This increases the above benchmark to 118 million context > switches per second. Well, the 1M -> 118M change does seem like a good reason for this series. The patches certainly don't look offensive to me, so Ack as far as I'm concerned, but honestly, it's been some time since I've personally been active on the idle and lazy TLB code, so that ack is probably largely worthless. If anything, my main reaction to this all is to wonder whether the config option is a good idea - maybe we could do this unconditionally, and make the source code (and logic) simpler to follow when you don't have to worry about the CONFIG_MMU_LAZY_TLB_REFCOUNT option. I wouldn't be surprised to hear that x86 can have the same issue where the mm_struct refcount is a bigger issue than the possibility of an extra TLB shootdown at the final exit time. But having the config options as a way to switch people over gradually (and perhaps then removing it later) doesn't sound wrong to me either. And I personally find the argument in patch 3/5 fairly convincing: Shootdown IPIs cost could be an issue, but they have not been observed to be a serious problem with this scheme, because short-lived processes tend not to migrate CPUs much, therefore they don't get much chance to leave lazy tlb mm references on remote CPUs. Andy? PeterZ? Catalin? Nick - it might be good to link to the actual benchmark, and let people who have access to big machines perhaps just try it out on non-powerpc platforms... Linus