From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A80BECAAD3 for ; Wed, 14 Sep 2022 14:23:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A8A1E8D0003; Wed, 14 Sep 2022 10:23:19 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A39B68D0001; Wed, 14 Sep 2022 10:23:19 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 901F28D0003; Wed, 14 Sep 2022 10:23:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 801128D0001 for ; Wed, 14 Sep 2022 10:23:19 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 5A28580363 for ; Wed, 14 Sep 2022 14:23:19 +0000 (UTC) X-FDA: 79910908518.02.FE92672 Received: from mail-pf1-f169.google.com (mail-pf1-f169.google.com [209.85.210.169]) by imf25.hostedemail.com (Postfix) with ESMTP id F1021A00C7 for ; Wed, 14 Sep 2022 14:23:18 +0000 (UTC) Received: by mail-pf1-f169.google.com with SMTP id c198so15065980pfc.13 for ; Wed, 14 Sep 2022 07:23:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastly.com; s=google; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:from:to:cc:subject:date; bh=7X3jI/FDmkrBs7DhtU4sYpQT4M91OAzd4bF9ysc8YQA=; b=kuTFYHmJmpQFxv2JP6mXl4t7RMOKICwKvTcwLslqUJQ/xIlbxwQYeNKTsYUk2K2fIw 16zOvbCrZ98E0IA/KYx5UktK/z6ClyAYPoHMSE7uJ30cU5o9IDFy857orY0dbk+oVpLL f/C15UDXNwotak1w8D3iBgvVTrcdx0jaXtqGY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:from:date:x-gm-message-state:from:to:cc :subject:date; bh=7X3jI/FDmkrBs7DhtU4sYpQT4M91OAzd4bF9ysc8YQA=; b=Mkr+ne6aPtSHqEN08YFXGw4eYfeFA67xxlWfxVdBb5e0zlcOPqazIK/H9fmlp37nBV j1KyJdLYlrrQJPcnijXKU4dtwy82VYcM1NBtVghtiQDrDiL9CTUwz3Pbq1vEFiEOEfnp AYosGmMv/U2uZXu2XJ5xCWYpdpGdf5QgJL6tQx+8HxlSL0AAOvWanpR+nKkpgH7bJ5VZ 0n7SgxYu9meMMdBaPVMHtTq7gPgfO0ih0UTNasjhWj3pYNSL1MQaPLD9lAKAhTTF8s+l vv9Amr7qmuFwBdLqesltnJFGOwynmZsQHhtnt8M/yg7ExRdpKPOTFy5twJs0OdQUsb9/ /kZA== X-Gm-Message-State: ACgBeo1l0rO466CJBSgdOOZtZ4VQbn3wnbcB9VBVVRJZ/y0+95ShTUPZ JvLnjoB6tHDRN/nFUM8NAmP3vA== X-Google-Smtp-Source: AA6agR5e0Wb81DIfJkmSV8Voh4NfgntHAxCLTOZXQzJ2lzogbO8SlJVIATVlUY6WISHAYjiZnqvEAw== X-Received: by 2002:a63:ed18:0:b0:439:4176:3ea6 with SMTP id d24-20020a63ed18000000b0043941763ea6mr7224872pgi.363.1663165397855; Wed, 14 Sep 2022 07:23:17 -0700 (PDT) Received: from fastly.com (c-73-223-190-181.hsd1.ca.comcast.net. [73.223.190.181]) by smtp.gmail.com with ESMTPSA id q6-20020aa78426000000b0053b9e5d365bsm5350864pfn.216.2022.09.14.07.23.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 14 Sep 2022 07:23:17 -0700 (PDT) Date: Wed, 14 Sep 2022 07:23:14 -0700 From: Joe Damato To: Peter Zijlstra Cc: Dave Hansen , x86@kernel.org, linux-mm@kvack.org, Dave Hansen , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , "H. Peter Anvin" , Juri Lelli , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Daniel Bristot de Oliveira , Valentin Schneider , linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org Subject: Re: [RFC 1/1] mm: Add per-task struct tlb counters Message-ID: <20220914142313.GB4422@fastly.com> References: <1663120270-2673-1-git-send-email-jdamato@fastly.com> <1663120270-2673-2-git-send-email-jdamato@fastly.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1663165399; a=rsa-sha256; cv=none; b=1xFTEQxR/VtEJsntz9Y8tf0xMcRNAGb94IVcX7gdO8K0VFonnZvFRJ96X1LexBx+1wEfF5 UiXLPU+vcYRfRlyaY/PHJD/YU4UTg+Yr/j2493AuF8GKVt55/WY4Z8mfCQ96iD7fsDq5Pw 7vef2+w/scktjMelwz9rmyEpZAhFADE= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=fastly.com header.s=google header.b=kuTFYHmJ; spf=pass (imf25.hostedemail.com: domain of jdamato@fastly.com designates 209.85.210.169 as permitted sender) smtp.mailfrom=jdamato@fastly.com; dmarc=pass (policy=reject) header.from=fastly.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1663165399; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7X3jI/FDmkrBs7DhtU4sYpQT4M91OAzd4bF9ysc8YQA=; b=0U1pDAZ/0EIKnU9j6TyPxbN8WHrc1ZxV2cjKC+hosvL5O5Ic8sk2rKAwkfrb4U1BHRPOp2 mSqvEGE+GB/C/3n6TcFYkPB906K3vrJVVpI4Uheoelf0AWg1zmqThFOIkOJYJcToHXS1Ov O2mexSzsEdhNDii5CDYpYJaDMLX2KLI= X-Rspam-User: Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=fastly.com header.s=google header.b=kuTFYHmJ; spf=pass (imf25.hostedemail.com: domain of jdamato@fastly.com designates 209.85.210.169 as permitted sender) smtp.mailfrom=jdamato@fastly.com; dmarc=pass (policy=reject) header.from=fastly.com X-Rspamd-Server: rspam02 X-Stat-Signature: pykeiihs45r1xpki765hsuf7qcys81fw X-Rspamd-Queue-Id: F1021A00C7 X-HE-Tag: 1663165398-138735 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Sep 14, 2022 at 01:58:27PM +0200, Peter Zijlstra wrote: > On Wed, Sep 14, 2022 at 12:40:55AM -0700, Dave Hansen wrote: > > Why didn't the tracepoints work for you? > > This; perf should be able to get you per-task slices of those events. Thanks for taking a look; I replied to Dave with a longer form response, but IMHO, tracepoints are helpful in specific circumstances. On a heavily loaded system with O(10,000) or O(100,000) tasks, tracepoints can be difficult to use... especially if the TLB shootdown events are anomalous events that happen in large bursts at unknown intervals and are difficult to reproduce. IMHO, I think that being able to periodically scrape /proc to see that a particular process has a large TLB shootdown storm can then instruct you as to when to apply perf (and to which specific tasks) in order to debug the issue.