From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41279C4345F for ; Fri, 19 Apr 2024 19:26:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CBE176B0089; Fri, 19 Apr 2024 15:26:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C45FE6B008A; Fri, 19 Apr 2024 15:26:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AE7296B0092; Fri, 19 Apr 2024 15:26:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 8F6656B0089 for ; Fri, 19 Apr 2024 15:26:18 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id E7CD98028E for ; Fri, 19 Apr 2024 19:26:17 +0000 (UTC) X-FDA: 82027262394.17.8817413 Received: from mail-lf1-f44.google.com (mail-lf1-f44.google.com [209.85.167.44]) by imf24.hostedemail.com (Postfix) with ESMTP id 42D99180015 for ; Fri, 19 Apr 2024 19:26:12 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="eF/akW/C"; spf=pass (imf24.hostedemail.com: domain of yosryahmed@google.com designates 209.85.167.44 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1713554773; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=5jD0EhyCifuBQLkCgtTMY21veWhe8WpFtZANsrLOqmw=; b=RUHNKTBNIWZpAGp0q+pZSCPuirP0StRgRqWeOfZdal5XJ5Jx95kqAGxCubEw3IIht4aCM7 z4bLmq6xNnRkHU7QEjr50EtoUm33YtiS2jH2lxnbGgsxCAnIvkxCf5nr6r9c0GTUgrK5Da lI5Ua/DR90yVYXxGuCM1e/S2vXRaMYA= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="eF/akW/C"; spf=pass (imf24.hostedemail.com: domain of yosryahmed@google.com designates 209.85.167.44 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1713554773; a=rsa-sha256; cv=none; b=QTA+YGis5uuskpCcuBrU5X5+XmR0Cdpm/PUN4BuKElEDeTCGVAfyqdsgp2T6pmL1a4HYDb UxT/OVG1aqz6NCveu1NtMcoehD9cxnLIs7mv2d3zHZu2c+/9KUCumzGeKXXsDpJd5LEmhP ArN23HHPf5SowPb9vs4joANFJSOANbk= Received: by mail-lf1-f44.google.com with SMTP id 2adb3069b0e04-51abd9fcbf6so1912924e87.1 for ; Fri, 19 Apr 2024 12:26:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1713554771; x=1714159571; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=5jD0EhyCifuBQLkCgtTMY21veWhe8WpFtZANsrLOqmw=; b=eF/akW/CMZVJnJggbkW6s5cITYbqRsd0VE9lAMbaLQqdMT6gqiPTjob0mrmX0emDwo wGRCrY49rPRoI4fS/F7zuTp/R7lzZeBHClzXQgpzrF1njtd1LE4FGjaWsfjMOIlsTi51 DSr1UcrZ6OE8LtzopP9Urpjq8oX6OcjOBU4IkQdVU/63zZvKepD4m9+BGxASKHuFneJ5 d90JXsFfI8Usa+sXK7Ewhq5gEchO1oGa5zwzy1uu0TdwIiB9QWfnguQOCxYUYaVD/Vyl k8yseKiSvtFbXvDIdMrStXHMrrQKnkkhaunmTvV7MUfa/UWYFJnXwziIqIfMUPJX/sCW HuIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713554771; x=1714159571; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5jD0EhyCifuBQLkCgtTMY21veWhe8WpFtZANsrLOqmw=; b=kADXEj+rKKL3tWxo+sbloPpsQOEQPEH9cgqkLFbxnSfQ2ek8zC9vpG5rWJ1on3brAw JpC4DeAoVOZSVzBef7u7iqoz7kGB/UMsVhsD9IULwr0OyOkWvO2owbeSU5pd5+IpUcin nzAZuztWsooF7bAo38Tu1UMRnRS1zbBFt/0iiyTq5KexkbqUsK5c4YmppCmDmKofaxl/ uJb8tAEbVJ8AJuQrj/oWeQm1ohkSN/d5cBAoTkB3Kuodl58MAhr1Dfk5stehcmi1osC9 RjnLLPMOYiUYMSLE7entgD4oGxUHgcyMDiL0tMg449pVtCwYvAEtETxad+JAp9e7ak+z 5aig== X-Forwarded-Encrypted: i=1; AJvYcCXHE9yLBl/PSnKWAYRThC6c3HnTuKhdNsY+0qaeZ8xSC8WUndoyNtKXj6GqCaFf2VcweMuohCR56lyJdKxTUG79QGg= X-Gm-Message-State: AOJu0Yxx+7J5SB+HizAkJBIc52r4DgCy3/qoLYRhGQ5N2fPA6lqd7jUb E+nFq5COlG0K5VFGA3sF13FXSc5fupBJde2OMmrzCU5nUf04qBj7kH4zXWcs2Z5zonmjT4Wbd6D +2S+f7xs+AN/88mkA8ImsgHXWL8rRwzNnVFr7 X-Google-Smtp-Source: AGHT+IG7VL6KZ/P5UdQBTbJCd+VjQRvtNpyhd38tP7Or3vQWzXSTZExyuSTBKhBZU9qfTY773qr5ITgLc3HCzj5YcEo= X-Received: by 2002:a05:6512:3f18:b0:51a:dee1:d41e with SMTP id y24-20020a0565123f1800b0051adee1d41emr294169lfa.61.1713554771092; Fri, 19 Apr 2024 12:26:11 -0700 (PDT) MIME-Version: 1.0 References: <171328983017.3930751.9484082608778623495.stgit@firesoul> <171328990014.3930751.10674097155895405137.stgit@firesoul> <72e4a55e-a246-4e28-9d2e-d4f1ef5637c2@kernel.org> <33295077-e969-427a-badb-3e29698f5cfb@kernel.org> In-Reply-To: <33295077-e969-427a-badb-3e29698f5cfb@kernel.org> From: Yosry Ahmed Date: Fri, 19 Apr 2024 12:25:32 -0700 Message-ID: Subject: Re: [PATCH v1 3/3] cgroup/rstat: introduce ratelimited rstat flushing To: Jesper Dangaard Brouer Cc: tj@kernel.org, hannes@cmpxchg.org, lizefan.x@bytedance.com, cgroups@vger.kernel.org, longman@redhat.com, netdev@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, shakeel.butt@linux.dev, kernel-team@cloudflare.com, Arnaldo Carvalho de Melo , Sebastian Andrzej Siewior , mhocko@kernel.org, Wei Xu Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Stat-Signature: gzh6y8uojhynbehahq9ekicyqs4gtdtg X-Rspamd-Queue-Id: 42D99180015 X-Rspamd-Server: rspam10 X-Rspam-User: X-HE-Tag: 1713554772-302006 X-HE-Meta: U2FsdGVkX1+tmybRSJ4XtlIJppt8zuuf6dYQYNiLJwOfz2KtwN7UlxaB4BgNYEt86H0jpdYKGvHT8Tc9Vxa8NJoLIiGcGZd3TfDdVAXMUdz4cRlilmBAN2TIuxr3gtPhDZCUTUn5AOZe8ntqTnypxo9/xhD3kF05IMRtHoN3d/RLiqfr+2bDncWongJp+Z5zioikH0iiW5PooEyvO8ypfjPKvedABJZNGugd5Mw7qHxML6FSenGjToa87N1AZBi+F0qYJcDRMOm9mUfzJ1MiWmtQAtKLvas5EKyrT0LeE5Vh3COuSqJQrffh2cbsbXt3owDAfYQ73iW3wN9FYRk1+VYVl8yMKvYKPqIkzA5M7w8prnBZP9eJR1igRAAVLCVEK7L4RZhzf0vVJohgX0PPgGnr655qOF1UvA7g+fHsPxph5ks474MkLHW5cOk5z9ob3I5/E/5fpDPljoXYIBSSilGvM5XIac8wp50/6ZsBAU8EgCLYP7s12XnxkIlsfkgCGbHxXsivr97uXaL0ON8TDcn10wAdoEZVdb4pomkbAbqMu8ye/Ia+lYoz8tVaghUMN7A0SrZdYG+KEujgdLcFdo5gDUkCTu4n13A6cyNv5BZAYOTS0R45aAC8MlC0ron1jaVwTYyUbHzkVxSnfcOx9kKcESx0OeuUhsZobOb5QJpHkUdW1ww9LouoHVB1rVezN8ySMyBd7Zjv0YbGQtCku3/Gqw2rMffde00V3mq4wNtOYWHstmxVQXqcvHDHOLSUZkf4xoUWkWEdXcgsQvo2GzVHUIx8cMLS9OGRRm/dPzJnvP5t7wztUTKEWwUkGkzeADhImRS4uOFtIBJ+XyZnHjnFGZ7kF6av/q0NHyJsrUNOopAYeNOO8liZUUagBCGVcep/LEsgEB/OxPbNOty79qNq+5wfwwsrTeyKauo+nHKvGSNwtwNbhzsfTSW8tNW8a3GF/ptXXENTPPPfol1 VYngo0Na Pd7OJXgzKAbMAkCVeURcqiPZmb2XXwizXUWUSPi3oNiYUkDTUTEakrT6+ISsc+iE1EbPAYmT0SkguXFVrPS74sC8FvHw7Udu+M5t451dRE6kyZtAlRTNqOg/U0ngPRBeWXfxgzdNkr9SZap+jLr7azr/9M3wo1F7/7wCL/1navIgGzvQcoinsL0pB0ZkXfNWedq3H1DzsPqZ0m+snRa8XTOtmcyQwe+zolqd605dj5DXXeL4GVyqh+zUvgpR3m64LiyjnnUDv7CiH5qIuMC+kGRhYwKP/8N6zltwP X-Bogosity: Ham, tests=bogofilter, spamicity=0.003666, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Apr 19, 2024 at 3:17=E2=80=AFAM Jesper Dangaard Brouer wrote: > > > On 18/04/2024 23.00, Yosry Ahmed wrote: > > On Thu, Apr 18, 2024 at 4:00=E2=80=AFAM Jesper Dangaard Brouer wrote: > >> On 18/04/2024 04.21, Yosry Ahmed wrote: > >>> On Tue, Apr 16, 2024 at 10:51=E2=80=AFAM Jesper Dangaard Brouer wrote: > >>>> This patch aims to reduce userspace-triggered pressure on the global > >>>> cgroup_rstat_lock by introducing a mechanism to limit how often read= ing > >>>> stat files causes cgroup rstat flushing. > >>>> > [...] > > > Taking a step back, I think this series is trying to address two > > issues in one go: interrupt handling latency and lock contention. > > Yes, patch 2 and 3 are essentially independent and address two different > aspects. > > > While both are related because reducing flushing reduces irq > > disablement, I think it would be better if we can fix that issue > > separately with a more fundamental solution (e.g. using a mutex or > > dropping the lock at each CPU boundary). > > > > After that, we can more clearly evaluate the lock contention problem > > with data purely about flushing latency, without taking into > > consideration the irq handling problem. > > > > Does this make sense to you? > > Yes, make sense. > > So, you are suggesting we start with the mutex change? (patch 2) > (which still needs some adjustments/tuning) Yes. Let's focus on (what I assume to be) the more important problem, IRQ serving latency. Once this is fixed, let's consider the tradeoffs for improving lock contention separately. Thanks! > > This make sense to me, as there are likely many solutions to reducing > the pressure on the lock. > > With the tracepoint patch in-place, I/we can measure the pressure on the > lock, and I plan to do this across our CF fleet. Then we can slowly > work on improving lock contention and evaluate this on our fleets. > > --Jesper > p.s. > Setting expectations: > - Going on vacation today, so will resume work after 29th April.