From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 33F66C3DA59 for ; Sat, 20 Jul 2024 04:53:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7DD896B0082; Sat, 20 Jul 2024 00:53:00 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 78E726B0085; Sat, 20 Jul 2024 00:53:00 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 653CE6B0088; Sat, 20 Jul 2024 00:53:00 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 47BFF6B0082 for ; Sat, 20 Jul 2024 00:53:00 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id D8E141C1CCB for ; Sat, 20 Jul 2024 04:52:59 +0000 (UTC) X-FDA: 82358911278.29.662D8DD Received: from mail-lf1-f49.google.com (mail-lf1-f49.google.com [209.85.167.49]) by imf14.hostedemail.com (Postfix) with ESMTP id 05A73100004 for ; Sat, 20 Jul 2024 04:52:57 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="visxl/ZT"; spf=pass (imf14.hostedemail.com: domain of yosryahmed@google.com designates 209.85.167.49 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1721451157; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fSRulvHWjoBy0HLcI6v4db2avURTI4j4zJY+qVX3PAA=; b=HKggw/uc+PCuPxGxD0h+b1VCivy+Oh9RxIvZivt5ImmHNgGhftEgc4ROHwSJbSLlelhFYa SHcEuxLb1jSzGbPdi5mRFKia/lG8KEiZGXgG/f3k0PQOzJFGFz7snSX9FXG+F3+Usp6Z7Z yGwewl8wRuydlBpyDqIOmdm+3Q7m8D8= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="visxl/ZT"; spf=pass (imf14.hostedemail.com: domain of yosryahmed@google.com designates 209.85.167.49 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1721451157; a=rsa-sha256; cv=none; b=lE+MGQGuxx0foz+4xR1T9UQBGPifLqm7GByLNzv0pYsopB8AGOuVKD8CQV37KphxGJocp+ LQ8xedyJUNc/3GJtPM/XOeKYh+HEsiZUNrVltNsiXIyLzzM6drofoFjr9MNyzTpfyBCNLl PdHTrvh/ZYFwHIPiBG9+1CvCQMcuKGw= Received: by mail-lf1-f49.google.com with SMTP id 2adb3069b0e04-52efabf5d7bso184517e87.1 for ; Fri, 19 Jul 2024 21:52:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1721451176; x=1722055976; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=fSRulvHWjoBy0HLcI6v4db2avURTI4j4zJY+qVX3PAA=; b=visxl/ZTtjwoZEf7F8UXhLwF9Zn+EmHNH57GXtqxjBA+PSaZA1QoXbWG1iCZRa50V5 tP3PL/DuqGeStwlZpvzECkeK6uP7ZCdqf/t+yviHptQ+YfcmPeITUHUFZvnkOs/MjYnn u2rdq6eMmPDmfAURikQql8yq2exrfMzW3fVtj4BEDgzexGo8HYOqujT2pUZPTfBl10zY AbFSJDtcUmztyUb/DUeoJJ1blc62DvHxx2umULkcw9OZvoHj/K/f4yawvVaioQtht9Lw GHwTCZcloJ4Epg5dhJ+hw0XVcQa4/xWiWbSFuxxo22egZen/ZFwvaIjstkXS5p8zmOTp pQtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721451176; x=1722055976; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fSRulvHWjoBy0HLcI6v4db2avURTI4j4zJY+qVX3PAA=; b=iyFbKblXh4ooCj5LrOWbuYx8cFDPC6WHpAAWMkt0SwckXWGsIRTLmP8d6lzQjDzau3 Ok1UXW4uFGDqaKDXtvTUfQg4ENyn3L6afK9i+ANnSSp+Sm1hZVTW/EFUuvccQ8I/6ORk hl4hFUxvfHAQQGZbzM7KlXhnYtwOF5JJFGOqD5Xvzi6bye1Mnc6WBOc8Jc8Q7MTjwYV4 LoJ3fBWfUTbkY1sCTzVi+mXSbPip2Qc0h8qIr+tmjo89qhuXudRZ+E7vD7lxhEhUPKh6 9vrgoEL/n+DI8kpeuXERaGBFkmdw5NqDLy/e5vNyhgThGZaJEkI4BOw/KzdfCmDh4DVu mfdg== X-Forwarded-Encrypted: i=1; AJvYcCXYPDe8GNf7cVF/QuxvDn0/K7wSczZO+Jbg/xJxuaW9BFsGZi8CLG33rneGbRgnsUIxN7PjHd/lgrX9g2VyhdoZ6d8= X-Gm-Message-State: AOJu0YzoYjpU7NmRkAsSJdhDlTmH0d8GYxCxeuRMajRR6O1ugyM0bYjh c1Kt5wMpepoMWP69KYNiHjXRvrDg4oRB39VP4FBnByiYSjLUbbwV6DnKRpJbm6bHYYFpExHzBZ2 C33D3AEVgxarAW0X8dvhVu8BUybn2mk5dWCvI X-Google-Smtp-Source: AGHT+IEuqDE8qtcuvEHCdMjPK+szxTU34wSvAmUMiAVDP0uGH3fdV3MdBRkljmhPdZlIdRmQlRwnnVbtKUVXHMJDaA0= X-Received: by 2002:a05:6512:2351:b0:52c:8920:875 with SMTP id 2adb3069b0e04-52ef8d960ffmr873936e87.20.1721451175516; Fri, 19 Jul 2024 21:52:55 -0700 (PDT) MIME-Version: 1.0 References: <172070450139.2992819.13210624094367257881.stgit@firesoul> <100caebf-c11c-45c9-b864-d8562e2a5ac5@kernel.org> <5ccc693a-2142-489d-b3f1-426758883c1e@kernel.org> In-Reply-To: From: Yosry Ahmed Date: Fri, 19 Jul 2024 21:52:17 -0700 Message-ID: Subject: Re: [PATCH V7 1/2] cgroup/rstat: Avoid thundering herd problem by kswapd across NUMA nodes To: Shakeel Butt Cc: Jesper Dangaard Brouer , tj@kernel.org, cgroups@vger.kernel.org, hannes@cmpxchg.org, lizefan.x@bytedance.com, longman@redhat.com, kernel-team@cloudflare.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 05A73100004 X-Stat-Signature: e9upgzesddpbt49rwk733hzqdr4trnrp X-HE-Tag: 1721451177-543399 X-HE-Meta: U2FsdGVkX1919yYbDABZkrEgZDISXhmlv01lIPu/UewUkPx7iAefRbJo3wC7hanKBd18pWuu2H4u0fogSvJXW0+euBvHAbIUkqgJn3/QcKv0ixY9g4m0cQadW/KyjvIfKfjkSpIz8kVlrRq+XugCDUoEPdxSqeXPS5OZpN5IDWglQ9zaWnqFtdAYJby5dZUvpw4yC0aGhHSN4lOnLFRJG5qF/rHJlweqLkbbqeCjjzDkIR0o96HpAJ9xPEpI8To4kmkxm8NGUwHrJwYe1CT4mdUPjdLlIRagvod0olXrN6z4bEYjcbmdIT+z5sBh9kMSOYkqKtjcKcDBP/SaqPBp6O7adKgNOi+mbqg9BgjyUDVk19SMi3lzql5R13ZnNrqBUP6n87mDGoAa93mcTpRHIQa3VHoexl8PXJo7dSsRBtuJNEkZPz20DsgtO4PgzU9Qz3+6YzSpOhdb5jvPmY2vdaZz+wwW+HO6vzaf3uuWPly6eBuTvZE9QuT0aU8UKfxKWtItHFKTFRCBtHJbSibaRchTSlZYWc9DrLtO2NBH15t0L1SeL45qap+NwUIOmsGveoTidwohM8crHmg/QHZQMxnA3yNGa699adtCh8AwpmqG3TKevtvQoL33i1lj4tSxGdg3XwZO4M2eVQI355lbDnygaz4zdoOqHTlg5HoEPUt8ozoJaWzxGM/OZ0M+oSBDDhX0F/3LgTdquvT16vyGjwqgkMCp1RDmUssca/C72odchRvp4G+BLIrlw4bAPXnJW4EXrsLw9sJQlqoLMsd1pMv/jnaKI9e7HbdOX3jqppO64WyA4dQLA/6zQ9bTvEx5HC/6zxGZUBFjgh+v6H+yzNYLOQcfxyAaG5CjqwRGSt5R+YBD/54HSHpoAON1Xgsvkd6EWU88s9UbIIylq0gGIlR+eK3yZAuoJYpmj/wChAOe3FwyBVCc9o/PX4iVsWt8nKxUXYXDHpqL+ImSJUI jd+ztW4E mQ1XxYeFDEwNLWusgFg/LLjIFFuolDZPbZSEnmwhdJeCPKYkt2yXrYpRL/IyjoQFS1i3k7Se8ROi0tLjmieMBfcgPYejKXT9Y8C4K/xZeD8h9lg1CXb4ovmluYfcSipxy6Xp2TvSVHp0NLpmUefw3h2ZJLOjb9n7kOl4bIA5cFjwMT46n1shznNoMQ7Es0VsWLKYpns3LwzdDDb7QWuTqZvAZhMTIY1aMYpAL X-Bogosity: Ham, tests=bogofilter, spamicity=0.371036, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Fri, Jul 19, 2024 at 3:48=E2=80=AFPM Shakeel Butt wrote: > > On Fri, Jul 19, 2024 at 09:54:41AM GMT, Jesper Dangaard Brouer wrote: > > > > > > On 19/07/2024 02.40, Shakeel Butt wrote: > > > Hi Jesper, > > > > > > On Wed, Jul 17, 2024 at 06:36:28PM GMT, Jesper Dangaard Brouer wrote: > > > > > > > [...] > > > > > > > > > > > > Looking at the production numbers for the time the lock is held for= level 0: > > > > > > > > @locked_time_level[0]: > > > > [4M, 8M) 623 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ = | > > > > [8M, 16M) 860 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@= @@@| > > > > [16M, 32M) 295 |@@@@@@@@@@@@@@@@@ = | > > > > [32M, 64M) 275 |@@@@@@@@@@@@@@@@ = | > > > > > > > > > > Is it possible to get the above histogram for other levels as well? > > > > Data from other levels available in [1]: > > [1] > > https://lore.kernel.org/all/8c123882-a5c5-409a-938b-cb5aec9b9ab5@kernel= .org/ > > > > IMHO the data shows we will get most out of skipping level-0 root-cgrou= p > > flushes. > > > > Thanks a lot of the data. Are all or most of these locked_time_level[0] > from kswapds? This just motivates me to strongly push the ratelimited > flush patch of mine (which would be orthogonal to your patch series). Jesper and I were discussing a better ratelimiting approach, whether it's measuring the time since the last flush, or only skipping if we have a lot of flushes in a specific time frame (using __ratelimit()). I believe this would be better than the current memcg ratelimiting approach, and we can remove the latter. WDYT? > > Shakeel