From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1BB1C6FD18 for ; Tue, 28 Mar 2023 14:15:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7A3EC6B0078; Tue, 28 Mar 2023 10:15:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 753AD6B007B; Tue, 28 Mar 2023 10:15:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 61BC76B007D; Tue, 28 Mar 2023 10:15:29 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 5070E6B0078 for ; Tue, 28 Mar 2023 10:15:29 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 25AA71202C8 for ; Tue, 28 Mar 2023 14:15:29 +0000 (UTC) X-FDA: 80618504778.23.641A265 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) by imf06.hostedemail.com (Postfix) with ESMTP id 281EF180005 for ; Tue, 28 Mar 2023 14:15:26 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=qUM+3vUW; spf=pass (imf06.hostedemail.com: domain of 3ffYiZAgKCB4MB4E88F5AIIAF8.6IGFCHOR-GGEP46E.ILA@flex--shakeelb.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3ffYiZAgKCB4MB4E88F5AIIAF8.6IGFCHOR-GGEP46E.ILA@flex--shakeelb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680012927; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=qjzC1AXnpW8l0xPrPEZPgxXh3c0otg2X0grqn38oTAU=; b=M2VzbsqIpe3rHUu2uAm7zKsWTaTJo/f5NV+ONkw2dYj7qn+hag8j1dW9Cd0lQtlHW0UXJR U2iJjLI8+ua4hcDy3rbYCnR+Xx+HIMJIr70hVHcmZehiJxTItkPUooNxQ8vepZGXGSVgyA Kc1Hjw9/TDi3gYZOHMMNiptNiSmFhwo= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=qUM+3vUW; spf=pass (imf06.hostedemail.com: domain of 3ffYiZAgKCB4MB4E88F5AIIAF8.6IGFCHOR-GGEP46E.ILA@flex--shakeelb.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3ffYiZAgKCB4MB4E88F5AIIAF8.6IGFCHOR-GGEP46E.ILA@flex--shakeelb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680012927; a=rsa-sha256; cv=none; b=mjRsDIO605AUR3z9Z/zlPMv3ohJH304oTnBiJGwQNs3NJEuTpbMC5how4UqVkb61ziWjw+ 0Td7lRyAu87KHdz9AqlKRQRtJo9P7tEMnN7cmOp2C3V1z6ff298le3zlw/89SKsTKYZ5US lV43PUHdm1dfzWEPFSwPvTx5jm7kZm0= Received: by mail-pj1-f73.google.com with SMTP id nm11-20020a17090b19cb00b0023d0c90d851so3381693pjb.1 for ; Tue, 28 Mar 2023 07:15:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1680012926; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=qjzC1AXnpW8l0xPrPEZPgxXh3c0otg2X0grqn38oTAU=; b=qUM+3vUWQkrigkhqTDTvvelfECv9Ygwv0i+n29LRD0qUqJiaUbOUUCegtJgko+lJPa ZtQCYU5u9ikCNn6Z+kBUMHeJWkGvrE++jHWXmrK5VfZWPiG5vLLvG9eeo027qTgCK6f2 6/RybNsWdVGwR5lq2x/FtSQnv+9idy/UkN3mr3DF15XueJDGP2qfXJwqTDsRvgrrVcSt jqHb20y3Tl9IJ5HgBcSl0lmWuVRaph6Jpu+5W4s8A1Ox35vsZRtzpBYZj6TDGdAAh1wg r0QWwu63iXb0Dvr/+hungxMKZJvyHyCmTeiodFTJiwIn3hxhjAN490SNbhljKM3Jv1E/ SseA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680012926; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=qjzC1AXnpW8l0xPrPEZPgxXh3c0otg2X0grqn38oTAU=; b=ArltLlbyl52xbacvQt/IKcJoX82ezihlM7+oeSM5QuChYzWjjRfNq0iAXH8QPObdsE hrHvLXmd0aMcanKzikE8jbHYb1FzTn+RQ5taBiv40tI/tJ8HAV16V6sOyMTYZBO05fwo ISAtHD0G4tSBqCudkroMY1kDVqUSzOUkUT6Im0lKLOxVpl5dZBaoRNuIkf6hgpnmGgyh qrGIhn+8Wo2Z9K1KT1gfT2hXx1ubByA4LIvbPTXDT4refridwbNl0SjdngYMCGGczz7m EJJHlgliaZezDAr87OOgMnyyKkgpinl9D8/anUFUk4otTwSQmdCv+6d0owcky5LIvDfg WGbA== X-Gm-Message-State: AAQBX9fDLrSgFqSgzvNoGu5gVcgYCkr661cy0xs3B+/mWAOBgR65hKoh KTXxNfU7oZJoG/eDBaFVODt8rENXTwEpqA== X-Google-Smtp-Source: AKy350YS7K4/XlK9G5uXTQxuk1NaV9HRM+JxBayP8kn09bcoJ8C7tEj08nHpJbsB9L4uvhkNT4kYeDqSgTfetg== X-Received: from shakeelb.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:262e]) (user=shakeelb job=sendgmr) by 2002:a63:5a43:0:b0:50a:c176:385b with SMTP id k3-20020a635a43000000b0050ac176385bmr4084662pgm.0.1680012925672; Tue, 28 Mar 2023 07:15:25 -0700 (PDT) Date: Tue, 28 Mar 2023 14:15:23 +0000 In-Reply-To: <20230328061638.203420-6-yosryahmed@google.com> Mime-Version: 1.0 References: <20230328061638.203420-1-yosryahmed@google.com> <20230328061638.203420-6-yosryahmed@google.com> Message-ID: <20230328141523.txyhl7wt7wtvssea@google.com> Subject: Re: [PATCH v1 5/9] memcg: replace stats_flush_lock with an atomic From: Shakeel Butt To: Yosry Ahmed Cc: Tejun Heo , Josef Bacik , Jens Axboe , Zefan Li , Johannes Weiner , Michal Hocko , Roman Gushchin , Muchun Song , Andrew Morton , "Michal =?utf-8?Q?Koutn=C3=BD?=" , Vasily Averin , cgroups@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, bpf@vger.kernel.org Content-Type: text/plain; charset="us-ascii" X-Stat-Signature: nwnn9rjoqnoywq3skmocs8j4pjx7ou5o X-Rspam-User: X-Rspamd-Queue-Id: 281EF180005 X-Rspamd-Server: rspam06 X-HE-Tag: 1680012926-505760 X-HE-Meta: U2FsdGVkX1/9b6Nc8D4fPBmp+jz/+cVeGs3Um0ASccA4SdySL+SLabGyc0KeEn4eFpUhw4Ohy0kjNpTl/n9kE7HB2SKxf6lvJ/g3IAZjXFdOUREr+u1PU/n0QwJy9nhGnwKiybY4yopzIlUyitmcPb/ywwAF5N0o01MNow6WgxMGHVwLyFiVEzbDwDRjW3uic4wyjMeeMJn1Lls8Z0arBYRlYdLg+vjKnV+kZQdkSrjDBzShaADRqTesUbEhMlvnmh1haZnFas3ipqYgLZrS7HziI8lS6t45AoBOONiPoTp3AptV+CY9m8JEyGAfVWf1RfdRgJ1Xr1YFyv+/2oenM0801xTuMpi40k/DkJOhbk133uP7QGcqhwYebxZZ/hNdW+/ZLIZ4BSA+UzmqCRwC8x3dtMdu+NesPqEdVMkF/+6N9iBapHjKFRgx9mtTHqknTVVlPLOR3ZXQrtF4f4IwwuS58vfmoyCcQIe+bWUVMdoTMHNvy6Lwul1CeRSkSN9fpEK46A5eUVQJ+2WR/XqRv9QQgOK9hjuQNsIUKjJcsLvf+r0N2uTgzePeGuR46uhyLqIO57PJLCk4X+LpeH/uilzJqgxSjwKLcO2XTTJ4TpdIWSw11HQWL8Xvl6lU+ou7vYbgxJvmlaSsjIFY5MsZ+IqJKSRmvYUKqAFUtCloSVTfF0yc3CEkqEiNvDJu4ussjS7/dTf8HvuFdHw/36h+vnq3ZNuutYWkNHgk3YvlK7AIaHGeMqUZYsKCTLbxPmaWBkH+UWlWihvOcYrivR66o1pGWx6Vb8n2fc3qW0pJJYC6p1sKi4xaGN5zKHo6XPwkrjAxDakOb2+6RZPLfFlb2NS3sAzZKSkh16Q/UO+ppgGWCIiqkDkpzm8T5RJVhZ3JxZbH3NjMq9psv1sLZnTUj7X0r3c5B2Q0kvPrcXQ7crpU1VRvdxpQ2L0PXWl3fwVRHWVRyf95BMd8zdA3pNr FGEJQml1 S7ZLJ/gL5olZ+5NtMMWe5czt3Wa16HzMIMEP7GNgykIRTft5IRCAYi73Ak372gLBdTJ+zx68AFHWQnyRfsZsd/RzU9+GiajD6N7FtgOgRWlgU9DhjzLAs+5s3CRPNpf8UfC0i+tsD2HUGghDxevLMbQnBF7JQcWuiTRKG2qJ0UTxfRnYfWDJNmgktBOpbFrDsFvJtw9aahE0pfQFm6ZtgIVn31bdZyAOT1pRgIy0AOmQs19V0mYnDY+SVXeJWbghjxQ32aMs/r9Fz5bsE3CwbJ5W/nHKSvN87DBPIabpaOZLlecyDBPkcFdx0zsFStJIiUhog3/E7CWuI/yYM3Vg8KKkEKVOn0JxUa2ijKIK2mjTu+YO4CzK/YxgEzUzFV7Pdew0tYxVSeEN+cliz27SgIuwv6vhLfmPdex2Ht4E+oSVpx7Ej05OE0TPGXKNEFrR3QFy9CdPIszg7AT88Bkplc9Geu12LGUqgG1SRL0krbWF3O3+KfLThPXRqNNL7BE+leRhHkA7ZNc45pL5ogYpB6vXZrQ/WkBprWcNwzd2iYbVSLx9gGvYc4dSw7G55gT46N2wgCDq8Zu6zem3oGqpHEtwOSChJfYT+bIud6ihdqSiTJZedQXJhjfjLWrddkAym9hkRtG+mUYaZibgIGC7KskfQYMfZThRXCxX9 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Mar 28, 2023 at 06:16:34AM +0000, Yosry Ahmed wrote: [...] > @@ -585,8 +585,8 @@ mem_cgroup_largest_soft_limit_node(struct mem_cgroup_tree_per_node *mctz) > */ > static void flush_memcg_stats_dwork(struct work_struct *w); > static DECLARE_DEFERRABLE_WORK(stats_flush_dwork, flush_memcg_stats_dwork); > -static DEFINE_SPINLOCK(stats_flush_lock); > static DEFINE_PER_CPU(unsigned int, stats_updates); > +static atomic_t stats_flush_ongoing = ATOMIC_INIT(0); > static atomic_t stats_flush_threshold = ATOMIC_INIT(0); > static u64 flush_next_time; > > @@ -636,15 +636,18 @@ static inline void memcg_rstat_updated(struct mem_cgroup *memcg, int val) > > static void __mem_cgroup_flush_stats(void) > { > - unsigned long flag; > - > - if (!spin_trylock_irqsave(&stats_flush_lock, flag)) > + /* > + * We always flush the entire tree, so concurrent flushers can just > + * skip. This avoids a thundering herd problem on the rstat global lock > + * from memcg flushers (e.g. reclaim, refault, etc). > + */ > + if (atomic_xchg(&stats_flush_ongoing, 1)) Have you profiled this? I wonder if we should replace the above with if (atomic_read(&stats_flush_ongoing) || atomic_xchg(&stats_flush_ongoing, 1)) to not always dirty the cacheline. This would not be an issue if there is no cacheline sharing but I suspect percpu stats_updates is sharing the cacheline with it and may cause false sharing with the parallel stat updaters (updaters only need to read the base percpu pointer). Other than that the patch looks good.