From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B26BC3DA6F for ; Fri, 25 Aug 2023 18:44:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 804352800C2; Fri, 25 Aug 2023 14:44:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 78DA42800C0; Fri, 25 Aug 2023 14:44:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5B9702800C2; Fri, 25 Aug 2023 14:44:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 457902800C0 for ; Fri, 25 Aug 2023 14:44:40 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id D6EEF1C9B3A for ; Fri, 25 Aug 2023 18:44:38 +0000 (UTC) X-FDA: 81163503036.05.ED72F78 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by imf19.hostedemail.com (Postfix) with ESMTP id C0D871A0012 for ; Fri, 25 Aug 2023 18:44:35 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=TbmrwaVB; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf19.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692989077; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1xfjXtaNhAs9Izi3V9PNqgVBs7ebPlEYdTa0N1qxD5A=; b=HUDJbBtfe9WDSnwLpzsAB9rDBtqsBRQfsbE1/BIrtTfGvlEz3y8mlRhpfFu1AV/AyA4nvN KurpSNU/w5lUZo2ECLO63dyOOOquc+ZNgvSI2gVbqngOmKyAFNNgmoaEIEvpArpnfojhiv Ty4aKSdonnZuyQiWxkuG3Y7sB0RhA84= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=suse.com header.s=susede1 header.b=TbmrwaVB; dmarc=pass (policy=quarantine) header.from=suse.com; spf=pass (imf19.hostedemail.com: domain of mhocko@suse.com designates 195.135.220.28 as permitted sender) smtp.mailfrom=mhocko@suse.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692989077; a=rsa-sha256; cv=none; b=ABkQfi2CRwwfCoste6z2+3NMFvN1HMN7RHd2vPyYDKzv9UdywpHYCp9JXrnAhiueerUpF0 f5aLntSewK4N1FFteZM9S47zEDVjxudV61KnTTjKkSTNftxdPx1n4V+YzxOV0Jmx5RLx0t QAxbcYLzc+bw1u0faTzAZ31b7UUN7es= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id AEC8E2212A; Fri, 25 Aug 2023 18:44:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1692989075; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1xfjXtaNhAs9Izi3V9PNqgVBs7ebPlEYdTa0N1qxD5A=; b=TbmrwaVBm/U0VHuwzCOgDkcqjniSNFpKEEWwmXZrYN7A1bQ6fYTMCItlOCo2TyeRl5M05C V4xRAdw449tIc2Gvk+yGlQMlUqZeHRoKFhnTBd2sr4OHz94bIpF6j9SeAFdrK/omaWJnB3 iLuCISHtWZvuk3bOJDuRo6pGC8KaBoU= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 8E8E6138F9; Fri, 25 Aug 2023 18:44:35 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id gSzGH5P26GSXBwAAMHmgww (envelope-from ); Fri, 25 Aug 2023 18:44:35 +0000 Date: Fri, 25 Aug 2023 20:44:34 +0200 From: Michal Hocko To: Yosry Ahmed Cc: Andrew Morton , Johannes Weiner , Roman Gushchin , Shakeel Butt , Muchun Song , Ivan Babrou , Tejun Heo , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 3/3] mm: memcg: use non-unified stats flushing for userspace reads Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Rspamd-Queue-Id: C0D871A0012 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: dbasc3pcdocnqeet5f35i4jgfg93zfh9 X-HE-Tag: 1692989075-914913 X-HE-Meta: U2FsdGVkX1/VF7fuVMS2TLAHM84xBH5skn4iQFVUapdyVa7XtnbBcZ/5llGdhUHyLSbIVkRKDeMHQTJEeAZpUDOQugCoyrIwnflxxdDvJK7yw/m2zi2TCjBfrxl7MffXl8h3LIa+SfrQR0anJnmsiTHGtncLMTh1JpZAP3UdXJN31DaZDTFBsyq4uWyr83b2sxIXmoMtAuX2jDs4H9Sr+D2dSygEdN3dsgls1ZV540IwM2gMW8s3wpR2Qgvuj9X7xx1CbzP3aatiaotArmZeiUvRQGq+q852i8WLTx+KVh4HQ7hBSITKA2qY8xI38PU/kh5F4O3EFSYSc1TvjhhXUYawccJE8su96J2JF6HKjIO6YBm/aOnMpGhZcq5LxqA6pjc57PlWVLP+bspWcK4DI6JU4WwmdvUqnGVfMIXiVtaXO0qD9Hy1cXKs3CLaPbrVOT6eOLQUyiMRc3zAaaePaCLCbBe1RvFL2tRiusSSl5gP2Uro0SXmpUT0ri/db/8DrSHZjhmlVopViuqMDyaTSZgGVq3ZORmzwP6p1Er/P2YerzIuF/0CNJdAfs1DHjHiGfRR6ycYXBn8GeVVeBgjjbqtIzkmpvy5aaptqywRQ+1JiqWjy9s/aswj1w1+LCnMIhg4VDqsA34yrwDy43HuMqVI+yUfScRKSPGbQIY+xz20Y97eTDiu6z4A5wK2ZM6pwrQZPVekkPtXrlNc7ykKqbVRenES2hf+bFe/+Vdl+RrTVEzwP4JOLPbg6ieNxtSOunkV6nBkRmC4YK+Scr0aIkm3mBtNPU9qudOtzsZBrfxcgvWj+HyRyn2CZR6PEP3mIYPZXmcjEWhqZEzj0Zf2eC6G9sn9U/qQtpFoSFyx7jpSMvVBCt50ZN0uq1/NjpU1L/hwXYS/17oIXXi4asZXqZEXQHI31OZ19H+fNRk7JxZXdpRjBMj9xoQU6Dfnw4s63QAULMu4FWtI2YV4qPd BcD3A339 7+Z53n/DE3NfU5CKvqftynPJLbe6S+TM5J4tVIbAAF8kTGOxsE7kS8+a6jUfdYF/CrszSAwwEjuTu1aQVllEc3p6hQeuodG/mTdChQZ4F+W8pBtXC26rLw1YQUcP7I8I+PHoEYWqBmRnw4160LFM+TqLWZzfcUNLOitTvph8iPN6rZBcmbWkJHStv67QJr9WxucAxA96fmVRZcZ8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri 25-08-23 20:43:02, Michal Hocko wrote: > On Fri 25-08-23 11:21:16, Yosry Ahmed wrote: > > On Fri, Aug 25, 2023 at 11:17 AM Michal Hocko wrote: > > > > > > On Fri 25-08-23 08:14:54, Yosry Ahmed wrote: > > > > On Fri, Aug 25, 2023 at 12:05 AM Michal Hocko wrote: > > > [...] > > > > > I might be wrong but the whole discussion so far suggests that the > > > > > global rstat lock should be reconsidered. From my personal experience > > > > > global locks easily triggerable from the userspace are just a receip for > > > > > problems. Stats reading shouldn't be interfering with the system runtime > > > > > as much as possible and they should be deterministic wrt runtime as > > > > > well. > > > > > > > > The problem is that the global lock also serializes the global > > > > counters that we flush to. I will talk from the memcg flushing > > > > perspective as that's what I am familiar with. I am not sure how much > > > > this is transferable to other flushers. > > > > > > > > On the memcg side (see mem_cgroup_css_rstat_flush()), the global lock > > > > synchronizes access to multiple counters, for this discussion what's > > > > most important are: > > > > - The global stat counters of the memcg being flushed (e.g. > > > > memcg->vmstats->state). > > > > - The pending stat counters of the parent being flushed (e.g. > > > > parent->vmstats->state_pending). > > > > > > I haven't digested the rest of the email yet (Friday brain, sorry) but I > > > do not think you are adressing this particular part so let me ask before > > > I dive more into the following. I really do not follow the serialization > > > requirement here because the lock doesn't really serialize the flushing, > > > does it? At least not in a sense of a single caller to do the flushing > > > atomicaly from other flushers. It is possible that the current flusher > > > simply drops the lock midway and another one retakes the lock and > > > performs the operation again. So what additional flushing > > > synchronization does it provide and why cannot parallel flushers simply > > > compete over pcp spinlocks? > > > > > > So what am I missing? > > > > Those counters are non-atomic. The lock makes sure we don't have two > > concurrent flushers updating the same counter locklessly and > > non-atomically, which would be possible if we flush the same cgroup on > > two different cpus in parallel. > > pcp lock (cpu_lock) guarantees the very same, doesn't it? Nope, it doesn't. I really need to have a deeper look. -- Michal Hocko SUSE Labs