From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52F88C6FD1D for ; Tue, 4 Apr 2023 21:58:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8567D6B0071; Tue, 4 Apr 2023 17:58:35 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8062A6B0074; Tue, 4 Apr 2023 17:58:35 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6CD896B0075; Tue, 4 Apr 2023 17:58:35 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 5C6106B0071 for ; Tue, 4 Apr 2023 17:58:35 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 2C027C01AA for ; Tue, 4 Apr 2023 21:58:35 +0000 (UTC) X-FDA: 80645073390.25.0B4D9D6 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf02.hostedemail.com (Postfix) with ESMTP id 5ABF28000F for ; Tue, 4 Apr 2023 21:58:33 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b="hXXYS/6K"; spf=pass (imf02.hostedemail.com: domain of akpm@linux-foundation.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680645513; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=B8qPpP/U/20JQ3ubkz3sVov6VNpgWV5pnhJ+Nka1p0c=; b=LCOVs24M1l9RoDus67vAAzqkGKzd5iJ9TqQluGpXU1+OA5e3DlVvPUQHshnHRoXxf219BU oJnxp0TgZF9kiu+NI+VJrJ0ylH3f9dloDCBICfny0+nxo4qLxmK6Xnt4YsFdYexJ32fNoO CUNGSKZ0Kd/uExjiJyo3CtEcgTJrXuE= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=linux-foundation.org header.s=korg header.b="hXXYS/6K"; spf=pass (imf02.hostedemail.com: domain of akpm@linux-foundation.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=akpm@linux-foundation.org; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680645513; a=rsa-sha256; cv=none; b=P4Gtx0p2bNhJvrPicP/8KyGu/pEX7DFcZk2Y4BnysVZjwEI1hqLdGon+nKnBJf6fP+ssJF bnuX30Oayr8t7Ur7/rKjIFZ8Yf1iC9NXW+IMcGxb4M5WorBGN92G8MBxgqzLfZDejU5snP 6Qoxhj1JDDeNcqgMum9E4gvbB7+thcM= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 626D063A4E; Tue, 4 Apr 2023 21:58:32 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1BB45C433D2; Tue, 4 Apr 2023 21:58:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linux-foundation.org; s=korg; t=1680645511; bh=M4j3aiWi53EC30SoD9UhI7oaz89YQljZpPWCkpXYH9I=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=hXXYS/6KJKvu7eFOvXg1x1TzISVO3S8h5QsgNIHHycdJ/kTKeOb+Dn5+dhVIu+Jux Hkos8yiVUSKC8/xGyKJTdTxXZ36EI5lVDi1Aw5OBy38rAMqj++P50jCZMlI22ylt4O PrwUvXef9eQtjqvMqw5mLD1gkY8zWgMXw+eoDk+o= Date: Tue, 4 Apr 2023 14:58:30 -0700 From: Andrew Morton To: Yosry Ahmed Cc: Alexander Viro , "Darrick J. Wong" , Christoph Lameter , David Rientjes , Joonsoo Kim , Vlastimil Babka , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Matthew Wilcox (Oracle)" , Miaohe Lin , David Hildenbrand , Johannes Weiner , Peter Xu , NeilBrown , Shakeel Butt , Michal Hocko , Yu Zhao , Dave Chinner , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-xfs@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH v4 0/3] Ignore non-LRU-based reclaim in memcg reclaim Message-Id: <20230404145830.b34afedb427921de2f0e2426@linux-foundation.org> In-Reply-To: References: <20230404001353.468224-1-yosryahmed@google.com> <20230404143824.a8c57452f04929da225a17d0@linux-foundation.org> X-Mailer: Sylpheed 3.8.0beta1 (GTK+ 2.24.33; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 5ABF28000F X-Stat-Signature: tmoahk1z6z7h5k5wg4o4ztb7xibrygkj X-Rspam-User: X-HE-Tag: 1680645513-59312 X-HE-Meta: U2FsdGVkX19ETJ395trPeDBAFBw1wlBO9rI3rsMfMGjY5YCawRjc4ZAbd/SBJSXDv1+oXPEtq8gCzX34emc2vQ/HZjlgff9IKeP8N3e6TZ8DiTIBYXpJfJl8scxhwyDL3a7OZ+Jgy5LU8mDZylSgCx38y6mfhkar6qdo45Pi2xl84UQF+sC1K4roZdoGqmRh2CIfKvxxAbxFE6MJYAmLM/QGZqVYPyDD1iRAIAcDVCEL+BKkwTFEMyUFe2lVuM9RBjO+soPYpFADIxGl1uR9bolkpmK8x66bvDGzlaw2WYXjJYH5JiGZase3GocON4WvractVyuNRoWqit0lyiW3extHyboj7dDYkL4GavHAskGXHRvHb9PRoBP14klzCkdAyA9mq56W6wJDJdF48MMQB8NPr+XUZavfD1aI/2Cxqw/fZZvmviXfIlbM1gfhvDVN4iyAOmpEjf8xHmiFHcextZ9G0SOjGvLd/oG96FfoJyuI8OT13IVg0XGmEEl7IrOQphvnT1m/g0aTkDQcDay7nxgUAXbwAmRzvbcI7ox1O9XdZ9Lu/1qAy13MfyzIFU1LLH8Q9FT3Lng5+snO2c5vBrkB0lLB56nDguWxS+hpwRdgeILFK8NJHZZGgN4PT+bvFtg4hSgtafy/F39ej5lwnv/bF5WoQVkIaNCSfysNdIn5ArPDQxi30wlgseSByh6G8GTw9aFbWbph/eKtzkZSUD3O09Wh14qE1HbNlovaXDEiqLHb0Rt5gTEwszQBr/rhIl3qY2p/mUPzqw5Oc9BQkPKb2hSiNVw79SqM51H+SWPWpWOWUpB0eqSyStIGJNjYc8asDEG/9A4KmsyGd0PUsbvi40rwZXTj5UxFYgM8f4xSeOc2rXBodMVu8zHxlEbgDEu44jjGCSZf4ui2ki8uk1ldYnJbIKvqzZ89WUAFE0RQL0cDJevVmM7SSMg3EkoPCtuJCQBKEKJvdkN2IKv ErRCqMZE aNwQXlS4V4m/iAhC8NlwehDhx2KY0cJiY6s6tivW7l6gobJsbVTjIFzO4LxcUxz6q6W7E5MhxysXE0eqPiybDhd2OqtxW+9GNHZgtQNtVojs1zSWRlkagayVOF+EwBjDu1+f8I9+ODIx22/22Mvd1r5BCetr4TQSULW4VK9zV0ta3vg6LLOufpjo/Md745eVkmJx4jDFgZue/q4kZjOqDNztfRGw9T8+GnBbb/NhV1wv7q4B65fzWFUkTxEPh4Jq/zgJbl5YIIvS80xi01YpRGi0yGxkksBUn1yvOI8e1JK8szaIfs2QUYb4HKCyJWWXRLqPh7Q2R0TwrXimI6UQQtEXC4trYHfZdk1q0CmZZmo9Ybo6VnpD+flNFdOpGq1XBqwizkaPg+k1oj84DWnE7gufd+iep9izOHZuREc4Pf3waduFQDw0+ORGTupAT4JTLV7HcP3stNGX7zkL+J7qZoi0gTDzrXO4bd0Jp7cYsT3mmPhUiwmPnGeV35A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, 4 Apr 2023 14:49:13 -0700 Yosry Ahmed wrote: > On Tue, Apr 4, 2023 at 2:38 PM Andrew Morton wrote: > > > > On Tue, 4 Apr 2023 00:13:50 +0000 Yosry Ahmed wrote: > > > > > Upon running some proactive reclaim tests using memory.reclaim, we > > > noticed some tests flaking where writing to memory.reclaim would be > > > successful even though we did not reclaim the requested amount fully. > > > Looking further into it, I discovered that *sometimes* we over-report > > > the number of reclaimed pages in memcg reclaim. > > > > > > Reclaimed pages through other means than LRU-based reclaim are tracked > > > through reclaim_state in struct scan_control, which is stashed in > > > current task_struct. These pages are added to the number of reclaimed > > > pages through LRUs. For memcg reclaim, these pages generally cannot be > > > linked to the memcg under reclaim and can cause an overestimated count > > > of reclaimed pages. This short series tries to address that. > > > > > > Patches 1-2 are just refactoring, they add helpers that wrap some > > > operations on current->reclaim_state, and rename > > > reclaim_state->reclaimed_slab to reclaim_state->reclaimed. > > > > > > Patch 3 ignores pages reclaimed outside of LRU reclaim in memcg reclaim. > > > The pages are uncharged anyway, so even if we end up under-reporting > > > reclaimed pages we will still succeed in making progress during > > > charging. > > > > > > Do not let the diff stat deceive you, the core of this series is patch 3, > > > which has one line of code change. All the rest is refactoring and one > > > huge comment. > > > > > > > Wouldn't it be better to do this as a single one-line patch for > > backportability? Then all the refactoring etcetera can be added on > > later. > > Without refactoring the code that adds reclaim_state->reclaimed to > scan_control->nr_reclaimed into a helper (flush_reclaim_state()), the > change would need to be done in two places instead of one, and I > wouldn't know where to put the huge comment. Well, all depends on how desirable it it that we backport. If "not desirable" then leave things as-is. If at least "possibly desirable" then a simple patch with the two changes and no elaborate comment will suit.