From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4C858C636D3 for ; Fri, 3 Feb 2023 00:01:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D08706B0074; Thu, 2 Feb 2023 19:01:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CB8B76B0075; Thu, 2 Feb 2023 19:01:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B58AE6B007D; Thu, 2 Feb 2023 19:01:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id A454A6B0074 for ; Thu, 2 Feb 2023 19:01:04 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 6F6BF1A081C for ; Fri, 3 Feb 2023 00:01:04 +0000 (UTC) X-FDA: 80424025248.04.76EE205 Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) by imf28.hostedemail.com (Postfix) with ESMTP id 791B3C0021 for ; Fri, 3 Feb 2023 00:01:01 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=fromorbit-com.20210112.gappssmtp.com header.s=20210112 header.b=DLV66Jzr; dmarc=none; spf=none (imf28.hostedemail.com: domain of david@fromorbit.com has no SPF policy when checking 209.85.216.47) smtp.mailfrom=david@fromorbit.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675382461; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=87LqBY1cmxJIDzlBwcArixEGRBUdoaZyq1gL8jLOlBo=; b=E9AavHBlpAcpVSKF4Gmb6/NZhI5yvKcZcrxExluV/8vpubBW1Ga0QnD7hBBSniqASM6/Cr M6e3YJAmbMcMUOH84tA2owgGYLRA1gC+jANRF7i3xGEjk+eewap1qNNtK2uNjQ9Gwttl5w YpUzevoaPfE1RodTRMzYsCYnpYFA3Gc= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=fromorbit-com.20210112.gappssmtp.com header.s=20210112 header.b=DLV66Jzr; dmarc=none; spf=none (imf28.hostedemail.com: domain of david@fromorbit.com has no SPF policy when checking 209.85.216.47) smtp.mailfrom=david@fromorbit.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675382461; a=rsa-sha256; cv=none; b=B+an8VxgBY/aYvbR60gG8oKl3eZYSjIppMjj6xA2rKjWdErgZnwWIQGg4d0XN+9ot7vILh QgiIf2diF5WPbNCihQskt1cupFtzp74f1Osgm7ZJxpT9JWyufePAapqJ5jv7EGYBkdCzkI QGXtLJIT8jHp183V0Sm5a+7ipXYW19I= Received: by mail-pj1-f47.google.com with SMTP id o13so3455307pjg.2 for ; Thu, 02 Feb 2023 16:01:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fromorbit-com.20210112.gappssmtp.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=87LqBY1cmxJIDzlBwcArixEGRBUdoaZyq1gL8jLOlBo=; b=DLV66Jzr/7xWnW/DB5gYUzwmub8DiPYt26/ICu8uqaIhyj/Y69zetvYZjUSTNA7JP4 oVFIW77wI167XlBLdmGFDdBQltemXKC02tY+062ITt7Fz7d2TNa4f42A+jhli5StHy08 4llhwpGMDi0YoGL3ASI82cb0n+Jneqd4L8TQs/k8Bl7K34gebF7CtuiqCdXKrXEMHzWG +78I7Fd6vJv9IAIj+P8oUAjNdRi92lx+gwfIj1aM6vDJk7BZXhxx+xsLu76rYXE4iMux BKyGx4MS/N9hjYr2+vYpvxDJcYByMMdDkUAB25zywwQdmKUb4WsI5MtyEeLjFgjw4eg+ aFbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=87LqBY1cmxJIDzlBwcArixEGRBUdoaZyq1gL8jLOlBo=; b=H0Hs9OVvOYQ7whaw5NvMK5mSTaZ3rr89SYoRZ9bV6jyxk2zfgjN5/A1VCT1nvlRzSi qKArn8LJPYVINGmK6wlhN0WKuJcuZc2vbokxm5vdlJRbmRYoq+Yd03uDILv9C9iS3Kdm xNc4QF6CZltuISrdsB2qKmR0RnUA40TzVSMw3lBlEJoNJLgzq6VjNVYHHXxsMBNIbs7Y BOgJtXzfpDkf7C5R5COn1w90UmNJfAd4P5LA/+KfYEY38cvG+r659KtXeLmgyYU0MHRy i+9vavzFrdtEMtiU16GCQ/xtus7PhuLVtLwZTFPjMsCqEjUq6IT8yriL4WTNtVvVoiuD YMpw== X-Gm-Message-State: AO0yUKXT36qLETjduh18KuozBuAwYfcTp/DNE4N21T+71w7utWaR5+4A zqdiOm9lKmFsjEJ+RslEaL2TEA== X-Google-Smtp-Source: AK7set9JaCKtnriracbwmug7u2ZHT3TTrBbzfIKwL1tiM1SyRqwMx3ca50uMJGBzPDJ4jy/LfeFDhg== X-Received: by 2002:a17:902:f0d1:b0:198:dec0:c926 with SMTP id v17-20020a170902f0d100b00198dec0c926mr231076pla.21.1675382460437; Thu, 02 Feb 2023 16:01:00 -0800 (PST) Received: from dread.disaster.area (pa49-181-4-128.pa.nsw.optusnet.com.au. [49.181.4.128]) by smtp.gmail.com with ESMTPSA id x190-20020a6263c7000000b00593a1f7c3dbsm292980pfb.10.2023.02.02.16.00.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 Feb 2023 16:00:59 -0800 (PST) Received: from dave by dread.disaster.area with local (Exim 4.92.3) (envelope-from ) id 1pNjVh-00AfBG-Cq; Fri, 03 Feb 2023 11:00:57 +1100 Date: Fri, 3 Feb 2023 11:00:57 +1100 From: Dave Chinner To: Yosry Ahmed Cc: Alexander Viro , "Darrick J. Wong" , Christoph Lameter , David Rientjes , Joonsoo Kim , Vlastimil Babka , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Matthew Wilcox (Oracle)" , Miaohe Lin , David Hildenbrand , Johannes Weiner , Peter Xu , NeilBrown , Shakeel Butt , Michal Hocko , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-xfs@vger.kernel.org, linux-mm@kvack.org Subject: Re: [RFC PATCH v1 0/2] Ignore non-LRU-based reclaim in memcg reclaim Message-ID: <20230203000057.GS360264@dread.disaster.area> References: <20230202233229.3895713-1-yosryahmed@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20230202233229.3895713-1-yosryahmed@google.com> X-Rspamd-Queue-Id: 791B3C0021 X-Rspamd-Server: rspam09 X-Rspam-User: X-Stat-Signature: 5385rhunu89d6qmjwe9detntrmh3omsp X-HE-Tag: 1675382461-172291 X-HE-Meta: U2FsdGVkX19u2lX3zqJ26WnEGtnLJenp5s2D+VQUByPvVFS05mO4vNmOyRM6CKlPb3NFE5q3ocypvwIQXwEeBoQTcjB9DRmylaW1BQ9iMF1VGmduywn3CgfyGAFEafto74TB8PjM4JEIJKmNgtXmYk+4Xjr+4culzO71g82HG+6xF950rvrQMKWeEyUx22Dx98/exwglIYK40BUWdvJN+6xSKvI6EC0U9iTC21vVY/EokMNQ7lBvxpIvfjA3QacZbMO4Tov2WQoHGSDZUb7dFpAmwdLsVxs1qDluKI9jgpm2i3Wbywj2XQ1o5rusATfwzJltXybTPZj5Hbaz3Qoz+dGssrF7x7NNusXqslYPXp/xACdU/+2NbaT6TYYBJZzuTVaIR/1b80CufNXWCzepHQaLhM84fw++E6o4Uoule59TojY+Ux19nsHGTczxXVFmMfetaa1uU8DMHoqMGV0r+vSICC0t+wxz2hIWv500gr2umET/HdW23PjeAmHjzazAWxqSfb7JSvULVLkcOOE0YMyXF75QwK/K6ll/UcNdf7HCR2dLchTomMetH5ziGx/xQmCkUyNhZZ4PzlgxUzeDZ9BXBLlk9YCmoTt/2/qM0CQl8xY/HKhcYM3toj+l5bC8VRtAFmXHqH68ZVz1SoB+TFE50xT1R7XBRNI0rmNJt0sOHgXXinYt2JuZ9BHqbaIQWMKjIzGXaCoG2bG9JE6Xzo7a1tn37xXt+kherAKfgPfZDg5vuu/4G5WttB+QWgabHqHFLv/u/Cc+AKmTBb35J4/rToLEsiOlP/aImhOV/X6mylag4fyW/WpSvSOd372EYTPVhaqyX+I/jaZEt+L6+MV4ebqCfAi5Ena2hD0ULf2tIYknckA9MWucbuP2FLeoohYo718xtCaFLDtA3cXHPqkWUqTUNnntZSVlGJInficFUSluV+EelgvGxFnECll/80m8ethHv4N4OS0bkUc 6oZ2FT31 rTCtMe2K8Da8NCjAY6TA9Vgt6Mu0dtgdmthl1Y8Jr42tzOwxbyi1JX2JhyJQzE+IaCze7yResv7VhSvnHPYzqhu24hdgXxz5eVNkIaVnDlpiQZxDwDzib8L/ZvZxS5+njMvSyzjULH05TfpE3BgBOIVlxjL04mcydDq1rWRqoCRVNu46u1DkFkO4vhXy/oTfWKjvwvkLozywQfBtxYHrzACt7ns87eN0/L58aF6TME8m3KkYL7xXFykwuOgtzNt3NULbSyg/xjVITVSnV79DGWugThnXUD2xmqRpxTtq91iyeVU1p4amuPgyxVwmXiUyuIaWizrGHlWuaqCEsk3WTsJwKtgx7TSek3Vurz0m3q0WmLe1gHW3Bsz2tx/Fzq/Wn+VD03klZG/CXUJWZ3HJZNbqvSFqeaVixKiQDpd3UaDzqkLUVCIZEwi7Ik9LVjf58dJ7JrQod6bQ1MrMxzI/GC2RuOvh+BPJ3WgP8VWgjBU4YCVvFhVr8A+fgaA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Thu, Feb 02, 2023 at 11:32:27PM +0000, Yosry Ahmed wrote: > Reclaimed pages through other means than LRU-based reclaim are tracked > through reclaim_state in struct scan_control, which is stashed in > current task_struct. These pages are added to the number of reclaimed > pages through LRUs. For memcg reclaim, these pages generally cannot be > linked to the memcg under reclaim and can cause an overestimated count > of reclaimed pages. This short series tries to address that. Can you explain why memcg specific reclaim is calling shrinkers that are not marked with SHRINKER_MEMCG_AWARE? i.e. only objects that are directly associated with memcg aware shrinkers should be accounted to the memcg, right? If the cache is global (e.g the xfs buffer cache) then they aren't marked with SHRINKER_MEMCG_AWARE and so should only be called for root memcg (i.e. global) reclaim contexts. So if you are having accounting problems caused by memcg specific reclaim on global caches freeing non-memcg accounted memory, isn't the problem the way the shrinkers are being called? > Patch 1 is just refactoring updating reclaim_state into a helper > function, and renames reclaimed_slab to just reclaimed, with a comment > describing its true purpose. > > Patch 2 ignores pages reclaimed outside of LRU reclaim in memcg reclaim. > > The original draft was a little bit different. It also kept track of > uncharged objcg pages, and reported them only in memcg reclaim and only > if the uncharged memcg is in the subtree of the memcg under reclaim. > This was an attempt to make reporting of memcg reclaim even more > accurate, but was dropped due to questionable complexity vs benefit > tradeoff. It can be revived if there is interest. > > Yosry Ahmed (2): > mm: vmscan: refactor updating reclaimed pages in reclaim_state > mm: vmscan: ignore non-LRU-based reclaim in memcg reclaim > > fs/inode.c | 3 +-- Inodes and inode mapping pages are directly charged to the memcg that allocated them and the shrinker is correctly marked as SHRINKER_MEMCG_AWARE. Freeing the pages attached to the inode will account them correctly to the related memcg, regardless of which memcg is triggering the reclaim. Hence I'm not sure that skipping the accounting of the reclaimed memory is even correct in this case; I think the code should still be accounting for all pages that belong to the memcg being scanned that are reclaimed, not ignoring them altogether... -Dave. -- Dave Chinner david@fromorbit.com