From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 359FDC6FD1D for ; Thu, 23 Mar 2023 04:06:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B21636B0075; Thu, 23 Mar 2023 00:06:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AD0576B0078; Thu, 23 Mar 2023 00:06:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9712E6B007B; Thu, 23 Mar 2023 00:06:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 886F56B0075 for ; Thu, 23 Mar 2023 00:06:25 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 5EB6BA02D7 for ; Thu, 23 Mar 2023 04:06:25 +0000 (UTC) X-FDA: 80598825930.16.0A2CC18 Received: from mail-ed1-f53.google.com (mail-ed1-f53.google.com [209.85.208.53]) by imf02.hostedemail.com (Postfix) with ESMTP id 3FB2F80019 for ; Thu, 23 Mar 2023 04:06:22 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=YNVfINBm; spf=pass (imf02.hostedemail.com: domain of yosryahmed@google.com designates 209.85.208.53 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1679544383; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=wZdX0zzJKzWZ5XKjbLc+5SlxbkCswtENUW050E1wjGk=; b=wl/vKwy3Ig3EKU1nYzXv9TyNtiN0HulabvJhaLzFhn6VK0hVVhLBxuLs3Qrwv7s5OXds1h O0d6iqWOVemJ64n7GsSEs3yZFnmi3OJZKSuqpztazop+M6y70rkiOuahpVCK0yoluZ/Kh4 hrDSKyXc9O40x1x6q0GLfbaWg5fPm4o= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=YNVfINBm; spf=pass (imf02.hostedemail.com: domain of yosryahmed@google.com designates 209.85.208.53 as permitted sender) smtp.mailfrom=yosryahmed@google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1679544383; a=rsa-sha256; cv=none; b=OD8Dwv6fGqVYLI7OnSihH0ZdGdKE8o/KUH4vSMZ3mk6Q4jpgrCaAqdiE9UW8NgQ2bRL+GG /sYIE0Ld55wO6NzIMGoJzwx7bcvEJQxhkqwD37qvjPfy3qIe+CG51K+i/B9489DtAQeIxa ol8Qu7/mqwQTgNcdsnLaMIg8IchxkRg= Received: by mail-ed1-f53.google.com with SMTP id r11so81351890edd.5 for ; Wed, 22 Mar 2023 21:06:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1679544381; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=wZdX0zzJKzWZ5XKjbLc+5SlxbkCswtENUW050E1wjGk=; b=YNVfINBm00Zd995dBeW0DDRzCfNhhwl3o+PiaTu4txCBPHZK4brFWKhX4BF6It/Wsb EPmopGFmlzfTAKS0dL++bEqFLsD4ToXQ44j6Wiv7sYiYv4Hy9/oxcmuOQK9AswHVWWpf rj4gX/+U0Onw1Qy4W6+3hbQAaD8Ch74ehfibMziF7Ti3W4x5svk5HFhyO8xyukr6OzmW Yav3yfqQKDe5KvKW5D5Ke2cfLkMwF0rG8SPKOfcygshjyGBWOBvW+cuSnZNJtsKcY+e1 /+Q2LwJu+2BsdXDdc2MlqSoJx+rapqg+0IbXcrAi7lnd5x8c6xqrSXzG1jdKw8q8RhUQ Cg/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679544381; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wZdX0zzJKzWZ5XKjbLc+5SlxbkCswtENUW050E1wjGk=; b=FQmVxl7pWtpF54PlAOkzgcPjfiE4GmURa0P38pDi8m5WKn2U3RYnyAMsOrqHR8eDaG RXpWDGpq1bSgD2gtFVHpaoR0mnNvnAm51gTnLce0+FMTdW0WROKnhHblwYT7Izvuj92P R2rhM9nuirOgqsNZ4hjPf4dEdNlWWOKOrUAG+c0K9PESL58SA/nLn+0HLfd6R6IODCbC xkcM09OexaKgWrr+rgs1maVt4IpgU34KS9OZKirU4370+z/vWPveCH0F87/ijN20zvRk hdH7ORVNcGLFi6GcgPLP38PLXsFpP7qjTabLfmNO4eUbdOrSYILJ5KmrY8Is51zemzgU YYww== X-Gm-Message-State: AO0yUKX4swYoEeTa6RS9BBYT7vfE+c8Nh7WkOHCO9zPuG/5iIQBGJewz YwHeFTJAFk6mCMhCiQz9PY3VfofhNCiuiwepKBd4gg== X-Google-Smtp-Source: AK7set/x0L6xL2vwQegBkGdVFVaSfGgafJFSRErVoiwJ/47Q3xkJsCVmfJEwHU4RiRF1MtOBxUxN57yV/jIPR3Fyiwc= X-Received: by 2002:a50:cc9b:0:b0:4fa:d8aa:74ad with SMTP id q27-20020a50cc9b000000b004fad8aa74admr4544724edi.8.1679544381396; Wed, 22 Mar 2023 21:06:21 -0700 (PDT) MIME-Version: 1.0 References: <20230309093109.3039327-1-yosryahmed@google.com> In-Reply-To: <20230309093109.3039327-1-yosryahmed@google.com> From: Yosry Ahmed Date: Wed, 22 Mar 2023 21:05:45 -0700 Message-ID: Subject: Re: [PATCH v2 0/3] Ignore non-LRU-based reclaim in memcg reclaim To: Alexander Viro , "Darrick J. Wong" , Christoph Lameter , David Rientjes , Joonsoo Kim , Vlastimil Babka , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Matthew Wilcox (Oracle)" , Miaohe Lin , David Hildenbrand , Johannes Weiner , Peter Xu , NeilBrown , Shakeel Butt , Michal Hocko , Yu Zhao Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-xfs@vger.kernel.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 3FB2F80019 X-Stat-Signature: jg7ryd9cqm8byfs5q9yabqafz6djkows X-HE-Tag: 1679544382-835591 X-HE-Meta: U2FsdGVkX1/wCJaThM8jsBkrnZWuudKoHjdrltNu7hATx899bcf9DrqD4WxzwcM/hP2fxoz+SB7KHGx+G1aXWROwkm3k9DRCpVN5GDWzetDq1xph1uMGNsyilrSnmrdiXXJC68f9j1+MjIMr/WSv1eGNH7HBKy4L+tQ0g3a/lMG5ZyLsldrOmjVMZtYiL8JgCotWuyaJqd8Cwc8tZLOwddqlcx4Z81x3xZk8AlbfWvydiIwxsOUCY+BqEfRbV1G8NTnB4NXN/y+nOWtCILUZGGicEMHHEN/0GXsKRGzileKm1jjDQN0mxJdWxIgyH2FDaYRRatbwMTePi0F8WXZf3kWTiabegTqkuJKXISogRICsNFCK5ZgUtPMRYyN1cd9LdPgDPDq/3yLruzoaF6rZqOb81CRtkTZwlrixAx1dXthE5Sug/VutIZ1m0uBWeCCrs15HKn75bAYGSTL62YVSg5oPb4OHfWeQDWzlJ1ci3Q3aHPCjQY+eRX6LaxVgwo5NBQ9iW3f80UfghwTtukOkMKdmS4XsLq3qKs6L4aD2KcNNDqh+JV24cW7cKYfHkMj+5Y32vDHlQNfyuarwaqUJJiRgb2pSdjlrJOHKhhnDHV7DLGJC+tmF7gjKY9JHNBgCuP52PUzcSSp2eXNZJFaa5vgQ0STC2atQlekCl4OLwSUE1smXyr9LDgiT0355Scj0GtIsyLzGhvfvAY3ZC7v3Qfaun8VJRkY8AKml6LUhJcVy8QqmOX9FAJp7Kv/KQn3IH/U8g45PBAkViKM5PvJZu5JjtE7bJS6yiX6MZz/6Ja+7iEcZmVRm0hOMyQ08LmP1Bi1aSO/mM6Ig40V8EG3gGH8z3A22De6rSJ2TrdVwPqysRgYjn9CkqmielDLKaElbvnfb+PVjZ84+ZrcLcNH8z6f39gDEa/VNEYRQqxyMsYRNKQK+OntMmlWNciWixs4tghMbs2PZW/JGQm4SSlZ e22I3Ocl 6sgDHtqpdyWegepg8v0jAq2Y2lrdZl+zgDUEQQ1/tuQdlIsNkAFSfVRiGbB14xxdJ04Y4DJni0mWtsEMJA9aYe5q0kcHF7QpHfFcmnz/4F2WkLSDEZh8ra9YqUEHQZQgEjZloFx62fyCLFmiAUQlx/+8idYRwwOtaXftCJhUQ7qWPnQoasj6GywB8W9RJt3Ql8FJHVCMw3Ikh22K+XbjNhXwdeTnAVIKeE+By/T0BQR8c7F091YEurQ/6MQnlpDlbC7Oyns7NZoSsuumUaLKztcuu0+TtE1yavqcmeHS2iBm6JKPgbkRCeiu9ajls0qM3UHW0sttiyZmgpKwTVWsgefBxyg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Any thoughts on this respin? On Thu, Mar 9, 2023 at 1:31=E2=80=AFAM Yosry Ahmed = wrote: > > Upon running some proactive reclaim tests using memory.reclaim, we > noticed some tests flaking where writing to memory.reclaim would be > successful even though we did not reclaim the requested amount fully. > Looking further into it, I discovered that *sometimes* we over-report > the number of reclaimed pages in memcg reclaim. > > Reclaimed pages through other means than LRU-based reclaim are tracked > through reclaim_state in struct scan_control, which is stashed in > current task_struct. These pages are added to the number of reclaimed > pages through LRUs. For memcg reclaim, these pages generally cannot be > linked to the memcg under reclaim and can cause an overestimated count > of reclaimed pages. This short series tries to address that. > > Patches 1-2 are just refactoring, they add helpers that wrap some > operations on current->reclaim_state, and rename > reclaim_state->reclaimed_slab to reclaim_state->reclaimed. > > Patch 3 ignores pages reclaimed outside of LRU reclaim in memcg reclaim. > The pages are uncharged anyway, so even if we end up under-reporting > reclaimed pages we will still succeed in making progress during > charging. > > Do not let the diff stat deceive you, the core of this series is patch 3, > which has one line of code change. All the rest is refactoring and one > huge comment. > > v1 -> v2: > - Renamed report_freed_pages() to mm_account_reclaimed_pages(), as > suggested by Dave Chinner. There were discussions about leaving > updating current->reclaim_state open-coded as it's not worth hiding > the current dereferencing to remove one line, but I'd rather have the > logic contained with mm/vmscan.c so that the next person that changes > this logic doesn't have to change 7 different files. > - Renamed add_non_vmscan_reclaimed() to flush_reclaim_state() (Johannes > Weiner). > - Added more context about how this problem was found in the cover > letter (Johannes Weiner). > - Added a patch to move set_task_reclaim_state() below the definition of > cgroup_reclaim(), and added additional helpers in the same position. > This way all the helpers for reclaim_state live together, and there is > no need to declare cgroup_reclaim() early or move its definition > around to call it from flush_reclaim_state(). This should also fix the > build error reported by the bot in !CONFIG_MEMCG. > > RFC -> v1: > - Exported report_freed_pages() in case XFS is built as a module (Matthew > Wilcox). > - Renamed reclaimed_slab to reclaim in previously missed MGLRU code. > - Refactored using reclaim_state to update sc->nr_reclaimed into a > helper and added an XL comment explaining why we ignore > reclaim_state->reclaimed in memcg reclaim (Johannes Weiner). > > Yosry Ahmed (3): > mm: vmscan: move set_task_reclaim_state() after cgroup_reclaim() > mm: vmscan: refactor updating reclaimed pages in reclaim_state > mm: vmscan: ignore non-LRU-based reclaim in memcg reclaim > > fs/inode.c | 3 +- > fs/xfs/xfs_buf.c | 3 +- > include/linux/swap.h | 5 ++- > mm/slab.c | 3 +- > mm/slob.c | 6 +-- > mm/slub.c | 5 +-- > mm/vmscan.c | 88 +++++++++++++++++++++++++++++++++++--------- > 7 files changed, 81 insertions(+), 32 deletions(-) > > -- > 2.40.0.rc0.216.gc4246ad0f0-goog >