From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B8ADC761A6 for ; Tue, 4 Apr 2023 22:01:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DD3916B0071; Tue, 4 Apr 2023 18:01:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D5B276B0074; Tue, 4 Apr 2023 18:01:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BD5A66B0075; Tue, 4 Apr 2023 18:01:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id AAB106B0071 for ; Tue, 4 Apr 2023 18:01:37 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 7F06D1C67B2 for ; Tue, 4 Apr 2023 22:01:37 +0000 (UTC) X-FDA: 80645081034.15.C8A8464 Received: from mail-ej1-f53.google.com (mail-ej1-f53.google.com [209.85.218.53]) by imf14.hostedemail.com (Postfix) with ESMTP id 53CD010002E for ; Tue, 4 Apr 2023 22:01:35 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=TOTaDgb1; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf14.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.53 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680645695; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kv2TSgddRLSEtUorKz+cZ0auOFXQwhCuKUblOc6sZ88=; b=XdMnaxBgLJAgXfo33+a2WtmIDigO2+oqCd0/gnu2tAUwduE4sHIWDqPzvAJQFGDHI/TCs9 ISCNHzG8rAOEkokCvAllJLGmEdSsEGeUPmoPbE9M1A6q5T6cBnhzlIyCJEjxqj4Jggh60S NhlMkU05FD55guAP9IgkmzxBpBLHAD8= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=TOTaDgb1; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf14.hostedemail.com: domain of yosryahmed@google.com designates 209.85.218.53 as permitted sender) smtp.mailfrom=yosryahmed@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680645695; a=rsa-sha256; cv=none; b=VR7GFQ9mFuY5bYKAP2exXNkbnD8lsCBBAuCcjzuDKNZyKyBoQ6LxoHI10TeS4moN4JUYr6 OFKwPDLWv2MfKaZCvav5BA43V9DuolkOyI3wpwsUb4Og5aLKJ1GrzsgbrRBswFn2OeqwzU lLWP3m68fP+UYBGSRm7vR26te2y30aU= Received: by mail-ej1-f53.google.com with SMTP id a640c23a62f3a-947abd74b10so2015766b.2 for ; Tue, 04 Apr 2023 15:01:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1680645694; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=kv2TSgddRLSEtUorKz+cZ0auOFXQwhCuKUblOc6sZ88=; b=TOTaDgb1ZEQy4TpifDssiU2hMo1Cm+YqC5+7Z6FE95Mi/bEuyXC1u/M4aU3edREnRj abGD92HeZ6xgeu7FjAYw6diUoPPyfBG1cfvNqWJVOrQvHCSc4aaLOHqlslc+kFKI4RFx KkHaaTCIGjHuAQOfyDZVwlMnIoE8lcpPoq9l8MqbaE8cFZVMbQ5G5WhWQ91Y4e4m3QTR g53IZwVWiElO21KaoaDIz3BRLFRVp/qOtK8ZbX/Pda6PygBlxrT0LK7JnQg0iB79/lEI vyKrAH5GxEeI5sbsE4u+nfFz7mscCPPQJQ9sMQtaD9tMMP1mJa8nZUZS50ftg878I7kc 6hOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680645694; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kv2TSgddRLSEtUorKz+cZ0auOFXQwhCuKUblOc6sZ88=; b=BeVCA7++C7VgWAK6JwhZgIeelkJdMHwdPDrjWsMczLP9Uqb5IT29479T/DpGXuvRCF 7ICGED1RYqaYN/BQws++co8RQ5SdKXtukC7m8Bs5P0GEoCjy0I8uhIY2HHUSKTWusTA7 S6BPAPhOxkP0hMPurhYDTsM21zK3C9fPsbqFZFzvmUDGJ5R+c56p7xQgYx2jUJoO1fhN YdVrnXllYwo+4sP4tato4xjTJqeeeRZJWPjehBET6p8zlXX8FJdLHucQJavQCrAeyaLt FIwcASOyoMS2CyPQoeA2giowR9pRsieD5JhRMglThOS//igBz3hP+eNEuBYJuqYNm+xr U17g== X-Gm-Message-State: AAQBX9dAh8sWqfS/bik0gOdT2SSHMIs3QDIlixs9kUrjCgCpLMkQiBfB YzdtY/HTd68XFoCXz/uh19hTfN/be/9alorxBv8rNQ== X-Google-Smtp-Source: AKy350ZHcX0TAVCqcpBXk84eCLSYS6TADo3x8F71eFEmC/F00AdU8mbkH42S6DeFTXDJYBsmoM0gtHE9yfbMO1NG5jU= X-Received: by 2002:a50:cc8e:0:b0:4fb:9735:f915 with SMTP id q14-20020a50cc8e000000b004fb9735f915mr32964edi.8.1680645693595; Tue, 04 Apr 2023 15:01:33 -0700 (PDT) MIME-Version: 1.0 References: <20230404001353.468224-1-yosryahmed@google.com> <20230404143824.a8c57452f04929da225a17d0@linux-foundation.org> <20230404145830.b34afedb427921de2f0e2426@linux-foundation.org> In-Reply-To: <20230404145830.b34afedb427921de2f0e2426@linux-foundation.org> From: Yosry Ahmed Date: Tue, 4 Apr 2023 15:00:57 -0700 Message-ID: Subject: Re: [PATCH v4 0/3] Ignore non-LRU-based reclaim in memcg reclaim To: Andrew Morton Cc: Alexander Viro , "Darrick J. Wong" , Christoph Lameter , David Rientjes , Joonsoo Kim , Vlastimil Babka , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Matthew Wilcox (Oracle)" , Miaohe Lin , David Hildenbrand , Johannes Weiner , Peter Xu , NeilBrown , Shakeel Butt , Michal Hocko , Yu Zhao , Dave Chinner , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-xfs@vger.kernel.org, linux-mm@kvack.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 53CD010002E X-Stat-Signature: 8tbr3x3yx871aprr7ipyj749m5tmcpjg X-HE-Tag: 1680645695-942815 X-HE-Meta: U2FsdGVkX18g9k1jZ2fajHagKQxAvPec+ckVeRHiYMi8EgxPCzGkXshcqqomrZFhe/VVYWOedVRfjMkem2857KUDCPhxyMCI+hHhEe/8zUqsS8XSODbLM6v+xO9E1kvhbFOX5NVkX694BbzAR4M3ekpQ+HysEf/pJlYqe19otbgbdLnpjmylqbReA6+W7wBTdO7Q70rvsiDfFQDmBesevuvOuqMIi3CRXHZ4jF/2+X59MzQvesUeZUkvfeSj0oSUz+R5ONxT1Nzbkz8S8kL8MOpK1oXPtxqtxkUq4HkH4IrNzNlOWAimx+EKw4oVAdtG2pjApxXCiZAOd2icRH5zU93fWf0duSo+dGVrTdViYkdjYgoCXHycJd+5VnkBkSUPcHC/hw19S78v8xSyx2k640r24C0mZ7a6MJ7K292pjPC7D4XWt53m9qlKyAq8fRgDNjHZGPxOpNqpQYnhxHOr5bPG2C1Y/JizRjH50gp5Fm1wJhTVH6jdllr4OJC3yqLDCUwoTbRvqJkKv0J6z2z3UE4NZOWWFLcz39mKumcSD6qjOuDEIWLjl9SPElXyF1lY0hJWeeKHZb2xQYrs0LHFO41cRVbxkdIlwr5flBPiOaeu7xWF/Fa+r2h7FmeUg72OqaCyooCihGcB6DNy2MaOhIeesN/+TbpKsKA9j2Ya8AknTvwy5iP2DBAGEeQxjShibsjf+p/ixLlYeQS9PS3yJFUiNbHiFiRLr9cKSosqiMMCRwiI+dtUTycBAp3O7Kj1zD5/hrZ5DGyRhyPygb5qfZcizMKYgk1QiPCSzsZvMpsXtRC3KgticTpDHJMErjmMYpQ9t8DBZotHFmn9+hcxBa2zKbSGNXd770f5+hyxEcjInmTfaxusLbRDDS64e33yJ/pO/vvKTTmhElmjPxGjyx7IDyhl4wcPfoOwy/j3FZYzoq87gSe7o8P9p5MYns9fgjQGMsXfZ6YeDbKkQRT RMereter K+G5RcYmeaC8CXLD4gj+zrA/Ztci1IRTaFAgA6NHTDUXl3chgVd7hArTwZ8FybvVRAJd/1XtTVl2uNNk0ucJHbayW5CT9Wej8OeFEtKb5tmgiupH6E+gDUo/L4oCQu8TcxS+XVCkoNf54nOSOiPAq14lmsIifL2wp7xYuRRsCWtCAuT09FntfJZxCTxTKFAwj9bdcsXMOLk7PJ8bdzb/LuKRCUhL1Ly06Jib8pEtk2jKU+Z43rWJJTL3UU3kf57U7hfrR1FGnUQQyjXgXkolDlTPdhk7dr0OuA0Le4HebvKROiQw/Pna/D4/TBFczT6rpRWHWsuNWEfzykf1YMzWySo6JaLz/INzNScEEq/US7fCMwkONqytOa/H1UA1he2tGWbDJhXHDPcNKgGGS3bRQxO1wOg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Tue, Apr 4, 2023 at 2:58=E2=80=AFPM Andrew Morton wrote: > > On Tue, 4 Apr 2023 14:49:13 -0700 Yosry Ahmed wro= te: > > > On Tue, Apr 4, 2023 at 2:38=E2=80=AFPM Andrew Morton wrote: > > > > > > On Tue, 4 Apr 2023 00:13:50 +0000 Yosry Ahmed wrote: > > > > > > > Upon running some proactive reclaim tests using memory.reclaim, we > > > > noticed some tests flaking where writing to memory.reclaim would be > > > > successful even though we did not reclaim the requested amount full= y. > > > > Looking further into it, I discovered that *sometimes* we over-repo= rt > > > > the number of reclaimed pages in memcg reclaim. > > > > > > > > Reclaimed pages through other means than LRU-based reclaim are trac= ked > > > > through reclaim_state in struct scan_control, which is stashed in > > > > current task_struct. These pages are added to the number of reclaim= ed > > > > pages through LRUs. For memcg reclaim, these pages generally cannot= be > > > > linked to the memcg under reclaim and can cause an overestimated co= unt > > > > of reclaimed pages. This short series tries to address that. > > > > > > > > Patches 1-2 are just refactoring, they add helpers that wrap some > > > > operations on current->reclaim_state, and rename > > > > reclaim_state->reclaimed_slab to reclaim_state->reclaimed. > > > > > > > > Patch 3 ignores pages reclaimed outside of LRU reclaim in memcg rec= laim. > > > > The pages are uncharged anyway, so even if we end up under-reportin= g > > > > reclaimed pages we will still succeed in making progress during > > > > charging. > > > > > > > > Do not let the diff stat deceive you, the core of this series is pa= tch 3, > > > > which has one line of code change. All the rest is refactoring and = one > > > > huge comment. > > > > > > > > > > Wouldn't it be better to do this as a single one-line patch for > > > backportability? Then all the refactoring etcetera can be added on > > > later. > > > > Without refactoring the code that adds reclaim_state->reclaimed to > > scan_control->nr_reclaimed into a helper (flush_reclaim_state()), the > > change would need to be done in two places instead of one, and I > > wouldn't know where to put the huge comment. > > Well, all depends on how desirable it it that we backport. If "not > desirable" then leave things as-is. If at least "possibly desirable" > then a simple patch with the two changes and no elaborate comment will > suit. > I would rather leave the current series as-is with an elaborate comment. I can send a separate single patch as a backport to stable if this is something that we usually do (though I am not sure how to format such patch).