From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2104C52D6F for ; Thu, 22 Aug 2024 00:53:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DEA866B0110; Wed, 21 Aug 2024 20:53:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D72576B0111; Wed, 21 Aug 2024 20:53:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C11A494000B; Wed, 21 Aug 2024 20:53:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id A20D46B0110 for ; Wed, 21 Aug 2024 20:53:01 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 4BC231C5096 for ; Thu, 22 Aug 2024 00:53:01 +0000 (UTC) X-FDA: 82478056962.03.D5E0AE4 Received: from mail-vs1-f49.google.com (mail-vs1-f49.google.com [209.85.217.49]) by imf25.hostedemail.com (Postfix) with ESMTP id 7AEDFA0013 for ; Thu, 22 Aug 2024 00:52:58 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=LHztOAPs; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf25.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.217.49 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1724287919; a=rsa-sha256; cv=none; b=0A2/m3o4IBYr8qlaodJxSqWTV5TZSFaNKrKLtH/21/q2163CrTtcoRsvvp24jTDu7m9GA4 rxtuvTfBc/qaQLTmCo8nLtRAAgIhAGgaDcJPdqoNaJKFUwW6z//jmtBmcYTmMNtRDmIlzZ 0dCzN6/d7iYmJ1SwUM7W1cgBhmI6sDo= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=LHztOAPs; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf25.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.217.49 as permitted sender) smtp.mailfrom=21cnbao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724287919; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RvNUmXh6QYHC9p2pKnLZfslXEAtuHNdoak5tt5thDe0=; b=5TsSgAXgYQafW+cU9ECtoopTBDZ+ktaKe2jLYtE67w6MI7BAZ1gVYhKFugRiHujVOeT2Sj exmSWWXAnOBb7sf0WixIBY0u6JvRPatyA/jWnJFL8q/1DXtf7OSnNf5WX+TMyOkYC9yCRq CNfA17QoT2ZBux3wbKcQK35mGOnCf1w= Received: by mail-vs1-f49.google.com with SMTP id ada2fe7eead31-498d6d67390so97233137.0 for ; Wed, 21 Aug 2024 17:52:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724287977; x=1724892777; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=RvNUmXh6QYHC9p2pKnLZfslXEAtuHNdoak5tt5thDe0=; b=LHztOAPsSkZb1ITw6ldl5uCgTyGe+w5wEpO1c/qCA9akHPXprc4PgPj39sjxy/JHnp /NDhdZarcEEbXNmv1lWPlMS2LztbNlNMGfdb8uKpV11JeKs2kjAOF8NCzWvM81Hg0Zt/ kr0hoNyi6SIWjS0I+7EdK/m3Os2vQH6iZvadrRMKACobmZGvsazk+/5K/fs+ZUZoE5/W pXauWUEb5Lf3wPThOZfZZ9mZfu0KamsiFTRtbelqmVwyg28+XV++7o2OioW7Lik2tQ/z hMS5JUdtdq+4NhTDX39JCVf4VNCXTkpJHWufKv0norgom4z68sC4bCzkiQp4uYiiWq0y LFuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724287977; x=1724892777; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RvNUmXh6QYHC9p2pKnLZfslXEAtuHNdoak5tt5thDe0=; b=k/7DGrJ+0BP2FtUjnFXua0zJnUoX/LWVzg+lNOnfAeOKpsA8AvY6n14LzG9Ugm3llj W0U9VTc1kK34XhbGppV9X8sY9oHIElpnldS94rvAnhGkcMG0Jox3Rq1JOiLmyQFGqsbg NDr35C8ESwtRahB/WzyLuBi90f6RJoY9jEpvSSwMdxh9vHNaqC40IVKrcvgTwNRXmqdq 5FDRwlmgUcRVjcBQsUrZvIZ/TcDH5DbOlnFDOWEfE+0kGKRZ2e4I92iF8HseUJmzcp9Y VTSfp74n6vhFKZJvRb1Y8phHscE9PRwkWvRVkygXS/9bMMzbs1yVzmWnVx/At78Hx6iL Mn5w== X-Forwarded-Encrypted: i=1; AJvYcCXtP2NVhAwF0SwoVqarjnlz+lBdsBkg11JdLEA5RIENoDNVJQsOb4r+uj3LyakpSpwjTufl3rYKdA==@kvack.org X-Gm-Message-State: AOJu0YzH5lah8r+KgiA6HNdWXyU+74vEufC/u4Qlut9LWkoU0niraezw 8fnDgxQBu3D5cFDGiXfoD7PEWVf5N2kGEDZD5EdjFIlXfNla3Nrp7zwCCkqWG7zkauPCIgKzGOR CigkzsYP4dJEx7Jhna/VouJs5l+w= X-Google-Smtp-Source: AGHT+IEl3MsbTYZQIajgTxyKIGV+TI3/pnxNw23DZ/6DGgfoWTK+gno5G+goww3B/BQ8SjYp+OzkBtSV2gx+2xET4I0= X-Received: by 2002:a05:6102:e0b:b0:493:b2b4:3708 with SMTP id ada2fe7eead31-498d2ffd644mr5148601137.27.1724287977448; Wed, 21 Aug 2024 17:52:57 -0700 (PDT) MIME-Version: 1.0 References: <20240811224940.39876-1-21cnbao@gmail.com> <20240811224940.39876-2-21cnbao@gmail.com> <3572ae2e-2141-4a70-99da-850b2e7ade41@redhat.com> In-Reply-To: <3572ae2e-2141-4a70-99da-850b2e7ade41@redhat.com> From: Barry Song <21cnbao@gmail.com> Date: Thu, 22 Aug 2024 08:52:44 +0800 Message-ID: Subject: Re: [PATCH v2 1/2] mm: collect the number of anon large folios To: David Hildenbrand Cc: akpm@linux-foundation.org, linux-mm@kvack.org, baolin.wang@linux.alibaba.com, chrisl@kernel.org, hanchuanhua@oppo.com, ioworker0@gmail.com, kaleshsingh@google.com, kasong@tencent.com, linux-kernel@vger.kernel.org, ryan.roberts@arm.com, v-songbaohua@oppo.com, ziy@nvidia.com, yuanshuai@oppo.com Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 7AEDFA0013 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: r4yctnigedj7tpyhtd9wyhgwze8u149i X-HE-Tag: 1724287978-622185 X-HE-Meta: U2FsdGVkX19FXMOrLHlCJ1zXcAEU3hYhmYlrgeGFsSab51GizFA/ZjXPBBQEz0+W3xtbsu7jOa1/h/uWzXsL+BdrUUZhol3adkdWOzm5izY0gdVV2SPWn+rTjjQJ+POMDZ3EYDFIJsKMBVTlXBeZjGzgYO0/PIKWoyuYXDaAvR0q+P7ljD6tH4gCd+V2YyQlO3Ztnp670f9FkV5Vw7QcngxSc0ko3P1Qm0rAjEpj4LBsW6+fJQPp2EO/ssCT6iRUb423fwwExWz8RkDLpy8hDaDAqTJXuPrzRWxXt3c1V/c3b45ZJTdJRfNoZBFtGT+W5XS9tlDkpsSUCF5idI6vxznJiA6vWcuVIi4q2gFCNb4vcK3/QllidS5c+6zuesz8AFJnmu3ixPxdzu3Dr8ptzGL8h8UgNnOhdGXviWS04MWrQYRnD/zv2W6yKMcnT19KHdI81vVd3hY3hERGxvPTjzYlpgwDSU9Q6nE01Q821n4A7nn60PIx0zeyx1ug9Z6wSdjzlAcKZMa424Q/iMIPQuT7OCqkeGjuRoJ1D5RjXGu7HUzvCm6fao4mjMaGW4/ml1RJZj7nC/AMc2eGtjfyzGgx/J7y8FC4gOPOo9k38JScDl4kuC7pfjVZoJeVayknMSaqqo1U9iBw5Rn1d2ASd3J6yuZp69pLEyDFMwZtUXdBY2uMb1p+lMMohWs5+PgAM+zQuW/PBAmyJiOLca0GIaFefP7LZs5vPq0mo+90ess5MggCrsJh2feQkLKq6+hp0mBYf0YsLrv8vn0Gr4xIIL1DQAsbmYjALwOzmASXum2IGAlpy124dZFeebXND+a9m0LiM3zOq+iLU6nbStwI/Noa4lfAo5Fp9G/fS6wN+75KzaH5jDqodEAE9w2iv5/IKW7ivyFpTMm2R4yqcDs1/ijmr/RxmXNdHovOmTQM/y5faaSInWOAmb3C3BaOLb9HI5Tm6dvB3zqn0snB9ok dkoJ9Quh AcDGUZqLLFUbJNCBZ7ANgpViF3yhQwCNbpbxr4fyIN0UyYPZN2f0CMBgcV/25hdRtei+wQm7qLjVkRSOiOggYMjchs3jgHXldbzGKv5kzll3BqHt+jchYazqv29W69pE/qQc0SiST/qrEeAy3lbI2v1kqIt8xnGMNZqNiG1oTchfVUVkH/2tqCv32ZDSrjgvttjyoL0UhOVqaI3dpCSTZ0yyKz97H43WFtLESXqh/+sxbnVsYrognfhlzp8N3FJ/OswCea5TldSgc3nFjeC/2rg4bhS70Qzan3th7ra+/BOGL2ha561mp+SjMqsCpVqN/eBiAXXncEwXCW0rfGIe+43EblQXQPPkWD/OMKzyS1CpXPmVXZ/Q/0USPFMP5UWAB1oa3 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: On Thu, Aug 22, 2024 at 5:34=E2=80=AFAM David Hildenbrand wrote: > > On 12.08.24 00:49, Barry Song wrote: > > From: Barry Song > > > > Anon large folios come from three places: > > 1. new allocated large folios in PF, they will call folio_add_new_anon_= rmap() > > for rmap; > > 2. a large folio is split into multiple lower-order large folios; > > 3. a large folio is migrated to a new large folio. > > > > In all above three counts, we increase nr_anon by 1; > > > > Anon large folios might go either because of be split or be put > > to free, in these cases, we reduce the count by 1. > > > > Folios that have been added to the swap cache but have not yet received > > an anon mapping won't be counted. This is consistent with the AnonPages > > statistics in /proc/meminfo. > > Thinking out loud, I wonder if we want to have something like that for > any anon folios (including small ones). > > Assume we longterm-pinned an anon folio and unmapped/zapped it. It would > be quite interesting to see that these are actually anon pages still > consuming memory. Same with memory leaks, when an anon folio doesn't get > freed for some reason. > > The whole "AnonPages" counter thingy is just confusing, it only counts > what's currently mapped ... so we'd want something different. > > But it's okay to start with large folios only, there we have a new > interface without that legacy stuff :) We have two options to do this: 1. add a new separate nr_anon_unmapped interface which counts unmapped anon memory only 2. let the nr_anon count both mapped and unmapped anon folios. I would assume 1 is clearer as right now AnonPages have been there for years. and counting all mapped and unmapped together, we are still lacking an approach to find out anon memory leak problem you mentioned. We are right now comparing nr_anon(including mapped folios only) with AnonPages to get the distribution of different folio sizes in performance profiling. unmapped_nr_anon should be normally always quite small. otherwise, something must be wrong. > > > > > Signed-off-by: Barry Song > > --- > > Documentation/admin-guide/mm/transhuge.rst | 5 +++++ > > include/linux/huge_mm.h | 15 +++++++++++++-- > > mm/huge_memory.c | 13 ++++++++++--- > > mm/migrate.c | 4 ++++ > > mm/page_alloc.c | 5 ++++- > > mm/rmap.c | 1 + > > 6 files changed, 37 insertions(+), 6 deletions(-) > > > > diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation= /admin-guide/mm/transhuge.rst > > index 058485daf186..9fdfb46e4560 100644 > > --- a/Documentation/admin-guide/mm/transhuge.rst > > +++ b/Documentation/admin-guide/mm/transhuge.rst > > @@ -527,6 +527,11 @@ split_deferred > > it would free up some memory. Pages on split queue are going = to > > be split under memory pressure, if splitting is possible. > > > > +nr_anon > > + the number of anon huge pages we have in the whole system. > > "transparent ..." otherwise people might confuse it with anon hugetlb > "huge pages" ... :) > > I briefly tried coming up with a better name than "nr_anon" but failed. > > if we might have unmapped_anon counter later, maybe rename it to nr_anon_mapped? and the new interface we will have in the future might be nr_anon_unmapped? > [...] > > > @@ -447,6 +449,8 @@ static int __folio_migrate_mapping(struct address_s= pace *mapping, > > */ > > newfolio->index =3D folio->index; > > newfolio->mapping =3D folio->mapping; > > + if (folio_test_anon(folio) && folio_test_large(folio)) > > + mod_mthp_stat(folio_order(folio), MTHP_STAT_NR_ANON, 1); > > folio_ref_add(newfolio, nr); /* add cache reference */ > > if (folio_test_swapbacked(folio)) { > > __folio_set_swapbacked(newfolio); > > diff --git a/mm/page_alloc.c b/mm/page_alloc.c > > index 84a7154fde93..382c364d3efa 100644 > > --- a/mm/page_alloc.c > > +++ b/mm/page_alloc.c > > @@ -1084,8 +1084,11 @@ __always_inline bool free_pages_prepare(struct p= age *page, > > (page + i)->flags &=3D ~PAGE_FLAGS_CHECK_AT_PREP; > > } > > } > > - if (PageMappingFlags(page)) > > + if (PageMappingFlags(page)) { > > + if (PageAnon(page) && compound) > > + mod_mthp_stat(order, MTHP_STAT_NR_ANON, -1); > > I wonder if you could even drop the "compound" check. mod_mthp_stat > would handle order =3D=3D 0 just fine. Not that I think it makes much > difference. i think either is fine as mod_mthp_stat will filter out order=3D=3D0 right now. > > > Nothing else jumped at me. > > Acked-by: David Hildenbrand > Thanks! > -- > Cheers, > > David / dhildenb > Barry