From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17FAAD1A431 for ; Sat, 12 Oct 2024 07:00:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4F71C6B0089; Sat, 12 Oct 2024 03:00:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4800B6B008A; Sat, 12 Oct 2024 03:00:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2FA176B0092; Sat, 12 Oct 2024 03:00:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 0CA686B0089 for ; Sat, 12 Oct 2024 03:00:33 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id E4919121ABA for ; Sat, 12 Oct 2024 07:00:27 +0000 (UTC) X-FDA: 82664051820.01.5D7D5EB Received: from mail-lf1-f45.google.com (mail-lf1-f45.google.com [209.85.167.45]) by imf09.hostedemail.com (Postfix) with ESMTP id F1B7E140023 for ; Sat, 12 Oct 2024 07:00:27 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=TSTwYWvj; spf=pass (imf09.hostedemail.com: domain of suhua.tanke@gmail.com designates 209.85.167.45 as permitted sender) smtp.mailfrom=suhua.tanke@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728716292; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=k+rjhvfejhKc66T4VNRLDbldGPyzCO+IeH4dTtI1lxA=; b=sDBie6Nv1uR6Bcd6vKgyvS2SuZjACy/xHORD5I4HjAwpIGb9XZHeoSURUzJ+B9t9TMxUeq qTE5R8yIwEowCyNKhfrM5NkunsSA+7ty1Mr9IRePy0E8QRCg91HECsPiJXXp8NgnJV/cWx V89YWWjWX8ByuwXWAk6g1TOLDxGjhCM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728716292; a=rsa-sha256; cv=none; b=OLFdTGyAl3knyG5kfFdHhnqxX3HByF8iAS94e6tC3l9i+j21+t2e0EgrlQPYSWtAQj91hT OiEyM5oxwXssf2BYznyDMYAGVIO3fNdvFk2gepJKFtfo5KxbtfsBLFoQtCIi+fx0jEzLOU CfOO7a8mVGsOSYva3HAjPVL6Lu+SB8c= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=TSTwYWvj; spf=pass (imf09.hostedemail.com: domain of suhua.tanke@gmail.com designates 209.85.167.45 as permitted sender) smtp.mailfrom=suhua.tanke@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-lf1-f45.google.com with SMTP id 2adb3069b0e04-5366fd6fdf1so3357189e87.0 for ; Sat, 12 Oct 2024 00:00:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1728716429; x=1729321229; darn=kvack.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=k+rjhvfejhKc66T4VNRLDbldGPyzCO+IeH4dTtI1lxA=; b=TSTwYWvjeAMrC7XsjkOraT1/t/IQm7ql68CIcGqEXdtQ5NBzSrAhlDLPIEEFACHo5K 43fmRlpl0EmW3+6UFHxgFhex/jYp3LbNHyZ9vwyVICxlHrdsu3oJjldWNpgfGHWq6DUO U/+kITY9dFu8pbohrwNAxj5hIzLSXdDTPlFws6507ZrVX6Zm00Zcp6Bb0x3DfKaAtNS/ /RfD7U2LQ+p8+YSTv+T3jpemahB0NhP/f/nD1o7DTN5Ni4gmuafBYr2PBGFoSHbOEER+ NhT2hdaAKi5p2/KDmGkQx/G56N3p2tg7+ZeU2K9LJd2+l5vHjPq2s3HYS9RVRnEKtn8i oDeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728716429; x=1729321229; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=k+rjhvfejhKc66T4VNRLDbldGPyzCO+IeH4dTtI1lxA=; b=TU7hWkhXiodEqxIiNnNfBuYm+IH9ZJWJ2i8XvYp6wYUMbbGczOuHZ00jsXyMiiVrj8 IcBAkvjH+4PBomox+lvjHZz2UUPauLyq3GrEp8IevSMm14ayORAwzXdExcC3yiRj6VTe dADVV09IEUpdooJTL2cjN+pKpusCMcNtXyUP52+159weIMbr9oA5ge4JXlhIWiCE9ftq SvVxhfAYgstkkCXbfwDy8EDLqkfM1i797/9iYihmW2eKTY+JnwODKK/+uPHGp1bxTsW+ r+zOcrGo1yH54j5jNRUh8SotIQP45hekASb8s3HlWrzdivbt6EWAXvTcULrgW79IFgej x8aw== X-Forwarded-Encrypted: i=1; AJvYcCUvUpIgNCVns8+YJ2J0uJ3Ih3xY7fToNjQ+IjucJadfApCdVphkDEYJVgcoLvoOHUmnvGhOO7UaNQ==@kvack.org X-Gm-Message-State: AOJu0YwEvh7Qv27HAWJrV7y1p7y3xnueW1zjKizTmcSHb/orR4+RBfYL qh47YMM6GV93W6KOc1PEwLGs/JhtfAAPzba2X0iMxrdfsIhKQKJR4hUtKgoWUp+4huwj9mGDU83 fkWi7o8eGDcHyownviopxYlDVxXI= X-Google-Smtp-Source: AGHT+IFUCULct4osFhNH4icf3XaVzR8jM+pcuE22BXGQHa+61qR0QNOQ/t5BDtAv2EI6ejnSzwoFg9M6B0Je2ISZpN0= X-Received: by 2002:ac2:4e16:0:b0:536:554a:24c2 with SMTP id 2adb3069b0e04-539da3c5ee1mr3094937e87.13.1728716428454; Sat, 12 Oct 2024 00:00:28 -0700 (PDT) MIME-Version: 1.0 References: <20241011080543.2422-1-suhua1@kingsoft.com> In-Reply-To: From: Su Hua Date: Sat, 12 Oct 2024 14:59:51 +0800 Message-ID: Subject: Re: [PATCH] mm/hugetlb: Concentrated releases memory when cmdline specifies node requests for large pages To: Muchun Song Cc: akpm@linux-foundation.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, suhua Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: F1B7E140023 X-Stat-Signature: 8wgkpams39f8oaco6953hxhuk86n3n5f X-Rspamd-Server: rspam09 X-Rspam-User: X-HE-Tag: 1728716427-998536 X-HE-Meta: U2FsdGVkX188Ot0ybbR9sUXT3eo1yJpUXP0CUJJLkgOq5mcov8xbdvtMfuNCZDwAAUq9qHQpZtqcFfcm/uLJiMzed227wCjCOJbMo1JCjdeoyp+QJqolPVDPupoXEO/X3G3P95o6NIBa4L+Kz0PPMjZjUl1Ax6dsLr/Qc/uVZk/oJ2o9fEmJNS6JJZ7gaRetjd1hM2EmEwhSMtO0X4AnktPucrHk2Q1yX0tX4ux6jotodQUr25O4iJQnV5US4ctZTnBslgrawvqdrPIal01XFC8U1YIzEge7dhwGG2+UZZB1QnzurBBz8xI25cVdFYSzkHSoW5uPMkTfw3Rn7A4ofZfHXNphd/8WzLlfh6lNSEmFmaqjvm4iQvEPMTjWR6UEoQmNu6/0hAVX+IPmn/A3aTe5gEUu81jRH3KIYI1wCUtyWm7ttk+SS6c6YVCL0pDlCOgwwRKK/qgPSgnP193nBNuPpMBWUjL/UzqNRfzwNCzaSeS2YvZID0K++kcPqjYqcHFukRyT5tfFsChtyHbLTsrQfixEr8SOrqMTvIAf6gCCHbBO/3VUc6pMLGlcRsGqzboeTdsT6pNjD28EmRoRbt30tZYEsuR2j2Sfzx6BBbdmynFLk/4QGr/9vjJC9nqRX1rbMQG5j0NrIjCCT6CagekHOrLtcwdohYP9g1gRAM8H9iFIAhDNJ7Bua+00aEQXaHfDytjtlJ/JB6ypkc4CsMPIKHnTa7miIZQn3407evHm8TG4E0gEG83E/6j8DTyFhvOvBphlc4hSa1OhLFTdXBI1O6A3NcRBLwop0bwfG+S8Wtf+0v3cXGkR85FZ9X33hl1gALHqxCcpDo9piAP2zMNHW29ULsATZ3FRxWAheHbY66CikA+XGzDekpFwE5Ki58ulnpziE2phK0XLDK1NnauFU5nda9IlQLdpg+B75DE077QITCLtFuIy1wgnqF6amMRhvx8eAKwmvCf+0ek 8nFBfnyr vZd7u46IbP79YknLlgOaGZNoknWVNxroFdkdT1DlXPolBqP7xC2TNHJ+xCsyVNecRuxNT0VUJLEArDyncgOiY9GbJT6nzSV19gx/Xb5Nn1+L8mV5SqMCmnpMAx5ErlgQvVil/IywyWr3wUKKCjB34iZLkGfzQPLTQA6BTLc1Z8LgH6klmKZOtmtq8hWJpvP+h0kERZ0SmWGn1nCoTMsP6dNXPZtnU6E3pyv/fgG/TDFTIzlazONSJXNu3BvSkk6vHBdbj7Lhxkw1OmtawQq7ztd6kz8LGEFYCcbwBY2NrprfiQO9ZYP4CszjJwQrbR54drq8UlN2k1ocagyEVQxqkRExDYwJUXkeiadoa6XdnQKQgx8CWblA9u+b8TbhAOudcshC1D9u7MkxBWEnDIUaYt3+d/tzzFC8TiLy4vzRScyVRy9aeW/BoPtJaFw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000003, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: It does seem to be for a newly allocated huge page just call enqueue_hugetlb_folio without the complicated decisions in free_huge_folio. Thanks Su Muchun Song =E4=BA=8E2024=E5=B9=B410=E6=9C=8812=E6= =97=A5=E5=91=A8=E5=85=AD 11:23=E5=86=99=E9=81=93=EF=BC=9A > > > > > On Oct 11, 2024, at 16:05, suhua wrote: > > > > When HVO is enabled and huge page memory allocs are made, the freed mem= ory > > can be aggregated into higher order memory in the following paths, whic= h > > facilitates further allocs for higher order memory. > > > > echo 200000 > /proc/sys/vm/nr_hugepages > > echo 200000 > /sys/devices/system/node/node*/hugepages/hugepages-2048kB= /nr_hugepages > > grub=EF=BC=9A default_hugepagesz=3D2M hugepagesz=3D2M hugepages=3D20000= 0 > > > > Currently not support for releasing aggregations to higher order in the > > following way, which will releasing to lower order. > > > > grub: default_hugepagesz=3D2M hugepagesz=3D2M hugepages=3D0:100000,1:10= 0000 > > > > This patch supports the release of huge page optimizations aggregates t= o > > higher order memory. > > > > eg: > > cat /proc/cmdline > > BOOT_IMAGE=3D/boot/vmlinuz-xxx ... default_hugepagesz=3D2M hugepagesz= =3D2M hugepages=3D0:100000,1:100000 > > > > Before: > > Free pages count per migrate type at order 0 1 2 3= 4 5 6 7 8 9 10 > > ... > > Node 0, zone Normal, type Unmovable 55282 97039 99307 0= 1 1 0 1 1 1 0 > > Node 0, zone Normal, type Movable 25 11 345 87= 48 21 2 20 9 3 75061 > > Node 0, zone Normal, type Reclaimable 4 2 2 4= 3 0 2 1 1 1 0 > > Node 0, zone Normal, type HighAtomic 0 0 0 0= 0 0 0 0 0 0 0 > > ... > > Free pages count per migrate type at order 0 1 2 3= 4 5 6 7 8 9 10 > > Node 1, zone Normal, type Unmovable 98888 99650 99679 2= 3 1 2 2 2 0 0 > > Node 1, zone Normal, type Movable 1 1 0 1= 1 0 1 0 1 1 75937 > > Node 1, zone Normal, type Reclaimable 0 0 0 0= 0 0 0 0 0 0 0 > > Node 1, zone Normal, type HighAtomic 0 0 0 0= 0 0 0 0 0 0 0 > > > > After: > > Free pages count per migrate type at order 0 1 2 3= 4 5 6 7 8 9 10 > > ... > > Node 0, zone Normal, type Unmovable 152 158 37 2= 2 0 3 4 2 6 717 > > Node 0, zone Normal, type Movable 1 37 53 3= 55 49 16 6 2 1 75000 > > Node 0, zone Normal, type Reclaimable 1 4 3 1= 2 1 1 1 1 1 0 > > Node 0, zone Normal, type HighAtomic 0 0 0 0= 0 0 0 0 0 0 0 > > ... > > Free pages count per migrate type at order 0 1 2 3= 4 5 6 7 8 9 10 > > Node 1, zone Normal, type Unmovable 5 3 2 1= 3 4 2 2 2 0 779 > > Node 1, zone Normal, type Movable 1 0 1 1= 1 0 1 0 1 1 75849 > > Node 1, zone Normal, type Reclaimable 0 0 0 0= 0 0 0 0 0 0 0 > > Node 1, zone Normal, type HighAtomic 0 0 0 0= 0 0 0 0 0 0 0 > > A good result. But the subject could be changed to: > > "mm/hugetlb: perform vmemmap optimization batchly for specific no= de allocation" > > > > > Signed-off-by: suhua > > --- > > mm/hugetlb.c | 37 +++++++++++++++++++++++++++++++++---- > > 1 file changed, 33 insertions(+), 4 deletions(-) > > > > diff --git a/mm/hugetlb.c b/mm/hugetlb.c > > index 190fa05635f4..3441d380c90b 100644 > > --- a/mm/hugetlb.c > > +++ b/mm/hugetlb.c > > @@ -2077,6 +2077,24 @@ static struct folio *only_alloc_fresh_hugetlb_fo= lio(struct hstate *h, > > return folio; > > } > > > > +static struct folio *only_alloc_and_account_fresh_hugetlb_folio( > > + struct hstate *h, gfp_t gfp_mask, > > + int nid, nodemask_t *nmask) > > +{ > > + struct folio *folio; > > + > > + folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask,= NULL); > > + if (!folio) > > + return NULL; > > + > > + spin_lock_irq(&hugetlb_lock); > > + h->nr_huge_pages++; > > + h->nr_huge_pages_node[nid]++; > > + spin_unlock_irq(&hugetlb_lock); > > + > > + return folio; > > +} > > + > > /* > > * Common helper to allocate a fresh hugetlb page. All specific allocat= ors > > * should use this function to get new hugetlb pages > > @@ -3301,23 +3319,34 @@ static void __init hugetlb_hstate_alloc_pages_o= nenode(struct hstate *h, int nid) > > { > > unsigned long i; > > char buf[32]; > > + LIST_HEAD(folio_list); > > + struct folio *folio, *tmp_f; > > > > for (i =3D 0; i < h->max_huge_pages_node[nid]; ++i) { > > if (hstate_is_gigantic(h)) { > > if (!alloc_bootmem_huge_page(h, nid)) > > break; > > } else { > > - struct folio *folio; > > gfp_t gfp_mask =3D htlb_alloc_mask(h) | __GFP_THI= SNODE; > > > > - folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, = nid, > > - &node_states[N_MEMORY]); > > + folio =3D only_alloc_and_account_fresh_hugetlb_fo= lio(h, > > + gfp_mask, nid, &node_states[N_MEM= ORY]); > > I think we could use only_alloc_fresh_hugetlb_folio plus prep_and_add_all= ocated_folios > to achieve the same goal but more simpler, right? > > > if (!folio) > > break; > > - free_huge_folio(folio); /* free it into the hugep= age allocator */ > > + list_add(&folio->lru, &folio_list); > > } > > cond_resched(); > > } > > + > > + if (!list_empty(&folio_list)) { > > + /* Send list for bulk vmemmap optimization processing */ > > + hugetlb_vmemmap_optimize_folios(h, &folio_list); > > + > > + list_for_each_entry_safe(folio, tmp_f, &folio_list, lru) = { > > + free_huge_folio(folio); /* free it into the hugep= age allocator */ > > + } > > + } > > We could use prep_and_add_allocated_folios here. > > Thanks. > > > + > > if (i =3D=3D h->max_huge_pages_node[nid]) > > return; > > > > -- > > 2.34.1 > > >