From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8AC9DD1A441 for ; Sat, 12 Oct 2024 07:08:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8D09B6B0092; Sat, 12 Oct 2024 03:08:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 87DEE6B0093; Sat, 12 Oct 2024 03:08:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 745576B0095; Sat, 12 Oct 2024 03:08:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 57C266B0092 for ; Sat, 12 Oct 2024 03:08:12 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 31406AD126 for ; Sat, 12 Oct 2024 07:08:01 +0000 (UTC) X-FDA: 82664071140.04.E033DA9 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) by imf10.hostedemail.com (Postfix) with ESMTP id 46B3BC0018 for ; Sat, 12 Oct 2024 07:08:08 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=EROYGIRL; spf=pass (imf10.hostedemail.com: domain of suhua.tanke@gmail.com designates 209.85.214.170 as permitted sender) smtp.mailfrom=suhua.tanke@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1728716818; a=rsa-sha256; cv=none; b=qb9oaFicYwvcdkBpkCrnGvqcVgIJ7I1ADhtNS+YSlmPZjaWaDfWBgc3i5wA1seTaPRx9/8 9iOF35xLj0g/Ngd5sBDtgzzgW0eF3TspjPsGNh2lwTYoQJUb8ZbLyXc3IiQmG/NSncBIs3 se2m//aecmJXpkhMczJFx63uTbCdwHc= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=EROYGIRL; spf=pass (imf10.hostedemail.com: domain of suhua.tanke@gmail.com designates 209.85.214.170 as permitted sender) smtp.mailfrom=suhua.tanke@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1728716818; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=DZmMCnNR2UPbWiVSXiubgc7IFfujo/BpYLBWN53BWGg=; b=w0bGUkZR5ZdNls3DkGsDX33EmIKeU3TWKhI3nQNes6roOMsJdfclyBooNmxCrkt95DNoa4 28iAaYVgMhhgGtFUkS2DYbU3do+wPzz/rORpZUt+MSmUmmJLlX8aSnPS1JtOP79buGZHbA D71aK+jEwTwWH7XXKmdqcXfH6j2hPnM= Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-20caea61132so9708785ad.2 for ; Sat, 12 Oct 2024 00:08:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1728716888; x=1729321688; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=DZmMCnNR2UPbWiVSXiubgc7IFfujo/BpYLBWN53BWGg=; b=EROYGIRLejFhfjMHdYyfcDL2iklqr1JKoEixz7YRvZvW6VICWR1iSnHAV8oLXR9Uqp oXFoXVPr8r28DvL8HwYpvPh7mIgvhQ5LwRZ0BjigWum1MFAzj3dMvrDVdCf72At3wVz+ pLFhKLFw1JlTy0l48DU4DHhjT8bznbbAYvI0V0OTWbzhU+xcXSBYXoHkheYDciBv5laL tU27nLCOny5j93NGU5ZHpvHC372/2kC6iIdNQUfvjZpJVwoXqKyULxC5StDeEW6QTkvg kRSSCmSjNbxMNhVniF5jtbWp/QbwTrOx//5KelDhBWI+qapjNMtswHxPR1teYCZJCWXi a+Jw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728716888; x=1729321688; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=DZmMCnNR2UPbWiVSXiubgc7IFfujo/BpYLBWN53BWGg=; b=LHuOpAq7x6541cODc/4jRpkweKJvtHnnFPJjWoiC3El0uEVgE5OYEV2+K9EYA7KsE3 ZKYJInLDVZr16WJm0s9gsuq3lPL8vr5YpD/StlSq3nFe8GrK28Hr244MsCA3oXSSd+9Q 7Ps9yj1/vr/fXM62EKkHvnMYvr1OkH9mH5Mh93fq+vOpyep3tEtUPymCHZAhSn/rR6b+ QSGM37TxKwbPu1WpaMyWeq3WZq7mtltGrM8uBPLGX9pmuP0t9cbVEVh71PYBvD1HGCU+ u+YMsQQEsvpFcv+1YzSv/Kd1O7XLU4cf3UZYB7owTvLXHVntLjDmIvWbo7xj7KPPtVCf Hv+Q== X-Gm-Message-State: AOJu0YyO6d4heLDwb/ZH3YfP9AqWQjXsgfe69dO60nNDD3xUaxabUvp3 9pbkBzCxdlNk17cR8NIS0t6dYDbADjvqFE2deAdXEoNxcIDZXqfR X-Google-Smtp-Source: AGHT+IFMmUxALkINzQWN8gf7Q3YK+P75Vuow7GtzNSGuILQu16IGYnty2dUkcPld7M0nulZzMOwUew== X-Received: by 2002:a17:902:e5cd:b0:20c:8dff:b4ed with SMTP id d9443c01a7336-20cbb1bb3a4mr31421915ad.16.1728716888441; Sat, 12 Oct 2024 00:08:08 -0700 (PDT) Received: from suhua-virtual-machine.localdomain ([36.112.24.11]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-20c8c33ff19sm32837545ad.272.2024.10.12.00.08.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 12 Oct 2024 00:08:07 -0700 (PDT) From: suhua X-Google-Original-From: suhua To: muchun.song@linux.dev, akpm@linux-foundation.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, suhua.tanke@gmail.com, suhua Subject: [PATCH] mm/hugetlb: Perform vmemmap optimization batchly for specific node allocation Date: Sat, 12 Oct 2024 15:08:02 +0800 Message-Id: <20241012070802.1876-1-suhua1@kingsoft.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Stat-Signature: kbcdzdrw5two9c8y9ipntsqj14nc55p7 X-Rspamd-Queue-Id: 46B3BC0018 X-Rspam-User: X-Rspamd-Server: rspam10 X-HE-Tag: 1728716888-883067 X-HE-Meta: U2FsdGVkX1+m3PbXmEJdQ6Dr7n199dy7DnOe+2B0OVEs3UCXqdnAEIKZatHWTsuGqwU2dSK+yojKkTdZlTPv2solQiQLyg5uftYKTjf0BQ0bLm7Gj7s81+Gv4MVvKHuCxS3Tzu4DCZqdQlUJgJ2szD1ceTV5Wu0fue/A/CzjLL+7v7x5NMMv7/9GatNKhDWSmFWi7zYmckqqPkLkIjtXObFmE9ZQ3qrtswkKKTgRoUA+B0Z0QLGW3NrOnnrjDDuF7atICH4qIX7MIk7Np7ZiJmCIgKkIA58/50P5EMYUcI53icLDCSQxjeFTRxlMHBSyrSrZecpqmxHIBJgqVqfo5h86MYgrMhVe0vhtgtRCktqXgOdrrg2ZSWXzMD17f8uyNOtmMHDKkqUPbgZnoZ3IBCiMx+4slDbetz9ij8YTs4Axt9+52t2MFvnhECju1R6TIMC1zIiciNL4yDSkCPXh8PCg7gNcuJNMLOl+RzdktEzaXXpSBRiuNMWg2qDPdWZ8lFq1nmWGvopHLIUMo0H4mtPrd/JeiMRoi8GEhRq7YopAAIWeVfVBF7IK242T5TRZKhxeii8efxnhTThLNVI6b2SGr6PDfcEwD+pSbS4jRmlhuSp7bXVFRhV1TfNAvSVl5ltxTXg9B8im+oCs6J8aOVlsqmjOFxI3aCVnx9bSGgsUexQsVtQYM4kSk3+wk+2iyvp+7laZOn1ijFGPftMz7WezNxMjWuSohqgpOydx2H1R5e0Jz9SeCLgX85WlvmHOsJ+la3WZBxN/lo5Pw44O0VVrDNojml22xKhBTMml/Gar5GigsmyR2tTYC80HQWhk9lnbOT5tf6xV28MfBTtAcN/KWAqUsPnDW7suLC+rDI866jH4+KnAFFUL8P1O46VPM2rSfeZIn/L3mlQ53RVZRMUdLei4fDoyUZLuRT0bEQX0zHAdw59GXgoEK0+YPi66SYiaXKasiJPfdcXFxy1 P/YK2dUi YGpC72E4Ql7C8WXph7ebywr8yeIhRydG9VHIXRdsgGU9ry2UXZYY81jzddS5ZJhVFsL35Csq3yQlo7ctTBpxravMnKIWnPMvKS6BJIyQYjLQn8zTLZfJxwr7YBH15n+a0rvNVDyeGoTD8uQX4ZhJ1uy4zpCIJhINTn1R9fV3Z3jqiEpter6T7vDfwbLqDPnJWWDYG/WHcRJZ4qZEASuEYWZdhx4VCGVgvTPOT5Y0q5VWgz5o7PhN0fUXZGEL6VLf7vn+NN8u44HolGh1T2Ra81znEzXBt7uoeLT3CVjsCkv97W/3bcRckPXuhalCFQR2bXtTU02D7YCXqEs6Y3R9p3Q5oiXYXRlBBuiTk6inQV+DhzahobEHlemmF3In13/MnsTAcqV4D+DDyuhMtzH9cNttN04+FRar2kXa//6FdE2QD8Km7wMYUHJdc+BXJ1RFe+W0emnsxo5HSmlmqPT8vYt2KdLWYL0dy4Sf9phi1es+AilCDo8pKUbiuhHQBU77YYKaE X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When HVO is enabled and huge page memory allocs are made, the freed memory can be aggregated into higher order memory in the following paths, which facilitates further allocs for higher order memory. echo 200000 > /proc/sys/vm/nr_hugepages echo 200000 > /sys/devices/system/node/node*/hugepages/hugepages-2048kB/nr_hugepages grub： default_hugepagesz=2M hugepagesz=2M hugepages=200000 Currently not support for releasing aggregations to higher order in the following way, which will releasing to lower order. grub: default_hugepagesz=2M hugepagesz=2M hugepages=0:100000,1:100000 This patch supports the release of huge page optimizations aggregates to higher order memory. eg: cat /proc/cmdline BOOT_IMAGE=/boot/vmlinuz-xxx ... default_hugepagesz=2M hugepagesz=2M hugepages=0:100000,1:100000 Before: Free pages count per migrate type at order 0 1 2 3 4 5 6 7 8 9 10 ... Node 0, zone Normal, type Unmovable 55282 97039 99307 0 1 1 0 1 1 1 0 Node 0, zone Normal, type Movable 25 11 345 87 48 21 2 20 9 3 75061 Node 0, zone Normal, type Reclaimable 4 2 2 4 3 0 2 1 1 1 0 Node 0, zone Normal, type HighAtomic 0 0 0 0 0 0 0 0 0 0 0 ... Free pages count per migrate type at order 0 1 2 3 4 5 6 7 8 9 10 Node 1, zone Normal, type Unmovable 98888 99650 99679 2 3 1 2 2 2 0 0 Node 1, zone Normal, type Movable 1 1 0 1 1 0 1 0 1 1 75937 Node 1, zone Normal, type Reclaimable 0 0 0 0 0 0 0 0 0 0 0 Node 1, zone Normal, type HighAtomic 0 0 0 0 0 0 0 0 0 0 0 After: Free pages count per migrate type at order 0 1 2 3 4 5 6 7 8 9 10 ... Node 0, zone Normal, type Unmovable 152 158 37 2 2 0 3 4 2 6 717 Node 0, zone Normal, type Movable 1 37 53 3 55 49 16 6 2 1 75000 Node 0, zone Normal, type Reclaimable 1 4 3 1 2 1 1 1 1 1 0 Node 0, zone Normal, type HighAtomic 0 0 0 0 0 0 0 0 0 0 0 ... Free pages count per migrate type at order 0 1 2 3 4 5 6 7 8 9 10 Node 1, zone Normal, type Unmovable 5 3 2 1 3 4 2 2 2 0 779 Node 1, zone Normal, type Movable 1 0 1 1 1 0 1 0 1 1 75849 Node 1, zone Normal, type Reclaimable 0 0 0 0 0 0 0 0 0 0 0 Node 1, zone Normal, type HighAtomic 0 0 0 0 0 0 0 0 0 0 0 Signed-off-by: suhua --- mm/hugetlb.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 190fa05635f4..906294ac85dc 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3301,6 +3301,7 @@ static void __init hugetlb_hstate_alloc_pages_onenode(struct hstate *h, int nid) { unsigned long i; char buf[32]; + LIST_HEAD(folio_list); for (i = 0; i < h->max_huge_pages_node[nid]; ++i) { if (hstate_is_gigantic(h)) { @@ -3310,14 +3311,18 @@ static void __init hugetlb_hstate_alloc_pages_onenode(struct hstate *h, int nid) struct folio *folio; gfp_t gfp_mask = htlb_alloc_mask(h) | __GFP_THISNODE; - folio = alloc_fresh_hugetlb_folio(h, gfp_mask, nid, - &node_states[N_MEMORY]); + folio = only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, + &node_states[N_MEMORY], NULL); if (!folio) break; - free_huge_folio(folio); /* free it into the hugepage allocator */ + list_add(&folio->lru, &folio_list); } cond_resched(); } + + if (!list_empty(&folio_list)) + prep_and_add_allocated_folios(h, &folio_list); + if (i == h->max_huge_pages_node[nid]) return; -- 2.34.1