From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 518EAC4332F for ; Wed, 19 Oct 2022 03:42:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CCD406B0075; Tue, 18 Oct 2022 23:42:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C7C636B0078; Tue, 18 Oct 2022 23:42:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AF6206B007B; Tue, 18 Oct 2022 23:42:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 9FA6A6B0075 for ; Tue, 18 Oct 2022 23:42:37 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 7804D16044A for ; Wed, 19 Oct 2022 03:42:37 +0000 (UTC) X-FDA: 80036301954.27.DA363D4 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by imf10.hostedemail.com (Postfix) with ESMTP id 0A0EDC0026 for ; Wed, 19 Oct 2022 03:42:35 +0000 (UTC) Received: from pps.filterd (m0109334.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 29J03wjr030247 for ; Tue, 18 Oct 2022 20:42:35 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding : content-type; s=facebook; bh=tEYaKDhGnah93pCbJzIZt981jxWGOApxCCyHZWI27w0=; b=GVp7KkPJnxsuIi11Js0538ku2p0dRnC61oZOv0dw5+sBZwj6MEwQOK881NeqUZI0Tzg8 ulUknSw/CmeRq6cM7cGCFW9gOMK9hfG/ztitFCGzC0AfGNXO1rxJ8EHvBdbEw/QxGWaR piHG5YtikadGIaeKLFzTkzg+64AtaOKgDeU= Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3k9tpehf8g-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Tue, 18 Oct 2022 20:42:34 -0700 Received: from twshared9269.07.ash9.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Tue, 18 Oct 2022 20:42:32 -0700 Received: by devvm6390.atn0.facebook.com (Postfix, from userid 352741) id 2AAC453324D8; Tue, 18 Oct 2022 20:42:22 -0700 (PDT) From: To: , CC: , , , Alexander Zhu Subject: [PATCH v4 3/3] mm: THP low utilization shrinker Date: Tue, 18 Oct 2022 20:42:20 -0700 Message-ID: X-Mailer: git-send-email 2.30.2 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-FB-Internal: Safe Content-Type: text/plain X-Proofpoint-ORIG-GUID: qpAw453BXAj4PvzTup53OXVCpK8QrjAA X-Proofpoint-GUID: qpAw453BXAj4PvzTup53OXVCpK8QrjAA X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1 definitions=2022-10-18_10,2022-10-18_01,2022-06-22_01 ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1666150956; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tEYaKDhGnah93pCbJzIZt981jxWGOApxCCyHZWI27w0=; b=SwDQ+b5TQr0reBIb+QA9m3X0/+pEaBaFVrsIjrUpTMIFvzOIEWXHx/G086ni75m6XmPPsp qzol92gtD9V/nFwg37WAWNTcsKbXeZq2PSefAMDE8PEfyJkFHcqwsPZOW17u41cdYGSFIm QEPjlMzJEcm7IYIk6HnZFuRF1LEOhAI= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=GVp7KkPJ; spf=pass (imf10.hostedemail.com: domain of "prvs=129168f899=alexlzhu@meta.com" designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=129168f899=alexlzhu@meta.com"; dmarc=pass (policy=reject) header.from=fb.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1666150956; a=rsa-sha256; cv=none; b=LK6CRP0zCTs6hYe9IxyjUqBH8NllyUsQzLItR0DnmJsc/ydKqj2AMlNaW9EyfCNIWvHhdF VWP47h7L9wEll9enP8nawKRBbOOl2KiS5zqGzvC4uCjZB76tHaIGrW00/pWbdOouN8cFBo Rh4mzivqAZdhj0mKJ/1s7gWKPm4/EU4= X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 0A0EDC0026 X-Rspam-User: Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=fb.com header.s=facebook header.b=GVp7KkPJ; spf=pass (imf10.hostedemail.com: domain of "prvs=129168f899=alexlzhu@meta.com" designates 67.231.145.42 as permitted sender) smtp.mailfrom="prvs=129168f899=alexlzhu@meta.com"; dmarc=pass (policy=reject) header.from=fb.com X-Stat-Signature: mf5t8sx7x9jksc3ywfgn45xgae7ebw9i X-HE-Tag: 1666150955-473731 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Alexander Zhu This patch introduces a shrinker that will remove THPs in the lowest utilization bucket. As previously mentioned, we have observed that almost all of the memory waste when THPs are always enabled is contained in the lowest utilization bucket. The shrinker will add these THPs to a list_lru and split anonymous THPs based off information from kswapd. It requires the changes from thp_utilization to identify the least utilized THPs, and the changes to split_huge_page to identify and free zero pages within THPs. Signed-off-by: Alexander Zhu --- v3 to v4 -added some comments regardling trylock -change the relock to be unconditional in low_util_free_page -only expose can_shrink_thp, abstract the thp_utilization and bucket logi= c to be private to mm/thp_utilization.c v2 to v3 -put_page() after trylock_page in low_util_free_page. put() to be called = after get() call=20 -removed spin_unlock_irq in low_util_free_page above LRU_SKIP. There was = a double unlock. =20 -moved spin_unlock_irq() to below list_lru_isolate() in low_util_free_pag= e. This is to shorten the critical section. -moved lock_page in add_underutilized_thp such that we only lock when all= ocating and adding to the list_lru =20 -removed list_lru_alloc in list_lru_add_page and list_lru_delete_page as = these are no longer needed.=20 v1 to v2 -Changed lru_lock to be irq safe. Added irq_save and restore around list_= lru adds/deletes. -Changed low_util_free_page() to trylock the page, and if it fails, unloc= k lru_lock and return LRU_SKIP. This is to avoid deadlock between reclaim= , which calls split_huge_page() and the THP Shrinker -Changed low_util_free_page() to unlock lru_lock, split_huge_page, then l= ock lru_lock. This way split_huge_page is not called with the lru_lock he= ld. That leads to deadlock as split_huge_page calls on_each_cpu_mask=20 -Changed list_lru_shrink_walk to list_lru_shrink_walk_irq.=20 RFC to v1 -Remove all THPs that are not in the top utilization bucket. This is what= we have found to perform the best in production testing, we have found t= hat there are an almost trivial number of THPs in the middle range of buc= kets that account for most of the memory waste.=20 -Added check for THP utilization prior to split_huge_page for the THP Shr= inker. This is to account for THPs that move to the top bucket, but were = underutilized at the time they were added to the list_lru.=20 -Multiply the shrink_count and scan_count by HPAGE_PMD_NR. This is becaus= e a THP is 512 pages, and should count as 512 objects in reclaim. This wa= y reclaim is triggered at a more appropriate frequency than in the RFC.=20 include/linux/huge_mm.h | 9 ++++ include/linux/list_lru.h | 24 +++++++++ include/linux/mm_types.h | 5 ++ mm/huge_memory.c | 110 ++++++++++++++++++++++++++++++++++++++- mm/list_lru.c | 49 +++++++++++++++++ mm/page_alloc.c | 6 +++ mm/thp_utilization.c | 13 +++++ 7 files changed, 214 insertions(+), 2 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index a1341fdcf666..1745c94eb103 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -178,6 +178,8 @@ bool hugepage_vma_check(struct vm_area_struct *vma, u= nsigned long vm_flags, unsigned long thp_get_unmapped_area(struct file *filp, unsigned long add= r, unsigned long len, unsigned long pgoff, unsigned long flags); =20 +bool can_shrink_thp(struct folio *folio); + void prep_transhuge_page(struct page *page); void free_transhuge_page(struct page *page); =20 @@ -189,6 +191,8 @@ static inline int split_huge_page(struct page *page) } void deferred_split_huge_page(struct page *page); =20 +void add_underutilized_thp(struct page *page); + void __split_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd, unsigned long address, bool freeze, struct folio *folio); =20 @@ -302,6 +306,11 @@ static inline struct list_head *page_deferred_list(s= truct page *page) return &page[2].deferred_list; } =20 +static inline struct list_head *page_underutilized_thp_list(struct page = *page) +{ + return &page[3].underutilized_thp_list; +} + #else /* CONFIG_TRANSPARENT_HUGEPAGE */ #define HPAGE_PMD_SHIFT ({ BUILD_BUG(); 0; }) #define HPAGE_PMD_MASK ({ BUILD_BUG(); 0; }) diff --git a/include/linux/list_lru.h b/include/linux/list_lru.h index b35968ee9fb5..c2cf146ea880 100644 --- a/include/linux/list_lru.h +++ b/include/linux/list_lru.h @@ -89,6 +89,18 @@ void memcg_reparent_list_lrus(struct mem_cgroup *memcg= , struct mem_cgroup *paren */ bool list_lru_add(struct list_lru *lru, struct list_head *item); =20 +/** + * list_lru_add_page: add an element to the lru list's tail + * @list_lru: the lru pointer + * @page: the page containing the item + * @item: the item to be deleted. + * + * This function works the same as list_lru_add in terms of list + * manipulation. Used for non slab objects contained in the page. + * + * Return value: true if the list was updated, false otherwise + */ +bool list_lru_add_page(struct list_lru *lru, struct page *page, struct l= ist_head *item); /** * list_lru_del: delete an element to the lru list * @list_lru: the lru pointer @@ -102,6 +114,18 @@ bool list_lru_add(struct list_lru *lru, struct list_= head *item); */ bool list_lru_del(struct list_lru *lru, struct list_head *item); =20 +/** + * list_lru_del_page: delete an element to the lru list + * @list_lru: the lru pointer + * @page: the page containing the item + * @item: the item to be deleted. + * + * This function works the same as list_lru_del in terms of list + * manipulation. Used for non slab objects contained in the page. + * + * Return value: true if the list was updated, false otherwise + */ +bool list_lru_del_page(struct list_lru *lru, struct page *page, struct l= ist_head *item); /** * list_lru_count_one: return the number of objects currently held by @l= ru * @lru: the lru pointer. diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 500e536796ca..da1d1cf42158 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -152,6 +152,11 @@ struct page { /* For both global and memcg */ struct list_head deferred_list; }; + struct { /* Third tail page of compound page */ + unsigned long _compound_pad_3; /* compound_head */ + unsigned long _compound_pad_4; + struct list_head underutilized_thp_list; + }; struct { /* Page table pages */ unsigned long _pt_pad_1; /* compound_head */ pgtable_t pmd_huge_pte; /* protected by page->ptl */ diff --git a/mm/huge_memory.c b/mm/huge_memory.c index f68a353e0adf..76d39ceceb05 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -71,6 +71,8 @@ static atomic_t huge_zero_refcount; struct page *huge_zero_page __read_mostly; unsigned long huge_zero_pfn __read_mostly =3D ~0UL; =20 +static struct list_lru huge_low_util_page_lru; + bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_fla= gs, bool smaps, bool in_pf, bool enforce_sysfs) { @@ -234,6 +236,53 @@ static struct shrinker huge_zero_page_shrinker =3D { .seeks =3D DEFAULT_SEEKS, }; =20 +static enum lru_status low_util_free_page(struct list_head *item, + struct list_lru_one *lru, + spinlock_t *lru_lock, + void *cb_arg) +{ + struct folio *folio =3D lru_to_folio(item); + struct page *head =3D &folio->page; + + if (get_page_unless_zero(head)) { + /* Inverse lock order from add_underutilized_thp() */ + if (!trylock_page(head)) { + put_page(head); + return LRU_SKIP; + } + list_lru_isolate(lru, item); + spin_unlock_irq(lru_lock); + if (can_shrink_thp(folio)) + split_huge_page(head); + spin_lock_irq(lru_lock); + unlock_page(head); + put_page(head); + } + + return LRU_REMOVED_RETRY; +} + +static unsigned long shrink_huge_low_util_page_count(struct shrinker *sh= rink, + struct shrink_control *sc) +{ + return HPAGE_PMD_NR * list_lru_shrink_count(&huge_low_util_page_lru, sc= ); +} + +static unsigned long shrink_huge_low_util_page_scan(struct shrinker *shr= ink, + struct shrink_control *sc) +{ + return HPAGE_PMD_NR * list_lru_shrink_walk_irq(&huge_low_util_page_lru, + sc, low_util_free_page, NULL); +} + +static struct shrinker huge_low_util_page_shrinker =3D { + .count_objects =3D shrink_huge_low_util_page_count, + .scan_objects =3D shrink_huge_low_util_page_scan, + .seeks =3D DEFAULT_SEEKS, + .flags =3D SHRINKER_NUMA_AWARE | SHRINKER_MEMCG_AWARE | + SHRINKER_NONSLAB, +}; + #ifdef CONFIG_SYSFS static ssize_t enabled_show(struct kobject *kobj, struct kobj_attribute *attr, char *buf) @@ -485,6 +534,9 @@ static int __init hugepage_init(void) if (err) goto err_slab; =20 + err =3D register_shrinker(&huge_low_util_page_shrinker, "thp-low-util")= ; + if (err) + goto err_low_util_shrinker; err =3D register_shrinker(&huge_zero_page_shrinker, "thp-zero"); if (err) goto err_hzp_shrinker; @@ -492,6 +544,9 @@ static int __init hugepage_init(void) if (err) goto err_split_shrinker; =20 + err =3D list_lru_init_memcg(&huge_low_util_page_lru, &huge_low_util_pag= e_shrinker); + if (err) + goto err_low_util_list_lru; /* * By default disable transparent hugepages on smaller systems, * where the extra memory used could hurt more than TLB overhead @@ -508,10 +563,14 @@ static int __init hugepage_init(void) =20 return 0; err_khugepaged: + list_lru_destroy(&huge_low_util_page_lru); +err_low_util_list_lru: unregister_shrinker(&deferred_split_shrinker); err_split_shrinker: unregister_shrinker(&huge_zero_page_shrinker); err_hzp_shrinker: + unregister_shrinker(&huge_low_util_page_shrinker); +err_low_util_shrinker: khugepaged_destroy(); err_slab: hugepage_exit_sysfs(hugepage_kobj); @@ -586,6 +645,7 @@ void prep_transhuge_page(struct page *page) */ =20 INIT_LIST_HEAD(page_deferred_list(page)); + INIT_LIST_HEAD(page_underutilized_thp_list(page)); set_compound_page_dtor(page, TRANSHUGE_PAGE_DTOR); } =20 @@ -2451,8 +2511,7 @@ static void __split_huge_page_tail(struct page *hea= d, int tail, LRU_GEN_MASK | LRU_REFS_MASK)); =20 /* ->mapping in first tail page is compound_mapcount */ - VM_BUG_ON_PAGE(tail > 2 && page_tail->mapping !=3D TAIL_MAPPING, - page_tail); + VM_BUG_ON_PAGE(tail > 3 && page_tail->mapping !=3D TAIL_MAPPING, page_t= ail); page_tail->mapping =3D head->mapping; page_tail->index =3D head->index + tail; page_tail->private =3D 0; @@ -2660,6 +2719,7 @@ int split_huge_page_to_list(struct page *page, stru= ct list_head *list) struct folio *folio =3D page_folio(page); struct deferred_split *ds_queue =3D get_deferred_split_queue(&folio->pa= ge); XA_STATE(xas, &folio->mapping->i_pages, folio->index); + struct list_head *underutilized_thp_list =3D page_underutilized_thp_lis= t(&folio->page); struct anon_vma *anon_vma =3D NULL; struct address_space *mapping =3D NULL; int extra_pins, ret; @@ -2767,6 +2827,10 @@ int split_huge_page_to_list(struct page *page, str= uct list_head *list) list_del(page_deferred_list(&folio->page)); } spin_unlock(&ds_queue->split_queue_lock); + /* Frozen refs lock out additions, test can be lockless */ + if (!list_empty(underutilized_thp_list)) + list_lru_del_page(&huge_low_util_page_lru, &folio->page, + underutilized_thp_list); if (mapping) { int nr =3D folio_nr_pages(folio); =20 @@ -2809,6 +2873,7 @@ int split_huge_page_to_list(struct page *page, stru= ct list_head *list) void free_transhuge_page(struct page *page) { struct deferred_split *ds_queue =3D get_deferred_split_queue(page); + struct list_head *underutilized_thp_list =3D page_underutilized_thp_lis= t(page); unsigned long flags; =20 spin_lock_irqsave(&ds_queue->split_queue_lock, flags); @@ -2817,6 +2882,13 @@ void free_transhuge_page(struct page *page) list_del(page_deferred_list(page)); } spin_unlock_irqrestore(&ds_queue->split_queue_lock, flags); + /* A dead page cannot be re-added to the THP shrinker, test can be lock= less */ + if (!list_empty(underutilized_thp_list)) + list_lru_del_page(&huge_low_util_page_lru, page, underutilized_thp_lis= t); + + if (PageLRU(page)) + __folio_clear_lru_flags(page_folio(page)); + free_compound_page(page); } =20 @@ -2857,6 +2929,40 @@ void deferred_split_huge_page(struct page *page) spin_unlock_irqrestore(&ds_queue->split_queue_lock, flags); } =20 +void add_underutilized_thp(struct page *page) +{ + VM_BUG_ON_PAGE(!PageTransHuge(page), page); + VM_BUG_ON_PAGE(!PageAnon(page), page); + + if (PageSwapCache(page)) + return; + + /* + * Need to take a reference on the page to prevent the page from gettin= g free'd from + * under us while we are adding the THP to the shrinker. + */ + if (!get_page_unless_zero(page)) + return; + + if (is_huge_zero_page(page)) + goto out_put; + + /* Stabilize page->memcg to allocate and add to the same list */ + lock_page(page); + +#ifdef CONFIG_MEMCG_KMEM + if (memcg_list_lru_alloc(page_memcg(page), &huge_low_util_page_lru, GFP= _KERNEL)) + goto out_unlock; +#endif + + list_lru_add_page(&huge_low_util_page_lru, page, page_underutilized_thp= _list(page)); + +out_unlock: + unlock_page(page); +out_put: + put_page(page); +} + static unsigned long deferred_split_count(struct shrinker *shrink, struct shrink_control *sc) { diff --git a/mm/list_lru.c b/mm/list_lru.c index a05e5bef3b40..8cc56a84b554 100644 --- a/mm/list_lru.c +++ b/mm/list_lru.c @@ -140,6 +140,32 @@ bool list_lru_add(struct list_lru *lru, struct list_= head *item) } EXPORT_SYMBOL_GPL(list_lru_add); =20 +bool list_lru_add_page(struct list_lru *lru, struct page *page, struct l= ist_head *item) +{ + int nid =3D page_to_nid(page); + struct list_lru_node *nlru =3D &lru->node[nid]; + struct list_lru_one *l; + struct mem_cgroup *memcg; + unsigned long flags; + + spin_lock_irqsave(&nlru->lock, flags); + if (list_empty(item)) { + memcg =3D page_memcg(page); + l =3D list_lru_from_memcg_idx(lru, nid, memcg_kmem_id(memcg)); + list_add_tail(item, &l->list); + /* Set shrinker bit if the first element was added */ + if (!l->nr_items++) + set_shrinker_bit(memcg, nid, + lru_shrinker_id(lru)); + nlru->nr_items++; + spin_unlock_irqrestore(&nlru->lock, flags); + return true; + } + spin_unlock_irqrestore(&nlru->lock, flags); + return false; +} +EXPORT_SYMBOL_GPL(list_lru_add_page); + bool list_lru_del(struct list_lru *lru, struct list_head *item) { int nid =3D page_to_nid(virt_to_page(item)); @@ -160,6 +186,29 @@ bool list_lru_del(struct list_lru *lru, struct list_= head *item) } EXPORT_SYMBOL_GPL(list_lru_del); =20 +bool list_lru_del_page(struct list_lru *lru, struct page *page, struct l= ist_head *item) +{ + int nid =3D page_to_nid(page); + struct list_lru_node *nlru =3D &lru->node[nid]; + struct list_lru_one *l; + struct mem_cgroup *memcg; + unsigned long flags; + + spin_lock_irqsave(&nlru->lock, flags); + if (!list_empty(item)) { + memcg =3D page_memcg(page); + l =3D list_lru_from_memcg_idx(lru, nid, memcg_kmem_id(memcg)); + list_del_init(item); + l->nr_items--; + nlru->nr_items--; + spin_unlock_irqrestore(&nlru->lock, flags); + return true; + } + spin_unlock_irqrestore(&nlru->lock, flags); + return false; +} +EXPORT_SYMBOL_GPL(list_lru_del_page); + void list_lru_isolate(struct list_lru_one *list, struct list_head *item) { list_del_init(item); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index e20ade858e71..31380526a9f4 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1335,6 +1335,12 @@ static int free_tail_pages_check(struct page *head= _page, struct page *page) * deferred_list.next -- ignore value. */ break; + case 3: + /* + * the third tail page: ->mapping is + * underutilized_thp_list.next -- ignore value. + */ + break; default: if (page->mapping !=3D TAIL_MAPPING) { bad_page(page, "corrupted mapping in tail page"); diff --git a/mm/thp_utilization.c b/mm/thp_utilization.c index 7b79f8759d12..d0efcffde50a 100644 --- a/mm/thp_utilization.c +++ b/mm/thp_utilization.c @@ -113,6 +113,19 @@ static int thp_number_utilized_pages(struct folio *f= olio) return thp_nr_utilized_pages; } =20 +bool can_shrink_thp(struct folio *folio) +{ + int bucket, num_utilized_pages; + + if (!folio || !folio_test_anon(folio) || !folio_test_transhuge(folio)) + return false; + + num_utilized_pages =3D thp_number_utilized_pages(folio); + bucket =3D thp_utilization_bucket(num_utilized_pages); + + return bucket < THP_UTIL_BUCKET_NR - 1; +} + static void thp_scan_next_zone(void) { struct timespec64 current_time; --=20 2.30.2