From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF749C4332F for ; Tue, 18 Oct 2022 04:29:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F0C306B0072; Tue, 18 Oct 2022 00:29:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EBAAC6B0075; Tue, 18 Oct 2022 00:29:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D5C6F6B0078; Tue, 18 Oct 2022 00:29:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id C75B66B0072 for ; Tue, 18 Oct 2022 00:29:24 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id A1D0AA0668 for ; Tue, 18 Oct 2022 04:29:24 +0000 (UTC) X-FDA: 80032791048.10.1769BE7 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by imf24.hostedemail.com (Postfix) with ESMTP id 71DC9180034 for ; Tue, 18 Oct 2022 04:29:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1666067363; x=1697603363; h=from:to:cc:subject:references:date:in-reply-to: message-id:mime-version:content-transfer-encoding; bh=WT5xBVv0590oScFKQVUC9PdcDlnAOTuUcrV3Am9jQOI=; b=YKb468xmiAZw/poTqcVwJAGKq1vJvI4krn8v1duP/hqfjBPWbB1ax+vE DxajbwiXHCoKE4MNRqirYWtCxXEqCPNPdu+ulCR41Y2tZHwhaKfDuNHL4 6zJBogHJYgpfxgy1YaJPjnkej4QqVXo1CqpqEJRDTCw8FDEthk1VBmFbk X+fuwCCyoQ5e7ke6MZNpk3FZNNMGK7yLsiMmYOqsoMfOznH8QkuWW/t+W KFxyeuccGtiRPJXAUrI8NXVBPoZil3kiq8acxkhA579IbpFZBvI+nIg3r Q12IBveMMt/eq4WD82mZwnKHeP4/4QkwUUeYlYis/XWSydHi5JcsYSiF1 Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10503"; a="285714615" X-IronPort-AV: E=Sophos;i="5.95,193,1661842800"; d="scan'208";a="285714615" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Oct 2022 21:29:21 -0700 X-IronPort-AV: E=McAfee;i="6500,9779,10503"; a="659625088" X-IronPort-AV: E=Sophos;i="5.95,193,1661842800"; d="scan'208";a="659625088" Received: from yhuang6-desk2.sh.intel.com (HELO yhuang6-desk2.ccr.corp.intel.com) ([10.238.208.55]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 17 Oct 2022 21:29:19 -0700 From: "Huang, Ying" To: "Alex Zhu (Kernel)" Cc: "Kirill A. Shutemov" , "linux-mm@kvack.org" , Kernel Team , "willy@infradead.org" , "hannes@cmpxchg.org" , "riel@surriel.com" Subject: Re: [PATCH v3 1/3] mm: add thp_utilization metrics to debugfs References: <1546999e21a418c2510b3ed02b2b1f76b2b0f5b7.1665614216.git.alexlzhu@fb.com> <20221013113548.65ofywprrjepk2t4@box.shutemov.name> <9891C13C-DC07-4372-B346-C353CE02B205@fb.com> Date: Tue, 18 Oct 2022 12:28:37 +0800 In-Reply-To: <9891C13C-DC07-4372-B346-C353CE02B205@fb.com> (Alex Zhu's message of "Thu, 13 Oct 2022 22:53:39 +0000") Message-ID: <87mt9tsqkq.fsf@yhuang6-desk2.ccr.corp.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=YKb468xm; spf=pass (imf24.hostedemail.com: domain of ying.huang@intel.com designates 192.55.52.136 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1666067364; a=rsa-sha256; cv=none; b=Ax2ndBpuJ8IvOtQGUkQdBOjjSFraFNij6t8y0RvY3GJm5YXAcE9sNtXIKtzO1VISlGc2Go pZi/upYe4oB13MS6w0TtoWMSWFedbouEpOPRF/Da7MhMc6qdJ/P8s3dhaLc5dhYw6zXBTw gSS/JP1UnBJ8dOhpRRBuQ6HVXRChKg0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1666067364; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=C3YOd+86xSFHsH926vRJKvxwUr6/8W1/Hb3FH7B3mAo=; b=h3CamioGR5GtDentyETlxMNGKlkUPaLfJHb31ZP/5hbrEQ1nMX5aIdIyHP4esmd2r15PTU wlaiN4pn2ji6Nf0t8ExS51o6n0GVmFaoA9eUIUn4XpCa/OMzGpy+MCKT2cVSCJFYYKSaX3 m/m7yO3LJC7NZyi05bxzho2iB1pViR8= X-Stat-Signature: 11do3fmg8tenq17etxj81j64othhc976 X-Rspamd-Queue-Id: 71DC9180034 Authentication-Results: imf24.hostedemail.com; dkim=none ("invalid DKIM record") header.d=intel.com header.s=Intel header.b=YKb468xm; spf=pass (imf24.hostedemail.com: domain of ying.huang@intel.com designates 192.55.52.136 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (policy=none) header.from=intel.com X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1666067363-836350 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: "Alex Zhu (Kernel)" writes: >> On Oct 13, 2022, at 4:35 AM, Kirill A. Shutemov w= rote: >>=20 >> On Wed, Oct 12, 2022 at 03:51:45PM -0700, alexlzhu@fb.com wrote: >>> From: Alexander Zhu >>>=20 >>> This change introduces a tool that scans through all of physical >>> memory for anonymous THPs and groups them into buckets based >>> on utilization. It also includes an interface under >>> /sys/kernel/debug/thp_utilization. >>>=20 >>> Sample Output: >>>=20 >>> Utilized[0-50]: 1331 680884 >>> Utilized[51-101]: 9 3983 >>> Utilized[102-152]: 3 1187 >>> Utilized[153-203]: 0 0 >>> Utilized[204-255]: 2 539 >>> Utilized[256-306]: 5 1135 >>> Utilized[307-357]: 1 192 >>> Utilized[358-408]: 0 0 >>> Utilized[409-459]: 1 57 >>> Utilized[460-512]: 400 13 >>> Last Scan Time: 223.98s >>> Last Scan Duration: 70.65s >>>=20 >>> This indicates that there are 1331 THPs that have between 0 and 50 >>> utilized (non zero) pages. In total there are 680884 zero pages in >>> this utilization bucket. THPs in the [0-50] bucket compose 76% of total >>> THPs, and are responsible for 99% of total zero pages across all >>> THPs. In other words, the least utilized THPs are responsible for almost >>> all of the memory waste when THP is always enabled. Similar results >>> have been observed across production workloads. >>>=20 >>> The last two lines indicate the timestamp and duration of the most rece= nt >>> scan through all of physical memory. Here we see that the last scan >>> occurred 223.98 seconds after boot time and took 70.65 seconds. >>>=20 >>> Utilization of a THP is defined as the percentage of nonzero >>> pages in the THP. The worker thread will scan through all >>> of physical memory and obtain utilization of all anonymous >>> THPs. It will gather this information by periodically scanning >>> through all of physical memory for anonymous THPs, group them >>> into buckets based on utilization, and report utilization >>> information through debugfs under /sys/kernel/debug/thp_utilization. >>>=20 >>> Signed-off-by: Alexander Zhu >>> --- >>> v1 to v2 >>> -reversed ordering of is_transparent_hugepage and PageAnon in is_anon_t= ransparent_hugepage, page->mapping is only meaningful for user pages >>>=20 >>> RFC to v1 >>> -Refactored out the code to obtain the thp_utilization_bucket, as that = now has to be used in multiple places. >>>=20 >>> Documentation/admin-guide/mm/transhuge.rst | 9 + >>> include/linux/huge_mm.h | 3 + >>> mm/huge_memory.c | 202 +++++++++++++++++++++ >>=20 >> Please, consider putting thp_scan functionality into a separate file. >> mm/thp_scan.c or something. > > I=E2=80=99ll consider it. Do you think this is necessary? It is huge page= related, but huge_memory has a lot of code already.=20 >>=20 >>> 3 files changed, 214 insertions(+) >>>=20 >>> diff --git a/Documentation/admin-guide/mm/transhuge.rst b/Documentation= /admin-guide/mm/transhuge.rst >>> index 8ee78ec232eb..21d86303c97e 100644 >>> --- a/Documentation/admin-guide/mm/transhuge.rst >>> +++ b/Documentation/admin-guide/mm/transhuge.rst >>> @@ -304,6 +304,15 @@ To identify what applications are mapping file tra= nsparent huge pages, it >>> is necessary to read ``/proc/PID/smaps`` and count the FileHugeMapped f= ields >>> for each mapping. >>>=20 >>> +The utilization of transparent hugepages can be viewed by reading >>> +``/sys/kernel/debug/thp_utilization``. The utilization of a THP is def= ined >>> +as the ratio of non zero filled 4kb pages to the total number of pages= in a >>> +THP. The buckets are labelled by the range of total utilized 4kb pages= with >>> +one line per utilization bucket. Each line contains the total number of >>> +THPs in that bucket and the total number of zero filled 4kb pages summ= ed >>> +over all THPs in that bucket. The last two lines show the timestamp and >>> +duration respectively of the most recent scan over all of physical mem= ory. >>> + >>=20 >> debugfs as a primary interface? Looks wrong to me. > > Where would you recommend? We had initially put it under /proc, and then = moved to debugfs.=20 > >>=20 >>> Note that reading the smaps file is expensive and reading it >>> frequently will incur overhead. >>>=20 >>> diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h >>> index a1341fdcf666..13ac7b2f29ae 100644 >>> --- a/include/linux/huge_mm.h >>> +++ b/include/linux/huge_mm.h >>> @@ -178,6 +178,9 @@ bool hugepage_vma_check(struct vm_area_struct *vma,= unsigned long vm_flags, >>> unsigned long thp_get_unmapped_area(struct file *filp, unsigned long ad= dr, >>> unsigned long len, unsigned long pgoff, unsigned long flags); >>>=20 >>> +int thp_number_utilized_pages(struct page *page); >>> +int thp_utilization_bucket(int num_utilized_pages); >>> + >>> void prep_transhuge_page(struct page *page); >>> void free_transhuge_page(struct page *page); >>>=20 >>> diff --git a/mm/huge_memory.c b/mm/huge_memory.c >>> index 1cc4a5f4791e..29e97df37c29 100644 >>> --- a/mm/huge_memory.c >>> +++ b/mm/huge_memory.c >>> @@ -46,6 +46,16 @@ >>> #define CREATE_TRACE_POINTS >>> #include >>>=20 >>> +/* >>> + * The number of utilization buckets THPs will be grouped in >>> + * under /sys/kernel/debug/thp_utilization. >>> + */ >>> +#define THP_UTIL_BUCKET_NR 10 >>> +/* >>> + * The number of PFNs (and hence hugepages) to scan through on each pe= riodic >>=20 >> PFNs here is misleading. They usually refer to base-pagesize frams. Just >> say hugepages. > > Sounds good.=20 > >>=20 >>> + * run of the scanner that generates /sys/kernel/debug/thp_utilization. >>> + */ >>> +#define THP_UTIL_SCAN_SIZE 256 >>> /* >>> * By default, transparent hugepage support is disabled in order to avo= id >>> * risking an increased memory footprint for applications that are not >>> @@ -71,6 +81,25 @@ static atomic_t huge_zero_refcount; >>> struct page *huge_zero_page __read_mostly; >>> unsigned long huge_zero_pfn __read_mostly =3D ~0UL; >>>=20 >>> +static void thp_utilization_workfn(struct work_struct *work); >>> +static DECLARE_DELAYED_WORK(thp_utilization_work, thp_utilization_work= fn); >>> + >>> +struct thp_scan_info_bucket { >>> + int nr_thps; >>> + int nr_zero_pages; >>> +}; >>> + >>> +struct thp_scan_info { >>> + struct thp_scan_info_bucket buckets[THP_UTIL_BUCKET_NR]; >>> + struct zone *scan_zone; >>> + struct timespec64 last_scan_duration; >>> + struct timespec64 last_scan_time; >>> + unsigned long pfn; >>> +}; >>> + >>> +static struct thp_scan_info thp_scan_debugfs; >>> +static struct thp_scan_info thp_scan; >>=20 >> Any explanation why there are two of them? It is not obvious to me. > > The reason we have two is that one of them is used for debugfs if =E2=80= =98cat /sys/kernel/debug/thp_utilization=E2=80=99 is called.=20 > The other is used to keep track of the current scan.=20 >>=20 >>> + >>> bool hugepage_vma_check(struct vm_area_struct *vma, unsigned long vm_fl= ags, >>> bool smaps, bool in_pf, bool enforce_sysfs) >>> { >>> @@ -485,6 +514,7 @@ static int __init hugepage_init(void) >>> if (err) >>> goto err_slab; >>>=20 >>> + schedule_delayed_work(&thp_utilization_work, HZ); >>> err =3D register_shrinker(&huge_zero_page_shrinker, "thp-zero"); >>> if (err) >>> goto err_hzp_shrinker; >>> @@ -599,6 +629,11 @@ static inline bool is_transparent_hugepage(struct = page *page) >>> page[1].compound_dtor =3D=3D TRANSHUGE_PAGE_DTOR; >>> } >>>=20 >>> +static inline bool is_anon_transparent_hugepage(struct page *page) >>> +{ >>> + return is_transparent_hugepage(page) && PageAnon(page); >>> +} >>> + >>> static unsigned long __thp_get_unmapped_area(struct file *filp, >>> unsigned long addr, unsigned long len, >>> loff_t off, unsigned long flags, unsigned long size) >>> @@ -649,6 +684,49 @@ unsigned long thp_get_unmapped_area(struct file *f= ilp, unsigned long addr, >>> } >>> EXPORT_SYMBOL_GPL(thp_get_unmapped_area); >>>=20 >>> +int thp_number_utilized_pages(struct page *page) >>> +{ >>> + struct folio *folio; >>> + unsigned long page_offset, value; >>> + int thp_nr_utilized_pages =3D HPAGE_PMD_NR; >>> + int step_size =3D sizeof(unsigned long); >>> + bool is_all_zeroes; >>> + void *kaddr; >>> + int i; >>> + >>> + if (!page || !is_anon_transparent_hugepage(page)) >>> + return -1; >>> + >>> + folio =3D page_folio(page); >>> + for (i =3D 0; i < folio_nr_pages(folio); i++) { >>> + kaddr =3D kmap_local_folio(folio, i); >>> + is_all_zeroes =3D true; >>> + for (page_offset =3D 0; page_offset < PAGE_SIZE; page_offset +=3D st= ep_size) { >>> + value =3D *(unsigned long *)(kaddr + page_offset); >>> + if (value !=3D 0) { >>> + is_all_zeroes =3D false; >>> + break; >>> + } >>=20 >> Uhmm.. memchr_inv()? > > I had considered that at the time but memchr_inv() used here would return= the address of the first nonzero byte.=20 > Here we are trying to find the utilization percentage of the THP. I do no= t believe memchr_inv() would be less code > as compared what we do here. In general, I think it's better to use library functions if possible. memchar_inv() can be used here to check whether the subpage is all zero via checking whether NULL is return. memchar_inv() isn't perfect for your purpose, but your code doesn't look like highly optimized too. Is it necessary to add another library function to check whether the contents of a range of memory are all zero? Then we can optimize the implementation. >>=20 >>> + } >>> + if (is_all_zeroes) >>> + thp_nr_utilized_pages--; >>> + >>> + kunmap_local(kaddr); >>> + } >>> + return thp_nr_utilized_pages; >>> +} >>> + >>> +int thp_utilization_bucket(int num_utilized_pages) >>> +{ >>> + int bucket; >>> + >>> + if (num_utilized_pages < 0 || num_utilized_pages > HPAGE_PMD_NR) >>=20 >> Shouldn't it be WARN() or something? >>=20 >>> + return -1; >>=20 >> >>=20 >>> + /* Group THPs into utilization buckets */ >>> + bucket =3D num_utilized_pages * THP_UTIL_BUCKET_NR / HPAGE_PMD_NR; >>> + return min(bucket, THP_UTIL_BUCKET_NR - 1); >>> +} >>> + >>> static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf, >>> struct page *page, gfp_t gfp) >>> { >>> @@ -3174,6 +3252,42 @@ static int __init split_huge_pages_debugfs(void) >>> return 0; >>> } >>> late_initcall(split_huge_pages_debugfs); >>> + >>> +static int thp_utilization_show(struct seq_file *seqf, void *pos) >>> +{ >>> + int i; >>> + int start; >>> + int end; >>> + >>> + for (i =3D 0; i < THP_UTIL_BUCKET_NR; i++) { >>> + start =3D i * HPAGE_PMD_NR / THP_UTIL_BUCKET_NR; >>> + end =3D (i + 1 =3D=3D THP_UTIL_BUCKET_NR) >>> + ? HPAGE_PMD_NR >>> + : ((i + 1) * HPAGE_PMD_NR / THP_UTIL_BUCKET_NR - 1); >>> + /* The last bucket will need to contain 100 */ >>> + seq_printf(seqf, "Utilized[%d-%d]: %d %d\n", start, end, >>> + thp_scan_debugfs.buckets[i].nr_thps, >>> + thp_scan_debugfs.buckets[i].nr_zero_pages); >>> + } >>=20 >> , again. Here and in many places below. Seriously, they are >> cheap. :P >>=20 >>> + seq_printf(seqf, "Last Scan Time: %lu.%02lus\n", >>> + (unsigned long)thp_scan_debugfs.last_scan_time.tv_sec, >>> + (thp_scan_debugfs.last_scan_time.tv_nsec / (NSEC_PER_SEC / 100))); >>> + >>> + seq_printf(seqf, "Last Scan Duration: %lu.%02lus\n", >>> + (unsigned long)thp_scan_debugfs.last_scan_duration.tv_sec, >>> + (thp_scan_debugfs.last_scan_duration.tv_nsec / (NSEC_PER_SEC / 10= 0))); >>> + >>> + return 0; >>> +} >>> +DEFINE_SHOW_ATTRIBUTE(thp_utilization); >>> + >>> +static int __init thp_utilization_debugfs(void) >>> +{ >>> + debugfs_create_file("thp_utilization", 0200, NULL, NULL, >>> + &thp_utilization_fops); >>> + return 0; >>> +} >>> +late_initcall(thp_utilization_debugfs); >>> #endif >>>=20 >>> #ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION >>> @@ -3269,3 +3383,91 @@ void remove_migration_pmd(struct page_vma_mapped= _walk *pvmw, struct page *new) >>> trace_remove_migration_pmd(address, pmd_val(pmde)); >>> } >>> #endif >>> + >>> +static void thp_scan_next_zone(void) >>> +{ >>> + struct timespec64 current_time; >>> + int i; >>> + bool update_debugfs; >>> + /* >>> + * THP utilization worker thread has reached the end >>> + * of the memory zone. Proceed to the next zone. >>> + */ >>> + thp_scan.scan_zone =3D next_zone(thp_scan.scan_zone); >>> + update_debugfs =3D !thp_scan.scan_zone; >>> + thp_scan.scan_zone =3D update_debugfs ? (first_online_pgdat())->node_= zones >>> + : thp_scan.scan_zone; >>=20 >> I don't follow what is going on. thp_scan vs thp_scan_debugfs looks >> confusing. >>=20 >>> + thp_scan.pfn =3D (thp_scan.scan_zone->zone_start_pfn + HPAGE_PMD_NR -= 1) >>> + & ~(HPAGE_PMD_SIZE - 1); >>> + if (!update_debugfs) >>> + return; >>> + /* >>> + * If the worker has scanned through all of physical >>> + * memory. Then update information displayed in /sys/kernel/debug/thp= _utilization >>> + */ >>> + ktime_get_ts64(¤t_time); >>> + thp_scan_debugfs.last_scan_duration =3D timespec64_sub(current_time, >>> + thp_scan_debugfs.last_scan_time); >>> + thp_scan_debugfs.last_scan_time =3D current_time; >>> + >>> + for (i =3D 0; i < THP_UTIL_BUCKET_NR; i++) { >>> + thp_scan_debugfs.buckets[i].nr_thps =3D thp_scan.buckets[i].nr_thps; >>> + thp_scan_debugfs.buckets[i].nr_zero_pages =3D thp_scan.buckets[i].nr= _zero_pages; >>> + thp_scan.buckets[i].nr_thps =3D 0; >>> + thp_scan.buckets[i].nr_zero_pages =3D 0; >>> + } >>> +} >>> + >>> +static void thp_util_scan(unsigned long pfn_end) >>> +{ >>> + struct page *page =3D NULL; >>> + int bucket, num_utilized_pages, current_pfn; >>> + int i; >>> + /* >>> + * Scan through each memory zone in chunks of THP_UTIL_SCAN_SIZE >>> + * PFNs every second looking for anonymous THPs. >>> + */ >>> + for (i =3D 0; i < THP_UTIL_SCAN_SIZE; i++) { >>> + current_pfn =3D thp_scan.pfn; >>> + thp_scan.pfn +=3D HPAGE_PMD_NR; >>> + if (current_pfn >=3D pfn_end) >>> + return; >>> + >>> + if (!pfn_valid(current_pfn)) >>> + continue; >>> + >>> + page =3D pfn_to_page(current_pfn); >>=20 >> pfn_valid() + pfn_to_page() has to be replaced to pfn_to_online_page(). > > Ah k thanks.=20 >>=20 >>> + num_utilized_pages =3D thp_number_utilized_pages(page); >>> + bucket =3D thp_utilization_bucket(num_utilized_pages); >>> + if (bucket < 0) >>> + continue; >>> + >>> + thp_scan.buckets[bucket].nr_thps++; >>> + thp_scan.buckets[bucket].nr_zero_pages +=3D (HPAGE_PMD_NR - num_util= ized_pages); >>> + } >>> +} >>> + >>> +static void thp_utilization_workfn(struct work_struct *work) >>> +{ >>> + unsigned long pfn_end; >>> + >>> + if (!thp_scan.scan_zone) >>> + thp_scan.scan_zone =3D (first_online_pgdat())->node_zones; >>> + /* >>> + * Worker function that scans through all of physical memory >>> + * for anonymous THPs. >>> + */ >>> + pfn_end =3D (thp_scan.scan_zone->zone_start_pfn + >>> + thp_scan.scan_zone->spanned_pages + HPAGE_PMD_NR - 1) >>> + & ~(HPAGE_PMD_SIZE - 1); >>> + /* If we have reached the end of the zone or end of physical memory >>> + * move on to the next zone. Otherwise, scan the next PFNs in the >>> + * current zone. >>> + */ >>> + if (!populated_zone(thp_scan.scan_zone) || thp_scan.pfn >=3D pfn_end) >>> + thp_scan_next_zone(); >>> + else >>> + thp_util_scan(pfn_end); >>> + >>> + schedule_delayed_work(&thp_utilization_work, HZ); >>=20 >> Why HZ? > > Scanning 256 PFNs per second is just what we have found to not have any n= oticeable effect on our hosts.=20 Better to show some performance data here. For example, ftrace callgraph can be used to collect the run time of thp_utilization_workfn() on a system full of THP. And, it will take about 512s to scan all memory on a system with 256 GB memory. This appears long too. >>=20 >>> +} Best Regards, Huang, Ying