Re: [Question] About memory.c: process_huge_page

["Huang, Ying" <ying.huang@linux.alibaba.com>]

Hi, Haoran,

Zhu Haoran <zhr1502@sjtu.edu.cn> writes:

> Hi!
>
> I recently noticed the process_huge_page function in memory.c, which was
> intended to keep the cache hotness of target page after processing. I compared
> the vm-scalability anon-cow-seq-hugetlb microbench using the default
> process_huge_page and sequential processing (code posted below).
>
> I ran test on epyc-7T83 with 36vCPUs and 64GB memory. Using default
> process_huge_page, the avg bandwidth is 1148 mb/s. However sequential
> processing yielded a better bandwidth of about 1255 mb/s and only
> one-third cache-miss rate compared with default one.
>
> The same test was run on epyc-9654 with 36vCPU and 64GB mem. The
> bandwidth result was similar but the difference was smaller: 1170mb/s
> for default and 1230 mb/s for sequential. Although we did find the cache
> miss rate here did the reverse, since the sequential processing seen 3
> times miss more than the default.
>
> These result seem really inconsitent with the what described in your
> patchset [1]. What factors might explain these behaviors?

One possible difference is cache topology.  Can you try to bind the test
process to the CPUs in one CCX (that is, share one LLC).  This make it
possible to hit the local cache.

> Thanks for your time.
>
> [1] https://lkml.org/lkml/2018/5/23/1072
>
> ---
> Sincere,
> Zhu Haoran
>
> ---
>
> static int process_huge_page(
>         unsigned long addr_hint, unsigned int nr_pages,
>         int (*process_subpage)(unsigned long addr, int idx, void *arg),
>         void *arg)
> {
>     int i, ret;
>     unsigned long addr = addr_hint &
>         ~(((unsigned long)nr_pages << PAGE_SHIFT) - 1);
>
>     might_sleep();
>     for (i = 0; i < nr_pages; i++) {
>             cond_resched();
>             ret = process_subpage(addr + i * PAGE_SIZE, i, arg);
>             if (ret)
>                     return ret;
>     }
>
>     return 0;
> }

---
Best Regards,
Huang, Ying