Re: [linux-next:master] [mm] 1111d46b5c: stress-ng.pthread.ops_per_sec -84.3% regression

[Yin Fengwei <fengwei.yin@intel.com>]

On 2024/1/4 09:32, Yang Shi wrote:
> On Thu, Dec 21, 2023 at 5:13 PM Yin, Fengwei <fengwei.yin@intel.com> wrote:
>>
>>
>>
>> On 12/22/2023 2:11 AM, Yang Shi wrote:
>>> On Thu, Dec 21, 2023 at 5:40 AM Yin, Fengwei <fengwei.yin@intel.com> wrote:
>>>>
>>>>
>>>>
>>>> On 12/21/2023 8:58 AM, Yin Fengwei wrote:
>>>>> But what I am not sure was whether it's worthy to do such kind of change
>>>>> as the regression only is seen obviously in micro-benchmark. No evidence
>>>>> showed the other regressionsin this report is related with madvise. At
>>>>> least from the perf statstics. Need to check more on stream/ramspeed.
>>>>> Thanks.
>>>>
>>>> With debugging patch (filter out the stack mapping from THP aligned),
>>>> the result of stream can be restored to around 2%:
>>>>
>>>> commit:
>>>>      30749e6fbb3d391a7939ac347e9612afe8c26e94
>>>>      1111d46b5cbad57486e7a3fab75888accac2f072
>>>>      89f60532d82b9ecd39303a74589f76e4758f176f  -> 1111d46b5cbad with
>>>> debugging patch
>>>>
>>>> 30749e6fbb3d391a 1111d46b5cbad57486e7a3fab75 89f60532d82b9ecd39303a74589
>>>> ---------------- --------------------------- ---------------------------
>>>>        350993           -15.6%     296081 ±  2%      -1.5%     345689
>>>>      stream.add_bandwidth_MBps
>>>>        349830           -16.1%     293492 ±  2%      -2.3%     341860 ±
>>>> 2%  stream.add_bandwidth_MBps_harmonicMean
>>>>        333973           -20.5%     265439 ±  3%      -1.7%     328403
>>>>      stream.copy_bandwidth_MBps
>>>>        332930           -21.7%     260548 ±  3%      -2.5%     324711 ±
>>>> 2%  stream.copy_bandwidth_MBps_harmonicMean
>>>>        302788           -16.2%     253817 ±  2%      -1.4%     298421
>>>>      stream.scale_bandwidth_MBps
>>>>        302157           -17.1%     250577 ±  2%      -2.0%     296054
>>>>      stream.scale_bandwidth_MBps_harmonicMean
>>>>        339047           -12.1%     298061            -1.4%     334206
>>>>      stream.triad_bandwidth_MBps
>>>>        338186           -12.4%     296218            -2.0%     331469
>>>>      stream.triad_bandwidth_MBps_harmonicMean
>>>>
>>>>
>>>> The regression of ramspeed is still there.
>>>
>>> Thanks for the debugging patch and the test. If no one has objection
>>> to honor MAP_STACK, I'm going to come up with a more formal patch.
>>> Even though thp_get_unmapped_area() is not called for MAP_STACK, stack
>>> area still may be allocated at 2M aligned address theoretically. And
>>> it may be worse with multi-sized THP, for 1M.
>> Right. Filtering out MAP_STACK can't make sure no THP for stack. Just
>> reduce the possibility of using THP for stack.
> 
> Can you please help test the below patch?
I can't access the testing box now. Oliver will help to test your patch.


Regards
Yin, Fengwei

> 
> diff --git a/include/linux/mman.h b/include/linux/mman.h
> index 40d94411d492..dc7048824be8 100644
> --- a/include/linux/mman.h
> +++ b/include/linux/mman.h
> @@ -156,6 +156,7 @@ calc_vm_flag_bits(unsigned long flags)
>          return _calc_vm_trans(flags, MAP_GROWSDOWN,  VM_GROWSDOWN ) |
>                 _calc_vm_trans(flags, MAP_LOCKED,     VM_LOCKED    ) |
>                 _calc_vm_trans(flags, MAP_SYNC,       VM_SYNC      ) |
> +              _calc_vm_trans(flags, MAP_STACK,      VM_NOHUGEPAGE) |
>                 arch_calc_vm_flag_bits(flags);
>   }
> 
> But I can't reproduce the pthread regression on my aarch64 VM. It
> might be due to the guard stack (the 64K guard stack is at 2M aligned,
> the 8M stack is right next to it which starts at 2M + 64K). But I can
> see the stack area is not THP eligible anymore with this patch. See:
> 
> fffd18e10000-fffd19610000 rw-p 00000000 00:00 0
> Size:               8192 kB
> KernelPageSize:        4 kB
> MMUPageSize:           4 kB
> Rss:                  12 kB
> Pss:                  12 kB
> Pss_Dirty:            12 kB
> Shared_Clean:          0 kB
> Shared_Dirty:          0 kB
> Private_Clean:         0 kB
> Private_Dirty:        12 kB
> Referenced:           12 kB
> Anonymous:            12 kB
> KSM:                   0 kB
> LazyFree:              0 kB
> AnonHugePages:         0 kB
> ShmemPmdMapped:        0 kB
> FilePmdMapped:         0 kB
> Shared_Hugetlb:        0 kB
> Private_Hugetlb:       0 kB
> Swap:                  0 kB
> SwapPss:               0 kB
> Locked:                0 kB
> THPeligible:           0
> VmFlags: rd wr mr mw me ac nh
> 
> The "nh" flag is set.
> 
>>
>>>
>>> Do you have any instructions regarding how to run ramspeed? Anyway I
>>> may not have time debug it until after holidays.
>> 0Day leverages phoronix-test-suite to run ramspeed. So I don't have
>> direct answer here.
>>
>> I suppose we could check the configuration of ramspeed in phoronix-test-
>> suite to understand what's the build options and command options to run
>> ramspeed:
>> https://openbenchmarking.org/test/pts/ramspeed
> 
> Downloaded the test suite. It looks phronix just runs test 3 (int) and
> 6 (float). They basically does 4 sub tests to benchmark memory
> bandwidth:
> 
>   * copy
>   * scale copy
>   * add copy
>   * triad copy
> 
> The source buffer is initialized (page fault is triggered), but the
> destination area is not. So the page fault + page clear time is
> accounted to the result. Clearing huge page may take a little bit more
> time. But I didn't see noticeable regression on my aarch64 VM either.
> Anyway I'm supposed such test should be run with THP off.
> 
>>
>>
>> Regards
>> Yin, Fengwei
>>
>>>
>>>>
>>>>
>>>> Regards
>>>> Yin, Fengwei