Re: [linux-next:master] [mm] 1111d46b5c: stress-ng.pthread.ops_per_sec -84.3% regression

["Huang, Ying" <ying.huang@intel.com>]

"Yin, Fengwei" <fengwei.yin@intel.com> writes:

> On 12/22/2023 2:14 AM, Matthew Wilcox wrote:
>> On Thu, Dec 21, 2023 at 10:07:09AM -0800, Yang Shi wrote:
>>> On Wed, Dec 20, 2023 at 8:49 PM Matthew Wilcox <willy@infradead.org> wrote:
>>>>
>>>> On Thu, Dec 21, 2023 at 08:58:42AM +0800, Yin Fengwei wrote:
>>>>> Yes. MAP_STACK is also mentioned in manpage of mmap. I did test to
>>>>> filter out of the MAP_STACK mapping based on this patch. The regression
>>>>> in stress-ng.pthread was gone. I suppose this is kind of safe because
>>>>> the madvise call is only applied to glibc allocated stack.
>>>>>
>>>>>
>>>>> But what I am not sure was whether it's worthy to do such kind of change
>>>>> as the regression only is seen obviously in micro-benchmark. No evidence
>>>>> showed the other regressionsin this report is related with madvise. At
>>>>> least from the perf statstics. Need to check more on stream/ramspeed.
>>>>
>>>> FWIW, we had a customer report a significant performance problem when
>>>> inadvertently using 2MB pages for stacks.  They were able to avoid it by
>>>> using 2044KiB sized stacks ...
>>>
>>> Thanks for the report. This provided more justification regarding
>>> honoring MAP_STACK on Linux. Some applications, for example, pthread,
>>> just allocate a fixed size area for stack. This confuses kernel
>>> because kernel tell stack by VM_GROWSDOWN | VM_GROWSUP.
>>>
>>> But I'm still a little confused by why THP for stack could result in
>>> significant performance problems. Unless the applications resize the
>>> stack quite often.
>> We didn't delve into what was causing the problem, only that it was
>> happening.  The application had many threads, so it could have been as
>> simple as consuming all the available THP and leaving fewer available
>> for other uses.  Or it could have been a memory consumption problem;
>> maybe the app would only have been using 16-32kB per thread but was
>> now using 2MB per thread and if there were, say, 100 threads, that's an
>> extra 199MB of memory in use.
> One thing I know is related with the memory zeroing. This is from
> the perf data in this report:
>
>       0.00           +16.7       16.69 ±  7%
>       perf-profile.calltrace.cycles-pp.clear_page_erms.clear_huge_page.__do_huge_pmd_anonymous_page.__handle_mm_fault.handle_mm_fault
>
> Zeroing 2M memory costs much more CPU than zeroing 16-32KB memory if
> there are many threads.

Using 2M stack may hurt performance of short-live threads with shallow
stack depth.  Imagine a network server which creates a new thread for
each incoming connection.  I understand that the performance will not be
great in this way anyway.  IIUC we should not make it too bad.

But, whether this is import depends on whether the use case is
important.  TBH, I don't know that.

--
Best Regards,
Huang, Ying