Re: [PATCH 2/2] lru: allow large batched add large folio to lru list

["Huang, Ying" <ying.huang@intel.com>]

Yin Fengwei <fengwei.yin@intel.com> writes:

> Add Ying who found out the large folio is not batched added to lru.

Thanks!

> On 4/18/23 09:57, Yin Fengwei wrote:
>> 
>> 
>> On 4/17/23 20:25, Matthew Wilcox wrote:
>>> On Mon, Apr 17, 2023 at 03:56:43PM +0800, Yin Fengwei wrote:
>>>> Currently, large folio is not batched added to lru list. Which
>>>> cause high lru lock contention after enable large folio for
>>>> anonymous mappping.
>>>
>>> Obviously, I think we should be doing a batched add, but I don't think
>>> this is right.
>>>
>>>> @@ -54,7 +57,12 @@ static inline unsigned pagevec_space(struct pagevec *pvec)
>>>>  static inline unsigned pagevec_add(struct pagevec *pvec, struct page *page)
>>>>  {
>>>>  	pvec->pages[pvec->nr++] = page;
>>>> -	return pagevec_space(pvec);
>>>> +	pvec->pages_nr += compound_nr(page);
>>>> +
>>>> +	if (pvec->pages_nr > PAGEVEC_SIZE)

nr_pages appears better for me.

>>>> +		return 0;
>>>> +	else
>>>> +		return pagevec_space(pvec);
>>>
>>> I assume your thinking here is that we should limit the number of pages
>>> in the batches, but I think we should allow the number of folios to reach
>>> PAGEVEC_SIZE before we drain the batch onto the LRU list.  That will
>>> reduce the contention on the LRU lock even further.
>> 
>> Yes. The first thought in my mind was to limit the number of folios also.
>> 
>> But the concern is that large folio has wider range of size. In the extreme
>> case, if all batched large folios has 2M size, with PAGEVEC_SIZE as 15,
>> one batch could have 30M memory. Which could be too large for some usages.

Yes.  I think that these are valid concerns.  One possibility to balance
between performance and lru cache size is to make nr_pages of per-CPU
lru cache < PAGEVEC_SIZE * N.  Where N can be determined according to
the intended base order of large folios.  For example, it can be 4 if we
use 2 as intended base order.

Just my 2 cents.

Best Regards,
Huang, Ying