Re: Prerequisites for Large Anon Folios

["Yin, Fengwei" <fengwei.yin@intel.com>]

On 8/31/2023 3:18 PM, Ryan Roberts wrote:
> On 31/08/2023 01:08, Yin, Fengwei wrote:
>>
>> On 8/30/2023 6:44 PM, Ryan Roberts wrote:
>>> Hi All,
>>>
>>>
>>> I want to get serious about getting large anon folios merged. To do that, there
>>> are a number of outstanding prerequistes. I'm hoping the respective owners may
>>> be able to provide an update on progress?
>>>
>>> I appreciate everyone is busy and likely juggling multiple things, so understand
>>> if no progress has been made or likely to be made - it would be good to know
>>> that though, so I can attempt to make alternative plans.
>>>
>>> See questions/comments below.
>>>
>>> Thanks!
>>>
>>>
> ...
>>>
>>>>
>>>> - item:
>>>>     mlock
>>>>
>>>>   priority:
>>>>     prerequisite
>>>>
>>>>   description: >-
>>>>     Large, pte-mapped folios are ignored when mlock is requested. Code comment
>>>>     for mlock_vma_folio() says "...filter out pte mappings of THPs, which cannot
>>>>     be consistently counted: a pte mapping of the THP head cannot be
>>>>     distinguished by the page alone."
>>>>
>>>>   location:
>>>>     - mlock_pte_range()
>>>>     - mlock_vma_folio()
>>>>
>>>>   links:
>>>>     - https://lore.kernel.org/linux-mm/20230712060144.3006358-1-fengwei.yin@intel.com/
>>>>
>>>>   assignee:
>>>>     Yin, Fengwei <fengwei.yin@intel.com>
>>>>
>>>>
>>>
>>> series on list at [2]. Does this series cover everything?
>> Yes. I suppose so. I already collected comment from you. And I am waiting for review comment
>> from Yu who is on vacation now. Then, I will work on v3.
> 
> Great -thanks for the fast reply!
> 
>>
>>>
>>> [2] https://lore.kernel.org/linux-mm/20230809061105.3369958-1-fengwei.yin@intel.com/
>>>
>>>
>>>>
>>>> - item:
>>>>     madvise
>>>>
>>>>   priority:
>>>>     prerequisite
>>>>
>>>>   description: >-
>>>>     MADV_COLD, MADV_PAGEOUT, MADV_FREE: For large folios, code assumes exclusive
>>>>     only if mapcount==1, else skips remainder of operation. For large,
>>>>     pte-mapped folios, exclusive folios can have mapcount upto nr_pages and
>>>>     still be exclusive. Even better; don't split the folio if it fits entirely
>>>>     within the range. Likely depends on "shared vs exclusive mappings".
>>>>
>>>>   links:
>>>>     - https://lore.kernel.org/linux-mm/20230713150558.200545-1-fengwei.yin@intel.com/
>>>>
>>>>   location:
>>>>     - madvise_cold_or_pageout_pte_range()
>>>>     - madvise_free_pte_range()
>>>>
>>>>   assignee:
>>>>     Yin, Fengwei <fengwei.yin@intel.com>
>>>
>>> As I understand it: initial solution based on folio_estimated_sharers() has gone
>>> into v6.5. Have a dependecy on David's precise shared vs exclusive work for an
>>> improved solution. And I think you mentioned you are planning to do a change
>>> that avoids splitting a large folio if it is entirely covered by the range?
>> The changes based on folio_estimated_sharers() is in. Once David's solution is
>> ready, will switch to new solution.
>>
>> For avoids splitting large folio, it was in the patchset I posted (before split
>> folio_estimated_sharers() part out).
> 
> The RFC version? Do you plan to post an updated version, or are you waiting for
> David's shared vs exclusive series before moving forwards?

For folio_estimated_sharers(), Once David's solution is ready. I will send patch
to switch to new solution.

For avoid splitting large folio, I don't think it blocks the anonymous large folio
merging as it's optimization instead of bug fix. My idea was demonstrated on the
first patchset (and folio_estimated_sharers() was separated from the first patchset
as it's a bug fixing) and wait for comments from Minchan.


Regards
Yin, Fengwei

> 
>>
>> Regards
>> Yin, Fengwei
>