Re: [RFC Patch 0/5] Make anon_vma operations testable

[David Hildenbrand <david@redhat.com>]

On 30.05.25 04:11, Wei Yang wrote:
> On Tue, May 27, 2025 at 01:31:47PM +0200, David Hildenbrand wrote:
>> On 27.05.25 08:34, Wei Yang wrote:
>>> On Wed, May 14, 2025 at 01:23:18AM +0000, Wei Yang wrote:
>>>> On Wed, Apr 30, 2025 at 09:47:16AM +0200, David Hildenbrand wrote:
>>>> [...]
>>>>>>>> Agreed, skimming over the tests there are some nice diagrams and cases.
>>>>>>>>
>>>>>>>> But I would hope that for most of these cases we could test on a higher
>>>>>>>> level: test our expectations when running real programs that we want to
>>>>>>>> check, especially when performing internal changes on how we handle anon
>>>>>>>> memory + rmap.
>>>>>>>>
>>>>>>>> E.g., do fork(), then test if we can successfully perform rmap
>>>>>>>> lookups/updates (e.g., migrate folio to a different numa node etc).
>>>>>>>>
>>>>>>>
>>>>>>> That's a great point! Wei - if you could look at making some self-tests
>>>>>>> (i.e. that live in tools/testing/selftests/mm) that try to recreate _real_
>>>>>>> scenarios that use the rmap like this and assert correct behaviour there,
>>>>>>> that could be a positive way of moving forward with this.
>>>>>>>
>>>>>>
>>>
>>> Ping
>>
>> Thanks for reminding me and sorry for the late reply.
>>
>>>
>>>>>> I am trying to understand what scenarios you want.
>>>>>
>>>>
>>>> Sorry for the late reply, I handled other things a while.
>>>>
>>>>> That is exactly the task to figure out: how can we actually test our rmap
>>>>> implementation from a higher level. The example regarding fork and migration
>>>>> is possibly a low-hanging fruit.
>>>>
>>>> If my understanding is correct, you suggested two high level way:
>>>>
>>>> 1. fork + migrate (move_pages)
>>
>> Yes, that should be one way of testing it. We could even get multiple child
>> processes into place.
>>
>>>>
>>>>>
>>>>> We might already have the functionality to achieve it, *maybe* we'd even want
>>>>> some extensions to make it all even easier to test.
>>>>>
>>>>> For example, MADV_PAGEOUT is refused on folios that are mapped into multiple
>>>>> processes. Maybe we'd want the option to *still* page it out, just like
>>>>> MPOL_MF_MOVE_ALL allows with CAP_SYS_NICE to *still* migrate a folio that is
>>>>> mapped into multiple processes.
>>>>>
>>>>
>>>> 2. madvise(MADV_PAGEOUT)
>>>>
>>>> Not fully get it here. You mean fork + madvise(MADV_PAGEOUT) + migrate ?
>>
>> fork + madvise(MADV_PAGEOUT) only.
>>
>> Then verify, that it was actually paged out.
>>
>> For example, we could have 10 processes mapping the page, then call
>> madvise(MADV_PAGEOUT) from one process. We can verify in all processes if the
>> page is actually no longer mapped using /proc/self/pagemap
>>
> 
> Hi, David

Hi!

> 
> I'd like to clarify some detail here.
> 
>> We could likely do it with anon pages (swap), shmem pages (swap) and
>> pagecache pages (file backed).
>>
> 
> I could understand the three cases here, but for the (swap) case, it is not
> clear to me. You mean we force the page to be swapped out before migration? I
> may not have an idea to ensure that.

No need to involve migration for that test. It's sufficient to trigger 
pageout to then check if the page was unmapped in all other processes.

> 
>> ... and possibly even with KSM pages!
>>
> 
> For KSM, it looks like an general background routine. I guess you want
> something like:
> 
>    * fork process tree
>    * do KSM and wait for it to finish?

Probably allocate memory in each process and fill it with the same value.

Then wait for KSM to deduplciate.

>    * migrate

Yes, or instead of migrate, trigger pageout from one process, then check 
if pageout in all other processes (/proc/self/pagemap)

 > > I guess the KSM duration would related to the total memory on the 
system. The
> bigger, the longer. While the selftest generally have timeout setting. Not
> sure it would be suitable for all situation.
> 
> Would you mind sharing more on KSM case?

Worth looking at tools/testing/selftests/mm/ksm_functional_tests.c (and 
the other KSM selftests)

In particular, in ksm_merge() we can detect whether KSM completed the 
scan when two full scans passed.

A lot of that functionality could probably be factored out into 
vm_utils.c to be reused in other tests.

-- 
Cheers,

David / dhildenb