Re: [PATCH 1/1] mm/ksm: add ksm_pages_sharing for each process to calculate profit more accurately

[Longlong Xia <xialonglong@kylinos.cn>]

在 2025/6/18 17:14, xu.xin16@zte.com.cn 写道:
>>>> and /proc/self/ksm_stat/ to indicate the saved pages of this process.
>>>> (not including ksm_zero_pages)
>>> Curious, why is updating ksm_process_profit() insufficient and we also
>>> have to expose ksm_pages_sharing?
>>>
>> Since ksm_process_profit() uses ksm_merging_pages(pages_sharing +
>> pages_shared) to calculate the profit for individual processes,
>>
>> while general_profit uses pages_sharing for profit calculation, this can
>> lead to the total profit calculated for each process being greater than
>> that of general_profit.
>>
>> Additionally, exposing ksm_pages_sharing under /proc/self/ksm_stat/ may
>> be sufficient.
>>
> Hi,
>
> Althorugh it's true, however, this patch maybe not okay. It can only ensure
> that the sum of each process's profit roughly equals the system's general_profit
> , but gives totally wrong profit result for some one process. For example, when
> two pages from two different processes are merged, one process's page_shared
> increments by +1, while the other's pages_sharing increments by +1, which
> resulting in different calculated profits for the two processes, even though
> their actual profits are identical. If in more extreme cases, this could even
> render a process's profit entirely unreadable.
>
> Lastly, do we really need each process’s profit sum to perfectly match the general
> profit, or we just want a rough estimate of the process’s profit from KSM ?
>
Hi,

In extreme cases, stable nodes may be distributed quite unevenly, which 
is due to stable nodes not being per mm, of course.
There are also situations where there are 1000 pairs of pages, with the 
pages within each pair being identical, while each pair is different 
from all other pages.
This results in the number of page_sharing and page_shared being the 
same. This way, using ksm_merging_pages(page_sharing + page_shared) 
averages a 50% error.
In practical testing, we may only need to enable KSM for specific 
applications and calculate the total benefits of these processes.
Since page_shared is also included in the statistics, this may lead to 
the calculated benefits being higher than the actual ones.
In practical testing, the error may reach 20%. For example, in one test, 
the total benefits of all processes were estimated to be around 528MB,
while the profit calculated through general_profit was only around 428MB.
The theoretical error may be around 50%.

If we expose the ksm_pages_sharing for each process, we can not only 
calculate the actual benefits

but also determine how many ksm_pages_shared there are by the difference 
between ksm_merging_pages and ksm_pages_sharing of each process.

>>
>>> Hm, I am wondering if that works. Stable nodes are not per MM, so
>>> can't we create an accounting imbalance for one MM somehow?
>>>
>>> (did not look into all the details, just something that came to mind)
>>>
>> Indeed, using the method in this patch to calculate ksm_pages_sharing
>> for each process to determine ksm_pages_shared
>>
>> can sometimes result in negative values for ksm_pages_shared.
>>
>> example for calculate mm->ksm_pages_shared：
>>
>>           if (rmap_item->hlist.next) {
>>               ksm_pages_sharing--;
>>               rmap_item->mm->ksm_pages_sharing--;
>>
>>           } else {
>>               ksm_pages_shared--;
>>                rmap_item->mm->ksm_pages_shared--; // can be negative
>>           }
>>
>>           rmap_item->mm->ksm_merging_pages--;
>>
>>
>> Would it be possible to compare the ratio of each process's rmap_item to
>> the total rmap_item and the ratio of the process's page_shared to the
>> total page_shared
>>
>> to assess this imbalance? For now, I don't have any better ideas.
> Although stable_node is not per-mm, if you really add ksm_shared to mm,
> it won't cause negative ksm_pages_shared, because the count of ksm_shared
> will only be attributed to the process of the first rmap_item.
Yes, it was the incorrect method I used during testing that led to the 
negative values.
After the improvement, it has not occurred again.

Thank you for your time.
Best regards,
Longlong Xia