答复: [PATCH 1/1] mm/ksm: add ksm_pages_sharing for each process to calculate profit more accurately

[<xu.xin16@zte.com.cn>]

> >>>> and /proc/self/ksm_stat/ to indicate the saved pages of this process.
> >>>> (not including ksm_zero_pages)
> >>> Curious, why is updating ksm_process_profit() insufficient and we also
> >>> have to expose ksm_pages_sharing?
> >>>
> >> Since ksm_process_profit() uses ksm_merging_pages(pages_sharing +
> >> pages_shared) to calculate the profit for individual processes,
> >>
> >> while general_profit uses pages_sharing for profit calculation, this can
> >> lead to the total profit calculated for each process being greater than
> >> that of general_profit.
> >>
> >> Additionally, exposing ksm_pages_sharing under /proc/self/ksm_stat/ may
> >> be sufficient.
> >>
> > Hi,
> >
> > Althorugh it's true, however, this patch maybe not okay. It can only ensure
> > that the sum of each process's profit roughly equals the system's general_profit
> > , but gives totally wrong profit result for some one process. For example, when
> > two pages from two different processes are merged, one process's page_shared
> > increments by +1, while the other's pages_sharing increments by +1, which
> > resulting in different calculated profits for the two processes, even though
> > their actual profits are identical. If in more extreme cases, this could even
> > render a process's profit entirely unreadable.
> >
> > Lastly, do we really need each process’s profit sum to perfectly match the general
> > profit, or we just want a rough estimate of the process’s profit from KSM ?
> >
> Hi,
> 
> In extreme cases, stable nodes may be distributed quite unevenly, which 
> is due to stable nodes not being per mm, of course.
> There are also situations where there are 1000 pairs of pages, with the 
> pages within each pair being identical, while each pair is different 
> from all other pages.
> This results in the number of page_sharing and page_shared being the 
> same. This way, using ksm_merging_pages(page_sharing + page_shared) 
> averages a 50% error.

In your tests, I don't agree 50% error because your
assumption that process benefit equals pages_sharing is fundamentally flawed.

The issue lies in what is the most accurate definition of process KSM profit.
Since stable_node isn't per-mm, we cannot calculate a process's
benefit solely based on pages_sharing. The cost of stable_node should be split
fairly among every process sharing this stable_node, rather than being assigned
to a single individual.

It's inaccurate to claim that when two processes' pages merge into a
single KSM page, one process gains 4k - sizeof(rmap_item) while
the other gains 0 ? This is unfair to the second process, as it actively
participated in the KSM merge.


The most accurate and fair profit caculation should be:

    profit = (ksm_merging_pages - united_stable_nodes)*PAGE_SIZE - sizeof(rmap_items)*ksm_rmap_items

where 'united_stable_nodes' is (stable_node)/shared_process. This is too complex.

For example: process A with one page is merged with process B with one page

process A      process B
    page          page
     \           /
      \         /
       \       /
        \     /
        Ksm Page

A: pages_sharing(1), pages_shared(0)
B: pages_sharing(0), pages_shared(1)

then

profit(A) = (pages_sharing + pages_shared - united_stable_nodes)*4K- sizeof(rmap_items)*ksm_rmap_items

          = (1 + 0 - 1/2)*4k-sizeof(rmap_items)*ksm_rmap_items

          = 0.5*4k - sizeof(rmap_items)*ksm_rmap_items

profit(A) = (pages_sharing + pages_shared - united_stable_nodes)*4K- sizeof(rmap_items)*ksm_rmap_items

	  = (0 + 1 - 1/2)*4k-sizeof(rmap_items)*ksm_rmap_items

          = 0.5*4k - sizeof(rmap_items)*ksm_rmap_items


> In practical testing, we may only need to enable KSM for specific 
> applications and calculate the total benefits of these processes.
> Since page_shared is also included in the statistics, this may lead to 
> the calculated benefits being higher than the actual ones.
> In practical testing, the error may reach 20%. For example, in one test, 
> the total benefits of all processes were estimated to be around 528MB,
> while the profit calculated through general_profit was only around 428MB.
> The theoretical error may be around 50%.
> 
> If we expose the ksm_pages_sharing for each process, we can not only 
> calculate the actual benefits
> 
> but also determine how many ksm_pages_shared there are by the difference 
> between ksm_merging_pages and ksm_pages_sharing of each process.