Re: mm performance with zram

[Luigi Semenzato <semenzato@google.com>]

Very nice!  But that looks like a tree.  What if a function is called
from multiple locations?

On Tue, Jan 13, 2015 at 1:18 AM, Vlastimil Babka <vbabka@suse.cz> wrote:
> On 01/08/2015 11:49 PM, Luigi Semenzato wrote:
>> I am taking a closer look at the performance of the Linux MM in the
>> context of heavy zram usage.  The bottom line is that there is
>> surprisingly high overhead (35-40%) from MM code other than
>> compression/decompression routines.  I'd like to share some results in
>> the hope they will be helpful in planning future development.
>>
>> SETUP
>>
>> I am running on an ASUS Chromebox with about 2GB RAM (actually 4GB,
>> but with mem=1931M).  The zram block device size is approx. 2.8GB
>> (uncompressed size).
>>
>> http://www.amazon.com/Asus-CHROMEBOX-M004U-ASUS-Desktop/dp/B00IT1WJZQ
>>
>> Intel(R) Celeron(R) 2955U @ 1.40GHz
>> MemTotal:        1930456 kB
>> SwapTotal:       2827816 kB
>>
>> I took the kernel from Linus's tree a few days ago: Linux localhost
>> 3.19.0-rc2+ (...) x86_64.  I also set maxcpus=1.  The kernel
>> configuration is available if needed.
>>
>> EXPERIMENTS
>>
>> I wrote a page walker (historically called "balloon") which allocates
>> a lot of memory, more than physical RAM, and fills it with a dump of
>> /dev/mem from a Chrome OS system running at capacity.  The memory
>> compresses down to about 35%.  I ran two main experiments.
>>
>> 1. Compression/decompression.  After filling the memory, the program
>> touches the first byte of all pages in a random permutation (I tried
>> sequentially too, it makes little difference).  At steady state, this
>> forces one page decompression and one compression (on average) at each
>> step of the walk.
>>
>> 2. Decompression only.  After filling the memory, the program walks
>> all pages sequentially.  Then it frees the second half of the pages
>> (the ones most recently touched), and walks the first half.  This
>> causes one page decompression at each step, and almost no
>> compressions.
>>
>> RESULTS
>>
>> The average time (real time) to walk a page in microseconds is
>>
>> experiment 1 (compress + decompress): 26.5  us/page
>> experiment 2 (decompress only): 9.3 us/page
>>
>> I ran "perf record -ag"during the relevant parts of the experiment.
>> (CAVEAT: the version of perf I used doesn't match the kernel, it's
>> quite a bit older, but that should be mostly OK).  I put the output of
>> "perf report" in this Google Drive folder:
>>
>> https://drive.google.com/folderview?id=0B6kmZ3mOd0bzVzJKeTV6eExfeFE&usp=sharing
>>
>> (You shouldn't need a Google ID to access it.  You may have to re-join
>> the link if the plain text mailer splits it into multiple lines.)
>>
>> I also tried to analyze cumulative graph profiles.  Interestingly the
>> only tool I found to do this is gprof2dot (any other suggestion?  I
>
> I think this could be useful for better graphs here:
>
> http://www.brendangregg.com/flamegraphs.html
>
>> would prefer a text-based tool).  The output is in the .png files in
>> the same folder.  The interesting numbers are:
>>
>> experiment 1
>> compression 43.2%
>> decompression 20.4%
>> everything else 36.4%
>>
>> experiment 2
>> decompression 61.7%
>> everything else 38.3%
>>
>> The graph profiles don't seem to show low-hanging fruits on any path.
>>
>> CONCLUSION
>>
>> Before zram, in a situation involving swapping, the MM overhead was
>> probably nearly invisible, especially with rotating disks.  But with
>> zram the MM is surprisingly close to being the main bottleneck.
>> Compression/decompression speeds will likely improve, and they are
>> tuneable (tradeoff between compression ratio and speed).  Compression
>> can happen often in the background, so decompression speed is more
>> important for latency, and LZ4 decompression can already be a lot
>> faster than LZO (the experiments use LZO, and LZ4 can be 2x faster).
>> This suggests that simplifying and speeding up the relevant code paths
>> in the Linux MM may be worth the effort.
>>
>> --
>> To unsubscribe, send a message with 'unsubscribe linux-mm' in
>> the body to majordomo@kvack.org.  For more info on Linux MM,
>> see: http://www.linux-mm.org/ .
>> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
>>
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>