Greedy kswapd reclaim behavior

[Lock Free <atomiclong64@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 1218 bytes --]

I'm trying to explain swapping out behavior that is causing
unpredictability in our app.  We’re running redhat kernel 2.6.32-431 (yes
older) on a host that has 24GB of physical memory and a swap space of 4GB.
Swappiness is set to 10, min_free_kbytes is 90112.   Over time free memory
drop down to ~180MB due to filesystem usage over a few hours, which is
immediately followed by 2GB or 4GB of memory being reclaimed.  We expect
the free memory to be used by the file system cache, and also expect kswapd
to be triggered when min_free_kbytes is breached.  However what was not
expected was the 2-4GB of memory being reclaimed.  Our understanding is
once free memory hits high water mark which is 2 x min_free_kbytes, kswapd
duty cycle finishes.   2-3GB is usually the file system cache pages,
however the other 1-2GB are anonymous pages.  It’s a issue for us to see
the anonymous pages swapped out because they correspond to a process (JVM)
whose performance is important to us.  This process virtual and resident
size is static at 15GB.  Why is kswapd so aggressive in reclaiming pages
when it clearly reclaimed more than high water immediately after the FS
cache was flushed?  Is this by design?

[-- Attachment #2: Type: text/html, Size: 1271 bytes --]