* 8 Gigabytes and constantly swapping
@ 2017-05-12 8:51 Arthur Marsh
2017-05-12 10:56 ` Tetsuo Handa
2017-05-15 8:09 ` Michal Hocko
0 siblings, 2 replies; 6+ messages in thread
From: Arthur Marsh @ 2017-05-12 8:51 UTC (permalink / raw)
To: linux-mm
I've been building the Linus git head kernels as the source gets updated
and the one built about 3 hours ago managed to get stuck with kswapd0 as
the highest consumer of CPU cycles (but still under 1 percent) of
processes listed by top for over 15 minutes, after which I hit the power
switch and rebooted with a Debian 4.11.0 kernel.
The previous kernel built less than 24 hours earlier did not have this
problem.
CPU is an Athlon64 (Athlon II X4, 4 cores), RAM is 8GiB, swap is 4GiB,
load was mainly firefox and chromium. Opening a new window in chromium
seemed to help trigger the problem.
It's not much information to go on, just wondered if anyone else had
experienced similar issues?
I'm happy to supply more configuration information and run tests
including with kernels built with test patches applied.
Arthur.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: 8 Gigabytes and constantly swapping
2017-05-12 8:51 8 Gigabytes and constantly swapping Arthur Marsh
@ 2017-05-12 10:56 ` Tetsuo Handa
2017-05-12 13:39 ` Arthur Marsh
2017-05-15 8:09 ` Michal Hocko
1 sibling, 1 reply; 6+ messages in thread
From: Tetsuo Handa @ 2017-05-12 10:56 UTC (permalink / raw)
To: Arthur Marsh; +Cc: linux-mm
On 2017/05/12 17:51, Arthur Marsh wrote:
> I've been building the Linus git head kernels as the source gets updated and
> the one built about 3 hours ago managed to get stuck with kswapd0 as the highest
> consumer of CPU cycles (but still under 1 percent) of processes listed by top for
> over 15 minutes, after which I hit the power switch and rebooted with a Debian
> 4.11.0 kernel.
Did the /bin/top process continue showing up-to-dated statistics rather than
refrain from showing up-to-dated statistics? (I wonder why you had to hit
the power switch before trying SysRq-m/SysRq-f etc.)
If yes, assuming that reading statistics involves memory allocation requests,
there was no load at at all despite firefox and chromium were running?
If no, all allocation requests got stuck waiting for memory reclaim?
>
> The previous kernel built less than 24 hours earlier did not have this problem.
>
> CPU is an Athlon64 (Athlon II X4, 4 cores), RAM is 8GiB, swap is 4GiB, load was
> mainly firefox and chromium. Opening a new window in chromium seemed to help
> trigger the problem.
>
> It's not much information to go on, just wondered if anyone else had experienced
> similar issues?
>
> I'm happy to supply more configuration information and run tests including with
> kernels built with test patches applied.
I don't know but http://lkml.kernel.org/r/20170316100409.GR802@shells.gnugeneration.com
or http://lkml.kernel.org/r/20170502041235.zqmywvj5tiiom3jk@merlins.org ?
More description of what happened and how you confirmed that
the /bin/top process continued working would be helpful.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: 8 Gigabytes and constantly swapping
2017-05-12 10:56 ` Tetsuo Handa
@ 2017-05-12 13:39 ` Arthur Marsh
2017-05-13 4:15 ` Arthur Marsh
0 siblings, 1 reply; 6+ messages in thread
From: Arthur Marsh @ 2017-05-12 13:39 UTC (permalink / raw)
To: Tetsuo Handa; +Cc: linux-mm
Tetsuo Handa wrote on 12/05/17 20:26:
> On 2017/05/12 17:51, Arthur Marsh wrote:
>> I've been building the Linus git head kernels as the source gets updated and
>> the one built about 3 hours ago managed to get stuck with kswapd0 as the highest
>> consumer of CPU cycles (but still under 1 percent) of processes listed by top for
>> over 15 minutes, after which I hit the power switch and rebooted with a Debian
>> 4.11.0 kernel.
>
> Did the /bin/top process continue showing up-to-dated statistics rather than
> refrain from showing up-to-dated statistics? (I wonder why you had to hit
> the power switch before trying SysRq-m/SysRq-f etc.)
Yes, the /bin/top process continued updating.
The load average reached a high figure (over 14).
kswapd0 was using more CPU cycles than anything else but was still under
1 percent.
There was still a lot of swap space free (about 3 GiB).
buffer/cache dropped below 1 GiB out of 8 GiB RAM.
I would have used alt-sysrq-f but had only recently enabled it in my
kernel builds and didn't think to look it up on my other pc.
>
> If yes, assuming that reading statistics involves memory allocation requests,
> there was no load at at all despite firefox and chromium were running?
Wait time was over 98 percent, load average over 14.
>
> If no, all allocation requests got stuck waiting for memory reclaim?
...
> More description of what happened and how you confirmed that
> the /bin/top process continued working would be helpful.
>
>
It was as if kswapd0 was just left waiting to swap pages in and out
without any other processes getting to complete what they were trying to do.
It does seem to be related to chromium starting up several processes
when opening extra windows/tabs.
Thanks for your help!
Arthur.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: 8 Gigabytes and constantly swapping
2017-05-12 13:39 ` Arthur Marsh
@ 2017-05-13 4:15 ` Arthur Marsh
0 siblings, 0 replies; 6+ messages in thread
From: Arthur Marsh @ 2017-05-13 4:15 UTC (permalink / raw)
To: Tetsuo Handa; +Cc: linux-mm
Arthur Marsh wrote on 12/05/17 23:09:
> It does seem to be related to chromium starting up several processes
> when opening extra windows/tabs.
I tried using ionice -c3 on chromium, which meant that disk I/O by
chromium should only occur when disk I/O was otherwise idle, and this
made a major difference.
It appears that chromium's use of multiple processes and threads means
that it effectively hogs all the disk I/O but ends up not being able to
do much as different processes/threads are making different demands on
disk I/O, leading to kswapd0 having the greatest CPU usage of any
process but still under 1 percent of available CPU time and the system
spending over 99 percent of the time in wait. All while about 1 GiB of 4
GiB available swap is being used.
Is there any existing way to limit the chromium process tree to have the
same amount of access to disk I/O as say firefox running as a single
process?
Arthur.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: 8 Gigabytes and constantly swapping
2017-05-12 8:51 8 Gigabytes and constantly swapping Arthur Marsh
2017-05-12 10:56 ` Tetsuo Handa
@ 2017-05-15 8:09 ` Michal Hocko
2017-05-17 1:27 ` Arthur Marsh
1 sibling, 1 reply; 6+ messages in thread
From: Michal Hocko @ 2017-05-15 8:09 UTC (permalink / raw)
To: Arthur Marsh; +Cc: linux-mm
On Fri 12-05-17 18:21:27, Arthur Marsh wrote:
> I've been building the Linus git head kernels as the source gets updated and
> the one built about 3 hours ago managed to get stuck with kswapd0 as the
> highest consumer of CPU cycles (but still under 1 percent) of processes
> listed by top for over 15 minutes, after which I hit the power switch and
> rebooted with a Debian 4.11.0 kernel.
>
> The previous kernel built less than 24 hours earlier did not have this
> problem.
>
> CPU is an Athlon64 (Athlon II X4, 4 cores), RAM is 8GiB, swap is 4GiB, load
> was mainly firefox and chromium. Opening a new window in chromium seemed to
> help trigger the problem.
>
> It's not much information to go on, just wondered if anyone else had
> experienced similar issues?
>
> I'm happy to supply more configuration information and run tests including
> with kernels built with test patches applied.
Is this 32b or 64b kernel? Could you take /proc/vmstat snapshots ever
second while the kswapd is active?
--
Michal Hocko
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: 8 Gigabytes and constantly swapping
2017-05-15 8:09 ` Michal Hocko
@ 2017-05-17 1:27 ` Arthur Marsh
0 siblings, 0 replies; 6+ messages in thread
From: Arthur Marsh @ 2017-05-17 1:27 UTC (permalink / raw)
To: Michal Hocko; +Cc: linux-mm
Michal Hocko wrote on 15/05/17 17:39:
> Is this 32b or 64b kernel? Could you take /proc/vmstat snapshots ever
> second while the kswapd is active?
>
This was with a 64 bit kernel.
Thankfully the problem does not seem to affect 4.12.0-rc1 and later
kernels, with the amount of swap used and amount of time spent waiting
on kswapd not being a problem even under heavy load.
Arthur.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2017-05-17 1:27 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-05-12 8:51 8 Gigabytes and constantly swapping Arthur Marsh
2017-05-12 10:56 ` Tetsuo Handa
2017-05-12 13:39 ` Arthur Marsh
2017-05-13 4:15 ` Arthur Marsh
2017-05-15 8:09 ` Michal Hocko
2017-05-17 1:27 ` Arthur Marsh
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox