ideas for a swapping daemon

ideas for a swapping daemon

[Benjamin LaHaise]

Date: Thu, 18 Dec 1997 15:28:07 +0100 (MET)
From: Rik van Riel <H.H.vanRiel@fys.ruu.nl>
X-Sender: riel@mirkwood.dummy.home
Reply-To: H.H.vanRiel@fys.ruu.nl
To: linux-mm <linux-mm@kvack.org>
cc: linux-kernel <linux-kernel@vger.rutgers.edu>
Subject: ideas for a swapping daemon
Message-ID: <Pine.LNX.3.91.971218151905.15652C-100000@mirkwood.dummy.home>
Approved: ObHack@localhost
Organization: none
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII

THIS FIRST MESSAGE WILL BE ON LINUX-KERNEL, I hope you'll all
F-up to linux-mm ONLY!!!
---------
Hi,

here it is, the thread about the swapping daemon <g>

I have the following ideas about the swapping daemon:

- it should swap out some processes if the paging daemon
  (kswapd) doesn't manage to keep nr_free_pages above
  (min_free_pages + free_pages_low)/2 for some time (ie: 5 seconds)
- then it should choose a low-priority, or long-sleeping program
  and swap it out (using the kswapd swapping mechanism, but using
  a 'force' flag, which could be implemented as an argument to the
  routines).
- as long as buffer+cache memory take up more than 1/3 of memory,
  it should try swapping that memory as well
- after a program has been swapped out, it can be swapped in by:
  - being swapped out some time (in case of a low-prio process swap)
  - being woken up (in case of a sleeping program being swapped out)

This is a very rough outline of what the swapping process should
do, and more ideas are needed...

looking forward to your comments,

Rik.

PS: I'd like this thread to continue on linux-mm only. The
linux-kernel people who'd like to follow it too, can easily
subscribe to linux-mm@kvack.org (majordomo@kvack.org) since
linux-mm tends to a quiet list...
+-----------------------------+------------------------------+
| For Linux mm-patches, go to | "I'm busy managing memory.." |
| my homepage (via LinuxHQ).  | H.H.vanRiel@fys.ruu.nl       |
| ...submissions welcome...   | http://www.fys.ruu.nl/~riel/ |
+-----------------------------+------------------------------+

Re: ideas for a swapping daemon

[Moshe Zadka]

Seems like a good idea. But experience with vhand
shows that what we should be most careful about is 
waking the swapping daemon too often, consuming 
resources. So, if anyone is writing it (I might
start by patching kswapd's routines to allow 
"forcing"), please keep an easily configurable
parameter (perhaps run-time adjusted through proc-fs)
controlling how often it is woken up, for easier
determining of the heuristics.

-- 
Moshe Zadka - moshez@math.huji.ac.il
Violating RFC2049 Should Be a Shooting Offense

Re: ideas for a swapping daemon

[Rik van Riel]

On Thu, 18 Dec 1997, Moshe Zadka wrote:

> Seems like a good idea. But experience with vhand
> shows that what we should be most careful about is 
> waking the swapping daemon too often, consuming 
> resources. So, if anyone is writing it (I might
> start by patching kswapd's routines to allow 
> "forcing"), please keep an easily configurable
> parameter (perhaps run-time adjusted through proc-fs)
> controlling how often it is woken up, for easier
> determining of the heuristics.

The kswapd part I can do in several hours too, the real
issue is scheduling... If we have the scheduling part
correct, the kswapd part is only optional...

We should start by adding SLEEP_TIME and IN_CORE time
entries to the task_struct. Then we have to write a
mechanism to wake up the daemon to decide to swap which
program.

Only then are the extra lines to kswapd functional
(otherwise, the system will be paging in the process we
just swapped out :-)

What we need first is a generally agreed upon algorithm
on how to decide:
- when to swap (tunable)
- what to swap (tunable?)
- how agressive we should be (tunable)
- how long programs can be swapped out/in (tunable)

Even though most of the above is tunable, we still need
to agree on the algoritms and design the program...
When the design is finished, we can have the daemon up and
running in a few days.

Rik.
+-----------------------------+------------------------------+
| For Linux mm-patches, go to | "I'm busy managing memory.." |
| my homepage (via LinuxHQ).  | H.H.vanRiel@fys.ruu.nl       |
| ...submissions welcome...   | http://www.fys.ruu.nl/~riel/ |
+-----------------------------+------------------------------+