Re: new kmod.c - debuggers and testers needed

Re: new kmod.c - debuggers and testers needed

[Rik van Riel]

On Tue, 7 Apr 1998, Perry Harrington wrote:

>                                                           Threads
> are useful in their appropriate context, and kswapd, and kmod would benefit
> from them.

Hmm, maybe it would be useful for kswapd and bdflush to fork()
off threads to do the actual disk I/O, so the main thread won't
be blocked and paused... This could remove some bottlenecks.

Rik.
+-------------------------------------------+--------------------------+
| Linux: - LinuxHQ MM-patches page          | Scouting       webmaster |
|        - kswapd ask-him & complain-to guy | Vries    cubscout leader |
|     http://www.fys.ruu.nl/~riel/          | <H.H.vanRiel@fys.ruu.nl> |
+-------------------------------------------+--------------------------+

Re: new kmod.c - debuggers and testers needed

[Perry Harrington]

> 
> On Tue, 7 Apr 1998, Perry Harrington wrote:
> 
> >                                                           Threads
> > are useful in their appropriate context, and kswapd, and kmod would benefit
> > from them.
> 
> Hmm, maybe it would be useful for kswapd and bdflush to fork()
> off threads to do the actual disk I/O, so the main thread won't
> be blocked and paused... This could remove some bottlenecks.

I was thinking that kswapd could use some of it's spare time to do an LRU
paging scan, consolidate free space, and possibly do remapping of process
memory spaces to make them more efficient (map pages to contiguous chunks
of memory and swap).

> 
> Rik.

--Perry

-- 
Perry Harrington       Linux rules all OSes.    APSoft      ()
email: perry@apsoft.com 			Think Blue. /\

Kswapd future (was: Re: new kmod.c - debuggers and testers needed)

[Rik van Riel]

On Tue, 14 Apr 1998, Perry Harrington wrote:

> > On Tue, 7 Apr 1998, Perry Harrington wrote:
> > 
> > >                                                           Threads
> > > are useful in their appropriate context, and kswapd, and kmod would benefit
> > > from them.
> > 
> > Hmm, maybe it would be useful for kswapd and bdflush to fork()
> > off threads to do the actual disk I/O, so the main thread won't
> > be blocked and paused... This could remove some bottlenecks.
> 
> I was thinking that kswapd could use some of it's spare time to do an LRU
> paging scan, consolidate free space, and possibly do remapping of process
> memory spaces to make them more efficient (map pages to contiguous chunks
> of memory and swap).

Unfortunately, kswapd doesn't have such a thing as 'spare time'.
It is a kernel daemon that does what's neccessary, but we want
to keep it's CPU/IO footprint absolutely minimal since all other
applications are more important...

Rik.
+-------------------------------------------+--------------------------+
| Linux: - LinuxHQ MM-patches page          | Scouting       webmaster |
|        - kswapd ask-him & complain-to guy | Vries    cubscout leader |
|     http://www.fys.ruu.nl/~riel/          | <H.H.vanRiel@fys.ruu.nl> |
+-------------------------------------------+--------------------------+

Re: new kmod.c - debuggers and testers needed

[Benjamin C.R. LaHaise]

On Tue, 14 Apr 1998, Perry Harrington wrote:
...
> > Hmm, maybe it would be useful for kswapd and bdflush to fork()
> > off threads to do the actual disk I/O, so the main thread won't
> > be blocked and paused... This could remove some bottlenecks.

It's not bottlenecks so much as it is the need to do some things
speculatively, but what I have in mind is too much to do for now.

> I was thinking that kswapd could use some of it's spare time to do an LRU
> paging scan, consolidate free space, and possibly do remapping of process
> memory spaces to make them more efficient (map pages to contiguous chunks
> of memory and swap).

Two techniques that I think should help are:
	1. make it easier to free contiguous pages (that's done and
	working for me)
	2. change the allocation strategy to make our common cases not
	fragment memory all over the place - ie switch to a zone allocator
	for pages

It's far too late in 2.1 for any truely radical changes to combat
fragmention to go into the mainstream kernel, hence we need to stick to
tweaking the behaviour of our existing code.  Perhaps get_free_page
shouldn't break any of the higher memory order allocations for GFP_USER
requests when such actions will leave insufficient high-order pages
around.  Couple that with kswapd now checking the available memory orders
and normal systems shouldn't suffer swapout attacks (or at least those
caused by a lack of free high-order page).

		-ben

Re: new kmod.c - debuggers and testers needed

[Stephen C. Tweedie]

Hi,

On Tue, 14 Apr 1998 20:02:09 +0200 (MET DST), Rik van Riel
<H.H.vanRiel@fys.ruu.nl> said:

> On Tue, 7 Apr 1998, Perry Harrington wrote:
>> Threads
>> are useful in their appropriate context, and kswapd, and kmod would benefit
>> from them.

> Hmm, maybe it would be useful for kswapd and bdflush to fork()
> off threads to do the actual disk I/O, so the main thread won't
> be blocked and paused... This could remove some bottlenecks.

bdflush does nothing except IO, so there's no real reason to
twin-thread it.  kswapd does indeed benefit from a separate IO thread,
and I've already got patches which implement a kswiod for IO and a
kswapd for page scanning.  I'll post them once I've got them ready
against the latest kernel: my current patches for this code are pretty
old.

--Stephen