Re: -mm merge plans for 2.6.23

Re: -mm merge plans for 2.6.23

[Con Kolivas]

On Tuesday 10 July 2007 18:31, Andrew Morton wrote:
> When replying, please rewrite the subject suitably and try to Cc: the
> appropriate developer(s).

~swap prefetch

Nick's only remaining issue which I could remotely identify was to make it 
cpuset aware:
http://marc.info/?l=linux-mm&m=117875557014098&w=2
as discussed with Paul Jackson it was cpuset aware:
http://marc.info/?l=linux-mm&m=117895463120843&w=2

I fixed all bugs I could find and improved it as much as I could last kernel 
cycle.

Put me and the users out of our misery and merge it now or delete it forever 
please. And if the meaningless handwaving that I 100% expect as a response 
begins again, then that's fine. I'll take that as a no and you can dump it.

-- 
-ck

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Re: -mm merge plans for 2.6.23

[Matthew Hawkins]

[-- Attachment #1.1: Type: text/plain, Size: 1089 bytes --]

On 7/10/07, Con Kolivas <kernel@kolivas.org> wrote:
>
> On Tuesday 10 July 2007 18:31, Andrew Morton wrote:
> > When replying, please rewrite the subject suitably and try to Cc: the
> > appropriate developer(s).
>
> ~swap prefetch

[snip]
Put me and the users out of our misery and merge it now
[snip]

For the record; it merges, builds, and runs cleanly on x86_64 vanilla+CFS
provided sched-add_above_background_load is also merged (you need the old
one that adds an inline to sched.h, not the new one that depends on
SD-isms).  I believe that is already merged with -mm anyway.

I'd also be interested to see if there is a better way of doing what
above_background_load() does with CFS, I think v18 added some functionality
along these lines...

We all know swap prefetch has been tested out the wazoo since Moses was a
little boy, is compile-time and runtime selectable, and gives an important
and quantifiable performance increase to desktop systems.  Save a Redhat
employee some time reinventing the wheel and just merge it.  This wheel
already has dope 21" rims, homes ;-)

-- 
Matt

[-- Attachment #1.2: Type: text/html, Size: 1438 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [ck] Re: -mm merge plans for 2.6.23

[Andrew Morton]

On Wed, 11 Jul 2007 11:02:56 +1000 "Matthew Hawkins" <darthmdh@gmail.com> wrote:

> We all know swap prefetch has been tested out the wazoo since Moses was a
> little boy, is compile-time and runtime selectable, and gives an important
> and quantifiable performance increase to desktop systems.

Always interested.  Please provide us more details on your usage and
testing of that code.  Amount of memory, workload, observed results,
etc?

>  Save a Redhat
> employee some time reinventing the wheel and just merge it.  This wheel
> already has dope 21" rims, homes ;-)

ooh, kernel bling.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: Re: -mm merge plans for 2.6.23

[André Goddard Rosa]

[-- Attachment #1.1: Type: text/plain, Size: 797 bytes --]

On 7/10/07, Andrew Morton <akpm@linux-foundation.org> wrote:
>
> On Wed, 11 Jul 2007 11:02:56 +1000 "Matthew Hawkins" <darthmdh@gmail.com>
> wrote:
>
> > We all know swap prefetch has been tested out the wazoo since Moses was
> a
> > little boy, is compile-time and runtime selectable, and gives an
> important
> > and quantifiable performance increase to desktop systems.
>
> Always interested.  Please provide us more details on your usage and
> testing of that code.  Amount of memory, workload, observed results,
> etc?
>
>
It keeps my machine responsive after some time of inactivity,
i.e.  when I try to use firefox in the morning after leaving it running
overnight with multiple tabs open. I have 1Gb of memory in this machine.

With regards,
-- 
[]s,
André Goddard

[-- Attachment #1.2: Type: text/html, Size: 1171 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [ck] Re: -mm merge plans for 2.6.23

[André Goddard Rosa]

On 7/10/07, Andre Goddard Rosa <andre.goddard@gmail.com> wrote:
> On 7/10/07, Andrew Morton <akpm@linux-foundation.org> wrote:
> > On Wed, 11 Jul 2007 11:02:56 +1000 "Matthew Hawkins" <darthmdh@gmail.com>
> wrote:
> >
> > > We all know swap prefetch has been tested out the wazoo since Moses was
> a
> > > little boy, is compile-time and runtime selectable, and gives an
> important
> > > and quantifiable performance increase to desktop systems.
> >
> > Always interested.  Please provide us more details on your usage and
> > testing of that code.  Amount of memory, workload, observed results,
> > etc?
> >

It keeps my machine responsive after some time of inactivity,
i.e.  when I try to use firefox in the morning after leaving it running
overnight with multiple tabs open. I have 1Gb of memory in this machine.

With regards,
-- 
[]s,
Andre Goddard

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Ira Snyder]

[-- Attachment #1: Type: text/plain, Size: 1376 bytes --]

On Tue, 10 Jul 2007 18:14:19 -0700
Andrew Morton <akpm@linux-foundation.org> wrote:

> On Wed, 11 Jul 2007 11:02:56 +1000 "Matthew Hawkins" <darthmdh@gmail.com> wrote:
> 
> > We all know swap prefetch has been tested out the wazoo since Moses was a
> > little boy, is compile-time and runtime selectable, and gives an important
> > and quantifiable performance increase to desktop systems.
> 
> Always interested.  Please provide us more details on your usage and
> testing of that code.  Amount of memory, workload, observed results,
> etc?
> 

I often leave long compiles running overnight (I'm a gentoo user). I always have the desktop running, with quite a few applications open, usually firefox, amarok, sylpheed, and liferea at the minimum. I've recently tried using a "stock" gentoo kernel, without the swap prefetch patch, and in the morning when I get on the computer, it hits the disk pretty hard pulling my applications (especially firefox) in from swap. With swap prefetch, the system responds like I expect: quick. It doesn't hit the swap at all, at least that I can tell.

Swap prefetch definitely makes a difference for me: it makes my experience MUCH better.

My system is a Core Duo 1.83GHz laptop, with 1GB ram and a 5400 rpm disk. With the disk being so slow, the less I hit swap, the better.

I'll cast my vote to merge swap prefetch.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [ck] Re: -mm merge plans for 2.6.23

[timotheus]

[-- Attachment #1: Type: text/plain, Size: 2484 bytes --]

Ira Snyder <kernel@irasnyder.com> writes:

>> Always interested.  Please provide us more details on your usage and
>> testing of that code.  Amount of memory, workload, observed results,
>> etc?
>> 
>
> I often leave long compiles running overnight (I'm a gentoo user). I
> always have the desktop running, with quite a few applications open,
> usually firefox, amarok, sylpheed, and liferea at the minimum. I've
> recently tried using a "stock" gentoo kernel, without the swap
> prefetch patch, and in the morning when I get on the computer, it hits
> the disk pretty hard pulling my applications (especially firefox) in
> from swap. With swap prefetch, the system responds like I expect:
> quick. It doesn't hit the swap at all, at least that I can tell.
>
> Swap prefetch definitely makes a difference for me: it makes my
> experience MUCH better.
>
> My system is a Core Duo 1.83GHz laptop, with 1GB ram and a 5400 rpm
> disk. With the disk being so slow, the less I hit swap, the better.
>
> I'll cast my vote to merge swap prefetch.

Very similar experiences. Other usage patterns that swap prefetch can
cause improvements with:

- Idling VMware session with large memory. Since VMware (server) can use
  mixed swap/RAM, the prefetch allows it swap back into RAM without
  having to make the application active in the foreground.

- Firefox, OO Office, long from-source compilations, all of the normal.

- My largest RAM capacity machine is a Core 2 Duo Laptop with 2 GB of
  RAM. It still benefits from the prefetch after running long
  compilations or backups.

- Also, I have an old Pentium 4 server (1.3 GHz, original RDRAM, ...)
  that uses the CK patches including swap prefetch. It has only 640 MB
  of RAM, and runs GBytes of data backup every night. The swap is split
  among multiple disks, and can easily fill .5 GBytes over
  night. Applications that run in a VNC session, web browsers, office
  programs, etc., all resume much faster with the prefetch. Even the
  intial ssh-login appears snappier; but I think that is just CK's fine
  work elsewhere. :)

I am curious how much of the benefit is due to prefetch, or due to CK
using `vm_mapped' rather than `vm_swappiness'. Swappiness always seemed
such an unbenificial hack to me...

(The past 6 months I've tried weeks/months of using various kernels,
-mm, -ck, vanilla, genpatches, combinations there of -- x86 and ppc.)

I vote for prefetch and `vm_mapped'.

[-- Attachment #2: Type: application/pgp-signature, Size: 188 bytes --]

Re: [ck] Re: -mm merge plans for 2.6.23

[Matthew Hawkins]

On 7/11/07, Andrew Morton <akpm@linux-foundation.org> wrote:
> On Wed, 11 Jul 2007 11:02:56 +1000 "Matthew Hawkins" <darthmdh@gmail.com> wrote:

> Always interested.  Please provide us more details on your usage and
> testing of that code.  Amount of memory, workload, observed results,
> etc?

My usual workstation has 1Gb of ram & 2Gb of swap (single partition -
though in the past with multiple drives I would spread swap around the
less-used disks & fiddle with the priority).  Its acting as server for
my home network too (so it has squid, cups, bind, dhcpd, apache, mysql
& postgresql) but for the most part I'll have Listen playing music
while I switch between Flock &/or Firefox, Thunderbird, and
xvncviewer.  On the odd occasion I'll fire up some game (gewled,
actioncube, critical mass).  Compiling these days has been mostly
limited to kernels, I've been building mostly -ck and -cfs - keeping
up-to-date and also doing some odd things (like patching the non-SD
-ck stuff on top of CFS).  Mainly just to get swap prefetch, but also
not to lose skills since I'm out of the daily coding routine now.

Anyhow with swap prefetch, applications that may have been sitting
there idle for a while become responsive in the single-digit seconds
rather than double-digit or worse.  The same goes for a morning wakeup
(ie after nightly cron jobs throw things out) and also after doing
basically anything that wants memory, like benchmarking the various
kernels I'm messing with or doing some local DB work or coding a
memory leak into a web application running under apache ;)

-- 
Matt

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Nick Piggin]

Matthew Hawkins wrote:
> On 7/11/07, Andrew Morton <akpm@linux-foundation.org> wrote:

> Anyhow with swap prefetch, applications that may have been sitting
> there idle for a while become responsive in the single-digit seconds
> rather than double-digit or worse.  The same goes for a morning wakeup
> (ie after nightly cron jobs throw things out)

OK that's a good data point. It would be really good to be able to
do an analysis on your overnight IO patterns and the corresponding
memory reclaim behaviour and see why things are getting evicted.

Not that swap prefetching isn't a good solution for this situation,
but the fact that things are getting swapped out for you also means
that mapped files and possibly important pagecache and dentries are
being flushed out, which we might be able to avoid.

Thanks,
Nick

-- 
SUSE Labs, Novell Inc.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Ray Lee]

On 7/10/07, Nick Piggin <nickpiggin@yahoo.com.au> wrote:
> Matthew Hawkins wrote:
> > On 7/11/07, Andrew Morton <akpm@linux-foundation.org> wrote:
>
> > Anyhow with swap prefetch, applications that may have been sitting
> > there idle for a while become responsive in the single-digit seconds
> > rather than double-digit or worse.  The same goes for a morning wakeup
> > (ie after nightly cron jobs throw things out)
>
> OK that's a good data point. It would be really good to be able to
> do an analysis on your overnight IO patterns and the corresponding
> memory reclaim behaviour and see why things are getting evicted.

Eviction can happen for multiple reasons, as I'm sure you're painfully
aware. It can happen because of poor balancing choices, or it can
happen because the system is just short of RAM for the workload. As
for the former, you're absolutely right, it would be good to know
where those come from and see if they can be addressed.

However, it's the latter that swap prefetch can help and no amount of
fiddling with the aging code can address.

As an honest question, what's it going to take here? If I were to
write something that watched the task stats at process exit (cool
feature, that), and recorded the IO wait time or some such, and showed
it was lower with a kernel with the prefetch, would *that* get us some
forward motion on this?

I mean, from my point of view, it's a simple mental proof to show that
if you're out of RAM for your working set, things that you'll
eventually need again will get kicked out, and prefetch will bring
those back in before normal access patterns would fault them back in
under today's behavior. That seems like an obvious win. Where's the
corresponding obvious loss that makes this a questionable addition to
the kernel?

Ray

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Nick Piggin]

Ray Lee wrote:
> On 7/10/07, Nick Piggin <nickpiggin@yahoo.com.au> wrote:
> 
>> Matthew Hawkins wrote:
>> > On 7/11/07, Andrew Morton <akpm@linux-foundation.org> wrote:
>>
>> > Anyhow with swap prefetch, applications that may have been sitting
>> > there idle for a while become responsive in the single-digit seconds
>> > rather than double-digit or worse.  The same goes for a morning wakeup
>> > (ie after nightly cron jobs throw things out)
>>
>> OK that's a good data point. It would be really good to be able to
>> do an analysis on your overnight IO patterns and the corresponding
>> memory reclaim behaviour and see why things are getting evicted.
> 
> 
> Eviction can happen for multiple reasons, as I'm sure you're painfully
> aware. It can happen because of poor balancing choices, or it can

s/balancing/reclaim, yes. And for the nightly cron job case, this is
could quite possibly be the cause. At least updatedb should be fairly
easy to apply use-once heuristics for, so if they're not working then
we should hopefully be able to improve it.

-- 
SUSE Labs, Novell Inc.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Ray Lee]

On 7/10/07, Nick Piggin <nickpiggin@yahoo.com.au> wrote:
> >> OK that's a good data point. It would be really good to be able to
> >> do an analysis on your overnight IO patterns and the corresponding
> >> memory reclaim behaviour and see why things are getting evicted.
> >
> > Eviction can happen for multiple reasons, as I'm sure you're painfully
> > aware. It can happen because of poor balancing choices, or it can
>
> s/balancing/reclaim, yes. And for the nightly cron job case, this is
> could quite possibly be the cause. At least updatedb should be fairly
> easy to apply use-once heuristics for, so if they're not working then
> we should hopefully be able to improve it.

<nod> Sorry, I'm not so clear on the terminology, am I.

So, that's one part of it: one could argue that for that bit swap
prefetch is a bit of a band-aid over the issue. A useful band-aid,
that works today, isn't invasive, and can be ripped out at some future
time if the underlying issue is eventually solved by a proper use-once
aging mechanism, but nevertheless a band-aid.

The other part is when I've got evolution and a few other things open,
then I run gimp on a raw photo and do some work on it, quit out of
gimp, do a couple of things in a shell to upload the photo to my
server, then switch back to evolution. Hang, waiting on swap in. Well,
the kernel had some free time there to repopulate evolution's working
set, and swap prefetch would help that, while better (or perfect!)
heuristics in the reclaim *won't*.

That's the real issue here.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Nick Piggin]

Ray Lee wrote:
> On 7/10/07, Nick Piggin <nickpiggin@yahoo.com.au> wrote:
> 
>> >> OK that's a good data point. It would be really good to be able to
>> >> do an analysis on your overnight IO patterns and the corresponding
>> >> memory reclaim behaviour and see why things are getting evicted.
>> >
>> > Eviction can happen for multiple reasons, as I'm sure you're painfully
>> > aware. It can happen because of poor balancing choices, or it can
>>
>> s/balancing/reclaim, yes. And for the nightly cron job case, this is
>> could quite possibly be the cause. At least updatedb should be fairly
>> easy to apply use-once heuristics for, so if they're not working then
>> we should hopefully be able to improve it.
> 
> 
> <nod> Sorry, I'm not so clear on the terminology, am I.
> 
> So, that's one part of it: one could argue that for that bit swap
> prefetch is a bit of a band-aid over the issue. A useful band-aid,
> that works today, isn't invasive, and can be ripped out at some future
> time if the underlying issue is eventually solved by a proper use-once
> aging mechanism, but nevertheless a band-aid.

I think for some workloads it is probably a bandaid, and for others
the concept of prefetching likely to be used again data back in is
undeniably going to be a win for others.

A lot of postitive reports I have seen about this say that desktop
the next morning is more responsive. So I kind of want to know what's
happening here -- as far as I can tell, swap prefetching shouldn't
help a huge amount to recover from a simple updatedb alone --
although if other cron stuff happened that used a bit more memory
afterwards and pushing out some of updatedb's cache, perhaps that's
when swap prefetching finds its niche. I don't know.

However, I don't like the fact that there is _any_ swap happening
on 1GB desktops after a single updatedb run. Is something else
running that hogs a huge amount of memory? Maybe that explains it,
but I don't know. I do know that we probably don't do very good
use-once algorithms on the dentry and inode caches, so updatedb
might cause them to push swap out. We could test that by winding
the vfs reclaim right up.


> The other part is when I've got evolution and a few other things open,
> then I run gimp on a raw photo and do some work on it, quit out of
> gimp, do a couple of things in a shell to upload the photo to my
> server, then switch back to evolution. Hang, waiting on swap in. Well,
> the kernel had some free time there to repopulate evolution's working
> set, and swap prefetch would help that, while better (or perfect!)
> heuristics in the reclaim *won't*.
> 
> That's the real issue here.

Yeah that's an issue, and swap prefetching has the potential to
help there no doubt at all. How much is the saving? I don't think
it will be like an order of magnitude because unfortunately we
also get mapped pagecache being thrown out as well as swap, so
for example all your evolution mailbox, libraries, executable,
etc. is still going to have to be paged back in.

Regarding swap prefetching. I'm not going to argue for or against
it anymore because I have really stopped following where it is
up to, for now. If the code and the results meet the standard that
Andrew wants then I don't particularly mind if he merges it.

It would be nice if some of you guys would still report and test
problems with reclaim when prefetching is turned off -- I have
never encountered the morning after sluggishness (although I don't
doubt for a minute that it is a problem for some).

-- 
SUSE Labs, Novell Inc.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: swap prefetch (Re: -mm merge plans for 2.6.23)

[Ingo Molnar]

* Nick Piggin <nickpiggin@yahoo.com.au> wrote:

> Regarding swap prefetching. I'm not going to argue for or against it 
> anymore because I have really stopped following where it is up to, for 
> now. If the code and the results meet the standard that Andrew wants 
> then I don't particularly mind if he merges it.

I have tested it and have read the code, and it looks fine to me. (i've 
reported my test results elsewhere already) We should include this in 
v2.6.23.

Acked-by: Ingo Molnar <mingo@elte.hu>

	Ingo

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Nick Piggin]

Ray Lee wrote:

> As an honest question, what's it going to take here? If I were to
> write something that watched the task stats at process exit (cool
> feature, that), and recorded the IO wait time or some such, and showed
> it was lower with a kernel with the prefetch, would *that* get us some
> forward motion on this?

Honest answer? Sure, why not. Numbers are good.

-- 
SUSE Labs, Novell Inc.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Grzegorz Kulewski]

On Tue, 10 Jul 2007, Andrew Morton wrote:
> On Wed, 11 Jul 2007 11:02:56 +1000 "Matthew Hawkins" <darthmdh@gmail.com> wrote:
>
>> We all know swap prefetch has been tested out the wazoo since Moses was a
>> little boy, is compile-time and runtime selectable, and gives an important
>> and quantifiable performance increase to desktop systems.
>
> Always interested.  Please provide us more details on your usage and
> testing of that code.  Amount of memory, workload, observed results,
> etc?

I am using swap prefetch in -ck kernels since it was introduced.

My machine: Athlon XP 2000MHz, 1GB DDR 266, fast SATA disk, different 
swap configurations but usually heaps of swap (2GB and/or 8GB).

My workload: desktop usage, KDE, software development, Firefox (HUGE 
memory hog), Eclipse and all that stuff (HUGE memory hog), sometimes other 
applications, sometimes some game such as Americas Army (that one will eat 
all your memory in any configuration), Konsole with heaps of tabs, usually 
some heavy compilations in the background.

Observed result (of not broken swap prefetch versions): after closing some 
memory hog (for example stopping playing game and starting to write some 
code or reloading Firefox after it leaked enough memory to nearly bring 
the system down) the disk will work for some time and after that 
everything works as expected, no heavy swap-in when switching between 
applications and so on, nearly no lags in desktop usage.

This is nearly unnoticable. Unless I have to run pure mainline. In that 
case I can notice that swap prefetch is off very quickly because after 
closing such memory hog and returning to some other application the system 
is slow for long time. Worse: after it starts to work reasonably and I try 
to switch to some other application or even try to use some dialog window 
or module of current application I have to wait, sometimes > 10s for it to 
swap back in (even if 70% of my RAM is free at that time, after memory hog 
is gone). It is painfull.

I observed similar results on my laptop (Athlon 64, 512MB RAM, slow ATA 
disk, similar workload but reduced because hardware is weak).

For me swap prefetch makes huge difference. The system lags a lot less in 
such circumstances.

Personaly I think swap prefetch is a hack. Maybe not very dirty and ugly 
but still a hack. But since:

* nobody proposed anything that can replace it and can be considered a 
no-hack,
* swap prefetch is rather well tested and shouldn't cause regressions (no 
known regressions as far as I know, the patch does not look very 
invasive, was reviewed several times, ...),
* Con said he won't make further -ck relases and won't port these patches 
to newer kernels,
* there are at least several people who see the difference,
* if somebody really hates it (s)he can turn it off

I think it could get merged, at least temporarily, before somebody can 
suggest some better or extended solution.

Personaly I would be very happy to see it in so people like me don't have 
to patch it in or (worse) port it (possibly causing bugs and filling 
additional bug reports and asking additional questions on these lists).

I even wonder if adding the opposite of swap prefetch too wouldn't be even 
better for many workloads. Something like: "when system and swap-disk is 
idle try to copy some pages to swap so when system needs memory swap-out 
could be much cheaper". I suspect patch like that can reduce startup times 
(and other operations) of great memory hogs because disk (the slowest 
device) will only have to read the application and won't have to swap-out 
half of the RAM at the same time.

I am happy to provide further info if needed.


Thanks,

Grzegorz Kulewski

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Kevin Winchester]

[-- Attachment #1: Type: text/plain, Size: 1411 bytes --]

On Tue, 10 Jul 2007 18:14:19 -0700
Andrew Morton <akpm@linux-foundation.org> wrote:

> On Wed, 11 Jul 2007 11:02:56 +1000 "Matthew Hawkins" <darthmdh@gmail.com> wrote:
> 
> > We all know swap prefetch has been tested out the wazoo since Moses was a
> > little boy, is compile-time and runtime selectable, and gives an important
> > and quantifiable performance increase to desktop systems.
> 
> Always interested.  Please provide us more details on your usage and
> testing of that code.  Amount of memory, workload, observed results,
> etc?
> 

I only have 512 MB of memory on my Athlon64 desktop box, and I switch between -mm and mainline kernels regularly.  I have noticed that -mm is always much more responsive, especially first thing in the morning.  I believe this has been due to the new schedulers in -mm (because I notice an improvement in mainline now that CFS has been merged), as well as swap prefetch.  I haven't tested swap prefetch alone to know for sure, but it seems pretty likely.

My workload is compiling kernels, with sylpheed, pidgin and firefox[1] open, and sometimes MonoDevelop if I want to slow my system to a crawl.

I will be getting another 512 MB of RAM at Christmas time, but from the other reports, it seems that swap prefetch will still be useful.

[1] Is there a graphical browser for linux that doesn't suck huge amounts of RAM?

-- 
Kevin Winchester

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [ck] Re: -mm merge plans for 2.6.23

[Jesper Juhl]

On 11/07/07, Kevin Winchester <kjwinchester@gmail.com> wrote:
[snip]
>
> [1] Is there a graphical browser for linux that doesn't suck huge amounts of RAM?
>

Dillo (http://www.dillo.org/) is really really tiny , a memory
footprint somewhere in the hundreds of K area IIRC.

links 2 (http://links.twibright.com/) has a graphical mode in addition
to the traditional text only mode (links -g) and the memory footprint
is really tiny.

-- 
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Kacper Wysocki]

On 7/11/07, Andrew Morton <akpm@linux-foundation.org> wrote:
> On Wed, 11 Jul 2007 11:02:56 +1000 "Matthew Hawkins" <darthmdh@gmail.com> wrote:
>
> > We all know swap prefetch has been tested out the wazoo since Moses was a
> > little boy, is compile-time and runtime selectable, and gives an important
> > and quantifiable performance increase to desktop systems.
>
> Always interested.  Please provide us more details on your usage and
> testing of that code.  Amount of memory, workload, observed results,
> etc?

Swap prefetch has been around for years, and it's a complete boon for
the desktop user and a noop in any other situation. In addition to the
sp_tester tool which consistently shows a definite advantage, there
are many user reports that show the noticeable improvements it has.
The many people who have tried it out have generally chosen to switch
to patched kernels because of the performance increase.

It's been discussed on the lkml many times before, we've been over
performance, testing and impact. The big question is: why *don't* we
merge it?

-Kacper

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Avuton Olrich]

On 7/12/07, Kacper Wysocki <kacperw@online.no> wrote:
> performance, testing and impact. The big question is: why *don't* we
> merge it?

Stranger thing to me is that this is like Deja Vu. Many have asked
this same question. When users were asked for their comment before
many end users and some developers have given rave reviews. I don't
remember anyone giving it the heavy thumbs-down, with exception of
when things needed fixing (over 6 months ago?). It continues to go
unmerged. Is there a clear answer on what needs to happen for it to
get merged?
-- 
avuton
--
 Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Con Kolivas]

On Tuesday 10 July 2007 20:15, Con Kolivas wrote:
> On Tuesday 10 July 2007 18:31, Andrew Morton wrote:
> > When replying, please rewrite the subject suitably and try to Cc: the
> > appropriate developer(s).
>
> ~swap prefetch
>
> Nick's only remaining issue which I could remotely identify was to make it
> cpuset aware:
> http://marc.info/?l=linux-mm&m=117875557014098&w=2
> as discussed with Paul Jackson it was cpuset aware:
> http://marc.info/?l=linux-mm&m=117895463120843&w=2
>
> I fixed all bugs I could find and improved it as much as I could last
> kernel cycle.
>
> Put me and the users out of our misery and merge it now or delete it
> forever please. And if the meaningless handwaving that I 100% expect as a
> response begins again, then that's fine. I'll take that as a no and you can
> dump it.

The window for 2.6.23 has now closed and your position on this is clear. I've 
been supporting this code in -mm for 21 months since 16-Oct-2005 without any 
obvious decision for this code forwards or backwards.

I am no longer part of your operating system's kernel's world; thus I cannot 
support this code any longer. Unless someone takes over the code base for 
swap prefetch you have to assume it is now unmaintained and should delete it.

Please respect my request to not be contacted further regarding this or any 
other kernel code.

-- 
-ck

Re: -mm merge plans for 2.6.23

[Jesper Juhl]

On 10/07/07, Con Kolivas <kernel@kolivas.org> wrote:
> On Tuesday 10 July 2007 18:31, Andrew Morton wrote:
> > When replying, please rewrite the subject suitably and try to Cc: the
> > appropriate developer(s).
>
> ~swap prefetch
>
> Nick's only remaining issue which I could remotely identify was to make it
> cpuset aware:
> http://marc.info/?l=linux-mm&m=117875557014098&w=2
> as discussed with Paul Jackson it was cpuset aware:
> http://marc.info/?l=linux-mm&m=117895463120843&w=2
>
> I fixed all bugs I could find and improved it as much as I could last kernel
> cycle.
>
> Put me and the users out of our misery and merge it now or delete it forever
> please. And if the meaningless handwaving that I 100% expect as a response
> begins again, then that's fine. I'll take that as a no and you can dump it.
>
For what it's worth; put me down as supporting the merger of swap
prefetch. I've found it useful in the past, Con has maintained it
nicely and cleaned up everything that people have pointed out - it's
mature, does no harm - let's just get it merged.  It's too late for
2.6.23-rc1 now, but let's try and get this in by -rc2 - it's long
overdue...

-- 
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Nick Piggin]

Jesper Juhl wrote:
> On 10/07/07, Con Kolivas <kernel@kolivas.org> wrote:
> 
>> On Tuesday 10 July 2007 18:31, Andrew Morton wrote:
>> > When replying, please rewrite the subject suitably and try to Cc: the
>> > appropriate developer(s).
>>
>> ~swap prefetch
>>
>> Nick's only remaining issue which I could remotely identify was to 
>> make it
>> cpuset aware:
>> http://marc.info/?l=linux-mm&m=117875557014098&w=2
>> as discussed with Paul Jackson it was cpuset aware:
>> http://marc.info/?l=linux-mm&m=117895463120843&w=2
>>
>> I fixed all bugs I could find and improved it as much as I could last 
>> kernel
>> cycle.
>>
>> Put me and the users out of our misery and merge it now or delete it 
>> forever
>> please. And if the meaningless handwaving that I 100% expect as a 
>> response
>> begins again, then that's fine. I'll take that as a no and you can 
>> dump it.
>>
> For what it's worth; put me down as supporting the merger of swap
> prefetch. I've found it useful in the past, Con has maintained it
> nicely and cleaned up everything that people have pointed out - it's
> mature, does no harm - let's just get it merged.  It's too late for
> 2.6.23-rc1 now, but let's try and get this in by -rc2 - it's long
> overdue...


Not talking about swap prefetch itself, but everytime I have asked
anyone to instrument or produce some workload where swap prefetch
helps, they never do.

Fair enough if swap prefetch helps them, but I also want to look at
why that is the case and try to improve page reclaim in some of
these situations (for example standard overnight cron jobs shouldn't
need swap prefetch on a 1 or 2GB system, I would hope).

Anyway, back to swap prefetch, I don't know why I've been singled out
as the bad guy here. I'm one of the only people who has had a look at
the damn thing and tried to point out areas where it could be improved
to the point of being included, and outlining things that are needed
for it to be merged (ie. numbers). If anyone thinks that makes me the
bad guy then they have an utterly inverted understanding of what peer
review is for.

Finally, everyone who has ever hacked on these heuristicy parts of the
VM has heaps of patches that help some workload or some silly test
case or (real or percieved) shortfall but have not been merged. It
really isn't anything personal.

If something really works, then it should be possible to get real
numbers in real situations where it helps (OK, swap prefetching won't
be as easy as a straight line performance improvement, but still much
easier than trying to measure something like scheduler interactivity).

Numbers are the best way to add weight to the pro-merge argument, so
for all the people who a whining about merging this and don't want
to actually work on the code -- post some numbers for where it helps
you!!

-- 
SUSE Labs, Novell Inc.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Ray Lee]

On 7/23/07, Nick Piggin <nickpiggin@yahoo.com.au> wrote:
> Not talking about swap prefetch itself, but everytime I have asked
> anyone to instrument or produce some workload where swap prefetch
> helps, they never do.
[...]
> so for all the people who a whining about merging this and don't want
> to actually work on the code -- post some numbers for where it helps
> you!!

<Raised eyebrow> You sound frustrated. Perhaps we could be
communicating better. I'll start.

Unlike others on the cc: line, I don't get paid to hack on the kernel,
not even indirectly. So if you find that my lack of providing numbers
is giving you heartache, I can only apologize and point at my paying
work that requires my attention.

That said, I'm willing to run my day to day life through both a swap
prefetch kernel and a normal one. *However*, before I go through all
the work of instrumenting the damn thing, I'd really like Andrew (or
Linus) to lay out his acceptance criteria on the feature. Exactly what
*should* I be paying attention to? I've suggested keeping track of
process swapin delay total time, and comparing with and without. Is
that reasonable? Is it incomplete?

Without Andrew's criteria, we're back to where we've been for a long
time: lots of work, no forward motion. Perhaps it's a character flaw
of mine, but I'd really like to know what would constitute proof here
before I invest the effort. Especially given that Con has already
written a test case that shows that swap prefetch works, and that I've
given you a clear argument for why better (or even perfect) page
reclaim can't provide full coverage to all the situations that swap
prefetch helps. (Also, it's not like I've got tons free time, y'know?
Just like all the rest of you all, I have to pick and choose my
battles if I'm going to be effective.)

Since this merge period has appeared particularly frazzling for
Andrew, I've been keeping silent and waiting for him to get to a point
where there's a breather. I didn't feel it would be polite to request
yet more work out of him while he had a mess on his hands.

But, given this has come to a head, I'm asking now.

Andrew? You've always given the impression that you want this run more
as an engineering effort than an artistic endeavour, so help us out
here. What are your concerns with swap prefetch? What sort of
comparative data would you like to see to justify its inclusion, or to
prove that it's not needed?

Or are we reading too much into the fact that it isn't merged? In
short, communicate please, it will help.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Jeremy Fitzhardinge]

Ray Lee wrote:
> That said, I'm willing to run my day to day life through both a swap
> prefetch kernel and a normal one. *However*, before I go through all
> the work of instrumenting the damn thing, I'd really like Andrew (or
> Linus) to lay out his acceptance criteria on the feature. Exactly what
> *should* I be paying attention to? I've suggested keeping track of
> process swapin delay total time, and comparing with and without. Is
> that reasonable? Is it incomplete?

Um, isn't it up to you?  The questions that need to be answered are:

   1. What are you trying to achieve?  Presumably you have some intended
      or desired effect you're trying to get.  What's the intended
      audience?  Who would be expected to see a benefit?  Who suffers?
   2. How does the code achieve that end?  Is it nasty or nice?  Has
      everyone who's interested in the affected areas at least looked at
      the changes, or ideally given them a good review?  Does it need
      lots of tunables, or is it set-and-forget?
   3. Does it achieve the intended end?  Numbers are helpful here.
   4. Does it make anything worse?  A lot or a little?  Rare corner
      cases, or a real world usage?  Again, numbers make the case most
      strongly.


I can't say I've been following this particular feature very closely,
but these are the fundamental questions that need to be dealt with in
merging any significant change.  And as Nick says, historically point 4
is very important in VM tuning changes, because "obvious" improvements
have often ended up giving pathologically bad results on unexpected
workloads.

    J

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Ray Lee]

On 7/23/07, Jeremy Fitzhardinge <jeremy@goop.org> wrote:
> Ray Lee wrote:
> > That said, I'm willing to run my day to day life through both a swap
> > prefetch kernel and a normal one. *However*, before I go through all
> > the work of instrumenting the damn thing, I'd really like Andrew (or
> > Linus) to lay out his acceptance criteria on the feature. Exactly what
> > *should* I be paying attention to? I've suggested keeping track of
> > process swapin delay total time, and comparing with and without. Is
> > that reasonable? Is it incomplete?
>
> Um, isn't it up to you?

Huh? I'm not Linus or Andrew, with the power to merge a patch to the
2.6 kernel, so I think that the answer to that is a really clear 'No.'

> 4. Does it make anything worse?  A lot or a little?  Rare corner
> cases, or a real world usage?  Again, numbers make the case most
> strongly.
>
> I can't say I've been following this particular feature very closely,
> but these are the fundamental questions that need to be dealt with in
> merging any significant change.  And as Nick says, historically point 4
> is very important in VM tuning changes, because "obvious" improvements
> have often ended up giving pathologically bad results on unexpected
> workloads.

Dude. My whole question was *what* numbers. Please go back and read it
all again. Maybe I was unclear, but I really don't think so.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Nick Piggin]

Ray Lee wrote:
> On 7/23/07, Nick Piggin <nickpiggin@yahoo.com.au> wrote:

> That said, I'm willing to run my day to day life through both a swap
> prefetch kernel and a normal one. *However*, before I go through all
> the work of instrumenting the damn thing, I'd really like Andrew (or
> Linus) to lay out his acceptance criteria on the feature. Exactly what
> *should* I be paying attention to? I've suggested keeping track of
> process swapin delay total time, and comparing with and without. Is
> that reasonable? Is it incomplete?

I don't feel it is so useful without more context. For example, in
most situations where pages get pushed to swap, there will *also* be
useful file backed pages being thrown out. Swap prefetch might
improve the total swapin delay time very significantly but that may
be just a tiny portion of the real problem.

Also a random day at the desktop, it is quite a broad scope and
pretty well impossible to analyse. If we can first try looking at
some specific problems that are easily identified.

Looking at your past email, you have a 1GB desktop system and your
overnight updatedb run is causing stuff to get swapped out such that
swap prefetch makes it significantly better. This is really
intriguing to me, and I would hope we can start by making this
particular workload "not suck" without swap prefetch (and hopefully
make it even better than it currently is with swap prefetch because
we'll try not to evict useful file backed pages as well).

After that we can look at other problems that swap prefetch helps
with, or think of some ways to measure your "whole day" scenario.

So when/if you have time, I can cook up a list of things to monitor
and possibly a patch to add some instrumentation over this updatedb
run.

Anyway, I realise swap prefetching has some situations where it will
fundamentally outperform even the page replacement oracle. This is
why I haven't asked for it to be dropped: it isn't a bad idea at all.

However, if we can improve basic page reclaim where it is obviously
lacking, that is always preferable. eg: being a highly speculative
operation, swap prefetch is not great for power efficiency -- but we
still want laptop users to have a good experience as well, right?

-- 
SUSE Labs, Novell Inc.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23 - Completely Fair Swap Prefetch

[Frank Kingswood]

Nick Piggin wrote:

> However, if we can improve basic page reclaim where it is obviously
> lacking, that is always preferable. eg: being a highly speculative
> operation, swap prefetch is not great for power efficiency -- but we
> still want laptop users to have a good experience as well, right?

Maybe we need someone (say, a Redhat engineer) to develop a "Completely 
Fair Swap Prefetch"?

Frank

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23 - Completely Fair Swap Prefetch

[Matthew Hawkins]

On 7/25/07, Frank Kingswood <frank@kingswood-consulting.co.uk> wrote:
> Nick Piggin wrote:
>
> > However, if we can improve basic page reclaim where it is obviously
> > lacking, that is always preferable. eg: being a highly speculative
> > operation, swap prefetch is not great for power efficiency -- but we
> > still want laptop users to have a good experience as well, right?
>
> Maybe we need someone (say, a Redhat engineer) to develop a "Completely
> Fair Swap Prefetch"?

swap prefetch is disabled by default on laptops.

What I like about swap prefetch is that, being completely runtime
selectable, it leaves it up to the sysadmin whether they want it or
not.  A distribution can ask a question at install time (or, better
still, since most distributions currently have separate server and
desktop installs, just do the appropriate thing depending on what is
being installed) and the sysadmin is free to alter that choice any
time in the future.  They can even set up a cron job to alter it for
them at multiple times in the future ;-)

-- 
Matt

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Ray Lee]

On 7/23/07, Nick Piggin <nickpiggin@yahoo.com.au> wrote:
> Ray Lee wrote:
> > That said, I'm willing to run my day to day life through both a swap
> > prefetch kernel and a normal one. *However*, before I go through all
> > the work of instrumenting the damn thing, I'd really like Andrew (or
> > Linus) to lay out his acceptance criteria on the feature. Exactly what
> > *should* I be paying attention to? I've suggested keeping track of
> > process swapin delay total time, and comparing with and without. Is
> > that reasonable? Is it incomplete?
>
> I don't feel it is so useful without more context. For example, in
> most situations where pages get pushed to swap, there will *also* be
> useful file backed pages being thrown out. Swap prefetch might
> improve the total swapin delay time very significantly but that may
> be just a tiny portion of the real problem.

Agreed, it's important to make sure we're not being penny-wise and
pound-foolish here.

> Also a random day at the desktop, it is quite a broad scope and
> pretty well impossible to analyse.

It is pretty broad, but that's also what swap prefetch is targetting.
As for hard to analyze, I'm not sure I agree. One can black-box test
this stuff with only a few controls. e.g., if I use the same apps each
day (mercurial, firefox, xorg, gcc), and the total I/O wait time
consistently goes down on a swap prefetch kernel (normalized by some
control statistic, such as application CPU time or total I/O, or
something), then that's a useful measurement.

> If we can first try looking at
> some specific problems that are easily identified.

Always easier, true. Let's start with "My mouse jerks around under
memory load." A Google Summer of Code student working on X.Org claims
that mlocking the mouse handling routines gives a smooth cursor under
load ([1]). It's surprising that the kernel would swap that out in the
first place.

[1] http://vignatti.wordpress.com/2007/07/06/xorg-input-thread-summary-or-something/

> Looking at your past email, you have a 1GB desktop system and your
> overnight updatedb run is causing stuff to get swapped out such that
> swap prefetch makes it significantly better. This is really
> intriguing to me, and I would hope we can start by making this
> particular workload "not suck" without swap prefetch (and hopefully
> make it even better than it currently is with swap prefetch because
> we'll try not to evict useful file backed pages as well).

updatedb is an annoying case, because one would hope that there would
be a better way to deal with that highly specific workload. It's also
pretty stat dominant, which puts it roughly in the same category as a
git diff. (They differ in that updatedb does a lot of open()s and
getdents on directories, git merely does a ton of lstat()s instead.)

Anyway, my point is that I worry that tuning for an unusual and
infrequent workload (which updatedb certainly is), is the wrong way to
go.

> After that we can look at other problems that swap prefetch helps
> with, or think of some ways to measure your "whole day" scenario.
>
> So when/if you have time, I can cook up a list of things to monitor
> and possibly a patch to add some instrumentation over this updatedb
> run.

That would be appreciated. Don't spend huge amounts of time on it,
okay? Point me the right direction, and we'll see how far I can run
with it.

> Anyway, I realise swap prefetching has some situations where it will
> fundamentally outperform even the page replacement oracle. This is
> why I haven't asked for it to be dropped: it isn't a bad idea at all.

<nod>

> However, if we can improve basic page reclaim where it is obviously
> lacking, that is always preferable. eg: being a highly speculative
> operation, swap prefetch is not great for power efficiency -- but we
> still want laptop users to have a good experience as well, right?

Absolutely. Disk I/O is the enemy, and the best I/O is one you never
had to do in the first place.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Nick Piggin]

Ray Lee wrote:
> On 7/23/07, Nick Piggin <nickpiggin@yahoo.com.au> wrote:

>> Also a random day at the desktop, it is quite a broad scope and
>> pretty well impossible to analyse.
> 
> 
> It is pretty broad, but that's also what swap prefetch is targetting.
> As for hard to analyze, I'm not sure I agree. One can black-box test
> this stuff with only a few controls. e.g., if I use the same apps each
> day (mercurial, firefox, xorg, gcc), and the total I/O wait time
> consistently goes down on a swap prefetch kernel (normalized by some
> control statistic, such as application CPU time or total I/O, or
> something), then that's a useful measurement.

I'm not saying that we can't try to tackle that problem, but first of
all you have a really nice narrow problem where updatedb seems to be
causing the kernel to completely do the wrong thing. So we start on
that.


>> If we can first try looking at
>> some specific problems that are easily identified.
> 
> 
> Always easier, true. Let's start with "My mouse jerks around under
> memory load." A Google Summer of Code student working on X.Org claims
> that mlocking the mouse handling routines gives a smooth cursor under
> load ([1]). It's surprising that the kernel would swap that out in the
> first place.
> 
> [1] 
> http://vignatti.wordpress.com/2007/07/06/xorg-input-thread-summary-or-something/ 

OK, I'm not sure what the point is though. Under heavy memory load,
things are going to get swapped out... and swap prefetch isn't going
to help there (at least, not during the memory load).

There are also other issues like whether the CPU scheduler is at fault,
etc. Interactive workloads are always the hardest to work out. updatedb
is a walk in the park by comparison.


>> Looking at your past email, you have a 1GB desktop system and your
>> overnight updatedb run is causing stuff to get swapped out such that
>> swap prefetch makes it significantly better. This is really
>> intriguing to me, and I would hope we can start by making this
>> particular workload "not suck" without swap prefetch (and hopefully
>> make it even better than it currently is with swap prefetch because
>> we'll try not to evict useful file backed pages as well).
> 
> 
> updatedb is an annoying case, because one would hope that there would
> be a better way to deal with that highly specific workload. It's also
> pretty stat dominant, which puts it roughly in the same category as a
> git diff. (They differ in that updatedb does a lot of open()s and
> getdents on directories, git merely does a ton of lstat()s instead.)

Yeah, and I suspect we might be able to do better use-once of
inode and dentry caches. It isn't really highly specific: lots
of things tend to just scan over a few files once -- updatedb
just scans a lot so the problem becomes more noticable.


> Anyway, my point is that I worry that tuning for an unusual and
> infrequent workload (which updatedb certainly is), is the wrong way to
> go.

Well it runs every day or so for every desktop Linux user, and
it has similarities with other workloads. We don't want to optimise
it at the expense of other things, but it _really_ should not be
pushing a 1-2GB desktop into swap, I don't think.


>> After that we can look at other problems that swap prefetch helps
>> with, or think of some ways to measure your "whole day" scenario.
>>
>> So when/if you have time, I can cook up a list of things to monitor
>> and possibly a patch to add some instrumentation over this updatedb
>> run.
> 
> 
> That would be appreciated. Don't spend huge amounts of time on it,
> okay? Point me the right direction, and we'll see how far I can run
> with it.

I guess /proc/meminfo, /proc/zoneinfo, /proc/vmstat, /proc/slabinfo
before and after the updatedb run with the latest kernel would be a
first step. top and vmstat output during the run wouldn't hurt either.

Thanks,
Nick

-- 
SUSE Labs, Novell Inc.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Rene Herman]

On 07/25/2007 06:06 AM, Nick Piggin wrote:

> Ray Lee wrote:

>> Anyway, my point is that I worry that tuning for an unusual and 
>> infrequent workload (which updatedb certainly is), is the wrong way to 
>> go.
> 
> Well it runs every day or so for every desktop Linux user, and it has
> similarities with other workloads.

It certainly doesn't run for me ever. Always kind of a "that's not the 
point" comment but I just keep wondering whenever I see anyone complain 
about updatedb why the _hell_ they are running it in the first place. If 
anyone who never uses "locate" for anything simply disable updatedb, the 
problem will for a large part be solved.

This not just meant as a cheap comment; while I can think of a few similar 
loads even on the desktop (scanning a browser cache, a media player indexing 
a large amount of media files, ...) I've never heard of problems _other_ 
than updatedb. So just junk that crap and be happy.

Rene.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Nick Piggin]

Rene Herman wrote:
> On 07/25/2007 06:06 AM, Nick Piggin wrote:
> 
>> Ray Lee wrote:
> 
> 
>>> Anyway, my point is that I worry that tuning for an unusual and 
>>> infrequent workload (which updatedb certainly is), is the wrong way 
>>> to go.
>>
>>
>> Well it runs every day or so for every desktop Linux user, and it has
>> similarities with other workloads.
> 
> 
> It certainly doesn't run for me ever. Always kind of a "that's not the 
> point" comment but I just keep wondering whenever I see anyone complain 
> about updatedb why the _hell_ they are running it in the first place. If 
> anyone who never uses "locate" for anything simply disable updatedb, the 
> problem will for a large part be solved.
> 
> This not just meant as a cheap comment; while I can think of a few 
> similar loads even on the desktop (scanning a browser cache, a media 
> player indexing a large amount of media files, ...) I've never heard of 
> problems _other_ than updatedb. So just junk that crap and be happy.

OK fair point, but the counter point that there are real patterns
that just use-once a lot of metadata (ls, for example. grep even.)

-- 
SUSE Labs, Novell Inc.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[david]

On Wed, 25 Jul 2007, Rene Herman wrote:

> On 07/25/2007 06:06 AM, Nick Piggin wrote:
>
>>  Ray Lee wrote:
>
>> >  Anyway, my point is that I worry that tuning for an unusual and 
>> >  infrequent workload (which updatedb certainly is), is the wrong way to 
>> >  go.
>>
>>  Well it runs every day or so for every desktop Linux user, and it has
>>  similarities with other workloads.
>
> It certainly doesn't run for me ever. Always kind of a "that's not the point" 
> comment but I just keep wondering whenever I see anyone complain about 
> updatedb why the _hell_ they are running it in the first place. If anyone who 
> never uses "locate" for anything simply disable updatedb, the problem will 
> for a large part be solved.
>
> This not just meant as a cheap comment; while I can think of a few similar 
> loads even on the desktop (scanning a browser cache, a media player indexing 
> a large amount of media files, ...) I've never heard of problems _other_ than 
> updatedb. So just junk that crap and be happy.

but if you do use locate then the alturnative becomes sitting around and 
waiting for find to complete on a regular basis.

David Lang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Rene Herman]

On 07/25/2007 07:12 AM, david@lang.hm wrote:

> On Wed, 25 Jul 2007, Rene Herman wrote:

>> It certainly doesn't run for me ever. Always kind of a "that's not the 
>> point" comment but I just keep wondering whenever I see anyone 
>> complain about updatedb why the _hell_ they are running it in the 
>> first place. If anyone who never uses "locate" for anything simply 
>> disable updatedb, the problem will for a large part be solved.
>>
>> This not just meant as a cheap comment; while I can think of a few 
>> similar loads even on the desktop (scanning a browser cache, a media 
>> player indexing a large amount of media files, ...) I've never heard 
>> of problems _other_ than updatedb. So just junk that crap and be happy.
> 
> but if you do use locate then the alturnative becomes sitting around and 
> waiting for find to complete on a regular basis.

Yes, but what's locate's usage scenario? I've never, ever wanted to use it. 
When do you know the name of something but not where it's located, other 
than situations which "which" wouldn't cover and after just having 
installed/unpacked something meaning locate doesn't know about it yet either?

Rene.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[david]

On Wed, 25 Jul 2007, Rene Herman wrote:

> On 07/25/2007 07:12 AM, david@lang.hm wrote:
>
>>  On Wed, 25 Jul 2007, Rene Herman wrote:
>
>> >  It certainly doesn't run for me ever. Always kind of a "that's not the 
>> >  point" comment but I just keep wondering whenever I see anyone complain 
>> >  about updatedb why the _hell_ they are running it in the first place. If 
>> >  anyone who never uses "locate" for anything simply disable updatedb, the 
>> >  problem will for a large part be solved.
>> > 
>> >  This not just meant as a cheap comment; while I can think of a few 
>> >  similar loads even on the desktop (scanning a browser cache, a media 
>> >  player indexing a large amount of media files, ...) I've never heard of 
>> >  problems _other_ than updatedb. So just junk that crap and be happy.
>>
>>  but if you do use locate then the alturnative becomes sitting around and
>>  waiting for find to complete on a regular basis.
>
> Yes, but what's locate's usage scenario? I've never, ever wanted to use it. 
> When do you know the name of something but not where it's located, other than 
> situations which "which" wouldn't cover and after just having 
> installed/unpacked something meaning locate doesn't know about it yet either?

which only finds executables that are in the path.

I commonly use locate to find config files (or sample config files) for 
packages that were installed at some point in the past with fairly default 
configs and now I want to go and tweak them. so I start reading 
documentation and then need to find out where $disto moved the files to 
this release (I commonly am working on machines with over a half dozen 
different distro releases, and none of them RedHat)

David Lang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1659 bytes --]

On Wed, 25 Jul 2007 07:30:37 +0200, Rene Herman said:

> Yes, but what's locate's usage scenario? I've never, ever wanted to use it. 
> When do you know the name of something but not where it's located, other 
> than situations which "which" wouldn't cover and after just having 
> installed/unpacked something meaning locate doesn't know about it yet either?

My favorite use - with 5 Fedora kernels and as many -mm kernels on my laptop,
doing a 'locate moby' finds all the moby.c and moby.o and moby.ko for
the various releases. For bonus points, something like:

ls -lt `locate iwl3945.ko`

to find all 19 copies that are on my system, and remind me which ones were
compiled when.  Or just when you remember the name of some one-off 100-line
Perl program that you wrote 6 months ago, but not sure which directory you
left it in... ;)

You want hard numbers? Here you go - 'locate' versus 'find'
(/usr/src/ has about 290K files on it):

%  strace locate iwl3945.ko  >| /tmp/foo3 2>&1
% wc /tmp/foo3
  96  592 6252 /tmp/foo3
% strace find /usr/src /lib -name iwl3945.ko >| /tmp/foo4 2>&1
% wc /tmp/foo4
  328380  1550032 15708205 /tmp/foo4

# echo 1 > /proc/sys/vm/drop_caches     (to empty the caches

% time locate iwl3945.ko > /dev/null

real    0m0.872s
user    0m0.867s
sys     0m0.008s

% time find /usr/src /lib -name iwl3945.ko > /dev/null
find: /usr/src/lost+found: Permission denied

real    1m12.241s
user    0m1.128s
sys     0m3.566s

So 96 system calls in 1 second, against 328K calls in a minute.  There's your
use case, right there.  Now if we can just find a way for that find/updatedb
to not be as painful to the rest of the system.....





[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: -mm merge plans for 2.6.23

[Rene Herman]

On 07/25/2007 09:14 AM, Valdis.Kletnieks@vt.edu wrote:

> On Wed, 25 Jul 2007 07:30:37 +0200, Rene Herman said:
> 
>> Yes, but what's locate's usage scenario? I've never, ever wanted to use
>> it. When do you know the name of something but not where it's located,
>> other than situations which "which" wouldn't cover and after just
>> having installed/unpacked something meaning locate doesn't know about
>> it yet either?
> 
> My favorite use - with 5 Fedora kernels and as many -mm kernels on my
> laptop, doing a 'locate moby' finds all the moby.c and moby.o and moby.ko
> for the various releases.

Supposing you know the path in one tree, you know the path in all of them, 
right? :-?

> You want hard numbers? Here you go - 'locate' versus 'find'

These are ofcourse not necesary. If you discount the time updatedb itself 
takes it's utterly obvious that _if_ you use it, it's going to be wildly 
faster than find.

Regardless, I'll stand by "[by disabling updatedb] the problem will for a 
large part be solved" as I expect approximately 94.372 percent of Linux 
desktop users couldn't care less about locate.

Rene.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Ingo Molnar]

* Rene Herman <rene.herman@gmail.com> wrote:

> Regardless, I'll stand by "[by disabling updatedb] the problem will 
> for a large part be solved" as I expect approximately 94.372 percent 
> of Linux desktop users couldn't care less about locate.

i think that approach is illogical: because Linux mis-handled a mixed 
workload the answer is to ... remove a portion of that workload?

To bring your approach to the extreme: what if Linux sucked at running 
more than two CPU-intense tasks at once. Most desktop users dont do 
that, so a probably larger than 94.372 percent of Linux desktop users 
couldn't care less about a proper scheduler. Still, anyone who builds a 
kernel (the average desktop user wont do that) while using firefox will 
attest to the fact that it's quite handy that the Linux scheduler can 
handle mixed workloads pretty well.

now, it might be the case that this mixed VM/VFS workload cannot be 
handled any more intelligently - but that wasnt your argument! The 
swap-prefetch patch certainly tried to do things more intelligently and 
the test-case (measurement app) Con provided showed visible improvements 
in swap-in latency. (and a good number of people posted those results)

	Ingo

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Rene Herman]

On 07/25/2007 10:28 AM, Ingo Molnar wrote:

>> Regardless, I'll stand by "[by disabling updatedb] the problem will 
>> for a large part be solved" as I expect approximately 94.372 percent 
>> of Linux desktop users couldn't care less about locate.
> 
> i think that approach is illogical: because Linux mis-handled a mixed 
> workload the answer is to ... remove a portion of that workload?

No. It got snipped but I introduced the comment by saying it was a "that's 
not the point" kind of thing. Sometimes things that aren't the point are 
still true though and in the case of Linux desktop users complaining about 
updatedb runs, a comment that says that for many an obvious solution would 
be to stop running the damned thing is not in any sense illogical.

Also note I'm not against swap prefetch or anything. I don't use it and do 
not believe I have a pressing need for it, but do suspect it has potential 
to make quite a bit of difference on some things -- if only to drastically 
reduce seeks if it means it's swapping in larger chunks than a randomly 
faulting program would.

Rene.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Jos Poortvliet]

[-- Attachment #1: Type: text/plain, Size: 3271 bytes --]

On 7/25/07, Rene Herman <rene.herman@gmail.com> wrote:
>
> On 07/25/2007 10:28 AM, Ingo Molnar wrote:
>
> >> Regardless, I'll stand by "[by disabling updatedb] the problem will
> >> for a large part be solved" as I expect approximately 94.372 percent
> >> of Linux desktop users couldn't care less about locate.
> >
> > i think that approach is illogical: because Linux mis-handled a mixed
> > workload the answer is to ... remove a portion of that workload?
>
> No. It got snipped but I introduced the comment by saying it was a "that's
> not the point" kind of thing. Sometimes things that aren't the point are
> still true though and in the case of Linux desktop users complaining about
> updatedb runs, a comment that says that for many an obvious solution would
> be to stop running the damned thing is not in any sense illogical.
>
> Also note I'm not against swap prefetch or anything. I don't use it and do
> not believe I have a pressing need for it, but do suspect it has potential
> to make quite a bit of difference on some things -- if only to drastically
> reduce seeks if it means it's swapping in larger chunks than a randomly
> faulting program would.


I wonder what your hardware is. Con talked about the diff in hardware
between most endusers and the kernel developers. Yes, swap prefetch doesn't
help if you have 3 GB ram, but it DOES do a lot on a 256 mb laptop... After
using OO.o, the system continues to be slow for a long time. With swap
prefetch, it's back up speed much faster. Con has showed a benchmark for
this with speedups of 10 times and more, users mentioned they liked it. Nick
has been talking about 'fixing the updatedb thing' for years now, no patch
yet. Besides, he won't fix OO.o nor all other userspace stuff - so actually,
he does NOT even promise an alternative. Not that I think fixing updatedb
would be cool, btw - it sure would, but it's no reason not to include swap
prefetch - it's mostly unrelated.

I think everyone with >1 gb ram should stop saying 'I don't need it' because
that's obvious for that hardware. Just like ppl having a dual- or quadcore
shouldn't even talk about scheduler interactivity stuff...

Desktop users want it, tests show it works, there is no alternative and the
maybe-promised-one won't even fix all cornercases. It's small, mostly
selfcontained. There is a maintainer. It's been stable for a long time. It's
been in MM for a long time.

Yet it doesn't make it. Andrew says 'some ppl have objections' (he means
Nick) and he doesn't see an advantage in it (at least 4 gig ram, right,
Andrew?).

Do I miss things?

Apparently, it didn't get in yet - and I find it hard to believe Andrew
holds swapprefetch for reasons like the above. So it must be something else.


Nick is saying tests have already proven swap prefetch to be helpfull,
that's not the problem. He calls the requirements to get in 'fuzzy'. OK.
Beer is fuzzy, do we need to offer beer to someone? If Andrew promises to
come to FOSDEM again next year, I'll offer him a beer, if that helps...
Anything else? A nice massage?


Rene.
> _______________________________________________
> http://ck.kolivas.org/faqs/replying-to-mailing-list.txt
> ck mailing list - mailto: ck@vds.kolivas.org
> http://vds.kolivas.org/mailman/listinfo/ck
>

[-- Attachment #2: Type: text/html, Size: 4124 bytes --]

Re: [ck] Re: -mm merge plans for 2.6.23

[Nick Piggin]

Jos Poortvliet wrote:

> Nick
> has been talking about 'fixing the updatedb thing' for years now, no patch
> yet.

Wrong Nick, I think.

First I heard about the updatedb problem was a few months ago with people
saying updatedb was causing their system to swap (that is, swap prefetching
helped after updatedb). I haven't been able to even try to fix it because I
can't reproduce it (I'm sitting on a machine with 256MB RAM), and nobody
has wanted to help me.


> Besides, he won't fix OO.o nor all other userspace stuff - so 
> actually,
> he does NOT even promise an alternative. Not that I think fixing updatedb
> would be cool, btw - it sure would, but it's no reason not to include swap
> prefetch - it's mostly unrelated.
> 
> I think everyone with >1 gb ram should stop saying 'I don't need it' 
> because
> that's obvious for that hardware. Just like ppl having a dual- or quadcore
> shouldn't even talk about scheduler interactivity stuff...

Actually there are people with >1GB of ram who are saying it helps. Why do
you want to shut people out of the discussion?


> Desktop users want it, tests show it works, there is no alternative and the
> maybe-promised-one won't even fix all cornercases. It's small, mostly
> selfcontained. There is a maintainer. It's been stable for a long time. 
> It's
> been in MM for a long time.
> 
> Yet it doesn't make it. Andrew says 'some ppl have objections' (he means
> Nick) and he doesn't see an advantage in it (at least 4 gig ram, right,
> Andrew?).
> 
> Do I miss things?

You could try constructively contributing?


> Apparently, it didn't get in yet - and I find it hard to believe Andrew
> holds swapprefetch for reasons like the above. So it must be something 
> else.
> 
> 
> Nick is saying tests have already proven swap prefetch to be helpfull,
> that's not the problem. He calls the requirements to get in 'fuzzy'. OK.

The test I have seen is the one that forces a huge amount of memory to
swap out, waits, then touches it. That speeds up, and that's fine. That's
a good sanity test to ensure it is working. Beyond that there are other
considerations to getting something merged.

-- 
SUSE Labs, Novell Inc.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Jos Poortvliet]

[-- Attachment #1: Type: text/plain, Size: 2464 bytes --]

Please ignore my mail... It was uninformed and not constructive. I should've
reread it and thought about it more. Sorry.

On 7/25/07, Nick Piggin <nickpiggin@yahoo.com.au> wrote:
>
> Jos Poortvliet wrote:
>
> > Nick
> > has been talking about 'fixing the updatedb thing' for years now, no
> patch
> > yet.
>
> Wrong Nick, I think.
>
> First I heard about the updatedb problem was a few months ago with people
> saying updatedb was causing their system to swap (that is, swap
> prefetching
> helped after updatedb). I haven't been able to even try to fix it because
> I
> can't reproduce it (I'm sitting on a machine with 256MB RAM), and nobody
> has wanted to help me.
>
>
> > Besides, he won't fix OO.o nor all other userspace stuff - so
> > actually,
> > he does NOT even promise an alternative. Not that I think fixing
> updatedb
> > would be cool, btw - it sure would, but it's no reason not to include
> swap
> > prefetch - it's mostly unrelated.
> >
> > I think everyone with >1 gb ram should stop saying 'I don't need it'
> > because
> > that's obvious for that hardware. Just like ppl having a dual- or
> quadcore
> > shouldn't even talk about scheduler interactivity stuff...
>
> Actually there are people with >1GB of ram who are saying it helps. Why do
> you want to shut people out of the discussion?
>
>
> > Desktop users want it, tests show it works, there is no alternative and
> the
> > maybe-promised-one won't even fix all cornercases. It's small, mostly
> > selfcontained. There is a maintainer. It's been stable for a long time.
> > It's
> > been in MM for a long time.
> >
> > Yet it doesn't make it. Andrew says 'some ppl have objections' (he means
> > Nick) and he doesn't see an advantage in it (at least 4 gig ram, right,
> > Andrew?).
> >
> > Do I miss things?
>
> You could try constructively contributing?
>
>
> > Apparently, it didn't get in yet - and I find it hard to believe Andrew
> > holds swapprefetch for reasons like the above. So it must be something
> > else.
> >
> >
> > Nick is saying tests have already proven swap prefetch to be helpfull,
> > that's not the problem. He calls the requirements to get in 'fuzzy'. OK.
>
> The test I have seen is the one that forces a huge amount of memory to
> swap out, waits, then touches it. That speeds up, and that's fine. That's
> a good sanity test to ensure it is working. Beyond that there are other
> considerations to getting something merged.
>
> --
> SUSE Labs, Novell Inc.
>

[-- Attachment #2: Type: text/html, Size: 3021 bytes --]

Re: [ck] Re: -mm merge plans for 2.6.23

[Rene Herman]

On 07/25/2007 12:53 PM, Jos Poortvliet wrote:

> On 7/25/07, *Rene Herman* <rene.herman@gmail.com 
> <mailto:rene.herman@gmail.com>> wrote:

>> Also note I'm not against swap prefetch or anything. I don't use it and
>> do not believe I have a pressing need for it, but do suspect it has 
>> potential to make quite a bit of difference on some things -- if only
>> to drastically reduce seeks if it means it's swapping in larger chunks
>> than a randomly faulting program would.
> 
> I wonder what your hardware is. Con talked about the diff in hardware 
> between most endusers and the kernel developers. 

I'm afraid you will need to categorize me more as an innocent bystander than 
a kernel developer and, as such, I have an endusery x86 with 768M (but I 
still think of myself as one of the cool kids!)

> Yes, swap prefetch doesn't help if you have 3 GB ram, but it DOES do a
> lot on a 256 mb laptop...

Rather, it does not help if you are not swapping or idling. Probably largely 
due to me being a rather serial person I seem to never even push my git tree 
from cache. Hence my belief that "I don't have a pressing need for it".

Taking a laptop as an example is interesting in itself by the way since a 
spundown disk (most applicable to laptops) is an argument against swap 
prefetch even when idle and when I'm not mistaken the feature actually 
disables itself when the machine's set to laptop mode...

> After using OO.o, the system continues to be slow for a long time. With
> swap prefetch, it's back up speed much faster. Con has showed a benchmark
> for this with speedups of 10 times and more, users mentioned they liked
> it.

After using and quiting OO.o. If you simply don't have any memory free to 
prefetch into swap prefetch won't help any. The fact that it helps the case 
of OO.o having pushed out firefox is fairly obvious.

> Nick has been talking about 'fixing the updatedb thing' for years now, no
> patch yet. Besides, he won't fix OO.o nor all other userspace stuff - so
> actually, he does NOT even promise an alternative. Not that I think
> fixing updatedb would be cool, btw - it sure would, but it's no reason
> not to include swap prefetch - it's mostly unrelated.

Well, the trouble at least to some is that they indeed seem to be rather 
unrelated. Why does the updatedb run even cause swapout? (itself ofcourse a 
pre-condition for swap-prefetch to help).

> I think everyone with >1 gb ram should stop saying 'I don't need it' 
> because that's obvious for that hardware. Just like ppl having a dual- 
> or quadcore shouldn't even talk about scheduler interactivity stuff...

Actually, interactivity is largely about latency and latency is largely or 
partly independent of CPU speed -- if something's keeping the system from 
scheduling for too long it's likely that it's hogging the CPU for a fixed 
number of usecs and those pass in the same amount of time on all CPUs (we 
hope...).

But that's a tangent anyway. I'm just glad that I get to say that I believe 
I don't need it with my 768M!

> Apparently, it didn't get in yet - and I find it hard to believe Andrew 
> holds swapprefetch for reasons like the above. So it must be something 
> else.
> 
> Nick is saying tests have already proven swap prefetch to be helpfull, 
> that's not the problem. He calls the requirements to get in 'fuzzy'. OK.
> Beer is fuzzy, do we need to offer beer to someone? If Andrew promises 
> to come to FOSDEM again next year, I'll offer him a beer, if that 
> helps... Anything else? A nice massage?

Personally I'd go for sexual favours directly (but then again, I always do).

But please also note that I even _literally_ said above that I myself am not 
against swap-prefetch or anything and yet I get what appears to be an least 
somewhat adversary rant directed at me. Which in itself is fine, but not too 
helpful...

Nick Piggin is the person to convince it seems and if I've read things right 
(I only stepped into this thing at the updatedb mention, so maybe I haven't) 
his main question is _why_ the hell it helps updatedb. As long as you don't 
know this, then even a solution that helps could be papering over a problem 
which you'd much rather fix at the root rather than at the symptom.

Rene.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Ingo Molnar]

* Rene Herman <rene.herman@gmail.com> wrote:

> Nick Piggin is the person to convince it seems and if I've read things 
> right (I only stepped into this thing at the updatedb mention, so 
> maybe I haven't) his main question is _why_ the hell it helps 
> updatedb. [...]

btw., i'd like to make this clear: if you want stuff to go upstream, do 
not concentrate on 'convincing the maintainer'.

Instead concentrate on understanding the _problem_, concentrate on 
making sure that both you and the maintainer understands the problem 
correctly, possibly write some testcase that clearly exposes it, and 
help the maintainer debug the problem. _Optionally_, if you find joy in 
it, you are also free to write a proposed solution for that problem and 
submit it to the maintainer.

But a "here is a solution, take it or leave it" approach, before having 
communicated the problem to the maintainer and before having debugged 
the problem is the wrong way around. It might still work out fine if the 
solution is correct (especially if the patch is small and obvious), but 
if there are any non-trivial tradeoffs involved, or if nontrivial amount 
of code is involved, you might see your patch at the end of a really 
long (and constantly growing) waiting list of patches.

	Ingo

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Satyam Sharma]

Hi Ingo,

[ Going off-topic, nothing related to swap/prefetch/etc. Just getting
a hang of how development goes on here ... ]

On 7/25/07, Ingo Molnar <mingo@elte.hu> wrote:
>
> * Rene Herman <rene.herman@gmail.com> wrote:
>
> > Nick Piggin is the person to convince it seems and if I've read things
> > right (I only stepped into this thing at the updatedb mention, so
> > maybe I haven't) his main question is _why_ the hell it helps
> > updatedb. [...]
>
> btw., i'd like to make this clear: if you want stuff to go upstream, do
> not concentrate on 'convincing the maintainer'.

It's not so easy or clear-cut, see below.

> Instead concentrate on understanding the _problem_,

Of course -- that's a given.

> concentrate on
> making sure that both you and the maintainer understands the problem
> correctly,

This itself may require some "convincing" to do. What if the maintainer
just doesn't recognize the problem? Note that the development model
here is more about the "social" thing than purely a "technical" thing.
People do handwave, possibly due to innocent misunderstandings,
possibly without. Often it's just a case of seeing different reasons behind
the "problematic behaviour". Or it could be a case of all of the above.

> possibly write some testcase that clearly exposes it, and

Oh yes -- that'll be helpful, but definitely not necessarily a prerequisite
for all issues, and then you can't even expect everybody to write or
test/benchmark with testcases. (oh, btw, this is assuming you do find
consensus on a testcase)

> help the maintainer debug the problem.

Umm ... well. Should this "dance-with-the-maintainer" and all be really
necessary? What you're saying is easy if a "bug" is simple and objective,
with mathematically few (probably just one) possible correct solutions.
Often (most often, in fact) it's a subjective issue -- could be about APIs,
high level design, tradeoffs, even little implementation nits ... with one
person wanting to do it one way, another thinks there's something hacky
or "band-aidy" about it and a more beautiful/elegant solution exists elsewhere.
I think there's a similar deadlock here (?)

> _Optionally_, if you find joy in
> it, you are also free to write a proposed solution for that problem

Oh yes. But why "optionally"? This is *precisely* what the spirit of
development in such open / distributed projects is ... unless Linux
wants to die the same, slow, ivory-towered, miserable death that
*BSD have.

> and
> submit it to the maintainer.

Umm, ok ... pretty unlikely Linus or Andrew would take patches for any
kernel subsystem (that isn't obvious/trivial) from anybody just like that,
so you do need to Cc: the ones they trust (maintainer) to ensure they
review/ack your work and pick it up.

> But a "here is a solution, take it or leave it" approach,

Agreed. That's definitely not the way to go.

> before having
> communicated the problem to the maintainer

Umm, well this could depend from problem-to-problem.

> and before having debugged
> the problem

Again, agreed -- but people can plausibly see different root causes for
the same symptoms -- and different solutions.

> is the wrong way around. It might still work out fine if the
> solution is correct (especially if the patch is small and obvious), but
> if there are any non-trivial tradeoffs involved, or if nontrivial amount
> of code is involved, you might see your patch at the end of a really
> long (and constantly growing) waiting list of patches.

That's the whole point. For non-trivial / non-obvious / subjective issues,
the "process" you laid out above could itself become a problem ...

Satyam

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Ingo Molnar]

* Satyam Sharma <satyam.sharma@gmail.com> wrote:

> > concentrate on making sure that both you and the maintainer 
> > understands the problem correctly,
> 
> This itself may require some "convincing" to do. What if the 
> maintainer just doesn't recognize the problem? Note that the 
> development model here is more about the "social" thing than purely a 
> "technical" thing. People do handwave, possibly due to innocent 
> misunderstandings, possibly without. Often it's just a case of seeing 
> different reasons behind the "problematic behaviour". Or it could be a 
> case of all of the above.

sure - but i was really not talking about from the user's perspective, 
but from the enterprising kernel developer's perspective who'd like to 
solve a particular problem. And the nice thing about concentrating on 
the problem: if you do that well, it does not really matter what the 
maintainer thinks!

( Talking to the maintainer can of course be of enormous help in the 
  quest for understanding the problem and figuring out the best fix - 
  the maintainer will most likely know more about the subject than 
  yourself. More communication never hurts. It's an additional bonus if 
  you manage to convince the maintainer to take up the matter for 
  himself. It's not a given right though - a maintainer's main task is 
  to judge code that is being submitted, to keep a subsystem running
  smoothly and to not let it regress - but otherwise there can easily be
  different priorities of what tasks to tackle first, and in that sense 
  the maintainer is just one of the many overworked kernel developers 
  who has no real obligation what to tackle first. )

If the maintainer rejects something despite it being well-reasoned, 
well-researched and robustly implemented with no tradeoffs and 
maintainance problems at all then it's a bad maintainer. (but from all 
i've seen in the past few years the VM maintainers do their job pretty 
damn fine.) And note that i _do_ disagree with them in this particular 
swap-prefetch case, but still, the non-merging of swap-prefetch was not 
a final decision at all. It was more of a "hm, dunno, i still dont 
really like it - shouldnt this be done differently? Could we debug this 
a bit better?" reaction. Yes, it can be frustrating after more than one 
year.

> > possibly write some testcase that clearly exposes it, and
> 
> Oh yes -- that'll be helpful, but definitely not necessarily a 
> prerequisite for all issues, and then you can't even expect everybody 
> to write or test/benchmark with testcases. (oh, btw, this is assuming 
> you do find consensus on a testcase)

no, but Con is/was certainly more than capable to write testcases and to 
debug various scenarios. That's the way how new maintainers are found 
within Linux: people take matters in their own hands and improve a 
subsystem so that they'll either peacefully co-work with the other 
maintainers or they replace them (sometimes not so peacefully - like in 
the IDE/SATA/PATA saga).

> > help the maintainer debug the problem.
> 
> Umm ... well. Should this "dance-with-the-maintainer" and all be 
> really necessary? What you're saying is easy if a "bug" is simple and 
> objective, with mathematically few (probably just one) possible 
> correct solutions. Often (most often, in fact) it's a subjective issue 
> -- could be about APIs, high level design, tradeoffs, even little 
> implementation nits ... with one person wanting to do it one way, 
> another thinks there's something hacky or "band-aidy" about it and a 
> more beautiful/elegant solution exists elsewhere. I think there's a 
> similar deadlock here (?)

you dont _have to_ cooperative with the maintainer, but it's certainly 
useful to work with good maintainers, if your goal is to improve Linux. 
Or if for some reason communication is not working out fine then grow 
into the job and replace the maintainer by doing a better job.

> > _Optionally_, if you find joy in it, you are also free to write a 
> > proposed solution for that problem
> 
> Oh yes. But why "optionally"? This is *precisely* what the spirit of 
> development in such open / distributed projects is ... unless Linux 
> wants to die the same, slow, ivory-towered, miserable death that *BSD 
> have.

perhaps you misunderstood how i meant the 'optional': it is certainly 
not required to write a solution for every problem you are reporting. 
Best-case the maintainer picks the issue up and solves it. Worst-case 
you get ignored. But you always have the option to take matters into 
your own hands and solve the problem.

> >and submit it to the maintainer.
> 
> Umm, ok ... pretty unlikely Linus or Andrew would take patches for any 
> kernel subsystem (that isn't obvious/trivial) from anybody just like 
> that, so you do need to Cc: the ones they trust (maintainer) to ensure 
> they review/ack your work and pick it up.

actually, it happens pretty frequently, and NACK-ing perfectly 
reasonable patches is a sure way towards getting replaced as a 
maintainer.

> > is the wrong way around. It might still work out fine if the 
> > solution is correct (especially if the patch is small and obvious), 
> > but if there are any non-trivial tradeoffs involved, or if 
> > nontrivial amount of code is involved, you might see your patch at 
> > the end of a really long (and constantly growing) waiting list of 
> > patches.
> 
> That's the whole point. For non-trivial / non-obvious / subjective 
> issues, the "process" you laid out above could itself become a problem 
> ...

firstly, there's rarely any 'subjective' issue in maintainance 
decisions, even when it comes to complex patches. The 'subjective' issue 
becomes a factor mostly when a problem has not been researched well 
enough, when it becomes more of a faith thing ('i believe it helps me') 
than a fully fact-backed argument. Maintainers tend to dodge such issues 
until they become more clearly fact-backed.

providing more and more facts gradually reduces the 'judgement/taste' 
leeway of maintainers, down to an almost algorithmic level.

but in any case there's always the ultimate way out: prove that you can 
do a better job yourself and replace the maintainer. But providing an 
overwhelming, irresistable body of facts in favor of a patch does the 
trick too in 99.9% of the cases.

	Ingo

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Bartlomiej Zolnierkiewicz]

Hi,

Some general thoughts about submitter/maintainer responsibilities,
not necessarily connected with the recents events (I hasn't been
following them closely - some people don't have that much free time
to burn at their hands ;)...

On Wednesday 25 July 2007, Ingo Molnar wrote:
> 
> * Satyam Sharma <satyam.sharma@gmail.com> wrote:
> 
> > > concentrate on making sure that both you and the maintainer 
> > > understands the problem correctly,
> > 
> > This itself may require some "convincing" to do. What if the 
> > maintainer just doesn't recognize the problem? Note that the 
> > development model here is more about the "social" thing than purely a 
> > "technical" thing. People do handwave, possibly due to innocent 
> > misunderstandings, possibly without. Often it's just a case of seeing 
> > different reasons behind the "problematic behaviour". Or it could be a 
> > case of all of the above.
> 
> sure - but i was really not talking about from the user's perspective, 
> but from the enterprising kernel developer's perspective who'd like to 
> solve a particular problem. And the nice thing about concentrating on 
> the problem: if you do that well, it does not really matter what the 
> maintainer thinks!

Yes, this is a really good strategy to get you changes upstream (and it
works) - just make changes so perfect that nobody can really complain. :)

The only problem is that the bigger the change becomes the less likely it
is to get it perfect so for really big changes it is also useful to show
maintainer that you take responsibility of your changes (by taking bugreports
and potential review issues very seriously instead of ignoring them, past
history of your merged changes has also a big influence here) so he will
know that you won't leave him in the cold with your code when bugreports
happen and be _sure_ that they will happen with bigger changes.

> ( Talking to the maintainer can of course be of enormous help in the 
>   quest for understanding the problem and figuring out the best fix - 
>   the maintainer will most likely know more about the subject than 
>   yourself. More communication never hurts. It's an additional bonus if 
>   you manage to convince the maintainer to take up the matter for 
>   himself. It's not a given right though - a maintainer's main task is 
>   to judge code that is being submitted, to keep a subsystem running
>   smoothly and to not let it regress - but otherwise there can easily be
>   different priorities of what tasks to tackle first, and in that sense 
>   the maintainer is just one of the many overworked kernel developers 
>   who has no real obligation what to tackle first. )

Yep, and patch author should try to help maintainer understand both the
problem he is trying to fix and the solution, i.e. throwing some undocumented
patches and screaming at maintainer to merge them is not a way to go.

> If the maintainer rejects something despite it being well-reasoned, 
> well-researched and robustly implemented with no tradeoffs and 
> maintainance problems at all then it's a bad maintainer. (but from all 
> i've seen in the past few years the VM maintainers do their job pretty 
> damn fine.) And note that i _do_ disagree with them in this particular 
> swap-prefetch case, but still, the non-merging of swap-prefetch was not 
> a final decision at all. It was more of a "hm, dunno, i still dont 
> really like it - shouldnt this be done differently? Could we debug this 
> a bit better?" reaction. Yes, it can be frustrating after more than one 
> year.
> 
> > > possibly write some testcase that clearly exposes it, and
> > 
> > Oh yes -- that'll be helpful, but definitely not necessarily a 
> > prerequisite for all issues, and then you can't even expect everybody 
> > to write or test/benchmark with testcases. (oh, btw, this is assuming 
> > you do find consensus on a testcase)
> 
> no, but Con is/was certainly more than capable to write testcases and to 
> debug various scenarios. That's the way how new maintainers are found 
> within Linux: people take matters in their own hands and improve a 
> subsystem so that they'll either peacefully co-work with the other 
> maintainers or they replace them (sometimes not so peacefully - like in 
> the IDE/SATA/PATA saga).

Heh, now that you've raised IDE saga I feel obligated to stand up
and say a few words...

The latest opening of IDE saga was quite interesting in the current context
because we had exactly the reversed situation there - "independent" maintainer
and "enterprise" developer (imagine the amount of frustration on both sides)
but the root source was quite similar (inability to get changes merged).

IMO the source root of the conflict lied in coming from different perspectives
and having a bit different priorities (stabilising/cleaning current code vs
adding new features on top of pile of crap).  In such situations it is very
important to be able to stop for a moment and look at the situation from
the other person's perspective.

In summary:

The IDE-wars are the thing of the past and lets learn from IDE-world
mistakes instead of repeating them in other subsystems, OK? :)

> > > help the maintainer debug the problem.
> > 
> > Umm ... well. Should this "dance-with-the-maintainer" and all be 
> > really necessary? What you're saying is easy if a "bug" is simple and 
> > objective, with mathematically few (probably just one) possible 
> > correct solutions. Often (most often, in fact) it's a subjective issue 
> > -- could be about APIs, high level design, tradeoffs, even little 
> > implementation nits ... with one person wanting to do it one way, 
> > another thinks there's something hacky or "band-aidy" about it and a 
> > more beautiful/elegant solution exists elsewhere. I think there's a 
> > similar deadlock here (?)
> 
> you dont _have to_ cooperative with the maintainer, but it's certainly 
> useful to work with good maintainers, if your goal is to improve Linux. 
> Or if for some reason communication is not working out fine then grow 
> into the job and replace the maintainer by doing a better job.

The idea of growing into the job and replacing the maintainer by proving
the you are doing better job was viable few years ago but may not be
feasible today.

If maintainer is "enterprise" developer and maintaining is part of his
job replacing him may be not possible et all because you simply lack
the time to do the job.  You may be actually better but you can't afford
to show it and without showing it you won't replace him (catch 22).

Oh, and it could happen that if maintainer works for a distro he sticks
his competing solution to the problem to the distro kernel and suddenly
gets order of magnitude more testers and sometimes even contributors.

How are you supposed to win such competition?  [ A: You can't. ]

I'm not even mentioning the situation when the maintainer is just a genius
and one of the best kernel hackers ever (I'm talking about you actually :)
so your chances are pretty slim from the start...

> > > _Optionally_, if you find joy in it, you are also free to write a 
> > > proposed solution for that problem
> > 
> > Oh yes. But why "optionally"? This is *precisely* what the spirit of 
> > development in such open / distributed projects is ... unless Linux 
> > wants to die the same, slow, ivory-towered, miserable death that *BSD 
> > have.
> 
> perhaps you misunderstood how i meant the 'optional': it is certainly 
> not required to write a solution for every problem you are reporting. 
> Best-case the maintainer picks the issue up and solves it. Worst-case 
> you get ignored. But you always have the option to take matters into 
> your own hands and solve the problem.
> 
> > >and submit it to the maintainer.
> > 
> > Umm, ok ... pretty unlikely Linus or Andrew would take patches for any 
> > kernel subsystem (that isn't obvious/trivial) from anybody just like 
> > that, so you do need to Cc: the ones they trust (maintainer) to ensure 
> > they review/ack your work and pick it up.
> 
> actually, it happens pretty frequently, and NACK-ing perfectly 

It actually happens really rarely (there are pretty good reasons for that).

> reasonable patches is a sure way towards getting replaced as a 
> maintainer.

"reasonable" is highly subjective

> > > is the wrong way around. It might still work out fine if the 
> > > solution is correct (especially if the patch is small and obvious), 
> > > but if there are any non-trivial tradeoffs involved, or if 
> > > nontrivial amount of code is involved, you might see your patch at 
> > > the end of a really long (and constantly growing) waiting list of 
> > > patches.
> > 
> > That's the whole point. For non-trivial / non-obvious / subjective 
> > issues, the "process" you laid out above could itself become a problem 
> > ...
> 
> firstly, there's rarely any 'subjective' issue in maintainance 
> decisions, even when it comes to complex patches. The 'subjective' issue 
> becomes a factor mostly when a problem has not been researched well 
> enough, when it becomes more of a faith thing ('i believe it helps me') 
> than a fully fact-backed argument. Maintainers tend to dodge such issues 
> until they become more clearly fact-backed.

Yep.

However there is a some reasonable time limit for this dodging, two years
isn't reasonable.  By being a maintainer you frequently have to sacrifice
your own goals and instead work on other people changes first (sometimes
even on changes that you don't find particulary interesting or important).
Sure it doesn't give you the same credit you'll get for your own changes
but you're investing in people who will help you in a long-term.

Could you allow the luxury of losing these people?

The another problem is that sometimes it seems that independent developers
has to go through more hops than entreprise ones and it is really frustrating
experience for them.  There is no conspiracy here - it is only the natural
mechanism of trusting more in the code of people who you are working with more.

> providing more and more facts gradually reduces the 'judgement/taste' 
> leeway of maintainers, down to an almost algorithmic level.
> but in any case there's always the ultimate way out: prove that you can 
> do a better job yourself and replace the maintainer. But providing an 

As stated before - this is nearly impossible in some cases.

I'm not proposing any kind of justice or fair chances here I'm just saying
that in the long-term it is gonna hurt said maintainer because he will lose
talented people willing to work on the code that he maintains.

> overwhelming, irresistable body of facts in favor of a patch does the 
> trick too in 99.9% of the cases.

Now could I ask people to stop all this -ck threads and give the developers
involved in the recent events some time to calmly rethink the whole case.

Please?

Thanks,
Bart

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Jeff Garzik]

Bartlomiej Zolnierkiewicz wrote:
> On Wednesday 25 July 2007, Ingo Molnar wrote:
>> you dont _have to_ cooperative with the maintainer, but it's certainly 
>> useful to work with good maintainers, if your goal is to improve Linux. 
>> Or if for some reason communication is not working out fine then grow 
>> into the job and replace the maintainer by doing a better job.
> 
> The idea of growing into the job and replacing the maintainer by proving
> the you are doing better job was viable few years ago but may not be
> feasible today.

IMO...  Tejun is an excellent counter-example.  He showed up as an 
independent developer, put a bunch of his own spare time and energy into 
the codebase, and is probably libata's main engineer (in terms of code 
output) today.  If I get hit by a bus tomorrow, I think the Linux 
community would be quite happy with him as the libata maintainer.


> The another problem is that sometimes it seems that independent developers
> has to go through more hops than entreprise ones and it is really frustrating
> experience for them.  There is no conspiracy here - it is only the natural
> mechanism of trusting more in the code of people who you are working with more.

I think Tejun is a counter-example here too :)  Everyone's experience is 
different, but from my perspective, Tejun "appeared out of nowhere" 
producing good code, and so, it got merged rapidly.

Personally, for merging code, I tend to trust people who are most in 
tune with "the Linux Way(tm)."  It is hard to quantify, but quite often, 
independent developers "get it" when enterprise developers do not.


> Now could I ask people to stop all this -ck threads and give the developers
> involved in the recent events some time to calmly rethink the whole case.

Indeed...

	Jeff


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Bartlomiej Zolnierkiewicz]

On Thursday 26 July 2007, Jeff Garzik wrote:
> Bartlomiej Zolnierkiewicz wrote:
> > On Wednesday 25 July 2007, Ingo Molnar wrote:
> >> you dont _have to_ cooperative with the maintainer, but it's certainly 
> >> useful to work with good maintainers, if your goal is to improve Linux. 
> >> Or if for some reason communication is not working out fine then grow 
> >> into the job and replace the maintainer by doing a better job.
> > 
> > The idea of growing into the job and replacing the maintainer by proving
> > the you are doing better job was viable few years ago but may not be
> > feasible today.
> 
> IMO...  Tejun is an excellent counter-example.  He showed up as an 

IMO this doesn't qualify as a counter-example here et all unless
you are trying to say that Tejun does your job much better and that
we should just replace you. ;)

> independent developer, put a bunch of his own spare time and energy into 
> the codebase, and is probably libata's main engineer (in terms of code 
> output) today.  If I get hit by a bus tomorrow, I think the Linux 
> community would be quite happy with him as the libata maintainer.

Fully agreed on this part.

> > The another problem is that sometimes it seems that independent developers
> > has to go through more hops than entreprise ones and it is really frustrating
> > experience for them.  There is no conspiracy here - it is only the natural
> > mechanism of trusting more in the code of people who you are working with more.
> 
> I think Tejun is a counter-example here too :)  Everyone's experience is 
> different, but from my perspective, Tejun "appeared out of nowhere" 
> producing good code, and so, it got merged rapidly.

Tejun (like any of other developers) spent some time in-the-making
and this time was in large part spent in the IDE-land, and yes I'm also
very glad of the effects. :)

Thanks,
Bart

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Ingo Molnar]

* Rene Herman <rene.herman@gmail.com> wrote:

> On 07/25/2007 10:28 AM, Ingo Molnar wrote:
> 
> >>Regardless, I'll stand by "[by disabling updatedb] the problem will 
> >>for a large part be solved" as I expect approximately 94.372 percent 
> >>of Linux desktop users couldn't care less about locate.
> >
> > i think that approach is illogical: because Linux mis-handled a 
> > mixed workload the answer is to ... remove a portion of that 
> > workload?
> 
> No. It got snipped but I introduced the comment by saying it was a 
> "that's not the point" kind of thing. [...]

ok - with that qualification i understand.

still, especially for someone like me who frequently deals with source 
code, 'locate' is indispensible.

and the fact is: updatedb discards a considerable portion of the cache 
completely unnecessarily: on a reasonably complex box no way do all the 
inodes and dentries fit into all of RAM, so we just trash everything. 
Maybe the kernel could be extended with a method of opening files in a 
'drop from the dcache after use' way. (beagled and backup tools could 
make use of that facility too.) (Or some other sort of 
file-cache-invalidation syscall that already exist, which would _also_ 
result in the immediate zapping of the dentry+inode from the dcache.)

	Ingo

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Rene Herman]

On 07/25/2007 01:34 PM, Ingo Molnar wrote:

> and the fact is: updatedb discards a considerable portion of the cache 
> completely unnecessarily: on a reasonably complex box no way do all the 
> inodes and dentries fit into all of RAM, so we just trash everything.

Okay, but unless I've now managed to really quite horribly confuse myself, 
that wouldn't have anything to do with _swap_ prefetch would it?

Rene.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Ingo Molnar]

* Rene Herman <rene.herman@gmail.com> wrote:

> > and the fact is: updatedb discards a considerable portion of the 
> > cache completely unnecessarily: on a reasonably complex box no way 
> > do all the inodes and dentries fit into all of RAM, so we just trash 
> > everything.
> 
> Okay, but unless I've now managed to really quite horribly confuse 
> myself, that wouldn't have anything to do with _swap_ prefetch would 
> it?

it's connected: it would remove updatedb from the VM picture altogether. 
(updatedb would just cycle through the files with leaving minimal cache 
disturbance.)

hence swap-prefetch could concentrate on the cases where it makes sense 
to start swap prefetching _without_ destroying other, already cached 
content: such as when a large app exits and frees gobs of memory back 
into the buddy allocator. _That_ would be a definitive "no costs and 
side-effects" point for swap-prefetch to kick in, and it would eliminate 
this pretty artificial (and unnecessary) 'desktop versus server' 
controversy and would turn it into a 'helps everyone' feature.

	Ingo

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 860 bytes --]

On Wed, 25 Jul 2007 13:34:01 +0200, Ingo Molnar said:

> Maybe the kernel could be extended with a method of opening files in a 
> 'drop from the dcache after use' way. (beagled and backup tools could 
> make use of that facility too.) (Or some other sort of 
> file-cache-invalidation syscall that already exist, which would _also_ 
> result in the immediate zapping of the dentry+inode from the dcache.)

The semantic that would benefit my work patterns the most would not be
"immediate zapping" - I have 2G of RAM, so often there's no memory pressure,
and often a 'find' will be followed by another similar 'find' that will hit a
lot of the same dentries and inodes, so may as well save them if we can.
Flagging it as "the first to be heaved over the side the instant there *is*
pressure" would suit just fine.

Or is that the semantic you actually meant?


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: -mm merge plans for 2.6.23

[Paul Jackson]

> and the fact is: updatedb discards a considerable portion of the cache 
> completely unnecessarily: on a reasonably complex box no way do all the 

I'm wondering how much of this updatedb problem is due to poor layout
of swap and other file systems across disk spindles.

I'll wager that those most impacted by updatedb have just one disk.

I have the following three boxes - three different setups, each with
different updatedb behaviour:

    The first box, with 1 GB ram, becomes dog slow as soon as it
    breaths on the swap device.  Updatedb and backups are painful
    intrusions on any interactive work on that system.  I sometimes
    wait a half minute for a response from an interactive application
    anytime it has to go to disk.  This box has a single disk spindle,
    on an old cheap slow disk, with swap on the opposite end of the
    disk from root and the main usr partition.  It's a worst case
    disk seek test device.

    The second box, also with 1 GB ram, has multiple disk spindles,
    and swap on its own spindle.  I can still notice updatedb and
    backup, but it's far far less painful.

    The third box has dual CPU cores and 4 GB ram.  Updatedb runs
    over the entire system in perhaps 30 seconds with no perceptible
    impact at all on interactive uses.  Everything is still in memory
    from the previous updatedb run; the disk is just used to write
    out new stuff.  Swap is never used on this (sweet) rig.

I'd think that prefetch would help in the single disk spindle
configuration, because it does the swap accesses separately, instead
of intermingling them with root or usr partition accesses, which
would require alot of disk head seeking.

Pretty much anytime that ordinary desktop users complain about
performance as much as they have about this one, it's either disk
head seeks or network delays.  Nothing else is -that- slow, to be so
noticeable to so many users just doing ordinary work.

Question:
  Could those who have found this prefetch helps them alot say how
  many disks they have?  In particular, is their swap on the same
  disk spindle as their root and user files?

Answer - for me:
  On my system where updatedb is a big problem, I have one, slow, disk.
  On my system where updatedb is a small problem, swap is on a separate
    spindle.
  On my system where updatedb is -no- problem, I have so much memory
    I never use swap.

I'd expect the laptop crowd to mostly have a single, slow, disk, and
hence to find updatedb more painful.

-- 
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <pj@sgi.com> 1.925.600.0401

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Zan Lynx]

[-- Attachment #1: Type: text/plain, Size: 953 bytes --]

On Wed, 2007-07-25 at 15:05 -0700, Paul Jackson wrote:
[snip]
> Question:
>   Could those who have found this prefetch helps them alot say how
>   many disks they have?  In particular, is their swap on the same
>   disk spindle as their root and user files?
> 
> Answer - for me:
>   On my system where updatedb is a big problem, I have one, slow, disk.
>   On my system where updatedb is a small problem, swap is on a separate
>     spindle.
>   On my system where updatedb is -no- problem, I have so much memory
>     I never use swap.
> 
> I'd expect the laptop crowd to mostly have a single, slow, disk, and
> hence to find updatedb more painful.

A well done swap-to-flash would help here.  I sometimes do it anyway to
a 4GB CF card but I can tell it's hitting the read/update/write cycles
on the flash blocks.  The sad thing is that it is still a speed
improvement over swapping to laptop disk.
-- 
Zan Lynx <zlynx@acm.org>

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: -mm merge plans for 2.6.23

[Jesper Juhl]

On 26/07/07, Paul Jackson <pj@sgi.com> wrote:
> > and the fact is: updatedb discards a considerable portion of the cache
> > completely unnecessarily: on a reasonably complex box no way do all the
>
> I'm wondering how much of this updatedb problem is due to poor layout
> of swap and other file systems across disk spindles.
>
> I'll wager that those most impacted by updatedb have just one disk.
>
[snip]
>
> Question:
>   Could those who have found this prefetch helps them alot say how
>   many disks they have?  In particular, is their swap on the same
>   disk spindle as their root and user files?
>

Swap prefetch helps me.

In my case I have a single (10K RPM, Ultra 160 SCSI) disk.

# fdisk -l /dev/sda

Disk /dev/sda: 36.7 GB, 36703918080 bytes
255 heads, 63 sectors/track, 4462 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1         974     7823623+  83  Linux
/dev/sda2             975        1218     1959930   83  Linux
/dev/sda3            1219        1341      987997+  82  Linux swap
/dev/sda4            1342        4462    25069432+  83  Linux

sda1 is "/", sda2 is "/usr/local/" and sda4 is "/home/"


But, I don't think updatedb is the problem, at least not just updatedb
on its own.
My machine has 2GB of RAM, so a single updatedb on its own will not
cause it to start swapping, but it does eat up a chunk of mem no doubt
about that.
The problem with updatedb is simply that it can be a contributing
factor to stuff being swapped out, but any memory hungry application
can do that - just try building an allyesconfig kernel and see how
much the linker eats towards the end.

What swap prefetch helps is not updatedb specifically, In my
experience it helps any case where you have applications running, then
start some memory hungry job that runs for a limited time, push the
previously started apps out to swap and then dies (like updatedb or a
compile job).

Without swap prefetch those apps that were pushed to swap won't be
brought back in before they are used (at which time the user is going
to have to sit there and wait for them).
With swap prefetch, the apps that got swapped out will slowly make
their way back once the mem hungry app has died and will then be fully
or partly back in memory when the user comes back to them.

That's how swap prefetch helps, it's got nothing to do with updatedb
as such - at least not as I see it.

-- 
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Michael Chang]

On 7/25/07, Paul Jackson <pj@sgi.com> wrote:
> Question:
>   Could those who have found this prefetch helps them alot say how
>   many disks they have?  In particular, is their swap on the same
>   disk spindle as their root and user files?

I have found that swap prefetch helped on all of the four machines
machine I have, although the effect is more noticeable on machines
with slower disks. They all have one hard disk, and root and swap were
always on the same disk. I have no idea how to determine how many disk
spindles they have, but since the drives are mainly low-end consumer
models sold with low-end sub $500 PCs...

-- 
Michael Chang

Please avoid sending me Word or PowerPoint attachments. Send me ODT,
RTF, or HTML instead.
See http://www.gnu.org/philosophy/no-word-attachments.html
Thank you.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[André Goddard Rosa]

> Question:
>   Could those who have found this prefetch helps them alot say how
>   many disks they have?  In particular, is their swap on the same
>   disk spindle as their root and user files?
>
> Answer - for me:
>   On my system where updatedb is a big problem, I have one, slow, disk.

On both desktop and laptop.

Cheers,
-- 
[]s,
Andre Goddard

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Ray Lee]

On 7/24/07, Rene Herman <rene.herman@gmail.com> wrote:
> Yes, but what's locate's usage scenario? I've never, ever wanted to use it.
> When do you know the name of something but not where it's located, other
> than situations which "which" wouldn't cover and after just having
> installed/unpacked something meaning locate doesn't know about it yet either?

I use it to find source files and documents all the time. One of my
work boxes has <runs a locate work | wc -l> ~38500 files and
directories under my source directory. And then there's the "I wrote
that tech doc two years ago, where was that. Hmm, what did I name it?
Bet it had 323 in the name, and doc in the path."

I'd just like updatedb to amortize its work better. If we had some way
to track all filesystem events, updatedb could keep a live and
accurate index on the filesystem. And this isn't just updatedb that
wants that, beagle and tracker et al also want to know filesystem
events so that they can index the documents themselves as well as the
metadata. And if they do it live, that spreads the cost out, including
the VM pressure.

Ray

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Zan Lynx]

[-- Attachment #1: Type: text/plain, Size: 854 bytes --]

On Wed, 2007-07-25 at 09:02 -0700, Ray Lee wrote:

> I'd just like updatedb to amortize its work better. If we had some way
> to track all filesystem events, updatedb could keep a live and
> accurate index on the filesystem. And this isn't just updatedb that
> wants that, beagle and tracker et al also want to know filesystem
> events so that they can index the documents themselves as well as the
> metadata. And if they do it live, that spreads the cost out, including
> the VM pressure.

That would be nice.  It'd be great if there was a per-filesystem inotify
mode.  I can't help but think it'd be more efficient than recursing
every directory and adding a watch.

Or maybe a netlink thing that could buffer events since filesystem mount
until a daemon could get around to starting, so none were lost.
-- 
Zan Lynx <zlynx@acm.org>

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: -mm merge plans for 2.6.23

[Ray Lee]

On 7/25/07, Zan Lynx <zlynx@acm.org> wrote:
> On Wed, 2007-07-25 at 09:02 -0700, Ray Lee wrote:
>
> > I'd just like updatedb to amortize its work better. If we had some way
> > to track all filesystem events, updatedb could keep a live and
> > accurate index on the filesystem. And this isn't just updatedb that
> > wants that, beagle and tracker et al also want to know filesystem
> > events so that they can index the documents themselves as well as the
> > metadata. And if they do it live, that spreads the cost out, including
> > the VM pressure.
>
> That would be nice.  It'd be great if there was a per-filesystem inotify
> mode.  I can't help but think it'd be more efficient than recursing
> every directory and adding a watch.
>
> Or maybe a netlink thing that could buffer events since filesystem mount
> until a daemon could get around to starting, so none were lost.

See "Filesystem Event Reporter" by Yi Yang, that does pretty much
exactly that. http://lkml.org/lkml/2006/9/30/98 . Author had things to
update, never resubmitted it as far as I can tell.

Ray

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Matthew Hawkins]

On 7/26/07, Ray Lee <ray-lk@madrabbit.org> wrote:
> I'd just like updatedb to amortize its work better. If we had some way
> to track all filesystem events, updatedb could keep a live and
> accurate index on the filesystem. And this isn't just updatedb that
> wants that, beagle and tracker et al also want to know filesystem
> events so that they can index the documents themselves as well as the
> metadata. And if they do it live, that spreads the cost out, including
> the VM pressure.

We already have this, its called inotify (and if I'm not mistaken,
beagle already uses it).  Several years ago when it was still a little
flakey patch, I built a custom filesystem indexer into an enterprise
search engine using it (I needed to pull apart Unix mbox files).  The
only trouble of course is the action is triggered immediately, which
may not always be ideal (but that's a userspace problem)

-- 
Matt

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Ray Lee]

On 7/25/07, Matthew Hawkins <darthmdh@gmail.com> wrote:
> On 7/26/07, Ray Lee <ray-lk@madrabbit.org> wrote:
> > I'd just like updatedb to amortize its work better. If we had some way
> > to track all filesystem events, updatedb could keep a live and
> > accurate index on the filesystem. And this isn't just updatedb that
> > wants that, beagle and tracker et al also want to know filesystem
> > events so that they can index the documents themselves as well as the
> > metadata. And if they do it live, that spreads the cost out, including
> > the VM pressure.
>
> We already have this, its called inotify (and if I'm not mistaken,
> beagle already uses it).

Yeah, I know about inotify, but it doesn't scale.

ray@phoenix:~$ find ~ -type d | wc -l
17933
ray@phoenix:~$

That's not fun with inotify, and that's just my home directory. The
vast majority of those are quiet the vast majority of the time, which
is the crux of the problem, and why inotify isn't a great fit for
on-demand virus scanners or indexers.

Ray

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Matthew Hawkins]

On 7/26/07, Ray Lee <ray-lk@madrabbit.org> wrote:
> Yeah, I know about inotify, but it doesn't scale.

Yeah, the nonrecursive behaviour is a bugger.  Also I found it helped
to queue operations in userspace and execute periodically rather than
trying to execute on every single notification.  Worked well for
indexing, for virus scanning though you'd want to do some risk
analysis.

It'd be nice to have a filesystem that handled that sort of thing
internally *cough*winfs*cough*.  That was my hope for reiserfs a very
long time ago with its pluggable fs modules feature.

-- 
Matt

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Michael Chang]

On 7/25/07, Matthew Hawkins <darthmdh@gmail.com> wrote:
> On 7/26/07, Ray Lee <ray-lk@madrabbit.org> wrote:
> > I'd just like updatedb to amortize its work better. If we had some way
> > to track all filesystem events, updatedb could keep a live and
> > accurate index on the filesystem. And this isn't just updatedb that
> > wants that, beagle and tracker et al also want to know filesystem
> > events so that they can index the documents themselves as well as the
> > metadata. And if they do it live, that spreads the cost out, including
> > the VM pressure.
>
> We already have this, its called inotify (and if I'm not mistaken,
> beagle already uses it).  Several years ago when it was still a little
> flakey patch, I built a custom filesystem indexer into an enterprise
> search engine using it (I needed to pull apart Unix mbox files).  The
> only trouble of course is the action is triggered immediately, which
> may not always be ideal (but that's a userspace problem)
>

With all this discussion about updatedb and locate and such, I thought
I'd do a Google search, (considering I've never heard of locate before
but I've seen updatedb here and there in ps lists) and I found this:

http://www.linux.com/articles/114029

That page mentions something called "rlocate", which seems to provide
some sort of almost-real-time mechanism, although the method it does
so bothers me -- it uses a 2.6 kernel module AND a userspace daemon.
And from what I can tell, there's no indication that this almost
"real-time" (--I see mentions of a 2 second lag--) system
replaces/eliminates updatedb in any way, shape, or form.

http://rlocate.sourceforge.net/ - Project "Web Site"
http://sourceforge.net/projects/rlocate/ - Source Forge Project Summary

The last release also appears a bit dated on sourceforge... release
0.4.0 on 2006-01-15.

Just thought I'd mention it.


-- 
Michael Chang

Please avoid sending me Word or PowerPoint attachments. Send me ODT,
RTF, or HTML instead.
See http://www.gnu.org/philosophy/no-word-attachments.html
Thank you.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Eric St-Laurent]

On Wed, 2007-25-07 at 06:55 +0200, Rene Herman wrote:

> It certainly doesn't run for me ever. Always kind of a "that's not the 
> point" comment but I just keep wondering whenever I see anyone complain 
> about updatedb why the _hell_ they are running it in the first place. If 
> anyone who never uses "locate" for anything simply disable updatedb, the 
> problem will for a large part be solved.
> 
> This not just meant as a cheap comment; while I can think of a few similar 
> loads even on the desktop (scanning a browser cache, a media player indexing 
> a large amount of media files, ...) I've never heard of problems _other_ 
> than updatedb. So just junk that crap and be happy.

>From my POV there's two different problems discussed recently:

- updatedb type of workloads that add tons of inodes and dentries in the
slab caches which of course use the pagecache.

- streaming large files (read or copying) that fill the pagecache with
useless used-once data

swap prefetch fix the first case, drop-behind fix the second case.

Both have the same symptoms but the cause is different.

Personally updatedb doesn't really hurt me.  But I don't have that many
files on my desktop.  I've tried the swap prefetch patch in the past and
it was not so noticeable for me. (I don't doubt it's helpful for others)

But every time I read or copy a large file around (usually from a
server) the slowdown is noticeable for some moments.

I just wanted to point this out, if it wasn't clean enough for everyone.
I hope both problems get fixed.


Best regards,

- Eric


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Nick Piggin]

Eric St-Laurent wrote:
> On Wed, 2007-25-07 at 06:55 +0200, Rene Herman wrote:
> 
> 
>>It certainly doesn't run for me ever. Always kind of a "that's not the 
>>point" comment but I just keep wondering whenever I see anyone complain 
>>about updatedb why the _hell_ they are running it in the first place. If 
>>anyone who never uses "locate" for anything simply disable updatedb, the 
>>problem will for a large part be solved.
>>
>>This not just meant as a cheap comment; while I can think of a few similar 
>>loads even on the desktop (scanning a browser cache, a media player indexing 
>>a large amount of media files, ...) I've never heard of problems _other_ 
>>than updatedb. So just junk that crap and be happy.
> 
> 
>>From my POV there's two different problems discussed recently:
> 
> - updatedb type of workloads that add tons of inodes and dentries in the
> slab caches which of course use the pagecache.
> 
> - streaming large files (read or copying) that fill the pagecache with
> useless used-once data
> 
> swap prefetch fix the first case, drop-behind fix the second case.

OK, this is where I start to worry. Swap prefetch AFAIKS doesn't fix
the updatedb problem very well, because if updatedb has caused swapout
then it has filled memory, and swap prefetch doesn't run unless there
is free memory (not to mention that updatedb would have paged out other
files as well).

And drop behind doesn't fix your usual problem where you are downloading
from a server, because that is use-once write(2) data which is the
problem. And this readahead-based drop behind also doesn't help if data
you were reading happened to be a sequence of small files, or otherwise
not in good readahead order.

Not to say that neither fix some problems, but for such conceptually
big changes, it should take a little more effort than a constructed test
case and no consideration of the alternatives to get it merged.

-- 
SUSE Labs, Novell Inc.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[david]

On Wed, 25 Jul 2007, Nick Piggin wrote:

> Eric St-Laurent wrote:
>>  On Wed, 2007-25-07 at 06:55 +0200, Rene Herman wrote:
>>
>> 
>> > It certainly doesn't run for me ever. Always kind of a "that's not the 
>> > point" comment but I just keep wondering whenever I see anyone complain 
>> > about updatedb why the _hell_ they are running it in the first place. If 
>> > anyone who never uses "locate" for anything simply disable updatedb, the 
>> > problem will for a large part be solved.
>> > 
>> > This not just meant as a cheap comment; while I can think of a few 
>> > similar loads even on the desktop (scanning a browser cache, a media 
>> > player indexing a large amount of media files, ...) I've never heard of 
>> > problems _other_ than updatedb. So just junk that crap and be happy.
>>
>> 
>> >From my POV there's two different problems discussed recently:
>>
>>  - updatedb type of workloads that add tons of inodes and dentries in the
>>  slab caches which of course use the pagecache.
>>
>>  - streaming large files (read or copying) that fill the pagecache with
>>  useless used-once data
>>
>>  swap prefetch fix the first case, drop-behind fix the second case.
>
> OK, this is where I start to worry. Swap prefetch AFAIKS doesn't fix
> the updatedb problem very well, because if updatedb has caused swapout
> then it has filled memory, and swap prefetch doesn't run unless there
> is free memory (not to mention that updatedb would have paged out other
> files as well).
>
> And drop behind doesn't fix your usual problem where you are downloading
> from a server, because that is use-once write(2) data which is the
> problem. And this readahead-based drop behind also doesn't help if data
> you were reading happened to be a sequence of small files, or otherwise
> not in good readahead order.
>
> Not to say that neither fix some problems, but for such conceptually
> big changes, it should take a little more effort than a constructed test
> case and no consideration of the alternatives to get it merged.

well, there appears to be a fairly large group of people who have 
subjective opinions that it helps them. but those were dismissed becouse 
they aren't measurements.

so now the measurements of the constructed test case aren't acceptable.

what sort of test case would be acceptable?

David Lang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Nick Piggin]

david@lang.hm wrote:
> On Wed, 25 Jul 2007, Nick Piggin wrote:

>> OK, this is where I start to worry. Swap prefetch AFAIKS doesn't fix
>> the updatedb problem very well, because if updatedb has caused swapout
>> then it has filled memory, and swap prefetch doesn't run unless there
>> is free memory (not to mention that updatedb would have paged out other
>> files as well).
>>
>> And drop behind doesn't fix your usual problem where you are downloading
>> from a server, because that is use-once write(2) data which is the
>> problem. And this readahead-based drop behind also doesn't help if data
>> you were reading happened to be a sequence of small files, or otherwise
>> not in good readahead order.
>>
>> Not to say that neither fix some problems, but for such conceptually
>> big changes, it should take a little more effort than a constructed test
>> case and no consideration of the alternatives to get it merged.
> 
> 
> well, there appears to be a fairly large group of people who have 
> subjective opinions that it helps them. but those were dismissed becouse 
> they aren't measurements.

Not at all. But there is also seems to be some people also experiencing
problems with basic page reclaim on some of the workloads where these
things help. I am not dismissing anybody's claims about anything; I want
to try to solve some of these problems.

Interestingly, some of the people ranting the most about how the VM sucks
are the ones helping least in solving these basic problems.


> so now the measurements of the constructed test case aren't acceptable.
> 
> what sort of test case would be acceptable?

Well I never said real world tests aren't acceptable, they are. There is
a difference between an "it feels better for me", and some actual real
measurement and analysis of said workload.

And constructed test cases of course are useful as well, I didn't say
they weren't. I don't know what you mean by "acceptable", but you should
read my last paragraph again.

-- 
SUSE Labs, Novell Inc.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[david]

On Wed, 25 Jul 2007, Nick Piggin wrote:

> david@lang.hm wrote:
>>  On Wed, 25 Jul 2007, Nick Piggin wrote:
>
>> >  OK, this is where I start to worry. Swap prefetch AFAIKS doesn't fix
>> >  the updatedb problem very well, because if updatedb has caused swapout
>> >  then it has filled memory, and swap prefetch doesn't run unless there
>> >  is free memory (not to mention that updatedb would have paged out other
>> >  files as well).
>> > 
>> >  And drop behind doesn't fix your usual problem where you are downloading
>> >  from a server, because that is use-once write(2) data which is the
>> >  problem. And this readahead-based drop behind also doesn't help if data
>> >  you were reading happened to be a sequence of small files, or otherwise
>> >  not in good readahead order.
>> > 
>> >  Not to say that neither fix some problems, but for such conceptually
>> >  big changes, it should take a little more effort than a constructed test
>> >  case and no consideration of the alternatives to get it merged.
>>
>>
>>  well, there appears to be a fairly large group of people who have
>>  subjective opinions that it helps them. but those were dismissed becouse
>>  they aren't measurements.
>
> Not at all. But there is also seems to be some people also experiencing
> problems with basic page reclaim on some of the workloads where these
> things help. I am not dismissing anybody's claims about anything; I want
> to try to solve some of these problems.
>
> Interestingly, some of the people ranting the most about how the VM sucks
> are the ones helping least in solving these basic problems.
>
>
>>  so now the measurements of the constructed test case aren't acceptable.
>>
>>  what sort of test case would be acceptable?
>
> Well I never said real world tests aren't acceptable, they are. There is
> a difference between an "it feels better for me", and some actual real
> measurement and analysis of said workload.
>
> And constructed test cases of course are useful as well, I didn't say
> they weren't. I don't know what you mean by "acceptable", but you should
> read my last paragraph again.

this problem has been around for many years, with many different people 
working on solutions. it's hardly a case of getting a proposal and trying 
to get it in without anyone looking at other options.

it seems that there are some people (not nessasarily including you) who 
will oppose this feature until a test is created that shows that it's 
better. the question is what sort of test will be accepted as valid? I'm 
not useing this patch, but it sounds as if the people who are useing it 
are interested in doing whatever testing is required, but so far the 
situation seems to be a series of "here's a test", "that test isn't valid, 
try again" loops. which don't seem to be doing anyone any good and are 
frustrating lots of people, so like several people over the last few days 
O'm asking the question, "what sort of test would be acceptable as proof 
that this patch does some good?"

David Lang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Nick Piggin]

david@lang.hm wrote:
> On Wed, 25 Jul 2007, Nick Piggin wrote:

>> And constructed test cases of course are useful as well, I didn't say
>> they weren't. I don't know what you mean by "acceptable", but you should
>> read my last paragraph again.
> 
> 
> this problem has been around for many years, with many different people 
> working on solutions. it's hardly a case of getting a proposal and 
> trying to get it in without anyone looking at other options.

What is "this problem"? People have an updatedb problem that is solved
by swap prefetching which I want to fix in a different way.

There would be a different problem of "run something that uses heaps of
memory and swap everything else out, then quit it, wait for a while, and
swap prefetching helps". OK, definitely swap prefetching would help there.
How much? I don't know. I'd be slightly surprised if it was like an order
of magnitude, because not only swap but everything else has been thrown
out too.


> it seems that there are some people (not nessasarily including you) who 
> will oppose this feature until a test is created that shows that it's 
> better. the question is what sort of test will be accepted as valid? I'm 
> not useing this patch, but it sounds as if the people who are useing it 
> are interested in doing whatever testing is required, but so far the 
> situation seems to be a series of "here's a test", "that test isn't 
> valid, try again" loops. which don't seem to be doing anyone any good 

And yet despite my repeated pleas, none of those people has yet spent a
bit of time with me to help analyse what is happening.


> and are frustrating lots of people, so like several people over the last 
> few days O'm asking the question, "what sort of test would be acceptable 
> as proof that this patch does some good?"

I don't think any further proof is needed that the patch does "some"
good. Rig up a test case and you could see some seconds shaved off it.
Maybe you want to know "how to get this patch merged"? And I don't know
that one. I do know that it is fuzzy, and probably doesn't include
demanding things of Andrew or Linus.

BTW. If you find out the answer to that one, let me know because I have
this lockless pagecache patch that has also been around for years, is
also just a few hundred lines in the VM, and does do some good too. I'm
sure the buffered AIO people and many others would also like to know.

-- 
SUSE Labs, Novell Inc.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Ingo Molnar]

* Nick Piggin <nickpiggin@yahoo.com.au> wrote:

> And yet despite my repeated pleas, none of those people has yet spent 
> a bit of time with me to help analyse what is happening.

btw., it might help to give specific, precise instructions about what 
people should do to help you analyze this problem.

	Ingo

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Nick Piggin]

Ingo Molnar wrote:
> * Nick Piggin <nickpiggin@yahoo.com.au> wrote:
> 
> 
>>And yet despite my repeated pleas, none of those people has yet spent 
>>a bit of time with me to help analyse what is happening.
> 
> 
> btw., it might help to give specific, precise instructions about what 
> people should do to help you analyze this problem.

Ray has been the first one to offer (thank you), and yes I have asked
him for precise details of info to collect to hopefully work out what
is happening with his first problem.

For the general "it feels better for me" it is harder, but not as hard
as CPU scheduler. We can measure various types of IO waits, swap in/out
events, swap prefetch events and successfulness; see what happens to
those as we change swappiness or vfs_cache_pressure etc.

-- 
SUSE Labs, Novell Inc.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Ingo Molnar]

* Nick Piggin <nickpiggin@yahoo.com.au> wrote:

> > > And yet despite my repeated pleas, none of those people has yet 
> > > spent a bit of time with me to help analyse what is happening.
> >
> > btw., it might help to give specific, precise instructions about 
> > what people should do to help you analyze this problem.
> 
> Ray has been the first one to offer (thank you), and yes I have asked 
> him for precise details of info to collect to hopefully work out what 
> is happening with his first problem.

do you mean this paragraph:

| I guess /proc/meminfo, /proc/zoneinfo, /proc/vmstat, /proc/slabinfo 
| before and after the updatedb run with the latest kernel would be a 
| first step. top and vmstat output during the run wouldn't hurt either.

correct? Does "latest kernel" mean v2.6.22.1, or does it have to be 
v2.6.23-rc1? I guess v2.6.22.1 would be fine as this is a VM problem, 
not a scheduling problem.

the following script will gather all the above information for a 10 
seconds interval:

  http://people.redhat.com/mingo/cfs-scheduler/tools/cfs-debug-info.sh

Ray, please run this script before the updatedb run, once during the 
updatedb run and once after the updatedb run, and send Nick the 3 files 
it creates. (feel free to Cc: me too)

	Ingo

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Jesper Juhl]

On 25/07/07, Nick Piggin <nickpiggin@yahoo.com.au> wrote:
[snip]
>
> Well I never said real world tests aren't acceptable, they are. There is
> a difference between an "it feels better for me", and some actual real
> measurement and analysis of said workload.
>

Let me tell you about the use-case where swap prefetch helps me. I
don't have actual numbers currently, only a subjective "it feels
better", but when I get home from work tonight I'll try to collect
some actual numbers for you.

Anyway, here's a description of the scenario (machine is a AMD
Athlon64 X2 4400+, 2GB RAM, 1GB swap, running 32bit kernel &
userspace):

A KDE desktop with the following running is common for me
 - A few (konsole) shells open running vim, pine, less, ssh sessions etc.
 - Eclipse (with CDT) with 20-30 files open in a project.
 - Firefox with 30+ tabs open.
 - LyX with a 200+ page document I'm working on open, is running.
 - Gimp running, usually with at least one or two images open (~1280x1024).
 - Amarok open and playing my playlist (a few days worth of music).
 - At least one Konqueror window in filemanager mode running.
 - More often than not OpenOffice is running with a spreadsheet or
text document open.
 - In the background the machine is running Apache, MySQL, BIND and
NFS services for my local LAN, but they see very little actual use.

Now, a thing I commonly do is fire up a new shell, pull the latest
changes from Linus' git tree and start a script running that builds a
allnoconfig kernel, a allmodconfig kernel, a allyesconfig kernel and
then 30 randconfig kernels.  Obviously that script takes quite a while
to run and loads the box quite a bit, so I usually just leave the box
alone for a few hours until it is done (sometimes I leave it over
night, in which case updatedb also gets added to the mix during the
night). This usually pushes the box to use some amount of swap.

Without swap prefetch; when I start working with one of the apps I had
running before starting the compile job it always feels a little laggy
at first. With swap prefetch app response time is not laggy when I
come back.  The "laggyness" doesn't last too long and is hard to
quantify, but I'll try getting some numbers (if in no other way, then
perhaps by using a stop watch)....

Fact is, this is a scenario that is common to me and one where swap
prefetch definately makes the box feel nicer to work with.


-- 
Jesper Juhl <jesper.juhl@gmail.com>
Don't top-post  http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please      http://www.expita.com/nomime.html

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Matthew Hawkins]

On 7/25/07, Nick Piggin <nickpiggin@yahoo.com.au> wrote:
> Not to say that neither fix some problems, but for such conceptually
> big changes, it should take a little more effort than a constructed test
> case and no consideration of the alternatives to get it merged.

Swap Prefetch has existed since September 5, 2005.  Please Nick,
enlighten us all with your "alternatives" which have been offered (in
practical, not theoretical form) in the past 23 months, along with
their non-constructed benchmarks proving their case and the hordes of
happy users and kernel developers who have tested them out the wazoo
and given their backing.  Or just take a nice steaming jug of STFU.

-- 
Matt

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Nick Piggin]

Matthew Hawkins wrote:
> On 7/25/07, Nick Piggin <nickpiggin@yahoo.com.au> wrote:
> 
>> Not to say that neither fix some problems, but for such conceptually
>> big changes, it should take a little more effort than a constructed test
>> case and no consideration of the alternatives to get it merged.
> 
> 
> Swap Prefetch has existed since September 5, 2005.  Please Nick,
> enlighten us all with your "alternatives" which have been offered (in
> practical, not theoretical form) in the past 23 months, along with
> their non-constructed benchmarks proving their case and the hordes of
> happy users and kernel developers who have tested them out the wazoo
> and given their backing.  Or just take a nice steaming jug of STFU.

The alternatives comment was in relation to the readahead based drop
behind patch,for which an alternative would be improving use-once,
possibly in the way I described.

As for swap prefetch, I don't know, I'm not in charge of it being
merged or not merged. I do know some people have reported that their
updatedb problem gets much better with swap prefetch turned on, and
I am trying to work on that too.

For you? You also have the alternative to help improve things yourself,
and you can modify your own kernel.

-- 
SUSE Labs, Novell Inc.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Mike Galbraith]

On Wed, 2007-07-25 at 16:19 +1000, Matthew Hawkins wrote:
> On 7/25/07, Nick Piggin <nickpiggin@yahoo.com.au> wrote:
> > Not to say that neither fix some problems, but for such conceptually
> > big changes, it should take a little more effort than a constructed test
> > case and no consideration of the alternatives to get it merged.
> 
> Swap Prefetch has existed since September 5, 2005.  Please Nick,
> enlighten us all with your "alternatives" which have been offered (in
> practical, not theoretical form) in the past 23 months, along with
> their non-constructed benchmarks proving their case and the hordes of
> happy users and kernel developers who have tested them out the wazoo
> and given their backing.  Or just take a nice steaming jug of STFU.

Heh.  Here we have a VM developer expressing his interest in the problem
space, and you offer him a steaming jug of STFU because he doesn't say
what you want to hear.  I wonder how many killfiles you just entered.

	-Mike

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Eric St-Laurent]

On Wed, 2007-25-07 at 08:47 +0200, Mike Galbraith wrote:

> Heh.  Here we have a VM developer expressing his interest in the problem
> space, and you offer him a steaming jug of STFU because he doesn't say
> what you want to hear.  I wonder how many killfiles you just entered.
> 

Agreed.

(a bit OT)

People should understand that it's not (I think) about a desktop
workload vs enterprise workloads war.

I see it mostly as a progression versus regressions trade-off. And
adding potentially useless or unmaintained code is a regression from the
maintainers POV.

The best way to justify a patch and have it integrated is to have a
scientific testing method with repeatable numbers.

Con has done so for his patch, his benchmark demonstrated good
improvements.

But I feel some of his supporters have indirectly harmed his cause by
their comments.  Also, the fact that Con recently stopped maintaining
his work out of frustration also don't help having his patch merged. 

Again I'm not personally pushing this patch, I don't need it.

Con has worked for many years on two area that still cause problems for
desktop users: scheduler interactivity and pagecache trashing.  Now that
the scheduler has been fixed, let's have the VM fixed too.

Sorry for the slightly OT post, and please don't start a flame war...


- Eric


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Eric St-Laurent]

On Wed, 2007-25-07 at 15:37 +1000, Nick Piggin wrote:

> OK, this is where I start to worry. Swap prefetch AFAIKS doesn't fix
> the updatedb problem very well, because if updatedb has caused swapout
> then it has filled memory, and swap prefetch doesn't run unless there
> is free memory (not to mention that updatedb would have paged out other
> files as well).
> 
> And drop behind doesn't fix your usual problem where you are downloading
> from a server, because that is use-once write(2) data which is the
> problem. And this readahead-based drop behind also doesn't help if data
> you were reading happened to be a sequence of small files, or otherwise
> not in good readahead order.
> 
> Not to say that neither fix some problems, but for such conceptually
> big changes, it should take a little more effort than a constructed test
> case and no consideration of the alternatives to get it merged.


Sorry for the confusion.

For swap prefetch I should have said "some people claim that it fix
their problem". I didn't want to hurt anybody feelings, some people are
tired to hear others speak hypothetically about this patch, as it
work-for-them (TM).

I don't experience the problem. Can't help.

For drop behind it fix half the problem. The read case is handled
perfectly by Peter's patch. And the copy (read+write) is unchanged. My
test case demonstrate it very easily, just look at the numbers.

So, I agree with you that drop behind doesn't fix the write() case.
Peter has said so himself when I offered to test his patch.

As I do experience this problem, I have written a small test program and
batch file to help push the patch for acceptance.  I'm very willing to
help improve the test cases, test patches and write code, time
permitting.

About this very subject, earlier this year this Andrew suggested me to
came up with a test case to demonstrate my problem, well finally I've
done so.

http://lkml.org/lkml/2007/3/3/164
http://lkml.org/lkml/2007/3/3/166

Lastly, I would go as far to say that the use-once read then copy fix
must also work with copies over NFS. I don't know if NFS change the
workload on the client station versus the local case, and I don't know
if it's still possible to consider data copied this way as use-once.


- Eric


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Ray Lee]

Hey Eric,

On 7/24/07, Nick Piggin <nickpiggin@yahoo.com.au> wrote:
> Eric St-Laurent wrote:
> > On Wed, 2007-25-07 at 06:55 +0200, Rene Herman wrote:
> >
> >
> >>It certainly doesn't run for me ever. Always kind of a "that's not the
> >>point" comment but I just keep wondering whenever I see anyone complain
> >>about updatedb why the _hell_ they are running it in the first place. If
> >>anyone who never uses "locate" for anything simply disable updatedb, the
> >>problem will for a large part be solved.
> >>
> >>This not just meant as a cheap comment; while I can think of a few similar
> >>loads even on the desktop (scanning a browser cache, a media player indexing
> >>a large amount of media files, ...) I've never heard of problems _other_
> >>than updatedb. So just junk that crap and be happy.
> >
> >
> >>From my POV there's two different problems discussed recently:
> >
> > - updatedb type of workloads that add tons of inodes and dentries in the
> > slab caches which of course use the pagecache.
> >
> > - streaming large files (read or copying) that fill the pagecache with
> > useless used-once data

No, there's a third case which I find the most annoying. I have
multiple working sets, the sum of which won't fit into RAM. When I
finish one, the kernel had time to preemptively swap back in the
other, and yet it didn't. So, I sit around, twiddling my thumbs,
waiting for my music player to come back to life, or thunderbird,
or...

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Andrew Morton]

On Wed, 25 Jul 2007 09:09:01 -0700
"Ray Lee" <ray-lk@madrabbit.org> wrote:

> No, there's a third case which I find the most annoying. I have
> multiple working sets, the sum of which won't fit into RAM. When I
> finish one, the kernel had time to preemptively swap back in the
> other, and yet it didn't. So, I sit around, twiddling my thumbs,
> waiting for my music player to come back to life, or thunderbird,
> or...

Yes, I'm thinking that's a good problem statement and it isn't something
which the kernel even vaguely attempts to address, apart from normal
demand paging.

We could perhaps improve things with larger and smarter fault readaround,
perhaps guided by refault-rate measurement.  But that's still demand-paged
rather than being proactive/predictive/whatever.

None of this is swap-specific though: exactly the same problem would need
to be solved for mmapped files and even plain old pagecache.

In fact I'd restate the problem as "system is in steady state A, then there
is a workload shift causing transition to state B, then the system goes
idle.  We now wish to reinstate state A in anticipation of a resumption of
the original workload".

swap-prefetch solves a part of that.

A complete solution for anon and file-backed memory could be implemented
(ta-da) in userspace using the kernel inspection tools in -mm's maps2-*
patches.  We would need to add a means by which userspace can repopulate
swapcache, but that doesn't sound too hard (especially when you haven't
thought about it).

And userspace can right now work out which pages from which files are in
pagecache so this application can handle pagecache, swap and file-backed
memory.  (file-backed memory might not even need special treatment, given
that it's pagecache anyway).


And userspace can do a much better implementation of this
how-to-handle-large-load-shifts problem, because it is really quite
complex.  The system needs to be monitored to determine what is the "usual"
state (ie: the thing we wish to reestablish when the transient workload
subsides).  The system then needs to be monitored to determine when the
exceptional workload has started, and when it has subsided, and userspace
then needs to decide when to start reestablishing the old working set, at
what rate, when to abort doing that, etc.

All this would end up needing runtime configurability and tweakability and
customisability.  All standard fare for userspace stuff - much easier than
patching the kernel.


So.  We can

a) provide a way for userspace to reload pagecache and

b) merge maps2 (once it's finished) (pokes mpm)

and we're done?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Nick Piggin]

Andrew Morton wrote:

> All this would end up needing runtime configurability and tweakability and
> customisability.  All standard fare for userspace stuff - much easier than
> patching the kernel.
> 
> 
> So.  We can
> 
> a) provide a way for userspace to reload pagecache and
> 
> b) merge maps2 (once it's finished) (pokes mpm)
> 
> and we're done?


The userspace solution has been brought up before. It could be a good way
to go. I was thinking about how to do refetching of file backed pages from
the kernel, and it isn't impossible, but it it seems like locking would be
quite hard and it would be pretty complex and inflexible compared to a
userspace solution. Userspace might know what to chuck out, what to keep,
what access patterns to use...

Not that I want to say anything about swap prefetch getting merged: my
inbox is already full of enough "helpful suggestions" about that, so I'll
just be happy to have a look at little things like updatedb.

-- 
SUSE Labs, Novell Inc.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Andrew Morton]

On Thu, 26 Jul 2007 15:53:37 +1000 Nick Piggin <nickpiggin@yahoo.com.au> wrote:

> Not that I want to say anything about swap prefetch getting merged: my
> inbox is already full of enough "helpful suggestions" about that,

give them the kernel interfaces, they can do it themselves ;)

> so I'll
> just be happy to have a look at little things like updatedb.

Yes, that is a little thing.  I mean, even if the kernel's behaviour
during an updatedb run was "perfect" (ie: does what we the designers
curently intend it to do (whatever that is)) then the core problem isn't
solved: short-term workload evicts your working set and you have to
synchronously reestablish it.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Nick Piggin]

Andrew Morton wrote:
> On Thu, 26 Jul 2007 15:53:37 +1000 Nick Piggin <nickpiggin@yahoo.com.au> wrote:
> 
> 
>>Not that I want to say anything about swap prefetch getting merged: my
>>inbox is already full of enough "helpful suggestions" about that,
> 
> 
> give them the kernel interfaces, they can do it themselves ;)

It is a good idea if we can give enough to get started. Then if
they run into something they really need to do in the kernel, we
can take a look.

Page eviction order / prefetch-back-in-order might be tricky to
expose.


>>so I'll
>>just be happy to have a look at little things like updatedb.
> 
> 
> Yes, that is a little thing.  I mean, even if the kernel's behaviour
> during an updatedb run was "perfect" (ie: does what we the designers
> curently intend it to do (whatever that is)) then the core problem isn't
> solved: short-term workload evicts your working set and you have to
> synchronously reestablish it.

Sure, I know and I was never against swap (and/or file) prefetching to
solve this problem. I'm just saying, I'm staying out of that :)

-- 
SUSE Labs, Novell Inc.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Ray Lee]

On 7/25/07, Andrew Morton <akpm@linux-foundation.org> wrote:
> On Wed, 25 Jul 2007 09:09:01 -0700
> "Ray Lee" <ray-lk@madrabbit.org> wrote:
>
> > No, there's a third case which I find the most annoying. I have
> > multiple working sets, the sum of which won't fit into RAM. When I
> > finish one, the kernel had time to preemptively swap back in the
> > other, and yet it didn't. So, I sit around, twiddling my thumbs,
> > waiting for my music player to come back to life, or thunderbird,
> > or...
>
> Yes, I'm thinking that's a good problem statement and it isn't something
> which the kernel even vaguely attempts to address, apart from normal
> demand paging.
>
> We could perhaps improve things with larger and smarter fault readaround,
> perhaps guided by refault-rate measurement.  But that's still demand-paged
> rather than being proactive/predictive/whatever.
>
> None of this is swap-specific though: exactly the same problem would need
> to be solved for mmapped files and even plain old pagecache.

<nod> Could be what I'm noticing, but it's important to note that as
others have shown improvement with Con's swap prefetch, it's easily
arguable that targeting just swap is good enough for a first
approximation.

> In fact I'd restate the problem as "system is in steady state A, then there
> is a workload shift causing transition to state B, then the system goes
> idle.  We now wish to reinstate state A in anticipation of a resumption of
> the original workload".

Yes, that's a fair transformation / generalization. It's always nice
talking to someone with more clarity than one's self.

> swap-prefetch solves a part of that.
>
> A complete solution for anon and file-backed memory could be implemented
> (ta-da) in userspace using the kernel inspection tools in -mm's maps2-*
> patches.
> We would need to add a means by which userspace can repopulate
> swapcache,

Okay, let's run with that for argument's sake.

> but that doesn't sound too hard (especially when you haven't
> thought about it).

I've always thought your sense of humor was underappreciated.

> And userspace can right now work out which pages from which files are in
> pagecache so this application can handle pagecache, swap and file-backed
> memory.  (file-backed memory might not even need special treatment, given
> that it's pagecache anyway).

So in your proposed scheme, would userspace be polling, er, <goes and
looks through email for maps2 stuff, only finds Rusty's patches to
it>, well, /proc/<pids>/something_or_another?

A userspace daemon that wakes up regularly to poll a bunch of proc
files fills me with glee. Wait, is that glee? I think, no... wait...
horror, yes, horror is what I'm feeling.

I'm wrong, right? I love being wrong about this kind of stuff.

> And userspace can do a much better implementation of this
> how-to-handle-large-load-shifts problem, because it is really quite
> complex.  The system needs to be monitored to determine what is the "usual"
> state (ie: the thing we wish to reestablish when the transient workload
> subsides).  The system then needs to be monitored to determine when the
> exceptional workload has started, and when it has subsided, and userspace
> then needs to decide when to start reestablishing the old working set, at
> what rate, when to abort doing that, etc.

Oy. I mean this in the most respectful way possible, but you're too
smart for your own good.

I mean, sure, it's possible one could have multiply-chained transient
workloads each of which have their optimum workingset, of which
there's little overlap with the previous. Mainframes made their names
on such loads. Workingset A starts, generates data, finishes and
invokes workingset B, of which the only thing they share in common is
said data. B finishes and invokes C, etc.

So, yeah, that's way too complex to stuff into the kernel. Even if it
were possible to do so, I cringe at the thought. And I can't believe
that would be a common enough pattern nowadays to justify any
hueristics on anyone's part. It's certainly complex enough that I'd
like to punt that scenario out of the conversation entirely -- I think
it has the potential to give a false impression as to how involved of
a process we're talking about here.

Let's go back to your restatement:

> In fact I'd restate the problem as "system is in steady state A, then there
> is a workload shift causing transition to state B, then the system goes
> idle.  We now wish to reinstate state A in anticipation of a resumption of
> the original workload".

I'll take an 80% solution for that one problem, and happily declare
that the kernel's job is done. In particular, when a resource hog
exits (or whatever hueristics prefetch is currently hooking in to),
the kernel (or userspace, if that interface could be made sane) could
exercise a completely workload agnostic refetch of the last n things
evicted, where n is determined by what's suddenly become free (or
whatever Con came up with).

Just, y'know, MRU style.

> All this would end up needing runtime configurability and tweakability and
> customisability.  All standard fare for userspace stuff - much easier than
> patching the kernel.

We're talking about patching the kernel for whatever API you're coming
up with to repopulate pagecache, swap, and inodes, aren't we? If we
are, it doesn't seem like we're saving any work here. Also we're
talking about a creating a new user-visible API instead of augmenting
a pre-existing hueristic -- page replacement -- that the kernel
doesn't export and so can change at a moment's notice. Augmenting an
opaque hueristic seems a lot more friendly to long-term maintenance.

> So.  We can
>
> a) provide a way for userspace to reload pagecache and
>
> b) merge maps2 (once it's finished) (pokes mpm)
>
> and we're done?

Eh, dunno. Maybe?

We're assuming we come up with an API for userspace to get
notifications of evictions (without polling, though poll() would be
fine -- you know what I mean), and an API for re-victing those things
on demand. If you think that adding that API and maintaining it is
simpler/better than including a variation on the above hueristic I
offered, then yeah, I guess we are. It'll all have that vague
userspace s2ram odor about it, but I'm sure it could be made to work.

As I think I've successfully Peter Principled my way through this
conversation to my level of incompetence, I'll shut up now.

Ray

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Andrew Morton]

On Wed, 25 Jul 2007 23:33:24 -0700 "Ray Lee" <ray-lk@madrabbit.org> wrote:

> > So.  We can
> >
> > a) provide a way for userspace to reload pagecache and
> >
> > b) merge maps2 (once it's finished) (pokes mpm)
> >
> > and we're done?
> 
> Eh, dunno. Maybe?
> 
> We're assuming we come up with an API for userspace to get
> notifications of evictions (without polling, though poll() would be
> fine -- you know what I mean), and an API for re-victing those things
> on demand.

I was assuming that polling would work OK.  I expect it would.

> If you think that adding that API and maintaining it is
> simpler/better than including a variation on the above hueristic I
> offered, then yeah, I guess we are. It'll all have that vague
> userspace s2ram odor about it, but I'm sure it could be made to work.

Actually, I overdesigned the API, I suspect.  What we _could_ do is to
provide a way of allowing userspace to say "pretend process A touched page
B": adopt its mm and go touch the page.  We in fact already have that:
PTRACE_PEEKTEXT.

So I suspect this could all be done by polling maps2 and using PEEKTEXT. 
The tricky part would be working out when to poll, and when to reestablish.

A neater implementation than PEEKTEXT would be to make the maps2 files
writeable(!) so as a party trick you could tar 'em up and then, when you
want to reestablish firefox's previous working set, do a untar in
/proc/$(pidof firefox)/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Ray Lee]

On 7/25/07, Andrew Morton <akpm@linux-foundation.org> wrote:
> On Wed, 25 Jul 2007 23:33:24 -0700 "Ray Lee" <ray-lk@madrabbit.org> wrote:
> > If you think that adding that API and maintaining it is
> > simpler/better than including a variation on the above hueristic I
> > offered, then yeah, I guess we are. It'll all have that vague
> > userspace s2ram odor about it, but I'm sure it could be made to work.
>
> Actually, I overdesigned the API, I suspect.  What we _could_ do is to
> provide a way of allowing userspace to say "pretend process A touched page
> B": adopt its mm and go touch the page.  We in fact already have that:
> PTRACE_PEEKTEXT.

Huh. All right.

> So I suspect this could all be done by polling maps2 and using PEEKTEXT.
> The tricky part would be working out when to poll, and when to reestablish.

Welllllll.... there is the taskstats interface. It's not required
right now, though, and lacks most of what userspace would need, I
think. It does at least currently provide a notification of process
exit, which is a clue for when to start reestablishment. Gotta be
another way we can get at that...

Oh, stat on /proc, does that work? Huh, it does, sort of. It seems to
be off by 12 or 13, but hey, that's something.

Wish I had the time to look at the maps2 stuff, but regardless, it
probably currently provides too much detail for continual polling? I
suspect what we'd want to do is to take a detailed snapshot a little
after the beginning of a process's lifetime (once the block-in counts
subside), then poll aggregate residency or evicition counts to know
which processes are suffering the burden of the transient workload.

Eh, wait, that doesn't help with inodes. No matter, I guess; I'm the
one who said targetting swap-in would be good enough for a first pass.

On process exit, if userspace can get a hold of an estimate of the
size of what just freed up, it could then spend
min(that,evicted_count) on repopulation. That's probably already
available by polling whatever `free` calls.

> A neater implementation than PEEKTEXT would be to make the maps2 files
> writeable(!) so as a party trick you could tar 'em up and then, when you
> want to reestablish firefox's previous working set, do a untar in
> /proc/$(pidof firefox)/

I'm going to get into trouble if I wake up the other person in the
house with my laughter. That's laughter in a positive sense, not a
"you're daft" kind of way.

Huh. <thinks> So, to go back a little bit, I guess one of my problems
with polling is that it means that userspace can only approximate an
MRU of what's been evicted. Perhaps an approximation is good enough, I
don't know, but that's something to keep in mind. (Hmm, how many pages
can an average desktop evict per second? If we poll everything once
per second, that's how off we could be.)

Another is a more philosophical hangup -- running a process that polls
periodically to improve system performance seems backward. Okay, so
that's my problem to get over, not yours.

Another problem is what poor sod would be willing to write and test
this, given that there's already a written and tested kernel patch to
do much the same thing? Yeah, that's sorta rhetorical, but it's sorta
not. Given that swap prefetch could be ripped out of 2.6.n+1 if it's
introduced in 2.6.n, and nothing in userspace would be the wiser,
where's the burden?  There is some, just as any kernel code has some,
and as it's core code (versus, say, a driver), the burden is
correspondingly greater per line, but given the massive changesets
flowing through each release now, I have to think that the burden this
introduces is marginal compared to the rest of the bulk sweeping
through the kernel weekly.

This is obviously where I'm totally conjecturing, and you'll know far,
far better than I.

Offline for about 20 hours or so, not that anyone would probably notice :-).

Ray

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Nick Piggin]

Ray Lee wrote:

> Another is a more philosophical hangup -- running a process that polls
> periodically to improve system performance seems backward.

You mean like the kprefetchd of swap prefetch? ;)


> Okay, so
> that's my problem to get over, not yours.

If it was a problem you could add some event trigger to wake it up.

-- 
SUSE Labs, Novell Inc.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Matt Mackall]

On Wed, Jul 25, 2007 at 11:50:37PM -0700, Andrew Morton wrote:
> On Wed, 25 Jul 2007 23:33:24 -0700 "Ray Lee" <ray-lk@madrabbit.org> wrote:
> 
> > > So.  We can
> > >
> > > a) provide a way for userspace to reload pagecache and
> > >
> > > b) merge maps2 (once it's finished) (pokes mpm)
> > >
> > > and we're done?
> > 
> > Eh, dunno. Maybe?
> > 
> > We're assuming we come up with an API for userspace to get
> > notifications of evictions (without polling, though poll() would be
> > fine -- you know what I mean), and an API for re-victing those things
> > on demand.
> 
> I was assuming that polling would work OK.  I expect it would.
> 
> > If you think that adding that API and maintaining it is
> > simpler/better than including a variation on the above hueristic I
> > offered, then yeah, I guess we are. It'll all have that vague
> > userspace s2ram odor about it, but I'm sure it could be made to work.
> 
> Actually, I overdesigned the API, I suspect.  What we _could_ do is to
> provide a way of allowing userspace to say "pretend process A touched page
> B": adopt its mm and go touch the page.  We in fact already have that:
> PTRACE_PEEKTEXT.
> 
> So I suspect this could all be done by polling maps2 and using PEEKTEXT. 
> The tricky part would be working out when to poll, and when to reestablish.
> 
> A neater implementation than PEEKTEXT would be to make the maps2 files
> writeable(!) so as a party trick you could tar 'em up and then, when you
> want to reestablish firefox's previous working set, do a untar in
> /proc/$(pidof firefox)/

Sick. But thankfully, unnecessary. The pagemaps give you more than
just a present bit, which is all we care about here. We simply need to
record which pages are mapped, then reference them all back to life..

-- 
Mathematics is the supreme nostalgia of our time.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Michael Chang]

On 7/26/07, Andrew Morton <akpm@linux-foundation.org> wrote:
> On Wed, 25 Jul 2007 09:09:01 -0700
> "Ray Lee" <ray-lk@madrabbit.org> wrote:
>
> > No, there's a third case which I find the most annoying. I have
> > multiple working sets, the sum of which won't fit into RAM. When I
> > finish one, the kernel had time to preemptively swap back in the
> > other, and yet it didn't. So, I sit around, twiddling my thumbs,
> > waiting for my music player to come back to life, or thunderbird,
> > or...
>
> In fact I'd restate the problem as "system is in steady state A, then there
> is a workload shift causing transition to state B, then the system goes
> idle.  We now wish to reinstate state A in anticipation of a resumption of
> the original workload".
>
> swap-prefetch solves a part of that.
>
> A complete solution for anon and file-backed memory could be implemented
> (ta-da) in userspace using the kernel inspection tools in -mm's maps2-*
> patches.  We would need to add a means by which userspace can repopulate
> swapcache, but that doesn't sound too hard (especially when you haven't
> thought about it).
>
> And userspace can right now work out which pages from which files are in
> pagecache so this application can handle pagecache, swap and file-backed
> memory.  (file-backed memory might not even need special treatment, given
> that it's pagecache anyway).
>
> And userspace can do a much better implementation of this
> how-to-handle-large-load-shifts problem, because it is really quite
> complex.  The system needs to be monitored to determine what is the "usual"
> state (ie: the thing we wish to reestablish when the transient workload
> subsides).  The system then needs to be monitored to determine when the
> exceptional workload has started, and when it has subsided, and userspace
> then needs to decide when to start reestablishing the old working set, at
> what rate, when to abort doing that, etc.
>
> All this would end up needing runtime configurability and tweakability and
> customisability.  All standard fare for userspace stuff - much easier than
> patching the kernel.

Maybe I'm missing something here, but if the problem is resource
allocation when switching from state A to state B, and from B to C,
etc.; wouldn't it be a bad thing if state B happened to be (in the
future) this state-shifting userspace daemon of which you speak? (Or
is that likely to be impossible/unlikely for some other reason which
alludes me at the moment?)

-- 
Michael Chang

Please avoid sending me Word or PowerPoint attachments. Send me ODT,
RTF, or HTML instead.
See http://www.gnu.org/philosophy/no-word-attachments.html
Thank you.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Andrew Morton]

On Thu, 26 Jul 2007 10:19:06 -0400 "Michael Chang" <thenewme91@gmail.com> wrote:

> > All this would end up needing runtime configurability and tweakability and
> > customisability.  All standard fare for userspace stuff - much easier than
> > patching the kernel.
> 
> Maybe I'm missing something here, but if the problem is resource
> allocation when switching from state A to state B, and from B to C,
> etc.; wouldn't it be a bad thing if state B happened to be (in the
> future) this state-shifting userspace daemon of which you speak? (Or
> is that likely to be impossible/unlikely for some other reason which
> alludes me at the moment?)

Well.  I was assuming that the daemon wouldn't be a great memory pig. 
I suspect it would do practically zero IO and would use little memory.
It could even be mlocked, but I doubt if that would be needed.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Dirk Schoebel]

[-- Attachment #1: Type: text/plain, Size: 1323 bytes --]

I don't really understand the reasons for all those discussions.
As long as you have a maintainer, why don't you just put swap prefetch into 
the kernel, marked experimental, default deactivated so anyone who just 
make[s] oldconfig (or defaultconfig) won't get it. If anyone finds a good 
solution for all those cache 'poisoning' problems and problems concerning 
swapin on workload changes and such swap prefetch can easily taken out again 
and no one has to complain and continuing maintaining it.
Actually the same goes for plugshed (having it might have kept Con as a 
valuable developer).
I am actually waiting for more than 2 years that reiser4 will make it into the 
kernel (sure, marked experimental or even highly experimental) so the 
patch-journey for every new kernel comes to an end. And most things in-kernel 
will surely be tested more intense so bugs will come up much faster. 
(Constantly running MM kernels is not really an option since many things in 
there can't be deactivated if they don't work as expected since lots of 
patches also concern 'vital' parts of the kernel.)

...just 2cents from a happy CK user for it made it possible to watch a movie 
while compiling the system - which was impossible with mainline kernel, even 
on dual core 2.2 GHz AMD64 with 4G RAM !

Dirk.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [ck] Re: -mm merge plans for 2.6.23

[Dirk Schoebel]

[-- Attachment #1: Type: text/plain, Size: 1126 bytes --]

...sorry for the reply to myself.

As Gentoo user i'm happy about the freedom of choice in almost every aspect of 
the system.
But with the kernel this freedom is taken away and i'm left with largely 
choices other people did. Sure, i can get the sources and patch and change 
the kernel myself in every aspect but thats more the possibility given by the 
freedom of the open source of the kernel.
I think the kernel should be more open for new ideas, and as long as the 
maintainer follows the kernel development things can be left in, if the 
maintainer can't follow anymore they are taken out quite fast again. (This 
statement mostly counts for parts of the kernel where a choice is possible or 
the coding overhead of making such choice possible is quite low.)
A problem of Linux is there are 100s of patches and patchsets out there but if 
you want to have more than one (since they offer advantages or functionality 
in different places) you are mostly left alone, start to integrate all 
patches by hand and solve so many rejects.

..still a happy CK user, but sad that Con left the scene.

Dirk.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [ck] Re: -mm merge plans for 2.6.23

[Jeff Garzik]

Dirk Schoebel wrote:
> as long as the 
> maintainer follows the kernel development things can be left in, if the 
> maintainer can't follow anymore they are taken out quite fast again. (This 
> statement mostly counts for parts of the kernel where a choice is possible or 
> the coding overhead of making such choice possible is quite low.)


This is just not good engineering.

It is axiomatic that it is easy to add code, but difficult to remove 
code.  It takes -years- to remove code that no one uses.  Long after the 
maintainer disappears, the users (and bug reports!) remain.

It is also axiomatic that adding code, particularly core code, often 
exponentially increases complexity.

	Jeff


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[david]

On Thu, 26 Jul 2007, Jeff Garzik wrote:

> Dirk Schoebel wrote:
>>  as long as the maintainer follows the kernel development things can be
>>  left in, if the maintainer can't follow anymore they are taken out quite
>>  fast again. (This statement mostly counts for parts of the kernel where a
>>  choice is possible or the coding overhead of making such choice possible
>>  is quite low.)
>
>
> This is just not good engineering.
>
> It is axiomatic that it is easy to add code, but difficult to remove code. 
> It takes -years- to remove code that no one uses.  Long after the maintainer 
> disappears, the users (and bug reports!) remain.

I'll point out that the code that's so hard to remove is the code that 
exposes an API to userspace.

code that's an internal implementation (like a couple of the things being 
discussed) gets removed much faster.

> It is also axiomatic that adding code, particularly core code, often 
> exponentially increases complexity.

this is true and may be a valid argument (depending on how large and how 
intrusive the proposed patch is)

David Lang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Jeff Garzik]

david@lang.hm wrote:
> On Thu, 26 Jul 2007, Jeff Garzik wrote:
> 
>> Dirk Schoebel wrote:
>>>  as long as the maintainer follows the kernel development things can be
>>>  left in, if the maintainer can't follow anymore they are taken out 
>>> quite
>>>  fast again. (This statement mostly counts for parts of the kernel 
>>> where a
>>>  choice is possible or the coding overhead of making such choice 
>>> possible
>>>  is quite low.)
>>
>>
>> This is just not good engineering.
>>
>> It is axiomatic that it is easy to add code, but difficult to remove 
>> code. It takes -years- to remove code that no one uses.  Long after 
>> the maintainer disappears, the users (and bug reports!) remain.
> 
> I'll point out that the code that's so hard to remove is the code that 
> exposes an API to userspace.

True.


> code that's an internal implementation (like a couple of the things 
> being discussed) gets removed much faster.

Not true.  It is highly unlikely that code will get removed if it has 
active users, even if the maintainer has disappeared.

The only things that get removed rapidly are those things mathematically 
guaranteed to be dead code.

_Behavior changes_, driver removals, feature removals happen more 
frequently than userspace ABI changes -- true -- but the rate of removal 
is still very, very slow.

It is axiomatic that we are automatically burdened with new code for at 
least 10 years :)  That's what you have to assume, when accepting anything.

	Jeff



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[david]

On Thu, 26 Jul 2007, Jeff Garzik wrote:

> david@lang.hm wrote:
>>  On Thu, 26 Jul 2007, Jeff Garzik wrote:
>> 
>> >  Dirk Schoebel wrote:
>> > >   as long as the maintainer follows the kernel development things can 
>> > >   be
>> > >   left in, if the maintainer can't follow anymore they are taken out 
>> > >  quite
>> > >  fast again. (This statement mostly counts for parts of the kernel 
>> > >  where a
>> > >  choice is possible or the coding overhead of making such choice 
>> > >  possible
>> > >   is quite low.)
>> > 
>> > 
>> >  This is just not good engineering.
>> > 
>> >  It is axiomatic that it is easy to add code, but difficult to remove 
>> >  code. It takes -years- to remove code that no one uses.  Long after the 
>> >  maintainer disappears, the users (and bug reports!) remain.
>>
>>  I'll point out that the code that's so hard to remove is the code that
>>  exposes an API to userspace.
>
> True.
>
>
>>  code that's an internal implementation (like a couple of the things being
>>  discussed) gets removed much faster.
>
> Not true.  It is highly unlikely that code will get removed if it has active 
> users, even if the maintainer has disappeared.

if you propose removing code in such a way that performance suffers then 
yes, it's hard to remove (and it should be).

but if it has no API the code is only visable to the users as a side 
effect of it's use. if the new code works better then it can be replaced.

the scheduler change that we're going through right now is an example, new 
code came along that was better and the old code went away very quickly.

the SLAB/SLOB/SLUB/S**B debate is another example. currently the different 
versions have different performance advantages and disadvantages, as one 
drops behind to the point where one of the others is better at all times, 
it goes away.

> The only things that get removed rapidly are those things mathematically 
> guaranteed to be dead code.
>
> _Behavior changes_, driver removals, feature removals happen more frequently 
> than userspace ABI changes -- true -- but the rate of removal is still very, 
> very slow.

a large part of this is that it's so hard to get a replacement that works 
better (this is very definantly a compliment to the kernel coders :-)



> It is axiomatic that we are automatically burdened with new code for at 
> least 10 years :)  That's what you have to assume, when accepting 
> anything.

for userspace API's 10 years is reasonable, for internal features it's 
not. there is a LOT of internal stuff that was in the kernel 10 (or even 
5) years ago that isn't there now. the key is that the behavior as far as 
users is concerned is better now.

David Lang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Matt Mackall]

On Wed, Jul 25, 2007 at 09:57:17PM -0700, Andrew Morton wrote:
> So.  We can
> 
> a) provide a way for userspace to reload pagecache and
> 
> b) merge maps2 (once it's finished) (pokes mpm)

Consider me poked, despite not being cc:ed.

-- 
Mathematics is the supreme nostalgia of our time.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Daniel Cheng]

Andrew Morton wrote:
[...]
> 
> And userspace can do a much better implementation of this
> how-to-handle-large-load-shifts problem, because it is really quite
> complex.  The system needs to be monitored to determine what is the "usual"
[...]
> All this would end up needing runtime configurability and tweakability and
> customisability.  All standard fare for userspace stuff - much easier than
> patching the kernel.

But a patch already exist.
Which is easier: (1) apply the patch ; or (2) write a new patch?

> 
> So.  We can
> a) provide a way for userspace to reload pagecache and
> b) merge maps2 (once it's finished) (pokes mpm)
> and we're done?

might be.
but merging maps2 have higher risk which should be done in a development
branch (er... 2.7, but we don't have it now).

-- 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Stefan Richter]

Daniel Cheng wrote:
> but merging maps2 have higher risk which should be done in a development
> branch (er... 2.7, but we don't have it now).

This is off-topic and has been discussed to death, but:  Rather than one
stable branch and one development branch, we have a few stable branches
and a lot of development branches.  Some are located at git.kernel.org.
Among else, this gives you a predictable release rythm and very timely
updated stable branches.
-- 
Stefan Richter
-=====-=-=== -=== ===--
http://arcgraph.de/sr/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Frank A. Kingswood]

Nick Piggin wrote:
> OK, this is where I start to worry. Swap prefetch AFAIKS doesn't fix
> the updatedb problem very well, because if updatedb has caused swapout
> then it has filled memory, and swap prefetch doesn't run unless there
> is free memory (not to mention that updatedb would have paged out other
> files as well).

It is *not* about updatedb. That is just a trivial case which people 
notice. Therefore fixing updatedb to be nicer, as was discussed at 
various points in this thread, is *not* the solution.
Most users are also *not*at*all* interested in kernel builds as a metric 
of system performance.

When I'm at work, I run a large, commercial, engineering application. 
While running, it takes most of the system memory (4GB and up), and it 
reads and writes very large files. Swap prefetch noticeably helps my 
desktop too. Can I measure it? Not sure. Can people on lkml fix the 
application? Certainly not.

Frank

Re: [ck] Re: -mm merge plans for 2.6.23

[Matthew Hawkins]

On 7/25/07, Nick Piggin <nickpiggin@yahoo.com.au> wrote:
> I'm not saying that we can't try to tackle that problem, but first of
> all you have a really nice narrow problem where updatedb seems to be
> causing the kernel to completely do the wrong thing. So we start on
> that.

updatedb isn't the only problem, its just an obvious one.  I like the
idea of looking into the vfs for this and other one-shot applications
(rather than looking at updatedb itself specifically)

Many modern applications have a lot of open file handles.  For
example, I just fired up my usual audio player and sys/fs/file-nr
showed another 600 open files (funnily enough, I have roughly that
many audio files :)  I'm not exactly sure what happens when this one
gets swapped out for whatever reason (firefox/java/vmware/etc chews
ram, updatedb, whatever) but I'm fairly confident what happens between
kswapd and the vfs and whatever else we're caching is not optimal come
time for this process to context-switch back in.  We're not running a
highly-optimised number-crunching scientific app on desktops, we're
running a full herd of poorly-coded hogs simultaneously through
smaller pens.

I don't think anyone is trying to claim that swap prefetch is the be
all and end all of this problem's solution, however without it the
effects are an order of magnitude worse (I've cited numbers elsewhere,
as have several others); its relatively non-intrusive (600+ lines of
the 755 changed ones are self-contained), is compile and runtime
selectable, and still has a maintainer now that Con has retired.  If
there was a better solution, it should have been developed sometime in
the past 23 months that swap prefetch has addressed it.  That's how we
got rmap versus aa, and so on.  But nobody chose to do so, and
continuing to hold out on merging it on the promise of vapourware is
ridiculous.  That has never been the way linux kernel development has
operated.

-- 
Matt

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[Nick Piggin]

Matthew Hawkins wrote:
> On 7/25/07, Nick Piggin <nickpiggin@yahoo.com.au> wrote:
> 
>> I'm not saying that we can't try to tackle that problem, but first of
>> all you have a really nice narrow problem where updatedb seems to be
>> causing the kernel to completely do the wrong thing. So we start on
>> that.
> 
> 
> updatedb isn't the only problem, its just an obvious one.  I like the
> idea of looking into the vfs for this and other one-shot applications
> (rather than looking at updatedb itself specifically)

That's the point, it is an obvious one. So it should be easy to work
out why it is going wrong, and fix it. (And hopefully that fixes some
of the less obvious problems too.)


> Many modern applications have a lot of open file handles.  For
> example, I just fired up my usual audio player and sys/fs/file-nr
> showed another 600 open files (funnily enough, I have roughly that
> many audio files :)  I'm not exactly sure what happens when this one
> gets swapped out for whatever reason (firefox/java/vmware/etc chews
> ram, updatedb, whatever) but I'm fairly confident what happens between
> kswapd and the vfs and whatever else we're caching is not optimal come
> time for this process to context-switch back in.  We're not running a
> highly-optimised number-crunching scientific app on desktops, we're
> running a full herd of poorly-coded hogs simultaneously through
> smaller pens.

And yet nobody wants to take the time to properly analyse why these
things are going wrong and reporting their findings? Or if they have,
where is that documented?

-- 
SUSE Labs, Novell Inc.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Ray Lee]

On 7/24/07, Nick Piggin <nickpiggin@yahoo.com.au> wrote:
> Ray Lee wrote:
> > On 7/23/07, Nick Piggin <nickpiggin@yahoo.com.au> wrote:
> >> If we can first try looking at
> >> some specific problems that are easily identified.
> >
> > Always easier, true. Let's start with "My mouse jerks around under
> > memory load." A Google Summer of Code student working on X.Org claims
> > that mlocking the mouse handling routines gives a smooth cursor under
> > load ([1]). It's surprising that the kernel would swap that out in the
> > first place.
> >
> > [1]
> > http://vignatti.wordpress.com/2007/07/06/xorg-input-thread-summary-or-something/
>
> OK, I'm not sure what the point is though. Under heavy memory load,
> things are going to get swapped out... and swap prefetch isn't going
> to help there (at least, not during the memory load).

Sorry, I headed slightly off-topic. Or perhaps 'up-topic' to the
larger issue, which is that the desktop experience has some suckiness
to it.

My point is that the page replacement algorithm has some choice as to
what to evict. The xorg input handler never should have been evicted.
It was hopefully a hard example of where the current page replacement
policy is falling flat on its face.

All that said, this could really easily be handled by xorg mlocking
the critical realtime stuff.

> There are also other issues like whether the CPU scheduler is at fault,
> etc. Interactive workloads are always the hardest to work out.

This one is not a scheduler issue, as mlock()ing the mouse handling
routines gives a smooth cursor. It's just a pure page replacement
problem, as the kernel should never have swapped that out in the first
place.

<snip things I agreed with>

<snip list of things to watch during updatedb run>

Ray

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Andi Kleen]

Nick Piggin <nickpiggin@yahoo.com.au> writes:

> Ray Lee wrote:
> > On 7/23/07, Nick Piggin <nickpiggin@yahoo.com.au> wrote:
> 
> >> Also a random day at the desktop, it is quite a broad scope and
> >> pretty well impossible to analyse.
> > It is pretty broad, but that's also what swap prefetch is targetting.
> > As for hard to analyze, I'm not sure I agree. One can black-box test
> > this stuff with only a few controls. e.g., if I use the same apps each
> > day (mercurial, firefox, xorg, gcc), and the total I/O wait time
> > consistently goes down on a swap prefetch kernel (normalized by some
> > control statistic, such as application CPU time or total I/O, or
> > something), then that's a useful measurement.
> 
> I'm not saying that we can't try to tackle that problem, but first of
> all you have a really nice narrow problem where updatedb seems to be
> causing the kernel to completely do the wrong thing. So we start on
> that.

One simple way to fix this would be to implement a fadvise() flag
that puts the dentry/inode on a "soon to be expired" list if there
are no other references. Then if a dentry allocation needs more
memory try to reuse dentries from that list (or better queue) first. Any other
access will remove the dentry from the list. 

Disadvantage would be that the userland would need to be patched,
but I guess it's better than adding very dubious heuristics to the
kernel.

Similar thing could be done for directory buffers although they
are probably less of a problem.

I expect that C.Lameter's directed dentry/inode freeing in slub will also
make a big difference. People who have problems with updatedb should
definitely try mm which has it I believe and enable SLUB.

-Andi (who always thought swap prefetch was just a workaround, not
a real solution) 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Frank Kingswood]

Andi Kleen wrote:
> One simple way to fix this would be to implement a fadvise() flag
> that puts the dentry/inode on a "soon to be expired" list if there
> are no other references. Then if a dentry allocation needs more
> memory try to reuse dentries from that list (or better queue) first. Any other
> access will remove the dentry from the list. 
> 
> Disadvantage would be that the userland would need to be patched,
> but I guess it's better than adding very dubious heuristics to the
> kernel.

Are you going to change every single large memory application in the 
world? As I wrote before, it is *not* about updatedb, but about all 
applications that use a lot of memory, and then terminate.

Frank

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Ingo Molnar]

* Frank Kingswood <frank@kingswood-consulting.co.uk> wrote:

> > Disadvantage would be that the userland would need to be patched, 
> > but I guess it's better than adding very dubious heuristics to the 
> > kernel.
> 
> Are you going to change every single large memory application in the 
> world? As I wrote before, it is *not* about updatedb, but about all 
> applications that use a lot of memory, and then terminate.

it is about multiple problems, _one_ problem is updatedb. The _second_ 
problem is large memory applications.

note that updatedb is not a "large memory application". It simply scans 
through the filesystem and has pretty minimal memory footprint.

the _kernel_ ends up blowing up the dentry cache to a rather large size 
(because it has no idea that updatedb uses every dentry only once).

Once we give the kernel the knowledge that the dentry wont be used again 
by this app, the kernel can do a lot more intelligent decision and not 
baloon the dentry cache.

( we _do_ want to baloon the dentry cache otherwise - for things like 
  "find" - having a fast VFS is important. But known-use-once things 
  like the daily updatedb job can clearly be annotated properly. )

the 'large memory apps' are a second category of problems. And those are 
where swap-prefetch could indeed help. (as long as it only 'fills up' 
the free memory that a large-memory-exit left behind it.)

the 'morning after' phenomenon that the majority of testers complained 
about will likely be resolved by the updatedb change. The second 
category is likely an improvement too, for swap-happy desktop (and 
server) workloads.

	Ingo

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Andrew Morton]

On Thu, 26 Jul 2007 11:20:25 +0200 Ingo Molnar <mingo@elte.hu> wrote:

> Once we give the kernel the knowledge that the dentry wont be used again 
> by this app, the kernel can do a lot more intelligent decision and not 
> baloon the dentry cache.
> 
> ( we _do_ want to baloon the dentry cache otherwise - for things like 
>   "find" - having a fast VFS is important. But known-use-once things 
>   like the daily updatedb job can clearly be annotated properly. )

Mutter.  /proc/sys/vm/vfs_cache_pressure has been there for what, three
years?  Are any distros raising it during the updatedb run yet?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Ingo Molnar]

* Andrew Morton <akpm@linux-foundation.org> wrote:

> On Thu, 26 Jul 2007 11:20:25 +0200 Ingo Molnar <mingo@elte.hu> wrote:
> 
> > Once we give the kernel the knowledge that the dentry wont be used again 
> > by this app, the kernel can do a lot more intelligent decision and not 
> > baloon the dentry cache.
> > 
> > ( we _do_ want to baloon the dentry cache otherwise - for things like 
> >   "find" - having a fast VFS is important. But known-use-once things 
> >   like the daily updatedb job can clearly be annotated properly. )
> 
> Mutter.  /proc/sys/vm/vfs_cache_pressure has been there for what, 
> three years?  Are any distros raising it during the updatedb run yet?

but ... that's system-wide, and the 'dont baloon the dcache' is only a 
property of updatedb. Still, it's useful to debug this thing.

below is an updatedb hack that sets vfs_cache_pressure down to 0 during 
an updatedb run. Could someone who is affected by the 'morning after' 
problem give it a try? If this works then we can think about any other 
measures ...

	Ingo

--- /etc/cron.daily/mlocate.cron.orig
+++ /etc/cron.daily/mlocate.cron
@@ -1,4 +1,7 @@
 #!/bin/sh
 nodevs=$(< /proc/filesystems awk '$1 == "nodev" { print $2 }')
 renice +19 -p $$ >/dev/null 2>&1
+PREV=`cat /proc/sys/vm/vfs_cache_pressure 2>/dev/null`
+echo 0 > /proc/sys/vm/vfs_cache_pressure 2>/dev/null
 /usr/bin/updatedb -f "$nodevs"
+[ "$PREV" != "" ] && echo $PREV > /proc/sys/vm/vfs_cache_pressure 2>/dev/null

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Andrew Morton]

On Thu, 26 Jul 2007 11:40:24 +0200 Ingo Molnar <mingo@elte.hu> wrote:

> 
> * Andrew Morton <akpm@linux-foundation.org> wrote:
> 
> > On Thu, 26 Jul 2007 11:20:25 +0200 Ingo Molnar <mingo@elte.hu> wrote:
> > 
> > > Once we give the kernel the knowledge that the dentry wont be used again 
> > > by this app, the kernel can do a lot more intelligent decision and not 
> > > baloon the dentry cache.
> > > 
> > > ( we _do_ want to baloon the dentry cache otherwise - for things like 
> > >   "find" - having a fast VFS is important. But known-use-once things 
> > >   like the daily updatedb job can clearly be annotated properly. )
> > 
> > Mutter.  /proc/sys/vm/vfs_cache_pressure has been there for what, 
> > three years?  Are any distros raising it during the updatedb run yet?
> 
> but ... that's system-wide, and the 'dont baloon the dcache' is only a 
> property of updatedb.

Sure, but it's practical, isn't it?  Who runs (and cares about)
vfs-intensive workloads during their wee-small-hours updatedb run?

(OK, I do, but I kill the damn thing if it goes off)

> Still, it's useful to debug this thing.
> 
> below is an updatedb hack that sets vfs_cache_pressure down to 0 during 
> an updatedb run. Could someone who is affected by the 'morning after' 
> problem give it a try? If this works then we can think about any other 
> measures ...
> 
> 	Ingo
> 
> --- /etc/cron.daily/mlocate.cron.orig
> +++ /etc/cron.daily/mlocate.cron
> @@ -1,4 +1,7 @@
>  #!/bin/sh
>  nodevs=$(< /proc/filesystems awk '$1 == "nodev" { print $2 }')
>  renice +19 -p $$ >/dev/null 2>&1
> +PREV=`cat /proc/sys/vm/vfs_cache_pressure 2>/dev/null`
> +echo 0 > /proc/sys/vm/vfs_cache_pressure 2>/dev/null
>  /usr/bin/updatedb -f "$nodevs"
> +[ "$PREV" != "" ] && echo $PREV > /proc/sys/vm/vfs_cache_pressure 2>/dev/null

Setting it to zero will maximise the preservation of the vfs caches.  You
wanted 10000 there.

<bets that nobody will test this>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Ingo Molnar]

* Andrew Morton <akpm@linux-foundation.org> wrote:

> Setting it to zero will maximise the preservation of the vfs caches.  
> You wanted 10000 there.

ok, updated patch below :-)

> <bets that nobody will test this>

wrong, it's active on three of my boxes already :) But then again, i 
never had these hangover problems. (not really expected with gigs of RAM 
anyway)

	Ingo

--- /etc/cron.daily/mlocate.cron.orig
+++ /etc/cron.daily/mlocate.cron
@@ -1,4 +1,7 @@
 #!/bin/sh
 nodevs=$(< /proc/filesystems awk '$1 == "nodev" { print $2 }')
 renice +19 -p $$ >/dev/null 2>&1
+PREV=`cat /proc/sys/vm/vfs_cache_pressure 2>/dev/null`
+echo 10000 > /proc/sys/vm/vfs_cache_pressure 2>/dev/null
 /usr/bin/updatedb -f "$nodevs"
+[ "$PREV" != "" ] && echo $PREV > /proc/sys/vm/vfs_cache_pressure 2>/dev/null

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Matthew Hawkins]

On 7/26/07, Ingo Molnar <mingo@elte.hu> wrote:
> wrong, it's active on three of my boxes already :) But then again, i
> never had these hangover problems. (not really expected with gigs of RAM
> anyway)
[...]
> --- /etc/cron.daily/mlocate.cron.orig
[...]

mlocate by design doesn't thrash the cache as much.  People using
slocate (distros other than Redhat ;) are going to be hit worse.  See
http://carolina.mff.cuni.cz/~trmac/blog/mlocate/

updatedb by itself doesn't really bug me, its just that on occasion
its still running at 7am which then doesn't assist my single spindle
come swapin of the other apps!  I'm considering getting one of the old
ide drives out in the garage and shifting swap onto it.  The swap
prefetch patch has mainly assisted me in the "state A -> B -> A"
scenario.  A lot.

-- 
Matt

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Helge Hafting]

Matthew Hawkins wrote:
> updatedb by itself doesn't really bug me, its just that on occasion
> its still running at 7am 
You should start it earlier then - assuming it doesn't
already start at the earliest opportunity?


Helge Hafting

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Ingo Molnar]

* Andrew Morton <akpm@linux-foundation.org> wrote:

> > > > ( we _do_ want to baloon the dentry cache otherwise - for things like 
> > > >   "find" - having a fast VFS is important. But known-use-once things 
> > > >   like the daily updatedb job can clearly be annotated properly. )
> > > 
> > > Mutter.  /proc/sys/vm/vfs_cache_pressure has been there for what, 
> > > three years?  Are any distros raising it during the updatedb run yet?
> > 
> > but ... that's system-wide, and the 'dont baloon the dcache' is only a 
> > property of updatedb.
> 
> Sure, but it's practical, isn't it?  Who runs (and cares about) 
> vfs-intensive workloads during their wee-small-hours updatedb run?

there's another side-effect: it likely results in the zapping of 
thousands of dentries that were cached nicely before. So we might 
exchange 'all my apps are swapped out' experience with 'all file access 
is slow'. The latter is _probably_ still an improvement over the 
balooning, but i'm not sure. What we _really_ want is an updatedb that 
does not disturb the dcache.

	Ingo

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Andrew Morton]

On Thu, 26 Jul 2007 12:27:30 +0200 Ingo Molnar <mingo@elte.hu> wrote:

> 
> * Andrew Morton <akpm@linux-foundation.org> wrote:
> 
> > > > > ( we _do_ want to baloon the dentry cache otherwise - for things like 
> > > > >   "find" - having a fast VFS is important. But known-use-once things 
> > > > >   like the daily updatedb job can clearly be annotated properly. )
> > > > 
> > > > Mutter.  /proc/sys/vm/vfs_cache_pressure has been there for what, 
> > > > three years?  Are any distros raising it during the updatedb run yet?
> > > 
> > > but ... that's system-wide, and the 'dont baloon the dcache' is only a 
> > > property of updatedb.
> > 
> > Sure, but it's practical, isn't it?  Who runs (and cares about) 
> > vfs-intensive workloads during their wee-small-hours updatedb run?
> 
> there's another side-effect: it likely results in the zapping of 
> thousands of dentries that were cached nicely before. So we might 
> exchange 'all my apps are swapped out' experience with 'all file access 
> is slow'. The latter is _probably_ still an improvement over the 
> balooning, but i'm not sure.

Yup.  Nobody has begun to think about preserving dcache/icache across load
shifts yet, afaik.  Hard.

> What we _really_ want is an updatedb that 
> does not disturb the dcache.

Well.  Hopefully this time next year you can prep a 16MB container and toss
your updatedb inside that.  Maybe set its peak disk bandwidth utilisation
too.  However that won't work ;) because I don't think anyone is looking
at containerisation of vfs cache memory yet.  Perhaps full-on openvz has it,
dunno.

But updatedb is a special case, because it is so vfs-intensive.  For lots
of other workloads (those which use heaps of pagecache), resource
management via containerisation will work nicely.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Mike Galbraith]

On Thu, 2007-07-26 at 03:09 -0700, Andrew Morton wrote:

> Setting it to zero will maximise the preservation of the vfs caches.  You
> wanted 10000 there.
> 
> <bets that nobody will test this>

drops caches prior to both updatedb runs.

root@Homer: df -i
Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/hdc3            12500992 1043544 11457448    9% /
udev                  129162    1567  127595    2% /dev
/dev/hdc1              26104      87   26017    1% /boot
/dev/hda1             108144   90676   17468   84% /windows/C
/dev/hda5              11136    3389    7747   31% /windows/D
/dev/hda6                  0       0       0    -  /windows/E

vfs_cache_pressure=10000, updatedb freshly completed:
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 1  0     48  76348 420356 104748    0    0     0     0 1137  912  3  1 97  0

ext3_inode_cache  315153 316274    524    7    1 : tunables   54   27    8 : slabdata  45182  45182      0
dentry_cache      224829 281358    136   29    1 : tunables  120   60    8 : slabdata   9702   9702      0
buffer_head       156624 159728     56   67    1 : tunables  120   60    8 : slabdata   2384   2384      0

vfs_cache_pressure=100 (stock), updatedb freshly completed:

procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
1  0    148  83824 270088 116340    0    0     0     0 1095  330  2  1 97  0
 
ext3_inode_cache  467257 502495    524    7    1 : tunables   54   27    8 : slabdata  71785  71785      0
dentry_cache      292695 408958    136   29    1 : tunables  120   60    8 : slabdata  14102  14102      0
buffer_head       118329 184384     56   67    1 : tunables  120   60    8 : slabdata   2752   2752      1

Note:  updatedb doesn't bother my box, not running enough leaky apps I
guess.

	-Mike

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Andrew Morton]

On Thu, 26 Jul 2007 14:46:58 +0200 Mike Galbraith <efault@gmx.de> wrote:

> On Thu, 2007-07-26 at 03:09 -0700, Andrew Morton wrote:
> 
> > Setting it to zero will maximise the preservation of the vfs caches.  You
> > wanted 10000 there.
> > 
> > <bets that nobody will test this>
> 
> drops caches prior to both updatedb runs.

I think that was the wrong thing to do.  That will leave gobs of free
memory for updatedb to populate with dentries and inodes.

Instead, fill all of memory up with pagecache, then do the updatedb.  See
how much pagecache is left behind and see how large the vfs caches end up.

> root@Homer: df -i
> Filesystem            Inodes   IUsed   IFree IUse% Mounted on
> /dev/hdc3            12500992 1043544 11457448    9% /
> udev                  129162    1567  127595    2% /dev
> /dev/hdc1              26104      87   26017    1% /boot
> /dev/hda1             108144   90676   17468   84% /windows/C
> /dev/hda5              11136    3389    7747   31% /windows/D
> /dev/hda6                  0       0       0    -  /windows/E
> 
> vfs_cache_pressure=10000, updatedb freshly completed:
> procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
>  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
>  1  0     48  76348 420356 104748    0    0     0     0 1137  912  3  1 97  0
> 
> ext3_inode_cache  315153 316274    524    7    1 : tunables   54   27    8 : slabdata  45182  45182      0
> dentry_cache      224829 281358    136   29    1 : tunables  120   60    8 : slabdata   9702   9702      0
> buffer_head       156624 159728     56   67    1 : tunables  120   60    8 : slabdata   2384   2384      0
> 
> vfs_cache_pressure=100 (stock), updatedb freshly completed:
> 
> procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
>  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
> 1  0    148  83824 270088 116340    0    0     0     0 1095  330  2  1 97  0
>  
> ext3_inode_cache  467257 502495    524    7    1 : tunables   54   27    8 : slabdata  71785  71785      0
> dentry_cache      292695 408958    136   29    1 : tunables  120   60    8 : slabdata  14102  14102      0
> buffer_head       118329 184384     56   67    1 : tunables  120   60    8 : slabdata   2752   2752      1
> 
> Note:  updatedb doesn't bother my box, not running enough leaky apps I
> guess.
> 

So you ended up with a couple hundred MB of pagecache preserved.

Capturing before-and-after /proc/meminfo would be nice - it's a useful
summary.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Mike Galbraith]

On Thu, 2007-07-26 at 11:05 -0700, Andrew Morton wrote:
> On Thu, 26 Jul 2007 14:46:58 +0200 Mike Galbraith <efault@gmx.de> wrote:
> 
> > On Thu, 2007-07-26 at 03:09 -0700, Andrew Morton wrote:
> > 
> > > Setting it to zero will maximise the preservation of the vfs caches.  You
> > > wanted 10000 there.
> > > 
> > > <bets that nobody will test this>
> > 
> > drops caches prior to both updatedb runs.
> 
> I think that was the wrong thing to do.  That will leave gobs of free
> memory for updatedb to populate with dentries and inodes.
> 
> Instead, fill all of memory up with pagecache, then do the updatedb.  See
> how much pagecache is left behind and see how large the vfs caches end up.

Yeah.  Before these two runs just to see what difference there was in
caches with those two settings, I tried running with a heavier than
normal (for me) desktop application mix, to see if it would start
swapping, but it didn't.   Seems that 1GB ram is enough space for
everything I do, and everything updatedb does as well.  You need a
larger working set to feel the pain I guess.

	-Mike

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Mike Galbraith]

On Fri, 2007-07-27 at 07:13 +0200, Mike Galbraith wrote:
> On Thu, 2007-07-26 at 11:05 -0700, Andrew Morton wrote:
> > > drops caches prior to both updatedb runs.
> > 
> > I think that was the wrong thing to do.  That will leave gobs of free
> > memory for updatedb to populate with dentries and inodes.
> > 
> > Instead, fill all of memory up with pagecache, then do the updatedb.  See
> > how much pagecache is left behind and see how large the vfs caches end up.

I didn't _fill_ memory, but loaded it up a bit with some real workload
data...

I tried time sh -c 'git diff v2.6.11 HEAD > /dev/null' to populate the
cache, and tried different values for vfs_cache_pressure.  Nothing
prevented git's data from being trashed by updatedb.  Turning the knob
downward rapidly became very unpleasant due to swap, (with 0 not
surprisingly being a true horror) but turning it up didn't help git one
bit.  The amount of data that had to be re-read with stock 100 or 10000
was the same, or at least so close that you couldn't see a difference in
vmstat and wall-clock.  Cache sizes varied, but the bottom line didn't.
(wasn't surprised, seems quite reasonable that git's data looks old and
useless to the reclaim logic when updatedb runs in between git runs)

	-Mike

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Andrew Morton]

On Fri, 27 Jul 2007 09:23:41 +0200 Mike Galbraith <efault@gmx.de> wrote:

> On Fri, 2007-07-27 at 07:13 +0200, Mike Galbraith wrote:
> > On Thu, 2007-07-26 at 11:05 -0700, Andrew Morton wrote:
> > > > drops caches prior to both updatedb runs.
> > > 
> > > I think that was the wrong thing to do.  That will leave gobs of free
> > > memory for updatedb to populate with dentries and inodes.
> > > 
> > > Instead, fill all of memory up with pagecache, then do the updatedb.  See
> > > how much pagecache is left behind and see how large the vfs caches end up.
> 
> I didn't _fill_ memory, but loaded it up a bit with some real workload
> data...
> 
> I tried time sh -c 'git diff v2.6.11 HEAD > /dev/null' to populate the
> cache, and tried different values for vfs_cache_pressure.  Nothing
> prevented git's data from being trashed by updatedb.  Turning the knob
> downward rapidly became very unpleasant due to swap, (with 0 not
> surprisingly being a true horror) but turning it up didn't help git one
> bit.  The amount of data that had to be re-read with stock 100 or 10000
> was the same, or at least so close that you couldn't see a difference in
> vmstat and wall-clock.  Cache sizes varied, but the bottom line didn't.
> (wasn't surprised, seems quite reasonable that git's data looks old and
> useless to the reclaim logic when updatedb runs in between git runs)
> 

Did a bit of playing with this with 128MB of memory.

- drop caches
- read a 1MB file
- run slocate.cron

With vfs_cache_pressure=100:

MemTotal:       116316 kB
MemFree:          3196 kB
Buffers:         54408 kB
Cached:           5128 kB
SwapCached:          0 kB
Active:          41728 kB
Inactive:        27540 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:       116316 kB
LowFree:          3196 kB
SwapTotal:     1020116 kB
SwapFree:      1019496 kB
Dirty:               0 kB
Writeback:           0 kB
AnonPages:        9760 kB
Mapped:           3808 kB
Slab:            40468 kB
SReclaimable:    34824 kB
SUnreclaim:       5644 kB
PageTables:        720 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   1078272 kB
Committed_AS:    25988 kB
VmallocTotal:   901112 kB
VmallocUsed:       656 kB
VmallocChunk:   900412 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     4096 kB

WIth vfs_cache_pressure=10000:

MemTotal:       116316 kB
MemFree:          3060 kB
Buffers:         80792 kB
Cached:           5052 kB
SwapCached:          0 kB
Active:          59432 kB
Inactive:        36140 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:       116316 kB
LowFree:          3060 kB
SwapTotal:     1020116 kB
SwapFree:      1019512 kB
Dirty:               0 kB
Writeback:           0 kB
AnonPages:        9756 kB
Mapped:           3832 kB
Slab:            14304 kB
SReclaimable:     7992 kB
SUnreclaim:       6312 kB
PageTables:        732 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   1078272 kB
Committed_AS:    26000 kB
VmallocTotal:   901112 kB
VmallocUsed:       656 kB
VmallocChunk:   900412 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     4096 kB

so we reaped quite a lot more slab with the higher vfs_cache_pressure.

What I think is killing us here is the blockdev pagecache: the pagecache
which backs those directory entries and inodes.  These pages get read
multiple times because they hold multiple directory entries and multiple
inodes.  These multiple touches will put those pages onto the active list
so they stick around for a long time and everything else gets evicted.


I've never been very sure about this policy for the metadata pagecache.  We
read the filesystem objects into the dcache and icache and then we won't
read from that page again for a long time (I expect).  But the page will
still hang around for a long time.

It could be that we should leave those pages inactive.

<tries it>

diff -puN include/linux/buffer_head.h~a include/linux/buffer_head.h
--- a/include/linux/buffer_head.h~a
+++ a/include/linux/buffer_head.h
@@ -130,7 +130,7 @@ BUFFER_FNS(Eopnotsupp, eopnotsupp)
 BUFFER_FNS(Unwritten, unwritten)
 
 #define bh_offset(bh)		((unsigned long)(bh)->b_data & ~PAGE_MASK)
-#define touch_buffer(bh)	mark_page_accessed(bh->b_page)
+#define touch_buffer(bh)	do { } while(0)
 
 /* If we *know* page->private refers to buffer_heads */
 #define page_buffers(page)					\
_

vfs_cache_pressure=100:

vmm:/home/akpm# cat /proc/meminfo 
MemTotal:       116524 kB
MemFree:          2692 kB
Buffers:         51044 kB
Cached:           5440 kB
SwapCached:          0 kB
Active:          19248 kB
Inactive:        46996 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:       116524 kB
LowFree:          2692 kB
SwapTotal:     1020116 kB
SwapFree:      1019492 kB
Dirty:            2008 kB
Writeback:           0 kB
AnonPages:        9772 kB
Mapped:           3812 kB
Slab:            44336 kB
SReclaimable:    38792 kB
SUnreclaim:       5544 kB
PageTables:        720 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   1078376 kB
Committed_AS:    26108 kB
VmallocTotal:   901112 kB
VmallocUsed:       648 kB
VmallocChunk:   900412 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     4096 kB


vfs_cache_pressure=10000

vmm:/home/akpm# cat /proc/meminfo                       
MemTotal:       116524 kB
MemFree:          3720 kB
Buffers:         79832 kB
Cached:           6260 kB
SwapCached:          0 kB
Active:          18276 kB
Inactive:        77584 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:       116524 kB
LowFree:          3720 kB
SwapTotal:     1020116 kB
SwapFree:      1019500 kB
Dirty:            2228 kB
Writeback:           0 kB
AnonPages:        9788 kB
Mapped:           3828 kB
Slab:            13680 kB
SReclaimable:     7676 kB
SUnreclaim:       6004 kB
PageTables:        736 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   1078376 kB
Committed_AS:    26112 kB
VmallocTotal:   901112 kB
VmallocUsed:       648 kB
VmallocChunk:   900412 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     4096 kB


So again, slab was trimmed a lot more, but all our pagecache still got
evicted.

ah, but I started that pagecache out on the inactive list.  Try again. 
This time, instead of reading a 1GB file once, let's read an 80MB file four
times.

<no difference>

OK, I saw what happened then.  The inode for my 80MB file got reclaimed
from icache and that instantly reclaimed all 80MB of pagecache.

A single large file probably isn't a good testcase, but the same will
happen with multiple files.  Higher vfs_cache_pressure will worsen this
effect.  But it won't happen with mapped files because their inodes aren't
reclaimable.  More sophisticated testing is needed - there's something in
ext3-tools which will mmap, page in and hold a file for you.


Anyway, blockdev pagecache is a problem, I expect.  It's worth playing with
that patch.

Another problem is atime updates.  You really do want to mount noatime. 
Because with atimes enabled, each touch of a file will touch its inode and
will keep its backing blockdev pagecache page in core.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Al Viro]

On Fri, Jul 27, 2007 at 01:47:49AM -0700, Andrew Morton wrote:
> What I think is killing us here is the blockdev pagecache: the pagecache
> which backs those directory entries and inodes.  These pages get read
> multiple times because they hold multiple directory entries and multiple
> inodes.  These multiple touches will put those pages onto the active list
> so they stick around for a long time and everything else gets evicted.

I wonder what happens if you try that on ext2.  There we'd get directory
contents in per-directory page cache, so the picture might change...

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Andrew Morton]

On Fri, 27 Jul 2007 09:54:41 +0100 Al Viro <viro@ftp.linux.org.uk> wrote:

> On Fri, Jul 27, 2007 at 01:47:49AM -0700, Andrew Morton wrote:
> > What I think is killing us here is the blockdev pagecache: the pagecache
> > which backs those directory entries and inodes.  These pages get read
> > multiple times because they hold multiple directory entries and multiple
> > inodes.  These multiple touches will put those pages onto the active list
> > so they stick around for a long time and everything else gets evicted.
> 
> I wonder what happens if you try that on ext2.  There we'd get directory
> contents in per-directory page cache, so the picture might change...

afacit ext2 just forgets to run mark_page_accessed for directory pages
altogether, so it'll be equivalent to ext3 with that one-liner, I expect.

The directory pagecache on ext2 might get reclaimed faster because those
pages are eligible for reclaiming via the reclaim of their inodes, whereas
ext3's directories are in blockdev pagecache, for which the reclaim-via-inode
mechanism cannot happen.

I should do some testing with mmapped files.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Mike Galbraith]

On Fri, 2007-07-27 at 01:47 -0700, Andrew Morton wrote:

> Anyway, blockdev pagecache is a problem, I expect.  It's worth playing with
> that patch.

(may tinker a bit, but i'm way rusty.  ain't had the urge to mutilate
anything down there in quite a while... works just fine for me these
days)

> Another problem is atime updates.  You really do want to mount noatime. 
> Because with atimes enabled, each touch of a file will touch its inode and
> will keep its backing blockdev pagecache page in core.

Yeah, I mount noatime,nodiratime,data=writeback.  ext3's journal with my
crusty old disk/fs is painful as heck.

	-Mike

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Andrew Morton]

On Fri, 27 Jul 2007 01:47:49 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:

> More sophisticated testing is needed - there's something in
> ext3-tools which will mmap, page in and hold a file for you.

So much for that theory.  afaict mmapped, active pagecache is immune to
updatedb activity.  It just sits there while updatedb continues munching
away at the slab and blockdev pagecache which it instantiated.  I assume
we're never getting the VM into enough trouble to tip it over the
start-reclaiming-mapped-pages threshold (ie: /proc/sys/vm/swappiness).

Start the updatedb on this 128MB machine with 80MB of mapped pagecache, it
falls to 55MB fairly soon and then never changes.

So hrm.  Are we sure that updatedb is the problem?  There are quite a few
heavyweight things which happen in the wee small hours.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Mike Galbraith]

On Fri, 2007-07-27 at 03:00 -0700, Andrew Morton wrote:
> On Fri, 27 Jul 2007 01:47:49 -0700 Andrew Morton <akpm@linux-foundation.org> wrote:
> 
> > More sophisticated testing is needed - there's something in
> > ext3-tools which will mmap, page in and hold a file for you.
> 
> So much for that theory.  afaict mmapped, active pagecache is immune to
> updatedb activity.  It just sits there while updatedb continues munching
> away at the slab and blockdev pagecache which it instantiated.  I assume
> we're never getting the VM into enough trouble to tip it over the
> start-reclaiming-mapped-pages threshold (ie: /proc/sys/vm/swappiness).
> 
> Start the updatedb on this 128MB machine with 80MB of mapped pagecache, it
> falls to 55MB fairly soon and then never changes.
> 
> So hrm.  Are we sure that updatedb is the problem?  There are quite a few
> heavyweight things which happen in the wee small hours.

The balance in _my_ world seems just fine.  I don't let any of those
system maintenance things run while I'm using the system, and it doesn't
bother me if my working set has to be reconstructed after heavy-weight
maintenance things are allowed to run.  I'm not seeing anything I
wouldn't expect to see when running a job the size of updatedb.

	-Mike

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Daniel Hazelton]

On Friday 27 July 2007 06:25:18 Mike Galbraith wrote:
> On Fri, 2007-07-27 at 03:00 -0700, Andrew Morton wrote:
> > On Fri, 27 Jul 2007 01:47:49 -0700 Andrew Morton 
<akpm@linux-foundation.org> wrote:
> > > More sophisticated testing is needed - there's something in
> > > ext3-tools which will mmap, page in and hold a file for you.
> >
> > So much for that theory.  afaict mmapped, active pagecache is immune to
> > updatedb activity.  It just sits there while updatedb continues munching
> > away at the slab and blockdev pagecache which it instantiated.  I assume
> > we're never getting the VM into enough trouble to tip it over the
> > start-reclaiming-mapped-pages threshold (ie: /proc/sys/vm/swappiness).
> >
> > Start the updatedb on this 128MB machine with 80MB of mapped pagecache,
> > it falls to 55MB fairly soon and then never changes.
> >
> > So hrm.  Are we sure that updatedb is the problem?  There are quite a few
> > heavyweight things which happen in the wee small hours.
>
> The balance in _my_ world seems just fine.  I don't let any of those
> system maintenance things run while I'm using the system, and it doesn't
> bother me if my working set has to be reconstructed after heavy-weight
> maintenance things are allowed to run.  I'm not seeing anything I
> wouldn't expect to see when running a job the size of updatedb.
>
> 	-Mike

Do you realize you've totally missed the point?

It isn't about what is fine in the Kernel Developers world, but what is fine 
in the *USERS* world. 

There are dozens of big businesses pushing Linux for Enterprise performance. 
Rather than discussing the merit of those patches - some of which just 
improve the performance of a specific application by 1 or 2  percent - they 
get a nod and go into the kernel. But when a group of users that don't 
represent one of those businesses says "Hey, this helps with problems I see 
on my system" there is a big discussion and ultimately those patches get 
rejected. Why? Because they'll give an example using a program that they see 
causing part of the problem and be told "Use program X - it does things 
differently and shouldn't cause the problem" or "But what causes the problem 
to happen? The patch treats a symptom of a larger problem".

The fucked up part of that is that the (mass of) kernel developers will see a 
similar report saying "mySQL has a performance problem because of X, this 
fixes it" and not blink twice - even if it is "treating the symptom and not 
the cause". It's this attitude more than anything that caused Con 
to "retire" - at least that is the impression I got from the interviews he's 
given. (The exact impression was "I'm sick of the kernel developers doing 
everything they can to help enterprise users and ignoring the home users")

So...
The problem:
Updatedb or another process that uses the FS heavily runs on a users 256MB 
P3-800 (when it is idle) and the VFS caches grow, causing memory pressure 
that causes other applications to be swapped to disk. In the morning the user 
has to wait for the system to swap those applications back in.

Questions about it:
Q) Does swap-prefetch help with this? 
A) [From all reports I've seen (*)] Yes, it does. 

Q) Why does it help? 
A) Because it pro-actively swaps stuff back-in when the memory pressure that 
caused it to be swapped out is gone. 

Q) What causes the problem? 
A) The VFS layer not keeping a limited cache. Instead the VFS will chew 
through available memory in the name of "increasing performance".

Solution(s) to the problem:
1) Limit the amount of memory the pagecache and other VFS caches can consume
2) Implement swap prefetch

If I had a (more) complete understanding of how the VFS cache(s) work I'd try 
to code a patch to do #1 myself. Patches to do #2 already exist and have been 
shown to work for the users that have tried it. My question is thus, simply: 
What is the reason that it is argued against?(**)

DRH
PS: Yes, I realize I've repeated some people from earlier in this thread, but 
it seems that people have been forgetting the point.

(*) I've followed this thread and all of its splinters. The reports that are 
in them, where the person making the report has a system that has the limited 
memory needed for the problem to exhibit itself, all show that swap-prefetch 
helps.

(**) No, I won't settle for "Its treating a symptom". The fact is that this 
isn't a *SYMPTOM* of anything. It treats the cause of the lag the users that 
have less than (for the sake of argument) 1G of memory are seeing. And no, 
changing userspace isn't a solution - updatedb may be the program that has 
been used as an example, but there are others. The proper fix is to change 
the kernel to either make the situation impossible (limit the VFS and other 
kernel caches) or make the situation as painless as possible (implement swap 
prefetch to alleviate the lag of swapping data back in).

-- 
Dialup is like pissing through a pipette. Slow and excruciatingly painful.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Rene Herman]

On 07/27/2007 07:45 PM, Daniel Hazelton wrote:

> Updatedb or another process that uses the FS heavily runs on a users
> 256MB P3-800 (when it is idle) and the VFS caches grow, causing memory
> pressure that causes other applications to be swapped to disk. In the
> morning the user has to wait for the system to swap those applications
> back in.
> 
> Questions about it:
> Q) Does swap-prefetch help with this? 
> A) [From all reports I've seen (*)] Yes, it does. 

No it does not. If updatedb filled memory to the point of causing swapping 
(which noone is reproducing anyway) it HAS FILLED MEMORY and swap-prefetch 
hasn't any memory to prefetch into -- updatedb itself doesn't use any 
significant memory.

Here's swap-prefetch's author saying the same:

http://lkml.org/lkml/2007/2/9/112

| It can't help the updatedb scenario. Updatedb leaves the ram full and
| swap prefetch wants to cost as little as possible so it will never
| move anything out of ram in preference for the pages it wants to swap
| back in.

Now please finally either understand this, or tell us how we're wrong.

Rene.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[david]

On Fri, 27 Jul 2007, Rene Herman wrote:

> On 07/27/2007 07:45 PM, Daniel Hazelton wrote:
>
>>  Updatedb or another process that uses the FS heavily runs on a users
>>  256MB P3-800 (when it is idle) and the VFS caches grow, causing memory
>>  pressure that causes other applications to be swapped to disk. In the
>>  morning the user has to wait for the system to swap those applications
>>  back in.
>>
>>  Questions about it:
>>  Q) Does swap-prefetch help with this? A) [From all reports I've seen (*)]
>>  Yes, it does. 
>
> No it does not. If updatedb filled memory to the point of causing swapping 
> (which noone is reproducing anyway) it HAS FILLED MEMORY and swap-prefetch 
> hasn't any memory to prefetch into -- updatedb itself doesn't use any 
> significant memory.

however there are other programs which are known to take up significant 
amounts of memory and will cause the issue being described (openoffice for 
example)

please don't get hung up on the text 'updatedb' and accept that there are 
programs that do run intermittently and do use a significant amount of ram 
and then free it.

David Lang

> Here's swap-prefetch's author saying the same:
>
> http://lkml.org/lkml/2007/2/9/112
>
> |  It can't help the updatedb scenario. Updatedb leaves the ram full and
> |  swap prefetch wants to cost as little as possible so it will never
> |  move anything out of ram in preference for the pages it wants to swap
> |  back in.
>
> Now please finally either understand this, or tell us how we're wrong.
>
> Rene.
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Rene Herman]

On 07/27/2007 09:43 PM, david@lang.hm wrote:

> On Fri, 27 Jul 2007, Rene Herman wrote:
> 
>> On 07/27/2007 07:45 PM, Daniel Hazelton wrote:
>>
>>>  Questions about it:
>>>  Q) Does swap-prefetch help with this?
>>>  A) [From all reports I've seen (*)]
>>>  Yes, it does. 
>>
>> No it does not. If updatedb filled memory to the point of causing 
>> swapping (which noone is reproducing anyway) it HAS FILLED MEMORY and 
>> swap-prefetch hasn't any memory to prefetch into -- updatedb itself 
>> doesn't use any significant memory.
> 
> however there are other programs which are known to take up significant 
> amounts of memory and will cause the issue being described (openoffice 
> for example)
> 
> please don't get hung up on the text 'updatedb' and accept that there 
> are programs that do run intermittently and do use a significant amount 
> of ram and then free it.

Different issue. One that's worth pursueing perhaps, but a different issue 
from the VFS caches issue that people have been trying to track down.

Rene.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[david]

On Sat, 28 Jul 2007, Rene Herman wrote:

> On 07/27/2007 09:43 PM, david@lang.hm wrote:
>
>>  On Fri, 27 Jul 2007, Rene Herman wrote:
>> 
>> >  On 07/27/2007 07:45 PM, Daniel Hazelton wrote:
>> > 
>> > >   Questions about it:
>> > >   Q) Does swap-prefetch help with this?
>> > >   A) [From all reports I've seen (*)]
>> > >   Yes, it does. 
>> > 
>> >  No it does not. If updatedb filled memory to the point of causing 
>> >  swapping (which noone is reproducing anyway) it HAS FILLED MEMORY and 
>> >  swap-prefetch hasn't any memory to prefetch into -- updatedb itself 
>> >  doesn't use any significant memory.
>>
>>  however there are other programs which are known to take up significant
>>  amounts of memory and will cause the issue being described (openoffice for
>>  example)
>>
>>  please don't get hung up on the text 'updatedb' and accept that there are
>>  programs that do run intermittently and do use a significant amount of ram
>>  and then free it.
>
> Different issue. One that's worth pursueing perhaps, but a different issue 
> from the VFS caches issue that people have been trying to track down.

people are trying to track down the problem of their machine being slow 
until enough data is swapped back in to operate normally.

in at some situations swap prefetch can help becouse something that used 
memory freed it so there is free memory that could be filled with data 
(which is something that Linux does agressivly in most other situations)

in some other situations swap prefetch cannot help becouse useless data is 
getting cached at the expense of useful data.

nobody is arguing that swap prefetch helps in the second cast.

what people are arguing is that there are situations where it helps for 
the first case. on some machines and version of updatedb the nighly run of 
updatedb can cause both sets of problems. but the nightly updatedb run is 
not the only thing that can cause problems


but let's talk about the concept here for a little bit

the design is to use CPU and I/O capacity that's otherwise idle to fill 
free memory with data from swap.

pro:
   more ram has potentially useful data in it

con:
   it takes a little extra effort to give this memory to another app (the 
page must be removed from the list and zeroed at the time it's needed, I 
assume that the data is left in swap so that it doesn't have to be written 
out again)

   it adds some complexity to the kernel (~500 lines IIRC from this thread)

   by undoing recent swapouts it can potentially mask problems with swapout

it looks to me like unless the code was really bad (and after 23 months in 
-mm it doesn't sound like it is) that the only significant con left is the 
potential to mask other problems.

however there are many legitimate cases where it is definantly dong the 
right thing (swapout was correct in pushing out the pages, but now the 
cause of that preasure is gone). the amount of benifit from this will vary 
from situation to situation, but it's not reasonable to claim that this 
provides no benifit (you have benchmark numbers that show it in synthetic 
benchmarks, and you have user reports that show it in the real-worlk)

there are lots of things in the kernel who's job is to pre-fill the memroy 
with data that may (or may not) be useful in the future. this is just 
another method of filling the cache. it does so my saying "the user wanted 
these pages in the recent past, so it's a reasonable guess to say that the 
user will want them again in the future"

David Lang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Rene Herman]

On 07/28/2007 10:55 AM, david@lang.hm wrote:

> in at some situations swap prefetch can help becouse something that used 
> memory freed it so there is free memory that could be filled with data 
> (which is something that Linux does agressivly in most other situations)
> 
> in some other situations swap prefetch cannot help becouse useless data 
> is getting cached at the expense of useful data.
> 
> nobody is arguing that swap prefetch helps in the second cast.

Oh yes they are. Daniel for example did twice, telling me to turn my brain 
on in between (if you read it, you may have noticed I got a little annoyed 
at that point).

> but let's talk about the concept here for a little bit
> 
> the design is to use CPU and I/O capacity that's otherwise idle to fill 
> free memory with data from swap.
> 
> pro:
>   more ram has potentially useful data in it
> 
> con:
>   it takes a little extra effort to give this memory to another app (the 
> page must be removed from the list and zeroed at the time it's needed, I 
> assume that the data is left in swap so that it doesn't have to be 
> written out again)

It is. Prefetched pages can be dropped on the floor without additional I/O.

>   it adds some complexity to the kernel (~500 lines IIRC from this thread)
> 
>   by undoing recent swapouts it can potentially mask problems with swapout
> 
> it looks to me like unless the code was really bad (and after 23 months 
> in -mm it doesn't sound like it is)

Not to sound pretentious or anything but I assume that Andrew has a fairly 
good overview of exactly how broken -mm can be at times. How many -mm users 
use it anyway? He himself said he's not convinced of usefulness having not 
seen it help for him (and notice that most developers are also users), 
turned it off due to it annoying him at some point and hasn't seen a serious 
investigation into potential downsides.

> that the only significant con left is the potential to mask other
> problems.

Which is not a madeup issue, mind you. As an example, I just now tried GNU 
locate and saw it's a complete pig and specifically unsuitable for the low 
memory boxes under discussion. Upon completion, it actually frees enough 
memory that swap-prefetch _could_ help on some boxes, while the real issue 
is that they should first and foremost dump GNU locate.

> however there are many legitimate cases where it is definantly dong the 
> right thing (swapout was correct in pushing out the pages, but now the 
> cause of that preasure is gone). the amount of benifit from this will 
> vary from situation to situation, but it's not reasonable to claim that 
> this provides no benifit (you have benchmark numbers that show it in 
> synthetic benchmarks, and you have user reports that show it in the 
> real-worlk)

I certainly would not want to argue anything of the sort no. As said a few 
times, I agree that swap-prefetch makes sense and has at least the potential 
to help some situations that you really wouldnt even want to try and fix any 
other way, simply because nothing's broken.

> there are lots of things in the kernel who's job is to pre-fill the 
> memroy with data that may (or may not) be useful in the future. this is 
> just another method of filling the cache. it does so my saying "the user
> wanted these pages in the recent past, so it's a reasonable guess to say
> that the user will want them again in the future"

Well, _that_ is what the kernel is already going to great lengths at doing, 
and it decided that those pages us poor overnight OO.o users want in in the 
morning weren't reasonable guesses. The kernel also won't any time soon be 
reading our minds, so any solution would need either user intervention (we 
could devise a way to tell the kernel "hey ho, I consider these pages to be 
very important -- try not to swap them out" possible even with a "and if you 
do, please pull them back in when possible") or we can let swap-prefetch do 
the "just in case" thing it is doing.

While swap-prefetch may not be the be all end all of solutions I agree that 
having a machine sit around with free memory and applications in swap seems 
not too useful if (as is the case) fetched pages can be dropped immediately 
when it turns out swap-prefetch made the wrong decision.

So that's for the concept. As to implementation, if I try and look at the 
code, it seems to be trying hard to really be free and as such, potential 
downsides seem limited. It's a rather core concept though and as such needs 
someone with a _lot_ more VM clue to ack. Sorry for not knowing, but who's 
maintaining/submitting the thing now that Con's not? He or she should 
preferably address any concerns it seems.

Rene.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Alan Cox]

> It is. Prefetched pages can be dropped on the floor without additional I/O.

Which is essentially free for most cases. In addition your disk access
may well have been in idle time (and should be for this sort of stuff)
and if it was in the same chunk as something nearby was effectively free
anyway. 

Actual physical disk ops are precious resource and anything that mostly
reduces the number will be a win - not to stay swap prefetch is the right
answer but accidentally or otherwise there are good reasons it may happen
to help.

Bigger more linear chunks of writeout/readin is much more important I
suspect than swap prefetching. 

> good overview of exactly how broken -mm can be at times. How many -mm users 
> use it anyway? He himself said he's not convinced of usefulness having not 

I've been using it for months with no noticed problem. I turn it on
because it might as well get tested. I've not done comparison tests so I
can't comment on if its worth it.

Lots of -mm testers turn *everything* on because its a test kernel.


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Ray Lee]

On 7/28/07, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> Actual physical disk ops are precious resource and anything that mostly
> reduces the number will be a win - not to stay swap prefetch is the right
> answer but accidentally or otherwise there are good reasons it may happen
> to help.
>
> Bigger more linear chunks of writeout/readin is much more important I
> suspect than swap prefetching.

<nod>. The larger the chunks are that we swap out, the less it
actually hurts to swap, which might make all this a moot point. Not
all I/O is created equal...

Ray

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[david]

On Sat, 28 Jul 2007, Alan Cox wrote:

>> It is. Prefetched pages can be dropped on the floor without additional I/O.
>
> Which is essentially free for most cases. In addition your disk access
> may well have been in idle time (and should be for this sort of stuff)
> and if it was in the same chunk as something nearby was effectively free
> anyway.

as I understand it the swap-prefetch only kicks in if the device is idle

> Actual physical disk ops are precious resource and anything that mostly
> reduces the number will be a win - not to stay swap prefetch is the right
> answer but accidentally or otherwise there are good reasons it may happen
> to help.
>
> Bigger more linear chunks of writeout/readin is much more important I
> suspect than swap prefetching.

I'm sure this is true while you are doing the swapout or swapin and the 
system is waiting for it. but with prefetch you may be able to avoid doing 
the swapin at a time when the system is waiting for it by doing it at a 
time when the system is otherwise idle.

David Lang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Rene Herman]

On 07/28/2007 01:21 PM, Alan Cox wrote:

>> It is. Prefetched pages can be dropped on the floor without additional
>> I/O.
> 
> Which is essentially free for most cases. In addition your disk access 
> may well have been in idle time (and should be for this sort of stuff)

Yes. The swap-prefetch patch ensures that the machine (well, the VM) is very 
idle before it allows itself to kick in.

> and if it was in the same chunk as something nearby was effectively free 
> anyway.
> 
> Actual physical disk ops are precious resource and anything that mostly 
> reduces the number will be a win - not to stay swap prefetch is the right
> answer but accidentally or otherwise there are good reasons it may
> happen to help.
> 
> Bigger more linear chunks of writeout/readin is much more important I 
> suspect than swap prefetching.

Yes, I believe this might be an important point. Earlier I posted a dumb 
little VM thrasher:

http://lkml.org/lkml/2007/7/25/85

Contrived thing and all, but what it does do is show exactly how bad seeking 
all over swap-space is. If you push it out before hitting enter, the time it 
takes easily grows past 10 minutes (with my 768M) versus sub-second (!) when 
it's all in to start with.

What are the tradeoffs here? What wants small chunks? Also, as far as I'm 
aware Linux does not do things like up the granularity when it notices it's 
swapping in heavily? That sounds sort of promising...

>> good overview of exactly how broken -mm can be at times. How many -mm users 
>> use it anyway? He himself said he's not convinced of usefulness having not 
> 
> I've been using it for months with no noticed problem. I turn it on
> because it might as well get tested. I've not done comparison tests so I
> can't comment on if its worth it.
> 
> Lots of -mm testers turn *everything* on because its a test kernel.

Okay.

Rene.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Alan Cox]

> Contrived thing and all, but what it does do is show exactly how bad seeking 
> all over swap-space is. If you push it out before hitting enter, the time it 
> takes easily grows past 10 minutes (with my 768M) versus sub-second (!) when 
> it's all in to start with.

Think in "operations/second" and you get a better view of the disk.

> What are the tradeoffs here? What wants small chunks? Also, as far as I'm 
> aware Linux does not do things like up the granularity when it notices it's 
> swapping in heavily? That sounds sort of promising...

Small chunks means you get better efficiency of memory use - large chunks
mean you may well page in a lot more than you needed to each time (and
cause more paging in turn). Your disk would prefer you fed it big linear
I/O's - 512KB would probably be my first guess at tuning a large box
under load for paging chunk size.

More radically if anyone wants to do real researchy type work - how about
log structured swap with a cleaner  ?

Alan

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Rene Herman]

On 07/29/2007 03:12 PM, Alan Cox wrote:

>> What are the tradeoffs here? What wants small chunks? Also, as far as
>> I'm aware Linux does not do things like up the granularity when it
>> notices it's swapping in heavily? That sounds sort of promising...
> 
> Small chunks means you get better efficiency of memory use - large chunks
> mean you may well page in a lot more than you needed to each time (and 
> cause more paging in turn). Your disk would prefer you fed it big linear
> I/O's - 512KB would probably be my first guess at tuning a large box 
> under load for paging chunk size.

That probably kills my momentary hope that I was looking at yet another good 
use of large soft-pages seeing as how 512K would be going overboard a bit 
right? :-/

> More radically if anyone wants to do real researchy type work - how about
> log structured swap with a cleaner  ?

Right over my head. Why does log-structure help anything?

Rene.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Ray Lee]

On 7/29/07, Rene Herman <rene.herman@gmail.com> wrote:
> On 07/29/2007 03:12 PM, Alan Cox wrote:
> > More radically if anyone wants to do real researchy type work - how about
> > log structured swap with a cleaner  ?
>
> Right over my head. Why does log-structure help anything?

Log structured disk layouts allow for better placement of writeout, so
that you cn eliminate most or all seeks. Seeks are the enemy when
trying to get full disk bandwidth.

google on log structured disk layout, or somesuch, for details.

Ray

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Rene Herman]

On 07/29/2007 04:58 PM, Ray Lee wrote:

> On 7/29/07, Rene Herman <rene.herman@gmail.com> wrote:
>> On 07/29/2007 03:12 PM, Alan Cox wrote:

>>> More radically if anyone wants to do real researchy type work - how about
>>> log structured swap with a cleaner  ?

>> Right over my head. Why does log-structure help anything?
> 
> Log structured disk layouts allow for better placement of writeout, so
> that you cn eliminate most or all seeks. Seeks are the enemy when
> trying to get full disk bandwidth.
> 
> google on log structured disk layout, or somesuch, for details.

I understand what log structure is generally, but how does it help swapin?

Rene.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Ray Lee]

On 7/29/07, Rene Herman <rene.herman@gmail.com> wrote:
> On 07/29/2007 04:58 PM, Ray Lee wrote:
> > On 7/29/07, Rene Herman <rene.herman@gmail.com> wrote:
> >> Right over my head. Why does log-structure help anything?
> >
> > Log structured disk layouts allow for better placement of writeout, so
> > that you cn eliminate most or all seeks. Seeks are the enemy when
> > trying to get full disk bandwidth.
> >
> > google on log structured disk layout, or somesuch, for details.
>
> I understand what log structure is generally, but how does it help swapin?

Look at the swap out case first.

Right now, when swapping out the kernel places whatever it can
wherever it can inside the swap space. The closer you are to filling
your swap space, the more likely that those swapped out blocks will be
all over place, rather than in one nice chunk. Contrast that with a
log structured scheme, where the writeout happens to sequential spaces
on the drive instead of scattered about.

So, at some point when the system needs to fault those blocks that
back in, it now has a linear span of sectors to read instead of asking
the drive to bounce over twenty tracks for a hundred blocks.

So, it eliminates the seeks. My laptop drive can read (huh, how odd,
it got slower, need to retest in single user mode), hmm, let's go with
about 25 MB/s. If we ask for a single block from each track, though,
that'll drop to 4k * (1 second / seek time) which is about a megabyte
a second if we're lucky enough to read from consecutive tracks. Even
worse if it's not.

Seeks are the enemy any time you need to hit the drive for anything,
be it swapping or optimizing a database.

Ray

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Rene Herman]

On 07/29/2007 05:20 PM, Ray Lee wrote:

>> I understand what log structure is generally, but how does it help swapin?
> 
> Look at the swap out case first.
> 
> Right now, when swapping out the kernel places whatever it can
> wherever it can inside the swap space. The closer you are to filling
> your swap space, the more likely that those swapped out blocks will be
> all over place, rather than in one nice chunk. Contrast that with a
> log structured scheme, where the writeout happens to sequential spaces
> on the drive instead of scattered about.

This seems to be now fixing the different problem of swap-space filling up. 
I'm quite willing to for now assume I've got plenty free.

> So, at some point when the system needs to fault those blocks that
> back in, it now has a linear span of sectors to read instead of asking
> the drive to bounce over twenty tracks for a hundred blocks.

Moreover though -- what I know about log structure is that generally it 
optimises for write (swapout) and might make read (swapin) worse due to 
fragmentation that wouldn't happen with a regular fs structure.

I guess that cleaner that Alan mentioned might be involved there -- I don't 
know how/what it would be doing.

> So, it eliminates the seeks. My laptop drive can read (huh, how odd,
> it got slower, need to retest in single user mode), hmm, let's go with
> about 25 MB/s. If we ask for a single block from each track, though,
> that'll drop to 4k * (1 second / seek time) which is about a megabyte
> a second if we're lucky enough to read from consecutive tracks. Even
> worse if it's not.
> 
> Seeks are the enemy any time you need to hit the drive for anything,
> be it swapping or optimizing a database.

I am very aware of the costs of seeks (on current magnetic media).

Rene.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Ray Lee]

On 7/29/07, Rene Herman <rene.herman@gmail.com> wrote:
> On 07/29/2007 05:20 PM, Ray Lee wrote:
> This seems to be now fixing the different problem of swap-space filling up.
> I'm quite willing to for now assume I've got plenty free.

I was trying to point out that currently, as an example, memory that
is linear in a process' space could be fragmented on disk when swapped
out. That's today.

Under a log-structured scheme, one could set it up such that something
that was linear in RAM could be swapped out linearly on the drive,
minimizing seeks on writeout, which will naturally minimize seeks on
swap in of that same data.

> > So, at some point when the system needs to fault those blocks that
> > back in, it now has a linear span of sectors to read instead of asking
> > the drive to bounce over twenty tracks for a hundred blocks.
>
> Moreover though -- what I know about log structure is that generally it
> optimises for write (swapout) and might make read (swapin) worse due to
> fragmentation that wouldn't happen with a regular fs structure.

It looks like I'm not doing a very good job of explaining this, I'm afraid.

Suffice it to say that a log structured swap would give optimization
options that we don't have today.

> I guess that cleaner that Alan mentioned might be involved there -- I don't
> know how/what it would be doing.

Then you should google on `log structured filesystem (primer OR
introduction)` and read a few of the links that pop up. You might find
it interesting.

> I am very aware of the costs of seeks (on current magnetic media).

Then perhaps you can just take it on faith -- log structured layouts
are designed to help minimize seeks, read and write.

Ray

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Rene Herman]

On 07/29/2007 06:04 PM, Ray Lee wrote:

>> I am very aware of the costs of seeks (on current magnetic media).
> 
> Then perhaps you can just take it on faith -- log structured layouts
> are designed to help minimize seeks, read and write.

I am particularly bad at faith. Let's take that stupid program that I posted:

http://lkml.org/lkml/2007/7/25/85

You push it out before you hit enter, it's written out to swap, at whatever 
speed. How should it be layed out so that it's swapped in most efficiently 
after hitting enter? Reading bigger chunks would quite obviously help, but 
the layout?

The program is not a real-world issue and if you do not consider it a useful 
boundary condition either (okay I guess), how would log structured swap help 
if I just assume I have plenty of free swap to begin with?

Rene.


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Ray Lee]

On 7/29/07, Rene Herman <rene.herman@gmail.com> wrote:
> On 07/29/2007 06:04 PM, Ray Lee wrote:
> >> I am very aware of the costs of seeks (on current magnetic media).
> >
> > Then perhaps you can just take it on faith -- log structured layouts
> > are designed to help minimize seeks, read and write.
>
> I am particularly bad at faith. Let's take that stupid program that I posted:

You only think you are :-). I'm sure there are lots of things you have
faith in. Gravity, for example :-).

> The program is not a real-world issue and if you do not consider it a useful
> boundary condition either (okay I guess), how would log structured swap help
> if I just assume I have plenty of free swap to begin with?

Is that generally the case on your systems? Every linux system I've
run, regardless of RAM, has always pushed things out to swap. And once
there's something already in swap, you now have a packing problem when
you want to swap something else out.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Rene Herman]

On 07/29/2007 07:19 PM, Ray Lee wrote:

>> The program is not a real-world issue and if you do not consider it a useful
>> boundary condition either (okay I guess), how would log structured swap help
>> if I just assume I have plenty of free swap to begin with?
> 
> Is that generally the case on your systems? Every linux system I've
> run, regardless of RAM, has always pushed things out to swap.

For me, it is generally the case yes. We are still discussing this in the 
context of desktop machines and their problems with being slow as things 
have been swapped out and generally I expect a desktop to have plenty of 
swap which it's not regularly going to fillup significantly since then the 
machine's unworkably slow as a desktop anyway.

> And once there's something already in swap, you now have a packing
> problem when you want to swap something else out.

Once we're crammed, it gets to be a different situation yes. As far as I'm 
concerned that's for another thread though. I'm spending too much time on 
LKML as it is...

Rene.



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Ray Lee]

On 7/29/07, Rene Herman <rene.herman@gmail.com> wrote:
> On 07/29/2007 07:19 PM, Ray Lee wrote:
> For me, it is generally the case yes. We are still discussing this in the
> context of desktop machines and their problems with being slow as things
> have been swapped out and generally I expect a desktop to have plenty of
> swap which it's not regularly going to fillup significantly since then the
> machine's unworkably slow as a desktop anyway.

<Shrug> Well, that doesn't match my systems. My laptop has 400MB in swap:

ray@phoenix:~$ free
             total       used       free     shared    buffers     cached
Mem:        894208     883920      10288          0       3044     163224
-/+ buffers/cache:     717652     176556
Swap:      1116476     393132     723344

> > And once there's something already in swap, you now have a packing
> > problem when you want to swap something else out.
>
> Once we're crammed, it gets to be a different situation yes. As far as I'm
> concerned that's for another thread though. I'm spending too much time on
> LKML as it is...

No, it's not even when crammed. It's just when there are holes.
mm/swapfile.c does try to cluster things, but doesn't work too hard at
it as we don't want to spend all our time looking for a perfect fit
that may not exist.

Ray

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Rene Herman]

On 07/29/2007 07:52 PM, Ray Lee wrote:

> <Shrug> Well, that doesn't match my systems. My laptop has 400MB in swap:

Which in your case is slightly more than 1/3 of available swap space. Quite 
a lot for a desktop indeed. And if it's more than a few percent fragmented, 
please fix current swapout instead of log structuring it.

Rene.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Alan Cox]

> > Is that generally the case on your systems? Every linux system I've
> > run, regardless of RAM, has always pushed things out to swap.
> 
> For me, it is generally the case yes. We are still discussing this in the 
> context of desktop machines and their problems with being slow as things 
> have been swapped out and generally I expect a desktop to have plenty of 
> swap which it's not regularly going to fillup significantly since then the 
> machine's unworkably slow as a desktop anyway.

A simple log optimises writeout (which is latency critical) and can
otherwise stall an enitre system. In a log you can also have multiple
copies of the same page on disk easily, some stale - so you can write out
chunks of data that are not all them removed from memory, just so you get
them back more easily if you then do (and I guess you'd mark them
accordingly)

The second element is a cleaner - something to go around removing stuff
from the log that is needed when the disks are idle - and also to repack
data in nice linear chunks. So instead of using the empty disk time for
page-in you use it for packing data and optimising future paging.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Paul Jackson]

Ray wrote:
> a log structured scheme, where the writeout happens to sequential spaces
> on the drive instead of scattered about.

If the problem is reading stuff back in from swap quickly when
needed, then this likely helps, by reducing the seeks needed.

If the problem is reading stuff back in from swap at the *same time*
that the application is reading stuff from some user file system, and if
that user file system is on the same drive as the swap partition
(typical on laptops), then interleaving the user file system accesses
with the swap partition accesses might overwhelm all other performance
problems, due to the frequent long seeks between the two.

In that case, swap layout and swap i/o block size are secondary.
However, pre-fetching, so that swap read back is not interleaved
with application file accesses, could help dramatically.

===

Perhaps we could have a 'wake-up' command, analogous to the various sleep
and hibernate commands.  The 'wake-up' command could do whatever of the
following it knew to do, in order to optimize for an anticipated change in
usage patterns:
 1) pre-fetch swap
 2) clean (write out) dirty pages
 3) maximize free memory
 4) layout swap nicely
 5) pre-fetch a favorite set of apps

Stumble out of bed in the morning, press 'wake-up', start boiling the
water for your coffee, and in another ten minutes, one is ready to rock
and roll.

In case Andrew is so bored he read this far -- yes this wake-up sounds
like user space code, with minimal kernel changes to support any
particular lower level operation that we can't do already.

-- 
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <pj@sgi.com> 1.925.600.0401

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Ray Lee]

On 7/29/07, Paul Jackson <pj@sgi.com> wrote:
> If the problem is reading stuff back in from swap at the *same time*
> that the application is reading stuff from some user file system, and if
> that user file system is on the same drive as the swap partition
> (typical on laptops), then interleaving the user file system accesses
> with the swap partition accesses might overwhelm all other performance
> problems, due to the frequent long seeks between the two.

Ah, so in a normal scenario where a working-set is getting faulted
back in, we have the swap storage as well as the file-backed stuff
that needs to be read as well. So even if swap is organized perfectly,
we're still seeking. Damn.

On the other hand, that explains another thing that swap prefetch
could be helping with -- if it preemptively faults the swap back in,
then the file-backed stuff can be faulted back more quickly, just by
the virtue of not needing to seek back and forth to swap for its
stuff. Hadn't thought of that.

That also implies that people running with swap files rather than swap
partitions will see less of an issue. I should dig out my old compact
flash card and try putting swap on that for a week.

> In that case, swap layout and swap i/o block size are secondary.
> However, pre-fetching, so that swap read back is not interleaved
> with application file accesses, could help dramatically.

<Nod>

> Perhaps we could have a 'wake-up' command, analogous to the various sleep
> and hibernate commands.
[...]
> In case Andrew is so bored he read this far -- yes this wake-up sounds
> like user space code, with minimal kernel changes to support any
> particular lower level operation that we can't do already.

He'd suggested using, uhm, ptrace_peek or somesuch for just such a
purpose. The second half of the issue is to know when and what to
target.

Ray

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Paul Jackson]

Ray wrote:
> Ah, so in a normal scenario where a working-set is getting faulted
> back in, we have the swap storage as well as the file-backed stuff
> that needs to be read as well. So even if swap is organized perfectly,
> we're still seeking. Damn.

Perhaps this applies in some cases ... perhaps.

-- 
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <pj@sgi.com> 1.925.600.0401

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Ray Lee]

On 7/29/07, Paul Jackson <pj@sgi.com> wrote:
> Ray wrote:
> > Ah, so in a normal scenario where a working-set is getting faulted
> > back in, we have the swap storage as well as the file-backed stuff
> > that needs to be read as well. So even if swap is organized perfectly,
> > we're still seeking. Damn.
>
> Perhaps this applies in some cases ... perhaps.

Yeah, point taken: better data would make this a lot easier to figure
out and target fixes.

Ray

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Daniel Hazelton]

On Sunday 29 July 2007 16:00:22 Ray Lee wrote:
> On 7/29/07, Paul Jackson <pj@sgi.com> wrote:
> > If the problem is reading stuff back in from swap at the *same time*
> > that the application is reading stuff from some user file system, and if
> > that user file system is on the same drive as the swap partition
> > (typical on laptops), then interleaving the user file system accesses
> > with the swap partition accesses might overwhelm all other performance
> > problems, due to the frequent long seeks between the two.
>
> Ah, so in a normal scenario where a working-set is getting faulted
> back in, we have the swap storage as well as the file-backed stuff
> that needs to be read as well. So even if swap is organized perfectly,
> we're still seeking. Damn.

That is one reason why I try to have swap on a device dedicated just for it. 
It helps keep the system from having to seek all over the drive for data. (I 
remember that this was recommended years ago with Windows - back when you 
could tell Windows where to put the swap file)

> On the other hand, that explains another thing that swap prefetch
> could be helping with -- if it preemptively faults the swap back in,
> then the file-backed stuff can be faulted back more quickly, just by
> the virtue of not needing to seek back and forth to swap for its
> stuff. Hadn't thought of that.

For it to really help swap-prefetch would have to be more aggressive. At the 
moment (if I'm reading the code correctly) the system has to have close to 
zero for it to kick in. A tunable knob controlling how much activity is too 
much for the prefetch to kick in would help with finding a sane default. IMHO 
it should be the one that provides the most benefit with the least hit to 
performance.

> That also implies that people running with swap files rather than swap
> partitions will see less of an issue. I should dig out my old compact
> flash card and try putting swap on that for a week.

Maybe. It all depends on how much seeking is needed to track down the pages in 
the swapfile and such. What would really help make the situation even better 
would be doing the log structured swap + cleaner. The log structured swap + 
cleaner should provide a performance boost by itself - add in the prefetch 
mechanism and the benefits are even more visible.

Another way to improve performance would require making the page replacement 
mechanism more intelligent. There are bounds to what can be done in the 
kernel without negatively impacting performance, but, if I've read the code 
correctly, there might be a better way to decide which pages to evict. One 
way to do this would be to implement some mechanism that allows the system to 
choose a single group of contiguous pages (or, say, a large soft-page) over 
swapping out a single page at a time.

(some form of memory defrag would also be nice, but I can't think of a way to 
do that without massively breaking everything)

<snip>
> > In case Andrew is so bored he read this far -- yes this wake-up sounds
> > like user space code, with minimal kernel changes to support any
> > particular lower level operation that we can't do already.
>
> He'd suggested using, uhm, ptrace_peek or somesuch for just such a
> purpose. The second half of the issue is to know when and what to
> target.

The userspace suggestion that was thrown out earlier would have been as 
error-prone and problematic as FUSE. A solution like you suggest would be 
workable - its small and does a task that is best done in userspace (IMHO). 
(IIRC, the original suggestion involved merging maps2 and another patchset 
into mainline and using that, combined with PEEKTEXT to provide for a 
userspace swap daemon. Swap, IMHO, should never be handled outside the 
kernel)

What might be useful is a userspace daemon that tracks memory pressure and 
uses a concise API to trigger various levels of prefetch and/or swap 
aggressiveness.

DRH

-- 
Dialup is like pissing through a pipette. Slow and excruciatingly painful.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[david]

On Sat, 28 Jul 2007, Rene Herman wrote:

> On 07/28/2007 10:55 AM, david@lang.hm wrote:
>
>>  it looks to me like unless the code was really bad (and after 23 months in
>>  -mm it doesn't sound like it is)
>
> Not to sound pretentious or anything but I assume that Andrew has a fairly 
> good overview of exactly how broken -mm can be at times. How many -mm users 
> use it anyway? He himself said he's not convinced of usefulness having not 
> seen it help for him (and notice that most developers are also users), turned 
> it off due to it annoying him at some point and hasn't seen a serious 
> investigation into potential downsides.

if that was the case then people should be responding to the request to 
get it merged with 'but it caused problems for me when I tried it'

I haven't seen any comments like that.

>>  that the only significant con left is the potential to mask other
>>  problems.
>
> Which is not a madeup issue, mind you. As an example, I just now tried GNU 
> locate and saw it's a complete pig and specifically unsuitable for the low 
> memory boxes under discussion. Upon completion, it actually frees enough 
> memory that swap-prefetch _could_ help on some boxes, while the real issue is 
> that they should first and foremost dump GNU locate.

I see the conclusion as being exactly the opposite.

here is a workload with some badly designed userspace software that the 
kernel can make much more pleasent for users.

arguing that users should never use badly designed software in userspace 
doesn't seem like an argument that will gain much traction. I'm not saying 
the kernel needs to fix the software itself (ala the sched_yeild issues), 
but the kernel should try and keep such software from hurting the rest of 
the system where it can.

in this case it can't help it while the bad software is running, but it 
could minimize the impact after it finishes.

>>  however there are many legitimate cases where it is definantly dong the
>>  right thing (swapout was correct in pushing out the pages, but now the
>>  cause of that preasure is gone). the amount of benifit from this will vary
>>  from situation to situation, but it's not reasonable to claim that this
>>  provides no benifit (you have benchmark numbers that show it in synthetic
>>  benchmarks, and you have user reports that show it in the real-worlk)
>
> I certainly would not want to argue anything of the sort no. As said a few 
> times, I agree that swap-prefetch makes sense and has at least the potential 
> to help some situations that you really wouldnt even want to try and fix any 
> other way, simply because nothing's broken.

so there is a legitimate situation where swap-prefetch will help 
significantly, what is the downside that prevents it from being included? 
(reading this thread it sometimes seems like the downside is that updatedb 
shouldn't cause this problem and so if you fixed updatedb there wold be no 
legitimate benifit, or alturnatly this patch doesn't help updatedb so 
there's no legitimate benifit)

>>  there are lots of things in the kernel who's job is to pre-fill the memroy
>>  with data that may (or may not) be useful in the future. this is just
>>  another method of filling the cache. it does so my saying "the user
>>  wanted these pages in the recent past, so it's a reasonable guess to say
>>  that the user will want them again in the future"
>
> Well, _that_ is what the kernel is already going to great lengths at doing, 
> and it decided that those pages us poor overnight OO.o users want in in the 
> morning weren't reasonable guesses. The kernel also won't any time soon be 
> reading our minds, so any solution would need either user intervention (we 
> could devise a way to tell the kernel "hey ho, I consider these pages to be 
> very important -- try not to swap them out" possible even with a "and if you 
> do, please pull them back in when possible") or we can let swap-prefetch do 
> the "just in case" thing it is doing.

it's not that they shouldn't have been swapped out (they should have 
been), it's that the reason they were swapped out no longer exists.

> While swap-prefetch may not be the be all end all of solutions I agree that 
> having a machine sit around with free memory and applications in swap seems 
> not too useful if (as is the case) fetched pages can be dropped immediately 
> when it turns out swap-prefetch made the wrong decision.
>
> So that's for the concept. As to implementation, if I try and look at the 
> code, it seems to be trying hard to really be free and as such, potential 
> downsides seem limited. It's a rather core concept though and as such needs 
> someone with a _lot_ more VM clue to ack. Sorry for not knowing, but who's 
> maintaining/submitting the thing now that Con's not? He or she should 
> preferably address any concerns it seems.

I've seen it mentioned that there is still a maintainer but I missed who 
it is, but I haven't seen any concerns that can be addressed, they all 
seem to be 'this is a core concept, people need to think about it' or 'but 
someone may find a better answer in the future' type of things. it's 
impossible to address these concerns directly.

David Lang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Rene Herman]

On 07/28/2007 11:00 PM, david@lang.hm wrote:

>> many -mm users use it anyway? He himself said he's not convinced of 
>> usefulness having not seen it help for him (and notice that most 
>> developers are also users), turned it off due to it annoying him at 
>> some point and hasn't seen a serious investigation into potential 
>> downsides.
> 
> if that was the case then people should be responding to the request to 
> get it merged with 'but it caused problems for me when I tried it'
> 
> I haven't seen any comments like that.

So you're saying Andrew did not say that? You're jumping to the conclusion 
that I am saying that it's causing problems.

>>>  that the only significant con left is the potential to mask other
>>>  problems.
>>
>> Which is not a madeup issue, mind you. As an example, I just now tried 
>> GNU locate and saw it's a complete pig and specifically unsuitable for 
>> the low memory boxes under discussion. Upon completion, it actually 
>> frees enough memory that swap-prefetch _could_ help on some boxes, 
>> while the real issue is that they should first and foremost dump GNU 
>> locate.
> 
> I see the conclusion as being exactly the opposite.

And now you do it again :-) There is no conclusion -- just the inescapable 
observation that swap-prefetch was (or may have been) masking the problem of 
GNU locate being a program that noone in their right mind should be using.

> so there is a legitimate situation where swap-prefetch will help 
> significantly, what is the downside that prevents it from being 
> included?

People being unconvinced it helps all that much, no serious investigation 
into possible downsides and no consideration of alternatives is three I've 
personally heard.

You don't want to merge a conceptually core VM feature if you're not really 
convinced. It's not a part of the kernel you can throw a feature into like 
you could some driver saying "ah, heck, if it makes someone happy" since 
everything in the VM ends up interacting -- that in fact is actually the 
hard part of VM as far as I've seen it.

And in this situation the proposed feature is something that "papers over a 
problem" by design -- where it could certainly be that the problem is not 
solveable in another way simply due to the kernel not growing the possiblity 
to read user's minds anytime soon (which some might even like to rephrase as 
"due to no problem existing") but that this gets people a bit anxious is not 
surprising.

> I've seen it mentioned that there is still a maintainer but I missed who
> it is, but I haven't seen any concerns that can be addressed, they all 
> seem to be 'this is a core concept, people need to think about it' or 
> 'but someone may find a better answer in the future' type of things. it's
> impossible to address these concerns directly.

So do it indirectly. But please don't just say "it help some people (not me 
mind you!) so merge it and if you don't it's all just politics and we can't 
do anything about it anyway". Because that's mostly what I've been hearing.

And no, I'm not subscribed to any ck mailinglists nor do I hang around its 
IRC community which will can account for part of that. I expect though that 
the same holds for the people that actually matter in this, such as Andrew 
Morton and Nick Piggin.

-- 1: people being unconvinced it helps all that much

At least partly caused by the updatedb i/dcache red herring that infected 
this issue. Also, at the point VM  pressure has mounted high enough to cause 
enough to be swapped out to give you a bad experience, a lot of other things 
have been dropped already as well.

It's unsurprising though that it would for example help the issue of 
openoffice with a large open spreadsheet having been thrown out overnight 
meaning it's a matter of deciding whether or not this is an important enough 
issue to fix inside the VM with something like swap-prefetch.

Personally -- no opinion, I do not experience the problem (I even switch off 
the machine at night and do not run cron at all).

-- 2: no serious investigation into possible downsides

Swap-prefetch tries hard to be as free as possible and it seems to largely 
be succeeding at that. Thing that (obviously -- as in I wouldn't want to 
state it's the only possible worry anyone could have left) remains is the 
"papering over effect" it has by design that one might not care for.

-- 3: no serious consideration of possible alternatives

Tweaking existing use-oce logic is one I've heard but if we consider the 
i/dcache issue dead, I believe that one is as well. Going to userspace is 
another one. Largest theoretical potential. I myself am extremely sceptical 
about the Linux userland, and largely equate it with "smallest _practical_ 
potential" -- but that might just be me.

A larger swap granularity, possible even a self-training granularity. Up to 
now, seeks only get costlier and costlier with respect to reads with every 
generation of disk (flash would largely overcome it though) and doing more 
in one read/write _greatly_ improves throughput, maybe up to the point that 
swap-prefetch is no longer very useful. I myself don't know about the 
tradeoffs involved.

Any other alternatives?

Any 4th and higher points?

Rene.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[david]

On Sun, 29 Jul 2007, Rene Herman wrote:

> On 07/28/2007 11:00 PM, david@lang.hm wrote:
>
>> >  many -mm users use it anyway? He himself said he's not convinced of 
>> >  usefulness having not seen it help for him (and notice that most 
>> >  developers are also users), turned it off due to it annoying him at some 
>> >  point and hasn't seen a serious investigation into potential downsides.
>>
>>  if that was the case then people should be responding to the request to
>>  get it merged with 'but it caused problems for me when I tried it'
>>
>>  I haven't seen any comments like that.
>
> So you're saying Andrew did not say that? You're jumping to the conclusion 
> that I am saying that it's causing problems.

I don't remember anyone saying that it actually caused problems (including 
both you and andrew). I (and others) have been trying to learn what 
problems people believe it has in the hope that they can be addressed one 
way or another.

>> > >   that the only significant con left is the potential to mask other
>> > >   problems.
>> > 
>> >  Which is not a madeup issue, mind you. As an example, I just now tried 
>> >  GNU locate and saw it's a complete pig and specifically unsuitable for 
>> >  the low memory boxes under discussion. Upon completion, it actually 
>> >  frees enough memory that swap-prefetch _could_ help on some boxes, while 
>> >  the real issue is that they should first and foremost dump GNU locate.
>>
>>  I see the conclusion as being exactly the opposite.
>
> And now you do it again :-) There is no conclusion -- just the inescapable 
> observation that swap-prefetch was (or may have been) masking the problem of 
> GNU locate being a program that noone in their right mind should be using.

isn't your conclusion then that if people just stopped useing that version 
of updatedb the problem would be solved and there would be no need for the 
swap prefetch patch? that seemed to be what you were strongly implying (if 
not saying outright)

>>  so there is a legitimate situation where swap-prefetch will help
>>  significantly, what is the downside that prevents it from being included?
>
> People being unconvinced it helps all that much, no serious investigation 
> into possible downsides and no consideration of alternatives is three I've 
> personally heard.
>
> You don't want to merge a conceptually core VM feature if you're not really 
> convinced. It's not a part of the kernel you can throw a feature into like 
> you could some driver saying "ah, heck, if it makes someone happy" since 
> everything in the VM ends up interacting -- that in fact is actually the hard 
> part of VM as far as I've seen it.
>
> And in this situation the proposed feature is something that "papers over a 
> problem" by design -- where it could certainly be that the problem is not 
> solveable in another way simply due to the kernel not growing the possiblity 
> to read user's minds anytime soon (which some might even like to rephrase as 
> "due to no problem existing") but that this gets people a bit anxious is not 
> surprising.

people who have lots of memory and so don't use swap will never see the 
benifit of this patch. over the years many people have investigated the 
problem and tried to address it in other ways (the better version of 
updatedb is an attempt to fix it for that program as an example), but 
there is still a problem.

I agree that tinkering with the core VM code should not be done lightly, 
but this has been put through the proper process and is stalled with no 
hints on how to move forward.

>>  I've seen it mentioned that there is still a maintainer but I missed who
>>  it is, but I haven't seen any concerns that can be addressed, they all
>>  seem to be 'this is a core concept, people need to think about it' or 'but
>>  someone may find a better answer in the future' type of things. it's
>>  impossible to address these concerns directly.
>
> So do it indirectly. But please don't just say "it help some people (not me 
> mind you!) so merge it and if you don't it's all just politics and we can't 
> do anything about it anyway". Because that's mostly what I've been hearing.
>
> And no, I'm not subscribed to any ck mailinglists nor do I hang around its 
> IRC community which will can account for part of that. I expect though that 
> the same holds for the people that actually matter in this, such as Andrew 
> Morton and Nick Piggin.
>
> -- 1: people being unconvinced it helps all that much
>
> At least partly caused by the updatedb i/dcache red herring that infected 
> this issue. Also, at the point VM  pressure has mounted high enough to cause 
> enough to be swapped out to give you a bad experience, a lot of other things 
> have been dropped already as well.
>
> It's unsurprising though that it would for example help the issue of 
> openoffice with a large open spreadsheet having been thrown out overnight 
> meaning it's a matter of deciding whether or not this is an important enough 
> issue to fix inside the VM with something like swap-prefetch.
>
> Personally -- no opinion, I do not experience the problem (I even switch off 
> the machine at night and do not run cron at all).

forget the nightly cron jobs for the moment. think of this scenerio. you 
have your memory fairly full with apps that you have open (including 
firefox with many tabs), you receive a spreadsheet you need to look at, so 
you fire up openoffice to look at it. then you exit openoffice and try to 
go back to firefox (after a pause while you walk to the printer to get 
the printout of the spreadsheet), only to find that it's going to be 
sluggish becouse it got swapped out due to the preasure from openoffice.

no nightly cron job needed, just enough of a memory hog or a small enough 
amount of ram to have your working set exceed it.

> -- 2: no serious investigation into possible downsides
>
> Swap-prefetch tries hard to be as free as possible and it seems to largely be 
> succeeding at that. Thing that (obviously -- as in I wouldn't want to state 
> it's the only possible worry anyone could have left) remains is the "papering 
> over effect" it has by design that one might not care for.
>
> -- 3: no serious consideration of possible alternatives
>
> Tweaking existing use-oce logic is one I've heard but if we consider the 
> i/dcache issue dead, I believe that one is as well. Going to userspace is 
> another one. Largest theoretical potential. I myself am extremely sceptical 
> about the Linux userland, and largely equate it with "smallest _practical_ 
> potential" -- but that might just be me.
>
> A larger swap granularity, possible even a self-training granularity. Up to 
> now, seeks only get costlier and costlier with respect to reads with every 
> generation of disk (flash would largely overcome it though) and doing more in 
> one read/write _greatly_ improves throughput, maybe up to the point that 
> swap-prefetch is no longer very useful. I myself don't know about the 
> tradeoffs involved.

larger swap granularity may help, but waiting for the user to need the ram 
and have to wait for it to be read back in is always going to be worse for 
the user then pre-populating the free memory (for the case where the 
pre-population is right, for other cases it's the same). so I see this as 
a red herring

> Any other alternatives?
>
> Any 4th and higher points?

there are fully legitimate situations where this is useful, the 'papering 
over' effect is not referring to these, it's referring to other possible 
problems in the future. I see this argument as being in the same catagory 
as people wanting to remove the old, working driver for some hardware in 
favor of the new, unreliable driver for the same hardware in order to get 
more bug reports to find the holes in the new driver. that's causing users 
unessasary pain and within the last week Linus was takeing a driver author 
to task for attempting exactly that IIRC (and yes, there does come a point 
where there are no further bugs known in the new driver, and it appears to 
do everything the old driver did that you do remove the old driver, but 
you don't remove it early to help the new driver stabilize)

David Lang

> Rene.
>
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Rene Herman]

On 07/29/2007 01:41 PM, david@lang.hm wrote:

>> And now you do it again :-) There is no conclusion -- just the 
>> inescapable observation that swap-prefetch was (or may have been) 
>> masking the problem of GNU locate being a program that noone in their 
>> right mind should be using.
> 
> isn't your conclusion then that if people just stopped useing that 
> version of updatedb the problem would be solved and there would be no 
> need for the swap prefetch patch? that seemed to be what you were 
> strongly implying (if not saying outright)

No. What I said outright, every single time, is that swap-prefetch in itself 
seems to make sense. And specifically that even if the _direct_ problem is a 
crummy program, it _still_ makes sense generally. Every single time.

But see -- you failed to notice this because you guys are stuck in this dumb 
adversary "us against them" thing so inherent of (online) communities, where 
you sit around your own habitats patting each other on the back for extended 
periods of time and then every once a while go out clinging on to each other 
vigorously and going "boo! hiss!" at the big bad outside world.

I already got overly violent at one point in this thread so I'll leave out 
any further references to sense-deprived fanboy-culture but please, I said 
every single time that I'm not against swap-prefetch. I cannot communicate 
when I'm not being read.

> I agree that tinkering with the core VM code should not be done lightly, 
> but this has been put through the proper process and is stalled with no 
> hints on how to move forward.

It has not. Concerns that were raised (by specifically Nick Piggin) weren't 
being addressed.

> forget the nightly cron jobs for the moment. think of this scenerio. you 
> have your memory fairly full with apps that you have open (including 
> firefox with many tabs), you receive a spreadsheet you need to look at, 
> so you fire up openoffice to look at it. then you exit openoffice and try
>  to go back to firefox (after a pause while you walk to the printer to
> get the printout of the spreadsheet)

And swinging a dead rat from its tail facing east-wards while reciting 
Documentation/CodingStyle.

Okay, very very sorry, that was particularly childish, but that "walking to 
the printer" is ofcourse completely constructed and this _is_ something to 
take into account. Swap-prefetch wants to be free, which (also again) it is 
doing a good job at it seems, but this also means that it waits for the VM 
to be _very_ idle before it does anything and as such, we cannot just forget 
the "nightly" scenario and pretend it's about something else entirely. As 
long as the machine's being used, swap-prefetch doesn't kick in.

Which is a good feature for swap-prefetch, but also something that needs to 
weighed alongside its other features in a discussion of alternatives, where 
for example something like a larger swap granularity would not have anything 
of the sort to take into account. If it were about walks to the printer, we 
could shelve the issue as being of too limited practical use for inclusion.

>> -- 2: no serious investigation into possible downsides
>>
>> Swap-prefetch tries hard to be as free as possible and it seems to 
>> largely be succeeding at that. Thing that (obviously -- as in I 
>> wouldn't want to state it's the only possible worry anyone could have 
>> left) remains is the "papering over effect" it has by design that one 
>> might not care for.

Arjan van de Ven made another point here about seeking away due to 
swap-prefetch (just) before the next request comes in, but that's probably a 
bit of a non-issue in practice with the "very idle" precondition.

>> -- 3: no serious consideration of possible alternatives
>>
>> Tweaking existing use-oce logic is one I've heard but if we consider 
>> the i/dcache issue dead, I believe that one is as well. Going to 
>> userspace is another one. Largest theoretical potential. I myself am 
>> extremely sceptical about the Linux userland, and largely equate it 
>> with "smallest _practical_ potential" -- but that might just be me.
>>
>> A larger swap granularity, possible even a self-training granularity. 
>> Up to now, seeks only get costlier and costlier with respect to reads 
>> with every generation of disk (flash would largely overcome it though) 
>> and doing more in one read/write _greatly_ improves throughput, maybe 
>> up to the point that swap-prefetch is no longer very useful. I myself 
>> don't know about the tradeoffs involved.
> 
> larger swap granularity may help, but waiting for the user to need the 
> ram and have to wait for it to be read back in is always going to be 
> worse for the user then pre-populating the free memory (for the case 
> where the pre-population is right, for other cases it's the same). so I 
> see this as a red herring

I saw Chris Snook make a good post here and am going to defer this part to 
that discussion:

http://lkml.org/lkml/2007/7/27/421

But no, it's not a red herring if _practically_ speaking the swapin is fast 
enough once started that people don't actually mind anymore since in that 
case you could simply do without yet more additional VM complexity (and 
kernel daemon).

> there are fully legitimate situations where this is useful, the 'papering
> over' effect is not referring to these, it's referring to other possible
> problems in the future.

No, it's not just future. Just look at the various things under discussion 
now such as improved use-once and better swapin.

Rene.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[david]

On Sun, 29 Jul 2007, Rene Herman wrote:

> On 07/29/2007 01:41 PM, david@lang.hm wrote:
>
>>  I agree that tinkering with the core VM code should not be done lightly,
>>  but this has been put through the proper process and is stalled with no
>>  hints on how to move forward.
>
> It has not. Concerns that were raised (by specifically Nick Piggin) weren't 
> being addressed.

I may have missed them, but what I saw from him weren't specific issues, 
but instead a nebulous 'something better may come along later'

>>  forget the nightly cron jobs for the moment. think of this scenerio. you
>>  have your memory fairly full with apps that you have open (including
>>  firefox with many tabs), you receive a spreadsheet you need to look at, so
>>  you fire up openoffice to look at it. then you exit openoffice and try
>>  to go back to firefox (after a pause while you walk to the printer to
>>  get the printout of the spreadsheet)
>
> And swinging a dead rat from its tail facing east-wards while reciting 
> Documentation/CodingStyle.
>
> Okay, very very sorry, that was particularly childish, but that "walking to 
> the printer" is ofcourse completely constructed and this _is_ something to 
> take into account.

yes it was contrived for simplicity.

the same effect would happen if instead of going back to firefox the user 
instead went to their e-mail software and read some mail. doing so should 
still make the machine idle enough to let prefetch kick in.

> Swap-prefetch wants to be free, which (also again) it is 
> doing a good job at it seems, but this also means that it waits for the VM to 
> be _very_ idle before it does anything and as such, we cannot just forget the 
> "nightly" scenario and pretend it's about something else entirely. As long as 
> the machine's being used, swap-prefetch doesn't kick in.

how long does the machine need to be idle? if someone spends 30 seconds 
reading an e-mail that's an incredibly long time for the system and I 
would think it should be enough to let the prefetch kick in.

>> >  -- 3: no serious consideration of possible alternatives
>> > 
>> >  Tweaking existing use-oce logic is one I've heard but if we consider 
>> >  the i/dcache issue dead, I believe that one is as well. Going to 
>> >  userspace is another one. Largest theoretical potential. I myself am 
>> >  extremely sceptical about the Linux userland, and largely equate it 
>> >  with "smallest _practical_ potential" -- but that might just be me.
>> > 
>> >  A larger swap granularity, possible even a self-training 
>> >  granularity. Up to now, seeks only get costlier and costlier with 
>> >  respect to reads with every generation of disk (flash would largely 
>> >  overcome it though) and doing more in one read/write _greatly_ 
>> >  improves throughput, maybe up to the point that swap-prefetch is no 
>> >  longer very useful. I myself don't know about the tradeoffs 
>> >  involved.
>>
>>  larger swap granularity may help, but waiting for the user to need the
>>  ram and have to wait for it to be read back in is always going to be
>>  worse for the user then pre-populating the free memory (for the case
>>  where the pre-population is right, for other cases it's the same). so
>>  I see this as a red herring
>
> I saw Chris Snook make a good post here and am going to defer this part to 
> that discussion:
>
> http://lkml.org/lkml/2007/7/27/421
>
> But no, it's not a red herring if _practically_ speaking the swapin is fast 
> enough once started that people don't actually mind anymore since in that 
> case you could simply do without yet more additional VM complexity (and 
> kernel daemon).

swapin will always require disk access, and avoiding doing disk access 
while the user is waiting for it by doing it when the system isn't useing 
the disk will always be a win (possibly not as large of a win, but still a 
win) on slow laptop drives where you may only get 20MB/second of reads 
under optimal situations it doesn't take much reading to be noticed by the 
user.

>>  there are fully legitimate situations where this is useful, the 'papering
>>  over' effect is not referring to these, it's referring to other possible
>>  problems in the future.
>
> No, it's not just future. Just look at the various things under discussion 
> now such as improved use-once and better swapin.

and these thing do not conflict with prefetch, they compliment it.

improved use-once will avoid pushing things out to swap in the first 
place. this will help during normal workloads so is valuble in any case.

better swapin (I assume you are talking about things like larger swap 
granularity) will also help during normal workloads when you are thrashing 
into swap.

prefetch will help when you have pushed things out to swap and now have 
free memory and a momentarily idle system.

David Lang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Nick Piggin]

david@lang.hm wrote:
> On Sun, 29 Jul 2007, Rene Herman wrote:
> 
>> On 07/29/2007 01:41 PM, david@lang.hm wrote:
>>
>>>  I agree that tinkering with the core VM code should not be done 
>>> lightly,
>>>  but this has been put through the proper process and is stalled with no
>>>  hints on how to move forward.
>>
>>
>> It has not. Concerns that were raised (by specifically Nick Piggin) 
>> weren't being addressed.
> 
> 
> I may have missed them, but what I saw from him weren't specific issues, 
> but instead a nebulous 'something better may come along later'

Something better, ie. the problems with page reclaim being fixed.
Why is that nebulous?

-- 
SUSE Labs, Novell Inc.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[david]

On Mon, 6 Aug 2007, Nick Piggin wrote:

> david@lang.hm wrote:
>>  On Sun, 29 Jul 2007, Rene Herman wrote:
>> 
>> >  On 07/29/2007 01:41 PM, david@lang.hm wrote:
>> > 
>> > >  I agree that tinkering with the core VM code should not be done 
>> > >  lightly,
>> > >   but this has been put through the proper process and is stalled with 
>> > >   no
>> > >   hints on how to move forward.
>> > 
>> > 
>> >  It has not. Concerns that were raised (by specifically Nick Piggin) 
>> >  weren't being addressed.
>>
>>
>>  I may have missed them, but what I saw from him weren't specific issues,
>>  but instead a nebulous 'something better may come along later'
>
> Something better, ie. the problems with page reclaim being fixed.
> Why is that nebulous?

becouse that doesn't begin to address all the benifits.

the approach of fixing page reclaim and updatedb is pretending that if you 
only do everything right pages won't get pushed to swap in the first 
place, and therefor swap prefetch won't be needed.

this completely ignores the use case where the swapping was exactly the 
right thing to do, but memory has been freed up from a program exiting so 
that you couldnow fill that empty ram with data that was swapped out.

David Lang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Nick Piggin]

--- david@lang.hm wrote:

> On Mon, 6 Aug 2007, Nick Piggin wrote:
> 
> > david@lang.hm wrote:
> >>  On Sun, 29 Jul 2007, Rene Herman wrote:
> >> 
> >> >  On 07/29/2007 01:41 PM, david@lang.hm wrote:
> >> > 
> >> > >  I agree that tinkering with the core VM code
> should not be done 
> >> > >  lightly,
> >> > >   but this has been put through the proper
> process and is stalled with 
> >> > >   no
> >> > >   hints on how to move forward.
> >> > 
> >> > 
> >> >  It has not. Concerns that were raised (by
> specifically Nick Piggin) 
> >> >  weren't being addressed.
> >>
> >>
> >>  I may have missed them, but what I saw from him
> weren't specific issues,
> >>  but instead a nebulous 'something better may
> come along later'
> >
> > Something better, ie. the problems with page
> reclaim being fixed.
> > Why is that nebulous?
> 
> becouse that doesn't begin to address all the
> benifits.

What do you mean "address the benefits"? What I want
to address is the page reclaim problems.


> the approach of fixing page reclaim and updatedb is
> pretending that if you 
> only do everything right pages won't get pushed to
> swap in the first 
> place, and therefor swap prefetch won't be needed.

You should read what I wrote.

Anyway, the fact of the matter is that there are still
fairly significant problems with page reclaim in this
workload which I would like to see fixed.

I personally still think some of the low hanging fruit
*might* be better fixed before swap prefetch gets
merged, but I've repeatedly said I'm sick of getting
dragged back into the whole debate so I'm happy with
whatever Andrew decides to do with it.

I think it is sad to turn it off for laptops, if it
really makes the "desktop" experience so much better.
Surely for _most_ workloads we should be able to
manage 1-2GB of RAM reasonably well.

 
> this completely ignores the use case where the
> swapping was exactly the 
> right thing to do, but memory has been freed up from
> a program exiting so 
> that you couldnow fill that empty ram with data that
> was swapped out.

Yeah. However, merging patches (especially when
changing heuristics, especially in page reclaim) is
not about just thinking up a use-case that it works
well for and telling people that they're putting their
heads in the sand if they say anything against it.
Read this thread and you'll find other examples of
patches that have been around for as long or longer
and also have some good use-cases and also have not
been merged.



      ____________________________________________________________________________________
Yahoo!7 Mail has just got even bigger and better with unlimited storage on all webmail accounts. 
http://au.docs.yahoo.com/mail/unlimitedstorage.html

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Paolo Ciarrocchi]

On 8/6/07, Nick Piggin <nickpiggin@yahoo.com.au> wrote:
[...]
> > this completely ignores the use case where the
> > swapping was exactly the
> > right thing to do, but memory has been freed up from
> > a program exiting so
> > that you couldnow fill that empty ram with data that
> > was swapped out.
>
> Yeah. However, merging patches (especially when
> changing heuristics, especially in page reclaim) is
> not about just thinking up a use-case that it works
> well for and telling people that they're putting their
> heads in the sand if they say anything against it.
> Read this thread and you'll find other examples of
> patches that have been around for as long or longer
> and also have some good use-cases and also have not
> been merged.

What do you think Andrew?
Swap prefetch is not the panacea, it's not going to solve all the
problems but it seems to improve the "desktop experience" and it has
been discussed and reviewed a lot (it's has even been discussed more
than it should have be).

Are you going to push upstream the patch?

Ciao,
-- 
Paolo
http://paolo.ciarrocchi.googlepages.com/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Daniel Hazelton]

On Saturday 28 July 2007 04:55:58 david@lang.hm wrote:
> On Sat, 28 Jul 2007, Rene Herman wrote:
> > On 07/27/2007 09:43 PM, david@lang.hm wrote:
> >>  On Fri, 27 Jul 2007, Rene Herman wrote:
> >> >  On 07/27/2007 07:45 PM, Daniel Hazelton wrote:
> >> > >   Questions about it:
> >> > >   Q) Does swap-prefetch help with this?
> >> > >   A) [From all reports I've seen (*)]
> >> > >   Yes, it does.
> >> >
> >> >  No it does not. If updatedb filled memory to the point of causing
> >> >  swapping (which noone is reproducing anyway) it HAS FILLED MEMORY and
> >> >  swap-prefetch hasn't any memory to prefetch into -- updatedb itself
> >> >  doesn't use any significant memory.
> >>
> >>  however there are other programs which are known to take up significant
> >>  amounts of memory and will cause the issue being described (openoffice
> >> for example)
> >>
> >>  please don't get hung up on the text 'updatedb' and accept that there
> >> are programs that do run intermittently and do use a significant amount
> >> of ram and then free it.
> >
> > Different issue. One that's worth pursueing perhaps, but a different
> > issue from the VFS caches issue that people have been trying to track
> > down.
>
> people are trying to track down the problem of their machine being slow
> until enough data is swapped back in to operate normally.
>
> in at some situations swap prefetch can help becouse something that used
> memory freed it so there is free memory that could be filled with data
> (which is something that Linux does agressivly in most other situations)
>
> in some other situations swap prefetch cannot help becouse useless data is
> getting cached at the expense of useful data.
>
> nobody is arguing that swap prefetch helps in the second cast.

Actually, I made a mistake when tracking the thread and reading the code for 
the patch and started to argue just that. But I have to admit I made a 
mistake - the patches author has stated (as Rene was kind enough to point 
out) that swap prefetch can't help when memory is filled.

> what people are arguing is that there are situations where it helps for
> the first case. on some machines and version of updatedb the nighly run of
> updatedb can cause both sets of problems. but the nightly updatedb run is
> not the only thing that can cause problems

Solving the cache filling memory case is difficult. There have been a number 
of discussions about it. The simplest solution, IMHO, would be to place a 
(configurable) hard limit on the maximum size any of the kernels caches can 
grow to. (The only solution that was discussed, however, is a complex beast)

>
> but let's talk about the concept here for a little bit
>
> the design is to use CPU and I/O capacity that's otherwise idle to fill
> free memory with data from swap.
>
> pro:
>    more ram has potentially useful data in it
>
> con:
>    it takes a little extra effort to give this memory to another app (the
> page must be removed from the list and zeroed at the time it's needed, I
> assume that the data is left in swap so that it doesn't have to be written
> out again)
>
>    it adds some complexity to the kernel (~500 lines IIRC from this thread)
>
>    by undoing recent swapouts it can potentially mask problems with swapout
>
> it looks to me like unless the code was really bad (and after 23 months in
> -mm it doesn't sound like it is) that the only significant con left is the
> potential to mask other problems.

I'll second this. But with the swap system itself having seen as heavy testing 
as it has I don't know if it would be masking other problems.

That is why I've been asking "What is so wrong with it?" - while it definately 
doesn't help with programs that cause caches to balloon (that problem does 
need another solution) it does help to speed things up when a memory hog has 
exited. (And since its a pretty safe assumption that swap is going to be 
noticeably slower than RAM this patch seems to me to be a rather visible and 
obvious solution to that problem)

> however there are many legitimate cases where it is definantly dong the
> right thing (swapout was correct in pushing out the pages, but now the
> cause of that preasure is gone). the amount of benifit from this will vary
> from situation to situation, but it's not reasonable to claim that this
> provides no benifit (you have benchmark numbers that show it in synthetic
> benchmarks, and you have user reports that show it in the real-worlk)

Exactly. Though I have seen posts which (to me at least) appear to claim 
exactly that. It was part of the reason why I got a bit incensed. (The other 
was that it looked like the kernel devs with the ultra-powerful machines were 
claiming 'I don't see the problem on my machine, so it doesn't exist'. That 
sort of attitude is fine, in some cases, but not, IMHO, where performance is 
concerned)

> there are lots of things in the kernel who's job is to pre-fill the memroy
> with data that may (or may not) be useful in the future. this is just
> another method of filling the cache. it does so my saying "the user wanted
> these pages in the recent past, so it's a reasonable guess to say that the
> user will want them again in the future"

Yep. And it's a pretty obvious step forward. The VFS system already does 
readahead and caching for mounted volumes to improve performance - why not do 
similar to improve the performance of swap?

The only real downside is that swap-prefetch won't be effective in all cases 
and it will cause some extra power consumption. (drives can't spin-down as 
soon as the would without it, etc...) While I can only make some suggestions 
as to how to fix the problem of ballooning caches (I've been wading through 
the VM code for a few days now and still don't fully understand any of it), 
the solution to the power consumption seems obvious - swap prefetch doesn't 
work when the system is running on battery (or UPS or whatever)

DRH

-- 
Dialup is like pissing through a pipette. Slow and excruciatingly painful.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[david]

On Sat, 28 Jul 2007, Daniel Hazelton wrote:

> 
> On Saturday 28 July 2007 04:55:58 david@lang.hm wrote:
>> On Sat, 28 Jul 2007, Rene Herman wrote:
>>> On 07/27/2007 09:43 PM, david@lang.hm wrote:
>>>>  On Fri, 27 Jul 2007, Rene Herman wrote:
>>>>>  On 07/27/2007 07:45 PM, Daniel Hazelton wrote:
>>
>> nobody is arguing that swap prefetch helps in the second cast.
>
> Actually, I made a mistake when tracking the thread and reading the code for
> the patch and started to argue just that. But I have to admit I made a
> mistake - the patches author has stated (as Rene was kind enough to point
> out) that swap prefetch can't help when memory is filled.

I stand corrected, thaks for speaking up and correcting your position.

>> what people are arguing is that there are situations where it helps for
>> the first case. on some machines and version of updatedb the nighly run of
>> updatedb can cause both sets of problems. but the nightly updatedb run is
>> not the only thing that can cause problems
>
> Solving the cache filling memory case is difficult. There have been a number
> of discussions about it. The simplest solution, IMHO, would be to place a
> (configurable) hard limit on the maximum size any of the kernels caches can
> grow to. (The only solution that was discussed, however, is a complex beast)

limiting the size of the cache is also the wrong thing to do in many 
situations. it's only right if the cache pushes out other data you care 
about, if you are trying to do one thing as fast as you can you really do 
want the system to use all the memory it can for the cache.

David Lang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Daniel Hazelton]

On Saturday 28 July 2007 17:06:50 david@lang.hm wrote:
> On Sat, 28 Jul 2007, Daniel Hazelton wrote:
> > On Saturday 28 July 2007 04:55:58 david@lang.hm wrote:
> >> On Sat, 28 Jul 2007, Rene Herman wrote:
> >>> On 07/27/2007 09:43 PM, david@lang.hm wrote:
> >>>>  On Fri, 27 Jul 2007, Rene Herman wrote:
> >>>>>  On 07/27/2007 07:45 PM, Daniel Hazelton wrote:
> >>
> >> nobody is arguing that swap prefetch helps in the second cast.
> >
> > Actually, I made a mistake when tracking the thread and reading the code
> > for the patch and started to argue just that. But I have to admit I made
> > a mistake - the patches author has stated (as Rene was kind enough to
> > point out) that swap prefetch can't help when memory is filled.
>
> I stand corrected, thaks for speaking up and correcting your position.

If you had made the statement before I decided to speak up you would have been 
correct :)

Anyway, I try to always admit when I've made a mistake - its part of my 
philosophy. (There have been times when I haven't done it, but I'm trying to 
make that stop entirely)

> >> what people are arguing is that there are situations where it helps for
> >> the first case. on some machines and version of updatedb the nighly run
> >> of updatedb can cause both sets of problems. but the nightly updatedb
> >> run is not the only thing that can cause problems
> >
> > Solving the cache filling memory case is difficult. There have been a
> > number of discussions about it. The simplest solution, IMHO, would be to
> > place a (configurable) hard limit on the maximum size any of the kernels
> > caches can grow to. (The only solution that was discussed, however, is a
> > complex beast)
>
> limiting the size of the cache is also the wrong thing to do in many
> situations. it's only right if the cache pushes out other data you care
> about, if you are trying to do one thing as fast as you can you really do
> want the system to use all the memory it can for the cache.

After thinking about this you are partially correct. There are those sorts of 
situations where you want the system to use all the memory it can for caches. 
OTOH, if those situations could be described in some sort of simple 
heuristic, then a soft-limit that uses those heuristics to determine when to 
let the cache expand could exploit the benefits of having both a limited and 
unlimited cache. (And, potentially, if the heuristic has allowed a cache to 
expand beyond the limit then, when the heuristic no longer shows the oversize 
cache is no longer necessary it could trigger and automatic reclaim of that 
memory.)

(I'm willing to help write and test code to do this exactly. There is no 
guarantee that I'll be able to help with more than testing - I don't 
understand the parts of the code involved all that well)

DRH

-- 
Dialup is like pissing through a pipette. Slow and excruciatingly painful.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Daniel Hazelton]

On Friday 27 July 2007 14:16:32 Rene Herman wrote:
> On 07/27/2007 07:45 PM, Daniel Hazelton wrote:
> > Updatedb or another process that uses the FS heavily runs on a users
> > 256MB P3-800 (when it is idle) and the VFS caches grow, causing memory
> > pressure that causes other applications to be swapped to disk. In the
> > morning the user has to wait for the system to swap those applications
> > back in.
> >
> > Questions about it:
> > Q) Does swap-prefetch help with this?
> > A) [From all reports I've seen (*)] Yes, it does.
>
> No it does not. If updatedb filled memory to the point of causing swapping
> (which noone is reproducing anyway) it HAS FILLED MEMORY and swap-prefetch
> hasn't any memory to prefetch into -- updatedb itself doesn't use any
> significant memory.

Check the attitude at the door then re-read what I actually said:
> > Updatedb or another process that uses the FS heavily runs on a users
> > 256MB P3-800 (when it is idle) and the VFS caches grow, causing memory
> > pressure that causes other applications to be swapped to disk. In the
> > morning the user has to wait for the system to swap those applications
> > back in.

I never said that it was the *program* itself - or *any* specific program (I 
used "Updatedb" because it has been the big name in the discussion) - doing 
the filling of memory. I actually said that the problem is that the kernel's 
caches - VFS and others - will grow *WITHOUT* *LIMIT*, filling all available 
memory. 

Swap prefetch on its own will not alleviate *all* of the problem, but it 
appears to fix enough of it that the problem doesn't seem to bother people 
anymore. (As I noted later on there are things that can be changes that would 
also fix things. Those changes, however, are quite tricky and involve changes 
to the page faulting mechanism, the way the various caches work and a number 
of other things)

In light of the fact that swap prefetch appears to solve the problem for the 
people that have been vocal about it, and because it is a less intrusive 
change than the other potential solutions, I'd like to know why all the 
complaints and arguments against it come down to "Its treating the symptom".

I mean it - because I fail to see how it isn't getting at the root of the 
problem - which is, pretty much, that Swap has classically been and, in the 
case of most modern systems, still is damned slow. By prefetching those pages 
that have most recently been evicted the problem of "slow swap" is being 
directly addressed.

You want to know what causes the problem? The current design of the caches. 
They will extend without much limit, to the point of actually pushing pages 
to disk so they can grow even more. 

> Here's swap-prefetch's author saying the same:
>
> http://lkml.org/lkml/2007/2/9/112
>
> | It can't help the updatedb scenario. Updatedb leaves the ram full and
> | swap prefetch wants to cost as little as possible so it will never
> | move anything out of ram in preference for the pages it wants to swap
> | back in.
>
> Now please finally either understand this, or tell us how we're wrong.
>
> Rene.

I already did. You completely ignored it because I happened to use the magic 
words "updatedb" and "swap prefetch". 

Did I ever say it was about "updatedb" in particular? You've got the statement 
in the part of my post that you quoted. Nope, appears that I used the name as 
a specific example - and one that has been used previously in the thread. Now 
drop the damned attitude and start using your brain. Okay?

DRH

-- 
Dialup is like pissing through a pipette. Slow and excruciatingly painful.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Rene Herman]

On 07/27/2007 10:28 PM, Daniel Hazelton wrote:

> Check the attitude at the door then re-read what I actually said:

Attitude? You wanted attitude dear boy?

>>> Updatedb or another process that uses the FS heavily runs on a users
>>> 256MB P3-800 (when it is idle) and the VFS caches grow, causing memory
>>> pressure that causes other applications to be swapped to disk. In the
>>> morning the user has to wait for the system to swap those applications
>>> back in.
> 
> I never said that it was the *program* itself - or *any* specific program (I 
> used "Updatedb" because it has been the big name in the discussion) - doing 
> the filling of memory. I actually said that the problem is that the kernel's 
> caches - VFS and others - will grow *WITHOUT* *LIMIT*, filling all available 
> memory. 

WHICH SWAP-PREFETCH DOES NOT HELP WITH.
WHICH SWAP-PREFETCH DOES NOT HELP WITH.
WHICH SWAP-PREFETCH DOES NOT HELP WITH.

And now finally get that through your thick scull or shut up, right fucking now.

> You want to know what causes the problem? The current design of the caches. 
> They will extend without much limit, to the point of actually pushing pages 
> to disk so they can grow even more. 

Due to being a generally nice guy, I am going to try _once_ more to try and 
make you understand. Not twice, once. So pay attention. Right now.

Those caches are NOT causing any problem under discussion. If any caches 
grow to the point of causing swap-out, they have filled memory and 
swap-prefetch cannot and will not do anything since it needs free (as in not 
occupied by caches) memory. As such, people maintaining that swap-prefetch 
helps their situation are not being hit by caches.

The only way swap-prefetch can (and will) do anything is when something that 
by itself takes up lots of memory runs and exits. So can we now please 
finally drop the fucking red herring and start talking about swap-prefetch?

If we accept that some of the people maintaining that swap-prefetch helps 
them are not in fact deluded -- a bit of a stretch seeing as how not a 
single one of them is substantiating anything -- we have a number of 
slightly different possibilities for "something" in the above.

-- 1)

It could be an inefficient updatedb. Although he isn't experiencing the 
problem, Bjoern Steinbrink is posting numbers (weeee!) that show that at 
least the GNU version spawns a large memory "sort" process meaning that on a 
low-memory box updatedb itself can be what causes the observed problem.

While in this situation switching to a different updatedb (slocate, mlocate) 
obviously makes sense it's the kind of situation where swap-prefetch will help.

-- 2)

It could be something else entirely such as a backup run. I suppose people 
would know if they were running anything of the sort though and wouldn't 
blaim anything on updatedb.

Other than that, it's again the situation where swap-prefetch would help.

-- 3)

The something else entirely can also run _after_ updatedb, kicking out the 
VFS caches and leaving free memory upon exit. I still suppose the same thing 
as under (2) but this is the only way how updatedb / VFS caches can even be 
part of any problem, if the _combined_ memory pressure is just enough to 
make the difference.

The direct problem is still just the "something else entirely" and needs 
someone affected to tell us what it is.

> I already did. You completely ignored it because I happened to use the magic 
> words "updatedb" and "swap prefetch". 

No I did not. This thread is about swap-prefetch and you used the magic 
words VFS caches. I don't give a fryin' fuck if their filling is caused by 
updatedb or the cat sleeping on the "find /<enter>" keys on your keyboard, 
they're still not causing anything swap-prefetch helps with.

This thread has seen input from a selection of knowledgeable people and 
Morton was even running benchmarks to look at this supposed VFS cache 
problem and not finding it. The only further input this thread needs is 
someone affected by the supposed problem.

Which I ofcourse notice in a followup of yours you are not either -- you're 
just here to blabber, not to solve anything.

Rene.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Björn Steinbrink]

On 2007.07.27 20:16:32 +0200, Rene Herman wrote:
> On 07/27/2007 07:45 PM, Daniel Hazelton wrote:
>
>> Updatedb or another process that uses the FS heavily runs on a users
>> 256MB P3-800 (when it is idle) and the VFS caches grow, causing memory
>> pressure that causes other applications to be swapped to disk. In the
>> morning the user has to wait for the system to swap those applications
>> back in.
>> Questions about it:
>> Q) Does swap-prefetch help with this? A) [From all reports I've seen (*)] 
>> Yes, it does. 
>
> No it does not. If updatedb filled memory to the point of causing swapping 
> (which noone is reproducing anyway) it HAS FILLED MEMORY and swap-prefetch 
> hasn't any memory to prefetch into -- updatedb itself doesn't use any 
> significant memory.
>
> Here's swap-prefetch's author saying the same:
>
> http://lkml.org/lkml/2007/2/9/112
>
> | It can't help the updatedb scenario. Updatedb leaves the ram full and
> | swap prefetch wants to cost as little as possible so it will never
> | move anything out of ram in preference for the pages it wants to swap
> | back in.
>
> Now please finally either understand this, or tell us how we're wrong.

Con might have been wrong there for boxes with really little memory.

My desktop box has not even 300k inodes in use (IIRC someone posted a df
-i output showing 1 million inodes in use). Still, the memory footprint
of the "sort" process grows up to about 50MB. Assuming that the average
filename length stays, that would mean 150MB for the 1 million inode
case, just for the "sort" process.

Now, sort cannot produce any output before its got all its input, so
that RSS usage exists at least as long as the VFS cache is growing due
to the ongoing search for files.

And then, all that memory that "sort" uses is required, because sort
needs to output its results. So if there's memory pressure, the VFS
cache is likely to be dropped, because "sort" needs its data, for
sorting and producing output. And then sort terminates and leaves that
whole lot of memory _unused_. The other actions of updatedb only touch
the locate db, which is just a few megs (4.5MB here) big so the cache
won't grow that much again.

OK, so we got about, say, at least 128MB of totally unused memory, maybe
even more. If you look at the vmstat output I sent, you see that I had
between 90MB and 128MB free, depending on the swappiness setting, with
increased inode usage, that could very well scale up.

Conclusion: updatedb does _not_ leave the RAM full. And for a box with
little memory (say 256MB) it might even be 50% or more memory that is
free after updatedb ran. Might that make swap prefetch kick in?


Any faults in that reasoning?

Thanks,
Bjorn

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Andi Kleen]

> Any faults in that reasoning?

GNU sort uses a merge sort with temporary files on disk. Not sure
how much it keeps in memory during that, but it's probably less
than 150MB. At some point the dirty limit should kick in and write back the 
data of the temporary files; so it's not quite the same as anonymous memory. 
But it's not that different given.

It would be better to measure than to guess. At least Andrew's measurements
on 128MB actually didn't show updatedb being really that big a problem.

Perhaps some people have much more files or simply a less efficient
updatedb implementation?

I guess the people who complain here that loudly really need to supply
some real numbers. 

-Andi

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Björn Steinbrink]

On 2007.07.28 01:29:19 +0200, Andi Kleen wrote:
> > Any faults in that reasoning?
> 
> GNU sort uses a merge sort with temporary files on disk. Not sure
> how much it keeps in memory during that, but it's probably less
> than 150MB. At some point the dirty limit should kick in and write back the 
> data of the temporary files; so it's not quite the same as anonymous memory. 
> But it's not that different given.

Hm, does that change anything? The files need to be read at the end (so
they go into the cache) and are delete afterwards (cache gets freed I
guess?).

> It would be better to measure than to guess. At least Andrew's measurements
> on 128MB actually didn't show updatedb being really that big a problem.

Here's a before/after memory usage for an updatedb run:
root@atjola:~# free -m
             total       used       free     shared    buffers     cached
Mem:          2011       1995         15          0        269        779
-/+ buffers/cache:        946       1064
Swap:         1945          0       1945
root@atjola:~# updatedb
root@atjola:~# free -m
             total       used       free     shared    buffers     cached
Mem:          2011       1914         96          0        209        746
-/+ buffers/cache:        958       1052
Swap:         1945          0       1944

81MB more unused RAM afterwards.

If anyone can make use of that, here's a snippet from /proc/$PID/smaps
of updatedb's sort process, when it was at about its peak memory usage
(according to the RSS column in top), which was about 50MB.

2b90ab3c1000-2b90ae4c3000 rw-p 2b90ab3c1000 00:00 0 
Size:              50184 kB
Rss:               50184 kB
Shared_Clean:          0 kB
Shared_Dirty:          0 kB
Private_Clean:         0 kB
Private_Dirty:     50184 kB
Referenced:        50184 kB

> Perhaps some people have much more files or simply a less efficient
> updatedb implementation?

sort (GNU coreutils) 5.97

GNU updatedb version 4.2.31

> I guess the people who complain here that loudly really need to supply
> some real numbers. 

Just to clarify: I'm not complaining either way, neither about not
merging swap prefetch, nor about someone wanting that to be merge. It
was rather the "discussion" that caught my attention... Just in case ;-)

Bjorn

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Daniel Hazelton]

On Friday 27 July 2007 19:29:19 Andi Kleen wrote:
> > Any faults in that reasoning?
>
> GNU sort uses a merge sort with temporary files on disk. Not sure
> how much it keeps in memory during that, but it's probably less
> than 150MB. At some point the dirty limit should kick in and write back the
> data of the temporary files; so it's not quite the same as anonymous
> memory. But it's not that different given.

Yes, this should occur. But how many programs use temporary files like that? 
>From what I can tell FireFox and OpenOffice both keep all their data in 
memory, only using a single file for some buffering purposes. When they get 
pushed out by a memory hog (either short term or long term) it takes several 
seconds for them to be swapped back in. (I'm on a P4-1.3GHz machine with 1G 
of ram and rarely run more than four programs (Mail Client, XChat, FireFox 
and a console window) and I've seen this lag in FireFox when switching to it 
after starting OOo. I've also seen the same sort of lag when exiting OOo. 
I'll see about getting some numbers about this)

> It would be better to measure than to guess. At least Andrew's measurements
> on 128MB actually didn't show updatedb being really that big a problem.

I agree. As I've said previously, it isn't updatedb itself which causes the 
problem. It's the way the VFS cache seems to just expand and expand - to the 
point of evicting pages to make room for itself. However, I may be wrong 
about that - I haven't actually tested it for myself, just looked at the 
numbers and other information that has been posted in this thread.

> Perhaps some people have much more files or simply a less efficient
> updatedb implementation?

Yes, it could be the proliferation of files. It could also be some other sort 
of problem that is exposing a corner-case in the VFS cache or the MM. I, 
personally, am of the opinion that it is likely the aforementioned corner 
case for people reporting the "updatedb" problem. If it is, then 
swap-prefetch is just papering over the problem. However I do not have the 
knowledge and understanding of the subsystems involved to be able to do much 
more than make a (probably wrong) guess.

> I guess the people who complain here that loudly really need to supply
> some real numbers.

I've seen numerous "real numbers" posted about this. As was said earlier in 
the thread "every time numbers are posted they are claimed to be no good". 
But hey, nobodies perfect :)

Anyway, the discussion seems to be turning to the technical merits of 
swap-prefetch...

Now, a completely different question:
During the research (and lots of thinking) I've been doing while this thread 
has been going on I've often wondered why swap prefetch wasn't already in the 
kernel. The problem of slow swap-in has long been known, and, given current 
hardware, the optimal solution would be some sort of data prefetch - similar 
to what is done to speed up normal disk reads. Swap prefetch looks like it 
does exactly that. The algo could be argued over and/or improved (to suggest 
ways to do that I'd have to give it more than a 10 minute look) but it does 
provide a speed-up.

This speed increase will probably be enjoyed more by the home users, but the 
performance increase could also help on enterprise systems.

Now I'll be the first one to admit that there is a trade-off there - it will 
cause more power to be used because the disk's don't get a chance to spin 
down (or go through a cycle every time the prefetch system starts) but that 
could, potentially, be alleviated by having "laptop mode" switch it off.

(And no, I'm not claiming that it is perfect - but then, what is when its 
first merged into the kernel?)

DRH

-- 
Dialup is like pissing through a pipette. Slow and excruciatingly painful.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Paul Jackson]

Andi wrote:
> GNU sort uses a merge sort with temporary files on disk. Not sure
> how much it keeps in memory during that, but it's probably less
> than 150MB. 

If I'm reading the source code for GNU sort correctly, then the
following snippet of shell code displays how much memory it uses
for its primary buffer on typical GNU/Linux systems:

    head -2 /proc/meminfo | awk '
	NR == 1 { memtotal = $2 }
	NR == 2 { memfree = $2 }
	END     {
		   if (memfree > memtotal/8)
		       m = memfree
		   else
		       m = memtotal/8
		   print "sort size:", m/2, "kB"
	}
    '

That is, over simplifying, GNU sort looks at the first two entries
in /proc/meminfo, which for example on a machine near me happen to be:

  MemTotal:      2336472 kB
  MemFree:        110600 kB

and then uses one-half of whichever is -greater- of MemTotal/8 or
MemFree.

... However ... for the typical GNU locate updatedb run, it is sorting
the list of pathnames for almost all files on the system, which is
usually larger than fits in one of these sized buffers.   So it ends up
using quite a few of the temporary files you mention, which tends to
chew up easily freed memory.

-- 
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <pj@sgi.com> 1.925.600.0401

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Rene Herman]

On 07/28/2007 01:15 AM, Bjorn Steinbrink wrote:

> On 2007.07.27 20:16:32 +0200, Rene Herman wrote:

>> Here's swap-prefetch's author saying the same:
>>
>> http://lkml.org/lkml/2007/2/9/112
>>
>> | It can't help the updatedb scenario. Updatedb leaves the ram full and
>> | swap prefetch wants to cost as little as possible so it will never
>> | move anything out of ram in preference for the pages it wants to swap
>> | back in.
>>
>> Now please finally either understand this, or tell us how we're wrong.
> 
> Con might have been wrong there for boxes with really little memory.

Note -- with "the updatedb scenario" both he in the above and I are talking 
about the "VFS caches filling memory cause the problem" not updatedb in 
particular.

> My desktop box has not even 300k inodes in use (IIRC someone posted a df 
> -i output showing 1 million inodes in use). Still, the memory footprint 
> of the "sort" process grows up to about 50MB. Assuming that the average 
> filename length stays, that would mean 150MB for the 1 million inode 
> case, just for the "sort" process.

Even if it's not 150MB, 50MB is already a lot on a 128 or even a 256MB box. 
So, yes, we're now at the expected scenario of some hog pushing out things 
and freeing it upon exit again and it's something swap-prefetch definitely 
has potential to help with.

Said early in the thread it's hard to imagine how it would not help in any 
such situation so that the discussion may as far as I'm concerned at that 
point concentrate on whether swap-prefetch hurts anything in others.

Some people I believe are not convinced it helps very significantly due to 
at that point _everything_ having been thrown out but a copy of openoffice 
with a large spreadsheet open should come back to life much quicker it would 
seem.

> Any faults in that reasoning?

No. If the machine goes idle after some memory hog _itself_ pushes things 
out and then exits, swap-prefetch helps, at the veryvery least potentially.

By the way -- I'm unable to make my slocate grow substantial here but I'll 
try what GNU locate does. If it's really as bad as I hear then regardless of 
anything else it should really be either fixed or dumped...

Rene.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Rene Herman]

On 07/28/2007 09:35 AM, Rene Herman wrote:

> By the way -- I'm unable to make my slocate grow substantial here but 
> I'll try what GNU locate does. If it's really as bad as I hear then 
> regardless of anything else it should really be either fixed or dumped...

Yes. GNU locate is broken and nobody should be using it. The updatedb from 
(my distribution standard) "slocate" uses around 2M allocated total during 
an entire run while GNU locate allocates some 30M to the sort process alone.

GNU locate is also close to 4 times as slow (although that ofcourse only 
matters on cached runs anyways).

So, GNU locate is just a pig pushing things out, with or without any added 
VFS cache pressure from the things it does by design. As such, we can trust 
people complaining about it but should first tell them to switch to halfway 
sane locate implementation. If you run memory hogs on small memory boxes, 
you're going to suffer.

Leaves the fact that swap-prefetch sometimes helps alleviate these and other 
kinds of memory-hog situations and as such, might not (again) be a bad idea 
in itself.

Rene.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Mike Galbraith]

On Fri, 2007-07-27 at 13:45 -0400, Daniel Hazelton wrote:
> On Friday 27 July 2007 06:25:18 Mike Galbraith wrote:
> > On Fri, 2007-07-27 at 03:00 -0700, Andrew Morton wrote:

> > > So hrm.  Are we sure that updatedb is the problem?  There are quite a few
> > > heavyweight things which happen in the wee small hours.
> >
> > The balance in _my_ world seems just fine.  I don't let any of those
> > system maintenance things run while I'm using the system, and it doesn't
> > bother me if my working set has to be reconstructed after heavy-weight
> > maintenance things are allowed to run.  I'm not seeing anything I
> > wouldn't expect to see when running a job the size of updatedb.
> >
> > 	-Mike
> 
> Do you realize you've totally missed the point?

Did you notice that I didn't make one disparaging remark about the patch
or the concept behind it?   Did you notice that I took _my time_  to
test, to actually look at  the problem?  No, you're too busy running
your mouth to appreciate the efforts of others.
 
<snips load of useless spleen venting>

Do yourself a favor, go dig into the VM source.  Read it, understand it
(not terribly easy), _then_ come back and preach to me.

Have a nice day.

	-Mike

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Daniel Hazelton]

On Friday 27 July 2007 18:08:44 Mike Galbraith wrote:
> On Fri, 2007-07-27 at 13:45 -0400, Daniel Hazelton wrote:
> > On Friday 27 July 2007 06:25:18 Mike Galbraith wrote:
> > > On Fri, 2007-07-27 at 03:00 -0700, Andrew Morton wrote:
> > > > So hrm.  Are we sure that updatedb is the problem?  There are quite a
> > > > few heavyweight things which happen in the wee small hours.
> > >
> > > The balance in _my_ world seems just fine.  I don't let any of those
> > > system maintenance things run while I'm using the system, and it
> > > doesn't bother me if my working set has to be reconstructed after
> > > heavy-weight maintenance things are allowed to run.  I'm not seeing
> > > anything I wouldn't expect to see when running a job the size of
> > > updatedb.
> > >
> > > 	-Mike
> >
> > Do you realize you've totally missed the point?
>
> Did you notice that I didn't make one disparaging remark about the patch
> or the concept behind it?   Did you notice that I took _my time_  to
> test, to actually look at  the problem?  No, you're too busy running
> your mouth to appreciate the efforts of others.

If you're done being an ass, take note of the fact that I never even said you 
were doing that. What I was commenting on was the fact that you (and a lot of 
the other developers) seem to keep saying "It doesn't happen here, so it 
doesn't matter!" - ie: If I don't see something happening, it doesn't matter.

> <snips load of useless spleen venting>
>
> Do yourself a favor, go dig into the VM source.  Read it, understand it
> (not terribly easy), _then_ come back and preach to me.

I've been trying to do that since the thread started. Note that you snipped 
where I said (and I'm going to paraphrase myself) "There is another way to 
fix this, but I don't have the understanding necessary".

Now, once more, I'm going to ask: What is so terribly wrong with swap 
prefetch? Why does it seem that everyone against it says "Its treating a 
symptom, so it can't go in"?

Try coming up with an answer that isn't "I don't see the problem on my $10K 
system" or similar - try explaining it based on the *technical* merits. Does 
it cause the processor cache to get thrashed? Does it create locking 
problems?

I stand by my statements, as vitriolic as you and Rene seem to want to get 
over it. So far in this thread I have not seen one bit of *technical* 
discussion over the merits, just the bits I've simplified and stated before.

> Have a nice day.

I am. You being nasty when somebody gets fed up with a line of BS doesn't stop 
me from having a nice day. Only thing that could make my life any better 
would be to have the questions I've asked answered, rather than having 
supposedly intelligent people act like trolls.

DRH

-- 
Dialup is like pissing through a pipette. Slow and excruciatingly painful.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Mike Galbraith]

On Fri, 2007-07-27 at 18:51 -0400, Daniel Hazelton wrote:

> Now, once more, I'm going to ask: What is so terribly wrong with swap 
> prefetch? Why does it seem that everyone against it says "Its treating a 
> symptom, so it can't go in"?

And once again, I personally have nothing against swap-prefetch, or
something like it. I can see how it or something like it could be made
to improve the lives of people who get up in the morning to find their
apps sitting on disk due to memory pressure generated by over-night
system maintenance operations.

The author himself however, says his implementation can't help with
updatedb (though people seem to be saying that it does), or anything
else that leaves memory full.  That IMHO, makes it of questionable value
toward solving what people are saying they want swap-prefetch for in the
first place.

I personally don't care if swap-prefetch goes in or not.

	-Mike

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Daniel Hazelton]

On Saturday 28 July 2007 03:48:13 Mike Galbraith wrote:
> On Fri, 2007-07-27 at 18:51 -0400, Daniel Hazelton wrote:
> > Now, once more, I'm going to ask: What is so terribly wrong with swap
> > prefetch? Why does it seem that everyone against it says "Its treating a
> > symptom, so it can't go in"?
>
> And once again, I personally have nothing against swap-prefetch, or
> something like it. I can see how it or something like it could be made
> to improve the lives of people who get up in the morning to find their
> apps sitting on disk due to memory pressure generated by over-night
> system maintenance operations.
>
> The author himself however, says his implementation can't help with
> updatedb (though people seem to be saying that it does), or anything
> else that leaves memory full.  That IMHO, makes it of questionable value
> toward solving what people are saying they want swap-prefetch for in the
> first place.

Okay. I have to agree with the author that, in such a situation, it wouldn't 
help. However there are, without a doubt, other situations where it would 
help immensely. (memory hogs forcing everything to disk and quitting, one off 
tasks that don't balloon the cache (kernel compiles, et al) - in those 
situations swap prefetch would really shine.)

DRH

-- 
Dialup is like pissing through a pipette. Slow and excruciatingly painful.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Rik van Riel]

Andrew Morton wrote:

> What I think is killing us here is the blockdev pagecache: the pagecache
> which backs those directory entries and inodes.  These pages get read
> multiple times because they hold multiple directory entries and multiple
> inodes.  These multiple touches will put those pages onto the active list
> so they stick around for a long time and everything else gets evicted.
> 
> I've never been very sure about this policy for the metadata pagecache.  We
> read the filesystem objects into the dcache and icache and then we won't
> read from that page again for a long time (I expect).  But the page will
> still hang around for a long time.
> 
> It could be that we should leave those pages inactive.

Good idea for updatedb.

However, it may be a bad idea for files that are often
written to.  Turning an inode write into a read plus a
write does not sound like such a hot idea, we really
want to keep those in the cache.

I think what you need is to ignore multiple references
to the same page when they all happen in one time
interval, counting them only if they happen in multiple
time intervals.

The use-once cleanup (which takes a page flag for PG_new,
I know...) would solve that problem.

However, it would introduce the problem of having to scan
all the pages on the list before a page becomes freeable.
We would have to add some background scanning (or a separate
list for PG_new pages) to make the initial pageout run use
an acceptable amount of CPU time.

Not sure that complexity will be worth it...

-- 
Politics is the struggle between those who want to make their country
the best in the world, and those who believe it already is.  Each group
calls the other unpatriotic.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Andrew Morton]

On Sat, 28 Jul 2007 21:33:59 -0400 Rik van Riel <riel@redhat.com> wrote:

> Andrew Morton wrote:
> 
> > What I think is killing us here is the blockdev pagecache: the pagecache
> > which backs those directory entries and inodes.  These pages get read
> > multiple times because they hold multiple directory entries and multiple
> > inodes.  These multiple touches will put those pages onto the active list
> > so they stick around for a long time and everything else gets evicted.
> > 
> > I've never been very sure about this policy for the metadata pagecache.  We
> > read the filesystem objects into the dcache and icache and then we won't
> > read from that page again for a long time (I expect).  But the page will
> > still hang around for a long time.
> > 
> > It could be that we should leave those pages inactive.
> 
> Good idea for updatedb.
> 
> However, it may be a bad idea for files that are often
> written to.  Turning an inode write into a read plus a
> write does not sound like such a hot idea, we really
> want to keep those in the cache.

Remember that this problem applies to both inode blocks and to directory
blocks.  Yes, it might be useful to hold onto an inode block for a future
write (atime, mtime, usually), but not a directory block.

> I think what you need is to ignore multiple references
> to the same page when they all happen in one time
> interval, counting them only if they happen in multiple
> time intervals.

Yes, the sudden burst of accesses for adjacent inode/dirents will be a
common pattern, and it'd make heaps of sense to treat that as a single
touch.  It'd have to be done in the fs I guess, and it might be a bit hard
to do.  And it turns out that embedding the touch_buffer() all the way down
in __find_get_block() was convenient, but it's going to be tricky to
change.

For now I'm fairly inclined to just nuke the touch_buffer() on the read side
and maybe add one on the modification codepaths and see what happens.

As always, testing is the problem.

> The use-once cleanup (which takes a page flag for PG_new,
> I know...) would solve that problem.
> 
> However, it would introduce the problem of having to scan
> all the pages on the list before a page becomes freeable.
> We would have to add some background scanning (or a separate
> list for PG_new pages) to make the initial pageout run use
> an acceptable amount of CPU time.
> 
> Not sure that complexity will be worth it...
> 

I suspect that the situation we have now is so bad that pretty much
anything we do will be an improvement.  I've always wondered "ytf is there
so much blockdev pagecache?"

This machine I'm typing at:

MemTotal:      3975080 kB
MemFree:        750400 kB
Buffers:        547736 kB
Cached:        1299532 kB
SwapCached:      12772 kB
Active:        1789864 kB
Inactive:       861420 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:      3975080 kB
LowFree:        750400 kB
SwapTotal:     4875716 kB
SwapFree:      4715660 kB
Dirty:              76 kB
Writeback:           0 kB
Mapped:         638036 kB
Slab:           522724 kB
CommitLimit:   6863256 kB
Committed_AS:  1115632 kB
PageTables:      14452 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     36432 kB
VmallocChunk: 34359696379 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

More that a quarter of my RAM in fs metadata!  Most of it I'll bet is on the
active list.  And the fs on which I do most of the work is mounted
noatime..


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Al Viro]

On Thu, Jul 26, 2007 at 11:40:24AM +0200, Ingo Molnar wrote:
> below is an updatedb hack that sets vfs_cache_pressure down to 0 during 
> an updatedb run. Could someone who is affected by the 'morning after' 
> problem give it a try? If this works then we can think about any other 
> measures ...

BTW, I really wonder how much pain could be avoided if updatedb recorded
mtime of directories and checked it.  I.e. instead of just doing blind
find(1), walk the stored directory tree comparing timestamps with those
in filesystem.  If directory mtime has not changed, don't bother rereading
it and just go for (stored) subdirectories.  If it has changed - reread the
sucker.  If we have a match for stored subdirectory of changed directory,
check inumber; if it doesn't match, consider the entire subtree as new
one.  AFAICS, that could eliminate quite a bit of IO...

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Andi Kleen]

> BTW, I really wonder how much pain could be avoided if updatedb recorded
> mtime of directories and checked it.  I.e. instead of just doing blind
> find(1), walk the stored directory tree comparing timestamps with those
> in filesystem.  If directory mtime has not changed, don't bother rereading
> it and just go for (stored) subdirectories.  If it has changed - reread the
> sucker.  If we have a match for stored subdirectory of changed directory,
> check inumber; if it doesn't match, consider the entire subtree as new
> one.  AFAICS, that could eliminate quite a bit of IO...

That would just save reading the directories. Not sure
it helps that much. Much better would be actually if it didn't stat the 
individual files (and force their dentries/inodes in). I bet it does that to 
find out if they are directories or not. But in a modern system it could just 
check the type in the dirent on file systems that support 
that and not do a stat. Then you would get much less dentries/inodes.

Also I expect in general the new slub dcache freeing that is pending
will improve things a lot.

But even if updatedb was fixed to be more efficient we probably
still need a general solution for other tree walking programs
that cannot be optimized this way.

-Andi

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Al Viro]

On Thu, Jul 26, 2007 at 02:23:30PM +0200, Andi Kleen wrote:
> That would just save reading the directories. Not sure
> it helps that much. Much better would be actually if it didn't stat the 
> individual files (and force their dentries/inodes in). I bet it does that to 
> find out if they are directories or not. But in a modern system it could just 
> check the type in the dirent on file systems that support 
> that and not do a stat. Then you would get much less dentries/inodes.
 
FWIW, find(1) does *not* stat non-directories (and neither would this
approach).  So it's just dentries for directories and you can't realistically
skip those.  OK, you could - if you had banned cross-directory rename
for directories and propagated "dirty since last look" towards root (note
that it would be a boolean, not a timestamp).  Then we could skip unchanged
subtrees completely...

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Pavel Machek]

Hi!

> > That would just save reading the directories. Not sure
> > it helps that much. Much better would be actually if it didn't stat the 
> > individual files (and force their dentries/inodes in). I bet it does that to 
> > find out if they are directories or not. But in a modern system it could just 
> > check the type in the dirent on file systems that support 
> > that and not do a stat. Then you would get much less dentries/inodes.
>  
> FWIW, find(1) does *not* stat non-directories (and neither would this
> approach).  So it's just dentries for directories and you can't realistically
> skip those.  OK, you could - if you had banned cross-directory rename
> for directories and propagated "dirty since last look" towards root (note
> that it would be a boolean, not a timestamp).  Then we could skip unchanged
> subtrees completely...

Could we help it a little from kernel and set 'dirty since last look'
on directory renames?

I mean, this is not only updatedb. KDE startup is limited by this,
too. It would be nice to have effective 'what change in tree'
operation.
							Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Paul Jackson]

Al Viro wrote:
> BTW, I really wonder how much pain could be avoided if updatedb recorded
> mtime of directories and checked it.

Someone mentioned a variant of slocate above that they called mlocate,
and that Red Hat ships, that seems to do this (if I understand you and
what mlocate does correctly.)

-- 
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <pj@sgi.com> 1.925.600.0401

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: RFT: updatedb "morning after" problem [was: Re: -mm merge plans for 2.6.23]

[Fredrik Klasson]

[-- Attachment #1.1: Type: text/plain, Size: 1796 bytes --]

On 7/26/07, Ingo Molnar <mingo@elte.hu> wrote:
>
> --- /etc/cron.daily/mlocate.cron.orig
> +++ /etc/cron.daily/mlocate.cron
> @@ -1,4 +1,7 @@
> #!/bin/sh
> nodevs=$(< /proc/filesystems awk '$1 == "nodev" { print $2 }')
> renice +19 -p $$ >/dev/null 2>&1
> +PREV=`cat /proc/sys/vm/vfs_cache_pressure 2>/dev/null`
> +echo 0 > /proc/sys/vm/vfs_cache_pressure 2>/dev/null
> /usr/bin/updatedb -f "$nodevs"
> +[ "$PREV" != "" ] && echo $PREV > /proc/sys/vm/vfs_cache_pressure
> 2>/dev/null
> _______________________________________________
> http://ck.kolivas.org/faqs/replying-to-mailing-list.txt
> ck mailing list - mailto: ck@vds.kolivas.org
> http://vds.kolivas.org/mailman/listinfo/ck


uhm... pardon my ignorance, but, doesn't this hack create a possible race
condition?
Ie, this job starts, and while updatedb runs some other app/script (let's
call it Gort) pokes with vfs_cache_pressure (saving 10000, as it's the
current value), then updatedb finishes, and then a while after that Gort
stops, "restoring" vfs_cache_pressure to 10000 instead of $PREV?

What we _really_ want is an updatedb that
> does not disturb the dcache.
>
just a thought, the problem seems (to me, a mere mortal at best) to be that
the functions/program doing the magic messes up caches. Wouldn't the
easiest/obvious solution/hack be to use/write an "uncached_" version of
those functions/apps? (or one that just reads the cache, but never upsets it
by writing to it). Or is that impossible/impractical for some reason? (or
worse yet, have I completely missed the point?)

-- 
... a professor saying: "use this proprietary software to learn computer
science" is the same as English professor handing you a copy of Shakespeare
and saying: "use this book to learn Shakespeare without opening the book
itself.
- Bradley Kuhn

[-- Attachment #1.2: Type: text/html, Size: 2575 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: [ck] Re: -mm merge plans for 2.6.23

[Matthew Hawkins]

[-- Attachment #1: Type: text/plain, Size: 1199 bytes --]

On 7/25/07, Nick Piggin <nickpiggin@yahoo.com.au> wrote:
> I guess /proc/meminfo, /proc/zoneinfo, /proc/vmstat, /proc/slabinfo
> before and after the updatedb run with the latest kernel would be a
> first step. top and vmstat output during the run wouldn't hurt either.

Hi Nick,

I've attached two files with this kind of info.  Being up at the cron
hours of the morning meant I got a better picture of what my system is
doing.  Here's a short summary of what I saw in top:

beagleindexer used gobs of ram.  600M or so (I have 1G)

updatedb didn't use much ram, but while it was running kswapd kept on
frequenting the top 10 cpu hogs - it would stick around for 5 seconds
or so then disappear for no more than 10 seconds, then come back
again.  This behaviour persisted during the run.  updatedb ran third
(beagleindexer was first, then update-dlocatedb)

I'm going to do this again, this time under a CFS kernel & use Ingo's
sched_debug script to see what the scheduler is doing also.

Let me know if there's anything else you wish to see.  The running
kernel at the time was 2.6.22.1-ck.  There's no slabinfo since I'm
using slub instead (and I don't have slub debug enabled).

Cheers,

-- 
Matt

[-- Attachment #2: beaglecron.ck --]
[-- Type: application/octet-stream, Size: 4539 bytes --]

MemTotal:      1028016 kB
MemFree:          9368 kB
Buffers:         56932 kB
Cached:         115820 kB
SwapCached:      19968 kB
Active:         463284 kB
Inactive:       418952 kB
SwapTotal:     2096472 kB
SwapFree:      1855632 kB
Dirty:            2272 kB
Writeback:           0 kB
AnonPages:      695744 kB
Mapped:          42732 kB
Slab:            89452 kB
SReclaimable:    74204 kB
SUnreclaim:      15248 kB
PageTables:      14656 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   2610480 kB
Committed_AS:  1478988 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    313292 kB
VmallocChunk: 34359424507 kB
nr_free_pages 3415
nr_inactive 102651
nr_active 115308
nr_anon_pages 169943
nr_mapped 10351
nr_file_pages 49521
nr_dirty 774
nr_writeback 0
nr_slab_reclaimable 20088
nr_slab_unreclaimable 3898
nr_page_table_pages 3664
nr_unstable 0
nr_bounce 0
nr_vmscan_write 64203
pgpgin 595981
pgpgout 472600
pswpin 3408
pswpout 63208
pgalloc_dma 3
pgalloc_dma32 1976896
pgalloc_normal 0
pgfree 1980436
pgactivate 117968
pgdeactivate 213723
pgfault 3468419
pgmajfault 3466
pgrefill_dma 0
pgrefill_dma32 562570
pgrefill_normal 0
pgsteal_dma 0
pgsteal_dma32 173803
pgsteal_normal 0
pgscan_kswapd_dma 0
pgscan_kswapd_dma32 241568
pgscan_kswapd_normal 0
pgscan_direct_dma 0
pgscan_direct_dma32 21792
pgscan_direct_normal 0
pginodesteal 3
slabs_scanned 455552
kswapd_steal 162131
kswapd_inodesteal 2519
pageoutrun 2497
allocstall 159
pgrotated 63202
00000000-0009f7ff : System RAM
  00000000-00000000 : Crash kernel
0009f800-0009ffff : reserved
000cc800-000cffff : pnp 00:0c
000f0000-000fffff : reserved
00100000-3ffeffff : System RAM
  00200000-0043db5f : Kernel code
  0043db60-0056f6cf : Kernel data
3fff0000-3fff2fff : ACPI Non-volatile Storage
3fff3000-3fffffff : ACPI Tables
d0000000-dfffffff : reserved
e0000000-efffffff : PCI Bus #05
  e0000000-efffffff : 0000:05:00.0
f0000000-f3ffffff : PCI Bus #05
  f0000000-f1ffffff : 0000:05:00.0
  f2000000-f2ffffff : 0000:05:00.0
    f2000000-f2ffffff : nvidia
  f3000000-f301ffff : 0000:05:00.0
f4000000-f4000fff : 0000:00:04.0
  f4000000-f4000fff : NVidia CK804
f4001000-f4001fff : 0000:00:07.0
  f4001000-f4001fff : sata_nv
f4002000-f4002fff : 0000:00:08.0
  f4002000-f4002fff : sata_nv
f4003000-f4003fff : 0000:00:0a.0
  f4003000-f4003fff : forcedeth
f4004000-f4004fff : 0000:00:02.0
  f4004000-f4004fff : ohci_hcd
feb00000-feb000ff : 0000:00:02.1
  feb00000-feb000ff : ehci_hcd
fec00000-fec00fff : IOAPIC 0
fee00000-fee00fff : Local APIC
Node 0, zone      DMA
  pages free     736
        min      11
        low      13
        high     16
        scanned  0 (a: 9 i: 9)
        spanned  4096
        present  2858
    nr_free_pages 736
    nr_inactive  0
    nr_active    0
    nr_anon_pages 0
    nr_mapped    1
    nr_file_pages 0
    nr_dirty     0
    nr_writeback 0
    nr_slab_reclaimable 0
    nr_slab_unreclaimable 2
    nr_page_table_pages 0
    nr_unstable  0
    nr_bounce    0
    nr_vmscan_write 0
        protection: (0, 994, 994)
  pagesets
    cpu: 0 pcp: 0
              count: 0
              high:  0
              batch: 1
    cpu: 0 pcp: 1
              count: 0
              high:  0
              batch: 1
  vm stats threshold: 4
    cpu: 1 pcp: 0
              count: 0
              high:  0
              batch: 1
    cpu: 1 pcp: 1
              count: 0
              high:  0
              batch: 1
  vm stats threshold: 4
  all_unreclaimable: 1
  prev_priority:     12
  start_pfn:         0
Node 0, zone    DMA32
  pages free     1786
        min      1002
        low      1252
        high     1503
        scanned  0 (a: 29 i: 23)
        spanned  258032
        present  254505
    nr_free_pages 1786
    nr_inactive  93406
    nr_active    118044
    nr_anon_pages 154539
    nr_mapped    9819
    nr_file_pages 58466
    nr_dirty     115
    nr_writeback 0
    nr_slab_reclaimable 27261
    nr_slab_unreclaimable 4205
    nr_page_table_pages 3664
    nr_unstable  0
    nr_bounce    0
    nr_vmscan_write 78966
        protection: (0, 0, 0)
  pagesets
    cpu: 0 pcp: 0
              count: 29
              high:  186
              batch: 31
    cpu: 0 pcp: 1
              count: 3
              high:  62
              batch: 15
  vm stats threshold: 16
    cpu: 1 pcp: 0
              count: 2
              high:  186
              batch: 31
    cpu: 1 pcp: 1
              count: 13
              high:  62
              batch: 15
  vm stats threshold: 16
  all_unreclaimable: 0
  prev_priority:     12
  start_pfn:         4096

[-- Attachment #3: updatedbcron.ck --]
[-- Type: application/octet-stream, Size: 5382 bytes --]

nr_free_pages 3478
nr_inactive 81057
nr_active 131617
nr_anon_pages 92818
nr_mapped 9092
nr_file_pages 130427
nr_dirty 4145
nr_writeback 0
nr_slab_reclaimable 22106
nr_slab_unreclaimable 5950
nr_page_table_pages 3572
nr_unstable 0
nr_bounce 0
nr_vmscan_write 145131
pgpgin 1305961
pgpgout 1321420
pswpin 21001
pswpout 144028
pgalloc_dma 3
pgalloc_dma32 2649338
pgalloc_normal 0
pgfree 2652884
pgactivate 251412
pgdeactivate 252293
pgfault 3565858
pgmajfault 5864
pgrefill_dma 0
pgrefill_dma32 836934
pgrefill_normal 0
pgsteal_dma 0
pgsteal_dma32 352216
pgsteal_normal 0
pgscan_kswapd_dma 0
pgscan_kswapd_dma32 510400
pgscan_kswapd_normal 0
pgscan_direct_dma 0
pgscan_direct_dma32 22112
pgscan_direct_normal 0
pginodesteal 3
slabs_scanned 809088
kswapd_steal 340352
kswapd_inodesteal 7286
pageoutrun 4661
allocstall 161
pgrotated 144024
00000000-0009f7ff : System RAM
  00000000-00000000 : Crash kernel
0009f800-0009ffff : reserved
000cc800-000cffff : pnp 00:0c
000f0000-000fffff : reserved
00100000-3ffeffff : System RAM
  00200000-0043db5f : Kernel code
  0043db60-0056f6cf : Kernel data
3fff0000-3fff2fff : ACPI Non-volatile Storage
3fff3000-3fffffff : ACPI Tables
d0000000-dfffffff : reserved
e0000000-efffffff : PCI Bus #05
  e0000000-efffffff : 0000:05:00.0
f0000000-f3ffffff : PCI Bus #05
  f0000000-f1ffffff : 0000:05:00.0
  f2000000-f2ffffff : 0000:05:00.0
    f2000000-f2ffffff : nvidia
  f3000000-f301ffff : 0000:05:00.0
f4000000-f4000fff : 0000:00:04.0
  f4000000-f4000fff : NVidia CK804
f4001000-f4001fff : 0000:00:07.0
  f4001000-f4001fff : sata_nv
f4002000-f4002fff : 0000:00:08.0
  f4002000-f4002fff : sata_nv
f4003000-f4003fff : 0000:00:0a.0
  f4003000-f4003fff : forcedeth
f4004000-f4004fff : 0000:00:02.0
  f4004000-f4004fff : ohci_hcd
feb00000-feb000ff : 0000:00:02.1
  feb00000-feb000ff : ehci_hcd
fec00000-fec00fff : IOAPIC 0
fee00000-fee00fff : Local APIC
MemTotal:      1028016 kB
MemFree:         26000 kB
Buffers:        352752 kB
Cached:          98672 kB
SwapCached:      54272 kB
Active:         513476 kB
Inactive:       321336 kB
SwapTotal:     2096472 kB
SwapFree:      1523124 kB
Dirty:             884 kB
Writeback:           0 kB
AnonPages:      371432 kB
Mapped:          36368 kB
Slab:           116016 kB
SReclaimable:    91864 kB
SUnreclaim:      24152 kB
PageTables:      14288 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   2610480 kB
Committed_AS:  1424352 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    313292 kB
VmallocChunk: 34359424507 kB
nr_free_pages 4210
nr_inactive 81387
nr_active 129217
nr_anon_pages 92867
nr_mapped 9100
nr_file_pages 128320
nr_dirty 548
nr_writeback 0
nr_slab_reclaimable 23296
nr_slab_unreclaimable 6120
nr_page_table_pages 3572
nr_unstable 0
nr_bounce 0
nr_vmscan_write 145131
pgpgin 1356125
pgpgout 1348944
pswpin 21066
pswpout 144028
pgalloc_dma 3
pgalloc_dma32 2687673
pgalloc_normal 0
pgfree 2691918
pgactivate 257106
pgdeactivate 260491
pgfault 3566843
pgmajfault 5880
pgrefill_dma 0
pgrefill_dma32 859380
pgrefill_normal 0
pgsteal_dma 0
pgsteal_dma32 367095
pgsteal_normal 0
pgscan_kswapd_dma 0
pgscan_kswapd_dma32 525312
pgscan_kswapd_normal 0
pgscan_direct_dma 0
pgscan_direct_dma32 22112
pgscan_direct_normal 0
pginodesteal 3
slabs_scanned 828928
kswapd_steal 355231
kswapd_inodesteal 7286
pageoutrun 4897
allocstall 161
pgrotated 144024
Node 0, zone      DMA
  pages free     736
        min      11
        low      13
        high     16
        scanned  0 (a: 18 i: 18)
        spanned  4096
        present  2858
    nr_free_pages 736
    nr_inactive  0
    nr_active    0
    nr_anon_pages 0
    nr_mapped    1
    nr_file_pages 0
    nr_dirty     0
    nr_writeback 0
    nr_slab_reclaimable 0
    nr_slab_unreclaimable 2
    nr_page_table_pages 0
    nr_unstable  0
    nr_bounce    0
    nr_vmscan_write 0
        protection: (0, 994, 994)
  pagesets
    cpu: 0 pcp: 0
              count: 0
              high:  0
              batch: 1
    cpu: 0 pcp: 1
              count: 0
              high:  0
              batch: 1
  vm stats threshold: 4
    cpu: 1 pcp: 0
              count: 0
              high:  0
              batch: 1
    cpu: 1 pcp: 1
              count: 0
              high:  0
              batch: 1
  vm stats threshold: 4
  all_unreclaimable: 1
  prev_priority:     12
  start_pfn:         0
Node 0, zone    DMA32
  pages free     2846
        min      1002
        low      1252
        high     1503
        scanned  0 (a: 0 i: 20)
        spanned  258032
        present  254505
    nr_free_pages 2846
    nr_inactive  78337
    nr_active    131382
    nr_anon_pages 92885
    nr_mapped    9099
    nr_file_pages 127433
    nr_dirty     411
    nr_writeback 0
    nr_slab_reclaimable 24595
    nr_slab_unreclaimable 6314
    nr_page_table_pages 3572
    nr_unstable  0
    nr_bounce    0
    nr_vmscan_write 145131
        protection: (0, 0, 0)
  pagesets
    cpu: 0 pcp: 0
              count: 29
              high:  186
              batch: 31
    cpu: 0 pcp: 1
              count: 13
              high:  62
              batch: 15
  vm stats threshold: 16
    cpu: 1 pcp: 0
              count: 9
              high:  186
              batch: 31
    cpu: 1 pcp: 1
              count: 12
              high:  62
              batch: 15
  vm stats threshold: 16
  all_unreclaimable: 0
  prev_priority:     12
  start_pfn:         4096

Re: [ck] Re: -mm merge plans for 2.6.23

[Nick Piggin]

Matthew Hawkins wrote:
> On 7/25/07, Nick Piggin <nickpiggin@yahoo.com.au> wrote:
> 
>>I guess /proc/meminfo, /proc/zoneinfo, /proc/vmstat, /proc/slabinfo
>>before and after the updatedb run with the latest kernel would be a
>>first step. top and vmstat output during the run wouldn't hurt either.
> 
> 
> Hi Nick,
> 
> I've attached two files with this kind of info.  Being up at the cron
> hours of the morning meant I got a better picture of what my system is
> doing.  Here's a short summary of what I saw in top:
> 
> beagleindexer used gobs of ram.  600M or so (I have 1G)

Hmm OK, beagleindexer. I thought beagle didn't need frequent reindexing
because of inotify? Oh well...


> updatedb didn't use much ram, but while it was running kswapd kept on
> frequenting the top 10 cpu hogs - it would stick around for 5 seconds
> or so then disappear for no more than 10 seconds, then come back
> again.  This behaviour persisted during the run.  updatedb ran third
> (beagleindexer was first, then update-dlocatedb)

Kswapd will use CPU when memory is low, even if there is no swapping.

Your "buffers" grew by 600% (from 50MB to 350MB), and slab also grew
by a few thousand entries. This is not just a problem when it pushes
out swap, it will also harm filebacked working set.

This (which Ray's traces also show) is a bit of a problem. As Andrew
noticed, use-once isn't working well for buffer cache, and it doesn't
really for dentry and inode cache either (although those don't seem
to be as much of a problem on your workload).

Andrew has done a little test patch for this in -mm, but it probably
wants more work and testing. If you can test the -mm kernel and see
if things are improved, that would help.

-- 
SUSE Labs, Novell Inc.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[david]

On Tue, 24 Jul 2007, Ray Lee wrote:

> On 7/23/07, Nick Piggin <nickpiggin@yahoo.com.au> wrote:
>>  Ray Lee wrote:
>
>>  Looking at your past email, you have a 1GB desktop system and your
>>  overnight updatedb run is causing stuff to get swapped out such that
>>  swap prefetch makes it significantly better. This is really
>>  intriguing to me, and I would hope we can start by making this
>>  particular workload "not suck" without swap prefetch (and hopefully
>>  make it even better than it currently is with swap prefetch because
>>  we'll try not to evict useful file backed pages as well).
>
> updatedb is an annoying case, because one would hope that there would
> be a better way to deal with that highly specific workload. It's also
> pretty stat dominant, which puts it roughly in the same category as a
> git diff. (They differ in that updatedb does a lot of open()s and
> getdents on directories, git merely does a ton of lstat()s instead.)
>
> Anyway, my point is that I worry that tuning for an unusual and
> infrequent workload (which updatedb certainly is), is the wrong way to
> go.

updatedb pushing out program data may be able to be improved on with drop 
behind or similar.

however another scenerio that causes a similar problem is when a user is 
busy useing one of the big memory hogs and then switches to another (think 
switching between openoffice and firefox)

>>  After that we can look at other problems that swap prefetch helps
>>  with, or think of some ways to measure your "whole day" scenario.
>>
>>  So when/if you have time, I can cook up a list of things to monitor
>>  and possibly a patch to add some instrumentation over this updatedb
>>  run.
>
> That would be appreciated. Don't spend huge amounts of time on it,
> okay? Point me the right direction, and we'll see how far I can run
> with it.

you could make a synthetic test by writing a memory hog that allocates 3/4 
of your ram then pauses waiting for input and then randomly accesses the 
memory for a while (say randomly accessing 2x # of pages allocated) and 
then pausing again before repeating

run two of these, alternating which one is running at any one time. time 
how long it takes to do the random accesses.

the difference in this time should be a fair example of how much it would 
impact the user.

by the way, I've also seen comments on the Postgres performance mailing 
list about how slow linux is compared to other OS's in pulling data back 
in that's been pushed out to swap (not a factor on dedicated database 
machines, but a big factor on multi-purpose machines)

>>  Anyway, I realise swap prefetching has some situations where it will
>>  fundamentally outperform even the page replacement oracle. This is
>>  why I haven't asked for it to be dropped: it isn't a bad idea at all.
>
> <nod>
>
>>  However, if we can improve basic page reclaim where it is obviously
>>  lacking, that is always preferable. eg: being a highly speculative
>>  operation, swap prefetch is not great for power efficiency -- but we
>>  still want laptop users to have a good experience as well, right?
>
> Absolutely. Disk I/O is the enemy, and the best I/O is one you never
> had to do in the first place.

almost always true, however there is some amount of I/O that is free with 
todays drives (remember, they read the entire track into ram and then 
give you the sectors on the track that you asked for). and if you have a 
raid array this is even more true.

if you read one sector in from a raid5 array you have done all the same 
I/O that you would have to do to read in the entire stripe, but I don't 
believe that the current system will keep it all around if it exceeds the 
readahead limit.

so in many cases readahead may end up being significantly cheaper then you 
expect.

David Lang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Rene Herman]

[-- Attachment #1: Type: text/plain, Size: 760 bytes --]

On 07/25/2007 06:46 AM, david@lang.hm wrote:

> you could make a synthetic test by writing a memory hog that allocates 
> 3/4 of your ram then pauses waiting for input and then randomly accesses 
> the memory for a while (say randomly accessing 2x # of pages allocated) 
> and then pausing again before repeating

Something like this?

> run two of these, alternating which one is running at any one time. time 
> how long it takes to do the random accesses.
> 
> the difference in this time should be a fair example of how much it 
> would impact the user.

Notenotenote, not sure what you're going to show with it (times are simply 
as horrendous as I'd expect) but thought I'd try to inject something other 
than steaming cups of 4-letter beverages.

Rene.

[-- Attachment #2: hog.c --]
[-- Type: text/plain, Size: 974 bytes --]

/* gcc -W -Wall -o hog hog.c */

#include <stdlib.h>
#include <stdio.h>

#include <sys/time.h>
#include <unistd.h>

int main(void)
{
	int pages, pagesize, i;
	unsigned char *mem;
	struct timeval tv;
	
	pages = sysconf(_SC_PHYS_PAGES);
	if (pages < 0) {
		perror("_SC_PHYS_PAGES");
		return EXIT_FAILURE;
	}
	pages = (3 * pages) / 4;

	pagesize = sysconf(_SC_PAGESIZE);
	if (pagesize < 0) {
		perror("_SC_PAGESIZE");
		return EXIT_FAILURE;
	}

	mem = malloc(pages * pagesize);
	if (!mem) {
		fprintf(stderr, "out of memory\n");
		return EXIT_FAILURE;
	}
	for (i = 0; i < pages; i++)
		mem[i * pagesize] = 0;

	gettimeofday(&tv, NULL);
	srand((unsigned int)tv.tv_sec);

	while (1) {
		struct timeval start;

		getchar();

		gettimeofday(&start, NULL);
		for (i = 0; i < 2 * pages; i++)
			mem[(rand() / (RAND_MAX / pages + 1)) * pagesize] = 0;
		gettimeofday(&tv, NULL);

		timersub(&tv, &start, &tv);
		printf("%lu.%lu\n", tv.tv_sec, tv.tv_usec);
	}

	return EXIT_SUCCESS;
}

Re: -mm merge plans for 2.6.23

[david]

On Wed, 25 Jul 2007, Rene Herman wrote:

> On 07/25/2007 06:46 AM, david@lang.hm wrote:
>
>>  you could make a synthetic test by writing a memory hog that allocates 3/4
>>  of your ram then pauses waiting for input and then randomly accesses the
>>  memory for a while (say randomly accessing 2x # of pages allocated) and
>>  then pausing again before repeating
>
> Something like this?
>
>>  run two of these, alternating which one is running at any one time. time
>>  how long it takes to do the random accesses.
>>
>>  the difference in this time should be a fair example of how much it would
>>  impact the user.
>
> Notenotenote, not sure what you're going to show with it (times are simply as 
> horrendous as I'd expect) but thought I'd try to inject something other than 
> steaming cups of 4-letter beverages.

when the swap readahead is enabled does it make a significant difference 
in the time to do the random access?

if it does that should show a direct benifit of the patch in a simulation 
of a relativly common workflow (startup a memory hog like openoffice then 
try and go back to your prior work)

David Lang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Rene Herman]

On 07/25/2007 10:07 AM, david@lang.hm wrote:

> On Wed, 25 Jul 2007, Rene Herman wrote:

>> Something like this?

[ ... ]

> when the swap readahead is enabled does it make a significant difference 
> in the time to do the random access?

I don't use swap prefetch (nor -ck or -mm). If someone who has the patch 
applied waits to hit enter until swap prefetch has prefetched it all back in 
again, it certainly will.

Swap prefetch's potential to do larger reads back from swapspace than a 
random segfaulting app could well be very significant. Reads are dwarved by 
seeks. If this program does what you wanted, please use it to show us.

Rene.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[david]

On Wed, 25 Jul 2007, Rene Herman wrote:

> On 07/25/2007 10:07 AM, david@lang.hm wrote:
>
>>  On Wed, 25 Jul 2007, Rene Herman wrote:
>
>> >  Something like this?
>
> [ ... ]
>
>>  when the swap readahead is enabled does it make a significant difference
>>  in the time to do the random access?
>
> I don't use swap prefetch (nor -ck or -mm). If someone who has the patch 
> applied waits to hit enter until swap prefetch has prefetched it all back in 
> again, it certainly will.
>
> Swap prefetch's potential to do larger reads back from swapspace than a 
> random segfaulting app could well be very significant. Reads are dwarved by 
> seeks. If this program does what you wanted, please use it to show us.

I haven't used swap prefetch either, the call was put out for what could 
be used to test the performance, and I was suggesting a test.

if nobody else follows up on this I'll try to get some time to test it 
myself in a day or two.

David Lang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[david]

On Wed, 25 Jul 2007, david@lang.hm wrote:

> Subject: Re: -mm merge plans for 2.6.23
> 
> On Wed, 25 Jul 2007, Rene Herman wrote:
>
>>  On 07/25/2007 10:07 AM, david@lang.hm wrote:
>> 
>> >   On Wed, 25 Jul 2007, Rene Herman wrote:
>> 
>> > >   Something like this?
>>
>>  [ ... ]
>> 
>> >   when the swap readahead is enabled does it make a significant 
>> >   difference
>> >   in the time to do the random access?
>>
>>  I don't use swap prefetch (nor -ck or -mm). If someone who has the patch
>>  applied waits to hit enter until swap prefetch has prefetched it all back
>>  in again, it certainly will.
>>
>>  Swap prefetch's potential to do larger reads back from swapspace than a
>>  random segfaulting app could well be very significant. Reads are dwarved
>>  by seeks. If this program does what you wanted, please use it to show us.
>
> I haven't used swap prefetch either, the call was put out for what could be 
> used to test the performance, and I was suggesting a test.
>
> if nobody else follows up on this I'll try to get some time to test it myself 
> in a day or two.

this assumes that this isn't ruled an invalid test in the meantime.

in any case thanks for codeing this up so quickly.

David Lang

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Rene Herman]

On 07/25/2007 10:33 AM, david@lang.hm wrote:

>> I haven't used swap prefetch either, the call was put out for what 
>> could be used to test the performance, and I was suggesting a test.
>>
>> if nobody else follows up on this I'll try to get some time to test it 
>> myself in a day or two.
> 
> this assumes that this isn't ruled an invalid test in the meantime.

Let's save a little time and guess. While two instances of the hog are 
running no physical memory is free (as together they take up 1.5x physical) 
meaning that swap-prefetch wouldn't get a change to do anything and wouldn't 
make a difference. As such, the two instances test as you suggested would in 
fact not be testing anything it seems.

However, if you quit one, and idle long enough to continue with the other 
one until swap-prefetch prefetched all its memory back in, it should be a 
difference on the order of minutes, even total if swap prefetch fetched it 
back in without seeking al over swap-space, and "total" isn't applicable if 
the idle time really is free.

A program randomly touching single pages all over memory is a contrived 
worst case scenario and not a real-world issue. It is a boundary condition 
though, and it's simply quite impossible to think of any example where 
swap-prefetch would _not_ give you a snappier feeling machine after you've 
been idling.

So really the only question would seem to be -- does it hurt any if you have 
_not_ been?

Rene.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Ray Lee]

Hoo boy, lots of messages this morning.

(Al? I've added you to the CC: because of your swap-in vs swap-out
speed report from January. See below -- half-way down or so -- for
more detals.)

On 7/24/07, david@lang.hm <david@lang.hm> wrote:
> On Tue, 24 Jul 2007, Ray Lee wrote:
>
> > On 7/23/07, Nick Piggin <nickpiggin@yahoo.com.au> wrote:
> >>  Ray Lee wrote:
> >
> >>  Looking at your past email, you have a 1GB desktop system and your
> >>  overnight updatedb run is causing stuff to get swapped out such that
> >>  swap prefetch makes it significantly better. This is really
> >>  intriguing to me, and I would hope we can start by making this
> >>  particular workload "not suck" without swap prefetch (and hopefully
> >>  make it even better than it currently is with swap prefetch because
> >>  we'll try not to evict useful file backed pages as well).
> >
> > updatedb is an annoying case, because one would hope that there would
> > be a better way to deal with that highly specific workload. It's also
> > pretty stat dominant, which puts it roughly in the same category as a
> > git diff. (They differ in that updatedb does a lot of open()s and
> > getdents on directories, git merely does a ton of lstat()s instead.)
> >
> > Anyway, my point is that I worry that tuning for an unusual and
> > infrequent workload (which updatedb certainly is), is the wrong way to
> > go.
>
> updatedb pushing out program data may be able to be improved on with drop
> behind or similar.

Hmm, I thought drop-behind wasn't going to be able to target metadata?

> however another scenerio that causes a similar problem is when a user is
> busy useing one of the big memory hogs and then switches to another (think
> switching between openoffice and firefox)

Yes, and that was the core of my original report months ago. I'm
working for a while on one task, go to openoffice to view a report, or
gimp to tweak the colors on a photo before uploading it, and then go
back to my email and... and... and... there we go. The faults that
occur when I context switch is what's most annoying.

> >>  After that we can look at other problems that swap prefetch helps
> >>  with, or think of some ways to measure your "whole day" scenario.
> >>
> >>  So when/if you have time, I can cook up a list of things to monitor
> >>  and possibly a patch to add some instrumentation over this updatedb
> >>  run.
> >
> > That would be appreciated. Don't spend huge amounts of time on it,
> > okay? Point me the right direction, and we'll see how far I can run
> > with it.
>
> you could make a synthetic test by writing a memory hog that allocates 3/4
> of your ram then pauses waiting for input and then randomly accesses the
> memory for a while (say randomly accessing 2x # of pages allocated) and
> then pausing again before repeating

Con wrote a benchmark much like that. It showed measurable improvement
with swap prefetch.

> by the way, I've also seen comments on the Postgres performance mailing
> list about how slow linux is compared to other OS's in pulling data back
> in that's been pushed out to swap (not a factor on dedicated database
> machines, but a big factor on multi-purpose machines)

Yeah, akpm and... one of the usual suspects, had mentioned something
such as 2.6 is half the speed of 2.4 for swapin. (Let's see if I can
find a reference for that, it's been a year or more...) Okay,
misremembered. Swap in is half the speed of swap out (
http://lkml.org/lkml/2007/1/22/173 ). Al Boldi (added to the CC:, poor
sod), is the one who knows how to measure that, I'm guessing.

Al? How are you coming up with those figures? I'm interested in
reproducing it. It could be due to something stupid, such as the VM
faulting things out in reverse order or something...

> >>  Anyway, I realise swap prefetching has some situations where it will
> >>  fundamentally outperform even the page replacement oracle. This is
> >>  why I haven't asked for it to be dropped: it isn't a bad idea at all.
> >
> > <nod>
> >
> >>  However, if we can improve basic page reclaim where it is obviously
> >>  lacking, that is always preferable. eg: being a highly speculative
> >>  operation, swap prefetch is not great for power efficiency -- but we
> >>  still want laptop users to have a good experience as well, right?
> >
> > Absolutely. Disk I/O is the enemy, and the best I/O is one you never
> > had to do in the first place.
>
> almost always true, however there is some amount of I/O that is free with
> todays drives (remember, they read the entire track into ram and then
> give you the sectors on the track that you asked for). and if you have a
> raid array this is even more true.

Yeah, I knew I'd get called on that one :-). It's the seeks that'll
really kill you, and as you say once you're on the track the rest is
practically free (which is why the VM should prefer to evict larger
chunks at a time rather than lots of small things, see
http://lkml.org/lkml/2007/7/23/214 for something that's heading the
right direction, though the side-effects are unfortunate.

> if you read one sector in from a raid5 array you have done all the same
> I/O that you would have to do to read in the entire stripe, but I don't
> believe that the current system will keep it all around if it exceeds the
> readahead limit.

Fengguang Wu is doing lots of active work on making the readahead suck
less. Ping him and he'll likely take an active interest in the RAID
stuff.

Ray

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Al Boldi]

Ray Lee wrote:
> On 7/24/07, david@lang.hm <david@lang.hm> wrote:
> > by the way, I've also seen comments on the Postgres performance mailing
> > list about how slow linux is compared to other OS's in pulling data back
> > in that's been pushed out to swap (not a factor on dedicated database
> > machines, but a big factor on multi-purpose machines)
>
> Yeah, akpm and... one of the usual suspects, had mentioned something
> such as 2.6 is half the speed of 2.4 for swapin. (Let's see if I can
> find a reference for that, it's been a year or more...) Okay,
> misremembered. Swap in is half the speed of swap out (
> http://lkml.org/lkml/2007/1/22/173 ). Al Boldi (added to the CC:, poor
> sod), is the one who knows how to measure that, I'm guessing.
>
> Al? How are you coming up with those figures? I'm interested in
> reproducing it. It could be due to something stupid, such as the VM
> faulting things out in reverse order or something...

Thanks for asking.  I'm rather surprised why nobody's noticing any of this 
slowdown.  To be fair, it's not really a regression, on the contrary, 2.4 is 
lot worse wrt swapin and swapout, and Rik van Riel even considers a 50% 
swapin slowdown wrt swapout something like better than expected (see thread 
'[RFC] kswapd: Kernel Swapper performance').  He probably meant random 
swapin, which seems to offer a 4x slowdown.

There are two ways to reproduce this:

1. swsusp to disk reports ~44mb/s swapout, and ~25mb/s swapin during resume

2. tmpfs swapout is superfast, whereas swapin is really slow
(see thread '[PATCH] free swap space when (re)activating page')

Here is an excerpt from that thread (note machine config in first line):

============================================
 RAM 512mb , SWAP 1G
 #mount -t tmpfs -o size=1G none /dev/shm
 #time cat /dev/full > /dev/shm/x.dmp
 15sec
 #time cat /dev/shm/x.dmp > /dev/null
 58sec
 #time cat /dev/shm/x.dmp > /dev/null
 72sec
 #time cat /dev/shm/x.dmp > /dev/null
 85sec
 #time cat /dev/shm/x.dmp > /dev/null
 93sec
 #time cat /dev/shm/x.dmp > /dev/null
 99sec
============================================

As you can see, swapout is running full wirespeed, whereas swapin not only is 
4x slower, but increasingly gets the VM tangled up to end at a ~6x slowdown.

So again, I'm really surprised people haven't noticed.


Thanks!

--
Al

Re: -mm merge plans for 2.6.23

[Magnus Naeslund]

Al Boldi wrote:
> 
> Thanks for asking.  I'm rather surprised why nobody's noticing any of this 
> slowdown.  To be fair, it's not really a regression, on the contrary, 2.4 is 
> lot worse wrt swapin and swapout, and Rik van Riel even considers a 50% 
> swapin slowdown wrt swapout something like better than expected (see thread 
> '[RFC] kswapd: Kernel Swapper performance').  He probably meant random 
> swapin, which seems to offer a 4x slowdown.
> 

Sorry for the late reply.
Well I think I reported this or another swap/tmpfs performance issue earlier ( http://marc.info/?t=116542915700004&r=1&w=2 ), we got the suggestion to increase /proc/sys/vm/page-cluster to 5, but we never came around to try it.
Maybe this was the reason for my report to be almost entirely ignored, sorry for that.

Regards,
Magnus

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Andrew Morton]

On Mon, 23 Jul 2007 21:53:38 -0700 "Ray Lee" <ray-lk@madrabbit.org> wrote:

> 
> Since this merge period has appeared particularly frazzling for
> Andrew, I've been keeping silent and waiting for him to get to a point
> where there's a breather. I didn't feel it would be polite to request
> yet more work out of him while he had a mess on his hands.

Let it just be noted that Con is not the only one who has expended effort
on this patch.  It's been in -mm for nearly two years and it has meant
ongoing effort for me and, to a lesser extent, other MM developers to keep
it alive.

> But, given this has come to a head, I'm asking now.
> 
> Andrew? You've always given the impression that you want this run more
> as an engineering effort than an artistic endeavour, so help us out
> here. What are your concerns with swap prefetch? What sort of
> comparative data would you like to see to justify its inclusion, or to
> prove that it's not needed?

Critera are different for each patch, but it usually comes down to a
cost/benefit judgement.  Does the benefit of the patch exceed its
maintenance cost over the lifetime of the kernel (whatever that is).

In this case the answer to that has never been clear to me.  The (much
older) fs-aio patches were (are) in a similar situation.

The other consideration here is, as Nick points out, are the problems which
people see this patch solving for them solveable in other, better ways? 
IOW, is this patch fixing up preexisting deficiencies post-facto?

To attack the second question we could start out with bug reports: system A
with workload B produces result C.  I think result C is wrong for <reasons>
and would prefer to see result D.

> Or are we reading too much into the fact that it isn't merged? In
> short, communicate please, it will help.

Well.  The above, plus there's always a lot of stuff happening in MM land,
and I haven't seen much in the way of enthusiasm from the usual MM
developers.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Ray Lee]

On 7/23/07, Andrew Morton <akpm@linux-foundation.org> wrote:
> Let it just be noted that Con is not the only one who has expended effort
> on this patch.  It's been in -mm for nearly two years and it has meant
> ongoing effort for me and, to a lesser extent, other MM developers to keep
> it alive.

<nods> Yes, keeping patches from crufting up and stepping on other
patches' toes is hard work; I did it for a bit, and it was one of the
more thankless tasks I've tried a hand at.

So, thanks.

> Critera are different for each patch, but it usually comes down to a
> cost/benefit judgement.  Does the benefit of the patch exceed its
> maintenance cost over the lifetime of the kernel (whatever that is).

Well, I suspect it's 'lifetime of the feature,' in this case as it's
no more user visible than the page replacement algorithm in the first
place.

> The other consideration here is, as Nick points out, are the problems which
> people see this patch solving for them solveable in other, better ways?
> IOW, is this patch fixing up preexisting deficiencies post-facto?

In some cases, it almost certainly is. It also has the troubling
aspect of mitigating future regressions without anyone terribly
noticing, due to it being able to paper over those hypothetical future
deficiencies when they're introduced.

> To attack the second question we could start out with bug reports: system A
> with workload B produces result C.  I think result C is wrong for <reasons>
> and would prefer to see result D.

I spend a lot of time each day watching my computer fault my
workingset back in when I switch contexts. I'd rather I didn't have to
do that. Unfortunately, that's a pretty subjective problem report. For
whatever it's worth, we have pretty subjective solution reports
pointing to swap prefetch as providing a fix for them.

My concern is that a subjective problem report may not be good enough.
So, what do I measure to make this an objective problem report? And if
I do that (and it shows a positive result), will that be good enough
to argue for inclusion?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Andrew Morton]

On Mon, 23 Jul 2007 23:01:41 -0700 "Ray Lee" <ray-lk@madrabbit.org> wrote:

> So, what do I measure to make this an objective problem report?

Ideal would be to find a reproducible-by-others testcase which does what you
believe to be the wrong thing.

> And if
> I do that (and it shows a positive result), will that be good enough
> to argue for inclusion?

That depends upon whether there are more suitable ways of fixing "the
wrong thing".

There may not be - it could well be that present behaviour
is correct for the testcase, but it leaves the system in the wrong
state for your large workload shift.  In that case, prefetching (ie:
restoring system state approximately to that which prevailed prior to
"testcase") might well be a suitable fix.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Tilman Schmidt]

[-- Attachment #1: Type: text/plain, Size: 1106 bytes --]

Ray Lee schrieb:
> I spend a lot of time each day watching my computer fault my
> workingset back in when I switch contexts. I'd rather I didn't have to
> do that. Unfortunately, that's a pretty subjective problem report. For
> whatever it's worth, we have pretty subjective solution reports
> pointing to swap prefetch as providing a fix for them.

Add me.

> My concern is that a subjective problem report may not be good enough.

That's my impression too, seeing the insistence on numbers.

> So, what do I measure to make this an objective problem report?

That seems to be the crux of the matter: how to measure subjective
usability issues (aka user experience) when simple reports along the
lines of "A is much better than B for everyday work" are not enough.
The same problem already impaired the "fair scheduler" discussion.
It would really help to have a clear direction there.

-- 
Tilman Schmidt                    E-Mail: tilman@imap.cc
Bonn, Germany
Diese Nachricht besteht zu 100% aus wiederverwerteten Bits.
Ungeöffnet mindestens haltbar bis: (siehe Rückseite)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 250 bytes --]

Re: [ck] Re: -mm merge plans for 2.6.23

[Matthew Hawkins]

On 7/24/07, Andrew Morton <akpm@linux-foundation.org> wrote:
> The other consideration here is, as Nick points out, are the problems which
> people see this patch solving for them solveable in other, better ways?
> IOW, is this patch fixing up preexisting deficiencies post-facto?

So let me get this straight - you don't want to merge swap prefetch
which exists now and solves issues many people are seeing, and has
been tested more than a gazillion other bits & pieces that do get
merged - because it could be possible that in the future some other
patch, which doesn't yet exist and nobody is working on, may solve the
problem better?

You know what, just release Linux 0.02 as 2.6.23 because, using your
logic, everything that was merged since October 5, 1991 could be
replaced by something better.  Perhaps.  So there's obviously no point
having it there in the first place & there'll be untold savings in
storage costs and compilation time for the kernel tree, also bandwidth
for the mirror sites etc. in the mean time while we wait for the magic
pixies to come and deliver the one true piece of code that cannot be
improved upon.

> Well.  The above, plus there's always a lot of stuff happening in MM land,
> and I haven't seen much in the way of enthusiasm from the usual MM
> developers.

I haven't seen much in the way of enthusiasm from developers, period.
People are tired of maintaining patches for years that never get
merged into mainline because of totally bullshit reasons (usually
amounting to NIH syndrome)

-- 
Matt

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [ck] Re: -mm merge plans for 2.6.23

[David Miller, Matthew Hawkins]

> On 7/24/07, Andrew Morton <akpm@linux-foundation.org> wrote:
> > The other consideration here is, as Nick points out, are the problems which
> > people see this patch solving for them solveable in other, better ways?
> > IOW, is this patch fixing up preexisting deficiencies post-facto?
> 
> So let me get this straight - you don't want to merge swap prefetch
> which exists now and solves issues many people are seeing, and has
> been tested more than a gazillion other bits & pieces that do get
> merged - because it could be possible that in the future some other
> patch, which doesn't yet exist and nobody is working on, may solve the
> problem better?

I have to generally agree that the objections to the swap prefetch
patches have been conjecture and in general wasting time and
frustrating people.

There is a point at which it might be wise to just step back and let
the river run it's course and see what happens.  Initially, it's good
to play games of "what if", but after several months it's not a
productive thing and slows down progress for no good reason.

If a better mechanism gets implemented, great!  We'll can easily
replace the swap prefetch stuff at such time.  But until then swap
prefetch is what we have and it's sat long enough in -mm with no major
problems to merge it.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: -mm merge plans for 2.6.23

[Con Kolivas]

On Tuesday 10 July 2007 20:15, Con Kolivas wrote:
> On Tuesday 10 July 2007 18:31, Andrew Morton wrote:
> > When replying, please rewrite the subject suitably and try to Cc: the
> > appropriate developer(s).
>
> ~swap prefetch
>
> Nick's only remaining issue which I could remotely identify was to make it
> cpuset aware:
> http://marc.info/?l=linux-mm&m=117875557014098&w=2
> as discussed with Paul Jackson it was cpuset aware:
> http://marc.info/?l=linux-mm&m=117895463120843&w=2
>
> I fixed all bugs I could find and improved it as much as I could last
> kernel cycle.
>
> Put me and the users out of our misery and merge it now or delete it
> forever please. And if the meaningless handwaving that I 100% expect as a
> response begins again, then that's fine. I'll take that as a no and you can
> dump it.

The window for 2.6.23 has now closed and your position on this is clear. I've 
been supporting this code in -mm for 21 months since 16-Oct-2005 without any 
obvious decision for this code forwards or backwards.

I am no longer part of your operating system's kernel's world; thus I cannot 
support this code any longer. Unless someone takes over the code base for 
swap prefetch you have to assume it is now unmaintained and should delete it.

Please respect my request to not be contacted further regarding this or any 
other kernel code.

-- 
-ck

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: buffered write patches, -mm merge plans for 2.6.23

[Christoph Hellwig]

>  pagefault-in-write deadlock fixes.  Will hold for 2.6.24.

Why that?  This stuff has been in forever and is needed at various
levels.  We need this in for anything to move forward on the buffered
write front.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: buffered write patches, -mm merge plans for 2.6.23

[Andrew Morton]

On Wed, 11 Jul 2007 13:39:44 +0200 Christoph Hellwig <hch@lst.de> wrote:

> >  pagefault-in-write deadlock fixes.  Will hold for 2.6.24.
> 
> Why that?

At Nick's request.  More work is needed and the code hasn't had a lot of
testing/thought/exposure/review.

>  This stuff has been in forever and is needed at various
> levels.  We need this in for anything to move forward on the buffered
> write front.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

lguest, Re: -mm merge plans for 2.6.23

[Christoph Hellwig]

> lguest-export-symbols-for-lguest-as-a-module.patch

__put_task_struct is one of those no way in hell should this be exported
things because we don't want modules messing with task lifetimes.

Fortunately I can't find anything actually using this in lguest, so
it looks the issue has been solved in the meantime.


I also have a rather bad feeling about exporting access_process_vm.
This is the proverbial sledge hammer for access to user vm addresses
and I'd rather keep it away from module programmers with "if all
you have is a hammer ..." in mind.

In lguest this is used by send_dma which from my short reading of the
code seems to be the central IPC mechanism.  The double copy here
doesn't look very efficient to me either.  Maybe some VM folks could
look into a better way to archive this that might be both more
efficient and not require the export.


> lguest-the-guest-code.patch
> lguest-the-host-code.patch
> lguest-the-host-code-lguest-vs-clockevents-fix-resume-logic.patch
> lguest-the-asm-offsets.patch
> lguest-the-makefile-and-kconfig.patch
> lguest-the-console-driver.patch
> lguest-the-net-driver.patch
> lguest-the-block-driver.patch
> lguest-the-documentation-example-launcher.patch

Just started to reading this (again) so no useful comment here, but it
would be nice if the code could follow CodingStyle and place the || and
&& at the end of the line in multiline conditionals instead of at the
beginning of the new one.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: lguest, Re: -mm merge plans for 2.6.23

[Randy Dunlap]

On Wed, 11 Jul 2007 14:23:24 +0200 Christoph Hellwig wrote:

...

> > lguest-the-guest-code.patch
> > lguest-the-host-code.patch
> > lguest-the-host-code-lguest-vs-clockevents-fix-resume-logic.patch
> > lguest-the-asm-offsets.patch
> > lguest-the-makefile-and-kconfig.patch
> > lguest-the-console-driver.patch
> > lguest-the-net-driver.patch
> > lguest-the-block-driver.patch
> > lguest-the-documentation-example-launcher.patch
> 
> Just started to reading this (again) so no useful comment here, but it
> would be nice if the code could follow CodingStyle and place the || and
> && at the end of the line in multiline conditionals instead of at the
> beginning of the new one.

I prefer them at the ends of lines also, but that's not in CodingStyle,
it's just how we do it most of the time (so "coding style", without
caps).

---
~Randy
*** Remember to use Documentation/SubmitChecklist when testing your code ***

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: lguest, Re: -mm merge plans for 2.6.23

[Andrew Morton]

On Wed, 11 Jul 2007 14:23:24 +0200
Christoph Hellwig <hch@lst.de> wrote:

> > lguest-export-symbols-for-lguest-as-a-module.patch
> 
> __put_task_struct is one of those no way in hell should this be exported
> things because we don't want modules messing with task lifetimes.
> 
> Fortunately I can't find anything actually using this in lguest, so
> it looks the issue has been solved in the meantime.
> 

Ther are a couple of calls to put_task_struct() in there, and that needs
__put_task_struct() exported.

> 
> I also have a rather bad feeling about exporting access_process_vm.
> This is the proverbial sledge hammer for access to user vm addresses
> and I'd rather keep it away from module programmers with "if all
> you have is a hammer ..." in mind.

hm, well, access_process_vm() is a convenience wrapper around
get_user_pages(), whcih is exported.

> In lguest this is used by send_dma which from my short reading of the
> code seems to be the central IPC mechanism.  The double copy here
> doesn't look very efficient to me either.  Maybe some VM folks could
> look into a better way to archive this that might be both more
> efficient and not require the export.
> 
> 
> > lguest-the-guest-code.patch
> > lguest-the-host-code.patch
> > lguest-the-host-code-lguest-vs-clockevents-fix-resume-logic.patch
> > lguest-the-asm-offsets.patch
> > lguest-the-makefile-and-kconfig.patch
> > lguest-the-console-driver.patch
> > lguest-the-net-driver.patch
> > lguest-the-block-driver.patch
> > lguest-the-documentation-example-launcher.patch
> 
> Just started to reading this (again) so no useful comment here, but it
> would be nice if the code could follow CodingStyle and place the || and
> && at the end of the line in multiline conditionals instead of at the
> beginning of the new one.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: lguest, Re: -mm merge plans for 2.6.23

[Rusty Russell]

On Wed, 2007-07-11 at 14:23 +0200, Christoph Hellwig wrote:
> > lguest-export-symbols-for-lguest-as-a-module.patch
> 
> __put_task_struct is one of those no way in hell should this be exported
> things because we don't want modules messing with task lifetimes.
> 
> Fortunately I can't find anything actually using this in lguest, so
> it looks the issue has been solved in the meantime.

To do inter-guest (ie. inter-process) I/O you really have to make sure
the other side doesn't go away.

> I also have a rather bad feeling about exporting access_process_vm.
> This is the proverbial sledge hammer for access to user vm addresses
> and I'd rather keep it away from module programmers with "if all
> you have is a hammer ..." in mind.
> 
> In lguest this is used by send_dma which from my short reading of the
> code seems to be the central IPC mechanism.  The double copy here
> doesn't look very efficient to me either.  Maybe some VM folks could
> look into a better way to archive this that might be both more
> efficient and not require the export.

It's not a double copy: it's a map & copy.

If KVM develops inter-guest I/O then this could all be extracted into a
helper function and made more efficient.

> Just started to reading this (again) so no useful comment here, but it
> would be nice if the code could follow CodingStyle and place the || and
> && at the end of the line in multiline conditionals instead of at the
> beginning of the new one.

Surprisingly, you have a point here.  Since the key purpose of lguest is
as demonstration code, it meticulously match kernel style.

I shall immediately prepare a patch to convert the rest of the kernel to
the correct "&& at beginning of line" style.

Rusty.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: lguest, Re: -mm merge plans for 2.6.23

[David Miller, Rusty Russell]

> To do inter-guest (ie. inter-process) I/O you really have to make sure
> the other side doesn't go away.

You should just let it exit and when it does you receive some kind of
exit notification that resets your virtual device channel.

I think the reference counting approach is error and deadlock prone.
Be more loose and let the events reset the virtual devices when
guests go splat.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: lguest, Re: -mm merge plans for 2.6.23

[Rusty Russell]

On Wed, 2007-07-11 at 19:28 -0700, David Miller wrote:
> From: Rusty Russell <rusty@rustcorp.com.au>
> Date: Thu, 12 Jul 2007 11:21:51 +1000
> 
> > To do inter-guest (ie. inter-process) I/O you really have to make sure
> > the other side doesn't go away.
> 
> You should just let it exit and when it does you receive some kind of
> exit notification that resets your virtual device channel.
> 
> I think the reference counting approach is error and deadlock prone.
> Be more loose and let the events reset the virtual devices when
> guests go splat.

There are two places where we grab task refcnt.  One might be avoidable
(will test and get back) but the deferred wakeup isn't really:

        /* We cache one process to wakeup: helps for batching & wakes outside locks. */
        void set_wakeup_process(struct lguest *lg, struct task_struct *p)
        {
        	if (p == lg->wake)
        		return;
        
        	if (lg->wake) {
        		wake_up_process(lg->wake);
        		put_task_struct(lg->wake);
        	}
        	lg->wake = p;
        	if (lg->wake)
        		get_task_struct(lg->wake);
        }

We drop the lock after I/O, and then do this wakeup.  Meanwhile the
other task might have exited.

I could get rid of it, but I don't think there's anything wrong with the
code...

Cheers,
Rusty.


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: lguest, Re: -mm merge plans for 2.6.23

[David Miller, Rusty Russell]

> We drop the lock after I/O, and then do this wakeup.  Meanwhile the
> other task might have exited.

I already understand what you're doing.

Is it possible to use exit notifiers to handle this case?
That's what I'm trying to suggest. :)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: lguest, Re: -mm merge plans for 2.6.23

[Rusty Russell]

On Wed, 2007-07-11 at 19:51 -0700, David Miller wrote:
> From: Rusty Russell <rusty@rustcorp.com.au>
> Date: Thu, 12 Jul 2007 12:48:41 +1000
> 
> > We drop the lock after I/O, and then do this wakeup.  Meanwhile the
> > other task might have exited.
> 
> I already understand what you're doing.
> 
> Is it possible to use exit notifiers to handle this case?
> That's what I'm trying to suggest. :)

Sure, the process has /dev/lguest open, so I can do something in the
close routine.  Instead of keeping a reference to the tsk, I can keep a
reference to the struct lguest (currently it doesn't have or need a
refcnt).  Then I need another lock, to protect lg->tsk.

This seems like a lot of dancing to avoid one export.  If it's that
important I'd far rather drop the code and do a normal wakeup under the
big lguest lock for 2.6.23.

Cheers,
Rusty.


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: lguest, Re: -mm merge plans for 2.6.23

[David Miller, Rusty Russell]

> Sure, the process has /dev/lguest open, so I can do something in the
> close routine.  Instead of keeping a reference to the tsk, I can keep a
> reference to the struct lguest (currently it doesn't have or need a
> refcnt).  Then I need another lock, to protect lg->tsk.
> 
> This seems like a lot of dancing to avoid one export.  If it's that
> important I'd far rather drop the code and do a normal wakeup under the
> big lguest lock for 2.6.23.

I'm not against the export, so use if it really helps.

Ref-counting just seems clumsy to me given how the hw assisted
virtualization stuff works on platforms I am intimately familiar with
:)

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: lguest, Re: -mm merge plans for 2.6.23

[Andrew Morton]

On Thu, 12 Jul 2007 12:48:41 +1000 Rusty Russell <rusty@rustcorp.com.au> wrote:

> On Wed, 2007-07-11 at 19:28 -0700, David Miller wrote:
> > From: Rusty Russell <rusty@rustcorp.com.au>
> > Date: Thu, 12 Jul 2007 11:21:51 +1000
> > 
> > > To do inter-guest (ie. inter-process) I/O you really have to make sure
> > > the other side doesn't go away.
> > 
> > You should just let it exit and when it does you receive some kind of
> > exit notification that resets your virtual device channel.
> > 
> > I think the reference counting approach is error and deadlock prone.
> > Be more loose and let the events reset the virtual devices when
> > guests go splat.
> 
> There are two places where we grab task refcnt.  One might be avoidable
> (will test and get back) but the deferred wakeup isn't really:
> 
>         /* We cache one process to wakeup: helps for batching & wakes outside locks. */
>         void set_wakeup_process(struct lguest *lg, struct task_struct *p)
>         {
>         	if (p == lg->wake)
>         		return;
>         
>         	if (lg->wake) {
>         		wake_up_process(lg->wake);
>         		put_task_struct(lg->wake);
>         	}
>         	lg->wake = p;
>         	if (lg->wake)
>         		get_task_struct(lg->wake);
>         }

<handwaving>

We seem to be taking the reference against the wrong thing here.  It should
be against the mm, not against a task_struct?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: lguest, Re: -mm merge plans for 2.6.23

[Rusty Russell]

On Wed, 2007-07-11 at 21:24 -0700, Andrew Morton wrote:
> We seem to be taking the reference against the wrong thing here.  It should
> be against the mm, not against a task_struct?

This is solely for the wakeup: you don't wake an mm 8)

The mm reference is held as well under the big lguest_mutex (mm gets
destroyed before files get closed, so we definitely do need to hold a
reference).

I just completed benchmarking: the cached wakeup with the current naive
drivers makes no difference (at one stage I was playing with batched
hypercalls, where it seemed to help).

Thanks Christoph, DaveM!
===
Remove export of __put_task_struct, and usage in lguest

lguest takes a reference count of tasks for two reasons.  The first is
bogus: the /dev/lguest close callback will be called before the task
is destroyed anyway, so no need to take a reference on open.

The second is code to defer waking up tasks for inter-guest I/O, but
the current lguest drivers are too simplistic to benefit (only batched
hypercalls will see an effect, and it's likely that lguests' entire
I/O model will be replaced with virtio and ringbuffers anyway).

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
---
 drivers/lguest/hypercalls.c  |    1 -
 drivers/lguest/io.c          |   18 +-----------------
 drivers/lguest/lg.h          |    1 -
 drivers/lguest/lguest_user.c |    2 --
 kernel/fork.c                |    1 -
 5 files changed, 1 insertion(+), 22 deletions(-)

===================================================================
--- a/drivers/lguest/hypercalls.c
+++ b/drivers/lguest/hypercalls.c
@@ -189,5 +189,4 @@ void do_hypercalls(struct lguest *lg)
 		do_hcall(lg, lg->regs);
 		clear_hcall(lg);
 	}
-	set_wakeup_process(lg, NULL);
 }
===================================================================
--- a/drivers/lguest/io.c
+++ b/drivers/lguest/io.c
@@ -296,7 +296,7 @@ static int dma_transfer(struct lguest *s
 
 	/* Do this last so dst doesn't simply sleep on lock. */
 	set_bit(dst->interrupt, dstlg->irqs_pending);
-	set_wakeup_process(srclg, dstlg->tsk);
+	wake_up_process(dstlg->tsk);
 	return i == dst->num_dmas;
 
 fail:
@@ -333,7 +333,6 @@ again:
 			/* Give any recipients one chance to restock. */
 			up_read(&current->mm->mmap_sem);
 			mutex_unlock(&lguest_lock);
-			set_wakeup_process(lg, NULL);
 			empty++;
 			goto again;
 		}
@@ -360,21 +359,6 @@ void release_all_dma(struct lguest *lg)
 			unlink_dma(&lg->dma[i]);
 	}
 	up_read(&lg->mm->mmap_sem);
-}
-
-/* We cache one process to wakeup: helps for batching & wakes outside locks. */
-void set_wakeup_process(struct lguest *lg, struct task_struct *p)
-{
-	if (p == lg->wake)
-		return;
-
-	if (lg->wake) {
-		wake_up_process(lg->wake);
-		put_task_struct(lg->wake);
-	}
-	lg->wake = p;
-	if (lg->wake)
-		get_task_struct(lg->wake);
 }
 
 /* Userspace wants a dma buffer from this guest. */
===================================================================
--- a/drivers/lguest/lg.h
+++ b/drivers/lguest/lg.h
@@ -240,7 +240,6 @@ void release_all_dma(struct lguest *lg);
 void release_all_dma(struct lguest *lg);
 unsigned long get_dma_buffer(struct lguest *lg, unsigned long key,
 			     unsigned long *interrupt);
-void set_wakeup_process(struct lguest *lg, struct task_struct *p);
 
 /* hypercalls.c: */
 void do_hypercalls(struct lguest *lg);
===================================================================
--- a/drivers/lguest/lguest_user.c
+++ b/drivers/lguest/lguest_user.c
@@ -141,7 +141,6 @@ static int initialize(struct file *file,
 	setup_guest_gdt(lg);
 	init_clockdev(lg);
 	lg->tsk = current;
-	get_task_struct(lg->tsk);
 	lg->mm = get_task_mm(lg->tsk);
 	init_waitqueue_head(&lg->break_wq);
 	lg->last_pages = NULL;
@@ -205,7 +204,6 @@ static int close(struct inode *inode, st
 	hrtimer_cancel(&lg->hrt);
 	release_all_dma(lg);
 	free_guest_pagetable(lg);
-	put_task_struct(lg->tsk);
 	mmput(lg->mm);
 	if (!IS_ERR(lg->dead))
 		kfree(lg->dead);
===================================================================
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -128,7 +128,6 @@ void __put_task_struct(struct task_struc
 	if (!profile_handoff_task(tsk))
 		free_task(tsk);
 }
-EXPORT_SYMBOL_GPL(__put_task_struct);
 
 void __init fork_init(unsigned long mempages)
 {


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: lguest, Re: -mm merge plans for 2.6.23

[Avi Kivity]

Rusty Russell wrote:
> Remove export of __put_task_struct, and usage in lguest
>
> lguest takes a reference count of tasks for two reasons.  The first is
> bogus: the /dev/lguest close callback will be called before the task
> is destroyed anyway, so no need to take a reference on open.
>
>   

What about

  Open /dev/lguest
  transfer fd using SCM_RIGHTS (or clone()?)
  close fd in original task
  exit()

?

My feeling is that if you want to be bound to a task, not a file, you 
need to use syscalls, not ioctls.

-- 
error compiling committee.c: too many arguments to function

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: lguest, Re: -mm merge plans for 2.6.23

[Christoph Hellwig]

On Thu, Jul 12, 2007 at 02:52:23PM +1000, Rusty Russell wrote:
> This is solely for the wakeup: you don't wake an mm 8)
> 
> The mm reference is held as well under the big lguest_mutex (mm gets
> destroyed before files get closed, so we definitely do need to hold a
> reference).
> 
> I just completed benchmarking: the cached wakeup with the current naive
> drivers makes no difference (at one stage I was playing with batched
> hypercalls, where it seemed to help).
> 
> Thanks Christoph, DaveM!

The version that just got into mainline still has the __put_task_struct
export despite not needing it anymore.  Care to fix this up?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: lguest, Re: -mm merge plans for 2.6.23

[Rusty Russell]

On Thu, 2007-07-19 at 19:27 +0200, Christoph Hellwig wrote:
> The version that just got into mainline still has the __put_task_struct
> export despite not needing it anymore.  Care to fix this up?

No, it got patched in then immediately patched out again.  Andrew
mis-mixed my patches, but there have been so many of them I find it hard
to blame him.

Rusty.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: lguest, Re: -mm merge plans for 2.6.23

[Christoph Hellwig]

On Fri, Jul 20, 2007 at 01:27:26PM +1000, Rusty Russell wrote:
> On Thu, 2007-07-19 at 19:27 +0200, Christoph Hellwig wrote:
> > The version that just got into mainline still has the __put_task_struct
> > export despite not needing it anymore.  Care to fix this up?
> 
> No, it got patched in then immediately patched out again.  Andrew
> mis-mixed my patches, but there have been so many of them I find it hard
> to blame him.

Indeed, the export is gone in last mainline gone.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

fault vs invalidate race (Re: -mm merge plans for 2.6.23)

[Nick Piggin]

Andrew Morton wrote:

> mm-fix-fault-vs-invalidate-race-for-linear-mappings.patch
> mm-merge-populate-and-nopage-into-fault-fixes-nonlinear.patch
> mm-merge-nopfn-into-fault.patch
> convert-hugetlbfs-to-use-vm_ops-fault.patch
> mm-remove-legacy-cruft.patch
> mm-debug-check-for-the-fault-vs-invalidate-race.patch
> mm-fix-clear_page_dirty_for_io-vs-fault-race.patch
> invalidate_mapping_pages-add-cond_resched.patch
> ocfs2-release-page-lock-before-calling-page_mkwrite.patch
> document-page_mkwrite-locking.patch
> 
>  The fault-vs-invalidate race fix.  I have belatedly learned that these need
>  more work, so their state is uncertain.

The more work may turn out being too much for you (although it is nothing
exactly tricky that would introduce subtle bugs, it is a fair amont of churn).

However, in that case we can still merge these two:

mm-fix-fault-vs-invalidate-race-for-linear-mappings.patch
mm-fix-clear_page_dirty_for_io-vs-fault-race.patch

Which fix real bugs that need fixing (and will at least help to get some of
my patches off your hands).

-- 
SUSE Labs, Novell Inc.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

block_page_mkwrite? (Re: fault vs invalidate race (Re: -mm merge plans for 2.6.23))

[David Chinner]

On Thu, Jul 12, 2007 at 10:54:57AM +1000, Nick Piggin wrote:
> Andrew Morton wrote:
> > The fault-vs-invalidate race fix.  I have belatedly learned that these 
> > need
> > more work, so their state is uncertain.
> 
> The more work may turn out being too much for you (although it is nothing
> exactly tricky that would introduce subtle bugs, it is a fair amont of 
> churn).

OK, so does that mean we can finally get the block_page_mkwrite
patches merged?

i.e.:

http://marc.info/?l=linux-kernel&m=117426058311032&w=2
http://marc.info/?l=linux-kernel&m=117426070111136&w=2

I've got up-to-date versions of them ready to go and they've been
consistently tested thanks to the XFSQA test I wrote for the bug
that it fixes. I've been holding them out-of-tree for months now
because ->fault was supposed to supercede this interface.....

Cheers,

Dave.
-- 
Dave Chinner
Principal Engineer
SGI Australian Software Group

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: block_page_mkwrite? (Re: fault vs invalidate race (Re: -mm merge plans for 2.6.23))

[Nick Piggin]

David Chinner wrote:
> On Thu, Jul 12, 2007 at 10:54:57AM +1000, Nick Piggin wrote:
> 
>>Andrew Morton wrote:
>>
>>>The fault-vs-invalidate race fix.  I have belatedly learned that these 
>>>need
>>>more work, so their state is uncertain.
>>
>>The more work may turn out being too much for you (although it is nothing
>>exactly tricky that would introduce subtle bugs, it is a fair amont of 
>>churn).
> 
> 
> OK, so does that mean we can finally get the block_page_mkwrite
> patches merged?
> 
> i.e.:
> 
> http://marc.info/?l=linux-kernel&m=117426058311032&w=2
> http://marc.info/?l=linux-kernel&m=117426070111136&w=2
> 
> I've got up-to-date versions of them ready to go and they've been
> consistently tested thanks to the XFSQA test I wrote for the bug
> that it fixes. I've been holding them out-of-tree for months now
> because ->fault was supposed to supercede this interface.....

Yeah, as I've said, don't hold them back because of me. They are
relatively simple enough that I don't see why they couldn't be
merged in this window.

-- 
SUSE Labs, Novell Inc.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>