Re: 2.6.6-rc3-mm1

Re: 2.6.6-rc3-mm1

[Christoph Hellwig]

On Fri, Apr 30, 2004 at 01:46:58AM -0700, Andrew Morton wrote:
> +rmap-14-i_shared_lock-fixes.patch
> +rmap-15-vma_adjust.patch
> +rmap-16-pretend-prio_tree.patch
> +rmap-17-real-prio_tree.patch
> +rmap-18-i_mmap_nonlinear.patch
> +rmap-19-arch-prio_tree.patch
> 
>  More VM work from Hugh

That's about 600 lines of additional code.  And that prio tree code is
used a lot, so even worse for that caches.

Do we have some benchmarks of real-life situation where the prio trees
show a big enough improvement or some 'exploits' where the linear list
walking leads to DoS situtations?

The bases objrmap/anonrmap changes keep the LOC pretty much the same as
the old pte-chain based code, but this is really a whole lot of code bloating
up the kernel and I'd prefer to see some numbers before it's going in..
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: 2.6.6-rc3-mm1

[Hugh Dickins]

On Wed, 5 May 2004, Christoph Hellwig wrote:
> On Fri, Apr 30, 2004 at 01:46:58AM -0700, Andrew Morton wrote:
> > +rmap-14-i_shared_lock-fixes.patch
> > +rmap-15-vma_adjust.patch
> > +rmap-16-pretend-prio_tree.patch
> > +rmap-17-real-prio_tree.patch
> > +rmap-18-i_mmap_nonlinear.patch
> > +rmap-19-arch-prio_tree.patch
> > 
> >  More VM work from Hugh

Of course, all the hard work here is actually from Rajesh.

> That's about 600 lines of additional code.  And that prio tree code is
> used a lot, so even worse for that caches.

Fair concern.  It has been discussed offlist, and you're certainly not
the only one to feel that way.  I'm neutral, just making the patches
available.  Even Rajesh is firmly of the opinion that it has to be
thrown out if it doesn't pay off.

> Do we have some benchmarks of real-life situation where the prio trees
> show a big enough improvement or some 'exploits' where the linear list
> walking leads to DoS situtations?

Andrew and Ingo wrote "exploits", though whether they amount to DoS
I'm not convinced.  We do feel vulnerable to corner cases, and more
secure with Rajesh's work in place.  But I don't believe anyone has
shown a _real-life_ case for it yet - nor a case against it either.
Rajesh has certainly shown its value in the corner cases.

> The bases objrmap/anonrmap changes keep the LOC pretty much the same as
> the old pte-chain based code, but this is really a whole lot of code bloating
> up the kernel and I'd prefer to see some numbers before it's going in..

I'm hoping someone (at OSDL?) will do those numbers: probably easiest
once the basic objrmap+anonrmap has gone into 2.6.7-pre, then that can
be compared against the same with the prio_tree patches added (and fair
to include Rajesh's latest work, the prefetching, in any such testing).

I don't place a lot of faith in numbers coming from me!

Hugh

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>

Re: 2.6.6-rc3-mm1

[Rajesh Venkatasubramanian]

>> +rmap-14-i_shared_lock-fixes.patch
>> +rmap-15-vma_adjust.patch
>> +rmap-16-pretend-prio_tree.patch
>> +rmap-17-real-prio_tree.patch
>> +rmap-18-i_mmap_nonlinear.patch
>> +rmap-19-arch-prio_tree.patch
>>
>>  More VM work from Hugh
>
> That's about 600 lines of additional code.

Agreed.

> And that prio tree code is
> used a lot, so even worse for that caches.

The prio tree code is designed to handle common cases (almost)
as well as the simple i_mmap lists. Although we add 600 lines
of code, many parts of the additional code are not exercised
in the common paths. In common cases, the tree has only one
or two nodes and all other vmas hang from these tree nodes
(check vm_set.list). So it more looks (and performs) like a
single list or two lists.

In cases where all processes map the same set of pages from a
file (e.g., libc), plain i_mmap lists are the best for objrmap.
I think we cannot do any better than a simple list in these cases.
The prio tree patch tries emulate a simple list in such cases.

My kernel compile tests with and without these patches did not
show much performance difference. The overhead due to prio tree
is in the noise level for kernel compiles. Martin's SDET tests
(after converting i_shared_sem to i_shared_lock) showed that
prio tree patch does not affect performance.

> Do we have some benchmarks of real-life situation where the prio trees
> show a big enough improvement or some 'exploits' where the linear list
> walking leads to DoS situtations?

Please check the first prio tree patch post. You will find pointers
to programs developed by Andrew and Ingo that show poor performance
of linear lists.

The first prio tree patch post link:

http://marc.theaimsgroup.com/?l=linux-kernel&m=107990752323873

Overall, prio tree patch increases kernel size. However, it seems not
to affect performance in the common cases. In addition, it handles
corner cases pretty well. I am confident that prio tree does not
affect performance significantly in the common cases. However, futher
testing may prove me wrong. I agree that these patches should
be ripped out of the kernel if someone shows significantly poor
performance or we find an easier way to handle corner cases.

Thanks,
Rajesh
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>