Re: mmap()/VM problems in 2.4.0

Re: mmap()/VM problems in 2.4.0

[Vlad Bolkhovitine]

Here is updated info for 2.4.1pre3:

Size is MB, BlkSz is Bytes, Read, Write, and Seeks are MB/sec

with mmap()

 File   Block  Num          Seq Read    Rand Read   Seq Write  Rand Write
 Dir    Size   Size    Thr Rate (CPU%) Rate (CPU%) Rate (CPU%) Rate (CPU%)
------- ------ ------- --- ----------- ----------- ----------- -----------
   .     1024   4096    2  1.089 1.24% 0.235 0.45% 1.118 4.11% 0.616 1.41%

without mmap()
   
 File   Block  Num          Seq Read    Rand Read   Seq Write  Rand Write
 Dir    Size   Size    Thr Rate (CPU%) Rate (CPU%) Rate (CPU%) Rate (CPU%)
------- ------ ------- --- ----------- ----------- ----------- -----------
   .     1024   4096    2  28.41 41.0% 0.547 1.15% 13.16 16.1% 0.652 1.46%


Mmap() performance dropped dramatically down to almost unusable level. Plus,
system was unusable during test: "vmstat 1" updated results every 1-2 _MINUTES_!

Problem one (impossible to run tiobench without swap) is still here with the
only difference that tiobench gets killed faster (just after start).

Regards,
Vlad

P.S. Sorry for overquoting, I hope it could be helpful for linux-mm subscribers.

Vlad Bolkhovitine wrote:
> 
> After upgrade from 2.4.0-test7 to 2.4.0 while running tiotest v0.3.1 I found two
> following problems.
> 
> 1. Tiotest is compiled for mmap() usage and there is no swap on the system with
> ~200Mb free memory. Tiotest tries to create mmap'ed file with size
> ~memory_size*2 and soon after start gets killed by OOM killer. If I add swap
> space, the kernel uses only a few Mb from it.
> 
> AFAIU, it is because out_of_memory() in oom_kill.c checks for amount swap space
> left, which is always 0 without swap. Apparently, it is not correct for
> "no-swap" systems.
> 
> 2. Second problem is related to mmap() performance.
> 
> I ran "./tiobench.pl --size 1024 --threads 2", which is translated to
> "./tiotest -t 2 -f 512 -r 2000 -b 4096 -d . -T", with tiotest compiled for
> mmap() and for conventional read()/write() usage on 2.4.0-test7 and 2.4.0. These
> are results:
> 
> Size is MB, BlkSz is Bytes, Read, Write, and Seeks are MB/sec
> 
> 2.4.0-test7 with mmap()
> 
> File   Block  Num  Seq Read    Rand Read   Seq Write  Rand Write
> Dir    Size   Size   Thr Rate (CPU%) Rate (CPU%) Rate (CPU%) Rate (CPU%)
> ------- ------ ------- --- ----------- ----------- ----------- -----------
>    .     1024   4096    2  22.44 14.7% 0.456 0.78% 10.66 22.5% 0.733 1.87%
> 
> 2.4.0 with mmap()
> 
> File   Block  Num  Seq Read    Rand Read   Seq Write  Rand Write
> Dir    Size   Size   Thr Rate (CPU%) Rate (CPU%) Rate (CPU%) Rate (CPU%)
> ------- ------ ------- --- ----------- ----------- ----------- -----------
>    .     1024   4096    2  12.53 9.02% 0.489 1.16% 10.82 15.3% 0.640 1.14%
> 
> 2.4.0-test7 without mmap()
> 
> File   Block  Num  Seq Read    Rand Read   Seq Write  Rand Write
> Dir    Size   Size   Thr Rate (CPU%) Rate (CPU%) Rate (CPU%) Rate (CPU%)
> ------- ------ ------- --- ----------- ----------- ----------- -----------
>    .     1024   4096    2  14.20 17.6% 0.502 1.28% 12.85 15.1% 0.643 1.31%
> 
> 2.4.0 without mmap()
> 
> File   Block  Num  Seq Read    Rand Read   Seq Write  Rand Write
> Dir    Size   Size   Thr Rate (CPU%) Rate (CPU%) Rate (CPU%) Rate (CPU%)
> ------- ------ ------- --- ----------- ----------- ----------- -----------
>    .     1024   4096    2  28.41 42.1% 0.541 1.35% 13.16 16.8% 0.645 1.52%
> 
> You can see, mmap() read performance dropped significantly as well as read() one
> raised. Plus, "interactivity" of 2.4.0 system was much worse during mmap'ed
> test, than using read() (everything was quite smooth here). 2.4.0-test7 was
> badly interactive in both cases.
> 
> I use /dev/hdc on IDE channel 2 for tests and /dev/hda IDE channel 2 for swap.
> hdparam output for both drives:
> 
>  multcount    =  0 (off)
>  I/O support  =  0 (default 16-bit)
>  unmaskirq    =  0 (off)
>  using_dma    =  1 (on)
>  keepsettings =  0 (off)
>  nowerr       =  0 (off)
>  readonly     =  0 (off)
>  readahead    =  8 (on)
> 
> 2.4.0 and 2.4.0-test7 were compiled with one .config via "make oldconfig".
> .config and dmesg you can find in the attachment.
> 
> Any comments?
> 
> Regards,
> Vlad
>
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: mmap()/VM problems in 2.4.0

[Zlatko Calusic]

"Vlad Bolkhovitine" <vladb@sw.com.sg> writes:

> Here is updated info for 2.4.1pre3:
> 
> Size is MB, BlkSz is Bytes, Read, Write, and Seeks are MB/sec
> 
> with mmap()
> 
>  File   Block  Num          Seq Read    Rand Read   Seq Write  Rand Write
>  Dir    Size   Size    Thr Rate (CPU%) Rate (CPU%) Rate (CPU%) Rate (CPU%)
> ------- ------ ------- --- ----------- ----------- ----------- -----------
>    .     1024   4096    2  1.089 1.24% 0.235 0.45% 1.118 4.11% 0.616 1.41%
> 
> without mmap()
>    
>  File   Block  Num          Seq Read    Rand Read   Seq Write  Rand Write
>  Dir    Size   Size    Thr Rate (CPU%) Rate (CPU%) Rate (CPU%) Rate (CPU%)
> ------- ------ ------- --- ----------- ----------- ----------- -----------
>    .     1024   4096    2  28.41 41.0% 0.547 1.15% 13.16 16.1% 0.652 1.46%
> 
> 
> Mmap() performance dropped dramatically down to almost unusable level. Plus,
> system was unusable during test: "vmstat 1" updated results every 1-2 _MINUTES_!
> 

You need Marcelo's patch. Please apply and retest.



--- linux.orig/mm/vmscan.c      Mon Jan 15 02:33:15 2001
+++ linux/mm/vmscan.c   Mon Jan 15 02:46:25 2001
@@ -153,7 +153,7 @@

                        if (VALID_PAGE(page) && !PageReserved(page)) {
                                try_to_swap_out(mm, vma, address, pte,
page);
-                               if (--count)
+                               if (!--count)
                                        break;
                        }
                }


-- 
Zlatko
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: mmap()/VM problems in 2.4.0

[Mike Galbraith]

On 15 Jan 2001, Zlatko Calusic wrote:

> "Vlad Bolkhovitine" <vladb@sw.com.sg> writes:
> 
> > Here is updated info for 2.4.1pre3:
> > 
> > Size is MB, BlkSz is Bytes, Read, Write, and Seeks are MB/sec
> > 
> > with mmap()
> > 
> >  File   Block  Num          Seq Read    Rand Read   Seq Write  Rand Write
> >  Dir    Size   Size    Thr Rate (CPU%) Rate (CPU%) Rate (CPU%) Rate (CPU%)
> > ------- ------ ------- --- ----------- ----------- ----------- -----------
> >    .     1024   4096    2  1.089 1.24% 0.235 0.45% 1.118 4.11% 0.616 1.41%
> > 
> > without mmap()
> >    
> >  File   Block  Num          Seq Read    Rand Read   Seq Write  Rand Write
> >  Dir    Size   Size    Thr Rate (CPU%) Rate (CPU%) Rate (CPU%) Rate (CPU%)
> > ------- ------ ------- --- ----------- ----------- ----------- -----------
> >    .     1024   4096    2  28.41 41.0% 0.547 1.15% 13.16 16.1% 0.652 1.46%
> > 
> > 
> > Mmap() performance dropped dramatically down to almost unusable level. Plus,
> > system was unusable during test: "vmstat 1" updated results every 1-2 _MINUTES_!
> > 
> 
> You need Marcelo's patch. Please apply and retest.

My box thinks quite highly of that patch fwiw, but insists that he needs
to apply Jens Axboes' blk patch first ;-)  (Not because of tiobench)

	-Mike

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: mmap()/VM problems in 2.4.0

[Vlad Bolkhovitine]

Mike Galbraith wrote:
> 
> On 15 Jan 2001, Zlatko Calusic wrote:
> 
> > "Vlad Bolkhovitine" <vladb@sw.com.sg> writes:
> >
> > > Here is updated info for 2.4.1pre3:
> > >
> > > Size is MB, BlkSz is Bytes, Read, Write, and Seeks are MB/sec
> > >
> > > with mmap()
> > >
> > >  File   Block  Num          Seq Read    Rand Read   Seq Write  Rand Write
> > >  Dir    Size   Size    Thr Rate (CPU%) Rate (CPU%) Rate (CPU%) Rate (CPU%)
> > > ------- ------ ------- --- ----------- ----------- ----------- -----------
> > >    .     1024   4096    2  1.089 1.24% 0.235 0.45% 1.118 4.11% 0.616 1.41%
> > >
[...]
> > > Mmap() performance dropped dramatically down to almost unusable level. Plus,
> > > system was unusable during test: "vmstat 1" updated results every 1-2 _MINUTES_!
> > >
> >
> > You need Marcelo's patch. Please apply and retest.
> 
> My box thinks quite highly of that patch fwiw, but insists that he needs
> to apply Jens Axboes' blk patch first ;-)  (Not because of tiobench)

New data:

2.4.1pre3 + Marcelo's patch

       File   Block  Num  Seq Read    Rand Read   Seq Write  Rand Write
Dir    Size   Size   Thr Rate (CPU%) Rate (CPU%) Rate (CPU%) Rate (CPU%)
------- ------ ------- --- ----------- ----------- ----------- -----------
   .     1024   4096    2  12.68 9.23% 0.497 0.92% 10.57 15.3% 0.594 1.44%

The same performance level as for 2.4.0. No improvement.

2.4.1pre3 + Marcelo's patch + Jens Axboes' blk-13B patch 

         File   Block  Num  Seq Read    Rand Read   Seq Write  Rand Write
Dir    Size   Size   Thr Rate (CPU%) Rate (CPU%) Rate (CPU%) Rate (CPU%)
------- ------ ------- --- ----------- ----------- ----------- -----------
   .     1024   4096    2  12.47 10.0% 0.504 1.13% 9.998 16.5% 0.735 2.21%

No significant difference, just noise. IMHO, it is expected, since 2 threads
simply aren't enough to launch blk patch's mechanisms

>         -Mike
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: mmap()/VM problems in 2.4.0

[Mike Galbraith]

On Tue, 16 Jan 2001, Vlad Bolkhovitine wrote:

> > My box thinks quite highly of that patch fwiw, but insists that he needs
> > to apply Jens Axboes' blk patch first ;-)  (Not because of tiobench)
> 
> New data:
> 
> 2.4.1pre3 + Marcelo's patch
> 
>        File   Block  Num  Seq Read    Rand Read   Seq Write  Rand Write
> Dir    Size   Size   Thr Rate (CPU%) Rate (CPU%) Rate (CPU%) Rate (CPU%)
> ------- ------ ------- --- ----------- ----------- ----------- -----------
>    .     1024   4096    2  12.68 9.23% 0.497 0.92% 10.57 15.3% 0.594 1.44%
> 
> The same performance level as for 2.4.0. No improvement.

I was refering to the stalls.. not throughput.

	-Mike

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/