Re: LINUX-MM

Re: LINUX-MM

[Rik van Riel]

On Tue, 23 Mar 1999, Matthias Arnold wrote:

> Thanks for your reply.Unfortunately the memory is not (at least
> not completely) returned to the system after the program has
> finished (as the OS comand 'free' tells me).

Hmm, what version of the kernel are you using?

IIRC there's a slight bug in some of the newer kernels
where the swap cache isn't being freed when you exit
your program, but only later on when the system tries
to reclaim memory...

Rik -- Open Source: you deserve to be in control of your data.
+-------------------------------------------------------------------+
| Linux Memory Management site:  http://humbolt.geo.uu.nl/Linux-MM/ |
| Nederlandse Linux documentatie:          http://www.nl.linux.org/ |
+-------------------------------------------------------------------+

--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address'
in the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Re: LINUX-MM

[Kev]

> > Thanks for your reply.Unfortunately the memory is not (at least
> > not completely) returned to the system after the program has
> > finished (as the OS comand 'free' tells me).
> 
> Hmm, what version of the kernel are you using?
> 
> IIRC there's a slight bug in some of the newer kernels
> where the swap cache isn't being freed when you exit
> your program, but only later on when the system tries
> to reclaim memory...

I believe the problem lies in the fact that there is not enough
SysV shared memory available.
-- 
Kevin L. Mitchell <klmitch@mit.edu>
-------------------------  -. .---- --.. ..- -..-  --------------------------
http://web.mit.edu/klmitch/www/               (PGP keys availiable from here)
    RSA AE87D37D/1024:  DE EA 1E 99 3F 2B F9 23  A0 D8 05 E0 6F BA B9 D2
    DSS ED0DB34E/1024: D9BF 0E74 FDCB 43F5 C597  878F 9455 EC24 ED0D B34E
    DH  2A2C31D4/2048: 1A77 4BA5 9E32 14AE 87DA  9FEC 7106 FC62 2A2C 31D4

--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address'
in the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Re: LINUX-MM

[Kev]

> I believe the problem lies in the fact that there is not enough
> SysV shared memory available.

blah, forget it; I'm paying attention to too many email conversations.
Sorry,
-- 
Kevin L. Mitchell <klmitch@mit.edu>
-------------------------  -. .---- --.. ..- -..-  --------------------------
http://web.mit.edu/klmitch/www/               (PGP keys availiable from here)
    RSA AE87D37D/1024:  DE EA 1E 99 3F 2B F9 23  A0 D8 05 E0 6F BA B9 D2
    DSS ED0DB34E/1024: D9BF 0E74 FDCB 43F5 C597  878F 9455 EC24 ED0D B34E
    DH  2A2C31D4/2048: 1A77 4BA5 9E32 14AE 87DA  9FEC 7106 FC62 2A2C 31D4

--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address'
in the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Re: LINUX-MM

[Stephen C. Tweedie]

Hi,

On Tue, 23 Mar 1999 10:49:11 EST, Kev <klmitch@MIT.EDU> said:

>> IIRC there's a slight bug in some of the newer kernels
>> where the swap cache isn't being freed when you exit
>> your program, but only later on when the system tries
>> to reclaim memory...

> I believe the problem lies in the fact that there is not enough
> SysV shared memory available.

It's nothing to do with SysV shared memory. 

The behaviour is there, but the only impact on the normal user will be
that "free" lies a little.  No big deal: it just shows up as cache.  The
effect is only a matter of when we recover the memory, not whether we
recover it.

--Stephen
--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address'
in the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Re: LINUX-MM

[Stephen C. Tweedie]

Hi,

On Wed, 24 Mar 1999 15:34:05 +0100, Matthias Arnold
<Matthias.Arnold@edda.imsid.uni-jena.de> said:

> The system lost a remarkable amount of memory after each run of my
> programs.After several runs the performance of the machine slowes down
> due to swapping (in other words the system hangs) and I have to
> reboot.

Which kernel precisely?  What does "vmstat 1" look like?  If you
swapoff/swapon between application runs does the effect persist?  What
does the application do?  What does /proc/sys/fs/inode-nr contain?

This does not sound like a result of the swap caching behaviour.  Once
you start swapping that memory _is_ returned (or something is not
working as it should).

--Stephen
--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address'
in the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Re: LINUX-MM

[Andrea Arcangeli]

On Tue, 23 Mar 1999, Rik van Riel wrote:

>On Tue, 23 Mar 1999, Matthias Arnold wrote:
>
>> Thanks for your reply.Unfortunately the memory is not (at least
>> not completely) returned to the system after the program has
>> finished (as the OS comand 'free' tells me).
>
>Hmm, what version of the kernel are you using?
>
>IIRC there's a slight bug in some of the newer kernels

I think at it as a feature and not a bug ;).

Andrea Arcangeli

--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address'
in the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Re: LINUX-MM

[Rik van Riel]

On Wed, 24 Mar 1999, Andrea Arcangeli wrote:
> On Tue, 23 Mar 1999, Rik van Riel wrote:
> >On Tue, 23 Mar 1999, Matthias Arnold wrote:
> >
> >> Thanks for your reply.Unfortunately the memory is not (at least
> >> not completely) returned to the system after the program has
> >> finished (as the OS comand 'free' tells me).
> >
> >Hmm, what version of the kernel are you using?
> >
> >IIRC there's a slight bug in some of the newer kernels
> 
> I think at it as a feature and not a bug ;).

It is a bug when it causes other programs to fail
miserably...

Rik -- Open Source: you deserve to be in control of your data.
+-------------------------------------------------------------------+
| Linux Memory Management site:  http://humbolt.geo.uu.nl/Linux-MM/ |
| Nederlandse Linux documentatie:          http://www.nl.linux.org/ |
+-------------------------------------------------------------------+

--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address'
in the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Re: LINUX-MM

[Stephen C. Tweedie]

Hi,

On Wed, 24 Mar 1999 01:33:53 +0100 (CET), Rik van Riel
<riel@nl.linux.org> said:

> It is a bug when it causes other programs to fail
> miserably...

Does this happen, and if so, in what way?

--Stephen
--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address'
in the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/

Re: LINUX-MM

[Andrea Arcangeli]

On Wed, 24 Mar 1999, Stephen C. Tweedie wrote:

>Hi,
>
>On Wed, 24 Mar 1999 01:33:53 +0100 (CET), Rik van Riel
><riel@nl.linux.org> said:
>
>> It is a bug when it causes other programs to fail
>> miserably...
>
>Does this happen, and if so, in what way?

I don't think that can happens. Anyway I just have a patch to return to
the old behavior, but I can't see any difference here (except that we
won't do the work on demand).

Just for the record my patch is this, probably it won't apply cleanly but
upporting it, it's trivial.

Index: linux/include/linux/swap.h
diff -c linux/include/linux/swap.h:1.1.1.1.14.3 linux/include/linux/swap.h:1.1.1.1.14.4
*** linux/include/linux/swap.h:1.1.1.1.14.3	Thu Oct 29 19:42:56 1998
--- linux/include/linux/swap.h	Sun Nov  1 18:20:48 1998
***************
*** 90,95 ****
--- 90,96 ----
  extern struct page * read_swap_cache_async(unsigned long, int);
  #define read_swap_cache(entry) read_swap_cache_async(entry, 1);
  extern int FASTCALL(swap_count(unsigned long));
+ extern void FASTCALL(try_to_free_last_swap_entry(unsigned long));
  /*
   * Make these inline later once they are working properly.
   */
Index: linux/mm/swap_state.c
diff -c linux/mm/swap_state.c:1.1.1.1.14.4 linux/mm/swap_state.c:1.1.1.1.14.5
*** linux/mm/swap_state.c:1.1.1.1.14.4	Fri Oct 30 19:11:15 1998
--- linux/mm/swap_state.c	Sun Nov  1 18:20:50 1998
***************
*** 287,292 ****
--- 287,316 ----
  	return 0;
  }
  
+ void try_to_free_last_swap_entry(unsigned long entry)
+ {
+ 	struct page * page = lookup_swap_cache(entry);
+ 	if (page)
+ 	{
+ 		/*
+ 		 * The last reference in the swap_map[entry] is caused
+ 		 * by this swap cache page.
+ 		 *
+ 		 * Decrease the page->count increased by __find_page().
+ 		 *						-arca
+ 		 */
+ 		__free_page(page);
+ 		if (atomic_read(&page->count) == 1)
+ 			/*
+ 			 * The page is resident in memory only because
+ 			 * it' s in the swap cache so we can remove it
+ 			 * because it can' t be useful anymore.
+ 			 *					-arca
+ 			 */
+ 			delete_from_swap_cache(page);
+ 	}
+ }
+ 
  /* 
   * Locate a page of swap in physical memory, reserving swap cache space
   * and reading the disk if it is not already cached.  If wait==0, we are
Index: linux/mm/swapfile.c
diff -c linux/mm/swapfile.c:1.1.1.1 linux/mm/swapfile.c:1.1.1.1.16.1
*** linux/mm/swapfile.c:1.1.1.1	Fri Oct  2 19:22:39 1998
--- linux/mm/swapfile.c	Sun Nov  1 18:20:50 1998
***************
*** 144,153 ****
  		p->highest_bit = offset;
  	if (!p->swap_map[offset])
  		goto bad_free;
! 	if (p->swap_map[offset] < SWAP_MAP_MAX) {
! 		if (!--p->swap_map[offset])
  			nr_swap_pages++;
! 	}
  #ifdef DEBUG_SWAP
  	printk("DebugVM: swap_free(entry %08lx, count now %d)\n",
  	       entry, p->swap_map[offset]);
--- 144,158 ----
  		p->highest_bit = offset;
  	if (!p->swap_map[offset])
  		goto bad_free;
! 	if (p->swap_map[offset] < SWAP_MAP_MAX)
! 		switch(--p->swap_map[offset])
! 		{
! 		case 0:
  			nr_swap_pages++;
! 			break;
! 		case 1:
! 			try_to_free_last_swap_entry(entry);
! 		}
  #ifdef DEBUG_SWAP
  	printk("DebugVM: swap_free(entry %08lx, count now %d)\n",
  	       entry, p->swap_map[offset]);

Andrea Arcangeli

--
To unsubscribe, send a message with 'unsubscribe linux-mm my@address'
in the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://humbolt.geo.uu.nl/Linux-MM/