[patch] memory hog protection

[patch] memory hog protection

[Rik van Riel]

Hi,

the patch below changes the mm->swap_cnt assignment to put
memory hogs at a disadvantage to programs with a smaller
RSS. In combination with my other patch (that still needs
some tuning) it has the property that memory pressure on
a task is proportional to the task's size.

If process A is 4 times smaller than process B, it's
memory pages will receive half the memory pressure the
pages of process B will receive. Or, put another way,
we'll spend 8 times as much effort trying to swap out
part of B as we'll spend on swapping out part of A.

It more or less preserves interactive response of the
machine and performance of smaller programs. It doesn't
work well with the standard agressiveness of shrink_mmap(),
but it seems to be quite decent with my shrink_mmap()
modifications ... but those need some tuning to deal well
with some specific situations :)

regards,

Rik
--
The Internet is not a network of computers. It is a network
of people. That is its real strength.

Wanna talk about the kernel?  irc.openprojects.net / #kernelnewbies
http://www.conectiva.com/		http://www.surriel.com/



--- linux/linux-2.3.99-pre6-3/mm/vmscan.c.orig	Mon Apr 17 12:21:46 2000
+++ linux/linux-2.3.99-pre6-3/mm/vmscan.c	Sun Apr 23 16:49:16 2000
@@ -34,7 +34,7 @@
  * using a process that no longer actually exists (it might
  * have died while we slept).
  */
-static int try_to_swap_out(struct vm_area_struct* vma, unsigned long address, pte_t * page_table, int gfp_mask)
+static int try_to_swap_out(struct mm_struct * mm, struct vm_area_struct* vma, unsigned long address, pte_t * page_table, int gfp_mask)
 {
 	pte_t pte;
 	swp_entry_t entry;
@@ -48,6 +48,7 @@
 	if ((page-mem_map >= max_mapnr) || PageReserved(page))
 		goto out_failed;
 
+	mm->swap_cnt--;
 	/* Don't look at this pte if it's been accessed recently. */
 	if (pte_young(pte)) {
 		/*
@@ -194,7 +195,7 @@
  * (C) 1993 Kai Petzke, wpp@marie.physik.tu-berlin.de
  */
 
-static inline int swap_out_pmd(struct vm_area_struct * vma, pmd_t *dir, unsigned long address, unsigned long end, int gfp_mask)
+static inline int swap_out_pmd(struct mm_struct * mm, struct vm_area_struct * vma, pmd_t *dir, unsigned long address, unsigned long end, int gfp_mask)
 {
 	pte_t * pte;
 	unsigned long pmd_end;
@@ -216,16 +217,18 @@
 	do {
 		int result;
 		vma->vm_mm->swap_address = address + PAGE_SIZE;
-		result = try_to_swap_out(vma, address, pte, gfp_mask);
+		result = try_to_swap_out(mm, vma, address, pte, gfp_mask);
 		if (result)
 			return result;
+		if (!mm->swap_cnt)
+			return 0;
 		address += PAGE_SIZE;
 		pte++;
 	} while (address && (address < end));
 	return 0;
 }
 
-static inline int swap_out_pgd(struct vm_area_struct * vma, pgd_t *dir, unsigned long address, unsigned long end, int gfp_mask)
+static inline int swap_out_pgd(struct mm_struct * mm, struct vm_area_struct * vma, pgd_t *dir, unsigned long address, unsigned long end, int gfp_mask)
 {
 	pmd_t * pmd;
 	unsigned long pgd_end;
@@ -245,16 +248,18 @@
 		end = pgd_end;
 	
 	do {
-		int result = swap_out_pmd(vma, pmd, address, end, gfp_mask);
+		int result = swap_out_pmd(mm, vma, pmd, address, end, gfp_mask);
 		if (result)
 			return result;
+		if (!mm->swap_cnt)
+			return 0;
 		address = (address + PMD_SIZE) & PMD_MASK;
 		pmd++;
 	} while (address && (address < end));
 	return 0;
 }
 
-static int swap_out_vma(struct vm_area_struct * vma, unsigned long address, int gfp_mask)
+static int swap_out_vma(struct mm_struct * mm, struct vm_area_struct * vma, unsigned long address, int gfp_mask)
 {
 	pgd_t *pgdir;
 	unsigned long end;
@@ -269,9 +274,11 @@
 	if (address >= end)
 		BUG();
 	do {
-		int result = swap_out_pgd(vma, pgdir, address, end, gfp_mask);
+		int result = swap_out_pgd(mm, vma, pgdir, address, end, gfp_mask);
 		if (result)
 			return result;
+		if (!mm->swap_cnt)
+			return 0;
 		address = (address + PGDIR_SIZE) & PGDIR_MASK;
 		pgdir++;
 	} while (address && (address < end));
@@ -299,7 +306,7 @@
 			address = vma->vm_start;
 
 		for (;;) {
-			int result = swap_out_vma(vma, address, gfp_mask);
+			int result = swap_out_vma(mm, vma, address, gfp_mask);
 			if (result)
 				return result;
 			vma = vma->vm_next;
@@ -369,9 +376,23 @@
 				pid = p->pid;
 			}
 		}
-		read_unlock(&tasklist_lock);
-		if (assign == 1)
+		if (assign == 1) {
+			/* we just assigned swap_cnt, normalise values */
 			assign = 2;
+			p = init_task.next_task;
+			for (; p != &init_task; p = p->next_task) {
+				int i = 0;
+				struct mm_struct *mm = p->mm;
+				if (!p->swappable || !mm || mm->rss <= 0)
+					continue;
+				/* small processes are swapped out less */
+				while ((mm->swap_cnt << 2 * i) < max_cnt)
+					i++;
+				mm->swap_cnt >>= i;
+				mm->swap_cnt += i; /* in case we reach 0 */
+			}
+		}
+		read_unlock(&tasklist_lock);
 		if (!best) {
 			if (!assign) {
 				assign = 1;

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: [patch] memory hog protection

[Jeff Garzik]

Rik van Riel wrote:
> the patch below changes the mm->swap_cnt assignment to put
> memory hogs at a disadvantage to programs with a smaller
> RSS.
[...]

There are many classes of problems where preserving interactivity at the
expense of a resource hog is a bad not good idea.  Think of obscure
situations like database servers for example :)

-- 
Jeff Garzik              | Nothing cures insomnia like the
Building 1024            | realization that it's time to get up.
MandrakeSoft, Inc.       |        -- random fortune
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: [patch] memory hog protection

[Arjan van de Ven]

In article <3903D353.D98969B7@mandrakesoft.com> you wrote:
>> the patch below changes the mm->swap_cnt assignment to put
>> memory hogs at a disadvantage to programs with a smaller

> There are many classes of problems where preserving interactivity at the
> expense of a resource hog is a bad not good idea.  Think of obscure
> situations like database servers for example :)

Is it really that bad to have a sysctl (or other /proc thingy) named
"boostinteractive" or whatever, and means that the owner of the machine
wants to favor interactive processes over memory/cpu hogs.
This can be used for
1) Rick's MM pressure stuff
2) The scheduler 
3) The OOM selection algorithm

Greetings,
    Arjan van de Ven
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: [patch] memory hog protection

[Rik van Riel]

On Mon, 24 Apr 2000, Jeff Garzik wrote:
> Rik van Riel wrote:
> > the patch below changes the mm->swap_cnt assignment to put
> > memory hogs at a disadvantage to programs with a smaller
> > RSS.
> [...]
> 
> There are many classes of problems where preserving
> interactivity at the expense of a resource hog is a bad not good
> idea.  Think of obscure situations like database servers for
> example :)

Firstly, if this code turns out to work, I'll make it sysctl
switchable.

Secondly, about the issue you bring up; suppose a database
server has 75% of memory and with this patch that would be
reduced to 70% of memory, that's only a very small difference
to the database server itself, but a BIG difference to the
dozen or so smaller processes in the system...

regards,

Rik
--
The Internet is not a network of computers. It is a network
of people. That is its real strength.

Wanna talk about the kernel?  irc.openprojects.net / #kernelnewbies
http://www.conectiva.com/		http://www.surriel.com/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: [patch] memory hog protection

[Jeff Garzik]

Rik van Riel wrote:
> Secondly, about the issue you bring up; suppose a database
> server has 75% of memory and with this patch that would be
> reduced to 70% of memory, that's only a very small difference
> to the database server itself, but a BIG difference to the
> dozen or so smaller processes in the system...

...dozen or so smaller processes which are far less important than the
database server :)

-- 
Jeff Garzik              | Nothing cures insomnia like the
Building 1024            | realization that it's time to get up.
MandrakeSoft, Inc.       |        -- random fortune
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/

Re: [patch] memory hog protection

[Stephen C. Tweedie]

Hi,

On Tue, Apr 25, 2000 at 04:45:30PM -0400, Jeff Garzik wrote:
> 
> ...dozen or so smaller processes which are far less important than the
> database server :)

That's exactly the property we need to avoid.  In many cases, when 
you have one large memory hog thrashing away, the one smaller process
which _really_ matters is the shell from which root is trying to sort
out the mess.

--Stephen
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/