From: Wolfgang Wander <wwc@rentec.com>
To: Andrew Morton <akpm@osdl.org>
Cc: Wolfgang Wander <wwc@rentec.com>,
kenneth.w.chen@intel.com, mingo@elte.hu, arjanv@redhat.com,
linux-mm@kvack.org
Subject: [PATCH] Avoiding mmap fragmentation (against 2.6.12-rc4) to
Date: Wed, 11 May 2005 10:36:03 -0400 [thread overview]
Message-ID: <17026.6227.225173.588629@gargle.gargle.HOWL> (raw)
In-Reply-To: <20050510125747.65b83b4c.akpm@osdl.org>
The patch below is against linux-2.6.12-rc4.
Ingo recently introduced a great speedup for allocating new
mmaps using the free_area_cache pointer which boosts the specweb
SSL benchmark by 4-5% and causes huge performance increases in
thread creation.
The downside of this patch is that it does lead to fragmentation
in the mmap-ed areas (visible via /proc/self/maps), such that
some applications that work fine under 2.4 kernels quickly run
out of memory on any 2.6 kernel.
The problem is twofold:
1) the free_area_cache is used to continue a search for
memory where the last search ended. Before the change
new areas were always searched from the base address on.
So now new small areas are cluttering holes of all sizes
throughout the whole mmap-able region whereas before small
holes tended to close holes near the base leaving holes
far from the base large and available for larger requests.
2) the free_area_cache also is set to the location of the last
munmap-ed area so in scenarios where we allocate e.g.
five regions of 1K each, then free regions 4 2 3 in this
order the next request for 1K will be placed in the position
of the old region 3, whereas before we appended it to the
still active region 1, placing it at the location of the old
region 2. Before we had 1 free region of 2K, now we only
get two free regions of 1K -> fragmentation.
The patch adresses thes issues by introducing yet another cache
descriptor cached_hole_size that contains the largest known hole
size below the current free_area_cache. If a new request comes
in the size is compared against the cached_hole_size and if the
request can be filled with a hole below free_area_cache the
search is started from the base instead.
The results look promising: Whereas 2.6.12-rc4 fragments
quickly and my (earlier posted) leakme.c test program terminates
after 50000+ iterations with 96 distinct and fragmented maps in
/proc/self/maps it performs nicely (as expected) with thread creation,
Ingo's test_str02 with 20000 threads requires 0.7s system time.
Taking out Ingo's patch (un-patch available per request) by basically
deleting all mentions of free_area_cache from the kernel and starting
the search for new memory always at the respective bases we observe:
leakme terminates successfully with 11 distinctive hardly fragmented
areas in /proc/self/maps but thread creating is gringdingly slow:
30+s(!) system time for Ingo's test_str02 with 20000 threads.
Now - drumroll ;-) the appended patch works fine with leakme: it
ends with only 7 distinct areas in /proc/self/maps and also thread
creation seems sufficiently fast with 0.71s for 20000 threads.
Wolfgang
----------------------------------------
Signed-off-by: Wolfgang Wander <wwc@rentec.com>
----------------------------------------
diff -rpu linux-2.6.12-rc4-vanilla/arch/arm/mm/mmap.c linux-2.6.12-rc4-wwc/arch/arm/mm/mmap.c
--- linux-2.6.12-rc4-vanilla/arch/arm/mm/mmap.c 2005-03-02 02:38:10.000000000 -0500
+++ linux-2.6.12-rc4-wwc/arch/arm/mm/mmap.c 2005-05-10 16:33:34.363204724 -0400
@@ -73,8 +73,13 @@ arch_get_unmapped_area(struct file *filp
(!vma || addr + len <= vma->vm_start))
return addr;
}
- start_addr = addr = mm->free_area_cache;
-
+ if( len > mm->cached_hole_size )
+ start_addr = addr = mm->free_area_cache;
+ else {
+ start_addr = TASK_UNMAPPED_BASE;
+ mm->cached_hole_size = 0;
+ }
+
full_search:
if (do_align)
addr = COLOUR_ALIGN(addr, pgoff);
@@ -90,6 +95,7 @@ full_search:
*/
if (start_addr != TASK_UNMAPPED_BASE) {
start_addr = addr = TASK_UNMAPPED_BASE;
+ mm->cached_hole_size = 0;
goto full_search;
}
return -ENOMEM;
@@ -101,6 +107,8 @@ full_search:
mm->free_area_cache = addr + len;
return addr;
}
+ if( addr + mm->cached_hole_size < vma->vm_start )
+ mm->cached_hole_size = vma->vm_start - addr;
addr = vma->vm_end;
if (do_align)
addr = COLOUR_ALIGN(addr, pgoff);
diff -rpu linux-2.6.12-rc4-vanilla/arch/i386/mm/hugetlbpage.c linux-2.6.12-rc4-wwc/arch/i386/mm/hugetlbpage.c
--- linux-2.6.12-rc4-vanilla/arch/i386/mm/hugetlbpage.c 2005-05-10 18:28:55.902605331 -0400
+++ linux-2.6.12-rc4-wwc/arch/i386/mm/hugetlbpage.c 2005-05-10 16:33:34.364204677 -0400
@@ -294,7 +294,12 @@ static unsigned long hugetlb_get_unmappe
struct vm_area_struct *vma;
unsigned long start_addr;
- start_addr = mm->free_area_cache;
+ if( len > mm->cached_hole_size )
+ start_addr = mm->free_area_cache;
+ else {
+ start_addr = TASK_UNMAPPED_BASE;
+ mm->cached_hole_size = 0;
+ }
full_search:
addr = ALIGN(start_addr, HPAGE_SIZE);
@@ -308,6 +313,7 @@ full_search:
*/
if (start_addr != TASK_UNMAPPED_BASE) {
start_addr = TASK_UNMAPPED_BASE;
+ mm->cached_hole_size = 0;
goto full_search;
}
return -ENOMEM;
@@ -316,6 +322,8 @@ full_search:
mm->free_area_cache = addr + len;
return addr;
}
+ if( addr + mm->cached_hole_size < vma->vm_start )
+ mm->cached_hole_size = vma->vm_start - addr;
addr = ALIGN(vma->vm_end, HPAGE_SIZE);
}
}
@@ -327,12 +335,17 @@ static unsigned long hugetlb_get_unmappe
struct mm_struct *mm = current->mm;
struct vm_area_struct *vma, *prev_vma;
unsigned long base = mm->mmap_base, addr = addr0;
+ unsigned long largest_hole = mm->cached_hole_size;
int first_time = 1;
/* don't allow allocations above current base */
if (mm->free_area_cache > base)
mm->free_area_cache = base;
+ if( len <= largest_hole ) {
+ largest_hole = 0;
+ mm->free_area_cache = base;
+ }
try_again:
/* make sure it can fit in the remaining address space */
if (mm->free_area_cache < len)
@@ -353,13 +366,20 @@ try_again:
* vma->vm_start, use it:
*/
if (addr + len <= vma->vm_start &&
- (!prev_vma || (addr >= prev_vma->vm_end)))
+ (!prev_vma || (addr >= prev_vma->vm_end))) {
/* remember the address as a hint for next time */
- return (mm->free_area_cache = addr);
- else
+ mm->cached_hole_size = largest_hole;
+ return (mm->free_area_cache = addr);
+ } else
/* pull free_area_cache down to the first hole */
- if (mm->free_area_cache == vma->vm_end)
+ if (mm->free_area_cache == vma->vm_end) {
mm->free_area_cache = vma->vm_start;
+ mm->cached_hole_size = largest_hole;
+ }
+
+ /* remember the largest hole we saw so far */
+ if( addr + largest_hole < vma->vm_start )
+ largest_hole = vma->vm_start - addr;
/* try just below the current vma->vm_start */
addr = (vma->vm_start - len) & HPAGE_MASK;
@@ -372,6 +392,7 @@ fail:
*/
if (first_time) {
mm->free_area_cache = base;
+ largest_hole = 0;
first_time = 0;
goto try_again;
}
@@ -382,6 +403,7 @@ fail:
* allocations.
*/
mm->free_area_cache = TASK_UNMAPPED_BASE;
+ mm->cached_hole_size = ~0UL;
addr = hugetlb_get_unmapped_area_bottomup(file, addr0,
len, pgoff, flags);
@@ -389,7 +411,8 @@ fail:
* Restore the topdown base:
*/
mm->free_area_cache = base;
-
+ mm->cached_hole_size = ~0UL;
+
return addr;
}
diff -rpu linux-2.6.12-rc4-vanilla/arch/ia64/kernel/sys_ia64.c linux-2.6.12-rc4-wwc/arch/ia64/kernel/sys_ia64.c
--- linux-2.6.12-rc4-vanilla/arch/ia64/kernel/sys_ia64.c 2005-05-10 18:28:55.929604069 -0400
+++ linux-2.6.12-rc4-wwc/arch/ia64/kernel/sys_ia64.c 2005-05-10 16:33:34.365204630 -0400
@@ -38,9 +38,15 @@ arch_get_unmapped_area (struct file *fil
if (REGION_NUMBER(addr) == REGION_HPAGE)
addr = 0;
#endif
- if (!addr)
- addr = mm->free_area_cache;
-
+ if (!addr) {
+ if( len > mm->cached_hole_size )
+ addr = mm->free_area_cache;
+ else {
+ addr = TASK_UNMAPPED_BASE;
+ mm->cached_hole_size = 0;
+ }
+ }
+
if (map_shared && (TASK_SIZE > 0xfffffffful))
/*
* For 64-bit tasks, align shared segments to 1MB to avoid potential
@@ -59,6 +65,7 @@ arch_get_unmapped_area (struct file *fil
if (start_addr != TASK_UNMAPPED_BASE) {
/* Start a new search --- just in case we missed some holes. */
addr = TASK_UNMAPPED_BASE;
+ mm->cached_hole_size = 0;
goto full_search;
}
return -ENOMEM;
@@ -68,6 +75,8 @@ arch_get_unmapped_area (struct file *fil
mm->free_area_cache = addr + len;
return addr;
}
+ if( addr + mm->cached_hole_size < vma->vm_start )
+ mm->cached_hole_size = vma->vm_start - addr;
addr = (vma->vm_end + align_mask) & ~align_mask;
}
}
diff -rpu linux-2.6.12-rc4-vanilla/arch/ppc64/mm/hugetlbpage.c linux-2.6.12-rc4-wwc/arch/ppc64/mm/hugetlbpage.c
--- linux-2.6.12-rc4-vanilla/arch/ppc64/mm/hugetlbpage.c 2005-05-10 18:28:56.186592052 -0400
+++ linux-2.6.12-rc4-wwc/arch/ppc64/mm/hugetlbpage.c 2005-05-10 16:33:34.366204583 -0400
@@ -468,7 +468,12 @@ unsigned long arch_get_unmapped_area(str
&& !is_hugepage_only_range(mm, addr,len))
return addr;
}
- start_addr = addr = mm->free_area_cache;
+ if( len > mm->cached_hole_size )
+ start_addr = addr = mm->free_area_cache;
+ else {
+ start_addr = addr = TASK_UNMAPPED_BASE;
+ mm->cached_hole_size = 0;
+ }
full_search:
vma = find_vma(mm, addr);
@@ -492,6 +497,8 @@ full_search:
mm->free_area_cache = addr + len;
return addr;
}
+ if( addr + mm->cached_hole_size < vma->vm_start )
+ mm->cached_hole_size = vma->vm_start - addr;
addr = vma->vm_end;
vma = vma->vm_next;
}
@@ -499,6 +506,7 @@ full_search:
/* Make sure we didn't miss any holes */
if (start_addr != TASK_UNMAPPED_BASE) {
start_addr = addr = TASK_UNMAPPED_BASE;
+ mm->cached_hole_size = 0;
goto full_search;
}
return -ENOMEM;
@@ -520,6 +528,7 @@ arch_get_unmapped_area_topdown(struct fi
struct vm_area_struct *vma, *prev_vma;
struct mm_struct *mm = current->mm;
unsigned long base = mm->mmap_base, addr = addr0;
+ unsigned long largest_hole = mm->cached_hole_size;
int first_time = 1;
/* requested length too big for entire address space */
@@ -540,6 +549,10 @@ arch_get_unmapped_area_topdown(struct fi
return addr;
}
+ if( len <= largest_hole ) {
+ largest_hole = 0;
+ mm->free_area_cache = base;
+ }
try_again:
/* make sure it can fit in the remaining address space */
if (mm->free_area_cache < len)
@@ -568,13 +581,21 @@ hugepage_recheck:
* vma->vm_start, use it:
*/
if (addr+len <= vma->vm_start &&
- (!prev_vma || (addr >= prev_vma->vm_end)))
+ (!prev_vma || (addr >= prev_vma->vm_end))) {
/* remember the address as a hint for next time */
- return (mm->free_area_cache = addr);
+ mm->cached_hole_size = largest_hole;
+ return (mm->free_area_cache = addr);
+ }
else
/* pull free_area_cache down to the first hole */
- if (mm->free_area_cache == vma->vm_end)
+ if (mm->free_area_cache == vma->vm_end) {
mm->free_area_cache = vma->vm_start;
+ mm->cached_hole_size = largest_hole;
+ }
+
+ /* remember the largest hole we saw so far */
+ if( addr + largest_hole < vma->vm_start )
+ largest_hole = vma->vm_start - addr;
/* try just below the current vma->vm_start */
addr = vma->vm_start-len;
@@ -587,6 +608,7 @@ fail:
*/
if (first_time) {
mm->free_area_cache = base;
+ largest_hole = 0;
first_time = 0;
goto try_again;
}
@@ -597,12 +619,14 @@ fail:
* allocations.
*/
mm->free_area_cache = TASK_UNMAPPED_BASE;
+ mm->cached_hole_size = ~0UL;
addr = arch_get_unmapped_area(filp, addr0, len, pgoff, flags);
/*
* Restore the topdown base:
*/
mm->free_area_cache = base;
-
+ mm->cached_hole_size = ~0UL;
+
return addr;
}
diff -rpu linux-2.6.12-rc4-vanilla/arch/sh/kernel/sys_sh.c linux-2.6.12-rc4-wwc/arch/sh/kernel/sys_sh.c
--- linux-2.6.12-rc4-vanilla/arch/sh/kernel/sys_sh.c 2005-03-02 02:38:34.000000000 -0500
+++ linux-2.6.12-rc4-wwc/arch/sh/kernel/sys_sh.c 2005-05-10 16:33:34.366204583 -0400
@@ -79,6 +79,10 @@ unsigned long arch_get_unmapped_area(str
(!vma || addr + len <= vma->vm_start))
return addr;
}
+ if( len <= mm->cached_hole_size ) {
+ mm->cached_hole_size = 0;
+ mm->free_area_cache = TASK_UNMAPPED_BASE;
+ }
if (flags & MAP_PRIVATE)
addr = PAGE_ALIGN(mm->free_area_cache);
else
@@ -95,6 +99,7 @@ full_search:
*/
if (start_addr != TASK_UNMAPPED_BASE) {
start_addr = addr = TASK_UNMAPPED_BASE;
+ mm->cached_hole_size = 0;
goto full_search;
}
return -ENOMEM;
@@ -106,6 +111,9 @@ full_search:
mm->free_area_cache = addr + len;
return addr;
}
+ if( addr + mm->cached_hole_size < vma->vm_start )
+ mm->cached_hole_size = vma->vm_start - addr;
+
addr = vma->vm_end;
if (!(flags & MAP_PRIVATE))
addr = COLOUR_ALIGN(addr);
diff -rpu linux-2.6.12-rc4-vanilla/arch/sparc64/kernel/sys_sparc.c linux-2.6.12-rc4-wwc/arch/sparc64/kernel/sys_sparc.c
--- linux-2.6.12-rc4-vanilla/arch/sparc64/kernel/sys_sparc.c 2005-03-02 02:38:10.000000000 -0500
+++ linux-2.6.12-rc4-wwc/arch/sparc64/kernel/sys_sparc.c 2005-05-10 16:33:34.367204536 -0400
@@ -84,6 +84,10 @@ unsigned long arch_get_unmapped_area(str
return addr;
}
+ if( len <= mm->cached_hole_size ) {
+ mm->cached_hole_size = 0;
+ mm->free_area_cache = TASK_UNMAPPED_BASE;
+ }
start_addr = addr = mm->free_area_cache;
task_size -= len;
@@ -103,6 +107,7 @@ full_search:
if (task_size < addr) {
if (start_addr != TASK_UNMAPPED_BASE) {
start_addr = addr = TASK_UNMAPPED_BASE;
+ mm->cached_hole_size = 0;
goto full_search;
}
return -ENOMEM;
@@ -114,6 +119,9 @@ full_search:
mm->free_area_cache = addr + len;
return addr;
}
+ if( addr + mm->cached_hole_size < vma->vm_start )
+ mm->cached_hole_size = vma->vm_start - addr;
+
addr = vma->vm_end;
if (do_color_align)
addr = COLOUR_ALIGN(addr, pgoff);
diff -rpu linux-2.6.12-rc4-vanilla/arch/x86_64/ia32/ia32_aout.c linux-2.6.12-rc4-wwc/arch/x86_64/ia32/ia32_aout.c
--- linux-2.6.12-rc4-vanilla/arch/x86_64/ia32/ia32_aout.c 2005-05-10 18:28:56.386582700 -0400
+++ linux-2.6.12-rc4-wwc/arch/x86_64/ia32/ia32_aout.c 2005-05-10 16:33:34.367204536 -0400
@@ -312,6 +312,7 @@ static int load_aout_binary(struct linux
current->mm->brk = ex.a_bss +
(current->mm->start_brk = N_BSSADDR(ex));
current->mm->free_area_cache = TASK_UNMAPPED_BASE;
+ current->mm->cached_hole_size = 0;
set_mm_counter(current->mm, rss, 0);
current->mm->mmap = NULL;
diff -rpu linux-2.6.12-rc4-vanilla/arch/x86_64/kernel/sys_x86_64.c linux-2.6.12-rc4-wwc/arch/x86_64/kernel/sys_x86_64.c
--- linux-2.6.12-rc4-vanilla/arch/x86_64/kernel/sys_x86_64.c 2005-05-10 18:28:56.406581765 -0400
+++ linux-2.6.12-rc4-wwc/arch/x86_64/kernel/sys_x86_64.c 2005-05-10 16:33:34.368204490 -0400
@@ -111,6 +111,10 @@ arch_get_unmapped_area(struct file *filp
(!vma || addr + len <= vma->vm_start))
return addr;
}
+ if( len <= mm->cached_hole_size ) {
+ mm->cached_hole_size = 0;
+ mm->free_area_cache = begin;
+ }
addr = mm->free_area_cache;
if (addr < begin)
addr = begin;
@@ -126,6 +130,7 @@ full_search:
*/
if (start_addr != begin) {
start_addr = addr = begin;
+ mm->cached_hole_size = 0;
goto full_search;
}
return -ENOMEM;
@@ -137,6 +142,9 @@ full_search:
mm->free_area_cache = addr + len;
return addr;
}
+ if( addr + mm->cached_hole_size < vma->vm_start )
+ mm->cached_hole_size = vma->vm_start - addr;
+
addr = vma->vm_end;
}
}
diff -rpu linux-2.6.12-rc4-vanilla/fs/binfmt_aout.c linux-2.6.12-rc4-wwc/fs/binfmt_aout.c
--- linux-2.6.12-rc4-vanilla/fs/binfmt_aout.c 2005-05-10 18:28:59.957415723 -0400
+++ linux-2.6.12-rc4-wwc/fs/binfmt_aout.c 2005-05-10 16:33:34.368204490 -0400
@@ -316,6 +316,7 @@ static int load_aout_binary(struct linux
current->mm->brk = ex.a_bss +
(current->mm->start_brk = N_BSSADDR(ex));
current->mm->free_area_cache = current->mm->mmap_base;
+ current->mm->cached_hole_size = current->mm->cached_hole_size;
set_mm_counter(current->mm, rss, 0);
current->mm->mmap = NULL;
diff -rpu linux-2.6.12-rc4-vanilla/fs/binfmt_elf.c linux-2.6.12-rc4-wwc/fs/binfmt_elf.c
--- linux-2.6.12-rc4-vanilla/fs/binfmt_elf.c 2005-05-10 18:28:59.958415676 -0400
+++ linux-2.6.12-rc4-wwc/fs/binfmt_elf.c 2005-05-10 16:34:23.696894470 -0400
@@ -775,6 +775,7 @@ static int load_elf_binary(struct linux_
change some of these later */
set_mm_counter(current->mm, rss, 0);
current->mm->free_area_cache = current->mm->mmap_base;
+ current->mm->cached_hole_size = current->mm->cached_hole_size;
retval = setup_arg_pages(bprm, randomize_stack_top(STACK_TOP),
executable_stack);
if (retval < 0) {
diff -rpu linux-2.6.12-rc4-vanilla/fs/hugetlbfs/inode.c linux-2.6.12-rc4-wwc/fs/hugetlbfs/inode.c
--- linux-2.6.12-rc4-vanilla/fs/hugetlbfs/inode.c 2005-05-10 18:29:00.032412216 -0400
+++ linux-2.6.12-rc4-wwc/fs/hugetlbfs/inode.c 2005-05-10 16:33:34.370204396 -0400
@@ -122,6 +122,11 @@ hugetlb_get_unmapped_area(struct file *f
start_addr = mm->free_area_cache;
+ if(len <= mm->cached_hole_size )
+ start_addr = TASK_UNMAPPED_BASE;
+
+
+
full_search:
addr = ALIGN(start_addr, HPAGE_SIZE);
diff -rpu linux-2.6.12-rc4-vanilla/include/linux/sched.h linux-2.6.12-rc4-wwc/include/linux/sched.h
--- linux-2.6.12-rc4-vanilla/include/linux/sched.h 2005-05-10 18:29:02.918277269 -0400
+++ linux-2.6.12-rc4-wwc/include/linux/sched.h 2005-05-10 16:33:34.371204349 -0400
@@ -219,8 +219,9 @@ struct mm_struct {
unsigned long addr, unsigned long len,
unsigned long pgoff, unsigned long flags);
void (*unmap_area) (struct vm_area_struct *area);
- unsigned long mmap_base; /* base of mmap area */
- unsigned long free_area_cache; /* first hole */
+ unsigned long mmap_base; /* base of mmap area */
+ unsigned long cached_hole_size; /* if non-zero, the largest hole below free_area_cache */
+ unsigned long free_area_cache; /* first hole of size cached_hole_size or larger */
pgd_t * pgd;
atomic_t mm_users; /* How many users with user space? */
atomic_t mm_count; /* How many references to "struct mm_struct" (users count as 1) */
diff -rpu linux-2.6.12-rc4-vanilla/kernel/fork.c linux-2.6.12-rc4-wwc/kernel/fork.c
--- linux-2.6.12-rc4-vanilla/kernel/fork.c 2005-05-10 18:29:02.994273715 -0400
+++ linux-2.6.12-rc4-wwc/kernel/fork.c 2005-05-10 16:33:34.372204302 -0400
@@ -194,6 +194,7 @@ static inline int dup_mmap(struct mm_str
mm->mmap = NULL;
mm->mmap_cache = NULL;
mm->free_area_cache = oldmm->mmap_base;
+ mm->cached_hole_size = ~0UL;
mm->map_count = 0;
set_mm_counter(mm, rss, 0);
set_mm_counter(mm, anon_rss, 0);
@@ -322,7 +323,8 @@ static struct mm_struct * mm_init(struct
mm->ioctx_list = NULL;
mm->default_kioctx = (struct kioctx)INIT_KIOCTX(mm->default_kioctx, *mm);
mm->free_area_cache = TASK_UNMAPPED_BASE;
-
+ mm->cached_hole_size = ~0UL;
+
if (likely(!mm_alloc_pgd(mm))) {
mm->def_flags = 0;
return mm;
diff -rpu linux-2.6.12-rc4-vanilla/mm/mmap.c linux-2.6.12-rc4-wwc/mm/mmap.c
--- linux-2.6.12-rc4-vanilla/mm/mmap.c 2005-05-10 18:29:03.031271985 -0400
+++ linux-2.6.12-rc4-wwc/mm/mmap.c 2005-05-10 17:57:17.869390186 -0400
@@ -1175,7 +1175,12 @@ arch_get_unmapped_area(struct file *filp
(!vma || addr + len <= vma->vm_start))
return addr;
}
- start_addr = addr = mm->free_area_cache;
+ if( len > mm->cached_hole_size )
+ start_addr = addr = mm->free_area_cache;
+ else {
+ start_addr = addr = TASK_UNMAPPED_BASE;
+ mm->cached_hole_size = 0;
+ }
full_search:
for (vma = find_vma(mm, addr); ; vma = vma->vm_next) {
@@ -1186,7 +1191,8 @@ full_search:
* some holes.
*/
if (start_addr != TASK_UNMAPPED_BASE) {
- start_addr = addr = TASK_UNMAPPED_BASE;
+ start_addr = addr = TASK_UNMAPPED_BASE;
+ mm->cached_hole_size = 0;
goto full_search;
}
return -ENOMEM;
@@ -1198,6 +1204,8 @@ full_search:
mm->free_area_cache = addr + len;
return addr;
}
+ if( addr + mm->cached_hole_size < vma->vm_start )
+ mm->cached_hole_size = vma->vm_start - addr;
addr = vma->vm_end;
}
}
@@ -1209,8 +1217,13 @@ void arch_unmap_area(struct vm_area_stru
* Is this a new hole at the lowest possible address?
*/
if (area->vm_start >= TASK_UNMAPPED_BASE &&
- area->vm_start < area->vm_mm->free_area_cache)
- area->vm_mm->free_area_cache = area->vm_start;
+ area->vm_start < area->vm_mm->free_area_cache) {
+ unsigned area_size = area->vm_end-area->vm_start;
+ if( area->vm_mm->cached_hole_size < area_size )
+ area->vm_mm->cached_hole_size = area_size;
+ else
+ area->vm_mm->cached_hole_size = ~0UL;
+ }
}
/*
@@ -1240,13 +1253,19 @@ arch_get_unmapped_area_topdown(struct fi
return addr;
}
+ /* check if free_area_cache is useful for us */
+ if( len <= mm->cached_hole_size ) {
+ mm->cached_hole_size = 0;
+ mm->free_area_cache = mm->mmap_base;
+ }
+
/* either no address requested or can't fit in requested address hole */
addr = mm->free_area_cache;
/* make sure it can fit in the remaining address space */
if (addr >= len) {
vma = find_vma(mm, addr-len);
- if (!vma || addr <= vma->vm_start)
+ if (!vma || addr <= vma->vm_start)
/* remember the address as a hint for next time */
return (mm->free_area_cache = addr-len);
}
@@ -1264,6 +1283,10 @@ arch_get_unmapped_area_topdown(struct fi
/* remember the address as a hint for next time */
return (mm->free_area_cache = addr);
+ /* remember the largest hole we saw so far */
+ if( addr + mm->cached_hole_size < vma->vm_start )
+ mm->cached_hole_size = vma->vm_start - addr;
+
/* try just below the current vma->vm_start */
addr = vma->vm_start-len;
} while (len <= vma->vm_start);
@@ -1274,13 +1297,15 @@ arch_get_unmapped_area_topdown(struct fi
* can happen with large stack limits and large mmap()
* allocations.
*/
- mm->free_area_cache = TASK_UNMAPPED_BASE;
+ mm->cached_hole_size = ~0UL;
+ mm->free_area_cache = TASK_UNMAPPED_BASE;
addr = arch_get_unmapped_area(filp, addr0, len, pgoff, flags);
/*
* Restore the topdown base:
*/
mm->free_area_cache = mm->mmap_base;
-
+ mm->cached_hole_size = ~0UL;
+
return addr;
}
#endif
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"aart@kvack.org"> aart@kvack.org </a>
next prev parent reply other threads:[~2005-05-11 14:36 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2005-05-09 13:30 Fw: [Bug 4520] New: /proc/*/maps fragments too quickly compared to Wolfgang Wander
2005-05-09 21:26 ` Andrew Morton
2005-05-09 21:30 ` Arjan van de Ven
2005-05-10 13:23 ` Wolfgang Wander
2005-05-10 15:35 ` Chen, Kenneth W
2005-05-10 18:58 ` Andrew Morton
2005-05-10 19:07 ` Arjan van de Ven
2005-05-10 19:34 ` Chen, Kenneth W
2005-05-10 19:43 ` Andrew Morton
2005-05-10 19:50 ` Wolfgang Wander
2005-05-10 19:57 ` Andrew Morton
2005-05-11 14:36 ` Wolfgang Wander [this message]
2005-05-11 15:05 ` [PATCH] Avoiding mmap fragmentation (against 2.6.12-rc4) to Chen, Kenneth W
2005-05-12 0:59 ` Andrew Morton
2005-05-12 1:33 ` Wolfgang Wander
2005-05-12 7:07 ` Ingo Molnar
2005-05-12 7:14 ` Ingo Molnar
2005-05-10 19:58 ` Fw: [Bug 4520] New: /proc/*/maps fragments too quickly compared to Ingo Molnar
2005-05-10 20:04 ` Wolfgang Wander
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=17026.6227.225173.588629@gargle.gargle.HOWL \
--to=wwc@rentec.com \
--cc=akpm@osdl.org \
--cc=arjanv@redhat.com \
--cc=kenneth.w.chen@intel.com \
--cc=linux-mm@kvack.org \
--cc=mingo@elte.hu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox