linux-mm.kvack.org archive mirror
 help / color / mirror / Atom feed
* [mmotm][PATCH 0/5] mm rss counting updates
@ 2009-12-15  9:09 KAMEZAWA Hiroyuki
  2009-12-15  9:11 ` [mmotm][PATCH 1/5] clean up mm_counter KAMEZAWA Hiroyuki
                   ` (4 more replies)
  0 siblings, 5 replies; 11+ messages in thread
From: KAMEZAWA Hiroyuki @ 2009-12-15  9:09 UTC (permalink / raw)
  To: linux-mm; +Cc: linux-kernel, akpm, cl, minchan.kim, Lee.Schermerhorn

[-- Attachment #1: Type: text/plain, Size: 1215 bytes --]


This is version 3 or 4....for rss counting. removed RFC.

My purpose is gathering more (rss-related) information per process without
scalability impact. (and improve oom-killer etc..)
The whole patch series is organized as

 [1/5] clean-up per mm stat counting.
 [2/5] making counter (a bit) more scalable with per-thread counting.
 [3/5] adding swap counter per mm
 [4/5] adding lowmem detection logic
 [5/5] adding lowmem usage counter per mm.

Big changes from previous one are...
  - removed per-cpu counter. added per-thread counter
  - synchronization point of a counter is moved to memory.c
    no hooks to ticks and scheduler.

Now, this patch is not very invasive as previous ones.

cache-miss/page fault with my benchmark on my box is

 [Before patch] 4.55 cache-miss/fault
 [After patch 2] 3.99 cache-miss/fault
 [After all patch] 4.06 cache-miss/fault

>From this numbers, I think swap/lowmem counters can be added.

My test program is attached (this is not modified from previous one)

[Future Plan]
 - add CONSTRAINT_LOWMEM oom killer.
 - add rss+swap based oom killer (with sysctl ?)
 - add some patch for perf ?
 - add mm_accessor patch.
 - improve page faults scalability, finally.

Thanks,
-Kame

[-- Attachment #2: multi-fault.c --]
[-- Type: text/x-csrc, Size: 1867 bytes --]

/*
 * multi-fault.c :: causes 60secs of parallel page fault in multi-thread.
 * % gcc -O2 -o multi-fault multi-fault.c -lpthread
 * % multi-fault # of cpus.
 */

#define _GNU_SOURCE
#include <stdio.h>
#include <pthread.h>
#include <sched.h>
#include <sys/mman.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <signal.h>

#define CORE_PER_SOCK	4
#define NR_THREADS	8
pthread_t threads[NR_THREADS];
/*
 * For avoiding contention in page table lock, FAULT area is
 * sparse. If FAULT_LENGTH is too large for your cpus, decrease it.
 */
#define MMAP_LENGTH	(8 * 1024 * 1024)
#define FAULT_LENGTH	(2 * 1024 * 1024)
void *mmap_area[NR_THREADS];
#define PAGE_SIZE	4096

pthread_barrier_t barrier;
int name[NR_THREADS];

void segv_handler(int sig)
{
	sleep(100);
}
void *worker(void *data)
{
	cpu_set_t set;
	int cpu;

	cpu = *(int *)data;

	CPU_ZERO(&set);
	CPU_SET(cpu, &set);
	sched_setaffinity(0, sizeof(set), &set);

	cpu /= CORE_PER_SOCK;

	while (1) {
		char *c;
		char *start = mmap_area[cpu];
		char *end = mmap_area[cpu] + FAULT_LENGTH;
		pthread_barrier_wait(&barrier);
		//printf("fault into %p-%p\n",start, end);

		for (c = start; c < end; c += PAGE_SIZE)
			*c = 0;
		pthread_barrier_wait(&barrier);

		madvise(start, FAULT_LENGTH, MADV_DONTNEED);
	}
	return NULL;
}

int main(int argc, char *argv[])
{
	int i, ret;
	unsigned int num;

	if (argc < 2)
		return 0;

	num = atoi(argv[1]);	
	pthread_barrier_init(&barrier, NULL, num);

	mmap_area[0] = mmap(NULL, MMAP_LENGTH * num, PROT_WRITE|PROT_READ,
				MAP_PRIVATE | MAP_ANONYMOUS, 0, 0);
	for (i = 1; i < num; i++) {
		mmap_area[i] = mmap_area[i - 1]+ MMAP_LENGTH;
	}

	for (i = 0; i < num; ++i) {
		name[i] = i * CORE_PER_SOCK;
		ret = pthread_create(&threads[i], NULL, worker, &name[i]);
		if (ret < 0) {
			perror("pthread create");
			return 0;
		}
	}
	sleep(60);
	return 0;
}

^ permalink raw reply	[flat|nested] 11+ messages in thread
end of thread, other threads:[~2009-12-15 23:56 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2009-12-15  9:09 [mmotm][PATCH 0/5] mm rss counting updates KAMEZAWA Hiroyuki
2009-12-15  9:11 ` [mmotm][PATCH 1/5] clean up mm_counter KAMEZAWA Hiroyuki
2009-12-15 23:25   ` Minchan Kim
2009-12-15 23:53     ` KAMEZAWA Hiroyuki
2009-12-15  9:13 ` [mmotm][PATCH 2/5] mm : avoid false sharing on mm_counter KAMEZAWA Hiroyuki
2009-12-15 15:25   ` Christoph Lameter
2009-12-15 16:54     ` KAMEZAWA Hiroyuki
2009-12-15 23:48     ` Minchan Kim
2009-12-15  9:14 ` [mmotm][PATCH 3/5] mm: count swap usage KAMEZAWA Hiroyuki
2009-12-15  9:15 ` [mmotm][PATCH 4/5] mm : add lowmem detection logic KAMEZAWA Hiroyuki
2009-12-15  9:16 ` [mmotm][PATCH 5/5] mm : count lowmem rss KAMEZAWA Hiroyuki

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox