RE: [patch] vmsig: notify user applications of virtual memory events via real-time signals

RE: [patch] vmsig: notify user applications of virtual memory events via real-time signals

[Emery Berger]

> That seems pretty high overhead.  I wonder if it wouldn't work
> similarly well for the kernel to simply notify the registrered
> apps that memory is running low and they should garbage collect
> _something_, without caring which pages.

Actually, it's quite important that the application know exactly which
page is being evicted, in order that it be "bookmarked". We found that
this particular aspect of the garbage collection algorithm was crucial
(it's in the paper).

Best,
-- emery

--
Emery Berger
Assistant Professor
Dept. of Computer Science
University of Massachusetts, Amherst
www.cs.umass.edu/~emery
 

> -----Original Message-----
> From: Rik van Riel [mailto:riel@redhat.com]
> Sent: Tuesday, November 22, 2005 4:45 PM
> To: Yi Feng
> Cc: linux-mm@kvack.org; 'Andrew Morton'; Emery Berger; 'Matthew Hertz'
> Subject: Re: [patch] vmsig: notify user applications of virtual memory
> events via real-time signals
> 
> On Tue, 22 Nov 2005, Yi Feng wrote:
> 
> > The user application can therefore maintain the residence
information of
> > all its pages and cooperate with the kernel under memory pressure.
> 
> That seems pretty high overhead.  I wonder if it wouldn't work
> similarly well for the kernel to simply notify the registrered
> apps that memory is running low and they should garbage collect
> _something_, without caring which pages.
> 
> Then the apps can "shoot holes" in their memory use by calling
> madvise with MADV_DONTNEED on the pages the application judges
> to be the least likely ones to be used again.
> 
> OTOH, maybe keeping state for each page is low enough overhead.
> I will have to read your patch to figure out the details ;)
> 
> --
> All Rights Reversed

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

RE: [patch] vmsig: notify user applications of virtual memory events via real-time signals

[Rohit Seth]

On Tue, 2005-11-22 at 13:53 -0800, Emery Berger wrote:
> > That seems pretty high overhead.  I wonder if it wouldn't work
> > similarly well for the kernel to simply notify the registrered
> > apps that memory is running low and they should garbage collect
> > _something_, without caring which pages.
> 
> Actually, it's quite important that the application know exactly which
> page is being evicted, in order that it be "bookmarked". We found that
> this particular aspect of the garbage collection algorithm was crucial
> (it's in the paper).
> 
Seems like a good idea for the notifications.

But for it to be useful for low memory conditions, I think it will
better if kernel knew (at possibly direct reclaim time) to swap out the
specific pages...so as to make it more cooperative between user and
kernel.  If kernel has to first notify user app (and possibly thousands
of them) that it is looking for free pages then it will probably be too
late or too expensive before an application actually completes the
operation of freeing the pages. 

-rohit

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

RE: [patch] vmsig: notify user applications of virtual memory events via real-time signals

[Yi Feng]

> -----Original Message-----
> From: Rohit Seth [mailto:rohit.seth@intel.com]
> Sent: Tuesday, November 22, 2005 9:30 PM
> To: Emery Berger
> Cc: Rik van Riel; Yi Feng; linux-mm@kvack.org; Andrew Morton; Matthew
> Hertz
> Subject: RE: [patch] vmsig: notify user applications of virtual
> memoryevents via real-time signals
> 
> On Tue, 2005-11-22 at 13:53 -0800, Emery Berger wrote:
> > > That seems pretty high overhead.  I wonder if it wouldn't work
> > > similarly well for the kernel to simply notify the registrered
> > > apps that memory is running low and they should garbage collect
> > > _something_, without caring which pages.
> >

The kernel overhead is low. The only space overhead is the rmap's we keep
for swapped-out pages - that's 8 bytes per swapped-out page plus the
associated radix tree structures we use for lookup. For time overhead, there
won't be any if the kernel is not swapping at all. If the kernel is
swapping, then we have to look at the rmap's of the involved pages and find
the process(es) to notify. However, this overhead is well justified because
when the system is swapping, the CPU is often under-utilized anyway.

> > Actually, it's quite important that the application know exactly which
> > page is being evicted, in order that it be "bookmarked". We found that
> > this particular aspect of the garbage collection algorithm was crucial
> > (it's in the paper).
> >
> Seems like a good idea for the notifications.
> 
> But for it to be useful for low memory conditions, I think it will
> better if kernel knew (at possibly direct reclaim time) to swap out the
> specific pages...so as to make it more cooperative between user and
> kernel.  If kernel has to first notify user app (and possibly thousands
> of them) that it is looking for free pages then it will probably be too
> late or too expensive before an application actually completes the
> operation of freeing the pages.
> 

We notify the user application of the impending eviction of a page before
the page is actually swapped out. We keep a boundary in the inactive list,
once a page slips behind the boundary and falls toward the end of the list,
we notify the user applications that this page is likely to be swapped out
soon. We made this boundary an adjustable parameter in /proc. 

When the application receives this notification and starts to process this
page, this page will stay in core (possibly for a fairly long time) because
it's been touched again. That's why we also added madvise(MADV_RELINQUISH)
to explicitly send the page to swap after the processing.


Yi Feng


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

RE: [patch] vmsig: notify user applications of virtual memory events via real-time signals

[Rik van Riel]

On Wed, 23 Nov 2005, Yi Feng wrote:

> When the application receives this notification and starts to process this
> page, this page will stay in core (possibly for a fairly long time) because
> it's been touched again. That's why we also added madvise(MADV_RELINQUISH)
> to explicitly send the page to swap after the processing.

Would it be better for the application to completely vacate
the page, so MADV_DONTNEED can be used instead, and swap IO
can be avoided ?

-- 
All Rights Reversed

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

RE: [patch] vmsig: notify user applications of virtual memory events via real-time signals

[Yi Feng]

> -----Original Message-----
> From: Rik van Riel [mailto:riel@redhat.com]
> Sent: Wednesday, November 23, 2005 8:11 AM
> To: Yi Feng
> Cc: 'Rohit Seth'; 'Emery Berger'; linux-mm@kvack.org; 'Andrew Morton';
> 'Matthew Hertz'
> Subject: RE: [patch] vmsig: notify user applications of virtual memory
> events via real-time signals
> 
> On Wed, 23 Nov 2005, Yi Feng wrote:
> 
> > When the application receives this notification and starts to process
> this
> > page, this page will stay in core (possibly for a fairly long time)
> because
> > it's been touched again. That's why we also added
> madvise(MADV_RELINQUISH)
> > to explicitly send the page to swap after the processing.
> 
> Would it be better for the application to completely vacate
> the page, so MADV_DONTNEED can be used instead, and swap IO
> can be avoided ?
> 

If the application can make the page completely useless, then it can use
MADV_DONTNEED. However, for some applications (e.g. our Bookmarking
Collection) the processed page may still contain useful data and can't be
simply discarded. And because it was chosen as an eviction victim before the
processing, it's deemed cold by the kernel, so we send it to swap with
MADV_RELINQUISH.


Yi Feng


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

RE: [patch] vmsig: notify user applications of virtual memory events via real-time signals

[Emery Berger]

> Would it be better for the application to completely vacate
> the page, so MADV_DONTNEED can be used instead, and swap IO
> can be avoided ?

Yes, but under severe memory pressure, it is not possible.

-- emery

--
Emery Berger
Assistant Professor
Dept. of Computer Science
University of Massachusetts, Amherst
www.cs.umass.edu/~emery


> -----Original Message-----
> From: Rik van Riel [mailto:riel@redhat.com]
> Sent: Wednesday, November 23, 2005 8:11 AM
> To: Yi Feng
> Cc: 'Rohit Seth'; Emery Berger; linux-mm@kvack.org; 'Andrew Morton';
> 'Matthew Hertz'
> Subject: RE: [patch] vmsig: notify user applications of virtual memory
> events via real-time signals
> 
> On Wed, 23 Nov 2005, Yi Feng wrote:
> 
> > When the application receives this notification and starts to
process
> this
> > page, this page will stay in core (possibly for a fairly long time)
> because
> > it's been touched again. That's why we also added
> madvise(MADV_RELINQUISH)
> > to explicitly send the page to swap after the processing.
> 
> Would it be better for the application to completely vacate
> the page, so MADV_DONTNEED can be used instead, and swap IO
> can be avoided ?
> 
> --
> All Rights Reversed

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[patch] vmsig: notify user applications of virtual memory events via real-time signals

[Yi Feng]

Linux-mm community:

I apologize for this long email. I'm presenting a kernel patch that the
community might be interested in.

We are a computer science research group at University of Massachusetts
Amherst, focusing on cooperative memory management across the OS kernel and
the user applications (particular the Java ones). We have developed a patch,
"vmsig", for the Linux kernel that enables kernel-to-user communication on
the virtual memory events. Currently our communication method is through an
unused real-time signal.

By using this patch, a user application can register with the kernel (via a
new system call vmsig_register) to receive RT signals on VM events on its
memory pages. There are 5 types of VM events as listed below:

VM_EVICTING_CLEAN: the page will soon be swapped out as a clean page
VM_EVICTING_DIRTY: the page will soon be swapped out as a dirty page
VM_SWAPPED_OUT: the page has been swapped out
VM_FAULTED_IN: the page has been faulted in by the application
VM_PREFETCHED: the page has been brought into the swap cache by the VM
prefetcher

The user application can therefore maintain the residence information of all
its pages and cooperate with the kernel under memory pressure. For example,
upon receiving the VM_EVICTING_* signals, the user application can
intelligently process the page, and optionally call madvise(MADV_DONTNEED)
to discard the page if it's no longer useful, or call madvise with our new
flag, MADV_RELIQUISH, to schedule the page to be swapped out (thus its
content will be saved on disk).

We have already developed a new garbage collector for Java based on this
kernel patch and published our work "Garbage Collection without Paging" on
PLDI 2005 (http://www.cs.umass.edu/~emery/pubs/f034-hertz.pdf). We would
expect more research work on this kind of cooperative memory management if
this patch can be merged into the main kernel.

Regarding the implementation details of the patch, we have included a brief
description in Documentation/vm/vmsig in the patch itself. The core part of
the patch is to maintain the ownership information of swapped-out pages as
well as resident pages. We naturally extended the existing rmap to swap
pages. This patch also complements mincore to work on anonymous pages.

The vmsig patch for Linux 2.6.14 is available at
http://www.cs.umass.edu/~yifeng/kernel/linux-2.6.14-vmsig.patch. It's fairly
large so it's probably not appropriate to include it in the body of this
email. Also because of the size of the patch, there are many details that I
didn't cover in this email. If this is the case, please send me email. I
will try to answer all your questions and clear the confusions.

Thanks and your comments and suggestions are welcome!


Yi Feng


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [patch] vmsig: notify user applications of virtual memory events via real-time signals

[Rik van Riel]

On Tue, 22 Nov 2005, Yi Feng wrote:

> The user application can therefore maintain the residence information of 
> all its pages and cooperate with the kernel under memory pressure.

That seems pretty high overhead.  I wonder if it wouldn't work
similarly well for the kernel to simply notify the registrered
apps that memory is running low and they should garbage collect
_something_, without caring which pages.

Then the apps can "shoot holes" in their memory use by calling
madvise with MADV_DONTNEED on the pages the application judges
to be the least likely ones to be used again.

OTOH, maybe keeping state for each page is low enough overhead.
I will have to read your patch to figure out the details ;)

-- 
All Rights Reversed

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>