Re: Page allocator: Single Zone optimizations

[Mel Gorman <mel@csn.ul.ie>]

On Thu, 2 Nov 2006, Christoph Lameter wrote:

> On Thu, 2 Nov 2006, Mel Gorman wrote:
>
>> Ok... list-based anti-frag identified three types of pages. From the leading
>> mail;
>>
>> EasyReclaimable - These are userspace pages that are easily reclaimable. This
>>         flag is set when it is known that the pages will be trivially
>> reclaimed
>>         by writing the page out to swap or syncing with backing storage
>>
>> KernelReclaimable - These are allocations for some kernel caches that are
>>         reclaimable or allocations that are known to be very short-lived.
>>
>> KernelNonReclaimable - These are pages that are allocated by the kernel that
>>         are not trivially reclaimed. For example, the memory allocated for a
>>         loaded module would be in this category. By default, allocations are
>>         considered to be of this type
>>
>> The EasyReclaimable and KernelReclaimable allocations are marked with __GFP
>> flags.
>>
>> Now, you want to separate pages according to movable and unmovable. Broadly
>> speaking, EasyReclaimable == Movable and
>> KernelReclaimable+KernelNonReclaimable == Non-Movable. However, while
>> KernelReclaimable are Non-Movable, they can be reclaimed by purging caches.
>> So, if we redefined the three terms to be Movable, Reclaimable and
>> Non-Movable, you get the separation you are looking for at least within a
>> MAX_ORDER_NR_PAGES.
>
> I think talking about reclaim here is not what you want. defragmentation
> is fundamentally about moving memor not reclaim.

Sure. That is why I called the mechanism anti-fragmentation, not 
defragmentation. However, If reclaimable pages are clustered together, you 
know they are moveable as well. Once the pages are clustered together in 
an intelligent manner, a hypothetical defragmenter would have less work to 
do. Additionally, once the defragmenter starts working, you know it's less 
likely to hit unmovable pages.

> Reclaim is a way of
> evicting pages from memory to avoid the move. This may be useful if memory
> is filled up because defragging can then do what swapping would have to
> do. However, evicting pages means that they have to be reread. Page
> migration can migrate pages at 1GB/sec which is certainly much higher
> than having to reread the page.
>

The reason why anti-frag currently reclaims is because reclaiming was easy 
and happens under memory pressure not because I thought pageout was free. 
As a proof-of-concept, I needed to show that pages clustered on 
reclaimability would free contiguous blocks of pages later. There was no 
point starting with defragmentation when I knew that unmovable pages would 
be with movable pages in the same MAX_ORDER_NR_PAGES block.

> Also I think the reclaim idea breaks down in the following cases:
>
> 1. An mlocked page. This is a page that is movable but not reclaimable.
> How does defrag handle that case right now? It should really move the
> page if necessary.
>

Defrag doesn't exist right now. If anti-frag got some traction, working on 
using page migration to handle movable-but-not-reclaimable pages would be 
the next step. Pages that are mlocked() will have been allocated with 
__GFP_EASYRCLM so will be clustered together with other movable pages.

> 2. There are a number of unreclaimable page types that are easily movable.
> F.e. page table pages are movable if you take a write-lock on mmap_sem
> and handle the tree carefully. These pages again are not reclaimable but
> they are movable.
>

Page tables are currently not allocated with __GFP_EASYRCLM because I knew 
I couldn't reclaim them without killing processes. However, if page 
migration within ranges was implemented, we'd start clustering based on 
movability instead of reclaimability.

> Various caching objects in the slab (cpucache align cache etc) are also
> easily movable. If we put them into a separate slab cache then we could
> make them movable.
>

As subsystems will have pointers to objects within the slab, I doubt they 
are easily movable but I'll take your word on it for the moment.

> Certain Device drivers may be able to shut down intermittendly releasing
> their memory and reallocating it later. This also may be used to move
> memory. Memory allocated by such a device driver is movable.
>

If such a driver existed in the future, their allocations could be marked 
and clustered together with other movable allocations.

> I would suggest to not categorize pages according to their reclaimability
> but according to their movability.

ok, I see your point. However, reclaimability seems a reasonable starting 
point. If I know pages of similar reclaimability are clustered together, I 
can work on using page migration to move pages out of the blocks of known 
reclaimability instead of paging them out. When that works, the __GFP_ 
flags identifying reclaimability can be renamed to marking movability and 
flag page table pages as well. This is a logical progression.

> The decision to evict a page (reclaim)
> is something that may be useful to avoid swap but it may be better to keep
> pages in memory.
>

Agreed, but swapping them out was an easier starting point.

-- 
Mel Gorman
Part-time Phd Student                          Linux Technology Center
University of Limerick                         IBM Dublin Software Lab

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>