Re: Need some help in understanding sparsemem.

[KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>]

On Tue, 6 Jul 2010 10:41:06 +0530
naren.mehra@gmail.com wrote:

> Hi,
> 
> I am trying to understand the sparsemem implementation in linux for
> NUMA/multiple node systems.
> 
> From the available documentation and the sparsemem patches, I am able
> to make out that sparsemem divides memory into different sections and
> if the whole section contains a hole then its marked as invalid
> section and if some pages in a section form a hole then those pages
> are marked reserved. My issue is that this classification, I am not
> able to map it to the code.
> 
> e.g. from arch specific code, we call memory_present()  to prepare a
> list of sections in a particular node. but unable to find where
> exactly some sections are marked invalid because they contain a hole.
> 
> Can somebody tell me where in the code are we identifying sections as
> invalid and where we are marking pages as reserved.
> 

As you wrote, memory_present() is just for setting flags 
"SECTION_MARKED_PRESENT". If a section contains both of valid pages and
holes, the section itself is marked as SECTION_MARKED_PRESENT.

This memory_present() is called in very early stage. The function which allocates
mem_map(array of struct page) is sparse_init(). It's called somewhere after
memory_present().
(In x86, it's called by paging_init(), in ARM, it's called by bootmem_init()).

After sparse_init(), mem_maps are allocated. (depends on config..plz see codes.)
But, here, mem_map is not initialized.
This is because initialization logic of memmap doesn't depend on
FLATMEM/DISCONTIGMEM/SPARSEMEM.

After sprase_init(), mem_map is allocated. It's not encouraged to detect a section
is valid or invalid but you can use pfn_valid() to check there are memmap or not.
(*) pfn_valid(pfn) is not for detecting there is memory but for detecting
    there is memmap.

Initializing mem_map is done by free_area_init_node(). This function initializes
memory range regitered by add_active_range() (see mm/page_alloc.c)
(*)There are architecutures which doesn't use add_active_range(), but this function
   is for generic use.

After free_area_init_node(), all mem_map are initialized as PG_reserved and
NODE_DATA(nid)->star_pfn, etc..are available.

When PG_reserved is cleared is at free_all_bootmem(). If you want to keep pages
as Reserved (because of holes), OR, don't register memory hole as bootmem.
Then, pages will be kept as Reserved.

clarification:
 memory_present().... prepare for section[] and mark up PRESENT.
 sparse_init()   .... allocates mem_map. but just allocates it.
 free_area_init_node() .... initizalize mem_map at el.
 free_all_bootmem() .... make pages available and put into buddy allocator.

 pfn_valid() ... useful for checking there are mem_map.
 
 How to make pages kept as Reserved ....
                         reserve bootmem or not register to bootmem.

All aboves may depend on CONFIG, I hope this can be a hint for you.

Hmm. unexpectedly long..

Thanks,
-Kame

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>