Re: sparsemem support for mips with highmem

["Jon Fraser" <jfraser@broadcom.com>]

David,

One of the reasons that we've gone to the HIGHMEM solution
is to conserve address space.  We were mapping up to 512Mb of dram space
into the kernel via wired TLB entries.  We have 256MB of dram and
256mb of IO space in kseg0.  This left only 512 mb to map pci space,
vmalloc'd memory, etc.  This wasn't enough, and we couldn't handle
systems with 1gb of memory.  And it was wiring a lot of tlb entries,
which means more tlb faults.

Our processors only had 32 tlb entries, although we've
recently increased that to 64.

We do have to setup a wired tlb entry for the base of the high mem so
that we have access to the page array at the beginning of the high mem
region.  There might be another way to do it, but it's only 1 tlb entry.

Jon,  who is still chasing cache alias issues on a 24k.



On Tue, 2008-08-19 at 16:38 -0700, David VomLehn wrote:
> Christoph Lameter wrote:
> > David VomLehn wrote:
> > 
> >> On MIPS processors, the kernel runs in unmapped memory, i.e. the TLB
> >> isn't even
> >> used, so I don't think you can use that trick. So, this comment doesn't
> >> apply to
> >> all processors.
> > 
> > In that case you have a choice between the overhead of sparsemem lookups in
> > every pfn_to_page or using TLB entries to create a virtually mapped memmap
> > which may create TLB pressure.
> > 
> > The virtually mapped memmap results in smaller code and is typically more
> > effective since the processor caches the TLB entries.
> 
> I'm pretty ignorant on this subject, but I think this is worth discussing. On a 
> MIPS processor, access to low memory bypasses the TLB entirely. I think what you 
> are suggesting is to use mapped addresses to make all of low memory virtually 
> contiguous. On a MIPS processor, we could do this by allocating a "wired" TLB 
> entry for each physically contiguous block of memory. Wired TLB entries are never 
> replaced, so they are very efficient for long-lived mappings such as this. Using 
> the TLB in this way does increase TLB pressure, but most platforms probably have 
> a very small number of "holes" in their memory. So, this may be a small overhead.
> 
> If we took this approach, we could then have a single, simple memmap array where 
> pfn_to_page looks just about the same as it looks with a flat memory model.
> 
> If I'm understand what you are suggesting correctly (a big if), the downside is 
> that we'd pay the cost of a TLB match for each non-cached low memory data access. 
> It seems to me that would be a higher cost than having the occasional, more 
> expensive, sparsemem lookup in pfn_to_page.
> 
> Anyone with more in-depth MIPS processor architecture knowledge care to weigh in 
> on this?
> --
> David VomLehn
> 
> 
> 


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>