From mboxrd@z Thu Jan 1 00:00:00 1970 From: Johannes Weiner Subject: Re: [PATCH 0/3] bootmem2 III References: <20080509151713.939253437@saeurebad.de> <20080509184044.GA19109@one.firstfloor.org> <87lk2gtzta.fsf@saeurebad.de> <48275493.40601@firstfloor.org> <874p92qsvn.fsf@saeurebad.de> <20080515191210.GE21787@shadowen.org> Date: Fri, 16 May 2008 22:42:51 +0200 In-Reply-To: <20080515191210.GE21787@shadowen.org> (Andy Whitcroft's message of "Thu, 15 May 2008 20:12:10 +0100") Message-ID: <87zlqqx9o4.fsf@saeurebad.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-linux-mm@kvack.org Return-Path: To: Andy Whitcroft Cc: Andi Kleen , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Ingo Molnar , Yinghai Lu , Andrew Morton , Linus Torvalds List-ID: Hi Andy, Andy Whitcroft writes: > On Tue, May 13, 2008 at 02:40:44PM +0200, Johannes Weiner wrote: >> Hi, >> >> Andi Kleen writes: >> >> > Johannes Weiner wrote: >> > >> >>> On Fri, May 09, 2008 at 05:17:13PM +0200, Johannes Weiner wrote: >> >>>> here is bootmem2, a memory block-oriented boot time allocator. >> >>>> >> >>>> Recent NUMA topologies broke the current bootmem's assumption that >> >>>> memory nodes provide non-overlapping and contiguous ranges of pages. >> >>> I'm still not sure that's a really good rationale for bootmem2. >> >>> e.g. the non continuous nodes are really special cases and there tends >> >>> to be enough memory at the beginning which is enough for boot time >> >>> use, so for those systems it would be quite reasonably to only >> >>> put the continuous starts of the nodes into bootmem. >> >> >> >> Hm, that would put the logic into arch-code. I have no strong opinion >> >> about it. >> > >> > In fact I suspect the current code will already work like that >> > implicitely. The aliasing is only a problem for the new "arbitary node >> > free_bootmem" right? >> >> And that alloc_bootmem_node() can not garuantee node-locality which is >> the much worse part, I think. >> >> >>> That said the bootmem code has gotten a little crufty and a clean >> >>> rewrite might be a good idea. >> >> >> >> I agree completely. >> > >> > The trouble is just that bootmem is used in early boot and early boot is >> > very subtle and getting it working over all architectures could be a >> > challenge. Not wanting to discourage you, but it's not exactly the >> > easiest part of the kernel to hack on. >> >> Bootmem seemed pretty self-contained to me, at least in the beginning. >> The bad thing is that I can test only the most simple configuration with >> it. >> >> I was wondering yesterday if it would be feasible to enforce >> contiguousness for nodes. So that arch-code does not create one pgdat >> for each node but one for each contiguous block. I have not yet looked > > That re-introduces the concept that a node is not a unit of numa locality, > but one of memory contiguity. The kernel pretty much assumes that a node > exhibits memory locality. Okay. >> deeper into it, but I suspect that other mm code has similar problems >> with nodes spanning other nodes. > > One thing we do know is that we already have systems in the wild with > overlapping nodes. PowerPC systems sometimes exhibit this behaviour, the > ones I have seen have node 1 embedded within node 0. x86_64 also enables > this support. This necessitated checks when initially freeing memory > into the allocator to make sure it ended up freed into the right node. > For non-sparsemem configurations these systems have some wasted mem_map, > but otherwise it does work. > > Check out NODES_SPAN_OTHER_NODES for the code to avoid miss-placing > memory. Will have a better look at all this. Thanks for the comment. Hannes -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org