Re: NUMA BOF @OLS

[Arnd Bergmann <arnd@arndb.de>]

On Friday 22 June 2007, Christoph Lameter wrote:
> 
> On Fri, 22 Jun 2007, Arnd Bergmann wrote:
> 
> > - Interface for preallocating hugetlbfs pages per node instead of system wide
> 
> We may want to get a bit higher level than that. General way of 
> controlling subsystem use on nodes. One wants to restrict the slab 
> allocator and the kernel etc on nodes too.
> 
> How will this interact with the other NUMA policy specifications?

I guess that's what I'd like to discuss at the BOF. I frequently
get requests from users that need to have some interface for it:
Application currently break if they try to use /proc/sys/vm/nr_hugepages
in combination with numactl --membind.

> > - architecture independent in-kernel API for enumerating CPU sockets with
> >   multicore processors (not sure if that's the same as your existing subject).
> 
> Not sure what you mean by this. We already have a topology interface and 
> the scheduler knows about these things.

I'm not referring to user interfaces or scheduling. It's probably not really
a NUMA topic, but we currently use the topology interfaces for enumerating
sockets on systems that are not really NUMA. This includes stuff like
per-socket 
 * cpufreq settings (these have their own logic currently)
 * IOMMU
 * performance counters
 * thermal management
 * local interrupt controller
 * PCI/HT host bridge

If you have a system with multiple CPUs in one socket and either multiple
sockets in one NUMA node or no NUMA at all,  you have no way of properly
enumerating the sockets.  I'd like to discuss what such an interface
would need to look like to be useful for all architectures.

	Arnd <><