NUMA BOF @OLS

NUMA BOF @OLS

[Christoph Lameter]

Everyone is invited to the NUMA BOF at the Ottawa Linux Symposium

Friday Jun 29th, 2007 19:00 - 20:00 in Rockhopper

The main interest seems to be a discussion on the use of memory policies. 
Lee Schermerhorn will talk a bit about his work and then I may say 
something about the problems with memory policies.

More subjects:

- Memoryless node support

- Restricting subsystems to not allocate on certain nodes
  (f.e. huge pages only on some nodes, slab only on some nodes,
  kernel memory only on some nodes).

- Cpusets and containers

- Do we need to have more support for multicore issues in the NUMA layers?

- Issues with the scheduler on NUMA. 

If you have another subject that should be brought up then please contact 
me.

Re: NUMA BOF @OLS

[Arnd Bergmann]

On Thursday 21 June 2007, Christoph Lameter wrote:
> If you have another subject that should be brought up then please contact 
> me.

- Interface for preallocating hugetlbfs pages per node instead of system wide

- architecture independent in-kernel API for enumerating CPU sockets with
  multicore processors (not sure if that's the same as your existing subject).

	Arnd <><

Re: NUMA BOF @OLS

[Christoph Lameter]

On Fri, 22 Jun 2007, Arnd Bergmann wrote:

> - Interface for preallocating hugetlbfs pages per node instead of system wide

We may want to get a bit higher level than that. General way of 
controlling subsystem use on nodes. One wants to restrict the slab 
allocator and the kernel etc on nodes too.

How will this interact with the other NUMA policy specifications?
 
> - architecture independent in-kernel API for enumerating CPU sockets with
>   multicore processors (not sure if that's the same as your existing subject).

Not sure what you mean by this. We already have a topology interface and 
the scheduler knows about these things.

Re: NUMA BOF @OLS

[Arnd Bergmann]

On Friday 22 June 2007, Christoph Lameter wrote:
> 
> On Fri, 22 Jun 2007, Arnd Bergmann wrote:
> 
> > - Interface for preallocating hugetlbfs pages per node instead of system wide
> 
> We may want to get a bit higher level than that. General way of 
> controlling subsystem use on nodes. One wants to restrict the slab 
> allocator and the kernel etc on nodes too.
> 
> How will this interact with the other NUMA policy specifications?

I guess that's what I'd like to discuss at the BOF. I frequently
get requests from users that need to have some interface for it:
Application currently break if they try to use /proc/sys/vm/nr_hugepages
in combination with numactl --membind.

> > - architecture independent in-kernel API for enumerating CPU sockets with
> >   multicore processors (not sure if that's the same as your existing subject).
> 
> Not sure what you mean by this. We already have a topology interface and 
> the scheduler knows about these things.

I'm not referring to user interfaces or scheduling. It's probably not really
a NUMA topic, but we currently use the topology interfaces for enumerating
sockets on systems that are not really NUMA. This includes stuff like
per-socket 
 * cpufreq settings (these have their own logic currently)
 * IOMMU
 * performance counters
 * thermal management
 * local interrupt controller
 * PCI/HT host bridge

If you have a system with multiple CPUs in one socket and either multiple
sockets in one NUMA node or no NUMA at all,  you have no way of properly
enumerating the sockets.  I'd like to discuss what such an interface
would need to look like to be useful for all architectures.

	Arnd <><

Re: NUMA BOF @OLS

[Takayoshi Kochi]

Hi all,

I'll host another mm-related BOF at OLS:

Discussion for the Future of Linux Memory Management
Saturday Jun 30th, 2007 14:45-15:30

I'll share some experiences with the MM-related real world issues there.
Anyone who have something to pitch in is welcome.
Please contact me or grab me at OLS.

Any topics spilled out of NUMA BOF are welcome!


2007/6/22, Arnd Bergmann <arnd@arndb.de>:
> On Friday 22 June 2007, Christoph Lameter wrote:
> >
> > On Fri, 22 Jun 2007, Arnd Bergmann wrote:
> >
> > > - Interface for preallocating hugetlbfs pages per node instead of system wide
> >
> > We may want to get a bit higher level than that. General way of
> > controlling subsystem use on nodes. One wants to restrict the slab
> > allocator and the kernel etc on nodes too.
> >
> > How will this interact with the other NUMA policy specifications?
>
> I guess that's what I'd like to discuss at the BOF. I frequently
> get requests from users that need to have some interface for it:
> Application currently break if they try to use /proc/sys/vm/nr_hugepages
> in combination with numactl --membind.
>
> > > - architecture independent in-kernel API for enumerating CPU sockets with
> > > multicore processors (not sure if that's the same as your existing subject).
> >
> > Not sure what you mean by this. We already have a topology interface and
> > the scheduler knows about these things.
>
> I'm not referring to user interfaces or scheduling. It's probably not really
> a NUMA topic, but we currently use the topology interfaces for enumerating
> sockets on systems that are not really NUMA. This includes stuff like
> per-socket
>  * cpufreq settings (these have their own logic currently)
>  * IOMMU
>  * performance counters
>  * thermal management
>  * local interrupt controller
>  * PCI/HT host bridge
>
> If you have a system with multiple CPUs in one socket and either multiple
> sockets in one NUMA node or no NUMA at all,  you have no way of properly
> enumerating the sockets.  I'd like to discuss what such an interface
> would need to look like to be useful for all architectures.
>
>         Arnd <><
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-- 
Takayoshi Kochi